2022-07-21T01:40:47.212 INFO:root:teuthology version: 1.1.1.dev383+g89d874d5 2022-07-21T01:40:47.214 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} --verbose --name yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365 -- /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/orig.config.yaml 2022-07-21T01:40:47.271 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-07-21T01:40:47.352 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365 branch: octopus-release description: fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} email: null first_in_suite: false job_id: '6941365' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: octopus-release ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: debug ms: 1 debug osd: 20 osd objectstore: filestore osd op complaint time: 180 osd sloppy crc: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - POOL_APP_NOT_ENABLED - has not responded to cap revoke by MDS for over - MDS_CLIENT_LATE_RELEASE - responding to mclientcaps sha1: 57544da114dbbe1c44f27b2ea18e3233e4e917e0 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: flavor: default sha1: 57544da114dbbe1c44f27b2ea18e3233e4e917e0 workunit: branch: octopus-release sha1: 57544da114dbbe1c44f27b2ea18e3233e4e917e0 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - - client.1 - - client.2 sha1: 57544da114dbbe1c44f27b2ea18e3233e4e917e0 sleep_before_teardown: 0 suite: fs suite_branch: octopus-release suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 57544da114dbbe1c44f27b2ea18e3233e4e917e0 targets: smithi140.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9x0WKcVST/l7vDaTfI7nKnhQhrkW71PXq8TlhG5T7KOTtcsgzvck9F1+BwHaspMsrLaH/UJ808y8KXdXmKqe+3aHtwOeg7PviU18S/6zlQm880RhjZwy8YtnRjxblQIbzteNDUVCvm9sFcAMMGzsqskG9QAfKWjU6FBESxzsvVtcu1x8JcBu1whatFbmvNMNj9conCia1GFon7cwjpY5uGwBKyNuhTTxYznh3TAOof9tvDwoCkLQ4yehTnCgtmN+nSOHHeaN+8GZIjT/nImnuSr84ZuKgbP38/4H/64lqvsV0qJPVJy6fHIxNTsSwu0LrTqrk/1vePLDIwdN8jYFOToqSAA17mCXO+CDrYlkgiOAi2hT72SqLdYKBUtlrPSv6aFBk+Y5drbBcavIheK/45DrXoxzFdLtAP1N4ZgjUn8/O8eX13nqLzfH9LAmzpkahWN/qaoonmnjIAgXZx3knISYY92ZHDeCaJxCtJCR5C9SGuwsuTclx2XmP12aD9Yk= smithi152.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfZ7/DsFfPLBGQi9jnB4Av06DPRwY6w371r4HYH4JVxi1d6qz968W2FpgfR5X+5D8OPcUgnl0Sgg4q1XkDdr9xaN+5TDzC3QCYlGUehLRdKYtOVXqJ8q370fXKkWu7CflgSHv7+5lLBmHRmZP8/7wpNSUPROct54sIkA/5xibapvvz6cTbwUBgbLDQNeJUndNrf8FV66HYiwuLXdOiFOKrK6dfaoSY6BsFEKiwcTi6dl1WKB+YemJmEejeFGFgD55ZbSGMgPwfU86b/AUOYuh0Qh5nWfk+WAQcu/dnKJqvzCwAvpaMkk240u3IbkBM3rQS+KjVVXWlSsvXw0ORH7UAc2z+1tI5CxSW3QLFBX68KCDftXy/dS9e2NzEGTMqMRLfKCGJFS5f18HaVdkGsTQF5AHMFtY9OmxRPlmaZmJZFdvonkLL9V/tFUVvxEjCTsnRRg5ef/9WgT6GVO/pNvVpIop60fjEPlre3pJuD8hHTzBs+u/P3bP1h0jWVDkGy30= smithi191.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUkUN0GSzJx2knn6eNQ/uLp5TG+T3LCJmskHcbyuI2Lwnk2gejB78bEz7Z1RimqnoPoRq2qEazNdM57IYZuA/DWYi3wbslshxwfd7B5n2lCYZoikj5BxJsn5Ygmmylojv1vL6VDCjRY3a8tZ2vCiVDc6xLkdiSYMdCS2UgBEUDS1UuZAZ9c3sQCmHPeqMRXbj9a0Rc3niR+VhP/cK7AAvM6sW6R6geX0BxlilPTC253E9p8s6dMMGusXUaUKP53AkwgAkbxqBePTppn4Ss1GgKxPswBFzX15t8/HgBOJWT7Z8PCRgMTKVs7AqJlTZsoCUBZYUzNn7WlgJ5Cpx6v12tkZsiQhxHrcd+Jkq5zqxWk+WlkaMay0cXqeLjx/QuUsdHxEjcPRQbXUuz30hZ/lD1XzFwVubPmW3eRLni/415UsftTj+rUKe+faIiOQ8buuGw+RvHYCxULMXRlY8P/gwNQDar8D1YrlV4RYCJAOM0gOThprKTHkzvd5YmEPqBSYs= smithi192.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+vDtwcDnkw0GjVDsV7muA7x/4sIK90gwk+QtVRqmYSn8Rd5zm/C4/4wEq+eaZaDkpHL282rFLP9oHqMTwI4iB3a9bhKdzhoEV/AI1JafUsZkcSMiKpoN9bqmlSyk61zEIXNWToBM4XI4LRM0LwJdHQqFz75ES94KrKCiVyq1HdKBAh+eD0STz0KRHHVWBqMj68SWqkQvqX58jWVwVTMeEHKYhZ6Gmm5mlGHpCPP6aiHJxSrQoAkOFyfyKP2xawkEYVNTMOx+TicJshHp9svqRg8jYwX3znWBydo4DYBk2DUHNvOTl8Y144B30/Vq5vIaF/hny0r/LX7HsiKP2tSLDwujR5xGtt1zRvfE7epMjIpF9yWdC4NkjLO9ieBJHxW7h49WbC2IPO53iKyVAz6svP2qh9ir2DtZFq8z+vuLUouLbl2kcfdk+BNjkFmCYGRPa/hz4tUlSIIDx9j2YafVDzsSDRKNFrcCLlW0/WYDjx+Jdu5scc4T9/4+NEblnY4M= smithi195.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChgDJokjWfgdtScLBpJnVPYk7hkd9iQCJNV5E1JbpsVje03XVrY6DQAegREfL/T7HBHc/kgkm/4I9KPXCqyQdQS80TqFrEcF2uMl2HtuWFL1oZ5EI5DvPmL7qXZ99E9DJ/D+8qS4fwIuu+RKJHreFiYaZ7zjJimRqQ4+vTLkdBblM+FlbrB9IQd4xpyrp3o3+89L+0Jp9IYMErnjkwF1p442rHv0rNfPD63XcA5/6t+YIvUV2nZCYrMf4oF7WPopNFs88UYvTVhqkUbv/KVunjoBQVzmOGUe90FAww0CHRrsi0gwFzqFekDzmVoD4vfo7BGn6jQknJWuVQK1AlT25GgviPI7ICW73/Ev3P+kvRr0WVbTdToRxk9kq6C6fugluM/6DpqSfjEVjQKzMjp4Btq8ouSukHrx7sIBnAMclrI7mEm8RKF7GjyNHvsA39OTn9DnXUP1b2wZMiw5h1yloMRZR4FVW8BxSBgLqQWOrKMAfTzlg1QMFxuR1zHK3K0K8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_misc teuthology_branch: main teuthology_sha1: 89d874d555f56b1c1f383df72487aecec0ee6cf9 timestamp: 2022-07-20_20:56:09 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1713 2022-07-21T01:40:47.352 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa; will attempt to use it 2022-07-21T01:40:47.353 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks 2022-07-21T01:40:47.354 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-07-21T01:40:47.356 INFO:teuthology.task.internal:Checking packages... 2022-07-21T01:40:47.377 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '57544da114dbbe1c44f27b2ea18e3233e4e917e0' 2022-07-21T01:40:47.377 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-07-21T01:40:47.378 INFO:teuthology.packaging:ref: None 2022-07-21T01:40:47.378 INFO:teuthology.packaging:tag: None 2022-07-21T01:40:47.379 INFO:teuthology.packaging:branch: octopus-release 2022-07-21T01:40:47.379 INFO:teuthology.packaging:sha1: 57544da114dbbe1c44f27b2ea18e3233e4e917e0 2022-07-21T01:40:47.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=octopus-release 2022-07-21T01:40:47.551 INFO:teuthology.task.internal:Found packages for ceph version 15.2.16-336-g57544da1-1focal 2022-07-21T01:40:47.551 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-07-21T01:40:47.559 INFO:teuthology.task.internal:no buildpackages task found 2022-07-21T01:40:47.559 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-07-21T01:40:47.597 INFO:teuthology.task.internal:Saving configuration 2022-07-21T01:40:47.616 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-07-21T01:40:47.632 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-07-21T01:40:47.667 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi140.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-07-21 01:35:17.139455', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBTKc/Tr7cluu5X/aXYNbB6IX7RcJ09JTut6+Fm6nlsA'} 2022-07-21T01:40:47.694 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-07-21 01:35:17.138099', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOvXnFHmQSeDJ12ZCOww/p1cDFzFgeVqfQE+/Gv2D9nM'} 2022-07-21T01:40:47.722 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-07-21 01:35:17.142639', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMF7ZuuwISohME4eH1GdYdGLv+o562xo4mEwpfFTL2e5'} 2022-07-21T01:40:47.749 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-07-21 01:35:17.141583', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJTJA/hRXeTeOq7oiyO174v3keaf9TdHavnIXhTvBqja'} 2022-07-21T01:40:47.776 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-07-21 01:35:17.140505', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKEicHgeFNjtECnhPay/VU/AP2Ky/LKA2sjgfQnRR9E8'} 2022-07-21T01:40:47.777 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-07-21T01:40:47.784 INFO:teuthology.task.internal:roles: ubuntu@smithi140.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2022-07-21T01:40:47.784 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2022-07-21T01:40:47.784 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['client.0'] 2022-07-21T01:40:47.785 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['client.1'] 2022-07-21T01:40:47.785 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['client.2'] 2022-07-21T01:40:47.785 INFO:teuthology.run_tasks:Running task console_log... 2022-07-21T01:40:48.028 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbd10f94d90>, signals=[15]) 2022-07-21T01:40:48.030 INFO:teuthology.run_tasks:Running task internal.connect... 2022-07-21T01:40:48.052 INFO:teuthology.task.internal:Opening connections... 2022-07-21T01:40:48.053 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi140.front.sepia.ceph.com 2022-07-21T01:40:48.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-07-21T01:40:48.171 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2022-07-21T01:40:48.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-07-21T01:40:48.276 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2022-07-21T01:40:48.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-07-21T01:40:48.373 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2022-07-21T01:40:48.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-07-21T01:40:48.473 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2022-07-21T01:40:48.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-07-21T01:40:48.573 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-07-21T01:40:48.581 DEBUG:teuthology.orchestra.run.smithi140:> uname -m 2022-07-21T01:40:48.588 INFO:teuthology.orchestra.run.smithi140.stdout:x86_64 2022-07-21T01:40:48.588 DEBUG:teuthology.orchestra.run.smithi140:> cat /etc/os-release 2022-07-21T01:40:48.638 INFO:teuthology.orchestra.run.smithi140.stdout:NAME="Ubuntu" 2022-07-21T01:40:48.638 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-21T01:40:48.638 INFO:teuthology.orchestra.run.smithi140.stdout:ID=ubuntu 2022-07-21T01:40:48.639 INFO:teuthology.orchestra.run.smithi140.stdout:ID_LIKE=debian 2022-07-21T01:40:48.639 INFO:teuthology.orchestra.run.smithi140.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-21T01:40:48.639 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION_ID="20.04" 2022-07-21T01:40:48.639 INFO:teuthology.orchestra.run.smithi140.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-21T01:40:48.640 INFO:teuthology.orchestra.run.smithi140.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-21T01:40:48.640 INFO:teuthology.orchestra.run.smithi140.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-21T01:40:48.640 INFO:teuthology.orchestra.run.smithi140.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-21T01:40:48.640 INFO:teuthology.orchestra.run.smithi140.stdout:VERSION_CODENAME=focal 2022-07-21T01:40:48.640 INFO:teuthology.orchestra.run.smithi140.stdout:UBUNTU_CODENAME=focal 2022-07-21T01:40:48.641 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2022-07-21T01:40:48.671 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2022-07-21T01:40:48.679 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2022-07-21T01:40:48.679 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2022-07-21T01:40:48.728 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="Ubuntu" 2022-07-21T01:40:48.729 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-21T01:40:48.729 INFO:teuthology.orchestra.run.smithi152.stdout:ID=ubuntu 2022-07-21T01:40:48.729 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE=debian 2022-07-21T01:40:48.730 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-21T01:40:48.730 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="20.04" 2022-07-21T01:40:48.730 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-21T01:40:48.731 INFO:teuthology.orchestra.run.smithi152.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-21T01:40:48.731 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-21T01:40:48.731 INFO:teuthology.orchestra.run.smithi152.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-21T01:40:48.731 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_CODENAME=focal 2022-07-21T01:40:48.732 INFO:teuthology.orchestra.run.smithi152.stdout:UBUNTU_CODENAME=focal 2022-07-21T01:40:48.732 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2022-07-21T01:40:48.751 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2022-07-21T01:40:48.759 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2022-07-21T01:40:48.760 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2022-07-21T01:40:48.806 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Ubuntu" 2022-07-21T01:40:48.806 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-21T01:40:48.807 INFO:teuthology.orchestra.run.smithi191.stdout:ID=ubuntu 2022-07-21T01:40:48.807 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE=debian 2022-07-21T01:40:48.807 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-21T01:40:48.807 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="20.04" 2022-07-21T01:40:48.808 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-21T01:40:48.808 INFO:teuthology.orchestra.run.smithi191.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-21T01:40:48.808 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-21T01:40:48.808 INFO:teuthology.orchestra.run.smithi191.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-21T01:40:48.809 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_CODENAME=focal 2022-07-21T01:40:48.809 INFO:teuthology.orchestra.run.smithi191.stdout:UBUNTU_CODENAME=focal 2022-07-21T01:40:48.809 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2022-07-21T01:40:48.829 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2022-07-21T01:40:48.836 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2022-07-21T01:40:48.836 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2022-07-21T01:40:48.884 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="Ubuntu" 2022-07-21T01:40:48.884 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-21T01:40:48.884 INFO:teuthology.orchestra.run.smithi192.stdout:ID=ubuntu 2022-07-21T01:40:48.884 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE=debian 2022-07-21T01:40:48.885 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-21T01:40:48.885 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="20.04" 2022-07-21T01:40:48.885 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-21T01:40:48.885 INFO:teuthology.orchestra.run.smithi192.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-21T01:40:48.886 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-21T01:40:48.886 INFO:teuthology.orchestra.run.smithi192.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-21T01:40:48.886 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_CODENAME=focal 2022-07-21T01:40:48.886 INFO:teuthology.orchestra.run.smithi192.stdout:UBUNTU_CODENAME=focal 2022-07-21T01:40:48.887 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2022-07-21T01:40:48.905 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2022-07-21T01:40:48.913 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2022-07-21T01:40:48.913 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2022-07-21T01:40:48.960 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="Ubuntu" 2022-07-21T01:40:48.960 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-07-21T01:40:48.961 INFO:teuthology.orchestra.run.smithi195.stdout:ID=ubuntu 2022-07-21T01:40:48.961 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE=debian 2022-07-21T01:40:48.961 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-07-21T01:40:48.961 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="20.04" 2022-07-21T01:40:48.962 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://www.ubuntu.com/" 2022-07-21T01:40:48.962 INFO:teuthology.orchestra.run.smithi195.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-07-21T01:40:48.962 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-07-21T01:40:48.962 INFO:teuthology.orchestra.run.smithi195.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-07-21T01:40:48.963 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_CODENAME=focal 2022-07-21T01:40:48.963 INFO:teuthology.orchestra.run.smithi195.stdout:UBUNTU_CODENAME=focal 2022-07-21T01:40:48.963 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2022-07-21T01:40:48.981 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-07-21T01:40:48.995 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-07-21T01:40:49.022 INFO:teuthology.task.internal:Checking for old test directory... 2022-07-21T01:40:49.023 DEBUG:teuthology.orchestra.run.smithi140:> test '!' -e /home/ubuntu/cephtest 2022-07-21T01:40:49.026 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2022-07-21T01:40:49.030 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2022-07-21T01:40:49.035 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2022-07-21T01:40:49.038 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2022-07-21T01:40:49.044 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-07-21T01:40:49.050 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-07-21T01:40:49.051 DEBUG:teuthology.orchestra.run.smithi140:> test -z $(ls -A /var/lib/ceph) 2022-07-21T01:40:49.054 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2022-07-21T01:40:49.057 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2022-07-21T01:40:49.082 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2022-07-21T01:40:49.086 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2022-07-21T01:40:49.096 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-07-21T01:40:49.226 INFO:teuthology.run_tasks:Running task kernel... 2022-07-21T01:40:49.253 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-07-21T01:40:49.254 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}, 'client.2': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-07-21T01:40:49.255 DEBUG:teuthology.orchestra.run.smithi140:> test -f /run/.containerenv -o -f /.dockerenv 2022-07-21T01:40:49.256 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2022-07-21T01:40:49.257 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2022-07-21T01:40:49.257 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2022-07-21T01:40:49.258 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2022-07-21T01:40:49.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:40:49.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:40:49.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:40:49.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:40:49.269 DEBUG:teuthology.orchestra.run.smithi140:> uname -r 2022-07-21T01:40:49.270 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2022-07-21T01:40:49.271 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2022-07-21T01:40:49.272 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2022-07-21T01:40:49.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:40:49.274 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2022-07-21T01:40:49.279 INFO:teuthology.orchestra.run.smithi140.stdout:5.4.0-122-generic 2022-07-21T01:40:49.279 INFO:teuthology.task.kernel:Running kernel on smithi140: 5.4.0-122-generic 2022-07-21T01:40:49.279 DEBUG:teuthology.orchestra.run.smithi140:> sudo apt-get clean 2022-07-21T01:40:49.281 INFO:teuthology.orchestra.run.smithi192.stdout:5.4.0-122-generic 2022-07-21T01:40:49.282 INFO:teuthology.orchestra.run.smithi152.stdout:5.4.0-122-generic 2022-07-21T01:40:49.282 INFO:teuthology.task.kernel:Running kernel on smithi192: 5.4.0-122-generic 2022-07-21T01:40:49.283 DEBUG:teuthology.orchestra.run.smithi192:> sudo apt-get clean 2022-07-21T01:40:49.283 INFO:teuthology.task.kernel:Running kernel on smithi152: 5.4.0-122-generic 2022-07-21T01:40:49.284 DEBUG:teuthology.orchestra.run.smithi152:> sudo apt-get clean 2022-07-21T01:40:49.318 INFO:teuthology.orchestra.run.smithi191.stdout:5.4.0-122-generic 2022-07-21T01:40:49.319 INFO:teuthology.task.kernel:Running kernel on smithi191: 5.4.0-122-generic 2022-07-21T01:40:49.319 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get clean 2022-07-21T01:40:49.323 INFO:teuthology.orchestra.run.smithi195.stdout:5.4.0-122-generic 2022-07-21T01:40:49.324 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.4.0-122-generic 2022-07-21T01:40:49.324 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2022-07-21T01:40:49.437 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2022-07-21T01:40:49.442 DEBUG:teuthology.orchestra.run.smithi140:> sudo apt-get update 2022-07-21T01:40:49.444 DEBUG:teuthology.orchestra.run.smithi152:> sudo apt-get update 2022-07-21T01:40:49.449 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get update 2022-07-21T01:40:49.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo apt-get update 2022-07-21T01:40:49.622 INFO:teuthology.orchestra.run.smithi140.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-07-21T01:40:49.622 INFO:teuthology.orchestra.run.smithi140.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-07-21T01:40:49.623 INFO:teuthology.orchestra.run.smithi140.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-07-21T01:40:49.686 INFO:teuthology.orchestra.run.smithi140.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-07-21T01:40:49.736 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-07-21T01:40:49.736 INFO:teuthology.orchestra.run.smithi191.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-07-21T01:40:49.737 INFO:teuthology.orchestra.run.smithi191.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-07-21T01:40:49.740 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-07-21T01:40:49.740 INFO:teuthology.orchestra.run.smithi152.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-07-21T01:40:49.743 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-07-21T01:40:49.743 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-07-21T01:40:49.750 INFO:teuthology.orchestra.run.smithi192.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-07-21T01:40:49.750 INFO:teuthology.orchestra.run.smithi192.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-07-21T01:40:49.773 INFO:teuthology.orchestra.run.smithi191.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-07-21T01:40:49.800 INFO:teuthology.orchestra.run.smithi152.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-07-21T01:40:49.801 INFO:teuthology.orchestra.run.smithi192.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-07-21T01:40:49.801 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-07-21T01:40:50.001 INFO:teuthology.orchestra.run.smithi140.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [471 kB] 2022-07-21T01:40:50.118 INFO:teuthology.orchestra.run.smithi191.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,630 kB] 2022-07-21T01:40:50.128 INFO:teuthology.orchestra.run.smithi140.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,630 kB] 2022-07-21T01:40:50.144 INFO:teuthology.orchestra.run.smithi140.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,990 kB] 2022-07-21T01:40:50.147 INFO:teuthology.orchestra.run.smithi152.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-07-21T01:40:50.148 INFO:teuthology.orchestra.run.smithi192.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-07-21T01:40:50.149 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-07-21T01:40:50.215 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,630 kB] 2022-07-21T01:40:50.216 INFO:teuthology.orchestra.run.smithi192.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [471 kB] 2022-07-21T01:40:50.225 INFO:teuthology.orchestra.run.smithi152.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,630 kB] 2022-07-21T01:40:50.256 INFO:teuthology.orchestra.run.smithi140.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [277 kB] 2022-07-21T01:40:50.257 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [696 kB] 2022-07-21T01:40:50.261 INFO:teuthology.orchestra.run.smithi140.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-07-21T01:40:50.262 INFO:teuthology.orchestra.run.smithi140.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,091 kB] 2022-07-21T01:40:50.277 INFO:teuthology.orchestra.run.smithi140.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [155 kB] 2022-07-21T01:40:50.277 INFO:teuthology.orchestra.run.smithi140.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [711 kB] 2022-07-21T01:40:50.286 INFO:teuthology.orchestra.run.smithi140.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [696 kB] 2022-07-21T01:40:50.290 INFO:teuthology.orchestra.run.smithi140.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [556 kB] 2022-07-21T01:40:50.295 INFO:teuthology.orchestra.run.smithi140.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [128 kB] 2022-07-21T01:40:50.296 INFO:teuthology.orchestra.run.smithi140.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [358 kB] 2022-07-21T01:40:50.298 INFO:teuthology.orchestra.run.smithi140.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-07-21T01:40:50.298 INFO:teuthology.orchestra.run.smithi140.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,178 kB] 2022-07-21T01:40:50.301 INFO:teuthology.orchestra.run.smithi140.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-07-21T01:40:50.313 INFO:teuthology.orchestra.run.smithi140.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [167 kB] 2022-07-21T01:40:50.314 INFO:teuthology.orchestra.run.smithi140.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [677 kB] 2022-07-21T01:40:50.328 INFO:teuthology.orchestra.run.smithi140.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [924 kB] 2022-07-21T01:40:50.345 INFO:teuthology.orchestra.run.smithi140.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [208 kB] 2022-07-21T01:40:50.348 INFO:teuthology.orchestra.run.smithi140.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-07-21T01:40:50.351 INFO:teuthology.orchestra.run.smithi192.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,990 kB] 2022-07-21T01:40:50.354 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,990 kB] 2022-07-21T01:40:50.364 INFO:teuthology.orchestra.run.smithi152.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [696 kB] 2022-07-21T01:40:50.460 INFO:teuthology.orchestra.run.smithi191.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [471 kB] 2022-07-21T01:40:50.508 INFO:teuthology.orchestra.run.smithi192.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,630 kB] 2022-07-21T01:40:50.527 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [277 kB] 2022-07-21T01:40:50.562 INFO:teuthology.orchestra.run.smithi191.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,990 kB] 2022-07-21T01:40:50.569 INFO:teuthology.orchestra.run.smithi191.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-07-21T01:40:50.570 INFO:teuthology.orchestra.run.smithi191.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,091 kB] 2022-07-21T01:40:50.649 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [471 kB] 2022-07-21T01:40:50.662 INFO:teuthology.orchestra.run.smithi152.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,990 kB] 2022-07-21T01:40:50.662 INFO:teuthology.orchestra.run.smithi152.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [471 kB] 2022-07-21T01:40:50.664 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [277 kB] 2022-07-21T01:40:50.672 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-07-21T01:40:50.673 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,091 kB] 2022-07-21T01:40:50.677 INFO:teuthology.orchestra.run.smithi152.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [277 kB] 2022-07-21T01:40:50.681 INFO:teuthology.orchestra.run.smithi192.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [277 kB] 2022-07-21T01:40:50.685 INFO:teuthology.orchestra.run.smithi152.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-07-21T01:40:50.686 INFO:teuthology.orchestra.run.smithi152.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,091 kB] 2022-07-21T01:40:50.694 INFO:teuthology.orchestra.run.smithi192.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [10.8 kB] 2022-07-21T01:40:50.695 INFO:teuthology.orchestra.run.smithi192.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,091 kB] 2022-07-21T01:40:50.733 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [155 kB] 2022-07-21T01:40:50.738 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [556 kB] 2022-07-21T01:40:50.746 INFO:teuthology.orchestra.run.smithi152.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [155 kB] 2022-07-21T01:40:50.747 INFO:teuthology.orchestra.run.smithi192.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [155 kB] 2022-07-21T01:40:50.751 INFO:teuthology.orchestra.run.smithi152.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [556 kB] 2022-07-21T01:40:50.753 INFO:teuthology.orchestra.run.smithi191.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [155 kB] 2022-07-21T01:40:50.754 INFO:teuthology.orchestra.run.smithi192.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [556 kB] 2022-07-21T01:40:50.756 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [711 kB] 2022-07-21T01:40:50.763 INFO:teuthology.orchestra.run.smithi192.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [696 kB] 2022-07-21T01:40:50.766 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [696 kB] 2022-07-21T01:40:50.770 INFO:teuthology.orchestra.run.smithi152.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [711 kB] 2022-07-21T01:40:50.770 INFO:teuthology.orchestra.run.smithi192.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [711 kB] 2022-07-21T01:40:50.774 INFO:teuthology.orchestra.run.smithi191.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [358 kB] 2022-07-21T01:40:50.775 INFO:teuthology.orchestra.run.smithi191.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [711 kB] 2022-07-21T01:40:50.778 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [128 kB] 2022-07-21T01:40:50.782 INFO:teuthology.orchestra.run.smithi192.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [358 kB] 2022-07-21T01:40:50.783 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-07-21T01:40:50.784 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [358 kB] 2022-07-21T01:40:50.787 INFO:teuthology.orchestra.run.smithi192.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [128 kB] 2022-07-21T01:40:50.790 INFO:teuthology.orchestra.run.smithi192.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-07-21T01:40:50.791 INFO:teuthology.orchestra.run.smithi191.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-07-21T01:40:50.793 INFO:teuthology.orchestra.run.smithi152.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [128 kB] 2022-07-21T01:40:50.794 INFO:teuthology.orchestra.run.smithi191.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,178 kB] 2022-07-21T01:40:50.794 INFO:teuthology.orchestra.run.smithi192.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-07-21T01:40:50.794 INFO:teuthology.orchestra.run.smithi192.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,178 kB] 2022-07-21T01:40:50.795 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-07-21T01:40:50.795 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,178 kB] 2022-07-21T01:40:50.796 INFO:teuthology.orchestra.run.smithi152.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-07-21T01:40:50.824 INFO:teuthology.orchestra.run.smithi152.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [358 kB] 2022-07-21T01:40:50.836 INFO:teuthology.orchestra.run.smithi152.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [15.6 kB] 2022-07-21T01:40:50.837 INFO:teuthology.orchestra.run.smithi152.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,178 kB] 2022-07-21T01:40:50.838 INFO:teuthology.orchestra.run.smithi191.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [167 kB] 2022-07-21T01:40:50.843 INFO:teuthology.orchestra.run.smithi191.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [924 kB] 2022-07-21T01:40:50.847 INFO:teuthology.orchestra.run.smithi192.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [167 kB] 2022-07-21T01:40:50.850 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [167 kB] 2022-07-21T01:40:50.851 INFO:teuthology.orchestra.run.smithi192.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [924 kB] 2022-07-21T01:40:50.854 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [924 kB] 2022-07-21T01:40:50.877 INFO:teuthology.orchestra.run.smithi192.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [677 kB] 2022-07-21T01:40:50.878 INFO:teuthology.orchestra.run.smithi152.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [167 kB] 2022-07-21T01:40:50.878 INFO:teuthology.orchestra.run.smithi191.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [677 kB] 2022-07-21T01:40:50.879 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [677 kB] 2022-07-21T01:40:50.880 INFO:teuthology.orchestra.run.smithi191.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [556 kB] 2022-07-21T01:40:50.883 INFO:teuthology.orchestra.run.smithi152.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [677 kB] 2022-07-21T01:40:50.894 INFO:teuthology.orchestra.run.smithi192.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [208 kB] 2022-07-21T01:40:50.897 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [208 kB] 2022-07-21T01:40:50.899 INFO:teuthology.orchestra.run.smithi192.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-07-21T01:40:50.901 INFO:teuthology.orchestra.run.smithi191.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [208 kB] 2022-07-21T01:40:50.901 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-07-21T01:40:50.904 INFO:teuthology.orchestra.run.smithi152.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [924 kB] 2022-07-21T01:40:50.908 INFO:teuthology.orchestra.run.smithi191.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-07-21T01:40:50.924 INFO:teuthology.orchestra.run.smithi152.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [208 kB] 2022-07-21T01:40:50.928 INFO:teuthology.orchestra.run.smithi152.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [20.9 kB] 2022-07-21T01:40:50.979 INFO:teuthology.orchestra.run.smithi191.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [128 kB] 2022-07-21T01:40:50.984 INFO:teuthology.orchestra.run.smithi191.stdout:Get:24 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [14.7 kB] 2022-07-21T01:40:53.808 INFO:teuthology.orchestra.run.smithi140.stdout:Fetched 11.6 MB in 2s (5,896 kB/s) 2022-07-21T01:40:53.952 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 11.6 MB in 2s (5,244 kB/s) 2022-07-21T01:40:54.053 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 11.6 MB in 2s (5,019 kB/s) 2022-07-21T01:40:54.059 INFO:teuthology.orchestra.run.smithi152.stdout:Fetched 11.6 MB in 2s (5,020 kB/s) 2022-07-21T01:40:54.142 INFO:teuthology.orchestra.run.smithi192.stdout:Fetched 11.6 MB in 2s (4,929 kB/s) 2022-07-21T01:40:55.081 INFO:teuthology.orchestra.run.smithi140.stdout:Reading package lists... 2022-07-21T01:40:55.107 DEBUG:teuthology.orchestra.run.smithi140:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-07-21T01:40:55.175 INFO:teuthology.orchestra.run.smithi140.stdout:Reading package lists... 2022-07-21T01:40:55.184 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2022-07-21T01:40:55.208 DEBUG:teuthology.orchestra.run.smithi191:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-07-21T01:40:55.273 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2022-07-21T01:40:55.310 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2022-07-21T01:40:55.335 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-07-21T01:40:55.347 INFO:teuthology.orchestra.run.smithi140.stdout:Building dependency tree... 2022-07-21T01:40:55.348 INFO:teuthology.orchestra.run.smithi140.stdout:Reading state information... 2022-07-21T01:40:55.356 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2022-07-21T01:40:55.379 DEBUG:teuthology.orchestra.run.smithi152:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-07-21T01:40:55.406 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2022-07-21T01:40:55.448 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2022-07-21T01:40:55.449 INFO:teuthology.orchestra.run.smithi191.stdout:Building dependency tree... 2022-07-21T01:40:55.449 INFO:teuthology.orchestra.run.smithi191.stdout:Reading state information... 2022-07-21T01:40:55.467 INFO:teuthology.orchestra.run.smithi192.stdout:Reading package lists... 2022-07-21T01:40:55.492 DEBUG:teuthology.orchestra.run.smithi192:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-07-21T01:40:55.550 INFO:teuthology.orchestra.run.smithi140.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-07-21T01:40:55.551 INFO:teuthology.orchestra.run.smithi140.stdout:linux-image-generic set to manually installed. 2022-07-21T01:40:55.551 INFO:teuthology.orchestra.run.smithi140.stdout:0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:40:55.553 DEBUG:teuthology.orchestra.run.smithi140:> dpkg -s linux-image-generic 2022-07-21T01:40:55.566 INFO:teuthology.orchestra.run.smithi192.stdout:Reading package lists... 2022-07-21T01:40:55.575 INFO:teuthology.orchestra.run.smithi140.stdout:Package: linux-image-generic 2022-07-21T01:40:55.575 INFO:teuthology.orchestra.run.smithi140.stdout:Status: install ok installed 2022-07-21T01:40:55.576 INFO:teuthology.orchestra.run.smithi140.stdout:Priority: optional 2022-07-21T01:40:55.576 INFO:teuthology.orchestra.run.smithi140.stdout:Section: kernel 2022-07-21T01:40:55.576 INFO:teuthology.orchestra.run.smithi140.stdout:Installed-Size: 19 2022-07-21T01:40:55.577 INFO:teuthology.orchestra.run.smithi140.stdout:Maintainer: Ubuntu Kernel Team 2022-07-21T01:40:55.577 INFO:teuthology.orchestra.run.smithi140.stdout:Architecture: amd64 2022-07-21T01:40:55.577 INFO:teuthology.orchestra.run.smithi140.stdout:Source: linux-meta 2022-07-21T01:40:55.577 INFO:teuthology.orchestra.run.smithi140.stdout:Version: 5.4.0.122.123 2022-07-21T01:40:55.578 INFO:teuthology.orchestra.run.smithi140.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-07-21T01:40:55.578 INFO:teuthology.orchestra.run.smithi140.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-07-21T01:40:55.578 INFO:teuthology.orchestra.run.smithi140.stdout:Recommends: thermald 2022-07-21T01:40:55.578 INFO:teuthology.orchestra.run.smithi140.stdout:Description: Generic Linux kernel image 2022-07-21T01:40:55.579 INFO:teuthology.orchestra.run.smithi140.stdout: This package will always depend on the latest generic kernel image 2022-07-21T01:40:55.579 INFO:teuthology.orchestra.run.smithi140.stdout: available. 2022-07-21T01:40:55.579 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-07-21T01:40:55.580 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2022-07-21T01:40:55.580 DEBUG:teuthology.orchestra.run.smithi140:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-07-21T01:40:55.581 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2022-07-21T01:40:55.582 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2022-07-21T01:40:55.627 INFO:teuthology.orchestra.run.smithi152.stdout:Building dependency tree... 2022-07-21T01:40:55.628 INFO:teuthology.orchestra.run.smithi152.stdout:Reading state information... 2022-07-21T01:40:55.636 INFO:teuthology.orchestra.run.smithi140.stdout:ttyS1 2022-07-21T01:40:55.655 INFO:teuthology.orchestra.run.smithi191.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-07-21T01:40:55.655 INFO:teuthology.orchestra.run.smithi191.stdout:linux-image-generic set to manually installed. 2022-07-21T01:40:55.656 INFO:teuthology.orchestra.run.smithi191.stdout:0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:40:55.657 DEBUG:teuthology.orchestra.run.smithi191:> dpkg -s linux-image-generic 2022-07-21T01:40:55.662 DEBUG:teuthology.parallel:result is None 2022-07-21T01:40:55.679 INFO:teuthology.orchestra.run.smithi191.stdout:Package: linux-image-generic 2022-07-21T01:40:55.680 INFO:teuthology.orchestra.run.smithi191.stdout:Status: install ok installed 2022-07-21T01:40:55.680 INFO:teuthology.orchestra.run.smithi191.stdout:Priority: optional 2022-07-21T01:40:55.680 INFO:teuthology.orchestra.run.smithi191.stdout:Section: kernel 2022-07-21T01:40:55.680 INFO:teuthology.orchestra.run.smithi191.stdout:Installed-Size: 19 2022-07-21T01:40:55.681 INFO:teuthology.orchestra.run.smithi191.stdout:Maintainer: Ubuntu Kernel Team 2022-07-21T01:40:55.681 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: amd64 2022-07-21T01:40:55.681 INFO:teuthology.orchestra.run.smithi191.stdout:Source: linux-meta 2022-07-21T01:40:55.681 INFO:teuthology.orchestra.run.smithi191.stdout:Version: 5.4.0.122.123 2022-07-21T01:40:55.682 INFO:teuthology.orchestra.run.smithi191.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-07-21T01:40:55.682 INFO:teuthology.orchestra.run.smithi191.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-07-21T01:40:55.682 INFO:teuthology.orchestra.run.smithi191.stdout:Recommends: thermald 2022-07-21T01:40:55.682 INFO:teuthology.orchestra.run.smithi191.stdout:Description: Generic Linux kernel image 2022-07-21T01:40:55.683 INFO:teuthology.orchestra.run.smithi191.stdout: This package will always depend on the latest generic kernel image 2022-07-21T01:40:55.683 INFO:teuthology.orchestra.run.smithi191.stdout: available. 2022-07-21T01:40:55.683 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-07-21T01:40:55.684 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2022-07-21T01:40:55.684 DEBUG:teuthology.orchestra.run.smithi191:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-07-21T01:40:55.710 INFO:teuthology.orchestra.run.smithi191.stdout:ttyS1 2022-07-21T01:40:55.735 DEBUG:teuthology.parallel:result is None 2022-07-21T01:40:55.743 INFO:teuthology.orchestra.run.smithi192.stdout:Building dependency tree... 2022-07-21T01:40:55.744 INFO:teuthology.orchestra.run.smithi192.stdout:Reading state information... 2022-07-21T01:40:55.787 INFO:teuthology.orchestra.run.smithi195.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-07-21T01:40:55.788 INFO:teuthology.orchestra.run.smithi195.stdout:linux-image-generic set to manually installed. 2022-07-21T01:40:55.788 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:40:55.790 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2022-07-21T01:40:55.813 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2022-07-21T01:40:55.813 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2022-07-21T01:40:55.814 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2022-07-21T01:40:55.814 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2022-07-21T01:40:55.814 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 19 2022-07-21T01:40:55.814 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2022-07-21T01:40:55.815 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2022-07-21T01:40:55.815 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2022-07-21T01:40:55.815 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.4.0.122.123 2022-07-21T01:40:55.815 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-07-21T01:40:55.815 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-07-21T01:40:55.816 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2022-07-21T01:40:55.816 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2022-07-21T01:40:55.816 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2022-07-21T01:40:55.816 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2022-07-21T01:40:55.817 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-07-21T01:40:55.817 INFO:teuthology.task.kernel:Enabling kdb on client.2... 2022-07-21T01:40:55.817 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-07-21T01:40:55.829 INFO:teuthology.orchestra.run.smithi152.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-07-21T01:40:55.829 INFO:teuthology.orchestra.run.smithi152.stdout:linux-image-generic set to manually installed. 2022-07-21T01:40:55.830 INFO:teuthology.orchestra.run.smithi152.stdout:0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:40:55.831 DEBUG:teuthology.orchestra.run.smithi152:> dpkg -s linux-image-generic 2022-07-21T01:40:55.847 INFO:teuthology.orchestra.run.smithi152.stdout:Package: linux-image-generic 2022-07-21T01:40:55.848 INFO:teuthology.orchestra.run.smithi152.stdout:Status: install ok installed 2022-07-21T01:40:55.848 INFO:teuthology.orchestra.run.smithi152.stdout:Priority: optional 2022-07-21T01:40:55.848 INFO:teuthology.orchestra.run.smithi152.stdout:Section: kernel 2022-07-21T01:40:55.848 INFO:teuthology.orchestra.run.smithi152.stdout:Installed-Size: 19 2022-07-21T01:40:55.848 INFO:teuthology.orchestra.run.smithi152.stdout:Maintainer: Ubuntu Kernel Team 2022-07-21T01:40:55.849 INFO:teuthology.orchestra.run.smithi152.stdout:Architecture: amd64 2022-07-21T01:40:55.849 INFO:teuthology.orchestra.run.smithi152.stdout:Source: linux-meta 2022-07-21T01:40:55.849 INFO:teuthology.orchestra.run.smithi152.stdout:Version: 5.4.0.122.123 2022-07-21T01:40:55.849 INFO:teuthology.orchestra.run.smithi152.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-07-21T01:40:55.850 INFO:teuthology.orchestra.run.smithi152.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-07-21T01:40:55.850 INFO:teuthology.orchestra.run.smithi152.stdout:Recommends: thermald 2022-07-21T01:40:55.850 INFO:teuthology.orchestra.run.smithi152.stdout:Description: Generic Linux kernel image 2022-07-21T01:40:55.850 INFO:teuthology.orchestra.run.smithi152.stdout: This package will always depend on the latest generic kernel image 2022-07-21T01:40:55.851 INFO:teuthology.orchestra.run.smithi152.stdout: available. 2022-07-21T01:40:55.851 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-07-21T01:40:55.852 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2022-07-21T01:40:55.853 DEBUG:teuthology.orchestra.run.smithi152:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-07-21T01:40:55.886 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2022-07-21T01:40:55.913 INFO:teuthology.orchestra.run.smithi152.stdout:ttyS1 2022-07-21T01:40:55.916 DEBUG:teuthology.parallel:result is None 2022-07-21T01:40:55.936 DEBUG:teuthology.parallel:result is None 2022-07-21T01:40:55.942 INFO:teuthology.orchestra.run.smithi192.stdout:linux-image-generic is already the newest version (5.4.0.122.123). 2022-07-21T01:40:55.943 INFO:teuthology.orchestra.run.smithi192.stdout:linux-image-generic set to manually installed. 2022-07-21T01:40:55.943 INFO:teuthology.orchestra.run.smithi192.stdout:0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:40:55.945 DEBUG:teuthology.orchestra.run.smithi192:> dpkg -s linux-image-generic 2022-07-21T01:40:55.965 INFO:teuthology.orchestra.run.smithi192.stdout:Package: linux-image-generic 2022-07-21T01:40:55.965 INFO:teuthology.orchestra.run.smithi192.stdout:Status: install ok installed 2022-07-21T01:40:55.966 INFO:teuthology.orchestra.run.smithi192.stdout:Priority: optional 2022-07-21T01:40:55.966 INFO:teuthology.orchestra.run.smithi192.stdout:Section: kernel 2022-07-21T01:40:55.967 INFO:teuthology.orchestra.run.smithi192.stdout:Installed-Size: 19 2022-07-21T01:40:55.967 INFO:teuthology.orchestra.run.smithi192.stdout:Maintainer: Ubuntu Kernel Team 2022-07-21T01:40:55.968 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: amd64 2022-07-21T01:40:55.968 INFO:teuthology.orchestra.run.smithi192.stdout:Source: linux-meta 2022-07-21T01:40:55.969 INFO:teuthology.orchestra.run.smithi192.stdout:Version: 5.4.0.122.123 2022-07-21T01:40:55.969 INFO:teuthology.orchestra.run.smithi192.stdout:Provides: virtualbox-guest-modules (= 6.1.32-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-07-21T01:40:55.970 INFO:teuthology.orchestra.run.smithi192.stdout:Depends: linux-image-5.4.0-122-generic, linux-modules-extra-5.4.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2022-07-21T01:40:55.970 INFO:teuthology.orchestra.run.smithi192.stdout:Recommends: thermald 2022-07-21T01:40:55.970 INFO:teuthology.orchestra.run.smithi192.stdout:Description: Generic Linux kernel image 2022-07-21T01:40:55.971 INFO:teuthology.orchestra.run.smithi192.stdout: This package will always depend on the latest generic kernel image 2022-07-21T01:40:55.971 INFO:teuthology.orchestra.run.smithi192.stdout: available. 2022-07-21T01:40:55.972 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-07-21T01:40:55.972 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2022-07-21T01:40:55.973 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-07-21T01:40:55.995 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2022-07-21T01:40:56.012 DEBUG:teuthology.parallel:result is None 2022-07-21T01:40:56.013 INFO:teuthology.run_tasks:Running task internal.base... 2022-07-21T01:40:56.025 INFO:teuthology.task.internal:Creating test directory... 2022-07-21T01:40:56.026 DEBUG:teuthology.orchestra.run.smithi140:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-07-21T01:40:56.030 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-07-21T01:40:56.034 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-07-21T01:40:56.039 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-07-21T01:40:56.044 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-07-21T01:40:56.052 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-07-21T01:40:56.066 INFO:teuthology.run_tasks:Running task internal.archive... 2022-07-21T01:40:56.092 INFO:teuthology.task.internal:Creating archive directory... 2022-07-21T01:40:56.093 DEBUG:teuthology.orchestra.run.smithi140:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-07-21T01:40:56.097 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-07-21T01:40:56.102 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-07-21T01:40:56.106 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-07-21T01:40:56.109 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-07-21T01:40:56.125 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-07-21T01:40:56.132 INFO:teuthology.task.internal:Enabling coredump saving... 2022-07-21T01:40:56.132 DEBUG:teuthology.orchestra.run.smithi140:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-07-21T01:40:56.135 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-07-21T01:40:56.137 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-07-21T01:40:56.140 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-07-21T01:40:56.142 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-07-21T01:40:56.169 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-07-21T01:40:56.170 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-07-21T01:40:56.170 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-07-21T01:40:56.171 INFO:teuthology.orchestra.run.smithi140.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-07-21T01:40:56.172 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-07-21T01:40:56.173 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-07-21T01:40:56.174 INFO:teuthology.orchestra.run.smithi140.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-07-21T01:40:56.176 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-07-21T01:40:56.192 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-07-21T01:40:56.203 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-07-21T01:40:56.205 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-07-21T01:40:56.213 INFO:teuthology.task.internal:Configuring sudo... 2022-07-21T01:40:56.214 DEBUG:teuthology.orchestra.run.smithi140:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-07-21T01:40:56.217 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-07-21T01:40:56.224 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-07-21T01:40:56.228 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-07-21T01:40:56.230 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-07-21T01:40:56.265 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-07-21T01:40:56.275 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-07-21T01:40:56.276 DEBUG:teuthology.orchestra.run.smithi140:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-07-21T01:40:56.279 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-07-21T01:40:56.282 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-07-21T01:40:56.285 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-07-21T01:40:56.288 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-07-21T01:40:56.295 DEBUG:teuthology.orchestra.run.smithi140:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-07-21T01:40:56.335 DEBUG:teuthology.orchestra.run.smithi140:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-07-21T01:40:56.343 DEBUG:teuthology.orchestra.run.smithi140:> set -ex 2022-07-21T01:40:56.344 DEBUG:teuthology.orchestra.run.smithi140:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-07-21T01:40:56.368 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-07-21T01:40:56.377 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-07-21T01:40:56.385 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2022-07-21T01:40:56.385 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-07-21T01:40:56.413 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-07-21T01:40:56.422 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-07-21T01:40:56.429 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-07-21T01:40:56.430 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-07-21T01:40:56.465 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-07-21T01:40:56.473 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-07-21T01:40:56.521 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2022-07-21T01:40:56.521 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-07-21T01:40:56.549 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-07-21T01:40:56.557 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-07-21T01:40:56.564 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2022-07-21T01:40:56.564 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-07-21T01:40:56.592 DEBUG:teuthology.orchestra.run.smithi140:> sudo service rsyslog restart 2022-07-21T01:40:56.595 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2022-07-21T01:40:56.598 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2022-07-21T01:40:56.600 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2022-07-21T01:40:56.603 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2022-07-21T01:40:56.687 INFO:teuthology.run_tasks:Running task internal.timer... 2022-07-21T01:40:56.695 INFO:teuthology.task.internal:Starting timer... 2022-07-21T01:40:56.695 INFO:teuthology.run_tasks:Running task pcp... 2022-07-21T01:40:56.744 INFO:teuthology.run_tasks:Running task selinux... 2022-07-21T01:40:56.767 DEBUG:teuthology.task.selinux:Excluding smithi140: OS 'ubuntu' does not support SELinux 2022-07-21T01:40:56.768 DEBUG:teuthology.task.selinux:Excluding smithi152: OS 'ubuntu' does not support SELinux 2022-07-21T01:40:56.768 DEBUG:teuthology.task.selinux:Excluding smithi191: OS 'ubuntu' does not support SELinux 2022-07-21T01:40:56.768 DEBUG:teuthology.task.selinux:Excluding smithi192: OS 'ubuntu' does not support SELinux 2022-07-21T01:40:56.769 DEBUG:teuthology.task.selinux:Excluding smithi195: OS 'ubuntu' does not support SELinux 2022-07-21T01:40:56.769 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-07-21T01:40:56.769 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2022-07-21T01:40:56.770 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-07-21T01:40:56.770 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-07-21T01:40:56.793 DEBUG:teuthology.repo_utils:Setting repo remote to git://git.ceph.com/git/ceph-cm-ansible.git 2022-07-21T01:40:56.801 INFO:teuthology.repo_utils:Fetching main from origin 2022-07-21T01:40:56.845 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main to origin/main 2022-07-21T01:40:56.875 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-07-21T01:40:56.876 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi140.front.sepia.ceph.com,smithi152.front.sepia.ceph.com,smithi191.front.sepia.ceph.com,smithi192.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_main/cephlab.yml 2022-07-21T01:45:38.495 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi140.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2022-07-21T01:45:38.500 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi140.front.sepia.ceph.com' 2022-07-21T01:45:38.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-07-21T01:45:38.616 DEBUG:teuthology.orchestra.run.smithi140:> true 2022-07-21T01:45:38.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2022-07-21T01:45:38.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-07-21T01:45:38.872 DEBUG:teuthology.orchestra.run.smithi152:> true 2022-07-21T01:45:38.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2022-07-21T01:45:39.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-07-21T01:45:39.139 DEBUG:teuthology.orchestra.run.smithi191:> true 2022-07-21T01:45:39.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2022-07-21T01:45:39.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-07-21T01:45:39.364 DEBUG:teuthology.orchestra.run.smithi192:> true 2022-07-21T01:45:39.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2022-07-21T01:45:39.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-07-21T01:45:39.590 DEBUG:teuthology.orchestra.run.smithi195:> true 2022-07-21T01:45:39.713 INFO:teuthology.run_tasks:Running task clock... 2022-07-21T01:45:39.732 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-07-21T01:45:39.733 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-07-21T01:45:39.738 DEBUG:teuthology.orchestra.run.smithi140:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-07-21T01:45:39.742 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-07-21T01:45:39.743 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-07-21T01:45:39.747 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-07-21T01:45:39.747 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-07-21T01:45:39.751 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-07-21T01:45:39.752 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-07-21T01:45:39.755 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-07-21T01:45:39.755 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-07-21T01:45:39.771 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-07-21T01:45:39.772 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: Command line: ntpd -gq 2022-07-21T01:45:39.772 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: proto: precision = 0.090 usec (-23) 2022-07-21T01:45:39.773 INFO:teuthology.orchestra.run.smithi140.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-07-21T01:45:39.781 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-07-21T01:45:39.782 INFO:teuthology.orchestra.run.smithi140.stderr:restrict ::: KOD does nothing without LIMITED. 2022-07-21T01:45:39.784 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: restrict ::: KOD does nothing without LIMITED. 2022-07-21T01:45:39.784 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: Listen and drop on 0 v6wildcard [::]:123 2022-07-21T01:45:39.784 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-07-21T01:45:39.784 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: Listen normally on 2 lo 127.0.0.1:123 2022-07-21T01:45:39.785 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: Listen normally on 3 ens1f1 172.21.15.140:123 2022-07-21T01:45:39.785 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: Listen normally on 4 lo [::1]:123 2022-07-21T01:45:39.785 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: Listen normally on 5 ens1f1 [fe80::ae1f:6bff:fea5:ca31%5]:123 2022-07-21T01:45:39.786 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:39 ntpd[17875]: Listening on routing socket on fd #22 for interface updates 2022-07-21T01:45:39.787 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-07-21T01:45:39.787 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: Command line: ntpd -gq 2022-07-21T01:45:39.787 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: proto: precision = 0.090 usec (-23) 2022-07-21T01:45:39.788 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-07-21T01:45:39.788 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: Command line: ntpd -gq 2022-07-21T01:45:39.788 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: proto: precision = 0.090 usec (-23) 2022-07-21T01:45:39.789 INFO:teuthology.orchestra.run.smithi191.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-07-21T01:45:39.789 INFO:teuthology.orchestra.run.smithi152.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-07-21T01:45:39.790 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-07-21T01:45:39.791 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-07-21T01:45:39.791 INFO:teuthology.orchestra.run.smithi191.stderr:restrict ::: KOD does nothing without LIMITED. 2022-07-21T01:45:39.792 INFO:teuthology.orchestra.run.smithi152.stderr:restrict ::: KOD does nothing without LIMITED. 2022-07-21T01:45:39.795 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-07-21T01:45:39.796 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: Command line: ntpd -gq 2022-07-21T01:45:39.796 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: proto: precision = 0.246 usec (-22) 2022-07-21T01:45:39.797 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: restrict ::: KOD does nothing without LIMITED. 2022-07-21T01:45:39.797 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: Listen and drop on 0 v6wildcard [::]:123 2022-07-21T01:45:39.797 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-07-21T01:45:39.797 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: Listen normally on 2 lo 127.0.0.1:123 2022-07-21T01:45:39.798 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: Listen normally on 3 enp3s0f1 172.21.15.191:123 2022-07-21T01:45:39.798 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: Listen normally on 4 lo [::1]:123 2022-07-21T01:45:39.798 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7211%5]:123 2022-07-21T01:45:39.799 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:39 ntpd[18098]: Listening on routing socket on fd #22 for interface updates 2022-07-21T01:45:39.799 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: restrict ::: KOD does nothing without LIMITED. 2022-07-21T01:45:39.799 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: Listen and drop on 0 v6wildcard [::]:123 2022-07-21T01:45:39.799 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-07-21T01:45:39.800 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: Listen normally on 2 lo 127.0.0.1:123 2022-07-21T01:45:39.800 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: Listen normally on 3 enp3s0f1 172.21.15.152:123 2022-07-21T01:45:39.800 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: Listen normally on 4 lo [::1]:123 2022-07-21T01:45:39.800 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6de3%5]:123 2022-07-21T01:45:39.801 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:39 ntpd[18087]: Listening on routing socket on fd #22 for interface updates 2022-07-21T01:45:39.801 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-07-21T01:45:39.801 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: Command line: ntpd -gq 2022-07-21T01:45:39.802 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: proto: precision = 0.120 usec (-23) 2022-07-21T01:45:39.802 INFO:teuthology.orchestra.run.smithi195.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-07-21T01:45:39.802 INFO:teuthology.orchestra.run.smithi192.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-07-21T01:45:39.803 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-07-21T01:45:39.804 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-07-21T01:45:39.804 INFO:teuthology.orchestra.run.smithi195.stderr:restrict ::: KOD does nothing without LIMITED. 2022-07-21T01:45:39.805 INFO:teuthology.orchestra.run.smithi192.stderr:restrict ::: KOD does nothing without LIMITED. 2022-07-21T01:45:39.807 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: restrict ::: KOD does nothing without LIMITED. 2022-07-21T01:45:39.807 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: Listen and drop on 0 v6wildcard [::]:123 2022-07-21T01:45:39.807 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-07-21T01:45:39.808 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: Listen normally on 2 lo 127.0.0.1:123 2022-07-21T01:45:39.808 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: Listen normally on 3 enp3s0f1 172.21.15.195:123 2022-07-21T01:45:39.808 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: Listen normally on 4 lo [::1]:123 2022-07-21T01:45:39.808 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8081%5]:123 2022-07-21T01:45:39.809 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:39 ntpd[18139]: Listening on routing socket on fd #22 for interface updates 2022-07-21T01:45:39.809 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: restrict ::: KOD does nothing without LIMITED. 2022-07-21T01:45:39.809 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: Listen and drop on 0 v6wildcard [::]:123 2022-07-21T01:45:39.810 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-07-21T01:45:39.810 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: Listen normally on 2 lo 127.0.0.1:123 2022-07-21T01:45:39.810 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: Listen normally on 3 enp3s0f1 172.21.15.192:123 2022-07-21T01:45:39.810 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: Listen normally on 4 lo [::1]:123 2022-07-21T01:45:39.811 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8049%5]:123 2022-07-21T01:45:39.811 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:39 ntpd[18054]: Listening on routing socket on fd #22 for interface updates 2022-07-21T01:45:40.773 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:40.773 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:40.773 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:40.774 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:40.774 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:40.774 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:40.775 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:40.775 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:40.775 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:40.775 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:40.776 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:40.776 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:40 ntpd[17875]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:40.786 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:40 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:40.786 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:40 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:40.786 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:40 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:40.787 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:40 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:40.787 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:40 ntpd[18098]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:40.787 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:40 ntpd[18098]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:40.790 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:40 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:40.790 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:40 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:40.791 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:40 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:40.791 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:40 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:40.791 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:40 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:40.791 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:40 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:40.792 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:40 ntpd[18139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:40.792 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:40 ntpd[18139]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:40.793 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:40 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:40.793 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:40 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:40.794 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:40 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:40.794 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:40 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:40.794 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:40 ntpd[18054]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:40.795 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:40 ntpd[18054]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:41.772 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:41 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:41.772 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:41 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:41.777 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:41 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:41.777 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:41 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:41.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:41 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:41.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:41 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:41.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:41 ntpd[18087]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:41.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:41 ntpd[18087]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:42.775 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.776 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.776 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.777 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.777 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.778 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.778 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.779 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.779 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:42.780 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:42.780 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.780 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.781 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:42.781 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:42.782 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.782 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.783 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:42.784 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:42 ntpd[17875]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.787 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.788 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.789 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:42.790 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:42 ntpd[18098]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:42.793 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.794 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.794 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.794 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.794 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.795 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.795 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.795 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.795 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:42.795 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:42.796 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.796 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.796 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:42.796 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:42 ntpd[18054]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:42.797 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.798 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.798 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.798 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.799 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.799 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.799 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.799 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.800 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:42.800 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:42.800 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.800 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.801 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:42.801 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:42.801 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:42.801 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:42.801 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:42.802 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:42 ntpd[18139]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:43.779 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:43 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:43.779 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:43 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:43.780 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:43 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:43.780 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:43 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:43.780 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:43 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:43.780 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:43 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:43.781 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:43 ntpd[18087]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:43.781 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:43 ntpd[18087]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:43.791 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:43 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:43.792 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:43 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:43.792 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:43 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:43.792 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:43 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:43.792 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:43 ntpd[18054]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:43.793 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:43 ntpd[18054]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.776 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.776 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.777 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.777 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.778 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.778 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.779 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.779 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.780 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.780 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.781 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.781 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.781 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.782 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.782 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.782 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.790 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.790 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:44 ntpd[17875]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.799 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.800 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.800 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.800 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.801 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.801 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.801 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.802 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.802 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.802 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.803 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.803 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:44 ntpd[18054]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.804 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.805 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.805 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.805 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.806 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.806 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.806 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.806 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.807 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.807 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.807 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.807 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:44 ntpd[18087]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.809 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.809 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.809 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.810 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.810 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.810 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.810 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.811 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.811 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.811 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.811 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.812 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.812 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.812 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.812 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.813 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.813 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.813 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:44 ntpd[18139]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.813 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.814 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.814 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.814 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.814 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.815 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.815 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.815 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.815 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.816 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.816 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.817 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.817 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.817 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:44.817 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:44.818 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:44.818 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:44.818 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:44 ntpd[18098]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:45.777 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:45 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:45.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:45 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:45.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:45 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:45.779 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:45 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:45.779 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:45 ntpd[18087]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:45.779 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:45 ntpd[18087]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:45.792 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:45 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:45.792 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:45 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:45.793 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:45 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:45.793 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:45 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:45.794 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:45 ntpd[18054]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:45.794 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:45 ntpd[18054]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:46.772 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:46 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.773 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:46 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.773 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:46 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.773 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:46 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.774 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:46 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.774 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:46 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.774 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:46 ntpd[17875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.774 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:46 ntpd[17875]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.775 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:46 ntpd[17875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:46.775 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:46 ntpd[17875]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:46.775 INFO:teuthology.orchestra.run.smithi140.stdout:21 Jul 01:45:46 ntpd[17875]: ntpd: time slew -0.002641 s 2022-07-21T01:45:46.776 INFO:teuthology.orchestra.run.smithi140.stdout:ntpd: time slew -0.002641s 2022-07-21T01:45:46.776 INFO:teuthology.orchestra.run.smithi140.stderr:21 Jul 01:45:46 ntpd[17875]: can't open /var/log/ntpstats/loopstats.20220721: Permission denied 2022-07-21T01:45:46.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.779 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.779 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.779 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.779 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:46.779 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:46.780 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.780 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.780 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:46.781 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:46 ntpd[18087]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:46.788 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.788 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.789 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.789 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.789 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.790 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.790 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.790 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.791 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:46.791 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:46.791 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.792 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.792 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:46.792 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:46.793 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.793 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.793 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:46.793 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:46.794 INFO:teuthology.orchestra.run.smithi191.stdout:21 Jul 01:45:46 ntpd[18098]: ntpd: time slew +0.004944 s 2022-07-21T01:45:46.794 INFO:teuthology.orchestra.run.smithi191.stdout:ntpd: time slew +0.004944s 2022-07-21T01:45:46.795 INFO:teuthology.orchestra.run.smithi191.stderr:21 Jul 01:45:46 ntpd[18098]: can't open /var/log/ntpstats/loopstats.20220721: Permission denied 2022-07-21T01:45:46.797 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:46 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.797 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:46 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.798 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:46 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.798 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:46 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.798 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:46 ntpd[18054]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.798 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:46 ntpd[18054]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.799 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:46 ntpd[18054]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:46.799 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:46 ntpd[18054]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:46.800 INFO:teuthology.orchestra.run.smithi192.stdout:21 Jul 01:45:46 ntpd[18054]: ntpd: time slew -0.001086 s 2022-07-21T01:45:46.800 INFO:teuthology.orchestra.run.smithi192.stdout:ntpd: time slew -0.001086s 2022-07-21T01:45:46.800 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:46 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.800 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:46 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.801 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:46 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.801 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:46 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.801 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:46 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.802 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:46 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.802 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:46 ntpd[18139]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:46.802 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:46 ntpd[18139]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:46.802 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:46 ntpd[18139]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:46.803 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:46 ntpd[18139]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:46.803 INFO:teuthology.orchestra.run.smithi195.stdout:21 Jul 01:45:46 ntpd[18139]: ntpd: time slew +0.001039 s 2022-07-21T01:45:46.804 INFO:teuthology.orchestra.run.smithi195.stdout:ntpd: time slew +0.001039s 2022-07-21T01:45:46.804 INFO:teuthology.orchestra.run.smithi192.stderr:21 Jul 01:45:46 ntpd[18054]: can't open /var/log/ntpstats/loopstats.20220721: Permission denied 2022-07-21T01:45:46.804 INFO:teuthology.orchestra.run.smithi195.stderr:21 Jul 01:45:46 ntpd[18139]: can't open /var/log/ntpstats/loopstats.20220721: Permission denied 2022-07-21T01:45:46.826 INFO:teuthology.orchestra.run.smithi140.stdout: remote refid st t when poll reach delay offset jitter 2022-07-21T01:45:46.827 INFO:teuthology.orchestra.run.smithi140.stdout:============================================================================== 2022-07-21T01:45:46.827 INFO:teuthology.orchestra.run.smithi140.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.827 INFO:teuthology.orchestra.run.smithi140.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.828 INFO:teuthology.orchestra.run.smithi140.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.828 INFO:teuthology.orchestra.run.smithi140.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.842 INFO:teuthology.orchestra.run.smithi191.stdout: remote refid st t when poll reach delay offset jitter 2022-07-21T01:45:46.842 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================== 2022-07-21T01:45:46.843 INFO:teuthology.orchestra.run.smithi191.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.843 INFO:teuthology.orchestra.run.smithi191.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.843 INFO:teuthology.orchestra.run.smithi191.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.843 INFO:teuthology.orchestra.run.smithi191.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2022-07-21T01:45:46.848 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2022-07-21T01:45:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: remote refid st t when poll reach delay offset jitter 2022-07-21T01:45:46.851 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================== 2022-07-21T01:45:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.851 INFO:teuthology.orchestra.run.smithi192.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:46.852 INFO:teuthology.orchestra.run.smithi192.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:47.777 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:47 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:47.777 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:47 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:47.777 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:47 ntpd[18087]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-07-21T01:45:47.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:47 ntpd[18087]: can't open /var/log/ntpstats/rawstats.20220721: Permission denied 2022-07-21T01:45:47.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:47 ntpd[18087]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-07-21T01:45:47.778 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:47 ntpd[18087]: can't open /var/log/ntpstats/peerstats.20220721: Permission denied 2022-07-21T01:45:47.779 INFO:teuthology.orchestra.run.smithi152.stdout:21 Jul 01:45:47 ntpd[18087]: ntpd: time slew +0.004398 s 2022-07-21T01:45:47.779 INFO:teuthology.orchestra.run.smithi152.stdout:ntpd: time slew +0.004398s 2022-07-21T01:45:47.779 INFO:teuthology.orchestra.run.smithi152.stderr:21 Jul 01:45:47 ntpd[18087]: can't open /var/log/ntpstats/loopstats.20220721: Permission denied 2022-07-21T01:45:47.837 INFO:teuthology.orchestra.run.smithi152.stdout: remote refid st t when poll reach delay offset jitter 2022-07-21T01:45:47.837 INFO:teuthology.orchestra.run.smithi152.stdout:============================================================================== 2022-07-21T01:45:47.838 INFO:teuthology.orchestra.run.smithi152.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:47.838 INFO:teuthology.orchestra.run.smithi152.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:47.838 INFO:teuthology.orchestra.run.smithi152.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:47.838 INFO:teuthology.orchestra.run.smithi152.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-07-21T01:45:47.840 INFO:teuthology.run_tasks:Running task install... 2022-07-21T01:45:47.853 DEBUG:teuthology.task.install:project ceph 2022-07-21T01:45:47.854 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '57544da114dbbe1c44f27b2ea18e3233e4e917e0'}} 2022-07-21T01:45:47.854 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '57544da114dbbe1c44f27b2ea18e3233e4e917e0'} 2022-07-21T01:45:47.855 INFO:teuthology.task.install:Using flavor: default 2022-07-21T01:45:47.862 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-07-21T01:45:47.863 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2022-07-21T01:45:47.864 DEBUG:teuthology.orchestra.run.smithi140:> sudo lsb_release -is 2022-07-21T01:45:47.916 INFO:teuthology.orchestra.run.smithi140.stdout:Ubuntu 2022-07-21T01:45:47.922 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:45:47.923 DEBUG:teuthology.orchestra.run.smithi152:> sudo lsb_release -is 2022-07-21T01:45:47.925 DEBUG:teuthology.orchestra.run.smithi140:> sudo apt-key list | grep Ceph 2022-07-21T01:45:47.976 INFO:teuthology.orchestra.run.smithi152.stdout:Ubuntu 2022-07-21T01:45:47.981 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:45:47.982 DEBUG:teuthology.orchestra.run.smithi191:> sudo lsb_release -is 2022-07-21T01:45:47.983 DEBUG:teuthology.orchestra.run.smithi152:> sudo apt-key list | grep Ceph 2022-07-21T01:45:48.024 INFO:teuthology.orchestra.run.smithi140.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-07-21T01:45:48.035 INFO:teuthology.orchestra.run.smithi191.stdout:Ubuntu 2022-07-21T01:45:48.040 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:45:48.041 DEBUG:teuthology.orchestra.run.smithi192:> sudo lsb_release -is 2022-07-21T01:45:48.042 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-key list | grep Ceph 2022-07-21T01:45:48.080 INFO:teuthology.orchestra.run.smithi152.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-07-21T01:45:48.084 INFO:teuthology.orchestra.run.smithi140.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-07-21T01:45:48.084 INFO:teuthology.orchestra.run.smithi140.stdout:uid [ unknown] Ceph.com (release key) 2022-07-21T01:45:48.086 DEBUG:teuthology.orchestra.run.smithi140:> sudo lsb_release -is 2022-07-21T01:45:48.093 INFO:teuthology.orchestra.run.smithi192.stdout:Ubuntu 2022-07-21T01:45:48.099 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:45:48.099 DEBUG:teuthology.orchestra.run.smithi195:> sudo lsb_release -is 2022-07-21T01:45:48.100 DEBUG:teuthology.orchestra.run.smithi192:> sudo apt-key list | grep Ceph 2022-07-21T01:45:48.135 INFO:teuthology.orchestra.run.smithi140.stdout:Ubuntu 2022-07-21T01:45:48.138 INFO:teuthology.orchestra.run.smithi191.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-07-21T01:45:48.140 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:45:48.140 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-07-21T01:45:48.141 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-07-21T01:45:48.142 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=57544da114dbbe1c44f27b2ea18e3233e4e917e0 2022-07-21T01:45:48.148 INFO:teuthology.orchestra.run.smithi152.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-07-21T01:45:48.148 INFO:teuthology.orchestra.run.smithi152.stdout:uid [ unknown] Ceph.com (release key) 2022-07-21T01:45:48.149 DEBUG:teuthology.orchestra.run.smithi152:> sudo lsb_release -is 2022-07-21T01:45:48.150 INFO:teuthology.orchestra.run.smithi195.stdout:Ubuntu 2022-07-21T01:45:48.154 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:45:48.154 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-key list | grep Ceph 2022-07-21T01:45:48.193 INFO:teuthology.orchestra.run.smithi191.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-07-21T01:45:48.194 INFO:teuthology.orchestra.run.smithi191.stdout:uid [ unknown] Ceph.com (release key) 2022-07-21T01:45:48.196 DEBUG:teuthology.orchestra.run.smithi191:> sudo lsb_release -is 2022-07-21T01:45:48.198 INFO:teuthology.orchestra.run.smithi192.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-07-21T01:45:48.199 INFO:teuthology.orchestra.run.smithi152.stdout:Ubuntu 2022-07-21T01:45:48.204 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:45:48.204 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-07-21T01:45:48.204 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-07-21T01:45:48.205 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=57544da114dbbe1c44f27b2ea18e3233e4e917e0 2022-07-21T01:45:48.244 INFO:teuthology.orchestra.run.smithi191.stdout:Ubuntu 2022-07-21T01:45:48.249 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:45:48.250 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-07-21T01:45:48.250 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-07-21T01:45:48.251 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=57544da114dbbe1c44f27b2ea18e3233e4e917e0 2022-07-21T01:45:48.254 INFO:teuthology.orchestra.run.smithi192.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-07-21T01:45:48.255 INFO:teuthology.orchestra.run.smithi192.stdout:uid [ unknown] Ceph.com (release key) 2022-07-21T01:45:48.257 DEBUG:teuthology.orchestra.run.smithi192:> sudo lsb_release -is 2022-07-21T01:45:48.295 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-07-21T01:45:48.305 INFO:teuthology.orchestra.run.smithi192.stdout:Ubuntu 2022-07-21T01:45:48.311 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:45:48.312 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-07-21T01:45:48.312 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-07-21T01:45:48.313 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=57544da114dbbe1c44f27b2ea18e3233e4e917e0 2022-07-21T01:45:48.336 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default/ 2022-07-21T01:45:48.337 INFO:teuthology.task.install.deb:Package version is 15.2.16-336-g57544da1-1focal 2022-07-21T01:45:48.358 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-07-21T01:45:48.358 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph.com (release key) 2022-07-21T01:45:48.359 DEBUG:teuthology.orchestra.run.smithi195:> sudo lsb_release -is 2022-07-21T01:45:48.377 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default/ 2022-07-21T01:45:48.378 INFO:teuthology.task.install.deb:Package version is 15.2.16-336-g57544da1-1focal 2022-07-21T01:45:48.412 INFO:teuthology.orchestra.run.smithi195.stdout:Ubuntu 2022-07-21T01:45:48.413 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:45:48.413 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2022-07-21T01:45:48.413 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2022-07-21T01:45:48.414 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=57544da114dbbe1c44f27b2ea18e3233e4e917e0 2022-07-21T01:45:48.421 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default/ 2022-07-21T01:45:48.421 INFO:teuthology.task.install.deb:Package version is 15.2.16-336-g57544da1-1focal 2022-07-21T01:45:48.478 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default/ 2022-07-21T01:45:48.479 INFO:teuthology.task.install.deb:Package version is 15.2.16-336-g57544da1-1focal 2022-07-21T01:45:48.485 DEBUG:teuthology.orchestra.run.smithi140:> set -ex 2022-07-21T01:45:48.485 DEBUG:teuthology.orchestra.run.smithi140:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-07-21T01:45:48.497 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2022-07-21T01:45:48.498 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-07-21T01:45:48.503 DEBUG:teuthology.orchestra.run.smithi140:> sudo apt-get update 2022-07-21T01:45:48.515 DEBUG:teuthology.orchestra.run.smithi152:> sudo apt-get update 2022-07-21T01:45:48.561 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default/ 2022-07-21T01:45:48.561 INFO:teuthology.task.install.deb:Package version is 15.2.16-336-g57544da1-1focal 2022-07-21T01:45:48.568 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-07-21T01:45:48.568 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-07-21T01:45:48.586 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get update 2022-07-21T01:45:48.622 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2022-07-21T01:45:48.622 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-07-21T01:45:48.640 DEBUG:teuthology.orchestra.run.smithi192:> sudo apt-get update 2022-07-21T01:45:48.683 INFO:teuthology.orchestra.run.smithi140.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-07-21T01:45:48.696 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2022-07-21T01:45:48.697 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-07-21T01:45:48.699 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-07-21T01:45:48.699 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-07-21T01:45:48.704 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-07-21T01:45:48.714 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2022-07-21T01:45:48.721 INFO:teuthology.orchestra.run.smithi152.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-07-21T01:45:48.727 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-07-21T01:45:48.728 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-07-21T01:45:48.731 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-07-21T01:45:48.748 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-07-21T01:45:48.799 INFO:teuthology.orchestra.run.smithi140.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal InRelease 2022-07-21T01:45:48.800 INFO:teuthology.orchestra.run.smithi140.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease 2022-07-21T01:45:48.808 INFO:teuthology.orchestra.run.smithi152.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal InRelease 2022-07-21T01:45:48.818 INFO:teuthology.orchestra.run.smithi192.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-07-21T01:45:48.819 INFO:teuthology.orchestra.run.smithi192.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-07-21T01:45:48.820 INFO:teuthology.orchestra.run.smithi191.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal InRelease 2022-07-21T01:45:48.827 INFO:teuthology.orchestra.run.smithi192.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-07-21T01:45:48.828 INFO:teuthology.orchestra.run.smithi140.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-07-21T01:45:48.845 INFO:teuthology.orchestra.run.smithi192.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-07-21T01:45:48.846 INFO:teuthology.orchestra.run.smithi152.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-07-21T01:45:48.846 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-07-21T01:45:48.852 INFO:teuthology.orchestra.run.smithi140.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal Release.gpg 2022-07-21T01:45:48.871 INFO:teuthology.orchestra.run.smithi152.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal Release.gpg 2022-07-21T01:45:48.875 INFO:teuthology.orchestra.run.smithi191.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal Release.gpg 2022-07-21T01:45:48.877 INFO:teuthology.orchestra.run.smithi140.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main i386 Packages [3,078 B] 2022-07-21T01:45:48.887 INFO:teuthology.orchestra.run.smithi140.stdout:Hit:7 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-07-21T01:45:48.898 INFO:teuthology.orchestra.run.smithi152.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main i386 Packages [3,078 B] 2022-07-21T01:45:48.899 INFO:teuthology.orchestra.run.smithi192.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal InRelease 2022-07-21T01:45:48.899 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-07-21T01:45:48.903 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main i386 Packages [3,078 B] 2022-07-21T01:45:48.915 INFO:teuthology.orchestra.run.smithi140.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-07-21T01:45:48.925 INFO:teuthology.orchestra.run.smithi192.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-07-21T01:45:48.934 INFO:teuthology.orchestra.run.smithi152.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-07-21T01:45:48.943 INFO:teuthology.orchestra.run.smithi191.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-07-21T01:45:48.956 INFO:teuthology.orchestra.run.smithi192.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal Release.gpg 2022-07-21T01:45:48.966 INFO:teuthology.orchestra.run.smithi140.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-07-21T01:45:48.974 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal InRelease 2022-07-21T01:45:48.981 INFO:teuthology.orchestra.run.smithi192.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-07-21T01:45:49.000 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-07-21T01:45:49.012 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal InRelease 2022-07-21T01:45:49.018 INFO:teuthology.orchestra.run.smithi192.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main i386 Packages [3,078 B] 2022-07-21T01:45:49.030 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal Release.gpg 2022-07-21T01:45:49.066 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main i386 Packages [3,078 B] 2022-07-21T01:45:49.091 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2022-07-21T01:45:49.092 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-07-21T01:45:49.172 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-07-21T01:45:49.316 INFO:teuthology.orchestra.run.smithi140.stdout:Fetched 26.6 kB in 1s (43.4 kB/s) 2022-07-21T01:45:49.346 INFO:teuthology.orchestra.run.smithi152.stdout:Fetched 26.6 kB in 1s (42.8 kB/s) 2022-07-21T01:45:49.385 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 26.6 kB in 1s (41.9 kB/s) 2022-07-21T01:45:49.457 INFO:teuthology.orchestra.run.smithi192.stdout:Fetched 26.6 kB in 1s (43.3 kB/s) 2022-07-21T01:45:49.538 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 26.6 kB in 1s (42.3 kB/s) 2022-07-21T01:45:50.204 INFO:teuthology.orchestra.run.smithi140.stdout:Reading package lists... 2022-07-21T01:45:50.227 DEBUG:teuthology.orchestra.run.smithi140:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-336-g57544da1-1focal cephadm=15.2.16-336-g57544da1-1focal ceph-mds=15.2.16-336-g57544da1-1focal ceph-mgr=15.2.16-336-g57544da1-1focal ceph-common=15.2.16-336-g57544da1-1focal ceph-fuse=15.2.16-336-g57544da1-1focal ceph-test=15.2.16-336-g57544da1-1focal radosgw=15.2.16-336-g57544da1-1focal python3-rados=15.2.16-336-g57544da1-1focal python3-rgw=15.2.16-336-g57544da1-1focal python3-cephfs=15.2.16-336-g57544da1-1focal python3-rbd=15.2.16-336-g57544da1-1focal libcephfs2=15.2.16-336-g57544da1-1focal libcephfs-dev=15.2.16-336-g57544da1-1focal librados2=15.2.16-336-g57544da1-1focal librbd1=15.2.16-336-g57544da1-1focal rbd-fuse=15.2.16-336-g57544da1-1focal python3-cephfs=15.2.16-336-g57544da1-1focal cephfs-shell=15.2.16-336-g57544da1-1focal 2022-07-21T01:45:50.237 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2022-07-21T01:45:50.259 DEBUG:teuthology.orchestra.run.smithi152:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-336-g57544da1-1focal cephadm=15.2.16-336-g57544da1-1focal ceph-mds=15.2.16-336-g57544da1-1focal ceph-mgr=15.2.16-336-g57544da1-1focal ceph-common=15.2.16-336-g57544da1-1focal ceph-fuse=15.2.16-336-g57544da1-1focal ceph-test=15.2.16-336-g57544da1-1focal radosgw=15.2.16-336-g57544da1-1focal python3-rados=15.2.16-336-g57544da1-1focal python3-rgw=15.2.16-336-g57544da1-1focal python3-cephfs=15.2.16-336-g57544da1-1focal python3-rbd=15.2.16-336-g57544da1-1focal libcephfs2=15.2.16-336-g57544da1-1focal libcephfs-dev=15.2.16-336-g57544da1-1focal librados2=15.2.16-336-g57544da1-1focal librbd1=15.2.16-336-g57544da1-1focal rbd-fuse=15.2.16-336-g57544da1-1focal python3-cephfs=15.2.16-336-g57544da1-1focal cephfs-shell=15.2.16-336-g57544da1-1focal 2022-07-21T01:45:50.269 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2022-07-21T01:45:50.291 DEBUG:teuthology.orchestra.run.smithi191:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-336-g57544da1-1focal cephadm=15.2.16-336-g57544da1-1focal ceph-mds=15.2.16-336-g57544da1-1focal ceph-mgr=15.2.16-336-g57544da1-1focal ceph-common=15.2.16-336-g57544da1-1focal ceph-fuse=15.2.16-336-g57544da1-1focal ceph-test=15.2.16-336-g57544da1-1focal radosgw=15.2.16-336-g57544da1-1focal python3-rados=15.2.16-336-g57544da1-1focal python3-rgw=15.2.16-336-g57544da1-1focal python3-cephfs=15.2.16-336-g57544da1-1focal python3-rbd=15.2.16-336-g57544da1-1focal libcephfs2=15.2.16-336-g57544da1-1focal libcephfs-dev=15.2.16-336-g57544da1-1focal librados2=15.2.16-336-g57544da1-1focal librbd1=15.2.16-336-g57544da1-1focal rbd-fuse=15.2.16-336-g57544da1-1focal python3-cephfs=15.2.16-336-g57544da1-1focal cephfs-shell=15.2.16-336-g57544da1-1focal 2022-07-21T01:45:50.294 INFO:teuthology.orchestra.run.smithi140.stdout:Reading package lists... 2022-07-21T01:45:50.326 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2022-07-21T01:45:50.334 INFO:teuthology.orchestra.run.smithi192.stdout:Reading package lists... 2022-07-21T01:45:50.355 DEBUG:teuthology.orchestra.run.smithi192:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-336-g57544da1-1focal cephadm=15.2.16-336-g57544da1-1focal ceph-mds=15.2.16-336-g57544da1-1focal ceph-mgr=15.2.16-336-g57544da1-1focal ceph-common=15.2.16-336-g57544da1-1focal ceph-fuse=15.2.16-336-g57544da1-1focal ceph-test=15.2.16-336-g57544da1-1focal radosgw=15.2.16-336-g57544da1-1focal python3-rados=15.2.16-336-g57544da1-1focal python3-rgw=15.2.16-336-g57544da1-1focal python3-cephfs=15.2.16-336-g57544da1-1focal python3-rbd=15.2.16-336-g57544da1-1focal libcephfs2=15.2.16-336-g57544da1-1focal libcephfs-dev=15.2.16-336-g57544da1-1focal librados2=15.2.16-336-g57544da1-1focal librbd1=15.2.16-336-g57544da1-1focal rbd-fuse=15.2.16-336-g57544da1-1focal python3-cephfs=15.2.16-336-g57544da1-1focal cephfs-shell=15.2.16-336-g57544da1-1focal 2022-07-21T01:45:50.361 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2022-07-21T01:45:50.422 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2022-07-21T01:45:50.426 INFO:teuthology.orchestra.run.smithi192.stdout:Reading package lists... 2022-07-21T01:45:50.445 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.16-336-g57544da1-1focal cephadm=15.2.16-336-g57544da1-1focal ceph-mds=15.2.16-336-g57544da1-1focal ceph-mgr=15.2.16-336-g57544da1-1focal ceph-common=15.2.16-336-g57544da1-1focal ceph-fuse=15.2.16-336-g57544da1-1focal ceph-test=15.2.16-336-g57544da1-1focal radosgw=15.2.16-336-g57544da1-1focal python3-rados=15.2.16-336-g57544da1-1focal python3-rgw=15.2.16-336-g57544da1-1focal python3-cephfs=15.2.16-336-g57544da1-1focal python3-rbd=15.2.16-336-g57544da1-1focal libcephfs2=15.2.16-336-g57544da1-1focal libcephfs-dev=15.2.16-336-g57544da1-1focal librados2=15.2.16-336-g57544da1-1focal librbd1=15.2.16-336-g57544da1-1focal rbd-fuse=15.2.16-336-g57544da1-1focal python3-cephfs=15.2.16-336-g57544da1-1focal cephfs-shell=15.2.16-336-g57544da1-1focal 2022-07-21T01:45:50.477 INFO:teuthology.orchestra.run.smithi140.stdout:Building dependency tree... 2022-07-21T01:45:50.477 INFO:teuthology.orchestra.run.smithi140.stdout:Reading state information... 2022-07-21T01:45:50.482 INFO:teuthology.orchestra.run.smithi152.stdout:Building dependency tree... 2022-07-21T01:45:50.483 INFO:teuthology.orchestra.run.smithi152.stdout:Reading state information... 2022-07-21T01:45:50.513 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2022-07-21T01:45:50.537 INFO:teuthology.orchestra.run.smithi191.stdout:Building dependency tree... 2022-07-21T01:45:50.538 INFO:teuthology.orchestra.run.smithi191.stdout:Reading state information... 2022-07-21T01:45:50.604 INFO:teuthology.orchestra.run.smithi192.stdout:Building dependency tree... 2022-07-21T01:45:50.604 INFO:teuthology.orchestra.run.smithi192.stdout:Reading state information... 2022-07-21T01:45:50.621 INFO:teuthology.orchestra.run.smithi152.stdout:The following packages were automatically installed and are no longer required: 2022-07-21T01:45:50.624 INFO:teuthology.orchestra.run.smithi152.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-07-21T01:45:50.625 INFO:teuthology.orchestra.run.smithi152.stdout:Use 'sudo apt autoremove' to remove them. 2022-07-21T01:45:50.625 INFO:teuthology.orchestra.run.smithi152.stdout:The following additional packages will be installed: 2022-07-21T01:45:50.625 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-07-21T01:45:50.625 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-07-21T01:45:50.626 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-07-21T01:45:50.626 INFO:teuthology.orchestra.run.smithi152.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-07-21T01:45:50.626 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-07-21T01:45:50.627 INFO:teuthology.orchestra.run.smithi152.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-07-21T01:45:50.627 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-07-21T01:45:50.627 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-07-21T01:45:50.627 INFO:teuthology.orchestra.run.smithi152.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-07-21T01:45:50.628 INFO:teuthology.orchestra.run.smithi152.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-07-21T01:45:50.628 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-07-21T01:45:50.628 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-07-21T01:45:50.628 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-07-21T01:45:50.629 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-07-21T01:45:50.629 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-07-21T01:45:50.629 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-07-21T01:45:50.630 INFO:teuthology.orchestra.run.smithi152.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-07-21T01:45:50.630 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-07-21T01:45:50.630 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-07-21T01:45:50.631 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet 2022-07-21T01:45:50.631 INFO:teuthology.orchestra.run.smithi152.stdout:Suggested packages: 2022-07-21T01:45:50.632 INFO:teuthology.orchestra.run.smithi152.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-07-21T01:45:50.632 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-07-21T01:45:50.632 INFO:teuthology.orchestra.run.smithi152.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-07-21T01:45:50.632 INFO:teuthology.orchestra.run.smithi152.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-07-21T01:45:50.633 INFO:teuthology.orchestra.run.smithi152.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-07-21T01:45:50.633 INFO:teuthology.orchestra.run.smithi152.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-07-21T01:45:50.633 INFO:teuthology.orchestra.run.smithi152.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-07-21T01:45:50.633 INFO:teuthology.orchestra.run.smithi152.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-07-21T01:45:50.634 INFO:teuthology.orchestra.run.smithi152.stdout:Recommended packages: 2022-07-21T01:45:50.634 INFO:teuthology.orchestra.run.smithi152.stdout: btrfs-tools 2022-07-21T01:45:50.635 INFO:teuthology.orchestra.run.smithi140.stdout:The following packages were automatically installed and are no longer required: 2022-07-21T01:45:50.636 INFO:teuthology.orchestra.run.smithi140.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-07-21T01:45:50.636 INFO:teuthology.orchestra.run.smithi140.stdout:Use 'sudo apt autoremove' to remove them. 2022-07-21T01:45:50.636 INFO:teuthology.orchestra.run.smithi140.stdout:The following additional packages will be installed: 2022-07-21T01:45:50.636 INFO:teuthology.orchestra.run.smithi140.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-07-21T01:45:50.637 INFO:teuthology.orchestra.run.smithi140.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-07-21T01:45:50.637 INFO:teuthology.orchestra.run.smithi140.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-07-21T01:45:50.637 INFO:teuthology.orchestra.run.smithi140.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-07-21T01:45:50.637 INFO:teuthology.orchestra.run.smithi140.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-07-21T01:45:50.638 INFO:teuthology.orchestra.run.smithi140.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-07-21T01:45:50.638 INFO:teuthology.orchestra.run.smithi140.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-07-21T01:45:50.638 INFO:teuthology.orchestra.run.smithi140.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-07-21T01:45:50.639 INFO:teuthology.orchestra.run.smithi140.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-07-21T01:45:50.639 INFO:teuthology.orchestra.run.smithi140.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-07-21T01:45:50.639 INFO:teuthology.orchestra.run.smithi140.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-07-21T01:45:50.639 INFO:teuthology.orchestra.run.smithi140.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-07-21T01:45:50.639 INFO:teuthology.orchestra.run.smithi140.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-07-21T01:45:50.640 INFO:teuthology.orchestra.run.smithi140.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-07-21T01:45:50.640 INFO:teuthology.orchestra.run.smithi140.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-07-21T01:45:50.640 INFO:teuthology.orchestra.run.smithi140.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-07-21T01:45:50.640 INFO:teuthology.orchestra.run.smithi140.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-07-21T01:45:50.641 INFO:teuthology.orchestra.run.smithi140.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-07-21T01:45:50.641 INFO:teuthology.orchestra.run.smithi140.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-07-21T01:45:50.641 INFO:teuthology.orchestra.run.smithi140.stdout: xmlstarlet 2022-07-21T01:45:50.642 INFO:teuthology.orchestra.run.smithi140.stdout:Suggested packages: 2022-07-21T01:45:50.643 INFO:teuthology.orchestra.run.smithi140.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-07-21T01:45:50.643 INFO:teuthology.orchestra.run.smithi140.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-07-21T01:45:50.643 INFO:teuthology.orchestra.run.smithi140.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-07-21T01:45:50.643 INFO:teuthology.orchestra.run.smithi140.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-07-21T01:45:50.643 INFO:teuthology.orchestra.run.smithi140.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-07-21T01:45:50.644 INFO:teuthology.orchestra.run.smithi140.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-07-21T01:45:50.644 INFO:teuthology.orchestra.run.smithi140.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-07-21T01:45:50.644 INFO:teuthology.orchestra.run.smithi140.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-07-21T01:45:50.645 INFO:teuthology.orchestra.run.smithi140.stdout:Recommended packages: 2022-07-21T01:45:50.645 INFO:teuthology.orchestra.run.smithi140.stdout: btrfs-tools 2022-07-21T01:45:50.679 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages were automatically installed and are no longer required: 2022-07-21T01:45:50.680 INFO:teuthology.orchestra.run.smithi191.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-07-21T01:45:50.680 INFO:teuthology.orchestra.run.smithi191.stdout:Use 'sudo apt autoremove' to remove them. 2022-07-21T01:45:50.680 INFO:teuthology.orchestra.run.smithi191.stdout:The following additional packages will be installed: 2022-07-21T01:45:50.681 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-07-21T01:45:50.681 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-07-21T01:45:50.683 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-07-21T01:45:50.684 INFO:teuthology.orchestra.run.smithi191.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-07-21T01:45:50.684 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-07-21T01:45:50.684 INFO:teuthology.orchestra.run.smithi191.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-07-21T01:45:50.685 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-07-21T01:45:50.685 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-07-21T01:45:50.685 INFO:teuthology.orchestra.run.smithi191.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-07-21T01:45:50.685 INFO:teuthology.orchestra.run.smithi191.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-07-21T01:45:50.686 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-07-21T01:45:50.686 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-07-21T01:45:50.686 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-07-21T01:45:50.687 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-07-21T01:45:50.687 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-07-21T01:45:50.687 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-07-21T01:45:50.688 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-07-21T01:45:50.688 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-07-21T01:45:50.688 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-07-21T01:45:50.689 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet 2022-07-21T01:45:50.689 INFO:teuthology.orchestra.run.smithi191.stdout:Suggested packages: 2022-07-21T01:45:50.689 INFO:teuthology.orchestra.run.smithi191.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-07-21T01:45:50.689 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-07-21T01:45:50.690 INFO:teuthology.orchestra.run.smithi191.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-07-21T01:45:50.690 INFO:teuthology.orchestra.run.smithi191.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-07-21T01:45:50.690 INFO:teuthology.orchestra.run.smithi191.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-07-21T01:45:50.691 INFO:teuthology.orchestra.run.smithi191.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-07-21T01:45:50.691 INFO:teuthology.orchestra.run.smithi191.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-07-21T01:45:50.692 INFO:teuthology.orchestra.run.smithi191.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-07-21T01:45:50.692 INFO:teuthology.orchestra.run.smithi191.stdout:Recommended packages: 2022-07-21T01:45:50.692 INFO:teuthology.orchestra.run.smithi191.stdout: btrfs-tools 2022-07-21T01:45:50.693 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2022-07-21T01:45:50.694 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2022-07-21T01:45:50.740 INFO:teuthology.orchestra.run.smithi152.stdout:The following NEW packages will be installed: 2022-07-21T01:45:50.740 INFO:teuthology.orchestra.run.smithi152.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-07-21T01:45:50.741 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-07-21T01:45:50.741 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-07-21T01:45:50.742 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-07-21T01:45:50.742 INFO:teuthology.orchestra.run.smithi152.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-07-21T01:45:50.742 INFO:teuthology.orchestra.run.smithi152.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-07-21T01:45:50.743 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-07-21T01:45:50.744 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-07-21T01:45:50.744 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-07-21T01:45:50.745 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-07-21T01:45:50.745 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-07-21T01:45:50.746 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-07-21T01:45:50.747 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-07-21T01:45:50.747 INFO:teuthology.orchestra.run.smithi152.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-07-21T01:45:50.747 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-07-21T01:45:50.748 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-07-21T01:45:50.749 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-07-21T01:45:50.749 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-07-21T01:45:50.750 INFO:teuthology.orchestra.run.smithi152.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-07-21T01:45:50.750 INFO:teuthology.orchestra.run.smithi152.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-07-21T01:45:50.751 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-07-21T01:45:50.751 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-07-21T01:45:50.752 INFO:teuthology.orchestra.run.smithi152.stdout:The following packages will be upgraded: 2022-07-21T01:45:50.752 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 librbd1 2022-07-21T01:45:50.755 INFO:teuthology.orchestra.run.smithi140.stdout:The following NEW packages will be installed: 2022-07-21T01:45:50.756 INFO:teuthology.orchestra.run.smithi140.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-07-21T01:45:50.756 INFO:teuthology.orchestra.run.smithi140.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-07-21T01:45:50.756 INFO:teuthology.orchestra.run.smithi140.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-07-21T01:45:50.756 INFO:teuthology.orchestra.run.smithi140.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-07-21T01:45:50.757 INFO:teuthology.orchestra.run.smithi140.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-07-21T01:45:50.757 INFO:teuthology.orchestra.run.smithi140.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-07-21T01:45:50.757 INFO:teuthology.orchestra.run.smithi140.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-07-21T01:45:50.757 INFO:teuthology.orchestra.run.smithi140.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-07-21T01:45:50.758 INFO:teuthology.orchestra.run.smithi140.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-07-21T01:45:50.758 INFO:teuthology.orchestra.run.smithi140.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-07-21T01:45:50.758 INFO:teuthology.orchestra.run.smithi140.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-07-21T01:45:50.758 INFO:teuthology.orchestra.run.smithi140.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-07-21T01:45:50.759 INFO:teuthology.orchestra.run.smithi140.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-07-21T01:45:50.759 INFO:teuthology.orchestra.run.smithi140.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-07-21T01:45:50.759 INFO:teuthology.orchestra.run.smithi140.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-07-21T01:45:50.760 INFO:teuthology.orchestra.run.smithi140.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-07-21T01:45:50.760 INFO:teuthology.orchestra.run.smithi140.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-07-21T01:45:50.760 INFO:teuthology.orchestra.run.smithi140.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-07-21T01:45:50.761 INFO:teuthology.orchestra.run.smithi140.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-07-21T01:45:50.761 INFO:teuthology.orchestra.run.smithi140.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-07-21T01:45:50.761 INFO:teuthology.orchestra.run.smithi140.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-07-21T01:45:50.761 INFO:teuthology.orchestra.run.smithi140.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-07-21T01:45:50.762 INFO:teuthology.orchestra.run.smithi140.stdout:The following packages will be upgraded: 2022-07-21T01:45:50.762 INFO:teuthology.orchestra.run.smithi140.stdout: librados2 librbd1 2022-07-21T01:45:50.764 INFO:teuthology.orchestra.run.smithi192.stdout:The following packages were automatically installed and are no longer required: 2022-07-21T01:45:50.765 INFO:teuthology.orchestra.run.smithi192.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-07-21T01:45:50.765 INFO:teuthology.orchestra.run.smithi192.stdout:Use 'sudo apt autoremove' to remove them. 2022-07-21T01:45:50.765 INFO:teuthology.orchestra.run.smithi192.stdout:The following additional packages will be installed: 2022-07-21T01:45:50.766 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-07-21T01:45:50.766 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-07-21T01:45:50.766 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-07-21T01:45:50.766 INFO:teuthology.orchestra.run.smithi192.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-07-21T01:45:50.767 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-07-21T01:45:50.767 INFO:teuthology.orchestra.run.smithi192.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-07-21T01:45:50.767 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-07-21T01:45:50.767 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-07-21T01:45:50.768 INFO:teuthology.orchestra.run.smithi192.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-07-21T01:45:50.768 INFO:teuthology.orchestra.run.smithi192.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-07-21T01:45:50.768 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-07-21T01:45:50.768 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-07-21T01:45:50.769 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-07-21T01:45:50.769 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-07-21T01:45:50.769 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-07-21T01:45:50.770 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-07-21T01:45:50.771 INFO:teuthology.orchestra.run.smithi192.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-07-21T01:45:50.771 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-07-21T01:45:50.771 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-07-21T01:45:50.771 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet 2022-07-21T01:45:50.772 INFO:teuthology.orchestra.run.smithi192.stdout:Suggested packages: 2022-07-21T01:45:50.772 INFO:teuthology.orchestra.run.smithi192.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-07-21T01:45:50.773 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-07-21T01:45:50.773 INFO:teuthology.orchestra.run.smithi192.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-07-21T01:45:50.773 INFO:teuthology.orchestra.run.smithi192.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-07-21T01:45:50.773 INFO:teuthology.orchestra.run.smithi192.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-07-21T01:45:50.774 INFO:teuthology.orchestra.run.smithi192.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-07-21T01:45:50.774 INFO:teuthology.orchestra.run.smithi192.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-07-21T01:45:50.774 INFO:teuthology.orchestra.run.smithi192.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-07-21T01:45:50.775 INFO:teuthology.orchestra.run.smithi192.stdout:Recommended packages: 2022-07-21T01:45:50.775 INFO:teuthology.orchestra.run.smithi192.stdout: btrfs-tools 2022-07-21T01:45:50.786 INFO:teuthology.orchestra.run.smithi191.stdout:The following NEW packages will be installed: 2022-07-21T01:45:50.786 INFO:teuthology.orchestra.run.smithi191.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-07-21T01:45:50.787 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-07-21T01:45:50.787 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-07-21T01:45:50.788 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-07-21T01:45:50.788 INFO:teuthology.orchestra.run.smithi191.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-07-21T01:45:50.789 INFO:teuthology.orchestra.run.smithi191.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-07-21T01:45:50.789 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-07-21T01:45:50.789 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-07-21T01:45:50.789 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-07-21T01:45:50.790 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-07-21T01:45:50.790 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-07-21T01:45:50.790 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-07-21T01:45:50.791 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-07-21T01:45:50.791 INFO:teuthology.orchestra.run.smithi191.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-07-21T01:45:50.791 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-07-21T01:45:50.791 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-07-21T01:45:50.792 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-07-21T01:45:50.792 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-07-21T01:45:50.792 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-07-21T01:45:50.793 INFO:teuthology.orchestra.run.smithi191.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-07-21T01:45:50.793 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-07-21T01:45:50.793 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-07-21T01:45:50.793 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages will be upgraded: 2022-07-21T01:45:50.794 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 librbd1 2022-07-21T01:45:50.798 INFO:teuthology.orchestra.run.smithi152.stdout:2 upgraded, 101 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:45:50.799 INFO:teuthology.orchestra.run.smithi152.stdout:Need to get 153 MB of archives. 2022-07-21T01:45:50.799 INFO:teuthology.orchestra.run.smithi152.stdout:After this operation, 699 MB of additional disk space will be used. 2022-07-21T01:45:50.799 INFO:teuthology.orchestra.run.smithi152.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-07-21T01:45:50.806 INFO:teuthology.orchestra.run.smithi140.stdout:2 upgraded, 101 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:45:50.807 INFO:teuthology.orchestra.run.smithi140.stdout:Need to get 153 MB of archives. 2022-07-21T01:45:50.807 INFO:teuthology.orchestra.run.smithi140.stdout:After this operation, 699 MB of additional disk space will be used. 2022-07-21T01:45:50.808 INFO:teuthology.orchestra.run.smithi140.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-07-21T01:45:50.817 INFO:teuthology.orchestra.run.smithi152.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-07-21T01:45:50.824 INFO:teuthology.orchestra.run.smithi140.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-07-21T01:45:50.825 INFO:teuthology.orchestra.run.smithi152.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-07-21T01:45:50.832 INFO:teuthology.orchestra.run.smithi140.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-07-21T01:45:50.842 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2022-07-21T01:45:50.844 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-07-21T01:45:50.844 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2022-07-21T01:45:50.844 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2022-07-21T01:45:50.845 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-07-21T01:45:50.845 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-07-21T01:45:50.845 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-07-21T01:45:50.846 INFO:teuthology.orchestra.run.smithi195.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-07-21T01:45:50.846 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2022-07-21T01:45:50.846 INFO:teuthology.orchestra.run.smithi195.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2022-07-21T01:45:50.847 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2022-07-21T01:45:50.847 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2022-07-21T01:45:50.847 INFO:teuthology.orchestra.run.smithi195.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-07-21T01:45:50.848 INFO:teuthology.orchestra.run.smithi195.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-07-21T01:45:50.848 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-07-21T01:45:50.848 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-07-21T01:45:50.848 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-07-21T01:45:50.849 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2022-07-21T01:45:50.849 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2022-07-21T01:45:50.849 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2022-07-21T01:45:50.850 INFO:teuthology.orchestra.run.smithi195.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2022-07-21T01:45:50.850 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2022-07-21T01:45:50.850 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2022-07-21T01:45:50.851 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet 2022-07-21T01:45:50.852 INFO:teuthology.orchestra.run.smithi191.stdout:2 upgraded, 101 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:45:50.852 INFO:teuthology.orchestra.run.smithi191.stdout:Need to get 153 MB of archives. 2022-07-21T01:45:50.852 INFO:teuthology.orchestra.run.smithi191.stdout:After this operation, 699 MB of additional disk space will be used. 2022-07-21T01:45:50.853 INFO:teuthology.orchestra.run.smithi191.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-07-21T01:45:50.853 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2022-07-21T01:45:50.853 INFO:teuthology.orchestra.run.smithi195.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-07-21T01:45:50.854 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-07-21T01:45:50.854 INFO:teuthology.orchestra.run.smithi195.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-07-21T01:45:50.854 INFO:teuthology.orchestra.run.smithi195.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-07-21T01:45:50.854 INFO:teuthology.orchestra.run.smithi195.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-07-21T01:45:50.855 INFO:teuthology.orchestra.run.smithi195.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-07-21T01:45:50.855 INFO:teuthology.orchestra.run.smithi195.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-07-21T01:45:50.855 INFO:teuthology.orchestra.run.smithi195.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-07-21T01:45:50.855 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2022-07-21T01:45:50.856 INFO:teuthology.orchestra.run.smithi195.stdout: btrfs-tools 2022-07-21T01:45:50.856 INFO:teuthology.orchestra.run.smithi152.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-07-21T01:45:50.860 INFO:teuthology.orchestra.run.smithi140.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-07-21T01:45:50.870 INFO:teuthology.orchestra.run.smithi191.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-07-21T01:45:50.874 INFO:teuthology.orchestra.run.smithi192.stdout:The following NEW packages will be installed: 2022-07-21T01:45:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-07-21T01:45:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-07-21T01:45:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-07-21T01:45:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-07-21T01:45:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-07-21T01:45:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-07-21T01:45:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-07-21T01:45:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-07-21T01:45:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-07-21T01:45:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-07-21T01:45:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-07-21T01:45:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-07-21T01:45:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-07-21T01:45:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-07-21T01:45:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-07-21T01:45:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-07-21T01:45:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-07-21T01:45:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-07-21T01:45:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-07-21T01:45:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-07-21T01:45:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-07-21T01:45:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-07-21T01:45:50.885 INFO:teuthology.orchestra.run.smithi192.stdout:The following packages will be upgraded: 2022-07-21T01:45:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 librbd1 2022-07-21T01:45:50.886 INFO:teuthology.orchestra.run.smithi191.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-07-21T01:45:50.890 INFO:teuthology.orchestra.run.smithi152.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-336-g57544da1-1focal [1,698 kB] 2022-07-21T01:45:50.896 INFO:teuthology.orchestra.run.smithi140.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-336-g57544da1-1focal [1,698 kB] 2022-07-21T01:45:50.903 INFO:teuthology.orchestra.run.smithi152.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-07-21T01:45:50.906 INFO:teuthology.orchestra.run.smithi152.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-07-21T01:45:50.907 INFO:teuthology.orchestra.run.smithi191.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-07-21T01:45:50.908 INFO:teuthology.orchestra.run.smithi152.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-07-21T01:45:50.910 INFO:teuthology.orchestra.run.smithi140.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-07-21T01:45:50.912 INFO:teuthology.orchestra.run.smithi140.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-07-21T01:45:50.915 INFO:teuthology.orchestra.run.smithi140.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-07-21T01:45:50.935 INFO:teuthology.orchestra.run.smithi191.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-07-21T01:45:50.936 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-07-21T01:45:50.939 INFO:teuthology.orchestra.run.smithi191.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-07-21T01:45:50.944 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-336-g57544da1-1focal [1,698 kB] 2022-07-21T01:45:50.948 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2022-07-21T01:45:50.948 INFO:teuthology.orchestra.run.smithi195.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-07-21T01:45:50.948 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2022-07-21T01:45:50.949 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2022-07-21T01:45:50.951 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2022-07-21T01:45:50.952 INFO:teuthology.orchestra.run.smithi195.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-07-21T01:45:50.952 INFO:teuthology.orchestra.run.smithi195.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2022-07-21T01:45:50.953 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2022-07-21T01:45:50.953 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2022-07-21T01:45:50.954 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2022-07-21T01:45:50.954 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2022-07-21T01:45:50.955 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-07-21T01:45:50.955 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-07-21T01:45:50.956 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-07-21T01:45:50.956 INFO:teuthology.orchestra.run.smithi195.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-07-21T01:45:50.957 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-07-21T01:45:50.957 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2022-07-21T01:45:50.958 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-07-21T01:45:50.958 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-07-21T01:45:50.959 INFO:teuthology.orchestra.run.smithi195.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-07-21T01:45:50.959 INFO:teuthology.orchestra.run.smithi195.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-07-21T01:45:50.960 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-07-21T01:45:50.960 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-07-21T01:45:50.961 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages will be upgraded: 2022-07-21T01:45:50.961 INFO:teuthology.orchestra.run.smithi195.stdout: librados2 librbd1 2022-07-21T01:45:50.962 INFO:teuthology.orchestra.run.smithi191.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-07-21T01:45:50.965 INFO:teuthology.orchestra.run.smithi191.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-07-21T01:45:50.967 INFO:teuthology.orchestra.run.smithi191.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-07-21T01:45:50.973 INFO:teuthology.orchestra.run.smithi191.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-07-21T01:45:50.973 INFO:teuthology.orchestra.run.smithi191.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-07-21T01:45:50.974 INFO:teuthology.orchestra.run.smithi152.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-07-21T01:45:50.978 INFO:teuthology.orchestra.run.smithi191.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-07-21T01:45:50.980 INFO:teuthology.orchestra.run.smithi191.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-07-21T01:45:50.981 INFO:teuthology.orchestra.run.smithi191.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-07-21T01:45:50.981 INFO:teuthology.orchestra.run.smithi191.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-07-21T01:45:50.988 INFO:teuthology.orchestra.run.smithi191.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-07-21T01:45:50.989 INFO:teuthology.orchestra.run.smithi191.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-07-21T01:45:50.989 INFO:teuthology.orchestra.run.smithi191.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-07-21T01:45:50.995 INFO:teuthology.orchestra.run.smithi140.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-07-21T01:45:50.996 INFO:teuthology.orchestra.run.smithi152.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-07-21T01:45:51.005 INFO:teuthology.orchestra.run.smithi191.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-07-21T01:45:51.006 INFO:teuthology.orchestra.run.smithi191.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-07-21T01:45:51.007 INFO:teuthology.orchestra.run.smithi191.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-07-21T01:45:51.009 INFO:teuthology.orchestra.run.smithi195.stdout:2 upgraded, 101 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:45:51.009 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 153 MB of archives. 2022-07-21T01:45:51.009 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 699 MB of additional disk space will be used. 2022-07-21T01:45:51.010 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-07-21T01:45:51.011 INFO:teuthology.orchestra.run.smithi191.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-07-21T01:45:51.012 INFO:teuthology.orchestra.run.smithi191.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-07-21T01:45:51.012 INFO:teuthology.orchestra.run.smithi191.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-07-21T01:45:51.013 INFO:teuthology.orchestra.run.smithi152.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-07-21T01:45:51.017 INFO:teuthology.orchestra.run.smithi191.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-07-21T01:45:51.018 INFO:teuthology.orchestra.run.smithi140.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-07-21T01:45:51.020 INFO:teuthology.orchestra.run.smithi140.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-07-21T01:45:51.021 INFO:teuthology.orchestra.run.smithi191.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-07-21T01:45:51.022 INFO:teuthology.orchestra.run.smithi191.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-07-21T01:45:51.023 INFO:teuthology.orchestra.run.smithi191.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-07-21T01:45:51.027 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-07-21T01:45:51.035 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-07-21T01:45:51.037 INFO:teuthology.orchestra.run.smithi152.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-07-21T01:45:51.038 INFO:teuthology.orchestra.run.smithi191.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-07-21T01:45:51.039 INFO:teuthology.orchestra.run.smithi152.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-07-21T01:45:51.041 INFO:teuthology.orchestra.run.smithi191.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-07-21T01:45:51.043 INFO:teuthology.orchestra.run.smithi191.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-07-21T01:45:51.043 INFO:teuthology.orchestra.run.smithi191.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-07-21T01:45:51.045 INFO:teuthology.orchestra.run.smithi191.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-07-21T01:45:51.046 INFO:teuthology.orchestra.run.smithi191.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-07-21T01:45:51.047 INFO:teuthology.orchestra.run.smithi191.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-07-21T01:45:51.055 INFO:teuthology.orchestra.run.smithi152.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-07-21T01:45:51.056 INFO:teuthology.orchestra.run.smithi191.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-07-21T01:45:51.057 INFO:teuthology.orchestra.run.smithi191.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-07-21T01:45:51.064 INFO:teuthology.orchestra.run.smithi192.stdout:2 upgraded, 101 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:45:51.064 INFO:teuthology.orchestra.run.smithi192.stdout:Need to get 153 MB of archives. 2022-07-21T01:45:51.065 INFO:teuthology.orchestra.run.smithi192.stdout:After this operation, 699 MB of additional disk space will be used. 2022-07-21T01:45:51.065 INFO:teuthology.orchestra.run.smithi192.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-336-g57544da1-1focal [1,698 kB] 2022-07-21T01:45:51.065 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-07-21T01:45:51.066 INFO:teuthology.orchestra.run.smithi192.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2022-07-21T01:45:51.074 INFO:teuthology.orchestra.run.smithi140.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-07-21T01:45:51.076 INFO:teuthology.orchestra.run.smithi140.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-07-21T01:45:51.078 INFO:teuthology.orchestra.run.smithi140.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-07-21T01:45:51.079 INFO:teuthology.orchestra.run.smithi152.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-336-g57544da1-1focal [3,257 kB] 2022-07-21T01:45:51.080 INFO:teuthology.orchestra.run.smithi152.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-07-21T01:45:51.080 INFO:teuthology.orchestra.run.smithi152.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-07-21T01:45:51.082 INFO:teuthology.orchestra.run.smithi152.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-07-21T01:45:51.082 INFO:teuthology.orchestra.run.smithi152.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-07-21T01:45:51.084 INFO:teuthology.orchestra.run.smithi152.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-07-21T01:45:51.085 INFO:teuthology.orchestra.run.smithi152.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-07-21T01:45:51.086 INFO:teuthology.orchestra.run.smithi152.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-07-21T01:45:51.093 INFO:teuthology.orchestra.run.smithi140.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-336-g57544da1-1focal [3,257 kB] 2022-07-21T01:45:51.100 INFO:teuthology.orchestra.run.smithi140.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-07-21T01:45:51.101 INFO:teuthology.orchestra.run.smithi140.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-07-21T01:45:51.102 INFO:teuthology.orchestra.run.smithi140.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-07-21T01:45:51.114 INFO:teuthology.orchestra.run.smithi152.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-07-21T01:45:51.114 INFO:teuthology.orchestra.run.smithi152.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-07-21T01:45:51.115 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-07-21T01:45:51.116 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-07-21T01:45:51.117 INFO:teuthology.orchestra.run.smithi140.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-07-21T01:45:51.118 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-07-21T01:45:51.119 INFO:teuthology.orchestra.run.smithi140.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-07-21T01:45:51.120 INFO:teuthology.orchestra.run.smithi140.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-07-21T01:45:51.121 INFO:teuthology.orchestra.run.smithi140.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-07-21T01:45:51.123 INFO:teuthology.orchestra.run.smithi152.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-07-21T01:45:51.128 INFO:teuthology.orchestra.run.smithi152.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-07-21T01:45:51.128 INFO:teuthology.orchestra.run.smithi152.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-07-21T01:45:51.140 INFO:teuthology.orchestra.run.smithi140.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-07-21T01:45:51.141 INFO:teuthology.orchestra.run.smithi140.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-07-21T01:45:51.145 INFO:teuthology.orchestra.run.smithi140.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-07-21T01:45:51.146 INFO:teuthology.orchestra.run.smithi192.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-07-21T01:45:51.163 INFO:teuthology.orchestra.run.smithi140.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-07-21T01:45:51.164 INFO:teuthology.orchestra.run.smithi140.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-07-21T01:45:51.165 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.16-336-g57544da1-1focal [1,698 kB] 2022-07-21T01:45:51.183 INFO:teuthology.orchestra.run.smithi192.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2022-07-21T01:45:51.184 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-07-21T01:45:51.206 INFO:teuthology.orchestra.run.smithi152.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-07-21T01:45:51.220 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-07-21T01:45:51.221 INFO:teuthology.orchestra.run.smithi152.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-07-21T01:45:51.222 INFO:teuthology.orchestra.run.smithi152.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-07-21T01:45:51.223 INFO:teuthology.orchestra.run.smithi152.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-07-21T01:45:51.227 INFO:teuthology.orchestra.run.smithi152.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-07-21T01:45:51.229 INFO:teuthology.orchestra.run.smithi140.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-07-21T01:45:51.230 INFO:teuthology.orchestra.run.smithi140.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-07-21T01:45:51.231 INFO:teuthology.orchestra.run.smithi140.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-07-21T01:45:51.236 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-07-21T01:45:51.246 INFO:teuthology.orchestra.run.smithi140.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-07-21T01:45:51.251 INFO:teuthology.orchestra.run.smithi140.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-07-21T01:45:51.261 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-07-21T01:45:51.264 INFO:teuthology.orchestra.run.smithi152.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-07-21T01:45:51.265 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-07-21T01:45:51.269 INFO:teuthology.orchestra.run.smithi152.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-07-21T01:45:51.269 INFO:teuthology.orchestra.run.smithi152.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-07-21T01:45:51.273 INFO:teuthology.orchestra.run.smithi140.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-07-21T01:45:51.279 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-07-21T01:45:51.287 INFO:teuthology.orchestra.run.smithi152.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-07-21T01:45:51.292 INFO:teuthology.orchestra.run.smithi140.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-07-21T01:45:51.293 INFO:teuthology.orchestra.run.smithi140.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-07-21T01:45:51.302 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-07-21T01:45:51.303 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-07-21T01:45:51.304 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-07-21T01:45:51.305 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-07-21T01:45:51.307 INFO:teuthology.orchestra.run.smithi152.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-07-21T01:45:51.308 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-07-21T01:45:51.308 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-07-21T01:45:51.309 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-07-21T01:45:51.311 INFO:teuthology.orchestra.run.smithi152.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-07-21T01:45:51.311 INFO:teuthology.orchestra.run.smithi152.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-07-21T01:45:51.312 INFO:teuthology.orchestra.run.smithi140.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-07-21T01:45:51.313 INFO:teuthology.orchestra.run.smithi152.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-07-21T01:45:51.315 INFO:teuthology.orchestra.run.smithi192.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-07-21T01:45:51.316 INFO:teuthology.orchestra.run.smithi140.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-07-21T01:45:51.327 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-07-21T01:45:51.327 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-07-21T01:45:51.340 INFO:teuthology.orchestra.run.smithi152.stdout:Get:41 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-336-g57544da1-1focal [463 kB] 2022-07-21T01:45:51.345 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-07-21T01:45:51.348 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-07-21T01:45:51.349 INFO:teuthology.orchestra.run.smithi140.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-07-21T01:45:51.350 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-07-21T01:45:51.351 INFO:teuthology.orchestra.run.smithi140.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-07-21T01:45:51.351 INFO:teuthology.orchestra.run.smithi140.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-07-21T01:45:51.352 INFO:teuthology.orchestra.run.smithi191.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-07-21T01:45:51.362 INFO:teuthology.orchestra.run.smithi191.stdout:Get:41 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-336-g57544da1-1focal [3,257 kB] 2022-07-21T01:45:51.378 INFO:teuthology.orchestra.run.smithi140.stdout:Get:41 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-336-g57544da1-1focal [463 kB] 2022-07-21T01:45:51.396 INFO:teuthology.orchestra.run.smithi152.stdout:Get:42 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-336-g57544da1-1focal [365 kB] 2022-07-21T01:45:51.407 INFO:teuthology.orchestra.run.smithi152.stdout:Get:43 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-336-g57544da1-1focal [20.9 kB] 2022-07-21T01:45:51.409 INFO:teuthology.orchestra.run.smithi152.stdout:Get:44 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-336-g57544da1-1focal [156 kB] 2022-07-21T01:45:51.410 INFO:teuthology.orchestra.run.smithi191.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-07-21T01:45:51.411 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-07-21T01:45:51.411 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-07-21T01:45:51.412 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-07-21T01:45:51.414 INFO:teuthology.orchestra.run.smithi152.stdout:Get:45 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-336-g57544da1-1focal [29.4 kB] 2022-07-21T01:45:51.414 INFO:teuthology.orchestra.run.smithi191.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-07-21T01:45:51.415 INFO:teuthology.orchestra.run.smithi152.stdout:Get:46 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-336-g57544da1-1focal [362 kB] 2022-07-21T01:45:51.425 INFO:teuthology.orchestra.run.smithi140.stdout:Get:42 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-336-g57544da1-1focal [365 kB] 2022-07-21T01:45:51.428 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-07-21T01:45:51.431 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-07-21T01:45:51.448 INFO:teuthology.orchestra.run.smithi140.stdout:Get:43 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-336-g57544da1-1focal [20.9 kB] 2022-07-21T01:45:51.449 INFO:teuthology.orchestra.run.smithi140.stdout:Get:44 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-336-g57544da1-1focal [156 kB] 2022-07-21T01:45:51.450 INFO:teuthology.orchestra.run.smithi192.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-07-21T01:45:51.452 INFO:teuthology.orchestra.run.smithi140.stdout:Get:45 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-336-g57544da1-1focal [29.4 kB] 2022-07-21T01:45:51.453 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-07-21T01:45:51.454 INFO:teuthology.orchestra.run.smithi140.stdout:Get:46 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-336-g57544da1-1focal [362 kB] 2022-07-21T01:45:51.456 INFO:teuthology.orchestra.run.smithi192.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-07-21T01:45:51.464 INFO:teuthology.orchestra.run.smithi191.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-07-21T01:45:51.465 INFO:teuthology.orchestra.run.smithi191.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-07-21T01:45:51.466 INFO:teuthology.orchestra.run.smithi191.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-07-21T01:45:51.467 INFO:teuthology.orchestra.run.smithi191.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-07-21T01:45:51.468 INFO:teuthology.orchestra.run.smithi191.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-07-21T01:45:51.469 INFO:teuthology.orchestra.run.smithi152.stdout:Get:47 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-336-g57544da1-1focal [3,145 kB] 2022-07-21T01:45:51.469 INFO:teuthology.orchestra.run.smithi191.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-07-21T01:45:51.470 INFO:teuthology.orchestra.run.smithi191.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-07-21T01:45:51.472 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-07-21T01:45:51.472 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-07-21T01:45:51.473 INFO:teuthology.orchestra.run.smithi191.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-07-21T01:45:51.473 INFO:teuthology.orchestra.run.smithi191.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-07-21T01:45:51.474 INFO:teuthology.orchestra.run.smithi192.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-07-21T01:45:51.482 INFO:teuthology.orchestra.run.smithi191.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-07-21T01:45:51.489 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-07-21T01:45:51.490 INFO:teuthology.orchestra.run.smithi191.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-07-21T01:45:51.491 INFO:teuthology.orchestra.run.smithi191.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-07-21T01:45:51.493 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-07-21T01:45:51.494 INFO:teuthology.orchestra.run.smithi191.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-07-21T01:45:51.495 INFO:teuthology.orchestra.run.smithi191.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-07-21T01:45:51.508 INFO:teuthology.orchestra.run.smithi191.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-07-21T01:45:51.509 INFO:teuthology.orchestra.run.smithi191.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-07-21T01:45:51.510 INFO:teuthology.orchestra.run.smithi140.stdout:Get:47 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-336-g57544da1-1focal [3,145 kB] 2022-07-21T01:45:51.511 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-07-21T01:45:51.511 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-07-21T01:45:51.512 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-07-21T01:45:51.513 INFO:teuthology.orchestra.run.smithi191.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-07-21T01:45:51.514 INFO:teuthology.orchestra.run.smithi191.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-07-21T01:45:51.527 INFO:teuthology.orchestra.run.smithi191.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-07-21T01:45:51.527 INFO:teuthology.orchestra.run.smithi191.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-07-21T01:45:51.528 INFO:teuthology.orchestra.run.smithi191.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-07-21T01:45:51.528 INFO:teuthology.orchestra.run.smithi191.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-07-21T01:45:51.529 INFO:teuthology.orchestra.run.smithi191.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-07-21T01:45:51.565 INFO:teuthology.orchestra.run.smithi191.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-07-21T01:45:51.565 INFO:teuthology.orchestra.run.smithi191.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-07-21T01:45:51.566 INFO:teuthology.orchestra.run.smithi191.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-07-21T01:45:51.566 INFO:teuthology.orchestra.run.smithi191.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-07-21T01:45:51.568 INFO:teuthology.orchestra.run.smithi191.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-07-21T01:45:51.569 INFO:teuthology.orchestra.run.smithi191.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-07-21T01:45:51.570 INFO:teuthology.orchestra.run.smithi191.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-07-21T01:45:51.575 INFO:teuthology.orchestra.run.smithi192.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-07-21T01:45:51.580 INFO:teuthology.orchestra.run.smithi191.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-07-21T01:45:51.596 INFO:teuthology.orchestra.run.smithi192.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-07-21T01:45:51.602 INFO:teuthology.orchestra.run.smithi192.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-07-21T01:45:51.627 INFO:teuthology.orchestra.run.smithi192.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-07-21T01:45:51.630 INFO:teuthology.orchestra.run.smithi192.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-07-21T01:45:51.634 INFO:teuthology.orchestra.run.smithi192.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-07-21T01:45:51.649 INFO:teuthology.orchestra.run.smithi192.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-07-21T01:45:51.650 INFO:teuthology.orchestra.run.smithi192.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-07-21T01:45:51.652 INFO:teuthology.orchestra.run.smithi192.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-07-21T01:45:51.690 INFO:teuthology.orchestra.run.smithi192.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2022-07-21T01:45:51.693 INFO:teuthology.orchestra.run.smithi192.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-07-21T01:45:51.693 INFO:teuthology.orchestra.run.smithi192.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-07-21T01:45:51.702 INFO:teuthology.orchestra.run.smithi192.stdout:Get:21 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-336-g57544da1-1focal [3,257 kB] 2022-07-21T01:45:51.756 INFO:teuthology.orchestra.run.smithi152.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-336-g57544da1-1focal [116 kB] 2022-07-21T01:45:51.759 INFO:teuthology.orchestra.run.smithi152.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-336-g57544da1-1focal [361 kB] 2022-07-21T01:45:51.770 INFO:teuthology.orchestra.run.smithi192.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-07-21T01:45:51.771 INFO:teuthology.orchestra.run.smithi152.stdout:Get:50 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-336-g57544da1-1focal [20.1 MB] 2022-07-21T01:45:51.773 INFO:teuthology.orchestra.run.smithi192.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-07-21T01:45:51.774 INFO:teuthology.orchestra.run.smithi192.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-07-21T01:45:51.777 INFO:teuthology.orchestra.run.smithi192.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-07-21T01:45:51.781 INFO:teuthology.orchestra.run.smithi192.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-07-21T01:45:51.782 INFO:teuthology.orchestra.run.smithi192.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-07-21T01:45:51.799 INFO:teuthology.orchestra.run.smithi192.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-07-21T01:45:51.851 INFO:teuthology.orchestra.run.smithi192.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-07-21T01:45:51.852 INFO:teuthology.orchestra.run.smithi192.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-07-21T01:45:51.852 INFO:teuthology.orchestra.run.smithi192.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-07-21T01:45:51.892 INFO:teuthology.orchestra.run.smithi140.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-336-g57544da1-1focal [116 kB] 2022-07-21T01:45:51.895 INFO:teuthology.orchestra.run.smithi140.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-336-g57544da1-1focal [361 kB] 2022-07-21T01:45:51.903 INFO:teuthology.orchestra.run.smithi140.stdout:Get:50 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-336-g57544da1-1focal [20.1 MB] 2022-07-21T01:45:51.930 INFO:teuthology.orchestra.run.smithi192.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-07-21T01:45:51.935 INFO:teuthology.orchestra.run.smithi192.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-07-21T01:45:51.938 INFO:teuthology.orchestra.run.smithi192.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-07-21T01:45:51.940 INFO:teuthology.orchestra.run.smithi192.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-07-21T01:45:51.943 INFO:teuthology.orchestra.run.smithi192.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-07-21T01:45:51.946 INFO:teuthology.orchestra.run.smithi192.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-07-21T01:45:51.949 INFO:teuthology.orchestra.run.smithi192.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-07-21T01:45:52.010 INFO:teuthology.orchestra.run.smithi192.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-07-21T01:45:52.011 INFO:teuthology.orchestra.run.smithi192.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-07-21T01:45:52.063 INFO:teuthology.orchestra.run.smithi191.stdout:Get:75 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-336-g57544da1-1focal [463 kB] 2022-07-21T01:45:52.133 INFO:teuthology.orchestra.run.smithi191.stdout:Get:76 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-336-g57544da1-1focal [365 kB] 2022-07-21T01:45:52.251 INFO:teuthology.orchestra.run.smithi191.stdout:Get:77 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-336-g57544da1-1focal [20.9 kB] 2022-07-21T01:45:52.252 INFO:teuthology.orchestra.run.smithi191.stdout:Get:78 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-336-g57544da1-1focal [156 kB] 2022-07-21T01:45:52.258 INFO:teuthology.orchestra.run.smithi191.stdout:Get:79 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-336-g57544da1-1focal [29.4 kB] 2022-07-21T01:45:52.259 INFO:teuthology.orchestra.run.smithi191.stdout:Get:80 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-336-g57544da1-1focal [362 kB] 2022-07-21T01:45:52.356 INFO:teuthology.orchestra.run.smithi192.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-07-21T01:45:52.382 INFO:teuthology.orchestra.run.smithi191.stdout:Get:81 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-336-g57544da1-1focal [3,145 kB] 2022-07-21T01:45:52.399 INFO:teuthology.orchestra.run.smithi192.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-07-21T01:45:52.401 INFO:teuthology.orchestra.run.smithi192.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-07-21T01:45:52.461 INFO:teuthology.orchestra.run.smithi192.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-07-21T01:45:52.462 INFO:teuthology.orchestra.run.smithi192.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-07-21T01:45:52.467 INFO:teuthology.orchestra.run.smithi192.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-07-21T01:45:52.468 INFO:teuthology.orchestra.run.smithi192.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-07-21T01:45:52.468 INFO:teuthology.orchestra.run.smithi192.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-07-21T01:45:52.469 INFO:teuthology.orchestra.run.smithi192.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-07-21T01:45:52.469 INFO:teuthology.orchestra.run.smithi192.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-07-21T01:45:52.517 INFO:teuthology.orchestra.run.smithi192.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-07-21T01:45:52.518 INFO:teuthology.orchestra.run.smithi192.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-07-21T01:45:52.597 INFO:teuthology.orchestra.run.smithi192.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-07-21T01:45:52.602 INFO:teuthology.orchestra.run.smithi192.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-07-21T01:45:52.605 INFO:teuthology.orchestra.run.smithi192.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-07-21T01:45:52.607 INFO:teuthology.orchestra.run.smithi192.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-07-21T01:45:52.607 INFO:teuthology.orchestra.run.smithi192.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-07-21T01:45:52.610 INFO:teuthology.orchestra.run.smithi192.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-07-21T01:45:52.611 INFO:teuthology.orchestra.run.smithi192.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-07-21T01:45:52.613 INFO:teuthology.orchestra.run.smithi192.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-07-21T01:45:52.677 INFO:teuthology.orchestra.run.smithi192.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-07-21T01:45:52.684 INFO:teuthology.orchestra.run.smithi192.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-07-21T01:45:52.714 INFO:teuthology.orchestra.run.smithi192.stdout:Get:63 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-336-g57544da1-1focal [463 kB] 2022-07-21T01:45:52.757 INFO:teuthology.orchestra.run.smithi192.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-07-21T01:45:52.757 INFO:teuthology.orchestra.run.smithi192.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-07-21T01:45:52.758 INFO:teuthology.orchestra.run.smithi192.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-07-21T01:45:52.760 INFO:teuthology.orchestra.run.smithi192.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-07-21T01:45:52.783 INFO:teuthology.orchestra.run.smithi192.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-07-21T01:45:52.783 INFO:teuthology.orchestra.run.smithi192.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-07-21T01:45:52.783 INFO:teuthology.orchestra.run.smithi192.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-07-21T01:45:52.784 INFO:teuthology.orchestra.run.smithi192.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-07-21T01:45:52.823 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.16-336-g57544da1-1focal [3,257 kB] 2022-07-21T01:45:52.837 INFO:teuthology.orchestra.run.smithi192.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-07-21T01:45:52.838 INFO:teuthology.orchestra.run.smithi192.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-07-21T01:45:52.854 INFO:teuthology.orchestra.run.smithi192.stdout:Get:74 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-336-g57544da1-1focal [365 kB] 2022-07-21T01:45:52.884 INFO:teuthology.orchestra.run.smithi152.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-07-21T01:45:52.916 INFO:teuthology.orchestra.run.smithi192.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-07-21T01:45:52.920 INFO:teuthology.orchestra.run.smithi192.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-07-21T01:45:52.929 INFO:teuthology.orchestra.run.smithi192.stdout:Get:77 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-336-g57544da1-1focal [20.9 kB] 2022-07-21T01:45:52.971 INFO:teuthology.orchestra.run.smithi192.stdout:Get:78 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-336-g57544da1-1focal [156 kB] 2022-07-21T01:45:52.999 INFO:teuthology.orchestra.run.smithi192.stdout:Get:79 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-336-g57544da1-1focal [29.4 kB] 2022-07-21T01:45:53.000 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-07-21T01:45:53.001 INFO:teuthology.orchestra.run.smithi192.stdout:Get:80 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-336-g57544da1-1focal [362 kB] 2022-07-21T01:45:53.077 INFO:teuthology.orchestra.run.smithi191.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-336-g57544da1-1focal [116 kB] 2022-07-21T01:45:53.080 INFO:teuthology.orchestra.run.smithi191.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-336-g57544da1-1focal [361 kB] 2022-07-21T01:45:53.095 INFO:teuthology.orchestra.run.smithi140.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-07-21T01:45:53.144 INFO:teuthology.orchestra.run.smithi192.stdout:Get:81 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-336-g57544da1-1focal [3,145 kB] 2022-07-21T01:45:53.157 INFO:teuthology.orchestra.run.smithi191.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-336-g57544da1-1focal [20.1 MB] 2022-07-21T01:45:53.171 INFO:teuthology.orchestra.run.smithi152.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-07-21T01:45:53.193 INFO:teuthology.orchestra.run.smithi152.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-07-21T01:45:53.316 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-07-21T01:45:53.352 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-07-21T01:45:53.395 INFO:teuthology.orchestra.run.smithi140.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-07-21T01:45:53.402 INFO:teuthology.orchestra.run.smithi140.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-07-21T01:45:53.435 INFO:teuthology.orchestra.run.smithi140.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-07-21T01:45:53.435 INFO:teuthology.orchestra.run.smithi140.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-07-21T01:45:53.436 INFO:teuthology.orchestra.run.smithi152.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-07-21T01:45:53.436 INFO:teuthology.orchestra.run.smithi140.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-07-21T01:45:53.437 INFO:teuthology.orchestra.run.smithi140.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-07-21T01:45:53.437 INFO:teuthology.orchestra.run.smithi140.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-07-21T01:45:53.437 INFO:teuthology.orchestra.run.smithi140.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-07-21T01:45:53.438 INFO:teuthology.orchestra.run.smithi140.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-07-21T01:45:53.439 INFO:teuthology.orchestra.run.smithi140.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-07-21T01:45:53.439 INFO:teuthology.orchestra.run.smithi140.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-07-21T01:45:53.449 INFO:teuthology.orchestra.run.smithi152.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-07-21T01:45:53.451 INFO:teuthology.orchestra.run.smithi152.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-07-21T01:45:53.452 INFO:teuthology.orchestra.run.smithi152.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-07-21T01:45:53.453 INFO:teuthology.orchestra.run.smithi140.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-07-21T01:45:53.454 INFO:teuthology.orchestra.run.smithi152.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-07-21T01:45:53.455 INFO:teuthology.orchestra.run.smithi152.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-07-21T01:45:53.455 INFO:teuthology.orchestra.run.smithi140.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-07-21T01:45:53.456 INFO:teuthology.orchestra.run.smithi152.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-07-21T01:45:53.470 INFO:teuthology.orchestra.run.smithi140.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-07-21T01:45:53.472 INFO:teuthology.orchestra.run.smithi140.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-07-21T01:45:53.473 INFO:teuthology.orchestra.run.smithi152.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-07-21T01:45:53.474 INFO:teuthology.orchestra.run.smithi140.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-07-21T01:45:53.474 INFO:teuthology.orchestra.run.smithi152.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-07-21T01:45:53.475 INFO:teuthology.orchestra.run.smithi152.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-07-21T01:45:53.478 INFO:teuthology.orchestra.run.smithi140.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-07-21T01:45:53.481 INFO:teuthology.orchestra.run.smithi140.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-07-21T01:45:53.483 INFO:teuthology.orchestra.run.smithi140.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-07-21T01:45:53.483 INFO:teuthology.orchestra.run.smithi140.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-07-21T01:45:53.486 INFO:teuthology.orchestra.run.smithi140.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-07-21T01:45:53.486 INFO:teuthology.orchestra.run.smithi140.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-07-21T01:45:53.487 INFO:teuthology.orchestra.run.smithi140.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-07-21T01:45:53.504 INFO:teuthology.orchestra.run.smithi140.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-07-21T01:45:53.505 INFO:teuthology.orchestra.run.smithi140.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-07-21T01:45:53.522 INFO:teuthology.orchestra.run.smithi140.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-07-21T01:45:53.522 INFO:teuthology.orchestra.run.smithi140.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-07-21T01:45:53.523 INFO:teuthology.orchestra.run.smithi140.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-07-21T01:45:53.523 INFO:teuthology.orchestra.run.smithi140.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-07-21T01:45:53.524 INFO:teuthology.orchestra.run.smithi140.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-07-21T01:45:53.525 INFO:teuthology.orchestra.run.smithi140.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-07-21T01:45:53.525 INFO:teuthology.orchestra.run.smithi140.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-07-21T01:45:53.526 INFO:teuthology.orchestra.run.smithi140.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-07-21T01:45:53.528 INFO:teuthology.orchestra.run.smithi152.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-07-21T01:45:53.534 INFO:teuthology.orchestra.run.smithi152.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-07-21T01:45:53.553 INFO:teuthology.orchestra.run.smithi152.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-07-21T01:45:53.555 INFO:teuthology.orchestra.run.smithi152.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-07-21T01:45:53.575 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-07-21T01:45:53.576 INFO:teuthology.orchestra.run.smithi195.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-07-21T01:45:53.591 INFO:teuthology.orchestra.run.smithi195.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-07-21T01:45:53.592 INFO:teuthology.orchestra.run.smithi195.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-07-21T01:45:53.593 INFO:teuthology.orchestra.run.smithi195.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-07-21T01:45:53.594 INFO:teuthology.orchestra.run.smithi195.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-07-21T01:45:53.595 INFO:teuthology.orchestra.run.smithi195.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-07-21T01:45:53.612 INFO:teuthology.orchestra.run.smithi195.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-07-21T01:45:53.628 INFO:teuthology.orchestra.run.smithi195.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-07-21T01:45:53.629 INFO:teuthology.orchestra.run.smithi195.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-07-21T01:45:53.668 INFO:teuthology.orchestra.run.smithi195.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-07-21T01:45:53.674 INFO:teuthology.orchestra.run.smithi195.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-07-21T01:45:53.682 INFO:teuthology.orchestra.run.smithi152.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-07-21T01:45:53.683 INFO:teuthology.orchestra.run.smithi152.stdout:Get:69 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-336-g57544da1-1focal [5,347 kB] 2022-07-21T01:45:53.685 INFO:teuthology.orchestra.run.smithi152.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-07-21T01:45:53.692 INFO:teuthology.orchestra.run.smithi195.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-07-21T01:45:53.694 INFO:teuthology.orchestra.run.smithi195.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-07-21T01:45:53.704 INFO:teuthology.orchestra.run.smithi152.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-07-21T01:45:53.705 INFO:teuthology.orchestra.run.smithi152.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-07-21T01:45:53.735 INFO:teuthology.orchestra.run.smithi195.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2022-07-21T01:45:53.744 INFO:teuthology.orchestra.run.smithi152.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-07-21T01:45:53.744 INFO:teuthology.orchestra.run.smithi152.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-07-21T01:45:53.745 INFO:teuthology.orchestra.run.smithi152.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-07-21T01:45:53.746 INFO:teuthology.orchestra.run.smithi152.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-07-21T01:45:53.754 INFO:teuthology.orchestra.run.smithi195.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-07-21T01:45:53.763 INFO:teuthology.orchestra.run.smithi152.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-07-21T01:45:53.792 INFO:teuthology.orchestra.run.smithi195.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2022-07-21T01:45:53.793 INFO:teuthology.orchestra.run.smithi195.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-07-21T01:45:53.832 INFO:teuthology.orchestra.run.smithi195.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-07-21T01:45:53.833 INFO:teuthology.orchestra.run.smithi195.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-07-21T01:45:53.835 INFO:teuthology.orchestra.run.smithi195.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-07-21T01:45:53.835 INFO:teuthology.orchestra.run.smithi195.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-07-21T01:45:53.851 INFO:teuthology.orchestra.run.smithi195.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-07-21T01:45:53.874 INFO:teuthology.orchestra.run.smithi140.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-336-g57544da1-1focal [5,347 kB] 2022-07-21T01:45:53.945 INFO:teuthology.orchestra.run.smithi152.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-07-21T01:45:53.946 INFO:teuthology.orchestra.run.smithi152.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-07-21T01:45:53.963 INFO:teuthology.orchestra.run.smithi152.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-07-21T01:45:53.963 INFO:teuthology.orchestra.run.smithi152.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-07-21T01:45:53.983 INFO:teuthology.orchestra.run.smithi152.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-07-21T01:45:53.985 INFO:teuthology.orchestra.run.smithi152.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-07-21T01:45:53.986 INFO:teuthology.orchestra.run.smithi152.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-07-21T01:45:54.007 INFO:teuthology.orchestra.run.smithi152.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-07-21T01:45:54.030 INFO:teuthology.orchestra.run.smithi195.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-07-21T01:45:54.031 INFO:teuthology.orchestra.run.smithi195.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-07-21T01:45:54.032 INFO:teuthology.orchestra.run.smithi195.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-07-21T01:45:54.032 INFO:teuthology.orchestra.run.smithi195.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-07-21T01:45:54.052 INFO:teuthology.orchestra.run.smithi195.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-07-21T01:45:54.054 INFO:teuthology.orchestra.run.smithi195.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-07-21T01:45:54.070 INFO:teuthology.orchestra.run.smithi195.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-07-21T01:45:54.091 INFO:teuthology.orchestra.run.smithi195.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-07-21T01:45:54.109 INFO:teuthology.orchestra.run.smithi195.stdout:Get:75 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.16-336-g57544da1-1focal [463 kB] 2022-07-21T01:45:54.147 INFO:teuthology.orchestra.run.smithi192.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-336-g57544da1-1focal [116 kB] 2022-07-21T01:45:54.176 INFO:teuthology.orchestra.run.smithi152.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-336-g57544da1-1focal [163 kB] 2022-07-21T01:45:54.181 INFO:teuthology.orchestra.run.smithi152.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-336-g57544da1-1focal [1,402 kB] 2022-07-21T01:45:54.210 INFO:teuthology.orchestra.run.smithi192.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-336-g57544da1-1focal [361 kB] 2022-07-21T01:45:54.267 INFO:teuthology.orchestra.run.smithi140.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-336-g57544da1-1focal [163 kB] 2022-07-21T01:45:54.294 INFO:teuthology.orchestra.run.smithi140.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-336-g57544da1-1focal [1,402 kB] 2022-07-21T01:45:54.303 INFO:teuthology.orchestra.run.smithi195.stdout:Get:76 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.16-336-g57544da1-1focal [365 kB] 2022-07-21T01:45:54.339 INFO:teuthology.orchestra.run.smithi152.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-336-g57544da1-1focal [6,164 kB] 2022-07-21T01:45:54.344 INFO:teuthology.orchestra.run.smithi192.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-336-g57544da1-1focal [20.1 MB] 2022-07-21T01:45:54.399 INFO:teuthology.orchestra.run.smithi140.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-336-g57544da1-1focal [6,164 kB] 2022-07-21T01:45:54.438 INFO:teuthology.orchestra.run.smithi195.stdout:Get:77 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.16-336-g57544da1-1focal [20.9 kB] 2022-07-21T01:45:54.439 INFO:teuthology.orchestra.run.smithi195.stdout:Get:78 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.16-336-g57544da1-1focal [156 kB] 2022-07-21T01:45:54.506 INFO:teuthology.orchestra.run.smithi195.stdout:Get:79 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.16-336-g57544da1-1focal [29.4 kB] 2022-07-21T01:45:54.508 INFO:teuthology.orchestra.run.smithi195.stdout:Get:80 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.16-336-g57544da1-1focal [362 kB] 2022-07-21T01:45:54.642 INFO:teuthology.orchestra.run.smithi195.stdout:Get:81 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.16-336-g57544da1-1focal [3,145 kB] 2022-07-21T01:45:54.945 INFO:teuthology.orchestra.run.smithi140.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-336-g57544da1-1focal [24.0 MB] 2022-07-21T01:45:55.055 INFO:teuthology.orchestra.run.smithi152.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-336-g57544da1-1focal [24.0 MB] 2022-07-21T01:45:55.703 INFO:teuthology.orchestra.run.smithi195.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.16-336-g57544da1-1focal [116 kB] 2022-07-21T01:45:55.706 INFO:teuthology.orchestra.run.smithi195.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.16-336-g57544da1-1focal [361 kB] 2022-07-21T01:45:55.839 INFO:teuthology.orchestra.run.smithi195.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.16-336-g57544da1-1focal [20.1 MB] 2022-07-21T01:45:57.298 INFO:teuthology.orchestra.run.smithi140.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-336-g57544da1-1focal [4,028 B] 2022-07-21T01:45:57.298 INFO:teuthology.orchestra.run.smithi140.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-336-g57544da1-1focal [537 kB] 2022-07-21T01:45:57.308 INFO:teuthology.orchestra.run.smithi140.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-336-g57544da1-1focal [2,006 kB] 2022-07-21T01:45:57.517 INFO:teuthology.orchestra.run.smithi140.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-336-g57544da1-1focal [53.2 kB] 2022-07-21T01:45:57.519 INFO:teuthology.orchestra.run.smithi140.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-336-g57544da1-1focal [59.1 kB] 2022-07-21T01:45:57.522 INFO:teuthology.orchestra.run.smithi140.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-336-g57544da1-1focal [3,218 kB] 2022-07-21T01:45:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-336-g57544da1-1focal [4,028 B] 2022-07-21T01:45:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-336-g57544da1-1focal [537 kB] 2022-07-21T01:45:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-336-g57544da1-1focal [2,006 kB] 2022-07-21T01:45:57.850 INFO:teuthology.orchestra.run.smithi152.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-336-g57544da1-1focal [53.2 kB] 2022-07-21T01:45:57.851 INFO:teuthology.orchestra.run.smithi152.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-336-g57544da1-1focal [59.1 kB] 2022-07-21T01:45:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-336-g57544da1-1focal [3,218 kB] 2022-07-21T01:45:57.939 INFO:teuthology.orchestra.run.smithi140.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-336-g57544da1-1focal [34.4 kB] 2022-07-21T01:45:57.940 INFO:teuthology.orchestra.run.smithi140.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-336-g57544da1-1focal [7,891 kB] 2022-07-21T01:45:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-336-g57544da1-1focal [34.4 kB] 2022-07-21T01:45:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-336-g57544da1-1focal [7,891 kB] 2022-07-21T01:45:59.136 INFO:teuthology.orchestra.run.smithi191.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-336-g57544da1-1focal [5,347 kB] 2022-07-21T01:45:59.223 INFO:teuthology.orchestra.run.smithi140.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-336-g57544da1-1focal [4,144 B] 2022-07-21T01:45:59.223 INFO:teuthology.orchestra.run.smithi140.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-336-g57544da1-1focal [40.2 MB] 2022-07-21T01:45:59.496 INFO:teuthology.orchestra.run.smithi152.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-336-g57544da1-1focal [4,144 B] 2022-07-21T01:45:59.497 INFO:teuthology.orchestra.run.smithi152.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-336-g57544da1-1focal [40.2 MB] 2022-07-21T01:46:00.366 INFO:teuthology.orchestra.run.smithi192.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-336-g57544da1-1focal [5,347 kB] 2022-07-21T01:46:00.543 INFO:teuthology.orchestra.run.smithi191.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-336-g57544da1-1focal [163 kB] 2022-07-21T01:46:00.574 INFO:teuthology.orchestra.run.smithi191.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-336-g57544da1-1focal [1,402 kB] 2022-07-21T01:46:00.920 INFO:teuthology.orchestra.run.smithi191.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-336-g57544da1-1focal [6,164 kB] 2022-07-21T01:46:01.390 INFO:teuthology.orchestra.run.smithi192.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-336-g57544da1-1focal [163 kB] 2022-07-21T01:46:01.423 INFO:teuthology.orchestra.run.smithi192.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-336-g57544da1-1focal [1,402 kB] 2022-07-21T01:46:01.474 INFO:teuthology.orchestra.run.smithi195.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.16-336-g57544da1-1focal [5,347 kB] 2022-07-21T01:46:01.677 INFO:teuthology.orchestra.run.smithi192.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-336-g57544da1-1focal [6,164 kB] 2022-07-21T01:46:02.318 INFO:teuthology.orchestra.run.smithi191.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-336-g57544da1-1focal [24.0 MB] 2022-07-21T01:46:02.557 INFO:teuthology.orchestra.run.smithi195.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.16-336-g57544da1-1focal [163 kB] 2022-07-21T01:46:02.590 INFO:teuthology.orchestra.run.smithi195.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.16-336-g57544da1-1focal [1,402 kB] 2022-07-21T01:46:02.857 INFO:teuthology.orchestra.run.smithi192.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-336-g57544da1-1focal [24.0 MB] 2022-07-21T01:46:02.883 INFO:teuthology.orchestra.run.smithi195.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.16-336-g57544da1-1focal [6,164 kB] 2022-07-21T01:46:04.424 INFO:teuthology.orchestra.run.smithi195.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.16-336-g57544da1-1focal [24.0 MB] 2022-07-21T01:46:04.636 INFO:teuthology.orchestra.run.smithi140.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-336-g57544da1-1focal [15.9 kB] 2022-07-21T01:46:04.637 INFO:teuthology.orchestra.run.smithi140.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-336-g57544da1-1focal [18.1 kB] 2022-07-21T01:46:04.638 INFO:teuthology.orchestra.run.smithi140.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-336-g57544da1-1focal [9,667 kB] 2022-07-21T01:46:05.071 INFO:teuthology.orchestra.run.smithi152.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-336-g57544da1-1focal [15.9 kB] 2022-07-21T01:46:05.071 INFO:teuthology.orchestra.run.smithi152.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-336-g57544da1-1focal [18.1 kB] 2022-07-21T01:46:05.072 INFO:teuthology.orchestra.run.smithi152.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-336-g57544da1-1focal [9,667 kB] 2022-07-21T01:46:06.964 INFO:teuthology.orchestra.run.smithi140.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-336-g57544da1-1focal [61.4 kB] 2022-07-21T01:46:07.061 INFO:teuthology.orchestra.run.smithi152.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-336-g57544da1-1focal [61.4 kB] 2022-07-21T01:46:07.578 INFO:teuthology.orchestra.run.smithi140.stdout:Fetched 153 MB in 16s (9,448 kB/s) 2022-07-21T01:46:07.654 INFO:teuthology.orchestra.run.smithi152.stdout:Fetched 153 MB in 16s (9,392 kB/s) 2022-07-21T01:46:07.693 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-pyperclip. 2022-07-21T01:46:07.741 INFO:teuthology.orchestra.run.smithi140.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-07-21T01:46:07.741 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-07-21T01:46:07.760 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pyperclip. 2022-07-21T01:46:07.774 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-07-21T01:46:07.809 INFO:teuthology.orchestra.run.smithi152.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-07-21T01:46:07.810 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-07-21T01:46:07.842 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-07-21T01:46:08.102 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-wcwidth. 2022-07-21T01:46:08.116 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-07-21T01:46:08.153 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-wcwidth. 2022-07-21T01:46:08.160 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-07-21T01:46:08.166 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-07-21T01:46:08.203 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-07-21T01:46:08.498 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-cmd2. 2022-07-21T01:46:08.506 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-cmd2. 2022-07-21T01:46:08.511 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-07-21T01:46:08.530 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-07-21T01:46:08.555 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-07-21T01:46:08.575 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-07-21T01:46:08.632 INFO:teuthology.orchestra.run.smithi191.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-336-g57544da1-1focal [4,028 B] 2022-07-21T01:46:08.633 INFO:teuthology.orchestra.run.smithi191.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-336-g57544da1-1focal [537 kB] 2022-07-21T01:46:08.743 INFO:teuthology.orchestra.run.smithi191.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-336-g57544da1-1focal [2,006 kB] 2022-07-21T01:46:08.867 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-07-21T01:46:08.882 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-07-21T01:46:08.882 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-07-21T01:46:08.896 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-07-21T01:46:08.951 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-07-21T01:46:08.999 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-07-21T01:46:09.302 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../004-librbd1_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:09.315 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../004-librbd1_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:09.325 INFO:teuthology.orchestra.run.smithi191.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-336-g57544da1-1focal [53.2 kB] 2022-07-21T01:46:09.328 INFO:teuthology.orchestra.run.smithi191.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-336-g57544da1-1focal [59.1 kB] 2022-07-21T01:46:09.333 INFO:teuthology.orchestra.run.smithi191.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-336-g57544da1-1focal [3,218 kB] 2022-07-21T01:46:09.420 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking librbd1 (15.2.16-336-g57544da1-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-07-21T01:46:09.430 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking librbd1 (15.2.16-336-g57544da1-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-07-21T01:46:09.754 INFO:teuthology.orchestra.run.smithi192.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-336-g57544da1-1focal [4,028 B] 2022-07-21T01:46:09.756 INFO:teuthology.orchestra.run.smithi192.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-336-g57544da1-1focal [537 kB] 2022-07-21T01:46:09.807 INFO:teuthology.orchestra.run.smithi192.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-336-g57544da1-1focal [2,006 kB] 2022-07-21T01:46:09.827 INFO:teuthology.orchestra.run.smithi191.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-336-g57544da1-1focal [34.4 kB] 2022-07-21T01:46:09.838 INFO:teuthology.orchestra.run.smithi191.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-336-g57544da1-1focal [7,891 kB] 2022-07-21T01:46:09.982 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../005-librados2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:09.988 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../005-librados2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:10.092 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking librados2 (15.2.16-336-g57544da1-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-07-21T01:46:10.199 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking librados2 (15.2.16-336-g57544da1-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-07-21T01:46:10.200 INFO:teuthology.orchestra.run.smithi192.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-336-g57544da1-1focal [53.2 kB] 2022-07-21T01:46:10.200 INFO:teuthology.orchestra.run.smithi192.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-336-g57544da1-1focal [59.1 kB] 2022-07-21T01:46:10.200 INFO:teuthology.orchestra.run.smithi192.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-336-g57544da1-1focal [3,218 kB] 2022-07-21T01:46:10.621 INFO:teuthology.orchestra.run.smithi192.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-336-g57544da1-1focal [34.4 kB] 2022-07-21T01:46:10.625 INFO:teuthology.orchestra.run.smithi192.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-336-g57544da1-1focal [7,891 kB] 2022-07-21T01:46:10.778 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libcephfs2. 2022-07-21T01:46:10.780 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libcephfs2. 2022-07-21T01:46:10.791 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:10.792 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:10.827 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libcephfs2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:10.829 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libcephfs2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:10.886 INFO:teuthology.orchestra.run.smithi191.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-336-g57544da1-1focal [4,144 B] 2022-07-21T01:46:10.887 INFO:teuthology.orchestra.run.smithi191.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-336-g57544da1-1focal [40.2 MB] 2022-07-21T01:46:11.072 INFO:teuthology.orchestra.run.smithi195.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.16-336-g57544da1-1focal [4,028 B] 2022-07-21T01:46:11.073 INFO:teuthology.orchestra.run.smithi195.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.16-336-g57544da1-1focal [537 kB] 2022-07-21T01:46:11.096 INFO:teuthology.orchestra.run.smithi195.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.16-336-g57544da1-1focal [2,006 kB] 2022-07-21T01:46:11.113 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-rados. 2022-07-21T01:46:11.127 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../007-python3-rados_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:11.157 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-rados. 2022-07-21T01:46:11.170 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../007-python3-rados_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:11.171 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-rados (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:11.207 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-rados (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:11.416 INFO:teuthology.orchestra.run.smithi195.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.16-336-g57544da1-1focal [53.2 kB] 2022-07-21T01:46:11.417 INFO:teuthology.orchestra.run.smithi195.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.16-336-g57544da1-1focal [59.1 kB] 2022-07-21T01:46:11.419 INFO:teuthology.orchestra.run.smithi195.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.16-336-g57544da1-1focal [3,218 kB] 2022-07-21T01:46:11.526 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-07-21T01:46:11.541 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:11.549 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-07-21T01:46:11.563 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:11.577 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-ceph-argparse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:11.607 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-ceph-argparse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:11.676 INFO:teuthology.orchestra.run.smithi192.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-336-g57544da1-1focal [4,144 B] 2022-07-21T01:46:11.677 INFO:teuthology.orchestra.run.smithi192.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-336-g57544da1-1focal [40.2 MB] 2022-07-21T01:46:11.790 INFO:teuthology.orchestra.run.smithi195.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.16-336-g57544da1-1focal [34.4 kB] 2022-07-21T01:46:11.791 INFO:teuthology.orchestra.run.smithi195.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.16-336-g57544da1-1focal [7,891 kB] 2022-07-21T01:46:11.895 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-cephfs. 2022-07-21T01:46:11.904 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-cephfs. 2022-07-21T01:46:11.911 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:11.918 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:11.945 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-cephfs (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:11.963 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-cephfs (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:12.290 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-ceph-common. 2022-07-21T01:46:12.298 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-ceph-common. 2022-07-21T01:46:12.306 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:12.313 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:12.349 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:12.356 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:12.608 INFO:teuthology.orchestra.run.smithi195.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.16-336-g57544da1-1focal [4,144 B] 2022-07-21T01:46:12.608 INFO:teuthology.orchestra.run.smithi195.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.16-336-g57544da1-1focal [40.2 MB] 2022-07-21T01:46:12.660 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-prettytable. 2022-07-21T01:46:12.674 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-07-21T01:46:12.676 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-prettytable. 2022-07-21T01:46:12.691 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-07-21T01:46:12.718 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-07-21T01:46:12.726 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-07-21T01:46:12.971 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-rbd. 2022-07-21T01:46:12.984 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:12.996 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-rbd. 2022-07-21T01:46:13.008 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:13.021 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-rbd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:13.045 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-rbd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:13.399 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-07-21T01:46:13.414 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-07-21T01:46:13.440 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-07-21T01:46:13.457 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-07-21T01:46:13.458 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-07-21T01:46:13.498 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-07-21T01:46:13.819 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-07-21T01:46:13.831 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-07-21T01:46:13.851 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-07-21T01:46:13.864 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-07-21T01:46:13.869 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-07-21T01:46:13.926 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-07-21T01:46:14.155 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package librgw2. 2022-07-21T01:46:14.168 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../015-librgw2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:14.206 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking librgw2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:14.288 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package librgw2. 2022-07-21T01:46:14.300 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../015-librgw2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:14.337 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking librgw2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:14.758 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-rgw. 2022-07-21T01:46:14.771 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:14.833 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-rgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:14.898 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-rgw. 2022-07-21T01:46:14.914 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:14.938 INFO:teuthology.orchestra.run.smithi191.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-336-g57544da1-1focal [15.9 kB] 2022-07-21T01:46:14.940 INFO:teuthology.orchestra.run.smithi191.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-336-g57544da1-1focal [18.1 kB] 2022-07-21T01:46:14.940 INFO:teuthology.orchestra.run.smithi191.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-336-g57544da1-1focal [9,667 kB] 2022-07-21T01:46:14.973 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-rgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:15.152 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-07-21T01:46:15.174 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-07-21T01:46:15.210 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-07-21T01:46:15.318 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-07-21T01:46:15.333 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-07-21T01:46:15.376 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-07-21T01:46:15.530 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package liboath0. 2022-07-21T01:46:15.543 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-07-21T01:46:15.572 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-07-21T01:46:15.746 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package liboath0. 2022-07-21T01:46:15.755 INFO:teuthology.orchestra.run.smithi191.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-336-g57544da1-1focal [61.4 kB] 2022-07-21T01:46:15.760 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-07-21T01:46:15.804 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-07-21T01:46:15.863 INFO:teuthology.orchestra.run.smithi192.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-336-g57544da1-1focal [15.9 kB] 2022-07-21T01:46:15.863 INFO:teuthology.orchestra.run.smithi192.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-336-g57544da1-1focal [18.1 kB] 2022-07-21T01:46:15.864 INFO:teuthology.orchestra.run.smithi192.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-336-g57544da1-1focal [9,667 kB] 2022-07-21T01:46:15.908 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libradosstriper1. 2022-07-21T01:46:15.922 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:15.966 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libradosstriper1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:16.073 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libradosstriper1. 2022-07-21T01:46:16.088 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:16.124 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libradosstriper1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:16.368 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 153 MB in 25s (6,137 kB/s) 2022-07-21T01:46:16.411 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-common. 2022-07-21T01:46:16.426 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../020-ceph-common_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:16.442 INFO:teuthology.orchestra.run.smithi192.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-336-g57544da1-1focal [61.4 kB] 2022-07-21T01:46:16.469 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:16.492 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pyperclip. 2022-07-21T01:46:16.535 INFO:teuthology.orchestra.run.smithi191.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-07-21T01:46:16.538 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-07-21T01:46:16.551 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-common. 2022-07-21T01:46:16.565 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../020-ceph-common_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:16.568 INFO:teuthology.orchestra.run.smithi195.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.16-336-g57544da1-1focal [15.9 kB] 2022-07-21T01:46:16.569 INFO:teuthology.orchestra.run.smithi195.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.16-336-g57544da1-1focal [18.1 kB] 2022-07-21T01:46:16.569 INFO:teuthology.orchestra.run.smithi195.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.16-336-g57544da1-1focal [9,667 kB] 2022-07-21T01:46:16.583 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-07-21T01:46:16.601 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:16.844 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-wcwidth. 2022-07-21T01:46:16.858 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-07-21T01:46:16.902 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-07-21T01:46:17.075 INFO:teuthology.orchestra.run.smithi192.stdout:Fetched 153 MB in 26s (5,988 kB/s) 2022-07-21T01:46:17.152 INFO:teuthology.orchestra.run.smithi195.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/57544da114dbbe1c44f27b2ea18e3233e4e917e0/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.16-336-g57544da1-1focal [61.4 kB] 2022-07-21T01:46:17.190 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cmd2. 2022-07-21T01:46:17.203 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-07-21T01:46:17.221 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pyperclip. 2022-07-21T01:46:17.240 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-07-21T01:46:17.263 INFO:teuthology.orchestra.run.smithi192.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-07-21T01:46:17.266 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-07-21T01:46:17.303 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-07-21T01:46:17.548 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-wcwidth. 2022-07-21T01:46:17.563 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-07-21T01:46:17.578 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-07-21T01:46:17.590 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-07-21T01:46:17.597 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-07-21T01:46:17.697 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-07-21T01:46:17.749 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 153 MB in 26s (5,853 kB/s) 2022-07-21T01:46:17.891 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyperclip. 2022-07-21T01:46:17.900 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-cmd2. 2022-07-21T01:46:17.913 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-07-21T01:46:17.935 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102338 files and directories currently installed.) 2022-07-21T01:46:17.939 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2022-07-21T01:46:17.950 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-07-21T01:46:17.981 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2022-07-21T01:46:18.020 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../004-librbd1_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:18.132 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librbd1 (15.2.16-336-g57544da1-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-07-21T01:46:18.295 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-07-21T01:46:18.309 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-07-21T01:46:18.310 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-wcwidth. 2022-07-21T01:46:18.322 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-07-21T01:46:18.358 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-07-21T01:46:18.395 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-07-21T01:46:18.442 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-base. 2022-07-21T01:46:18.457 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../021-ceph-base_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:18.511 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-base (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:18.652 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../005-librados2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cmd2. 2022-07-21T01:46:18.678 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2022-07-21T01:46:18.682 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-base. 2022-07-21T01:46:18.705 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../021-ceph-base_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:18.720 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2022-07-21T01:46:18.742 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-base (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:18.751 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../004-librbd1_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:18.788 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librados2 (15.2.16-336-g57544da1-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-07-21T01:46:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking librbd1 (15.2.16-336-g57544da1-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-07-21T01:46:19.064 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-07-21T01:46:19.077 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-07-21T01:46:19.157 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-07-21T01:46:19.351 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../005-librados2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:19.455 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-cryptography. 2022-07-21T01:46:19.459 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libcephfs2. 2022-07-21T01:46:19.470 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-07-21T01:46:19.475 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:19.478 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking librados2 (15.2.16-336-g57544da1-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-07-21T01:46:19.483 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../004-librbd1_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:19.513 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-07-21T01:46:19.517 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libcephfs2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:19.612 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-cryptography. 2022-07-21T01:46:19.625 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-07-21T01:46:19.635 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librbd1 (15.2.16-336-g57544da1-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-07-21T01:46:19.678 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-07-21T01:46:19.892 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-openssl. 2022-07-21T01:46:19.906 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-07-21T01:46:19.950 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-07-21T01:46:19.979 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rados. 2022-07-21T01:46:19.994 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../007-python3-rados_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:20.029 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rados (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:20.049 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-openssl. 2022-07-21T01:46:20.063 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-07-21T01:46:20.099 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-07-21T01:46:20.127 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../005-librados2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:20.148 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libcephfs2. 2022-07-21T01:46:20.165 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:20.206 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libcephfs2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:20.236 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-07-21T01:46:20.249 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:20.256 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librados2 (15.2.16-336-g57544da1-1focal) over (15.2.16-0ubuntu0.20.04.1) ... 2022-07-21T01:46:20.286 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr-modules-core (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:20.333 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-07-21T01:46:20.345 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-07-21T01:46:20.347 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:20.361 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:20.397 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-ceph-argparse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:20.401 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-mgr-modules-core (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:20.517 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-rados. 2022-07-21T01:46:20.532 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../007-python3-rados_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:20.575 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-rados (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:20.641 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-bcrypt. 2022-07-21T01:46:20.656 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-07-21T01:46:20.699 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-07-21T01:46:20.728 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cephfs. 2022-07-21T01:46:20.742 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:20.786 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cephfs (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:20.798 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-bcrypt. 2022-07-21T01:46:20.811 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-07-21T01:46:20.856 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-07-21T01:46:20.927 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs2. 2022-07-21T01:46:20.928 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-07-21T01:46:20.940 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../006-libcephfs2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:20.942 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:20.978 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-ceph-argparse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:20.984 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:21.082 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-ceph-common. 2022-07-21T01:46:21.094 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:21.148 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:21.192 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-cherrypy3. 2022-07-21T01:46:21.208 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-07-21T01:46:21.243 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-07-21T01:46:21.245 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rados. 2022-07-21T01:46:21.259 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../007-python3-rados_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:21.281 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-cephfs. 2022-07-21T01:46:21.294 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:21.295 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rados (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:21.356 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-cephfs (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:21.394 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-prettytable. 2022-07-21T01:46:21.407 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-07-21T01:46:21.444 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-07-21T01:46:21.596 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-jwt. 2022-07-21T01:46:21.611 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-07-21T01:46:21.615 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-07-21T01:46:21.630 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:21.636 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-cherrypy3. 2022-07-21T01:46:21.650 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-07-21T01:46:21.651 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-ceph-common. 2022-07-21T01:46:21.654 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-07-21T01:46:21.666 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:21.673 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-argparse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:21.694 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-07-21T01:46:21.697 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rbd. 2022-07-21T01:46:21.700 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:21.710 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:21.752 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rbd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:21.941 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-logutils. 2022-07-21T01:46:21.955 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-07-21T01:46:21.975 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cephfs. 2022-07-21T01:46:21.988 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../009-python3-cephfs_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:21.990 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-07-21T01:46:22.003 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-prettytable. 2022-07-21T01:46:22.019 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-07-21T01:46:22.022 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-jwt. 2022-07-21T01:46:22.039 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-07-21T01:46:22.040 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cephfs (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:22.061 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-07-21T01:46:22.081 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-07-21T01:46:22.101 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-07-21T01:46:22.116 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-07-21T01:46:22.151 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-07-21T01:46:22.302 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-markupsafe. 2022-07-21T01:46:22.314 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-rbd. 2022-07-21T01:46:22.317 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-07-21T01:46:22.328 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:22.360 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-07-21T01:46:22.364 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-rbd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:22.375 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-logutils. 2022-07-21T01:46:22.378 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-common. 2022-07-21T01:46:22.388 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-07-21T01:46:22.392 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:22.418 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-07-21T01:46:22.437 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:22.480 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-07-21T01:46:22.498 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-07-21T01:46:22.547 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-07-21T01:46:22.662 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-markupsafe. 2022-07-21T01:46:22.679 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-07-21T01:46:22.688 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-mako. 2022-07-21T01:46:22.703 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-07-21T01:46:22.712 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-07-21T01:46:22.714 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-prettytable. 2022-07-21T01:46:22.716 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-07-21T01:46:22.728 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2022-07-21T01:46:22.732 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-07-21T01:46:22.746 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-07-21T01:46:22.764 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-07-21T01:46:22.775 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-07-21T01:46:22.883 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package librgw2. 2022-07-21T01:46:22.896 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../015-librgw2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:22.933 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librgw2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:23.000 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rbd. 2022-07-21T01:46:23.015 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-mako. 2022-07-21T01:46:23.016 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../012-python3-rbd_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:23.027 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-07-21T01:46:23.033 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-simplegeneric. 2022-07-21T01:46:23.048 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-07-21T01:46:23.050 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rbd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:23.065 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-07-21T01:46:23.083 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-07-21T01:46:23.089 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-07-21T01:46:23.104 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-07-21T01:46:23.138 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-07-21T01:46:23.403 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-singledispatch. 2022-07-21T01:46:23.404 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-simplegeneric. 2022-07-21T01:46:23.411 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-07-21T01:46:23.417 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-07-21T01:46:23.424 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package librgw2. 2022-07-21T01:46:23.438 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../015-librgw2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:23.445 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-07-21T01:46:23.451 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-07-21T01:46:23.457 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2022-07-21T01:46:23.460 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-07-21T01:46:23.474 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking librgw2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:23.494 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-07-21T01:46:23.511 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rgw. 2022-07-21T01:46:23.524 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:23.562 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:23.729 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-singledispatch. 2022-07-21T01:46:23.742 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-07-21T01:46:23.771 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-07-21T01:46:23.780 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-webob. 2022-07-21T01:46:23.796 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-07-21T01:46:23.838 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-07-21T01:46:23.847 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-07-21T01:46:23.861 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-07-21T01:46:23.882 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-07-21T01:46:23.894 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-07-21T01:46:23.905 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-07-21T01:46:23.932 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-07-21T01:46:24.007 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-webob. 2022-07-21T01:46:24.019 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-07-21T01:46:24.044 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-rgw. 2022-07-21T01:46:24.057 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-07-21T01:46:24.057 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:24.102 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-rgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:24.166 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-soupsieve. 2022-07-21T01:46:24.182 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-07-21T01:46:24.216 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-07-21T01:46:24.225 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librgw2. 2022-07-21T01:46:24.237 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../015-librgw2_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:24.244 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package liboath0. 2022-07-21T01:46:24.258 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-07-21T01:46:24.291 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librgw2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:24.302 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-07-21T01:46:24.385 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-soupsieve. 2022-07-21T01:46:24.400 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-07-21T01:46:24.430 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-07-21T01:46:24.442 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-07-21T01:46:24.444 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-07-21T01:46:24.480 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-07-21T01:46:24.519 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-bs4. 2022-07-21T01:46:24.532 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-07-21T01:46:24.569 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-07-21T01:46:24.639 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libradosstriper1. 2022-07-21T01:46:24.651 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:24.690 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libradosstriper1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:24.762 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-bs4. 2022-07-21T01:46:24.767 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-07-21T01:46:24.774 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package liboath0. 2022-07-21T01:46:24.791 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-07-21T01:46:24.805 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-07-21T01:46:24.836 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rgw. 2022-07-21T01:46:24.841 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-07-21T01:46:24.851 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../016-python3-rgw_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:24.894 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:25.060 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-common. 2022-07-21T01:46:25.072 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../020-ceph-common_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:25.102 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:25.135 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libradosstriper1. 2022-07-21T01:46:25.149 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:25.157 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-waitress. 2022-07-21T01:46:25.170 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-07-21T01:46:25.210 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-07-21T01:46:25.377 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libradosstriper1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:25.531 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-waitress. 2022-07-21T01:46:25.536 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-tempita. 2022-07-21T01:46:25.536 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-07-21T01:46:25.551 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-07-21T01:46:25.575 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-07-21T01:46:25.586 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-07-21T01:46:25.831 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-paste. 2022-07-21T01:46:25.847 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-07-21T01:46:25.880 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-07-21T01:46:25.901 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-tempita. 2022-07-21T01:46:25.913 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-07-21T01:46:25.955 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-07-21T01:46:25.965 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-07-21T01:46:25.976 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-07-21T01:46:26.005 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-07-21T01:46:26.175 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-07-21T01:46:26.190 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-07-21T01:46:26.225 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-07-21T01:46:26.231 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-common. 2022-07-21T01:46:26.245 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../020-ceph-common_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:26.270 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-paste. 2022-07-21T01:46:26.285 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-07-21T01:46:26.289 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:26.328 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-07-21T01:46:26.333 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liboath0. 2022-07-21T01:46:26.346 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2022-07-21T01:46:26.383 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-07-21T01:46:26.520 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pastedeploy. 2022-07-21T01:46:26.534 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-07-21T01:46:26.578 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-07-21T01:46:26.615 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-07-21T01:46:26.628 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-07-21T01:46:26.656 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-07-21T01:46:26.694 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libradosstriper1. 2022-07-21T01:46:26.708 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../019-libradosstriper1_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:26.753 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libradosstriper1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:26.943 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-pastedeploy. 2022-07-21T01:46:26.956 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-07-21T01:46:26.993 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-07-21T01:46:27.106 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-webtest. 2022-07-21T01:46:27.121 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-07-21T01:46:27.164 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-07-21T01:46:27.180 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-common. 2022-07-21T01:46:27.196 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../020-ceph-common_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:27.230 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:27.275 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-base. 2022-07-21T01:46:27.289 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../021-ceph-base_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:27.329 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-webtest. 2022-07-21T01:46:27.343 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-07-21T01:46:27.344 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-base (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:27.388 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-07-21T01:46:27.500 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pecan. 2022-07-21T01:46:27.514 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-07-21T01:46:27.559 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-07-21T01:46:27.707 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-pecan. 2022-07-21T01:46:27.720 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-07-21T01:46:27.757 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-07-21T01:46:27.889 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-werkzeug. 2022-07-21T01:46:27.902 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-07-21T01:46:27.939 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-07-21T01:46:28.079 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-werkzeug. 2022-07-21T01:46:28.092 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-07-21T01:46:28.120 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-07-21T01:46:28.232 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cryptography. 2022-07-21T01:46:28.246 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-07-21T01:46:28.290 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-07-21T01:46:28.301 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr. 2022-07-21T01:46:28.314 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:28.329 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-base. 2022-07-21T01:46:28.345 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../021-ceph-base_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:28.359 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:28.408 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-base (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:28.465 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-mgr. 2022-07-21T01:46:28.479 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:28.523 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-mgr (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:28.687 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-openssl. 2022-07-21T01:46:28.702 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-07-21T01:46:28.744 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-07-21T01:46:28.779 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mon. 2022-07-21T01:46:28.794 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:28.820 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mon (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:28.943 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-mon. 2022-07-21T01:46:28.959 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:28.973 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-07-21T01:46:28.987 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:28.993 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-mon (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:29.023 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-modules-core (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:29.195 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-base. 2022-07-21T01:46:29.199 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-cryptography. 2022-07-21T01:46:29.211 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../021-ceph-base_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:29.214 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-07-21T01:46:29.263 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-base (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:29.270 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-07-21T01:46:29.362 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-bcrypt. 2022-07-21T01:46:29.377 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-07-21T01:46:29.420 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-07-21T01:46:29.604 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-openssl. 2022-07-21T01:46:29.617 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-07-21T01:46:29.645 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-07-21T01:46:29.715 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cherrypy3. 2022-07-21T01:46:29.722 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-07-21T01:46:29.757 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-07-21T01:46:29.824 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-osd. 2022-07-21T01:46:29.851 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:29.882 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-osd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:29.891 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-07-21T01:46:29.904 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:29.950 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-modules-core (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:30.022 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-osd. 2022-07-21T01:46:30.039 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:30.080 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-osd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:30.129 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jwt. 2022-07-21T01:46:30.142 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-07-21T01:46:30.173 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cryptography. 2022-07-21T01:46:30.186 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-07-21T01:46:30.187 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-07-21T01:46:30.228 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-07-21T01:46:30.327 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-bcrypt. 2022-07-21T01:46:30.342 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-07-21T01:46:30.385 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-07-21T01:46:30.449 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-logutils. 2022-07-21T01:46:30.464 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-07-21T01:46:30.499 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-07-21T01:46:30.577 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-openssl. 2022-07-21T01:46:30.591 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2022-07-21T01:46:30.627 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-07-21T01:46:30.680 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-cherrypy3. 2022-07-21T01:46:30.695 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-07-21T01:46:30.738 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-07-21T01:46:30.820 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-markupsafe. 2022-07-21T01:46:30.841 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-07-21T01:46:30.878 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-07-21T01:46:30.891 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-07-21T01:46:30.902 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:30.948 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-modules-core (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:31.109 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-jwt. 2022-07-21T01:46:31.134 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-07-21T01:46:31.158 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-07-21T01:46:31.174 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-mako. 2022-07-21T01:46:31.189 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-07-21T01:46:31.232 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-07-21T01:46:31.267 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-bcrypt. 2022-07-21T01:46:31.284 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-07-21T01:46:31.320 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-07-21T01:46:31.479 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-logutils. 2022-07-21T01:46:31.482 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-07-21T01:46:31.520 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-07-21T01:46:31.536 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-simplegeneric. 2022-07-21T01:46:31.549 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-07-21T01:46:31.578 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-07-21T01:46:31.595 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cherrypy3. 2022-07-21T01:46:31.612 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2022-07-21T01:46:31.653 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-07-21T01:46:31.781 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-markupsafe. 2022-07-21T01:46:31.795 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-07-21T01:46:31.822 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-07-21T01:46:31.875 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-singledispatch. 2022-07-21T01:46:31.887 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-07-21T01:46:31.926 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-07-21T01:46:31.965 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jwt. 2022-07-21T01:46:31.980 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-07-21T01:46:32.024 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-07-21T01:46:32.109 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-mako. 2022-07-21T01:46:32.122 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-07-21T01:46:32.159 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-07-21T01:46:32.211 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-webob. 2022-07-21T01:46:32.224 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-07-21T01:46:32.260 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-07-21T01:46:32.304 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph. 2022-07-21T01:46:32.317 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../047-ceph_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:32.334 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-logutils. 2022-07-21T01:46:32.349 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2022-07-21T01:46:32.354 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:32.409 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-07-21T01:46:32.412 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-simplegeneric. 2022-07-21T01:46:32.425 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-07-21T01:46:32.443 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph. 2022-07-21T01:46:32.456 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../047-ceph_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:32.462 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-07-21T01:46:32.493 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:32.555 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-soupsieve. 2022-07-21T01:46:32.571 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-07-21T01:46:32.604 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-07-21T01:46:32.649 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-fuse. 2022-07-21T01:46:32.662 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:32.699 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:32.712 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-markupsafe. 2022-07-21T01:46:32.726 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-07-21T01:46:32.771 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-07-21T01:46:32.781 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-singledispatch. 2022-07-21T01:46:32.797 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-07-21T01:46:32.838 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-fuse. 2022-07-21T01:46:32.840 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-07-21T01:46:32.851 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:32.888 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:32.907 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-bs4. 2022-07-21T01:46:32.922 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-07-21T01:46:32.966 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-07-21T01:46:33.060 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mds. 2022-07-21T01:46:33.064 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-mako. 2022-07-21T01:46:33.074 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:33.080 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2022-07-21T01:46:33.115 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-07-21T01:46:33.118 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mds (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:33.143 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-webob. 2022-07-21T01:46:33.156 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-07-21T01:46:33.201 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-07-21T01:46:33.233 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-mds. 2022-07-21T01:46:33.247 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:33.283 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-mds (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:33.327 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-waitress. 2022-07-21T01:46:33.343 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-07-21T01:46:33.368 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplegeneric. 2022-07-21T01:46:33.382 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2022-07-21T01:46:33.388 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-07-21T01:46:33.418 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-07-21T01:46:33.496 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-soupsieve. 2022-07-21T01:46:33.511 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-07-21T01:46:33.545 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-07-21T01:46:33.639 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package cephadm. 2022-07-21T01:46:33.651 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../050-cephadm_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:33.688 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:33.696 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-tempita. 2022-07-21T01:46:33.712 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-07-21T01:46:33.745 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-singledispatch. 2022-07-21T01:46:33.755 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-07-21T01:46:33.760 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-07-21T01:46:33.761 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package cephadm. 2022-07-21T01:46:33.776 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../050-cephadm_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:33.790 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-bs4. 2022-07-21T01:46:33.804 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-07-21T01:46:33.812 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-07-21T01:46:33.840 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-07-21T01:46:33.844 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:34.008 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-jinja2. 2022-07-21T01:46:34.022 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-07-21T01:46:34.049 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webob. 2022-07-21T01:46:34.065 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2022-07-21T01:46:34.066 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-paste. 2022-07-21T01:46:34.067 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-07-21T01:46:34.084 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-07-21T01:46:34.098 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-07-21T01:46:34.125 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-07-21T01:46:34.168 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-waitress. 2022-07-21T01:46:34.182 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-07-21T01:46:34.190 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-jinja2. 2022-07-21T01:46:34.203 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-07-21T01:46:34.221 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-07-21T01:46:34.264 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-07-21T01:46:34.351 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-soupsieve. 2022-07-21T01:46:34.367 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-07-21T01:46:34.386 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-07-21T01:46:34.401 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:34.410 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-07-21T01:46:34.419 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-07-21T01:46:34.434 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-07-21T01:46:34.446 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr-cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:34.477 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-07-21T01:46:34.565 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-tempita. 2022-07-21T01:46:34.577 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-07-21T01:46:34.609 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-07-21T01:46:34.621 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-07-21T01:46:34.623 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:34.667 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-mgr-cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:34.737 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-bs4. 2022-07-21T01:46:34.750 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2022-07-21T01:46:34.787 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-07-21T01:46:34.798 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-repoze.lru. 2022-07-21T01:46:34.805 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pastedeploy. 2022-07-21T01:46:34.814 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-07-21T01:46:34.819 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-07-21T01:46:34.858 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-07-21T01:46:34.862 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-07-21T01:46:34.916 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-paste. 2022-07-21T01:46:34.929 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-07-21T01:46:34.966 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-07-21T01:46:34.979 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-repoze.lru. 2022-07-21T01:46:34.994 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-07-21T01:46:35.029 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-07-21T01:46:35.100 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libjs-underscore. 2022-07-21T01:46:35.116 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-07-21T01:46:35.149 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-waitress. 2022-07-21T01:46:35.149 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-webtest. 2022-07-21T01:46:35.150 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-07-21T01:46:35.167 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-07-21T01:46:35.167 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-07-21T01:46:35.211 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-07-21T01:46:35.215 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-07-21T01:46:35.256 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libjs-underscore. 2022-07-21T01:46:35.271 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-07-21T01:46:35.277 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-07-21T01:46:35.290 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-07-21T01:46:35.315 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-07-21T01:46:35.327 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-07-21T01:46:35.361 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-07-21T01:46:35.374 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-07-21T01:46:35.403 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-07-21T01:46:35.518 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pecan. 2022-07-21T01:46:35.531 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-07-21T01:46:35.543 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempita. 2022-07-21T01:46:35.557 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2022-07-21T01:46:35.568 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-07-21T01:46:35.601 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-07-21T01:46:35.602 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-07-21T01:46:35.615 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-07-21T01:46:35.659 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-07-21T01:46:35.664 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pastedeploy. 2022-07-21T01:46:35.678 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-07-21T01:46:35.698 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-routes. 2022-07-21T01:46:35.713 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-07-21T01:46:35.714 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-07-21T01:46:35.756 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-07-21T01:46:35.848 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-werkzeug. 2022-07-21T01:46:35.860 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-07-21T01:46:35.889 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-07-21T01:46:35.938 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-paste. 2022-07-21T01:46:35.951 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-07-21T01:46:35.988 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-07-21T01:46:36.025 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-webtest. 2022-07-21T01:46:36.031 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-07-21T01:46:36.067 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-07-21T01:46:36.092 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-07-21T01:46:36.110 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:36.151 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr-dashboard (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:36.237 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-routes. 2022-07-21T01:46:36.251 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr. 2022-07-21T01:46:36.252 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-07-21T01:46:36.264 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:36.295 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-07-21T01:46:36.300 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:36.307 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-07-21T01:46:36.328 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-07-21T01:46:36.360 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-07-21T01:46:36.545 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pecan. 2022-07-21T01:46:36.558 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-07-21T01:46:36.595 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-07-21T01:46:36.657 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-07-21T01:46:36.671 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:36.693 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastedeploy. 2022-07-21T01:46:36.709 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2022-07-21T01:46:36.748 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-mgr-dashboard (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:36.761 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-07-21T01:46:36.787 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mon. 2022-07-21T01:46:36.800 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:36.837 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mon (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:36.866 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-werkzeug. 2022-07-21T01:46:36.879 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-07-21T01:46:36.917 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-07-21T01:46:37.030 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webtest. 2022-07-21T01:46:37.043 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2022-07-21T01:46:37.072 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-07-21T01:46:37.079 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-07-21T01:46:37.089 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:37.126 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:37.286 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr. 2022-07-21T01:46:37.302 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:37.344 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:37.358 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pecan. 2022-07-21T01:46:37.371 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-07-21T01:46:37.408 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-07-21T01:46:37.446 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-decorator. 2022-07-21T01:46:37.459 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-07-21T01:46:37.496 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-07-21T01:46:37.675 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-07-21T01:46:37.693 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:37.731 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:37.737 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-werkzeug. 2022-07-21T01:46:37.750 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-07-21T01:46:37.765 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-osd. 2022-07-21T01:46:37.779 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:37.781 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mon. 2022-07-21T01:46:37.787 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-07-21T01:46:37.794 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:37.815 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-osd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:37.824 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-07-21T01:46:37.831 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mon (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:37.837 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-07-21T01:46:37.882 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-07-21T01:46:38.077 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-decorator. 2022-07-21T01:46:38.090 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-07-21T01:46:38.179 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-07-21T01:46:38.191 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr. 2022-07-21T01:46:38.194 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-scipy. 2022-07-21T01:46:38.209 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../044-ceph-mgr_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:38.210 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-07-21T01:46:38.249 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:38.252 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-07-21T01:46:38.513 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-07-21T01:46:38.527 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-07-21T01:46:38.571 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-07-21T01:46:38.744 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mon. 2022-07-21T01:46:38.757 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../045-ceph-mon_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:38.803 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-osd. 2022-07-21T01:46:38.803 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mon (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:38.814 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:38.851 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-osd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:38.875 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-scipy. 2022-07-21T01:46:38.888 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-07-21T01:46:38.925 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-07-21T01:46:39.486 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-07-21T01:46:39.500 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-07-21T01:46:39.544 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-07-21T01:46:39.831 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-osd. 2022-07-21T01:46:39.845 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../046-ceph-osd_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:39.906 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-osd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:40.041 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-joblib. 2022-07-21T01:46:40.053 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-07-21T01:46:40.099 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-07-21T01:46:40.262 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph. 2022-07-21T01:46:40.275 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-07-21T01:46:40.275 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../047-ceph_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:40.291 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-07-21T01:46:40.303 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:40.333 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-07-21T01:46:40.452 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-sklearn. 2022-07-21T01:46:40.466 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-07-21T01:46:40.514 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-07-21T01:46:40.599 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-fuse. 2022-07-21T01:46:40.611 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:40.647 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:40.912 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-joblib. 2022-07-21T01:46:40.926 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-07-21T01:46:41.000 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-07-21T01:46:41.051 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mds. 2022-07-21T01:46:41.066 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:41.101 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mds (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:41.149 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph. 2022-07-21T01:46:41.165 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../047-ceph_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:41.187 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-07-21T01:46:41.203 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:41.207 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:41.246 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:41.449 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-sklearn. 2022-07-21T01:46:41.466 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-07-21T01:46:41.507 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-07-21T01:46:41.526 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-fuse. 2022-07-21T01:46:41.540 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:41.618 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:41.628 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package cephadm. 2022-07-21T01:46:41.642 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../050-cephadm_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:41.686 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:41.940 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jinja2. 2022-07-21T01:46:41.953 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-07-21T01:46:41.971 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mds. 2022-07-21T01:46:41.981 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-07-21T01:46:41.985 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:42.031 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mds (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:42.216 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-07-21T01:46:42.233 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:42.274 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:42.319 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph. 2022-07-21T01:46:42.333 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../047-ceph_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:42.334 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-07-21T01:46:42.347 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:42.368 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-cachetools. 2022-07-21T01:46:42.377 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:42.380 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-07-21T01:46:42.383 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:42.431 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-07-21T01:46:42.500 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package cephadm. 2022-07-21T01:46:42.513 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../050-cephadm_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:42.557 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:42.659 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pyasn1. 2022-07-21T01:46:42.671 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-fuse. 2022-07-21T01:46:42.673 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-07-21T01:46:42.685 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../048-ceph-fuse_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:42.703 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-repoze.lru. 2022-07-21T01:46:42.709 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-07-21T01:46:42.718 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-07-21T01:46:42.729 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:42.755 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-07-21T01:46:42.878 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-jinja2. 2022-07-21T01:46:42.893 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-07-21T01:46:42.935 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-07-21T01:46:42.972 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libjs-underscore. 2022-07-21T01:46:42.985 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-07-21T01:46:43.014 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-07-21T01:46:43.028 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-07-21T01:46:43.041 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-07-21T01:46:43.075 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mds. 2022-07-21T01:46:43.078 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-07-21T01:46:43.090 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../049-ceph-mds_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:43.133 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mds (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:43.255 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-07-21T01:46:43.258 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-07-21T01:46:43.269 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:43.272 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-07-21T01:46:43.313 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:43.316 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-07-21T01:46:43.356 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-rsa. 2022-07-21T01:46:43.370 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-07-21T01:46:43.393 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-cachetools. 2022-07-21T01:46:43.408 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-07-21T01:46:43.413 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-rsa (4.0-3) ... 2022-07-21T01:46:43.451 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-07-21T01:46:43.629 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephadm. 2022-07-21T01:46:43.641 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../050-cephadm_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:43.668 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-repoze.lru. 2022-07-21T01:46:43.669 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-routes. 2022-07-21T01:46:43.681 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-07-21T01:46:43.682 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-07-21T01:46:43.685 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:43.719 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-07-21T01:46:43.725 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-07-21T01:46:43.726 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-google-auth. 2022-07-21T01:46:43.739 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-07-21T01:46:43.756 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-pyasn1. 2022-07-21T01:46:43.768 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-07-21T01:46:43.783 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-07-21T01:46:43.830 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-07-21T01:46:43.937 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libjs-underscore. 2022-07-21T01:46:43.949 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-07-21T01:46:43.978 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-07-21T01:46:44.013 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jinja2. 2022-07-21T01:46:44.026 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-07-21T01:46:44.063 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-07-21T01:46:44.064 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-07-21T01:46:44.076 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:44.108 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-07-21T01:46:44.113 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-dashboard (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:44.123 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-07-21T01:46:44.129 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-blinker. 2022-07-21T01:46:44.142 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-07-21T01:46:44.158 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-07-21T01:46:44.186 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-07-21T01:46:44.247 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-07-21T01:46:44.263 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-07-21T01:46:44.305 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-07-21T01:46:44.367 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-07-21T01:46:44.381 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:44.425 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:44.452 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-rsa. 2022-07-21T01:46:44.461 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-oauthlib. 2022-07-21T01:46:44.467 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-07-21T01:46:44.472 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-07-21T01:46:44.508 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-07-21T01:46:44.510 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-rsa (4.0-3) ... 2022-07-21T01:46:44.617 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-routes. 2022-07-21T01:46:44.629 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-07-21T01:46:44.666 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-07-21T01:46:44.720 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-repoze.lru. 2022-07-21T01:46:44.735 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-07-21T01:46:44.778 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-07-21T01:46:44.839 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-google-auth. 2022-07-21T01:46:44.842 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-07-21T01:46:44.855 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-07-21T01:46:44.857 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-07-21T01:46:44.922 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-07-21T01:46:44.925 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-07-21T01:46:44.995 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-07-21T01:46:45.010 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:45.030 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjs-underscore. 2022-07-21T01:46:45.044 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-07-21T01:46:45.076 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-dashboard (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:45.089 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-07-21T01:46:45.096 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-07-21T01:46:45.116 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:45.154 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:45.195 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-websocket. 2022-07-21T01:46:45.207 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-07-21T01:46:45.225 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-blinker. 2022-07-21T01:46:45.242 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-07-21T01:46:45.244 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-07-21T01:46:45.309 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-07-21T01:46:45.395 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-07-21T01:46:45.405 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-07-21T01:46:45.449 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-07-21T01:46:45.460 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-decorator. 2022-07-21T01:46:45.470 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-07-21T01:46:45.498 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-kubernetes. 2022-07-21T01:46:45.511 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-07-21T01:46:45.515 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-07-21T01:46:45.547 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-07-21T01:46:45.615 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-oauthlib. 2022-07-21T01:46:45.628 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-07-21T01:46:45.686 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-07-21T01:46:45.744 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-routes. 2022-07-21T01:46:45.757 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-07-21T01:46:45.793 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-07-21T01:46:45.809 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-07-21T01:46:45.810 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-07-21T01:46:45.843 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-07-21T01:46:46.015 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-07-21T01:46:46.031 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-07-21T01:46:46.070 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-07-21T01:46:46.073 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-07-21T01:46:46.085 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-07-21T01:46:46.087 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:46.099 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:46.128 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:46.143 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:46.145 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-scipy. 2022-07-21T01:46:46.147 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-07-21T01:46:46.160 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-07-21T01:46:46.161 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:46.195 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-07-21T01:46:46.205 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-dashboard (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:46.378 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-websocket. 2022-07-21T01:46:46.394 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-07-21T01:46:46.430 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libonig5:amd64. 2022-07-21T01:46:46.432 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-decorator. 2022-07-21T01:46:46.434 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-07-21T01:46:46.443 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-07-21T01:46:46.444 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-07-21T01:46:46.482 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-07-21T01:46:46.487 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-07-21T01:46:46.759 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-07-21T01:46:46.773 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-07-21T01:46:46.780 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-kubernetes. 2022-07-21T01:46:46.793 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-07-21T01:46:46.807 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libjq1:amd64. 2022-07-21T01:46:46.822 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-07-21T01:46:46.830 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-07-21T01:46:46.834 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-07-21T01:46:46.874 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:46:47.137 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-scipy. 2022-07-21T01:46:47.152 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-07-21T01:46:47.168 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package jq. 2022-07-21T01:46:47.180 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-07-21T01:46:47.195 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-07-21T01:46:47.196 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2022-07-21T01:46:47.212 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:47.218 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:46:47.255 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:47.393 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-07-21T01:46:47.408 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:47.470 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package socat. 2022-07-21T01:46:47.476 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:47.483 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-07-21T01:46:47.520 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking socat (1.7.3.3-2) ... 2022-07-21T01:46:47.529 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-07-21T01:46:47.544 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-07-21T01:46:47.558 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-decorator. 2022-07-21T01:46:47.571 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-07-21T01:46:47.607 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-07-21T01:46:47.621 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-07-21T01:46:47.829 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libonig5:amd64. 2022-07-21T01:46:47.845 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-07-21T01:46:47.895 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-07-21T01:46:47.910 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-07-21T01:46:47.925 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-07-21T01:46:47.992 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-07-21T01:46:48.002 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-07-21T01:46:48.008 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-07-21T01:46:48.050 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-07-21T01:46:48.132 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-joblib. 2022-07-21T01:46:48.150 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-07-21T01:46:48.190 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-07-21T01:46:48.226 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libjq1:amd64. 2022-07-21T01:46:48.240 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-07-21T01:46:48.282 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:46:48.320 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package xmlstarlet. 2022-07-21T01:46:48.332 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-scipy. 2022-07-21T01:46:48.334 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-07-21T01:46:48.347 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-07-21T01:46:48.378 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-07-21T01:46:48.379 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-07-21T01:46:48.463 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-07-21T01:46:48.479 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-07-21T01:46:48.538 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-07-21T01:46:48.577 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-sklearn. 2022-07-21T01:46:48.585 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package jq. 2022-07-21T01:46:48.593 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-07-21T01:46:48.599 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-07-21T01:46:48.627 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-07-21T01:46:48.643 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:46:48.673 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package ceph-test. 2022-07-21T01:46:48.687 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../083-ceph-test_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:48.723 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking ceph-test (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:48.920 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package socat. 2022-07-21T01:46:48.935 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-07-21T01:46:48.992 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking socat (1.7.3.3-2) ... 2022-07-21T01:46:49.067 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-joblib. 2022-07-21T01:46:49.081 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-07-21T01:46:49.125 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-07-21T01:46:49.360 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-07-21T01:46:49.377 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:49.419 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:49.484 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-07-21T01:46:49.498 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-07-21T01:46:49.522 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-sklearn. 2022-07-21T01:46:49.536 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-07-21T01:46:49.536 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-07-21T01:46:49.578 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-07-21T01:46:49.664 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-07-21T01:46:49.678 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-07-21T01:46:49.722 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-07-21T01:46:49.840 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package xmlstarlet. 2022-07-21T01:46:49.853 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-07-21T01:46:49.890 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-07-21T01:46:50.217 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-joblib. 2022-07-21T01:46:50.230 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2022-07-21T01:46:50.266 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package ceph-test. 2022-07-21T01:46:50.267 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-07-21T01:46:50.275 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-07-21T01:46:50.285 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../083-ceph-test_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:50.287 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:50.320 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking ceph-test (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:50.329 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:50.563 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cachetools. 2022-07-21T01:46:50.577 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-07-21T01:46:50.613 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-07-21T01:46:50.631 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn. 2022-07-21T01:46:50.644 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-07-21T01:46:50.712 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-07-21T01:46:50.891 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pyasn1. 2022-07-21T01:46:50.905 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-07-21T01:46:50.940 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-07-21T01:46:51.329 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-07-21T01:46:51.343 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:51.395 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:51.404 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-07-21T01:46:51.421 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-07-21T01:46:51.432 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-cachetools. 2022-07-21T01:46:51.448 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-07-21T01:46:51.482 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-07-21T01:46:51.577 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-07-21T01:46:51.760 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pyasn1. 2022-07-21T01:46:51.776 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-07-21T01:46:51.869 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-07-21T01:46:51.898 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rsa. 2022-07-21T01:46:51.913 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-07-21T01:46:51.955 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rsa (4.0-3) ... 2022-07-21T01:46:52.160 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-colorama. 2022-07-21T01:46:52.174 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-07-21T01:46:52.210 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-07-21T01:46:52.214 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-07-21T01:46:52.230 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-07-21T01:46:52.272 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-07-21T01:46:52.274 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-google-auth. 2022-07-21T01:46:52.288 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-07-21T01:46:52.324 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-07-21T01:46:52.471 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package cephfs-shell. 2022-07-21T01:46:52.486 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:52.529 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking cephfs-shell (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:52.557 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cachetools. 2022-07-21T01:46:52.567 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-rsa. 2022-07-21T01:46:52.570 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2022-07-21T01:46:52.583 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-07-21T01:46:52.607 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-07-21T01:46:52.627 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-rsa (4.0-3) ... 2022-07-21T01:46:52.646 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-blinker. 2022-07-21T01:46:52.660 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-07-21T01:46:52.703 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-07-21T01:46:52.757 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libcephfs-dev. 2022-07-21T01:46:52.773 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:52.806 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libcephfs-dev (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:52.934 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1. 2022-07-21T01:46:52.951 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-07-21T01:46:52.962 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-google-auth. 2022-07-21T01:46:52.979 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-07-21T01:46:52.989 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-oauthlib. 2022-07-21T01:46:52.992 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-07-21T01:46:53.001 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-07-21T01:46:53.020 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-07-21T01:46:53.063 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-07-21T01:46:53.210 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package nvme-cli. 2022-07-21T01:46:53.227 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-07-21T01:46:53.285 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-07-21T01:46:53.315 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-07-21T01:46:53.325 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-blinker. 2022-07-21T01:46:53.330 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-07-21T01:46:53.337 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-07-21T01:46:53.381 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-07-21T01:46:53.392 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-07-21T01:46:53.398 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-07-21T01:46:53.406 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-07-21T01:46:53.450 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-07-21T01:46:53.656 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-atomicwrites. 2022-07-21T01:46:53.672 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-07-21T01:46:53.684 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-oauthlib. 2022-07-21T01:46:53.699 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-07-21T01:46:53.709 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rsa. 2022-07-21T01:46:53.714 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-07-21T01:46:53.722 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2022-07-21T01:46:53.742 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-07-21T01:46:53.752 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-websocket. 2022-07-21T01:46:53.759 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rsa (4.0-3) ... 2022-07-21T01:46:53.769 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-07-21T01:46:53.798 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-colorama. 2022-07-21T01:46:53.803 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-07-21T01:46:53.815 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-07-21T01:46:53.864 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-07-21T01:46:54.025 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-attr. 2022-07-21T01:46:54.038 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-07-21T01:46:54.054 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-google-auth. 2022-07-21T01:46:54.070 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2022-07-21T01:46:54.082 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-kubernetes. 2022-07-21T01:46:54.083 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-07-21T01:46:54.096 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-07-21T01:46:54.097 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-07-21T01:46:54.104 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-07-21T01:46:54.110 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-07-21T01:46:54.126 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package cephfs-shell. 2022-07-21T01:46:54.139 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-07-21T01:46:54.140 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:54.155 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-07-21T01:46:54.174 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking cephfs-shell (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:54.357 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-blinker. 2022-07-21T01:46:54.372 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-07-21T01:46:54.411 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-webencodings. 2022-07-21T01:46:54.415 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-07-21T01:46:54.426 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-07-21T01:46:54.427 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libcephfs-dev. 2022-07-21T01:46:54.441 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-websocket. 2022-07-21T01:46:54.442 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:54.456 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-07-21T01:46:54.461 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-07-21T01:46:54.485 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libcephfs-dev (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:54.491 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-07-21T01:46:54.694 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-07-21T01:46:54.708 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:54.730 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-html5lib. 2022-07-21T01:46:54.743 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-oauthlib. 2022-07-21T01:46:54.743 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-07-21T01:46:54.753 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:54.756 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-07-21T01:46:54.782 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-07-21T01:46:54.793 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-07-21T01:46:54.806 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-kubernetes. 2022-07-21T01:46:54.816 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-07-21T01:46:54.852 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-07-21T01:46:54.872 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package nvme-cli. 2022-07-21T01:46:54.888 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-07-21T01:46:54.948 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-07-21T01:46:55.071 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libonig5:amd64. 2022-07-21T01:46:55.086 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-07-21T01:46:55.103 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-07-21T01:46:55.116 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-07-21T01:46:55.120 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-07-21T01:46:55.121 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-07-21T01:46:55.137 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-07-21T01:46:55.158 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-07-21T01:46:55.179 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-07-21T01:46:55.352 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-atomicwrites. 2022-07-21T01:46:55.369 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-07-21T01:46:55.399 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-07-21T01:46:55.414 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:55.428 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-07-21T01:46:55.457 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:55.474 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libjq1:amd64. 2022-07-21T01:46:55.483 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-websocket. 2022-07-21T01:46:55.487 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-07-21T01:46:55.495 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-07-21T01:46:55.524 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-07-21T01:46:55.532 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:46:55.594 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-packaging. 2022-07-21T01:46:55.607 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-07-21T01:46:55.644 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-packaging (20.3-1) ... 2022-07-21T01:46:55.705 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-attr. 2022-07-21T01:46:55.719 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-07-21T01:46:55.771 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-07-21T01:46:55.786 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-kubernetes. 2022-07-21T01:46:55.801 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-07-21T01:46:55.827 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libonig5:amd64. 2022-07-21T01:46:55.834 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package jq. 2022-07-21T01:46:55.841 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-07-21T01:46:55.843 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-07-21T01:46:55.847 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-07-21T01:46:55.884 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:46:55.885 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-07-21T01:46:55.910 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pastescript. 2022-07-21T01:46:55.919 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-07-21T01:46:55.962 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-07-21T01:46:56.115 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package socat. 2022-07-21T01:46:56.124 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-webencodings. 2022-07-21T01:46:56.125 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-07-21T01:46:56.141 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-07-21T01:46:56.153 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking socat (1.7.3.3-2) ... 2022-07-21T01:46:56.182 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-07-21T01:46:56.234 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pluggy. 2022-07-21T01:46:56.238 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libjq1:amd64. 2022-07-21T01:46:56.249 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-07-21T01:46:56.253 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-07-21T01:46:56.296 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:46:56.300 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-07-21T01:46:56.432 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-07-21T01:46:56.448 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:46:56.490 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-k8sevents (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:56.519 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-html5lib. 2022-07-21T01:46:56.532 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-07-21T01:46:56.574 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package jq. 2022-07-21T01:46:56.577 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-07-21T01:46:56.589 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-07-21T01:46:56.611 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-psutil. 2022-07-21T01:46:56.625 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-07-21T01:46:56.632 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:46:56.634 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-07-21T01:46:56.649 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-07-21T01:46:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-07-21T01:46:56.692 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-07-21T01:46:56.869 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package socat. 2022-07-21T01:46:56.884 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-07-21T01:46:56.922 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-07-21T01:46:56.926 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking socat (1.7.3.3-2) ... 2022-07-21T01:46:56.938 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-07-21T01:46:56.971 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package xmlstarlet. 2022-07-21T01:46:56.973 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-07-21T01:46:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-py. 2022-07-21T01:46:56.986 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-07-21T01:46:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-07-21T01:46:57.035 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-07-21T01:46:57.035 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-07-21T01:46:57.060 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libonig5:amd64. 2022-07-21T01:46:57.074 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2022-07-21T01:46:57.117 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-07-21T01:46:57.361 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-07-21T01:46:57.366 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-test. 2022-07-21T01:46:57.374 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-07-21T01:46:57.383 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../083-ceph-test_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pyinotify. 2022-07-21T01:46:57.409 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-07-21T01:46:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-07-21T01:46:57.412 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjq1:amd64. 2022-07-21T01:46:57.424 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-test (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:57.426 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-07-21T01:46:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-07-21T01:46:57.462 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:46:57.478 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-packaging. 2022-07-21T01:46:57.492 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-07-21T01:46:57.534 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-packaging (20.3-1) ... 2022-07-21T01:46:57.653 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package xmlstarlet. 2022-07-21T01:46:57.668 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-07-21T01:46:57.715 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-07-21T01:46:57.773 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package jq. 2022-07-21T01:46:57.789 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-07-21T01:46:57.832 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:46:57.836 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-pastescript. 2022-07-21T01:46:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-pytest. 2022-07-21T01:46:57.850 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-07-21T01:46:57.851 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-07-21T01:46:57.878 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-07-21T01:46:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-07-21T01:46:58.023 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package ceph-test. 2022-07-21T01:46:58.039 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../083-ceph-test_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:58.081 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking ceph-test (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:58.084 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package socat. 2022-07-21T01:46:58.098 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2022-07-21T01:46:58.142 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking socat (1.7.3.3-2) ... 2022-07-21T01:46:58.198 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-pluggy. 2022-07-21T01:46:58.214 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-07-21T01:46:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package python3-simplejson. 2022-07-21T01:46:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-07-21T01:46:58.297 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-07-21T01:46:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-07-21T01:46:58.606 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-07-21T01:46:58.621 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2022-07-21T01:46:58.623 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-psutil. 2022-07-21T01:46:58.639 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-07-21T01:46:58.656 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2022-07-21T01:46:58.684 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-07-21T01:46:58.734 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package radosgw. 2022-07-21T01:46:58.749 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../101-radosgw_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:58.783 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking radosgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:58.969 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package xmlstarlet. 2022-07-21T01:46:58.986 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-07-21T01:46:59.012 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-py. 2022-07-21T01:46:59.025 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-07-21T01:46:59.025 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-07-21T01:46:59.061 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-07-21T01:46:59.330 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-test. 2022-07-21T01:46:59.343 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../083-ceph-test_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:59.380 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-test (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:46:59.402 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-pyinotify. 2022-07-21T01:46:59.414 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-07-21T01:46:59.456 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-07-21T01:46:59.759 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-pytest. 2022-07-21T01:46:59.774 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-07-21T01:46:59.809 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-07-21T01:46:59.812 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package rbd-fuse. 2022-07-21T01:46:59.826 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:46:59.870 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking rbd-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:00.165 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-attr (19.3.0-2) ... 2022-07-21T01:47:00.171 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package python3-simplejson. 2022-07-21T01:47:00.184 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-07-21T01:47:00.229 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-07-21T01:47:00.475 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-07-21T01:47:00.600 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-07-21T01:47:00.608 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package radosgw. 2022-07-21T01:47:00.623 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../101-radosgw_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:47:00.664 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking radosgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:00.730 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-07-21T01:47:00.819 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-colorama. 2022-07-21T01:47:00.834 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-07-21T01:47:00.877 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-07-21T01:47:00.919 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:01.147 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package cephfs-shell. 2022-07-21T01:47:01.163 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:47:01.197 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking cephfs-shell (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:01.405 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libcephfs-dev. 2022-07-21T01:47:01.418 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:47:01.457 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libcephfs-dev (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:01.535 INFO:teuthology.orchestra.run.smithi152.stdout:Adding system user cephadm....done 2022-07-21T01:47:01.677 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-colorama. 2022-07-21T01:47:01.690 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-07-21T01:47:01.692 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-07-21T01:47:01.710 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package rbd-fuse. 2022-07-21T01:47:01.735 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-07-21T01:47:01.735 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:47:01.768 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking rbd-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:01.885 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package nvme-cli. 2022-07-21T01:47:01.899 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-07-21T01:47:01.961 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-07-21T01:47:01.988 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package cephfs-shell. 2022-07-21T01:47:02.004 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:47:02.043 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-07-21T01:47:02.046 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking cephfs-shell (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:02.089 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-attr (19.3.0-2) ... 2022-07-21T01:47:02.258 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libcephfs-dev. 2022-07-21T01:47:02.271 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:47:02.300 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libcephfs-dev (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:02.349 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-07-21T01:47:02.351 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-atomicwrites. 2022-07-21T01:47:02.371 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-07-21T01:47:02.407 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-07-21T01:47:02.419 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-07-21T01:47:02.496 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-07-21T01:47:02.556 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-07-21T01:47:02.659 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-07-21T01:47:02.695 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package nvme-cli. 2022-07-21T01:47:02.713 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-07-21T01:47:02.727 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-attr. 2022-07-21T01:47:02.748 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-07-21T01:47:02.760 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-colorama. 2022-07-21T01:47:02.770 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-07-21T01:47:02.773 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2022-07-21T01:47:02.781 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-07-21T01:47:02.817 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2022-07-21T01:47:02.822 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-07-21T01:47:02.844 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:03.072 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-webencodings. 2022-07-21T01:47:03.085 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-07-21T01:47:03.103 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephfs-shell. 2022-07-21T01:47:03.120 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../085-cephfs-shell_15.2.16-336-g57544da1-1focal_all.deb ... 2022-07-21T01:47:03.129 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-07-21T01:47:03.152 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-atomicwrites. 2022-07-21T01:47:03.163 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephfs-shell (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:03.169 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-07-21T01:47:03.203 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-07-21T01:47:03.232 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-07-21T01:47:03.380 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs-dev. 2022-07-21T01:47:03.394 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:47:03.398 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-html5lib. 2022-07-21T01:47:03.413 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-07-21T01:47:03.430 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs-dev (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:03.455 INFO:teuthology.orchestra.run.smithi140.stdout:Adding system user cephadm....done 2022-07-21T01:47:03.456 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-07-21T01:47:03.496 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-attr. 2022-07-21T01:47:03.508 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-07-21T01:47:03.511 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-07-21T01:47:03.554 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-07-21T01:47:03.608 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-07-21T01:47:03.793 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-07-21T01:47:03.818 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-07-21T01:47:03.833 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-07-21T01:47:03.834 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package nvme-cli. 2022-07-21T01:47:03.846 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-07-21T01:47:03.873 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-webencodings. 2022-07-21T01:47:03.876 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-07-21T01:47:03.888 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-07-21T01:47:03.901 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-07-21T01:47:03.931 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-07-21T01:47:04.000 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-07-21T01:47:04.069 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-07-21T01:47:04.235 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-html5lib. 2022-07-21T01:47:04.249 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-07-21T01:47:04.288 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-atomicwrites. 2022-07-21T01:47:04.302 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-07-21T01:47:04.303 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-07-21T01:47:04.318 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-07-21T01:47:04.321 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-packaging. 2022-07-21T01:47:04.335 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-07-21T01:47:04.345 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-07-21T01:47:04.370 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-packaging (20.3-1) ... 2022-07-21T01:47:04.371 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-07-21T01:47:04.454 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-07-21T01:47:04.637 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-ceph-argparse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:04.646 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-07-21T01:47:04.648 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pastescript. 2022-07-21T01:47:04.649 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-attr. 2022-07-21T01:47:04.660 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-07-21T01:47:04.662 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-07-21T01:47:04.663 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2022-07-21T01:47:04.695 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-07-21T01:47:04.701 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-07-21T01:47:04.707 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-07-21T01:47:04.779 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-07-21T01:47:04.939 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-07-21T01:47:04.986 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webencodings. 2022-07-21T01:47:05.001 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-07-21T01:47:05.035 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-07-21T01:47:05.052 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pluggy. 2022-07-21T01:47:05.064 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-07-21T01:47:05.070 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-07-21T01:47:05.136 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-07-21T01:47:05.141 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-packaging. 2022-07-21T01:47:05.156 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-07-21T01:47:05.182 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-07-21T01:47:05.210 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-packaging (20.3-1) ... 2022-07-21T01:47:05.330 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-html5lib. 2022-07-21T01:47:05.341 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-07-21T01:47:05.347 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2022-07-21T01:47:05.398 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-07-21T01:47:05.446 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-psutil. 2022-07-21T01:47:05.458 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-07-21T01:47:05.463 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-07-21T01:47:05.505 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-07-21T01:47:05.510 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pastescript. 2022-07-21T01:47:05.522 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-07-21T01:47:05.551 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-07-21T01:47:05.600 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-07-21T01:47:05.750 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-07-21T01:47:05.766 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-07-21T01:47:05.767 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-07-21T01:47:05.808 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-07-21T01:47:05.846 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pluggy. 2022-07-21T01:47:05.857 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-py. 2022-07-21T01:47:05.861 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-07-21T01:47:05.872 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-07-21T01:47:05.885 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-07-21T01:47:05.896 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-07-21T01:47:05.917 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-07-21T01:47:06.052 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-07-21T01:47:06.233 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-psutil. 2022-07-21T01:47:06.236 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pyinotify. 2022-07-21T01:47:06.248 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-07-21T01:47:06.250 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-07-21T01:47:06.252 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-07-21T01:47:06.253 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-packaging. 2022-07-21T01:47:06.266 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2022-07-21T01:47:06.290 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-07-21T01:47:06.293 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-07-21T01:47:06.311 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-packaging (20.3-1) ... 2022-07-21T01:47:06.372 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-07-21T01:47:06.528 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-07-21T01:47:06.604 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pytest. 2022-07-21T01:47:06.619 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-py. 2022-07-21T01:47:06.621 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-07-21T01:47:06.631 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastescript. 2022-07-21T01:47:06.635 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-07-21T01:47:06.645 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2022-07-21T01:47:06.664 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-07-21T01:47:06.677 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-07-21T01:47:06.681 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-07-21T01:47:06.695 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-ceph-argparse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:06.897 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-07-21T01:47:06.976 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pluggy. 2022-07-21T01:47:06.992 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2022-07-21T01:47:07.008 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-simplejson. 2022-07-21T01:47:07.013 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pyinotify. 2022-07-21T01:47:07.027 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-07-21T01:47:07.028 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-07-21T01:47:07.059 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-07-21T01:47:07.071 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-07-21T01:47:07.073 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-07-21T01:47:07.074 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-07-21T01:47:07.181 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-07-21T01:47:07.206 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-07-21T01:47:07.370 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-psutil. 2022-07-21T01:47:07.387 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-07-21T01:47:07.428 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-07-21T01:47:07.490 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-07-21T01:47:07.516 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-07-21T01:47:07.528 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package radosgw. 2022-07-21T01:47:07.543 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../101-radosgw_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:47:07.586 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking radosgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:07.626 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-pytest. 2022-07-21T01:47:07.638 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-07-21T01:47:07.682 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-07-21T01:47:07.749 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-py. 2022-07-21T01:47:07.763 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../097-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-07-21T01:47:07.798 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-07-21T01:47:07.811 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-07-21T01:47:07.812 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-packaging (20.3-1) ... 2022-07-21T01:47:08.045 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package python3-simplejson. 2022-07-21T01:47:08.061 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-07-21T01:47:08.061 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyinotify. 2022-07-21T01:47:08.077 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-07-21T01:47:08.084 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-07-21T01:47:08.104 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-07-21T01:47:08.118 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-07-21T01:47:08.126 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-07-21T01:47:08.310 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-07-21T01:47:08.456 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pytest. 2022-07-21T01:47:08.470 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2022-07-21T01:47:08.511 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-07-21T01:47:08.512 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-07-21T01:47:08.532 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package radosgw. 2022-07-21T01:47:08.545 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../101-radosgw_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:47:08.588 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking radosgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:08.669 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-07-21T01:47:08.698 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package rbd-fuse. 2022-07-21T01:47:08.715 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:47:08.747 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking rbd-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:08.811 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-07-21T01:47:08.832 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplejson. 2022-07-21T01:47:08.846 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-07-21T01:47:08.890 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-07-21T01:47:09.071 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-07-21T01:47:09.108 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-attr (19.3.0-2) ... 2022-07-21T01:47:09.121 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-07-21T01:47:09.335 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package radosgw. 2022-07-21T01:47:09.349 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../101-radosgw_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:47:09.372 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-07-21T01:47:09.390 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-07-21T01:47:09.393 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking radosgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:09.460 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-07-21T01:47:09.569 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-07-21T01:47:09.695 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-07-21T01:47:09.706 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package rbd-fuse. 2022-07-21T01:47:09.715 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:47:09.732 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-07-21T01:47:09.742 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-07-21T01:47:09.751 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking rbd-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:09.914 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:10.043 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-attr (19.3.0-2) ... 2022-07-21T01:47:10.050 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-packaging (20.3-1) ... 2022-07-21T01:47:10.105 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up socat (1.7.3.3-2) ... 2022-07-21T01:47:10.203 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:10.334 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-07-21T01:47:10.394 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-07-21T01:47:10.480 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package rbd-fuse. 2022-07-21T01:47:10.485 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-07-21T01:47:10.494 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../102-rbd-fuse_15.2.16-336-g57544da1-1focal_amd64.deb ... 2022-07-21T01:47:10.517 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-07-21T01:47:10.528 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-07-21T01:47:10.538 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking rbd-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:10.541 INFO:teuthology.orchestra.run.smithi191.stdout:Adding system user cephadm....done 2022-07-21T01:47:10.615 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-07-21T01:47:10.661 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-07-21T01:47:10.786 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-07-21T01:47:10.801 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:10.828 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-attr (19.3.0-2) ... 2022-07-21T01:47:10.854 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-07-21T01:47:10.971 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-07-21T01:47:11.012 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-07-21T01:47:11.151 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-07-21T01:47:11.179 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-07-21T01:47:11.256 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-07-21T01:47:11.261 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-07-21T01:47:11.305 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-07-21T01:47:11.370 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-07-21T01:47:11.381 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-07-21T01:47:11.433 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-07-21T01:47:11.444 INFO:teuthology.orchestra.run.smithi192.stdout:Adding system user cephadm....done 2022-07-21T01:47:11.481 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-07-21T01:47:11.555 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:11.605 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-07-21T01:47:11.716 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-07-21T01:47:11.766 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-07-21T01:47:11.850 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-07-21T01:47:11.923 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-07-21T01:47:12.001 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-07-21T01:47:12.210 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-07-21T01:47:12.211 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-07-21T01:47:12.220 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user cephadm....done 2022-07-21T01:47:12.285 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-07-21T01:47:12.317 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up socat (1.7.3.3-2) ... 2022-07-21T01:47:12.342 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-07-21T01:47:12.386 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-07-21T01:47:12.418 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:12.486 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-07-21T01:47:12.494 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-07-21T01:47:12.645 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-07-21T01:47:12.710 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-07-21T01:47:12.728 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-07-21T01:47:12.782 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-07-21T01:47:12.846 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-07-21T01:47:12.913 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-07-21T01:47:13.054 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-07-21T01:47:13.084 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-07-21T01:47:13.114 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-07-21T01:47:13.147 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-07-21T01:47:13.205 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-07-21T01:47:13.223 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-07-21T01:47:13.238 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-07-21T01:47:13.330 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-07-21T01:47:13.439 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-07-21T01:47:13.518 INFO:teuthology.orchestra.run.smithi152.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-07-21T01:47:13.531 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-07-21T01:47:13.541 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-07-21T01:47:13.623 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-ceph-argparse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:13.685 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-07-21T01:47:13.730 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-07-21T01:47:13.808 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-07-21T01:47:13.909 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-07-21T01:47:13.933 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-07-21T01:47:13.951 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-07-21T01:47:13.992 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-07-21T01:47:14.001 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-07-21T01:47:14.058 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-07-21T01:47:14.202 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-07-21T01:47:14.236 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-07-21T01:47:14.275 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-07-21T01:47:14.302 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-07-21T01:47:14.335 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-07-21T01:47:14.528 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-07-21T01:47:14.529 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2022-07-21T01:47:14.559 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-ceph-argparse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:14.569 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-07-21T01:47:14.612 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-07-21T01:47:14.654 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-07-21T01:47:14.721 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-07-21T01:47:14.805 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-07-21T01:47:14.813 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-07-21T01:47:14.852 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-07-21T01:47:14.878 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-07-21T01:47:14.985 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-07-21T01:47:15.013 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-07-21T01:47:15.114 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-07-21T01:47:15.139 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-07-21T01:47:15.238 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-07-21T01:47:15.263 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-07-21T01:47:15.348 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-07-21T01:47:15.407 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-argparse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:15.432 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up librados2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:15.514 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-07-21T01:47:15.547 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-07-21T01:47:15.558 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-07-21T01:47:15.646 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-07-21T01:47:15.708 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-07-21T01:47:15.840 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-07-21T01:47:15.856 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-07-21T01:47:15.860 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2022-07-21T01:47:15.974 INFO:teuthology.orchestra.run.smithi140.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-07-21T01:47:16.135 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-07-21T01:47:16.143 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-07-21T01:47:16.224 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-07-21T01:47:16.228 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-07-21T01:47:16.327 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-07-21T01:47:16.453 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-07-21T01:47:16.478 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-07-21T01:47:16.534 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-07-21T01:47:16.662 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-07-21T01:47:16.737 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-07-21T01:47:16.787 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-packaging (20.3-1) ... 2022-07-21T01:47:16.860 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-07-21T01:47:17.005 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up librgw2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:17.047 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-07-21T01:47:17.079 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-07-21T01:47:17.131 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:47:17.136 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-07-21T01:47:17.147 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-07-21T01:47:17.198 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-07-21T01:47:17.239 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-07-21T01:47:17.265 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-07-21T01:47:17.325 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-07-21T01:47:17.407 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-07-21T01:47:17.454 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-07-21T01:47:17.506 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-07-21T01:47:17.642 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-07-21T01:47:17.643 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-07-21T01:47:17.733 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-07-21T01:47:17.743 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-packaging (20.3-1) ... 2022-07-21T01:47:17.908 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-07-21T01:47:17.918 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up librados2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:17.931 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-07-21T01:47:18.025 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-07-21T01:47:18.027 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-07-21T01:47:18.031 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-07-21T01:47:18.157 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-07-21T01:47:18.192 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-07-21T01:47:18.202 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-07-21T01:47:18.360 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-07-21T01:47:18.475 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-07-21T01:47:18.520 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libcephfs2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:18.585 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-07-21T01:47:18.646 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-07-21T01:47:18.686 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-packaging (20.3-1) ... 2022-07-21T01:47:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-07-21T01:47:18.946 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up socat (1.7.3.3-2) ... 2022-07-21T01:47:18.962 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-07-21T01:47:18.973 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libradosstriper1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:19.071 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:19.088 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-07-21T01:47:19.107 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-07-21T01:47:19.161 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-07-21T01:47:19.373 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-07-21T01:47:19.381 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-07-21T01:47:19.433 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-07-21T01:47:19.476 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up librgw2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:19.496 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-07-21T01:47:19.617 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:47:19.709 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-07-21T01:47:19.727 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-07-21T01:47:19.758 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2022-07-21T01:47:19.809 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-07-21T01:47:19.855 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up socat (1.7.3.3-2) ... 2022-07-21T01:47:20.014 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:20.093 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-07-21T01:47:20.095 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-07-21T01:47:20.126 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-07-21T01:47:20.252 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-07-21T01:47:20.316 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-07-21T01:47:20.396 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-07-21T01:47:20.477 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-07-21T01:47:20.479 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up librbd1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:20.609 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-07-21T01:47:20.610 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-07-21T01:47:20.684 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-07-21T01:47:20.698 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-07-21T01:47:20.731 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-07-21T01:47:20.785 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-07-21T01:47:20.845 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up socat (1.7.3.3-2) ... 2022-07-21T01:47:20.858 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-07-21T01:47:20.888 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-07-21T01:47:20.971 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:21.049 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libcephfs2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:21.086 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-07-21T01:47:21.167 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:47:21.175 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-07-21T01:47:21.187 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-07-21T01:47:21.197 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-07-21T01:47:21.256 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-07-21T01:47:21.268 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-rsa (4.0-3) ... 2022-07-21T01:47:21.332 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-07-21T01:47:21.504 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libradosstriper1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:21.507 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-07-21T01:47:21.536 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-07-21T01:47:21.574 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-07-21T01:47:21.583 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-07-21T01:47:21.603 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-07-21T01:47:21.700 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-07-21T01:47:21.798 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-07-21T01:47:21.822 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr-modules-core (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:21.858 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-07-21T01:47:21.946 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:21.954 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-07-21T01:47:21.999 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2022-07-21T01:47:22.085 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-07-21T01:47:22.110 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-07-21T01:47:22.185 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-07-21T01:47:22.208 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-07-21T01:47:22.209 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-07-21T01:47:22.209 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-07-21T01:47:22.428 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-07-21T01:47:22.502 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-07-21T01:47:22.641 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libcephfs-dev (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:22.746 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-07-21T01:47:22.756 INFO:teuthology.orchestra.run.smithi191.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-07-21T01:47:22.765 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-07-21T01:47:22.802 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-07-21T01:47:23.012 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-07-21T01:47:23.018 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up librbd1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:23.022 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-07-21T01:47:23.066 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-07-21T01:47:23.078 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-07-21T01:47:23.124 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-07-21T01:47:23.139 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-07-21T01:47:23.161 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-07-21T01:47:23.270 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-07-21T01:47:23.379 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-07-21T01:47:23.384 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-07-21T01:47:23.396 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-07-21T01:47:23.417 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-07-21T01:47:23.457 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-07-21T01:47:23.639 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-07-21T01:47:23.717 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-07-21T01:47:23.731 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:47:23.736 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-rados (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:23.756 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-07-21T01:47:23.803 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-07-21T01:47:23.845 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-rbd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:23.882 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-rsa (4.0-3) ... 2022-07-21T01:47:23.900 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-07-21T01:47:23.971 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up rbd-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:24.018 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-07-21T01:47:24.039 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-07-21T01:47:24.074 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-07-21T01:47:24.122 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-routes (2.4.1-2) ... 2022-07-21T01:47:24.200 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-07-21T01:47:24.310 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-07-21T01:47:24.369 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-07-21T01:47:24.434 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-07-21T01:47:24.500 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-07-21T01:47:24.534 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-mgr-modules-core (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:24.560 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-07-21T01:47:24.571 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-07-21T01:47:24.643 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:24.661 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librados2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:24.686 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-07-21T01:47:24.688 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2022-07-21T01:47:24.788 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-07-21T01:47:24.801 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-rgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:24.852 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-07-21T01:47:24.852 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-07-21T01:47:24.918 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-cephfs (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:24.990 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-07-21T01:47:25.012 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-07-21T01:47:25.203 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up cephfs-shell (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:25.271 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up librados2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:25.303 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libcephfs-dev (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:25.324 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-07-21T01:47:25.329 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-07-21T01:47:25.372 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-07-21T01:47:25.429 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-07-21T01:47:25.451 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-07-21T01:47:25.576 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-07-21T01:47:25.739 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-07-21T01:47:25.903 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-07-21T01:47:26.057 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-07-21T01:47:26.187 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librados2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:26.248 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:26.268 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librgw2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:26.313 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-07-21T01:47:26.394 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:47:26.434 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-rados (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:26.520 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-07-21T01:47:26.551 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-rbd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:26.688 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up rbd-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:26.792 INFO:teuthology.orchestra.run.smithi152.stdout:Adding group ceph....done 2022-07-21T01:47:26.804 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-routes (2.4.1-2) ... 2022-07-21T01:47:26.837 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up librgw2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:26.930 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-07-21T01:47:26.996 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:47:27.122 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-07-21T01:47:27.177 INFO:teuthology.orchestra.run.smithi152.stdout:Adding system user ceph....done 2022-07-21T01:47:27.191 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-07-21T01:47:27.222 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-07-21T01:47:27.339 INFO:teuthology.orchestra.run.smithi152.stdout:Setting system user ceph properties....done 2022-07-21T01:47:27.354 INFO:teuthology.orchestra.run.smithi152.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-07-21T01:47:27.498 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-07-21T01:47:27.499 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-rgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:27.506 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-07-21T01:47:27.532 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-07-21T01:47:27.643 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-cephfs (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:27.760 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librgw2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:27.823 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-07-21T01:47:27.824 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-07-21T01:47:27.858 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libcephfs2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:27.963 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up cephfs-shell (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:28.009 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-07-21T01:47:28.119 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-07-21T01:47:28.143 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-07-21T01:47:28.261 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:47:28.276 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-test (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:28.328 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libradosstriper1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:28.388 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-07-21T01:47:28.402 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up radosgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:28.453 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-07-21T01:47:28.469 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libcephfs2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:28.587 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-07-21T01:47:28.672 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-07-21T01:47:28.673 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-07-21T01:47:28.765 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-07-21T01:47:28.792 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-07-21T01:47:28.929 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libradosstriper1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:29.014 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:29.042 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-07-21T01:47:29.067 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-07-21T01:47:29.320 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-base (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:29.350 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-07-21T01:47:29.399 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-07-21T01:47:29.591 INFO:teuthology.orchestra.run.smithi140.stdout:Adding group ceph....done 2022-07-21T01:47:29.692 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs2 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:29.700 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-07-21T01:47:29.819 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-07-21T01:47:29.836 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librbd1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:29.935 INFO:teuthology.orchestra.run.smithi140.stdout:Adding system user ceph....done 2022-07-21T01:47:29.961 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-07-21T01:47:30.110 INFO:teuthology.orchestra.run.smithi140.stdout:Setting system user ceph properties....done 2022-07-21T01:47:30.111 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-07-21T01:47:30.124 INFO:teuthology.orchestra.run.smithi140.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-07-21T01:47:30.137 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libradosstriper1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:30.156 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mds (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:30.238 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-07-21T01:47:30.263 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-07-21T01:47:30.268 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-07-21T01:47:30.399 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-07-21T01:47:30.399 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-07-21T01:47:30.462 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up librbd1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:30.556 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:47:30.588 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-07-21T01:47:30.589 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-07-21T01:47:30.598 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-07-21T01:47:30.707 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rsa (4.0-3) ... 2022-07-21T01:47:30.714 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-07-21T01:47:30.840 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-07-21T01:47:30.841 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:30.991 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-07-21T01:47:31.057 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-test (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:31.062 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-07-21T01:47:31.063 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-07-21T01:47:31.175 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:47:31.182 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up radosgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:31.268 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-modules-core (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:31.318 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-rsa (4.0-3) ... 2022-07-21T01:47:31.402 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:31.479 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-07-21T01:47:31.479 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-07-21T01:47:31.525 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:31.603 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-07-21T01:47:31.615 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-07-21T01:47:31.616 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-07-21T01:47:31.661 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librbd1 (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:31.668 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-osd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:31.787 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-07-21T01:47:31.887 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-modules-core (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:31.922 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-07-21T01:47:32.013 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:32.053 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libcephfs-dev (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:32.065 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-07-21T01:47:32.119 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-base (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:32.166 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-07-21T01:47:32.167 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-07-21T01:47:32.179 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-07-21T01:47:32.235 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-07-21T01:47:32.237 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-07-21T01:47:32.366 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-07-21T01:47:32.464 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-07-21T01:47:32.469 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rsa (4.0-3) ... 2022-07-21T01:47:32.498 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-07-21T01:47:32.609 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr-k8sevents (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:32.697 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libcephfs-dev (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:32.729 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:32.752 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-07-21T01:47:32.790 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-07-21T01:47:32.840 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-07-21T01:47:32.872 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mon (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:32.945 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-mds (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.041 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-modules-core (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.100 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-07-21T01:47:33.100 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-07-21T01:47:33.125 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-07-21T01:47:33.142 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rados (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.162 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.193 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-07-21T01:47:33.193 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-07-21T01:47:33.244 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rbd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.394 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up rbd-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.396 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-07-21T01:47:33.396 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-07-21T01:47:33.442 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-07-21T01:47:33.547 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr-cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.553 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-routes (2.4.1-2) ... 2022-07-21T01:47:33.629 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-mgr (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.682 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.799 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up ceph-mgr-dashboard (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.804 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-rados (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.856 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs-dev (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.891 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-07-21T01:47:33.891 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-07-21T01:47:33.918 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-07-21T01:47:33.930 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-07-21T01:47:33.940 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-rbd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:33.982 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-07-21T01:47:34.055 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up rbd-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:34.156 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-routes (2.4.1-2) ... 2022-07-21T01:47:34.206 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-07-21T01:47:34.257 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:34.284 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-07-21T01:47:34.332 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:34.358 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cephfs (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:34.499 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-osd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:34.534 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-07-21T01:47:34.584 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-07-21T01:47:34.637 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up cephfs-shell (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:34.760 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-07-21T01:47:34.843 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-rgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:34.937 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rados (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:34.970 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-cephfs (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:35.006 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-07-21T01:47:35.006 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-07-21T01:47:35.063 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rbd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:35.197 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up rbd-fuse (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:35.262 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up cephfs-shell (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:35.323 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-routes (2.4.1-2) ... 2022-07-21T01:47:35.380 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-07-21T01:47:35.459 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-mgr-k8sevents (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:35.609 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:35.680 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:35.717 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-07-21T01:47:35.752 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-mon (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:36.000 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-07-21T01:47:36.001 INFO:teuthology.orchestra.run.smithi140.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-07-21T01:47:36.004 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:36.153 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cephfs (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:36.241 INFO:teuthology.orchestra.run.smithi191.stdout:Adding group ceph....done 2022-07-21T01:47:36.266 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:36.453 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-mgr-cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:36.487 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephfs-shell (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:36.617 INFO:teuthology.orchestra.run.smithi191.stdout:Adding system user ceph....done 2022-07-21T01:47:36.621 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:36.622 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-07-21T01:47:36.747 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up ceph-mgr-dashboard (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:36.810 INFO:teuthology.orchestra.run.smithi191.stdout:Setting system user ceph properties....done 2022-07-21T01:47:36.823 INFO:teuthology.orchestra.run.smithi191.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-07-21T01:47:36.828 INFO:teuthology.orchestra.run.smithi192.stdout:Adding group ceph....done 2022-07-21T01:47:36.866 INFO:teuthology.orchestra.run.smithi140.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-07-21T01:47:36.889 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-07-21T01:47:36.976 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-07-21T01:47:37.154 INFO:teuthology.orchestra.run.smithi192.stdout:Adding system user ceph....done 2022-07-21T01:47:37.167 INFO:teuthology.orchestra.run.smithi140.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-07-21T01:47:37.291 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-07-21T01:47:37.295 INFO:teuthology.orchestra.run.smithi152.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-07-21T01:47:37.298 DEBUG:teuthology.orchestra.run.smithi152:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-07-21T01:47:37.364 INFO:teuthology.orchestra.run.smithi192.stdout:Setting system user ceph properties....done 2022-07-21T01:47:37.370 INFO:teuthology.orchestra.run.smithi152.stdout:Reading package lists... 2022-07-21T01:47:37.378 INFO:teuthology.orchestra.run.smithi192.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-07-21T01:47:37.519 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-07-21T01:47:37.550 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-common (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:37.553 INFO:teuthology.orchestra.run.smithi152.stdout:Building dependency tree... 2022-07-21T01:47:37.554 INFO:teuthology.orchestra.run.smithi152.stdout:Reading state information... 2022-07-21T01:47:37.710 INFO:teuthology.orchestra.run.smithi152.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-07-21T01:47:37.711 INFO:teuthology.orchestra.run.smithi152.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-07-21T01:47:37.711 INFO:teuthology.orchestra.run.smithi152.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-07-21T01:47:37.711 INFO:teuthology.orchestra.run.smithi152.stdout:The following packages were automatically installed and are no longer required: 2022-07-21T01:47:37.712 INFO:teuthology.orchestra.run.smithi152.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-07-21T01:47:37.712 INFO:teuthology.orchestra.run.smithi152.stdout:Use 'sudo apt autoremove' to remove them. 2022-07-21T01:47:37.713 INFO:teuthology.orchestra.run.smithi152.stdout:Suggested packages: 2022-07-21T01:47:37.713 INFO:teuthology.orchestra.run.smithi152.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-07-21T01:47:37.745 INFO:teuthology.orchestra.run.smithi152.stdout:The following NEW packages will be installed: 2022-07-21T01:47:37.747 INFO:teuthology.orchestra.run.smithi152.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-07-21T01:47:37.747 INFO:teuthology.orchestra.run.smithi152.stdout: libssl-dev 2022-07-21T01:47:37.764 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-test (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:37.834 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-07-21T01:47:37.890 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up radosgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:37.950 INFO:teuthology.orchestra.run.smithi152.stdout:0 upgraded, 10 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:47:37.951 INFO:teuthology.orchestra.run.smithi152.stdout:Need to get 2,914 kB of archives. 2022-07-21T01:47:37.951 INFO:teuthology.orchestra.run.smithi152.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-07-21T01:47:37.951 INFO:teuthology.orchestra.run.smithi152.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-07-21T01:47:38.036 INFO:teuthology.orchestra.run.smithi195.stdout:Adding group ceph....done 2022-07-21T01:47:38.165 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-07-21T01:47:38.165 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-07-21T01:47:38.379 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user ceph....done 2022-07-21T01:47:38.431 INFO:teuthology.orchestra.run.smithi152.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-07-21T01:47:38.506 INFO:teuthology.orchestra.run.smithi195.stdout:Setting system user ceph properties....done 2022-07-21T01:47:38.522 INFO:teuthology.orchestra.run.smithi195.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-07-21T01:47:38.534 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-test (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:38.570 INFO:teuthology.orchestra.run.smithi152.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-07-21T01:47:38.578 INFO:teuthology.orchestra.run.smithi152.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-07-21T01:47:38.597 INFO:teuthology.orchestra.run.smithi152.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-07-21T01:47:38.602 INFO:teuthology.orchestra.run.smithi152.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-07-21T01:47:38.603 INFO:teuthology.orchestra.run.smithi152.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-07-21T01:47:38.607 INFO:teuthology.orchestra.run.smithi152.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-07-21T01:47:38.608 INFO:teuthology.orchestra.run.smithi152.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-07-21T01:47:38.609 INFO:teuthology.orchestra.run.smithi152.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-07-21T01:47:38.659 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-07-21T01:47:38.661 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up radosgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:38.826 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-base (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:38.934 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-07-21T01:47:38.934 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-07-21T01:47:38.962 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-07-21T01:47:39.056 INFO:teuthology.orchestra.run.smithi152.stdout:Fetched 2,914 kB in 1s (3,045 kB/s) 2022-07-21T01:47:39.183 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-07-21T01:47:39.236 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package flex. 2022-07-21T01:47:39.280 INFO:teuthology.orchestra.run.smithi152.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-07-21T01:47:39.283 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:39.325 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking flex (2.6.4-6.2) ... 2022-07-21T01:47:39.455 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-test (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:39.572 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-base (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:39.588 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up radosgw (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:39.637 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mds (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:39.830 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package bison. 2022-07-21T01:47:39.843 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-07-21T01:47:39.845 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-07-21T01:47:39.845 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-07-21T01:47:39.853 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-07-21T01:47:39.854 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-07-21T01:47:39.885 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-07-21T01:47:39.912 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-07-21T01:47:40.054 INFO:teuthology.orchestra.run.smithi140.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-07-21T01:47:40.284 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-07-21T01:47:40.295 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:40.300 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-07-21T01:47:40.356 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mds (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:40.426 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-07-21T01:47:40.489 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-base (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:40.497 INFO:teuthology.orchestra.run.smithi140.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-07-21T01:47:40.500 DEBUG:teuthology.orchestra.run.smithi140:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-07-21T01:47:40.528 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-07-21T01:47:40.529 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-07-21T01:47:40.573 INFO:teuthology.orchestra.run.smithi140.stdout:Reading package lists... 2022-07-21T01:47:40.580 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-07-21T01:47:40.581 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-07-21T01:47:40.745 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package dump. 2022-07-21T01:47:40.759 INFO:teuthology.orchestra.run.smithi140.stdout:Building dependency tree... 2022-07-21T01:47:40.760 INFO:teuthology.orchestra.run.smithi140.stdout:Reading state information... 2022-07-21T01:47:40.760 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-07-21T01:47:40.795 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking dump (0.4b46-6) ... 2022-07-21T01:47:40.851 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-07-21T01:47:40.919 INFO:teuthology.orchestra.run.smithi140.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-07-21T01:47:40.919 INFO:teuthology.orchestra.run.smithi140.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-07-21T01:47:40.919 INFO:teuthology.orchestra.run.smithi140.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-07-21T01:47:40.920 INFO:teuthology.orchestra.run.smithi140.stdout:The following packages were automatically installed and are no longer required: 2022-07-21T01:47:40.920 INFO:teuthology.orchestra.run.smithi140.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-07-21T01:47:40.921 INFO:teuthology.orchestra.run.smithi140.stdout:Use 'sudo apt autoremove' to remove them. 2022-07-21T01:47:40.921 INFO:teuthology.orchestra.run.smithi140.stdout:Suggested packages: 2022-07-21T01:47:40.921 INFO:teuthology.orchestra.run.smithi140.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-07-21T01:47:40.953 INFO:teuthology.orchestra.run.smithi140.stdout:The following NEW packages will be installed: 2022-07-21T01:47:40.953 INFO:teuthology.orchestra.run.smithi140.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-07-21T01:47:40.954 INFO:teuthology.orchestra.run.smithi140.stdout: libssl-dev 2022-07-21T01:47:40.988 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:41.008 INFO:teuthology.orchestra.run.smithi140.stdout:0 upgraded, 10 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:47:41.009 INFO:teuthology.orchestra.run.smithi140.stdout:Need to get 2,914 kB of archives. 2022-07-21T01:47:41.009 INFO:teuthology.orchestra.run.smithi140.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-07-21T01:47:41.009 INFO:teuthology.orchestra.run.smithi140.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-07-21T01:47:41.041 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:41.065 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package indent. 2022-07-21T01:47:41.082 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-07-21T01:47:41.128 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking indent (2.2.12-1) ... 2022-07-21T01:47:41.131 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-osd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:41.142 INFO:teuthology.orchestra.run.smithi140.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-07-21T01:47:41.269 INFO:teuthology.orchestra.run.smithi140.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-07-21T01:47:41.270 INFO:teuthology.orchestra.run.smithi140.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-07-21T01:47:41.292 INFO:teuthology.orchestra.run.smithi140.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-07-21T01:47:41.293 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-07-21T01:47:41.293 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-07-21T01:47:41.311 INFO:teuthology.orchestra.run.smithi140.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-07-21T01:47:41.311 INFO:teuthology.orchestra.run.smithi140.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-07-21T01:47:41.314 INFO:teuthology.orchestra.run.smithi140.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-07-21T01:47:41.315 INFO:teuthology.orchestra.run.smithi140.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-07-21T01:47:41.316 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mds (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:41.316 INFO:teuthology.orchestra.run.smithi140.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-07-21T01:47:41.393 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-07-21T01:47:41.407 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-07-21T01:47:41.452 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-07-21T01:47:41.526 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-07-21T01:47:41.527 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-07-21T01:47:41.627 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-07-21T01:47:41.628 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-07-21T01:47:41.737 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-07-21T01:47:41.752 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-07-21T01:47:41.795 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-07-21T01:47:41.844 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:41.918 INFO:teuthology.orchestra.run.smithi140.stdout:Fetched 2,914 kB in 1s (4,860 kB/s) 2022-07-21T01:47:41.967 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:41.970 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-osd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:42.058 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package flex. 2022-07-21T01:47:42.102 INFO:teuthology.orchestra.run.smithi140.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-07-21T01:47:42.105 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:42.106 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libfl2:amd64. 2022-07-21T01:47:42.116 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-k8sevents (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:42.121 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:42.149 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking flex (2.6.4-6.2) ... 2022-07-21T01:47:42.156 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:42.194 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-07-21T01:47:42.195 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-07-21T01:47:42.225 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:42.352 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mon (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:42.392 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-07-21T01:47:42.406 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:42.436 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-07-21T01:47:42.436 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-07-21T01:47:42.450 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:42.580 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-07-21T01:47:42.581 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-07-21T01:47:42.652 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:42.670 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package bison. 2022-07-21T01:47:42.685 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-07-21T01:47:42.720 INFO:teuthology.orchestra.run.smithi152.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-07-21T01:47:42.732 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-07-21T01:47:42.733 INFO:teuthology.orchestra.run.smithi152.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-07-21T01:47:42.778 INFO:teuthology.orchestra.run.smithi152.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-07-21T01:47:42.819 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-osd (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:42.907 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-k8sevents (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:43.014 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:43.078 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:43.140 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mon (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:43.214 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:43.252 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up flex (2.6.4-6.2) ... 2022-07-21T01:47:43.303 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-07-21T01:47:43.315 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-07-21T01:47:43.316 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-07-21T01:47:43.316 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-07-21T01:47:43.338 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-dashboard (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:43.371 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-07-21T01:47:43.371 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-07-21T01:47:43.372 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-07-21T01:47:43.408 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-07-21T01:47:43.467 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-07-21T01:47:43.524 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-07-21T01:47:43.585 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-07-21T01:47:43.753 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-07-21T01:47:43.761 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package dump. 2022-07-21T01:47:43.775 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-07-21T01:47:43.788 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-k8sevents (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:43.816 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-07-21T01:47:43.819 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking dump (0.4b46-6) ... 2022-07-21T01:47:43.835 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:43.862 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:43.908 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-diskprediction-local (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:43.968 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:43.987 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up dump (0.4b46-6) ... 2022-07-21T01:47:44.035 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mon (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:44.082 INFO:teuthology.orchestra.run.smithi152.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-07-21T01:47:44.094 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up ceph-mgr-dashboard (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:44.139 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-07-21T01:47:44.148 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package indent. 2022-07-21T01:47:44.165 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-07-21T01:47:44.206 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking indent (2.2.12-1) ... 2022-07-21T01:47:44.228 INFO:teuthology.orchestra.run.smithi152.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-07-21T01:47:44.237 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-07-21T01:47:44.267 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-07-21T01:47:44.268 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-07-21T01:47:44.384 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up indent (2.2.12-1) ... 2022-07-21T01:47:44.501 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-07-21T01:47:44.514 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-07-21T01:47:44.530 INFO:teuthology.orchestra.run.smithi152.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:44.546 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-07-21T01:47:44.550 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-07-21T01:47:44.635 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-07-21T01:47:44.726 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-cephadm (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:44.795 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-07-21T01:47:44.821 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-07-21T01:47:44.834 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-07-21T01:47:44.870 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-07-21T01:47:44.894 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:45.026 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-dashboard (15.2.16-336-g57544da1-1focal) ... 2022-07-21T01:47:45.164 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2022-07-21T01:47:45.233 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libfl2:amd64. 2022-07-21T01:47:45.247 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:45.291 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:45.481 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-07-21T01:47:45.569 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-07-21T01:47:45.582 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:45.630 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:45.897 INFO:teuthology.orchestra.run.smithi140.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-07-21T01:47:45.910 INFO:teuthology.orchestra.run.smithi140.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-07-21T01:47:45.955 INFO:teuthology.orchestra.run.smithi140.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-07-21T01:47:46.472 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up flex (2.6.4-6.2) ... 2022-07-21T01:47:46.485 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-07-21T01:47:46.597 INFO:teuthology.orchestra.run.smithi152.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-07-21T01:47:46.618 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-07-21T01:47:46.744 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-07-21T01:47:46.870 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-07-21T01:47:46.914 INFO:teuthology.orchestra.run.smithi191.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-07-21T01:47:46.918 DEBUG:teuthology.orchestra.run.smithi191:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-07-21T01:47:46.991 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2022-07-21T01:47:47.021 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-07-21T01:47:47.147 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:47.169 INFO:teuthology.orchestra.run.smithi191.stdout:Building dependency tree... 2022-07-21T01:47:47.170 INFO:teuthology.orchestra.run.smithi191.stdout:Reading state information... 2022-07-21T01:47:47.224 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-07-21T01:47:47.293 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up dump (0.4b46-6) ... 2022-07-21T01:47:47.326 INFO:teuthology.orchestra.run.smithi191.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-07-21T01:47:47.327 INFO:teuthology.orchestra.run.smithi191.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-07-21T01:47:47.327 INFO:teuthology.orchestra.run.smithi191.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-07-21T01:47:47.327 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages were automatically installed and are no longer required: 2022-07-21T01:47:47.328 INFO:teuthology.orchestra.run.smithi191.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-07-21T01:47:47.328 INFO:teuthology.orchestra.run.smithi191.stdout:Use 'sudo apt autoremove' to remove them. 2022-07-21T01:47:47.328 INFO:teuthology.orchestra.run.smithi191.stdout:Suggested packages: 2022-07-21T01:47:47.329 INFO:teuthology.orchestra.run.smithi191.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-07-21T01:47:47.334 INFO:teuthology.orchestra.run.smithi152.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-07-21T01:47:47.337 DEBUG:teuthology.parallel:result is None 2022-07-21T01:47:47.362 INFO:teuthology.orchestra.run.smithi191.stdout:The following NEW packages will be installed: 2022-07-21T01:47:47.362 INFO:teuthology.orchestra.run.smithi191.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-07-21T01:47:47.363 INFO:teuthology.orchestra.run.smithi191.stdout: libssl-dev 2022-07-21T01:47:47.394 INFO:teuthology.orchestra.run.smithi140.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-07-21T01:47:47.418 INFO:teuthology.orchestra.run.smithi191.stdout:0 upgraded, 10 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:47:47.419 INFO:teuthology.orchestra.run.smithi191.stdout:Need to get 2,914 kB of archives. 2022-07-21T01:47:47.419 INFO:teuthology.orchestra.run.smithi191.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-07-21T01:47:47.419 INFO:teuthology.orchestra.run.smithi191.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-07-21T01:47:47.449 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-07-21T01:47:47.522 INFO:teuthology.orchestra.run.smithi140.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-07-21T01:47:47.553 INFO:teuthology.orchestra.run.smithi191.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-07-21T01:47:47.593 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up indent (2.2.12-1) ... 2022-07-21T01:47:47.646 INFO:teuthology.orchestra.run.smithi192.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-07-21T01:47:47.650 DEBUG:teuthology.orchestra.run.smithi192:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-07-21T01:47:47.679 INFO:teuthology.orchestra.run.smithi191.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-07-21T01:47:47.682 INFO:teuthology.orchestra.run.smithi191.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-07-21T01:47:47.704 INFO:teuthology.orchestra.run.smithi191.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-07-21T01:47:47.721 INFO:teuthology.orchestra.run.smithi192.stdout:Reading package lists... 2022-07-21T01:47:47.722 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-07-21T01:47:47.722 INFO:teuthology.orchestra.run.smithi191.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-07-21T01:47:47.726 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-07-21T01:47:47.727 INFO:teuthology.orchestra.run.smithi191.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-07-21T01:47:47.728 INFO:teuthology.orchestra.run.smithi191.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-07-21T01:47:47.752 INFO:teuthology.orchestra.run.smithi140.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:47.878 INFO:teuthology.orchestra.run.smithi140.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-07-21T01:47:47.900 INFO:teuthology.orchestra.run.smithi192.stdout:Building dependency tree... 2022-07-21T01:47:47.901 INFO:teuthology.orchestra.run.smithi192.stdout:Reading state information... 2022-07-21T01:47:48.056 INFO:teuthology.orchestra.run.smithi192.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-07-21T01:47:48.056 INFO:teuthology.orchestra.run.smithi192.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-07-21T01:47:48.057 INFO:teuthology.orchestra.run.smithi192.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-07-21T01:47:48.057 INFO:teuthology.orchestra.run.smithi192.stdout:The following packages were automatically installed and are no longer required: 2022-07-21T01:47:48.059 INFO:teuthology.orchestra.run.smithi192.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-07-21T01:47:48.059 INFO:teuthology.orchestra.run.smithi192.stdout:Use 'sudo apt autoremove' to remove them. 2022-07-21T01:47:48.060 INFO:teuthology.orchestra.run.smithi192.stdout:Suggested packages: 2022-07-21T01:47:48.060 INFO:teuthology.orchestra.run.smithi192.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-07-21T01:47:48.063 INFO:teuthology.orchestra.run.smithi140.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-07-21T01:47:48.089 INFO:teuthology.orchestra.run.smithi192.stdout:The following NEW packages will be installed: 2022-07-21T01:47:48.090 INFO:teuthology.orchestra.run.smithi192.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-07-21T01:47:48.091 INFO:teuthology.orchestra.run.smithi192.stdout: libssl-dev 2022-07-21T01:47:48.237 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-07-21T01:47:48.270 INFO:teuthology.orchestra.run.smithi192.stdout:0 upgraded, 10 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:47:48.270 INFO:teuthology.orchestra.run.smithi192.stdout:Need to get 2,914 kB of archives. 2022-07-21T01:47:48.270 INFO:teuthology.orchestra.run.smithi192.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-07-21T01:47:48.271 INFO:teuthology.orchestra.run.smithi192.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-07-21T01:47:48.331 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 2,914 kB in 1s (4,811 kB/s) 2022-07-21T01:47:48.474 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package flex. 2022-07-21T01:47:48.518 INFO:teuthology.orchestra.run.smithi191.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-07-21T01:47:48.521 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:48.564 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking flex (2.6.4-6.2) ... 2022-07-21T01:47:48.641 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-07-21T01:47:48.643 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2022-07-21T01:47:48.718 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2022-07-21T01:47:48.748 INFO:teuthology.orchestra.run.smithi192.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-07-21T01:47:48.887 INFO:teuthology.orchestra.run.smithi192.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-07-21T01:47:48.893 INFO:teuthology.orchestra.run.smithi192.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-07-21T01:47:48.897 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2022-07-21T01:47:48.898 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2022-07-21T01:47:48.911 INFO:teuthology.orchestra.run.smithi192.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-07-21T01:47:48.918 INFO:teuthology.orchestra.run.smithi192.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-07-21T01:47:48.919 INFO:teuthology.orchestra.run.smithi192.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-07-21T01:47:48.924 INFO:teuthology.orchestra.run.smithi192.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-07-21T01:47:48.924 INFO:teuthology.orchestra.run.smithi192.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-07-21T01:47:48.925 INFO:teuthology.orchestra.run.smithi192.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-07-21T01:47:49.044 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package bison. 2022-07-21T01:47:49.056 INFO:teuthology.orchestra.run.smithi195.stdout:libtool-bin is already the newest version (2.4.6-14). 2022-07-21T01:47:49.057 INFO:teuthology.orchestra.run.smithi195.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2022-07-21T01:47:49.057 INFO:teuthology.orchestra.run.smithi195.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.3). 2022-07-21T01:47:49.057 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2022-07-21T01:47:49.058 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-07-21T01:47:49.058 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2022-07-21T01:47:49.059 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2022-07-21T01:47:49.059 INFO:teuthology.orchestra.run.smithi195.stdout: bison-doc flex-doc indent-doc libssl-doc 2022-07-21T01:47:49.060 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-07-21T01:47:49.090 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2022-07-21T01:47:49.090 INFO:teuthology.orchestra.run.smithi195.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2022-07-21T01:47:49.091 INFO:teuthology.orchestra.run.smithi195.stdout: libssl-dev 2022-07-21T01:47:49.106 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-07-21T01:47:49.147 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 10 newly installed, 0 to remove and 2 not upgraded. 2022-07-21T01:47:49.148 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 2,914 kB of archives. 2022-07-21T01:47:49.148 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 12.4 MB of additional disk space will be used. 2022-07-21T01:47:49.148 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2022-07-21T01:47:49.246 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2022-07-21T01:47:49.279 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2022-07-21T01:47:49.290 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2022-07-21T01:47:49.292 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2022-07-21T01:47:49.294 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2022-07-21T01:47:49.294 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2022-07-21T01:47:49.307 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2022-07-21T01:47:49.308 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2022-07-21T01:47:49.308 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.16 [1,584 kB] 2022-07-21T01:47:49.385 INFO:teuthology.orchestra.run.smithi192.stdout:Fetched 2,914 kB in 1s (3,136 kB/s) 2022-07-21T01:47:49.479 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package flex. 2022-07-21T01:47:49.526 INFO:teuthology.orchestra.run.smithi192.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-07-21T01:47:49.530 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:49.539 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-07-21T01:47:49.554 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-07-21T01:47:49.569 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking flex (2.6.4-6.2) ... 2022-07-21T01:47:49.639 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-07-21T01:47:49.749 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 2,914 kB in 0s (9,783 kB/s) 2022-07-21T01:47:49.890 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package flex. 2022-07-21T01:47:49.908 INFO:teuthology.orchestra.run.smithi140.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-07-21T01:47:49.935 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 109762 files and directories currently installed.) 2022-07-21T01:47:49.943 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../0-flex_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:49.980 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking flex (2.6.4-6.2) ... 2022-07-21T01:47:49.986 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package dump. 2022-07-21T01:47:49.998 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-07-21T01:47:50.042 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking dump (0.4b46-6) ... 2022-07-21T01:47:50.057 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package bison. 2022-07-21T01:47:50.072 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-07-21T01:47:50.161 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-07-21T01:47:50.374 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package indent. 2022-07-21T01:47:50.378 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-07-21T01:47:50.410 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package bison. 2022-07-21T01:47:50.412 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking indent (2.2.12-1) ... 2022-07-21T01:47:50.425 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../1-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2022-07-21T01:47:50.464 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2022-07-21T01:47:50.595 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-07-21T01:47:50.603 INFO:teuthology.orchestra.run.smithi140.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-07-21T01:47:50.606 DEBUG:teuthology.parallel:result is None 2022-07-21T01:47:50.608 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-07-21T01:47:50.656 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-07-21T01:47:50.672 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-07-21T01:47:50.695 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-07-21T01:47:50.715 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-07-21T01:47:50.814 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblzo2-2:amd64. 2022-07-21T01:47:50.832 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../2-liblzo2-2_2.10-2_amd64.deb ... 2022-07-21T01:47:50.948 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2022-07-21T01:47:50.996 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-07-21T01:47:51.008 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-07-21T01:47:51.031 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package dump. 2022-07-21T01:47:51.044 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-07-21T01:47:51.051 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-07-21T01:47:51.089 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking dump (0.4b46-6) ... 2022-07-21T01:47:51.278 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package dump. 2022-07-21T01:47:51.292 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../3-dump_0.4b46-6_amd64.deb ... 2022-07-21T01:47:51.334 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking dump (0.4b46-6) ... 2022-07-21T01:47:51.401 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package indent. 2022-07-21T01:47:51.404 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libfl2:amd64. 2022-07-21T01:47:51.414 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-07-21T01:47:51.417 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:51.451 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking indent (2.2.12-1) ... 2022-07-21T01:47:51.462 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:51.654 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package indent. 2022-07-21T01:47:51.668 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../4-indent_2.2.12-1_amd64.deb ... 2022-07-21T01:47:51.704 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking indent (2.2.12-1) ... 2022-07-21T01:47:51.712 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-07-21T01:47:51.715 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-07-21T01:47:51.727 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-07-21T01:47:51.728 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:51.756 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:51.761 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-07-21T01:47:51.948 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libaio-dev:amd64. 2022-07-21T01:47:51.962 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../5-libaio-dev_0.3.112-5_amd64.deb ... 2022-07-21T01:47:52.007 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2022-07-21T01:47:52.008 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-07-21T01:47:52.009 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-07-21T01:47:52.020 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-07-21T01:47:52.022 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-07-21T01:47:52.059 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-07-21T01:47:52.082 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-07-21T01:47:52.285 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libelf-dev:amd64. 2022-07-21T01:47:52.300 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../6-libelf-dev_0.176-1.1build1_amd64.deb ... 2022-07-21T01:47:52.343 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2022-07-21T01:47:52.443 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libfl2:amd64. 2022-07-21T01:47:52.457 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:52.505 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:52.524 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up flex (2.6.4-6.2) ... 2022-07-21T01:47:52.647 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-07-21T01:47:52.688 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libfl2:amd64. 2022-07-21T01:47:52.702 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../7-libfl2_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:52.746 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:52.753 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-07-21T01:47:52.765 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-07-21T01:47:52.767 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:52.818 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:52.899 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-07-21T01:47:53.024 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libfl-dev:amd64. 2022-07-21T01:47:53.025 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-07-21T01:47:53.038 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../8-libfl-dev_2.6.4-6.2_amd64.deb ... 2022-07-21T01:47:53.065 INFO:teuthology.orchestra.run.smithi192.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-07-21T01:47:53.077 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-07-21T01:47:53.082 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:53.114 INFO:teuthology.orchestra.run.smithi192.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-07-21T01:47:53.151 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:53.337 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libssl-dev:amd64. 2022-07-21T01:47:53.350 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../9-libssl-dev_1.1.1f-1ubuntu2.16_amd64.deb ... 2022-07-21T01:47:53.393 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-07-21T01:47:53.581 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up flex (2.6.4-6.2) ... 2022-07-21T01:47:53.641 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up dump (0.4b46-6) ... 2022-07-21T01:47:53.710 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-07-21T01:47:53.815 INFO:teuthology.orchestra.run.smithi191.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-07-21T01:47:53.871 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-07-21T01:47:53.937 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-07-21T01:47:53.949 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up flex (2.6.4-6.2) ... 2022-07-21T01:47:53.961 INFO:teuthology.orchestra.run.smithi191.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-07-21T01:47:54.031 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up indent (2.2.12-1) ... 2022-07-21T01:47:54.047 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-07-21T01:47:54.057 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2022-07-21T01:47:54.148 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:54.159 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2022-07-21T01:47:54.172 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-07-21T01:47:54.284 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-07-21T01:47:54.299 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:54.309 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.16) ... 2022-07-21T01:47:54.427 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up dump (0.4b46-6) ... 2022-07-21T01:47:54.436 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-07-21T01:47:54.452 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2022-07-21T01:47:54.514 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-07-21T01:47:54.570 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-07-21T01:47:54.578 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:54.643 INFO:teuthology.orchestra.run.smithi192.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-07-21T01:47:54.707 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up indent (2.2.12-1) ... 2022-07-21T01:47:54.707 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up dump (0.4b46-6) ... 2022-07-21T01:47:54.800 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2022-07-21T01:47:54.847 INFO:teuthology.orchestra.run.smithi192.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:54.859 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2022-07-21T01:47:54.953 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2022-07-21T01:47:54.974 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-07-21T01:47:55.016 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up indent (2.2.12-1) ... 2022-07-21T01:47:55.117 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2022-07-21T01:47:55.126 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-07-21T01:47:55.235 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-07-21T01:47:55.395 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-07-21T01:47:56.305 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-07-21T01:47:56.928 INFO:teuthology.orchestra.run.smithi192.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-07-21T01:47:56.970 INFO:teuthology.orchestra.run.smithi191.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-07-21T01:47:56.972 DEBUG:teuthology.parallel:result is None 2022-07-21T01:47:57.189 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2022-07-21T01:47:57.642 INFO:teuthology.orchestra.run.smithi192.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-07-21T01:47:57.649 DEBUG:teuthology.parallel:result is None 2022-07-21T01:47:57.903 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-07-21T01:47:57.906 DEBUG:teuthology.parallel:result is None 2022-07-21T01:47:57.907 DEBUG:teuthology.orchestra.run.smithi140:> sudo lsb_release -is 2022-07-21T01:47:57.962 INFO:teuthology.orchestra.run.smithi140.stdout:Ubuntu 2022-07-21T01:47:57.966 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:47:57.967 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=57544da114dbbe1c44f27b2ea18e3233e4e917e0 2022-07-21T01:47:58.144 DEBUG:teuthology.orchestra.run.smithi140:> dpkg-query -W -f '${Version}' ceph 2022-07-21T01:47:58.170 INFO:teuthology.orchestra.run.smithi140.stdout:15.2.16-336-g57544da1-1focal 2022-07-21T01:47:58.171 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-336-g57544da1-1focal 2022-07-21T01:47:58.172 INFO:teuthology.task.install:The correct ceph version 15.2.16-336-g57544da1-1focal is installed. 2022-07-21T01:47:58.172 DEBUG:teuthology.orchestra.run.smithi152:> sudo lsb_release -is 2022-07-21T01:47:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:Ubuntu 2022-07-21T01:47:58.231 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:47:58.232 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=57544da114dbbe1c44f27b2ea18e3233e4e917e0 2022-07-21T01:47:58.395 DEBUG:teuthology.orchestra.run.smithi152:> dpkg-query -W -f '${Version}' ceph 2022-07-21T01:47:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:15.2.16-336-g57544da1-1focal 2022-07-21T01:47:58.426 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-336-g57544da1-1focal 2022-07-21T01:47:58.427 INFO:teuthology.task.install:The correct ceph version 15.2.16-336-g57544da1-1focal is installed. 2022-07-21T01:47:58.427 DEBUG:teuthology.orchestra.run.smithi191:> sudo lsb_release -is 2022-07-21T01:47:58.481 INFO:teuthology.orchestra.run.smithi191.stdout:Ubuntu 2022-07-21T01:47:58.487 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:47:58.488 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=57544da114dbbe1c44f27b2ea18e3233e4e917e0 2022-07-21T01:47:58.656 DEBUG:teuthology.orchestra.run.smithi191:> dpkg-query -W -f '${Version}' ceph 2022-07-21T01:47:58.684 INFO:teuthology.orchestra.run.smithi191.stdout:15.2.16-336-g57544da1-1focal 2022-07-21T01:47:58.684 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-336-g57544da1-1focal 2022-07-21T01:47:58.686 INFO:teuthology.task.install:The correct ceph version 15.2.16-336-g57544da1-1focal is installed. 2022-07-21T01:47:58.687 DEBUG:teuthology.orchestra.run.smithi192:> sudo lsb_release -is 2022-07-21T01:47:58.740 INFO:teuthology.orchestra.run.smithi192.stdout:Ubuntu 2022-07-21T01:47:58.746 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:47:58.747 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=57544da114dbbe1c44f27b2ea18e3233e4e917e0 2022-07-21T01:47:58.907 DEBUG:teuthology.orchestra.run.smithi192:> dpkg-query -W -f '${Version}' ceph 2022-07-21T01:47:58.931 INFO:teuthology.orchestra.run.smithi192.stdout:15.2.16-336-g57544da1-1focal 2022-07-21T01:47:58.931 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-336-g57544da1-1focal 2022-07-21T01:47:58.932 INFO:teuthology.task.install:The correct ceph version 15.2.16-336-g57544da1-1focal is installed. 2022-07-21T01:47:58.932 DEBUG:teuthology.orchestra.run.smithi195:> sudo lsb_release -is 2022-07-21T01:47:58.988 INFO:teuthology.orchestra.run.smithi195.stdout:Ubuntu 2022-07-21T01:47:58.993 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-07-21T01:47:58.994 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=57544da114dbbe1c44f27b2ea18e3233e4e917e0 2022-07-21T01:47:59.164 DEBUG:teuthology.orchestra.run.smithi195:> dpkg-query -W -f '${Version}' ceph 2022-07-21T01:47:59.193 INFO:teuthology.orchestra.run.smithi195.stdout:15.2.16-336-g57544da1-1focal 2022-07-21T01:47:59.194 INFO:teuthology.packaging:The installed version of ceph is 15.2.16-336-g57544da1-1focal 2022-07-21T01:47:59.195 INFO:teuthology.task.install:The correct ceph version 15.2.16-336-g57544da1-1focal is installed. 2022-07-21T01:47:59.196 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-07-21T01:47:59.196 DEBUG:teuthology.orchestra.run.smithi140:> set -ex 2022-07-21T01:47:59.197 DEBUG:teuthology.orchestra.run.smithi140:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-07-21T01:47:59.215 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2022-07-21T01:47:59.216 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-07-21T01:47:59.237 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-07-21T01:47:59.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-07-21T01:47:59.264 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2022-07-21T01:47:59.265 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-07-21T01:47:59.285 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2022-07-21T01:47:59.286 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-07-21T01:47:59.306 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-07-21T01:47:59.308 DEBUG:teuthology.orchestra.run.smithi140:> set -ex 2022-07-21T01:47:59.308 DEBUG:teuthology.orchestra.run.smithi140:> sudo dd of=/usr/bin/daemon-helper 2022-07-21T01:47:59.331 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-07-21T01:47:59.389 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2022-07-21T01:47:59.390 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2022-07-21T01:47:59.408 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-07-21T01:47:59.469 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-07-21T01:47:59.470 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2022-07-21T01:47:59.490 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-07-21T01:47:59.507 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2022-07-21T01:47:59.508 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2022-07-21T01:47:59.529 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-07-21T01:47:59.591 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2022-07-21T01:47:59.592 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2022-07-21T01:47:59.609 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-07-21T01:47:59.626 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-07-21T01:47:59.626 DEBUG:teuthology.orchestra.run.smithi140:> set -ex 2022-07-21T01:47:59.627 DEBUG:teuthology.orchestra.run.smithi140:> sudo dd of=/usr/bin/adjust-ulimits 2022-07-21T01:47:59.647 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-07-21T01:47:59.659 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2022-07-21T01:47:59.660 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2022-07-21T01:47:59.677 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-07-21T01:47:59.738 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-07-21T01:47:59.739 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2022-07-21T01:47:59.756 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-07-21T01:47:59.773 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2022-07-21T01:47:59.775 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2022-07-21T01:47:59.796 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-07-21T01:47:59.812 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2022-07-21T01:47:59.813 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2022-07-21T01:47:59.831 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-07-21T01:47:59.845 INFO:teuthology.run_tasks:Running task ceph... 2022-07-21T01:47:59.941 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-07-21T01:47:59.943 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 777 /var/log/ceph 2022-07-21T01:47:59.948 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /var/log/ceph 2022-07-21T01:47:59.952 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /var/log/ceph 2022-07-21T01:47:59.957 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /var/log/ceph 2022-07-21T01:47:59.962 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /var/log/ceph 2022-07-21T01:47:59.979 INFO:tasks.ceph:Disabling ceph logrotate... 2022-07-21T01:47:59.980 DEBUG:teuthology.orchestra.run.smithi140:> sudo rm -f -- /etc/logrotate.d/ceph 2022-07-21T01:47:59.988 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/logrotate.d/ceph 2022-07-21T01:48:00.010 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/logrotate.d/ceph 2022-07-21T01:48:00.014 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/logrotate.d/ceph 2022-07-21T01:48:00.017 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/logrotate.d/ceph 2022-07-21T01:48:00.035 INFO:tasks.ceph:Creating extra log directories... 2022-07-21T01:48:00.036 DEBUG:teuthology.orchestra.run.smithi140:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-07-21T01:48:00.043 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-07-21T01:48:00.064 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-07-21T01:48:00.074 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-07-21T01:48:00.078 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-07-21T01:48:00.100 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-07-21T01:48:00.101 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-07-21T01:48:00.104 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-07-21T01:48:00.104 DEBUG:teuthology.orchestra.run.smithi140:> set -ex 2022-07-21T01:48:00.104 DEBUG:teuthology.orchestra.run.smithi140:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-07-21T01:48:00.112 DEBUG:teuthology.orchestra.run.smithi140:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-07-21T01:48:00.148 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2022-07-21T01:48:00.149 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-07-21T01:48:00.169 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-07-21T01:48:00.216 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-07-21T01:48:00.221 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-07-21T01:48:00.243 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-07-21T01:48:00.335 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2022-07-21T01:48:00.336 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-07-21T01:48:00.405 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-07-21T01:48:00.454 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2022-07-21T01:48:00.455 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-07-21T01:48:00.504 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-07-21T01:48:00.596 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-07-21T01:48:00.604 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd objectstore': 'filestore', 'osd op complaint time': 180, 'osd sloppy crc': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-07-21T01:48:00.605 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365', 'branch': 'octopus-release', 'description': 'fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/filestore-xfs overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}', 'email': None, 'first_in_suite': False, 'job_id': '6941365', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'octopus-release'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd objectstore': 'filestore', 'osd op complaint time': 180, 'osd sloppy crc': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'sha1': '57544da114dbbe1c44f27b2ea18e3233e4e917e0'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'osd objectstore': 'filestore', 'osd sloppy crc': True}}, 'filestore': True, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '57544da114dbbe1c44f27b2ea18e3233e4e917e0'}}, 'workunit': {'branch': 'octopus-release', 'sha1': '57544da114dbbe1c44f27b2ea18e3233e4e917e0'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0'], ['client.1'], ['client.2']], 'sha1': '57544da114dbbe1c44f27b2ea18e3233e4e917e0', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'octopus-release', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '57544da114dbbe1c44f27b2ea18e3233e4e917e0', 'targets': {'smithi140.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9x0WKcVST/l7vDaTfI7nKnhQhrkW71PXq8TlhG5T7KOTtcsgzvck9F1+BwHaspMsrLaH/UJ808y8KXdXmKqe+3aHtwOeg7PviU18S/6zlQm880RhjZwy8YtnRjxblQIbzteNDUVCvm9sFcAMMGzsqskG9QAfKWjU6FBESxzsvVtcu1x8JcBu1whatFbmvNMNj9conCia1GFon7cwjpY5uGwBKyNuhTTxYznh3TAOof9tvDwoCkLQ4yehTnCgtmN+nSOHHeaN+8GZIjT/nImnuSr84ZuKgbP38/4H/64lqvsV0qJPVJy6fHIxNTsSwu0LrTqrk/1vePLDIwdN8jYFOToqSAA17mCXO+CDrYlkgiOAi2hT72SqLdYKBUtlrPSv6aFBk+Y5drbBcavIheK/45DrXoxzFdLtAP1N4ZgjUn8/O8eX13nqLzfH9LAmzpkahWN/qaoonmnjIAgXZx3knISYY92ZHDeCaJxCtJCR5C9SGuwsuTclx2XmP12aD9Yk=', 'smithi152.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDfZ7/DsFfPLBGQi9jnB4Av06DPRwY6w371r4HYH4JVxi1d6qz968W2FpgfR5X+5D8OPcUgnl0Sgg4q1XkDdr9xaN+5TDzC3QCYlGUehLRdKYtOVXqJ8q370fXKkWu7CflgSHv7+5lLBmHRmZP8/7wpNSUPROct54sIkA/5xibapvvz6cTbwUBgbLDQNeJUndNrf8FV66HYiwuLXdOiFOKrK6dfaoSY6BsFEKiwcTi6dl1WKB+YemJmEejeFGFgD55ZbSGMgPwfU86b/AUOYuh0Qh5nWfk+WAQcu/dnKJqvzCwAvpaMkk240u3IbkBM3rQS+KjVVXWlSsvXw0ORH7UAc2z+1tI5CxSW3QLFBX68KCDftXy/dS9e2NzEGTMqMRLfKCGJFS5f18HaVdkGsTQF5AHMFtY9OmxRPlmaZmJZFdvonkLL9V/tFUVvxEjCTsnRRg5ef/9WgT6GVO/pNvVpIop60fjEPlre3pJuD8hHTzBs+u/P3bP1h0jWVDkGy30=', 'smithi191.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUkUN0GSzJx2knn6eNQ/uLp5TG+T3LCJmskHcbyuI2Lwnk2gejB78bEz7Z1RimqnoPoRq2qEazNdM57IYZuA/DWYi3wbslshxwfd7B5n2lCYZoikj5BxJsn5Ygmmylojv1vL6VDCjRY3a8tZ2vCiVDc6xLkdiSYMdCS2UgBEUDS1UuZAZ9c3sQCmHPeqMRXbj9a0Rc3niR+VhP/cK7AAvM6sW6R6geX0BxlilPTC253E9p8s6dMMGusXUaUKP53AkwgAkbxqBePTppn4Ss1GgKxPswBFzX15t8/HgBOJWT7Z8PCRgMTKVs7AqJlTZsoCUBZYUzNn7WlgJ5Cpx6v12tkZsiQhxHrcd+Jkq5zqxWk+WlkaMay0cXqeLjx/QuUsdHxEjcPRQbXUuz30hZ/lD1XzFwVubPmW3eRLni/415UsftTj+rUKe+faIiOQ8buuGw+RvHYCxULMXRlY8P/gwNQDar8D1YrlV4RYCJAOM0gOThprKTHkzvd5YmEPqBSYs=', 'smithi192.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+vDtwcDnkw0GjVDsV7muA7x/4sIK90gwk+QtVRqmYSn8Rd5zm/C4/4wEq+eaZaDkpHL282rFLP9oHqMTwI4iB3a9bhKdzhoEV/AI1JafUsZkcSMiKpoN9bqmlSyk61zEIXNWToBM4XI4LRM0LwJdHQqFz75ES94KrKCiVyq1HdKBAh+eD0STz0KRHHVWBqMj68SWqkQvqX58jWVwVTMeEHKYhZ6Gmm5mlGHpCPP6aiHJxSrQoAkOFyfyKP2xawkEYVNTMOx+TicJshHp9svqRg8jYwX3znWBydo4DYBk2DUHNvOTl8Y144B30/Vq5vIaF/hny0r/LX7HsiKP2tSLDwujR5xGtt1zRvfE7epMjIpF9yWdC4NkjLO9ieBJHxW7h49WbC2IPO53iKyVAz6svP2qh9ir2DtZFq8z+vuLUouLbl2kcfdk+BNjkFmCYGRPa/hz4tUlSIIDx9j2YafVDzsSDRKNFrcCLlW0/WYDjx+Jdu5scc4T9/4+NEblnY4M=', 'smithi195.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQChgDJokjWfgdtScLBpJnVPYk7hkd9iQCJNV5E1JbpsVje03XVrY6DQAegREfL/T7HBHc/kgkm/4I9KPXCqyQdQS80TqFrEcF2uMl2HtuWFL1oZ5EI5DvPmL7qXZ99E9DJ/D+8qS4fwIuu+RKJHreFiYaZ7zjJimRqQ4+vTLkdBblM+FlbrB9IQd4xpyrp3o3+89L+0Jp9IYMErnjkwF1p442rHv0rNfPD63XcA5/6t+YIvUV2nZCYrMf4oF7WPopNFs88UYvTVhqkUbv/KVunjoBQVzmOGUe90FAww0CHRrsi0gwFzqFekDzmVoD4vfo7BGn6jQknJWuVQK1AlT25GgviPI7ICW73/Ev3P+kvRr0WVbTdToRxk9kq6C6fugluM/6DpqSfjEVjQKzMjp4Btq8ouSukHrx7sIBnAMclrI7mEm8RKF7GjyNHvsA39OTn9DnXUP1b2wZMiw5h1yloMRZR4FVW8BxSBgLqQWOrKMAfTzlg1QMFxuR1zHK3K0K8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '57544da114dbbe1c44f27b2ea18e3233e4e917e0'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_misc']}}], 'teuthology_branch': 'main', 'teuthology_sha1': '89d874d555f56b1c1f383df72487aecec0ee6cf9', 'timestamp': '2022-07-20_20:56:09', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1713'} 2022-07-21T01:48:00.607 DEBUG:teuthology.orchestra.run.smithi140:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-07-21T01:48:00.620 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-07-21T01:48:00.637 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-07-21T01:48:00.657 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-07-21T01:48:00.665 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-07-21T01:48:00.679 DEBUG:teuthology.orchestra.run.smithi140:> sudo install -d -m0777 -- /var/run/ceph 2022-07-21T01:48:00.691 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/run/ceph 2022-07-21T01:48:00.706 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/run/ceph 2022-07-21T01:48:00.717 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/run/ceph 2022-07-21T01:48:00.749 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/run/ceph 2022-07-21T01:48:00.774 DEBUG:teuthology.orchestra.run.smithi140:> set -ex 2022-07-21T01:48:00.775 DEBUG:teuthology.orchestra.run.smithi140:> dd if=/scratch_devs of=/dev/stdout 2022-07-21T01:48:00.799 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-07-21T01:48:00.800 DEBUG:teuthology.orchestra.run.smithi140:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-07-21T01:48:00.818 INFO:teuthology.orchestra.run.smithi140.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-07-21T01:48:00.819 INFO:teuthology.orchestra.run.smithi140.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-07-21T01:48:00.819 INFO:teuthology.orchestra.run.smithi140.stdout:Device: 6h/6d Inode: 358 Links: 1 2022-07-21T01:48:00.820 INFO:teuthology.orchestra.run.smithi140.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-07-21T01:48:00.820 INFO:teuthology.orchestra.run.smithi140.stdout:Access: 2022-07-21 01:44:53.794305013 +0000 2022-07-21T01:48:00.821 INFO:teuthology.orchestra.run.smithi140.stdout:Modify: 2022-07-21 01:44:53.486305003 +0000 2022-07-21T01:48:00.821 INFO:teuthology.orchestra.run.smithi140.stdout:Change: 2022-07-21 01:44:53.486305003 +0000 2022-07-21T01:48:00.822 INFO:teuthology.orchestra.run.smithi140.stdout: Birth: - 2022-07-21T01:48:00.832 INFO:teuthology.orchestra.run.smithi140.stderr:1+0 records in 2022-07-21T01:48:00.832 INFO:teuthology.orchestra.run.smithi140.stderr:1+0 records out 2022-07-21T01:48:00.833 INFO:teuthology.orchestra.run.smithi140.stderr:512 bytes copied, 0.000431101 s, 1.2 MB/s 2022-07-21T01:48:00.835 DEBUG:teuthology.orchestra.run.smithi140:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-07-21T01:48:00.845 INFO:teuthology.orchestra.run.smithi140.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-07-21T01:48:00.846 INFO:teuthology.orchestra.run.smithi140.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-07-21T01:48:00.846 INFO:teuthology.orchestra.run.smithi140.stdout:Device: 6h/6d Inode: 365 Links: 1 2022-07-21T01:48:00.847 INFO:teuthology.orchestra.run.smithi140.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-07-21T01:48:00.847 INFO:teuthology.orchestra.run.smithi140.stdout:Access: 2022-07-21 01:44:54.354305031 +0000 2022-07-21T01:48:00.848 INFO:teuthology.orchestra.run.smithi140.stdout:Modify: 2022-07-21 01:44:54.038305020 +0000 2022-07-21T01:48:00.849 INFO:teuthology.orchestra.run.smithi140.stdout:Change: 2022-07-21 01:44:54.038305020 +0000 2022-07-21T01:48:00.849 INFO:teuthology.orchestra.run.smithi140.stdout: Birth: - 2022-07-21T01:48:00.858 INFO:teuthology.orchestra.run.smithi140.stderr:1+0 records in 2022-07-21T01:48:00.859 INFO:teuthology.orchestra.run.smithi140.stderr:1+0 records out 2022-07-21T01:48:00.859 INFO:teuthology.orchestra.run.smithi140.stderr:512 bytes copied, 0.000389104 s, 1.3 MB/s 2022-07-21T01:48:00.862 DEBUG:teuthology.orchestra.run.smithi140:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-07-21T01:48:00.919 INFO:teuthology.orchestra.run.smithi140.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-07-21T01:48:00.919 INFO:teuthology.orchestra.run.smithi140.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-07-21T01:48:00.920 INFO:teuthology.orchestra.run.smithi140.stdout:Device: 6h/6d Inode: 372 Links: 1 2022-07-21T01:48:00.920 INFO:teuthology.orchestra.run.smithi140.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-07-21T01:48:00.920 INFO:teuthology.orchestra.run.smithi140.stdout:Access: 2022-07-21 01:44:54.902305048 +0000 2022-07-21T01:48:00.920 INFO:teuthology.orchestra.run.smithi140.stdout:Modify: 2022-07-21 01:44:54.586305038 +0000 2022-07-21T01:48:00.921 INFO:teuthology.orchestra.run.smithi140.stdout:Change: 2022-07-21 01:44:54.586305038 +0000 2022-07-21T01:48:00.921 INFO:teuthology.orchestra.run.smithi140.stdout: Birth: - 2022-07-21T01:48:00.929 INFO:teuthology.orchestra.run.smithi140.stderr:1+0 records in 2022-07-21T01:48:00.930 INFO:teuthology.orchestra.run.smithi140.stderr:1+0 records out 2022-07-21T01:48:00.931 INFO:teuthology.orchestra.run.smithi140.stderr:512 bytes copied, 0.000394791 s, 1.3 MB/s 2022-07-21T01:48:00.935 DEBUG:teuthology.orchestra.run.smithi140:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-07-21T01:48:00.958 INFO:teuthology.orchestra.run.smithi140.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-07-21T01:48:00.959 INFO:teuthology.orchestra.run.smithi140.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-07-21T01:48:00.959 INFO:teuthology.orchestra.run.smithi140.stdout:Device: 6h/6d Inode: 376 Links: 1 2022-07-21T01:48:00.960 INFO:teuthology.orchestra.run.smithi140.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-07-21T01:48:00.960 INFO:teuthology.orchestra.run.smithi140.stdout:Access: 2022-07-21 01:44:55.138305056 +0000 2022-07-21T01:48:00.960 INFO:teuthology.orchestra.run.smithi140.stdout:Modify: 2022-07-21 01:44:55.134305056 +0000 2022-07-21T01:48:00.960 INFO:teuthology.orchestra.run.smithi140.stdout:Change: 2022-07-21 01:44:55.134305056 +0000 2022-07-21T01:48:00.961 INFO:teuthology.orchestra.run.smithi140.stdout: Birth: - 2022-07-21T01:48:00.963 INFO:teuthology.orchestra.run.smithi140.stderr:1+0 records in 2022-07-21T01:48:00.963 INFO:teuthology.orchestra.run.smithi140.stderr:1+0 records out 2022-07-21T01:48:00.963 INFO:teuthology.orchestra.run.smithi140.stderr:512 bytes copied, 0.000426954 s, 1.2 MB/s 2022-07-21T01:48:00.964 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-07-21T01:48:00.965 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2022-07-21T01:48:00.966 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2022-07-21T01:48:00.977 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-07-21T01:48:00.978 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2022-07-21T01:48:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-07-21T01:48:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-07-21T01:48:00.988 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 358 Links: 1 2022-07-21T01:48:00.989 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-07-21T01:48:00.989 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2022-07-21 01:45:00.639684767 +0000 2022-07-21T01:48:00.990 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2022-07-21 01:45:00.331684759 +0000 2022-07-21T01:48:00.990 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2022-07-21 01:45:00.331684759 +0000 2022-07-21T01:48:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2022-07-21T01:48:00.997 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2022-07-21T01:48:01.004 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2022-07-21T01:48:01.004 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.00039991 s, 1.3 MB/s 2022-07-21T01:48:01.007 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2022-07-21T01:48:01.063 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-07-21T01:48:01.064 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-07-21T01:48:01.065 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 362 Links: 1 2022-07-21T01:48:01.065 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-07-21T01:48:01.065 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2022-07-21 01:45:01.147684780 +0000 2022-07-21T01:48:01.066 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2022-07-21 01:45:00.839684772 +0000 2022-07-21T01:48:01.066 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2022-07-21 01:45:00.839684772 +0000 2022-07-21T01:48:01.067 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2022-07-21T01:48:01.074 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2022-07-21T01:48:01.074 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2022-07-21T01:48:01.075 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000403545 s, 1.3 MB/s 2022-07-21T01:48:01.079 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2022-07-21T01:48:01.087 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-07-21T01:48:01.087 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-07-21T01:48:01.088 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 371 Links: 1 2022-07-21T01:48:01.088 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-07-21T01:48:01.088 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2022-07-21 01:45:01.407684787 +0000 2022-07-21T01:48:01.088 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2022-07-21 01:45:01.403684787 +0000 2022-07-21T01:48:01.089 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2022-07-21 01:45:01.403684787 +0000 2022-07-21T01:48:01.089 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2022-07-21T01:48:01.094 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2022-07-21T01:48:01.094 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2022-07-21T01:48:01.095 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000392655 s, 1.3 MB/s 2022-07-21T01:48:01.100 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2022-07-21T01:48:01.111 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-07-21T01:48:01.111 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-07-21T01:48:01.111 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 376 Links: 1 2022-07-21T01:48:01.112 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-07-21T01:48:01.112 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2022-07-21 01:45:02.267684810 +0000 2022-07-21T01:48:01.113 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2022-07-21 01:45:01.923684801 +0000 2022-07-21T01:48:01.113 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2022-07-21 01:45:01.923684801 +0000 2022-07-21T01:48:01.113 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2022-07-21T01:48:01.120 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2022-07-21T01:48:01.120 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2022-07-21T01:48:01.120 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000409661 s, 1.2 MB/s 2022-07-21T01:48:01.125 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-07-21T01:48:01.125 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi140.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-07-21T01:48:01.126 INFO:tasks.ceph:Generating config... 2022-07-21T01:48:01.127 INFO:tasks.ceph:[client] client mount timeout = 600 2022-07-21T01:48:01.128 INFO:tasks.ceph:[client] debug client = 20 2022-07-21T01:48:01.129 INFO:tasks.ceph:[client] debug ms = 1 2022-07-21T01:48:01.129 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2022-07-21T01:48:01.129 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2022-07-21T01:48:01.130 INFO:tasks.ceph:[mds] debug mds = 20 2022-07-21T01:48:01.130 INFO:tasks.ceph:[mds] debug ms = 1 2022-07-21T01:48:01.130 INFO:tasks.ceph:[mds] mds bal frag = True 2022-07-21T01:48:01.131 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2022-07-21T01:48:01.131 INFO:tasks.ceph:[mds] mds bal merge size = 5 2022-07-21T01:48:01.131 INFO:tasks.ceph:[mds] mds bal split bits = 3 2022-07-21T01:48:01.132 INFO:tasks.ceph:[mds] mds bal split size = 100 2022-07-21T01:48:01.132 INFO:tasks.ceph:[mds] mds debug frag = True 2022-07-21T01:48:01.132 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-07-21T01:48:01.133 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-07-21T01:48:01.133 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-07-21T01:48:01.133 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-07-21T01:48:01.134 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2022-07-21T01:48:01.134 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2022-07-21T01:48:01.134 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-07-21T01:48:01.134 INFO:tasks.ceph:[mgr] debug ms = 1 2022-07-21T01:48:01.135 INFO:tasks.ceph:[mon] debug mon = 20 2022-07-21T01:48:01.135 INFO:tasks.ceph:[mon] debug ms = 1 2022-07-21T01:48:01.135 INFO:tasks.ceph:[mon] debug paxos = 20 2022-07-21T01:48:01.136 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-07-21T01:48:01.136 INFO:tasks.ceph:[osd] debug ms = 1 2022-07-21T01:48:01.136 INFO:tasks.ceph:[osd] debug osd = 20 2022-07-21T01:48:01.137 INFO:tasks.ceph:[osd] osd objectstore = filestore 2022-07-21T01:48:01.137 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-07-21T01:48:01.137 INFO:tasks.ceph:[osd] osd sloppy crc = True 2022-07-21T01:48:01.138 INFO:tasks.ceph:Setting up mon.a... 2022-07-21T01:48:01.138 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-07-21T01:48:01.190 INFO:teuthology.orchestra.run.smithi140.stdout:creating /etc/ceph/ceph.keyring 2022-07-21T01:48:01.193 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-07-21T01:48:01.226 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-07-21T01:48:01.286 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.140'), ('mon.b', '172.21.15.152'), ('mon.c', '[v2:172.21.15.152:3301,v1:172.21.15.152:6790]')] 2022-07-21T01:48:01.286 DEBUG:teuthology.orchestra.run.smithi140:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.140 --add b 172.21.15.152 --addv c '[v2:172.21.15.152:3301,v1:172.21.15.152:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:01.317 INFO:teuthology.orchestra.run.smithi140.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:01.318 INFO:teuthology.orchestra.run.smithi140.stdout:monmaptool: generated fsid 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:48:01.318 INFO:teuthology.orchestra.run.smithi140.stdout:epoch 0 2022-07-21T01:48:01.319 INFO:teuthology.orchestra.run.smithi140.stdout:fsid 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:48:01.319 INFO:teuthology.orchestra.run.smithi140.stdout:last_changed 2022-07-21T01:48:01.320808+0000 2022-07-21T01:48:01.319 INFO:teuthology.orchestra.run.smithi140.stdout:created 2022-07-21T01:48:01.320808+0000 2022-07-21T01:48:01.320 INFO:teuthology.orchestra.run.smithi140.stdout:min_mon_release 0 (unknown) 2022-07-21T01:48:01.320 INFO:teuthology.orchestra.run.smithi140.stdout:0: [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] mon.a 2022-07-21T01:48:01.321 INFO:teuthology.orchestra.run.smithi140.stdout:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2022-07-21T01:48:01.321 INFO:teuthology.orchestra.run.smithi140.stdout:2: [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] mon.c 2022-07-21T01:48:01.321 INFO:teuthology.orchestra.run.smithi140.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-07-21T01:48:01.326 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 87d2f829-64cd-4d49-9e42-a1dfafceefb8... 2022-07-21T01:48:01.329 DEBUG:teuthology.orchestra.run.smithi140:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:48:01.337 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:48:01.341 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:48:01.354 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:48:01.358 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:48:01.369 INFO:teuthology.orchestra.run.smithi140.stdout:[global] 2022-07-21T01:48:01.370 INFO:teuthology.orchestra.run.smithi140.stdout: chdir = "" 2022-07-21T01:48:01.370 INFO:teuthology.orchestra.run.smithi140.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:48:01.371 INFO:teuthology.orchestra.run.smithi140.stdout: auth supported = cephx 2022-07-21T01:48:01.371 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.372 INFO:teuthology.orchestra.run.smithi140.stdout: filestore xattr use omap = true 2022-07-21T01:48:01.372 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.373 INFO:teuthology.orchestra.run.smithi140.stdout: mon clock drift allowed = 1.000 2022-07-21T01:48:01.373 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.374 INFO:teuthology.orchestra.run.smithi140.stdout: osd crush chooseleaf type = 0 2022-07-21T01:48:01.374 INFO:teuthology.orchestra.run.smithi140.stdout: auth debug = true 2022-07-21T01:48:01.375 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.375 INFO:teuthology.orchestra.run.smithi140.stdout: ms die on old message = true 2022-07-21T01:48:01.376 INFO:teuthology.orchestra.run.smithi140.stdout: ms die on bug = true 2022-07-21T01:48:01.376 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.377 INFO:teuthology.orchestra.run.smithi140.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:48:01.377 INFO:teuthology.orchestra.run.smithi140.stdout: mon pg warn max object skew = 0 2022-07-21T01:48:01.378 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.378 INFO:teuthology.orchestra.run.smithi140.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:48:01.379 INFO:teuthology.orchestra.run.smithi140.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:48:01.379 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.380 INFO:teuthology.orchestra.run.smithi140.stdout: osd pool default size = 2 2022-07-21T01:48:01.380 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.381 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd allow primary affinity = true 2022-07-21T01:48:01.381 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd allow pg remap = true 2022-07-21T01:48:01.382 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:48:01.382 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:48:01.383 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on no sortbitwise = false 2022-07-21T01:48:01.383 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:48:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on too few osds = false 2022-07-21T01:48:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:48:01.385 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:48:01.385 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.388 INFO:teuthology.orchestra.run.smithi140.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:48:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: osd default data pool replay window = 5 2022-07-21T01:48:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: mon allow pool delete = true 2022-07-21T01:48:01.390 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.390 INFO:teuthology.orchestra.run.smithi140.stdout: mon cluster log file level = debug 2022-07-21T01:48:01.390 INFO:teuthology.orchestra.run.smithi140.stdout: debug asserts on shutdown = true 2022-07-21T01:48:01.390 INFO:teuthology.orchestra.run.smithi140.stdout: mon health detail to clog = false 2022-07-21T01:48:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:48:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:48:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.392 INFO:teuthology.orchestra.run.smithi140.stdout:[osd] 2022-07-21T01:48:01.392 INFO:teuthology.orchestra.run.smithi140.stdout: osd journal size = 100 2022-07-21T01:48:01.392 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.392 INFO:teuthology.orchestra.run.smithi140.stdout: osd scrub load threshold = 5.0 2022-07-21T01:48:01.393 INFO:teuthology.orchestra.run.smithi140.stdout: osd scrub max interval = 600 2022-07-21T01:48:01.394 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: osd recover clone overlap = true 2022-07-21T01:48:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: osd recovery max chunk = 1048576 2022-07-21T01:48:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug shutdown = true 2022-07-21T01:48:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug op order = true 2022-07-21T01:48:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug verify stray on activate = true 2022-07-21T01:48:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: osd open classes on start = true 2022-07-21T01:48:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug pg log writeout = true 2022-07-21T01:48:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:48:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: osd map max advance = 10 2022-07-21T01:48:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: journal zero on create = true 2022-07-21T01:48:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:48:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: filestore apply finisher threads = 3 2022-07-21T01:48:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: bdev debug aio = true 2022-07-21T01:48:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug misdirected ops = true 2022-07-21T01:48:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:48:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: debug osd = 20 2022-07-21T01:48:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: osd objectstore = filestore 2022-07-21T01:48:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: osd op complaint time = 180 2022-07-21T01:48:01.401 INFO:teuthology.orchestra.run.smithi140.stdout: osd sloppy crc = True 2022-07-21T01:48:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.402 INFO:teuthology.orchestra.run.smithi140.stdout:[mgr] 2022-07-21T01:48:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:48:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: debug mgr = 20 2022-07-21T01:48:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: debug mon = 20 2022-07-21T01:48:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: debug auth = 20 2022-07-21T01:48:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:48:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:48:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: mgr/telemetry/nag = false 2022-07-21T01:48:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.404 INFO:teuthology.orchestra.run.smithi140.stdout:[mon] 2022-07-21T01:48:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:48:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: debug mon = 20 2022-07-21T01:48:01.405 INFO:teuthology.orchestra.run.smithi140.stdout: debug paxos = 20 2022-07-21T01:48:01.405 INFO:teuthology.orchestra.run.smithi140.stdout: debug auth = 20 2022-07-21T01:48:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: mon data avail warn = 5 2022-07-21T01:48:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: mon mgr mkfs grace = 240 2022-07-21T01:48:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:48:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd reporter subtree level = osd 2022-07-21T01:48:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd prime pg temp = true 2022-07-21T01:48:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:48:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:48:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:48:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:48:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:48:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:48:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:48:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: mon op complaint time = 120 2022-07-21T01:48:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:01.409 INFO:teuthology.orchestra.run.smithi140.stdout:[client] 2022-07-21T01:48:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: rgw cache enabled = true 2022-07-21T01:48:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: rgw enable ops log = true 2022-07-21T01:48:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: rgw enable usage log = true 2022-07-21T01:48:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:48:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:48:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: client mount timeout = 600 2022-07-21T01:48:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: debug client = 20 2022-07-21T01:48:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:48:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: rados mon op timeout = 15m 2022-07-21T01:48:01.412 INFO:teuthology.orchestra.run.smithi140.stdout: rados osd op timeout = 15m 2022-07-21T01:48:01.412 INFO:teuthology.orchestra.run.smithi140.stdout:[mon.a] 2022-07-21T01:48:01.412 INFO:teuthology.orchestra.run.smithi140.stdout:[mon.b] 2022-07-21T01:48:01.412 INFO:teuthology.orchestra.run.smithi140.stdout:[mon.c] 2022-07-21T01:48:01.413 INFO:teuthology.orchestra.run.smithi140.stdout:[mds.a] 2022-07-21T01:48:01.413 INFO:teuthology.orchestra.run.smithi140.stdout:[mds.b] 2022-07-21T01:48:01.413 INFO:teuthology.orchestra.run.smithi140.stdout:[mds] 2022-07-21T01:48:01.413 INFO:teuthology.orchestra.run.smithi140.stdout: debug mds = 20 2022-07-21T01:48:01.413 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:48:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal frag = True 2022-07-21T01:48:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal fragment size max = 10000 2022-07-21T01:48:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal merge size = 5 2022-07-21T01:48:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal split bits = 3 2022-07-21T01:48:01.415 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal split size = 100 2022-07-21T01:48:01.415 INFO:teuthology.orchestra.run.smithi140.stdout: mds debug frag = True 2022-07-21T01:48:01.415 INFO:teuthology.orchestra.run.smithi140.stdout: mds debug scatterstat = True 2022-07-21T01:48:01.416 INFO:teuthology.orchestra.run.smithi140.stdout: mds op complaint time = 180 2022-07-21T01:48:01.416 INFO:teuthology.orchestra.run.smithi140.stdout: mds verify scatter = True 2022-07-21T01:48:01.416 INFO:teuthology.orchestra.run.smithi140.stdout: osd op complaint time = 180 2022-07-21T01:48:01.416 INFO:teuthology.orchestra.run.smithi140.stdout: rados mon op timeout = 15m 2022-07-21T01:48:01.417 INFO:teuthology.orchestra.run.smithi140.stdout: rados osd op timeout = 15m 2022-07-21T01:48:01.421 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2022-07-21T01:48:01.421 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2022-07-21T01:48:01.421 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:48:01.421 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2022-07-21T01:48:01.422 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.422 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2022-07-21T01:48:01.422 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.422 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2022-07-21T01:48:01.423 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.423 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2022-07-21T01:48:01.423 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2022-07-21T01:48:01.423 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.424 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2022-07-21T01:48:01.424 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2022-07-21T01:48:01.424 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.424 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:48:01.424 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2022-07-21T01:48:01.425 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.425 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:48:01.425 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:48:01.425 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.426 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2022-07-21T01:48:01.426 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.426 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2022-07-21T01:48:01.426 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2022-07-21T01:48:01.427 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:48:01.427 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:48:01.427 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2022-07-21T01:48:01.427 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:48:01.427 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2022-07-21T01:48:01.428 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:48:01.428 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:48:01.428 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.428 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:48:01.429 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.429 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2022-07-21T01:48:01.429 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.429 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2022-07-21T01:48:01.429 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.430 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2022-07-21T01:48:01.430 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2022-07-21T01:48:01.430 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2022-07-21T01:48:01.430 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:48:01.431 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:48:01.431 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.431 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2022-07-21T01:48:01.431 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2022-07-21T01:48:01.432 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.432 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2022-07-21T01:48:01.432 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2022-07-21T01:48:01.432 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.432 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2022-07-21T01:48:01.433 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2022-07-21T01:48:01.433 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.433 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2022-07-21T01:48:01.433 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2022-07-21T01:48:01.434 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2022-07-21T01:48:01.434 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.434 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2022-07-21T01:48:01.434 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2022-07-21T01:48:01.435 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.435 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:48:01.435 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.435 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2022-07-21T01:48:01.435 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.436 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2022-07-21T01:48:01.436 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.436 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:48:01.436 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2022-07-21T01:48:01.437 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.437 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2022-07-21T01:48:01.437 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2022-07-21T01:48:01.437 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:48:01.437 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2022-07-21T01:48:01.438 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = filestore 2022-07-21T01:48:01.438 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2022-07-21T01:48:01.438 INFO:teuthology.orchestra.run.smithi195.stdout: osd sloppy crc = True 2022-07-21T01:48:01.438 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.439 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2022-07-21T01:48:01.439 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:48:01.439 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2022-07-21T01:48:01.439 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2022-07-21T01:48:01.440 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2022-07-21T01:48:01.440 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:48:01.440 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:48:01.440 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2022-07-21T01:48:01.440 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.441 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2022-07-21T01:48:01.441 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:48:01.441 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2022-07-21T01:48:01.441 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2022-07-21T01:48:01.442 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2022-07-21T01:48:01.442 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2022-07-21T01:48:01.442 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2022-07-21T01:48:01.442 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:48:01.443 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2022-07-21T01:48:01.443 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2022-07-21T01:48:01.443 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:48:01.443 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.444 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:48:01.444 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:48:01.444 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:48:01.444 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.445 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:48:01.445 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:48:01.445 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:48:01.445 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2022-07-21T01:48:01.445 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:48:01.446 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2022-07-21T01:48:01.446 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2022-07-21T01:48:01.446 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2022-07-21T01:48:01.446 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2022-07-21T01:48:01.447 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:48:01.447 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:48:01.447 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2022-07-21T01:48:01.447 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2022-07-21T01:48:01.448 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:48:01.448 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 15m 2022-07-21T01:48:01.448 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 15m 2022-07-21T01:48:01.448 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2022-07-21T01:48:01.448 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2022-07-21T01:48:01.449 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2022-07-21T01:48:01.449 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2022-07-21T01:48:01.449 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2022-07-21T01:48:01.450 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2022-07-21T01:48:01.450 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2022-07-21T01:48:01.450 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:48:01.450 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal frag = True 2022-07-21T01:48:01.450 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal fragment size max = 10000 2022-07-21T01:48:01.451 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal merge size = 5 2022-07-21T01:48:01.451 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal split bits = 3 2022-07-21T01:48:01.451 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal split size = 100 2022-07-21T01:48:01.451 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2022-07-21T01:48:01.452 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = True 2022-07-21T01:48:01.452 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2022-07-21T01:48:01.452 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = True 2022-07-21T01:48:01.452 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2022-07-21T01:48:01.453 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 15m 2022-07-21T01:48:01.453 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 15m 2022-07-21T01:48:01.453 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2022-07-21T01:48:01.453 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2022-07-21T01:48:01.454 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:48:01.454 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2022-07-21T01:48:01.454 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.454 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2022-07-21T01:48:01.455 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.455 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2022-07-21T01:48:01.455 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.455 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2022-07-21T01:48:01.456 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2022-07-21T01:48:01.456 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.456 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2022-07-21T01:48:01.457 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2022-07-21T01:48:01.457 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.457 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:48:01.457 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2022-07-21T01:48:01.457 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.458 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:48:01.458 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:48:01.458 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.458 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2022-07-21T01:48:01.459 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.459 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2022-07-21T01:48:01.459 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2022-07-21T01:48:01.459 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:48:01.460 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:48:01.460 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2022-07-21T01:48:01.460 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:48:01.460 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2022-07-21T01:48:01.461 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:48:01.461 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:48:01.461 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.462 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:48:01.462 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.462 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2022-07-21T01:48:01.462 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.463 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2022-07-21T01:48:01.463 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.463 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2022-07-21T01:48:01.463 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2022-07-21T01:48:01.464 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2022-07-21T01:48:01.464 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:48:01.464 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:48:01.464 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.465 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2022-07-21T01:48:01.465 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2022-07-21T01:48:01.465 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.465 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2022-07-21T01:48:01.465 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2022-07-21T01:48:01.466 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.466 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2022-07-21T01:48:01.466 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2022-07-21T01:48:01.466 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.467 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2022-07-21T01:48:01.467 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2022-07-21T01:48:01.467 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2022-07-21T01:48:01.467 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.468 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2022-07-21T01:48:01.468 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2022-07-21T01:48:01.468 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.468 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:48:01.468 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.469 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2022-07-21T01:48:01.469 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.469 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2022-07-21T01:48:01.469 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.470 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:48:01.470 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2022-07-21T01:48:01.470 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.470 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2022-07-21T01:48:01.470 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2022-07-21T01:48:01.471 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:48:01.471 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2022-07-21T01:48:01.471 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = filestore 2022-07-21T01:48:01.471 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2022-07-21T01:48:01.472 INFO:teuthology.orchestra.run.smithi192.stdout: osd sloppy crc = True 2022-07-21T01:48:01.472 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.472 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2022-07-21T01:48:01.472 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:48:01.472 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2022-07-21T01:48:01.473 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2022-07-21T01:48:01.473 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2022-07-21T01:48:01.473 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:48:01.473 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:48:01.474 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2022-07-21T01:48:01.474 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.475 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2022-07-21T01:48:01.475 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:48:01.475 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2022-07-21T01:48:01.475 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2022-07-21T01:48:01.476 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2022-07-21T01:48:01.476 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2022-07-21T01:48:01.476 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2022-07-21T01:48:01.476 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:48:01.477 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2022-07-21T01:48:01.477 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2022-07-21T01:48:01.477 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:48:01.477 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.478 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:48:01.478 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:48:01.478 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:48:01.478 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.479 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:48:01.479 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:48:01.479 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:48:01.479 INFO:teuthology.orchestra.run.smithi192.stdout: mon op complaint time = 120 2022-07-21T01:48:01.480 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:48:01.480 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2022-07-21T01:48:01.480 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2022-07-21T01:48:01.480 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2022-07-21T01:48:01.480 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2022-07-21T01:48:01.481 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:48:01.481 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:48:01.481 INFO:teuthology.orchestra.run.smithi192.stdout: client mount timeout = 600 2022-07-21T01:48:01.481 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2022-07-21T01:48:01.481 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:48:01.482 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2022-07-21T01:48:01.482 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2022-07-21T01:48:01.482 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2022-07-21T01:48:01.482 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2022-07-21T01:48:01.483 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2022-07-21T01:48:01.483 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2022-07-21T01:48:01.483 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2022-07-21T01:48:01.483 INFO:teuthology.orchestra.run.smithi192.stdout:[mds] 2022-07-21T01:48:01.484 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds = 20 2022-07-21T01:48:01.484 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:48:01.484 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal frag = True 2022-07-21T01:48:01.484 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal fragment size max = 10000 2022-07-21T01:48:01.485 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal merge size = 5 2022-07-21T01:48:01.485 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split bits = 3 2022-07-21T01:48:01.485 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split size = 100 2022-07-21T01:48:01.485 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug frag = True 2022-07-21T01:48:01.485 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug scatterstat = True 2022-07-21T01:48:01.486 INFO:teuthology.orchestra.run.smithi192.stdout: mds op complaint time = 180 2022-07-21T01:48:01.486 INFO:teuthology.orchestra.run.smithi192.stdout: mds verify scatter = True 2022-07-21T01:48:01.487 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2022-07-21T01:48:01.487 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2022-07-21T01:48:01.487 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2022-07-21T01:48:01.488 INFO:teuthology.orchestra.run.smithi152.stdout:[global] 2022-07-21T01:48:01.488 INFO:teuthology.orchestra.run.smithi152.stdout: chdir = "" 2022-07-21T01:48:01.489 INFO:teuthology.orchestra.run.smithi152.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:48:01.489 INFO:teuthology.orchestra.run.smithi152.stdout: auth supported = cephx 2022-07-21T01:48:01.489 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.490 INFO:teuthology.orchestra.run.smithi152.stdout: filestore xattr use omap = true 2022-07-21T01:48:01.490 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.491 INFO:teuthology.orchestra.run.smithi152.stdout: mon clock drift allowed = 1.000 2022-07-21T01:48:01.491 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.492 INFO:teuthology.orchestra.run.smithi152.stdout: osd crush chooseleaf type = 0 2022-07-21T01:48:01.492 INFO:teuthology.orchestra.run.smithi152.stdout: auth debug = true 2022-07-21T01:48:01.496 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.496 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on old message = true 2022-07-21T01:48:01.497 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on bug = true 2022-07-21T01:48:01.497 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.498 INFO:teuthology.orchestra.run.smithi152.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:48:01.498 INFO:teuthology.orchestra.run.smithi152.stdout: mon pg warn max object skew = 0 2022-07-21T01:48:01.499 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.499 INFO:teuthology.orchestra.run.smithi152.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:48:01.499 INFO:teuthology.orchestra.run.smithi152.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:48:01.500 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.500 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default size = 2 2022-07-21T01:48:01.500 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.500 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow primary affinity = true 2022-07-21T01:48:01.500 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow pg remap = true 2022-07-21T01:48:01.501 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:48:01.501 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:48:01.501 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on no sortbitwise = false 2022-07-21T01:48:01.501 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:48:01.501 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on too few osds = false 2022-07-21T01:48:01.502 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:48:01.502 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:48:01.503 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.503 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:48:01.504 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.504 INFO:teuthology.orchestra.run.smithi152.stdout: osd default data pool replay window = 5 2022-07-21T01:48:01.505 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.505 INFO:teuthology.orchestra.run.smithi152.stdout: mon allow pool delete = true 2022-07-21T01:48:01.507 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.508 INFO:teuthology.orchestra.run.smithi152.stdout: mon cluster log file level = debug 2022-07-21T01:48:01.508 INFO:teuthology.orchestra.run.smithi152.stdout: debug asserts on shutdown = true 2022-07-21T01:48:01.508 INFO:teuthology.orchestra.run.smithi152.stdout: mon health detail to clog = false 2022-07-21T01:48:01.509 INFO:teuthology.orchestra.run.smithi152.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:48:01.509 INFO:teuthology.orchestra.run.smithi152.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:48:01.510 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.511 INFO:teuthology.orchestra.run.smithi152.stdout:[osd] 2022-07-21T01:48:01.511 INFO:teuthology.orchestra.run.smithi152.stdout: osd journal size = 100 2022-07-21T01:48:01.512 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.512 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub load threshold = 5.0 2022-07-21T01:48:01.513 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub max interval = 600 2022-07-21T01:48:01.514 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.514 INFO:teuthology.orchestra.run.smithi152.stdout: osd recover clone overlap = true 2022-07-21T01:48:01.515 INFO:teuthology.orchestra.run.smithi152.stdout: osd recovery max chunk = 1048576 2022-07-21T01:48:01.515 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.516 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug shutdown = true 2022-07-21T01:48:01.517 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug op order = true 2022-07-21T01:48:01.518 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify stray on activate = true 2022-07-21T01:48:01.518 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.519 INFO:teuthology.orchestra.run.smithi152.stdout: osd open classes on start = true 2022-07-21T01:48:01.520 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug pg log writeout = true 2022-07-21T01:48:01.520 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.520 INFO:teuthology.orchestra.run.smithi152.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:48:01.521 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.521 INFO:teuthology.orchestra.run.smithi152.stdout: osd map max advance = 10 2022-07-21T01:48:01.522 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.522 INFO:teuthology.orchestra.run.smithi152.stdout: journal zero on create = true 2022-07-21T01:48:01.523 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.523 INFO:teuthology.orchestra.run.smithi152.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:48:01.524 INFO:teuthology.orchestra.run.smithi152.stdout: filestore apply finisher threads = 3 2022-07-21T01:48:01.524 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.525 INFO:teuthology.orchestra.run.smithi152.stdout: bdev debug aio = true 2022-07-21T01:48:01.525 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug misdirected ops = true 2022-07-21T01:48:01.525 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:48:01.526 INFO:teuthology.orchestra.run.smithi152.stdout: debug osd = 20 2022-07-21T01:48:01.527 INFO:teuthology.orchestra.run.smithi152.stdout: osd objectstore = filestore 2022-07-21T01:48:01.527 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2022-07-21T01:48:01.528 INFO:teuthology.orchestra.run.smithi152.stdout: osd sloppy crc = True 2022-07-21T01:48:01.528 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.529 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr] 2022-07-21T01:48:01.529 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:48:01.530 INFO:teuthology.orchestra.run.smithi152.stdout: debug mgr = 20 2022-07-21T01:48:01.531 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2022-07-21T01:48:01.531 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2022-07-21T01:48:01.532 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:48:01.534 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:48:01.534 INFO:teuthology.orchestra.run.smithi152.stdout: mgr/telemetry/nag = false 2022-07-21T01:48:01.535 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.535 INFO:teuthology.orchestra.run.smithi152.stdout:[mon] 2022-07-21T01:48:01.536 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:48:01.536 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2022-07-21T01:48:01.537 INFO:teuthology.orchestra.run.smithi152.stdout: debug paxos = 20 2022-07-21T01:48:01.538 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2022-07-21T01:48:01.538 INFO:teuthology.orchestra.run.smithi152.stdout: mon data avail warn = 5 2022-07-21T01:48:01.539 INFO:teuthology.orchestra.run.smithi152.stdout: mon mgr mkfs grace = 240 2022-07-21T01:48:01.539 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:48:01.540 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd reporter subtree level = osd 2022-07-21T01:48:01.541 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd prime pg temp = true 2022-07-21T01:48:01.541 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:48:01.542 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.542 INFO:teuthology.orchestra.run.smithi152.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:48:01.543 INFO:teuthology.orchestra.run.smithi152.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:48:01.543 INFO:teuthology.orchestra.run.smithi152.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:48:01.544 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.544 INFO:teuthology.orchestra.run.smithi152.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:48:01.545 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:48:01.545 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:48:01.546 INFO:teuthology.orchestra.run.smithi152.stdout: mon op complaint time = 120 2022-07-21T01:48:01.546 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:48:01.547 INFO:teuthology.orchestra.run.smithi152.stdout:[client] 2022-07-21T01:48:01.547 INFO:teuthology.orchestra.run.smithi152.stdout: rgw cache enabled = true 2022-07-21T01:48:01.548 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable ops log = true 2022-07-21T01:48:01.548 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable usage log = true 2022-07-21T01:48:01.549 INFO:teuthology.orchestra.run.smithi152.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:48:01.549 INFO:teuthology.orchestra.run.smithi152.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:48:01.550 INFO:teuthology.orchestra.run.smithi152.stdout: client mount timeout = 600 2022-07-21T01:48:01.550 INFO:teuthology.orchestra.run.smithi152.stdout: debug client = 20 2022-07-21T01:48:01.550 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:48:01.551 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 15m 2022-07-21T01:48:01.551 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 15m 2022-07-21T01:48:01.552 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.a] 2022-07-21T01:48:01.552 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.b] 2022-07-21T01:48:01.553 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.c] 2022-07-21T01:48:01.553 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.a] 2022-07-21T01:48:01.554 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.b] 2022-07-21T01:48:01.555 INFO:teuthology.orchestra.run.smithi152.stdout:[mds] 2022-07-21T01:48:01.555 INFO:teuthology.orchestra.run.smithi152.stdout: debug mds = 20 2022-07-21T01:48:01.556 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:48:01.556 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal frag = True 2022-07-21T01:48:01.556 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal fragment size max = 10000 2022-07-21T01:48:01.557 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal merge size = 5 2022-07-21T01:48:01.557 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal split bits = 3 2022-07-21T01:48:01.557 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal split size = 100 2022-07-21T01:48:01.558 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug frag = True 2022-07-21T01:48:01.558 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug scatterstat = True 2022-07-21T01:48:01.558 INFO:teuthology.orchestra.run.smithi152.stdout: mds op complaint time = 180 2022-07-21T01:48:01.558 INFO:teuthology.orchestra.run.smithi152.stdout: mds verify scatter = True 2022-07-21T01:48:01.559 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2022-07-21T01:48:01.559 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 15m 2022-07-21T01:48:01.559 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 15m 2022-07-21T01:48:01.560 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2022-07-21T01:48:01.561 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2022-07-21T01:48:01.561 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:48:01.562 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2022-07-21T01:48:01.562 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.563 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2022-07-21T01:48:01.563 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.564 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2022-07-21T01:48:01.564 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.565 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2022-07-21T01:48:01.565 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2022-07-21T01:48:01.565 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.566 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2022-07-21T01:48:01.566 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2022-07-21T01:48:01.567 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.567 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:48:01.568 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2022-07-21T01:48:01.568 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.570 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:48:01.570 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:48:01.571 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.571 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2022-07-21T01:48:01.571 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.572 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2022-07-21T01:48:01.572 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2022-07-21T01:48:01.572 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:48:01.572 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:48:01.573 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2022-07-21T01:48:01.573 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:48:01.573 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2022-07-21T01:48:01.573 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:48:01.574 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:48:01.574 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.574 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:48:01.575 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.575 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2022-07-21T01:48:01.576 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.576 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2022-07-21T01:48:01.577 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.578 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2022-07-21T01:48:01.578 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2022-07-21T01:48:01.579 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2022-07-21T01:48:01.579 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:48:01.580 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:48:01.580 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.581 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2022-07-21T01:48:01.581 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2022-07-21T01:48:01.582 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.582 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2022-07-21T01:48:01.583 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2022-07-21T01:48:01.584 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.584 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2022-07-21T01:48:01.585 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2022-07-21T01:48:01.585 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.586 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2022-07-21T01:48:01.586 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2022-07-21T01:48:01.587 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2022-07-21T01:48:01.587 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.588 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2022-07-21T01:48:01.588 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2022-07-21T01:48:01.589 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.589 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:48:01.590 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.590 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2022-07-21T01:48:01.591 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.591 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2022-07-21T01:48:01.592 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.592 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:48:01.593 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2022-07-21T01:48:01.593 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.594 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2022-07-21T01:48:01.594 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2022-07-21T01:48:01.595 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:48:01.595 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2022-07-21T01:48:01.596 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = filestore 2022-07-21T01:48:01.596 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2022-07-21T01:48:01.597 INFO:teuthology.orchestra.run.smithi191.stdout: osd sloppy crc = True 2022-07-21T01:48:01.597 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.598 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2022-07-21T01:48:01.598 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:48:01.599 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2022-07-21T01:48:01.599 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2022-07-21T01:48:01.600 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2022-07-21T01:48:01.600 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:48:01.601 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:48:01.601 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2022-07-21T01:48:01.602 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.602 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2022-07-21T01:48:01.603 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:48:01.603 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2022-07-21T01:48:01.604 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2022-07-21T01:48:01.604 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2022-07-21T01:48:01.605 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2022-07-21T01:48:01.605 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2022-07-21T01:48:01.606 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:48:01.607 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2022-07-21T01:48:01.607 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2022-07-21T01:48:01.608 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:48:01.608 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.609 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:48:01.609 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:48:01.610 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:48:01.610 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.611 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:48:01.611 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:48:01.612 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:48:01.612 INFO:teuthology.orchestra.run.smithi191.stdout: mon op complaint time = 120 2022-07-21T01:48:01.613 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:48:01.613 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2022-07-21T01:48:01.614 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2022-07-21T01:48:01.614 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2022-07-21T01:48:01.615 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2022-07-21T01:48:01.615 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:48:01.616 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi191.stdout: client mount timeout = 600 2022-07-21T01:48:01.617 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2022-07-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:48:01.618 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 15m 2022-07-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 15m 2022-07-21T01:48:01.619 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2022-07-21T01:48:01.620 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.b] 2022-07-21T01:48:01.620 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.c] 2022-07-21T01:48:01.621 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2022-07-21T01:48:01.621 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2022-07-21T01:48:01.622 INFO:teuthology.orchestra.run.smithi191.stdout:[mds] 2022-07-21T01:48:01.622 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds = 20 2022-07-21T01:48:01.623 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:48:01.623 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal frag = True 2022-07-21T01:48:01.624 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal fragment size max = 10000 2022-07-21T01:48:01.624 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal merge size = 5 2022-07-21T01:48:01.625 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal split bits = 3 2022-07-21T01:48:01.625 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal split size = 100 2022-07-21T01:48:01.626 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug frag = True 2022-07-21T01:48:01.626 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug scatterstat = True 2022-07-21T01:48:01.627 INFO:teuthology.orchestra.run.smithi191.stdout: mds op complaint time = 180 2022-07-21T01:48:01.627 INFO:teuthology.orchestra.run.smithi191.stdout: mds verify scatter = True 2022-07-21T01:48:01.628 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2022-07-21T01:48:01.628 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 15m 2022-07-21T01:48:01.629 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 15m 2022-07-21T01:48:01.630 INFO:tasks.ceph:Creating admin key on mon.a... 2022-07-21T01:48:01.631 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-07-21T01:48:01.682 INFO:tasks.ceph:Copying monmap to all nodes... 2022-07-21T01:48:01.696 DEBUG:teuthology.orchestra.remote:smithi140:/etc/ceph/ceph.keyring is 206B 2022-07-21T01:48:01.813 DEBUG:teuthology.orchestra.remote:smithi140:/home/ubuntu/cephtest/ceph.monmap is 379B 2022-07-21T01:48:01.837 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi140.front.sepia.ceph.com 2022-07-21T01:48:01.838 DEBUG:teuthology.orchestra.run.smithi140:> set -ex 2022-07-21T01:48:01.839 DEBUG:teuthology.orchestra.run.smithi140:> sudo dd of=/etc/ceph/ceph.keyring 2022-07-21T01:48:01.839 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-07-21T01:48:01.864 DEBUG:teuthology.orchestra.run.smithi140:> set -ex 2022-07-21T01:48:01.865 DEBUG:teuthology.orchestra.run.smithi140:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:01.912 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi152.front.sepia.ceph.com 2022-07-21T01:48:01.912 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2022-07-21T01:48:01.913 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.keyring 2022-07-21T01:48:01.913 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-07-21T01:48:01.940 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2022-07-21T01:48:01.941 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:01.948 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi191.front.sepia.ceph.com 2022-07-21T01:48:01.949 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-07-21T01:48:01.950 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.keyring 2022-07-21T01:48:01.950 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-07-21T01:48:01.986 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-07-21T01:48:01.986 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:02.042 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi192.front.sepia.ceph.com 2022-07-21T01:48:02.043 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2022-07-21T01:48:02.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.keyring 2022-07-21T01:48:02.044 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-07-21T01:48:02.073 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2022-07-21T01:48:02.074 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:02.120 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi195.front.sepia.ceph.com 2022-07-21T01:48:02.121 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2022-07-21T01:48:02.121 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.keyring 2022-07-21T01:48:02.122 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-07-21T01:48:02.149 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2022-07-21T01:48:02.150 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:02.199 INFO:tasks.ceph:Setting up mon nodes... 2022-07-21T01:48:02.199 INFO:tasks.ceph:Setting up mgr nodes... 2022-07-21T01:48:02.200 DEBUG:teuthology.orchestra.run.smithi140:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-07-21T01:48:02.242 INFO:teuthology.orchestra.run.smithi140.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-07-21T01:48:02.246 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-07-21T01:48:02.296 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-07-21T01:48:02.300 INFO:tasks.ceph:Setting up mds nodes... 2022-07-21T01:48:02.301 DEBUG:teuthology.orchestra.run.smithi140:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-07-21T01:48:02.351 INFO:teuthology.orchestra.run.smithi140.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-07-21T01:48:02.352 DEBUG:teuthology.orchestra.run.smithi140:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-07-21T01:48:02.369 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-07-21T01:48:02.416 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-07-21T01:48:02.422 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-07-21T01:48:02.480 INFO:tasks.ceph_client:Setting up client nodes... 2022-07-21T01:48:02.482 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-07-21T01:48:02.521 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-07-21T01:48:02.535 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-07-21T01:48:02.579 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-07-21T01:48:02.592 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2022-07-21T01:48:02.644 INFO:teuthology.orchestra.run.smithi195.stdout:creating /etc/ceph/ceph.client.2.keyring 2022-07-21T01:48:02.658 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-07-21T01:48:02.659 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi140.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-07-21T01:48:02.659 DEBUG:teuthology.orchestra.run.smithi140:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-07-21T01:48:02.678 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-07-21T01:48:02.678 INFO:tasks.ceph:role: osd.0 2022-07-21T01:48:02.679 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi140.front.sepia.ceph.com 2022-07-21T01:48:02.679 DEBUG:teuthology.orchestra.run.smithi140:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-07-21T01:48:03.241 INFO:teuthology.orchestra.run.smithi140.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-07-21T01:48:03.242 INFO:teuthology.orchestra.run.smithi140.stdout: = sectsz=512 attr=2, projid32bit=1 2022-07-21T01:48:03.242 INFO:teuthology.orchestra.run.smithi140.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-07-21T01:48:03.242 INFO:teuthology.orchestra.run.smithi140.stdout: = reflink=1 2022-07-21T01:48:03.243 INFO:teuthology.orchestra.run.smithi140.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-07-21T01:48:03.243 INFO:teuthology.orchestra.run.smithi140.stdout: = sunit=0 swidth=0 blks 2022-07-21T01:48:03.244 INFO:teuthology.orchestra.run.smithi140.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-07-21T01:48:03.244 INFO:teuthology.orchestra.run.smithi140.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-07-21T01:48:03.244 INFO:teuthology.orchestra.run.smithi140.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-07-21T01:48:03.246 INFO:teuthology.orchestra.run.smithi140.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-07-21T01:48:03.248 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi140.front.sepia.ceph.com -o noatime 2022-07-21T01:48:03.248 DEBUG:teuthology.orchestra.run.smithi140:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-07-21T01:48:03.351 DEBUG:teuthology.orchestra.run.smithi140:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-07-21T01:48:03.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:48:03.368 INFO:teuthology.orchestra.run.smithi140.stderr:sudo: /sbin/restorecon: command not found 2022-07-21T01:48:03.369 DEBUG:teuthology.orchestra.run.smithi140:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-07-21T01:48:03.426 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-07-21T01:48:03.427 INFO:tasks.ceph:role: osd.1 2022-07-21T01:48:03.427 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi140.front.sepia.ceph.com 2022-07-21T01:48:03.428 DEBUG:teuthology.orchestra.run.smithi140:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-07-21T01:48:03.920 INFO:teuthology.orchestra.run.smithi140.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-07-21T01:48:03.920 INFO:teuthology.orchestra.run.smithi140.stdout: = sectsz=512 attr=2, projid32bit=1 2022-07-21T01:48:03.921 INFO:teuthology.orchestra.run.smithi140.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-07-21T01:48:03.921 INFO:teuthology.orchestra.run.smithi140.stdout: = reflink=1 2022-07-21T01:48:03.922 INFO:teuthology.orchestra.run.smithi140.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-07-21T01:48:03.930 INFO:teuthology.orchestra.run.smithi140.stdout: = sunit=0 swidth=0 blks 2022-07-21T01:48:03.931 INFO:teuthology.orchestra.run.smithi140.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-07-21T01:48:03.931 INFO:teuthology.orchestra.run.smithi140.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-07-21T01:48:03.932 INFO:teuthology.orchestra.run.smithi140.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-07-21T01:48:03.941 INFO:teuthology.orchestra.run.smithi140.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-07-21T01:48:03.944 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi140.front.sepia.ceph.com -o noatime 2022-07-21T01:48:03.949 DEBUG:teuthology.orchestra.run.smithi140:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-07-21T01:48:03.982 DEBUG:teuthology.orchestra.run.smithi140:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-07-21T01:48:03.992 INFO:teuthology.orchestra.run.smithi140.stderr:sudo: /sbin/restorecon: command not found 2022-07-21T01:48:03.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:48:03.994 DEBUG:teuthology.orchestra.run.smithi140:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-07-21T01:48:04.059 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-07-21T01:48:04.059 INFO:tasks.ceph:role: osd.2 2022-07-21T01:48:04.060 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi140.front.sepia.ceph.com 2022-07-21T01:48:04.061 DEBUG:teuthology.orchestra.run.smithi140:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-07-21T01:48:04.581 INFO:teuthology.orchestra.run.smithi140.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-07-21T01:48:04.581 INFO:teuthology.orchestra.run.smithi140.stdout: = sectsz=512 attr=2, projid32bit=1 2022-07-21T01:48:04.582 INFO:teuthology.orchestra.run.smithi140.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-07-21T01:48:04.582 INFO:teuthology.orchestra.run.smithi140.stdout: = reflink=1 2022-07-21T01:48:04.583 INFO:teuthology.orchestra.run.smithi140.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-07-21T01:48:04.583 INFO:teuthology.orchestra.run.smithi140.stdout: = sunit=0 swidth=0 blks 2022-07-21T01:48:04.584 INFO:teuthology.orchestra.run.smithi140.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-07-21T01:48:04.586 INFO:teuthology.orchestra.run.smithi140.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-07-21T01:48:04.587 INFO:teuthology.orchestra.run.smithi140.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-07-21T01:48:04.587 INFO:teuthology.orchestra.run.smithi140.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-07-21T01:48:04.590 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi140.front.sepia.ceph.com -o noatime 2022-07-21T01:48:04.590 DEBUG:teuthology.orchestra.run.smithi140:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-07-21T01:48:04.655 DEBUG:teuthology.orchestra.run.smithi140:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-07-21T01:48:04.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:48:04.670 INFO:teuthology.orchestra.run.smithi140.stderr:sudo: /sbin/restorecon: command not found 2022-07-21T01:48:04.670 DEBUG:teuthology.orchestra.run.smithi140:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-07-21T01:48:04.728 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-07-21T01:48:04.729 INFO:tasks.ceph:role: osd.3 2022-07-21T01:48:04.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi140.front.sepia.ceph.com 2022-07-21T01:48:04.729 DEBUG:teuthology.orchestra.run.smithi140:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-07-21T01:48:05.252 INFO:teuthology.orchestra.run.smithi140.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-07-21T01:48:05.252 INFO:teuthology.orchestra.run.smithi140.stdout: = sectsz=512 attr=2, projid32bit=1 2022-07-21T01:48:05.253 INFO:teuthology.orchestra.run.smithi140.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-07-21T01:48:05.254 INFO:teuthology.orchestra.run.smithi140.stdout: = reflink=1 2022-07-21T01:48:05.254 INFO:teuthology.orchestra.run.smithi140.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-07-21T01:48:05.255 INFO:teuthology.orchestra.run.smithi140.stdout: = sunit=0 swidth=0 blks 2022-07-21T01:48:05.255 INFO:teuthology.orchestra.run.smithi140.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-07-21T01:48:05.256 INFO:teuthology.orchestra.run.smithi140.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-07-21T01:48:05.256 INFO:teuthology.orchestra.run.smithi140.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-07-21T01:48:05.257 INFO:teuthology.orchestra.run.smithi140.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-07-21T01:48:05.259 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi140.front.sepia.ceph.com -o noatime 2022-07-21T01:48:05.260 DEBUG:teuthology.orchestra.run.smithi140:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-07-21T01:48:05.318 DEBUG:teuthology.orchestra.run.smithi140:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-07-21T01:48:05.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:48:05.333 INFO:teuthology.orchestra.run.smithi140.stderr:sudo: /sbin/restorecon: command not found 2022-07-21T01:48:05.333 DEBUG:teuthology.orchestra.run.smithi140:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:05.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:05.427+0000 7f26633fed80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-07-21T01:48:05.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:05.431+0000 7f26633fed80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-07-21T01:48:05.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:05.463+0000 7f26633fed80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:05.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:05.463+0000 7f26633fed80 -1 journal FileJournal::_open_file : zeroing journal 2022-07-21T01:48:05.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:05.763+0000 7f26633fed80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:05.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:05.767+0000 7f26633fed80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:05.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:05.767+0000 7f26633fed80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:05.821 DEBUG:teuthology.orchestra.run.smithi140:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-07-21T01:48:05.857 DEBUG:teuthology.orchestra.run.smithi140:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:05.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:05.955+0000 7f2a52d11d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-07-21T01:48:05.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:05.955+0000 7f2a52d11d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-07-21T01:48:05.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:05.987+0000 7f2a52d11d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:05.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:05.987+0000 7f2a52d11d80 -1 journal FileJournal::_open_file : zeroing journal 2022-07-21T01:48:06.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.199+0000 7f2a52d11d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:06.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.199+0000 7f2a52d11d80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:06.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.199+0000 7f2a52d11d80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:06.220 DEBUG:teuthology.orchestra.run.smithi140:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-07-21T01:48:06.289 DEBUG:teuthology.orchestra.run.smithi140:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:06.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.403+0000 7f414cb2bd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-07-21T01:48:06.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.403+0000 7f414cb2bd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-07-21T01:48:06.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.415+0000 7f414cb2bd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:06.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.415+0000 7f414cb2bd80 -1 journal FileJournal::_open_file : zeroing journal 2022-07-21T01:48:06.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.551+0000 7f414cb2bd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:06.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.551+0000 7f414cb2bd80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:06.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.551+0000 7f414cb2bd80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:06.575 DEBUG:teuthology.orchestra.run.smithi140:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-07-21T01:48:06.610 DEBUG:teuthology.orchestra.run.smithi140:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:06.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.699+0000 7f4bcdb0bd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-07-21T01:48:06.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.699+0000 7f4bcdb0bd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-07-21T01:48:06.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.715+0000 7f4bcdb0bd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:06.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.715+0000 7f4bcdb0bd80 -1 journal FileJournal::_open_file : zeroing journal 2022-07-21T01:48:06.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.855+0000 7f4bcdb0bd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:06.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.855+0000 7f4bcdb0bd80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:06.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:06.855+0000 7f4bcdb0bd80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:06.871 DEBUG:teuthology.orchestra.run.smithi140:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-07-21T01:48:06.889 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-07-21T01:48:06.917 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-07-21T01:48:06.920 INFO:tasks.ceph:role: osd.4 2022-07-21T01:48:06.920 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com 2022-07-21T01:48:06.921 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-07-21T01:48:07.438 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-07-21T01:48:07.439 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2022-07-21T01:48:07.439 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-07-21T01:48:07.439 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 2022-07-21T01:48:07.440 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-07-21T01:48:07.440 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2022-07-21T01:48:07.440 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-07-21T01:48:07.440 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-07-21T01:48:07.440 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-07-21T01:48:07.441 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-07-21T01:48:07.442 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2022-07-21T01:48:07.442 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-07-21T01:48:07.524 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-07-21T01:48:07.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:48:07.546 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: /sbin/restorecon: command not found 2022-07-21T01:48:07.547 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-07-21T01:48:07.609 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-07-21T01:48:07.609 INFO:tasks.ceph:role: osd.5 2022-07-21T01:48:07.609 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com 2022-07-21T01:48:07.612 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-07-21T01:48:08.140 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-07-21T01:48:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2022-07-21T01:48:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-07-21T01:48:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 2022-07-21T01:48:08.142 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-07-21T01:48:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2022-07-21T01:48:08.142 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-07-21T01:48:08.142 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-07-21T01:48:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-07-21T01:48:08.143 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-07-21T01:48:08.144 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2022-07-21T01:48:08.145 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-07-21T01:48:08.185 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-07-21T01:48:08.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:48:08.248 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: /sbin/restorecon: command not found 2022-07-21T01:48:08.249 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-07-21T01:48:08.313 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-07-21T01:48:08.314 INFO:tasks.ceph:role: osd.6 2022-07-21T01:48:08.314 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com 2022-07-21T01:48:08.314 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-07-21T01:48:08.845 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-07-21T01:48:08.845 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2022-07-21T01:48:08.846 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-07-21T01:48:08.847 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 2022-07-21T01:48:08.847 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-07-21T01:48:08.847 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2022-07-21T01:48:08.848 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-07-21T01:48:08.848 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-07-21T01:48:08.848 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-07-21T01:48:08.848 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-07-21T01:48:08.850 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2022-07-21T01:48:08.851 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-07-21T01:48:08.928 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-07-21T01:48:08.940 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: /sbin/restorecon: command not found 2022-07-21T01:48:08.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:48:08.942 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-07-21T01:48:08.966 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-07-21T01:48:08.966 INFO:tasks.ceph:role: osd.7 2022-07-21T01:48:08.968 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com 2022-07-21T01:48:08.968 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-07-21T01:48:09.520 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-07-21T01:48:09.520 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2022-07-21T01:48:09.520 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-07-21T01:48:09.521 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 2022-07-21T01:48:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-07-21T01:48:09.521 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2022-07-21T01:48:09.521 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-07-21T01:48:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-07-21T01:48:09.522 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-07-21T01:48:09.522 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-07-21T01:48:09.522 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2022-07-21T01:48:09.523 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-07-21T01:48:09.583 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-07-21T01:48:09.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:48:09.600 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: /sbin/restorecon: command not found 2022-07-21T01:48:09.601 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:09.659 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:09.664+0000 7fe7d6d75d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-07-21T01:48:09.662 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:09.668+0000 7fe7d6d75d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-07-21T01:48:09.695 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:09.700+0000 7fe7d6d75d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:09.695 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:09.700+0000 7fe7d6d75d80 -1 journal FileJournal::_open_file : zeroing journal 2022-07-21T01:48:09.982 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:09.988+0000 7fe7d6d75d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:09.985 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:09.992+0000 7fe7d6d75d80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:09.985 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:09.992+0000 7fe7d6d75d80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:10.025 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-07-21T01:48:10.046 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:10.083 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.088+0000 7f55d8f72d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-07-21T01:48:10.084 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.088+0000 7f55d8f72d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-07-21T01:48:10.117 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.120+0000 7f55d8f72d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:10.118 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.120+0000 7f55d8f72d80 -1 journal FileJournal::_open_file : zeroing journal 2022-07-21T01:48:10.378 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.384+0000 7f55d8f72d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:10.382 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.388+0000 7f55d8f72d80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:10.382 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.388+0000 7f55d8f72d80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:10.431 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-07-21T01:48:10.447 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:10.528 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.532+0000 7f50f429fd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-07-21T01:48:10.528 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.532+0000 7f50f429fd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-07-21T01:48:10.542 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.548+0000 7f50f429fd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:10.542 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.548+0000 7f50f429fd80 -1 journal FileJournal::_open_file : zeroing journal 2022-07-21T01:48:10.688 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.692+0000 7f50f429fd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:10.691 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.696+0000 7f50f429fd80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:10.691 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.696+0000 7f50f429fd80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:10.710 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-07-21T01:48:10.726 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:10.763 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.768+0000 7ff38dfcbd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-07-21T01:48:10.764 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.768+0000 7ff38dfcbd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-07-21T01:48:10.774 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.780+0000 7ff38dfcbd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:10.775 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.780+0000 7ff38dfcbd80 -1 journal FileJournal::_open_file : zeroing journal 2022-07-21T01:48:10.927 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.932+0000 7ff38dfcbd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:10.929 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.932+0000 7ff38dfcbd80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:10.929 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:10.932+0000 7ff38dfcbd80 -1 journal do_read_entry(4096): bad header magic 2022-07-21T01:48:10.946 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-07-21T01:48:10.970 INFO:tasks.ceph:Reading keys from all nodes... 2022-07-21T01:48:10.970 DEBUG:teuthology.orchestra.run.smithi140:> mktemp 2022-07-21T01:48:10.985 INFO:teuthology.orchestra.run.smithi140.stdout:/tmp/tmp.SGuv0bbbAV 2022-07-21T01:48:10.985 DEBUG:teuthology.orchestra.run.smithi140:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.SGuv0bbbAV 2022-07-21T01:48:11.013 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0666 /tmp/tmp.SGuv0bbbAV 2022-07-21T01:48:11.116 DEBUG:teuthology.orchestra.remote:smithi140:/tmp/tmp.SGuv0bbbAV is 56B 2022-07-21T01:48:11.156 DEBUG:teuthology.orchestra.run.smithi140:> rm -fr /tmp/tmp.SGuv0bbbAV 2022-07-21T01:48:11.168 DEBUG:teuthology.orchestra.run.smithi140:> mktemp 2022-07-21T01:48:11.234 INFO:teuthology.orchestra.run.smithi140.stdout:/tmp/tmp.v6qJqf01CE 2022-07-21T01:48:11.236 DEBUG:teuthology.orchestra.run.smithi140:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.v6qJqf01CE 2022-07-21T01:48:11.257 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0666 /tmp/tmp.v6qJqf01CE 2022-07-21T01:48:11.388 DEBUG:teuthology.orchestra.remote:smithi140:/tmp/tmp.v6qJqf01CE is 56B 2022-07-21T01:48:11.423 DEBUG:teuthology.orchestra.run.smithi140:> rm -fr /tmp/tmp.v6qJqf01CE 2022-07-21T01:48:11.433 DEBUG:teuthology.orchestra.run.smithi140:> mktemp 2022-07-21T01:48:11.485 INFO:teuthology.orchestra.run.smithi140.stdout:/tmp/tmp.662q5W7yUl 2022-07-21T01:48:11.486 DEBUG:teuthology.orchestra.run.smithi140:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.662q5W7yUl 2022-07-21T01:48:11.511 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0666 /tmp/tmp.662q5W7yUl 2022-07-21T01:48:11.585 DEBUG:teuthology.orchestra.remote:smithi140:/tmp/tmp.662q5W7yUl is 56B 2022-07-21T01:48:11.620 DEBUG:teuthology.orchestra.run.smithi140:> rm -fr /tmp/tmp.662q5W7yUl 2022-07-21T01:48:11.635 DEBUG:teuthology.orchestra.run.smithi140:> mktemp 2022-07-21T01:48:11.699 INFO:teuthology.orchestra.run.smithi140.stdout:/tmp/tmp.mSAVzeNOUo 2022-07-21T01:48:11.701 DEBUG:teuthology.orchestra.run.smithi140:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.mSAVzeNOUo 2022-07-21T01:48:11.759 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0666 /tmp/tmp.mSAVzeNOUo 2022-07-21T01:48:11.829 DEBUG:teuthology.orchestra.remote:smithi140:/tmp/tmp.mSAVzeNOUo is 56B 2022-07-21T01:48:11.851 DEBUG:teuthology.orchestra.run.smithi140:> rm -fr /tmp/tmp.mSAVzeNOUo 2022-07-21T01:48:11.858 DEBUG:teuthology.orchestra.run.smithi140:> mktemp 2022-07-21T01:48:11.907 INFO:teuthology.orchestra.run.smithi140.stdout:/tmp/tmp.nvIweiGXBo 2022-07-21T01:48:11.907 DEBUG:teuthology.orchestra.run.smithi140:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.nvIweiGXBo 2022-07-21T01:48:11.921 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0666 /tmp/tmp.nvIweiGXBo 2022-07-21T01:48:11.947 DEBUG:teuthology.orchestra.remote:smithi140:/tmp/tmp.nvIweiGXBo is 56B 2022-07-21T01:48:11.999 DEBUG:teuthology.orchestra.run.smithi140:> rm -fr /tmp/tmp.nvIweiGXBo 2022-07-21T01:48:12.007 DEBUG:teuthology.orchestra.run.smithi140:> mktemp 2022-07-21T01:48:12.057 INFO:teuthology.orchestra.run.smithi140.stdout:/tmp/tmp.nDcFuQ4mnc 2022-07-21T01:48:12.057 DEBUG:teuthology.orchestra.run.smithi140:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.nDcFuQ4mnc 2022-07-21T01:48:12.073 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0666 /tmp/tmp.nDcFuQ4mnc 2022-07-21T01:48:12.140 DEBUG:teuthology.orchestra.remote:smithi140:/tmp/tmp.nDcFuQ4mnc is 56B 2022-07-21T01:48:12.156 DEBUG:teuthology.orchestra.run.smithi140:> rm -fr /tmp/tmp.nDcFuQ4mnc 2022-07-21T01:48:12.164 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T01:48:12.171 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.BFumePbvZG 2022-07-21T01:48:12.172 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.BFumePbvZG 2022-07-21T01:48:12.187 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.BFumePbvZG 2022-07-21T01:48:12.256 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.BFumePbvZG is 56B 2022-07-21T01:48:12.275 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.BFumePbvZG 2022-07-21T01:48:12.283 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T01:48:12.338 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.MZcRd0sZOa 2022-07-21T01:48:12.339 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.MZcRd0sZOa 2022-07-21T01:48:12.356 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.MZcRd0sZOa 2022-07-21T01:48:12.428 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.MZcRd0sZOa is 56B 2022-07-21T01:48:12.483 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.MZcRd0sZOa 2022-07-21T01:48:12.491 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T01:48:12.538 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.cJp6KKhQXX 2022-07-21T01:48:12.539 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.cJp6KKhQXX 2022-07-21T01:48:12.555 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.cJp6KKhQXX 2022-07-21T01:48:12.636 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.cJp6KKhQXX is 56B 2022-07-21T01:48:12.663 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.cJp6KKhQXX 2022-07-21T01:48:12.681 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T01:48:12.727 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.yUfu8QgVO0 2022-07-21T01:48:12.728 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.yUfu8QgVO0 2022-07-21T01:48:12.744 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.yUfu8QgVO0 2022-07-21T01:48:12.808 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.yUfu8QgVO0 is 56B 2022-07-21T01:48:12.826 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.yUfu8QgVO0 2022-07-21T01:48:12.834 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T01:48:12.882 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.kh5h1p0CCy 2022-07-21T01:48:12.882 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.kh5h1p0CCy 2022-07-21T01:48:12.899 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.kh5h1p0CCy 2022-07-21T01:48:12.920 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.kh5h1p0CCy is 56B 2022-07-21T01:48:12.940 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.kh5h1p0CCy 2022-07-21T01:48:12.947 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T01:48:12.995 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.6DA5TkOrqB 2022-07-21T01:48:12.995 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.6DA5TkOrqB 2022-07-21T01:48:13.010 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.6DA5TkOrqB 2022-07-21T01:48:13.083 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.6DA5TkOrqB is 56B 2022-07-21T01:48:13.105 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.6DA5TkOrqB 2022-07-21T01:48:13.131 DEBUG:teuthology.orchestra.remote:smithi191:/etc/ceph/ceph.client.0.keyring is 59B 2022-07-21T01:48:13.169 DEBUG:teuthology.orchestra.remote:smithi192:/etc/ceph/ceph.client.1.keyring is 59B 2022-07-21T01:48:13.241 DEBUG:teuthology.orchestra.remote:smithi195:/etc/ceph/ceph.client.2.keyring is 59B 2022-07-21T01:48:13.301 INFO:tasks.ceph:Adding keys to all mons... 2022-07-21T01:48:13.302 DEBUG:teuthology.orchestra.run.smithi140:> sudo tee -a /etc/ceph/ceph.keyring 2022-07-21T01:48:13.306 DEBUG:teuthology.orchestra.run.smithi152:> sudo tee -a /etc/ceph/ceph.keyring 2022-07-21T01:48:13.318 INFO:teuthology.orchestra.run.smithi140.stdout:[mgr.y] 2022-07-21T01:48:13.318 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ== 2022-07-21T01:48:13.318 INFO:teuthology.orchestra.run.smithi140.stdout:[mds.a] 2022-07-21T01:48:13.319 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w== 2022-07-21T01:48:13.319 INFO:teuthology.orchestra.run.smithi140.stdout:[osd.0] 2022-07-21T01:48:13.319 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA== 2022-07-21T01:48:13.319 INFO:teuthology.orchestra.run.smithi140.stdout:[osd.1] 2022-07-21T01:48:13.320 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ== 2022-07-21T01:48:13.320 INFO:teuthology.orchestra.run.smithi140.stdout:[osd.2] 2022-07-21T01:48:13.320 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg== 2022-07-21T01:48:13.321 INFO:teuthology.orchestra.run.smithi140.stdout:[osd.3] 2022-07-21T01:48:13.321 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w== 2022-07-21T01:48:13.321 INFO:teuthology.orchestra.run.smithi140.stdout:[mgr.x] 2022-07-21T01:48:13.322 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw== 2022-07-21T01:48:13.322 INFO:teuthology.orchestra.run.smithi140.stdout:[mds.b] 2022-07-21T01:48:13.322 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw== 2022-07-21T01:48:13.323 INFO:teuthology.orchestra.run.smithi140.stdout:[osd.4] 2022-07-21T01:48:13.324 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw== 2022-07-21T01:48:13.324 INFO:teuthology.orchestra.run.smithi140.stdout:[osd.5] 2022-07-21T01:48:13.325 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ== 2022-07-21T01:48:13.325 INFO:teuthology.orchestra.run.smithi140.stdout:[osd.6] 2022-07-21T01:48:13.326 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q== 2022-07-21T01:48:13.326 INFO:teuthology.orchestra.run.smithi140.stdout:[osd.7] 2022-07-21T01:48:13.326 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw== 2022-07-21T01:48:13.327 INFO:teuthology.orchestra.run.smithi140.stdout:[client.0] 2022-07-21T01:48:13.327 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw== 2022-07-21T01:48:13.328 INFO:teuthology.orchestra.run.smithi140.stdout:[client.1] 2022-07-21T01:48:13.328 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g== 2022-07-21T01:48:13.328 INFO:teuthology.orchestra.run.smithi140.stdout:[client.2] 2022-07-21T01:48:13.329 INFO:teuthology.orchestra.run.smithi140.stdout: key = AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g== 2022-07-21T01:48:13.332 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.y] 2022-07-21T01:48:13.332 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ== 2022-07-21T01:48:13.333 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.a] 2022-07-21T01:48:13.333 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w== 2022-07-21T01:48:13.333 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.0] 2022-07-21T01:48:13.333 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA== 2022-07-21T01:48:13.334 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.1] 2022-07-21T01:48:13.334 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ== 2022-07-21T01:48:13.334 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.2] 2022-07-21T01:48:13.335 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg== 2022-07-21T01:48:13.335 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.3] 2022-07-21T01:48:13.335 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w== 2022-07-21T01:48:13.335 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.x] 2022-07-21T01:48:13.335 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw== 2022-07-21T01:48:13.336 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.b] 2022-07-21T01:48:13.336 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw== 2022-07-21T01:48:13.336 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.4] 2022-07-21T01:48:13.336 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw== 2022-07-21T01:48:13.337 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.5] 2022-07-21T01:48:13.337 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ== 2022-07-21T01:48:13.337 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.6] 2022-07-21T01:48:13.337 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q== 2022-07-21T01:48:13.338 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.7] 2022-07-21T01:48:13.338 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw== 2022-07-21T01:48:13.338 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2022-07-21T01:48:13.338 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw== 2022-07-21T01:48:13.339 INFO:teuthology.orchestra.run.smithi152.stdout:[client.1] 2022-07-21T01:48:13.339 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g== 2022-07-21T01:48:13.339 INFO:teuthology.orchestra.run.smithi152.stdout:[client.2] 2022-07-21T01:48:13.340 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g== 2022-07-21T01:48:13.340 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-07-21T01:48:13.375 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-07-21T01:48:13.421 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-07-21T01:48:13.425 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-07-21T01:48:13.510 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:13.518 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:13.564 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:13.607 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:13.650 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:13.695 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:13.741 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:13.745 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:13.828 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-07-21T01:48:13.831 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-07-21T01:48:13.877 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-07-21T01:48:13.880 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-07-21T01:48:13.964 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:13.969 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:14.011 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:14.014 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:14.056 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:14.060 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:14.142 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:14.145 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2022-07-21T01:48:14.188 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-07-21T01:48:14.193 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-07-21T01:48:14.276 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-07-21T01:48:14.279 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-07-21T01:48:14.323 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-07-21T01:48:14.325 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-07-21T01:48:14.366 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-07-21T01:48:14.367 DEBUG:teuthology.orchestra.run.smithi140:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-07-21T01:48:14.430 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-07-21T01:48:14.487 DEBUG:teuthology.orchestra.run.smithi140:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-07-21T01:48:14.504 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-07-21T01:48:14.522 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-07-21T01:48:14.582 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-07-21T01:48:14.597 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-07-21T01:48:14.608 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-07-21T01:48:14.696 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-07-21T01:48:14.724 DEBUG:teuthology.orchestra.run.smithi140:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:14.727 DEBUG:teuthology.orchestra.run.smithi152:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-07-21T01:48:14.737 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-07-21T01:48:14.737 INFO:tasks.ceph.mon.a:Restarting daemon 2022-07-21T01:48:14.738 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-07-21T01:48:14.744 INFO:tasks.ceph.mon.a:Started 2022-07-21T01:48:14.744 INFO:tasks.ceph.mon.b:Restarting daemon 2022-07-21T01:48:14.745 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-07-21T01:48:14.782 INFO:tasks.ceph.mon.b:Started 2022-07-21T01:48:14.782 INFO:tasks.ceph.mon.c:Restarting daemon 2022-07-21T01:48:14.783 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-07-21T01:48:14.785 INFO:tasks.ceph.mon.c:Started 2022-07-21T01:48:14.786 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-07-21T01:48:14.786 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-07-21T01:48:14.787 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-07-21T01:48:14.791 INFO:tasks.ceph.mgr.y:Started 2022-07-21T01:48:14.791 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-07-21T01:48:14.792 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-07-21T01:48:14.796 INFO:tasks.ceph.mgr.x:Started 2022-07-21T01:48:14.796 INFO:tasks.ceph:Setting crush tunables to default 2022-07-21T01:48:14.797 DEBUG:teuthology.orchestra.run.smithi140:> sudo ceph --cluster ceph osd crush tunables default 2022-07-21T01:48:14.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfaffcf700 1 Processor -- start 2022-07-21T01:48:14.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfaffcf700 1 -- start start 2022-07-21T01:48:14.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfaffcf700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 0x7fbfa80af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:14.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfaffcf700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa8143550 con 0x7fbfa80b1e60 2022-07-21T01:48:14.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfaffcf700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa8142b70 con 0x7fbfa80b1460 2022-07-21T01:48:14.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfaffcf700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa8142e10 con 0x7fbfa8142660 2022-07-21T01:48:14.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfadd6b700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbfa80b1460 0x7fbfa80af090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:46972/0 (socket says 172.21.15.140:46972) 2022-07-21T01:48:14.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfadd6b700 1 -- 172.21.15.140:0/3498915670 learned_addr learned my addr 172.21.15.140:0/3498915670 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:14.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfae56c700 1 --1- 172.21.15.140:0/3498915670 >> v1:172.21.15.140:6789/0 conn(0x7fbfa80b1e60 0x7fbfa813f090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:45862/0 (socket says 172.21.15.140:45862) 2022-07-21T01:48:14.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfad56a700 1 --2- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 0x7fbfa80af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:14.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfad56a700 1 -- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 msgr2=0x7fbfa80af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-07-21T01:48:14.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfad56a700 1 -- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 msgr2=0x7fbfa80af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-07-21T01:48:14.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfad56a700 1 --2- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 0x7fbfa80af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-07-21T01:48:14.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:14.891+0000 7fbfad56a700 1 --2- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 0x7fbfa80af7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-07-21T01:48:15.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.095+0000 7fbfad56a700 1 --2- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 0x7fbfa80af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:15.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.095+0000 7fbfad56a700 1 -- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 msgr2=0x7fbfa80af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-07-21T01:48:15.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.095+0000 7fbfad56a700 1 -- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 msgr2=0x7fbfa80af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-07-21T01:48:15.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.095+0000 7fbfad56a700 1 --2- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 0x7fbfa80af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-07-21T01:48:15.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.095+0000 7fbfad56a700 1 --2- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 0x7fbfa80af7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-07-21T01:48:15.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154511422 0 0) 0x7fbfa8143550 con 0x7fbfa80b1e60 2022-07-21T01:48:15.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf90003260 con 0x7fbfa80b1e60 2022-07-21T01:48:15.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572907244 0 0) 0x7fbf90003260 con 0x7fbfa80b1e60 2022-07-21T01:48:15.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 msgr2=0x7fbfa80af7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-07-21T01:48:15.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 --2- 172.21.15.140:0/3498915670 >> v2:172.21.15.152:3300/0 conn(0x7fbfa8142660 0x7fbfa80af7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:15.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 >> v1:172.21.15.152:6789/0 conn(0x7fbfa80b1460 legacy=0x7fbfa80af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:15.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa81430b0 con 0x7fbfa80b1e60 2022-07-21T01:48:15.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fbfa00019e0 con 0x7fbfa80b1e60 2022-07-21T01:48:15.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 >> v1:172.21.15.140:6789/0 conn(0x7fbfa80b1e60 legacy=0x7fbfa813f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:15.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 --2- 172.21.15.140:0/3498915670 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90003750 0x7fbf90003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:15.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 --2- 172.21.15.140:0/3498915670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf90004500 0x7fbf90004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:15.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 --2- 172.21.15.140:0/3498915670 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 0x7fbf9000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:15.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa8142e10 con 0x7fbf90004ec0 2022-07-21T01:48:15.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9000be50 con 0x7fbf90004500 2022-07-21T01:48:15.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.135+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9000c010 con 0x7fbf90003750 2022-07-21T01:48:15.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfae56c700 1 --2- 172.21.15.140:0/3498915670 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 0x7fbf9000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:15.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfadd6b700 1 --2- 172.21.15.140:0/3498915670 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90003750 0x7fbf90003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:15.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfad56a700 1 --2- 172.21.15.140:0/3498915670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf90004500 0x7fbf90004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:15.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfae56c700 1 -- 172.21.15.140:0/3498915670 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90003750 msgr2=0x7fbf90003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:15.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfae56c700 1 --2- 172.21.15.140:0/3498915670 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90003750 0x7fbf90003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:15.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfae56c700 1 -- 172.21.15.140:0/3498915670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf90004500 msgr2=0x7fbf90004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:15.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfae56c700 1 --2- 172.21.15.140:0/3498915670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf90004500 0x7fbf90004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:15.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfae56c700 1 -- 172.21.15.140:0/3498915670 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf90003260 con 0x7fbf90004ec0 2022-07-21T01:48:15.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfae56c700 1 -- 172.21.15.140:0/3498915670 reap_dead start 2022-07-21T01:48:15.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfae56c700 1 --2- 172.21.15.140:0/3498915670 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 0x7fbf9000b750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fbfa00016f0 tx=0x7fbfa8143240).ready entity=mon.0 client_cookie=b2c04a5c11a5573 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:15.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfa000cbb0 con 0x7fbf90004ec0 2022-07-21T01:48:15.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa000cd10 con 0x7fbf90004ec0 2022-07-21T01:48:15.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfacd69700 1 -- 172.21.15.140:0/3498915670 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfa0005ac0 con 0x7fbf90004ec0 2022-07-21T01:48:15.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfaffcf700 1 -- 172.21.15.140:0/3498915670 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 msgr2=0x7fbf9000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:15.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfaffcf700 1 --2- 172.21.15.140:0/3498915670 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 0x7fbf9000b750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fbfa00016f0 tx=0x7fbfa8143240).stop 2022-07-21T01:48:15.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfaffcf700 1 -- 172.21.15.140:0/3498915670 shutdown_connections 2022-07-21T01:48:15.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfaffcf700 1 --2- 172.21.15.140:0/3498915670 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 0x7fbf9000b750 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:15.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfaffcf700 1 -- 172.21.15.140:0/3498915670 >> 172.21.15.140:0/3498915670 conn(0x7fbfa80b23b0 msgr2=0x7fbfa80b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:15.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfaffcf700 1 -- 172.21.15.140:0/3498915670 shutdown_connections 2022-07-21T01:48:15.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.139+0000 7fbfaffcf700 1 -- 172.21.15.140:0/3498915670 wait complete. 2022-07-21T01:48:15.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfaffcf700 1 Processor -- start 2022-07-21T01:48:15.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfaffcf700 1 -- start start 2022-07-21T01:48:15.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfaffcf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 0x7fbfa8066da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:15.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfaffcf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa814ea00 0x7fbfa8065df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:15.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfaffcf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa80662e0 0x7fbfa8064280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:15.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfaffcf700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa813ae40 con 0x7fbf90004ec0 2022-07-21T01:48:15.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfaffcf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa813fbd0 con 0x7fbfa814ea00 2022-07-21T01:48:15.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfaffcf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa80667f0 con 0x7fbfa80662e0 2022-07-21T01:48:15.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfadd6b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 0x7fbfa8066da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:15.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfadd6b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 0x7fbfa8066da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:59966/0 (socket says 172.21.15.140:59966) 2022-07-21T01:48:15.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfadd6b700 1 -- 172.21.15.140:0/2306215761 learned_addr learned my addr 172.21.15.140:0/2306215761 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:15.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfad56a700 1 --2- 172.21.15.140:0/2306215761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa814ea00 0x7fbfa8065df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:15.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfadd6b700 1 -- 172.21.15.140:0/2306215761 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa80662e0 msgr2=0x7fbfa8064280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-07-21T01:48:15.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfadd6b700 1 --2- 172.21.15.140:0/2306215761 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa80662e0 0x7fbfa8064280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:48:15.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfadd6b700 1 -- 172.21.15.140:0/2306215761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa814ea00 msgr2=0x7fbfa8065df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:15.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfadd6b700 1 --2- 172.21.15.140:0/2306215761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa814ea00 0x7fbfa8065df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:15.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfadd6b700 1 -- 172.21.15.140:0/2306215761 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa8066a90 con 0x7fbf90004ec0 2022-07-21T01:48:15.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfad56a700 1 --2- 172.21.15.140:0/2306215761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa814ea00 0x7fbfa8065df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:15.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.143+0000 7fbfadd6b700 1 --2- 172.21.15.140:0/2306215761 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 0x7fbfa8066da0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fbf9c00d000 tx=0x7fbf9c007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:15.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.147+0000 7fbf9affd700 1 -- 172.21.15.140:0/2306215761 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf9c015410 con 0x7fbf90004ec0 2022-07-21T01:48:15.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.147+0000 7fbf9affd700 1 -- 172.21.15.140:0/2306215761 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9c015570 con 0x7fbf90004ec0 2022-07-21T01:48:15.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.147+0000 7fbfaffcf700 1 -- 172.21.15.140:0/2306215761 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa813e9d0 con 0x7fbf90004ec0 2022-07-21T01:48:15.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.147+0000 7fbfaffcf700 1 -- 172.21.15.140:0/2306215761 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa813f990 con 0x7fbf90004ec0 2022-07-21T01:48:15.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.147+0000 7fbf9affd700 1 -- 172.21.15.140:0/2306215761 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf9c0158d0 con 0x7fbf90004ec0 2022-07-21T01:48:15.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.147+0000 7fbfaffcf700 1 -- 172.21.15.140:0/2306215761 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa8067760 con 0x7fbf90004ec0 2022-07-21T01:48:15.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.147+0000 7fbf9affd700 1 -- 172.21.15.140:0/2306215761 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fbf9c01b020 con 0x7fbf90004ec0 2022-07-21T01:48:15.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.147+0000 7fbf9affd700 1 -- 172.21.15.140:0/2306215761 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fbf9c01eb50 con 0x7fbf90004ec0 2022-07-21T01:48:15.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.151+0000 7fbf9affd700 1 -- 172.21.15.140:0/2306215761 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fbfa813f990 con 0x7fbf90004ec0 2022-07-21T01:48:15.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.223+0000 7fbfaffcf700 1 -- 172.21.15.140:0/2306215761 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbfa80574e0 con 0x7fbf90004ec0 2022-07-21T01:48:15.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.227+0000 7fbf9affd700 1 -- 172.21.15.140:0/2306215761 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbf9c01e5c0 con 0x7fbf90004ec0 2022-07-21T01:48:15.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.231+0000 7fbfaffcf700 1 -- 172.21.15.140:0/2306215761 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 msgr2=0x7fbfa8066da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:15.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.231+0000 7fbfaffcf700 1 --2- 172.21.15.140:0/2306215761 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 0x7fbfa8066da0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fbf9c00d000 tx=0x7fbf9c007ab0).stop 2022-07-21T01:48:15.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.231+0000 7fbfaffcf700 1 -- 172.21.15.140:0/2306215761 shutdown_connections 2022-07-21T01:48:15.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.231+0000 7fbfaffcf700 1 --2- 172.21.15.140:0/2306215761 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa80662e0 0x7fbfa8064280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:48:15.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.231+0000 7fbfaffcf700 1 --2- 172.21.15.140:0/2306215761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa814ea00 0x7fbfa8065df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:15.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.231+0000 7fbfaffcf700 1 --2- 172.21.15.140:0/2306215761 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf90004ec0 0x7fbfa8066da0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:15.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.231+0000 7fbfaffcf700 1 -- 172.21.15.140:0/2306215761 >> 172.21.15.140:0/2306215761 conn(0x7fbfa80b23b0 msgr2=0x7fbfa80aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:15.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.231+0000 7fbfaffcf700 1 -- 172.21.15.140:0/2306215761 shutdown_connections 2022-07-21T01:48:15.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:15.231+0000 7fbfaffcf700 1 -- 172.21.15.140:0/2306215761 wait complete. 2022-07-21T01:48:15.233 INFO:teuthology.orchestra.run.smithi140.stderr:adjusted tunables profile to default 2022-07-21T01:48:15.241 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-07-21T01:48:15.242 DEBUG:teuthology.orchestra.run.smithi140:> mktemp 2022-07-21T01:48:15.249 INFO:teuthology.orchestra.run.smithi140.stdout:/tmp/tmp.HF3pjSbpLk 2022-07-21T01:48:15.250 DEBUG:teuthology.orchestra.run.smithi140:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.HF3pjSbpLk 2022-07-21T01:48:15.266 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0666 /tmp/tmp.HF3pjSbpLk 2022-07-21T01:48:15.339 DEBUG:teuthology.orchestra.remote:smithi140:/tmp/tmp.HF3pjSbpLk is 37B 2022-07-21T01:48:15.390 DEBUG:teuthology.orchestra.run.smithi140:> rm -fr /tmp/tmp.HF3pjSbpLk 2022-07-21T01:48:15.398 DEBUG:teuthology.orchestra.run.smithi140:> mktemp 2022-07-21T01:48:15.451 INFO:teuthology.orchestra.run.smithi140.stdout:/tmp/tmp.4p8bfYOgSF 2022-07-21T01:48:15.452 DEBUG:teuthology.orchestra.run.smithi140:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.4p8bfYOgSF 2022-07-21T01:48:15.468 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0666 /tmp/tmp.4p8bfYOgSF 2022-07-21T01:48:15.533 DEBUG:teuthology.orchestra.remote:smithi140:/tmp/tmp.4p8bfYOgSF is 37B 2022-07-21T01:48:15.548 DEBUG:teuthology.orchestra.run.smithi140:> rm -fr /tmp/tmp.4p8bfYOgSF 2022-07-21T01:48:15.555 DEBUG:teuthology.orchestra.run.smithi140:> mktemp 2022-07-21T01:48:15.603 INFO:teuthology.orchestra.run.smithi140.stdout:/tmp/tmp.BtNW0aDE1J 2022-07-21T01:48:15.604 DEBUG:teuthology.orchestra.run.smithi140:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.BtNW0aDE1J 2022-07-21T01:48:15.619 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0666 /tmp/tmp.BtNW0aDE1J 2022-07-21T01:48:15.692 DEBUG:teuthology.orchestra.remote:smithi140:/tmp/tmp.BtNW0aDE1J is 37B 2022-07-21T01:48:15.709 DEBUG:teuthology.orchestra.run.smithi140:> rm -fr /tmp/tmp.BtNW0aDE1J 2022-07-21T01:48:15.718 DEBUG:teuthology.orchestra.run.smithi140:> mktemp 2022-07-21T01:48:15.768 INFO:teuthology.orchestra.run.smithi140.stdout:/tmp/tmp.DPrLpTI1CX 2022-07-21T01:48:15.769 DEBUG:teuthology.orchestra.run.smithi140:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.DPrLpTI1CX 2022-07-21T01:48:15.784 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0666 /tmp/tmp.DPrLpTI1CX 2022-07-21T01:48:15.855 DEBUG:teuthology.orchestra.remote:smithi140:/tmp/tmp.DPrLpTI1CX is 37B 2022-07-21T01:48:15.868 DEBUG:teuthology.orchestra.run.smithi140:> rm -fr /tmp/tmp.DPrLpTI1CX 2022-07-21T01:48:15.875 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T01:48:15.883 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.G8oMSzrFkZ 2022-07-21T01:48:15.884 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.G8oMSzrFkZ 2022-07-21T01:48:15.939 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.G8oMSzrFkZ 2022-07-21T01:48:15.960 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.G8oMSzrFkZ is 37B 2022-07-21T01:48:15.980 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.G8oMSzrFkZ 2022-07-21T01:48:15.988 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T01:48:16.041 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.0m6zAB9Les 2022-07-21T01:48:16.042 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.0m6zAB9Les 2022-07-21T01:48:16.058 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.0m6zAB9Les 2022-07-21T01:48:16.126 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.0m6zAB9Les is 37B 2022-07-21T01:48:16.144 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.0m6zAB9Les 2022-07-21T01:48:16.157 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T01:48:16.203 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.CFi2xxEJ4p 2022-07-21T01:48:16.203 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.CFi2xxEJ4p 2022-07-21T01:48:16.218 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.CFi2xxEJ4p 2022-07-21T01:48:16.283 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.CFi2xxEJ4p is 37B 2022-07-21T01:48:16.331 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.CFi2xxEJ4p 2022-07-21T01:48:16.340 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T01:48:16.391 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.jKN9BOkVHa 2022-07-21T01:48:16.392 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.jKN9BOkVHa 2022-07-21T01:48:16.408 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.jKN9BOkVHa 2022-07-21T01:48:16.474 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.jKN9BOkVHa is 37B 2022-07-21T01:48:16.525 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.jKN9BOkVHa 2022-07-21T01:48:16.531 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new f32f4b5f-243e-4377-b3e7-6d2876095aec 0 2022-07-21T01:48:16.686 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.676+0000 7fa67a55c700 1 Processor -- start 2022-07-21T01:48:16.687 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.676+0000 7fa67a55c700 1 -- start start 2022-07-21T01:48:16.688 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa67a55c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740a9690 0x7fa6740ac510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:16.688 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa67a55c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6740b4290 con 0x7fa6740af0d0 2022-07-21T01:48:16.689 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa67a55c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6740af4b0 con 0x7fa6740b4c20 2022-07-21T01:48:16.689 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa67a55c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6740a9340 con 0x7fa6740a9690 2022-07-21T01:48:16.690 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa678d59700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740a9690 0x7fa6740ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:16.690 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa678d59700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740a9690 0x7fa6740ac510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:39344/0 (socket says 172.21.15.152:39344) 2022-07-21T01:48:16.691 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa678d59700 1 -- 172.21.15.152:0/4003353621 learned_addr learned my addr 172.21.15.152:0/4003353621 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:16.691 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa678d59700 1 -- 172.21.15.152:0/4003353621 >> v1:172.21.15.152:6789/0 conn(0x7fa6740b4c20 legacy=0x7fa6741409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:16.692 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa678d59700 1 -- 172.21.15.152:0/4003353621 >> v1:172.21.15.140:6789/0 conn(0x7fa6740af0d0 legacy=0x7fa6740abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:16.692 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa678d59700 1 -- 172.21.15.152:0/4003353621 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6740b44e0 con 0x7fa6740a9690 2022-07-21T01:48:16.693 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa678d59700 1 --2- 172.21.15.152:0/4003353621 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740a9690 0x7fa6740ac510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fa664002f70 tx=0x7fa664006eb0).ready entity=mon.2 client_cookie=ca4c30c5f6fecb3b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:16.693 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa663fff700 1 -- 172.21.15.152:0/4003353621 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa664009600 con 0x7fa6740a9690 2022-07-21T01:48:16.694 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa663fff700 1 -- 172.21.15.152:0/4003353621 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa664000e90 con 0x7fa6740a9690 2022-07-21T01:48:16.694 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa663fff700 1 -- 172.21.15.152:0/4003353621 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa664009990 con 0x7fa6740a9690 2022-07-21T01:48:16.695 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa67a55c700 1 -- 172.21.15.152:0/4003353621 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740a9690 msgr2=0x7fa6740ac510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:16.695 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa67a55c700 1 --2- 172.21.15.152:0/4003353621 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740a9690 0x7fa6740ac510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fa664002f70 tx=0x7fa664006eb0).stop 2022-07-21T01:48:16.696 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa67a55c700 1 -- 172.21.15.152:0/4003353621 shutdown_connections 2022-07-21T01:48:16.696 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa67a55c700 1 --2- 172.21.15.152:0/4003353621 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740a9690 0x7fa6740ac510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.697 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa67a55c700 1 -- 172.21.15.152:0/4003353621 >> 172.21.15.152:0/4003353621 conn(0x7fa674063150 msgr2=0x7fa674063530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:16.697 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa67a55c700 1 -- 172.21.15.152:0/4003353621 shutdown_connections 2022-07-21T01:48:16.698 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.680+0000 7fa67a55c700 1 -- 172.21.15.152:0/4003353621 wait complete. 2022-07-21T01:48:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67a55c700 1 Processor -- start 2022-07-21T01:48:16.711 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67a55c700 1 -- start start 2022-07-21T01:48:16.712 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67a55c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6740a9690 0x7fa67413fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:16.712 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67a55c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6740af0d0 0x7fa6741537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:16.713 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67a55c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740b4c20 0x7fa674153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:16.713 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67a55c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa674066f20 con 0x7fa6740af0d0 2022-07-21T01:48:16.714 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67a55c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6741541a0 con 0x7fa6740a9690 2022-07-21T01:48:16.714 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67a55c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa674063fc0 con 0x7fa6740b4c20 2022-07-21T01:48:16.714 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa679d5b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740b4c20 0x7fa674153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:16.715 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67955a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6740a9690 0x7fa67413fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:16.715 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa679d5b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740b4c20 0x7fa674153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:39352/0 (socket says 172.21.15.152:39352) 2022-07-21T01:48:16.715 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa679d5b700 1 -- 172.21.15.152:0/692086966 learned_addr learned my addr 172.21.15.152:0/692086966 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:16.715 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa678d59700 1 --2- 172.21.15.152:0/692086966 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6740af0d0 0x7fa6741537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:16.716 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa679d5b700 1 -- 172.21.15.152:0/692086966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6740a9690 msgr2=0x7fa67413fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:16.716 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa679d5b700 1 --2- 172.21.15.152:0/692086966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6740a9690 0x7fa67413fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.716 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa679d5b700 1 -- 172.21.15.152:0/692086966 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6740af0d0 msgr2=0x7fa6741537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:16.716 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa679d5b700 1 --2- 172.21.15.152:0/692086966 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6740af0d0 0x7fa6741537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.717 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa679d5b700 1 -- 172.21.15.152:0/692086966 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66400b1e0 con 0x7fa6740b4c20 2022-07-21T01:48:16.717 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67955a700 1 --2- 172.21.15.152:0/692086966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6740a9690 0x7fa67413fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:16.717 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa679d5b700 1 --2- 172.21.15.152:0/692086966 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740b4c20 0x7fa674153cb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fa674063f30 tx=0x7fa66c007f70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:16.717 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa6627fc700 1 -- 172.21.15.152:0/692086966 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa66c00cea0 con 0x7fa6740b4c20 2022-07-21T01:48:16.718 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa6627fc700 1 -- 172.21.15.152:0/692086966 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c006aa0 con 0x7fa6740b4c20 2022-07-21T01:48:16.718 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67a55c700 1 -- 172.21.15.152:0/692086966 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa674143ba0 con 0x7fa6740b4c20 2022-07-21T01:48:16.718 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa6627fc700 1 -- 172.21.15.152:0/692086966 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa66c006e00 con 0x7fa6740b4c20 2022-07-21T01:48:16.718 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa67a55c700 1 -- 172.21.15.152:0/692086966 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa674064260 con 0x7fa6740b4c20 2022-07-21T01:48:16.719 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa6627fc700 1 -- 172.21.15.152:0/692086966 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fa66c010030 con 0x7fa6740b4c20 2022-07-21T01:48:16.719 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.684+0000 7fa6627fc700 1 -- 172.21.15.152:0/692086966 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7fa66c01d7b0 con 0x7fa6740b4c20 2022-07-21T01:48:16.720 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.688+0000 7fa67a55c700 1 -- 172.21.15.152:0/692086966 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6740b4780 con 0x7fa6740b4c20 2022-07-21T01:48:16.720 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.692+0000 7fa6627fc700 1 -- 172.21.15.152:0/692086966 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fa66c0076b0 con 0x7fa6740b4c20 2022-07-21T01:48:16.750 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.756+0000 7fa67a55c700 1 -- 172.21.15.152:0/692086966 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", "id": 0} v 0) v1 -- 0x7fa6740574e0 con 0x7fa6740b4c20 2022-07-21T01:48:16.759 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.764+0000 7fa6627fc700 1 -- 172.21.15.152:0/692086966 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa66c01d970 con 0x7fa6740b4c20 2022-07-21T01:48:16.760 INFO:teuthology.orchestra.run.smithi152.stdout:0 2022-07-21T01:48:16.762 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.764+0000 7fa67a55c700 1 -- 172.21.15.152:0/692086966 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740b4c20 msgr2=0x7fa674153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:16.762 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.764+0000 7fa67a55c700 1 --2- 172.21.15.152:0/692086966 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740b4c20 0x7fa674153cb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fa674063f30 tx=0x7fa66c007f70).stop 2022-07-21T01:48:16.763 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.764+0000 7fa67a55c700 1 -- 172.21.15.152:0/692086966 shutdown_connections 2022-07-21T01:48:16.763 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.764+0000 7fa67a55c700 1 --2- 172.21.15.152:0/692086966 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6740b4c20 0x7fa674153cb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.764 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.764+0000 7fa67a55c700 1 --2- 172.21.15.152:0/692086966 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6740af0d0 0x7fa6741537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.764 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.764+0000 7fa67a55c700 1 --2- 172.21.15.152:0/692086966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6740a9690 0x7fa67413fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.764 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.764+0000 7fa67a55c700 1 -- 172.21.15.152:0/692086966 >> 172.21.15.152:0/692086966 conn(0x7fa674063150 msgr2=0x7fa6740abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:16.765 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.764+0000 7fa67a55c700 1 -- 172.21.15.152:0/692086966 shutdown_connections 2022-07-21T01:48:16.765 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.764+0000 7fa67a55c700 1 -- 172.21.15.152:0/692086966 wait complete. 2022-07-21T01:48:16.773 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 50f0b7fd-5258-4d4d-8075-828f0b34d254 1 2022-07-21T01:48:16.869 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 Processor -- start 2022-07-21T01:48:16.870 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 -- start start 2022-07-21T01:48:16.877 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbe4c05c7c0 0x7fbe4c0b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:16.877 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbe4c1430b0 0x7fbe4c1469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:16.877 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4c0b0900 con 0x7fbe4c0ab250 2022-07-21T01:48:16.877 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4c0b2a30 con 0x7fbe4c1430b0 2022-07-21T01:48:16.878 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4c094c50 con 0x7fbe4c05c7c0 2022-07-21T01:48:16.878 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4a59c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbe4c05c7c0 0x7fbe4c0b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:16.878 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4a59c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbe4c05c7c0 0x7fbe4c0b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51760/0 (socket says 172.21.15.152:51760) 2022-07-21T01:48:16.879 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4a59c700 1 -- 172.21.15.152:0/1706473538 learned_addr learned my addr 172.21.15.152:0/1706473538 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:16.879 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4ad9d700 1 --2- 172.21.15.152:0/1706473538 >> v2:172.21.15.140:3300/0 conn(0x7fbe4c1430b0 0x7fbe4c1469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:16.879 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4a59c700 1 -- 172.21.15.152:0/1706473538 >> v2:172.21.15.140:3300/0 conn(0x7fbe4c1430b0 msgr2=0x7fbe4c1469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:16.879 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4a59c700 1 --2- 172.21.15.152:0/1706473538 >> v2:172.21.15.140:3300/0 conn(0x7fbe4c1430b0 0x7fbe4c1469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.880 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4a59c700 1 -- 172.21.15.152:0/1706473538 >> v1:172.21.15.140:6789/0 conn(0x7fbe4c0ab250 legacy=0x7fbe4c1462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:16.880 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4a59c700 1 -- 172.21.15.152:0/1706473538 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c0b0c50 con 0x7fbe4c05c7c0 2022-07-21T01:48:16.880 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4a59c700 1 --2- 172.21.15.152:0/1706473538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe4c05c7c0 0x7fbe4c0b1390 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fbe3c002f70 tx=0x7fbe3c006eb0).ready entity=mon.1 client_cookie=d56a445411b227ae server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:16.880 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4959a700 1 -- 172.21.15.152:0/1706473538 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe3c00d610 con 0x7fbe4c05c7c0 2022-07-21T01:48:16.881 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4959a700 1 -- 172.21.15.152:0/1706473538 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c000e90 con 0x7fbe4c05c7c0 2022-07-21T01:48:16.881 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4959a700 1 -- 172.21.15.152:0/1706473538 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe3c00d9a0 con 0x7fbe4c05c7c0 2022-07-21T01:48:16.881 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 -- 172.21.15.152:0/1706473538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe4c05c7c0 msgr2=0x7fbe4c0b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:16.881 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 --2- 172.21.15.152:0/1706473538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe4c05c7c0 0x7fbe4c0b1390 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fbe3c002f70 tx=0x7fbe3c006eb0).stop 2022-07-21T01:48:16.882 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 -- 172.21.15.152:0/1706473538 shutdown_connections 2022-07-21T01:48:16.882 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 --2- 172.21.15.152:0/1706473538 >> v2:172.21.15.140:3300/0 conn(0x7fbe4c1430b0 0x7fbe4c1469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.882 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 --2- 172.21.15.152:0/1706473538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe4c05c7c0 0x7fbe4c0b1390 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.882 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 -- 172.21.15.152:0/1706473538 >> 172.21.15.152:0/1706473538 conn(0x7fbe4c0b6210 msgr2=0x7fbe4c0b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:16.882 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 -- 172.21.15.152:0/1706473538 shutdown_connections 2022-07-21T01:48:16.883 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.876+0000 7fbe4b59e700 1 -- 172.21.15.152:0/1706473538 wait complete. 2022-07-21T01:48:16.883 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4b59e700 1 Processor -- start 2022-07-21T01:48:16.883 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4b59e700 1 -- start start 2022-07-21T01:48:16.883 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe4c05c7c0 0x7fbe4c09b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:16.884 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe4c0ab250 0x7fbe4c065ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:16.884 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe4c1430b0 0x7fbe4c0663d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:16.884 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c09b480 con 0x7fbe4c0ab250 2022-07-21T01:48:16.884 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c09b210 con 0x7fbe4c1430b0 2022-07-21T01:48:16.885 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c099cf0 con 0x7fbe4c05c7c0 2022-07-21T01:48:16.885 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4ad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe4c1430b0 0x7fbe4c0663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:16.885 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4ad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe4c1430b0 0x7fbe4c0663d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51766/0 (socket says 172.21.15.152:51766) 2022-07-21T01:48:16.885 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4ad9d700 1 -- 172.21.15.152:0/2595343563 learned_addr learned my addr 172.21.15.152:0/2595343563 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:16.885 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4a59c700 1 --2- 172.21.15.152:0/2595343563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe4c05c7c0 0x7fbe4c09b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:16.886 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe49d9b700 1 --2- 172.21.15.152:0/2595343563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe4c0ab250 0x7fbe4c065ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:16.886 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4ad9d700 1 -- 172.21.15.152:0/2595343563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe4c05c7c0 msgr2=0x7fbe4c09b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:16.886 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4ad9d700 1 --2- 172.21.15.152:0/2595343563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe4c05c7c0 0x7fbe4c09b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.886 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4ad9d700 1 -- 172.21.15.152:0/2595343563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe4c0ab250 msgr2=0x7fbe4c065ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:16.887 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4ad9d700 1 --2- 172.21.15.152:0/2595343563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe4c0ab250 0x7fbe4c065ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.887 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4ad9d700 1 -- 172.21.15.152:0/2595343563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c00b1e0 con 0x7fbe4c1430b0 2022-07-21T01:48:16.887 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4ad9d700 1 --2- 172.21.15.152:0/2595343563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe4c1430b0 0x7fbe4c0663d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fbe4400a000 tx=0x7fbe44008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:16.887 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe3b7fe700 1 -- 172.21.15.152:0/2595343563 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe440127e0 con 0x7fbe4c1430b0 2022-07-21T01:48:16.888 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe3b7fe700 1 -- 172.21.15.152:0/2595343563 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe44012940 con 0x7fbe4c1430b0 2022-07-21T01:48:16.888 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4b59e700 1 -- 172.21.15.152:0/2595343563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4c099f90 con 0x7fbe4c1430b0 2022-07-21T01:48:16.888 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe3b7fe700 1 -- 172.21.15.152:0/2595343563 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe44012ca0 con 0x7fbe4c1430b0 2022-07-21T01:48:16.888 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4b59e700 1 -- 172.21.15.152:0/2595343563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4c067900 con 0x7fbe4c1430b0 2022-07-21T01:48:16.889 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe3b7fe700 1 -- 172.21.15.152:0/2595343563 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fbe44005a70 con 0x7fbe4c1430b0 2022-07-21T01:48:16.889 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe3b7fe700 1 -- 172.21.15.152:0/2595343563 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fbe44006530 con 0x7fbe4c1430b0 2022-07-21T01:48:16.889 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.880+0000 7fbe4b59e700 1 -- 172.21.15.152:0/2595343563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4c0b0ef0 con 0x7fbe4c1430b0 2022-07-21T01:48:16.890 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.884+0000 7fbe3b7fe700 1 -- 172.21.15.152:0/2595343563 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fbe44006740 con 0x7fbe4c1430b0 2022-07-21T01:48:16.943 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.948+0000 7fbe4b59e700 1 -- 172.21.15.152:0/2595343563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", "id": 1} v 0) v1 -- 0x7fbe4c0574e0 con 0x7fbe4c1430b0 2022-07-21T01:48:16.951 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.956+0000 7fbe3b7fe700 1 -- 172.21.15.152:0/2595343563 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbe4402b0b0 con 0x7fbe4c1430b0 2022-07-21T01:48:16.951 INFO:teuthology.orchestra.run.smithi152.stdout:1 2022-07-21T01:48:16.953 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.960+0000 7fbe4b59e700 1 -- 172.21.15.152:0/2595343563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe4c1430b0 msgr2=0x7fbe4c0663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:16.953 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.960+0000 7fbe4b59e700 1 --2- 172.21.15.152:0/2595343563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe4c1430b0 0x7fbe4c0663d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fbe4400a000 tx=0x7fbe44008040).stop 2022-07-21T01:48:16.953 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.960+0000 7fbe4b59e700 1 -- 172.21.15.152:0/2595343563 shutdown_connections 2022-07-21T01:48:16.954 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.960+0000 7fbe4b59e700 1 --2- 172.21.15.152:0/2595343563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe4c1430b0 0x7fbe4c0663d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.954 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.960+0000 7fbe4b59e700 1 --2- 172.21.15.152:0/2595343563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe4c0ab250 0x7fbe4c065ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.954 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.960+0000 7fbe4b59e700 1 --2- 172.21.15.152:0/2595343563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe4c05c7c0 0x7fbe4c09b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:16.954 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.960+0000 7fbe4b59e700 1 -- 172.21.15.152:0/2595343563 >> 172.21.15.152:0/2595343563 conn(0x7fbe4c0b6210 msgr2=0x7fbe4c141150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:16.955 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.960+0000 7fbe4b59e700 1 -- 172.21.15.152:0/2595343563 shutdown_connections 2022-07-21T01:48:16.955 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:16.960+0000 7fbe4b59e700 1 -- 172.21.15.152:0/2595343563 wait complete. 2022-07-21T01:48:16.965 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 0c4190c0-0c1b-49da-ba40-93dab22993a2 2 2022-07-21T01:48:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.072+0000 7fb4a68d0700 1 Processor -- start 2022-07-21T01:48:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.072+0000 7fb4a68d0700 1 -- start start 2022-07-21T01:48:17.076 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.072+0000 7fb4a68d0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb4a00b3ed0 0x7fb4a0141e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.076 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.072+0000 7fb4a68d0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 0x7fb4a01433f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.076 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.072+0000 7fb4a68d0700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a00b42b0 con 0x7fb4a01422f0 2022-07-21T01:48:17.076 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.072+0000 7fb4a68d0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a01460d0 con 0x7fb4a00b3ed0 2022-07-21T01:48:17.077 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.072+0000 7fb4a68d0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00acb80 con 0x7fb4a0142780 2022-07-21T01:48:17.077 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.072+0000 7fb4a58ce700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb4a00b3ed0 0x7fb4a0141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.077 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.072+0000 7fb4a60cf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 0x7fb4a01433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.077 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.072+0000 7fb4a58ce700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb4a00b3ed0 0x7fb4a0141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51772/0 (socket says 172.21.15.152:51772) 2022-07-21T01:48:17.078 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.072+0000 7fb4a60cf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 0x7fb4a01433f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:39370/0 (socket says 172.21.15.152:39370) 2022-07-21T01:48:17.078 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a58ce700 1 -- 172.21.15.152:0/2248898501 learned_addr learned my addr 172.21.15.152:0/2248898501 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:17.078 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a58ce700 1 -- 172.21.15.152:0/2248898501 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 msgr2=0x7fb4a01433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:17.078 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a58ce700 1 --2- 172.21.15.152:0/2248898501 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 0x7fb4a01433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.079 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a58ce700 1 -- 172.21.15.152:0/2248898501 >> v1:172.21.15.140:6789/0 conn(0x7fb4a01422f0 legacy=0x7fb4a0142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.079 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a58ce700 1 -- 172.21.15.152:0/2248898501 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a00ace20 con 0x7fb4a00b3ed0 2022-07-21T01:48:17.079 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a58ce700 1 --2- 172.21.15.152:0/2248898501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a00b3ed0 0x7fb4a0141e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fb49c002f70 tx=0x7fb49c006eb0).ready entity=mon.1 client_cookie=5bf0ff75e6649f04 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:17.079 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a48cc700 1 -- 172.21.15.152:0/2248898501 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb49c00d570 con 0x7fb4a00b3ed0 2022-07-21T01:48:17.080 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a48cc700 1 -- 172.21.15.152:0/2248898501 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49c000e90 con 0x7fb4a00b3ed0 2022-07-21T01:48:17.080 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a48cc700 1 -- 172.21.15.152:0/2248898501 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb49c00d900 con 0x7fb4a00b3ed0 2022-07-21T01:48:17.080 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 -- 172.21.15.152:0/2248898501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a00b3ed0 msgr2=0x7fb4a0141e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.080 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 --2- 172.21.15.152:0/2248898501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a00b3ed0 0x7fb4a0141e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fb49c002f70 tx=0x7fb49c006eb0).stop 2022-07-21T01:48:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 -- 172.21.15.152:0/2248898501 shutdown_connections 2022-07-21T01:48:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 --2- 172.21.15.152:0/2248898501 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 0x7fb4a01433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 --2- 172.21.15.152:0/2248898501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a00b3ed0 0x7fb4a0141e00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 -- 172.21.15.152:0/2248898501 >> 172.21.15.152:0/2248898501 conn(0x7fb4a0064c00 msgr2=0x7fb4a0064fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 -- 172.21.15.152:0/2248898501 shutdown_connections 2022-07-21T01:48:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 -- 172.21.15.152:0/2248898501 wait complete. 2022-07-21T01:48:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 Processor -- start 2022-07-21T01:48:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 -- start start 2022-07-21T01:48:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a00b3ed0 0x7fb4a00b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.076+0000 7fb4a68d0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4a01422f0 0x7fb4a00af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a68d0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 0x7fb4a0155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a68d0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00b5ec0 con 0x7fb4a01422f0 2022-07-21T01:48:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a58ce700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a00b3ed0 0x7fb4a00b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a68d0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00b5110 con 0x7fb4a00b3ed0 2022-07-21T01:48:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a58ce700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a00b3ed0 0x7fb4a00b6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51778/0 (socket says 172.21.15.152:51778) 2022-07-21T01:48:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a60cf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 0x7fb4a0155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a58ce700 1 -- 172.21.15.152:0/210351228 learned_addr learned my addr 172.21.15.152:0/210351228 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a68d0700 1 -- 172.21.15.152:0/210351228 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00b54a0 con 0x7fb4a0142780 2022-07-21T01:48:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a50cd700 1 --2- 172.21.15.152:0/210351228 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4a01422f0 0x7fb4a00af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a60cf700 1 -- 172.21.15.152:0/210351228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a00b3ed0 msgr2=0x7fb4a00b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a60cf700 1 --2- 172.21.15.152:0/210351228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a00b3ed0 0x7fb4a00b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a60cf700 1 -- 172.21.15.152:0/210351228 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4a01422f0 msgr2=0x7fb4a00af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a60cf700 1 --2- 172.21.15.152:0/210351228 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4a01422f0 0x7fb4a00af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a60cf700 1 -- 172.21.15.152:0/210351228 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c00b1e0 con 0x7fb4a0142780 2022-07-21T01:48:17.087 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a60cf700 1 --2- 172.21.15.152:0/210351228 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 0x7fb4a0155460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fb49800c000 tx=0x7fb4980091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:17.087 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb48effd700 1 -- 172.21.15.152:0/210351228 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb49800c610 con 0x7fb4a0142780 2022-07-21T01:48:17.087 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb48effd700 1 -- 172.21.15.152:0/210351228 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49800c770 con 0x7fb4a0142780 2022-07-21T01:48:17.087 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a68d0700 1 -- 172.21.15.152:0/210351228 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a00b0ff0 con 0x7fb4a0142780 2022-07-21T01:48:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb48effd700 1 -- 172.21.15.152:0/210351228 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb49800cad0 con 0x7fb4a0142780 2022-07-21T01:48:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a68d0700 1 -- 172.21.15.152:0/210351228 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a00ae8d0 con 0x7fb4a0142780 2022-07-21T01:48:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb48effd700 1 -- 172.21.15.152:0/210351228 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fb498022070 con 0x7fb4a0142780 2022-07-21T01:48:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb48effd700 1 -- 172.21.15.152:0/210351228 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7fb49800ce00 con 0x7fb4a0142780 2022-07-21T01:48:17.089 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.080+0000 7fb4a68d0700 1 -- 172.21.15.152:0/210351228 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a00b60c0 con 0x7fb4a0142780 2022-07-21T01:48:17.089 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.084+0000 7fb48effd700 1 -- 172.21.15.152:0/210351228 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fb498039870 con 0x7fb4a0142780 2022-07-21T01:48:17.143 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.148+0000 7fb4a68d0700 1 -- 172.21.15.152:0/210351228 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", "id": 2} v 0) v1 -- 0x7fb4a00b2230 con 0x7fb4a0142780 2022-07-21T01:48:17.152 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.156+0000 7fb48effd700 1 -- 172.21.15.152:0/210351228 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb498029030 con 0x7fb4a0142780 2022-07-21T01:48:17.153 INFO:teuthology.orchestra.run.smithi152.stdout:2 2022-07-21T01:48:17.154 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.160+0000 7fb4a68d0700 1 -- 172.21.15.152:0/210351228 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 msgr2=0x7fb4a0155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.154 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.160+0000 7fb4a68d0700 1 --2- 172.21.15.152:0/210351228 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 0x7fb4a0155460 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fb49800c000 tx=0x7fb4980091d0).stop 2022-07-21T01:48:17.154 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.160+0000 7fb4a68d0700 1 -- 172.21.15.152:0/210351228 shutdown_connections 2022-07-21T01:48:17.155 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.160+0000 7fb4a68d0700 1 --2- 172.21.15.152:0/210351228 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4a0142780 0x7fb4a0155460 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.155 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.160+0000 7fb4a68d0700 1 --2- 172.21.15.152:0/210351228 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4a01422f0 0x7fb4a00af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.155 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.160+0000 7fb4a68d0700 1 --2- 172.21.15.152:0/210351228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4a00b3ed0 0x7fb4a00b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.155 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.160+0000 7fb4a68d0700 1 -- 172.21.15.152:0/210351228 >> 172.21.15.152:0/210351228 conn(0x7fb4a0064c00 msgr2=0x7fb4a0141bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:17.156 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.160+0000 7fb4a68d0700 1 -- 172.21.15.152:0/210351228 shutdown_connections 2022-07-21T01:48:17.156 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.160+0000 7fb4a68d0700 1 -- 172.21.15.152:0/210351228 wait complete. 2022-07-21T01:48:17.168 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 6318d81c-39e1-4e05-8ada-4bb0536c536f 3 2022-07-21T01:48:17.268 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.272+0000 7f2290fc9700 1 Processor -- start 2022-07-21T01:48:17.274 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.272+0000 7f2290fc9700 1 -- start start 2022-07-21T01:48:17.286 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f228c05c960 0x7f228c060070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.286 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c146e20 0x7f228c13c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.286 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228c142f90 con 0x7f228c13ced0 2022-07-21T01:48:17.287 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228c05f840 con 0x7f228c05c960 2022-07-21T01:48:17.287 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c05fae0 con 0x7f228c146e20 2022-07-21T01:48:17.287 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f228a59c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f228c05c960 0x7f228c060070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.287 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2289d9b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c146e20 0x7f228c13c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.288 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2289d9b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c146e20 0x7f228c13c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:39380/0 (socket says 172.21.15.152:39380) 2022-07-21T01:48:17.288 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f228a59c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f228c05c960 0x7f228c060070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51784/0 (socket says 172.21.15.152:51784) 2022-07-21T01:48:17.288 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2289d9b700 1 -- 172.21.15.152:0/3566617305 learned_addr learned my addr 172.21.15.152:0/3566617305 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:17.288 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f228a59c700 1 -- 172.21.15.152:0/3566617305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c146e20 msgr2=0x7f228c13c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:17.289 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f228a59c700 1 --2- 172.21.15.152:0/3566617305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c146e20 0x7f228c13c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.289 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f228a59c700 1 -- 172.21.15.152:0/3566617305 >> v1:172.21.15.140:6789/0 conn(0x7f228c13ced0 legacy=0x7f228c1433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.289 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f228a59c700 1 -- 172.21.15.152:0/3566617305 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c144050 con 0x7f228c05c960 2022-07-21T01:48:17.289 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f228a59c700 1 --2- 172.21.15.152:0/3566617305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c05c960 0x7f228c060070 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f227c002f70 tx=0x7f227c006df0).ready entity=mon.1 client_cookie=4df567011ca69fda server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:17.289 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f228959a700 1 -- 172.21.15.152:0/3566617305 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f227c00d570 con 0x7f228c05c960 2022-07-21T01:48:17.290 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f228959a700 1 -- 172.21.15.152:0/3566617305 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c000e90 con 0x7f228c05c960 2022-07-21T01:48:17.290 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f228959a700 1 -- 172.21.15.152:0/3566617305 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f227c00d8c0 con 0x7f228c05c960 2022-07-21T01:48:17.290 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 -- 172.21.15.152:0/3566617305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c05c960 msgr2=0x7f228c060070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.291 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 --2- 172.21.15.152:0/3566617305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c05c960 0x7f228c060070 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f227c002f70 tx=0x7f227c006df0).stop 2022-07-21T01:48:17.291 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 -- 172.21.15.152:0/3566617305 shutdown_connections 2022-07-21T01:48:17.291 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 --2- 172.21.15.152:0/3566617305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c146e20 0x7f228c13c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.291 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 --2- 172.21.15.152:0/3566617305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c05c960 0x7f228c060070 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.291 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 -- 172.21.15.152:0/3566617305 >> 172.21.15.152:0/3566617305 conn(0x7f228c063bb0 msgr2=0x7f228c063f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:17.292 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 -- 172.21.15.152:0/3566617305 shutdown_connections 2022-07-21T01:48:17.292 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 -- 172.21.15.152:0/3566617305 wait complete. 2022-07-21T01:48:17.292 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 Processor -- start 2022-07-21T01:48:17.292 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.276+0000 7f2290fc9700 1 -- start start 2022-07-21T01:48:17.293 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2290fc9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c05c960 0x7f228c0b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.293 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2290fc9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c13ced0 0x7f228c155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.293 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2290fc9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f228c146e20 0x7f228c1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.293 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2290fc9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0651d0 con 0x7f228c146e20 2022-07-21T01:48:17.293 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2290fc9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c13e6d0 con 0x7f228c13ced0 2022-07-21T01:48:17.294 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2290fc9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c13e430 con 0x7f228c05c960 2022-07-21T01:48:17.294 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2289d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c13ced0 0x7f228c155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.294 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f228a59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c05c960 0x7f228c0b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.294 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2289d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c13ced0 0x7f228c155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51792/0 (socket says 172.21.15.152:51792) 2022-07-21T01:48:17.295 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f228a59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c05c960 0x7f228c0b5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:39384/0 (socket says 172.21.15.152:39384) 2022-07-21T01:48:17.295 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2289d9b700 1 -- 172.21.15.152:0/1354732648 learned_addr learned my addr 172.21.15.152:0/1354732648 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:17.295 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f228ad9d700 1 --2- 172.21.15.152:0/1354732648 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f228c146e20 0x7f228c1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.295 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2289d9b700 1 -- 172.21.15.152:0/1354732648 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c05c960 msgr2=0x7f228c0b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.296 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2289d9b700 1 --2- 172.21.15.152:0/1354732648 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c05c960 0x7f228c0b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.296 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2289d9b700 1 -- 172.21.15.152:0/1354732648 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f228c146e20 msgr2=0x7f228c1561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.296 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2289d9b700 1 --2- 172.21.15.152:0/1354732648 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f228c146e20 0x7f228c1561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.296 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2289d9b700 1 -- 172.21.15.152:0/1354732648 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c00b180 con 0x7f228c13ced0 2022-07-21T01:48:17.297 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2289d9b700 1 --2- 172.21.15.152:0/1354732648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c13ced0 0x7f228c155cd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f228c13fff0 tx=0x7f2280008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:17.297 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f227b7fe700 1 -- 172.21.15.152:0/1354732648 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f228000fe60 con 0x7f228c13ced0 2022-07-21T01:48:17.297 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f227b7fe700 1 -- 172.21.15.152:0/1354732648 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2280013610 con 0x7f228c13ced0 2022-07-21T01:48:17.297 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2290fc9700 1 -- 172.21.15.152:0/1354732648 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c1400d0 con 0x7f228c13ced0 2022-07-21T01:48:17.298 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f227b7fe700 1 -- 172.21.15.152:0/1354732648 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2280013970 con 0x7f228c13ced0 2022-07-21T01:48:17.298 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2290fc9700 1 -- 172.21.15.152:0/1354732648 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c064b00 con 0x7f228c13ced0 2022-07-21T01:48:17.298 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f227b7fe700 1 -- 172.21.15.152:0/1354732648 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f2280013ca0 con 0x7f228c13ced0 2022-07-21T01:48:17.298 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f227b7fe700 1 -- 172.21.15.152:0/1354732648 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f228001b870 con 0x7f228c13ced0 2022-07-21T01:48:17.299 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.280+0000 7f2290fc9700 1 -- 172.21.15.152:0/1354732648 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c05cd40 con 0x7f228c13ced0 2022-07-21T01:48:17.299 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.284+0000 7f227b7fe700 1 -- 172.21.15.152:0/1354732648 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f2280017020 con 0x7f228c13ced0 2022-07-21T01:48:17.343 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.348+0000 7f2290fc9700 1 -- 172.21.15.152:0/1354732648 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", "id": 3} v 0) v1 -- 0x7f228c0574e0 con 0x7f228c13ced0 2022-07-21T01:48:17.352 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.360+0000 7f227b7fe700 1 -- 172.21.15.152:0/1354732648 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f228001b3b0 con 0x7f228c13ced0 2022-07-21T01:48:17.353 INFO:teuthology.orchestra.run.smithi152.stdout:3 2022-07-21T01:48:17.354 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.360+0000 7f2290fc9700 1 -- 172.21.15.152:0/1354732648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c13ced0 msgr2=0x7f228c155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.354 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.360+0000 7f2290fc9700 1 --2- 172.21.15.152:0/1354732648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c13ced0 0x7f228c155cd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f228c13fff0 tx=0x7f2280008040).stop 2022-07-21T01:48:17.354 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.360+0000 7f2290fc9700 1 -- 172.21.15.152:0/1354732648 shutdown_connections 2022-07-21T01:48:17.355 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.360+0000 7f2290fc9700 1 --2- 172.21.15.152:0/1354732648 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f228c146e20 0x7f228c1561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.355 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.360+0000 7f2290fc9700 1 --2- 172.21.15.152:0/1354732648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f228c13ced0 0x7f228c155cd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.355 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.360+0000 7f2290fc9700 1 --2- 172.21.15.152:0/1354732648 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f228c05c960 0x7f228c0b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.355 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.360+0000 7f2290fc9700 1 -- 172.21.15.152:0/1354732648 >> 172.21.15.152:0/1354732648 conn(0x7f228c063bb0 msgr2=0x7f228c146360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:17.356 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.360+0000 7f2290fc9700 1 -- 172.21.15.152:0/1354732648 shutdown_connections 2022-07-21T01:48:17.356 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.360+0000 7f2290fc9700 1 -- 172.21.15.152:0/1354732648 wait complete. 2022-07-21T01:48:17.375 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new fc54a43f-8a54-40e1-9db5-e4bb3c84ed26 4 2022-07-21T01:48:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 Processor -- start 2022-07-21T01:48:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 -- start start 2022-07-21T01:48:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 0x7fde6813e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fde68066090 0x7fde68066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.489 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde680b4800 con 0x7fde680b4420 2022-07-21T01:48:17.489 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde680ad000 con 0x7fde68066090 2022-07-21T01:48:17.489 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde680ad2a0 con 0x7fde68144030 2022-07-21T01:48:17.491 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde667fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 0x7fde6813e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.492 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde667fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 0x7fde6813e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:39392/0 (socket says 172.21.15.152:39392) 2022-07-21T01:48:17.492 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde66ffd700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fde680b4420 0x7fde6813d980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:37750/0 (socket says 172.21.15.152:37750) 2022-07-21T01:48:17.493 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde667fc700 1 -- 172.21.15.152:0/439947050 learned_addr learned my addr 172.21.15.152:0/439947050 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:17.493 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde677fe700 1 --2- 172.21.15.152:0/439947050 >> v2:172.21.15.140:3300/0 conn(0x7fde68066090 0x7fde68066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.494 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde65ffb700 1 -- 172.21.15.152:0/439947050 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358568469 0 0) 0x7fde680b4800 con 0x7fde680b4420 2022-07-21T01:48:17.494 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde667fc700 1 -- 172.21.15.152:0/439947050 >> v2:172.21.15.140:3300/0 conn(0x7fde68066090 msgr2=0x7fde68066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:17.495 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde667fc700 1 --2- 172.21.15.152:0/439947050 >> v2:172.21.15.140:3300/0 conn(0x7fde68066090 0x7fde68066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.495 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde667fc700 1 -- 172.21.15.152:0/439947050 >> v1:172.21.15.152:6789/0 conn(0x7fde680b4420 legacy=0x7fde6813d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.496 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde667fc700 1 -- 172.21.15.152:0/439947050 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde680b1910 con 0x7fde68144030 2022-07-21T01:48:17.497 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde667fc700 1 --2- 172.21.15.152:0/439947050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 0x7fde6813e090 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fde5c002f70 tx=0x7fde5c006eb0).ready entity=mon.2 client_cookie=ac3af6ccf558fc1d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:17.497 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde65ffb700 1 -- 172.21.15.152:0/439947050 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde5c00d570 con 0x7fde68144030 2022-07-21T01:48:17.497 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde65ffb700 1 -- 172.21.15.152:0/439947050 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c000e90 con 0x7fde68144030 2022-07-21T01:48:17.498 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde65ffb700 1 -- 172.21.15.152:0/439947050 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde5c00d8c0 con 0x7fde68144030 2022-07-21T01:48:17.498 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 -- 172.21.15.152:0/439947050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 msgr2=0x7fde6813e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.498 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 --2- 172.21.15.152:0/439947050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 0x7fde6813e090 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fde5c002f70 tx=0x7fde5c006eb0).stop 2022-07-21T01:48:17.499 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 -- 172.21.15.152:0/439947050 shutdown_connections 2022-07-21T01:48:17.499 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 --2- 172.21.15.152:0/439947050 >> v2:172.21.15.140:3300/0 conn(0x7fde68066090 0x7fde68066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.499 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 --2- 172.21.15.152:0/439947050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 0x7fde6813e090 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.499 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 -- 172.21.15.152:0/439947050 >> 172.21.15.152:0/439947050 conn(0x7fde68064d00 msgr2=0x7fde68065110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:17.500 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 -- 172.21.15.152:0/439947050 shutdown_connections 2022-07-21T01:48:17.500 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.480+0000 7fde6d4dd700 1 -- 172.21.15.152:0/439947050 wait complete. 2022-07-21T01:48:17.500 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde6d4dd700 1 Processor -- start 2022-07-21T01:48:17.500 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde6d4dd700 1 -- start start 2022-07-21T01:48:17.500 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde6d4dd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fde68066090 0x7fde6813b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.501 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde6d4dd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde680b4420 0x7fde681552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.501 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde6d4dd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 0x7fde681557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.501 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde6d4dd700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde68146240 con 0x7fde68066090 2022-07-21T01:48:17.501 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde6d4dd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde68146470 con 0x7fde680b4420 2022-07-21T01:48:17.502 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde6d4dd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde68060c20 con 0x7fde68144030 2022-07-21T01:48:17.502 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde667fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde680b4420 0x7fde681552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.502 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde677fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 0x7fde681557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.502 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde667fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde680b4420 0x7fde681552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51804/0 (socket says 172.21.15.152:51804) 2022-07-21T01:48:17.502 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde667fc700 1 -- 172.21.15.152:0/232918466 learned_addr learned my addr 172.21.15.152:0/232918466 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:17.503 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde66ffd700 1 --2- 172.21.15.152:0/232918466 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fde68066090 0x7fde6813b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.503 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde667fc700 1 -- 172.21.15.152:0/232918466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 msgr2=0x7fde681557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.503 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde667fc700 1 --2- 172.21.15.152:0/232918466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 0x7fde681557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.504 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde667fc700 1 -- 172.21.15.152:0/232918466 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fde68066090 msgr2=0x7fde6813b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.504 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde667fc700 1 --2- 172.21.15.152:0/232918466 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fde68066090 0x7fde6813b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.504 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde667fc700 1 -- 172.21.15.152:0/232918466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c00b1e0 con 0x7fde680b4420 2022-07-21T01:48:17.504 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde667fc700 1 --2- 172.21.15.152:0/232918466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde680b4420 0x7fde681552d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fde5c002f70 tx=0x7fde5c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:17.504 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde57fff700 1 -- 172.21.15.152:0/232918466 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde5c0068b0 con 0x7fde680b4420 2022-07-21T01:48:17.505 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde57fff700 1 -- 172.21.15.152:0/232918466 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c00d8f0 con 0x7fde680b4420 2022-07-21T01:48:17.505 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde57fff700 1 -- 172.21.15.152:0/232918466 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde5c011d70 con 0x7fde680b4420 2022-07-21T01:48:17.505 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde6d4dd700 1 -- 172.21.15.152:0/232918466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde68147d20 con 0x7fde680b4420 2022-07-21T01:48:17.505 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde6d4dd700 1 -- 172.21.15.152:0/232918466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde68155ed0 con 0x7fde680b4420 2022-07-21T01:48:17.506 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde57fff700 1 -- 172.21.15.152:0/232918466 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fde5c01a040 con 0x7fde680b4420 2022-07-21T01:48:17.506 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.484+0000 7fde57fff700 1 -- 172.21.15.152:0/232918466 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7fde5c01e630 con 0x7fde680b4420 2022-07-21T01:48:17.506 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.488+0000 7fde6d4dd700 1 -- 172.21.15.152:0/232918466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde680ae330 con 0x7fde680b4420 2022-07-21T01:48:17.507 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.488+0000 7fde57fff700 1 -- 172.21.15.152:0/232918466 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fde5c023070 con 0x7fde680b4420 2022-07-21T01:48:17.552 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.556+0000 7fde6d4dd700 1 -- 172.21.15.152:0/232918466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", "id": 4} v 0) v1 -- 0x7fde680574e0 con 0x7fde680b4420 2022-07-21T01:48:17.560 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.568+0000 7fde57fff700 1 -- 172.21.15.152:0/232918466 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fde5c006020 con 0x7fde680b4420 2022-07-21T01:48:17.561 INFO:teuthology.orchestra.run.smithi152.stdout:4 2022-07-21T01:48:17.563 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.568+0000 7fde6d4dd700 1 -- 172.21.15.152:0/232918466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde680b4420 msgr2=0x7fde681552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.563 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.568+0000 7fde6d4dd700 1 --2- 172.21.15.152:0/232918466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde680b4420 0x7fde681552d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fde5c002f70 tx=0x7fde5c015000).stop 2022-07-21T01:48:17.563 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.568+0000 7fde6d4dd700 1 -- 172.21.15.152:0/232918466 shutdown_connections 2022-07-21T01:48:17.564 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.568+0000 7fde6d4dd700 1 --2- 172.21.15.152:0/232918466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde68144030 0x7fde681557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.564 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.568+0000 7fde6d4dd700 1 --2- 172.21.15.152:0/232918466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde680b4420 0x7fde681552d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.564 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.568+0000 7fde6d4dd700 1 --2- 172.21.15.152:0/232918466 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fde68066090 0x7fde6813b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.565 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.568+0000 7fde6d4dd700 1 -- 172.21.15.152:0/232918466 >> 172.21.15.152:0/232918466 conn(0x7fde68064d00 msgr2=0x7fde6813da10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:17.565 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.568+0000 7fde6d4dd700 1 -- 172.21.15.152:0/232918466 shutdown_connections 2022-07-21T01:48:17.566 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.568+0000 7fde6d4dd700 1 -- 172.21.15.152:0/232918466 wait complete. 2022-07-21T01:48:17.574 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new d63edac8-54f9-468a-8748-faf4231a7cb8 5 2022-07-21T01:48:17.672 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 Processor -- start 2022-07-21T01:48:17.673 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 -- start start 2022-07-21T01:48:17.676 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7face80b5320 0x7face805c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.676 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7face805c990 0x7face8146430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.676 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face80b0c20 con 0x7face80b9eb0 2022-07-21T01:48:17.677 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face80ba290 con 0x7face80b5320 2022-07-21T01:48:17.677 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face8142e00 con 0x7face805c990 2022-07-21T01:48:17.677 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facedcfb700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7face80b5320 0x7face805c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.677 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facedcfb700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7face80b5320 0x7face805c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51810/0 (socket says 172.21.15.152:51810) 2022-07-21T01:48:17.678 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facedcfb700 1 -- 172.21.15.152:0/3472330879 learned_addr learned my addr 172.21.15.152:0/3472330879 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:17.678 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7faceecfd700 1 --2- 172.21.15.152:0/3472330879 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7face805c990 0x7face8146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.678 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7faceecfd700 1 -- 172.21.15.152:0/3472330879 >> v2:172.21.15.152:3300/0 conn(0x7face80b5320 msgr2=0x7face805c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:17.678 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7faceecfd700 1 --2- 172.21.15.152:0/3472330879 >> v2:172.21.15.152:3300/0 conn(0x7face80b5320 0x7face805c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.679 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7faceecfd700 1 -- 172.21.15.152:0/3472330879 >> v1:172.21.15.152:6789/0 conn(0x7face80b9eb0 legacy=0x7face805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.679 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7faceecfd700 1 -- 172.21.15.152:0/3472330879 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face80afed0 con 0x7face805c990 2022-07-21T01:48:17.679 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facedcfb700 1 --2- 172.21.15.152:0/3472330879 >> v2:172.21.15.152:3300/0 conn(0x7face80b5320 0x7face805c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:17.679 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7faceecfd700 1 --2- 172.21.15.152:0/3472330879 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7face805c990 0x7face8146430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7face00032f0 tx=0x7face00091d0).ready entity=mon.2 client_cookie=5fb7ec78c92daec2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:17.680 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7faced4fa700 1 -- 172.21.15.152:0/3472330879 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7face000b6c0 con 0x7face805c990 2022-07-21T01:48:17.680 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7faced4fa700 1 -- 172.21.15.152:0/3472330879 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face000b820 con 0x7face805c990 2022-07-21T01:48:17.680 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7faced4fa700 1 -- 172.21.15.152:0/3472330879 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7face000b990 con 0x7face805c990 2022-07-21T01:48:17.680 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 -- 172.21.15.152:0/3472330879 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7face805c990 msgr2=0x7face8146430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.681 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 --2- 172.21.15.152:0/3472330879 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7face805c990 0x7face8146430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7face00032f0 tx=0x7face00091d0).stop 2022-07-21T01:48:17.681 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 -- 172.21.15.152:0/3472330879 shutdown_connections 2022-07-21T01:48:17.681 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 --2- 172.21.15.152:0/3472330879 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7face805c990 0x7face8146430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.681 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 --2- 172.21.15.152:0/3472330879 >> v2:172.21.15.152:3300/0 conn(0x7face80b5320 0x7face805c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.682 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 -- 172.21.15.152:0/3472330879 >> 172.21.15.152:0/3472330879 conn(0x7face8067980 msgr2=0x7face80b9460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:17.682 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 -- 172.21.15.152:0/3472330879 shutdown_connections 2022-07-21T01:48:17.682 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.680+0000 7facf0760700 1 -- 172.21.15.152:0/3472330879 wait complete. 2022-07-21T01:48:17.685 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facf0760700 1 Processor -- start 2022-07-21T01:48:17.686 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facf0760700 1 -- start start 2022-07-21T01:48:17.686 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facf0760700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7face805c990 0x7face81557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.686 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facf0760700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7face80b5320 0x7face81453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.687 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facf0760700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7face80b9eb0 0x7face80acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.687 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facf0760700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face8146410 con 0x7face805c990 2022-07-21T01:48:17.687 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facf0760700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face8153100 con 0x7face80b5320 2022-07-21T01:48:17.687 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facf0760700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face8144f60 con 0x7face80b9eb0 2022-07-21T01:48:17.687 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facedcfb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7face80b5320 0x7face81453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.688 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facedcfb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7face80b5320 0x7face81453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51816/0 (socket says 172.21.15.152:51816) 2022-07-21T01:48:17.688 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facedcfb700 1 -- 172.21.15.152:0/1228654098 learned_addr learned my addr 172.21.15.152:0/1228654098 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:17.688 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7faceecfd700 1 --2- 172.21.15.152:0/1228654098 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7face80b9eb0 0x7face80acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.688 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facee4fc700 1 --2- 172.21.15.152:0/1228654098 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7face805c990 0x7face81557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.689 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facedcfb700 1 -- 172.21.15.152:0/1228654098 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7face80b9eb0 msgr2=0x7face80acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.689 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facedcfb700 1 --2- 172.21.15.152:0/1228654098 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7face80b9eb0 0x7face80acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.689 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facedcfb700 1 -- 172.21.15.152:0/1228654098 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7face805c990 msgr2=0x7face81557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.689 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facedcfb700 1 --2- 172.21.15.152:0/1228654098 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7face805c990 0x7face81557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.690 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facedcfb700 1 -- 172.21.15.152:0/1228654098 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd800b1e0 con 0x7face80b5320 2022-07-21T01:48:17.690 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facedcfb700 1 --2- 172.21.15.152:0/1228654098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7face80b5320 0x7face81453c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7facd800c000 tx=0x7facd8006eb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:17.690 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facd77fe700 1 -- 172.21.15.152:0/1228654098 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facd8012600 con 0x7face80b5320 2022-07-21T01:48:17.690 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facd77fe700 1 -- 172.21.15.152:0/1228654098 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd8012760 con 0x7face80b5320 2022-07-21T01:48:17.690 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facf0760700 1 -- 172.21.15.152:0/1228654098 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face0009470 con 0x7face80b5320 2022-07-21T01:48:17.691 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facd77fe700 1 -- 172.21.15.152:0/1228654098 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7facd8012ac0 con 0x7face80b5320 2022-07-21T01:48:17.691 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.684+0000 7facf0760700 1 -- 172.21.15.152:0/1228654098 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face8155cd0 con 0x7face80b5320 2022-07-21T01:48:17.691 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.688+0000 7facd77fe700 1 -- 172.21.15.152:0/1228654098 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7facd8022070 con 0x7face80b5320 2022-07-21T01:48:17.691 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.688+0000 7facd77fe700 1 -- 172.21.15.152:0/1228654098 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7facd801bbb0 con 0x7face80b5320 2022-07-21T01:48:17.692 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.688+0000 7facf0760700 1 -- 172.21.15.152:0/1228654098 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face8145a50 con 0x7face80b5320 2022-07-21T01:48:17.692 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.688+0000 7facd77fe700 1 -- 172.21.15.152:0/1228654098 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7facd8039870 con 0x7face80b5320 2022-07-21T01:48:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.752+0000 7facf0760700 1 -- 172.21.15.152:0/1228654098 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", "id": 5} v 0) v1 -- 0x7face80b0170 con 0x7face80b5320 2022-07-21T01:48:17.752 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.760+0000 7facd77fe700 1 -- 172.21.15.152:0/1228654098 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7facd8029070 con 0x7face80b5320 2022-07-21T01:48:17.753 INFO:teuthology.orchestra.run.smithi152.stdout:5 2022-07-21T01:48:17.754 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.760+0000 7facf0760700 1 -- 172.21.15.152:0/1228654098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7face80b5320 msgr2=0x7face81453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.755 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.760+0000 7facf0760700 1 --2- 172.21.15.152:0/1228654098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7face80b5320 0x7face81453c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7facd800c000 tx=0x7facd8006eb0).stop 2022-07-21T01:48:17.755 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.760+0000 7facf0760700 1 -- 172.21.15.152:0/1228654098 shutdown_connections 2022-07-21T01:48:17.755 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.760+0000 7facf0760700 1 --2- 172.21.15.152:0/1228654098 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7face80b9eb0 0x7face80acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.755 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.760+0000 7facf0760700 1 --2- 172.21.15.152:0/1228654098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7face80b5320 0x7face81453c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.755 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.760+0000 7facf0760700 1 --2- 172.21.15.152:0/1228654098 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7face805c990 0x7face81557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.760+0000 7facf0760700 1 -- 172.21.15.152:0/1228654098 >> 172.21.15.152:0/1228654098 conn(0x7face8067980 msgr2=0x7face8143000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.760+0000 7facf0760700 1 -- 172.21.15.152:0/1228654098 shutdown_connections 2022-07-21T01:48:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.760+0000 7facf0760700 1 -- 172.21.15.152:0/1228654098 wait complete. 2022-07-21T01:48:17.771 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 81228326-ddc7-4683-83f7-c1c6eb0232d4 6 2022-07-21T01:48:17.876 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67aae4d700 1 Processor -- start 2022-07-21T01:48:17.876 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67aae4d700 1 -- start start 2022-07-21T01:48:17.876 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67aae4d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f67a40a9690 0x7f67a40ac510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.876 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67aae4d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40b4c20 0x7f67a41409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.877 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67aae4d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a40b4290 con 0x7f67a40af0d0 2022-07-21T01:48:17.877 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67aae4d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a40af4b0 con 0x7f67a40a9690 2022-07-21T01:48:17.877 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67aae4d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40a9340 con 0x7f67a40b4c20 2022-07-21T01:48:17.877 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67a964a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f67a40a9690 0x7f67a40ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.878 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67aa64c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40b4c20 0x7f67a41409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.878 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67aa64c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40b4c20 0x7f67a41409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:39418/0 (socket says 172.21.15.152:39418) 2022-07-21T01:48:17.878 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67a964a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f67a40a9690 0x7f67a40ac510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51822/0 (socket says 172.21.15.152:51822) 2022-07-21T01:48:17.879 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.876+0000 7f67aa64c700 1 -- 172.21.15.152:0/2678559574 learned_addr learned my addr 172.21.15.152:0/2678559574 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:17.879 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aa64c700 1 -- 172.21.15.152:0/2678559574 >> v2:172.21.15.152:3300/0 conn(0x7f67a40a9690 msgr2=0x7f67a40ac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:17.880 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aa64c700 1 --2- 172.21.15.152:0/2678559574 >> v2:172.21.15.152:3300/0 conn(0x7f67a40a9690 0x7f67a40ac510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.880 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aa64c700 1 -- 172.21.15.152:0/2678559574 >> v1:172.21.15.140:6789/0 conn(0x7f67a40af0d0 legacy=0x7f67a40abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.880 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aa64c700 1 -- 172.21.15.152:0/2678559574 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a40b44e0 con 0x7f67a40b4c20 2022-07-21T01:48:17.880 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67a964a700 1 --2- 172.21.15.152:0/2678559574 >> v2:172.21.15.152:3300/0 conn(0x7f67a40a9690 0x7f67a40ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:17.881 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aa64c700 1 --2- 172.21.15.152:0/2678559574 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40b4c20 0x7f67a41409b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f679c0032f0 tx=0x7f679c009290).ready entity=mon.2 client_cookie=d8903cc6e8ced527 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:17.881 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67a8e49700 1 -- 172.21.15.152:0/2678559574 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679c00b8a0 con 0x7f67a40b4c20 2022-07-21T01:48:17.881 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67a8e49700 1 -- 172.21.15.152:0/2678559574 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c00ba00 con 0x7f67a40b4c20 2022-07-21T01:48:17.881 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67a8e49700 1 -- 172.21.15.152:0/2678559574 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679c00bb70 con 0x7f67a40b4c20 2022-07-21T01:48:17.882 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aae4d700 1 -- 172.21.15.152:0/2678559574 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40b4c20 msgr2=0x7f67a41409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.882 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aae4d700 1 --2- 172.21.15.152:0/2678559574 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40b4c20 0x7f67a41409b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f679c0032f0 tx=0x7f679c009290).stop 2022-07-21T01:48:17.882 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aae4d700 1 -- 172.21.15.152:0/2678559574 shutdown_connections 2022-07-21T01:48:17.883 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aae4d700 1 --2- 172.21.15.152:0/2678559574 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40b4c20 0x7f67a41409b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.883 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aae4d700 1 --2- 172.21.15.152:0/2678559574 >> v2:172.21.15.152:3300/0 conn(0x7f67a40a9690 0x7f67a40ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.883 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aae4d700 1 -- 172.21.15.152:0/2678559574 >> 172.21.15.152:0/2678559574 conn(0x7f67a4063150 msgr2=0x7f67a4063530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:17.883 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aae4d700 1 -- 172.21.15.152:0/2678559574 shutdown_connections 2022-07-21T01:48:17.884 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aae4d700 1 -- 172.21.15.152:0/2678559574 wait complete. 2022-07-21T01:48:17.884 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aae4d700 1 Processor -- start 2022-07-21T01:48:17.884 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.880+0000 7f67aae4d700 1 -- start start 2022-07-21T01:48:17.887 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67aae4d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67a40a9690 0x7f67a40aa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.887 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67aae4d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40af0d0 0x7f67a40a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.888 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67aae4d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67a40b4c20 0x7f67a40a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:17.888 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67aae4d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4143820 con 0x7f67a40a9690 2022-07-21T01:48:17.888 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67aae4d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a40b2d20 con 0x7f67a40b4c20 2022-07-21T01:48:17.888 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67aae4d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4142800 con 0x7f67a40af0d0 2022-07-21T01:48:17.889 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67a964a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40af0d0 0x7f67a40a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.889 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67aa64c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67a40b4c20 0x7f67a40a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.889 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67a964a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40af0d0 0x7f67a40a83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:39422/0 (socket says 172.21.15.152:39422) 2022-07-21T01:48:17.889 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67a9e4b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67a40a9690 0x7f67a40aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:17.890 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67a964a700 1 -- 172.21.15.152:0/2952377109 learned_addr learned my addr 172.21.15.152:0/2952377109 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:17.890 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67a964a700 1 -- 172.21.15.152:0/2952377109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67a40b4c20 msgr2=0x7f67a40a88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.890 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67a964a700 1 --2- 172.21.15.152:0/2952377109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67a40b4c20 0x7f67a40a88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.890 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67a964a700 1 -- 172.21.15.152:0/2952377109 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67a40a9690 msgr2=0x7f67a40aa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.890 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67a964a700 1 --2- 172.21.15.152:0/2952377109 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67a40a9690 0x7f67a40aa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.891 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67a964a700 1 -- 172.21.15.152:0/2952377109 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679400b180 con 0x7f67a40af0d0 2022-07-21T01:48:17.891 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67aa64c700 1 --2- 172.21.15.152:0/2952377109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67a40b4c20 0x7f67a40a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:17.891 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67a964a700 1 --2- 172.21.15.152:0/2952377109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40af0d0 0x7f67a40a83c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f679400c000 tx=0x7f6794006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:17.891 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f6792ffd700 1 -- 172.21.15.152:0/2952377109 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679400c7e0 con 0x7f67a40af0d0 2022-07-21T01:48:17.892 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f6792ffd700 1 -- 172.21.15.152:0/2952377109 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679400c940 con 0x7f67a40af0d0 2022-07-21T01:48:17.892 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67aae4d700 1 -- 172.21.15.152:0/2952377109 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c009530 con 0x7f67a40af0d0 2022-07-21T01:48:17.892 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f6792ffd700 1 -- 172.21.15.152:0/2952377109 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679400cca0 con 0x7f67a40af0d0 2022-07-21T01:48:17.892 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67aae4d700 1 -- 172.21.15.152:0/2952377109 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a4142aa0 con 0x7f67a40af0d0 2022-07-21T01:48:17.892 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f6792ffd700 1 -- 172.21.15.152:0/2952377109 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f679401c460 con 0x7f67a40af0d0 2022-07-21T01:48:17.893 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f6792ffd700 1 -- 172.21.15.152:0/2952377109 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7f679401c6f0 con 0x7f67a40af0d0 2022-07-21T01:48:17.893 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.884+0000 7f67aae4d700 1 -- 172.21.15.152:0/2952377109 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a409e0f0 con 0x7f67a40af0d0 2022-07-21T01:48:17.893 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.888+0000 7f6792ffd700 1 -- 172.21.15.152:0/2952377109 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f679401cba0 con 0x7f67a40af0d0 2022-07-21T01:48:17.894 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.896+0000 7f6792ffd700 1 -- 172.21.15.152:0/2952377109 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mgrmap(e 2) v1 ==== 32894+0+0 (secure 0 0 0) 0x7f679401cd40 con 0x7f67a40af0d0 2022-07-21T01:48:17.944 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.952+0000 7f67aae4d700 1 -- 172.21.15.152:0/2952377109 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd new", "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", "id": 6} v 0) v1 -- 0x7f67a40a8f40 con 0x7f67a40af0d0 2022-07-21T01:48:17.953 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.960+0000 7f6792ffd700 1 -- 172.21.15.152:0/2952377109 <== mon.2 v2:172.21.15.152:3301/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f67940253e0 con 0x7f67a40af0d0 2022-07-21T01:48:17.953 INFO:teuthology.orchestra.run.smithi152.stdout:6 2022-07-21T01:48:17.955 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.960+0000 7f67aae4d700 1 -- 172.21.15.152:0/2952377109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40af0d0 msgr2=0x7f67a40a83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:17.955 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.960+0000 7f67aae4d700 1 --2- 172.21.15.152:0/2952377109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40af0d0 0x7f67a40a83c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f679400c000 tx=0x7f6794006df0).stop 2022-07-21T01:48:17.955 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.960+0000 7f67aae4d700 1 -- 172.21.15.152:0/2952377109 shutdown_connections 2022-07-21T01:48:17.956 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.960+0000 7f67aae4d700 1 --2- 172.21.15.152:0/2952377109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67a40b4c20 0x7f67a40a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.956 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.960+0000 7f67aae4d700 1 --2- 172.21.15.152:0/2952377109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67a40af0d0 0x7f67a40a83c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.956 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.960+0000 7f67aae4d700 1 --2- 172.21.15.152:0/2952377109 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67a40a9690 0x7f67a40aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:17.956 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.960+0000 7f67aae4d700 1 -- 172.21.15.152:0/2952377109 >> 172.21.15.152:0/2952377109 conn(0x7f67a4063150 msgr2=0x7f67a40b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:17.957 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.960+0000 7f67aae4d700 1 -- 172.21.15.152:0/2952377109 shutdown_connections 2022-07-21T01:48:17.957 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:17.960+0000 7f67aae4d700 1 -- 172.21.15.152:0/2952377109 wait complete. 2022-07-21T01:48:17.966 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 4efb2119-4ea0-4c46-95e2-662416bc7990 7 2022-07-21T01:48:18.061 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 Processor -- start 2022-07-21T01:48:18.062 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 -- start start 2022-07-21T01:48:18.078 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f430c0b6800 0x7f430c0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:18.079 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f430c0acf80 0x7f430c145f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:18.079 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f430c1428a0 con 0x7f430c0aed00 2022-07-21T01:48:18.079 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430c0b4120 con 0x7f430c0acf80 2022-07-21T01:48:18.080 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430c0b6be0 con 0x7f430c0b6800 2022-07-21T01:48:18.082 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430b7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f430c0b6800 0x7f430c0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:18.083 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430bfff700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f430c0aed00 0x7f430c05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:37790/0 (socket says 172.21.15.152:37790) 2022-07-21T01:48:18.083 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430bfff700 1 -- 172.21.15.152:0/1819522582 learned_addr learned my addr 172.21.15.152:0/1819522582 (peer_addr_for_me v1:172.21.15.152:0/0) 2022-07-21T01:48:18.084 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430affd700 1 --2- 172.21.15.152:0/1819522582 >> v2:172.21.15.140:3300/0 conn(0x7f430c0acf80 0x7f430c145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:18.084 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430b7fe700 1 -- 172.21.15.152:0/1819522582 >> v2:172.21.15.140:3300/0 conn(0x7f430c0acf80 msgr2=0x7f430c145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:18.084 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430b7fe700 1 --2- 172.21.15.152:0/1819522582 >> v2:172.21.15.140:3300/0 conn(0x7f430c0acf80 0x7f430c145f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:18.085 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430b7fe700 1 -- 172.21.15.152:0/1819522582 >> v1:172.21.15.152:6789/0 conn(0x7f430c0aed00 legacy=0x7f430c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:18.085 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430a7fc700 1 -- 172.21.15.152:0/1819522582 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940839161 0 0) 0x7f42f4001110 con 0x7f430c0aed00 2022-07-21T01:48:18.086 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430b7fe700 1 -- 172.21.15.152:0/1819522582 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c0b4800 con 0x7f430c0b6800 2022-07-21T01:48:18.086 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430affd700 1 --2- 172.21.15.152:0/1819522582 >> v2:172.21.15.140:3300/0 conn(0x7f430c0acf80 0x7f430c145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:18.087 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430b7fe700 1 --2- 172.21.15.152:0/1819522582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430c0b6800 0x7f430c0b51e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f430c066600 tx=0x7f4304006df0).ready entity=mon.1 client_cookie=be291d8f02058895 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:18.087 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430a7fc700 1 -- 172.21.15.152:0/1819522582 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f430400d5c0 con 0x7f430c0b6800 2022-07-21T01:48:18.088 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430a7fc700 1 -- 172.21.15.152:0/1819522582 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4304000e90 con 0x7f430c0b6800 2022-07-21T01:48:18.088 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f430a7fc700 1 -- 172.21.15.152:0/1819522582 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f430400d910 con 0x7f430c0b6800 2022-07-21T01:48:18.088 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 -- 172.21.15.152:0/1819522582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430c0b6800 msgr2=0x7f430c0b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:18.089 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 --2- 172.21.15.152:0/1819522582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430c0b6800 0x7f430c0b51e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f430c066600 tx=0x7f4304006df0).stop 2022-07-21T01:48:18.089 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 -- 172.21.15.152:0/1819522582 shutdown_connections 2022-07-21T01:48:18.090 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 --2- 172.21.15.152:0/1819522582 >> v2:172.21.15.140:3300/0 conn(0x7f430c0acf80 0x7f430c145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:18.090 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 --2- 172.21.15.152:0/1819522582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430c0b6800 0x7f430c0b51e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:18.091 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.068+0000 7f4310bcd700 1 -- 172.21.15.152:0/1819522582 >> 172.21.15.152:0/1819522582 conn(0x7f430c0b9de0 msgr2=0x7f430c0b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:18.091 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f4310bcd700 1 -- 172.21.15.152:0/1819522582 shutdown_connections 2022-07-21T01:48:18.092 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f4310bcd700 1 -- 172.21.15.152:0/1819522582 wait complete. 2022-07-21T01:48:18.092 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f4310bcd700 1 Processor -- start 2022-07-21T01:48:18.093 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f4310bcd700 1 -- start start 2022-07-21T01:48:18.093 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f4310bcd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f430c0acf80 0x7f430c1489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:18.093 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f4310bcd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430c0aed00 0x7f430c113ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:18.094 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f4310bcd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f430c0b6800 0x7f430c1551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:18.094 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f4310bcd700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c151c20 con 0x7f430c0acf80 2022-07-21T01:48:18.095 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f4310bcd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c0b1a20 con 0x7f430c0aed00 2022-07-21T01:48:18.095 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f4310bcd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430c1556d0 con 0x7f430c0b6800 2022-07-21T01:48:18.095 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430c0aed00 0x7f430c113ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:18.096 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430c0aed00 0x7f430c113ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51840/0 (socket says 172.21.15.152:51840) 2022-07-21T01:48:18.096 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430affd700 1 -- 172.21.15.152:0/1125306154 learned_addr learned my addr 172.21.15.152:0/1125306154 (peer_addr_for_me v2:172.21.15.152:0/0) 2022-07-21T01:48:18.097 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430bfff700 1 --2- 172.21.15.152:0/1125306154 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f430c0b6800 0x7f430c1551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:18.097 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430b7fe700 1 --2- 172.21.15.152:0/1125306154 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f430c0acf80 0x7f430c1489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:18.097 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430affd700 1 -- 172.21.15.152:0/1125306154 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f430c0b6800 msgr2=0x7f430c1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:18.098 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430affd700 1 --2- 172.21.15.152:0/1125306154 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f430c0b6800 0x7f430c1551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:18.098 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430affd700 1 -- 172.21.15.152:0/1125306154 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f430c0acf80 msgr2=0x7f430c1489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:18.098 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430affd700 1 --2- 172.21.15.152:0/1125306154 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f430c0acf80 0x7f430c1489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:18.099 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430affd700 1 -- 172.21.15.152:0/1125306154 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430400b180 con 0x7f430c0aed00 2022-07-21T01:48:18.099 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430b7fe700 1 --2- 172.21.15.152:0/1125306154 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f430c0acf80 0x7f430c1489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:18.099 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.072+0000 7f430affd700 1 --2- 172.21.15.152:0/1125306154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430c0aed00 0x7f430c113ce0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f42fc006cc0 tx=0x7f42fc008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:18.099 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.076+0000 7f4308ff9700 1 -- 172.21.15.152:0/1125306154 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42fc00c7d0 con 0x7f430c0aed00 2022-07-21T01:48:18.100 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.076+0000 7f4308ff9700 1 -- 172.21.15.152:0/1125306154 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42fc00c930 con 0x7f430c0aed00 2022-07-21T01:48:18.100 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.076+0000 7f4310bcd700 1 -- 172.21.15.152:0/1125306154 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430c0b13f0 con 0x7f430c0aed00 2022-07-21T01:48:18.100 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.076+0000 7f4308ff9700 1 -- 172.21.15.152:0/1125306154 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42fc00cc90 con 0x7f430c0aed00 2022-07-21T01:48:18.100 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.076+0000 7f4310bcd700 1 -- 172.21.15.152:0/1125306154 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430c0b0f50 con 0x7f430c0aed00 2022-07-21T01:48:18.101 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.076+0000 7f4308ff9700 1 -- 172.21.15.152:0/1125306154 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 2) v1 ==== 32894+0+0 (secure 0 0 0) 0x7f42fc01d460 con 0x7f430c0aed00 2022-07-21T01:48:18.101 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.076+0000 7f4310bcd700 1 -- 172.21.15.152:0/1125306154 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430c0b1200 con 0x7f430c0aed00 2022-07-21T01:48:18.101 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.076+0000 7f4308ff9700 1 -- 172.21.15.152:0/1125306154 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1935+0+0 (secure 0 0 0) 0x7f42fc03a930 con 0x7f430c0aed00 2022-07-21T01:48:18.101 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.080+0000 7f4308ff9700 1 -- 172.21.15.152:0/1125306154 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f42fc01d6f0 con 0x7f430c0aed00 2022-07-21T01:48:18.143 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.148+0000 7f4310bcd700 1 -- 172.21.15.152:0/1125306154 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", "id": 7} v 0) v1 -- 0x7f430c0b4aa0 con 0x7f430c0aed00 2022-07-21T01:48:18.152 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.160+0000 7f4308ff9700 1 -- 172.21.15.152:0/1125306154 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f42fc01a310 con 0x7f430c0aed00 2022-07-21T01:48:18.153 INFO:teuthology.orchestra.run.smithi152.stdout:7 2022-07-21T01:48:18.154 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.160+0000 7f4310bcd700 1 -- 172.21.15.152:0/1125306154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430c0aed00 msgr2=0x7f430c113ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:18.155 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.160+0000 7f4310bcd700 1 --2- 172.21.15.152:0/1125306154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430c0aed00 0x7f430c113ce0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f42fc006cc0 tx=0x7f42fc008040).stop 2022-07-21T01:48:18.155 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.160+0000 7f4310bcd700 1 -- 172.21.15.152:0/1125306154 shutdown_connections 2022-07-21T01:48:18.155 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.160+0000 7f4310bcd700 1 --2- 172.21.15.152:0/1125306154 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f430c0b6800 0x7f430c1551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:18.155 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.160+0000 7f4310bcd700 1 --2- 172.21.15.152:0/1125306154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f430c0aed00 0x7f430c113ce0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:18.155 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.160+0000 7f4310bcd700 1 --2- 172.21.15.152:0/1125306154 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f430c0acf80 0x7f430c1489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:18.156 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.160+0000 7f4310bcd700 1 -- 172.21.15.152:0/1125306154 >> 172.21.15.152:0/1125306154 conn(0x7f430c0b9de0 msgr2=0x7f430c141820 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:18.156 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.160+0000 7f4310bcd700 1 -- 172.21.15.152:0/1125306154 shutdown_connections 2022-07-21T01:48:18.156 INFO:teuthology.orchestra.run.smithi152.stderr:2022-07-21T01:48:18.160+0000 7f4310bcd700 1 -- 172.21.15.152:0/1125306154 wait complete. 2022-07-21T01:48:18.164 INFO:tasks.ceph.osd.0:Restarting daemon 2022-07-21T01:48:18.165 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-07-21T01:48:18.169 INFO:tasks.ceph.osd.0:Started 2022-07-21T01:48:18.170 INFO:tasks.ceph.osd.1:Restarting daemon 2022-07-21T01:48:18.170 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-07-21T01:48:18.175 INFO:tasks.ceph.osd.1:Started 2022-07-21T01:48:18.176 INFO:tasks.ceph.osd.2:Restarting daemon 2022-07-21T01:48:18.176 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-07-21T01:48:18.179 INFO:tasks.ceph.osd.2:Started 2022-07-21T01:48:18.179 INFO:tasks.ceph.osd.3:Restarting daemon 2022-07-21T01:48:18.180 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-07-21T01:48:18.182 INFO:tasks.ceph.osd.3:Started 2022-07-21T01:48:18.183 INFO:tasks.ceph.osd.4:Restarting daemon 2022-07-21T01:48:18.183 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-07-21T01:48:18.185 INFO:tasks.ceph.osd.4:Started 2022-07-21T01:48:18.186 INFO:tasks.ceph.osd.5:Restarting daemon 2022-07-21T01:48:18.186 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-07-21T01:48:18.189 INFO:tasks.ceph.osd.5:Started 2022-07-21T01:48:18.190 INFO:tasks.ceph.osd.6:Restarting daemon 2022-07-21T01:48:18.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-07-21T01:48:18.193 INFO:tasks.ceph.osd.6:Started 2022-07-21T01:48:18.194 INFO:tasks.ceph.osd.7:Restarting daemon 2022-07-21T01:48:18.195 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-07-21T01:48:18.198 INFO:tasks.ceph.osd.7:Started 2022-07-21T01:48:18.199 INFO:tasks.ceph:Waiting for OSDs to come up 2022-07-21T01:48:18.234 INFO:tasks.ceph.osd.0.smithi140.stderr:2022-07-21T01:48:18.235+0000 7f3f70f30d80 -1 Falling back to public interface 2022-07-21T01:48:18.249 INFO:tasks.ceph.osd.2.smithi140.stderr:2022-07-21T01:48:18.251+0000 7fc4d1b88d80 -1 Falling back to public interface 2022-07-21T01:48:18.257 INFO:tasks.ceph.osd.0.smithi140.stderr:2022-07-21T01:48:18.255+0000 7f3f70f30d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:18.257 INFO:tasks.ceph.osd.0.smithi140.stderr:2022-07-21T01:48:18.255+0000 7f3f70f30d80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.258 INFO:tasks.ceph.osd.0.smithi140.stderr:2022-07-21T01:48:18.255+0000 7f3f70f30d80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.259 INFO:tasks.ceph.osd.3.smithi140.stderr:2022-07-21T01:48:18.255+0000 7f3dfff58d80 -1 Falling back to public interface 2022-07-21T01:48:18.261 INFO:tasks.ceph.osd.4.smithi152.stderr:2022-07-21T01:48:18.268+0000 7fe800db8d80 -1 Falling back to public interface 2022-07-21T01:48:18.264 INFO:tasks.ceph.osd.0.smithi140.stderr:2022-07-21T01:48:18.263+0000 7f3f70f30d80 -1 osd.0 0 log_to_monitors {default=true} 2022-07-21T01:48:18.270 INFO:tasks.ceph.osd.6.smithi152.stderr:2022-07-21T01:48:18.276+0000 7f1740eecd80 -1 Falling back to public interface 2022-07-21T01:48:18.271 INFO:tasks.ceph.osd.5.smithi152.stderr:2022-07-21T01:48:18.276+0000 7f837e4aed80 -1 Falling back to public interface 2022-07-21T01:48:18.273 INFO:tasks.ceph.osd.3.smithi140.stderr:2022-07-21T01:48:18.271+0000 7f3dfff58d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:18.273 INFO:tasks.ceph.osd.3.smithi140.stderr:2022-07-21T01:48:18.271+0000 7f3dfff58d80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.274 INFO:tasks.ceph.osd.3.smithi140.stderr:2022-07-21T01:48:18.271+0000 7f3dfff58d80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.279 INFO:tasks.ceph.osd.4.smithi152.stderr:2022-07-21T01:48:18.280+0000 7fe800db8d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:18.279 INFO:tasks.ceph.osd.4.smithi152.stderr:2022-07-21T01:48:18.280+0000 7fe800db8d80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.279 INFO:tasks.ceph.osd.4.smithi152.stderr:2022-07-21T01:48:18.280+0000 7fe800db8d80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.280 INFO:tasks.ceph.osd.3.smithi140.stderr:2022-07-21T01:48:18.279+0000 7f3dfff58d80 -1 osd.3 0 log_to_monitors {default=true} 2022-07-21T01:48:18.282 INFO:tasks.ceph.osd.4.smithi152.stderr:2022-07-21T01:48:18.288+0000 7fe800db8d80 -1 osd.4 0 log_to_monitors {default=true} 2022-07-21T01:48:18.283 INFO:tasks.ceph.osd.2.smithi140.stderr:2022-07-21T01:48:18.279+0000 7fc4d1b88d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:18.283 INFO:tasks.ceph.osd.2.smithi140.stderr:2022-07-21T01:48:18.279+0000 7fc4d1b88d80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.283 INFO:tasks.ceph.osd.2.smithi140.stderr:2022-07-21T01:48:18.279+0000 7fc4d1b88d80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.284 INFO:tasks.ceph.osd.2.smithi140.stderr:2022-07-21T01:48:18.283+0000 7fc4d1b88d80 -1 osd.2 0 log_to_monitors {default=true} 2022-07-21T01:48:18.286 INFO:tasks.ceph.osd.6.smithi152.stderr:2022-07-21T01:48:18.292+0000 7f1740eecd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:18.286 INFO:tasks.ceph.osd.6.smithi152.stderr:2022-07-21T01:48:18.292+0000 7f1740eecd80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.287 INFO:tasks.ceph.osd.6.smithi152.stderr:2022-07-21T01:48:18.292+0000 7f1740eecd80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.287 INFO:tasks.ceph.osd.5.smithi152.stderr:2022-07-21T01:48:18.292+0000 7f837e4aed80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:18.288 INFO:tasks.ceph.osd.5.smithi152.stderr:2022-07-21T01:48:18.292+0000 7f837e4aed80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.288 INFO:tasks.ceph.osd.5.smithi152.stderr:2022-07-21T01:48:18.292+0000 7f837e4aed80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.290 INFO:tasks.ceph.osd.5.smithi152.stderr:2022-07-21T01:48:18.296+0000 7f837e4aed80 -1 osd.5 0 log_to_monitors {default=true} 2022-07-21T01:48:18.291 INFO:tasks.ceph.osd.6.smithi152.stderr:2022-07-21T01:48:18.296+0000 7f1740eecd80 -1 osd.6 0 log_to_monitors {default=true} 2022-07-21T01:48:18.296 INFO:tasks.ceph.osd.1.smithi140.stderr:2022-07-21T01:48:18.295+0000 7fdc4d3edd80 -1 Falling back to public interface 2022-07-21T01:48:18.297 INFO:tasks.ceph.osd.7.smithi152.stderr:2022-07-21T01:48:18.300+0000 7f97100a4d80 -1 Falling back to public interface 2022-07-21T01:48:18.314 INFO:tasks.ceph.osd.7.smithi152.stderr:2022-07-21T01:48:18.320+0000 7f97100a4d80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:18.314 INFO:tasks.ceph.osd.7.smithi152.stderr:2022-07-21T01:48:18.320+0000 7f97100a4d80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.314 INFO:tasks.ceph.osd.7.smithi152.stderr:2022-07-21T01:48:18.320+0000 7f97100a4d80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.316 INFO:tasks.ceph.osd.1.smithi140.stderr:2022-07-21T01:48:18.315+0000 7fdc4d3edd80 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2022-07-21T01:48:18.316 INFO:tasks.ceph.osd.1.smithi140.stderr:2022-07-21T01:48:18.315+0000 7fdc4d3edd80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.317 INFO:tasks.ceph.osd.1.smithi140.stderr:2022-07-21T01:48:18.315+0000 7fdc4d3edd80 -1 journal do_read_entry(8192): bad header magic 2022-07-21T01:48:18.318 INFO:tasks.ceph.osd.7.smithi152.stderr:2022-07-21T01:48:18.324+0000 7f97100a4d80 -1 osd.7 0 log_to_monitors {default=true} 2022-07-21T01:48:18.320 INFO:tasks.ceph.osd.1.smithi140.stderr:2022-07-21T01:48:18.319+0000 7fdc4d3edd80 -1 osd.1 0 log_to_monitors {default=true} 2022-07-21T01:48:19.006 DEBUG:teuthology.orchestra.run.smithi140:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:19.108 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 Processor -- start 2022-07-21T01:48:19.109 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 -- start start 2022-07-21T01:48:19.109 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f728c0b2390 0x7f728c0b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:19.110 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f728c0aacf0 0x7f728c0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:19.110 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728c0b2770 con 0x7f728c147ae0 2022-07-21T01:48:19.111 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c147100 con 0x7f728c0aacf0 2022-07-21T01:48:19.111 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c1473a0 con 0x7f728c0b2390 2022-07-21T01:48:19.111 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728ad9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f728c0aacf0 0x7f728c0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:19.111 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728ad9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f728c0aacf0 0x7f728c0b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60090/0 (socket says 172.21.15.140:60090) 2022-07-21T01:48:19.112 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728ad9d700 1 -- 172.21.15.140:0/3303767468 learned_addr learned my addr 172.21.15.140:0/3303767468 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:19.112 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728a59c700 1 --2- 172.21.15.140:0/3303767468 >> v2:172.21.15.152:3300/0 conn(0x7f728c0b2390 0x7f728c0b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:19.112 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728ad9d700 1 -- 172.21.15.140:0/3303767468 >> v2:172.21.15.152:3300/0 conn(0x7f728c0b2390 msgr2=0x7f728c0b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:19.112 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728ad9d700 1 --2- 172.21.15.140:0/3303767468 >> v2:172.21.15.152:3300/0 conn(0x7f728c0b2390 0x7f728c0b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:19.113 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728ad9d700 1 -- 172.21.15.140:0/3303767468 >> v1:172.21.15.152:6789/0 conn(0x7f728c147ae0 legacy=0x7f728c05bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:19.113 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728ad9d700 1 -- 172.21.15.140:0/3303767468 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c147640 con 0x7f728c0aacf0 2022-07-21T01:48:19.113 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728ad9d700 1 --2- 172.21.15.140:0/3303767468 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728c0aacf0 0x7f728c0b1010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f727c002f70 tx=0x7f727c006eb0).ready entity=mon.0 client_cookie=c393ca60294af8da server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:19.113 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728959a700 1 -- 172.21.15.140:0/3303767468 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f727c00d5c0 con 0x7f728c0aacf0 2022-07-21T01:48:19.113 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728959a700 1 -- 172.21.15.140:0/3303767468 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c000e90 con 0x7f728c0aacf0 2022-07-21T01:48:19.114 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728959a700 1 -- 172.21.15.140:0/3303767468 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f727c00d950 con 0x7f728c0aacf0 2022-07-21T01:48:19.114 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 -- 172.21.15.140:0/3303767468 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728c0aacf0 msgr2=0x7f728c0b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:19.114 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 --2- 172.21.15.140:0/3303767468 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728c0aacf0 0x7f728c0b1010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f727c002f70 tx=0x7f727c006eb0).stop 2022-07-21T01:48:19.114 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 -- 172.21.15.140:0/3303767468 shutdown_connections 2022-07-21T01:48:19.115 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 --2- 172.21.15.140:0/3303767468 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728c0aacf0 0x7f728c0b1010 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:19.115 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 --2- 172.21.15.140:0/3303767468 >> v2:172.21.15.152:3300/0 conn(0x7f728c0b2390 0x7f728c0b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:19.115 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.103+0000 7f728b59e700 1 -- 172.21.15.140:0/3303767468 >> 172.21.15.140:0/3303767468 conn(0x7f728c0b5920 msgr2=0x7f728c0b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:19.115 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 -- 172.21.15.140:0/3303767468 shutdown_connections 2022-07-21T01:48:19.116 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 -- 172.21.15.140:0/3303767468 wait complete. 2022-07-21T01:48:19.116 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 Processor -- start 2022-07-21T01:48:19.116 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 -- start start 2022-07-21T01:48:19.116 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f728c0aacf0 0x7f728c145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:19.116 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f728c0b2390 0x7f728c149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:19.117 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728c147ae0 0x7f728c1482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:19.117 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0aed90 con 0x7f728c147ae0 2022-07-21T01:48:19.117 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c15a840 con 0x7f728c0aacf0 2022-07-21T01:48:19.117 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c0ae8f0 con 0x7f728c0b2390 2022-07-21T01:48:19.117 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728c147ae0 0x7f728c1482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:19.118 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f7289d9b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f728c0b2390 0x7f728c149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:19.118 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728a59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f728c0aacf0 0x7f728c145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:19.118 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728c147ae0 0x7f728c1482c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60096/0 (socket says 172.21.15.140:60096) 2022-07-21T01:48:19.118 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728ad9d700 1 -- 172.21.15.140:0/1651160723 learned_addr learned my addr 172.21.15.140:0/1651160723 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:19.119 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728ad9d700 1 -- 172.21.15.140:0/1651160723 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f728c0b2390 msgr2=0x7f728c149380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:19.119 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728ad9d700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f728c0b2390 0x7f728c149380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:19.119 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728ad9d700 1 -- 172.21.15.140:0/1651160723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f728c0aacf0 msgr2=0x7f728c145ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:19.119 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728ad9d700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f728c0aacf0 0x7f728c145ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:19.120 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728ad9d700 1 -- 172.21.15.140:0/1651160723 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c00b1e0 con 0x7f728c147ae0 2022-07-21T01:48:19.120 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728ad9d700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728c147ae0 0x7f728c1482c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f727c002f70 tx=0x7f727c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:19.120 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f727b7fe700 1 -- 172.21.15.140:0/1651160723 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f727c011760 con 0x7f728c147ae0 2022-07-21T01:48:19.120 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f727b7fe700 1 -- 172.21.15.140:0/1651160723 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c0118c0 con 0x7f728c147ae0 2022-07-21T01:48:19.120 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 -- 172.21.15.140:0/1651160723 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c15a4b0 con 0x7f728c147ae0 2022-07-21T01:48:19.121 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f727b7fe700 1 -- 172.21.15.140:0/1651160723 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f727c011c20 con 0x7f728c147ae0 2022-07-21T01:48:19.121 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.107+0000 7f728b59e700 1 -- 172.21.15.140:0/1651160723 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c149fc0 con 0x7f728c147ae0 2022-07-21T01:48:19.122 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.111+0000 7f727b7fe700 1 -- 172.21.15.140:0/1651160723 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 3) v1 ==== 65884+0+0 (secure 0 0 0) 0x7f727c025030 con 0x7f728c147ae0 2022-07-21T01:48:19.122 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.111+0000 7f728b59e700 1 -- 172.21.15.140:0/1651160723 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728c141820 con 0x7f728c147ae0 2022-07-21T01:48:19.123 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.111+0000 7f727b7fe700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72680539a0 0x7f7268055e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:19.123 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.111+0000 7f728a59c700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72680539a0 0x7f7268055e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:19.123 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.111+0000 7f727b7fe700 1 -- 172.21.15.140:0/1651160723 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2074+0+0 (secure 0 0 0) 0x7f727c06c050 con 0x7f728c147ae0 2022-07-21T01:48:19.123 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.111+0000 7f728a59c700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72680539a0 0x7f7268055e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f728000b000 tx=0x7f7280006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:19.124 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.115+0000 7f727b7fe700 1 -- 172.21.15.140:0/1651160723 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f728c141820 con 0x7f728c147ae0 2022-07-21T01:48:19.241 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.239+0000 7f728b59e700 1 -- 172.21.15.140:0/1651160723 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f728c141820 con 0x7f728c147ae0 2022-07-21T01:48:19.242 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f727b7fe700 1 -- 172.21.15.140:0/1651160723 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+8275 (secure 0 0 0) 0x7f728c141820 con 0x7f728c147ae0 2022-07-21T01:48:19.243 INFO:teuthology.misc.health.smithi140.stdout: 2022-07-21T01:48:19.244 INFO:teuthology.misc.health.smithi140.stdout:{"epoch":11,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:19.154402+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:19.246 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 -- 172.21.15.140:0/1651160723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72680539a0 msgr2=0x7f7268055e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:19.247 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72680539a0 0x7f7268055e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f728000b000 tx=0x7f7280006df0).stop 2022-07-21T01:48:19.247 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 -- 172.21.15.140:0/1651160723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728c147ae0 msgr2=0x7f728c1482c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:19.248 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728c147ae0 0x7f728c1482c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f727c002f70 tx=0x7f727c015000).stop 2022-07-21T01:48:19.248 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 -- 172.21.15.140:0/1651160723 shutdown_connections 2022-07-21T01:48:19.249 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72680539a0 0x7f7268055e60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:19.249 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728c147ae0 0x7f728c1482c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:19.250 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f728c0b2390 0x7f728c149380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:19.250 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 --2- 172.21.15.140:0/1651160723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f728c0aacf0 0x7f728c145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:19.251 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 -- 172.21.15.140:0/1651160723 >> 172.21.15.140:0/1651160723 conn(0x7f728c0b5920 msgr2=0x7f728c05c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:19.251 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 -- 172.21.15.140:0/1651160723 shutdown_connections 2022-07-21T01:48:19.252 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:19.243+0000 7f728b59e700 1 -- 172.21.15.140:0/1651160723 wait complete. 2022-07-21T01:48:19.256 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-07-21T01:48:20.167 INFO:tasks.ceph.osd.3.smithi140.stderr:2022-07-21T01:48:20.167+0000 7f3dea01e700 -1 osd.3 0 waiting for initial osdmap 2022-07-21T01:48:20.168 INFO:tasks.ceph.osd.2.smithi140.stderr:2022-07-21T01:48:20.167+0000 7fc4bbc4e700 -1 osd.2 0 waiting for initial osdmap 2022-07-21T01:48:20.172 INFO:tasks.ceph.osd.0.smithi140.stderr:2022-07-21T01:48:20.171+0000 7f3f5aff6700 -1 osd.0 0 waiting for initial osdmap 2022-07-21T01:48:20.174 INFO:tasks.ceph.osd.4.smithi152.stderr:2022-07-21T01:48:20.180+0000 7fe7eae7e700 -1 osd.4 0 waiting for initial osdmap 2022-07-21T01:48:20.174 INFO:tasks.ceph.osd.6.smithi152.stderr:2022-07-21T01:48:20.180+0000 7f172afb2700 -1 osd.6 0 waiting for initial osdmap 2022-07-21T01:48:20.175 INFO:tasks.ceph.osd.5.smithi152.stderr:2022-07-21T01:48:20.180+0000 7f8368574700 -1 osd.5 0 waiting for initial osdmap 2022-07-21T01:48:20.175 INFO:tasks.ceph.osd.7.smithi152.stderr:2022-07-21T01:48:20.180+0000 7f96fa16a700 -1 osd.7 0 waiting for initial osdmap 2022-07-21T01:48:20.177 INFO:tasks.ceph.osd.2.smithi140.stderr:2022-07-21T01:48:20.175+0000 7fc4cb0ce700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-07-21T01:48:20.178 INFO:tasks.ceph.osd.4.smithi152.stderr:2022-07-21T01:48:20.184+0000 7fe7fa2fe700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-07-21T01:48:20.180 INFO:tasks.ceph.osd.3.smithi140.stderr:2022-07-21T01:48:20.175+0000 7f3df949e700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-07-21T01:48:20.181 INFO:tasks.ceph.osd.1.smithi140.stderr:2022-07-21T01:48:20.179+0000 7fdc374b3700 -1 osd.1 0 waiting for initial osdmap 2022-07-21T01:48:20.181 INFO:tasks.ceph.osd.1.smithi140.stderr:2022-07-21T01:48:20.179+0000 7fdc46933700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-07-21T01:48:20.182 INFO:tasks.ceph.osd.5.smithi152.stderr:2022-07-21T01:48:20.184+0000 7f83779f4700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-07-21T01:48:20.183 INFO:tasks.ceph.osd.7.smithi152.stderr:2022-07-21T01:48:20.184+0000 7f97095ea700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-07-21T01:48:20.184 INFO:tasks.ceph.osd.6.smithi152.stderr:2022-07-21T01:48:20.184+0000 7f173a432700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-07-21T01:48:20.201 INFO:tasks.ceph.osd.0.smithi140.stderr:2022-07-21T01:48:20.203+0000 7f3f6a476700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-07-21T01:48:26.062 DEBUG:teuthology.orchestra.run.smithi140:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:26.172 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa03cfb6700 1 Processor -- start 2022-07-21T01:48:26.173 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa03cfb6700 1 -- start start 2022-07-21T01:48:26.173 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa03cfb6700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa0380af680 0x7fa03805ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:26.174 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa03cfb6700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0380b2970 con 0x7fa0380ad730 2022-07-21T01:48:26.174 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa03cfb6700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0380afb00 con 0x7fa038139780 2022-07-21T01:48:26.174 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa03cfb6700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0380afd30 con 0x7fa0380af680 2022-07-21T01:48:26.175 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa035d9b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa0380ad730 0x7fa03805f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46214/0 (socket says 172.21.15.140:46214) 2022-07-21T01:48:26.175 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa035d9b700 1 -- 172.21.15.140:0/1191902820 learned_addr learned my addr 172.21.15.140:0/1191902820 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:26.176 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa036d9d700 1 --2- 172.21.15.140:0/1191902820 >> v2:172.21.15.152:3300/0 conn(0x7fa0380af680 0x7fa03805ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:26.176 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa03559a700 1 -- 172.21.15.140:0/1191902820 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644260316 0 0) 0x7fa0380b2970 con 0x7fa0380ad730 2022-07-21T01:48:26.176 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa03559a700 1 -- 172.21.15.140:0/1191902820 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa018003260 con 0x7fa0380ad730 2022-07-21T01:48:26.176 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa03559a700 1 -- 172.21.15.140:0/1191902820 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371156497 0 0) 0x7fa0380afb00 con 0x7fa038139780 2022-07-21T01:48:26.176 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa03559a700 1 -- 172.21.15.140:0/1191902820 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0380b2970 con 0x7fa038139780 2022-07-21T01:48:26.177 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa036d9d700 1 -- 172.21.15.140:0/1191902820 >> v1:172.21.15.152:6789/0 conn(0x7fa038139780 legacy=0x7fa0380aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.177 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa03559a700 1 -- 172.21.15.140:0/1191902820 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3084187885 0 0) 0x7fa018003260 con 0x7fa0380ad730 2022-07-21T01:48:26.178 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa036d9d700 1 -- 172.21.15.140:0/1191902820 >> v1:172.21.15.140:6789/0 conn(0x7fa0380ad730 legacy=0x7fa03805f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.178 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa036d9d700 1 -- 172.21.15.140:0/1191902820 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa038140270 con 0x7fa0380af680 2022-07-21T01:48:26.179 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.163+0000 7fa036d9d700 1 --2- 172.21.15.140:0/1191902820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0380af680 0x7fa03805ff70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa030001110 tx=0x7fa028006df0).ready entity=mon.1 client_cookie=ecaca61442e802af server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:26.179 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03559a700 1 -- 172.21.15.140:0/1191902820 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa030001660 con 0x7fa0380af680 2022-07-21T01:48:26.180 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03559a700 1 -- 172.21.15.140:0/1191902820 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa028000e90 con 0x7fa0380af680 2022-07-21T01:48:26.180 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03559a700 1 -- 172.21.15.140:0/1191902820 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa028012460 con 0x7fa0380af680 2022-07-21T01:48:26.181 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03cfb6700 1 -- 172.21.15.140:0/1191902820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0380af680 msgr2=0x7fa03805ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.181 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03cfb6700 1 --2- 172.21.15.140:0/1191902820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0380af680 0x7fa03805ff70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fa030001110 tx=0x7fa028006df0).stop 2022-07-21T01:48:26.182 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03cfb6700 1 -- 172.21.15.140:0/1191902820 shutdown_connections 2022-07-21T01:48:26.182 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03cfb6700 1 --2- 172.21.15.140:0/1191902820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0380af680 0x7fa03805ff70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.183 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03cfb6700 1 -- 172.21.15.140:0/1191902820 >> 172.21.15.140:0/1191902820 conn(0x7fa0380b6150 msgr2=0x7fa0380b6530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:26.183 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03cfb6700 1 -- 172.21.15.140:0/1191902820 shutdown_connections 2022-07-21T01:48:26.184 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03cfb6700 1 -- 172.21.15.140:0/1191902820 wait complete. 2022-07-21T01:48:26.184 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03cfb6700 1 Processor -- start 2022-07-21T01:48:26.184 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.167+0000 7fa03cfb6700 1 -- start start 2022-07-21T01:48:26.185 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03cfb6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0380ad730 0x7fa0381445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:26.185 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03cfb6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0380af680 0x7fa038155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:26.185 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03cfb6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa038139780 0x7fa038066c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:26.185 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03cfb6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0381410c0 con 0x7fa0380ad730 2022-07-21T01:48:26.186 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03cfb6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa038142040 con 0x7fa0380af680 2022-07-21T01:48:26.186 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03cfb6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0381424e0 con 0x7fa038139780 2022-07-21T01:48:26.187 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0380ad730 0x7fa0381445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:26.187 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0380ad730 0x7fa0381445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60302/0 (socket says 172.21.15.140:60302) 2022-07-21T01:48:26.188 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03659c700 1 -- 172.21.15.140:0/2189596124 learned_addr learned my addr 172.21.15.140:0/2189596124 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:26.188 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa035d9b700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0380af680 0x7fa038155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:26.189 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa036d9d700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa038139780 0x7fa038066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:26.189 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03659c700 1 -- 172.21.15.140:0/2189596124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa038139780 msgr2=0x7fa038066c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.189 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03659c700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa038139780 0x7fa038066c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.190 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03659c700 1 -- 172.21.15.140:0/2189596124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0380af680 msgr2=0x7fa038155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.190 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03659c700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0380af680 0x7fa038155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.190 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03659c700 1 -- 172.21.15.140:0/2189596124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02800b180 con 0x7fa0380ad730 2022-07-21T01:48:26.191 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03659c700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0380ad730 0x7fa0381445c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fa020001490 tx=0x7fa020007ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:26.191 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa02f7fe700 1 -- 172.21.15.140:0/2189596124 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa020010510 con 0x7fa0380ad730 2022-07-21T01:48:26.192 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa02f7fe700 1 -- 172.21.15.140:0/2189596124 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa020010670 con 0x7fa0380ad730 2022-07-21T01:48:26.192 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03cfb6700 1 -- 172.21.15.140:0/2189596124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0381414c0 con 0x7fa0380ad730 2022-07-21T01:48:26.193 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa02f7fe700 1 -- 172.21.15.140:0/2189596124 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0200107e0 con 0x7fa0380ad730 2022-07-21T01:48:26.193 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03cfb6700 1 -- 172.21.15.140:0/2189596124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa038140e40 con 0x7fa0380ad730 2022-07-21T01:48:26.194 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa03cfb6700 1 -- 172.21.15.140:0/2189596124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa038142200 con 0x7fa0380ad730 2022-07-21T01:48:26.194 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.171+0000 7fa02f7fe700 1 -- 172.21.15.140:0/2189596124 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa020010070 con 0x7fa0380ad730 2022-07-21T01:48:26.195 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.175+0000 7fa02f7fe700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa01c053950 0x7fa01c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:26.196 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.175+0000 7fa02f7fe700 1 -- 172.21.15.140:0/2189596124 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7fa02006b740 con 0x7fa0380ad730 2022-07-21T01:48:26.196 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.175+0000 7fa035d9b700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa01c053950 0x7fa01c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:26.196 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.175+0000 7fa035d9b700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa01c053950 0x7fa01c055e10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa03000b000 tx=0x7fa0300014c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:26.197 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.179+0000 7fa02f7fe700 1 -- 172.21.15.140:0/2189596124 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa020042a90 con 0x7fa0380ad730 2022-07-21T01:48:26.302 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.303+0000 7fa03cfb6700 1 -- 172.21.15.140:0/2189596124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa038140510 con 0x7fa0380ad730 2022-07-21T01:48:26.304 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.303+0000 7fa02f7fe700 1 -- 172.21.15.140:0/2189596124 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11901 (secure 0 0 0) 0x7fa0200460b0 con 0x7fa0380ad730 2022-07-21T01:48:26.305 INFO:teuthology.misc.health.smithi140.stdout: 2022-07-21T01:48:26.305 INFO:teuthology.misc.health.smithi140.stdout:{"epoch":15,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:23.202228+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:26.308 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 -- 172.21.15.140:0/2189596124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa01c053950 msgr2=0x7fa01c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.308 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa01c053950 0x7fa01c055e10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa03000b000 tx=0x7fa0300014c0).stop 2022-07-21T01:48:26.308 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 -- 172.21.15.140:0/2189596124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0380ad730 msgr2=0x7fa0381445c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.309 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0380ad730 0x7fa0381445c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fa020001490 tx=0x7fa020007ee0).stop 2022-07-21T01:48:26.309 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 -- 172.21.15.140:0/2189596124 shutdown_connections 2022-07-21T01:48:26.309 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa01c053950 0x7fa01c055e10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.309 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa038139780 0x7fa038066c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.310 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0380af680 0x7fa038155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.310 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 --2- 172.21.15.140:0/2189596124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0380ad730 0x7fa0381445c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.310 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 -- 172.21.15.140:0/2189596124 >> 172.21.15.140:0/2189596124 conn(0x7fa0380b6150 msgr2=0x7fa0380b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:26.310 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 -- 172.21.15.140:0/2189596124 shutdown_connections 2022-07-21T01:48:26.311 INFO:teuthology.misc.health.smithi140.stderr:2022-07-21T01:48:26.307+0000 7fa03cfb6700 1 -- 172.21.15.140:0/2189596124 wait complete. 2022-07-21T01:48:26.321 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-07-21T01:48:26.322 INFO:tasks.ceph:Creating RBD pool 2022-07-21T01:48:26.322 DEBUG:teuthology.orchestra.run.smithi140:> sudo ceph --cluster ceph osd pool create rbd 8 2022-07-21T01:48:26.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efefaa76700 1 Processor -- start 2022-07-21T01:48:26.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efefaa76700 1 -- start start 2022-07-21T01:48:26.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efefaa76700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7efef40988b0 0x7efef41422c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:26.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efefaa76700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efef41427b0 0x7efef4060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:26.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efefaa76700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef40afdb0 con 0x7efef41462a0 2022-07-21T01:48:26.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efefaa76700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef40b0050 con 0x7efef40988b0 2022-07-21T01:48:26.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efefaa76700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef405fcd0 con 0x7efef41427b0 2022-07-21T01:48:26.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efef3fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7efef40988b0 0x7efef41422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:26.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efef3fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7efef40988b0 0x7efef41422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60312/0 (socket says 172.21.15.140:60312) 2022-07-21T01:48:26.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efef3fff700 1 -- 172.21.15.140:0/1169624546 learned_addr learned my addr 172.21.15.140:0/1169624546 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:26.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efef8812700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7efef41462a0 0x7efef4146680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46224/0 (socket says 172.21.15.140:46224) 2022-07-21T01:48:26.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efef9013700 1 --2- 172.21.15.140:0/1169624546 >> v2:172.21.15.152:3300/0 conn(0x7efef41427b0 0x7efef4060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:26.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efef3fff700 1 -- 172.21.15.140:0/1169624546 >> v2:172.21.15.152:3300/0 conn(0x7efef41427b0 msgr2=0x7efef4060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:26.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efef3fff700 1 --2- 172.21.15.140:0/1169624546 >> v2:172.21.15.152:3300/0 conn(0x7efef41427b0 0x7efef4060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efef3fff700 1 -- 172.21.15.140:0/1169624546 >> v1:172.21.15.140:6789/0 conn(0x7efef41462a0 legacy=0x7efef4146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efef37fe700 1 -- 172.21.15.140:0/1169624546 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936202297 0 0) 0x7efef40afdb0 con 0x7efef41462a0 2022-07-21T01:48:26.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efef3fff700 1 -- 172.21.15.140:0/1169624546 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef40afae0 con 0x7efef40988b0 2022-07-21T01:48:26.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.431+0000 7efef3fff700 1 --2- 172.21.15.140:0/1169624546 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef40988b0 0x7efef41422c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7efee4002f70 tx=0x7efee4006eb0).ready entity=mon.0 client_cookie=29dda078832eba93 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:26.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efef37fe700 1 -- 172.21.15.140:0/1169624546 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee400d610 con 0x7efef40988b0 2022-07-21T01:48:26.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efef37fe700 1 -- 172.21.15.140:0/1169624546 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee4000e90 con 0x7efef40988b0 2022-07-21T01:48:26.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efef37fe700 1 -- 172.21.15.140:0/1169624546 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee400d960 con 0x7efef40988b0 2022-07-21T01:48:26.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efefaa76700 1 -- 172.21.15.140:0/1169624546 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef40988b0 msgr2=0x7efef41422c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efefaa76700 1 --2- 172.21.15.140:0/1169624546 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef40988b0 0x7efef41422c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7efee4002f70 tx=0x7efee4006eb0).stop 2022-07-21T01:48:26.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efefaa76700 1 -- 172.21.15.140:0/1169624546 shutdown_connections 2022-07-21T01:48:26.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efefaa76700 1 --2- 172.21.15.140:0/1169624546 >> v2:172.21.15.152:3300/0 conn(0x7efef41427b0 0x7efef4060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efefaa76700 1 --2- 172.21.15.140:0/1169624546 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef40988b0 0x7efef41422c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efefaa76700 1 -- 172.21.15.140:0/1169624546 >> 172.21.15.140:0/1169624546 conn(0x7efef40b5f00 msgr2=0x7efef40b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:26.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efefaa76700 1 -- 172.21.15.140:0/1169624546 shutdown_connections 2022-07-21T01:48:26.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efefaa76700 1 -- 172.21.15.140:0/1169624546 wait complete. 2022-07-21T01:48:26.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efefaa76700 1 Processor -- start 2022-07-21T01:48:26.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.435+0000 7efefaa76700 1 -- start start 2022-07-21T01:48:26.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efefaa76700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef40988b0 0x7efef409c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:26.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efefaa76700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef41427b0 0x7efef414a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:26.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efefaa76700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef41462a0 0x7efef4155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:26.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efefaa76700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef41563c0 con 0x7efef40988b0 2022-07-21T01:48:26.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efefaa76700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef40b3470 con 0x7efef41462a0 2022-07-21T01:48:26.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efefaa76700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef40b3710 con 0x7efef41427b0 2022-07-21T01:48:26.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef8812700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef40988b0 0x7efef409c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:26.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef8812700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef40988b0 0x7efef409c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60318/0 (socket says 172.21.15.140:60318) 2022-07-21T01:48:26.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef8812700 1 -- 172.21.15.140:0/749632755 learned_addr learned my addr 172.21.15.140:0/749632755 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:26.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef3fff700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef41427b0 0x7efef414a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:26.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef9013700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef41462a0 0x7efef4155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:26.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef8812700 1 -- 172.21.15.140:0/749632755 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef41427b0 msgr2=0x7efef414a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef8812700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef41427b0 0x7efef414a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef8812700 1 -- 172.21.15.140:0/749632755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef41462a0 msgr2=0x7efef4155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef8812700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef41462a0 0x7efef4155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef8812700 1 -- 172.21.15.140:0/749632755 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee400b1e0 con 0x7efef40988b0 2022-07-21T01:48:26.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef8812700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef40988b0 0x7efef409c490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7efedc000ed0 tx=0x7efedc007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:26.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef1ffb700 1 -- 172.21.15.140:0/749632755 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efedc00ce50 con 0x7efef40988b0 2022-07-21T01:48:26.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef1ffb700 1 -- 172.21.15.140:0/749632755 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc0070d0 con 0x7efef40988b0 2022-07-21T01:48:26.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efef1ffb700 1 -- 172.21.15.140:0/749632755 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efedc007430 con 0x7efef40988b0 2022-07-21T01:48:26.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efefaa76700 1 -- 172.21.15.140:0/749632755 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef4151f30 con 0x7efef40988b0 2022-07-21T01:48:26.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.439+0000 7efefaa76700 1 -- 172.21.15.140:0/749632755 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef40b3130 con 0x7efef40988b0 2022-07-21T01:48:26.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.443+0000 7efefaa76700 1 -- 172.21.15.140:0/749632755 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef4146d90 con 0x7efef40988b0 2022-07-21T01:48:26.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.443+0000 7efef1ffb700 1 -- 172.21.15.140:0/749632755 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efedc010070 con 0x7efef40988b0 2022-07-21T01:48:26.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.443+0000 7efef1ffb700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed8053950 0x7efed8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:26.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.443+0000 7efef3fff700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed8053950 0x7efed8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:26.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.443+0000 7efef1ffb700 1 -- 172.21.15.140:0/749632755 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4900+0+0 (secure 0 0 0) 0x7efedc06b100 con 0x7efef40988b0 2022-07-21T01:48:26.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.443+0000 7efef3fff700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed8053950 0x7efed8055e10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7efee401c000 tx=0x7efee4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:26.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.451+0000 7efef1ffb700 1 -- 172.21.15.140:0/749632755 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efedc0423a0 con 0x7efef40988b0 2022-07-21T01:48:26.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.567+0000 7efefaa76700 1 -- 172.21.15.140:0/749632755 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7efef40574e0 con 0x7efef40988b0 2022-07-21T01:48:26.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.959+0000 7efef1ffb700 1 -- 172.21.15.140:0/749632755 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7efedc0459c0 con 0x7efef40988b0 2022-07-21T01:48:26.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 -- 172.21.15.140:0/749632755 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed8053950 msgr2=0x7efed8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed8053950 0x7efed8055e10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7efee401c000 tx=0x7efee4015000).stop 2022-07-21T01:48:26.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 -- 172.21.15.140:0/749632755 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef40988b0 msgr2=0x7efef409c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:26.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef40988b0 0x7efef409c490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7efedc000ed0 tx=0x7efedc007c80).stop 2022-07-21T01:48:26.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 -- 172.21.15.140:0/749632755 shutdown_connections 2022-07-21T01:48:26.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed8053950 0x7efed8055e10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef41462a0 0x7efef4155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef41427b0 0x7efef414a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 --2- 172.21.15.140:0/749632755 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef40988b0 0x7efef409c490 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:26.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 -- 172.21.15.140:0/749632755 >> 172.21.15.140:0/749632755 conn(0x7efef40b5f00 msgr2=0x7efef405bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:26.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 -- 172.21.15.140:0/749632755 shutdown_connections 2022-07-21T01:48:26.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:26.963+0000 7efefaa76700 1 -- 172.21.15.140:0/749632755 wait complete. 2022-07-21T01:48:26.967 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'rbd' created 2022-07-21T01:48:26.974 DEBUG:teuthology.orchestra.run.smithi140:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-07-21T01:48:27.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb465d5700 1 Processor -- start 2022-07-21T01:48:27.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb465d5700 1 -- start start 2022-07-21T01:48:27.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb465d5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcb400ad070 0x7fcb40147290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:27.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb465d5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcb4005bfa0 0x7fcb40144220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:27.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb465d5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb400b5f60 con 0x7fcb400b5b80 2022-07-21T01:48:27.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb465d5700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb400ac570 con 0x7fcb4005bfa0 2022-07-21T01:48:27.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb465d5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4005c5f0 con 0x7fcb400ad070 2022-07-21T01:48:27.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb3ffff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fcb400b5b80 0x7fcb400ac960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46238/0 (socket says 172.21.15.140:46238) 2022-07-21T01:48:27.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb3ffff700 1 -- 172.21.15.140:0/308854808 learned_addr learned my addr 172.21.15.140:0/308854808 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:27.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb44b72700 1 --2- 172.21.15.140:0/308854808 >> v2:172.21.15.140:3300/0 conn(0x7fcb4005bfa0 0x7fcb40144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:27.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb377fe700 1 --2- 172.21.15.140:0/308854808 >> v2:172.21.15.152:3300/0 conn(0x7fcb400ad070 0x7fcb40147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:27.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb3f7fe700 1 -- 172.21.15.140:0/308854808 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607358405 0 0) 0x7fcb400b5f60 con 0x7fcb400b5b80 2022-07-21T01:48:27.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb3f7fe700 1 -- 172.21.15.140:0/308854808 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb24003260 con 0x7fcb400b5b80 2022-07-21T01:48:27.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb44b72700 1 -- 172.21.15.140:0/308854808 >> v2:172.21.15.152:3300/0 conn(0x7fcb400ad070 msgr2=0x7fcb40147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:27.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb44b72700 1 --2- 172.21.15.140:0/308854808 >> v2:172.21.15.152:3300/0 conn(0x7fcb400ad070 0x7fcb40147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:27.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb44b72700 1 -- 172.21.15.140:0/308854808 >> v1:172.21.15.140:6789/0 conn(0x7fcb400b5b80 legacy=0x7fcb400ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:27.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb44b72700 1 -- 172.21.15.140:0/308854808 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4005c890 con 0x7fcb4005bfa0 2022-07-21T01:48:27.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb44b72700 1 --2- 172.21.15.140:0/308854808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb4005bfa0 0x7fcb40144220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fcb380032f0 tx=0x7fcb380091d0).ready entity=mon.0 client_cookie=1a1ef536646e21b3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:27.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb3f7fe700 1 -- 172.21.15.140:0/308854808 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb3800b600 con 0x7fcb4005bfa0 2022-07-21T01:48:27.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb3f7fe700 1 -- 172.21.15.140:0/308854808 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3800b760 con 0x7fcb4005bfa0 2022-07-21T01:48:27.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.091+0000 7fcb3f7fe700 1 -- 172.21.15.140:0/308854808 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb3800b8d0 con 0x7fcb4005bfa0 2022-07-21T01:48:27.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 -- 172.21.15.140:0/308854808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb4005bfa0 msgr2=0x7fcb40144220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:27.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 --2- 172.21.15.140:0/308854808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb4005bfa0 0x7fcb40144220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fcb380032f0 tx=0x7fcb380091d0).stop 2022-07-21T01:48:27.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 -- 172.21.15.140:0/308854808 shutdown_connections 2022-07-21T01:48:27.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 --2- 172.21.15.140:0/308854808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb4005bfa0 0x7fcb40144220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:27.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 --2- 172.21.15.140:0/308854808 >> v2:172.21.15.152:3300/0 conn(0x7fcb400ad070 0x7fcb40147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:27.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 -- 172.21.15.140:0/308854808 >> 172.21.15.140:0/308854808 conn(0x7fcb400635e0 msgr2=0x7fcb400b5130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:27.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 -- 172.21.15.140:0/308854808 shutdown_connections 2022-07-21T01:48:27.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 -- 172.21.15.140:0/308854808 wait complete. 2022-07-21T01:48:27.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 Processor -- start 2022-07-21T01:48:27.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 -- start start 2022-07-21T01:48:27.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb4005bfa0 0x7fcb400ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:27.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb400ad070 0x7fcb400a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:27.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb400b5b80 0x7fcb400a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:27.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4014a2f0 con 0x7fcb400ad070 2022-07-21T01:48:27.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400a0270 con 0x7fcb4005bfa0 2022-07-21T01:48:27.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.095+0000 7fcb465d5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4014a880 con 0x7fcb400b5b80 2022-07-21T01:48:27.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb377fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb400ad070 0x7fcb400a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:27.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb377fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb400ad070 0x7fcb400a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60332/0 (socket says 172.21.15.140:60332) 2022-07-21T01:48:27.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb377fe700 1 -- 172.21.15.140:0/2482039612 learned_addr learned my addr 172.21.15.140:0/2482039612 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:27.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb44b72700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb400b5b80 0x7fcb400a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:27.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb3ffff700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb4005bfa0 0x7fcb400ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:27.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb377fe700 1 -- 172.21.15.140:0/2482039612 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb400b5b80 msgr2=0x7fcb400a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:27.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb377fe700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb400b5b80 0x7fcb400a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:27.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb377fe700 1 -- 172.21.15.140:0/2482039612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb4005bfa0 msgr2=0x7fcb400ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:27.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb377fe700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb4005bfa0 0x7fcb400ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:27.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb377fe700 1 -- 172.21.15.140:0/2482039612 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb38009470 con 0x7fcb400ad070 2022-07-21T01:48:27.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb44b72700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb400b5b80 0x7fcb400a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:27.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb377fe700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb400ad070 0x7fcb400a1100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcb2800c000 tx=0x7fcb28006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:27.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb3dffb700 1 -- 172.21.15.140:0/2482039612 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb28010780 con 0x7fcb400ad070 2022-07-21T01:48:27.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb3dffb700 1 -- 172.21.15.140:0/2482039612 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb280108e0 con 0x7fcb400ad070 2022-07-21T01:48:27.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb465d5700 1 -- 172.21.15.140:0/2482039612 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb400a15f0 con 0x7fcb400ad070 2022-07-21T01:48:27.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb3dffb700 1 -- 172.21.15.140:0/2482039612 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb28010c40 con 0x7fcb400ad070 2022-07-21T01:48:27.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb465d5700 1 -- 172.21.15.140:0/2482039612 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4014a070 con 0x7fcb400ad070 2022-07-21T01:48:27.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb465d5700 1 -- 172.21.15.140:0/2482039612 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb400a0f40 con 0x7fcb400ad070 2022-07-21T01:48:27.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.099+0000 7fcb3dffb700 1 -- 172.21.15.140:0/2482039612 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcb2801c460 con 0x7fcb400ad070 2022-07-21T01:48:27.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.103+0000 7fcb3dffb700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb18053b30 0x7fcb18055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:27.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.103+0000 7fcb3ffff700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb18053b30 0x7fcb18055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:27.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.103+0000 7fcb3dffb700 1 -- 172.21.15.140:0/2482039612 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5242+0+0 (secure 0 0 0) 0x7fcb2806c100 con 0x7fcb400ad070 2022-07-21T01:48:27.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.103+0000 7fcb3ffff700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb18053b30 0x7fcb18055ff0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcb3000d000 tx=0x7fcb30009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:27.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.107+0000 7fcb3dffb700 1 -- 172.21.15.140:0/2482039612 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcb400a0f40 con 0x7fcb400ad070 2022-07-21T01:48:27.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.239+0000 7fcb465d5700 1 -- 172.21.15.140:0/2482039612 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcb400a0f40 con 0x7fcb400ad070 2022-07-21T01:48:27.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.959+0000 7fcb3dffb700 1 -- 172.21.15.140:0/2482039612 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7fcb400a0f40 con 0x7fcb400ad070 2022-07-21T01:48:27.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 -- 172.21.15.140:0/2482039612 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb18053b30 msgr2=0x7fcb18055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:27.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb18053b30 0x7fcb18055ff0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcb3000d000 tx=0x7fcb30009040).stop 2022-07-21T01:48:27.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 -- 172.21.15.140:0/2482039612 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb400ad070 msgr2=0x7fcb400a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:27.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb400ad070 0x7fcb400a1100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fcb2800c000 tx=0x7fcb28006df0).stop 2022-07-21T01:48:27.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 -- 172.21.15.140:0/2482039612 shutdown_connections 2022-07-21T01:48:27.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb18053b30 0x7fcb18055ff0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:27.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb400b5b80 0x7fcb400a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:27.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb400ad070 0x7fcb400a1100 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0x7fcb2800c000 tx=0x7fcb28006df0).stop 2022-07-21T01:48:27.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 --2- 172.21.15.140:0/2482039612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb4005bfa0 0x7fcb400ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:27.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 -- 172.21.15.140:0/2482039612 >> 172.21.15.140:0/2482039612 conn(0x7fcb400635e0 msgr2=0x7fcb400b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:27.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 -- 172.21.15.140:0/2482039612 shutdown_connections 2022-07-21T01:48:27.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:27.967+0000 7fcb465d5700 1 -- 172.21.15.140:0/2482039612 wait complete. 2022-07-21T01:48:27.973 INFO:teuthology.orchestra.run.smithi140.stderr:enabled application 'rbd' on pool 'rbd' 2022-07-21T01:48:27.985 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-07-21T01:48:27.986 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:48:27.986 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:48:27.990 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:48:27.991 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:48:27.992 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:48:27.996 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:48:27.997 INFO:tasks.ceph:Setting up CephFS filesystem... 2022-07-21T01:48:27.998 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:28.059 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:48:28.074 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:48:28.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.107+0000 7f730d975700 1 Processor -- start 2022-07-21T01:48:28.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.107+0000 7f730d975700 1 -- start start 2022-07-21T01:48:28.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.107+0000 7f730d975700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73080b1ef0 0x7f7308142090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.107+0000 7f730d975700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7308142580 0x7f730805c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.107+0000 7f730d975700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73080b22d0 con 0x7f730805cb10 2022-07-21T01:48:28.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.107+0000 7f730d975700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73081472a0 con 0x7f7308142580 2022-07-21T01:48:28.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.107+0000 7f730d975700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73080ac0c0 con 0x7f73080b1ef0 2022-07-21T01:48:28.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.107+0000 7f7307fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7308142580 0x7f730805c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.107+0000 7f730d174700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f730805cb10 0x7f7308061130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46268/0 (socket says 172.21.15.140:46268) 2022-07-21T01:48:28.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.107+0000 7f730d174700 1 -- 172.21.15.140:0/415256150 learned_addr learned my addr 172.21.15.140:0/415256150 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:28.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.107+0000 7f730c973700 1 --2- 172.21.15.140:0/415256150 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73080b1ef0 0x7f7308142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f7307fff700 1 -- 172.21.15.140:0/415256150 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73080b1ef0 msgr2=0x7f7308142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:28.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f7307fff700 1 --2- 172.21.15.140:0/415256150 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73080b1ef0 0x7f7308142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f7307fff700 1 -- 172.21.15.140:0/415256150 >> v1:172.21.15.140:6789/0 conn(0x7f730805cb10 legacy=0x7f7308061130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f73077fe700 1 -- 172.21.15.140:0/415256150 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257605829 0 0) 0x7f73080b22d0 con 0x7f730805cb10 2022-07-21T01:48:28.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f7307fff700 1 -- 172.21.15.140:0/415256150 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080ac360 con 0x7f7308142580 2022-07-21T01:48:28.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f7307fff700 1 --2- 172.21.15.140:0/415256150 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7308142580 0x7f730805c620 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f72f0002f70 tx=0x7f72f0006df0).ready entity=mon.0 client_cookie=8fbe0f8f4ee94cf8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f73077fe700 1 -- 172.21.15.140:0/415256150 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72f000d570 con 0x7f7308142580 2022-07-21T01:48:28.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f73077fe700 1 -- 172.21.15.140:0/415256150 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f0000e90 con 0x7f7308142580 2022-07-21T01:48:28.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f73077fe700 1 -- 172.21.15.140:0/415256150 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72f000d8c0 con 0x7f7308142580 2022-07-21T01:48:28.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f730d975700 1 -- 172.21.15.140:0/415256150 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7308142580 msgr2=0x7f730805c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f730d975700 1 --2- 172.21.15.140:0/415256150 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7308142580 0x7f730805c620 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f72f0002f70 tx=0x7f72f0006df0).stop 2022-07-21T01:48:28.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f730d975700 1 -- 172.21.15.140:0/415256150 shutdown_connections 2022-07-21T01:48:28.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f730d975700 1 --2- 172.21.15.140:0/415256150 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7308142580 0x7f730805c620 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f730d975700 1 --2- 172.21.15.140:0/415256150 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73080b1ef0 0x7f7308142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f730d975700 1 -- 172.21.15.140:0/415256150 >> 172.21.15.140:0/415256150 conn(0x7f73080b2ca0 msgr2=0x7f73080b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:28.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f730d975700 1 -- 172.21.15.140:0/415256150 shutdown_connections 2022-07-21T01:48:28.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f730d975700 1 -- 172.21.15.140:0/415256150 wait complete. 2022-07-21T01:48:28.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f730d975700 1 Processor -- start 2022-07-21T01:48:28.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.111+0000 7f730d975700 1 -- start start 2022-07-21T01:48:28.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f730d975700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f730805cb10 0x7f730809add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f730d975700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73080b1ef0 0x7f73080aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f730d975700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7308142580 0x7f7308099d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f730d975700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73080aec20 con 0x7f73080b1ef0 2022-07-21T01:48:28.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f730d975700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730809a5a0 con 0x7f7308142580 2022-07-21T01:48:28.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f730d975700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730809a840 con 0x7f730805cb10 2022-07-21T01:48:28.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7307fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73080b1ef0 0x7f73080aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7307fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73080b1ef0 0x7f73080aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60358/0 (socket says 172.21.15.140:60358) 2022-07-21T01:48:28.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7307fff700 1 -- 172.21.15.140:0/4140218057 learned_addr learned my addr 172.21.15.140:0/4140218057 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:28.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f730c973700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f730805cb10 0x7f730809add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f730d174700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7308142580 0x7f7308099d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7307fff700 1 -- 172.21.15.140:0/4140218057 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f730805cb10 msgr2=0x7f730809add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7307fff700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f730805cb10 0x7f730809add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7307fff700 1 -- 172.21.15.140:0/4140218057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7308142580 msgr2=0x7f7308099d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7307fff700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7308142580 0x7f7308099d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7307fff700 1 -- 172.21.15.140:0/4140218057 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f000b180 con 0x7f73080b1ef0 2022-07-21T01:48:28.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7307fff700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73080b1ef0 0x7f73080aedd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f72f0002f70 tx=0x7f72f0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7305ffb700 1 -- 172.21.15.140:0/4140218057 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72f000dea0 con 0x7f73080b1ef0 2022-07-21T01:48:28.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f730d975700 1 -- 172.21.15.140:0/4140218057 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73080af2c0 con 0x7f73080b1ef0 2022-07-21T01:48:28.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7305ffb700 1 -- 172.21.15.140:0/4140218057 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f0013e90 con 0x7f73080b1ef0 2022-07-21T01:48:28.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7305ffb700 1 -- 172.21.15.140:0/4140218057 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72f001e5f0 con 0x7f73080b1ef0 2022-07-21T01:48:28.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f730d975700 1 -- 172.21.15.140:0/4140218057 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730809a200 con 0x7f73080b1ef0 2022-07-21T01:48:28.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f730d975700 1 -- 172.21.15.140:0/4140218057 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73080ac600 con 0x7f73080b1ef0 2022-07-21T01:48:28.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7305ffb700 1 -- 172.21.15.140:0/4140218057 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f72f001c070 con 0x7f73080b1ef0 2022-07-21T01:48:28.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7305ffb700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ec053950 0x7f72ec055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.115+0000 7f7305ffb700 1 -- 172.21.15.140:0/4140218057 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f72f006c620 con 0x7f73080b1ef0 2022-07-21T01:48:28.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.119+0000 7f730c973700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ec053950 0x7f72ec055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.119+0000 7f730c973700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ec053950 0x7f72ec055e10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f730000b000 tx=0x7f73000091a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.123+0000 7f7305ffb700 1 -- 172.21.15.140:0/4140218057 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72f00437d0 con 0x7f73080b1ef0 2022-07-21T01:48:28.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.247+0000 7f730d975700 1 -- 172.21.15.140:0/4140218057 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73080574e0 con 0x7f73080b1ef0 2022-07-21T01:48:28.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.247+0000 7f7305ffb700 1 -- 172.21.15.140:0/4140218057 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13287 (secure 0 0 0) 0x7f72f0023020 con 0x7f73080b1ef0 2022-07-21T01:48:28.249 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:28.249 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":17,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:27.954230+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:28.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 -- 172.21.15.140:0/4140218057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ec053950 msgr2=0x7f72ec055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ec053950 0x7f72ec055e10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f730000b000 tx=0x7f73000091a0).stop 2022-07-21T01:48:28.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 -- 172.21.15.140:0/4140218057 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73080b1ef0 msgr2=0x7f73080aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73080b1ef0 0x7f73080aedd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f72f0002f70 tx=0x7f72f0014000).stop 2022-07-21T01:48:28.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 -- 172.21.15.140:0/4140218057 shutdown_connections 2022-07-21T01:48:28.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ec053950 0x7f72ec055e10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7308142580 0x7f7308099d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73080b1ef0 0x7f73080aedd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 --2- 172.21.15.140:0/4140218057 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f730805cb10 0x7f730809add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 -- 172.21.15.140:0/4140218057 >> 172.21.15.140:0/4140218057 conn(0x7f73080b2ca0 msgr2=0x7f7308140380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:28.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 -- 172.21.15.140:0/4140218057 shutdown_connections 2022-07-21T01:48:28.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.251+0000 7f730d975700 1 -- 172.21.15.140:0/4140218057 wait complete. 2022-07-21T01:48:28.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:48:28.264 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:48:28.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda6f39700 1 Processor -- start 2022-07-21T01:48:28.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda6f39700 1 -- start start 2022-07-21T01:48:28.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda6f39700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdda00b59f0 0x7fdda01473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda6f39700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda005cc70 con 0x7fdda00af940 2022-07-21T01:48:28.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda6f39700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda00b5dd0 con 0x7fdda00b1400 2022-07-21T01:48:28.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda6f39700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda00b1070 con 0x7fdda00b59f0 2022-07-21T01:48:28.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5736700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fdda00af940 0x7fdda014a3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46280/0 (socket says 172.21.15.140:46280) 2022-07-21T01:48:28.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5736700 1 -- 172.21.15.140:0/4032769821 learned_addr learned my addr 172.21.15.140:0/4032769821 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:28.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5f37700 1 --2- 172.21.15.140:0/4032769821 >> v2:172.21.15.152:3300/0 conn(0x7fdda00b59f0 0x7fdda01473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358982155 0 0) 0x7fdda005cc70 con 0x7fdda00af940 2022-07-21T01:48:28.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd84003260 con 0x7fdda00af940 2022-07-21T01:48:28.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3082437588 0 0) 0x7fdd84003260 con 0x7fdda00af940 2022-07-21T01:48:28.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 >> v2:172.21.15.152:3300/0 conn(0x7fdda00b59f0 msgr2=0x7fdda01473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:28.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 --2- 172.21.15.140:0/4032769821 >> v2:172.21.15.152:3300/0 conn(0x7fdda00b59f0 0x7fdda01473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 >> v1:172.21.15.152:6789/0 conn(0x7fdda00b1400 legacy=0x7fdda014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda005c760 con 0x7fdda00af940 2022-07-21T01:48:28.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fdd9c001660 con 0x7fdda00af940 2022-07-21T01:48:28.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 >> v1:172.21.15.140:6789/0 conn(0x7fdda00af940 legacy=0x7fdda014a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd84003750 0x7fdd84003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd84004500 0x7fdd84004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd84004e70 0x7fdd8400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda00b1070 con 0x7fdd84003750 2022-07-21T01:48:28.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8400bdd0 con 0x7fdd84004500 2022-07-21T01:48:28.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8400bf60 con 0x7fdd84004e70 2022-07-21T01:48:28.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5f37700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd84003750 0x7fdd84003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5736700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd84004500 0x7fdd84004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda6738700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd84004e70 0x7fdd8400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5f37700 1 -- 172.21.15.140:0/4032769821 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd84004e70 msgr2=0x7fdd8400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:28.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5f37700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd84004e70 0x7fdd8400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5f37700 1 -- 172.21.15.140:0/4032769821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd84004500 msgr2=0x7fdd84004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:28.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5f37700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd84004500 0x7fdd84004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5f37700 1 -- 172.21.15.140:0/4032769821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84003260 con 0x7fdd84003750 2022-07-21T01:48:28.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda6738700 1 -- 172.21.15.140:0/4032769821 reap_dead start 2022-07-21T01:48:28.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5736700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd84004500 0x7fdd84004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:28.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda5f37700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd84003750 0x7fdd84003b70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fdd9000c000 tx=0x7fdd90006df0).ready entity=mon.0 client_cookie=7cf18245b37b42a4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd9000e830 con 0x7fdd84003750 2022-07-21T01:48:28.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9000e990 con 0x7fdd84003750 2022-07-21T01:48:28.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.371+0000 7fdda4f35700 1 -- 172.21.15.140:0/4032769821 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd9000ecf0 con 0x7fdd84003750 2022-07-21T01:48:28.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 -- 172.21.15.140:0/4032769821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd84003750 msgr2=0x7fdd84003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd84003750 0x7fdd84003b70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fdd9000c000 tx=0x7fdd90006df0).stop 2022-07-21T01:48:28.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 -- 172.21.15.140:0/4032769821 shutdown_connections 2022-07-21T01:48:28.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 --2- 172.21.15.140:0/4032769821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd84003750 0x7fdd84003b70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 -- 172.21.15.140:0/4032769821 >> 172.21.15.140:0/4032769821 conn(0x7fdda0063480 msgr2=0x7fdda00b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:28.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 -- 172.21.15.140:0/4032769821 shutdown_connections 2022-07-21T01:48:28.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 -- 172.21.15.140:0/4032769821 wait complete. 2022-07-21T01:48:28.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 Processor -- start 2022-07-21T01:48:28.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 -- start start 2022-07-21T01:48:28.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd84003750 0x7fdda0141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdda0142e00 0x7fdda0143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdda0143a00 0x7fdda0156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda010f230 con 0x7fdda0143a00 2022-07-21T01:48:28.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda010f4d0 con 0x7fdd84003750 2022-07-21T01:48:28.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6f39700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda010f770 con 0x7fdda0142e00 2022-07-21T01:48:28.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6738700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdda0143a00 0x7fdda0156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6738700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdda0143a00 0x7fdda0156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60378/0 (socket says 172.21.15.140:60378) 2022-07-21T01:48:28.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6738700 1 -- 172.21.15.140:0/1911837900 learned_addr learned my addr 172.21.15.140:0/1911837900 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:28.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda5f37700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd84003750 0x7fdda0141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda5736700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdda0142e00 0x7fdda0143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6738700 1 -- 172.21.15.140:0/1911837900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdda0142e00 msgr2=0x7fdda0143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6738700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdda0142e00 0x7fdda0143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6738700 1 -- 172.21.15.140:0/1911837900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd84003750 msgr2=0x7fdda0141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6738700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd84003750 0x7fdda0141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6738700 1 -- 172.21.15.140:0/1911837900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda0143e80 con 0x7fdda0143a00 2022-07-21T01:48:28.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda5f37700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd84003750 0x7fdda0141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:28.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda5736700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdda0142e00 0x7fdda0143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:28.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdda6738700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdda0143a00 0x7fdda0156480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fdd98001210 tx=0x7fdd98009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.375+0000 7fdd96ffd700 1 -- 172.21.15.140:0/1911837900 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd98003220 con 0x7fdda0143a00 2022-07-21T01:48:28.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.379+0000 7fdd96ffd700 1 -- 172.21.15.140:0/1911837900 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd98003380 con 0x7fdda0143a00 2022-07-21T01:48:28.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.379+0000 7fdda6f39700 1 -- 172.21.15.140:0/1911837900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda010fc10 con 0x7fdda0143a00 2022-07-21T01:48:28.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.379+0000 7fdd96ffd700 1 -- 172.21.15.140:0/1911837900 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd98001980 con 0x7fdda0143a00 2022-07-21T01:48:28.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.379+0000 7fdda6f39700 1 -- 172.21.15.140:0/1911837900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda010eeb0 con 0x7fdda0143a00 2022-07-21T01:48:28.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.379+0000 7fdd96ffd700 1 -- 172.21.15.140:0/1911837900 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdd9801c060 con 0x7fdda0143a00 2022-07-21T01:48:28.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.379+0000 7fdda6f39700 1 -- 172.21.15.140:0/1911837900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda010f930 con 0x7fdda0143a00 2022-07-21T01:48:28.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.379+0000 7fdd96ffd700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd88053a70 0x7fdd88055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.379+0000 7fdda5f37700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd88053a70 0x7fdd88055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.379+0000 7fdd96ffd700 1 -- 172.21.15.140:0/1911837900 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fdd9806b720 con 0x7fdda0143a00 2022-07-21T01:48:28.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.379+0000 7fdda5f37700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd88053a70 0x7fdd88055f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fdd9001b000 tx=0x7fdd90019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.383+0000 7fdd96ffd700 1 -- 172.21.15.140:0/1911837900 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdd980428f0 con 0x7fdda0143a00 2022-07-21T01:48:28.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.507+0000 7fdda6f39700 1 -- 172.21.15.140:0/1911837900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdda005c9d0 con 0x7fdda0143a00 2022-07-21T01:48:28.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.507+0000 7fdd96ffd700 1 -- 172.21.15.140:0/1911837900 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdd9801a070 con 0x7fdda0143a00 2022-07-21T01:48:28.507 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:48:28.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 -- 172.21.15.140:0/1911837900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd88053a70 msgr2=0x7fdd88055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd88053a70 0x7fdd88055f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fdd9001b000 tx=0x7fdd90019000).stop 2022-07-21T01:48:28.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 -- 172.21.15.140:0/1911837900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdda0143a00 msgr2=0x7fdda0156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdda0143a00 0x7fdda0156480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fdd98001210 tx=0x7fdd98009040).stop 2022-07-21T01:48:28.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 -- 172.21.15.140:0/1911837900 shutdown_connections 2022-07-21T01:48:28.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd88053a70 0x7fdd88055f30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdda0143a00 0x7fdda0156480 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdda0142e00 0x7fdda0143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 --2- 172.21.15.140:0/1911837900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd84003750 0x7fdda0141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 -- 172.21.15.140:0/1911837900 >> 172.21.15.140:0/1911837900 conn(0x7fdda0063480 msgr2=0x7fdda00add40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:28.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 -- 172.21.15.140:0/1911837900 shutdown_connections 2022-07-21T01:48:28.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.511+0000 7fdda6f39700 1 -- 172.21.15.140:0/1911837900 wait complete. 2022-07-21T01:48:28.520 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:48:28.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9dffb700 1 Processor -- start 2022-07-21T01:48:28.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9dffb700 1 -- start start 2022-07-21T01:48:28.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9dffb700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c98142c40 0x7f8c9813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9dffb700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c980b5d00 con 0x7f8c980b5920 2022-07-21T01:48:28.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9dffb700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9813f090 con 0x7f8c980b4ee0 2022-07-21T01:48:28.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9dffb700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9813f330 con 0x7f8c98142c40 2022-07-21T01:48:28.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9d7fa700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c98142c40 0x7f8c9813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9cff9700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f8c980b5920 0x7f8c98142530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46298/0 (socket says 172.21.15.140:46298) 2022-07-21T01:48:28.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9cff9700 1 -- 172.21.15.140:0/961572998 learned_addr learned my addr 172.21.15.140:0/961572998 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:28.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c977fe700 1 -- 172.21.15.140:0/961572998 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904195865 0 0) 0x7f8c980b5d00 con 0x7f8c980b5920 2022-07-21T01:48:28.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9d7fa700 1 -- 172.21.15.140:0/961572998 >> v1:172.21.15.152:6789/0 conn(0x7f8c980b4ee0 legacy=0x7f8c9813e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9d7fa700 1 -- 172.21.15.140:0/961572998 >> v1:172.21.15.140:6789/0 conn(0x7f8c980b5920 legacy=0x7f8c98142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9d7fa700 1 -- 172.21.15.140:0/961572998 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9813f5d0 con 0x7f8c98142c40 2022-07-21T01:48:28.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9d7fa700 1 --2- 172.21.15.140:0/961572998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c98142c40 0x7f8c9813fa70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8c900032f0 tx=0x7f8c9000a370).ready entity=mon.0 client_cookie=ac23f457286fe3e9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c977fe700 1 -- 172.21.15.140:0/961572998 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c9000cc70 con 0x7f8c98142c40 2022-07-21T01:48:28.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c977fe700 1 -- 172.21.15.140:0/961572998 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9000cdd0 con 0x7f8c98142c40 2022-07-21T01:48:28.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c977fe700 1 -- 172.21.15.140:0/961572998 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c90010600 con 0x7f8c98142c40 2022-07-21T01:48:28.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9dffb700 1 -- 172.21.15.140:0/961572998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c98142c40 msgr2=0x7f8c9813fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.631+0000 7f8c9dffb700 1 --2- 172.21.15.140:0/961572998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c98142c40 0x7f8c9813fa70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8c900032f0 tx=0x7f8c9000a370).stop 2022-07-21T01:48:28.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 -- 172.21.15.140:0/961572998 shutdown_connections 2022-07-21T01:48:28.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 --2- 172.21.15.140:0/961572998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c98142c40 0x7f8c9813fa70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 -- 172.21.15.140:0/961572998 >> 172.21.15.140:0/961572998 conn(0x7f8c980632b0 msgr2=0x7f8c98063690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:28.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 -- 172.21.15.140:0/961572998 shutdown_connections 2022-07-21T01:48:28.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 -- 172.21.15.140:0/961572998 wait complete. 2022-07-21T01:48:28.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 Processor -- start 2022-07-21T01:48:28.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 -- start start 2022-07-21T01:48:28.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c980b4ee0 0x7f8c980b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c980b5920 0x7f8c98136990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c98142c40 0x7f8c98136e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98154040 con 0x7f8c98142c40 2022-07-21T01:48:28.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98153b30 con 0x7f8c980b5920 2022-07-21T01:48:28.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98153da0 con 0x7f8c980b4ee0 2022-07-21T01:48:28.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9d7fa700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c98142c40 0x7f8c98136e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9d7fa700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c98142c40 0x7f8c98136e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60394/0 (socket says 172.21.15.140:60394) 2022-07-21T01:48:28.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9d7fa700 1 -- 172.21.15.140:0/3556609803 learned_addr learned my addr 172.21.15.140:0/3556609803 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:28.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c97fff700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c980b5920 0x7f8c98136990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9cff9700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c980b4ee0 0x7f8c980b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9d7fa700 1 -- 172.21.15.140:0/3556609803 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c980b4ee0 msgr2=0x7f8c980b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9d7fa700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c980b4ee0 0x7f8c980b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9d7fa700 1 -- 172.21.15.140:0/3556609803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c980b5920 msgr2=0x7f8c98136990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9d7fa700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c980b5920 0x7f8c98136990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9d7fa700 1 -- 172.21.15.140:0/3556609803 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9000a610 con 0x7f8c98142c40 2022-07-21T01:48:28.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9d7fa700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c98142c40 0x7f8c98136e80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f8c900032f0 tx=0x7f8c980acb30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c95ffb700 1 -- 172.21.15.140:0/3556609803 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c9001a630 con 0x7f8c98142c40 2022-07-21T01:48:28.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c95ffb700 1 -- 172.21.15.140:0/3556609803 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9001a790 con 0x7f8c98142c40 2022-07-21T01:48:28.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 -- 172.21.15.140:0/3556609803 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c98137370 con 0x7f8c98142c40 2022-07-21T01:48:28.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c95ffb700 1 -- 172.21.15.140:0/3556609803 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c9001aaf0 con 0x7f8c98142c40 2022-07-21T01:48:28.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.635+0000 7f8c9dffb700 1 -- 172.21.15.140:0/3556609803 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c98064280 con 0x7f8c98142c40 2022-07-21T01:48:28.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.639+0000 7f8c9dffb700 1 -- 172.21.15.140:0/3556609803 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9809ba80 con 0x7f8c98142c40 2022-07-21T01:48:28.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.639+0000 7f8c95ffb700 1 -- 172.21.15.140:0/3556609803 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8c9002a070 con 0x7f8c98142c40 2022-07-21T01:48:28.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.639+0000 7f8c95ffb700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c7c053a70 0x7f8c7c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.639+0000 7f8c95ffb700 1 -- 172.21.15.140:0/3556609803 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f8c9006d890 con 0x7f8c98142c40 2022-07-21T01:48:28.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.639+0000 7f8c9cff9700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c7c053a70 0x7f8c7c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.639+0000 7f8c9cff9700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c7c053a70 0x7f8c7c055f30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f8c8000d000 tx=0x7f8c800011a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.643+0000 7f8c95ffb700 1 -- 172.21.15.140:0/3556609803 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c90044ae0 con 0x7f8c98142c40 2022-07-21T01:48:28.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.767+0000 7f8c9dffb700 1 -- 172.21.15.140:0/3556609803 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8c980574e0 con 0x7f8c98142c40 2022-07-21T01:48:28.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.767+0000 7f8c95ffb700 1 -- 172.21.15.140:0/3556609803 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8c90048100 con 0x7f8c98142c40 2022-07-21T01:48:28.767 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:28.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 -- 172.21.15.140:0/3556609803 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c7c053a70 msgr2=0x7f8c7c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c7c053a70 0x7f8c7c055f30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f8c8000d000 tx=0x7f8c800011a0).stop 2022-07-21T01:48:28.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 -- 172.21.15.140:0/3556609803 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c98142c40 msgr2=0x7f8c98136e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c98142c40 0x7f8c98136e80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f8c900032f0 tx=0x7f8c980acb30).stop 2022-07-21T01:48:28.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 -- 172.21.15.140:0/3556609803 shutdown_connections 2022-07-21T01:48:28.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c7c053a70 0x7f8c7c055f30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c98142c40 0x7f8c98136e80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c980b5920 0x7f8c98136990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 --2- 172.21.15.140:0/3556609803 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c980b4ee0 0x7f8c980b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 -- 172.21.15.140:0/3556609803 >> 172.21.15.140:0/3556609803 conn(0x7f8c980632b0 msgr2=0x7f8c98137b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:28.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 -- 172.21.15.140:0/3556609803 shutdown_connections 2022-07-21T01:48:28.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.771+0000 7f8c9dffb700 1 -- 172.21.15.140:0/3556609803 wait complete. 2022-07-21T01:48:28.780 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:48:28.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 Processor -- start 2022-07-21T01:48:28.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 -- start start 2022-07-21T01:48:28.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f22680b1ef0 0x7f22680605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 0x7f22680abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2268140940 con 0x7f22680ac4e0 2022-07-21T01:48:28.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22680b22d0 con 0x7f22680b1ef0 2022-07-21T01:48:28.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226805fbc0 con 0x7f226813f980 2022-07-21T01:48:28.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226e69c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f22680ac4e0 0x7f22680ab180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46316/0 (socket says 172.21.15.140:46316) 2022-07-21T01:48:28.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226e69c700 1 -- 172.21.15.140:0/3950218778 learned_addr learned my addr 172.21.15.140:0/3950218778 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:28.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226de9b700 1 --2- 172.21.15.140:0/3950218778 >> v2:172.21.15.152:3300/0 conn(0x7f22680b1ef0 0x7f22680605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226d69a700 1 --2- 172.21.15.140:0/3950218778 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 0x7f22680abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ce99700 1 -- 172.21.15.140:0/3950218778 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180620800 0 0) 0x7f2268140940 con 0x7f22680ac4e0 2022-07-21T01:48:28.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ce99700 1 -- 172.21.15.140:0/3950218778 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f224c003260 con 0x7f22680ac4e0 2022-07-21T01:48:28.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226de9b700 1 -- 172.21.15.140:0/3950218778 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 msgr2=0x7f22680abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:28.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226de9b700 1 --2- 172.21.15.140:0/3950218778 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 0x7f22680abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226de9b700 1 -- 172.21.15.140:0/3950218778 >> v1:172.21.15.140:6789/0 conn(0x7f22680ac4e0 legacy=0x7f22680ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226de9b700 1 -- 172.21.15.140:0/3950218778 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226805fe60 con 0x7f22680b1ef0 2022-07-21T01:48:28.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226d69a700 1 --2- 172.21.15.140:0/3950218778 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 0x7f22680abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:28.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226de9b700 1 --2- 172.21.15.140:0/3950218778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22680b1ef0 0x7f22680605a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2258002f70 tx=0x7f2258006df0).ready entity=mon.1 client_cookie=120959f67fbd78e8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ce99700 1 -- 172.21.15.140:0/3950218778 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f225800d4a0 con 0x7f22680b1ef0 2022-07-21T01:48:28.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ce99700 1 -- 172.21.15.140:0/3950218778 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2258000e90 con 0x7f22680b1ef0 2022-07-21T01:48:28.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ce99700 1 -- 172.21.15.140:0/3950218778 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f225800d7f0 con 0x7f22680b1ef0 2022-07-21T01:48:28.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 -- 172.21.15.140:0/3950218778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22680b1ef0 msgr2=0x7f22680605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 --2- 172.21.15.140:0/3950218778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22680b1ef0 0x7f22680605a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2258002f70 tx=0x7f2258006df0).stop 2022-07-21T01:48:28.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 -- 172.21.15.140:0/3950218778 shutdown_connections 2022-07-21T01:48:28.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 --2- 172.21.15.140:0/3950218778 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 0x7f22680abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 --2- 172.21.15.140:0/3950218778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22680b1ef0 0x7f22680605a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 -- 172.21.15.140:0/3950218778 >> 172.21.15.140:0/3950218778 conn(0x7f22680b2ca0 msgr2=0x7f22680b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:28.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 -- 172.21.15.140:0/3950218778 shutdown_connections 2022-07-21T01:48:28.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.883+0000 7f226ee9d700 1 -- 172.21.15.140:0/3950218778 wait complete. 2022-07-21T01:48:28.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226ee9d700 1 Processor -- start 2022-07-21T01:48:28.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226ee9d700 1 -- start start 2022-07-21T01:48:28.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226ee9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22680ac4e0 0x7f226809b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226ee9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f22680b1ef0 0x7f226813b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226ee9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 0x7f226813b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226ee9d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226814fd10 con 0x7f22680b1ef0 2022-07-21T01:48:28.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226ee9d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22680ad640 con 0x7f22680ac4e0 2022-07-21T01:48:28.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226ee9d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22680abcb0 con 0x7f226813f980 2022-07-21T01:48:28.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226d69a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f22680b1ef0 0x7f226813b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226de9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22680ac4e0 0x7f226809b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226e69c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 0x7f226813b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226d69a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f22680b1ef0 0x7f226813b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60408/0 (socket says 172.21.15.140:60408) 2022-07-21T01:48:28.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226e69c700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 0x7f226813b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:57136/0 (socket says 172.21.15.140:57136) 2022-07-21T01:48:28.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226d69a700 1 -- 172.21.15.140:0/338187022 learned_addr learned my addr 172.21.15.140:0/338187022 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:28.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226e69c700 1 -- 172.21.15.140:0/338187022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22680ac4e0 msgr2=0x7f226809b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226e69c700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22680ac4e0 0x7f226809b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226e69c700 1 -- 172.21.15.140:0/338187022 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f22680b1ef0 msgr2=0x7f226813b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226e69c700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f22680b1ef0 0x7f226813b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226e69c700 1 -- 172.21.15.140:0/338187022 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2260009470 con 0x7f226813f980 2022-07-21T01:48:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226e69c700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 0x7f226813b970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f2264001110 tx=0x7f22640079c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f225effd700 1 -- 172.21.15.140:0/338187022 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f226400cdb0 con 0x7f226813f980 2022-07-21T01:48:28.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f225effd700 1 -- 172.21.15.140:0/338187022 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22640070d0 con 0x7f226813f980 2022-07-21T01:48:28.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226ee9d700 1 -- 172.21.15.140:0/338187022 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225800b180 con 0x7f226813f980 2022-07-21T01:48:28.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f225effd700 1 -- 172.21.15.140:0/338187022 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2264007430 con 0x7f226813f980 2022-07-21T01:48:28.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.887+0000 7f226ee9d700 1 -- 172.21.15.140:0/338187022 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226813be60 con 0x7f226813f980 2022-07-21T01:48:28.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.891+0000 7f226ee9d700 1 -- 172.21.15.140:0/338187022 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2268060100 con 0x7f226813f980 2022-07-21T01:48:28.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.891+0000 7f225effd700 1 -- 172.21.15.140:0/338187022 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2264010070 con 0x7f226813f980 2022-07-21T01:48:28.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.891+0000 7f225effd700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2250053a70 0x7f2250055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:28.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.891+0000 7f225effd700 1 -- 172.21.15.140:0/338187022 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f226406b2d0 con 0x7f226813f980 2022-07-21T01:48:28.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.891+0000 7f226de9b700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2250053a70 0x7f2250055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:28.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.891+0000 7f226de9b700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2250053a70 0x7f2250055f30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f225801a000 tx=0x7f2258014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:28.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:28.903+0000 7f225effd700 1 -- 172.21.15.140:0/338187022 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2264042390 con 0x7f226813f980 2022-07-21T01:48:29.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f226ee9d700 1 -- 172.21.15.140:0/338187022 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f22680574e0 con 0x7f226813f980 2022-07-21T01:48:29.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f225effd700 1 -- 172.21.15.140:0/338187022 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+143 (secure 0 0 0) 0x7f22640459b0 con 0x7f226813f980 2022-07-21T01:48:29.031 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:29.031 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:48:29.031 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:48:29.031 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:48:29.032 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:48:29.032 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:48:29.033 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:48:29.033 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:48:29.034 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:48:29.034 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:48:29.034 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:48:29.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f226ee9d700 1 -- 172.21.15.140:0/338187022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2250053a70 msgr2=0x7f2250055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:29.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f226ee9d700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2250053a70 0x7f2250055f30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f225801a000 tx=0x7f2258014000).stop 2022-07-21T01:48:29.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f226ee9d700 1 -- 172.21.15.140:0/338187022 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 msgr2=0x7f226813b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:29.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f226ee9d700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 0x7f226813b970 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f2264001110 tx=0x7f22640079c0).stop 2022-07-21T01:48:29.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f226ee9d700 1 -- 172.21.15.140:0/338187022 shutdown_connections 2022-07-21T01:48:29.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f226ee9d700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2250053a70 0x7f2250055f30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:29.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f226ee9d700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f226813f980 0x7f226813b970 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:29.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f226ee9d700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f22680b1ef0 0x7f226813b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:29.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f226ee9d700 1 --2- 172.21.15.140:0/338187022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22680ac4e0 0x7f226809b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:29.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.031+0000 7f226ee9d700 1 -- 172.21.15.140:0/338187022 >> 172.21.15.140:0/338187022 conn(0x7f22680b2ca0 msgr2=0x7f22680b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:29.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.035+0000 7f226ee9d700 1 -- 172.21.15.140:0/338187022 shutdown_connections 2022-07-21T01:48:29.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.035+0000 7f226ee9d700 1 -- 172.21.15.140:0/338187022 wait complete. 2022-07-21T01:48:29.044 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:48:29.045 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:48:29.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.151+0000 7fc782a60700 1 Processor -- start 2022-07-21T01:48:29.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.151+0000 7fc782a60700 1 -- start start 2022-07-21T01:48:29.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.151+0000 7fc782a60700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc77c0b59f0 0x7fc77c1473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:29.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.151+0000 7fc782a60700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc77c0af940 0x7fc77c147890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:29.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.151+0000 7fc782a60700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c05cc70 con 0x7fc77c0b1400 2022-07-21T01:48:29.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.151+0000 7fc782a60700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77c0b1070 con 0x7fc77c0b59f0 2022-07-21T01:48:29.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.151+0000 7fc782a60700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77c0b5dd0 con 0x7fc77c0af940 2022-07-21T01:48:29.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.151+0000 7fc781a5e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc77c0b59f0 0x7fc77c1473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:29.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.151+0000 7fc781a5e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc77c0b59f0 0x7fc77c1473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60412/0 (socket says 172.21.15.140:60412) 2022-07-21T01:48:29.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.151+0000 7fc781a5e700 1 -- 172.21.15.140:0/3370959651 learned_addr learned my addr 172.21.15.140:0/3370959651 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:29.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc78125d700 1 --2- 172.21.15.140:0/3370959651 >> v2:172.21.15.152:3300/0 conn(0x7fc77c0af940 0x7fc77c147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:29.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc781a5e700 1 -- 172.21.15.140:0/3370959651 >> v2:172.21.15.152:3300/0 conn(0x7fc77c0af940 msgr2=0x7fc77c147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:29.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc781a5e700 1 --2- 172.21.15.140:0/3370959651 >> v2:172.21.15.152:3300/0 conn(0x7fc77c0af940 0x7fc77c147890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:29.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc781a5e700 1 -- 172.21.15.140:0/3370959651 >> v1:172.21.15.152:6789/0 conn(0x7fc77c0b1400 legacy=0x7fc77c14a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:29.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc781a5e700 1 -- 172.21.15.140:0/3370959651 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c05c760 con 0x7fc77c0b59f0 2022-07-21T01:48:29.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc781a5e700 1 --2- 172.21.15.140:0/3370959651 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc77c0b59f0 0x7fc77c1473a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fc770002f70 tx=0x7fc770006eb0).ready entity=mon.0 client_cookie=fc1d49d15d415fcd server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:29.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc780a5c700 1 -- 172.21.15.140:0/3370959651 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc77000d5c0 con 0x7fc77c0b59f0 2022-07-21T01:48:29.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc780a5c700 1 -- 172.21.15.140:0/3370959651 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc770000e90 con 0x7fc77c0b59f0 2022-07-21T01:48:29.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc780a5c700 1 -- 172.21.15.140:0/3370959651 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc77000d950 con 0x7fc77c0b59f0 2022-07-21T01:48:29.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 -- 172.21.15.140:0/3370959651 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc77c0b59f0 msgr2=0x7fc77c1473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:29.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 --2- 172.21.15.140:0/3370959651 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc77c0b59f0 0x7fc77c1473a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fc770002f70 tx=0x7fc770006eb0).stop 2022-07-21T01:48:29.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 -- 172.21.15.140:0/3370959651 shutdown_connections 2022-07-21T01:48:29.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 --2- 172.21.15.140:0/3370959651 >> v2:172.21.15.152:3300/0 conn(0x7fc77c0af940 0x7fc77c147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:29.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 --2- 172.21.15.140:0/3370959651 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc77c0b59f0 0x7fc77c1473a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:29.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 -- 172.21.15.140:0/3370959651 >> 172.21.15.140:0/3370959651 conn(0x7fc77c063480 msgr2=0x7fc77c0b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:29.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 -- 172.21.15.140:0/3370959651 shutdown_connections 2022-07-21T01:48:29.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 -- 172.21.15.140:0/3370959651 wait complete. 2022-07-21T01:48:29.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 Processor -- start 2022-07-21T01:48:29.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 -- start start 2022-07-21T01:48:29.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc77c0af940 0x7fc77c10e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:29.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc77c0b1400 0x7fc77c157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:29.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc77c0b59f0 0x7fc77c110e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:29.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c111a30 con 0x7fc77c0b59f0 2022-07-21T01:48:29.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c111790 con 0x7fc77c0af940 2022-07-21T01:48:29.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc782a60700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c111cb0 con 0x7fc77c0b1400 2022-07-21T01:48:29.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc78225f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc77c0b59f0 0x7fc77c110e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:29.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc78225f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc77c0b59f0 0x7fc77c110e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60420/0 (socket says 172.21.15.140:60420) 2022-07-21T01:48:29.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc78225f700 1 -- 172.21.15.140:0/3531018340 learned_addr learned my addr 172.21.15.140:0/3531018340 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:29.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc781a5e700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc77c0af940 0x7fc77c10e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:29.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.155+0000 7fc78125d700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc77c0b1400 0x7fc77c157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:29.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc78225f700 1 -- 172.21.15.140:0/3531018340 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc77c0b1400 msgr2=0x7fc77c157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:29.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc78225f700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc77c0b1400 0x7fc77c157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:29.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc78225f700 1 -- 172.21.15.140:0/3531018340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc77c0af940 msgr2=0x7fc77c10e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:29.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc78225f700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc77c0af940 0x7fc77c10e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:29.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc78225f700 1 -- 172.21.15.140:0/3531018340 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77000b1e0 con 0x7fc77c0b59f0 2022-07-21T01:48:29.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc78225f700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc77c0b59f0 0x7fc77c110e60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc77c068460 tx=0x7fc774007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:29.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc77affd700 1 -- 172.21.15.140:0/3531018340 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc774012400 con 0x7fc77c0b59f0 2022-07-21T01:48:29.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc77affd700 1 -- 172.21.15.140:0/3531018340 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc774012560 con 0x7fc77c0b59f0 2022-07-21T01:48:29.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc782a60700 1 -- 172.21.15.140:0/3531018340 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c112150 con 0x7fc77c0b59f0 2022-07-21T01:48:29.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc77affd700 1 -- 172.21.15.140:0/3531018340 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7740128c0 con 0x7fc77c0b59f0 2022-07-21T01:48:29.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc782a60700 1 -- 172.21.15.140:0/3531018340 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c1578d0 con 0x7fc77c0b59f0 2022-07-21T01:48:29.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc782a60700 1 -- 172.21.15.140:0/3531018340 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c05c9d0 con 0x7fc77c0b59f0 2022-07-21T01:48:29.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc77affd700 1 -- 172.21.15.140:0/3531018340 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc774019030 con 0x7fc77c0b59f0 2022-07-21T01:48:29.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc77affd700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc75c053b30 0x7fc75c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:29.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc77affd700 1 -- 172.21.15.140:0/3531018340 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc77406ade0 con 0x7fc77c0b59f0 2022-07-21T01:48:29.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc781a5e700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc75c053b30 0x7fc75c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:29.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.159+0000 7fc781a5e700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc75c053b30 0x7fc75c055ff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc77001c000 tx=0x7fc770015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:29.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.167+0000 7fc77affd700 1 -- 172.21.15.140:0/3531018340 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc774042030 con 0x7fc77c0b59f0 2022-07-21T01:48:29.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:29.291+0000 7fc782a60700 1 -- 172.21.15.140:0/3531018340 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fc77c0574e0 con 0x7fc77c0b59f0 2022-07-21T01:48:30.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.095+0000 7fc77affd700 1 -- 172.21.15.140:0/3531018340 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v19) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc774045650 con 0x7fc77c0b59f0 2022-07-21T01:48:30.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 -- 172.21.15.140:0/3531018340 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc75c053b30 msgr2=0x7fc75c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:30.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc75c053b30 0x7fc75c055ff0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc77001c000 tx=0x7fc770015000).stop 2022-07-21T01:48:30.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 -- 172.21.15.140:0/3531018340 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc77c0b59f0 msgr2=0x7fc77c110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:30.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc77c0b59f0 0x7fc77c110e60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc77c068460 tx=0x7fc774007bc0).stop 2022-07-21T01:48:30.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 -- 172.21.15.140:0/3531018340 shutdown_connections 2022-07-21T01:48:30.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc75c053b30 0x7fc75c055ff0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:30.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc77c0b59f0 0x7fc77c110e60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:30.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc77c0b1400 0x7fc77c157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:30.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 --2- 172.21.15.140:0/3531018340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc77c0af940 0x7fc77c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:30.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 -- 172.21.15.140:0/3531018340 >> 172.21.15.140:0/3531018340 conn(0x7fc77c063480 msgr2=0x7fc77c0b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:30.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 -- 172.21.15.140:0/3531018340 shutdown_connections 2022-07-21T01:48:30.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.103+0000 7fc782a60700 1 -- 172.21.15.140:0/3531018340 wait complete. 2022-07-21T01:48:30.114 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:48:30.119 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:48:30.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13e5f8700 1 Processor -- start 2022-07-21T01:48:30.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13e5f8700 1 -- start start 2022-07-21T01:48:30.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13e5f8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1380b59d0 0x7fe1380b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:30.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13e5f8700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe1380b14f0 0x7fe138147050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:30.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13e5f8700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1380afbc0 con 0x7fe13814ada0 2022-07-21T01:48:30.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13e5f8700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe138148080 con 0x7fe1380b14f0 2022-07-21T01:48:30.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13e5f8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380b00e0 con 0x7fe1380b59d0 2022-07-21T01:48:30.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13d5f6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1380b59d0 0x7fe1380b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:30.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13ddf7700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe1380b14f0 0x7fe138147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:30.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13d5f6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1380b59d0 0x7fe1380b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:57156/0 (socket says 172.21.15.140:57156) 2022-07-21T01:48:30.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13d5f6700 1 -- 172.21.15.140:0/4145795977 learned_addr learned my addr 172.21.15.140:0/4145795977 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:30.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13d5f6700 1 -- 172.21.15.140:0/4145795977 >> v2:172.21.15.152:3300/0 conn(0x7fe1380b14f0 msgr2=0x7fe138147050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:30.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13d5f6700 1 --2- 172.21.15.140:0/4145795977 >> v2:172.21.15.152:3300/0 conn(0x7fe1380b14f0 0x7fe138147050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:30.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13d5f6700 1 -- 172.21.15.140:0/4145795977 >> v1:172.21.15.152:6789/0 conn(0x7fe13814ada0 legacy=0x7fe13805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:30.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13d5f6700 1 -- 172.21.15.140:0/4145795977 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1380b5db0 con 0x7fe1380b59d0 2022-07-21T01:48:30.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13ddf7700 1 --2- 172.21.15.140:0/4145795977 >> v2:172.21.15.152:3300/0 conn(0x7fe1380b14f0 0x7fe138147050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:30.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe13d5f6700 1 --2- 172.21.15.140:0/4145795977 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1380b59d0 0x7fe1380b07e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe120002f70 tx=0x7fe120006eb0).ready entity=mon.2 client_cookie=5e84776f9b6a3ded server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:30.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe137fff700 1 -- 172.21.15.140:0/4145795977 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe12000d570 con 0x7fe1380b59d0 2022-07-21T01:48:30.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe137fff700 1 -- 172.21.15.140:0/4145795977 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe120000e90 con 0x7fe1380b59d0 2022-07-21T01:48:30.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.231+0000 7fe137fff700 1 -- 172.21.15.140:0/4145795977 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe12000d900 con 0x7fe1380b59d0 2022-07-21T01:48:30.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 -- 172.21.15.140:0/4145795977 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1380b59d0 msgr2=0x7fe1380b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:30.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 --2- 172.21.15.140:0/4145795977 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1380b59d0 0x7fe1380b07e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe120002f70 tx=0x7fe120006eb0).stop 2022-07-21T01:48:30.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 -- 172.21.15.140:0/4145795977 shutdown_connections 2022-07-21T01:48:30.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 --2- 172.21.15.140:0/4145795977 >> v2:172.21.15.152:3300/0 conn(0x7fe1380b14f0 0x7fe138147050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:30.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 --2- 172.21.15.140:0/4145795977 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1380b59d0 0x7fe1380b07e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:30.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 -- 172.21.15.140:0/4145795977 >> 172.21.15.140:0/4145795977 conn(0x7fe1380634c0 msgr2=0x7fe1380b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:30.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 -- 172.21.15.140:0/4145795977 shutdown_connections 2022-07-21T01:48:30.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 -- 172.21.15.140:0/4145795977 wait complete. 2022-07-21T01:48:30.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 Processor -- start 2022-07-21T01:48:30.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 -- start start 2022-07-21T01:48:30.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1380b14f0 0x7fe1380645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:30.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1380b59d0 0x7fe1381116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:30.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe13814ada0 0x7fe138111bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:30.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe138066960 con 0x7fe1380b14f0 2022-07-21T01:48:30.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13810e480 con 0x7fe1380b59d0 2022-07-21T01:48:30.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13e5f8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13810e720 con 0x7fe13814ada0 2022-07-21T01:48:30.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13d5f6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1380b14f0 0x7fe1380645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:30.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13d5f6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1380b14f0 0x7fe1380645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60432/0 (socket says 172.21.15.140:60432) 2022-07-21T01:48:30.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13d5f6700 1 -- 172.21.15.140:0/1335472269 learned_addr learned my addr 172.21.15.140:0/1335472269 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:30.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13ddf7700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe13814ada0 0x7fe138111bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:30.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13cdf5700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1380b59d0 0x7fe1381116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:30.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13d5f6700 1 -- 172.21.15.140:0/1335472269 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe13814ada0 msgr2=0x7fe138111bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:30.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.235+0000 7fe13d5f6700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe13814ada0 0x7fe138111bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:30.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe13d5f6700 1 -- 172.21.15.140:0/1335472269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1380b59d0 msgr2=0x7fe1381116e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:30.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe13d5f6700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1380b59d0 0x7fe1381116e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:30.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe13d5f6700 1 -- 172.21.15.140:0/1335472269 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13000b180 con 0x7fe1380b14f0 2022-07-21T01:48:30.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe13d5f6700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1380b14f0 0x7fe1380645d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe120002f70 tx=0x7fe120015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:30.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe1367fc700 1 -- 172.21.15.140:0/1335472269 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1200068b0 con 0x7fe1380b14f0 2022-07-21T01:48:30.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe1367fc700 1 -- 172.21.15.140:0/1335472269 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12000d8f0 con 0x7fe1380b14f0 2022-07-21T01:48:30.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe1367fc700 1 -- 172.21.15.140:0/1335472269 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe120011d70 con 0x7fe1380b14f0 2022-07-21T01:48:30.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe13e5f8700 1 -- 172.21.15.140:0/1335472269 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12000b1e0 con 0x7fe1380b14f0 2022-07-21T01:48:30.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe13e5f8700 1 -- 172.21.15.140:0/1335472269 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13810f100 con 0x7fe1380b14f0 2022-07-21T01:48:30.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe13e5f8700 1 -- 172.21.15.140:0/1335472269 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13810e8e0 con 0x7fe1380b14f0 2022-07-21T01:48:30.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe1367fc700 1 -- 172.21.15.140:0/1335472269 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe12001a040 con 0x7fe1380b14f0 2022-07-21T01:48:30.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe1367fc700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe11c053aa0 0x7fe11c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:30.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe1367fc700 1 -- 172.21.15.140:0/1335472269 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fe12006cb30 con 0x7fe1380b14f0 2022-07-21T01:48:30.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.239+0000 7fe13cdf5700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe11c053aa0 0x7fe11c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:30.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.243+0000 7fe13cdf5700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe11c053aa0 0x7fe11c055f60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fe12800d000 tx=0x7fe128007ef0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:30.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.247+0000 7fe1367fc700 1 -- 172.21.15.140:0/1335472269 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe13810e8e0 con 0x7fe1380b14f0 2022-07-21T01:48:30.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:30.371+0000 7fe13e5f8700 1 -- 172.21.15.140:0/1335472269 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fe13810e8e0 con 0x7fe1380b14f0 2022-07-21T01:48:30.609 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:48:30.612 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:48:30.615 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:48:30.619 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:48:30.622 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:48:31.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.099+0000 7fe1367fc700 1 -- 172.21.15.140:0/1335472269 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v20) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe13810e8e0 con 0x7fe1380b14f0 2022-07-21T01:48:31.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.107+0000 7fe13e5f8700 1 -- 172.21.15.140:0/1335472269 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe11c053aa0 msgr2=0x7fe11c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.107+0000 7fe13e5f8700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe11c053aa0 0x7fe11c055f60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fe12800d000 tx=0x7fe128007ef0).stop 2022-07-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.107+0000 7fe13e5f8700 1 -- 172.21.15.140:0/1335472269 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1380b14f0 msgr2=0x7fe1380645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.107+0000 7fe13e5f8700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1380b14f0 0x7fe1380645d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe120002f70 tx=0x7fe120015000).stop 2022-07-21T01:48:31.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.107+0000 7fe13e5f8700 1 -- 172.21.15.140:0/1335472269 shutdown_connections 2022-07-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.107+0000 7fe13e5f8700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe11c053aa0 0x7fe11c055f60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.107+0000 7fe13e5f8700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe13814ada0 0x7fe138111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.107+0000 7fe13e5f8700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1380b59d0 0x7fe1381116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.107+0000 7fe13e5f8700 1 --2- 172.21.15.140:0/1335472269 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1380b14f0 0x7fe1380645d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.107+0000 7fe13e5f8700 1 -- 172.21.15.140:0/1335472269 >> 172.21.15.140:0/1335472269 conn(0x7fe1380634c0 msgr2=0x7fe138145170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.111+0000 7fe13e5f8700 1 -- 172.21.15.140:0/1335472269 shutdown_connections 2022-07-21T01:48:31.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.111+0000 7fe13e5f8700 1 -- 172.21.15.140:0/1335472269 wait complete. 2022-07-21T01:48:31.112 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:48:31.122 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:48:31.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.231+0000 7fa6c48bb700 1 Processor -- start 2022-07-21T01:48:31.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.231+0000 7fa6c48bb700 1 -- start start 2022-07-21T01:48:31.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6c48bb700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa6c00b1090 0x7fa6c013c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6c48bb700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c005d060 con 0x7fa6c00b1b00 2022-07-21T01:48:31.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6c48bb700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c005ca80 con 0x7fa6c0142a10 2022-07-21T01:48:31.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6c48bb700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c0067450 con 0x7fa6c00b1090 2022-07-21T01:48:31.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6bf59e700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa6c00b1b00 0x7fa6c014d380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46356/0 (socket says 172.21.15.140:46356) 2022-07-21T01:48:31.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6bed9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa6c00b1090 0x7fa6c013c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6bf59e700 1 -- 172.21.15.140:0/1001592361 learned_addr learned my addr 172.21.15.140:0/1001592361 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:31.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6bdd9b700 1 -- 172.21.15.140:0/1001592361 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750347587 0 0) 0x7fa6c005ca80 con 0x7fa6c0142a10 2022-07-21T01:48:31.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6bed9d700 1 -- 172.21.15.140:0/1001592361 >> v1:172.21.15.152:6789/0 conn(0x7fa6c0142a10 legacy=0x7fa6c0146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6bed9d700 1 -- 172.21.15.140:0/1001592361 >> v1:172.21.15.140:6789/0 conn(0x7fa6c00b1b00 legacy=0x7fa6c014d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6bed9d700 1 -- 172.21.15.140:0/1001592361 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c00676f0 con 0x7fa6c00b1090 2022-07-21T01:48:31.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6bed9d700 1 --2- 172.21.15.140:0/1001592361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6c00b1090 0x7fa6c013c610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fa6b8001490 tx=0x7fa6b0006df0).ready entity=mon.0 client_cookie=7fd54ce41a02f2ec server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:31.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6bdd9b700 1 -- 172.21.15.140:0/1001592361 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6b000d5c0 con 0x7fa6c00b1090 2022-07-21T01:48:31.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6bdd9b700 1 -- 172.21.15.140:0/1001592361 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b0000e90 con 0x7fa6c00b1090 2022-07-21T01:48:31.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6bdd9b700 1 -- 172.21.15.140:0/1001592361 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6b000d910 con 0x7fa6c00b1090 2022-07-21T01:48:31.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6c48bb700 1 -- 172.21.15.140:0/1001592361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6c00b1090 msgr2=0x7fa6c013c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6c48bb700 1 --2- 172.21.15.140:0/1001592361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6c00b1090 0x7fa6c013c610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fa6b8001490 tx=0x7fa6b0006df0).stop 2022-07-21T01:48:31.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6c48bb700 1 -- 172.21.15.140:0/1001592361 shutdown_connections 2022-07-21T01:48:31.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6c48bb700 1 --2- 172.21.15.140:0/1001592361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6c00b1090 0x7fa6c013c610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6c48bb700 1 -- 172.21.15.140:0/1001592361 >> 172.21.15.140:0/1001592361 conn(0x7fa6c0064c00 msgr2=0x7fa6c0064fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:31.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6c48bb700 1 -- 172.21.15.140:0/1001592361 shutdown_connections 2022-07-21T01:48:31.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.235+0000 7fa6c48bb700 1 -- 172.21.15.140:0/1001592361 wait complete. 2022-07-21T01:48:31.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.239+0000 7fa6c48bb700 1 Processor -- start 2022-07-21T01:48:31.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.239+0000 7fa6c48bb700 1 -- start start 2022-07-21T01:48:31.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.239+0000 7fa6c48bb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6c00b1090 0x7fa6c0155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.239+0000 7fa6c48bb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6c00b1b00 0x7fa6c0144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.239+0000 7fa6c48bb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c00b66e0 0x7fa6c0110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.239+0000 7fa6c48bb700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c00b1fb0 con 0x7fa6c00b1090 2022-07-21T01:48:31.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.239+0000 7fa6c48bb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c0144f40 con 0x7fa6c00b66e0 2022-07-21T01:48:31.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.239+0000 7fa6c48bb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c0141fb0 con 0x7fa6c00b1b00 2022-07-21T01:48:31.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6bed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6c00b1090 0x7fa6c0155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6bf59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c00b66e0 0x7fa6c0110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6be59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6c00b1b00 0x7fa6c0144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6bed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6c00b1090 0x7fa6c0155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60448/0 (socket says 172.21.15.140:60448) 2022-07-21T01:48:31.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6bed9d700 1 -- 172.21.15.140:0/708305582 learned_addr learned my addr 172.21.15.140:0/708305582 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:31.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6bed9d700 1 -- 172.21.15.140:0/708305582 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6c00b1b00 msgr2=0x7fa6c0144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6bed9d700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6c00b1b00 0x7fa6c0144360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6bed9d700 1 -- 172.21.15.140:0/708305582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c00b66e0 msgr2=0x7fa6c0110550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6bed9d700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c00b66e0 0x7fa6c0110550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6bed9d700 1 -- 172.21.15.140:0/708305582 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b000b180 con 0x7fa6c00b1090 2022-07-21T01:48:31.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6bf59e700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c00b66e0 0x7fa6c0110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:31.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6bed9d700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6c00b1090 0x7fa6c0155240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa6b8001490 tx=0x7fa6b0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:31.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6b7fff700 1 -- 172.21.15.140:0/708305582 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6b0016070 con 0x7fa6c00b1090 2022-07-21T01:48:31.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6b7fff700 1 -- 172.21.15.140:0/708305582 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b0002f50 con 0x7fa6c00b1090 2022-07-21T01:48:31.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6c48bb700 1 -- 172.21.15.140:0/708305582 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c0142250 con 0x7fa6c00b1090 2022-07-21T01:48:31.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6b7fff700 1 -- 172.21.15.140:0/708305582 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6b0013de0 con 0x7fa6c00b1090 2022-07-21T01:48:31.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6c48bb700 1 -- 172.21.15.140:0/708305582 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c00b62d0 con 0x7fa6c00b1090 2022-07-21T01:48:31.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.243+0000 7fa6c48bb700 1 -- 172.21.15.140:0/708305582 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c0110350 con 0x7fa6c00b1090 2022-07-21T01:48:31.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.247+0000 7fa6b7fff700 1 -- 172.21.15.140:0/708305582 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa6b000f030 con 0x7fa6c00b1090 2022-07-21T01:48:31.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.247+0000 7fa6b7fff700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6a4053b00 0x7fa6a4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.247+0000 7fa6be59c700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6a4053b00 0x7fa6a4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.247+0000 7fa6b7fff700 1 -- 172.21.15.140:0/708305582 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fa6b006cd20 con 0x7fa6c00b1090 2022-07-21T01:48:31.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.247+0000 7fa6be59c700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6a4053b00 0x7fa6a4055fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa6a800d000 tx=0x7fa6a8007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:31.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.255+0000 7fa6b7fff700 1 -- 172.21.15.140:0/708305582 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa6b0043b20 con 0x7fa6c00b1090 2022-07-21T01:48:31.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.419+0000 7fa6c48bb700 1 -- 172.21.15.140:0/708305582 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fa6c0143b40 con 0x7fa6c00b1090 2022-07-21T01:48:31.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.427+0000 7fa6b7fff700 1 -- 172.21.15.140:0/708305582 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7fa6b0023020 con 0x7fa6c00b1090 2022-07-21T01:48:31.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.435+0000 7fa6c48bb700 1 -- 172.21.15.140:0/708305582 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6a4053b00 msgr2=0x7fa6a4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.435+0000 7fa6c48bb700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6a4053b00 0x7fa6a4055fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa6a800d000 tx=0x7fa6a8007b70).stop 2022-07-21T01:48:31.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.435+0000 7fa6c48bb700 1 -- 172.21.15.140:0/708305582 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6c00b1090 msgr2=0x7fa6c0155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.435+0000 7fa6c48bb700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6c00b1090 0x7fa6c0155240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fa6b8001490 tx=0x7fa6b0014000).stop 2022-07-21T01:48:31.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.435+0000 7fa6c48bb700 1 -- 172.21.15.140:0/708305582 shutdown_connections 2022-07-21T01:48:31.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.435+0000 7fa6c48bb700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6a4053b00 0x7fa6a4055fc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.435+0000 7fa6c48bb700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6c00b66e0 0x7fa6c0110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.435+0000 7fa6c48bb700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa6c00b1b00 0x7fa6c0144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.435+0000 7fa6c48bb700 1 --2- 172.21.15.140:0/708305582 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa6c00b1090 0x7fa6c0155240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.435+0000 7fa6c48bb700 1 -- 172.21.15.140:0/708305582 >> 172.21.15.140:0/708305582 conn(0x7fa6c0064c00 msgr2=0x7fa6c013fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:31.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.435+0000 7fa6c48bb700 1 -- 172.21.15.140:0/708305582 shutdown_connections 2022-07-21T01:48:31.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.439+0000 7fa6c48bb700 1 -- 172.21.15.140:0/708305582 wait complete. 2022-07-21T01:48:31.441 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 3 and data pool 4 2022-07-21T01:48:31.452 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:31.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d2b0c3700 1 Processor -- start 2022-07-21T01:48:31.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d2b0c3700 1 -- start start 2022-07-21T01:48:31.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d2b0c3700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d240afa10 0x7f9d240b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d2b0c3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d240b3de0 0x7f9d241452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d2b0c3700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d240b42a0 con 0x7f9d240608b0 2022-07-21T01:48:31.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d2b0c3700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d24060c90 con 0x7f9d240afa10 2022-07-21T01:48:31.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d2b0c3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240af410 con 0x7f9d240b3de0 2022-07-21T01:48:31.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d23fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d240afa10 0x7f9d240b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d29660700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d240b3de0 0x7f9d241452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d29660700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d240b3de0 0x7f9d241452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:57188/0 (socket says 172.21.15.140:57188) 2022-07-21T01:48:31.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d23fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d240afa10 0x7f9d240b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:37962/0 (socket says 172.21.15.140:37962) 2022-07-21T01:48:31.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d28e5f700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9d240608b0 0x7f9d2405bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:47482/0 (socket says 172.21.15.140:47482) 2022-07-21T01:48:31.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d29660700 1 -- 172.21.15.140:0/2229068737 learned_addr learned my addr 172.21.15.140:0/2229068737 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:31.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d23fff700 1 -- 172.21.15.140:0/2229068737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d240b3de0 msgr2=0x7f9d241452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:31.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d23fff700 1 --2- 172.21.15.140:0/2229068737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d240b3de0 0x7f9d241452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d23fff700 1 -- 172.21.15.140:0/2229068737 >> v1:172.21.15.152:6789/0 conn(0x7f9d240608b0 legacy=0x7f9d2405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d23fff700 1 -- 172.21.15.140:0/2229068737 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d240af6b0 con 0x7f9d240afa10 2022-07-21T01:48:31.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d23fff700 1 --2- 172.21.15.140:0/2229068737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d240afa10 0x7f9d240b46d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f9d140032f0 tx=0x7f9d140091d0).ready entity=mon.1 client_cookie=c03d7e10894c37f3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:31.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d237fe700 1 -- 172.21.15.140:0/2229068737 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d1400b6c0 con 0x7f9d240afa10 2022-07-21T01:48:31.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d237fe700 1 -- 172.21.15.140:0/2229068737 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1400b820 con 0x7f9d240afa10 2022-07-21T01:48:31.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d237fe700 1 -- 172.21.15.140:0/2229068737 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d1400b990 con 0x7f9d240afa10 2022-07-21T01:48:31.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/2229068737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d240afa10 msgr2=0x7f9d240b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.547+0000 7f9d2b0c3700 1 --2- 172.21.15.140:0/2229068737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d240afa10 0x7f9d240b46d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f9d140032f0 tx=0x7f9d140091d0).stop 2022-07-21T01:48:31.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/2229068737 shutdown_connections 2022-07-21T01:48:31.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 --2- 172.21.15.140:0/2229068737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d240b3de0 0x7f9d241452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 --2- 172.21.15.140:0/2229068737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d240afa10 0x7f9d240b46d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/2229068737 >> 172.21.15.140:0/2229068737 conn(0x7f9d240b5ee0 msgr2=0x7f9d240b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:31.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/2229068737 shutdown_connections 2022-07-21T01:48:31.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/2229068737 wait complete. 2022-07-21T01:48:31.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 Processor -- start 2022-07-21T01:48:31.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 -- start start 2022-07-21T01:48:31.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d240608b0 0x7f9d24156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d240afa10 0x7f9d24093230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d240b3de0 0x7f9d241442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24093720 con 0x7f9d240608b0 2022-07-21T01:48:31.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d241556a0 con 0x7f9d240afa10 2022-07-21T01:48:31.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d2b0c3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24155940 con 0x7f9d240b3de0 2022-07-21T01:48:31.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d28e5f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d240608b0 0x7f9d24156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d28e5f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d240608b0 0x7f9d24156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60460/0 (socket says 172.21.15.140:60460) 2022-07-21T01:48:31.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d28e5f700 1 -- 172.21.15.140:0/3516045507 learned_addr learned my addr 172.21.15.140:0/3516045507 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:31.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d23fff700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d240afa10 0x7f9d24093230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d29660700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d240b3de0 0x7f9d241442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d28e5f700 1 -- 172.21.15.140:0/3516045507 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d240b3de0 msgr2=0x7f9d241442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d28e5f700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d240b3de0 0x7f9d241442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d28e5f700 1 -- 172.21.15.140:0/3516045507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d240afa10 msgr2=0x7f9d24093230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d28e5f700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d240afa10 0x7f9d24093230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d28e5f700 1 -- 172.21.15.140:0/3516045507 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d14009470 con 0x7f9d240608b0 2022-07-21T01:48:31.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d23fff700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d240afa10 0x7f9d24093230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:31.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.551+0000 7f9d28e5f700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d240608b0 0x7f9d24156080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f9d2406ad80 tx=0x7f9d0c007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:31.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.555+0000 7f9d21ffb700 1 -- 172.21.15.140:0/3516045507 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d0c007170 con 0x7f9d240608b0 2022-07-21T01:48:31.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.555+0000 7f9d21ffb700 1 -- 172.21.15.140:0/3516045507 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c0072d0 con 0x7f9d240608b0 2022-07-21T01:48:31.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.555+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/3516045507 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d24155be0 con 0x7f9d240608b0 2022-07-21T01:48:31.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.555+0000 7f9d21ffb700 1 -- 172.21.15.140:0/3516045507 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d0c004710 con 0x7f9d240608b0 2022-07-21T01:48:31.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.555+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/3516045507 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d241449d0 con 0x7f9d240608b0 2022-07-21T01:48:31.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.555+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/3516045507 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d240b3880 con 0x7f9d240608b0 2022-07-21T01:48:31.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.555+0000 7f9d21ffb700 1 -- 172.21.15.140:0/3516045507 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9d0c004870 con 0x7f9d240608b0 2022-07-21T01:48:31.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.555+0000 7f9d21ffb700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9d08053a70 0x7f9d08055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.555+0000 7f9d21ffb700 1 -- 172.21.15.140:0/3516045507 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9d0c06b800 con 0x7f9d240608b0 2022-07-21T01:48:31.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.555+0000 7f9d23fff700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9d08053a70 0x7f9d08055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.559+0000 7f9d23fff700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9d08053a70 0x7f9d08055f30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f9d1401b000 tx=0x7f9d14016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:31.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.563+0000 7f9d21ffb700 1 -- 172.21.15.140:0/3516045507 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9d240b3880 con 0x7f9d240608b0 2022-07-21T01:48:31.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.683+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/3516045507 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d240b3880 con 0x7f9d240608b0 2022-07-21T01:48:31.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d21ffb700 1 -- 172.21.15.140:0/3516045507 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16147 (secure 0 0 0) 0x7f9d240b3880 con 0x7f9d240608b0 2022-07-21T01:48:31.687 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:31.687 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":21,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:31.422698+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:31.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/3516045507 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9d08053a70 msgr2=0x7f9d08055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9d08053a70 0x7f9d08055f30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f9d1401b000 tx=0x7f9d14016000).stop 2022-07-21T01:48:31.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/3516045507 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d240608b0 msgr2=0x7f9d24156080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d240608b0 0x7f9d24156080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f9d2406ad80 tx=0x7f9d0c007bd0).stop 2022-07-21T01:48:31.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/3516045507 shutdown_connections 2022-07-21T01:48:31.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9d08053a70 0x7f9d08055f30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d240b3de0 0x7f9d241442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d240afa10 0x7f9d24093230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 --2- 172.21.15.140:0/3516045507 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d240608b0 0x7f9d24156080 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/3516045507 >> 172.21.15.140:0/3516045507 conn(0x7f9d240b5ee0 msgr2=0x7f9d24145610 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:31.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/3516045507 shutdown_connections 2022-07-21T01:48:31.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.687+0000 7f9d2b0c3700 1 -- 172.21.15.140:0/3516045507 wait complete. 2022-07-21T01:48:31.705 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:31.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.807+0000 7fdbd842c700 1 Processor -- start 2022-07-21T01:48:31.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.807+0000 7fdbd842c700 1 -- start start 2022-07-21T01:48:31.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd842c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdbd009b8e0 0x7fdbd00ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd842c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdbd00ad350 0x7fdbd00abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd842c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd00afb60 con 0x7fdbd00b1480 2022-07-21T01:48:31.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd842c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd01411b0 con 0x7fdbd00ad350 2022-07-21T01:48:31.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd842c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd00af130 con 0x7fdbd009b8e0 2022-07-21T01:48:31.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd61c8700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fdbd00b1480 0x7fdbd0140aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46382/0 (socket says 172.21.15.140:46382) 2022-07-21T01:48:31.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd61c8700 1 -- 172.21.15.140:0/1564112275 learned_addr learned my addr 172.21.15.140:0/1564112275 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:31.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd69c9700 1 --2- 172.21.15.140:0/1564112275 >> v2:172.21.15.140:3300/0 conn(0x7fdbd00ad350 0x7fdbd00abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd59c7700 1 --2- 172.21.15.140:0/1564112275 >> v2:172.21.15.152:3300/0 conn(0x7fdbd009b8e0 0x7fdbd00ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946146643 0 0) 0x7fdbd00afb60 con 0x7fdbd00b1480 2022-07-21T01:48:31.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb8003260 con 0x7fdbd00b1480 2022-07-21T01:48:31.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1247611297 0 0) 0x7fdbb8003260 con 0x7fdbd00b1480 2022-07-21T01:48:31.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 >> v2:172.21.15.152:3300/0 conn(0x7fdbd009b8e0 msgr2=0x7fdbd00ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:31.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 --2- 172.21.15.140:0/1564112275 >> v2:172.21.15.152:3300/0 conn(0x7fdbd009b8e0 0x7fdbd00ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 >> v2:172.21.15.140:3300/0 conn(0x7fdbd00ad350 msgr2=0x7fdbd00abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:31.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 --2- 172.21.15.140:0/1564112275 >> v2:172.21.15.140:3300/0 conn(0x7fdbd00ad350 0x7fdbd00abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00af3d0 con 0x7fdbd00b1480 2022-07-21T01:48:31.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fdbc80019e0 con 0x7fdbd00b1480 2022-07-21T01:48:31.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 >> v1:172.21.15.140:6789/0 conn(0x7fdbd00b1480 legacy=0x7fdbd0140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 --2- 172.21.15.140:0/1564112275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbb80037d0 0x7fdbb8003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 --2- 172.21.15.140:0/1564112275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbb8004500 0x7fdbb8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 --2- 172.21.15.140:0/1564112275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbb8004e70 0x7fdbb800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01411b0 con 0x7fdbb8004e70 2022-07-21T01:48:31.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00af130 con 0x7fdbb8004500 2022-07-21T01:48:31.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb800bd70 con 0x7fdbb80037d0 2022-07-21T01:48:31.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd59c7700 1 --2- 172.21.15.140:0/1564112275 >> v2:172.21.15.152:3300/0 conn(0x7fdbd009b8e0 0x7fdbd00ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:31.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd69c9700 1 --2- 172.21.15.140:0/1564112275 >> v2:172.21.15.140:3300/0 conn(0x7fdbd00ad350 0x7fdbd00abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:31.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd61c8700 1 --2- 172.21.15.140:0/1564112275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbb80037d0 0x7fdbb8003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd69c9700 1 --2- 172.21.15.140:0/1564112275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbb8004e70 0x7fdbb800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd59c7700 1 --2- 172.21.15.140:0/1564112275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbb8004500 0x7fdbb8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd61c8700 1 -- 172.21.15.140:0/1564112275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbb8004500 msgr2=0x7fdbb8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:31.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd61c8700 1 --2- 172.21.15.140:0/1564112275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbb8004500 0x7fdbb8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd61c8700 1 -- 172.21.15.140:0/1564112275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbb8004e70 msgr2=0x7fdbb800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:31.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd61c8700 1 --2- 172.21.15.140:0/1564112275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbb8004e70 0x7fdbb800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd61c8700 1 -- 172.21.15.140:0/1564112275 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb8003260 con 0x7fdbb80037d0 2022-07-21T01:48:31.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd69c9700 1 -- 172.21.15.140:0/1564112275 reap_dead start 2022-07-21T01:48:31.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd61c8700 1 --2- 172.21.15.140:0/1564112275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbb80037d0 0x7fdbb8003bb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fdbb80054d0 tx=0x7fdbc8008dc0).ready entity=mon.2 client_cookie=49ab00ac4cc787e1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:31.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbc8005990 con 0x7fdbb80037d0 2022-07-21T01:48:31.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc8005af0 con 0x7fdbb80037d0 2022-07-21T01:48:31.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.811+0000 7fdbd51c6700 1 -- 172.21.15.140:0/1564112275 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbc8005e50 con 0x7fdbb80037d0 2022-07-21T01:48:31.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.815+0000 7fdbd842c700 1 -- 172.21.15.140:0/1564112275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbb80037d0 msgr2=0x7fdbb8003bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.815+0000 7fdbd842c700 1 --2- 172.21.15.140:0/1564112275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbb80037d0 0x7fdbb8003bb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fdbb80054d0 tx=0x7fdbc8008dc0).stop 2022-07-21T01:48:31.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.815+0000 7fdbd842c700 1 -- 172.21.15.140:0/1564112275 shutdown_connections 2022-07-21T01:48:31.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.815+0000 7fdbd842c700 1 --2- 172.21.15.140:0/1564112275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbb80037d0 0x7fdbb8003bb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.815+0000 7fdbd842c700 1 -- 172.21.15.140:0/1564112275 >> 172.21.15.140:0/1564112275 conn(0x7fdbd00b23d0 msgr2=0x7fdbd00b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:31.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.815+0000 7fdbd842c700 1 -- 172.21.15.140:0/1564112275 shutdown_connections 2022-07-21T01:48:31.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.815+0000 7fdbd842c700 1 -- 172.21.15.140:0/1564112275 wait complete. 2022-07-21T01:48:31.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.815+0000 7fdbd842c700 1 Processor -- start 2022-07-21T01:48:31.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.815+0000 7fdbd842c700 1 -- start start 2022-07-21T01:48:31.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd842c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbb80037d0 0x7fdbd00b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd842c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd0091410 0x7fdbd013faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd842c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd0143890 0x7fdbd013ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd842c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd009aee0 con 0x7fdbd0143890 2022-07-21T01:48:31.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd842c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd009b180 con 0x7fdbb80037d0 2022-07-21T01:48:31.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd842c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd0143e80 con 0x7fdbd0091410 2022-07-21T01:48:31.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd69c9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd0143890 0x7fdbd013ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd69c9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd0143890 0x7fdbd013ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60484/0 (socket says 172.21.15.140:60484) 2022-07-21T01:48:31.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd69c9700 1 -- 172.21.15.140:0/2505043085 learned_addr learned my addr 172.21.15.140:0/2505043085 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:31.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd61c8700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbb80037d0 0x7fdbd00b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd59c7700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd0091410 0x7fdbd013faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd69c9700 1 -- 172.21.15.140:0/2505043085 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd0091410 msgr2=0x7fdbd013faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd69c9700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd0091410 0x7fdbd013faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd69c9700 1 -- 172.21.15.140:0/2505043085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbb80037d0 msgr2=0x7fdbd00b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd69c9700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbb80037d0 0x7fdbd00b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd69c9700 1 -- 172.21.15.140:0/2505043085 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd0140480 con 0x7fdbd0143890 2022-07-21T01:48:31.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd59c7700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd0091410 0x7fdbd013faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:31.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd69c9700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd0143890 0x7fdbd013ff90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fdbcc00b000 tx=0x7fdbcc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:31.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbc37fe700 1 -- 172.21.15.140:0/2505043085 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbcc019070 con 0x7fdbd0143890 2022-07-21T01:48:31.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbc37fe700 1 -- 172.21.15.140:0/2505043085 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc000e90 con 0x7fdbd0143890 2022-07-21T01:48:31.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd842c700 1 -- 172.21.15.140:0/2505043085 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd01532a0 con 0x7fdbd0143890 2022-07-21T01:48:31.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbc37fe700 1 -- 172.21.15.140:0/2505043085 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbcc014840 con 0x7fdbd0143890 2022-07-21T01:48:31.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.819+0000 7fdbd842c700 1 -- 172.21.15.140:0/2505043085 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd0153750 con 0x7fdbd0143890 2022-07-21T01:48:31.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.823+0000 7fdbd842c700 1 -- 172.21.15.140:0/2505043085 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd00917f0 con 0x7fdbd0143890 2022-07-21T01:48:31.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.823+0000 7fdbc37fe700 1 -- 172.21.15.140:0/2505043085 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdbcc0149a0 con 0x7fdbd0143890 2022-07-21T01:48:31.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.823+0000 7fdbc37fe700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb0053a70 0x7fdbb0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:31.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.823+0000 7fdbc37fe700 1 -- 172.21.15.140:0/2505043085 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdbcc06ba80 con 0x7fdbd0143890 2022-07-21T01:48:31.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.823+0000 7fdbd61c8700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb0053a70 0x7fdbb0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:31.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.823+0000 7fdbd61c8700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb0053a70 0x7fdbb0055f30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fdbc801c000 tx=0x7fdbc8016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:31.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.827+0000 7fdbc37fe700 1 -- 172.21.15.140:0/2505043085 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdbd00917f0 con 0x7fdbd0143890 2022-07-21T01:48:31.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.951+0000 7fdbd842c700 1 -- 172.21.15.140:0/2505043085 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbd00917f0 con 0x7fdbd0143890 2022-07-21T01:48:31.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbc37fe700 1 -- 172.21.15.140:0/2505043085 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16147 (secure 0 0 0) 0x7fdbcc027f30 con 0x7fdbd0143890 2022-07-21T01:48:31.954 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:31.954 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":21,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:31.422698+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:31.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 -- 172.21.15.140:0/2505043085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb0053a70 msgr2=0x7fdbb0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb0053a70 0x7fdbb0055f30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fdbc801c000 tx=0x7fdbc8016000).stop 2022-07-21T01:48:31.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 -- 172.21.15.140:0/2505043085 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd0143890 msgr2=0x7fdbd013ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:31.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd0143890 0x7fdbd013ff90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fdbcc00b000 tx=0x7fdbcc006df0).stop 2022-07-21T01:48:31.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 -- 172.21.15.140:0/2505043085 shutdown_connections 2022-07-21T01:48:31.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb0053a70 0x7fdbb0055f30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd0143890 0x7fdbd013ff90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd0091410 0x7fdbd013faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 --2- 172.21.15.140:0/2505043085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbb80037d0 0x7fdbd00b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:31.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 -- 172.21.15.140:0/2505043085 >> 172.21.15.140:0/2505043085 conn(0x7fdbd00b23d0 msgr2=0x7fdbd00abf40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:31.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 -- 172.21.15.140:0/2505043085 shutdown_connections 2022-07-21T01:48:31.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:31.955+0000 7fdbd842c700 1 -- 172.21.15.140:0/2505043085 wait complete. 2022-07-21T01:48:31.967 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:48:32.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 Processor -- start 2022-07-21T01:48:32.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 -- start start 2022-07-21T01:48:32.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3a280b3f30 0x7f3a280b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a28065db0 con 0x7f3a2805c0b0 2022-07-21T01:48:32.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a280b4310 con 0x7f3a28066210 2022-07-21T01:48:32.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2805d180 con 0x7f3a280b3f30 2022-07-21T01:48:32.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a1e7fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f3a2805c0b0 0x7f3a2814a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46402/0 (socket says 172.21.15.140:46402) 2022-07-21T01:48:32.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a1e7fc700 1 -- 172.21.15.140:0/780533309 learned_addr learned my addr 172.21.15.140:0/780533309 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:32.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a26ffd700 1 --2- 172.21.15.140:0/780533309 >> v2:172.21.15.152:3300/0 conn(0x7f3a280b3f30 0x7f3a280b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a267fc700 1 -- 172.21.15.140:0/780533309 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831906527 0 0) 0x7f3a28065db0 con 0x7f3a2805c0b0 2022-07-21T01:48:32.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a267fc700 1 -- 172.21.15.140:0/780533309 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a08003260 con 0x7f3a2805c0b0 2022-07-21T01:48:32.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a267fc700 1 -- 172.21.15.140:0/780533309 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3461010418 0 0) 0x7f3a08003260 con 0x7f3a2805c0b0 2022-07-21T01:48:32.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a26ffd700 1 -- 172.21.15.140:0/780533309 >> v1:172.21.15.152:6789/0 conn(0x7f3a28066210 legacy=0x7f3a28146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a26ffd700 1 -- 172.21.15.140:0/780533309 >> v1:172.21.15.140:6789/0 conn(0x7f3a2805c0b0 legacy=0x7f3a2814a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a26ffd700 1 -- 172.21.15.140:0/780533309 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2805cee0 con 0x7f3a280b3f30 2022-07-21T01:48:32.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a26ffd700 1 --2- 172.21.15.140:0/780533309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a280b3f30 0x7f3a280b24f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3a10001820 tx=0x7f3a18006df0).ready entity=mon.1 client_cookie=29f40294fddbac67 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:32.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a267fc700 1 -- 172.21.15.140:0/780533309 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a10001680 con 0x7f3a280b3f30 2022-07-21T01:48:32.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a267fc700 1 -- 172.21.15.140:0/780533309 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a18000e90 con 0x7f3a280b3f30 2022-07-21T01:48:32.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a267fc700 1 -- 172.21.15.140:0/780533309 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a1800d4f0 con 0x7f3a280b3f30 2022-07-21T01:48:32.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 -- 172.21.15.140:0/780533309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a280b3f30 msgr2=0x7f3a280b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 --2- 172.21.15.140:0/780533309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a280b3f30 0x7f3a280b24f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3a10001820 tx=0x7f3a18006df0).stop 2022-07-21T01:48:32.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 -- 172.21.15.140:0/780533309 shutdown_connections 2022-07-21T01:48:32.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 --2- 172.21.15.140:0/780533309 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a280b3f30 0x7f3a280b24f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 -- 172.21.15.140:0/780533309 >> 172.21.15.140:0/780533309 conn(0x7f3a28064c20 msgr2=0x7f3a28065000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:32.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 -- 172.21.15.140:0/780533309 shutdown_connections 2022-07-21T01:48:32.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.079+0000 7f3a27fff700 1 -- 172.21.15.140:0/780533309 wait complete. 2022-07-21T01:48:32.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a27fff700 1 Processor -- start 2022-07-21T01:48:32.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a27fff700 1 -- start start 2022-07-21T01:48:32.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a27fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a2805c0b0 0x7f3a28066fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a27fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a28066210 0x7f3a28159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a27fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a280b3f30 0x7f3a281561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a27fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28110bf0 con 0x7f3a2805c0b0 2022-07-21T01:48:32.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a27fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28112440 con 0x7f3a280b3f30 2022-07-21T01:48:32.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a27fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28159860 con 0x7f3a28066210 2022-07-21T01:48:32.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a26ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a2805c0b0 0x7f3a28066fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a26ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a2805c0b0 0x7f3a28066fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60494/0 (socket says 172.21.15.140:60494) 2022-07-21T01:48:32.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a26ffd700 1 -- 172.21.15.140:0/656922311 learned_addr learned my addr 172.21.15.140:0/656922311 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:32.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a1e7fc700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a28066210 0x7f3a28159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a277fe700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a280b3f30 0x7f3a281561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a26ffd700 1 -- 172.21.15.140:0/656922311 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a28066210 msgr2=0x7f3a28159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a26ffd700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a28066210 0x7f3a28159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a26ffd700 1 -- 172.21.15.140:0/656922311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a280b3f30 msgr2=0x7f3a281561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a26ffd700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a280b3f30 0x7f3a281561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a26ffd700 1 -- 172.21.15.140:0/656922311 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1800b180 con 0x7f3a2805c0b0 2022-07-21T01:48:32.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a26ffd700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a2805c0b0 0x7f3a28066fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3a10001820 tx=0x7f3a18014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:32.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a24ff9700 1 -- 172.21.15.140:0/656922311 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a18016070 con 0x7f3a2805c0b0 2022-07-21T01:48:32.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a24ff9700 1 -- 172.21.15.140:0/656922311 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a18002f50 con 0x7f3a2805c0b0 2022-07-21T01:48:32.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a27fff700 1 -- 172.21.15.140:0/656922311 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a28159b00 con 0x7f3a2805c0b0 2022-07-21T01:48:32.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a24ff9700 1 -- 172.21.15.140:0/656922311 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a1800d4f0 con 0x7f3a2805c0b0 2022-07-21T01:48:32.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.083+0000 7f3a27fff700 1 -- 172.21.15.140:0/656922311 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a280aee50 con 0x7f3a2805c0b0 2022-07-21T01:48:32.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.087+0000 7f3a27fff700 1 -- 172.21.15.140:0/656922311 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2805d430 con 0x7f3a2805c0b0 2022-07-21T01:48:32.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.087+0000 7f3a24ff9700 1 -- 172.21.15.140:0/656922311 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3a18025030 con 0x7f3a2805c0b0 2022-07-21T01:48:32.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.087+0000 7f3a24ff9700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a0c053b00 0x7f3a0c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.087+0000 7f3a1e7fc700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a0c053b00 0x7f3a0c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.087+0000 7f3a24ff9700 1 -- 172.21.15.140:0/656922311 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3a1806ce70 con 0x7f3a2805c0b0 2022-07-21T01:48:32.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.087+0000 7f3a1e7fc700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a0c053b00 0x7f3a0c055fc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3a1000b000 tx=0x7f3a10000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:32.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.095+0000 7f3a24ff9700 1 -- 172.21.15.140:0/656922311 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a18043dd0 con 0x7f3a2805c0b0 2022-07-21T01:48:32.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.259+0000 7f3a27fff700 1 -- 172.21.15.140:0/656922311 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3a280574e0 con 0x7f3a2805c0b0 2022-07-21T01:48:32.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.435+0000 7f3a24ff9700 1 -- 172.21.15.140:0/656922311 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3a18023070 con 0x7f3a2805c0b0 2022-07-21T01:48:32.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 -- 172.21.15.140:0/656922311 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a0c053b00 msgr2=0x7f3a0c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a0c053b00 0x7f3a0c055fc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3a1000b000 tx=0x7f3a10000bc0).stop 2022-07-21T01:48:32.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 -- 172.21.15.140:0/656922311 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a2805c0b0 msgr2=0x7f3a28066fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a2805c0b0 0x7f3a28066fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3a10001820 tx=0x7f3a18014000).stop 2022-07-21T01:48:32.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 -- 172.21.15.140:0/656922311 shutdown_connections 2022-07-21T01:48:32.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a0c053b00 0x7f3a0c055fc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a280b3f30 0x7f3a281561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a28066210 0x7f3a28159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 --2- 172.21.15.140:0/656922311 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a2805c0b0 0x7f3a28066fc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 -- 172.21.15.140:0/656922311 >> 172.21.15.140:0/656922311 conn(0x7f3a28064c20 msgr2=0x7f3a280b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:32.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 -- 172.21.15.140:0/656922311 shutdown_connections 2022-07-21T01:48:32.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.439+0000 7f3a27fff700 1 -- 172.21.15.140:0/656922311 wait complete. 2022-07-21T01:48:32.453 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:48:32.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.559+0000 7efdaa2b8700 1 Processor -- start 2022-07-21T01:48:32.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.559+0000 7efdaa2b8700 1 -- start start 2022-07-21T01:48:32.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.559+0000 7efdaa2b8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efda40b5650 0x7efda40b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.559+0000 7efdaa2b8700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda4147010 con 0x7efda40adff0 2022-07-21T01:48:32.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.559+0000 7efdaa2b8700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda41472b0 con 0x7efda40af8e0 2022-07-21T01:48:32.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.559+0000 7efdaa2b8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda4146d40 con 0x7efda40b5650 2022-07-21T01:48:32.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efda8ab5700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7efda40adff0 0x7efda4147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46416/0 (socket says 172.21.15.140:46416) 2022-07-21T01:48:32.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efda8ab5700 1 -- 172.21.15.140:0/1715539204 learned_addr learned my addr 172.21.15.140:0/1715539204 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:32.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efda92b6700 1 --2- 172.21.15.140:0/1715539204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efda40b5650 0x7efda40b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efd97fff700 1 -- 172.21.15.140:0/1715539204 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310414293 0 0) 0x7efda4147010 con 0x7efda40adff0 2022-07-21T01:48:32.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efd97fff700 1 -- 172.21.15.140:0/1715539204 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd88003260 con 0x7efda40adff0 2022-07-21T01:48:32.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efd97fff700 1 -- 172.21.15.140:0/1715539204 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347080838 0 0) 0x7efda41472b0 con 0x7efda40af8e0 2022-07-21T01:48:32.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efd97fff700 1 -- 172.21.15.140:0/1715539204 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efda4147010 con 0x7efda40af8e0 2022-07-21T01:48:32.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efda92b6700 1 -- 172.21.15.140:0/1715539204 >> v1:172.21.15.152:6789/0 conn(0x7efda40af8e0 legacy=0x7efda405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efda92b6700 1 -- 172.21.15.140:0/1715539204 >> v1:172.21.15.140:6789/0 conn(0x7efda40adff0 legacy=0x7efda4147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efda92b6700 1 -- 172.21.15.140:0/1715539204 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda405c4a0 con 0x7efda40b5650 2022-07-21T01:48:32.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efda92b6700 1 --2- 172.21.15.140:0/1715539204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efda40b5650 0x7efda40b5a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7efd98002f70 tx=0x7efd98006df0).ready entity=mon.2 client_cookie=c607a33f26c3d61 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:32.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efd97fff700 1 -- 172.21.15.140:0/1715539204 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd9800d5c0 con 0x7efda40b5650 2022-07-21T01:48:32.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efd97fff700 1 -- 172.21.15.140:0/1715539204 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd98000e90 con 0x7efda40b5650 2022-07-21T01:48:32.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efd97fff700 1 -- 172.21.15.140:0/1715539204 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd9800d910 con 0x7efda40b5650 2022-07-21T01:48:32.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efdaa2b8700 1 -- 172.21.15.140:0/1715539204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efda40b5650 msgr2=0x7efda40b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efdaa2b8700 1 --2- 172.21.15.140:0/1715539204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efda40b5650 0x7efda40b5a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7efd98002f70 tx=0x7efd98006df0).stop 2022-07-21T01:48:32.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efdaa2b8700 1 -- 172.21.15.140:0/1715539204 shutdown_connections 2022-07-21T01:48:32.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efdaa2b8700 1 --2- 172.21.15.140:0/1715539204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efda40b5650 0x7efda40b5a30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efdaa2b8700 1 -- 172.21.15.140:0/1715539204 >> 172.21.15.140:0/1715539204 conn(0x7efda40634e0 msgr2=0x7efda40b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:32.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efdaa2b8700 1 -- 172.21.15.140:0/1715539204 shutdown_connections 2022-07-21T01:48:32.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.563+0000 7efdaa2b8700 1 -- 172.21.15.140:0/1715539204 wait complete. 2022-07-21T01:48:32.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efdaa2b8700 1 Processor -- start 2022-07-21T01:48:32.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efdaa2b8700 1 -- start start 2022-07-21T01:48:32.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efdaa2b8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efda40adff0 0x7efda4067680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efdaa2b8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efda40af8e0 0x7efda414a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efdaa2b8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efda40b5650 0x7efda41443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efdaa2b8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40652e0 con 0x7efda40af8e0 2022-07-21T01:48:32.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efdaa2b8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40643f0 con 0x7efda40adff0 2022-07-21T01:48:32.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efdaa2b8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda4064690 con 0x7efda40b5650 2022-07-21T01:48:32.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efda8ab5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efda40af8e0 0x7efda414a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efda8ab5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efda40af8e0 0x7efda414a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60510/0 (socket says 172.21.15.140:60510) 2022-07-21T01:48:32.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efda8ab5700 1 -- 172.21.15.140:0/3050667204 learned_addr learned my addr 172.21.15.140:0/3050667204 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:32.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efda92b6700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efda40adff0 0x7efda4067680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efda9ab7700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efda40b5650 0x7efda41443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efda8ab5700 1 -- 172.21.15.140:0/3050667204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efda40b5650 msgr2=0x7efda41443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efda8ab5700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efda40b5650 0x7efda41443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efda8ab5700 1 -- 172.21.15.140:0/3050667204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efda40adff0 msgr2=0x7efda4067680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efda8ab5700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efda40adff0 0x7efda4067680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efda8ab5700 1 -- 172.21.15.140:0/3050667204 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9800b180 con 0x7efda40af8e0 2022-07-21T01:48:32.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efda8ab5700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efda40af8e0 0x7efda414a690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7efda0001610 tx=0x7efda0007cd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:32.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.567+0000 7efd967fc700 1 -- 172.21.15.140:0/3050667204 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efda0005a70 con 0x7efda40af8e0 2022-07-21T01:48:32.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.571+0000 7efdaa2b8700 1 -- 172.21.15.140:0/3050667204 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda4064fd0 con 0x7efda40af8e0 2022-07-21T01:48:32.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.571+0000 7efd967fc700 1 -- 172.21.15.140:0/3050667204 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda0005bd0 con 0x7efda40af8e0 2022-07-21T01:48:32.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.571+0000 7efd967fc700 1 -- 172.21.15.140:0/3050667204 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efda00070d0 con 0x7efda40af8e0 2022-07-21T01:48:32.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.571+0000 7efdaa2b8700 1 -- 172.21.15.140:0/3050667204 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda4066080 con 0x7efda40af8e0 2022-07-21T01:48:32.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.571+0000 7efdaa2b8700 1 -- 172.21.15.140:0/3050667204 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda40574e0 con 0x7efda40af8e0 2022-07-21T01:48:32.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.571+0000 7efd967fc700 1 -- 172.21.15.140:0/3050667204 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efda0010070 con 0x7efda40af8e0 2022-07-21T01:48:32.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.571+0000 7efd967fc700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd8c053b00 0x7efd8c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.571+0000 7efd967fc700 1 -- 172.21.15.140:0/3050667204 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efda006b9d0 con 0x7efda40af8e0 2022-07-21T01:48:32.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.571+0000 7efda92b6700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd8c053b00 0x7efd8c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.571+0000 7efda92b6700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd8c053b00 0x7efd8c055fc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7efd98018000 tx=0x7efd98014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:32.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.579+0000 7efd967fc700 1 -- 172.21.15.140:0/3050667204 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efda00427f0 con 0x7efda40af8e0 2022-07-21T01:48:32.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.739+0000 7efdaa2b8700 1 -- 172.21.15.140:0/3050667204 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efda4067ea0 con 0x7efda40af8e0 2022-07-21T01:48:32.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efd967fc700 1 -- 172.21.15.140:0/3050667204 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7efda0018030 con 0x7efda40af8e0 2022-07-21T01:48:32.743 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:32.743 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/665899293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":665899293},{"type":"v1","addr":"172.21.15.140:6835","nonce":665899293}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:48:31.423028+0000","modified":"2022-07-21T01:48:32.427116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.152:6833/1276387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1276387778},{"type":"v1","addr":"172.21.15.152:6833","nonce":1276387778}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-07-21T01:48:32.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 -- 172.21.15.140:0/3050667204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd8c053b00 msgr2=0x7efd8c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd8c053b00 0x7efd8c055fc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7efd98018000 tx=0x7efd98014000).stop 2022-07-21T01:48:32.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 -- 172.21.15.140:0/3050667204 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efda40af8e0 msgr2=0x7efda414a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efda40af8e0 0x7efda414a690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7efda0001610 tx=0x7efda0007cd0).stop 2022-07-21T01:48:32.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 -- 172.21.15.140:0/3050667204 shutdown_connections 2022-07-21T01:48:32.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd8c053b00 0x7efd8c055fc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efda40b5650 0x7efda41443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efda40af8e0 0x7efda414a690 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 --2- 172.21.15.140:0/3050667204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efda40adff0 0x7efda4067680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 -- 172.21.15.140:0/3050667204 >> 172.21.15.140:0/3050667204 conn(0x7efda40634e0 msgr2=0x7efda41458d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:32.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 -- 172.21.15.140:0/3050667204 shutdown_connections 2022-07-21T01:48:32.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.743+0000 7efdaa2b8700 1 -- 172.21.15.140:0/3050667204 wait complete. 2022-07-21T01:48:32.751 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 5 2022-07-21T01:48:32.754 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:32.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.855+0000 7f82bc72c700 1 Processor -- start 2022-07-21T01:48:32.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.855+0000 7f82bc72c700 1 -- start start 2022-07-21T01:48:32.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40b1e00 0x7f82b414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82b405c3f0 0x7f82b40ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b405d480 con 0x7f82b405bd90 2022-07-21T01:48:32.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b40b21e0 con 0x7f82b405c3f0 2022-07-21T01:48:32.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b405cf40 con 0x7f82b40b1e00 2022-07-21T01:48:32.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82ba4c8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40b1e00 0x7f82b414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82b9cc7700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82b405c3f0 0x7f82b40ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82ba4c8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40b1e00 0x7f82b414a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:57246/0 (socket says 172.21.15.140:57246) 2022-07-21T01:48:32.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82ba4c8700 1 -- 172.21.15.140:0/544318852 learned_addr learned my addr 172.21.15.140:0/544318852 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:32.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82ba4c8700 1 -- 172.21.15.140:0/544318852 >> v2:172.21.15.152:3300/0 conn(0x7f82b405c3f0 msgr2=0x7f82b40ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:32.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82ba4c8700 1 --2- 172.21.15.140:0/544318852 >> v2:172.21.15.152:3300/0 conn(0x7f82b405c3f0 0x7f82b40ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82ba4c8700 1 -- 172.21.15.140:0/544318852 >> v1:172.21.15.152:6789/0 conn(0x7f82b405bd90 legacy=0x7f82b4146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82ba4c8700 1 -- 172.21.15.140:0/544318852 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b405d1e0 con 0x7f82b40b1e00 2022-07-21T01:48:32.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82ba4c8700 1 --2- 172.21.15.140:0/544318852 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40b1e00 0x7f82b414a6d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f82ac0032f0 tx=0x7f82ac009290).ready entity=mon.2 client_cookie=db7e545717b0996e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:32.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82b94c6700 1 -- 172.21.15.140:0/544318852 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82ac00b8a0 con 0x7f82b40b1e00 2022-07-21T01:48:32.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82b94c6700 1 -- 172.21.15.140:0/544318852 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac00ba00 con 0x7f82b40b1e00 2022-07-21T01:48:32.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82b94c6700 1 -- 172.21.15.140:0/544318852 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82ac00bb70 con 0x7f82b40b1e00 2022-07-21T01:48:32.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 -- 172.21.15.140:0/544318852 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40b1e00 msgr2=0x7f82b414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 --2- 172.21.15.140:0/544318852 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40b1e00 0x7f82b414a6d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f82ac0032f0 tx=0x7f82ac009290).stop 2022-07-21T01:48:32.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 -- 172.21.15.140:0/544318852 shutdown_connections 2022-07-21T01:48:32.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 --2- 172.21.15.140:0/544318852 >> v2:172.21.15.152:3300/0 conn(0x7f82b405c3f0 0x7f82b40ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 --2- 172.21.15.140:0/544318852 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40b1e00 0x7f82b414a6d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 -- 172.21.15.140:0/544318852 >> 172.21.15.140:0/544318852 conn(0x7f82b40611c0 msgr2=0x7f82b4062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:32.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 -- 172.21.15.140:0/544318852 shutdown_connections 2022-07-21T01:48:32.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.859+0000 7f82bc72c700 1 -- 172.21.15.140:0/544318852 wait complete. 2022-07-21T01:48:32.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bc72c700 1 Processor -- start 2022-07-21T01:48:32.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bc72c700 1 -- start start 2022-07-21T01:48:32.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bc72c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b405c3f0 0x7f82b40a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bc72c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82b40b1e00 0x7f82b40672e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bc72c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40639e0 0x7f82b40b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bc72c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b409e120 con 0x7f82b40b1e00 2022-07-21T01:48:32.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bc72c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b409e380 con 0x7f82b405c3f0 2022-07-21T01:48:32.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bc72c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b415aa20 con 0x7f82b40639e0 2022-07-21T01:48:32.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82ba4c8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b405c3f0 0x7f82b40a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82b9cc7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82b40b1e00 0x7f82b40672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82ba4c8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b405c3f0 0x7f82b40a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:38030/0 (socket says 172.21.15.140:38030) 2022-07-21T01:48:32.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82ba4c8700 1 -- 172.21.15.140:0/2690911586 learned_addr learned my addr 172.21.15.140:0/2690911586 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:32.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bacc9700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40639e0 0x7f82b40b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82ba4c8700 1 -- 172.21.15.140:0/2690911586 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40639e0 msgr2=0x7f82b40b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82ba4c8700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40639e0 0x7f82b40b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82ba4c8700 1 -- 172.21.15.140:0/2690911586 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82b40b1e00 msgr2=0x7f82b40672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:32.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82ba4c8700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82b40b1e00 0x7f82b40672e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:32.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82ba4c8700 1 -- 172.21.15.140:0/2690911586 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac009530 con 0x7f82b405c3f0 2022-07-21T01:48:32.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82ba4c8700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b405c3f0 0x7f82b40a1d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f82ac0032f0 tx=0x7f82ac016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:32.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82a37fe700 1 -- 172.21.15.140:0/2690911586 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82ac006b20 con 0x7f82b405c3f0 2022-07-21T01:48:32.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82a37fe700 1 -- 172.21.15.140:0/2690911586 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac021440 con 0x7f82b405c3f0 2022-07-21T01:48:32.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bc72c700 1 -- 172.21.15.140:0/2690911586 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b40ada30 con 0x7f82b405c3f0 2022-07-21T01:48:32.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82a37fe700 1 -- 172.21.15.140:0/2690911586 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82ac0217a0 con 0x7f82b405c3f0 2022-07-21T01:48:32.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bc72c700 1 -- 172.21.15.140:0/2690911586 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b4067970 con 0x7f82b405c3f0 2022-07-21T01:48:32.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.863+0000 7f82bc72c700 1 -- 172.21.15.140:0/2690911586 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b40b04b0 con 0x7f82b405c3f0 2022-07-21T01:48:32.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.867+0000 7f82a37fe700 1 -- 172.21.15.140:0/2690911586 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f82ac027020 con 0x7f82b405c3f0 2022-07-21T01:48:32.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.867+0000 7f82a37fe700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8294053a70 0x7f8294055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:32.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.867+0000 7f82a37fe700 1 -- 172.21.15.140:0/2690911586 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f82ac06dca0 con 0x7f82b405c3f0 2022-07-21T01:48:32.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.867+0000 7f82b9cc7700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8294053a70 0x7f8294055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:32.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.867+0000 7f82b9cc7700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8294053a70 0x7f8294055f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f82a400b000 tx=0x7f82a4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:32.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.875+0000 7f82a37fe700 1 -- 172.21.15.140:0/2690911586 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f82ac044bc0 con 0x7f82b405c3f0 2022-07-21T01:48:32.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:32.999+0000 7f82bc72c700 1 -- 172.21.15.140:0/2690911586 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82b40574e0 con 0x7f82b405c3f0 2022-07-21T01:48:33.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.003+0000 7f82a37fe700 1 -- 172.21.15.140:0/2690911586 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f82ac015020 con 0x7f82b405c3f0 2022-07-21T01:48:33.002 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:33.003 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:33.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 -- 172.21.15.140:0/2690911586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8294053a70 msgr2=0x7f8294055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8294053a70 0x7f8294055f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f82a400b000 tx=0x7f82a4006df0).stop 2022-07-21T01:48:33.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 -- 172.21.15.140:0/2690911586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b405c3f0 msgr2=0x7f82b40a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b405c3f0 0x7f82b40a1d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f82ac0032f0 tx=0x7f82ac016000).stop 2022-07-21T01:48:33.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 -- 172.21.15.140:0/2690911586 shutdown_connections 2022-07-21T01:48:33.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8294053a70 0x7f8294055f30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82b40639e0 0x7f82b40b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82b40b1e00 0x7f82b40672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 --2- 172.21.15.140:0/2690911586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82b405c3f0 0x7f82b40a1d90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 -- 172.21.15.140:0/2690911586 >> 172.21.15.140:0/2690911586 conn(0x7f82b405bd90 msgr2=0x7f82b4147a70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:33.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 -- 172.21.15.140:0/2690911586 shutdown_connections 2022-07-21T01:48:33.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.007+0000 7f82bc72c700 1 -- 172.21.15.140:0/2690911586 wait complete. 2022-07-21T01:48:33.023 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:33.025 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2022-07-21T01:48:33.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.135+0000 7f87c054b700 1 Processor -- start 2022-07-21T01:48:33.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.135+0000 7f87c054b700 1 -- start start 2022-07-21T01:48:33.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.135+0000 7f87c054b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80675f0 0x7f87b81411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.135+0000 7f87c054b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f87b80b1a70 0x7f87b80b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.135+0000 7f87c054b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b80aee00 con 0x7f87b80aea20 2022-07-21T01:48:33.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.135+0000 7f87c054b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b8144470 con 0x7f87b80b1a70 2022-07-21T01:48:33.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.135+0000 7f87c054b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b8144710 con 0x7f87b80675f0 2022-07-21T01:48:33.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.135+0000 7f87beae8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f87b80b1a70 0x7f87b80b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.135+0000 7f87be2e7700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f87b80aea20 0x7f87b8140a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46444/0 (socket says 172.21.15.140:46444) 2022-07-21T01:48:33.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.135+0000 7f87be2e7700 1 -- 172.21.15.140:0/2183260976 learned_addr learned my addr 172.21.15.140:0/2183260976 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:33.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.135+0000 7f87bdae6700 1 --2- 172.21.15.140:0/2183260976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80675f0 0x7f87b81411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87beae8700 1 -- 172.21.15.140:0/2183260976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80675f0 msgr2=0x7f87b81411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:33.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87beae8700 1 --2- 172.21.15.140:0/2183260976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80675f0 0x7f87b81411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87beae8700 1 -- 172.21.15.140:0/2183260976 >> v1:172.21.15.140:6789/0 conn(0x7f87b80aea20 legacy=0x7f87b8140a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87bd2e5700 1 -- 172.21.15.140:0/2183260976 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194946675 0 0) 0x7f87b80aee00 con 0x7f87b80aea20 2022-07-21T01:48:33.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87beae8700 1 -- 172.21.15.140:0/2183260976 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b81449b0 con 0x7f87b80b1a70 2022-07-21T01:48:33.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87bdae6700 1 --2- 172.21.15.140:0/2183260976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80675f0 0x7f87b81411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:33.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87beae8700 1 --2- 172.21.15.140:0/2183260976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87b80b1a70 0x7f87b80b1f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f87b00032f0 tx=0x7f87b00091d0).ready entity=mon.0 client_cookie=a35923cbd5183011 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87bd2e5700 1 -- 172.21.15.140:0/2183260976 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87b000b6c0 con 0x7f87b80b1a70 2022-07-21T01:48:33.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87bd2e5700 1 -- 172.21.15.140:0/2183260976 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b000b820 con 0x7f87b80b1a70 2022-07-21T01:48:33.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87bd2e5700 1 -- 172.21.15.140:0/2183260976 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87b000b990 con 0x7f87b80b1a70 2022-07-21T01:48:33.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 -- 172.21.15.140:0/2183260976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87b80b1a70 msgr2=0x7f87b80b1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 --2- 172.21.15.140:0/2183260976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87b80b1a70 0x7f87b80b1f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f87b00032f0 tx=0x7f87b00091d0).stop 2022-07-21T01:48:33.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 -- 172.21.15.140:0/2183260976 shutdown_connections 2022-07-21T01:48:33.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 --2- 172.21.15.140:0/2183260976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87b80b1a70 0x7f87b80b1f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 --2- 172.21.15.140:0/2183260976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80675f0 0x7f87b81411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 -- 172.21.15.140:0/2183260976 >> 172.21.15.140:0/2183260976 conn(0x7f87b80611c0 msgr2=0x7f87b8062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:33.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 -- 172.21.15.140:0/2183260976 shutdown_connections 2022-07-21T01:48:33.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 -- 172.21.15.140:0/2183260976 wait complete. 2022-07-21T01:48:33.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 Processor -- start 2022-07-21T01:48:33.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 -- start start 2022-07-21T01:48:33.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80aea20 0x7f87b8150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87b80b1a70 0x7f87b813e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87b8066810 0x7f87b813ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b813f0f0 con 0x7f87b8066810 2022-07-21T01:48:33.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b813b1f0 con 0x7f87b80b1a70 2022-07-21T01:48:33.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87c054b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b813b420 con 0x7f87b80aea20 2022-07-21T01:48:33.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.139+0000 7f87beae8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87b8066810 0x7f87b813ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87be2e7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80aea20 0x7f87b8150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87beae8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87b8066810 0x7f87b813ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60540/0 (socket says 172.21.15.140:60540) 2022-07-21T01:48:33.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87beae8700 1 -- 172.21.15.140:0/3640225448 learned_addr learned my addr 172.21.15.140:0/3640225448 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:33.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87bdae6700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87b80b1a70 0x7f87b813e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87be2e7700 1 -- 172.21.15.140:0/3640225448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87b80b1a70 msgr2=0x7f87b813e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87be2e7700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87b80b1a70 0x7f87b813e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87be2e7700 1 -- 172.21.15.140:0/3640225448 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87b8066810 msgr2=0x7f87b813ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87be2e7700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87b8066810 0x7f87b813ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87be2e7700 1 -- 172.21.15.140:0/3640225448 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a800b180 con 0x7f87b80aea20 2022-07-21T01:48:33.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87bdae6700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87b80b1a70 0x7f87b813e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:33.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87beae8700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87b8066810 0x7f87b813ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:33.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87be2e7700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80aea20 0x7f87b8150c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f87b80665d0 tx=0x7f87b4007a00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87a77fe700 1 -- 172.21.15.140:0/3640225448 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87b40070d0 con 0x7f87b80aea20 2022-07-21T01:48:33.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87a77fe700 1 -- 172.21.15.140:0/3640225448 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b4007230 con 0x7f87b80aea20 2022-07-21T01:48:33.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87c054b700 1 -- 172.21.15.140:0/3640225448 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b0009470 con 0x7f87b80aea20 2022-07-21T01:48:33.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87a77fe700 1 -- 172.21.15.140:0/3640225448 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87b4005610 con 0x7f87b80aea20 2022-07-21T01:48:33.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87c054b700 1 -- 172.21.15.140:0/3640225448 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b8151140 con 0x7f87b80aea20 2022-07-21T01:48:33.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87c054b700 1 -- 172.21.15.140:0/3640225448 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b8144e50 con 0x7f87b80aea20 2022-07-21T01:48:33.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.143+0000 7f87a77fe700 1 -- 172.21.15.140:0/3640225448 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f87b401a060 con 0x7f87b80aea20 2022-07-21T01:48:33.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.147+0000 7f87a77fe700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8798053950 0x7f8798055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.147+0000 7f87bdae6700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8798053950 0x7f8798055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.147+0000 7f87a77fe700 1 -- 172.21.15.140:0/3640225448 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f87b406b620 con 0x7f87b80aea20 2022-07-21T01:48:33.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.147+0000 7f87bdae6700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8798053950 0x7f8798055e10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f87a800b000 tx=0x7f87a8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.155+0000 7f87a77fe700 1 -- 172.21.15.140:0/3640225448 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f87b40424c0 con 0x7f87b80aea20 2022-07-21T01:48:33.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.279+0000 7f87c054b700 1 -- 172.21.15.140:0/3640225448 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87b80af4f0 con 0x7f87b80aea20 2022-07-21T01:48:33.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.279+0000 7f87a77fe700 1 -- 172.21.15.140:0/3640225448 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f87b4018070 con 0x7f87b80aea20 2022-07-21T01:48:33.281 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:33.282 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:33.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.283+0000 7f87c054b700 1 -- 172.21.15.140:0/3640225448 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8798053950 msgr2=0x7f8798055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.283+0000 7f87c054b700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8798053950 0x7f8798055e10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f87a800b000 tx=0x7f87a8006df0).stop 2022-07-21T01:48:33.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.283+0000 7f87c054b700 1 -- 172.21.15.140:0/3640225448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80aea20 msgr2=0x7f87b8150c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.283+0000 7f87c054b700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80aea20 0x7f87b8150c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f87b80665d0 tx=0x7f87b4007a00).stop 2022-07-21T01:48:33.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.283+0000 7f87c054b700 1 -- 172.21.15.140:0/3640225448 shutdown_connections 2022-07-21T01:48:33.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.283+0000 7f87c054b700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8798053950 0x7f8798055e10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.283+0000 7f87c054b700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87b8066810 0x7f87b813ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.283+0000 7f87c054b700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87b80b1a70 0x7f87b813e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.283+0000 7f87c054b700 1 --2- 172.21.15.140:0/3640225448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87b80aea20 0x7f87b8150c50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.283+0000 7f87c054b700 1 -- 172.21.15.140:0/3640225448 >> 172.21.15.140:0/3640225448 conn(0x7f87b80675f0 msgr2=0x7f87b80b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:33.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.283+0000 7f87c054b700 1 -- 172.21.15.140:0/3640225448 shutdown_connections 2022-07-21T01:48:33.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.287+0000 7f87c054b700 1 -- 172.21.15.140:0/3640225448 wait complete. 2022-07-21T01:48:33.299 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:48:29.344690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:48:30.427054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:48:33.300 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:48:33.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff67759e700 1 Processor -- start 2022-07-21T01:48:33.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff67759e700 1 -- start start 2022-07-21T01:48:33.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff67759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6780b0580 0x7ff67805c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff67759e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff6781468a0 0x7ff6780b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff67759e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6780b16a0 con 0x7ff6780b23b0 2022-07-21T01:48:33.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff67759e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff678067f70 con 0x7ff6781468a0 2022-07-21T01:48:33.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff67759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6780ad0e0 con 0x7ff6780b0580 2022-07-21T01:48:33.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff675d9b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6780b0580 0x7ff67805c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff676d9d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff6781468a0 0x7ff6780b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff675d9b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6780b0580 0x7ff67805c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:57276/0 (socket says 172.21.15.140:57276) 2022-07-21T01:48:33.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff675d9b700 1 -- 172.21.15.140:0/2447806467 learned_addr learned my addr 172.21.15.140:0/2447806467 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:33.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff676d9d700 1 -- 172.21.15.140:0/2447806467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6780b0580 msgr2=0x7ff67805c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:33.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff676d9d700 1 --2- 172.21.15.140:0/2447806467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6780b0580 0x7ff67805c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff676d9d700 1 -- 172.21.15.140:0/2447806467 >> v1:172.21.15.152:6789/0 conn(0x7ff6780b23b0 legacy=0x7ff678146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff676d9d700 1 -- 172.21.15.140:0/2447806467 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6780ae660 con 0x7ff6781468a0 2022-07-21T01:48:33.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.403+0000 7ff67559a700 1 -- 172.21.15.140:0/2447806467 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930966892 0 0) 0x7ff6780b16a0 con 0x7ff6780b23b0 2022-07-21T01:48:33.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff675d9b700 1 --2- 172.21.15.140:0/2447806467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6780b0580 0x7ff67805c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:33.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff676d9d700 1 --2- 172.21.15.140:0/2447806467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6781468a0 0x7ff6780b1bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7ff6700032f0 tx=0x7ff6700091d0).ready entity=mon.1 client_cookie=faa351b48608cee7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67559a700 1 -- 172.21.15.140:0/2447806467 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67000b6c0 con 0x7ff6781468a0 2022-07-21T01:48:33.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67559a700 1 -- 172.21.15.140:0/2447806467 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67000b820 con 0x7ff6781468a0 2022-07-21T01:48:33.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67559a700 1 -- 172.21.15.140:0/2447806467 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff67000b990 con 0x7ff6781468a0 2022-07-21T01:48:33.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67759e700 1 -- 172.21.15.140:0/2447806467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6781468a0 msgr2=0x7ff6780b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67759e700 1 --2- 172.21.15.140:0/2447806467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6781468a0 0x7ff6780b1bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7ff6700032f0 tx=0x7ff6700091d0).stop 2022-07-21T01:48:33.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67759e700 1 -- 172.21.15.140:0/2447806467 shutdown_connections 2022-07-21T01:48:33.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67759e700 1 --2- 172.21.15.140:0/2447806467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6780b0580 0x7ff67805c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67759e700 1 --2- 172.21.15.140:0/2447806467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6781468a0 0x7ff6780b1bd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67759e700 1 -- 172.21.15.140:0/2447806467 >> 172.21.15.140:0/2447806467 conn(0x7ff6780632d0 msgr2=0x7ff6780636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:33.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67759e700 1 -- 172.21.15.140:0/2447806467 shutdown_connections 2022-07-21T01:48:33.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67759e700 1 -- 172.21.15.140:0/2447806467 wait complete. 2022-07-21T01:48:33.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67759e700 1 Processor -- start 2022-07-21T01:48:33.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.407+0000 7ff67759e700 1 -- start start 2022-07-21T01:48:33.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff67759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6780b0580 0x7ff678066ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff67759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff6780b23b0 0x7ff678112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff67759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6781468a0 0x7ff6781189c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff67759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff678116370 con 0x7ff6780b23b0 2022-07-21T01:48:33.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff67759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff678118520 con 0x7ff6780b0580 2022-07-21T01:48:33.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff67759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6781132d0 con 0x7ff6781468a0 2022-07-21T01:48:33.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff675d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff6780b23b0 0x7ff678112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff675d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff6780b23b0 0x7ff678112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60552/0 (socket says 172.21.15.140:60552) 2022-07-21T01:48:33.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff675d9b700 1 -- 172.21.15.140:0/3939647097 learned_addr learned my addr 172.21.15.140:0/3939647097 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:33.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff67659c700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6780b0580 0x7ff678066ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff676d9d700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6781468a0 0x7ff6781189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff675d9b700 1 -- 172.21.15.140:0/3939647097 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6781468a0 msgr2=0x7ff6781189c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff675d9b700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6781468a0 0x7ff6781189c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff675d9b700 1 -- 172.21.15.140:0/3939647097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6780b0580 msgr2=0x7ff678066ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff675d9b700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6780b0580 0x7ff678066ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff675d9b700 1 -- 172.21.15.140:0/3939647097 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c00b1e0 con 0x7ff6780b23b0 2022-07-21T01:48:33.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff675d9b700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff6780b23b0 0x7ff678112de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff66c00c000 tx=0x7ff66c006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff6677fe700 1 -- 172.21.15.140:0/3939647097 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff66c012600 con 0x7ff6780b23b0 2022-07-21T01:48:33.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff6677fe700 1 -- 172.21.15.140:0/3939647097 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c012760 con 0x7ff6780b23b0 2022-07-21T01:48:33.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff67759e700 1 -- 172.21.15.140:0/3939647097 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff670009470 con 0x7ff6780b23b0 2022-07-21T01:48:33.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff6677fe700 1 -- 172.21.15.140:0/3939647097 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff66c012ac0 con 0x7ff6780b23b0 2022-07-21T01:48:33.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.411+0000 7ff67759e700 1 -- 172.21.15.140:0/3939647097 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff678115e70 con 0x7ff6780b23b0 2022-07-21T01:48:33.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.415+0000 7ff67759e700 1 -- 172.21.15.140:0/3939647097 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6781186e0 con 0x7ff6780b23b0 2022-07-21T01:48:33.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.415+0000 7ff6677fe700 1 -- 172.21.15.140:0/3939647097 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff66c012c20 con 0x7ff6780b23b0 2022-07-21T01:48:33.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.415+0000 7ff6677fe700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff6540539e0 0x7ff654055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.415+0000 7ff6677fe700 1 -- 172.21.15.140:0/3939647097 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff66c06c2d0 con 0x7ff6780b23b0 2022-07-21T01:48:33.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.415+0000 7ff67659c700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff6540539e0 0x7ff654055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.415+0000 7ff67659c700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff6540539e0 0x7ff654055ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff66800d000 tx=0x7ff668007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.423+0000 7ff6677fe700 1 -- 172.21.15.140:0/3939647097 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff66c0430f0 con 0x7ff6780b23b0 2022-07-21T01:48:33.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.551+0000 7ff67759e700 1 -- 172.21.15.140:0/3939647097 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff6780b1390 con 0x7ff6780b23b0 2022-07-21T01:48:33.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff6677fe700 1 -- 172.21.15.140:0/3939647097 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff66c046710 con 0x7ff6780b23b0 2022-07-21T01:48:33.553 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:48:33.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 -- 172.21.15.140:0/3939647097 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff6540539e0 msgr2=0x7ff654055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff6540539e0 0x7ff654055ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff66800d000 tx=0x7ff668007a00).stop 2022-07-21T01:48:33.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 -- 172.21.15.140:0/3939647097 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff6780b23b0 msgr2=0x7ff678112de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff6780b23b0 0x7ff678112de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff66c00c000 tx=0x7ff66c006eb0).stop 2022-07-21T01:48:33.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 -- 172.21.15.140:0/3939647097 shutdown_connections 2022-07-21T01:48:33.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff6540539e0 0x7ff654055ea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff6781468a0 0x7ff6781189c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff6780b23b0 0x7ff678112de0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 --2- 172.21.15.140:0/3939647097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6780b0580 0x7ff678066ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 -- 172.21.15.140:0/3939647097 >> 172.21.15.140:0/3939647097 conn(0x7ff6780632d0 msgr2=0x7ff67805f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:33.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 -- 172.21.15.140:0/3939647097 shutdown_connections 2022-07-21T01:48:33.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.555+0000 7ff67759e700 1 -- 172.21.15.140:0/3939647097 wait complete. 2022-07-21T01:48:33.566 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:48:33.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb2c92700 1 Processor -- start 2022-07-21T01:48:33.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb2c92700 1 -- start start 2022-07-21T01:48:33.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb2c92700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4eac0b59d0 0x7f4eac0b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb2c92700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4eac14ada0 0x7f4eac0b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb2c92700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eac0afbc0 con 0x7f4eac0b14f0 2022-07-21T01:48:33.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb2c92700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eac148080 con 0x7f4eac0b59d0 2022-07-21T01:48:33.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb2c92700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac0b00e0 con 0x7f4eac14ada0 2022-07-21T01:48:33.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb1c90700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4eac0b59d0 0x7f4eac0b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb1c90700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4eac0b59d0 0x7f4eac0b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60558/0 (socket says 172.21.15.140:60558) 2022-07-21T01:48:33.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb1c90700 1 -- 172.21.15.140:0/3586711768 learned_addr learned my addr 172.21.15.140:0/3586711768 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:33.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb148f700 1 --2- 172.21.15.140:0/3586711768 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4eac14ada0 0x7f4eac0b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb0c8e700 1 -- 172.21.15.140:0/3586711768 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899956574 0 0) 0x7f4eac0afbc0 con 0x7f4eac0b14f0 2022-07-21T01:48:33.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb0c8e700 1 -- 172.21.15.140:0/3586711768 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e94003260 con 0x7f4eac0b14f0 2022-07-21T01:48:33.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb1c90700 1 -- 172.21.15.140:0/3586711768 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4eac14ada0 msgr2=0x7f4eac0b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:33.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb1c90700 1 --2- 172.21.15.140:0/3586711768 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4eac14ada0 0x7f4eac0b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.675+0000 7f4eb1c90700 1 -- 172.21.15.140:0/3586711768 >> v1:172.21.15.152:6789/0 conn(0x7f4eac0b14f0 legacy=0x7f4eac05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb1c90700 1 -- 172.21.15.140:0/3586711768 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac0b5db0 con 0x7f4eac0b59d0 2022-07-21T01:48:33.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb1c90700 1 --2- 172.21.15.140:0/3586711768 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4eac0b59d0 0x7f4eac0b07e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f4ea40032f0 tx=0x7f4ea4009290).ready entity=mon.0 client_cookie=9dd6ad2e7aaf4863 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb0c8e700 1 -- 172.21.15.140:0/3586711768 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ea400b6c0 con 0x7f4eac0b59d0 2022-07-21T01:48:33.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb0c8e700 1 -- 172.21.15.140:0/3586711768 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea400b820 con 0x7f4eac0b59d0 2022-07-21T01:48:33.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb0c8e700 1 -- 172.21.15.140:0/3586711768 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ea400b990 con 0x7f4eac0b59d0 2022-07-21T01:48:33.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb2c92700 1 -- 172.21.15.140:0/3586711768 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4eac0b59d0 msgr2=0x7f4eac0b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb2c92700 1 --2- 172.21.15.140:0/3586711768 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4eac0b59d0 0x7f4eac0b07e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f4ea40032f0 tx=0x7f4ea4009290).stop 2022-07-21T01:48:33.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb2c92700 1 -- 172.21.15.140:0/3586711768 shutdown_connections 2022-07-21T01:48:33.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb2c92700 1 --2- 172.21.15.140:0/3586711768 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4eac14ada0 0x7f4eac0b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb2c92700 1 --2- 172.21.15.140:0/3586711768 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4eac0b59d0 0x7f4eac0b07e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb2c92700 1 -- 172.21.15.140:0/3586711768 >> 172.21.15.140:0/3586711768 conn(0x7f4eac0634c0 msgr2=0x7f4eac0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:33.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb2c92700 1 -- 172.21.15.140:0/3586711768 shutdown_connections 2022-07-21T01:48:33.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb2c92700 1 -- 172.21.15.140:0/3586711768 wait complete. 2022-07-21T01:48:33.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb2c92700 1 Processor -- start 2022-07-21T01:48:33.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.679+0000 7f4eb2c92700 1 -- start start 2022-07-21T01:48:33.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2c92700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eac0b14f0 0x7f4eac1493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2c92700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4eac0b59d0 0x7f4eac156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2c92700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4eac14ada0 0x7f4eac1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2c92700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac067a60 con 0x7f4eac14ada0 2022-07-21T01:48:33.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2c92700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac1563b0 con 0x7f4eac0b14f0 2022-07-21T01:48:33.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2c92700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac156650 con 0x7f4eac0b59d0 2022-07-21T01:48:33.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2491700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4eac14ada0 0x7f4eac1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2491700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4eac14ada0 0x7f4eac1595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60566/0 (socket says 172.21.15.140:60566) 2022-07-21T01:48:33.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2491700 1 -- 172.21.15.140:0/4274935811 learned_addr learned my addr 172.21.15.140:0/4274935811 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:33.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb1c90700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eac0b14f0 0x7f4eac1493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb148f700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4eac0b59d0 0x7f4eac156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2491700 1 -- 172.21.15.140:0/4274935811 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4eac0b59d0 msgr2=0x7f4eac156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2491700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4eac0b59d0 0x7f4eac156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2491700 1 -- 172.21.15.140:0/4274935811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eac0b14f0 msgr2=0x7f4eac1493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2491700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eac0b14f0 0x7f4eac1493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2491700 1 -- 172.21.15.140:0/4274935811 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea4009530 con 0x7f4eac14ada0 2022-07-21T01:48:33.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb148f700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4eac0b59d0 0x7f4eac156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:33.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2491700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4eac14ada0 0x7f4eac1595f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f4ea8001110 tx=0x7f4ea8007a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4e9affd700 1 -- 172.21.15.140:0/4274935811 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ea800ce30 con 0x7f4eac14ada0 2022-07-21T01:48:33.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4e9affd700 1 -- 172.21.15.140:0/4274935811 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea8012400 con 0x7f4eac14ada0 2022-07-21T01:48:33.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2c92700 1 -- 172.21.15.140:0/4274935811 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eac156070 con 0x7f4eac14ada0 2022-07-21T01:48:33.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4e9affd700 1 -- 172.21.15.140:0/4274935811 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ea8012760 con 0x7f4eac14ada0 2022-07-21T01:48:33.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2c92700 1 -- 172.21.15.140:0/4274935811 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac157200 con 0x7f4eac14ada0 2022-07-21T01:48:33.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb2c92700 1 -- 172.21.15.140:0/4274935811 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eac0b0340 con 0x7f4eac14ada0 2022-07-21T01:48:33.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4e9affd700 1 -- 172.21.15.140:0/4274935811 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4ea8019030 con 0x7f4eac14ada0 2022-07-21T01:48:33.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4e9affd700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e8c053b00 0x7f4e8c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4e9affd700 1 -- 172.21.15.140:0/4274935811 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4ea806b2d0 con 0x7f4eac14ada0 2022-07-21T01:48:33.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.683+0000 7f4eb1c90700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e8c053b00 0x7f4e8c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.687+0000 7f4eb1c90700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e8c053b00 0x7f4e8c055fc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f4ea401b000 tx=0x7f4ea4016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.691+0000 7f4e9affd700 1 -- 172.21.15.140:0/4274935811 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4ea8042150 con 0x7f4eac14ada0 2022-07-21T01:48:33.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.819+0000 7f4eb2c92700 1 -- 172.21.15.140:0/4274935811 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4eac156200 con 0x7f4eac14ada0 2022-07-21T01:48:33.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.819+0000 7f4e9affd700 1 -- 172.21.15.140:0/4274935811 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4ea8017020 con 0x7f4eac14ada0 2022-07-21T01:48:33.827 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:33.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.819+0000 7f4eb2c92700 1 -- 172.21.15.140:0/4274935811 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e8c053b00 msgr2=0x7f4e8c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.819+0000 7f4eb2c92700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e8c053b00 0x7f4e8c055fc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f4ea401b000 tx=0x7f4ea4016000).stop 2022-07-21T01:48:33.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.819+0000 7f4eb2c92700 1 -- 172.21.15.140:0/4274935811 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4eac14ada0 msgr2=0x7f4eac1595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.819+0000 7f4eb2c92700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4eac14ada0 0x7f4eac1595f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f4ea8001110 tx=0x7f4ea8007a20).stop 2022-07-21T01:48:33.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.823+0000 7f4eb2c92700 1 -- 172.21.15.140:0/4274935811 shutdown_connections 2022-07-21T01:48:33.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.823+0000 7f4eb2c92700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e8c053b00 0x7f4e8c055fc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.823+0000 7f4eb2c92700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4eac14ada0 0x7f4eac1595f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.823+0000 7f4eb2c92700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4eac0b59d0 0x7f4eac156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.823+0000 7f4eb2c92700 1 --2- 172.21.15.140:0/4274935811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4eac0b14f0 0x7f4eac1493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.823+0000 7f4eb2c92700 1 -- 172.21.15.140:0/4274935811 >> 172.21.15.140:0/4274935811 conn(0x7f4eac0634c0 msgr2=0x7f4eac05c580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:33.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.823+0000 7f4eb2c92700 1 -- 172.21.15.140:0/4274935811 shutdown_connections 2022-07-21T01:48:33.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.823+0000 7f4eb2c92700 1 -- 172.21.15.140:0/4274935811 wait complete. 2022-07-21T01:48:33.832 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:48:33.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.939+0000 7f363714b700 1 Processor -- start 2022-07-21T01:48:33.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.939+0000 7f363714b700 1 -- start start 2022-07-21T01:48:33.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36300acfa0 0x7f3630142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f36300b1030 0x7f3630142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36300ad380 con 0x7f36300b02e0 2022-07-21T01:48:33.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363013f030 con 0x7f36300b1030 2022-07-21T01:48:33.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36300af470 con 0x7f36300acfa0 2022-07-21T01:48:33.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635948700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f36300b1030 0x7f3630142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635948700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f36300b1030 0x7f3630142fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60576/0 (socket says 172.21.15.140:60576) 2022-07-21T01:48:33.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635948700 1 -- 172.21.15.140:0/2370737506 learned_addr learned my addr 172.21.15.140:0/2370737506 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:33.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3636149700 1 --2- 172.21.15.140:0/2370737506 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36300acfa0 0x7f3630142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635948700 1 -- 172.21.15.140:0/2370737506 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36300acfa0 msgr2=0x7f3630142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:33.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635948700 1 --2- 172.21.15.140:0/2370737506 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36300acfa0 0x7f3630142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635948700 1 -- 172.21.15.140:0/2370737506 >> v1:172.21.15.152:6789/0 conn(0x7f36300b02e0 legacy=0x7f36300ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635147700 1 -- 172.21.15.140:0/2370737506 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34869049 0 0) 0x7f36300ad380 con 0x7f36300b02e0 2022-07-21T01:48:33.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635948700 1 -- 172.21.15.140:0/2370737506 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36300b1630 con 0x7f36300b1030 2022-07-21T01:48:33.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635948700 1 --2- 172.21.15.140:0/2370737506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36300b1030 0x7f3630142fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f3620002f70 tx=0x7f3620006eb0).ready entity=mon.0 client_cookie=f2e6f4b465fee4f8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635147700 1 -- 172.21.15.140:0/2370737506 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f362000d570 con 0x7f36300b1030 2022-07-21T01:48:33.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635147700 1 -- 172.21.15.140:0/2370737506 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3620000e90 con 0x7f36300b1030 2022-07-21T01:48:33.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f3635147700 1 -- 172.21.15.140:0/2370737506 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f362000d8c0 con 0x7f36300b1030 2022-07-21T01:48:33.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 -- 172.21.15.140:0/2370737506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36300b1030 msgr2=0x7f3630142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 --2- 172.21.15.140:0/2370737506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36300b1030 0x7f3630142fb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f3620002f70 tx=0x7f3620006eb0).stop 2022-07-21T01:48:33.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 -- 172.21.15.140:0/2370737506 shutdown_connections 2022-07-21T01:48:33.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 --2- 172.21.15.140:0/2370737506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36300b1030 0x7f3630142fb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 --2- 172.21.15.140:0/2370737506 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36300acfa0 0x7f3630142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 -- 172.21.15.140:0/2370737506 >> 172.21.15.140:0/2370737506 conn(0x7f36300632d0 msgr2=0x7f36300636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:33.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 -- 172.21.15.140:0/2370737506 shutdown_connections 2022-07-21T01:48:33.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.943+0000 7f363714b700 1 -- 172.21.15.140:0/2370737506 wait complete. 2022-07-21T01:48:33.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f363714b700 1 Processor -- start 2022-07-21T01:48:33.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f363714b700 1 -- start start 2022-07-21T01:48:33.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f363714b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36300acfa0 0x7f3630065f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f363714b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36300b02e0 0x7f3630112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f363714b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36300b1030 0x7f36301156d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f363714b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36301169a0 con 0x7f36300b02e0 2022-07-21T01:48:33.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f363714b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3630114f90 con 0x7f36300acfa0 2022-07-21T01:48:33.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f363714b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3630115230 con 0x7f36300b1030 2022-07-21T01:48:33.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f3635948700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36300b02e0 0x7f3630112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f3635948700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36300b02e0 0x7f3630112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60580/0 (socket says 172.21.15.140:60580) 2022-07-21T01:48:33.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f3635948700 1 -- 172.21.15.140:0/1364729041 learned_addr learned my addr 172.21.15.140:0/1364729041 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:33.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f3636149700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36300acfa0 0x7f3630065f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f3635948700 1 -- 172.21.15.140:0/1364729041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36300b1030 msgr2=0x7f36301156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f3635948700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36300b1030 0x7f36301156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:48:33.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f3635948700 1 -- 172.21.15.140:0/1364729041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36300acfa0 msgr2=0x7f3630065f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:33.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f3635948700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36300acfa0 0x7f3630065f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:33.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f3635948700 1 -- 172.21.15.140:0/1364729041 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362000b1e0 con 0x7f36300b02e0 2022-07-21T01:48:33.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f3636149700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36300acfa0 0x7f3630065f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:33.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f3635948700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36300b02e0 0x7f3630112de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f3620002f70 tx=0x7f3620015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f361f7fe700 1 -- 172.21.15.140:0/1364729041 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36200118a0 con 0x7f36300b02e0 2022-07-21T01:48:33.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f361f7fe700 1 -- 172.21.15.140:0/1364729041 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3620011a00 con 0x7f36300b02e0 2022-07-21T01:48:33.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f363714b700 1 -- 172.21.15.140:0/1364729041 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3630114300 con 0x7f36300b02e0 2022-07-21T01:48:33.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f361f7fe700 1 -- 172.21.15.140:0/1364729041 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3620011d60 con 0x7f36300b02e0 2022-07-21T01:48:33.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f363714b700 1 -- 172.21.15.140:0/1364729041 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3630067190 con 0x7f36300b02e0 2022-07-21T01:48:33.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.947+0000 7f363714b700 1 -- 172.21.15.140:0/1364729041 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3630114490 con 0x7f36300b02e0 2022-07-21T01:48:33.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.951+0000 7f361f7fe700 1 -- 172.21.15.140:0/1364729041 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3620025030 con 0x7f36300b02e0 2022-07-21T01:48:33.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.951+0000 7f361f7fe700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3610053a70 0x7f3610055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:33.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.951+0000 7f361f7fe700 1 -- 172.21.15.140:0/1364729041 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f362006d0e0 con 0x7f36300b02e0 2022-07-21T01:48:33.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.951+0000 7f3636149700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3610053a70 0x7f3610055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:33.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.951+0000 7f3636149700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3610053a70 0x7f3610055f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f362c00b000 tx=0x7f362c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:33.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:33.955+0000 7f361f7fe700 1 -- 172.21.15.140:0/1364729041 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3620043ef0 con 0x7f36300b02e0 2022-07-21T01:48:34.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.083+0000 7f363714b700 1 -- 172.21.15.140:0/1364729041 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36301153f0 con 0x7f36300b02e0 2022-07-21T01:48:34.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.083+0000 7f361f7fe700 1 -- 172.21.15.140:0/1364729041 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f36200230a0 con 0x7f36300b02e0 2022-07-21T01:48:34.083 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:34.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.083+0000 7f363714b700 1 -- 172.21.15.140:0/1364729041 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3610053a70 msgr2=0x7f3610055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.083+0000 7f363714b700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3610053a70 0x7f3610055f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f362c00b000 tx=0x7f362c008040).stop 2022-07-21T01:48:34.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.083+0000 7f363714b700 1 -- 172.21.15.140:0/1364729041 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36300b02e0 msgr2=0x7f3630112de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.083+0000 7f363714b700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36300b02e0 0x7f3630112de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f3620002f70 tx=0x7f3620015000).stop 2022-07-21T01:48:34.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.087+0000 7f363714b700 1 -- 172.21.15.140:0/1364729041 shutdown_connections 2022-07-21T01:48:34.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.087+0000 7f363714b700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3610053a70 0x7f3610055f30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.087+0000 7f363714b700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36300b1030 0x7f36301156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:48:34.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.087+0000 7f363714b700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36300b02e0 0x7f3630112de0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.087+0000 7f363714b700 1 --2- 172.21.15.140:0/1364729041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36300acfa0 0x7f3630065f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.087+0000 7f363714b700 1 -- 172.21.15.140:0/1364729041 >> 172.21.15.140:0/1364729041 conn(0x7f36300632d0 msgr2=0x7f363005fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:34.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.087+0000 7f363714b700 1 -- 172.21.15.140:0/1364729041 shutdown_connections 2022-07-21T01:48:34.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.087+0000 7f363714b700 1 -- 172.21.15.140:0/1364729041 wait complete. 2022-07-21T01:48:34.096 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:48:34.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.207+0000 7f11b6e3c700 1 Processor -- start 2022-07-21T01:48:34.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.207+0000 7f11b6e3c700 1 -- start start 2022-07-21T01:48:34.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11b014ada0 0x7f11b0146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f11b00b14f0 0x7f11b0146530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b00afbc0 con 0x7f11b00b59d0 2022-07-21T01:48:34.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b0148080 con 0x7f11b00b14f0 2022-07-21T01:48:34.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b00b00e0 con 0x7f11b014ada0 2022-07-21T01:48:34.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b5639700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11b014ada0 0x7f11b0146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b663b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f11b00b14f0 0x7f11b0146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b663b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f11b00b14f0 0x7f11b0146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:38096/0 (socket says 172.21.15.140:38096) 2022-07-21T01:48:34.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b5639700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11b014ada0 0x7f11b0146040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:57318/0 (socket says 172.21.15.140:57318) 2022-07-21T01:48:34.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b663b700 1 -- 172.21.15.140:0/2326971640 learned_addr learned my addr 172.21.15.140:0/2326971640 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:34.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b5e3a700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f11b00b59d0 0x7f11b00b07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:47614/0 (socket says 172.21.15.140:47614) 2022-07-21T01:48:34.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b663b700 1 -- 172.21.15.140:0/2326971640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11b014ada0 msgr2=0x7f11b0146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:34.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b663b700 1 --2- 172.21.15.140:0/2326971640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11b014ada0 0x7f11b0146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b663b700 1 -- 172.21.15.140:0/2326971640 >> v1:172.21.15.152:6789/0 conn(0x7f11b00b59d0 legacy=0x7f11b00b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b663b700 1 -- 172.21.15.140:0/2326971640 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b00b5db0 con 0x7f11b00b14f0 2022-07-21T01:48:34.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b663b700 1 --2- 172.21.15.140:0/2326971640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b00b14f0 0x7f11b0146530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f11a80032f0 tx=0x7f11a8009290).ready entity=mon.1 client_cookie=65c944636a2e5fea server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:34.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b4e38700 1 -- 172.21.15.140:0/2326971640 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11a800b8a0 con 0x7f11b00b14f0 2022-07-21T01:48:34.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b4e38700 1 -- 172.21.15.140:0/2326971640 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a800ba00 con 0x7f11b00b14f0 2022-07-21T01:48:34.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b4e38700 1 -- 172.21.15.140:0/2326971640 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11a800bb70 con 0x7f11b00b14f0 2022-07-21T01:48:34.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 -- 172.21.15.140:0/2326971640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b00b14f0 msgr2=0x7f11b0146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 --2- 172.21.15.140:0/2326971640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b00b14f0 0x7f11b0146530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f11a80032f0 tx=0x7f11a8009290).stop 2022-07-21T01:48:34.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 -- 172.21.15.140:0/2326971640 shutdown_connections 2022-07-21T01:48:34.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 --2- 172.21.15.140:0/2326971640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11b014ada0 0x7f11b0146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 --2- 172.21.15.140:0/2326971640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b00b14f0 0x7f11b0146530 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 -- 172.21.15.140:0/2326971640 >> 172.21.15.140:0/2326971640 conn(0x7f11b00634c0 msgr2=0x7f11b00b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:34.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 -- 172.21.15.140:0/2326971640 shutdown_connections 2022-07-21T01:48:34.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.211+0000 7f11b6e3c700 1 -- 172.21.15.140:0/2326971640 wait complete. 2022-07-21T01:48:34.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b6e3c700 1 Processor -- start 2022-07-21T01:48:34.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b6e3c700 1 -- start start 2022-07-21T01:48:34.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b6e3c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b00b14f0 0x7f11b0063d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b6e3c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11b00b59d0 0x7f11b0156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b6e3c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11b014ada0 0x7f11b0156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b6e3c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b0156280 con 0x7f11b00b59d0 2022-07-21T01:48:34.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b6e3c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b0065c50 con 0x7f11b00b14f0 2022-07-21T01:48:34.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b6e3c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b0065ef0 con 0x7f11b014ada0 2022-07-21T01:48:34.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b5639700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11b00b59d0 0x7f11b0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b5639700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11b00b59d0 0x7f11b0156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60594/0 (socket says 172.21.15.140:60594) 2022-07-21T01:48:34.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b5639700 1 -- 172.21.15.140:0/959549467 learned_addr learned my addr 172.21.15.140:0/959549467 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:34.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b5e3a700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b00b14f0 0x7f11b0063d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b663b700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11b014ada0 0x7f11b0156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b5639700 1 -- 172.21.15.140:0/959549467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11b014ada0 msgr2=0x7f11b0156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b5639700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11b014ada0 0x7f11b0156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b5639700 1 -- 172.21.15.140:0/959549467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b00b14f0 msgr2=0x7f11b0063d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b5639700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b00b14f0 0x7f11b0063d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b5639700 1 -- 172.21.15.140:0/959549467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a8009530 con 0x7f11b00b59d0 2022-07-21T01:48:34.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b5639700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11b00b59d0 0x7f11b0156680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f11a400c000 tx=0x7f11a4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:34.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11aeffd700 1 -- 172.21.15.140:0/959549467 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11a4011750 con 0x7f11b00b59d0 2022-07-21T01:48:34.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11aeffd700 1 -- 172.21.15.140:0/959549467 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a40118b0 con 0x7f11b00b59d0 2022-07-21T01:48:34.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b6e3c700 1 -- 172.21.15.140:0/959549467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b0067830 con 0x7f11b00b59d0 2022-07-21T01:48:34.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11b6e3c700 1 -- 172.21.15.140:0/959549467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b0067260 con 0x7f11b00b59d0 2022-07-21T01:48:34.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.215+0000 7f11aeffd700 1 -- 172.21.15.140:0/959549467 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11a4011c10 con 0x7f11b00b59d0 2022-07-21T01:48:34.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.219+0000 7f11b6e3c700 1 -- 172.21.15.140:0/959549467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b0065a80 con 0x7f11b00b59d0 2022-07-21T01:48:34.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.219+0000 7f11aeffd700 1 -- 172.21.15.140:0/959549467 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f11a4011d70 con 0x7f11b00b59d0 2022-07-21T01:48:34.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.219+0000 7f11aeffd700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1190053a70 0x7f1190055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.219+0000 7f11aeffd700 1 -- 172.21.15.140:0/959549467 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f11a406c410 con 0x7f11b00b59d0 2022-07-21T01:48:34.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.219+0000 7f11b5e3a700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1190053a70 0x7f1190055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.219+0000 7f11b5e3a700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1190053a70 0x7f1190055f30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f11a000d000 tx=0x7f11a0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:34.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.227+0000 7f11aeffd700 1 -- 172.21.15.140:0/959549467 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f11a4043290 con 0x7f11b00b59d0 2022-07-21T01:48:34.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 -- 172.21.15.140:0/959549467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11b0156420 con 0x7f11b00b59d0 2022-07-21T01:48:34.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11aeffd700 1 -- 172.21.15.140:0/959549467 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f11a40468b0 con 0x7f11b00b59d0 2022-07-21T01:48:34.350 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:34.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 -- 172.21.15.140:0/959549467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1190053a70 msgr2=0x7f1190055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1190053a70 0x7f1190055f30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f11a000d000 tx=0x7f11a0009040).stop 2022-07-21T01:48:34.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 -- 172.21.15.140:0/959549467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11b00b59d0 msgr2=0x7f11b0156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11b00b59d0 0x7f11b0156680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f11a400c000 tx=0x7f11a4006df0).stop 2022-07-21T01:48:34.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 -- 172.21.15.140:0/959549467 shutdown_connections 2022-07-21T01:48:34.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1190053a70 0x7f1190055f30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11b014ada0 0x7f11b0156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11b00b59d0 0x7f11b0156680 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 --2- 172.21.15.140:0/959549467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11b00b14f0 0x7f11b0063d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 -- 172.21.15.140:0/959549467 >> 172.21.15.140:0/959549467 conn(0x7f11b00634c0 msgr2=0x7f11b0146d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:34.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 -- 172.21.15.140:0/959549467 shutdown_connections 2022-07-21T01:48:34.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.351+0000 7f11b6e3c700 1 -- 172.21.15.140:0/959549467 wait complete. 2022-07-21T01:48:34.363 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2022-07-21T01:48:34.363 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2022-07-21T01:48:34.364 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2022-07-21T01:48:34.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86f2634700 1 Processor -- start 2022-07-21T01:48:34.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86f2634700 1 -- start start 2022-07-21T01:48:34.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86f2634700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec0aed20 0x7f86ec0b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86f2634700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86ec0b1520 con 0x7f86ec063b00 2022-07-21T01:48:34.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86f2634700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86ec0b1750 con 0x7f86ec0ad3a0 2022-07-21T01:48:34.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86f2634700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec0adb60 con 0x7f86ec0aed20 2022-07-21T01:48:34.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86f0bd1700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f86ec063b00 0x7f86ec13ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46516/0 (socket says 172.21.15.140:46516) 2022-07-21T01:48:34.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86f0bd1700 1 -- 172.21.15.140:0/1147682188 learned_addr learned my addr 172.21.15.140:0/1147682188 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:34.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86ebfff700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec0aed20 0x7f86ec0b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497935038 0 0) 0x7f86ec0b1520 con 0x7f86ec063b00 2022-07-21T01:48:34.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86cc003260 con 0x7f86ec063b00 2022-07-21T01:48:34.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 53604933 0 0) 0x7f86ec0b1750 con 0x7f86ec0ad3a0 2022-07-21T01:48:34.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86ec0b1520 con 0x7f86ec0ad3a0 2022-07-21T01:48:34.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 557254920 0 0) 0x7f86cc003260 con 0x7f86ec063b00 2022-07-21T01:48:34.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec0aed20 msgr2=0x7f86ec0b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:34.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec0aed20 0x7f86ec0b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 >> v1:172.21.15.152:6789/0 conn(0x7f86ec0ad3a0 legacy=0x7f86ec143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec0b19f0 con 0x7f86ec063b00 2022-07-21T01:48:34.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f86e4001a00 con 0x7f86ec063b00 2022-07-21T01:48:34.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 >> v1:172.21.15.140:6789/0 conn(0x7f86ec063b00 legacy=0x7f86ec13ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86cc003910 0x7f86cc003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86cc004500 0x7f86cc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec0aed20 0x7f86cc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec0adb60 con 0x7f86cc003910 2022-07-21T01:48:34.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc00d990 con 0x7f86cc004500 2022-07-21T01:48:34.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc00db50 con 0x7f86ec0aed20 2022-07-21T01:48:34.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86ebfff700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86cc003910 0x7f86cc003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86e3fff700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86cc004500 0x7f86cc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86f0bd1700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec0aed20 0x7f86cc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86ebfff700 1 -- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec0aed20 msgr2=0x7f86cc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:34.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86ebfff700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec0aed20 0x7f86cc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86ebfff700 1 -- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86cc004500 msgr2=0x7f86cc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:34.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86ebfff700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86cc004500 0x7f86cc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86ebfff700 1 -- 172.21.15.140:0/1147682188 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc003260 con 0x7f86cc003910 2022-07-21T01:48:34.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86f0bd1700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec0aed20 0x7f86cc004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:34.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86ebfff700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86cc003910 0x7f86cc003d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f86dc00c000 tx=0x7f86dc006df0).ready entity=mon.0 client_cookie=4713d3dfb127ec9c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:34.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86dc00e830 con 0x7f86cc003910 2022-07-21T01:48:34.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc00e990 con 0x7f86cc003910 2022-07-21T01:48:34.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.467+0000 7f86eb7fe700 1 -- 172.21.15.140:0/1147682188 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86dc00ecf0 con 0x7f86cc003910 2022-07-21T01:48:34.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 -- 172.21.15.140:0/1147682188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86cc003910 msgr2=0x7f86cc003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 --2- 172.21.15.140:0/1147682188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86cc003910 0x7f86cc003d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f86dc00c000 tx=0x7f86dc006df0).stop 2022-07-21T01:48:34.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f0bd1700 1 -- 172.21.15.140:0/1147682188 reap_dead start 2022-07-21T01:48:34.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 -- 172.21.15.140:0/1147682188 shutdown_connections 2022-07-21T01:48:34.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 -- 172.21.15.140:0/1147682188 >> 172.21.15.140:0/1147682188 conn(0x7f86ec0611c0 msgr2=0x7f86ec062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:34.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 -- 172.21.15.140:0/1147682188 shutdown_connections 2022-07-21T01:48:34.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 -- 172.21.15.140:0/1147682188 wait complete. 2022-07-21T01:48:34.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 Processor -- start 2022-07-21T01:48:34.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 -- start start 2022-07-21T01:48:34.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec13d300 0x7f86ec1419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86ec063fb0 0x7f86ec1385f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86ec138ae0 0x7f86ec064f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec064b40 con 0x7f86ec063fb0 2022-07-21T01:48:34.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec13d6e0 con 0x7f86ec138ae0 2022-07-21T01:48:34.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f2634700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ec13c7c0 con 0x7f86ec13d300 2022-07-21T01:48:34.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86e3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86ec063fb0 0x7f86ec1385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86e3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86ec063fb0 0x7f86ec1385f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60616/0 (socket says 172.21.15.140:60616) 2022-07-21T01:48:34.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86e3fff700 1 -- 172.21.15.140:0/374574270 learned_addr learned my addr 172.21.15.140:0/374574270 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:34.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86ebfff700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec13d300 0x7f86ec1419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f0bd1700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86ec138ae0 0x7f86ec064f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86e3fff700 1 -- 172.21.15.140:0/374574270 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec13d300 msgr2=0x7f86ec1419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86e3fff700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec13d300 0x7f86ec1419f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86e3fff700 1 -- 172.21.15.140:0/374574270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86ec138ae0 msgr2=0x7f86ec064f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86e3fff700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86ec138ae0 0x7f86ec064f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86e3fff700 1 -- 172.21.15.140:0/374574270 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ec13ca60 con 0x7f86ec063fb0 2022-07-21T01:48:34.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86ebfff700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec13d300 0x7f86ec1419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:34.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86f0bd1700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86ec138ae0 0x7f86ec064f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:34.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.471+0000 7f86e3fff700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86ec063fb0 0x7f86ec1385f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f86d400d000 tx=0x7f86d4007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:34.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.475+0000 7f86e9ffb700 1 -- 172.21.15.140:0/374574270 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86d400fac0 con 0x7f86ec063fb0 2022-07-21T01:48:34.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.475+0000 7f86e9ffb700 1 -- 172.21.15.140:0/374574270 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d400fc20 con 0x7f86ec063fb0 2022-07-21T01:48:34.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.475+0000 7f86e9ffb700 1 -- 172.21.15.140:0/374574270 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86d4014450 con 0x7f86ec063fb0 2022-07-21T01:48:34.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.475+0000 7f86f2634700 1 -- 172.21.15.140:0/374574270 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ec141ff0 con 0x7f86ec063fb0 2022-07-21T01:48:34.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.475+0000 7f86f2634700 1 -- 172.21.15.140:0/374574270 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ec139100 con 0x7f86ec063fb0 2022-07-21T01:48:34.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.475+0000 7f86f2634700 1 -- 172.21.15.140:0/374574270 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ec0574e0 con 0x7f86ec063fb0 2022-07-21T01:48:34.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.475+0000 7f86e9ffb700 1 -- 172.21.15.140:0/374574270 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f86d401b030 con 0x7f86ec063fb0 2022-07-21T01:48:34.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.475+0000 7f86e9ffb700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d0053a70 0x7f86d0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.475+0000 7f86e9ffb700 1 -- 172.21.15.140:0/374574270 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f86d406c1e0 con 0x7f86ec063fb0 2022-07-21T01:48:34.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.475+0000 7f86ebfff700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d0053a70 0x7f86d0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.479+0000 7f86ebfff700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d0053a70 0x7f86d0055f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f86dc01a000 tx=0x7f86dc019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:34.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.479+0000 7f86e9ffb700 1 -- 172.21.15.140:0/374574270 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f86d40430a0 con 0x7f86ec063fb0 2022-07-21T01:48:34.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.631+0000 7f86f2634700 1 -- 172.21.15.140:0/374574270 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f86ec0b3780 con 0x7f86ec063fb0 2022-07-21T01:48:34.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.635+0000 7f86e9ffb700 1 -- 172.21.15.140:0/374574270 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+131782 (secure 0 0 0) 0x7f86d40466c0 con 0x7f86ec063fb0 2022-07-21T01:48:34.641 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:34.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 -- 172.21.15.140:0/374574270 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d0053a70 msgr2=0x7f86d0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d0053a70 0x7f86d0055f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f86dc01a000 tx=0x7f86dc019000).stop 2022-07-21T01:48:34.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 -- 172.21.15.140:0/374574270 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86ec063fb0 msgr2=0x7f86ec1385f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86ec063fb0 0x7f86ec1385f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f86d400d000 tx=0x7f86d4007a00).stop 2022-07-21T01:48:34.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 -- 172.21.15.140:0/374574270 shutdown_connections 2022-07-21T01:48:34.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d0053a70 0x7f86d0055f30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86ec138ae0 0x7f86ec064f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86ec063fb0 0x7f86ec1385f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 --2- 172.21.15.140:0/374574270 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86ec13d300 0x7f86ec1419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 -- 172.21.15.140:0/374574270 >> 172.21.15.140:0/374574270 conn(0x7f86cc003910 msgr2=0x7f86ec0afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:34.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 -- 172.21.15.140:0/374574270 shutdown_connections 2022-07-21T01:48:34.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.639+0000 7f86f2634700 1 -- 172.21.15.140:0/374574270 wait complete. 2022-07-21T01:48:34.652 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":4,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6800","nonce":22519},{"type":"v1","addr":"172.21.15.140:6801","nonce":22519}]},"active_addr":"172.21.15.140:6801/22519","active_change":"2022-07-21T01:48:17.886106+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.140:0","nonce":3492780042}]},{"addrvec":[{"type":"v2","addr":"172.21.15.140:0","nonce":2781194106}]},{"addrvec":[{"type":"v2","addr":"172.21.15.140:0","nonce":3457595944}]}]}} 2022-07-21T01:48:34.658 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2022-07-21T01:48:34.659 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2022-07-21T01:48:34.659 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:34.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.763+0000 7f6182d62700 1 Processor -- start 2022-07-21T01:48:34.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.763+0000 7f6182d62700 1 -- start start 2022-07-21T01:48:34.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f6182d62700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f617c0acef0 0x7f617c14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f6182d62700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f617c05d480 con 0x7f617c05c3f0 2022-07-21T01:48:34.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f6182d62700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f617c0b21e0 con 0x7f617c0b1e00 2022-07-21T01:48:34.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f6182d62700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617c05cf40 con 0x7f617c0acef0 2022-07-21T01:48:34.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f61812ff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f617c0acef0 0x7f617c14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f617bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f617c05c3f0 0x7f617c146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46536/0 (socket says 172.21.15.140:46536) 2022-07-21T01:48:34.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f617bfff700 1 -- 172.21.15.140:0/1023722077 learned_addr learned my addr 172.21.15.140:0/1023722077 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:34.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f617b7fe700 1 -- 172.21.15.140:0/1023722077 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085969508 0 0) 0x7f617c05d480 con 0x7f617c05c3f0 2022-07-21T01:48:34.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f617b7fe700 1 -- 172.21.15.140:0/1023722077 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f615c003260 con 0x7f617c05c3f0 2022-07-21T01:48:34.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f61812ff700 1 -- 172.21.15.140:0/1023722077 >> v1:172.21.15.152:6789/0 conn(0x7f617c0b1e00 legacy=0x7f617c0ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f61812ff700 1 -- 172.21.15.140:0/1023722077 >> v1:172.21.15.140:6789/0 conn(0x7f617c05c3f0 legacy=0x7f617c146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f617b7fe700 1 -- 172.21.15.140:0/1023722077 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3662346571 0 0) 0x7f615c003260 con 0x7f617c05c3f0 2022-07-21T01:48:34.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f61812ff700 1 -- 172.21.15.140:0/1023722077 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c05d1e0 con 0x7f617c0acef0 2022-07-21T01:48:34.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f61812ff700 1 --2- 172.21.15.140:0/1023722077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f617c0acef0 0x7f617c14a6d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f6164001800 tx=0x7f6174006df0).ready entity=mon.0 client_cookie=49fc6c118e47d2f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:34.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f617b7fe700 1 -- 172.21.15.140:0/1023722077 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6164001660 con 0x7f617c0acef0 2022-07-21T01:48:34.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f617b7fe700 1 -- 172.21.15.140:0/1023722077 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6174000e90 con 0x7f617c0acef0 2022-07-21T01:48:34.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f617b7fe700 1 -- 172.21.15.140:0/1023722077 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f617400d4f0 con 0x7f617c0acef0 2022-07-21T01:48:34.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f6182d62700 1 -- 172.21.15.140:0/1023722077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f617c0acef0 msgr2=0x7f617c14a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f6182d62700 1 --2- 172.21.15.140:0/1023722077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f617c0acef0 0x7f617c14a6d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f6164001800 tx=0x7f6174006df0).stop 2022-07-21T01:48:34.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f6182d62700 1 -- 172.21.15.140:0/1023722077 shutdown_connections 2022-07-21T01:48:34.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f6182d62700 1 --2- 172.21.15.140:0/1023722077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f617c0acef0 0x7f617c14a6d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f6182d62700 1 -- 172.21.15.140:0/1023722077 >> 172.21.15.140:0/1023722077 conn(0x7f617c0611c0 msgr2=0x7f617c062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:34.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f6182d62700 1 -- 172.21.15.140:0/1023722077 shutdown_connections 2022-07-21T01:48:34.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.767+0000 7f6182d62700 1 -- 172.21.15.140:0/1023722077 wait complete. 2022-07-21T01:48:34.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6182d62700 1 Processor -- start 2022-07-21T01:48:34.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6182d62700 1 -- start start 2022-07-21T01:48:34.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6182d62700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f617c0acef0 0x7f617c066f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6182d62700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617c0b1e00 0x7f617c148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6182d62700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f617c0675c0 0x7f617c13d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6182d62700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c064e90 con 0x7f617c0acef0 2022-07-21T01:48:34.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6182d62700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c063e30 con 0x7f617c0b1e00 2022-07-21T01:48:34.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6182d62700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617c0647e0 con 0x7f617c0675c0 2022-07-21T01:48:34.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6180afe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f617c0acef0 0x7f617c066f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6180afe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f617c0acef0 0x7f617c066f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60626/0 (socket says 172.21.15.140:60626) 2022-07-21T01:48:34.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f617bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617c0b1e00 0x7f617c148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6180afe700 1 -- 172.21.15.140:0/2735250774 learned_addr learned my addr 172.21.15.140:0/2735250774 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:34.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f61812ff700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f617c0675c0 0x7f617c13d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6180afe700 1 -- 172.21.15.140:0/2735250774 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f617c0675c0 msgr2=0x7f617c13d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6180afe700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f617c0675c0 0x7f617c13d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6180afe700 1 -- 172.21.15.140:0/2735250774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617c0b1e00 msgr2=0x7f617c148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6180afe700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617c0b1e00 0x7f617c148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6180afe700 1 -- 172.21.15.140:0/2735250774 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617400b180 con 0x7f617c0acef0 2022-07-21T01:48:34.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6180afe700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f617c0acef0 0x7f617c066f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f617c065a90 tx=0x7f616c001400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:34.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6179ffb700 1 -- 172.21.15.140:0/2735250774 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f616c00f470 con 0x7f617c0acef0 2022-07-21T01:48:34.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6179ffb700 1 -- 172.21.15.140:0/2735250774 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616c00f5d0 con 0x7f617c0acef0 2022-07-21T01:48:34.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6182d62700 1 -- 172.21.15.140:0/2735250774 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617c064380 con 0x7f617c0acef0 2022-07-21T01:48:34.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6179ffb700 1 -- 172.21.15.140:0/2735250774 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f616c00f930 con 0x7f617c0acef0 2022-07-21T01:48:34.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.771+0000 7f6182d62700 1 -- 172.21.15.140:0/2735250774 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617c0aebf0 con 0x7f617c0acef0 2022-07-21T01:48:34.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.775+0000 7f6182d62700 1 -- 172.21.15.140:0/2735250774 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617c0574e0 con 0x7f617c0acef0 2022-07-21T01:48:34.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.775+0000 7f6179ffb700 1 -- 172.21.15.140:0/2735250774 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f616c019030 con 0x7f617c0acef0 2022-07-21T01:48:34.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.775+0000 7f6179ffb700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61600539e0 0x7f6160055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:34.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.775+0000 7f617bfff700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61600539e0 0x7f6160055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:34.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.775+0000 7f6179ffb700 1 -- 172.21.15.140:0/2735250774 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f616c06b5f0 con 0x7f617c0acef0 2022-07-21T01:48:34.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.775+0000 7f617bfff700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61600539e0 0x7f6160055ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f616400d000 tx=0x7f6164000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:34.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.783+0000 7f6179ffb700 1 -- 172.21.15.140:0/2735250774 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f616c0423a0 con 0x7f617c0acef0 2022-07-21T01:48:34.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.907+0000 7f6182d62700 1 -- 172.21.15.140:0/2735250774 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f617c064510 con 0x7f617c0acef0 2022-07-21T01:48:34.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.907+0000 7f6179ffb700 1 -- 172.21.15.140:0/2735250774 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f616c0459c0 con 0x7f617c0acef0 2022-07-21T01:48:34.907 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:34.908 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:34.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 -- 172.21.15.140:0/2735250774 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61600539e0 msgr2=0x7f6160055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61600539e0 0x7f6160055ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f616400d000 tx=0x7f6164000bc0).stop 2022-07-21T01:48:34.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 -- 172.21.15.140:0/2735250774 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f617c0acef0 msgr2=0x7f617c066f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:34.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f617c0acef0 0x7f617c066f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f617c065a90 tx=0x7f616c001400).stop 2022-07-21T01:48:34.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 -- 172.21.15.140:0/2735250774 shutdown_connections 2022-07-21T01:48:34.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61600539e0 0x7f6160055ea0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f617c0675c0 0x7f617c13d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f617c0b1e00 0x7f617c148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 --2- 172.21.15.140:0/2735250774 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f617c0acef0 0x7f617c066f40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:34.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 -- 172.21.15.140:0/2735250774 >> 172.21.15.140:0/2735250774 conn(0x7f617c05c3f0 msgr2=0x7f617c069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:34.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 -- 172.21.15.140:0/2735250774 shutdown_connections 2022-07-21T01:48:34.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:34.911+0000 7f6182d62700 1 -- 172.21.15.140:0/2735250774 wait complete. 2022-07-21T01:48:34.921 INFO:tasks.ceph.ceph_manager.ceph:all up! 2022-07-21T01:48:34.921 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:35.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 Processor -- start 2022-07-21T01:48:35.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 -- start start 2022-07-21T01:48:35.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f96e805d160 0x7f96e80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f96e80b2960 0x7f96e8142210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e80ba2d0 con 0x7f96e80b9ef0 2022-07-21T01:48:35.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80b4560 con 0x7f96e805d160 2022-07-21T01:48:35.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80b4800 con 0x7f96e80b2960 2022-07-21T01:48:35.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ec9a7700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f96e805d160 0x7f96e80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ed1a8700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f96e80b9ef0 0x7f96e80b2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46548/0 (socket says 172.21.15.140:46548) 2022-07-21T01:48:35.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ed1a8700 1 -- 172.21.15.140:0/388435262 learned_addr learned my addr 172.21.15.140:0/388435262 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:35.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ed9a9700 1 --2- 172.21.15.140:0/388435262 >> v2:172.21.15.152:3300/0 conn(0x7f96e80b2960 0x7f96e8142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ec9a7700 1 -- 172.21.15.140:0/388435262 >> v2:172.21.15.152:3300/0 conn(0x7f96e80b2960 msgr2=0x7f96e8142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:35.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ec9a7700 1 --2- 172.21.15.140:0/388435262 >> v2:172.21.15.152:3300/0 conn(0x7f96e80b2960 0x7f96e8142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96dffff700 1 -- 172.21.15.140:0/388435262 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696120987 0 0) 0x7f96e80ba2d0 con 0x7f96e80b9ef0 2022-07-21T01:48:35.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ec9a7700 1 -- 172.21.15.140:0/388435262 >> v1:172.21.15.140:6789/0 conn(0x7f96e80b9ef0 legacy=0x7f96e80b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ec9a7700 1 -- 172.21.15.140:0/388435262 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80b1db0 con 0x7f96e805d160 2022-07-21T01:48:35.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ec9a7700 1 --2- 172.21.15.140:0/388435262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96e805d160 0x7f96e80b4e50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f96d8002f70 tx=0x7f96d8006df0).ready entity=mon.0 client_cookie=ca3697c3637448ec server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96dffff700 1 -- 172.21.15.140:0/388435262 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d800d5c0 con 0x7f96e805d160 2022-07-21T01:48:35.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96dffff700 1 -- 172.21.15.140:0/388435262 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d8000e90 con 0x7f96e805d160 2022-07-21T01:48:35.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96dffff700 1 -- 172.21.15.140:0/388435262 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d800d910 con 0x7f96e805d160 2022-07-21T01:48:35.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 -- 172.21.15.140:0/388435262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96e805d160 msgr2=0x7f96e80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 --2- 172.21.15.140:0/388435262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96e805d160 0x7f96e80b4e50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f96d8002f70 tx=0x7f96d8006df0).stop 2022-07-21T01:48:35.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 -- 172.21.15.140:0/388435262 shutdown_connections 2022-07-21T01:48:35.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 --2- 172.21.15.140:0/388435262 >> v2:172.21.15.152:3300/0 conn(0x7f96e80b2960 0x7f96e8142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 --2- 172.21.15.140:0/388435262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96e805d160 0x7f96e80b4e50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 -- 172.21.15.140:0/388435262 >> 172.21.15.140:0/388435262 conn(0x7f96e80679d0 msgr2=0x7f96e80b94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 -- 172.21.15.140:0/388435262 shutdown_connections 2022-07-21T01:48:35.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.027+0000 7f96ef40c700 1 -- 172.21.15.140:0/388435262 wait complete. 2022-07-21T01:48:35.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ef40c700 1 Processor -- start 2022-07-21T01:48:35.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ef40c700 1 -- start start 2022-07-21T01:48:35.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ef40c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96e805d160 0x7f96e80b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ef40c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96e80b2960 0x7f96e8152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ef40c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96e80b9ef0 0x7f96e8140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ef40c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8147450 con 0x7f96e805d160 2022-07-21T01:48:35.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ef40c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e813f680 con 0x7f96e80b9ef0 2022-07-21T01:48:35.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ef40c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e81476f0 con 0x7f96e80b2960 2022-07-21T01:48:35.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ed1a8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96e805d160 0x7f96e80b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ed1a8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96e805d160 0x7f96e80b7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60644/0 (socket says 172.21.15.140:60644) 2022-07-21T01:48:35.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ed1a8700 1 -- 172.21.15.140:0/3831565031 learned_addr learned my addr 172.21.15.140:0/3831565031 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:35.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ed9a9700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96e80b9ef0 0x7f96e8140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ec9a7700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96e80b2960 0x7f96e8152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ed1a8700 1 -- 172.21.15.140:0/3831565031 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96e80b2960 msgr2=0x7f96e8152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ed1a8700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96e80b2960 0x7f96e8152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ed1a8700 1 -- 172.21.15.140:0/3831565031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96e80b9ef0 msgr2=0x7f96e8140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ed1a8700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96e80b9ef0 0x7f96e8140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ed1a8700 1 -- 172.21.15.140:0/3831565031 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d800b180 con 0x7f96e805d160 2022-07-21T01:48:35.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ed9a9700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96e80b9ef0 0x7f96e8140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:35.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ed1a8700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96e805d160 0x7f96e80b7f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f96e4001250 tx=0x7f96e40013f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96de7fc700 1 -- 172.21.15.140:0/3831565031 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e4005ba0 con 0x7f96e805d160 2022-07-21T01:48:35.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96de7fc700 1 -- 172.21.15.140:0/3831565031 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e4005d00 con 0x7f96e805d160 2022-07-21T01:48:35.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96de7fc700 1 -- 172.21.15.140:0/3831565031 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e40072c0 con 0x7f96e805d160 2022-07-21T01:48:35.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ef40c700 1 -- 172.21.15.140:0/3831565031 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e813fb20 con 0x7f96e805d160 2022-07-21T01:48:35.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.031+0000 7f96ef40c700 1 -- 172.21.15.140:0/3831565031 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e8152d30 con 0x7f96e805d160 2022-07-21T01:48:35.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.035+0000 7f96ef40c700 1 -- 172.21.15.140:0/3831565031 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e80b3780 con 0x7f96e805d160 2022-07-21T01:48:35.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.035+0000 7f96de7fc700 1 -- 172.21.15.140:0/3831565031 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f96e4010070 con 0x7f96e805d160 2022-07-21T01:48:35.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.035+0000 7f96de7fc700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d0053b00 0x7f96d0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.035+0000 7f96de7fc700 1 -- 172.21.15.140:0/3831565031 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f96e406b730 con 0x7f96e805d160 2022-07-21T01:48:35.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.039+0000 7f96ec9a7700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d0053b00 0x7f96d0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.039+0000 7f96ec9a7700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d0053b00 0x7f96d0055fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f96d801a000 tx=0x7f96d8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.043+0000 7f96de7fc700 1 -- 172.21.15.140:0/3831565031 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96e4042650 con 0x7f96e805d160 2022-07-21T01:48:35.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.167+0000 7f96ef40c700 1 -- 172.21.15.140:0/3831565031 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96e8069a60 con 0x7f96e805d160 2022-07-21T01:48:35.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.171+0000 7f96de7fc700 1 -- 172.21.15.140:0/3831565031 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f96e4018070 con 0x7f96e805d160 2022-07-21T01:48:35.170 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:35.171 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:35.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 -- 172.21.15.140:0/3831565031 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d0053b00 msgr2=0x7f96d0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d0053b00 0x7f96d0055fc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f96d801a000 tx=0x7f96d8014000).stop 2022-07-21T01:48:35.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 -- 172.21.15.140:0/3831565031 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96e805d160 msgr2=0x7f96e80b7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96e805d160 0x7f96e80b7f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f96e4001250 tx=0x7f96e40013f0).stop 2022-07-21T01:48:35.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 -- 172.21.15.140:0/3831565031 shutdown_connections 2022-07-21T01:48:35.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d0053b00 0x7f96d0055fc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96e80b9ef0 0x7f96e8140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96e80b2960 0x7f96e8152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 --2- 172.21.15.140:0/3831565031 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96e805d160 0x7f96e80b7f70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 -- 172.21.15.140:0/3831565031 >> 172.21.15.140:0/3831565031 conn(0x7f96e80679d0 msgr2=0x7f96e8143850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 -- 172.21.15.140:0/3831565031 shutdown_connections 2022-07-21T01:48:35.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.175+0000 7f96ef40c700 1 -- 172.21.15.140:0/3831565031 wait complete. 2022-07-21T01:48:35.188 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2022-07-21T01:48:35.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.291+0000 7fbf17fff700 1 Processor -- start 2022-07-21T01:48:35.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.291+0000 7fbf17fff700 1 -- start start 2022-07-21T01:48:35.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.291+0000 7fbf17fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbf180acae0 0x7fbf1813ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.291+0000 7fbf17fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf180ada70 con 0x7fbf180b5970 2022-07-21T01:48:35.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.291+0000 7fbf17fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf180b5d50 con 0x7fbf180b1060 2022-07-21T01:48:35.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.291+0000 7fbf17fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf180af8c0 con 0x7fbf180acae0 2022-07-21T01:48:35.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.291+0000 7fbf16ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fbf180b5970 0x7fbf1813c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46562/0 (socket says 172.21.15.140:46562) 2022-07-21T01:48:35.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.291+0000 7fbf16ffd700 1 -- 172.21.15.140:0/1333200899 learned_addr learned my addr 172.21.15.140:0/1333200899 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:35.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.291+0000 7fbf177fe700 1 --2- 172.21.15.140:0/1333200899 >> v2:172.21.15.140:3300/0 conn(0x7fbf180acae0 0x7fbf1813ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf15ffb700 1 -- 172.21.15.140:0/1333200899 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608094874 0 0) 0x7fbf180ada70 con 0x7fbf180b5970 2022-07-21T01:48:35.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf15ffb700 1 -- 172.21.15.140:0/1333200899 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbefc003260 con 0x7fbf180b5970 2022-07-21T01:48:35.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf15ffb700 1 -- 172.21.15.140:0/1333200899 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573640988 0 0) 0x7fbf180b5d50 con 0x7fbf180b1060 2022-07-21T01:48:35.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf15ffb700 1 -- 172.21.15.140:0/1333200899 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf180ada70 con 0x7fbf180b1060 2022-07-21T01:48:35.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf177fe700 1 -- 172.21.15.140:0/1333200899 >> v1:172.21.15.152:6789/0 conn(0x7fbf180b1060 legacy=0x7fbf1813f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf177fe700 1 -- 172.21.15.140:0/1333200899 >> v1:172.21.15.140:6789/0 conn(0x7fbf180b5970 legacy=0x7fbf1813c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf177fe700 1 -- 172.21.15.140:0/1333200899 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180adcd0 con 0x7fbf180acae0 2022-07-21T01:48:35.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf177fe700 1 --2- 172.21.15.140:0/1333200899 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf180acae0 0x7fbf1813ffd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fbf100032f0 tx=0x7fbf100091d0).ready entity=mon.0 client_cookie=47740cca8a18a1ed server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf15ffb700 1 -- 172.21.15.140:0/1333200899 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf1000b600 con 0x7fbf180acae0 2022-07-21T01:48:35.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf15ffb700 1 -- 172.21.15.140:0/1333200899 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1000b760 con 0x7fbf180acae0 2022-07-21T01:48:35.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf15ffb700 1 -- 172.21.15.140:0/1333200899 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf1000b8d0 con 0x7fbf180acae0 2022-07-21T01:48:35.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf17fff700 1 -- 172.21.15.140:0/1333200899 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf180acae0 msgr2=0x7fbf1813ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf17fff700 1 --2- 172.21.15.140:0/1333200899 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf180acae0 0x7fbf1813ffd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fbf100032f0 tx=0x7fbf100091d0).stop 2022-07-21T01:48:35.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf17fff700 1 -- 172.21.15.140:0/1333200899 shutdown_connections 2022-07-21T01:48:35.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf17fff700 1 --2- 172.21.15.140:0/1333200899 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf180acae0 0x7fbf1813ffd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf17fff700 1 -- 172.21.15.140:0/1333200899 >> 172.21.15.140:0/1333200899 conn(0x7fbf18063490 msgr2=0x7fbf180b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf17fff700 1 -- 172.21.15.140:0/1333200899 shutdown_connections 2022-07-21T01:48:35.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf17fff700 1 -- 172.21.15.140:0/1333200899 wait complete. 2022-07-21T01:48:35.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf17fff700 1 Processor -- start 2022-07-21T01:48:35.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.295+0000 7fbf17fff700 1 -- start start 2022-07-21T01:48:35.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf17fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf180acae0 0x7fbf18066040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf17fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf180b1060 0x7fbf181383b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf17fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf180b5970 0x7fbf1814fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf17fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf18140ef0 con 0x7fbf180acae0 2022-07-21T01:48:35.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf17fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180649e0 con 0x7fbf180b5970 2022-07-21T01:48:35.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf17fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180652e0 con 0x7fbf180b1060 2022-07-21T01:48:35.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf16ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf180acae0 0x7fbf18066040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf16ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf180acae0 0x7fbf18066040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60654/0 (socket says 172.21.15.140:60654) 2022-07-21T01:48:35.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf16ffd700 1 -- 172.21.15.140:0/1230050698 learned_addr learned my addr 172.21.15.140:0/1230050698 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:35.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf167fc700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf180b1060 0x7fbf181383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf177fe700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf180b5970 0x7fbf1814fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf16ffd700 1 -- 172.21.15.140:0/1230050698 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf180b1060 msgr2=0x7fbf181383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf16ffd700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf180b1060 0x7fbf181383b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf16ffd700 1 -- 172.21.15.140:0/1230050698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf180b5970 msgr2=0x7fbf1814fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf16ffd700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf180b5970 0x7fbf1814fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf16ffd700 1 -- 172.21.15.140:0/1230050698 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009470 con 0x7fbf180acae0 2022-07-21T01:48:35.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf177fe700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf180b5970 0x7fbf1814fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:35.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf16ffd700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf180acae0 0x7fbf18066040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fbf08001110 tx=0x7fbf08007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf07fff700 1 -- 172.21.15.140:0/1230050698 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf080070d0 con 0x7fbf180acae0 2022-07-21T01:48:35.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf07fff700 1 -- 172.21.15.140:0/1230050698 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf08007230 con 0x7fbf180acae0 2022-07-21T01:48:35.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf17fff700 1 -- 172.21.15.140:0/1230050698 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf18141120 con 0x7fbf180acae0 2022-07-21T01:48:35.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf07fff700 1 -- 172.21.15.140:0/1230050698 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf08005610 con 0x7fbf180acae0 2022-07-21T01:48:35.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf17fff700 1 -- 172.21.15.140:0/1230050698 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf180639e0 con 0x7fbf180acae0 2022-07-21T01:48:35.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf17fff700 1 -- 172.21.15.140:0/1230050698 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbf180af4c0 con 0x7fbf180acae0 2022-07-21T01:48:35.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.299+0000 7fbf07fff700 1 -- 172.21.15.140:0/1230050698 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbf08010070 con 0x7fbf180acae0 2022-07-21T01:48:35.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.303+0000 7fbf07fff700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbef4053a70 0x7fbef4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.303+0000 7fbf07fff700 1 -- 172.21.15.140:0/1230050698 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbf0806b5c0 con 0x7fbf180acae0 2022-07-21T01:48:35.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.303+0000 7fbf167fc700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbef4053a70 0x7fbef4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.303+0000 7fbf07fff700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x7fbef405add0 0x7fbef405d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.303+0000 7fbf07fff700 1 -- 172.21.15.140:0/1230050698 --> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbef405d810 con 0x7fbef405add0 2022-07-21T01:48:35.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.303+0000 7fbf07fff700 1 -- 172.21.15.140:0/1230050698 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf0806dc50 con 0x7fbf180acae0 2022-07-21T01:48:35.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.303+0000 7fbf177fe700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x7fbef405add0 0x7fbef405d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.303+0000 7fbf167fc700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbef4053a70 0x7fbef4055f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fbf0c00b000 tx=0x7fbf0c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.303+0000 7fbf177fe700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x7fbef405add0 0x7fbef405d230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.303+0000 7fbf07fff700 1 -- 172.21.15.140:0/1230050698 <== osd.0 v2:172.21.15.140:6802/22710 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7fbef405d810 con 0x7fbef405add0 2022-07-21T01:48:35.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.319+0000 7fbf17fff700 1 -- 172.21.15.140:0/1230050698 --> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbf180650d0 con 0x7fbef405add0 2022-07-21T01:48:35.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.319+0000 7fbf07fff700 1 -- 172.21.15.140:0/1230050698 <== osd.0 v2:172.21.15.140:6802/22710 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbf180650d0 con 0x7fbef405add0 2022-07-21T01:48:35.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.319+0000 7fbf17fff700 1 -- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x7fbef405add0 msgr2=0x7fbef405d230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.319+0000 7fbf17fff700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x7fbef405add0 0x7fbef405d230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.319+0000 7fbf17fff700 1 -- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbef4053a70 msgr2=0x7fbef4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.319+0000 7fbf17fff700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbef4053a70 0x7fbef4055f30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fbf0c00b000 tx=0x7fbf0c009040).stop 2022-07-21T01:48:35.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.323+0000 7fbf17fff700 1 -- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf180acae0 msgr2=0x7fbf18066040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.323+0000 7fbf17fff700 1 --2- 172.21.15.140:0/1230050698 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf180acae0 0x7fbf18066040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fbf08001110 tx=0x7fbf08007ad0).stop 2022-07-21T01:48:35.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.323+0000 7fbf177fe700 1 -- 172.21.15.140:0/1230050698 reap_dead start 2022-07-21T01:48:35.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.323+0000 7fbf17fff700 1 -- 172.21.15.140:0/1230050698 shutdown_connections 2022-07-21T01:48:35.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.323+0000 7fbf17fff700 1 -- 172.21.15.140:0/1230050698 >> 172.21.15.140:0/1230050698 conn(0x7fbf18063490 msgr2=0x7fbf181452d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.323+0000 7fbf17fff700 1 -- 172.21.15.140:0/1230050698 shutdown_connections 2022-07-21T01:48:35.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.323+0000 7fbf17fff700 1 -- 172.21.15.140:0/1230050698 wait complete. 2022-07-21T01:48:35.331 INFO:teuthology.orchestra.run.smithi140.stdout:55834574854 2022-07-21T01:48:35.332 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2022-07-21T01:48:35.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6ae1924700 1 Processor -- start 2022-07-21T01:48:35.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6ae1924700 1 -- start start 2022-07-21T01:48:35.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6ae1924700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6adc0af8c0 0x7f6adc0ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6ae1924700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6adc143a10 0x7f6adc143e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6ae1924700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6adc0b0b80 con 0x7f6adc0b5970 2022-07-21T01:48:35.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6ae1924700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6adc05d4d0 con 0x7f6adc0af8c0 2022-07-21T01:48:35.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6ae1924700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6adc0b5d50 con 0x7f6adc143a10 2022-07-21T01:48:35.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6adbfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6adc0af8c0 0x7f6adc0ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6adbfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6adc0af8c0 0x7f6adc0ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60664/0 (socket says 172.21.15.140:60664) 2022-07-21T01:48:35.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6adbfff700 1 -- 172.21.15.140:0/3329556188 learned_addr learned my addr 172.21.15.140:0/3329556188 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:35.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6ae1123700 1 --2- 172.21.15.140:0/3329556188 >> v2:172.21.15.152:3300/0 conn(0x7f6adc143a10 0x7f6adc143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6adbfff700 1 -- 172.21.15.140:0/3329556188 >> v2:172.21.15.152:3300/0 conn(0x7f6adc143a10 msgr2=0x7f6adc143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:35.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6adbfff700 1 --2- 172.21.15.140:0/3329556188 >> v2:172.21.15.152:3300/0 conn(0x7f6adc143a10 0x7f6adc143e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6adbfff700 1 -- 172.21.15.140:0/3329556188 >> v1:172.21.15.152:6789/0 conn(0x7f6adc0b5970 legacy=0x7f6adc0ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:35.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6adbfff700 1 -- 172.21.15.140:0/3329556188 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc0b0db0 con 0x7f6adc0af8c0 2022-07-21T01:48:35.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6adbfff700 1 --2- 172.21.15.140:0/3329556188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6adc0af8c0 0x7f6adc0ae810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6adc05d010 tx=0x7f6ac4009290).ready entity=mon.0 client_cookie=ee1d66ffe0388c62 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6adb7fe700 1 -- 172.21.15.140:0/3329556188 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac400b830 con 0x7f6adc0af8c0 2022-07-21T01:48:35.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6adb7fe700 1 -- 172.21.15.140:0/3329556188 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac400b990 con 0x7f6adc0af8c0 2022-07-21T01:48:35.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.439+0000 7f6adb7fe700 1 -- 172.21.15.140:0/3329556188 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac400bb40 con 0x7f6adc0af8c0 2022-07-21T01:48:35.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 -- 172.21.15.140:0/3329556188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6adc0af8c0 msgr2=0x7f6adc0ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 --2- 172.21.15.140:0/3329556188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6adc0af8c0 0x7f6adc0ae810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6adc05d010 tx=0x7f6ac4009290).stop 2022-07-21T01:48:35.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 -- 172.21.15.140:0/3329556188 shutdown_connections 2022-07-21T01:48:35.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 --2- 172.21.15.140:0/3329556188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6adc0af8c0 0x7f6adc0ae810 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 --2- 172.21.15.140:0/3329556188 >> v2:172.21.15.152:3300/0 conn(0x7f6adc143a10 0x7f6adc143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 -- 172.21.15.140:0/3329556188 >> 172.21.15.140:0/3329556188 conn(0x7f6adc063490 msgr2=0x7f6adc0b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 -- 172.21.15.140:0/3329556188 shutdown_connections 2022-07-21T01:48:35.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 -- 172.21.15.140:0/3329556188 wait complete. 2022-07-21T01:48:35.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 Processor -- start 2022-07-21T01:48:35.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 -- start start 2022-07-21T01:48:35.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6adc0af8c0 0x7f6adc0ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6adc0b5970 0x7f6adc156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6adc143a10 0x7f6adc156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc10e110 con 0x7f6adc0af8c0 2022-07-21T01:48:35.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc10e340 con 0x7f6adc0b5970 2022-07-21T01:48:35.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae1924700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc10e5e0 con 0x7f6adc143a10 2022-07-21T01:48:35.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae0922700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6adc0af8c0 0x7f6adc0ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae0922700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6adc0af8c0 0x7f6adc0ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60670/0 (socket says 172.21.15.140:60670) 2022-07-21T01:48:35.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.443+0000 7f6ae0922700 1 -- 172.21.15.140:0/2481125407 learned_addr learned my addr 172.21.15.140:0/2481125407 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:35.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6adbfff700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6adc0b5970 0x7f6adc156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ae1123700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6adc143a10 0x7f6adc156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ae0922700 1 -- 172.21.15.140:0/2481125407 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6adc143a10 msgr2=0x7f6adc156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ae0922700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6adc143a10 0x7f6adc156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ae0922700 1 -- 172.21.15.140:0/2481125407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6adc0b5970 msgr2=0x7f6adc156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ae0922700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6adc0b5970 0x7f6adc156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ae0922700 1 -- 172.21.15.140:0/2481125407 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac4009530 con 0x7f6adc0af8c0 2022-07-21T01:48:35.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ae0922700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6adc0af8c0 0x7f6adc0ae360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f6ad4001fa0 tx=0x7f6ad4007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ad9ffb700 1 -- 172.21.15.140:0/2481125407 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ad4007030 con 0x7f6adc0af8c0 2022-07-21T01:48:35.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ad9ffb700 1 -- 172.21.15.140:0/2481125407 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad4007190 con 0x7f6adc0af8c0 2022-07-21T01:48:35.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ae1924700 1 -- 172.21.15.140:0/2481125407 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc10e880 con 0x7f6adc0af8c0 2022-07-21T01:48:35.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ad9ffb700 1 -- 172.21.15.140:0/2481125407 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ad4006510 con 0x7f6adc0af8c0 2022-07-21T01:48:35.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ae1924700 1 -- 172.21.15.140:0/2481125407 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc145280 con 0x7f6adc0af8c0 2022-07-21T01:48:35.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ae1924700 1 -- 172.21.15.140:0/2481125407 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6adc142f90 con 0x7f6adc0af8c0 2022-07-21T01:48:35.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ad9ffb700 1 -- 172.21.15.140:0/2481125407 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6ad4010030 con 0x7f6adc0af8c0 2022-07-21T01:48:35.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ad9ffb700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ab40539e0 0x7f6ab4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6adbfff700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ab40539e0 0x7f6ab4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ad9ffb700 1 -- 172.21.15.140:0/2481125407 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6ad406bb80 con 0x7f6adc0af8c0 2022-07-21T01:48:35.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ad9ffb700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] conn(0x7f6ab405ad40 0x7f6ab405d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ad9ffb700 1 -- 172.21.15.140:0/2481125407 --> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ab405d820 con 0x7f6ab405ad40 2022-07-21T01:48:35.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.447+0000 7f6ad9ffb700 1 -- 172.21.15.140:0/2481125407 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ad406e210 con 0x7f6adc0af8c0 2022-07-21T01:48:35.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.451+0000 7f6ae1123700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] conn(0x7f6ab405ad40 0x7f6ab405d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.451+0000 7f6adbfff700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ab40539e0 0x7f6ab4055ea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f6ac401b000 tx=0x7f6ac4016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.451+0000 7f6ae1123700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] conn(0x7f6ab405ad40 0x7f6ab405d1a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.451+0000 7f6ad9ffb700 1 -- 172.21.15.140:0/2481125407 <== osd.1 v2:172.21.15.140:6826/22712 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f6ab405d820 con 0x7f6ab405ad40 2022-07-21T01:48:35.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ae1924700 1 -- 172.21.15.140:0/2481125407 --> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6adc10ea10 con 0x7f6ab405ad40 2022-07-21T01:48:35.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ad9ffb700 1 -- 172.21.15.140:0/2481125407 <== osd.1 v2:172.21.15.140:6826/22712 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6acc000e90 con 0x7f6ab405ad40 2022-07-21T01:48:35.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ae1924700 1 -- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] conn(0x7f6ab405ad40 msgr2=0x7f6ab405d1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ae1924700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] conn(0x7f6ab405ad40 0x7f6ab405d1a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ae1924700 1 -- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ab40539e0 msgr2=0x7f6ab4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ae1924700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ab40539e0 0x7f6ab4055ea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f6ac401b000 tx=0x7f6ac4016000).stop 2022-07-21T01:48:35.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ae1924700 1 -- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6adc0af8c0 msgr2=0x7f6adc0ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ae1924700 1 --2- 172.21.15.140:0/2481125407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6adc0af8c0 0x7f6adc0ae360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f6ad4001fa0 tx=0x7f6ad4007c30).stop 2022-07-21T01:48:35.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ae1123700 1 -- 172.21.15.140:0/2481125407 reap_dead start 2022-07-21T01:48:35.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ae1924700 1 -- 172.21.15.140:0/2481125407 shutdown_connections 2022-07-21T01:48:35.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ae1924700 1 -- 172.21.15.140:0/2481125407 >> 172.21.15.140:0/2481125407 conn(0x7f6adc063490 msgr2=0x7f6adc0b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.463+0000 7f6ae1924700 1 -- 172.21.15.140:0/2481125407 shutdown_connections 2022-07-21T01:48:35.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.467+0000 7f6ae1924700 1 -- 172.21.15.140:0/2481125407 wait complete. 2022-07-21T01:48:35.476 INFO:teuthology.orchestra.run.smithi140.stdout:55834574853 2022-07-21T01:48:35.476 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2022-07-21T01:48:35.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa2b40d700 1 Processor -- start 2022-07-21T01:48:35.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa2b40d700 1 -- start start 2022-07-21T01:48:35.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa2b40d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ffa240acc10 0x7ffa240acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa2b40d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffa240966b0 0x7ffa240b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa2b40d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa240abde0 con 0x7ffa24095b10 2022-07-21T01:48:35.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa2b40d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa24096c80 con 0x7ffa240acc10 2022-07-21T01:48:35.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa2b40d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa24098910 con 0x7ffa240966b0 2022-07-21T01:48:35.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa2a40b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ffa240acc10 0x7ffa240acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa29c0a700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ffa24095b10 0x7ffa24095fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46596/0 (socket says 172.21.15.140:46596) 2022-07-21T01:48:35.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa2a40b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ffa240acc10 0x7ffa240acff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60680/0 (socket says 172.21.15.140:60680) 2022-07-21T01:48:35.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa29c0a700 1 -- 172.21.15.140:0/3537865564 learned_addr learned my addr 172.21.15.140:0/3537865564 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:35.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.583+0000 7ffa2ac0c700 1 --2- 172.21.15.140:0/3537865564 >> v2:172.21.15.152:3300/0 conn(0x7ffa240966b0 0x7ffa240b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa29409700 1 -- 172.21.15.140:0/3537865564 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035641011 0 0) 0x7ffa240abde0 con 0x7ffa24095b10 2022-07-21T01:48:35.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2a40b700 1 -- 172.21.15.140:0/3537865564 >> v2:172.21.15.152:3300/0 conn(0x7ffa240966b0 msgr2=0x7ffa240b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:35.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2a40b700 1 --2- 172.21.15.140:0/3537865564 >> v2:172.21.15.152:3300/0 conn(0x7ffa240966b0 0x7ffa240b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2a40b700 1 -- 172.21.15.140:0/3537865564 >> v1:172.21.15.140:6789/0 conn(0x7ffa24095b10 legacy=0x7ffa24095fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2a40b700 1 -- 172.21.15.140:0/3537865564 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa24098b20 con 0x7ffa240acc10 2022-07-21T01:48:35.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2a40b700 1 --2- 172.21.15.140:0/3537865564 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa240acc10 0x7ffa240acff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7ffa20002f70 tx=0x7ffa20006e80).ready entity=mon.0 client_cookie=9b53e692554008b8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa29409700 1 -- 172.21.15.140:0/3537865564 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa2000b980 con 0x7ffa240acc10 2022-07-21T01:48:35.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa29409700 1 -- 172.21.15.140:0/3537865564 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa20000e90 con 0x7ffa240acc10 2022-07-21T01:48:35.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa29409700 1 -- 172.21.15.140:0/3537865564 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa2000bcd0 con 0x7ffa240acc10 2022-07-21T01:48:35.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2b40d700 1 -- 172.21.15.140:0/3537865564 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa240acc10 msgr2=0x7ffa240acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2b40d700 1 --2- 172.21.15.140:0/3537865564 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa240acc10 0x7ffa240acff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7ffa20002f70 tx=0x7ffa20006e80).stop 2022-07-21T01:48:35.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2b40d700 1 -- 172.21.15.140:0/3537865564 shutdown_connections 2022-07-21T01:48:35.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2b40d700 1 --2- 172.21.15.140:0/3537865564 >> v2:172.21.15.152:3300/0 conn(0x7ffa240966b0 0x7ffa240b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2b40d700 1 --2- 172.21.15.140:0/3537865564 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa240acc10 0x7ffa240acff0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2b40d700 1 -- 172.21.15.140:0/3537865564 >> 172.21.15.140:0/3537865564 conn(0x7ffa240b6210 msgr2=0x7ffa240b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2b40d700 1 -- 172.21.15.140:0/3537865564 shutdown_connections 2022-07-21T01:48:35.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2b40d700 1 -- 172.21.15.140:0/3537865564 wait complete. 2022-07-21T01:48:35.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2b40d700 1 Processor -- start 2022-07-21T01:48:35.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.587+0000 7ffa2b40d700 1 -- start start 2022-07-21T01:48:35.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2b40d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa24095b10 0x7ffa24140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2b40d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa240966b0 0x7ffa2409a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2b40d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa240acc10 0x7ffa241517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2b40d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa24065700 con 0x7ffa24095b10 2022-07-21T01:48:35.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2b40d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa240606f0 con 0x7ffa240966b0 2022-07-21T01:48:35.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2b40d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2409afd0 con 0x7ffa240acc10 2022-07-21T01:48:35.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2a40b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa24095b10 0x7ffa24140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2ac0c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa240acc10 0x7ffa241517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa29c0a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa240966b0 0x7ffa2409a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2a40b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa24095b10 0x7ffa24140200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60686/0 (socket says 172.21.15.140:60686) 2022-07-21T01:48:35.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2a40b700 1 -- 172.21.15.140:0/74846501 learned_addr learned my addr 172.21.15.140:0/74846501 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:35.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2ac0c700 1 -- 172.21.15.140:0/74846501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa240966b0 msgr2=0x7ffa2409a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2ac0c700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa240966b0 0x7ffa2409a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2ac0c700 1 -- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa24095b10 msgr2=0x7ffa24140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2ac0c700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa24095b10 0x7ffa24140200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2ac0c700 1 -- 172.21.15.140:0/74846501 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2000d410 con 0x7ffa240acc10 2022-07-21T01:48:35.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2ac0c700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa240acc10 0x7ffa241517f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ffa2409a200 tx=0x7ffa1c0091d0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa137fe700 1 -- 172.21.15.140:0/74846501 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa1c012600 con 0x7ffa240acc10 2022-07-21T01:48:35.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa137fe700 1 -- 172.21.15.140:0/74846501 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1c012760 con 0x7ffa240acc10 2022-07-21T01:48:35.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa137fe700 1 -- 172.21.15.140:0/74846501 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa1c012ac0 con 0x7ffa240acc10 2022-07-21T01:48:35.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2b40d700 1 -- 172.21.15.140:0/74846501 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2409b480 con 0x7ffa240acc10 2022-07-21T01:48:35.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2b40d700 1 -- 172.21.15.140:0/74846501 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa24140770 con 0x7ffa240acc10 2022-07-21T01:48:35.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.591+0000 7ffa2b40d700 1 -- 172.21.15.140:0/74846501 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa2409b920 con 0x7ffa240acc10 2022-07-21T01:48:35.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.595+0000 7ffa137fe700 1 -- 172.21.15.140:0/74846501 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffa1c022070 con 0x7ffa240acc10 2022-07-21T01:48:35.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.595+0000 7ffa137fe700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffa04053b00 0x7ffa04055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.595+0000 7ffa137fe700 1 -- 172.21.15.140:0/74846501 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffa1c06cc20 con 0x7ffa240acc10 2022-07-21T01:48:35.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.595+0000 7ffa2a40b700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffa04053b00 0x7ffa04055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.595+0000 7ffa137fe700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] conn(0x7ffa0405ae60 0x7ffa0405d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.595+0000 7ffa137fe700 1 -- 172.21.15.140:0/74846501 --> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa0405d8a0 con 0x7ffa0405ae60 2022-07-21T01:48:35.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.595+0000 7ffa137fe700 1 -- 172.21.15.140:0/74846501 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa1c06cef0 con 0x7ffa240acc10 2022-07-21T01:48:35.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.595+0000 7ffa29c0a700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] conn(0x7ffa0405ae60 0x7ffa0405d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.595+0000 7ffa29c0a700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] conn(0x7ffa0405ae60 0x7ffa0405d2c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.595+0000 7ffa2a40b700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffa04053b00 0x7ffa04055fc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ffa2001e000 tx=0x7ffa240abf70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.595+0000 7ffa137fe700 1 -- 172.21.15.140:0/74846501 <== osd.2 v2:172.21.15.140:6810/22724 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7ffa0405d8a0 con 0x7ffa0405ae60 2022-07-21T01:48:35.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2b40d700 1 -- 172.21.15.140:0/74846501 --> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa240ac360 con 0x7ffa0405ae60 2022-07-21T01:48:35.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa137fe700 1 -- 172.21.15.140:0/74846501 <== osd.2 v2:172.21.15.140:6810/22724 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffa240ac360 con 0x7ffa0405ae60 2022-07-21T01:48:35.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2b40d700 1 -- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] conn(0x7ffa0405ae60 msgr2=0x7ffa0405d2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2b40d700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] conn(0x7ffa0405ae60 0x7ffa0405d2c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2b40d700 1 -- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffa04053b00 msgr2=0x7ffa04055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2b40d700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffa04053b00 0x7ffa04055fc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ffa2001e000 tx=0x7ffa240abf70).stop 2022-07-21T01:48:35.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2b40d700 1 -- 172.21.15.140:0/74846501 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa240acc10 msgr2=0x7ffa241517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2b40d700 1 --2- 172.21.15.140:0/74846501 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa240acc10 0x7ffa241517f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ffa2409a200 tx=0x7ffa1c0091d0).stop 2022-07-21T01:48:35.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2ac0c700 1 -- 172.21.15.140:0/74846501 reap_dead start 2022-07-21T01:48:35.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2b40d700 1 -- 172.21.15.140:0/74846501 shutdown_connections 2022-07-21T01:48:35.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2b40d700 1 -- 172.21.15.140:0/74846501 >> 172.21.15.140:0/74846501 conn(0x7ffa240b6210 msgr2=0x7ffa240b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2b40d700 1 -- 172.21.15.140:0/74846501 shutdown_connections 2022-07-21T01:48:35.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.611+0000 7ffa2b40d700 1 -- 172.21.15.140:0/74846501 wait complete. 2022-07-21T01:48:35.622 INFO:teuthology.orchestra.run.smithi140.stdout:55834574854 2022-07-21T01:48:35.623 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2022-07-21T01:48:35.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff342609700 1 Processor -- start 2022-07-21T01:48:35.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff342609700 1 -- start start 2022-07-21T01:48:35.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff342609700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff33c060ca0 0x7ff33c061080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff342609700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff33c0afb80 0x7ff33c14b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff342609700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c14a750 con 0x7ff33c097a80 2022-07-21T01:48:35.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff342609700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33c148430 con 0x7ff33c0afb80 2022-07-21T01:48:35.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff342609700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c0b2840 con 0x7ff33c060ca0 2022-07-21T01:48:35.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff33b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff33c097a80 0x7ff33c14ab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46610/0 (socket says 172.21.15.140:46610) 2022-07-21T01:48:35.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff33b7fe700 1 -- 172.21.15.140:0/4059609520 learned_addr learned my addr 172.21.15.140:0/4059609520 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:35.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff33bfff700 1 --2- 172.21.15.140:0/4059609520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff33c060ca0 0x7ff33c061080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff340ba6700 1 --2- 172.21.15.140:0/4059609520 >> v2:172.21.15.152:3300/0 conn(0x7ff33c0afb80 0x7ff33c14b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff33affd700 1 -- 172.21.15.140:0/4059609520 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201354657 0 0) 0x7ff33c14a750 con 0x7ff33c097a80 2022-07-21T01:48:35.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff33affd700 1 -- 172.21.15.140:0/4059609520 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff328003260 con 0x7ff33c097a80 2022-07-21T01:48:35.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff33bfff700 1 -- 172.21.15.140:0/4059609520 >> v2:172.21.15.152:3300/0 conn(0x7ff33c0afb80 msgr2=0x7ff33c14b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:35.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff33bfff700 1 --2- 172.21.15.140:0/4059609520 >> v2:172.21.15.152:3300/0 conn(0x7ff33c0afb80 0x7ff33c14b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff33bfff700 1 -- 172.21.15.140:0/4059609520 >> v1:172.21.15.140:6789/0 conn(0x7ff33c097a80 legacy=0x7ff33c14ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.731+0000 7ff33bfff700 1 -- 172.21.15.140:0/4059609520 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c0ac110 con 0x7ff33c060ca0 2022-07-21T01:48:35.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff33bfff700 1 --2- 172.21.15.140:0/4059609520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff33c060ca0 0x7ff33c061080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ff330002f70 tx=0x7ff330006df0).ready entity=mon.2 client_cookie=92a5419d745af1fe server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff33affd700 1 -- 172.21.15.140:0/4059609520 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff33000d4a0 con 0x7ff33c060ca0 2022-07-21T01:48:35.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff33affd700 1 -- 172.21.15.140:0/4059609520 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330000e90 con 0x7ff33c060ca0 2022-07-21T01:48:35.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff33affd700 1 -- 172.21.15.140:0/4059609520 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff33000d7f0 con 0x7ff33c060ca0 2022-07-21T01:48:35.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff342609700 1 -- 172.21.15.140:0/4059609520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff33c060ca0 msgr2=0x7ff33c061080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff342609700 1 --2- 172.21.15.140:0/4059609520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff33c060ca0 0x7ff33c061080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ff330002f70 tx=0x7ff330006df0).stop 2022-07-21T01:48:35.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff342609700 1 -- 172.21.15.140:0/4059609520 shutdown_connections 2022-07-21T01:48:35.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff342609700 1 --2- 172.21.15.140:0/4059609520 >> v2:172.21.15.152:3300/0 conn(0x7ff33c0afb80 0x7ff33c14b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff342609700 1 --2- 172.21.15.140:0/4059609520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff33c060ca0 0x7ff33c061080 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff342609700 1 -- 172.21.15.140:0/4059609520 >> 172.21.15.140:0/4059609520 conn(0x7ff33c0b5cb0 msgr2=0x7ff33c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff342609700 1 -- 172.21.15.140:0/4059609520 shutdown_connections 2022-07-21T01:48:35.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff342609700 1 -- 172.21.15.140:0/4059609520 wait complete. 2022-07-21T01:48:35.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff342609700 1 Processor -- start 2022-07-21T01:48:35.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.735+0000 7ff342609700 1 -- start start 2022-07-21T01:48:35.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff342609700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c060ca0 0x7ff33c05c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff342609700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff33c097a80 0x7ff33c099b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff342609700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff33c0afb80 0x7ff33c09a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff342609700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c098f30 con 0x7ff33c0afb80 2022-07-21T01:48:35.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff342609700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c0993d0 con 0x7ff33c060ca0 2022-07-21T01:48:35.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff342609700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c099130 con 0x7ff33c097a80 2022-07-21T01:48:35.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff340ba6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff33c0afb80 0x7ff33c09a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff340ba6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff33c0afb80 0x7ff33c09a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60706/0 (socket says 172.21.15.140:60706) 2022-07-21T01:48:35.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff340ba6700 1 -- 172.21.15.140:0/2042329484 learned_addr learned my addr 172.21.15.140:0/2042329484 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:35.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff33bfff700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c060ca0 0x7ff33c05c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff33b7fe700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff33c097a80 0x7ff33c099b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff340ba6700 1 -- 172.21.15.140:0/2042329484 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff33c097a80 msgr2=0x7ff33c099b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff340ba6700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff33c097a80 0x7ff33c099b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff340ba6700 1 -- 172.21.15.140:0/2042329484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c060ca0 msgr2=0x7ff33c05c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff340ba6700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff33c060ca0 0x7ff33c05c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff340ba6700 1 -- 172.21.15.140:0/2042329484 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33000b180 con 0x7ff33c0afb80 2022-07-21T01:48:35.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff33b7fe700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff33c097a80 0x7ff33c099b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:35.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff340ba6700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff33c0afb80 0x7ff33c09a050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff33c098440 tx=0x7ff334006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff3397fa700 1 -- 172.21.15.140:0/2042329484 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff334011750 con 0x7ff33c0afb80 2022-07-21T01:48:35.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff3397fa700 1 -- 172.21.15.140:0/2042329484 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3340118b0 con 0x7ff33c0afb80 2022-07-21T01:48:35.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff342609700 1 -- 172.21.15.140:0/2042329484 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c09a540 con 0x7ff33c0afb80 2022-07-21T01:48:35.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff3397fa700 1 -- 172.21.15.140:0/2042329484 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff334011c10 con 0x7ff33c0afb80 2022-07-21T01:48:35.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff342609700 1 -- 172.21.15.140:0/2042329484 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c09a960 con 0x7ff33c0afb80 2022-07-21T01:48:35.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.739+0000 7ff342609700 1 -- 172.21.15.140:0/2042329484 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff33c09b1b0 con 0x7ff33c0afb80 2022-07-21T01:48:35.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.743+0000 7ff3397fa700 1 -- 172.21.15.140:0/2042329484 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff334011d70 con 0x7ff33c0afb80 2022-07-21T01:48:35.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.743+0000 7ff3397fa700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff31c053aa0 0x7ff31c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.743+0000 7ff3397fa700 1 -- 172.21.15.140:0/2042329484 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff33406c310 con 0x7ff33c0afb80 2022-07-21T01:48:35.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.743+0000 7ff33bfff700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff31c053aa0 0x7ff31c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.743+0000 7ff3397fa700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] conn(0x7ff31c05ae00 0x7ff31c05d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.743+0000 7ff3397fa700 1 -- 172.21.15.140:0/2042329484 --> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff31c05d840 con 0x7ff31c05ae00 2022-07-21T01:48:35.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.743+0000 7ff3397fa700 1 -- 172.21.15.140:0/2042329484 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff33406e890 con 0x7ff33c0afb80 2022-07-21T01:48:35.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.743+0000 7ff33b7fe700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] conn(0x7ff31c05ae00 0x7ff31c05d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.743+0000 7ff33bfff700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff31c053aa0 0x7ff31c055f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff33001a000 tx=0x7ff330014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.743+0000 7ff33b7fe700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] conn(0x7ff31c05ae00 0x7ff31c05d260 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.743+0000 7ff3397fa700 1 -- 172.21.15.140:0/2042329484 <== osd.3 v2:172.21.15.140:6818/22721 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7ff31c05d840 con 0x7ff31c05ae00 2022-07-21T01:48:35.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff342609700 1 -- 172.21.15.140:0/2042329484 --> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff33c1598b0 con 0x7ff31c05ae00 2022-07-21T01:48:35.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff3397fa700 1 -- 172.21.15.140:0/2042329484 <== osd.3 v2:172.21.15.140:6818/22721 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff33c1598b0 con 0x7ff31c05ae00 2022-07-21T01:48:35.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff342609700 1 -- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] conn(0x7ff31c05ae00 msgr2=0x7ff31c05d260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff342609700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] conn(0x7ff31c05ae00 0x7ff31c05d260 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff342609700 1 -- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff31c053aa0 msgr2=0x7ff31c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff342609700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff31c053aa0 0x7ff31c055f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff33001a000 tx=0x7ff330014000).stop 2022-07-21T01:48:35.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff342609700 1 -- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff33c0afb80 msgr2=0x7ff33c09a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff342609700 1 --2- 172.21.15.140:0/2042329484 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff33c0afb80 0x7ff33c09a050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff33c098440 tx=0x7ff334006df0).stop 2022-07-21T01:48:35.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff340ba6700 1 -- 172.21.15.140:0/2042329484 reap_dead start 2022-07-21T01:48:35.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff342609700 1 -- 172.21.15.140:0/2042329484 shutdown_connections 2022-07-21T01:48:35.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff342609700 1 -- 172.21.15.140:0/2042329484 >> 172.21.15.140:0/2042329484 conn(0x7ff33c0b5cb0 msgr2=0x7ff33c0b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff342609700 1 -- 172.21.15.140:0/2042329484 shutdown_connections 2022-07-21T01:48:35.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.759+0000 7ff342609700 1 -- 172.21.15.140:0/2042329484 wait complete. 2022-07-21T01:48:35.771 INFO:teuthology.orchestra.run.smithi140.stdout:55834574853 2022-07-21T01:48:35.771 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2022-07-21T01:48:35.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216a25700 1 Processor -- start 2022-07-21T01:48:35.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216a25700 1 -- start start 2022-07-21T01:48:35.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216a25700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f32100b5970 0x7f32100ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216a25700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3210143a10 0x7f3210143e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216a25700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32100b0b80 con 0x7f32100af8c0 2022-07-21T01:48:35.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216a25700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321005d4d0 con 0x7f3210143a10 2022-07-21T01:48:35.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216a25700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32100b5d50 con 0x7f32100b5970 2022-07-21T01:48:35.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216224700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3210143a10 0x7f3210143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216224700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3210143a10 0x7f3210143e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60716/0 (socket says 172.21.15.140:60716) 2022-07-21T01:48:35.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216224700 1 -- 172.21.15.140:0/1609086686 learned_addr learned my addr 172.21.15.140:0/1609086686 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:35.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3215a23700 1 --2- 172.21.15.140:0/1609086686 >> v2:172.21.15.152:3300/0 conn(0x7f32100b5970 0x7f32100ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216224700 1 -- 172.21.15.140:0/1609086686 >> v2:172.21.15.152:3300/0 conn(0x7f32100b5970 msgr2=0x7f32100ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:35.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3214a21700 1 -- 172.21.15.140:0/1609086686 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007828248 0 0) 0x7f32100b0b80 con 0x7f32100af8c0 2022-07-21T01:48:35.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216224700 1 --2- 172.21.15.140:0/1609086686 >> v2:172.21.15.152:3300/0 conn(0x7f32100b5970 0x7f32100ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216224700 1 -- 172.21.15.140:0/1609086686 >> v1:172.21.15.152:6789/0 conn(0x7f32100af8c0 legacy=0x7f32100ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216224700 1 -- 172.21.15.140:0/1609086686 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100b0db0 con 0x7f3210143a10 2022-07-21T01:48:35.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3216224700 1 --2- 172.21.15.140:0/1609086686 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3210143a10 0x7f3210143e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f3208002f70 tx=0x7f3208006eb0).ready entity=mon.0 client_cookie=9a11e403b11a2d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3214a21700 1 -- 172.21.15.140:0/1609086686 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f320800d5c0 con 0x7f3210143a10 2022-07-21T01:48:35.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.883+0000 7f3214a21700 1 -- 172.21.15.140:0/1609086686 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208000e90 con 0x7f3210143a10 2022-07-21T01:48:35.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.887+0000 7f3214a21700 1 -- 172.21.15.140:0/1609086686 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f320800d910 con 0x7f3210143a10 2022-07-21T01:48:35.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.887+0000 7f3216a25700 1 -- 172.21.15.140:0/1609086686 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3210143a10 msgr2=0x7f3210143e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.887+0000 7f3216a25700 1 --2- 172.21.15.140:0/1609086686 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3210143a10 0x7f3210143e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f3208002f70 tx=0x7f3208006eb0).stop 2022-07-21T01:48:35.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.887+0000 7f3216a25700 1 -- 172.21.15.140:0/1609086686 shutdown_connections 2022-07-21T01:48:35.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.887+0000 7f3216a25700 1 --2- 172.21.15.140:0/1609086686 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3210143a10 0x7f3210143e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.887+0000 7f3216a25700 1 --2- 172.21.15.140:0/1609086686 >> v2:172.21.15.152:3300/0 conn(0x7f32100b5970 0x7f32100ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.887+0000 7f3216a25700 1 -- 172.21.15.140:0/1609086686 >> 172.21.15.140:0/1609086686 conn(0x7f3210063490 msgr2=0x7f32100b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.887+0000 7f3216a25700 1 -- 172.21.15.140:0/1609086686 shutdown_connections 2022-07-21T01:48:35.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.887+0000 7f3216a25700 1 -- 172.21.15.140:0/1609086686 wait complete. 2022-07-21T01:48:35.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.887+0000 7f3216a25700 1 Processor -- start 2022-07-21T01:48:35.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.887+0000 7f3216a25700 1 -- start start 2022-07-21T01:48:35.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3216a25700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32100af8c0 0x7f321010f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3216a25700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32100b5970 0x7f3210156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3216a25700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3210143a10 0x7f3210159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3216a25700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3210156980 con 0x7f32100b5970 2022-07-21T01:48:35.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3216a25700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321013c750 con 0x7f32100af8c0 2022-07-21T01:48:35.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3216a25700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321010f7c0 con 0x7f3210143a10 2022-07-21T01:48:35.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3215222700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32100b5970 0x7f3210156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3215222700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32100b5970 0x7f3210156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60720/0 (socket says 172.21.15.140:60720) 2022-07-21T01:48:35.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3215222700 1 -- 172.21.15.140:0/603735678 learned_addr learned my addr 172.21.15.140:0/603735678 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:35.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3215a23700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32100af8c0 0x7f321010f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3216224700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3210143a10 0x7f3210159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3215222700 1 -- 172.21.15.140:0/603735678 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3210143a10 msgr2=0x7f3210159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3215222700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3210143a10 0x7f3210159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3215222700 1 -- 172.21.15.140:0/603735678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32100af8c0 msgr2=0x7f321010f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3215222700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32100af8c0 0x7f321010f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3215222700 1 -- 172.21.15.140:0/603735678 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320800b1e0 con 0x7f32100b5970 2022-07-21T01:48:35.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3215222700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32100b5970 0x7f3210156ba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f320c001250 tx=0x7f320c007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3202ffd700 1 -- 172.21.15.140:0/603735678 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f320c010510 con 0x7f32100b5970 2022-07-21T01:48:35.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3202ffd700 1 -- 172.21.15.140:0/603735678 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c010670 con 0x7f32100b5970 2022-07-21T01:48:35.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3216a25700 1 -- 172.21.15.140:0/603735678 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321010fa60 con 0x7f32100b5970 2022-07-21T01:48:35.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3202ffd700 1 -- 172.21.15.140:0/603735678 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f320c010820 con 0x7f32100b5970 2022-07-21T01:48:35.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.891+0000 7f3216a25700 1 -- 172.21.15.140:0/603735678 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32101430e0 con 0x7f32100b5970 2022-07-21T01:48:35.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.895+0000 7f3216a25700 1 -- 172.21.15.140:0/603735678 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f321010fbf0 con 0x7f32100b5970 2022-07-21T01:48:35.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.895+0000 7f3202ffd700 1 -- 172.21.15.140:0/603735678 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f320c010070 con 0x7f32100b5970 2022-07-21T01:48:35.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.895+0000 7f3202ffd700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f31f00539e0 0x7f31f0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.895+0000 7f3215a23700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f31f00539e0 0x7f31f0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.895+0000 7f3202ffd700 1 -- 172.21.15.140:0/603735678 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f320c06bdb0 con 0x7f32100b5970 2022-07-21T01:48:35.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.895+0000 7f3202ffd700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] conn(0x7f31f005ad40 0x7f31f005d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:35.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.895+0000 7f3202ffd700 1 -- 172.21.15.140:0/603735678 --> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31f005d820 con 0x7f31f005ad40 2022-07-21T01:48:35.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.895+0000 7f3202ffd700 1 -- 172.21.15.140:0/603735678 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f320c06e330 con 0x7f32100b5970 2022-07-21T01:48:35.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.895+0000 7f3215a23700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f31f00539e0 0x7f31f0055ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f320400b000 tx=0x7f3204006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.895+0000 7f3216224700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] conn(0x7f31f005ad40 0x7f31f005d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:35.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.895+0000 7f3216224700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] conn(0x7f31f005ad40 0x7f31f005d1a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:35.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.899+0000 7f3202ffd700 1 -- 172.21.15.140:0/603735678 <== osd.4 v2:172.21.15.152:6800/23141 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f31f005d820 con 0x7f31f005ad40 2022-07-21T01:48:35.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.915+0000 7f3216a25700 1 -- 172.21.15.140:0/603735678 --> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3210141280 con 0x7f31f005ad40 2022-07-21T01:48:35.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.915+0000 7f3202ffd700 1 -- 172.21.15.140:0/603735678 <== osd.4 v2:172.21.15.152:6800/23141 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3210141280 con 0x7f31f005ad40 2022-07-21T01:48:35.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.915+0000 7f3216a25700 1 -- 172.21.15.140:0/603735678 >> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] conn(0x7f31f005ad40 msgr2=0x7f31f005d1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.915+0000 7f3216a25700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] conn(0x7f31f005ad40 0x7f31f005d1a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:35.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.919+0000 7f3216a25700 1 -- 172.21.15.140:0/603735678 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f31f00539e0 msgr2=0x7f31f0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.919+0000 7f3216a25700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f31f00539e0 0x7f31f0055ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f320400b000 tx=0x7f3204006df0).stop 2022-07-21T01:48:35.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.919+0000 7f3216a25700 1 -- 172.21.15.140:0/603735678 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32100b5970 msgr2=0x7f3210156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:35.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.919+0000 7f3216a25700 1 --2- 172.21.15.140:0/603735678 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32100b5970 0x7f3210156ba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f320c001250 tx=0x7f320c007e40).stop 2022-07-21T01:48:35.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.919+0000 7f3216224700 1 -- 172.21.15.140:0/603735678 reap_dead start 2022-07-21T01:48:35.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.919+0000 7f3216a25700 1 -- 172.21.15.140:0/603735678 shutdown_connections 2022-07-21T01:48:35.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.919+0000 7f3216a25700 1 -- 172.21.15.140:0/603735678 >> 172.21.15.140:0/603735678 conn(0x7f3210063490 msgr2=0x7f32100b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:35.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.919+0000 7f3216a25700 1 -- 172.21.15.140:0/603735678 shutdown_connections 2022-07-21T01:48:35.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:35.919+0000 7f3216a25700 1 -- 172.21.15.140:0/603735678 wait complete. 2022-07-21T01:48:35.934 INFO:teuthology.orchestra.run.smithi140.stdout:55834574854 2022-07-21T01:48:35.935 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2022-07-21T01:48:36.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.047+0000 7f370ffff700 1 Processor -- start 2022-07-21T01:48:36.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.047+0000 7f370ffff700 1 -- start start 2022-07-21T01:48:36.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3710148580 0x7f371014a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370ffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3710096930 con 0x7f37100acc10 2022-07-21T01:48:36.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370ffff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37100abde0 con 0x7f37100acff0 2022-07-21T01:48:36.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370ffff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3710096c80 con 0x7f3710148580 2022-07-21T01:48:36.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3710148580 0x7f371014a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3710148580 0x7f371014a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60732/0 (socket says 172.21.15.140:60732) 2022-07-21T01:48:36.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370f7fe700 1 -- 172.21.15.140:0/3549541920 learned_addr learned my addr 172.21.15.140:0/3549541920 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:36.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370effd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f37100acc10 0x7f3710095b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46644/0 (socket says 172.21.15.140:46644) 2022-07-21T01:48:36.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370f7fe700 1 -- 172.21.15.140:0/3549541920 >> v1:172.21.15.152:6789/0 conn(0x7f37100acff0 legacy=0x7f3710096220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370f7fe700 1 -- 172.21.15.140:0/3549541920 >> v1:172.21.15.140:6789/0 conn(0x7f37100acc10 legacy=0x7f3710095b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370dffb700 1 -- 172.21.15.140:0/3549541920 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056516255 0 0) 0x7f3710096930 con 0x7f37100acc10 2022-07-21T01:48:36.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370f7fe700 1 -- 172.21.15.140:0/3549541920 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3710098b20 con 0x7f3710148580 2022-07-21T01:48:36.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370f7fe700 1 --2- 172.21.15.140:0/3549541920 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3710148580 0x7f371014a970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f37080032f0 tx=0x7f3708009290).ready entity=mon.0 client_cookie=a226a001a30db873 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370dffb700 1 -- 172.21.15.140:0/3549541920 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370800b8a0 con 0x7f3710148580 2022-07-21T01:48:36.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370dffb700 1 -- 172.21.15.140:0/3549541920 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370800ba00 con 0x7f3710148580 2022-07-21T01:48:36.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370dffb700 1 -- 172.21.15.140:0/3549541920 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370800bb70 con 0x7f3710148580 2022-07-21T01:48:36.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370ffff700 1 -- 172.21.15.140:0/3549541920 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3710148580 msgr2=0x7f371014a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370ffff700 1 --2- 172.21.15.140:0/3549541920 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3710148580 0x7f371014a970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f37080032f0 tx=0x7f3708009290).stop 2022-07-21T01:48:36.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370ffff700 1 -- 172.21.15.140:0/3549541920 shutdown_connections 2022-07-21T01:48:36.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370ffff700 1 --2- 172.21.15.140:0/3549541920 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3710148580 0x7f371014a970 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370ffff700 1 -- 172.21.15.140:0/3549541920 >> 172.21.15.140:0/3549541920 conn(0x7f37100b6210 msgr2=0x7f37100b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:36.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370ffff700 1 -- 172.21.15.140:0/3549541920 shutdown_connections 2022-07-21T01:48:36.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.051+0000 7f370ffff700 1 -- 172.21.15.140:0/3549541920 wait complete. 2022-07-21T01:48:36.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370ffff700 1 Processor -- start 2022-07-21T01:48:36.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370ffff700 1 -- start start 2022-07-21T01:48:36.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37100acc10 0x7f371013f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37100acff0 0x7f3710152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3710148580 0x7f37100600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37101530d0 con 0x7f37100acc10 2022-07-21T01:48:36.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3710152200 con 0x7f37100acff0 2022-07-21T01:48:36.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37101524a0 con 0x7f3710148580 2022-07-21T01:48:36.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37100acc10 0x7f371013f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370e7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37100acff0 0x7f3710152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370f7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3710148580 0x7f37100600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370e7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37100acff0 0x7f3710152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:38242/0 (socket says 172.21.15.140:38242) 2022-07-21T01:48:36.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37100acc10 0x7f371013f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60736/0 (socket says 172.21.15.140:60736) 2022-07-21T01:48:36.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370e7fc700 1 -- 172.21.15.140:0/1706644579 learned_addr learned my addr 172.21.15.140:0/1706644579 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:36.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370effd700 1 -- 172.21.15.140:0/1706644579 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3710148580 msgr2=0x7f37100600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370effd700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3710148580 0x7f37100600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370effd700 1 -- 172.21.15.140:0/1706644579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37100acff0 msgr2=0x7f3710152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370effd700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37100acff0 0x7f3710152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370effd700 1 -- 172.21.15.140:0/1706644579 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3708009530 con 0x7f37100acc10 2022-07-21T01:48:36.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370effd700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37100acc10 0x7f371013f610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f36f8000ed0 tx=0x7f36f8007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f3707fff700 1 -- 172.21.15.140:0/1706644579 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36f80070d0 con 0x7f37100acc10 2022-07-21T01:48:36.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f3707fff700 1 -- 172.21.15.140:0/1706644579 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f8007230 con 0x7f37100acc10 2022-07-21T01:48:36.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370ffff700 1 -- 172.21.15.140:0/1706644579 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3710152740 con 0x7f37100acc10 2022-07-21T01:48:36.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f3707fff700 1 -- 172.21.15.140:0/1706644579 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36f8005550 con 0x7f37100acc10 2022-07-21T01:48:36.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.055+0000 7f370ffff700 1 -- 172.21.15.140:0/1706644579 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371005f840 con 0x7f37100acc10 2022-07-21T01:48:36.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.059+0000 7f370ffff700 1 -- 172.21.15.140:0/1706644579 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3710155a00 con 0x7f37100acc10 2022-07-21T01:48:36.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.059+0000 7f3707fff700 1 -- 172.21.15.140:0/1706644579 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f36f801a060 con 0x7f37100acc10 2022-07-21T01:48:36.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.059+0000 7f3707fff700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36f4053a10 0x7f36f4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.059+0000 7f3707fff700 1 -- 172.21.15.140:0/1706644579 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f36f806b560 con 0x7f37100acc10 2022-07-21T01:48:36.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.059+0000 7f370e7fc700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36f4053a10 0x7f36f4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.059+0000 7f3707fff700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] conn(0x7f36f405ad70 0x7f36f405d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.059+0000 7f3707fff700 1 -- 172.21.15.140:0/1706644579 --> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36f405d850 con 0x7f36f405ad70 2022-07-21T01:48:36.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.059+0000 7f3707fff700 1 -- 172.21.15.140:0/1706644579 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36f806dbf0 con 0x7f37100acc10 2022-07-21T01:48:36.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.059+0000 7f370f7fe700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] conn(0x7f36f405ad70 0x7f36f405d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.059+0000 7f370e7fc700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36f4053a10 0x7f36f4055ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f370000d000 tx=0x7f3700007c50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.059+0000 7f370f7fe700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] conn(0x7f36f405ad70 0x7f36f405d1d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.063+0000 7f3707fff700 1 -- 172.21.15.140:0/1706644579 <== osd.5 v2:172.21.15.152:6810/23143 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f36f405d850 con 0x7f36f405ad70 2022-07-21T01:48:36.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370ffff700 1 -- 172.21.15.140:0/1706644579 --> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f37100608d0 con 0x7f36f405ad70 2022-07-21T01:48:36.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f3707fff700 1 -- 172.21.15.140:0/1706644579 <== osd.5 v2:172.21.15.152:6810/23143 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f37100608d0 con 0x7f36f405ad70 2022-07-21T01:48:36.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370ffff700 1 -- 172.21.15.140:0/1706644579 >> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] conn(0x7f36f405ad70 msgr2=0x7f36f405d1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370ffff700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] conn(0x7f36f405ad70 0x7f36f405d1d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370ffff700 1 -- 172.21.15.140:0/1706644579 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36f4053a10 msgr2=0x7f36f4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370ffff700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36f4053a10 0x7f36f4055ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f370000d000 tx=0x7f3700007c50).stop 2022-07-21T01:48:36.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370ffff700 1 -- 172.21.15.140:0/1706644579 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37100acc10 msgr2=0x7f371013f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370ffff700 1 --2- 172.21.15.140:0/1706644579 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37100acc10 0x7f371013f610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f36f8000ed0 tx=0x7f36f8007a00).stop 2022-07-21T01:48:36.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370f7fe700 1 -- 172.21.15.140:0/1706644579 reap_dead start 2022-07-21T01:48:36.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370ffff700 1 -- 172.21.15.140:0/1706644579 shutdown_connections 2022-07-21T01:48:36.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370ffff700 1 -- 172.21.15.140:0/1706644579 >> 172.21.15.140:0/1706644579 conn(0x7f37100b6210 msgr2=0x7f3710144580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:36.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370ffff700 1 -- 172.21.15.140:0/1706644579 shutdown_connections 2022-07-21T01:48:36.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.079+0000 7f370ffff700 1 -- 172.21.15.140:0/1706644579 wait complete. 2022-07-21T01:48:36.090 INFO:teuthology.orchestra.run.smithi140.stdout:55834574853 2022-07-21T01:48:36.091 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2022-07-21T01:48:36.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.203+0000 7fab3359e700 1 Processor -- start 2022-07-21T01:48:36.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.203+0000 7fab3359e700 1 -- start start 2022-07-21T01:48:36.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab340b2170 0x7fab340b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fab340b2550 0x7fab340aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fab340b3d40 0x7fab340b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab340b5020 con 0x7fab340b2550 2022-07-21T01:48:36.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab340adad0 con 0x7fab340b3d40 2022-07-21T01:48:36.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab340b3640 con 0x7fab340b2170 2022-07-21T01:48:36.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab29d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fab340b2550 0x7fab340aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab29d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fab340b2550 0x7fab340aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60746/0 (socket says 172.21.15.140:60746) 2022-07-21T01:48:36.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab29d9b700 1 -- 172.21.15.140:0/160530929 learned_addr learned my addr 172.21.15.140:0/160530929 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:36.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3259c700 1 --2- 172.21.15.140:0/160530929 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab340b2170 0x7fab340b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab32d9d700 1 --2- 172.21.15.140:0/160530929 >> v2:172.21.15.152:3300/0 conn(0x7fab340b3d40 0x7fab340b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab29d9b700 1 -- 172.21.15.140:0/160530929 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab340b2170 msgr2=0x7fab340b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:36.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab29d9b700 1 --2- 172.21.15.140:0/160530929 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab340b2170 0x7fab340b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab29d9b700 1 -- 172.21.15.140:0/160530929 >> v2:172.21.15.152:3300/0 conn(0x7fab340b3d40 msgr2=0x7fab340b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:36.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab29d9b700 1 --2- 172.21.15.140:0/160530929 >> v2:172.21.15.152:3300/0 conn(0x7fab340b3d40 0x7fab340b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab29d9b700 1 -- 172.21.15.140:0/160530929 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340add00 con 0x7fab340b2550 2022-07-21T01:48:36.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab29d9b700 1 --2- 172.21.15.140:0/160530929 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab340b2550 0x7fab340aeae0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fab1c002f70 tx=0x7fab1c006f40).ready entity=mon.0 client_cookie=5e8e7e75bfe857b3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab31d9b700 1 -- 172.21.15.140:0/160530929 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab1c010810 con 0x7fab340b2550 2022-07-21T01:48:36.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab31d9b700 1 -- 172.21.15.140:0/160530929 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c000e90 con 0x7fab340b2550 2022-07-21T01:48:36.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab31d9b700 1 -- 172.21.15.140:0/160530929 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab1c010ba0 con 0x7fab340b2550 2022-07-21T01:48:36.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 -- 172.21.15.140:0/160530929 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab340b2550 msgr2=0x7fab340aeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 --2- 172.21.15.140:0/160530929 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab340b2550 0x7fab340aeae0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fab1c002f70 tx=0x7fab1c006f40).stop 2022-07-21T01:48:36.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 -- 172.21.15.140:0/160530929 shutdown_connections 2022-07-21T01:48:36.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 --2- 172.21.15.140:0/160530929 >> v2:172.21.15.152:3300/0 conn(0x7fab340b3d40 0x7fab340b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 --2- 172.21.15.140:0/160530929 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab340b2550 0x7fab340aeae0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 --2- 172.21.15.140:0/160530929 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab340b2170 0x7fab340b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 -- 172.21.15.140:0/160530929 >> 172.21.15.140:0/160530929 conn(0x7fab34063150 msgr2=0x7fab34063530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:36.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 -- 172.21.15.140:0/160530929 shutdown_connections 2022-07-21T01:48:36.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.207+0000 7fab3359e700 1 -- 172.21.15.140:0/160530929 wait complete. 2022-07-21T01:48:36.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.211+0000 7fab3359e700 1 Processor -- start 2022-07-21T01:48:36.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.211+0000 7fab3359e700 1 -- start start 2022-07-21T01:48:36.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab3359e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab340b2170 0x7fab3414e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab3359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab340b2550 0x7fab340651b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab3359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab340b3d40 0x7fab340656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab3359e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab34142af0 con 0x7fab340b2550 2022-07-21T01:48:36.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab3359e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab34140530 con 0x7fab340b2170 2022-07-21T01:48:36.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab3359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab34140720 con 0x7fab340b3d40 2022-07-21T01:48:36.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab29d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab340b2550 0x7fab340651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab29d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab340b2550 0x7fab340651b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60752/0 (socket says 172.21.15.140:60752) 2022-07-21T01:48:36.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab29d9b700 1 -- 172.21.15.140:0/1232027877 learned_addr learned my addr 172.21.15.140:0/1232027877 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:36.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab3259c700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab340b2170 0x7fab3414e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab32d9d700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab340b3d40 0x7fab340656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab29d9b700 1 -- 172.21.15.140:0/1232027877 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab340b3d40 msgr2=0x7fab340656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab29d9b700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab340b3d40 0x7fab340656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab29d9b700 1 -- 172.21.15.140:0/1232027877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab340b2170 msgr2=0x7fab3414e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab29d9b700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab340b2170 0x7fab3414e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab29d9b700 1 -- 172.21.15.140:0/1232027877 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c00a360 con 0x7fab340b2550 2022-07-21T01:48:36.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab29d9b700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab340b2550 0x7fab340651b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fab1c002f70 tx=0x7fab340b31d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab2bfff700 1 -- 172.21.15.140:0/1232027877 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab1c020070 con 0x7fab340b2550 2022-07-21T01:48:36.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab2bfff700 1 -- 172.21.15.140:0/1232027877 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c010bd0 con 0x7fab340b2550 2022-07-21T01:48:36.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab3359e700 1 -- 172.21.15.140:0/1232027877 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab341409c0 con 0x7fab340b2550 2022-07-21T01:48:36.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab2bfff700 1 -- 172.21.15.140:0/1232027877 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab1c00f560 con 0x7fab340b2550 2022-07-21T01:48:36.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab3359e700 1 -- 172.21.15.140:0/1232027877 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab34141800 con 0x7fab340b2550 2022-07-21T01:48:36.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.215+0000 7fab3359e700 1 -- 172.21.15.140:0/1232027877 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab3414e660 con 0x7fab340b2550 2022-07-21T01:48:36.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.219+0000 7fab2bfff700 1 -- 172.21.15.140:0/1232027877 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fab1c00e040 con 0x7fab340b2550 2022-07-21T01:48:36.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.219+0000 7fab2bfff700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fab18053a70 0x7fab18055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.219+0000 7fab2bfff700 1 -- 172.21.15.140:0/1232027877 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fab1c06e9d0 con 0x7fab340b2550 2022-07-21T01:48:36.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.219+0000 7fab3259c700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fab18053a70 0x7fab18055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.219+0000 7fab2bfff700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] conn(0x7fab1805add0 0x7fab1805d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.219+0000 7fab2bfff700 1 -- 172.21.15.140:0/1232027877 --> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab1805d810 con 0x7fab1805add0 2022-07-21T01:48:36.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.219+0000 7fab2bfff700 1 -- 172.21.15.140:0/1232027877 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab1c073240 con 0x7fab340b2550 2022-07-21T01:48:36.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.219+0000 7fab32d9d700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] conn(0x7fab1805add0 0x7fab1805d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.219+0000 7fab3259c700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fab18053a70 0x7fab18055f30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fab2400b000 tx=0x7fab24006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.219+0000 7fab32d9d700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] conn(0x7fab1805add0 0x7fab1805d230 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.223+0000 7fab2bfff700 1 -- 172.21.15.140:0/1232027877 <== osd.6 v2:172.21.15.152:6808/23144 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7fab1805d810 con 0x7fab1805add0 2022-07-21T01:48:36.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.239+0000 7fab3359e700 1 -- 172.21.15.140:0/1232027877 --> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab340b0c10 con 0x7fab1805add0 2022-07-21T01:48:36.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.239+0000 7fab2bfff700 1 -- 172.21.15.140:0/1232027877 <== osd.6 v2:172.21.15.152:6808/23144 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fab340b0c10 con 0x7fab1805add0 2022-07-21T01:48:36.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.243+0000 7fab3359e700 1 -- 172.21.15.140:0/1232027877 >> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] conn(0x7fab1805add0 msgr2=0x7fab1805d230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.243+0000 7fab3359e700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] conn(0x7fab1805add0 0x7fab1805d230 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.243+0000 7fab3359e700 1 -- 172.21.15.140:0/1232027877 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fab18053a70 msgr2=0x7fab18055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.243+0000 7fab3359e700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fab18053a70 0x7fab18055f30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fab2400b000 tx=0x7fab24006df0).stop 2022-07-21T01:48:36.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.243+0000 7fab3359e700 1 -- 172.21.15.140:0/1232027877 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab340b2550 msgr2=0x7fab340651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.243+0000 7fab3359e700 1 --2- 172.21.15.140:0/1232027877 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab340b2550 0x7fab340651b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fab1c002f70 tx=0x7fab340b31d0).stop 2022-07-21T01:48:36.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.243+0000 7fab32d9d700 1 -- 172.21.15.140:0/1232027877 reap_dead start 2022-07-21T01:48:36.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.243+0000 7fab3359e700 1 -- 172.21.15.140:0/1232027877 shutdown_connections 2022-07-21T01:48:36.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.243+0000 7fab3359e700 1 -- 172.21.15.140:0/1232027877 >> 172.21.15.140:0/1232027877 conn(0x7fab34063150 msgr2=0x7fab340b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:36.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.243+0000 7fab3359e700 1 -- 172.21.15.140:0/1232027877 shutdown_connections 2022-07-21T01:48:36.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.243+0000 7fab3359e700 1 -- 172.21.15.140:0/1232027877 wait complete. 2022-07-21T01:48:36.257 INFO:teuthology.orchestra.run.smithi140.stdout:55834574854 2022-07-21T01:48:36.258 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2022-07-21T01:48:36.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4ef59e700 1 Processor -- start 2022-07-21T01:48:36.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4ef59e700 1 -- start start 2022-07-21T01:48:36.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4ef59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4f00b5630 0x7fa4f00b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4ef59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa4f013c7c0 0x7fa4f005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4ef59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f00b0a10 con 0x7fa4f013cc50 2022-07-21T01:48:36.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4ef59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f00b0cb0 con 0x7fa4f013c7c0 2022-07-21T01:48:36.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4ef59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00b0f50 con 0x7fa4f00b5630 2022-07-21T01:48:36.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4edd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa4f013c7c0 0x7fa4f005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4edd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa4f013c7c0 0x7fa4f005bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60760/0 (socket says 172.21.15.140:60760) 2022-07-21T01:48:36.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4edd9b700 1 -- 172.21.15.140:0/3046630147 learned_addr learned my addr 172.21.15.140:0/3046630147 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:36.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4ee59c700 1 --2- 172.21.15.140:0/3046630147 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4f00b5630 0x7fa4f00b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4edd9b700 1 -- 172.21.15.140:0/3046630147 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4f00b5630 msgr2=0x7fa4f00b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:36.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4edd9b700 1 --2- 172.21.15.140:0/3046630147 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4f00b5630 0x7fa4f00b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4edd9b700 1 -- 172.21.15.140:0/3046630147 >> v1:172.21.15.152:6789/0 conn(0x7fa4f013cc50 legacy=0x7fa4f005c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4edd9b700 1 -- 172.21.15.140:0/3046630147 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f00afa10 con 0x7fa4f013c7c0 2022-07-21T01:48:36.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4edd9b700 1 --2- 172.21.15.140:0/3046630147 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4f013c7c0 0x7fa4f005bd90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fa4d80032f0 tx=0x7fa4d8009290).ready entity=mon.0 client_cookie=3e1f2b036805fc88 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4ed59a700 1 -- 172.21.15.140:0/3046630147 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4d800b8a0 con 0x7fa4f013c7c0 2022-07-21T01:48:36.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4ed59a700 1 -- 172.21.15.140:0/3046630147 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d800ba00 con 0x7fa4f013c7c0 2022-07-21T01:48:36.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.371+0000 7fa4ed59a700 1 -- 172.21.15.140:0/3046630147 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4d800bb70 con 0x7fa4f013c7c0 2022-07-21T01:48:36.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3046630147 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4f013c7c0 msgr2=0x7fa4f005bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 --2- 172.21.15.140:0/3046630147 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4f013c7c0 0x7fa4f005bd90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fa4d80032f0 tx=0x7fa4d8009290).stop 2022-07-21T01:48:36.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3046630147 shutdown_connections 2022-07-21T01:48:36.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 --2- 172.21.15.140:0/3046630147 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4f013c7c0 0x7fa4f005bd90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 --2- 172.21.15.140:0/3046630147 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4f00b5630 0x7fa4f00b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3046630147 >> 172.21.15.140:0/3046630147 conn(0x7fa4f0063490 msgr2=0x7fa4f00b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:36.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3046630147 shutdown_connections 2022-07-21T01:48:36.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3046630147 wait complete. 2022-07-21T01:48:36.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 Processor -- start 2022-07-21T01:48:36.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 -- start start 2022-07-21T01:48:36.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f00b5630 0x7fa4f01520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4f013c7c0 0x7fa4f013fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4f013cc50 0x7fa4f01527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0140610 con 0x7fa4f013c7c0 2022-07-21T01:48:36.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f013b8e0 con 0x7fa4f00b5630 2022-07-21T01:48:36.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0141940 con 0x7fa4f013cc50 2022-07-21T01:48:36.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4edd9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4f013c7c0 0x7fa4f013fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ee59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f00b5630 0x7fa4f01520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4edd9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4f013c7c0 0x7fa4f013fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60770/0 (socket says 172.21.15.140:60770) 2022-07-21T01:48:36.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4edd9b700 1 -- 172.21.15.140:0/3956335876 learned_addr learned my addr 172.21.15.140:0/3956335876 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:36.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4eed9d700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4f013cc50 0x7fa4f01527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4eed9d700 1 -- 172.21.15.140:0/3956335876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f00b5630 msgr2=0x7fa4f01520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4eed9d700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4f00b5630 0x7fa4f01520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4eed9d700 1 -- 172.21.15.140:0/3956335876 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4f013c7c0 msgr2=0x7fa4f013fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4eed9d700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4f013c7c0 0x7fa4f013fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4eed9d700 1 -- 172.21.15.140:0/3956335876 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d8009530 con 0x7fa4f013cc50 2022-07-21T01:48:36.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4edd9b700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4f013c7c0 0x7fa4f013fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:36.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4eed9d700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4f013cc50 0x7fa4f01527f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa4f0062550 tx=0x7fa4e8007b90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4e77fe700 1 -- 172.21.15.140:0/3956335876 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e800cd20 con 0x7fa4f013cc50 2022-07-21T01:48:36.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4e77fe700 1 -- 172.21.15.140:0/3956335876 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e800ce80 con 0x7fa4f013cc50 2022-07-21T01:48:36.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3956335876 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f0141be0 con 0x7fa4f013cc50 2022-07-21T01:48:36.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.375+0000 7fa4e77fe700 1 -- 172.21.15.140:0/3956335876 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e8007220 con 0x7fa4f013cc50 2022-07-21T01:48:36.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.379+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3956335876 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f0067640 con 0x7fa4f013cc50 2022-07-21T01:48:36.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.379+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3956335876 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa4f0140760 con 0x7fa4f013cc50 2022-07-21T01:48:36.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.379+0000 7fa4e77fe700 1 -- 172.21.15.140:0/3956335876 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa4e8010030 con 0x7fa4f013cc50 2022-07-21T01:48:36.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.379+0000 7fa4e77fe700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4d4053b00 0x7fa4d4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.379+0000 7fa4e77fe700 1 -- 172.21.15.140:0/3956335876 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa4e806c240 con 0x7fa4f013cc50 2022-07-21T01:48:36.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.379+0000 7fa4ee59c700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4d4053b00 0x7fa4d4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.379+0000 7fa4e77fe700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] conn(0x7fa4d405ade0 0x7fa4d405d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.379+0000 7fa4e77fe700 1 -- 172.21.15.140:0/3956335876 --> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4d405d820 con 0x7fa4d405ade0 2022-07-21T01:48:36.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.379+0000 7fa4e77fe700 1 -- 172.21.15.140:0/3956335876 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4e806c510 con 0x7fa4f013cc50 2022-07-21T01:48:36.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.379+0000 7fa4ee59c700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4d4053b00 0x7fa4d4055fc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa4e000b000 tx=0x7fa4e0006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.403+0000 7fa4edd9b700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] conn(0x7fa4d405ade0 0x7fa4d405d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.407+0000 7fa4edd9b700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] conn(0x7fa4d405ade0 0x7fa4d405d240 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.407+0000 7fa4e77fe700 1 -- 172.21.15.140:0/3956335876 <== osd.7 v2:172.21.15.152:6824/23142 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7fa4d405d820 con 0x7fa4d405ade0 2022-07-21T01:48:36.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.427+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3956335876 --> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa4f0063d20 con 0x7fa4d405ade0 2022-07-21T01:48:36.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.427+0000 7fa4e77fe700 1 -- 172.21.15.140:0/3956335876 <== osd.7 v2:172.21.15.152:6824/23142 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa4f0063d20 con 0x7fa4d405ade0 2022-07-21T01:48:36.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.431+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3956335876 >> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] conn(0x7fa4d405ade0 msgr2=0x7fa4d405d240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.431+0000 7fa4ef59e700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] conn(0x7fa4d405ade0 0x7fa4d405d240 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.431+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3956335876 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4d4053b00 msgr2=0x7fa4d4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.431+0000 7fa4ef59e700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4d4053b00 0x7fa4d4055fc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa4e000b000 tx=0x7fa4e0006df0).stop 2022-07-21T01:48:36.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.431+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3956335876 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4f013cc50 msgr2=0x7fa4f01527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.431+0000 7fa4ef59e700 1 --2- 172.21.15.140:0/3956335876 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4f013cc50 0x7fa4f01527f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa4f0062550 tx=0x7fa4e8007b90).stop 2022-07-21T01:48:36.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.431+0000 7fa4eed9d700 1 -- 172.21.15.140:0/3956335876 reap_dead start 2022-07-21T01:48:36.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.431+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3956335876 shutdown_connections 2022-07-21T01:48:36.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.431+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3956335876 >> 172.21.15.140:0/3956335876 conn(0x7fa4f0063490 msgr2=0x7fa4f00b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:36.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.431+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3956335876 shutdown_connections 2022-07-21T01:48:36.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.431+0000 7fa4ef59e700 1 -- 172.21.15.140:0/3956335876 wait complete. 2022-07-21T01:48:36.443 INFO:teuthology.orchestra.run.smithi140.stdout:55834574852 2022-07-21T01:48:36.444 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2022-07-21T01:48:36.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f6220b4f700 1 Processor -- start 2022-07-21T01:48:36.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f6220b4f700 1 -- start start 2022-07-21T01:48:36.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f6220b4f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f621c0b3cb0 0x7f621c0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f6220b4f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f621c0ad0e0 0x7f621c1418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f6220b4f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621c0ae280 con 0x7f621c0b6800 2022-07-21T01:48:36.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f6220b4f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f621c0b6be0 con 0x7f621c0b3cb0 2022-07-21T01:48:36.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f6220b4f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0ae4b0 con 0x7f621c0ad0e0 2022-07-21T01:48:36.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621affd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f621c0b3cb0 0x7f621c0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621affd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f621c0b3cb0 0x7f621c0aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60776/0 (socket says 172.21.15.140:60776) 2022-07-21T01:48:36.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621affd700 1 -- 172.21.15.140:0/2253074044 learned_addr learned my addr 172.21.15.140:0/2253074044 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:36.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621bfff700 1 --2- 172.21.15.140:0/2253074044 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f621c0ad0e0 0x7f621c1418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621affd700 1 -- 172.21.15.140:0/2253074044 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f621c0ad0e0 msgr2=0x7f621c1418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:36.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621affd700 1 --2- 172.21.15.140:0/2253074044 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f621c0ad0e0 0x7f621c1418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621affd700 1 -- 172.21.15.140:0/2253074044 >> v1:172.21.15.152:6789/0 conn(0x7f621c0b6800 legacy=0x7f621c145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621affd700 1 -- 172.21.15.140:0/2253074044 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c0ae750 con 0x7f621c0b3cb0 2022-07-21T01:48:36.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621affd700 1 --2- 172.21.15.140:0/2253074044 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f621c0b3cb0 0x7f621c0aebf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f62040032f0 tx=0x7f6204009290).ready entity=mon.0 client_cookie=89ce293b54b374f3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621a7fc700 1 -- 172.21.15.140:0/2253074044 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f620400b8a0 con 0x7f621c0b3cb0 2022-07-21T01:48:36.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621a7fc700 1 -- 172.21.15.140:0/2253074044 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620400ba00 con 0x7f621c0b3cb0 2022-07-21T01:48:36.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.551+0000 7f621a7fc700 1 -- 172.21.15.140:0/2253074044 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f620400bb70 con 0x7f621c0b3cb0 2022-07-21T01:48:36.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.555+0000 7f6220b4f700 1 -- 172.21.15.140:0/2253074044 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f621c0b3cb0 msgr2=0x7f621c0aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.555+0000 7f6220b4f700 1 --2- 172.21.15.140:0/2253074044 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f621c0b3cb0 0x7f621c0aebf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f62040032f0 tx=0x7f6204009290).stop 2022-07-21T01:48:36.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.555+0000 7f6220b4f700 1 -- 172.21.15.140:0/2253074044 shutdown_connections 2022-07-21T01:48:36.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.555+0000 7f6220b4f700 1 --2- 172.21.15.140:0/2253074044 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f621c0ad0e0 0x7f621c1418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.555+0000 7f6220b4f700 1 --2- 172.21.15.140:0/2253074044 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f621c0b3cb0 0x7f621c0aebf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.555+0000 7f6220b4f700 1 -- 172.21.15.140:0/2253074044 >> 172.21.15.140:0/2253074044 conn(0x7f621c0b9dd0 msgr2=0x7f621c0b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:36.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.555+0000 7f6220b4f700 1 -- 172.21.15.140:0/2253074044 shutdown_connections 2022-07-21T01:48:36.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.555+0000 7f6220b4f700 1 -- 172.21.15.140:0/2253074044 wait complete. 2022-07-21T01:48:36.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.555+0000 7f6220b4f700 1 Processor -- start 2022-07-21T01:48:36.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.555+0000 7f6220b4f700 1 -- start start 2022-07-21T01:48:36.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f6220b4f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c0ad0e0 0x7f621c144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f6220b4f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f621c0b3cb0 0x7f621c0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f6220b4f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f621c0b6800 0x7f621c0b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f6220b4f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0b0320 con 0x7f621c0b3cb0 2022-07-21T01:48:36.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f6220b4f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0b0080 con 0x7f621c0ad0e0 2022-07-21T01:48:36.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f6220b4f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0b1000 con 0x7f621c0b6800 2022-07-21T01:48:36.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f621c0b3cb0 0x7f621c0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c0ad0e0 0x7f621c144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f621c0b3cb0 0x7f621c0b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60786/0 (socket says 172.21.15.140:60786) 2022-07-21T01:48:36.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621b7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c0ad0e0 0x7f621c144180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:38290/0 (socket says 172.21.15.140:38290) 2022-07-21T01:48:36.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621affd700 1 -- 172.21.15.140:0/1458292802 learned_addr learned my addr 172.21.15.140:0/1458292802 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:36.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621bfff700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f621c0b6800 0x7f621c0b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621affd700 1 -- 172.21.15.140:0/1458292802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f621c0b6800 msgr2=0x7f621c0b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621affd700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f621c0b6800 0x7f621c0b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621affd700 1 -- 172.21.15.140:0/1458292802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c0ad0e0 msgr2=0x7f621c144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621affd700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c0ad0e0 0x7f621c144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621affd700 1 -- 172.21.15.140:0/1458292802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6204009530 con 0x7f621c0b3cb0 2022-07-21T01:48:36.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f621affd700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f621c0b3cb0 0x7f621c0b2720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f62040032f0 tx=0x7f6204016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f6218ff9700 1 -- 172.21.15.140:0/1458292802 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6204021440 con 0x7f621c0b3cb0 2022-07-21T01:48:36.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f6218ff9700 1 -- 172.21.15.140:0/1458292802 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62040215a0 con 0x7f621c0b3cb0 2022-07-21T01:48:36.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f6220b4f700 1 -- 172.21.15.140:0/1458292802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c146ac0 con 0x7f621c0b3cb0 2022-07-21T01:48:36.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f6218ff9700 1 -- 172.21.15.140:0/1458292802 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6204021900 con 0x7f621c0b3cb0 2022-07-21T01:48:36.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.559+0000 7f6220b4f700 1 -- 172.21.15.140:0/1458292802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c0af6f0 con 0x7f621c0b3cb0 2022-07-21T01:48:36.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.563+0000 7f6220b4f700 1 -- 172.21.15.140:0/1458292802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621c0b0e40 con 0x7f621c0b3cb0 2022-07-21T01:48:36.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.563+0000 7f6218ff9700 1 -- 172.21.15.140:0/1458292802 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6204029070 con 0x7f621c0b3cb0 2022-07-21T01:48:36.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.563+0000 7f6218ff9700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62000539e0 0x7f6200055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.563+0000 7f6218ff9700 1 -- 172.21.15.140:0/1458292802 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f620406d390 con 0x7f621c0b3cb0 2022-07-21T01:48:36.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.563+0000 7f621b7fe700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62000539e0 0x7f6200055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.563+0000 7f621b7fe700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62000539e0 0x7f6200055ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f621400d000 tx=0x7f6214007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.567+0000 7f6218ff9700 1 -- 172.21.15.140:0/1458292802 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f62040442f0 con 0x7f621c0b3cb0 2022-07-21T01:48:36.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.691+0000 7f6220b4f700 1 -- 172.21.15.140:0/1458292802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f621c0b11c0 con 0x7f621c0b3cb0 2022-07-21T01:48:36.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.695+0000 7f6218ff9700 1 -- 172.21.15.140:0/1458292802 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6204015070 con 0x7f621c0b3cb0 2022-07-21T01:48:36.695 INFO:teuthology.orchestra.run.smithi140.stdout:55834574854 2022-07-21T01:48:36.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.695+0000 7f6220b4f700 1 -- 172.21.15.140:0/1458292802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62000539e0 msgr2=0x7f6200055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.695+0000 7f6220b4f700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62000539e0 0x7f6200055ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f621400d000 tx=0x7f6214007b90).stop 2022-07-21T01:48:36.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.695+0000 7f6220b4f700 1 -- 172.21.15.140:0/1458292802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f621c0b3cb0 msgr2=0x7f621c0b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.695+0000 7f6220b4f700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f621c0b3cb0 0x7f621c0b2720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f62040032f0 tx=0x7f6204016000).stop 2022-07-21T01:48:36.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.695+0000 7f6220b4f700 1 -- 172.21.15.140:0/1458292802 shutdown_connections 2022-07-21T01:48:36.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.695+0000 7f6220b4f700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62000539e0 0x7f6200055ea0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.695+0000 7f6220b4f700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f621c0b6800 0x7f621c0b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.695+0000 7f6220b4f700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f621c0b3cb0 0x7f621c0b2720 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.695+0000 7f6220b4f700 1 --2- 172.21.15.140:0/1458292802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f621c0ad0e0 0x7f621c144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.695+0000 7f6220b4f700 1 -- 172.21.15.140:0/1458292802 >> 172.21.15.140:0/1458292802 conn(0x7f621c0b9dd0 msgr2=0x7f621c0b7140 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:36.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.699+0000 7f6220b4f700 1 -- 172.21.15.140:0/1458292802 shutdown_connections 2022-07-21T01:48:36.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.699+0000 7f6220b4f700 1 -- 172.21.15.140:0/1458292802 wait complete. 2022-07-21T01:48:36.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2022-07-21T01:48:36.715 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2022-07-21T01:48:36.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.819+0000 7f1746e94700 1 Processor -- start 2022-07-21T01:48:36.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.819+0000 7f1746e94700 1 -- start start 2022-07-21T01:48:36.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17400b02a0 0x7f1740146a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f17400afb80 0x7f1740146f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1740146340 con 0x7f1740060fe0 2022-07-21T01:48:36.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17400613c0 con 0x7f17400afb80 2022-07-21T01:48:36.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17400ac110 con 0x7f17400b02a0 2022-07-21T01:48:36.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1745431700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f17400afb80 0x7f1740146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1745431700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f17400afb80 0x7f1740146f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60794/0 (socket says 172.21.15.140:60794) 2022-07-21T01:48:36.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1745431700 1 -- 172.21.15.140:0/3494816012 learned_addr learned my addr 172.21.15.140:0/3494816012 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:36.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1737fff700 1 --2- 172.21.15.140:0/3494816012 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17400b02a0 0x7f1740146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f17377fe700 1 -- 172.21.15.140:0/3494816012 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963138342 0 0) 0x7f1740146340 con 0x7f1740060fe0 2022-07-21T01:48:36.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1745431700 1 -- 172.21.15.140:0/3494816012 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17400b02a0 msgr2=0x7f1740146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:36.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1745431700 1 --2- 172.21.15.140:0/3494816012 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17400b02a0 0x7f1740146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1745431700 1 -- 172.21.15.140:0/3494816012 >> v1:172.21.15.152:6789/0 conn(0x7f1740060fe0 legacy=0x7f17400b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1745431700 1 -- 172.21.15.140:0/3494816012 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1740146570 con 0x7f17400afb80 2022-07-21T01:48:36.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1745431700 1 --2- 172.21.15.140:0/3494816012 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17400afb80 0x7f1740146f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f17380032f0 tx=0x7f1738009290).ready entity=mon.0 client_cookie=ca2c1f2884b61b13 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f17377fe700 1 -- 172.21.15.140:0/3494816012 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f173800b8a0 con 0x7f17400afb80 2022-07-21T01:48:36.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f17377fe700 1 -- 172.21.15.140:0/3494816012 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173800ba00 con 0x7f17400afb80 2022-07-21T01:48:36.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f17377fe700 1 -- 172.21.15.140:0/3494816012 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f173800bb70 con 0x7f17400afb80 2022-07-21T01:48:36.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 -- 172.21.15.140:0/3494816012 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17400afb80 msgr2=0x7f1740146f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 --2- 172.21.15.140:0/3494816012 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17400afb80 0x7f1740146f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f17380032f0 tx=0x7f1738009290).stop 2022-07-21T01:48:36.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 -- 172.21.15.140:0/3494816012 shutdown_connections 2022-07-21T01:48:36.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 --2- 172.21.15.140:0/3494816012 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17400afb80 0x7f1740146f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 --2- 172.21.15.140:0/3494816012 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17400b02a0 0x7f1740146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 -- 172.21.15.140:0/3494816012 >> 172.21.15.140:0/3494816012 conn(0x7f17400b5cb0 msgr2=0x7f17400b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:36.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 -- 172.21.15.140:0/3494816012 shutdown_connections 2022-07-21T01:48:36.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.823+0000 7f1746e94700 1 -- 172.21.15.140:0/3494816012 wait complete. 2022-07-21T01:48:36.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1746e94700 1 Processor -- start 2022-07-21T01:48:36.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1746e94700 1 -- start start 2022-07-21T01:48:36.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1746e94700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1740060fe0 0x7f174013feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1746e94700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17400afb80 0x7f1740098220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1746e94700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17400b02a0 0x7f17400ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1746e94700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174013f220 con 0x7f17400afb80 2022-07-21T01:48:36.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1746e94700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17400af370 con 0x7f1740060fe0 2022-07-21T01:48:36.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1746e94700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17400b1980 con 0x7f17400b02a0 2022-07-21T01:48:36.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1737fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17400afb80 0x7f1740098220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1737fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17400afb80 0x7f1740098220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60798/0 (socket says 172.21.15.140:60798) 2022-07-21T01:48:36.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1745431700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17400b02a0 0x7f17400ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1737fff700 1 -- 172.21.15.140:0/1010141636 learned_addr learned my addr 172.21.15.140:0/1010141636 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:36.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1744c30700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1740060fe0 0x7f174013feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1737fff700 1 -- 172.21.15.140:0/1010141636 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17400b02a0 msgr2=0x7f17400ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1737fff700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17400b02a0 0x7f17400ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1737fff700 1 -- 172.21.15.140:0/1010141636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1740060fe0 msgr2=0x7f174013feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1737fff700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1740060fe0 0x7f174013feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1737fff700 1 -- 172.21.15.140:0/1010141636 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1738009530 con 0x7f17400afb80 2022-07-21T01:48:36.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1737fff700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17400afb80 0x7f1740098220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f174009b2e0 tx=0x7f1730006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1735ffb700 1 -- 172.21.15.140:0/1010141636 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1730011750 con 0x7f17400afb80 2022-07-21T01:48:36.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1735ffb700 1 -- 172.21.15.140:0/1010141636 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17300118b0 con 0x7f17400afb80 2022-07-21T01:48:36.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1746e94700 1 -- 172.21.15.140:0/1010141636 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17400ade70 con 0x7f17400afb80 2022-07-21T01:48:36.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1735ffb700 1 -- 172.21.15.140:0/1010141636 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1730011c10 con 0x7f17400afb80 2022-07-21T01:48:36.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.827+0000 7f1746e94700 1 -- 172.21.15.140:0/1010141636 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174013f3b0 con 0x7f17400afb80 2022-07-21T01:48:36.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.831+0000 7f1746e94700 1 -- 172.21.15.140:0/1010141636 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17400abe40 con 0x7f17400afb80 2022-07-21T01:48:36.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.831+0000 7f1735ffb700 1 -- 172.21.15.140:0/1010141636 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1730011d70 con 0x7f17400afb80 2022-07-21T01:48:36.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.831+0000 7f1735ffb700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1720053b30 0x7f1720055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:36.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.831+0000 7f1735ffb700 1 -- 172.21.15.140:0/1010141636 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f173006c260 con 0x7f17400afb80 2022-07-21T01:48:36.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.831+0000 7f1744c30700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1720053b30 0x7f1720055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:36.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.831+0000 7f1744c30700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1720053b30 0x7f1720055ff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f173c00d000 tx=0x7f173c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:36.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.839+0000 7f1735ffb700 1 -- 172.21.15.140:0/1010141636 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1730043070 con 0x7f17400afb80 2022-07-21T01:48:36.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.963+0000 7f1746e94700 1 -- 172.21.15.140:0/1010141636 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f17400574e0 con 0x7f17400afb80 2022-07-21T01:48:36.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.963+0000 7f1735ffb700 1 -- 172.21.15.140:0/1010141636 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1730046690 con 0x7f17400afb80 2022-07-21T01:48:36.968 INFO:teuthology.orchestra.run.smithi140.stdout:55834574853 2022-07-21T01:48:36.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 -- 172.21.15.140:0/1010141636 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1720053b30 msgr2=0x7f1720055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1720053b30 0x7f1720055ff0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f173c00d000 tx=0x7f173c009040).stop 2022-07-21T01:48:36.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 -- 172.21.15.140:0/1010141636 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17400afb80 msgr2=0x7f1740098220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:36.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17400afb80 0x7f1740098220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f174009b2e0 tx=0x7f1730006df0).stop 2022-07-21T01:48:36.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 -- 172.21.15.140:0/1010141636 shutdown_connections 2022-07-21T01:48:36.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1720053b30 0x7f1720055ff0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17400b02a0 0x7f17400ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17400afb80 0x7f1740098220 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 --2- 172.21.15.140:0/1010141636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1740060fe0 0x7f174013feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:36.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 -- 172.21.15.140:0/1010141636 >> 172.21.15.140:0/1010141636 conn(0x7f17400b5cb0 msgr2=0x7f1740060280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:36.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 -- 172.21.15.140:0/1010141636 shutdown_connections 2022-07-21T01:48:36.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:36.967+0000 7f1746e94700 1 -- 172.21.15.140:0/1010141636 wait complete. 2022-07-21T01:48:36.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2022-07-21T01:48:36.979 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2022-07-21T01:48:37.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 Processor -- start 2022-07-21T01:48:37.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 -- start start 2022-07-21T01:48:37.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1e80af8c0 0x7fa1e80ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa1e8143a10 0x7fa1e8143e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e80b0b80 con 0x7fa1e80b5970 2022-07-21T01:48:37.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e805d4d0 con 0x7fa1e8143a10 2022-07-21T01:48:37.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80b5d50 con 0x7fa1e80af8c0 2022-07-21T01:48:37.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa1e8143a10 0x7fa1e8143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa1e8143a10 0x7fa1e8143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60806/0 (socket says 172.21.15.140:60806) 2022-07-21T01:48:37.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e7fff700 1 -- 172.21.15.140:0/2443330429 learned_addr learned my addr 172.21.15.140:0/2443330429 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:37.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e6ffd700 1 --2- 172.21.15.140:0/2443330429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1e80af8c0 0x7fa1e80ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e7fff700 1 -- 172.21.15.140:0/2443330429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1e80af8c0 msgr2=0x7fa1e80ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:37.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e7fff700 1 --2- 172.21.15.140:0/2443330429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1e80af8c0 0x7fa1e80ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e7fff700 1 -- 172.21.15.140:0/2443330429 >> v1:172.21.15.152:6789/0 conn(0x7fa1e80b5970 legacy=0x7fa1e80ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e7fff700 1 -- 172.21.15.140:0/2443330429 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e80b0db0 con 0x7fa1e8143a10 2022-07-21T01:48:37.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e6ffd700 1 --2- 172.21.15.140:0/2443330429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1e80af8c0 0x7fa1e80ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:37.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e7fff700 1 --2- 172.21.15.140:0/2443330429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1e8143a10 0x7fa1e8143e80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fa1d8002f70 tx=0x7fa1d8006eb0).ready entity=mon.0 client_cookie=cff1610ea073df7a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:37.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e67fc700 1 -- 172.21.15.140:0/2443330429 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1d800d5c0 con 0x7fa1e8143a10 2022-07-21T01:48:37.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e67fc700 1 -- 172.21.15.140:0/2443330429 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8000e90 con 0x7fa1e8143a10 2022-07-21T01:48:37.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1e67fc700 1 -- 172.21.15.140:0/2443330429 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1d800d950 con 0x7fa1e8143a10 2022-07-21T01:48:37.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/2443330429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1e8143a10 msgr2=0x7fa1e8143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 --2- 172.21.15.140:0/2443330429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1e8143a10 0x7fa1e8143e80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fa1d8002f70 tx=0x7fa1d8006eb0).stop 2022-07-21T01:48:37.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/2443330429 shutdown_connections 2022-07-21T01:48:37.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 --2- 172.21.15.140:0/2443330429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1e8143a10 0x7fa1e8143e80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 --2- 172.21.15.140:0/2443330429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1e80af8c0 0x7fa1e80ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/2443330429 >> 172.21.15.140:0/2443330429 conn(0x7fa1e8063490 msgr2=0x7fa1e80b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:37.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/2443330429 shutdown_connections 2022-07-21T01:48:37.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.087+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/2443330429 wait complete. 2022-07-21T01:48:37.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1ecc0c700 1 Processor -- start 2022-07-21T01:48:37.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1ecc0c700 1 -- start start 2022-07-21T01:48:37.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1ecc0c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1e80af8c0 0x7fa1e8147170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1ecc0c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1e80b5970 0x7fa1e81404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1ecc0c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1e8143a10 0x7fa1e8159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1ecc0c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8140990 con 0x7fa1e80b5970 2022-07-21T01:48:37.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1ecc0c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8145970 con 0x7fa1e8143a10 2022-07-21T01:48:37.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1ecc0c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e81597b0 con 0x7fa1e80af8c0 2022-07-21T01:48:37.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1e80b5970 0x7fa1e81404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1e80b5970 0x7fa1e81404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60810/0 (socket says 172.21.15.140:60810) 2022-07-21T01:48:37.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e6ffd700 1 -- 172.21.15.140:0/1550086376 learned_addr learned my addr 172.21.15.140:0/1550086376 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:37.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e77fe700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1e80af8c0 0x7fa1e8147170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e7fff700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1e8143a10 0x7fa1e8159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e6ffd700 1 -- 172.21.15.140:0/1550086376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1e80af8c0 msgr2=0x7fa1e8147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e6ffd700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1e80af8c0 0x7fa1e8147170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e6ffd700 1 -- 172.21.15.140:0/1550086376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1e8143a10 msgr2=0x7fa1e8159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e6ffd700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1e8143a10 0x7fa1e8159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e6ffd700 1 -- 172.21.15.140:0/1550086376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d800b1e0 con 0x7fa1e80b5970 2022-07-21T01:48:37.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e6ffd700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1e80b5970 0x7fa1e81404a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fa1dc007000 tx=0x7fa1dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:37.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e4ff9700 1 -- 172.21.15.140:0/1550086376 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1dc00ce80 con 0x7fa1e80b5970 2022-07-21T01:48:37.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e4ff9700 1 -- 172.21.15.140:0/1550086376 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc010580 con 0x7fa1e80b5970 2022-07-21T01:48:37.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/1550086376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e81455a0 con 0x7fa1e80b5970 2022-07-21T01:48:37.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1e4ff9700 1 -- 172.21.15.140:0/1550086376 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1dc0108e0 con 0x7fa1e80b5970 2022-07-21T01:48:37.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.091+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/1550086376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e8145170 con 0x7fa1e80b5970 2022-07-21T01:48:37.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.095+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/1550086376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e80acae0 con 0x7fa1e80b5970 2022-07-21T01:48:37.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.095+0000 7fa1e4ff9700 1 -- 172.21.15.140:0/1550086376 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa1dc010a40 con 0x7fa1e80b5970 2022-07-21T01:48:37.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.095+0000 7fa1e4ff9700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1c8053a70 0x7fa1c8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.095+0000 7fa1e4ff9700 1 -- 172.21.15.140:0/1550086376 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa1dc06bc00 con 0x7fa1e80b5970 2022-07-21T01:48:37.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.095+0000 7fa1e77fe700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1c8053a70 0x7fa1c8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.095+0000 7fa1e77fe700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1c8053a70 0x7fa1c8055f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa1e000d000 tx=0x7fa1e0000f40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:37.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.103+0000 7fa1e4ff9700 1 -- 172.21.15.140:0/1550086376 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa1dc071020 con 0x7fa1e80b5970 2022-07-21T01:48:37.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/1550086376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa1e80574e0 con 0x7fa1e80b5970 2022-07-21T01:48:37.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1e4ff9700 1 -- 172.21.15.140:0/1550086376 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa1dc046550 con 0x7fa1e80b5970 2022-07-21T01:48:37.229 INFO:teuthology.orchestra.run.smithi140.stdout:55834574854 2022-07-21T01:48:37.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/1550086376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1c8053a70 msgr2=0x7fa1c8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1c8053a70 0x7fa1c8055f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa1e000d000 tx=0x7fa1e0000f40).stop 2022-07-21T01:48:37.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/1550086376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1e80b5970 msgr2=0x7fa1e81404a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1e80b5970 0x7fa1e81404a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fa1dc007000 tx=0x7fa1dc008040).stop 2022-07-21T01:48:37.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/1550086376 shutdown_connections 2022-07-21T01:48:37.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1c8053a70 0x7fa1c8055f30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1e8143a10 0x7fa1e8159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1e80b5970 0x7fa1e81404a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 --2- 172.21.15.140:0/1550086376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1e80af8c0 0x7fa1e8147170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/1550086376 >> 172.21.15.140:0/1550086376 conn(0x7fa1e8063490 msgr2=0x7fa1e80b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:37.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/1550086376 shutdown_connections 2022-07-21T01:48:37.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.231+0000 7fa1ecc0c700 1 -- 172.21.15.140:0/1550086376 wait complete. 2022-07-21T01:48:37.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2022-07-21T01:48:37.248 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2022-07-21T01:48:37.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 Processor -- start 2022-07-21T01:48:37.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 -- start start 2022-07-21T01:48:37.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f73dc0af8c0 0x7f73dc0ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc0b0b80 con 0x7f73dc143a10 2022-07-21T01:48:37.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc05d4d0 con 0x7f73dc0b5970 2022-07-21T01:48:37.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73dc0b5d50 con 0x7f73dc0af8c0 2022-07-21T01:48:37.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73db59e700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f73dc143a10 0x7f73dc143e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46736/0 (socket says 172.21.15.140:46736) 2022-07-21T01:48:37.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73da59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f73dc0af8c0 0x7f73dc0ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73db59e700 1 -- 172.21.15.140:0/3707737462 learned_addr learned my addr 172.21.15.140:0/3707737462 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:37.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73d9d9b700 1 -- 172.21.15.140:0/3707737462 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141357436 0 0) 0x7f73dc0b0b80 con 0x7f73dc143a10 2022-07-21T01:48:37.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73d9d9b700 1 -- 172.21.15.140:0/3707737462 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73bc003260 con 0x7f73dc143a10 2022-07-21T01:48:37.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73d9d9b700 1 -- 172.21.15.140:0/3707737462 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817777398 0 0) 0x7f73dc05d4d0 con 0x7f73dc0b5970 2022-07-21T01:48:37.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73da59c700 1 -- 172.21.15.140:0/3707737462 >> v1:172.21.15.152:6789/0 conn(0x7f73dc0b5970 legacy=0x7f73dc0ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73da59c700 1 -- 172.21.15.140:0/3707737462 >> v1:172.21.15.140:6789/0 conn(0x7f73dc143a10 legacy=0x7f73dc143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73da59c700 1 -- 172.21.15.140:0/3707737462 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc0b0db0 con 0x7f73dc0af8c0 2022-07-21T01:48:37.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73da59c700 1 --2- 172.21.15.140:0/3707737462 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73dc0af8c0 0x7f73dc0ae810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f73cc0032f0 tx=0x7f73cc0091d0).ready entity=mon.0 client_cookie=6b984498051f528c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:37.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73d9d9b700 1 -- 172.21.15.140:0/3707737462 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73cc00b6c0 con 0x7f73dc0af8c0 2022-07-21T01:48:37.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73d9d9b700 1 -- 172.21.15.140:0/3707737462 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc00b820 con 0x7f73dc0af8c0 2022-07-21T01:48:37.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73d9d9b700 1 -- 172.21.15.140:0/3707737462 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73cc00b990 con 0x7f73dc0af8c0 2022-07-21T01:48:37.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 -- 172.21.15.140:0/3707737462 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73dc0af8c0 msgr2=0x7f73dc0ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 --2- 172.21.15.140:0/3707737462 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73dc0af8c0 0x7f73dc0ae810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f73cc0032f0 tx=0x7f73cc0091d0).stop 2022-07-21T01:48:37.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 -- 172.21.15.140:0/3707737462 shutdown_connections 2022-07-21T01:48:37.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 --2- 172.21.15.140:0/3707737462 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73dc0af8c0 0x7f73dc0ae810 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 -- 172.21.15.140:0/3707737462 >> 172.21.15.140:0/3707737462 conn(0x7f73dc063490 msgr2=0x7f73dc0b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:37.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 -- 172.21.15.140:0/3707737462 shutdown_connections 2022-07-21T01:48:37.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.355+0000 7f73e0817700 1 -- 172.21.15.140:0/3707737462 wait complete. 2022-07-21T01:48:37.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73e0817700 1 Processor -- start 2022-07-21T01:48:37.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73e0817700 1 -- start start 2022-07-21T01:48:37.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73e0817700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73dc0af8c0 0x7f73dc14b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73e0817700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73dc0b5970 0x7f73dc140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73e0817700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73dc143a10 0x7f73dc1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73e0817700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc141e40 con 0x7f73dc143a10 2022-07-21T01:48:37.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73e0817700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc155e90 con 0x7f73dc0b5970 2022-07-21T01:48:37.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73e0817700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc142850 con 0x7f73dc0af8c0 2022-07-21T01:48:37.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73db59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73dc143a10 0x7f73dc1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73db59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73dc143a10 0x7f73dc1565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60828/0 (socket says 172.21.15.140:60828) 2022-07-21T01:48:37.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73db59e700 1 -- 172.21.15.140:0/3316277810 learned_addr learned my addr 172.21.15.140:0/3316277810 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:37.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73da59c700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73dc0b5970 0x7f73dc140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73dad9d700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73dc0af8c0 0x7f73dc14b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73db59e700 1 -- 172.21.15.140:0/3316277810 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73dc0af8c0 msgr2=0x7f73dc14b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73db59e700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73dc0af8c0 0x7f73dc14b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73db59e700 1 -- 172.21.15.140:0/3316277810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73dc0b5970 msgr2=0x7f73dc140cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73db59e700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73dc0b5970 0x7f73dc140cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73db59e700 1 -- 172.21.15.140:0/3316277810 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc009470 con 0x7f73dc143a10 2022-07-21T01:48:37.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73db59e700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73dc143a10 0x7f73dc1565b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f73d4001110 tx=0x7f73d4007a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:37.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73d3fff700 1 -- 172.21.15.140:0/3316277810 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73d40070d0 con 0x7f73dc143a10 2022-07-21T01:48:37.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73d3fff700 1 -- 172.21.15.140:0/3316277810 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d4007230 con 0x7f73dc143a10 2022-07-21T01:48:37.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73e0817700 1 -- 172.21.15.140:0/3316277810 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73dc156110 con 0x7f73dc143a10 2022-07-21T01:48:37.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73d3fff700 1 -- 172.21.15.140:0/3316277810 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73d4005610 con 0x7f73dc143a10 2022-07-21T01:48:37.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.359+0000 7f73e0817700 1 -- 172.21.15.140:0/3316277810 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73dc10e620 con 0x7f73dc143a10 2022-07-21T01:48:37.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.363+0000 7f73e0817700 1 -- 172.21.15.140:0/3316277810 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73dc1562a0 con 0x7f73dc143a10 2022-07-21T01:48:37.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.363+0000 7f73d3fff700 1 -- 172.21.15.140:0/3316277810 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f73d4010070 con 0x7f73dc143a10 2022-07-21T01:48:37.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.363+0000 7f73d3fff700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f73c0053980 0x7f73c0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.363+0000 7f73dad9d700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f73c0053980 0x7f73c0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.363+0000 7f73d3fff700 1 -- 172.21.15.140:0/3316277810 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f73d406b610 con 0x7f73dc143a10 2022-07-21T01:48:37.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.363+0000 7f73dad9d700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f73c0053980 0x7f73c0055e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f73c400b000 tx=0x7f73c4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:37.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.367+0000 7f73d3fff700 1 -- 172.21.15.140:0/3316277810 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f73d40423c0 con 0x7f73dc143a10 2022-07-21T01:48:37.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.491+0000 7f73e0817700 1 -- 172.21.15.140:0/3316277810 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f73dc0acae0 con 0x7f73dc143a10 2022-07-21T01:48:37.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73d3fff700 1 -- 172.21.15.140:0/3316277810 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f73d40459e0 con 0x7f73dc143a10 2022-07-21T01:48:37.502 INFO:teuthology.orchestra.run.smithi140.stdout:55834574853 2022-07-21T01:48:37.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 -- 172.21.15.140:0/3316277810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f73c0053980 msgr2=0x7f73c0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f73c0053980 0x7f73c0055e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f73c400b000 tx=0x7f73c4009040).stop 2022-07-21T01:48:37.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 -- 172.21.15.140:0/3316277810 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73dc143a10 msgr2=0x7f73dc1565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73dc143a10 0x7f73dc1565b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f73d4001110 tx=0x7f73d4007a20).stop 2022-07-21T01:48:37.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 -- 172.21.15.140:0/3316277810 shutdown_connections 2022-07-21T01:48:37.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f73c0053980 0x7f73c0055e40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73dc143a10 0x7f73dc1565b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73dc0b5970 0x7f73dc140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 --2- 172.21.15.140:0/3316277810 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73dc0af8c0 0x7f73dc14b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 -- 172.21.15.140:0/3316277810 >> 172.21.15.140:0/3316277810 conn(0x7f73dc063490 msgr2=0x7f73dc14c900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:37.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 -- 172.21.15.140:0/3316277810 shutdown_connections 2022-07-21T01:48:37.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.495+0000 7f73e0817700 1 -- 172.21.15.140:0/3316277810 wait complete. 2022-07-21T01:48:37.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2022-07-21T01:48:37.514 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2022-07-21T01:48:37.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.623+0000 7f947ee6b700 1 Processor -- start 2022-07-21T01:48:37.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 -- start start 2022-07-21T01:48:37.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f94780b9eb0 0x7f94781471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f94780b52d0 0x7f9478147a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94780b3e00 0x7f9478147f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94781476b0 con 0x7f94780b52d0 2022-07-21T01:48:37.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94780afef0 con 0x7f94780b9eb0 2022-07-21T01:48:37.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947814a7b0 con 0x7f94780b3e00 2022-07-21T01:48:37.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f94780b52d0 0x7f9478147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f94780b52d0 0x7f9478147a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60834/0 (socket says 172.21.15.140:60834) 2022-07-21T01:48:37.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946ffff700 1 -- 172.21.15.140:0/1825884346 learned_addr learned my addr 172.21.15.140:0/1825884346 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:37.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947d408700 1 --2- 172.21.15.140:0/1825884346 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94780b3e00 0x7f9478147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947cc07700 1 --2- 172.21.15.140:0/1825884346 >> v2:172.21.15.152:3300/0 conn(0x7f94780b9eb0 0x7f94781471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946ffff700 1 -- 172.21.15.140:0/1825884346 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94780b3e00 msgr2=0x7f9478147f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:37.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946ffff700 1 --2- 172.21.15.140:0/1825884346 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94780b3e00 0x7f9478147f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946ffff700 1 -- 172.21.15.140:0/1825884346 >> v2:172.21.15.152:3300/0 conn(0x7f94780b9eb0 msgr2=0x7f94781471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:37.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946ffff700 1 --2- 172.21.15.140:0/1825884346 >> v2:172.21.15.152:3300/0 conn(0x7f94780b9eb0 0x7f94781471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946ffff700 1 -- 172.21.15.140:0/1825884346 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9478148420 con 0x7f94780b52d0 2022-07-21T01:48:37.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946ffff700 1 --2- 172.21.15.140:0/1825884346 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94780b52d0 0x7f9478147a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f9468002f70 tx=0x7f9468006eb0).ready entity=mon.0 client_cookie=dbae6a24c0bf0f86 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:37.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946f7fe700 1 -- 172.21.15.140:0/1825884346 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946800d570 con 0x7f94780b52d0 2022-07-21T01:48:37.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946f7fe700 1 -- 172.21.15.140:0/1825884346 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468000e90 con 0x7f94780b52d0 2022-07-21T01:48:37.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f946f7fe700 1 -- 172.21.15.140:0/1825884346 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946800d900 con 0x7f94780b52d0 2022-07-21T01:48:37.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 -- 172.21.15.140:0/1825884346 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94780b52d0 msgr2=0x7f9478147a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 --2- 172.21.15.140:0/1825884346 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94780b52d0 0x7f9478147a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f9468002f70 tx=0x7f9468006eb0).stop 2022-07-21T01:48:37.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 -- 172.21.15.140:0/1825884346 shutdown_connections 2022-07-21T01:48:37.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 --2- 172.21.15.140:0/1825884346 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94780b3e00 0x7f9478147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 --2- 172.21.15.140:0/1825884346 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94780b52d0 0x7f9478147a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 --2- 172.21.15.140:0/1825884346 >> v2:172.21.15.152:3300/0 conn(0x7f94780b9eb0 0x7f94781471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 -- 172.21.15.140:0/1825884346 >> 172.21.15.140:0/1825884346 conn(0x7f9478067980 msgr2=0x7f94780b9460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:37.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 -- 172.21.15.140:0/1825884346 shutdown_connections 2022-07-21T01:48:37.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.627+0000 7f947ee6b700 1 -- 172.21.15.140:0/1825884346 wait complete. 2022-07-21T01:48:37.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f947ee6b700 1 Processor -- start 2022-07-21T01:48:37.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f947ee6b700 1 -- start start 2022-07-21T01:48:37.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f947ee6b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94780b3e00 0x7f9478145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f947ee6b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94780b52d0 0x7f94780ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f947ee6b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94780b9eb0 0x7f9478159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f947ee6b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9478063110 con 0x7f94780b52d0 2022-07-21T01:48:37.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f947ee6b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947814abd0 con 0x7f94780b9eb0 2022-07-21T01:48:37.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f947ee6b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947814ae70 con 0x7f94780b3e00 2022-07-21T01:48:37.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f946ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94780b52d0 0x7f94780ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f947cc07700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94780b3e00 0x7f9478145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f946ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94780b52d0 0x7f94780ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60840/0 (socket says 172.21.15.140:60840) 2022-07-21T01:48:37.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f947cc07700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94780b3e00 0x7f9478145440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:57568/0 (socket says 172.21.15.140:57568) 2022-07-21T01:48:37.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f946ffff700 1 -- 172.21.15.140:0/123781385 learned_addr learned my addr 172.21.15.140:0/123781385 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:37.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f947d408700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94780b9eb0 0x7f9478159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f946ffff700 1 -- 172.21.15.140:0/123781385 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94780b3e00 msgr2=0x7f9478145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f946ffff700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94780b3e00 0x7f9478145440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f946ffff700 1 -- 172.21.15.140:0/123781385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94780b9eb0 msgr2=0x7f9478159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f946ffff700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94780b9eb0 0x7f9478159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f946ffff700 1 -- 172.21.15.140:0/123781385 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946800b1e0 con 0x7f94780b52d0 2022-07-21T01:48:37.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.631+0000 7f946ffff700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94780b52d0 0x7f94780ad130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f9468002f70 tx=0x7f9468015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:37.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.635+0000 7f946dffb700 1 -- 172.21.15.140:0/123781385 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9468011610 con 0x7f94780b52d0 2022-07-21T01:48:37.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.635+0000 7f946dffb700 1 -- 172.21.15.140:0/123781385 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468011770 con 0x7f94780b52d0 2022-07-21T01:48:37.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.635+0000 7f947ee6b700 1 -- 172.21.15.140:0/123781385 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9478062d60 con 0x7f94780b52d0 2022-07-21T01:48:37.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.635+0000 7f946dffb700 1 -- 172.21.15.140:0/123781385 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9468011ad0 con 0x7f94780b52d0 2022-07-21T01:48:37.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.635+0000 7f947ee6b700 1 -- 172.21.15.140:0/123781385 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947814b710 con 0x7f94780b52d0 2022-07-21T01:48:37.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.635+0000 7f947ee6b700 1 -- 172.21.15.140:0/123781385 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94780b0140 con 0x7f94780b52d0 2022-07-21T01:48:37.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.635+0000 7f946dffb700 1 -- 172.21.15.140:0/123781385 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9468025030 con 0x7f94780b52d0 2022-07-21T01:48:37.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.635+0000 7f946dffb700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9458053b50 0x7f9458056010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:37.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.635+0000 7f946dffb700 1 -- 172.21.15.140:0/123781385 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f946806cf20 con 0x7f94780b52d0 2022-07-21T01:48:37.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.635+0000 7f947cc07700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9458053b50 0x7f9458056010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:37.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.635+0000 7f947cc07700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9458053b50 0x7f9458056010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f947400c000 tx=0x7f9474008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:37.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.643+0000 7f946dffb700 1 -- 172.21.15.140:0/123781385 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9468043e40 con 0x7f94780b52d0 2022-07-21T01:48:37.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.771+0000 7f947ee6b700 1 -- 172.21.15.140:0/123781385 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f947814b710 con 0x7f94780b52d0 2022-07-21T01:48:37.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.771+0000 7f946dffb700 1 -- 172.21.15.140:0/123781385 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f947814b710 con 0x7f94780b52d0 2022-07-21T01:48:37.778 INFO:teuthology.orchestra.run.smithi140.stdout:55834574853 2022-07-21T01:48:37.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.771+0000 7f947ee6b700 1 -- 172.21.15.140:0/123781385 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9458053b50 msgr2=0x7f9458056010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.771+0000 7f947ee6b700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9458053b50 0x7f9458056010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f947400c000 tx=0x7f9474008040).stop 2022-07-21T01:48:37.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.771+0000 7f947ee6b700 1 -- 172.21.15.140:0/123781385 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94780b52d0 msgr2=0x7f94780ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:37.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.771+0000 7f947ee6b700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94780b52d0 0x7f94780ad130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f9468002f70 tx=0x7f9468015000).stop 2022-07-21T01:48:37.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.775+0000 7f947ee6b700 1 -- 172.21.15.140:0/123781385 shutdown_connections 2022-07-21T01:48:37.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.775+0000 7f947ee6b700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9458053b50 0x7f9458056010 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.775+0000 7f947ee6b700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94780b9eb0 0x7f9478159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.775+0000 7f947ee6b700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94780b52d0 0x7f94780ad130 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.775+0000 7f947ee6b700 1 --2- 172.21.15.140:0/123781385 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94780b3e00 0x7f9478145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:37.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.775+0000 7f947ee6b700 1 -- 172.21.15.140:0/123781385 >> 172.21.15.140:0/123781385 conn(0x7f9478067980 msgr2=0x7f94781463d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:37.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.775+0000 7f947ee6b700 1 -- 172.21.15.140:0/123781385 shutdown_connections 2022-07-21T01:48:37.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:37.775+0000 7f947ee6b700 1 -- 172.21.15.140:0/123781385 wait complete. 2022-07-21T01:48:37.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2022-07-21T01:48:38.795 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2022-07-21T01:48:38.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb564a4e700 1 Processor -- start 2022-07-21T01:48:38.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb564a4e700 1 -- start start 2022-07-21T01:48:38.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb564a4e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb5600b3cb0 0x7fb5600aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:38.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb564a4e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb5600ad0e0 0x7fb5601418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:38.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb564a4e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5600ae280 con 0x7fb5600b6800 2022-07-21T01:48:38.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb564a4e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5600b6be0 con 0x7fb5600ad0e0 2022-07-21T01:48:38.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb564a4e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600ae4b0 con 0x7fb5600b3cb0 2022-07-21T01:48:38.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55effd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb5600b3cb0 0x7fb5600aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:38.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb5600ad0e0 0x7fb5601418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:38.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55f7fe700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb5600b6800 0x7fb560145f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:47874/0 (socket says 172.21.15.140:47874) 2022-07-21T01:48:38.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55f7fe700 1 -- 172.21.15.140:0/1660809042 learned_addr learned my addr 172.21.15.140:0/1660809042 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:38.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55ffff700 1 -- 172.21.15.140:0/1660809042 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb5600b3cb0 msgr2=0x7fb5600aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:38.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55ffff700 1 --2- 172.21.15.140:0/1660809042 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb5600b3cb0 0x7fb5600aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:38.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55ffff700 1 -- 172.21.15.140:0/1660809042 >> v1:172.21.15.152:6789/0 conn(0x7fb5600b6800 legacy=0x7fb560145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:38.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55e7fc700 1 -- 172.21.15.140:0/1660809042 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915899877 0 0) 0x7fb5600ae280 con 0x7fb5600b6800 2022-07-21T01:48:38.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55ffff700 1 -- 172.21.15.140:0/1660809042 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5600ae750 con 0x7fb5600ad0e0 2022-07-21T01:48:38.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55ffff700 1 --2- 172.21.15.140:0/1660809042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600ad0e0 0x7fb5601418f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb550002f70 tx=0x7fb550006df0).ready entity=mon.1 client_cookie=3260a4550407b68b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:38.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55e7fc700 1 -- 172.21.15.140:0/1660809042 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb55000d570 con 0x7fb5600ad0e0 2022-07-21T01:48:38.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55e7fc700 1 -- 172.21.15.140:0/1660809042 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb550000e90 con 0x7fb5600ad0e0 2022-07-21T01:48:38.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.911+0000 7fb55e7fc700 1 -- 172.21.15.140:0/1660809042 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb55000d8c0 con 0x7fb5600ad0e0 2022-07-21T01:48:38.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 -- 172.21.15.140:0/1660809042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600ad0e0 msgr2=0x7fb5601418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:38.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 --2- 172.21.15.140:0/1660809042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600ad0e0 0x7fb5601418f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb550002f70 tx=0x7fb550006df0).stop 2022-07-21T01:48:38.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 -- 172.21.15.140:0/1660809042 shutdown_connections 2022-07-21T01:48:38.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 --2- 172.21.15.140:0/1660809042 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb5600b3cb0 0x7fb5600aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:38.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 --2- 172.21.15.140:0/1660809042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600ad0e0 0x7fb5601418f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:38.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 -- 172.21.15.140:0/1660809042 >> 172.21.15.140:0/1660809042 conn(0x7fb5600b9dd0 msgr2=0x7fb5600b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:38.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 -- 172.21.15.140:0/1660809042 shutdown_connections 2022-07-21T01:48:38.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 -- 172.21.15.140:0/1660809042 wait complete. 2022-07-21T01:48:38.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 Processor -- start 2022-07-21T01:48:38.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 -- start start 2022-07-21T01:48:38.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb5600ad0e0 0x7fb5600b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:38.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600b3cb0 0x7fb56013d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:38.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb5600b6800 0x7fb56013be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:38.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600b7570 con 0x7fb5600b6800 2022-07-21T01:48:38.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600b8c70 con 0x7fb5600b3cb0 2022-07-21T01:48:38.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb564a4e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600b8f10 con 0x7fb5600ad0e0 2022-07-21T01:48:38.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb5600b6800 0x7fb56013be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:38.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb5600b6800 0x7fb56013be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60854/0 (socket says 172.21.15.140:60854) 2022-07-21T01:48:38.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55ffff700 1 -- 172.21.15.140:0/2186178105 learned_addr learned my addr 172.21.15.140:0/2186178105 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:38.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55f7fe700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb5600ad0e0 0x7fb5600b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:38.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55effd700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600b3cb0 0x7fb56013d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:38.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55ffff700 1 -- 172.21.15.140:0/2186178105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb5600ad0e0 msgr2=0x7fb5600b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:38.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55ffff700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb5600ad0e0 0x7fb5600b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:38.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55ffff700 1 -- 172.21.15.140:0/2186178105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600b3cb0 msgr2=0x7fb56013d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:38.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55ffff700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600b3cb0 0x7fb56013d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:38.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55ffff700 1 -- 172.21.15.140:0/2186178105 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55000b180 con 0x7fb5600b6800 2022-07-21T01:48:38.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55ffff700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb5600b6800 0x7fb56013be30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fb550002f70 tx=0x7fb550014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:38.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55cff9700 1 -- 172.21.15.140:0/2186178105 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb550013ac0 con 0x7fb5600b6800 2022-07-21T01:48:38.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.915+0000 7fb55cff9700 1 -- 172.21.15.140:0/2186178105 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb550013c20 con 0x7fb5600b6800 2022-07-21T01:48:38.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.919+0000 7fb564a4e700 1 -- 172.21.15.140:0/2186178105 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5600b7790 con 0x7fb5600b6800 2022-07-21T01:48:38.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.919+0000 7fb55cff9700 1 -- 172.21.15.140:0/2186178105 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb55001e400 con 0x7fb5600b6800 2022-07-21T01:48:38.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.919+0000 7fb564a4e700 1 -- 172.21.15.140:0/2186178105 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb560140380 con 0x7fb5600b6800 2022-07-21T01:48:38.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.919+0000 7fb564a4e700 1 -- 172.21.15.140:0/2186178105 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5600b8ac0 con 0x7fb5600b6800 2022-07-21T01:48:38.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.919+0000 7fb55cff9700 1 -- 172.21.15.140:0/2186178105 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb550025030 con 0x7fb5600b6800 2022-07-21T01:48:38.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.919+0000 7fb55cff9700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb538053b00 0x7fb538055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:38.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.919+0000 7fb55cff9700 1 -- 172.21.15.140:0/2186178105 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb55006cda0 con 0x7fb5600b6800 2022-07-21T01:48:38.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.919+0000 7fb55f7fe700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb538053b00 0x7fb538055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:38.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.919+0000 7fb55f7fe700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb538053b00 0x7fb538055fc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb55800d000 tx=0x7fb558001360).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:38.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:38.923+0000 7fb55cff9700 1 -- 172.21.15.140:0/2186178105 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb550043cc0 con 0x7fb5600b6800 2022-07-21T01:48:39.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 -- 172.21.15.140:0/2186178105 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb5600b5530 con 0x7fb5600b6800 2022-07-21T01:48:39.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb55cff9700 1 -- 172.21.15.140:0/2186178105 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb550023070 con 0x7fb5600b6800 2022-07-21T01:48:39.052 INFO:teuthology.orchestra.run.smithi140.stdout:55834574855 2022-07-21T01:48:39.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 -- 172.21.15.140:0/2186178105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb538053b00 msgr2=0x7fb538055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb538053b00 0x7fb538055fc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb55800d000 tx=0x7fb558001360).stop 2022-07-21T01:48:39.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 -- 172.21.15.140:0/2186178105 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb5600b6800 msgr2=0x7fb56013be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb5600b6800 0x7fb56013be30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fb550002f70 tx=0x7fb550014000).stop 2022-07-21T01:48:39.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 -- 172.21.15.140:0/2186178105 shutdown_connections 2022-07-21T01:48:39.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb538053b00 0x7fb538055fc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb5600b6800 0x7fb56013be30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5600b3cb0 0x7fb56013d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 --2- 172.21.15.140:0/2186178105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb5600ad0e0 0x7fb5600b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 -- 172.21.15.140:0/2186178105 >> 172.21.15.140:0/2186178105 conn(0x7fb5600b9dd0 msgr2=0x7fb560141620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:39.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 -- 172.21.15.140:0/2186178105 shutdown_connections 2022-07-21T01:48:39.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.051+0000 7fb564a4e700 1 -- 172.21.15.140:0/2186178105 wait complete. 2022-07-21T01:48:39.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.4 2022-07-21T01:48:39.064 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2022-07-21T01:48:39.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 Processor -- start 2022-07-21T01:48:39.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 -- start start 2022-07-21T01:48:39.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29300b5970 0x7f29300ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f29300af8c0 0x7f29300ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29300b0b80 con 0x7f29300aeae0 2022-07-21T01:48:39.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f293005d4d0 con 0x7f29300af8c0 2022-07-21T01:48:39.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29300b5d50 con 0x7f29300b5970 2022-07-21T01:48:39.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f2925d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f29300af8c0 0x7f29300ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f2925d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f29300af8c0 0x7f29300ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60860/0 (socket says 172.21.15.140:60860) 2022-07-21T01:48:39.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f2925d9b700 1 -- 172.21.15.140:0/3397950823 learned_addr learned my addr 172.21.15.140:0/3397950823 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:39.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292e59c700 1 --2- 172.21.15.140:0/3397950823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29300b5970 0x7f29300ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f2925d9b700 1 -- 172.21.15.140:0/3397950823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29300b5970 msgr2=0x7f29300ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:39.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f2925d9b700 1 --2- 172.21.15.140:0/3397950823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29300b5970 0x7f29300ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f2925d9b700 1 -- 172.21.15.140:0/3397950823 >> v1:172.21.15.152:6789/0 conn(0x7f29300aeae0 legacy=0x7f293005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f2925d9b700 1 -- 172.21.15.140:0/3397950823 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29300b0db0 con 0x7f29300af8c0 2022-07-21T01:48:39.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292dd9b700 1 -- 172.21.15.140:0/3397950823 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157600527 0 0) 0x7f29300b0b80 con 0x7f29300aeae0 2022-07-21T01:48:39.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f2925d9b700 1 --2- 172.21.15.140:0/3397950823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29300af8c0 0x7f29300ae5f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f2918002f70 tx=0x7f2918006eb0).ready entity=mon.0 client_cookie=eb56aaea60872ad2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292dd9b700 1 -- 172.21.15.140:0/3397950823 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f291800d570 con 0x7f29300af8c0 2022-07-21T01:48:39.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292dd9b700 1 -- 172.21.15.140:0/3397950823 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2918000e90 con 0x7f29300af8c0 2022-07-21T01:48:39.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292dd9b700 1 -- 172.21.15.140:0/3397950823 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f291800d8c0 con 0x7f29300af8c0 2022-07-21T01:48:39.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 -- 172.21.15.140:0/3397950823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29300af8c0 msgr2=0x7f29300ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 --2- 172.21.15.140:0/3397950823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29300af8c0 0x7f29300ae5f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f2918002f70 tx=0x7f2918006eb0).stop 2022-07-21T01:48:39.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 -- 172.21.15.140:0/3397950823 shutdown_connections 2022-07-21T01:48:39.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 --2- 172.21.15.140:0/3397950823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29300af8c0 0x7f29300ae5f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 --2- 172.21.15.140:0/3397950823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29300b5970 0x7f29300ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 -- 172.21.15.140:0/3397950823 >> 172.21.15.140:0/3397950823 conn(0x7f2930063490 msgr2=0x7f29300b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:39.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 -- 172.21.15.140:0/3397950823 shutdown_connections 2022-07-21T01:48:39.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.187+0000 7f292f59e700 1 -- 172.21.15.140:0/3397950823 wait complete. 2022-07-21T01:48:39.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f292f59e700 1 Processor -- start 2022-07-21T01:48:39.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f292f59e700 1 -- start start 2022-07-21T01:48:39.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f292f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29300aeae0 0x7f2930140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f292f59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29300af8c0 0x7f293013f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f292f59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29300b5970 0x7f293013bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f292f59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2930140fa0 con 0x7f29300af8c0 2022-07-21T01:48:39.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f292f59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2930143120 con 0x7f29300b5970 2022-07-21T01:48:39.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f292f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2930140370 con 0x7f29300aeae0 2022-07-21T01:48:39.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f2925d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29300af8c0 0x7f293013f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f2925d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29300af8c0 0x7f293013f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60868/0 (socket says 172.21.15.140:60868) 2022-07-21T01:48:39.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f2925d9b700 1 -- 172.21.15.140:0/2419260742 learned_addr learned my addr 172.21.15.140:0/2419260742 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:39.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f292ed9d700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29300b5970 0x7f293013bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.191+0000 7f292e59c700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29300aeae0 0x7f2930140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2925d9b700 1 -- 172.21.15.140:0/2419260742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29300aeae0 msgr2=0x7f2930140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2925d9b700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29300aeae0 0x7f2930140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2925d9b700 1 -- 172.21.15.140:0/2419260742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29300b5970 msgr2=0x7f293013bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2925d9b700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29300b5970 0x7f293013bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2925d9b700 1 -- 172.21.15.140:0/2419260742 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291800b1e0 con 0x7f29300af8c0 2022-07-21T01:48:39.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f292ed9d700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29300b5970 0x7f293013bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:39.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2925d9b700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29300af8c0 0x7f293013f830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f2918002f70 tx=0x7f2918015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2927fff700 1 -- 172.21.15.140:0/2419260742 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2918011990 con 0x7f29300af8c0 2022-07-21T01:48:39.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2927fff700 1 -- 172.21.15.140:0/2419260742 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2918011af0 con 0x7f29300af8c0 2022-07-21T01:48:39.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f292f59e700 1 -- 172.21.15.140:0/2419260742 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2930140610 con 0x7f29300af8c0 2022-07-21T01:48:39.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2927fff700 1 -- 172.21.15.140:0/2419260742 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2918011e50 con 0x7f29300af8c0 2022-07-21T01:48:39.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f292f59e700 1 -- 172.21.15.140:0/2419260742 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2930142de0 con 0x7f29300af8c0 2022-07-21T01:48:39.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f292f59e700 1 -- 172.21.15.140:0/2419260742 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29300acae0 con 0x7f29300af8c0 2022-07-21T01:48:39.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2927fff700 1 -- 172.21.15.140:0/2419260742 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2918025030 con 0x7f29300af8c0 2022-07-21T01:48:39.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2927fff700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2914053b00 0x7f2914055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f2927fff700 1 -- 172.21.15.140:0/2419260742 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f291806d180 con 0x7f29300af8c0 2022-07-21T01:48:39.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f292e59c700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2914053b00 0x7f2914055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.195+0000 7f292e59c700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2914053b00 0x7f2914055fc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f292000b000 tx=0x7f2920008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.203+0000 7f2927fff700 1 -- 172.21.15.140:0/2419260742 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f29180440a0 con 0x7f29300af8c0 2022-07-21T01:48:39.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.323+0000 7f292f59e700 1 -- 172.21.15.140:0/2419260742 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f29300574e0 con 0x7f29300af8c0 2022-07-21T01:48:39.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.323+0000 7f2927fff700 1 -- 172.21.15.140:0/2419260742 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2918023090 con 0x7f29300af8c0 2022-07-21T01:48:39.327 INFO:teuthology.orchestra.run.smithi140.stdout:55834574854 2022-07-21T01:48:39.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.323+0000 7f292f59e700 1 -- 172.21.15.140:0/2419260742 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2914053b00 msgr2=0x7f2914055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.323+0000 7f292f59e700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2914053b00 0x7f2914055fc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f292000b000 tx=0x7f2920008040).stop 2022-07-21T01:48:39.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.323+0000 7f292f59e700 1 -- 172.21.15.140:0/2419260742 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29300af8c0 msgr2=0x7f293013f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.323+0000 7f292f59e700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29300af8c0 0x7f293013f830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f2918002f70 tx=0x7f2918015000).stop 2022-07-21T01:48:39.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.323+0000 7f292f59e700 1 -- 172.21.15.140:0/2419260742 shutdown_connections 2022-07-21T01:48:39.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.323+0000 7f292f59e700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2914053b00 0x7f2914055fc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.327+0000 7f292f59e700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29300b5970 0x7f293013bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.327+0000 7f292f59e700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29300af8c0 0x7f293013f830 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.327+0000 7f292f59e700 1 --2- 172.21.15.140:0/2419260742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29300aeae0 0x7f2930140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.327+0000 7f292f59e700 1 -- 172.21.15.140:0/2419260742 >> 172.21.15.140:0/2419260742 conn(0x7f2930063490 msgr2=0x7f293014c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:39.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.327+0000 7f292f59e700 1 -- 172.21.15.140:0/2419260742 shutdown_connections 2022-07-21T01:48:39.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.327+0000 7f292f59e700 1 -- 172.21.15.140:0/2419260742 wait complete. 2022-07-21T01:48:39.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.5 2022-07-21T01:48:39.338 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2022-07-21T01:48:39.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b87042700 1 Processor -- start 2022-07-21T01:48:39.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b87042700 1 -- start start 2022-07-21T01:48:39.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b87042700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1b8005c7c0 0x7f1b800b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b87042700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b800b0900 con 0x7f1b801430b0 2022-07-21T01:48:39.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b87042700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b801470e0 con 0x7f1b800ab250 2022-07-21T01:48:39.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b87042700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b800b2a30 con 0x7f1b8005c7c0 2022-07-21T01:48:39.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b86040700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1b8005c7c0 0x7f1b800b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b86040700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1b8005c7c0 0x7f1b800b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60874/0 (socket says 172.21.15.140:60874) 2022-07-21T01:48:39.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b86841700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1b801430b0 0x7f1b801469d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46790/0 (socket says 172.21.15.140:46790) 2022-07-21T01:48:39.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b86040700 1 -- 172.21.15.140:0/4142432163 learned_addr learned my addr 172.21.15.140:0/4142432163 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:39.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b86040700 1 -- 172.21.15.140:0/4142432163 >> v1:172.21.15.152:6789/0 conn(0x7f1b800ab250 legacy=0x7f1b801462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b86040700 1 -- 172.21.15.140:0/4142432163 >> v1:172.21.15.140:6789/0 conn(0x7f1b801430b0 legacy=0x7f1b801469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b86040700 1 -- 172.21.15.140:0/4142432163 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80094c50 con 0x7f1b8005c7c0 2022-07-21T01:48:39.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b86040700 1 --2- 172.21.15.140:0/4142432163 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b8005c7c0 0x7f1b800b1390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f1b680032f0 tx=0x7f1b68009290).ready entity=mon.0 client_cookie=475406249d7088d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b8503e700 1 -- 172.21.15.140:0/4142432163 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b6800b8a0 con 0x7f1b8005c7c0 2022-07-21T01:48:39.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b8503e700 1 -- 172.21.15.140:0/4142432163 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6800ba00 con 0x7f1b8005c7c0 2022-07-21T01:48:39.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b8503e700 1 -- 172.21.15.140:0/4142432163 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b6800bb70 con 0x7f1b8005c7c0 2022-07-21T01:48:39.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b87042700 1 -- 172.21.15.140:0/4142432163 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b8005c7c0 msgr2=0x7f1b800b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b87042700 1 --2- 172.21.15.140:0/4142432163 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b8005c7c0 0x7f1b800b1390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f1b680032f0 tx=0x7f1b68009290).stop 2022-07-21T01:48:39.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.455+0000 7f1b87042700 1 -- 172.21.15.140:0/4142432163 shutdown_connections 2022-07-21T01:48:39.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 --2- 172.21.15.140:0/4142432163 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b8005c7c0 0x7f1b800b1390 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 -- 172.21.15.140:0/4142432163 >> 172.21.15.140:0/4142432163 conn(0x7f1b800b6210 msgr2=0x7f1b800b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:39.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 -- 172.21.15.140:0/4142432163 shutdown_connections 2022-07-21T01:48:39.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 -- 172.21.15.140:0/4142432163 wait complete. 2022-07-21T01:48:39.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 Processor -- start 2022-07-21T01:48:39.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 -- start start 2022-07-21T01:48:39.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8005c7c0 0x7f1b80060390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b800ab250 0x7f1b800af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b801430b0 0x7f1b800afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b80098a40 con 0x7f1b800ab250 2022-07-21T01:48:39.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800af400 con 0x7f1b8005c7c0 2022-07-21T01:48:39.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800af160 con 0x7f1b801430b0 2022-07-21T01:48:39.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b8583f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b800ab250 0x7f1b800af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b8583f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b800ab250 0x7f1b800af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60882/0 (socket says 172.21.15.140:60882) 2022-07-21T01:48:39.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b8583f700 1 -- 172.21.15.140:0/1111876237 learned_addr learned my addr 172.21.15.140:0/1111876237 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:39.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b86040700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8005c7c0 0x7f1b80060390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b86841700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b801430b0 0x7f1b800afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b8583f700 1 -- 172.21.15.140:0/1111876237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b801430b0 msgr2=0x7f1b800afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b8583f700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b801430b0 0x7f1b800afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b8583f700 1 -- 172.21.15.140:0/1111876237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8005c7c0 msgr2=0x7f1b80060390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b8583f700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8005c7c0 0x7f1b80060390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b8583f700 1 -- 172.21.15.140:0/1111876237 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b68009530 con 0x7f1b800ab250 2022-07-21T01:48:39.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b8583f700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b800ab250 0x7f1b800af920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f1b8005d080 tx=0x7f1b70007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b7f7fe700 1 -- 172.21.15.140:0/1111876237 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b70012400 con 0x7f1b800ab250 2022-07-21T01:48:39.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b7f7fe700 1 -- 172.21.15.140:0/1111876237 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b70012560 con 0x7f1b800ab250 2022-07-21T01:48:39.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 -- 172.21.15.140:0/1111876237 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b80096f20 con 0x7f1b800ab250 2022-07-21T01:48:39.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b7f7fe700 1 -- 172.21.15.140:0/1111876237 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b700128c0 con 0x7f1b800ab250 2022-07-21T01:48:39.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.459+0000 7f1b87042700 1 -- 172.21.15.140:0/1111876237 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b80155b30 con 0x7f1b800ab250 2022-07-21T01:48:39.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.463+0000 7f1b87042700 1 -- 172.21.15.140:0/1111876237 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b800970b0 con 0x7f1b800ab250 2022-07-21T01:48:39.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.463+0000 7f1b7f7fe700 1 -- 172.21.15.140:0/1111876237 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1b70019030 con 0x7f1b800ab250 2022-07-21T01:48:39.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.463+0000 7f1b7f7fe700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b64053b00 0x7f1b64055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.463+0000 7f1b7f7fe700 1 -- 172.21.15.140:0/1111876237 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b7006b270 con 0x7f1b800ab250 2022-07-21T01:48:39.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.463+0000 7f1b86040700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b64053b00 0x7f1b64055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.463+0000 7f1b86040700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b64053b00 0x7f1b64055fc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f1b6801b000 tx=0x7f1b68016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.471+0000 7f1b7f7fe700 1 -- 172.21.15.140:0/1111876237 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1b70042190 con 0x7f1b800ab250 2022-07-21T01:48:39.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b87042700 1 -- 172.21.15.140:0/1111876237 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1b800b0c50 con 0x7f1b800ab250 2022-07-21T01:48:39.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b7f7fe700 1 -- 172.21.15.140:0/1111876237 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b700457b0 con 0x7f1b800ab250 2022-07-21T01:48:39.594 INFO:teuthology.orchestra.run.smithi140.stdout:55834574855 2022-07-21T01:48:39.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b87042700 1 -- 172.21.15.140:0/1111876237 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b64053b00 msgr2=0x7f1b64055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b87042700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b64053b00 0x7f1b64055fc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f1b6801b000 tx=0x7f1b68016000).stop 2022-07-21T01:48:39.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b87042700 1 -- 172.21.15.140:0/1111876237 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b800ab250 msgr2=0x7f1b800af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b87042700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b800ab250 0x7f1b800af920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f1b8005d080 tx=0x7f1b70007c50).stop 2022-07-21T01:48:39.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b87042700 1 -- 172.21.15.140:0/1111876237 shutdown_connections 2022-07-21T01:48:39.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b87042700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b64053b00 0x7f1b64055fc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b87042700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b801430b0 0x7f1b800afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b87042700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b800ab250 0x7f1b800af920 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b87042700 1 --2- 172.21.15.140:0/1111876237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b8005c7c0 0x7f1b80060390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.595+0000 7f1b87042700 1 -- 172.21.15.140:0/1111876237 >> 172.21.15.140:0/1111876237 conn(0x7f1b800b6210 msgr2=0x7f1b800ab7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:39.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.599+0000 7f1b87042700 1 -- 172.21.15.140:0/1111876237 shutdown_connections 2022-07-21T01:48:39.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.599+0000 7f1b87042700 1 -- 172.21.15.140:0/1111876237 wait complete. 2022-07-21T01:48:39.612 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.6 2022-07-21T01:48:39.613 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2022-07-21T01:48:39.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 Processor -- start 2022-07-21T01:48:39.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 -- start start 2022-07-21T01:48:39.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0b6800 0x7f7b9c0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7b9c0b3cb0 0x7f7b9c145f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c0ae280 con 0x7f7b9c0ad0e0 2022-07-21T01:48:39.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9c0b6be0 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9c0ae4b0 con 0x7f7b9c0b6800 2022-07-21T01:48:39.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba17a6700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7b9c0b3cb0 0x7f7b9c145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba17a6700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7b9c0b3cb0 0x7f7b9c145f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60888/0 (socket says 172.21.15.140:60888) 2022-07-21T01:48:39.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba17a6700 1 -- 172.21.15.140:0/3373888673 learned_addr learned my addr 172.21.15.140:0/3373888673 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:39.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba1fa7700 1 --2- 172.21.15.140:0/3373888673 >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0b6800 0x7f7b9c0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba0fa5700 1 -- 172.21.15.140:0/3373888673 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843021221 0 0) 0x7f7b9c0ae280 con 0x7f7b9c0ad0e0 2022-07-21T01:48:39.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba17a6700 1 -- 172.21.15.140:0/3373888673 >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0b6800 msgr2=0x7f7b9c0aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:39.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba17a6700 1 --2- 172.21.15.140:0/3373888673 >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0b6800 0x7f7b9c0aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba17a6700 1 -- 172.21.15.140:0/3373888673 >> v1:172.21.15.152:6789/0 conn(0x7f7b9c0ad0e0 legacy=0x7f7b9c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba17a6700 1 -- 172.21.15.140:0/3373888673 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c0ae750 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba17a6700 1 --2- 172.21.15.140:0/3373888673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0b3cb0 0x7f7b9c145f40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f7b8c002f70 tx=0x7f7b8c006eb0).ready entity=mon.0 client_cookie=8e11bad023bacd65 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba0fa5700 1 -- 172.21.15.140:0/3373888673 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8c00d5c0 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba0fa5700 1 -- 172.21.15.140:0/3373888673 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c000e90 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba0fa5700 1 -- 172.21.15.140:0/3373888673 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8c00d910 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/3373888673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0b3cb0 msgr2=0x7f7b9c145f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 --2- 172.21.15.140:0/3373888673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0b3cb0 0x7f7b9c145f40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f7b8c002f70 tx=0x7f7b8c006eb0).stop 2022-07-21T01:48:39.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/3373888673 shutdown_connections 2022-07-21T01:48:39.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 --2- 172.21.15.140:0/3373888673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0b3cb0 0x7f7b9c145f40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 --2- 172.21.15.140:0/3373888673 >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0b6800 0x7f7b9c0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/3373888673 >> 172.21.15.140:0/3373888673 conn(0x7f7b9c0b9dd0 msgr2=0x7f7b9c0b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:39.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/3373888673 shutdown_connections 2022-07-21T01:48:39.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.723+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/3373888673 wait complete. 2022-07-21T01:48:39.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba2fa9700 1 Processor -- start 2022-07-21T01:48:39.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba2fa9700 1 -- start start 2022-07-21T01:48:39.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba2fa9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c0ad0e0 0x7f7b9c0b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba2fa9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0b3cb0 0x7f7b9c142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba2fa9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c0b6800 0x7f7b9c151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba2fa9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c13de50 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba2fa9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c0b1020 con 0x7f7b9c0b6800 2022-07-21T01:48:39.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba2fa9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c0b12c0 con 0x7f7b9c0ad0e0 2022-07-21T01:48:39.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba17a6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0b3cb0 0x7f7b9c142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba17a6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0b3cb0 0x7f7b9c142990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60894/0 (socket says 172.21.15.140:60894) 2022-07-21T01:48:39.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba17a6700 1 -- 172.21.15.140:0/1600034751 learned_addr learned my addr 172.21.15.140:0/1600034751 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:39.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba1fa7700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c0ad0e0 0x7f7b9c0b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba27a8700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c0b6800 0x7f7b9c151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba17a6700 1 -- 172.21.15.140:0/1600034751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c0ad0e0 msgr2=0x7f7b9c0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba17a6700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c0ad0e0 0x7f7b9c0b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba17a6700 1 -- 172.21.15.140:0/1600034751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c0b6800 msgr2=0x7f7b9c151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba17a6700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c0b6800 0x7f7b9c151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba17a6700 1 -- 172.21.15.140:0/1600034751 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c00b1e0 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba17a6700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0b3cb0 0x7f7b9c142990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f7b8c002f70 tx=0x7f7b8c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7b8affd700 1 -- 172.21.15.140:0/1600034751 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8c011760 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7b8affd700 1 -- 172.21.15.140:0/1600034751 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c0118c0 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/1600034751 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c0b1560 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7b8affd700 1 -- 172.21.15.140:0/1600034751 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8c011c20 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.727+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/1600034751 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c1430a0 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.731+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/1600034751 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c147c30 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.731+0000 7f7b8affd700 1 -- 172.21.15.140:0/1600034751 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7b8c025030 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.731+0000 7f7b8affd700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b7c053b30 0x7f7b7c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.731+0000 7f7b8affd700 1 -- 172.21.15.140:0/1600034751 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7b8c06d1e0 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.731+0000 7f7ba1fa7700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b7c053b30 0x7f7b7c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.731+0000 7f7ba1fa7700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b7c053b30 0x7f7b7c055ff0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f7b9800b000 tx=0x7f7b98006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.735+0000 7f7b8affd700 1 -- 172.21.15.140:0/1600034751 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7b8c044100 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.855+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/1600034751 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7b9c0af0e0 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.855+0000 7f7b8affd700 1 -- 172.21.15.140:0/1600034751 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b9c0af0e0 con 0x7f7b9c0b3cb0 2022-07-21T01:48:39.856 INFO:teuthology.orchestra.run.smithi140.stdout:55834574853 2022-07-21T01:48:39.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/1600034751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b7c053b30 msgr2=0x7f7b7c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b7c053b30 0x7f7b7c055ff0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f7b9800b000 tx=0x7f7b98006df0).stop 2022-07-21T01:48:39.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/1600034751 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0b3cb0 msgr2=0x7f7b9c142990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0b3cb0 0x7f7b9c142990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f7b8c002f70 tx=0x7f7b8c015000).stop 2022-07-21T01:48:39.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/1600034751 shutdown_connections 2022-07-21T01:48:39.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b7c053b30 0x7f7b7c055ff0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c0b6800 0x7f7b9c151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0b3cb0 0x7f7b9c142990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 --2- 172.21.15.140:0/1600034751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c0ad0e0 0x7f7b9c0b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/1600034751 >> 172.21.15.140:0/1600034751 conn(0x7f7b9c0b9dd0 msgr2=0x7f7b9c0b8520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:39.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/1600034751 shutdown_connections 2022-07-21T01:48:39.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.859+0000 7f7ba2fa9700 1 -- 172.21.15.140:0/1600034751 wait complete. 2022-07-21T01:48:39.869 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.7 2022-07-21T01:48:39.870 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2022-07-21T01:48:39.870 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-07-21T01:48:39.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f5113325700 1 Processor -- start 2022-07-21T01:48:39.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f5113325700 1 -- start start 2022-07-21T01:48:39.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f5113325700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0b3620 0x7f510c0ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f5113325700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f510c096fa0 con 0x7f510c141200 2022-07-21T01:48:39.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f5113325700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f510c0b22d0 con 0x7f510c0ac260 2022-07-21T01:48:39.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f5113325700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c0b26d0 con 0x7f510c0b3620 2022-07-21T01:48:39.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f51118c2700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f510c141200 0x7f510c1400c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46816/0 (socket says 172.21.15.140:46816) 2022-07-21T01:48:39.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f51118c2700 1 -- 172.21.15.140:0/3656203923 learned_addr learned my addr 172.21.15.140:0/3656203923 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:39.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f51108c0700 1 --2- 172.21.15.140:0/3656203923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0b3620 0x7f510c0ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f5103fff700 1 -- 172.21.15.140:0/3656203923 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539857906 0 0) 0x7f510c096fa0 con 0x7f510c141200 2022-07-21T01:48:39.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f5103fff700 1 -- 172.21.15.140:0/3656203923 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50f0003260 con 0x7f510c141200 2022-07-21T01:48:39.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f5103fff700 1 -- 172.21.15.140:0/3656203923 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626413327 0 0) 0x7f510c0b22d0 con 0x7f510c0ac260 2022-07-21T01:48:39.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f5103fff700 1 -- 172.21.15.140:0/3656203923 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f510c096fa0 con 0x7f510c0ac260 2022-07-21T01:48:39.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f51108c0700 1 -- 172.21.15.140:0/3656203923 >> v1:172.21.15.152:6789/0 conn(0x7f510c0ac260 legacy=0x7f510c096890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f51108c0700 1 -- 172.21.15.140:0/3656203923 >> v1:172.21.15.140:6789/0 conn(0x7f510c141200 legacy=0x7f510c1400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f51108c0700 1 -- 172.21.15.140:0/3656203923 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c140ac0 con 0x7f510c0b3620 2022-07-21T01:48:39.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.967+0000 7f51108c0700 1 --2- 172.21.15.140:0/3656203923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0b3620 0x7f510c0ac640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f50fc002f70 tx=0x7f50fc006d90).ready entity=mon.2 client_cookie=7315fb2c83e41ea8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5103fff700 1 -- 172.21.15.140:0/3656203923 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50fc00d4a0 con 0x7f510c0b3620 2022-07-21T01:48:39.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5103fff700 1 -- 172.21.15.140:0/3656203923 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc000e90 con 0x7f510c0b3620 2022-07-21T01:48:39.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5103fff700 1 -- 172.21.15.140:0/3656203923 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50fc00d7f0 con 0x7f510c0b3620 2022-07-21T01:48:39.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 -- 172.21.15.140:0/3656203923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0b3620 msgr2=0x7f510c0ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 --2- 172.21.15.140:0/3656203923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0b3620 0x7f510c0ac640 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f50fc002f70 tx=0x7f50fc006d90).stop 2022-07-21T01:48:39.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 -- 172.21.15.140:0/3656203923 shutdown_connections 2022-07-21T01:48:39.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 --2- 172.21.15.140:0/3656203923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0b3620 0x7f510c0ac640 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 -- 172.21.15.140:0/3656203923 >> 172.21.15.140:0/3656203923 conn(0x7f510c0b47d0 msgr2=0x7f510c05bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:39.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 -- 172.21.15.140:0/3656203923 shutdown_connections 2022-07-21T01:48:39.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 -- 172.21.15.140:0/3656203923 wait complete. 2022-07-21T01:48:39.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 Processor -- start 2022-07-21T01:48:39.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 -- start start 2022-07-21T01:48:39.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0ac260 0x7f510c144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f510c0b3620 0x7f510c0aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c141200 0x7f510c14fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c0ae890 con 0x7f510c0b3620 2022-07-21T01:48:39.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c0aeaf0 con 0x7f510c141200 2022-07-21T01:48:39.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f5113325700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c097ce0 con 0x7f510c0ac260 2022-07-21T01:48:39.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f51110c1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0ac260 0x7f510c144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f51108c0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f510c0b3620 0x7f510c0aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f51110c1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0ac260 0x7f510c144540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:57636/0 (socket says 172.21.15.140:57636) 2022-07-21T01:48:39.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f51110c1700 1 -- 172.21.15.140:0/2616278760 learned_addr learned my addr 172.21.15.140:0/2616278760 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:39.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.971+0000 7f51118c2700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c141200 0x7f510c14fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f51108c0700 1 -- 172.21.15.140:0/2616278760 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0ac260 msgr2=0x7f510c144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f51108c0700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0ac260 0x7f510c144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f51108c0700 1 -- 172.21.15.140:0/2616278760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c141200 msgr2=0x7f510c14fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:39.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f51108c0700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c141200 0x7f510c14fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:39.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f51108c0700 1 -- 172.21.15.140:0/2616278760 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc00b120 con 0x7f510c0b3620 2022-07-21T01:48:39.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f51108c0700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f510c0b3620 0x7f510c0aef90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f50fc002f70 tx=0x7f50fc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f51027fc700 1 -- 172.21.15.140:0/2616278760 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50fc011ae0 con 0x7f510c0b3620 2022-07-21T01:48:39.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f51027fc700 1 -- 172.21.15.140:0/2616278760 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc00de90 con 0x7f510c0b3620 2022-07-21T01:48:39.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f5113325700 1 -- 172.21.15.140:0/2616278760 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510c098d60 con 0x7f510c0b3620 2022-07-21T01:48:39.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f51027fc700 1 -- 172.21.15.140:0/2616278760 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50fc011e30 con 0x7f510c0b3620 2022-07-21T01:48:39.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f5113325700 1 -- 172.21.15.140:0/2616278760 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510c098860 con 0x7f510c0b3620 2022-07-21T01:48:39.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f5113325700 1 -- 172.21.15.140:0/2616278760 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510c098bc0 con 0x7f510c0b3620 2022-07-21T01:48:39.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f51027fc700 1 -- 172.21.15.140:0/2616278760 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f50fc00f070 con 0x7f510c0b3620 2022-07-21T01:48:39.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.975+0000 7f51027fc700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50f4053b00 0x7f50f4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:39.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.979+0000 7f51027fc700 1 -- 172.21.15.140:0/2616278760 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f50fc06c2d0 con 0x7f510c0b3620 2022-07-21T01:48:39.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.979+0000 7f51110c1700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50f4053b00 0x7f50f4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:39.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.979+0000 7f51110c1700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50f4053b00 0x7f50f4055fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f510400d000 tx=0x7f5104007e60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:39.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:39.983+0000 7f51027fc700 1 -- 172.21.15.140:0/2616278760 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f510c098bc0 con 0x7f510c0b3620 2022-07-21T01:48:40.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.107+0000 7f5113325700 1 -- 172.21.15.140:0/2616278760 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f510c098bc0 con 0x7f50f4053b00 2022-07-21T01:48:40.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.107+0000 7f51027fc700 1 -- 172.21.15.140:0/2616278760 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82228 (secure 0 0 0) 0x7f510c098bc0 con 0x7f50f4053b00 2022-07-21T01:48:40.110 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:40.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 -- 172.21.15.140:0/2616278760 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50f4053b00 msgr2=0x7f50f4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50f4053b00 0x7f50f4055fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f510400d000 tx=0x7f5104007e60).stop 2022-07-21T01:48:40.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 -- 172.21.15.140:0/2616278760 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f510c0b3620 msgr2=0x7f510c0aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f510c0b3620 0x7f510c0aef90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f50fc002f70 tx=0x7f50fc014000).stop 2022-07-21T01:48:40.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 -- 172.21.15.140:0/2616278760 shutdown_connections 2022-07-21T01:48:40.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50f4053b00 0x7f50f4055fc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c141200 0x7f510c14fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f510c0b3620 0x7f510c0aef90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 --2- 172.21.15.140:0/2616278760 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f510c0ac260 0x7f510c144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 -- 172.21.15.140:0/2616278760 >> 172.21.15.140:0/2616278760 conn(0x7f510c0b47d0 msgr2=0x7f510c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:40.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 -- 172.21.15.140:0/2616278760 shutdown_connections 2022-07-21T01:48:40.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.111+0000 7f5113325700 1 -- 172.21.15.140:0/2616278760 wait complete. 2022-07-21T01:48:40.115 INFO:teuthology.orchestra.run.smithi140.stderr:dumped all 2022-07-21T01:48:40.121 INFO:teuthology.orchestra.run.smithi140.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2022-07-21T01:48:39.900210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":6366784,"kb_used_data":743228256,"kb_used_omap":604,"kb_used_meta":0,"kb_avail":743228256,"statfs":{"total":767585320960,"available":761065734144,"internally_reserved":0,"allocated":761065734144,"data_stored":761065734144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":619440,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.471739"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.441638+0000","last_change":"2022-07-21T01:48:31.441432+0000","last_active":"2022-07-21T01:48:31.441638+0000","last_peered":"2022-07-21T01:48:31.441638+0000","last_clean":"2022-07-21T01:48:31.441638+0000","last_became_active":"2022-07-21T01:48:31.440766+0000","last_became_peered":"2022-07-21T01:48:31.440766+0000","last_unstale":"2022-07-21T01:48:31.441638+0000","last_undegraded":"2022-07-21T01:48:31.441638+0000","last_fullsized":"2022-07-21T01:48:31.441638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.445217+0000","last_change":"2022-07-21T01:48:22.241647+0000","last_active":"2022-07-21T01:48:31.445217+0000","last_peered":"2022-07-21T01:48:31.445217+0000","last_clean":"2022-07-21T01:48:31.445217+0000","last_became_active":"2022-07-21T01:48:22.240745+0000","last_became_peered":"2022-07-21T01:48:22.240745+0000","last_unstale":"2022-07-21T01:48:31.445217+0000","last_undegraded":"2022-07-21T01:48:31.445217+0000","last_fullsized":"2022-07-21T01:48:31.445217+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:17.950031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:17.950031+0000","last_clean_scrub_stamp":"2022-07-21T01:48:17.950031+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.433823+0000","last_change":"2022-07-21T01:48:27.974374+0000","last_active":"2022-07-21T01:48:31.433823+0000","last_peered":"2022-07-21T01:48:31.433823+0000","last_clean":"2022-07-21T01:48:31.433823+0000","last_became_active":"2022-07-21T01:48:27.973977+0000","last_became_peered":"2022-07-21T01:48:27.973977+0000","last_unstale":"2022-07-21T01:48:31.433823+0000","last_undegraded":"2022-07-21T01:48:31.433823+0000","last_fullsized":"2022-07-21T01:48:31.433823+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.442125+0000","last_change":"2022-07-21T01:48:31.110631+0000","last_active":"2022-07-21T01:48:31.442125+0000","last_peered":"2022-07-21T01:48:31.442125+0000","last_clean":"2022-07-21T01:48:31.442125+0000","last_became_active":"2022-07-21T01:48:31.109569+0000","last_became_peered":"2022-07-21T01:48:31.109569+0000","last_unstale":"2022-07-21T01:48:31.442125+0000","last_undegraded":"2022-07-21T01:48:31.442125+0000","last_fullsized":"2022-07-21T01:48:31.442125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.438469+0000","last_change":"2022-07-21T01:48:31.438048+0000","last_active":"2022-07-21T01:48:31.438469+0000","last_peered":"2022-07-21T01:48:31.438469+0000","last_clean":"2022-07-21T01:48:31.438469+0000","last_became_active":"2022-07-21T01:48:31.437396+0000","last_became_peered":"2022-07-21T01:48:31.437396+0000","last_unstale":"2022-07-21T01:48:31.438469+0000","last_undegraded":"2022-07-21T01:48:31.438469+0000","last_fullsized":"2022-07-21T01:48:31.438469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.445055+0000","last_change":"2022-07-21T01:48:27.974429+0000","last_active":"2022-07-21T01:48:31.445055+0000","last_peered":"2022-07-21T01:48:31.445055+0000","last_clean":"2022-07-21T01:48:31.445055+0000","last_became_active":"2022-07-21T01:48:27.973957+0000","last_became_peered":"2022-07-21T01:48:27.973957+0000","last_unstale":"2022-07-21T01:48:31.445055+0000","last_undegraded":"2022-07-21T01:48:31.445055+0000","last_fullsized":"2022-07-21T01:48:31.445055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.435635+0000","last_change":"2022-07-21T01:48:31.110488+0000","last_active":"2022-07-21T01:48:31.435635+0000","last_peered":"2022-07-21T01:48:31.435635+0000","last_clean":"2022-07-21T01:48:31.435635+0000","last_became_active":"2022-07-21T01:48:31.110137+0000","last_became_peered":"2022-07-21T01:48:31.110137+0000","last_unstale":"2022-07-21T01:48:31.435635+0000","last_undegraded":"2022-07-21T01:48:31.435635+0000","last_fullsized":"2022-07-21T01:48:31.435635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.439764+0000","last_change":"2022-07-21T01:48:31.438987+0000","last_active":"2022-07-21T01:48:31.439764+0000","last_peered":"2022-07-21T01:48:31.439764+0000","last_clean":"2022-07-21T01:48:31.439764+0000","last_became_active":"2022-07-21T01:48:31.438554+0000","last_became_peered":"2022-07-21T01:48:31.438554+0000","last_unstale":"2022-07-21T01:48:31.439764+0000","last_undegraded":"2022-07-21T01:48:31.439764+0000","last_fullsized":"2022-07-21T01:48:31.439764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.429350+0000","last_change":"2022-07-21T01:48:27.968845+0000","last_active":"2022-07-21T01:48:31.429350+0000","last_peered":"2022-07-21T01:48:31.429350+0000","last_clean":"2022-07-21T01:48:31.429350+0000","last_became_active":"2022-07-21T01:48:27.967411+0000","last_became_peered":"2022-07-21T01:48:27.967411+0000","last_unstale":"2022-07-21T01:48:31.429350+0000","last_undegraded":"2022-07-21T01:48:31.429350+0000","last_fullsized":"2022-07-21T01:48:31.429350+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.448695+0000","last_change":"2022-07-21T01:48:31.109985+0000","last_active":"2022-07-21T01:48:31.448695+0000","last_peered":"2022-07-21T01:48:31.448695+0000","last_clean":"2022-07-21T01:48:31.448695+0000","last_became_active":"2022-07-21T01:48:31.108696+0000","last_became_peered":"2022-07-21T01:48:31.108696+0000","last_unstale":"2022-07-21T01:48:31.448695+0000","last_undegraded":"2022-07-21T01:48:31.448695+0000","last_fullsized":"2022-07-21T01:48:31.448695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.447026+0000","last_change":"2022-07-21T01:48:31.439876+0000","last_active":"2022-07-21T01:48:31.447026+0000","last_peered":"2022-07-21T01:48:31.447026+0000","last_clean":"2022-07-21T01:48:31.447026+0000","last_became_active":"2022-07-21T01:48:31.439300+0000","last_became_peered":"2022-07-21T01:48:31.439300+0000","last_unstale":"2022-07-21T01:48:31.447026+0000","last_undegraded":"2022-07-21T01:48:31.447026+0000","last_fullsized":"2022-07-21T01:48:31.447026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.434061+0000","last_change":"2022-07-21T01:48:27.974179+0000","last_active":"2022-07-21T01:48:31.434061+0000","last_peered":"2022-07-21T01:48:31.434061+0000","last_clean":"2022-07-21T01:48:31.434061+0000","last_became_active":"2022-07-21T01:48:27.973744+0000","last_became_peered":"2022-07-21T01:48:27.973744+0000","last_unstale":"2022-07-21T01:48:31.434061+0000","last_undegraded":"2022-07-21T01:48:31.434061+0000","last_fullsized":"2022-07-21T01:48:31.434061+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.449297+0000","last_change":"2022-07-21T01:48:31.115414+0000","last_active":"2022-07-21T01:48:31.449297+0000","last_peered":"2022-07-21T01:48:31.449297+0000","last_clean":"2022-07-21T01:48:31.449297+0000","last_became_active":"2022-07-21T01:48:31.115117+0000","last_became_peered":"2022-07-21T01:48:31.115117+0000","last_unstale":"2022-07-21T01:48:31.449297+0000","last_undegraded":"2022-07-21T01:48:31.449297+0000","last_fullsized":"2022-07-21T01:48:31.449297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.440466+0000","last_change":"2022-07-21T01:48:31.439589+0000","last_active":"2022-07-21T01:48:31.440466+0000","last_peered":"2022-07-21T01:48:31.440466+0000","last_clean":"2022-07-21T01:48:31.440466+0000","last_became_active":"2022-07-21T01:48:31.439191+0000","last_became_peered":"2022-07-21T01:48:31.439191+0000","last_unstale":"2022-07-21T01:48:31.440466+0000","last_undegraded":"2022-07-21T01:48:31.440466+0000","last_fullsized":"2022-07-21T01:48:31.440466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.450055+0000","last_change":"2022-07-21T01:48:31.113428+0000","last_active":"2022-07-21T01:48:31.450055+0000","last_peered":"2022-07-21T01:48:31.450055+0000","last_clean":"2022-07-21T01:48:31.450055+0000","last_became_active":"2022-07-21T01:48:31.113150+0000","last_became_peered":"2022-07-21T01:48:31.113150+0000","last_unstale":"2022-07-21T01:48:31.450055+0000","last_undegraded":"2022-07-21T01:48:31.450055+0000","last_fullsized":"2022-07-21T01:48:31.450055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.435994+0000","last_change":"2022-07-21T01:48:27.992744+0000","last_active":"2022-07-21T01:48:31.435994+0000","last_peered":"2022-07-21T01:48:31.435994+0000","last_clean":"2022-07-21T01:48:31.435994+0000","last_became_active":"2022-07-21T01:48:27.992216+0000","last_became_peered":"2022-07-21T01:48:27.992216+0000","last_unstale":"2022-07-21T01:48:31.435994+0000","last_undegraded":"2022-07-21T01:48:31.435994+0000","last_fullsized":"2022-07-21T01:48:31.435994+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.444348+0000","last_change":"2022-07-21T01:48:31.440982+0000","last_active":"2022-07-21T01:48:31.444348+0000","last_peered":"2022-07-21T01:48:31.444348+0000","last_clean":"2022-07-21T01:48:31.444348+0000","last_became_active":"2022-07-21T01:48:31.437988+0000","last_became_peered":"2022-07-21T01:48:31.437988+0000","last_unstale":"2022-07-21T01:48:31.444348+0000","last_undegraded":"2022-07-21T01:48:31.444348+0000","last_fullsized":"2022-07-21T01:48:31.444348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.448900+0000","last_change":"2022-07-21T01:48:31.109232+0000","last_active":"2022-07-21T01:48:31.448900+0000","last_peered":"2022-07-21T01:48:31.448900+0000","last_clean":"2022-07-21T01:48:31.448900+0000","last_became_active":"2022-07-21T01:48:31.108745+0000","last_became_peered":"2022-07-21T01:48:31.108745+0000","last_unstale":"2022-07-21T01:48:31.448900+0000","last_undegraded":"2022-07-21T01:48:31.448900+0000","last_fullsized":"2022-07-21T01:48:31.448900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.444448+0000","last_change":"2022-07-21T01:48:27.996870+0000","last_active":"2022-07-21T01:48:31.444448+0000","last_peered":"2022-07-21T01:48:31.444448+0000","last_clean":"2022-07-21T01:48:31.444448+0000","last_became_active":"2022-07-21T01:48:27.996456+0000","last_became_peered":"2022-07-21T01:48:27.996456+0000","last_unstale":"2022-07-21T01:48:31.444448+0000","last_undegraded":"2022-07-21T01:48:31.444448+0000","last_fullsized":"2022-07-21T01:48:31.444448+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.444331+0000","last_change":"2022-07-21T01:48:31.443991+0000","last_active":"2022-07-21T01:48:31.444331+0000","last_peered":"2022-07-21T01:48:31.444331+0000","last_clean":"2022-07-21T01:48:31.444331+0000","last_became_active":"2022-07-21T01:48:31.443606+0000","last_became_peered":"2022-07-21T01:48:31.443606+0000","last_unstale":"2022-07-21T01:48:31.444331+0000","last_undegraded":"2022-07-21T01:48:31.444331+0000","last_fullsized":"2022-07-21T01:48:31.444331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.446683+0000","last_change":"2022-07-21T01:48:31.112093+0000","last_active":"2022-07-21T01:48:31.446683+0000","last_peered":"2022-07-21T01:48:31.446683+0000","last_clean":"2022-07-21T01:48:31.446683+0000","last_became_active":"2022-07-21T01:48:31.111515+0000","last_became_peered":"2022-07-21T01:48:31.111515+0000","last_unstale":"2022-07-21T01:48:31.446683+0000","last_undegraded":"2022-07-21T01:48:31.446683+0000","last_fullsized":"2022-07-21T01:48:31.446683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.436709+0000","last_change":"2022-07-21T01:48:27.970841+0000","last_active":"2022-07-21T01:48:31.436709+0000","last_peered":"2022-07-21T01:48:31.436709+0000","last_clean":"2022-07-21T01:48:31.436709+0000","last_became_active":"2022-07-21T01:48:27.970498+0000","last_became_peered":"2022-07-21T01:48:27.970498+0000","last_unstale":"2022-07-21T01:48:31.436709+0000","last_undegraded":"2022-07-21T01:48:31.436709+0000","last_fullsized":"2022-07-21T01:48:31.436709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.442277+0000","last_change":"2022-07-21T01:48:31.441954+0000","last_active":"2022-07-21T01:48:31.442277+0000","last_peered":"2022-07-21T01:48:31.442277+0000","last_clean":"2022-07-21T01:48:31.442277+0000","last_became_active":"2022-07-21T01:48:31.441653+0000","last_became_peered":"2022-07-21T01:48:31.441653+0000","last_unstale":"2022-07-21T01:48:31.442277+0000","last_undegraded":"2022-07-21T01:48:31.442277+0000","last_fullsized":"2022-07-21T01:48:31.442277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2022-07-21T01:48:33.084963+0000","last_change":"2022-07-21T01:48:31.113239+0000","last_active":"2022-07-21T01:48:33.084963+0000","last_peered":"2022-07-21T01:48:33.084963+0000","last_clean":"2022-07-21T01:48:33.084963+0000","last_became_active":"2022-07-21T01:48:31.112660+0000","last_became_peered":"2022-07-21T01:48:31.112660+0000","last_unstale":"2022-07-21T01:48:33.084963+0000","last_undegraded":"2022-07-21T01:48:33.084963+0000","last_fullsized":"2022-07-21T01:48:33.084963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.435045+0000","last_change":"2022-07-21T01:48:27.969947+0000","last_active":"2022-07-21T01:48:31.435045+0000","last_peered":"2022-07-21T01:48:31.435045+0000","last_clean":"2022-07-21T01:48:31.435045+0000","last_became_active":"2022-07-21T01:48:27.968884+0000","last_became_peered":"2022-07-21T01:48:27.968884+0000","last_unstale":"2022-07-21T01:48:31.435045+0000","last_undegraded":"2022-07-21T01:48:31.435045+0000","last_fullsized":"2022-07-21T01:48:31.435045+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795852,"kb_used_data":92903528,"kb_used_omap":106,"kb_used_meta":0,"kb_avail":92903528,"statfs":{"total":95948165120,"available":95133212672,"internally_reserved":0,"allocated":95133212672,"data_stored":95133212672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":109023,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]}]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795848,"kb_used_data":92903532,"kb_used_omap":76,"kb_used_meta":0,"kb_avail":92903532,"statfs":{"total":95948165120,"available":95133216768,"internally_reserved":0,"allocated":95133216768,"data_stored":95133216768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77824,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795840,"kb_used_data":92903540,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903540,"statfs":{"total":95948165120,"available":95133224960,"internally_reserved":0,"allocated":95133224960,"data_stored":95133224960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43381,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795848,"kb_used_data":92903532,"kb_used_omap":77,"kb_used_meta":0,"kb_avail":92903532,"statfs":{"total":95948165120,"available":95133216768,"internally_reserved":0,"allocated":95133216768,"data_stored":95133216768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":79374,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":27.279}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795844,"kb_used_data":92903536,"kb_used_omap":61,"kb_used_meta":0,"kb_avail":92903536,"statfs":{"total":95948165120,"available":95133220864,"internally_reserved":0,"allocated":95133220864,"data_stored":95133220864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":62992,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795856,"kb_used_data":92903524,"kb_used_omap":69,"kb_used_meta":0,"kb_avail":92903524,"statfs":{"total":95948165120,"available":95133208576,"internally_reserved":0,"allocated":95133208576,"data_stored":95133208576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":71200,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795852,"kb_used_data":92903528,"kb_used_omap":72,"kb_used_meta":0,"kb_avail":92903528,"statfs":{"total":95948165120,"available":95133212672,"internally_reserved":0,"allocated":95133212672,"data_stored":95133212672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":74018,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795844,"kb_used_data":92903536,"kb_used_omap":99,"kb_used_meta":0,"kb_avail":92903536,"statfs":{"total":95948165120,"available":95133220864,"internally_reserved":0,"allocated":95133220864,"data_stored":95133220864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":101628,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]}]}]}} 2022-07-21T01:48:40.140 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-07-21T01:48:40.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 Processor -- start 2022-07-21T01:48:40.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 -- start start 2022-07-21T01:48:40.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96f413a5b0 0x7f96f413a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f96f40b20c0 0x7f96f405d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f40b4410 con 0x7f96f4140810 2022-07-21T01:48:40.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f4141ac0 con 0x7f96f40b20c0 2022-07-21T01:48:40.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40b4170 con 0x7f96f413a5b0 2022-07-21T01:48:40.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f37fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f96f4140810 0x7f96f40b3650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46830/0 (socket says 172.21.15.140:46830) 2022-07-21T01:48:40.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f37fe700 1 -- 172.21.15.140:0/2098464159 learned_addr learned my addr 172.21.15.140:0/2098464159 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:40.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f8d37700 1 --2- 172.21.15.140:0/2098464159 >> v2:172.21.15.140:3300/0 conn(0x7f96f40b20c0 0x7f96f405d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f3fff700 1 --2- 172.21.15.140:0/2098464159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96f413a5b0 0x7f96f413a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f2ffd700 1 -- 172.21.15.140:0/2098464159 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210484179 0 0) 0x7f96f40b4410 con 0x7f96f4140810 2022-07-21T01:48:40.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f2ffd700 1 -- 172.21.15.140:0/2098464159 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96e0003260 con 0x7f96f4140810 2022-07-21T01:48:40.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f8d37700 1 -- 172.21.15.140:0/2098464159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96f413a5b0 msgr2=0x7f96f413a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:40.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f8d37700 1 --2- 172.21.15.140:0/2098464159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96f413a5b0 0x7f96f413a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f8d37700 1 -- 172.21.15.140:0/2098464159 >> v1:172.21.15.140:6789/0 conn(0x7f96f4140810 legacy=0x7f96f40b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f2ffd700 1 -- 172.21.15.140:0/2098464159 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4002595440 0 0) 0x7f96e0003260 con 0x7f96f4140810 2022-07-21T01:48:40.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f8d37700 1 -- 172.21.15.140:0/2098464159 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f405bfc0 con 0x7f96f40b20c0 2022-07-21T01:48:40.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f8d37700 1 --2- 172.21.15.140:0/2098464159 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96f40b20c0 0x7f96f405d0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f96ec001b80 tx=0x7f96e4006df0).ready entity=mon.0 client_cookie=b3c46b16f9967f3c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:40.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f2ffd700 1 -- 172.21.15.140:0/2098464159 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96ec0019e0 con 0x7f96f40b20c0 2022-07-21T01:48:40.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f2ffd700 1 -- 172.21.15.140:0/2098464159 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e4000e90 con 0x7f96f40b20c0 2022-07-21T01:48:40.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f2ffd700 1 -- 172.21.15.140:0/2098464159 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96e4012460 con 0x7f96f40b20c0 2022-07-21T01:48:40.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 -- 172.21.15.140:0/2098464159 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96f40b20c0 msgr2=0x7f96f405d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 --2- 172.21.15.140:0/2098464159 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96f40b20c0 0x7f96f405d0c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f96ec001b80 tx=0x7f96e4006df0).stop 2022-07-21T01:48:40.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 -- 172.21.15.140:0/2098464159 shutdown_connections 2022-07-21T01:48:40.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 --2- 172.21.15.140:0/2098464159 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96f40b20c0 0x7f96f405d0c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 --2- 172.21.15.140:0/2098464159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96f413a5b0 0x7f96f413a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 -- 172.21.15.140:0/2098464159 >> 172.21.15.140:0/2098464159 conn(0x7f96f4064be0 msgr2=0x7f96f4064fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:40.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 -- 172.21.15.140:0/2098464159 shutdown_connections 2022-07-21T01:48:40.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.251+0000 7f96f9538700 1 -- 172.21.15.140:0/2098464159 wait complete. 2022-07-21T01:48:40.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f9538700 1 Processor -- start 2022-07-21T01:48:40.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f9538700 1 -- start start 2022-07-21T01:48:40.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f9538700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96f40b20c0 0x7f96f40b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f9538700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96f413a5b0 0x7f96f40b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f9538700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96f4140810 0x7f96f40b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f9538700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f413c3b0 con 0x7f96f413a5b0 2022-07-21T01:48:40.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f9538700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f413b9d0 con 0x7f96f4140810 2022-07-21T01:48:40.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f9538700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f413bc70 con 0x7f96f40b20c0 2022-07-21T01:48:40.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96f413a5b0 0x7f96f40b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96f413a5b0 0x7f96f40b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60920/0 (socket says 172.21.15.140:60920) 2022-07-21T01:48:40.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f37fe700 1 -- 172.21.15.140:0/1426500149 learned_addr learned my addr 172.21.15.140:0/1426500149 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:40.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f3fff700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96f40b20c0 0x7f96f40b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f8d37700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96f4140810 0x7f96f40b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f37fe700 1 -- 172.21.15.140:0/1426500149 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96f40b20c0 msgr2=0x7f96f40b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f37fe700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96f40b20c0 0x7f96f40b7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f37fe700 1 -- 172.21.15.140:0/1426500149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96f4140810 msgr2=0x7f96f40b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f37fe700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96f4140810 0x7f96f40b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f37fe700 1 -- 172.21.15.140:0/1426500149 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e400b180 con 0x7f96f413a5b0 2022-07-21T01:48:40.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f3fff700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96f40b20c0 0x7f96f40b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:40.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f37fe700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96f413a5b0 0x7f96f40b09c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f96ec001c40 tx=0x7f96ec0021d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:40.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f17fa700 1 -- 172.21.15.140:0/1426500149 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96ec004ab0 con 0x7f96f413a5b0 2022-07-21T01:48:40.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f17fa700 1 -- 172.21.15.140:0/1426500149 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec004c10 con 0x7f96f413a5b0 2022-07-21T01:48:40.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f9538700 1 -- 172.21.15.140:0/1426500149 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f413bf10 con 0x7f96f413a5b0 2022-07-21T01:48:40.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f17fa700 1 -- 172.21.15.140:0/1426500149 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96ec007170 con 0x7f96f413a5b0 2022-07-21T01:48:40.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.255+0000 7f96f9538700 1 -- 172.21.15.140:0/1426500149 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f413b570 con 0x7f96f413a5b0 2022-07-21T01:48:40.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.259+0000 7f96f9538700 1 -- 172.21.15.140:0/1426500149 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f40b6a40 con 0x7f96f413a5b0 2022-07-21T01:48:40.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.259+0000 7f96f17fa700 1 -- 172.21.15.140:0/1426500149 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f96ec01b060 con 0x7f96f413a5b0 2022-07-21T01:48:40.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.259+0000 7f96f17fa700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d4053b00 0x7f96d4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.259+0000 7f96f17fa700 1 -- 172.21.15.140:0/1426500149 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f96ec06bce0 con 0x7f96f413a5b0 2022-07-21T01:48:40.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.259+0000 7f96f3fff700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d4053b00 0x7f96d4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.259+0000 7f96f3fff700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d4053b00 0x7f96d4055fc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f96e800b000 tx=0x7f96e8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:40.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.263+0000 7f96f17fa700 1 -- 172.21.15.140:0/1426500149 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f96f40b6a40 con 0x7f96f413a5b0 2022-07-21T01:48:40.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.387+0000 7f96f9538700 1 -- 172.21.15.140:0/1426500149 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96f40b6a40 con 0x7f96d4053b00 2022-07-21T01:48:40.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.395+0000 7f96f17fa700 1 -- 172.21.15.140:0/1426500149 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82228 (secure 0 0 0) 0x7f96f40b6a40 con 0x7f96d4053b00 2022-07-21T01:48:40.394 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:40.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 -- 172.21.15.140:0/1426500149 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d4053b00 msgr2=0x7f96d4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d4053b00 0x7f96d4055fc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f96e800b000 tx=0x7f96e8006df0).stop 2022-07-21T01:48:40.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 -- 172.21.15.140:0/1426500149 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96f413a5b0 msgr2=0x7f96f40b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96f413a5b0 0x7f96f40b09c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f96ec001c40 tx=0x7f96ec0021d0).stop 2022-07-21T01:48:40.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 -- 172.21.15.140:0/1426500149 shutdown_connections 2022-07-21T01:48:40.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f96d4053b00 0x7f96d4055fc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96f4140810 0x7f96f40b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96f413a5b0 0x7f96f40b09c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 --2- 172.21.15.140:0/1426500149 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96f40b20c0 0x7f96f40b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 -- 172.21.15.140:0/1426500149 >> 172.21.15.140:0/1426500149 conn(0x7f96f4064be0 msgr2=0x7f96f4142680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:40.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 -- 172.21.15.140:0/1426500149 shutdown_connections 2022-07-21T01:48:40.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.399+0000 7f96f9538700 1 -- 172.21.15.140:0/1426500149 wait complete. 2022-07-21T01:48:40.405 INFO:teuthology.orchestra.run.smithi140.stderr:dumped all 2022-07-21T01:48:40.409 INFO:teuthology.orchestra.run.smithi140.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2022-07-21T01:48:39.900210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":6366784,"kb_used_data":743228256,"kb_used_omap":604,"kb_used_meta":0,"kb_avail":743228256,"statfs":{"total":767585320960,"available":761065734144,"internally_reserved":0,"allocated":761065734144,"data_stored":761065734144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":619440,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.471739"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.441638+0000","last_change":"2022-07-21T01:48:31.441432+0000","last_active":"2022-07-21T01:48:31.441638+0000","last_peered":"2022-07-21T01:48:31.441638+0000","last_clean":"2022-07-21T01:48:31.441638+0000","last_became_active":"2022-07-21T01:48:31.440766+0000","last_became_peered":"2022-07-21T01:48:31.440766+0000","last_unstale":"2022-07-21T01:48:31.441638+0000","last_undegraded":"2022-07-21T01:48:31.441638+0000","last_fullsized":"2022-07-21T01:48:31.441638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"14'2","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.445217+0000","last_change":"2022-07-21T01:48:22.241647+0000","last_active":"2022-07-21T01:48:31.445217+0000","last_peered":"2022-07-21T01:48:31.445217+0000","last_clean":"2022-07-21T01:48:31.445217+0000","last_became_active":"2022-07-21T01:48:22.240745+0000","last_became_peered":"2022-07-21T01:48:22.240745+0000","last_unstale":"2022-07-21T01:48:31.445217+0000","last_undegraded":"2022-07-21T01:48:31.445217+0000","last_fullsized":"2022-07-21T01:48:31.445217+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:17.950031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:17.950031+0000","last_clean_scrub_stamp":"2022-07-21T01:48:17.950031+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.433823+0000","last_change":"2022-07-21T01:48:27.974374+0000","last_active":"2022-07-21T01:48:31.433823+0000","last_peered":"2022-07-21T01:48:31.433823+0000","last_clean":"2022-07-21T01:48:31.433823+0000","last_became_active":"2022-07-21T01:48:27.973977+0000","last_became_peered":"2022-07-21T01:48:27.973977+0000","last_unstale":"2022-07-21T01:48:31.433823+0000","last_undegraded":"2022-07-21T01:48:31.433823+0000","last_fullsized":"2022-07-21T01:48:31.433823+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.442125+0000","last_change":"2022-07-21T01:48:31.110631+0000","last_active":"2022-07-21T01:48:31.442125+0000","last_peered":"2022-07-21T01:48:31.442125+0000","last_clean":"2022-07-21T01:48:31.442125+0000","last_became_active":"2022-07-21T01:48:31.109569+0000","last_became_peered":"2022-07-21T01:48:31.109569+0000","last_unstale":"2022-07-21T01:48:31.442125+0000","last_undegraded":"2022-07-21T01:48:31.442125+0000","last_fullsized":"2022-07-21T01:48:31.442125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.438469+0000","last_change":"2022-07-21T01:48:31.438048+0000","last_active":"2022-07-21T01:48:31.438469+0000","last_peered":"2022-07-21T01:48:31.438469+0000","last_clean":"2022-07-21T01:48:31.438469+0000","last_became_active":"2022-07-21T01:48:31.437396+0000","last_became_peered":"2022-07-21T01:48:31.437396+0000","last_unstale":"2022-07-21T01:48:31.438469+0000","last_undegraded":"2022-07-21T01:48:31.438469+0000","last_fullsized":"2022-07-21T01:48:31.438469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.445055+0000","last_change":"2022-07-21T01:48:27.974429+0000","last_active":"2022-07-21T01:48:31.445055+0000","last_peered":"2022-07-21T01:48:31.445055+0000","last_clean":"2022-07-21T01:48:31.445055+0000","last_became_active":"2022-07-21T01:48:27.973957+0000","last_became_peered":"2022-07-21T01:48:27.973957+0000","last_unstale":"2022-07-21T01:48:31.445055+0000","last_undegraded":"2022-07-21T01:48:31.445055+0000","last_fullsized":"2022-07-21T01:48:31.445055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.435635+0000","last_change":"2022-07-21T01:48:31.110488+0000","last_active":"2022-07-21T01:48:31.435635+0000","last_peered":"2022-07-21T01:48:31.435635+0000","last_clean":"2022-07-21T01:48:31.435635+0000","last_became_active":"2022-07-21T01:48:31.110137+0000","last_became_peered":"2022-07-21T01:48:31.110137+0000","last_unstale":"2022-07-21T01:48:31.435635+0000","last_undegraded":"2022-07-21T01:48:31.435635+0000","last_fullsized":"2022-07-21T01:48:31.435635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.439764+0000","last_change":"2022-07-21T01:48:31.438987+0000","last_active":"2022-07-21T01:48:31.439764+0000","last_peered":"2022-07-21T01:48:31.439764+0000","last_clean":"2022-07-21T01:48:31.439764+0000","last_became_active":"2022-07-21T01:48:31.438554+0000","last_became_peered":"2022-07-21T01:48:31.438554+0000","last_unstale":"2022-07-21T01:48:31.439764+0000","last_undegraded":"2022-07-21T01:48:31.439764+0000","last_fullsized":"2022-07-21T01:48:31.439764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.429350+0000","last_change":"2022-07-21T01:48:27.968845+0000","last_active":"2022-07-21T01:48:31.429350+0000","last_peered":"2022-07-21T01:48:31.429350+0000","last_clean":"2022-07-21T01:48:31.429350+0000","last_became_active":"2022-07-21T01:48:27.967411+0000","last_became_peered":"2022-07-21T01:48:27.967411+0000","last_unstale":"2022-07-21T01:48:31.429350+0000","last_undegraded":"2022-07-21T01:48:31.429350+0000","last_fullsized":"2022-07-21T01:48:31.429350+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.448695+0000","last_change":"2022-07-21T01:48:31.109985+0000","last_active":"2022-07-21T01:48:31.448695+0000","last_peered":"2022-07-21T01:48:31.448695+0000","last_clean":"2022-07-21T01:48:31.448695+0000","last_became_active":"2022-07-21T01:48:31.108696+0000","last_became_peered":"2022-07-21T01:48:31.108696+0000","last_unstale":"2022-07-21T01:48:31.448695+0000","last_undegraded":"2022-07-21T01:48:31.448695+0000","last_fullsized":"2022-07-21T01:48:31.448695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.447026+0000","last_change":"2022-07-21T01:48:31.439876+0000","last_active":"2022-07-21T01:48:31.447026+0000","last_peered":"2022-07-21T01:48:31.447026+0000","last_clean":"2022-07-21T01:48:31.447026+0000","last_became_active":"2022-07-21T01:48:31.439300+0000","last_became_peered":"2022-07-21T01:48:31.439300+0000","last_unstale":"2022-07-21T01:48:31.447026+0000","last_undegraded":"2022-07-21T01:48:31.447026+0000","last_fullsized":"2022-07-21T01:48:31.447026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.434061+0000","last_change":"2022-07-21T01:48:27.974179+0000","last_active":"2022-07-21T01:48:31.434061+0000","last_peered":"2022-07-21T01:48:31.434061+0000","last_clean":"2022-07-21T01:48:31.434061+0000","last_became_active":"2022-07-21T01:48:27.973744+0000","last_became_peered":"2022-07-21T01:48:27.973744+0000","last_unstale":"2022-07-21T01:48:31.434061+0000","last_undegraded":"2022-07-21T01:48:31.434061+0000","last_fullsized":"2022-07-21T01:48:31.434061+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'5","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.449297+0000","last_change":"2022-07-21T01:48:31.115414+0000","last_active":"2022-07-21T01:48:31.449297+0000","last_peered":"2022-07-21T01:48:31.449297+0000","last_clean":"2022-07-21T01:48:31.449297+0000","last_became_active":"2022-07-21T01:48:31.115117+0000","last_became_peered":"2022-07-21T01:48:31.115117+0000","last_unstale":"2022-07-21T01:48:31.449297+0000","last_undegraded":"2022-07-21T01:48:31.449297+0000","last_fullsized":"2022-07-21T01:48:31.449297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.440466+0000","last_change":"2022-07-21T01:48:31.439589+0000","last_active":"2022-07-21T01:48:31.440466+0000","last_peered":"2022-07-21T01:48:31.440466+0000","last_clean":"2022-07-21T01:48:31.440466+0000","last_became_active":"2022-07-21T01:48:31.439191+0000","last_became_peered":"2022-07-21T01:48:31.439191+0000","last_unstale":"2022-07-21T01:48:31.440466+0000","last_undegraded":"2022-07-21T01:48:31.440466+0000","last_fullsized":"2022-07-21T01:48:31.440466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"21'6","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.450055+0000","last_change":"2022-07-21T01:48:31.113428+0000","last_active":"2022-07-21T01:48:31.450055+0000","last_peered":"2022-07-21T01:48:31.450055+0000","last_clean":"2022-07-21T01:48:31.450055+0000","last_became_active":"2022-07-21T01:48:31.113150+0000","last_became_peered":"2022-07-21T01:48:31.113150+0000","last_unstale":"2022-07-21T01:48:31.450055+0000","last_undegraded":"2022-07-21T01:48:31.450055+0000","last_fullsized":"2022-07-21T01:48:31.450055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.435994+0000","last_change":"2022-07-21T01:48:27.992744+0000","last_active":"2022-07-21T01:48:31.435994+0000","last_peered":"2022-07-21T01:48:31.435994+0000","last_clean":"2022-07-21T01:48:31.435994+0000","last_became_active":"2022-07-21T01:48:27.992216+0000","last_became_peered":"2022-07-21T01:48:27.992216+0000","last_unstale":"2022-07-21T01:48:31.435994+0000","last_undegraded":"2022-07-21T01:48:31.435994+0000","last_fullsized":"2022-07-21T01:48:31.435994+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.444348+0000","last_change":"2022-07-21T01:48:31.440982+0000","last_active":"2022-07-21T01:48:31.444348+0000","last_peered":"2022-07-21T01:48:31.444348+0000","last_clean":"2022-07-21T01:48:31.444348+0000","last_became_active":"2022-07-21T01:48:31.437988+0000","last_became_peered":"2022-07-21T01:48:31.437988+0000","last_unstale":"2022-07-21T01:48:31.444348+0000","last_undegraded":"2022-07-21T01:48:31.444348+0000","last_fullsized":"2022-07-21T01:48:31.444348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"21'8","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.448900+0000","last_change":"2022-07-21T01:48:31.109232+0000","last_active":"2022-07-21T01:48:31.448900+0000","last_peered":"2022-07-21T01:48:31.448900+0000","last_clean":"2022-07-21T01:48:31.448900+0000","last_became_active":"2022-07-21T01:48:31.108745+0000","last_became_peered":"2022-07-21T01:48:31.108745+0000","last_unstale":"2022-07-21T01:48:31.448900+0000","last_undegraded":"2022-07-21T01:48:31.448900+0000","last_fullsized":"2022-07-21T01:48:31.448900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.444448+0000","last_change":"2022-07-21T01:48:27.996870+0000","last_active":"2022-07-21T01:48:31.444448+0000","last_peered":"2022-07-21T01:48:31.444448+0000","last_clean":"2022-07-21T01:48:31.444448+0000","last_became_active":"2022-07-21T01:48:27.996456+0000","last_became_peered":"2022-07-21T01:48:27.996456+0000","last_unstale":"2022-07-21T01:48:31.444448+0000","last_undegraded":"2022-07-21T01:48:31.444448+0000","last_fullsized":"2022-07-21T01:48:31.444448+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.444331+0000","last_change":"2022-07-21T01:48:31.443991+0000","last_active":"2022-07-21T01:48:31.444331+0000","last_peered":"2022-07-21T01:48:31.444331+0000","last_clean":"2022-07-21T01:48:31.444331+0000","last_became_active":"2022-07-21T01:48:31.443606+0000","last_became_peered":"2022-07-21T01:48:31.443606+0000","last_unstale":"2022-07-21T01:48:31.444331+0000","last_undegraded":"2022-07-21T01:48:31.444331+0000","last_fullsized":"2022-07-21T01:48:31.444331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"21'7","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.446683+0000","last_change":"2022-07-21T01:48:31.112093+0000","last_active":"2022-07-21T01:48:31.446683+0000","last_peered":"2022-07-21T01:48:31.446683+0000","last_clean":"2022-07-21T01:48:31.446683+0000","last_became_active":"2022-07-21T01:48:31.111515+0000","last_became_peered":"2022-07-21T01:48:31.111515+0000","last_unstale":"2022-07-21T01:48:31.446683+0000","last_undegraded":"2022-07-21T01:48:31.446683+0000","last_fullsized":"2022-07-21T01:48:31.446683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.436709+0000","last_change":"2022-07-21T01:48:27.970841+0000","last_active":"2022-07-21T01:48:31.436709+0000","last_peered":"2022-07-21T01:48:31.436709+0000","last_clean":"2022-07-21T01:48:31.436709+0000","last_became_active":"2022-07-21T01:48:27.970498+0000","last_became_peered":"2022-07-21T01:48:27.970498+0000","last_unstale":"2022-07-21T01:48:31.436709+0000","last_undegraded":"2022-07-21T01:48:31.436709+0000","last_fullsized":"2022-07-21T01:48:31.436709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.442277+0000","last_change":"2022-07-21T01:48:31.441954+0000","last_active":"2022-07-21T01:48:31.442277+0000","last_peered":"2022-07-21T01:48:31.442277+0000","last_clean":"2022-07-21T01:48:31.442277+0000","last_became_active":"2022-07-21T01:48:31.441653+0000","last_became_peered":"2022-07-21T01:48:31.441653+0000","last_unstale":"2022-07-21T01:48:31.442277+0000","last_undegraded":"2022-07-21T01:48:31.442277+0000","last_fullsized":"2022-07-21T01:48:31.442277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:31.095105+0000","last_clean_scrub_stamp":"2022-07-21T01:48:31.095105+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2022-07-21T01:48:33.084963+0000","last_change":"2022-07-21T01:48:31.113239+0000","last_active":"2022-07-21T01:48:33.084963+0000","last_peered":"2022-07-21T01:48:33.084963+0000","last_clean":"2022-07-21T01:48:33.084963+0000","last_became_active":"2022-07-21T01:48:31.112660+0000","last_became_peered":"2022-07-21T01:48:31.112660+0000","last_unstale":"2022-07-21T01:48:33.084963+0000","last_undegraded":"2022-07-21T01:48:33.084963+0000","last_fullsized":"2022-07-21T01:48:33.084963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:30.085780+0000","last_clean_scrub_stamp":"2022-07-21T01:48:30.085780+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2022-07-21T01:48:31.435045+0000","last_change":"2022-07-21T01:48:27.969947+0000","last_active":"2022-07-21T01:48:31.435045+0000","last_peered":"2022-07-21T01:48:31.435045+0000","last_clean":"2022-07-21T01:48:31.435045+0000","last_became_active":"2022-07-21T01:48:27.968884+0000","last_became_peered":"2022-07-21T01:48:27.968884+0000","last_unstale":"2022-07-21T01:48:31.435045+0000","last_undegraded":"2022-07-21T01:48:31.435045+0000","last_fullsized":"2022-07-21T01:48:31.435045+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795852,"kb_used_data":92903528,"kb_used_omap":106,"kb_used_meta":0,"kb_avail":92903528,"statfs":{"total":95948165120,"available":95133212672,"internally_reserved":0,"allocated":95133212672,"data_stored":95133212672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":109023,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]}]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795848,"kb_used_data":92903532,"kb_used_omap":76,"kb_used_meta":0,"kb_avail":92903532,"statfs":{"total":95948165120,"available":95133216768,"internally_reserved":0,"allocated":95133216768,"data_stored":95133216768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":77824,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795840,"kb_used_data":92903540,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":92903540,"statfs":{"total":95948165120,"available":95133224960,"internally_reserved":0,"allocated":95133224960,"data_stored":95133224960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43381,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795848,"kb_used_data":92903532,"kb_used_omap":77,"kb_used_meta":0,"kb_avail":92903532,"statfs":{"total":95948165120,"available":95133216768,"internally_reserved":0,"allocated":95133216768,"data_stored":95133216768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":79374,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":27.279}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795844,"kb_used_data":92903536,"kb_used_omap":61,"kb_used_meta":0,"kb_avail":92903536,"statfs":{"total":95948165120,"available":95133220864,"internally_reserved":0,"allocated":95133220864,"data_stored":95133220864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":62992,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795856,"kb_used_data":92903524,"kb_used_omap":69,"kb_used_meta":0,"kb_avail":92903524,"statfs":{"total":95948165120,"available":95133208576,"internally_reserved":0,"allocated":95133208576,"data_stored":95133208576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":71200,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795852,"kb_used_data":92903528,"kb_used_omap":72,"kb_used_meta":0,"kb_avail":92903528,"statfs":{"total":95948165120,"available":95133212672,"internally_reserved":0,"allocated":95133212672,"data_stored":95133212672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":74018,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":795844,"kb_used_data":92903536,"kb_used_omap":99,"kb_used_meta":0,"kb_avail":92903536,"statfs":{"total":95948165120,"available":95133220864,"internally_reserved":0,"allocated":95133220864,"data_stored":95133220864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":101628,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]}]}]}} 2022-07-21T01:48:40.413 INFO:tasks.ceph.ceph_manager.ceph:clean! 2022-07-21T01:48:40.413 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2022-07-21T01:48:40.414 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2022-07-21T01:48:40.415 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2022-07-21T01:48:40.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.515+0000 7f2da826f700 1 Processor -- start 2022-07-21T01:48:40.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.515+0000 7f2da826f700 1 -- start start 2022-07-21T01:48:40.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da826f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0060c00 0x7f2da009cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da826f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2da005cda0 0x7f2da0146a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da826f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da0060fe0 con 0x7f2da0061410 2022-07-21T01:48:40.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da826f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da009e590 con 0x7f2da005cda0 2022-07-21T01:48:40.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da826f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00b39c0 con 0x7f2da0060c00 2022-07-21T01:48:40.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da580a700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2da0061410 0x7f2da0146330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46842/0 (socket says 172.21.15.140:46842) 2022-07-21T01:48:40.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da580a700 1 -- 172.21.15.140:0/3578990298 learned_addr learned my addr 172.21.15.140:0/3578990298 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:40.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da680c700 1 --2- 172.21.15.140:0/3578990298 >> v2:172.21.15.152:3300/0 conn(0x7f2da005cda0 0x7f2da0146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da600b700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0060c00 0x7f2da009cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184122463 0 0) 0x7f2da0060fe0 con 0x7f2da0061410 2022-07-21T01:48:40.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d88003260 con 0x7f2da0061410 2022-07-21T01:48:40.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 203412519 0 0) 0x7f2d88003260 con 0x7f2da0061410 2022-07-21T01:48:40.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0060c00 msgr2=0x7f2da009cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:40.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0060c00 0x7f2da009cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 >> v2:172.21.15.152:3300/0 conn(0x7f2da005cda0 msgr2=0x7f2da0146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:40.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 --2- 172.21.15.140:0/3578990298 >> v2:172.21.15.152:3300/0 conn(0x7f2da005cda0 0x7f2da0146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da009dbc0 con 0x7f2da0061410 2022-07-21T01:48:40.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f2d9c001660 con 0x7f2da0061410 2022-07-21T01:48:40.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 >> v1:172.21.15.140:6789/0 conn(0x7f2da0061410 legacy=0x7f2da0146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0060c00 0x7f2d88003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d88003c80 0x7f2d88004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d880049f0 0x7f2d88004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00b39c0 con 0x7f2d880049f0 2022-07-21T01:48:40.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8800b750 con 0x7f2d88003c80 2022-07-21T01:48:40.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8800b890 con 0x7f2da0060c00 2022-07-21T01:48:40.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da680c700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d880049f0 0x7f2d88004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da580a700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d88003c80 0x7f2d88004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da600b700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0060c00 0x7f2d88003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da680c700 1 -- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0060c00 msgr2=0x7f2d88003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:40.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da680c700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0060c00 0x7f2d88003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da680c700 1 -- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d88003c80 msgr2=0x7f2d88004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:40.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da680c700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d88003c80 0x7f2d88004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da680c700 1 -- 172.21.15.140:0/3578990298 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d88003260 con 0x7f2d880049f0 2022-07-21T01:48:40.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da600b700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0060c00 0x7f2d88003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:40.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da680c700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d880049f0 0x7f2d88004e70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f2d880054d0 tx=0x7f2d980091d0).ready entity=mon.0 client_cookie=6df55311d26df23d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:40.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d9800e7e0 con 0x7f2d880049f0 2022-07-21T01:48:40.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9800e940 con 0x7f2d880049f0 2022-07-21T01:48:40.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da5009700 1 -- 172.21.15.140:0/3578990298 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d9800eca0 con 0x7f2d880049f0 2022-07-21T01:48:40.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da826f700 1 -- 172.21.15.140:0/3578990298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d880049f0 msgr2=0x7f2d88004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da826f700 1 --2- 172.21.15.140:0/3578990298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d880049f0 0x7f2d88004e70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f2d880054d0 tx=0x7f2d980091d0).stop 2022-07-21T01:48:40.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da680c700 1 -- 172.21.15.140:0/3578990298 reap_dead start 2022-07-21T01:48:40.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da826f700 1 -- 172.21.15.140:0/3578990298 shutdown_connections 2022-07-21T01:48:40.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da826f700 1 -- 172.21.15.140:0/3578990298 >> 172.21.15.140:0/3578990298 conn(0x7f2da00b5f30 msgr2=0x7f2da00b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:40.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da826f700 1 -- 172.21.15.140:0/3578990298 shutdown_connections 2022-07-21T01:48:40.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.519+0000 7f2da826f700 1 -- 172.21.15.140:0/3578990298 wait complete. 2022-07-21T01:48:40.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da826f700 1 Processor -- start 2022-07-21T01:48:40.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da826f700 1 -- start start 2022-07-21T01:48:40.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da826f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0141750 0x7f2da013f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da826f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2da0145980 0x7f2da00af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da826f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2da00afa10 0x7f2da00b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da826f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00af0c0 con 0x7f2da00afa10 2022-07-21T01:48:40.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da826f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0141b30 con 0x7f2da0145980 2022-07-21T01:48:40.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da826f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0141010 con 0x7f2da0141750 2022-07-21T01:48:40.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da680c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2da00afa10 0x7f2da00b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da680c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2da00afa10 0x7f2da00b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60942/0 (socket says 172.21.15.140:60942) 2022-07-21T01:48:40.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da600b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0141750 0x7f2da013f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da680c700 1 -- 172.21.15.140:0/4265701428 learned_addr learned my addr 172.21.15.140:0/4265701428 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:40.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da580a700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2da0145980 0x7f2da00af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da680c700 1 -- 172.21.15.140:0/4265701428 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0141750 msgr2=0x7f2da013f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da680c700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0141750 0x7f2da013f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da680c700 1 -- 172.21.15.140:0/4265701428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2da0145980 msgr2=0x7f2da00af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da680c700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2da0145980 0x7f2da00af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da680c700 1 -- 172.21.15.140:0/4265701428 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da0140d70 con 0x7f2da00afa10 2022-07-21T01:48:40.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da680c700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2da00afa10 0x7f2da00b2210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f2da01411a0 tx=0x7f2d9801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:40.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2d8f7fe700 1 -- 172.21.15.140:0/4265701428 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d9800e7e0 con 0x7f2da00afa10 2022-07-21T01:48:40.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2d8f7fe700 1 -- 172.21.15.140:0/4265701428 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9800eeb0 con 0x7f2da00afa10 2022-07-21T01:48:40.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da826f700 1 -- 172.21.15.140:0/4265701428 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da013f8e0 con 0x7f2da00afa10 2022-07-21T01:48:40.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2d8f7fe700 1 -- 172.21.15.140:0/4265701428 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d98009600 con 0x7f2da00afa10 2022-07-21T01:48:40.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.523+0000 7f2da826f700 1 -- 172.21.15.140:0/4265701428 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da0145600 con 0x7f2da00afa10 2022-07-21T01:48:40.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.527+0000 7f2da826f700 1 -- 172.21.15.140:0/4265701428 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da009de60 con 0x7f2da00afa10 2022-07-21T01:48:40.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.527+0000 7f2d8f7fe700 1 -- 172.21.15.140:0/4265701428 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2d9801b850 con 0x7f2da00afa10 2022-07-21T01:48:40.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.527+0000 7f2d8f7fe700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d80053980 0x7f2d80055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.527+0000 7f2d8f7fe700 1 -- 172.21.15.140:0/4265701428 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2d9806dd50 con 0x7f2da00afa10 2022-07-21T01:48:40.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.527+0000 7f2da600b700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d80053980 0x7f2d80055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.527+0000 7f2da600b700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d80053980 0x7f2d80055e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f2d9000b000 tx=0x7f2d90008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:40.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.535+0000 7f2d8f7fe700 1 -- 172.21.15.140:0/4265701428 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d98044ae0 con 0x7f2da00afa10 2022-07-21T01:48:40.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2da826f700 1 -- 172.21.15.140:0/4265701428 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2da00574e0 con 0x7f2da00afa10 2022-07-21T01:48:40.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2d8f7fe700 1 -- 172.21.15.140:0/4265701428 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f2d98016020 con 0x7f2da00afa10 2022-07-21T01:48:40.703 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:40.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2da826f700 1 -- 172.21.15.140:0/4265701428 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d80053980 msgr2=0x7f2d80055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2da826f700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d80053980 0x7f2d80055e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f2d9000b000 tx=0x7f2d90008040).stop 2022-07-21T01:48:40.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2da826f700 1 -- 172.21.15.140:0/4265701428 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2da00afa10 msgr2=0x7f2da00b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2da826f700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2da00afa10 0x7f2da00b2210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f2da01411a0 tx=0x7f2d9801a000).stop 2022-07-21T01:48:40.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2da826f700 1 -- 172.21.15.140:0/4265701428 shutdown_connections 2022-07-21T01:48:40.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2da826f700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d80053980 0x7f2d80055e40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2da826f700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2da00afa10 0x7f2da00b2210 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2da826f700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2da0145980 0x7f2da00af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2da826f700 1 --2- 172.21.15.140:0/4265701428 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2da0141750 0x7f2da013f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.703+0000 7f2da826f700 1 -- 172.21.15.140:0/4265701428 >> 172.21.15.140:0/4265701428 conn(0x7f2da00b5f30 msgr2=0x7f2da00b83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:40.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.707+0000 7f2da826f700 1 -- 172.21.15.140:0/4265701428 shutdown_connections 2022-07-21T01:48:40.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.707+0000 7f2da826f700 1 -- 172.21.15.140:0/4265701428 wait complete. 2022-07-21T01:48:40.715 INFO:teuthology.orchestra.run.smithi140.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2022-07-21T01:48:40.715 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2022-07-21T01:48:40.716 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:40.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa27fff700 1 Processor -- start 2022-07-21T01:48:40.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa27fff700 1 -- start start 2022-07-21T01:48:40.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa27fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faa280af8e0 0x7faa280b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa27fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa28147010 con 0x7faa280b5650 2022-07-21T01:48:40.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa27fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa281472b0 con 0x7faa280adff0 2022-07-21T01:48:40.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa27fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa28146d40 con 0x7faa280af8e0 2022-07-21T01:48:40.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa26ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7faa280b5650 0x7faa28147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46862/0 (socket says 172.21.15.140:46862) 2022-07-21T01:48:40.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa277fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faa280af8e0 0x7faa280b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa26ffd700 1 -- 172.21.15.140:0/945870100 learned_addr learned my addr 172.21.15.140:0/945870100 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:40.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa277fe700 1 -- 172.21.15.140:0/945870100 >> v1:172.21.15.152:6789/0 conn(0x7faa280adff0 legacy=0x7faa2805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:40.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa277fe700 1 -- 172.21.15.140:0/945870100 >> v1:172.21.15.140:6789/0 conn(0x7faa280b5650 legacy=0x7faa28147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa277fe700 1 -- 172.21.15.140:0/945870100 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2805c4a0 con 0x7faa280af8e0 2022-07-21T01:48:40.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa25ffb700 1 -- 172.21.15.140:0/945870100 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502873710 0 0) 0x7faa28147010 con 0x7faa280b5650 2022-07-21T01:48:40.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa277fe700 1 --2- 172.21.15.140:0/945870100 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa280af8e0 0x7faa280b5a30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7faa280622d0 tx=0x7faa18006df0).ready entity=mon.0 client_cookie=fa294d12f264d33b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:40.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa25ffb700 1 -- 172.21.15.140:0/945870100 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1800d5c0 con 0x7faa280af8e0 2022-07-21T01:48:40.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa25ffb700 1 -- 172.21.15.140:0/945870100 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa18000e90 con 0x7faa280af8e0 2022-07-21T01:48:40.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa25ffb700 1 -- 172.21.15.140:0/945870100 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1800d910 con 0x7faa280af8e0 2022-07-21T01:48:40.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa27fff700 1 -- 172.21.15.140:0/945870100 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa280af8e0 msgr2=0x7faa280b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa27fff700 1 --2- 172.21.15.140:0/945870100 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa280af8e0 0x7faa280b5a30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7faa280622d0 tx=0x7faa18006df0).stop 2022-07-21T01:48:40.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa27fff700 1 -- 172.21.15.140:0/945870100 shutdown_connections 2022-07-21T01:48:40.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa27fff700 1 --2- 172.21.15.140:0/945870100 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa280af8e0 0x7faa280b5a30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.823+0000 7faa27fff700 1 -- 172.21.15.140:0/945870100 >> 172.21.15.140:0/945870100 conn(0x7faa280634e0 msgr2=0x7faa280b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:40.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 -- 172.21.15.140:0/945870100 shutdown_connections 2022-07-21T01:48:40.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 -- 172.21.15.140:0/945870100 wait complete. 2022-07-21T01:48:40.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 Processor -- start 2022-07-21T01:48:40.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 -- start start 2022-07-21T01:48:40.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faa280adff0 0x7faa280646d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa280af8e0 0x7faa28143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa280b5650 0x7faa28067050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa28111190 con 0x7faa280af8e0 2022-07-21T01:48:40.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa28143e00 con 0x7faa280b5650 2022-07-21T01:48:40.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa28110c50 con 0x7faa280adff0 2022-07-21T01:48:40.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa267fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa280af8e0 0x7faa28143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa267fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa280af8e0 0x7faa28143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60958/0 (socket says 172.21.15.140:60958) 2022-07-21T01:48:40.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa267fc700 1 -- 172.21.15.140:0/855412178 learned_addr learned my addr 172.21.15.140:0/855412178 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:40.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa277fe700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa280b5650 0x7faa28067050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa26ffd700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faa280adff0 0x7faa280646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa267fc700 1 -- 172.21.15.140:0/855412178 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faa280adff0 msgr2=0x7faa280646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa267fc700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faa280adff0 0x7faa280646d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa267fc700 1 -- 172.21.15.140:0/855412178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa280b5650 msgr2=0x7faa28067050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa267fc700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa280b5650 0x7faa28067050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa267fc700 1 -- 172.21.15.140:0/855412178 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1800b180 con 0x7faa280af8e0 2022-07-21T01:48:40.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa267fc700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa280af8e0 0x7faa28143480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7faa1c001ff0 tx=0x7faa1c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:40.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa17fff700 1 -- 172.21.15.140:0/855412178 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1c007030 con 0x7faa280af8e0 2022-07-21T01:48:40.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa17fff700 1 -- 172.21.15.140:0/855412178 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c007190 con 0x7faa280af8e0 2022-07-21T01:48:40.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa17fff700 1 -- 172.21.15.140:0/855412178 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1c006560 con 0x7faa280af8e0 2022-07-21T01:48:40.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 -- 172.21.15.140:0/855412178 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa28110ef0 con 0x7faa280af8e0 2022-07-21T01:48:40.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.827+0000 7faa27fff700 1 -- 172.21.15.140:0/855412178 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa28063990 con 0x7faa280af8e0 2022-07-21T01:48:40.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.831+0000 7faa27fff700 1 -- 172.21.15.140:0/855412178 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa280574e0 con 0x7faa280af8e0 2022-07-21T01:48:40.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.831+0000 7faa17fff700 1 -- 172.21.15.140:0/855412178 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faa1c010030 con 0x7faa280af8e0 2022-07-21T01:48:40.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.831+0000 7faa17fff700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa040539e0 0x7faa04055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:40.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.831+0000 7faa17fff700 1 -- 172.21.15.140:0/855412178 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faa1c06bac0 con 0x7faa280af8e0 2022-07-21T01:48:40.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.831+0000 7faa26ffd700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa040539e0 0x7faa04055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:40.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.831+0000 7faa26ffd700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa040539e0 0x7faa04055ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7faa2000d000 tx=0x7faa20001380).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:40.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.839+0000 7faa17fff700 1 -- 172.21.15.140:0/855412178 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faa1c0429e0 con 0x7faa280af8e0 2022-07-21T01:48:40.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.963+0000 7faa27fff700 1 -- 172.21.15.140:0/855412178 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa28067ea0 con 0x7faa280af8e0 2022-07-21T01:48:40.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.963+0000 7faa17fff700 1 -- 172.21.15.140:0/855412178 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7faa1c046000 con 0x7faa280af8e0 2022-07-21T01:48:40.967 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:40.968 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:40.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 -- 172.21.15.140:0/855412178 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa040539e0 msgr2=0x7faa04055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa040539e0 0x7faa04055ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7faa2000d000 tx=0x7faa20001380).stop 2022-07-21T01:48:40.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 -- 172.21.15.140:0/855412178 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa280af8e0 msgr2=0x7faa28143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:40.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa280af8e0 0x7faa28143480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7faa1c001ff0 tx=0x7faa1c007c50).stop 2022-07-21T01:48:40.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 -- 172.21.15.140:0/855412178 shutdown_connections 2022-07-21T01:48:40.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa040539e0 0x7faa04055ea0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa280b5650 0x7faa28067050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa280af8e0 0x7faa28143480 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 --2- 172.21.15.140:0/855412178 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faa280adff0 0x7faa280646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:40.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 -- 172.21.15.140:0/855412178 >> 172.21.15.140:0/855412178 conn(0x7faa280634e0 msgr2=0x7faa28142730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:40.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 -- 172.21.15.140:0/855412178 shutdown_connections 2022-07-21T01:48:40.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:40.967+0000 7faa27fff700 1 -- 172.21.15.140:0/855412178 wait complete. 2022-07-21T01:48:40.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:48:29.344690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:48:30.427054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:48:40.982 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:48:41.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.095+0000 7f87ed326700 1 Processor -- start 2022-07-21T01:48:41.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.095+0000 7f87ed326700 1 -- start start 2022-07-21T01:48:41.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.095+0000 7f87ed326700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87e80b9da0 0x7f87e81468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.095+0000 7f87ed326700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f87e81419a0 0x7f87e8141e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.095+0000 7f87ed326700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e80ba180 con 0x7f87e80b3cf0 2022-07-21T01:48:41.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.095+0000 7f87ed326700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e80ad5b0 con 0x7f87e81419a0 2022-07-21T01:48:41.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.095+0000 7f87ed326700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e81461b0 con 0x7f87e80b9da0 2022-07-21T01:48:41.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.095+0000 7f87ecb25700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f87e81419a0 0x7f87e8141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.095+0000 7f87ecb25700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f87e81419a0 0x7f87e8141e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60966/0 (socket says 172.21.15.140:60966) 2022-07-21T01:48:41.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.095+0000 7f87ecb25700 1 -- 172.21.15.140:0/2943527358 learned_addr learned my addr 172.21.15.140:0/2943527358 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:41.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87e7fff700 1 --2- 172.21.15.140:0/2943527358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87e80b9da0 0x7f87e81468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ecb25700 1 -- 172.21.15.140:0/2943527358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87e80b9da0 msgr2=0x7f87e81468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:41.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ecb25700 1 --2- 172.21.15.140:0/2943527358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87e80b9da0 0x7f87e81468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ecb25700 1 -- 172.21.15.140:0/2943527358 >> v1:172.21.15.152:6789/0 conn(0x7f87e80b3cf0 legacy=0x7f87e8142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ecb25700 1 -- 172.21.15.140:0/2943527358 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e8146450 con 0x7f87e81419a0 2022-07-21T01:48:41.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87e7fff700 1 --2- 172.21.15.140:0/2943527358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87e80b9da0 0x7f87e81468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:41.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ecb25700 1 --2- 172.21.15.140:0/2943527358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87e81419a0 0x7f87e8141e50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f87e00032f0 tx=0x7f87e0009290).ready entity=mon.0 client_cookie=b12a25a8c77720c5 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87e6ffd700 1 -- 172.21.15.140:0/2943527358 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87e000b8a0 con 0x7f87e81419a0 2022-07-21T01:48:41.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87e6ffd700 1 -- 172.21.15.140:0/2943527358 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e000ba00 con 0x7f87e81419a0 2022-07-21T01:48:41.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87e6ffd700 1 -- 172.21.15.140:0/2943527358 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87e000bb70 con 0x7f87e81419a0 2022-07-21T01:48:41.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ed326700 1 -- 172.21.15.140:0/2943527358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87e81419a0 msgr2=0x7f87e8141e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ed326700 1 --2- 172.21.15.140:0/2943527358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87e81419a0 0x7f87e8141e50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f87e00032f0 tx=0x7f87e0009290).stop 2022-07-21T01:48:41.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ed326700 1 -- 172.21.15.140:0/2943527358 shutdown_connections 2022-07-21T01:48:41.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ed326700 1 --2- 172.21.15.140:0/2943527358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87e81419a0 0x7f87e8141e50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ed326700 1 --2- 172.21.15.140:0/2943527358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87e80b9da0 0x7f87e81468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ed326700 1 -- 172.21.15.140:0/2943527358 >> 172.21.15.140:0/2943527358 conn(0x7f87e80678c0 msgr2=0x7f87e80b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:41.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ed326700 1 -- 172.21.15.140:0/2943527358 shutdown_connections 2022-07-21T01:48:41.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ed326700 1 -- 172.21.15.140:0/2943527358 wait complete. 2022-07-21T01:48:41.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ed326700 1 Processor -- start 2022-07-21T01:48:41.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.099+0000 7f87ed326700 1 -- start start 2022-07-21T01:48:41.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87ed326700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87e80b3cf0 0x7f87e80afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87ed326700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87e80b9da0 0x7f87e80b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87ed326700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e81419a0 0x7f87e80b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87ed326700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80b0000 con 0x7f87e80b9da0 2022-07-21T01:48:41.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87ed326700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80b1110 con 0x7f87e81419a0 2022-07-21T01:48:41.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87ed326700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e8113290 con 0x7f87e80b3cf0 2022-07-21T01:48:41.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87e80b9da0 0x7f87e80b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87e80b9da0 0x7f87e80b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60970/0 (socket says 172.21.15.140:60970) 2022-07-21T01:48:41.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e77fe700 1 -- 172.21.15.140:0/473035409 learned_addr learned my addr 172.21.15.140:0/473035409 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:41.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87ecb25700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e81419a0 0x7f87e80b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e7fff700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87e80b3cf0 0x7f87e80afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e77fe700 1 -- 172.21.15.140:0/473035409 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87e80b3cf0 msgr2=0x7f87e80afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e77fe700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87e80b3cf0 0x7f87e80afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e77fe700 1 -- 172.21.15.140:0/473035409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e81419a0 msgr2=0x7f87e80b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e77fe700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e81419a0 0x7f87e80b1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e77fe700 1 -- 172.21.15.140:0/473035409 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e0009530 con 0x7f87e80b9da0 2022-07-21T01:48:41.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87ecb25700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e81419a0 0x7f87e80b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:41.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e77fe700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87e80b9da0 0x7f87e80b16b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f87e80af020 tx=0x7f87d8007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e57fa700 1 -- 172.21.15.140:0/473035409 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87d800ce30 con 0x7f87e80b9da0 2022-07-21T01:48:41.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e57fa700 1 -- 172.21.15.140:0/473035409 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d800f470 con 0x7f87e80b9da0 2022-07-21T01:48:41.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87ed326700 1 -- 172.21.15.140:0/473035409 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e8113530 con 0x7f87e80b9da0 2022-07-21T01:48:41.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87e57fa700 1 -- 172.21.15.140:0/473035409 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87d800f7d0 con 0x7f87e80b9da0 2022-07-21T01:48:41.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.103+0000 7f87ed326700 1 -- 172.21.15.140:0/473035409 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e80b0220 con 0x7f87e80b9da0 2022-07-21T01:48:41.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.107+0000 7f87ed326700 1 -- 172.21.15.140:0/473035409 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e80af910 con 0x7f87e80b9da0 2022-07-21T01:48:41.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.107+0000 7f87e57fa700 1 -- 172.21.15.140:0/473035409 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f87d8019030 con 0x7f87e80b9da0 2022-07-21T01:48:41.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.107+0000 7f87e57fa700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87c80539e0 0x7f87c8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.107+0000 7f87e57fa700 1 -- 172.21.15.140:0/473035409 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f87d806b330 con 0x7f87e80b9da0 2022-07-21T01:48:41.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.107+0000 7f87e7fff700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87c80539e0 0x7f87c8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.107+0000 7f87e7fff700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87c80539e0 0x7f87c8055ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f87dc00c000 tx=0x7f87dc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.111+0000 7f87e57fa700 1 -- 172.21.15.140:0/473035409 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f87d8042250 con 0x7f87e80b9da0 2022-07-21T01:48:41.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.239+0000 7f87ed326700 1 -- 172.21.15.140:0/473035409 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f87e81130d0 con 0x7f87e80b9da0 2022-07-21T01:48:41.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.239+0000 7f87e57fa700 1 -- 172.21.15.140:0/473035409 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f87d8045870 con 0x7f87e80b9da0 2022-07-21T01:48:41.240 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:48:41.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 -- 172.21.15.140:0/473035409 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87c80539e0 msgr2=0x7f87c8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87c80539e0 0x7f87c8055ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f87dc00c000 tx=0x7f87dc008040).stop 2022-07-21T01:48:41.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 -- 172.21.15.140:0/473035409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87e80b9da0 msgr2=0x7f87e80b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87e80b9da0 0x7f87e80b16b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f87e80af020 tx=0x7f87d8007c20).stop 2022-07-21T01:48:41.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 -- 172.21.15.140:0/473035409 shutdown_connections 2022-07-21T01:48:41.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87c80539e0 0x7f87c8055ea0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e81419a0 0x7f87e80b1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87e80b9da0 0x7f87e80b16b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 --2- 172.21.15.140:0/473035409 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87e80b3cf0 0x7f87e80afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 -- 172.21.15.140:0/473035409 >> 172.21.15.140:0/473035409 conn(0x7f87e80678c0 msgr2=0x7f87e8141db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:41.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 -- 172.21.15.140:0/473035409 shutdown_connections 2022-07-21T01:48:41.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.243+0000 7f87ed326700 1 -- 172.21.15.140:0/473035409 wait complete. 2022-07-21T01:48:41.252 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:48:41.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35e169700 1 Processor -- start 2022-07-21T01:48:41.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35e169700 1 -- start start 2022-07-21T01:48:41.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35e169700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc3580b59d0 0x7fc3580b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35e169700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc3580af920 0x7fc3580ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35e169700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3580adf70 con 0x7fc35805d0c0 2022-07-21T01:48:41.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35e169700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3580b5db0 con 0x7fc3580af920 2022-07-21T01:48:41.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35e169700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc358146300 con 0x7fc3580b59d0 2022-07-21T01:48:41.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35c966700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc3580af920 0x7fc3580ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35c966700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc3580af920 0x7fc3580ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60976/0 (socket says 172.21.15.140:60976) 2022-07-21T01:48:41.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35c966700 1 -- 172.21.15.140:0/328154007 learned_addr learned my addr 172.21.15.140:0/328154007 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:41.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35d167700 1 --2- 172.21.15.140:0/328154007 >> v2:172.21.15.152:3300/0 conn(0x7fc3580b59d0 0x7fc3580b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35c966700 1 -- 172.21.15.140:0/328154007 >> v2:172.21.15.152:3300/0 conn(0x7fc3580b59d0 msgr2=0x7fc3580b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:41.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35c966700 1 --2- 172.21.15.140:0/328154007 >> v2:172.21.15.152:3300/0 conn(0x7fc3580b59d0 0x7fc3580b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35c966700 1 -- 172.21.15.140:0/328154007 >> v1:172.21.15.152:6789/0 conn(0x7fc35805d0c0 legacy=0x7fc3580ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:41.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35c966700 1 -- 172.21.15.140:0/328154007 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc358146060 con 0x7fc3580af920 2022-07-21T01:48:41.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35d167700 1 --2- 172.21.15.140:0/328154007 >> v2:172.21.15.152:3300/0 conn(0x7fc3580b59d0 0x7fc3580b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:41.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc35c966700 1 --2- 172.21.15.140:0/328154007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3580af920 0x7fc3580ae3d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fc3580639a0 tx=0x7fc34c006eb0).ready entity=mon.0 client_cookie=ce1ebf3850c093c0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc34bfff700 1 -- 172.21.15.140:0/328154007 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc34c00d5c0 con 0x7fc3580af920 2022-07-21T01:48:41.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc34bfff700 1 -- 172.21.15.140:0/328154007 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c000e90 con 0x7fc3580af920 2022-07-21T01:48:41.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.363+0000 7fc34bfff700 1 -- 172.21.15.140:0/328154007 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc34c00d910 con 0x7fc3580af920 2022-07-21T01:48:41.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 -- 172.21.15.140:0/328154007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3580af920 msgr2=0x7fc3580ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 --2- 172.21.15.140:0/328154007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3580af920 0x7fc3580ae3d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fc3580639a0 tx=0x7fc34c006eb0).stop 2022-07-21T01:48:41.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 -- 172.21.15.140:0/328154007 shutdown_connections 2022-07-21T01:48:41.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 --2- 172.21.15.140:0/328154007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3580af920 0x7fc3580ae3d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 --2- 172.21.15.140:0/328154007 >> v2:172.21.15.152:3300/0 conn(0x7fc3580b59d0 0x7fc3580b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 -- 172.21.15.140:0/328154007 >> 172.21.15.140:0/328154007 conn(0x7fc3580634c0 msgr2=0x7fc3580b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:41.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 -- 172.21.15.140:0/328154007 shutdown_connections 2022-07-21T01:48:41.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 -- 172.21.15.140:0/328154007 wait complete. 2022-07-21T01:48:41.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 Processor -- start 2022-07-21T01:48:41.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 -- start start 2022-07-21T01:48:41.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc35805d0c0 0x7fc3581130b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3580af920 0x7fc3581135a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3580b59d0 0x7fc358155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc358113d40 con 0x7fc3580af920 2022-07-21T01:48:41.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc358112ec0 con 0x7fc3580b59d0 2022-07-21T01:48:41.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35e169700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc358112c20 con 0x7fc35805d0c0 2022-07-21T01:48:41.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35c966700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3580af920 0x7fc3581135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35c966700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3580af920 0x7fc3581135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60984/0 (socket says 172.21.15.140:60984) 2022-07-21T01:48:41.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35c966700 1 -- 172.21.15.140:0/1407018952 learned_addr learned my addr 172.21.15.140:0/1407018952 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:41.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35d167700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc35805d0c0 0x7fc3581130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.367+0000 7fc35d968700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3580b59d0 0x7fc358155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc35c966700 1 -- 172.21.15.140:0/1407018952 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc35805d0c0 msgr2=0x7fc3581130b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc35c966700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc35805d0c0 0x7fc3581130b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc35c966700 1 -- 172.21.15.140:0/1407018952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3580b59d0 msgr2=0x7fc358155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc35c966700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3580b59d0 0x7fc358155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc35c966700 1 -- 172.21.15.140:0/1407018952 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c00b1e0 con 0x7fc3580af920 2022-07-21T01:48:41.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc35d167700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc35805d0c0 0x7fc3581130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:41.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc35c966700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3580af920 0x7fc3581135a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fc3580639a0 tx=0x7fc34c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc34a7fc700 1 -- 172.21.15.140:0/1407018952 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc34c011930 con 0x7fc3580af920 2022-07-21T01:48:41.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc34a7fc700 1 -- 172.21.15.140:0/1407018952 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c011a90 con 0x7fc3580af920 2022-07-21T01:48:41.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc35e169700 1 -- 172.21.15.140:0/1407018952 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc358113a90 con 0x7fc3580af920 2022-07-21T01:48:41.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc34a7fc700 1 -- 172.21.15.140:0/1407018952 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc34c011df0 con 0x7fc3580af920 2022-07-21T01:48:41.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc35e169700 1 -- 172.21.15.140:0/1407018952 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc358066ce0 con 0x7fc3580af920 2022-07-21T01:48:41.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc35e169700 1 -- 172.21.15.140:0/1407018952 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3580673e0 con 0x7fc3580af920 2022-07-21T01:48:41.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc34a7fc700 1 -- 172.21.15.140:0/1407018952 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc34c025030 con 0x7fc3580af920 2022-07-21T01:48:41.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc34a7fc700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3400539e0 0x7fc340055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.371+0000 7fc35d167700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3400539e0 0x7fc340055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.375+0000 7fc34a7fc700 1 -- 172.21.15.140:0/1407018952 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc34c06d1b0 con 0x7fc3580af920 2022-07-21T01:48:41.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.375+0000 7fc35d167700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3400539e0 0x7fc340055ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fc35400b000 tx=0x7fc354008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.379+0000 7fc34a7fc700 1 -- 172.21.15.140:0/1407018952 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc34c0440d0 con 0x7fc3580af920 2022-07-21T01:48:41.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.503+0000 7fc35e169700 1 -- 172.21.15.140:0/1407018952 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc3580af2d0 con 0x7fc3580af920 2022-07-21T01:48:41.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.503+0000 7fc34a7fc700 1 -- 172.21.15.140:0/1407018952 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc34c023020 con 0x7fc3580af920 2022-07-21T01:48:41.505 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:41.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 -- 172.21.15.140:0/1407018952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3400539e0 msgr2=0x7fc340055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3400539e0 0x7fc340055ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fc35400b000 tx=0x7fc354008040).stop 2022-07-21T01:48:41.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 -- 172.21.15.140:0/1407018952 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3580af920 msgr2=0x7fc3581135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3580af920 0x7fc3581135a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fc3580639a0 tx=0x7fc34c015000).stop 2022-07-21T01:48:41.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 -- 172.21.15.140:0/1407018952 shutdown_connections 2022-07-21T01:48:41.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3400539e0 0x7fc340055ea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3580b59d0 0x7fc358155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3580af920 0x7fc3581135a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 --2- 172.21.15.140:0/1407018952 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc35805d0c0 0x7fc3581130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 -- 172.21.15.140:0/1407018952 >> 172.21.15.140:0/1407018952 conn(0x7fc3580634c0 msgr2=0x7fc358140310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:41.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 -- 172.21.15.140:0/1407018952 shutdown_connections 2022-07-21T01:48:41.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.507+0000 7fc35e169700 1 -- 172.21.15.140:0/1407018952 wait complete. 2022-07-21T01:48:41.522 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:48:41.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbfa55ad700 1 Processor -- start 2022-07-21T01:48:41.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbfa55ad700 1 -- start start 2022-07-21T01:48:41.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbfa55ad700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa00b5b50 0x7fbfa00accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbfa55ad700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfa00b0480 0x7fbfa014a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbfa55ad700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa00ac5e0 con 0x7fbfa014ab60 2022-07-21T01:48:41.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbfa55ad700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa00b5f30 con 0x7fbfa00b0480 2022-07-21T01:48:41.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbfa55ad700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa005cf70 con 0x7fbfa00b5b50 2022-07-21T01:48:41.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9effd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa00b5b50 0x7fbfa00accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9e7fc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfa00b0480 0x7fbfa014a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9effd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa00b5b50 0x7fbfa00accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:57720/0 (socket says 172.21.15.140:57720) 2022-07-21T01:48:41.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9effd700 1 -- 172.21.15.140:0/2376911703 learned_addr learned my addr 172.21.15.140:0/2376911703 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:41.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9e7fc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbfa00b0480 0x7fbfa014a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:38498/0 (socket says 172.21.15.140:38498) 2022-07-21T01:48:41.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9effd700 1 -- 172.21.15.140:0/2376911703 >> v2:172.21.15.152:3300/0 conn(0x7fbfa00b0480 msgr2=0x7fbfa014a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:41.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9effd700 1 --2- 172.21.15.140:0/2376911703 >> v2:172.21.15.152:3300/0 conn(0x7fbfa00b0480 0x7fbfa014a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9effd700 1 -- 172.21.15.140:0/2376911703 >> v1:172.21.15.152:6789/0 conn(0x7fbfa014ab60 legacy=0x7fbfa0144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9effd700 1 -- 172.21.15.140:0/2376911703 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa00ac810 con 0x7fbfa00b5b50 2022-07-21T01:48:41.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9effd700 1 --2- 172.21.15.140:0/2376911703 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa00b5b50 0x7fbfa00accb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fbf900032f0 tx=0x7fbf90009290).ready entity=mon.2 client_cookie=2f76044f9c52a176 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9dffb700 1 -- 172.21.15.140:0/2376911703 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf9000b8a0 con 0x7fbfa00b5b50 2022-07-21T01:48:41.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9dffb700 1 -- 172.21.15.140:0/2376911703 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9000ba00 con 0x7fbfa00b5b50 2022-07-21T01:48:41.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.627+0000 7fbf9dffb700 1 -- 172.21.15.140:0/2376911703 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf9000bb70 con 0x7fbfa00b5b50 2022-07-21T01:48:41.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 -- 172.21.15.140:0/2376911703 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa00b5b50 msgr2=0x7fbfa00accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 --2- 172.21.15.140:0/2376911703 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa00b5b50 0x7fbfa00accb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fbf900032f0 tx=0x7fbf90009290).stop 2022-07-21T01:48:41.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 -- 172.21.15.140:0/2376911703 shutdown_connections 2022-07-21T01:48:41.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 --2- 172.21.15.140:0/2376911703 >> v2:172.21.15.152:3300/0 conn(0x7fbfa00b0480 0x7fbfa014a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 --2- 172.21.15.140:0/2376911703 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa00b5b50 0x7fbfa00accb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 -- 172.21.15.140:0/2376911703 >> 172.21.15.140:0/2376911703 conn(0x7fbfa00635e0 msgr2=0x7fbfa00b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:41.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 -- 172.21.15.140:0/2376911703 shutdown_connections 2022-07-21T01:48:41.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 -- 172.21.15.140:0/2376911703 wait complete. 2022-07-21T01:48:41.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 Processor -- start 2022-07-21T01:48:41.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 -- start start 2022-07-21T01:48:41.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbfa00b0480 0x7fbfa0142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa00b5b50 0x7fbfa0159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa014ab60 0x7fbfa0148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa015a180 con 0x7fbfa00b0480 2022-07-21T01:48:41.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa014a0c0 con 0x7fbfa00b5b50 2022-07-21T01:48:41.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbfa55ad700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa014a360 con 0x7fbfa014ab60 2022-07-21T01:48:41.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbf9effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbfa00b0480 0x7fbfa0142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbf9effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbfa00b0480 0x7fbfa0142b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:60996/0 (socket says 172.21.15.140:60996) 2022-07-21T01:48:41.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbf9effd700 1 -- 172.21.15.140:0/1328018720 learned_addr learned my addr 172.21.15.140:0/1328018720 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:41.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbf9e7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa00b5b50 0x7fbfa0159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.631+0000 7fbf9f7fe700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa014ab60 0x7fbfa0148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf9effd700 1 -- 172.21.15.140:0/1328018720 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa014ab60 msgr2=0x7fbfa0148f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf9effd700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa014ab60 0x7fbfa0148f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf9effd700 1 -- 172.21.15.140:0/1328018720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa00b5b50 msgr2=0x7fbfa0159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf9effd700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa00b5b50 0x7fbfa0159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf9effd700 1 -- 172.21.15.140:0/1328018720 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf90009530 con 0x7fbfa00b0480 2022-07-21T01:48:41.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf9e7fc700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa00b5b50 0x7fbfa0159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:41.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf9effd700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbfa00b0480 0x7fbfa0142b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fbf900032f0 tx=0x7fbf90016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf97fff700 1 -- 172.21.15.140:0/1328018720 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf9000b8a0 con 0x7fbfa00b0480 2022-07-21T01:48:41.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf97fff700 1 -- 172.21.15.140:0/1328018720 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf90021440 con 0x7fbfa00b0480 2022-07-21T01:48:41.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbfa55ad700 1 -- 172.21.15.140:0/1328018720 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa00659b0 con 0x7fbfa00b0480 2022-07-21T01:48:41.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf97fff700 1 -- 172.21.15.140:0/1328018720 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf900217a0 con 0x7fbfa00b0480 2022-07-21T01:48:41.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbfa55ad700 1 -- 172.21.15.140:0/1328018720 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa01493f0 con 0x7fbfa00b0480 2022-07-21T01:48:41.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbfa55ad700 1 -- 172.21.15.140:0/1328018720 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa0065b40 con 0x7fbfa00b0480 2022-07-21T01:48:41.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf97fff700 1 -- 172.21.15.140:0/1328018720 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbf9000f070 con 0x7fbfa00b0480 2022-07-21T01:48:41.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf97fff700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf84053b00 0x7fbf84055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.635+0000 7fbf97fff700 1 -- 172.21.15.140:0/1328018720 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbf9006d950 con 0x7fbfa00b0480 2022-07-21T01:48:41.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.639+0000 7fbf9e7fc700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf84053b00 0x7fbf84055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.639+0000 7fbf9e7fc700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf84053b00 0x7fbf84055fc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fbf8800b000 tx=0x7fbf88006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.643+0000 7fbf97fff700 1 -- 172.21.15.140:0/1328018720 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf90044760 con 0x7fbfa00b0480 2022-07-21T01:48:41.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.763+0000 7fbfa55ad700 1 -- 172.21.15.140:0/1328018720 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbfa00574e0 con 0x7fbfa00b0480 2022-07-21T01:48:41.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.763+0000 7fbf97fff700 1 -- 172.21.15.140:0/1328018720 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbf90015070 con 0x7fbfa00b0480 2022-07-21T01:48:41.764 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:41.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 -- 172.21.15.140:0/1328018720 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf84053b00 msgr2=0x7fbf84055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf84053b00 0x7fbf84055fc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fbf8800b000 tx=0x7fbf88006df0).stop 2022-07-21T01:48:41.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 -- 172.21.15.140:0/1328018720 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbfa00b0480 msgr2=0x7fbfa0142b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbfa00b0480 0x7fbfa0142b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fbf900032f0 tx=0x7fbf90016000).stop 2022-07-21T01:48:41.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 -- 172.21.15.140:0/1328018720 shutdown_connections 2022-07-21T01:48:41.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf84053b00 0x7fbf84055fc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbfa014ab60 0x7fbfa0148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbfa00b5b50 0x7fbfa0159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 --2- 172.21.15.140:0/1328018720 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbfa00b0480 0x7fbfa0142b50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 -- 172.21.15.140:0/1328018720 >> 172.21.15.140:0/1328018720 conn(0x7fbfa00635e0 msgr2=0x7fbfa0145010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:41.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 -- 172.21.15.140:0/1328018720 shutdown_connections 2022-07-21T01:48:41.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.767+0000 7fbfa55ad700 1 -- 172.21.15.140:0/1328018720 wait complete. 2022-07-21T01:48:41.777 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:48:41.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.883+0000 7f476759e700 1 Processor -- start 2022-07-21T01:48:41.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.883+0000 7f476759e700 1 -- start start 2022-07-21T01:48:41.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f47680b2ec0 0x7f47680b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476759e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f476805d160 0x7f4768142210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476759e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47680b3f10 con 0x7f4768142700 2022-07-21T01:48:41.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476759e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4768142b90 con 0x7f476805d160 2022-07-21T01:48:41.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47680b4ea0 con 0x7f47680b2ec0 2022-07-21T01:48:41.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f4766d9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4768142700 0x7f47681461c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46920/0 (socket says 172.21.15.140:46920) 2022-07-21T01:48:41.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f4766d9d700 1 -- 172.21.15.140:0/3446536978 learned_addr learned my addr 172.21.15.140:0/3446536978 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:41.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476659c700 1 --2- 172.21.15.140:0/3446536978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f47680b2ec0 0x7f47680b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f4765d9b700 1 --2- 172.21.15.140:0/3446536978 >> v2:172.21.15.152:3300/0 conn(0x7f476805d160 0x7f4768142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476559a700 1 -- 172.21.15.140:0/3446536978 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948532987 0 0) 0x7f47680b3f10 con 0x7f4768142700 2022-07-21T01:48:41.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476559a700 1 -- 172.21.15.140:0/3446536978 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f474c003260 con 0x7f4768142700 2022-07-21T01:48:41.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476659c700 1 -- 172.21.15.140:0/3446536978 >> v2:172.21.15.152:3300/0 conn(0x7f476805d160 msgr2=0x7f4768142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:41.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476659c700 1 --2- 172.21.15.140:0/3446536978 >> v2:172.21.15.152:3300/0 conn(0x7f476805d160 0x7f4768142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476659c700 1 -- 172.21.15.140:0/3446536978 >> v1:172.21.15.140:6789/0 conn(0x7f4768142700 legacy=0x7f47681461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476659c700 1 -- 172.21.15.140:0/3446536978 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47680b3790 con 0x7f47680b2ec0 2022-07-21T01:48:41.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f4765d9b700 1 --2- 172.21.15.140:0/3446536978 >> v2:172.21.15.152:3300/0 conn(0x7f476805d160 0x7f4768142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:41.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476659c700 1 --2- 172.21.15.140:0/3446536978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f47680b2ec0 0x7f47680b32a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f47600032f0 tx=0x7f47600091d0).ready entity=mon.2 client_cookie=ba7ad20bc0ffa7f1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476559a700 1 -- 172.21.15.140:0/3446536978 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f476000b600 con 0x7f47680b2ec0 2022-07-21T01:48:41.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476559a700 1 -- 172.21.15.140:0/3446536978 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476000b760 con 0x7f47680b2ec0 2022-07-21T01:48:41.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476559a700 1 -- 172.21.15.140:0/3446536978 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f476000b8d0 con 0x7f47680b2ec0 2022-07-21T01:48:41.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476759e700 1 -- 172.21.15.140:0/3446536978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f47680b2ec0 msgr2=0x7f47680b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476759e700 1 --2- 172.21.15.140:0/3446536978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f47680b2ec0 0x7f47680b32a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f47600032f0 tx=0x7f47600091d0).stop 2022-07-21T01:48:41.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476759e700 1 -- 172.21.15.140:0/3446536978 shutdown_connections 2022-07-21T01:48:41.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476759e700 1 --2- 172.21.15.140:0/3446536978 >> v2:172.21.15.152:3300/0 conn(0x7f476805d160 0x7f4768142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476759e700 1 --2- 172.21.15.140:0/3446536978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f47680b2ec0 0x7f47680b32a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.887+0000 7f476759e700 1 -- 172.21.15.140:0/3446536978 >> 172.21.15.140:0/3446536978 conn(0x7f4768066e20 msgr2=0x7f4768067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:41.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476759e700 1 -- 172.21.15.140:0/3446536978 shutdown_connections 2022-07-21T01:48:41.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476759e700 1 -- 172.21.15.140:0/3446536978 wait complete. 2022-07-21T01:48:41.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476759e700 1 Processor -- start 2022-07-21T01:48:41.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476759e700 1 -- start start 2022-07-21T01:48:41.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f476805d160 0x7f476813ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f47680b2ec0 0x7f4768152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4768142700 0x7f4768156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47680ae940 con 0x7f4768142700 2022-07-21T01:48:41.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47680ad650 con 0x7f476805d160 2022-07-21T01:48:41.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47680ad8f0 con 0x7f47680b2ec0 2022-07-21T01:48:41.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f476805d160 0x7f476813ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f4766d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4768142700 0x7f4768156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f476805d160 0x7f476813ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:38518/0 (socket says 172.21.15.140:38518) 2022-07-21T01:48:41.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f476659c700 1 -- 172.21.15.140:0/2593248652 learned_addr learned my addr 172.21.15.140:0/2593248652 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:41.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f4765d9b700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f47680b2ec0 0x7f4768152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f4766d9d700 1 -- 172.21.15.140:0/2593248652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f47680b2ec0 msgr2=0x7f4768152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.891+0000 7f4766d9d700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f47680b2ec0 0x7f4768152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f4766d9d700 1 -- 172.21.15.140:0/2593248652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f476805d160 msgr2=0x7f476813ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:41.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f4766d9d700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f476805d160 0x7f476813ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:41.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f4766d9d700 1 -- 172.21.15.140:0/2593248652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c00b180 con 0x7f4768142700 2022-07-21T01:48:41.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f476659c700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f476805d160 0x7f476813ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:41.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f4766d9d700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4768142700 0x7f4768156170 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f4758001110 tx=0x7f47580079c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f47577fe700 1 -- 172.21.15.140:0/2593248652 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f475800cdb0 con 0x7f4768142700 2022-07-21T01:48:41.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f47577fe700 1 -- 172.21.15.140:0/2593248652 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47580070d0 con 0x7f4768142700 2022-07-21T01:48:41.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f476759e700 1 -- 172.21.15.140:0/2593248652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4760009470 con 0x7f4768142700 2022-07-21T01:48:41.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f47577fe700 1 -- 172.21.15.140:0/2593248652 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4758007430 con 0x7f4768142700 2022-07-21T01:48:41.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f476759e700 1 -- 172.21.15.140:0/2593248652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4768152b70 con 0x7f4768142700 2022-07-21T01:48:41.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f476759e700 1 -- 172.21.15.140:0/2593248652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47680b2620 con 0x7f4768142700 2022-07-21T01:48:41.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f47577fe700 1 -- 172.21.15.140:0/2593248652 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4758010070 con 0x7f4768142700 2022-07-21T01:48:41.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f47577fe700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4744053a70 0x7f4744055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:41.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f47577fe700 1 -- 172.21.15.140:0/2593248652 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f475806b520 con 0x7f4768142700 2022-07-21T01:48:41.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.895+0000 7f476659c700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4744053a70 0x7f4744055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:41.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.899+0000 7f476659c700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4744053a70 0x7f4744055f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f476001b000 tx=0x7f4760016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:41.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:41.903+0000 7f47577fe700 1 -- 172.21.15.140:0/2593248652 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4758042330 con 0x7f4768142700 2022-07-21T01:48:42.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.027+0000 7f476759e700 1 -- 172.21.15.140:0/2593248652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47680574e0 con 0x7f4768142700 2022-07-21T01:48:42.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.027+0000 7f47577fe700 1 -- 172.21.15.140:0/2593248652 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4758045950 con 0x7f4768142700 2022-07-21T01:48:42.029 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:42.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 -- 172.21.15.140:0/2593248652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4744053a70 msgr2=0x7f4744055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4744053a70 0x7f4744055f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f476001b000 tx=0x7f4760016000).stop 2022-07-21T01:48:42.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 -- 172.21.15.140:0/2593248652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4768142700 msgr2=0x7f4768156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4768142700 0x7f4768156170 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f4758001110 tx=0x7f47580079c0).stop 2022-07-21T01:48:42.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 -- 172.21.15.140:0/2593248652 shutdown_connections 2022-07-21T01:48:42.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4744053a70 0x7f4744055f30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4768142700 0x7f4768156170 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f47680b2ec0 0x7f4768152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 --2- 172.21.15.140:0/2593248652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f476805d160 0x7f476813ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 -- 172.21.15.140:0/2593248652 >> 172.21.15.140:0/2593248652 conn(0x7f4768066e20 msgr2=0x7f476805be20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:42.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 -- 172.21.15.140:0/2593248652 shutdown_connections 2022-07-21T01:48:42.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.031+0000 7f476759e700 1 -- 172.21.15.140:0/2593248652 wait complete. 2022-07-21T01:48:42.042 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:48:42.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2cd7a9700 1 Processor -- start 2022-07-21T01:48:42.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2cd7a9700 1 -- start start 2022-07-21T01:48:42.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2cd7a9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2c80b21f0 0x7fd2c80b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2cd7a9700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2c813ffa0 0x7fd2c8143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2cd7a9700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c80ab430 con 0x7fd2c80ac860 2022-07-21T01:48:42.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2cd7a9700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c80ab6d0 con 0x7fd2c813ffa0 2022-07-21T01:48:42.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2cd7a9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80ab970 con 0x7fd2c80b21f0 2022-07-21T01:48:42.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c77fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2c813ffa0 0x7fd2c8143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2be7fc700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd2c80ac860 0x7fd2c8142f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:48046/0 (socket says 172.21.15.140:48046) 2022-07-21T01:48:42.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2be7fc700 1 -- 172.21.15.140:0/921720679 learned_addr learned my addr 172.21.15.140:0/921720679 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:42.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c6ffd700 1 --2- 172.21.15.140:0/921720679 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2c80b21f0 0x7fd2c80b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c67fc700 1 -- 172.21.15.140:0/921720679 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199284972 0 0) 0x7fd2c80ab430 con 0x7fd2c80ac860 2022-07-21T01:48:42.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c67fc700 1 -- 172.21.15.140:0/921720679 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ac003260 con 0x7fd2c80ac860 2022-07-21T01:48:42.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c77fe700 1 -- 172.21.15.140:0/921720679 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2c80b21f0 msgr2=0x7fd2c80b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:42.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c77fe700 1 --2- 172.21.15.140:0/921720679 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2c80b21f0 0x7fd2c80b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c77fe700 1 -- 172.21.15.140:0/921720679 >> v1:172.21.15.152:6789/0 conn(0x7fd2c80ac860 legacy=0x7fd2c8142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c77fe700 1 -- 172.21.15.140:0/921720679 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80ab0b0 con 0x7fd2c813ffa0 2022-07-21T01:48:42.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c6ffd700 1 --2- 172.21.15.140:0/921720679 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2c80b21f0 0x7fd2c80b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:42.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c77fe700 1 --2- 172.21.15.140:0/921720679 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2c813ffa0 0x7fd2c8143690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fd2c00032f0 tx=0x7fd2c00091d0).ready entity=mon.0 client_cookie=8c7050461190b75 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:42.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c67fc700 1 -- 172.21.15.140:0/921720679 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2c000b600 con 0x7fd2c813ffa0 2022-07-21T01:48:42.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c67fc700 1 -- 172.21.15.140:0/921720679 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c000b760 con 0x7fd2c813ffa0 2022-07-21T01:48:42.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2c67fc700 1 -- 172.21.15.140:0/921720679 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2c000b8d0 con 0x7fd2c813ffa0 2022-07-21T01:48:42.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/921720679 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2c813ffa0 msgr2=0x7fd2c8143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.143+0000 7fd2cd7a9700 1 --2- 172.21.15.140:0/921720679 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2c813ffa0 0x7fd2c8143690 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fd2c00032f0 tx=0x7fd2c00091d0).stop 2022-07-21T01:48:42.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/921720679 shutdown_connections 2022-07-21T01:48:42.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 --2- 172.21.15.140:0/921720679 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2c813ffa0 0x7fd2c8143690 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 --2- 172.21.15.140:0/921720679 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2c80b21f0 0x7fd2c80b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/921720679 >> 172.21.15.140:0/921720679 conn(0x7fd2c80b3690 msgr2=0x7fd2c80b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:42.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/921720679 shutdown_connections 2022-07-21T01:48:42.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/921720679 wait complete. 2022-07-21T01:48:42.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 Processor -- start 2022-07-21T01:48:42.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 -- start start 2022-07-21T01:48:42.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2c80ac860 0x7fd2c80ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2c80b21f0 0x7fd2c80ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2c813ffa0 0x7fd2c80aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80ae2f0 con 0x7fd2c813ffa0 2022-07-21T01:48:42.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80ae520 con 0x7fd2c80b21f0 2022-07-21T01:48:42.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2cd7a9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8141630 con 0x7fd2c80ac860 2022-07-21T01:48:42.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2c77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2c813ffa0 0x7fd2c80aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2be7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2c80b21f0 0x7fd2c80ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2be7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2c80b21f0 0x7fd2c80ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:38534/0 (socket says 172.21.15.140:38534) 2022-07-21T01:48:42.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2be7fc700 1 -- 172.21.15.140:0/206132381 learned_addr learned my addr 172.21.15.140:0/206132381 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:42.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2c6ffd700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2c80ac860 0x7fd2c80ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2be7fc700 1 -- 172.21.15.140:0/206132381 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2c80ac860 msgr2=0x7fd2c80ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2be7fc700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2c80ac860 0x7fd2c80ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2be7fc700 1 -- 172.21.15.140:0/206132381 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2c813ffa0 msgr2=0x7fd2c80aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2be7fc700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2c813ffa0 0x7fd2c80aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2be7fc700 1 -- 172.21.15.140:0/206132381 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b800b180 con 0x7fd2c80b21f0 2022-07-21T01:48:42.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.147+0000 7fd2be7fc700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2c80b21f0 0x7fd2c80ae9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd2b0001110 tx=0x7fd2b0007ad0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:42.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.151+0000 7fd2c4ff9700 1 -- 172.21.15.140:0/206132381 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2b00070d0 con 0x7fd2c80b21f0 2022-07-21T01:48:42.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.151+0000 7fd2c4ff9700 1 -- 172.21.15.140:0/206132381 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b0007230 con 0x7fd2c80b21f0 2022-07-21T01:48:42.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.151+0000 7fd2c4ff9700 1 -- 172.21.15.140:0/206132381 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2b0005610 con 0x7fd2c80b21f0 2022-07-21T01:48:42.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.151+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/206132381 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c0009470 con 0x7fd2c80b21f0 2022-07-21T01:48:42.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.151+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/206132381 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c8141900 con 0x7fd2c80b21f0 2022-07-21T01:48:42.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.151+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/206132381 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c80574e0 con 0x7fd2c80b21f0 2022-07-21T01:48:42.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.151+0000 7fd2c4ff9700 1 -- 172.21.15.140:0/206132381 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd2b0010070 con 0x7fd2c80b21f0 2022-07-21T01:48:42.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.151+0000 7fd2c4ff9700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2a0053aa0 0x7fd2a0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.151+0000 7fd2c4ff9700 1 -- 172.21.15.140:0/206132381 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd2b006b490 con 0x7fd2c80b21f0 2022-07-21T01:48:42.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.151+0000 7fd2c6ffd700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2a0053aa0 0x7fd2a0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.151+0000 7fd2c6ffd700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2a0053aa0 0x7fd2a0055f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fd2b800b000 tx=0x7fd2b8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:42.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.159+0000 7fd2c4ff9700 1 -- 172.21.15.140:0/206132381 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd2b00423b0 con 0x7fd2c80b21f0 2022-07-21T01:48:42.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.319+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/206132381 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2c80ae6f0 con 0x7fd2c80b21f0 2022-07-21T01:48:42.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.319+0000 7fd2c4ff9700 1 -- 172.21.15.140:0/206132381 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7fd2b0018870 con 0x7fd2c80b21f0 2022-07-21T01:48:42.321 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:42.321 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/665899293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":665899293},{"type":"v1","addr":"172.21.15.140:6835","nonce":665899293}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:48:31.423028+0000","modified":"2022-07-21T01:48:32.427116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.152:6833/1276387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1276387778},{"type":"v1","addr":"172.21.15.152:6833","nonce":1276387778}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-07-21T01:48:42.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/206132381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2a0053aa0 msgr2=0x7fd2a0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2a0053aa0 0x7fd2a0055f60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fd2b800b000 tx=0x7fd2b8006df0).stop 2022-07-21T01:48:42.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/206132381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2c80b21f0 msgr2=0x7fd2c80ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2c80b21f0 0x7fd2c80ae9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fd2b0001110 tx=0x7fd2b0007ad0).stop 2022-07-21T01:48:42.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/206132381 shutdown_connections 2022-07-21T01:48:42.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2a0053aa0 0x7fd2a0055f60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2c813ffa0 0x7fd2c80aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2c80b21f0 0x7fd2c80ae9c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 --2- 172.21.15.140:0/206132381 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2c80ac860 0x7fd2c80ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/206132381 >> 172.21.15.140:0/206132381 conn(0x7fd2c80b3690 msgr2=0x7fd2c8096ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:42.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/206132381 shutdown_connections 2022-07-21T01:48:42.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.323+0000 7fd2cd7a9700 1 -- 172.21.15.140:0/206132381 wait complete. 2022-07-21T01:48:42.330 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 5 2022-07-21T01:48:42.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:48:31.423028+0000', 'modified': '2022-07-21T01:48:32.427116+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.152:6833/1276387778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 1276387778}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 1276387778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:48:42.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:48:42.538 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:48:42.663 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":4282,"want_state":"up:active","state":"up:active","rank_uptime":11.223008694000001,"mdsmap_epoch":5,"osdmap_epoch":22,"osdmap_epoch_barrier":21,"uptime":14.570709299000001} 2022-07-21T01:48:42.664 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 4282, "mdsmap_epoch": 5, "osdmap_epoch": 22, "osdmap_epoch_barrier": 21, "rank_uptime": 11.223008694, "state": "up:active", "uptime": 14.570709299, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:48:42.665 INFO:teuthology.run_tasks:Running task ceph-fuse... 2022-07-21T01:48:42.700 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2022-07-21T01:48:42.701 INFO:tasks.ceph_fuse:config is None 2022-07-21T01:48:42.702 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None, 'client.2': None} 2022-07-21T01:48:42.703 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2022-07-21T01:48:42.704 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:42.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91e011700 1 Processor -- start 2022-07-21T01:48:42.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91e011700 1 -- start start 2022-07-21T01:48:42.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91e011700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9180adff0 0x7fe9180b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91e011700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe9180af8e0 0x7fe91805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91e011700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe918147010 con 0x7fe9180b5650 2022-07-21T01:48:42.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91e011700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9181472b0 con 0x7fe9180af8e0 2022-07-21T01:48:42.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91e011700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe918146d40 con 0x7fe9180adff0 2022-07-21T01:48:42.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91d810700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe9180af8e0 0x7fe91805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91d810700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe9180af8e0 0x7fe91805bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32804/0 (socket says 172.21.15.140:32804) 2022-07-21T01:48:42.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91d810700 1 -- 172.21.15.140:0/1548477199 learned_addr learned my addr 172.21.15.140:0/1548477199 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:42.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91c80e700 1 --2- 172.21.15.140:0/1548477199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9180adff0 0x7fe9180b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91d810700 1 -- 172.21.15.140:0/1548477199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9180adff0 msgr2=0x7fe9180b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:42.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91d810700 1 --2- 172.21.15.140:0/1548477199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9180adff0 0x7fe9180b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91d810700 1 -- 172.21.15.140:0/1548477199 >> v1:172.21.15.152:6789/0 conn(0x7fe9180b5650 legacy=0x7fe918147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91d810700 1 -- 172.21.15.140:0/1548477199 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9180acae0 con 0x7fe9180af8e0 2022-07-21T01:48:42.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91c80e700 1 --2- 172.21.15.140:0/1548477199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9180adff0 0x7fe9180b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:42.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe91d810700 1 --2- 172.21.15.140:0/1548477199 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe9180af8e0 0x7fe91805bd90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe918060b70 tx=0x7fe914006eb0).ready entity=mon.0 client_cookie=565f5ae3c314511a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:42.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.819+0000 7fe90ffff700 1 -- 172.21.15.140:0/1548477199 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe91400d570 con 0x7fe9180af8e0 2022-07-21T01:48:42.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe90ffff700 1 -- 172.21.15.140:0/1548477199 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe914000e90 con 0x7fe9180af8e0 2022-07-21T01:48:42.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe90ffff700 1 -- 172.21.15.140:0/1548477199 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe91400d8c0 con 0x7fe9180af8e0 2022-07-21T01:48:42.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 -- 172.21.15.140:0/1548477199 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe9180af8e0 msgr2=0x7fe91805bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 --2- 172.21.15.140:0/1548477199 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe9180af8e0 0x7fe91805bd90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe918060b70 tx=0x7fe914006eb0).stop 2022-07-21T01:48:42.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 -- 172.21.15.140:0/1548477199 shutdown_connections 2022-07-21T01:48:42.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 --2- 172.21.15.140:0/1548477199 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe9180af8e0 0x7fe91805bd90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 --2- 172.21.15.140:0/1548477199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9180adff0 0x7fe9180b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 -- 172.21.15.140:0/1548477199 >> 172.21.15.140:0/1548477199 conn(0x7fe9180634e0 msgr2=0x7fe9180b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:42.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 -- 172.21.15.140:0/1548477199 shutdown_connections 2022-07-21T01:48:42.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 -- 172.21.15.140:0/1548477199 wait complete. 2022-07-21T01:48:42.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 Processor -- start 2022-07-21T01:48:42.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 -- start start 2022-07-21T01:48:42.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9180adff0 0x7fe918111c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe9180af8e0 0x7fe918110de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9180b5650 0x7fe918142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe918064d60 con 0x7fe9180af8e0 2022-07-21T01:48:42.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91810e130 con 0x7fe9180adff0 2022-07-21T01:48:42.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91e011700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe918063e30 con 0x7fe9180b5650 2022-07-21T01:48:42.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91c80e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe9180af8e0 0x7fe918110de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91c80e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe9180af8e0 0x7fe918110de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32808/0 (socket says 172.21.15.140:32808) 2022-07-21T01:48:42.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91c80e700 1 -- 172.21.15.140:0/228604504 learned_addr learned my addr 172.21.15.140:0/228604504 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:42.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91d00f700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9180adff0 0x7fe918111c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91d810700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9180b5650 0x7fe918142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91c80e700 1 -- 172.21.15.140:0/228604504 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9180b5650 msgr2=0x7fe918142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91c80e700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9180b5650 0x7fe918142da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91c80e700 1 -- 172.21.15.140:0/228604504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9180adff0 msgr2=0x7fe918111c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91c80e700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9180adff0 0x7fe918111c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91c80e700 1 -- 172.21.15.140:0/228604504 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91400b1e0 con 0x7fe9180af8e0 2022-07-21T01:48:42.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.823+0000 7fe91c80e700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe9180af8e0 0x7fe918110de0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fe908007000 tx=0x7fe908008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:42.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.827+0000 7fe90e7fc700 1 -- 172.21.15.140:0/228604504 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe908010580 con 0x7fe9180af8e0 2022-07-21T01:48:42.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.827+0000 7fe90e7fc700 1 -- 172.21.15.140:0/228604504 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9080106e0 con 0x7fe9180af8e0 2022-07-21T01:48:42.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.827+0000 7fe91e011700 1 -- 172.21.15.140:0/228604504 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9180640d0 con 0x7fe9180af8e0 2022-07-21T01:48:42.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.827+0000 7fe90e7fc700 1 -- 172.21.15.140:0/228604504 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe908010a40 con 0x7fe9180af8e0 2022-07-21T01:48:42.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.827+0000 7fe91e011700 1 -- 172.21.15.140:0/228604504 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe918110030 con 0x7fe9180af8e0 2022-07-21T01:48:42.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.827+0000 7fe91e011700 1 -- 172.21.15.140:0/228604504 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9180574e0 con 0x7fe9180af8e0 2022-07-21T01:48:42.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.827+0000 7fe90e7fc700 1 -- 172.21.15.140:0/228604504 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe908010ba0 con 0x7fe9180af8e0 2022-07-21T01:48:42.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.827+0000 7fe90e7fc700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe900053a30 0x7fe900055ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:42.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.827+0000 7fe90e7fc700 1 -- 172.21.15.140:0/228604504 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe90806b7b0 con 0x7fe9180af8e0 2022-07-21T01:48:42.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.827+0000 7fe91d00f700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe900053a30 0x7fe900055ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:42.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.831+0000 7fe91d00f700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe900053a30 0x7fe900055ef0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fe91000d000 tx=0x7fe910007f10).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:42.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.835+0000 7fe90e7fc700 1 -- 172.21.15.140:0/228604504 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe908042670 con 0x7fe9180af8e0 2022-07-21T01:48:42.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.959+0000 7fe91e011700 1 -- 172.21.15.140:0/228604504 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe918067ea0 con 0x7fe9180af8e0 2022-07-21T01:48:42.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe90e7fc700 1 -- 172.21.15.140:0/228604504 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7fe908045c90 con 0x7fe9180af8e0 2022-07-21T01:48:42.961 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:42.962 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:42.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 -- 172.21.15.140:0/228604504 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe900053a30 msgr2=0x7fe900055ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe900053a30 0x7fe900055ef0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fe91000d000 tx=0x7fe910007f10).stop 2022-07-21T01:48:42.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 -- 172.21.15.140:0/228604504 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe9180af8e0 msgr2=0x7fe918110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:42.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe9180af8e0 0x7fe918110de0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fe908007000 tx=0x7fe908008040).stop 2022-07-21T01:48:42.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 -- 172.21.15.140:0/228604504 shutdown_connections 2022-07-21T01:48:42.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe900053a30 0x7fe900055ef0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9180b5650 0x7fe918142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe9180af8e0 0x7fe918110de0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 --2- 172.21.15.140:0/228604504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9180adff0 0x7fe918111c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:42.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 -- 172.21.15.140:0/228604504 >> 172.21.15.140:0/228604504 conn(0x7fe9180634e0 msgr2=0x7fe9180aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:42.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 -- 172.21.15.140:0/228604504 shutdown_connections 2022-07-21T01:48:42.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:42.963+0000 7fe91e011700 1 -- 172.21.15.140:0/228604504 wait complete. 2022-07-21T01:48:42.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:48:29.344690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:48:30.427054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:48:42.978 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:48:43.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.083+0000 7f9693fff700 1 Processor -- start 2022-07-21T01:48:43.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.083+0000 7f9693fff700 1 -- start start 2022-07-21T01:48:43.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f96940b59f0 0x7f96941473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96940af940 0x7f9694147890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f96940b1400 0x7f969414a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969405cc70 con 0x7f96940b59f0 2022-07-21T01:48:43.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96940b5dd0 con 0x7f96940b1400 2022-07-21T01:48:43.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969405c760 con 0x7f96940af940 2022-07-21T01:48:43.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9692ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f96940b59f0 0x7f96941473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9692ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f96940b59f0 0x7f96941473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32816/0 (socket says 172.21.15.140:32816) 2022-07-21T01:48:43.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9692ffd700 1 -- 172.21.15.140:0/201606600 learned_addr learned my addr 172.21.15.140:0/201606600 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:43.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f96927fc700 1 --2- 172.21.15.140:0/201606600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96940af940 0x7f9694147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f96937fe700 1 --2- 172.21.15.140:0/201606600 >> v2:172.21.15.152:3300/0 conn(0x7f96940b1400 0x7f969414a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9692ffd700 1 -- 172.21.15.140:0/201606600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96940af940 msgr2=0x7f9694147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:43.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9692ffd700 1 --2- 172.21.15.140:0/201606600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96940af940 0x7f9694147890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9692ffd700 1 -- 172.21.15.140:0/201606600 >> v2:172.21.15.152:3300/0 conn(0x7f96940b1400 msgr2=0x7f969414a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:43.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9692ffd700 1 --2- 172.21.15.140:0/201606600 >> v2:172.21.15.152:3300/0 conn(0x7f96940b1400 0x7f969414a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9692ffd700 1 -- 172.21.15.140:0/201606600 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9694147d80 con 0x7f96940b59f0 2022-07-21T01:48:43.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9692ffd700 1 --2- 172.21.15.140:0/201606600 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96940b59f0 0x7f96941473a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f968c002f70 tx=0x7f968c006eb0).ready entity=mon.0 client_cookie=c718c237b88de741 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9691ffb700 1 -- 172.21.15.140:0/201606600 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f968c00d570 con 0x7f96940b59f0 2022-07-21T01:48:43.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9691ffb700 1 -- 172.21.15.140:0/201606600 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968c000e90 con 0x7f96940b59f0 2022-07-21T01:48:43.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9691ffb700 1 -- 172.21.15.140:0/201606600 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f968c00d900 con 0x7f96940b59f0 2022-07-21T01:48:43.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 -- 172.21.15.140:0/201606600 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96940b59f0 msgr2=0x7f96941473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 --2- 172.21.15.140:0/201606600 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96940b59f0 0x7f96941473a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f968c002f70 tx=0x7f968c006eb0).stop 2022-07-21T01:48:43.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 -- 172.21.15.140:0/201606600 shutdown_connections 2022-07-21T01:48:43.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 --2- 172.21.15.140:0/201606600 >> v2:172.21.15.152:3300/0 conn(0x7f96940b1400 0x7f969414a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 --2- 172.21.15.140:0/201606600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96940af940 0x7f9694147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 --2- 172.21.15.140:0/201606600 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96940b59f0 0x7f96941473a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 -- 172.21.15.140:0/201606600 >> 172.21.15.140:0/201606600 conn(0x7f9694063480 msgr2=0x7f96940b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:43.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 -- 172.21.15.140:0/201606600 shutdown_connections 2022-07-21T01:48:43.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.087+0000 7f9693fff700 1 -- 172.21.15.140:0/201606600 wait complete. 2022-07-21T01:48:43.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9693fff700 1 Processor -- start 2022-07-21T01:48:43.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9693fff700 1 -- start start 2022-07-21T01:48:43.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9693fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96940af940 0x7f9694063d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9693fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96940b1400 0x7f969413d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9693fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96940b59f0 0x7f969413da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9693fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9694110480 con 0x7f96940b59f0 2022-07-21T01:48:43.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9693fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969410e920 con 0x7f96940af940 2022-07-21T01:48:43.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9693fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969410ebc0 con 0x7f96940b1400 2022-07-21T01:48:43.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f96937fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96940b59f0 0x7f969413da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f96937fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96940b59f0 0x7f969413da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32824/0 (socket says 172.21.15.140:32824) 2022-07-21T01:48:43.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f96937fe700 1 -- 172.21.15.140:0/3006662112 learned_addr learned my addr 172.21.15.140:0/3006662112 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:43.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f96927fc700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96940b1400 0x7f969413d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9692ffd700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96940af940 0x7f9694063d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f96937fe700 1 -- 172.21.15.140:0/3006662112 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96940b1400 msgr2=0x7f969413d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f96937fe700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96940b1400 0x7f969413d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f96937fe700 1 -- 172.21.15.140:0/3006662112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96940af940 msgr2=0x7f9694063d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f96937fe700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96940af940 0x7f9694063d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f96937fe700 1 -- 172.21.15.140:0/3006662112 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c00b1e0 con 0x7f96940b59f0 2022-07-21T01:48:43.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9692ffd700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96940af940 0x7f9694063d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:43.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f96937fe700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96940b59f0 0x7f969413da50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f968400c000 tx=0x7f9684006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f968bfff700 1 -- 172.21.15.140:0/3006662112 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9684010780 con 0x7f96940b59f0 2022-07-21T01:48:43.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f968bfff700 1 -- 172.21.15.140:0/3006662112 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96840108e0 con 0x7f96940b59f0 2022-07-21T01:48:43.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9693fff700 1 -- 172.21.15.140:0/3006662112 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969410ee60 con 0x7f96940b59f0 2022-07-21T01:48:43.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f968bfff700 1 -- 172.21.15.140:0/3006662112 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9684010c40 con 0x7f96940b59f0 2022-07-21T01:48:43.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.091+0000 7f9693fff700 1 -- 172.21.15.140:0/3006662112 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9694066ae0 con 0x7f96940b59f0 2022-07-21T01:48:43.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.095+0000 7f9693fff700 1 -- 172.21.15.140:0/3006662112 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969410eff0 con 0x7f96940b59f0 2022-07-21T01:48:43.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.095+0000 7f968bfff700 1 -- 172.21.15.140:0/3006662112 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f968401c460 con 0x7f96940b59f0 2022-07-21T01:48:43.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.095+0000 7f968bfff700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f966c053aa0 0x7f966c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.095+0000 7f968bfff700 1 -- 172.21.15.140:0/3006662112 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f968406c360 con 0x7f96940b59f0 2022-07-21T01:48:43.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.095+0000 7f9692ffd700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f966c053aa0 0x7f966c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.095+0000 7f9692ffd700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f966c053aa0 0x7f966c055f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f968c01a000 tx=0x7f968c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.099+0000 7f968bfff700 1 -- 172.21.15.140:0/3006662112 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9684043220 con 0x7f96940b59f0 2022-07-21T01:48:43.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.223+0000 7f9693fff700 1 -- 172.21.15.140:0/3006662112 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f969405c9d0 con 0x7f96940b59f0 2022-07-21T01:48:43.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.223+0000 7f968bfff700 1 -- 172.21.15.140:0/3006662112 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9684046840 con 0x7f96940b59f0 2022-07-21T01:48:43.227 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:48:43.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 -- 172.21.15.140:0/3006662112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f966c053aa0 msgr2=0x7f966c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f966c053aa0 0x7f966c055f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f968c01a000 tx=0x7f968c015000).stop 2022-07-21T01:48:43.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 -- 172.21.15.140:0/3006662112 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96940b59f0 msgr2=0x7f969413da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96940b59f0 0x7f969413da50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f968400c000 tx=0x7f9684006df0).stop 2022-07-21T01:48:43.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 -- 172.21.15.140:0/3006662112 shutdown_connections 2022-07-21T01:48:43.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f966c053aa0 0x7f966c055f60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f96940b59f0 0x7f969413da50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f96940b1400 0x7f969413d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 --2- 172.21.15.140:0/3006662112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96940af940 0x7f9694063d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 -- 172.21.15.140:0/3006662112 >> 172.21.15.140:0/3006662112 conn(0x7f9694063480 msgr2=0x7f96940b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:43.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 -- 172.21.15.140:0/3006662112 shutdown_connections 2022-07-21T01:48:43.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.227+0000 7f9693fff700 1 -- 172.21.15.140:0/3006662112 wait complete. 2022-07-21T01:48:43.253 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:48:43.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db359e700 1 Processor -- start 2022-07-21T01:48:43.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db359e700 1 -- start start 2022-07-21T01:48:43.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db359e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6db40b5920 0x7f6db413fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db359e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6db40b4ee0 0x7f6db4142530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db359e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db40b5d00 con 0x7f6db40ab990 2022-07-21T01:48:43.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db359e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db413f090 con 0x7f6db40b4ee0 2022-07-21T01:48:43.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db359e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db413f330 con 0x7f6db40b5920 2022-07-21T01:48:43.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db1d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6db40b4ee0 0x7f6db4142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db1d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6db40b4ee0 0x7f6db4142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32830/0 (socket says 172.21.15.140:32830) 2022-07-21T01:48:43.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db1d9b700 1 -- 172.21.15.140:0/2705682020 learned_addr learned my addr 172.21.15.140:0/2705682020 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:43.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db259c700 1 --2- 172.21.15.140:0/2705682020 >> v2:172.21.15.152:3300/0 conn(0x7f6db40b5920 0x7f6db413fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db1d9b700 1 -- 172.21.15.140:0/2705682020 >> v2:172.21.15.152:3300/0 conn(0x7f6db40b5920 msgr2=0x7f6db413fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:43.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db1d9b700 1 --2- 172.21.15.140:0/2705682020 >> v2:172.21.15.152:3300/0 conn(0x7f6db40b5920 0x7f6db413fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db1d9b700 1 -- 172.21.15.140:0/2705682020 >> v1:172.21.15.152:6789/0 conn(0x7f6db40ab990 legacy=0x7f6db413e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db1d9b700 1 -- 172.21.15.140:0/2705682020 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db413f5d0 con 0x7f6db40b4ee0 2022-07-21T01:48:43.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.375+0000 7f6db1d9b700 1 --2- 172.21.15.140:0/2705682020 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6db40b4ee0 0x7f6db4142530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f6d9c002f70 tx=0x7f6d9c006eb0).ready entity=mon.0 client_cookie=574821eaa4366f0f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db159a700 1 -- 172.21.15.140:0/2705682020 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d9c00d5c0 con 0x7f6db40b4ee0 2022-07-21T01:48:43.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db159a700 1 -- 172.21.15.140:0/2705682020 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c000e90 con 0x7f6db40b4ee0 2022-07-21T01:48:43.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db159a700 1 -- 172.21.15.140:0/2705682020 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d9c00d950 con 0x7f6db40b4ee0 2022-07-21T01:48:43.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 -- 172.21.15.140:0/2705682020 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6db40b4ee0 msgr2=0x7f6db4142530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 --2- 172.21.15.140:0/2705682020 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6db40b4ee0 0x7f6db4142530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f6d9c002f70 tx=0x7f6d9c006eb0).stop 2022-07-21T01:48:43.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 -- 172.21.15.140:0/2705682020 shutdown_connections 2022-07-21T01:48:43.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 --2- 172.21.15.140:0/2705682020 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6db40b4ee0 0x7f6db4142530 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 --2- 172.21.15.140:0/2705682020 >> v2:172.21.15.152:3300/0 conn(0x7f6db40b5920 0x7f6db413fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 -- 172.21.15.140:0/2705682020 >> 172.21.15.140:0/2705682020 conn(0x7f6db40632b0 msgr2=0x7f6db4063690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:43.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 -- 172.21.15.140:0/2705682020 shutdown_connections 2022-07-21T01:48:43.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 -- 172.21.15.140:0/2705682020 wait complete. 2022-07-21T01:48:43.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 Processor -- start 2022-07-21T01:48:43.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 -- start start 2022-07-21T01:48:43.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6db40ab990 0x7f6db40b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6db40b4ee0 0x7f6db4145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6db40b5920 0x7f6db4145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db41406d0 con 0x7f6db40ab990 2022-07-21T01:48:43.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4140970 con 0x7f6db40b4ee0 2022-07-21T01:48:43.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4140c10 con 0x7f6db40b5920 2022-07-21T01:48:43.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.379+0000 7f6db259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6db40ab990 0x7f6db40b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6db40ab990 0x7f6db40b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32832/0 (socket says 172.21.15.140:32832) 2022-07-21T01:48:43.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db259c700 1 -- 172.21.15.140:0/4178523018 learned_addr learned my addr 172.21.15.140:0/4178523018 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:43.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db1d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6db40b4ee0 0x7f6db4145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db2d9d700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6db40b5920 0x7f6db4145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db1d9b700 1 -- 172.21.15.140:0/4178523018 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6db40b5920 msgr2=0x7f6db4145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db1d9b700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6db40b5920 0x7f6db4145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db1d9b700 1 -- 172.21.15.140:0/4178523018 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6db40ab990 msgr2=0x7f6db40b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db1d9b700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6db40ab990 0x7f6db40b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db1d9b700 1 -- 172.21.15.140:0/4178523018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c00b1e0 con 0x7f6db40b4ee0 2022-07-21T01:48:43.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db2d9d700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6db40b5920 0x7f6db4145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:43.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db1d9b700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6db40b4ee0 0x7f6db4145720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f6d9c002f70 tx=0x7f6d9c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6dab7fe700 1 -- 172.21.15.140:0/4178523018 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d9c011860 con 0x7f6db40b4ee0 2022-07-21T01:48:43.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6dab7fe700 1 -- 172.21.15.140:0/4178523018 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c0119c0 con 0x7f6db40b4ee0 2022-07-21T01:48:43.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db359e700 1 -- 172.21.15.140:0/4178523018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db4140eb0 con 0x7f6db40b4ee0 2022-07-21T01:48:43.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6dab7fe700 1 -- 172.21.15.140:0/4178523018 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d9c011d20 con 0x7f6db40b4ee0 2022-07-21T01:48:43.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db359e700 1 -- 172.21.15.140:0/4178523018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db4141350 con 0x7f6db40b4ee0 2022-07-21T01:48:43.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6db359e700 1 -- 172.21.15.140:0/4178523018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db41462a0 con 0x7f6db40b4ee0 2022-07-21T01:48:43.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.383+0000 7f6dab7fe700 1 -- 172.21.15.140:0/4178523018 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6d9c01a040 con 0x7f6db40b4ee0 2022-07-21T01:48:43.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.387+0000 7f6dab7fe700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6d98053980 0x7f6d98055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.387+0000 7f6dab7fe700 1 -- 172.21.15.140:0/4178523018 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6d9c06ef50 con 0x7f6db40b4ee0 2022-07-21T01:48:43.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.387+0000 7f6db259c700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6d98053980 0x7f6d98055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.387+0000 7f6db259c700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6d98053980 0x7f6d98055e40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f6da400c000 tx=0x7f6da400a370).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.395+0000 7f6dab7fe700 1 -- 172.21.15.140:0/4178523018 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6d9c045e70 con 0x7f6db40b4ee0 2022-07-21T01:48:43.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.523+0000 7f6db359e700 1 -- 172.21.15.140:0/4178523018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6db4141040 con 0x7f6db40b4ee0 2022-07-21T01:48:43.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.523+0000 7f6dab7fe700 1 -- 172.21.15.140:0/4178523018 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6d9c049490 con 0x7f6db40b4ee0 2022-07-21T01:48:43.522 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:43.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 -- 172.21.15.140:0/4178523018 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6d98053980 msgr2=0x7f6d98055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6d98053980 0x7f6d98055e40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f6da400c000 tx=0x7f6da400a370).stop 2022-07-21T01:48:43.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 -- 172.21.15.140:0/4178523018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6db40b4ee0 msgr2=0x7f6db4145720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6db40b4ee0 0x7f6db4145720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f6d9c002f70 tx=0x7f6d9c015000).stop 2022-07-21T01:48:43.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 -- 172.21.15.140:0/4178523018 shutdown_connections 2022-07-21T01:48:43.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6d98053980 0x7f6d98055e40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6db40b5920 0x7f6db4145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6db40b4ee0 0x7f6db4145720 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 --2- 172.21.15.140:0/4178523018 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6db40ab990 0x7f6db40b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 -- 172.21.15.140:0/4178523018 >> 172.21.15.140:0/4178523018 conn(0x7f6db40632b0 msgr2=0x7f6db40abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:43.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 -- 172.21.15.140:0/4178523018 shutdown_connections 2022-07-21T01:48:43.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.527+0000 7f6db359e700 1 -- 172.21.15.140:0/4178523018 wait complete. 2022-07-21T01:48:43.541 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:48:43.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.651+0000 7f5e87233700 1 Processor -- start 2022-07-21T01:48:43.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.651+0000 7f5e87233700 1 -- start start 2022-07-21T01:48:43.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.651+0000 7f5e87233700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5e80096810 0x7f5e800ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.651+0000 7f5e87233700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5e800abcb0 0x7f5e8013fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.651+0000 7f5e87233700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e80095e30 con 0x7f5e800b1ba0 2022-07-21T01:48:43.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.651+0000 7f5e87233700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e800960d0 con 0x7f5e800abcb0 2022-07-21T01:48:43.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.651+0000 7f5e87233700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e80096370 con 0x7f5e80096810 2022-07-21T01:48:43.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.651+0000 7f5e86231700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f5e800b1ba0 0x7f5e800ab0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:46986/0 (socket says 172.21.15.140:46986) 2022-07-21T01:48:43.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.651+0000 7f5e86231700 1 -- 172.21.15.140:0/1515604904 learned_addr learned my addr 172.21.15.140:0/1515604904 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:43.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.651+0000 7f5e86a32700 1 --2- 172.21.15.140:0/1515604904 >> v2:172.21.15.140:3300/0 conn(0x7f5e800abcb0 0x7f5e8013fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e85a30700 1 --2- 172.21.15.140:0/1515604904 >> v2:172.21.15.152:3300/0 conn(0x7f5e80096810 0x7f5e800ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e8522f700 1 -- 172.21.15.140:0/1515604904 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357073563 0 0) 0x7f5e80095e30 con 0x7f5e800b1ba0 2022-07-21T01:48:43.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e8522f700 1 -- 172.21.15.140:0/1515604904 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e68003260 con 0x7f5e800b1ba0 2022-07-21T01:48:43.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e86a32700 1 -- 172.21.15.140:0/1515604904 >> v2:172.21.15.152:3300/0 conn(0x7f5e80096810 msgr2=0x7f5e800ab7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:43.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e86a32700 1 --2- 172.21.15.140:0/1515604904 >> v2:172.21.15.152:3300/0 conn(0x7f5e80096810 0x7f5e800ab7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e86a32700 1 -- 172.21.15.140:0/1515604904 >> v1:172.21.15.140:6789/0 conn(0x7f5e800b1ba0 legacy=0x7f5e800ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e86a32700 1 -- 172.21.15.140:0/1515604904 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e80095ab0 con 0x7f5e800abcb0 2022-07-21T01:48:43.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e85a30700 1 --2- 172.21.15.140:0/1515604904 >> v2:172.21.15.152:3300/0 conn(0x7f5e80096810 0x7f5e800ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:43.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e86a32700 1 --2- 172.21.15.140:0/1515604904 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e800abcb0 0x7f5e8013fec0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f5e780032f0 tx=0x7f5e780091d0).ready entity=mon.0 client_cookie=a6dfd524c6c7c0d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e8522f700 1 -- 172.21.15.140:0/1515604904 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e7800b600 con 0x7f5e800abcb0 2022-07-21T01:48:43.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e8522f700 1 -- 172.21.15.140:0/1515604904 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7800b760 con 0x7f5e800abcb0 2022-07-21T01:48:43.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e8522f700 1 -- 172.21.15.140:0/1515604904 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e7800b8d0 con 0x7f5e800abcb0 2022-07-21T01:48:43.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e87233700 1 -- 172.21.15.140:0/1515604904 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e800abcb0 msgr2=0x7f5e8013fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e87233700 1 --2- 172.21.15.140:0/1515604904 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e800abcb0 0x7f5e8013fec0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f5e780032f0 tx=0x7f5e780091d0).stop 2022-07-21T01:48:43.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e87233700 1 -- 172.21.15.140:0/1515604904 shutdown_connections 2022-07-21T01:48:43.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e87233700 1 --2- 172.21.15.140:0/1515604904 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e800abcb0 0x7f5e8013fec0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e87233700 1 --2- 172.21.15.140:0/1515604904 >> v2:172.21.15.152:3300/0 conn(0x7f5e80096810 0x7f5e800ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e87233700 1 -- 172.21.15.140:0/1515604904 >> 172.21.15.140:0/1515604904 conn(0x7f5e800b2af0 msgr2=0x7f5e800b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:43.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e87233700 1 -- 172.21.15.140:0/1515604904 shutdown_connections 2022-07-21T01:48:43.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.655+0000 7f5e87233700 1 -- 172.21.15.140:0/1515604904 wait complete. 2022-07-21T01:48:43.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e87233700 1 Processor -- start 2022-07-21T01:48:43.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e87233700 1 -- start start 2022-07-21T01:48:43.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e87233700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e80096810 0x7f5e800afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e87233700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e800abcb0 0x7f5e8009b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e87233700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e800b1ba0 0x7f5e801511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e87233700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80063cf0 con 0x7f5e800b1ba0 2022-07-21T01:48:43.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e87233700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8009bf30 con 0x7f5e800abcb0 2022-07-21T01:48:43.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e87233700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8009ba40 con 0x7f5e80096810 2022-07-21T01:48:43.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e86a32700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e800b1ba0 0x7f5e801511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e86a32700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e800b1ba0 0x7f5e801511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32848/0 (socket says 172.21.15.140:32848) 2022-07-21T01:48:43.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e86a32700 1 -- 172.21.15.140:0/1389167383 learned_addr learned my addr 172.21.15.140:0/1389167383 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:43.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e86231700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e80096810 0x7f5e800afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e85a30700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e800abcb0 0x7f5e8009b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e86a32700 1 -- 172.21.15.140:0/1389167383 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e80096810 msgr2=0x7f5e800afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e86a32700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e80096810 0x7f5e800afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e86a32700 1 -- 172.21.15.140:0/1389167383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e800abcb0 msgr2=0x7f5e8009b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e86a32700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e800abcb0 0x7f5e8009b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e86a32700 1 -- 172.21.15.140:0/1389167383 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c00b180 con 0x7f5e800b1ba0 2022-07-21T01:48:43.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e86a32700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e800b1ba0 0x7f5e801511d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f5e80153300 tx=0x7f5e78016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e737fe700 1 -- 172.21.15.140:0/1389167383 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e78011c80 con 0x7f5e800b1ba0 2022-07-21T01:48:43.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e737fe700 1 -- 172.21.15.140:0/1389167383 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e78011de0 con 0x7f5e800b1ba0 2022-07-21T01:48:43.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e87233700 1 -- 172.21.15.140:0/1389167383 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e78009470 con 0x7f5e800b1ba0 2022-07-21T01:48:43.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e737fe700 1 -- 172.21.15.140:0/1389167383 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e7801f5e0 con 0x7f5e800b1ba0 2022-07-21T01:48:43.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.659+0000 7f5e87233700 1 -- 172.21.15.140:0/1389167383 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8009bc90 con 0x7f5e800b1ba0 2022-07-21T01:48:43.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.663+0000 7f5e87233700 1 -- 172.21.15.140:0/1389167383 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e80153580 con 0x7f5e800b1ba0 2022-07-21T01:48:43.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.663+0000 7f5e737fe700 1 -- 172.21.15.140:0/1389167383 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e78026030 con 0x7f5e800b1ba0 2022-07-21T01:48:43.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.663+0000 7f5e737fe700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e60053a10 0x7f5e60055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.663+0000 7f5e86231700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e60053a10 0x7f5e60055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.663+0000 7f5e737fe700 1 -- 172.21.15.140:0/1389167383 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e7806cb40 con 0x7f5e800b1ba0 2022-07-21T01:48:43.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.663+0000 7f5e86231700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e60053a10 0x7f5e60055ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5e7400b000 tx=0x7f5e74009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.671+0000 7f5e737fe700 1 -- 172.21.15.140:0/1389167383 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e780439d0 con 0x7f5e800b1ba0 2022-07-21T01:48:43.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.791+0000 7f5e87233700 1 -- 172.21.15.140:0/1389167383 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5e800574e0 con 0x7f5e800b1ba0 2022-07-21T01:48:43.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.791+0000 7f5e737fe700 1 -- 172.21.15.140:0/1389167383 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5e780150a0 con 0x7f5e800b1ba0 2022-07-21T01:48:43.792 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:43.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 -- 172.21.15.140:0/1389167383 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e60053a10 msgr2=0x7f5e60055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e60053a10 0x7f5e60055ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f5e7400b000 tx=0x7f5e74009040).stop 2022-07-21T01:48:43.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 -- 172.21.15.140:0/1389167383 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e800b1ba0 msgr2=0x7f5e801511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e800b1ba0 0x7f5e801511d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f5e80153300 tx=0x7f5e78016000).stop 2022-07-21T01:48:43.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 -- 172.21.15.140:0/1389167383 shutdown_connections 2022-07-21T01:48:43.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e60053a10 0x7f5e60055ed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e800b1ba0 0x7f5e801511d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e800abcb0 0x7f5e8009b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 --2- 172.21.15.140:0/1389167383 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e80096810 0x7f5e800afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 -- 172.21.15.140:0/1389167383 >> 172.21.15.140:0/1389167383 conn(0x7f5e800b2af0 msgr2=0x7f5e8013ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:43.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 -- 172.21.15.140:0/1389167383 shutdown_connections 2022-07-21T01:48:43.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.795+0000 7f5e87233700 1 -- 172.21.15.140:0/1389167383 wait complete. 2022-07-21T01:48:43.810 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:48:43.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673e62e700 1 Processor -- start 2022-07-21T01:48:43.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673e62e700 1 -- start start 2022-07-21T01:48:43.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673e62e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f67381462a0 0x7f6738146680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673e62e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67380988b0 0x7f6738146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673e62e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f67381422c0 0x7f6738142730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673e62e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67380afdb0 con 0x7f67381422c0 2022-07-21T01:48:43.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673e62e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67380b0050 con 0x7f67381462a0 2022-07-21T01:48:43.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673e62e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673805fcd0 con 0x7f67380988b0 2022-07-21T01:48:43.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673cbcb700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f67381422c0 0x7f6738142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673cbcb700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f67381422c0 0x7f6738142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32858/0 (socket says 172.21.15.140:32858) 2022-07-21T01:48:43.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f6737fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f67381462a0 0x7f6738146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f67377fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67380988b0 0x7f6738146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673cbcb700 1 -- 172.21.15.140:0/3291318865 learned_addr learned my addr 172.21.15.140:0/3291318865 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:43.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673cbcb700 1 -- 172.21.15.140:0/3291318865 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67380988b0 msgr2=0x7f6738146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:43.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673cbcb700 1 --2- 172.21.15.140:0/3291318865 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67380988b0 0x7f6738146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673cbcb700 1 -- 172.21.15.140:0/3291318865 >> v2:172.21.15.152:3300/0 conn(0x7f67381462a0 msgr2=0x7f6738146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:43.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673cbcb700 1 --2- 172.21.15.140:0/3291318865 >> v2:172.21.15.152:3300/0 conn(0x7f67381462a0 0x7f6738146680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673cbcb700 1 -- 172.21.15.140:0/3291318865 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67380afae0 con 0x7f67381422c0 2022-07-21T01:48:43.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f6737fff700 1 --2- 172.21.15.140:0/3291318865 >> v2:172.21.15.152:3300/0 conn(0x7f67381462a0 0x7f6738146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:43.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f673cbcb700 1 --2- 172.21.15.140:0/3291318865 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67381422c0 0x7f6738142730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f67300032f0 tx=0x7f6730009290).ready entity=mon.0 client_cookie=f1a2fc69404abcfe server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f6736ffd700 1 -- 172.21.15.140:0/3291318865 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f673000b680 con 0x7f67381422c0 2022-07-21T01:48:43.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f6736ffd700 1 -- 172.21.15.140:0/3291318865 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673000b7e0 con 0x7f67381422c0 2022-07-21T01:48:43.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.919+0000 7f6736ffd700 1 -- 172.21.15.140:0/3291318865 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f673000b950 con 0x7f67381422c0 2022-07-21T01:48:43.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 -- 172.21.15.140:0/3291318865 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67381422c0 msgr2=0x7f6738142730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 --2- 172.21.15.140:0/3291318865 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67381422c0 0x7f6738142730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f67300032f0 tx=0x7f6730009290).stop 2022-07-21T01:48:43.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 -- 172.21.15.140:0/3291318865 shutdown_connections 2022-07-21T01:48:43.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 --2- 172.21.15.140:0/3291318865 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67381422c0 0x7f6738142730 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 --2- 172.21.15.140:0/3291318865 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67380988b0 0x7f6738146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 --2- 172.21.15.140:0/3291318865 >> v2:172.21.15.152:3300/0 conn(0x7f67381462a0 0x7f6738146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 -- 172.21.15.140:0/3291318865 >> 172.21.15.140:0/3291318865 conn(0x7f67380b5f00 msgr2=0x7f67380b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:43.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 -- 172.21.15.140:0/3291318865 shutdown_connections 2022-07-21T01:48:43.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 -- 172.21.15.140:0/3291318865 wait complete. 2022-07-21T01:48:43.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 Processor -- start 2022-07-21T01:48:43.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 -- start start 2022-07-21T01:48:43.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67380988b0 0x7f67380990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67381422c0 0x7f673809c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67381462a0 0x7f673809cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67380b19c0 con 0x7f67380988b0 2022-07-21T01:48:43.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67380b1c60 con 0x7f67381462a0 2022-07-21T01:48:43.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673e62e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67380b1f00 con 0x7f67381422c0 2022-07-21T01:48:43.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f6737fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67380988b0 0x7f67380990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f6737fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67380988b0 0x7f67380990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32862/0 (socket says 172.21.15.140:32862) 2022-07-21T01:48:43.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f6737fff700 1 -- 172.21.15.140:0/741377759 learned_addr learned my addr 172.21.15.140:0/741377759 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:43.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f67377fe700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67381422c0 0x7f673809c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673cbcb700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67381462a0 0x7f673809cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f6737fff700 1 -- 172.21.15.140:0/741377759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67381422c0 msgr2=0x7f673809c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f6737fff700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67381422c0 0x7f673809c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f6737fff700 1 -- 172.21.15.140:0/741377759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67381462a0 msgr2=0x7f673809cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:43.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f6737fff700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67381462a0 0x7f673809cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:43.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f6737fff700 1 -- 172.21.15.140:0/741377759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672000b180 con 0x7f67380988b0 2022-07-21T01:48:43.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.923+0000 7f673cbcb700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67381462a0 0x7f673809cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:43.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.927+0000 7f6737fff700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67380988b0 0x7f67380990e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f672000c000 tx=0x7f6720006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.927+0000 7f67357fa700 1 -- 172.21.15.140:0/741377759 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f672000c860 con 0x7f67380988b0 2022-07-21T01:48:43.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.927+0000 7f67357fa700 1 -- 172.21.15.140:0/741377759 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672000c9c0 con 0x7f67380988b0 2022-07-21T01:48:43.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.927+0000 7f673e62e700 1 -- 172.21.15.140:0/741377759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6730009530 con 0x7f67380988b0 2022-07-21T01:48:43.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.927+0000 7f67357fa700 1 -- 172.21.15.140:0/741377759 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f672000cd20 con 0x7f67380988b0 2022-07-21T01:48:43.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.927+0000 7f673e62e700 1 -- 172.21.15.140:0/741377759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67380b2870 con 0x7f67380988b0 2022-07-21T01:48:43.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.927+0000 7f673e62e700 1 -- 172.21.15.140:0/741377759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67380b20c0 con 0x7f67380988b0 2022-07-21T01:48:43.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.927+0000 7f67357fa700 1 -- 172.21.15.140:0/741377759 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f672001c460 con 0x7f67380988b0 2022-07-21T01:48:43.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.927+0000 7f67357fa700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f671c0539e0 0x7f671c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:43.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.931+0000 7f67357fa700 1 -- 172.21.15.140:0/741377759 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f672006c580 con 0x7f67380988b0 2022-07-21T01:48:43.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.931+0000 7f67377fe700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f671c0539e0 0x7f671c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:43.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.931+0000 7f67377fe700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f671c0539e0 0x7f671c055ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f672800b000 tx=0x7f6728006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:43.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:43.935+0000 7f67357fa700 1 -- 172.21.15.140:0/741377759 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6720043440 con 0x7f67380988b0 2022-07-21T01:48:44.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.059+0000 7f673e62e700 1 -- 172.21.15.140:0/741377759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67380574e0 con 0x7f67380988b0 2022-07-21T01:48:44.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f67357fa700 1 -- 172.21.15.140:0/741377759 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6720046a60 con 0x7f67380988b0 2022-07-21T01:48:44.061 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:44.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 -- 172.21.15.140:0/741377759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f671c0539e0 msgr2=0x7f671c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:44.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f671c0539e0 0x7f671c055ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f672800b000 tx=0x7f6728006df0).stop 2022-07-21T01:48:44.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 -- 172.21.15.140:0/741377759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67380988b0 msgr2=0x7f67380990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:44.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67380988b0 0x7f67380990e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f672000c000 tx=0x7f6720006df0).stop 2022-07-21T01:48:44.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 -- 172.21.15.140:0/741377759 shutdown_connections 2022-07-21T01:48:44.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f671c0539e0 0x7f671c055ea0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67381462a0 0x7f673809cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67381422c0 0x7f673809c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 --2- 172.21.15.140:0/741377759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67380988b0 0x7f67380990e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 -- 172.21.15.140:0/741377759 >> 172.21.15.140:0/741377759 conn(0x7f67380b5f00 msgr2=0x7f6738146680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:44.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 -- 172.21.15.140:0/741377759 shutdown_connections 2022-07-21T01:48:44.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.063+0000 7f673e62e700 1 -- 172.21.15.140:0/741377759 wait complete. 2022-07-21T01:48:44.073 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:48:44.074 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:48:44.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.179+0000 7f4f78a42700 1 Processor -- start 2022-07-21T01:48:44.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.179+0000 7f4f78a42700 1 -- start start 2022-07-21T01:48:44.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f740b1ef0 0x7f4f74142f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:44.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4f7405fa50 0x7f4f7413e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:44.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f74060980 con 0x7f4f740ac3a0 2022-07-21T01:48:44.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f740b22d0 con 0x7f4f7405fa50 2022-07-21T01:48:44.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f74098810 con 0x7f4f740b1ef0 2022-07-21T01:48:44.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f73fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4f7405fa50 0x7f4f7413e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:44.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f72ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4f740ac3a0 0x7f4f74143470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47014/0 (socket says 172.21.15.140:47014) 2022-07-21T01:48:44.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f72ffd700 1 -- 172.21.15.140:0/655190752 learned_addr learned my addr 172.21.15.140:0/655190752 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:44.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f737fe700 1 --2- 172.21.15.140:0/655190752 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f740b1ef0 0x7f4f74142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:44.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f727fc700 1 -- 172.21.15.140:0/655190752 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286744844 0 0) 0x7f4f74060980 con 0x7f4f740ac3a0 2022-07-21T01:48:44.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f727fc700 1 -- 172.21.15.140:0/655190752 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f60003260 con 0x7f4f740ac3a0 2022-07-21T01:48:44.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f73fff700 1 -- 172.21.15.140:0/655190752 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f740b1ef0 msgr2=0x7f4f74142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:44.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f73fff700 1 --2- 172.21.15.140:0/655190752 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f740b1ef0 0x7f4f74142f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f73fff700 1 -- 172.21.15.140:0/655190752 >> v1:172.21.15.140:6789/0 conn(0x7f4f740ac3a0 legacy=0x7f4f74143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:44.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f73fff700 1 -- 172.21.15.140:0/655190752 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f74060bb0 con 0x7f4f7405fa50 2022-07-21T01:48:44.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f73fff700 1 --2- 172.21.15.140:0/655190752 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f7405fa50 0x7f4f7413e9a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f4f68002f70 tx=0x7f4f68006df0).ready entity=mon.0 client_cookie=936b2b3374183a25 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:44.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f727fc700 1 -- 172.21.15.140:0/655190752 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f6800d4a0 con 0x7f4f7405fa50 2022-07-21T01:48:44.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f727fc700 1 -- 172.21.15.140:0/655190752 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f68000e90 con 0x7f4f7405fa50 2022-07-21T01:48:44.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f727fc700 1 -- 172.21.15.140:0/655190752 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f6800d7f0 con 0x7f4f7405fa50 2022-07-21T01:48:44.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 -- 172.21.15.140:0/655190752 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f7405fa50 msgr2=0x7f4f7413e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:44.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 --2- 172.21.15.140:0/655190752 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f7405fa50 0x7f4f7413e9a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f4f68002f70 tx=0x7f4f68006df0).stop 2022-07-21T01:48:44.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 -- 172.21.15.140:0/655190752 shutdown_connections 2022-07-21T01:48:44.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 --2- 172.21.15.140:0/655190752 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f7405fa50 0x7f4f7413e9a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 --2- 172.21.15.140:0/655190752 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f740b1ef0 0x7f4f74142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 -- 172.21.15.140:0/655190752 >> 172.21.15.140:0/655190752 conn(0x7f4f740b2ca0 msgr2=0x7f4f740b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:44.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 -- 172.21.15.140:0/655190752 shutdown_connections 2022-07-21T01:48:44.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.183+0000 7f4f78a42700 1 -- 172.21.15.140:0/655190752 wait complete. 2022-07-21T01:48:44.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f78a42700 1 Processor -- start 2022-07-21T01:48:44.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f78a42700 1 -- start start 2022-07-21T01:48:44.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f78a42700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f7405fa50 0x7f4f740af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:44.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f78a42700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f740ac3a0 0x7f4f7409b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:44.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f78a42700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f740b1ef0 0x7f4f74065690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:44.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f78a42700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7409aa80 con 0x7f4f740ac3a0 2022-07-21T01:48:44.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f78a42700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7409b460 con 0x7f4f740b1ef0 2022-07-21T01:48:44.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f78a42700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740afde0 con 0x7f4f7405fa50 2022-07-21T01:48:44.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f72ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f740ac3a0 0x7f4f7409b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:44.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f72ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f740ac3a0 0x7f4f7409b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32876/0 (socket says 172.21.15.140:32876) 2022-07-21T01:48:44.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f72ffd700 1 -- 172.21.15.140:0/1274250895 learned_addr learned my addr 172.21.15.140:0/1274250895 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:44.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f737fe700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f7405fa50 0x7f4f740af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:44.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f73fff700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f740b1ef0 0x7f4f74065690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:44.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f72ffd700 1 -- 172.21.15.140:0/1274250895 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f7405fa50 msgr2=0x7f4f740af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:44.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f72ffd700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f7405fa50 0x7f4f740af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f72ffd700 1 -- 172.21.15.140:0/1274250895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f740b1ef0 msgr2=0x7f4f74065690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:44.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f72ffd700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f740b1ef0 0x7f4f74065690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f72ffd700 1 -- 172.21.15.140:0/1274250895 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6800b180 con 0x7f4f740ac3a0 2022-07-21T01:48:44.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f73fff700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f740b1ef0 0x7f4f74065690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:44.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f72ffd700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f740ac3a0 0x7f4f7409b900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f4f64001110 tx=0x7f4f64007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:44.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f70ff9700 1 -- 172.21.15.140:0/1274250895 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f640070d0 con 0x7f4f740ac3a0 2022-07-21T01:48:44.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f70ff9700 1 -- 172.21.15.140:0/1274250895 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f64007230 con 0x7f4f740ac3a0 2022-07-21T01:48:44.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f78a42700 1 -- 172.21.15.140:0/1274250895 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f740b4ba0 con 0x7f4f740ac3a0 2022-07-21T01:48:44.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f70ff9700 1 -- 172.21.15.140:0/1274250895 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f64005610 con 0x7f4f740ac3a0 2022-07-21T01:48:44.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.187+0000 7f4f78a42700 1 -- 172.21.15.140:0/1274250895 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f74136270 con 0x7f4f740ac3a0 2022-07-21T01:48:44.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.191+0000 7f4f78a42700 1 -- 172.21.15.140:0/1274250895 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f7409ac80 con 0x7f4f740ac3a0 2022-07-21T01:48:44.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.191+0000 7f4f70ff9700 1 -- 172.21.15.140:0/1274250895 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4f64010070 con 0x7f4f740ac3a0 2022-07-21T01:48:44.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.191+0000 7f4f70ff9700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f54053950 0x7f4f54055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:44.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.191+0000 7f4f70ff9700 1 -- 172.21.15.140:0/1274250895 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4f6406b780 con 0x7f4f740ac3a0 2022-07-21T01:48:44.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.191+0000 7f4f737fe700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f54053950 0x7f4f54055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:44.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.191+0000 7f4f737fe700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f54053950 0x7f4f54055e10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4f6c00b000 tx=0x7f4f6c0091a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:44.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.195+0000 7f4f70ff9700 1 -- 172.21.15.140:0/1274250895 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4f64042510 con 0x7f4f740ac3a0 2022-07-21T01:48:44.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.351+0000 7f4f78a42700 1 -- 172.21.15.140:0/1274250895 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f74062270 con 0x7f4f740ac3a0 2022-07-21T01:48:44.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.351+0000 7f4f70ff9700 1 -- 172.21.15.140:0/1274250895 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7f4f64018070 con 0x7f4f740ac3a0 2022-07-21T01:48:44.351 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:44.351 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/665899293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":665899293},{"type":"v1","addr":"172.21.15.140:6835","nonce":665899293}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:48:31.423028+0000","modified":"2022-07-21T01:48:32.427116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.152:6833/1276387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1276387778},{"type":"v1","addr":"172.21.15.152:6833","nonce":1276387778}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-07-21T01:48:44.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 -- 172.21.15.140:0/1274250895 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f54053950 msgr2=0x7f4f54055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:44.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f54053950 0x7f4f54055e10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4f6c00b000 tx=0x7f4f6c0091a0).stop 2022-07-21T01:48:44.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 -- 172.21.15.140:0/1274250895 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f740ac3a0 msgr2=0x7f4f7409b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:44.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f740ac3a0 0x7f4f7409b900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f4f64001110 tx=0x7f4f64007aa0).stop 2022-07-21T01:48:44.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 -- 172.21.15.140:0/1274250895 shutdown_connections 2022-07-21T01:48:44.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f54053950 0x7f4f54055e10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f740b1ef0 0x7f4f74065690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f740ac3a0 0x7f4f7409b900 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 --2- 172.21.15.140:0/1274250895 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f7405fa50 0x7f4f740af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:44.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 -- 172.21.15.140:0/1274250895 >> 172.21.15.140:0/1274250895 conn(0x7f4f740b2ca0 msgr2=0x7f4f740ab2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:44.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 -- 172.21.15.140:0/1274250895 shutdown_connections 2022-07-21T01:48:44.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:44.355+0000 7f4f78a42700 1 -- 172.21.15.140:0/1274250895 wait complete. 2022-07-21T01:48:44.358 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 5 2022-07-21T01:48:44.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:48:31.423028+0000', 'modified': '2022-07-21T01:48:32.427116+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.152:6833/1276387778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 1276387778}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 1276387778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:48:44.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:48:44.568 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:48:44.684 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":4282,"want_state":"up:active","state":"up:active","rank_uptime":13.249083292,"mdsmap_epoch":5,"osdmap_epoch":22,"osdmap_epoch_barrier":21,"uptime":16.596760578000001} 2022-07-21T01:48:44.685 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 4282, "mdsmap_epoch": 5, "osdmap_epoch": 22, "osdmap_epoch_barrier": 21, "rank_uptime": 13.249083292, "state": "up:active", "uptime": 16.596760578, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:48:44.686 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:48:44.687 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:48:44.689 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:48:44.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:44.690 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:48:44.699 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:48:44.760 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:48:44.760 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:48:44.776 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:48:44.778 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:48:44.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:44.780 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:48:44.786 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:48:44.787 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:48:44.831 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:48:44.849 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:48:44.850 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:48:44.907 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:48:44.908 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:48:44.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:44.910 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:48:44.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:48:44.923+0000 7f870fa49080 -1 init, newargv = 0x558eb7c78720 newargc=9 2022-07-21T01:48:44.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22164]: starting ceph client 2022-07-21T01:48:44.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22164]: starting fuse 2022-07-21T01:48:46.026 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:48:46.045 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:48:46.045 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:48:46.111 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:48:46.112 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:48:46.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:46.114 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:48:46.122 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:48:46.122 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:48:46.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:46.123 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:48:46.124 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:48:46.124 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:48:46.124 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:48:46.125 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:48:46.125 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:48:46.126 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:48:46.127 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:48:46.127 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:48:46.127 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:48:46.128 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:48:46.128 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:48:46.128 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:48:46.129 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:48:46.129 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:48:46.129 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:48:46.130 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:48:46.130 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:48:46.131 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:48:46.131 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:48:46.132 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:48:46.132 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:48:46.132 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:48:46.174 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.22164.asok 2022-07-21T01:48:46.180 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22164.asok 2022-07-21T01:48:46.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:46.181 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22164.asok status 2022-07-21T01:48:46.317 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:48:46.317 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:48:46.317 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:48:46.318 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:48:46.318 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:48:46.318 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:48:46.319 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:48:46.319 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "22164", 2022-07-21T01:48:46.319 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:48:46.319 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:48:46.320 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:48:46.320 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:48:46.320 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 4593, 2022-07-21T01:48:46.320 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:48:46.321 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:48:46.321 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:48:46.321 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 4593 2022-07-21T01:48:46.322 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:48:46.322 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:48:46.322 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:48:46.322 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:48:46.322 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 4048699977 2022-07-21T01:48:46.323 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:48:46.323 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:48:46.323 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:48:46.323 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:48:46.324 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:48:46.324 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 4048699977 2022-07-21T01:48:46.324 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:48:46.324 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.4593 172.21.15.191:0/4048699977", 2022-07-21T01:48:46.325 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/4048699977", 2022-07-21T01:48:46.325 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:48:46.325 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 5, 2022-07-21T01:48:46.325 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 22, 2022-07-21T01:48:46.326 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:48:46.326 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:48:46.326 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:48:46.327 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:48:46.329 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:46.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.443+0000 7f7ed4873700 1 Processor -- start 2022-07-21T01:48:46.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.443+0000 7f7ed4873700 1 -- start start 2022-07-21T01:48:46.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.443+0000 7f7ed4873700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00af8e0 0x7f7ed00b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.443+0000 7f7ed4873700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed0147010 con 0x7f7ed00adff0 2022-07-21T01:48:46.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.443+0000 7f7ed4873700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed01472b0 con 0x7f7ed00b5650 2022-07-21T01:48:46.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.443+0000 7f7ed4873700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed0146d40 con 0x7f7ed00af8e0 2022-07-21T01:48:46.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.443+0000 7f7eceffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f7ed00adff0 0x7f7ed005bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47028/0 (socket says 172.21.15.140:47028) 2022-07-21T01:48:46.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.443+0000 7f7eceffd700 1 -- 172.21.15.140:0/2248150113 learned_addr learned my addr 172.21.15.140:0/2248150113 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:46.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.443+0000 7f7ecffff700 1 --2- 172.21.15.140:0/2248150113 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00af8e0 0x7f7ed00b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.443+0000 7f7ece7fc700 1 -- 172.21.15.140:0/2248150113 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226102397 0 0) 0x7f7ed0147010 con 0x7f7ed00adff0 2022-07-21T01:48:46.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ece7fc700 1 -- 172.21.15.140:0/2248150113 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eb0003260 con 0x7f7ed00adff0 2022-07-21T01:48:46.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ece7fc700 1 -- 172.21.15.140:0/2248150113 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029104292 0 0) 0x7f7ed01472b0 con 0x7f7ed00b5650 2022-07-21T01:48:46.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ece7fc700 1 -- 172.21.15.140:0/2248150113 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ed0147010 con 0x7f7ed00b5650 2022-07-21T01:48:46.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ecffff700 1 -- 172.21.15.140:0/2248150113 >> v1:172.21.15.152:6789/0 conn(0x7f7ed00b5650 legacy=0x7f7ed0147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ecffff700 1 -- 172.21.15.140:0/2248150113 >> v1:172.21.15.140:6789/0 conn(0x7f7ed00adff0 legacy=0x7f7ed005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ecffff700 1 -- 172.21.15.140:0/2248150113 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed005c4a0 con 0x7f7ed00af8e0 2022-07-21T01:48:46.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ecffff700 1 --2- 172.21.15.140:0/2248150113 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00af8e0 0x7f7ed00b5a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7ec80032f0 tx=0x7f7ec80091d0).ready entity=mon.2 client_cookie=137502c3003ceafa server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:46.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ece7fc700 1 -- 172.21.15.140:0/2248150113 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec800b640 con 0x7f7ed00af8e0 2022-07-21T01:48:46.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ece7fc700 1 -- 172.21.15.140:0/2248150113 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec800b7a0 con 0x7f7ed00af8e0 2022-07-21T01:48:46.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ece7fc700 1 -- 172.21.15.140:0/2248150113 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ec800b910 con 0x7f7ed00af8e0 2022-07-21T01:48:46.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 -- 172.21.15.140:0/2248150113 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00af8e0 msgr2=0x7f7ed00b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 --2- 172.21.15.140:0/2248150113 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00af8e0 0x7f7ed00b5a30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7ec80032f0 tx=0x7f7ec80091d0).stop 2022-07-21T01:48:46.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 -- 172.21.15.140:0/2248150113 shutdown_connections 2022-07-21T01:48:46.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 --2- 172.21.15.140:0/2248150113 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00af8e0 0x7f7ed00b5a30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 -- 172.21.15.140:0/2248150113 >> 172.21.15.140:0/2248150113 conn(0x7f7ed00634e0 msgr2=0x7f7ed00b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:46.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 -- 172.21.15.140:0/2248150113 shutdown_connections 2022-07-21T01:48:46.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 -- 172.21.15.140:0/2248150113 wait complete. 2022-07-21T01:48:46.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 Processor -- start 2022-07-21T01:48:46.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 -- start start 2022-07-21T01:48:46.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ed00adff0 0x7f7ed00640f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ed00af8e0 0x7f7ed013d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00b5650 0x7f7ed013daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed013dfe0 con 0x7f7ed00af8e0 2022-07-21T01:48:46.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed0063990 con 0x7f7ed00adff0 2022-07-21T01:48:46.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.447+0000 7f7ed4873700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed0111db0 con 0x7f7ed00b5650 2022-07-21T01:48:46.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eceffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ed00af8e0 0x7f7ed013d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7ecffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00b5650 0x7f7ed013daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eceffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ed00af8e0 0x7f7ed013d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32890/0 (socket says 172.21.15.140:32890) 2022-07-21T01:48:46.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7ecffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00b5650 0x7f7ed013daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:57854/0 (socket says 172.21.15.140:57854) 2022-07-21T01:48:46.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7ecf7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ed00adff0 0x7f7ed00640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eceffd700 1 -- 172.21.15.140:0/4219708129 learned_addr learned my addr 172.21.15.140:0/4219708129 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:46.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eceffd700 1 -- 172.21.15.140:0/4219708129 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00b5650 msgr2=0x7f7ed013daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eceffd700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00b5650 0x7f7ed013daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eceffd700 1 -- 172.21.15.140:0/4219708129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ed00adff0 msgr2=0x7f7ed00640f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eceffd700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ed00adff0 0x7f7ed00640f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eceffd700 1 -- 172.21.15.140:0/4219708129 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec8009470 con 0x7f7ed00af8e0 2022-07-21T01:48:46.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eceffd700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ed00af8e0 0x7f7ed013d600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f7eb8001290 tx=0x7f7eb80028b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:46.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eccff9700 1 -- 172.21.15.140:0/4219708129 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7eb800ce90 con 0x7f7ed00af8e0 2022-07-21T01:48:46.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eccff9700 1 -- 172.21.15.140:0/4219708129 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb80070d0 con 0x7f7ed00af8e0 2022-07-21T01:48:46.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7ed4873700 1 -- 172.21.15.140:0/4219708129 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed00b13d0 con 0x7f7ed00af8e0 2022-07-21T01:48:46.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eccff9700 1 -- 172.21.15.140:0/4219708129 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7eb80054e0 con 0x7f7ed00af8e0 2022-07-21T01:48:46.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7ed4873700 1 -- 172.21.15.140:0/4219708129 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed010e130 con 0x7f7ed00af8e0 2022-07-21T01:48:46.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7ed4873700 1 -- 172.21.15.140:0/4219708129 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed0110f60 con 0x7f7ed00af8e0 2022-07-21T01:48:46.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.451+0000 7f7eccff9700 1 -- 172.21.15.140:0/4219708129 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7eb801a060 con 0x7f7ed00af8e0 2022-07-21T01:48:46.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.455+0000 7f7eccff9700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7eb40539e0 0x7f7eb4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.455+0000 7f7eccff9700 1 -- 172.21.15.140:0/4219708129 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7eb806b5f0 con 0x7f7ed00af8e0 2022-07-21T01:48:46.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.455+0000 7f7ecf7fe700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7eb40539e0 0x7f7eb4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.455+0000 7f7ecf7fe700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7eb40539e0 0x7f7eb4055ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f7ec000d000 tx=0x7f7ec0007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:46.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.459+0000 7f7eccff9700 1 -- 172.21.15.140:0/4219708129 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7eb8042510 con 0x7f7ed00af8e0 2022-07-21T01:48:46.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.587+0000 7f7ed4873700 1 -- 172.21.15.140:0/4219708129 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ed0111f70 con 0x7f7ed00af8e0 2022-07-21T01:48:46.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7eccff9700 1 -- 172.21.15.140:0/4219708129 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f7eb8018020 con 0x7f7ed00af8e0 2022-07-21T01:48:46.589 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:46.589 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:46.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 -- 172.21.15.140:0/4219708129 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7eb40539e0 msgr2=0x7f7eb4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7eb40539e0 0x7f7eb4055ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f7ec000d000 tx=0x7f7ec0007a00).stop 2022-07-21T01:48:46.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 -- 172.21.15.140:0/4219708129 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ed00af8e0 msgr2=0x7f7ed013d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ed00af8e0 0x7f7ed013d600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f7eb8001290 tx=0x7f7eb80028b0).stop 2022-07-21T01:48:46.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 -- 172.21.15.140:0/4219708129 shutdown_connections 2022-07-21T01:48:46.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7eb40539e0 0x7f7eb4055ea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ed00b5650 0x7f7ed013daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ed00af8e0 0x7f7ed013d600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 --2- 172.21.15.140:0/4219708129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ed00adff0 0x7f7ed00640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 -- 172.21.15.140:0/4219708129 >> 172.21.15.140:0/4219708129 conn(0x7f7ed00634e0 msgr2=0x7f7ed00ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:46.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 -- 172.21.15.140:0/4219708129 shutdown_connections 2022-07-21T01:48:46.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.591+0000 7f7ed4873700 1 -- 172.21.15.140:0/4219708129 wait complete. 2022-07-21T01:48:46.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:48:29.344690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:48:30.427054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:48:46.603 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:48:46.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.711+0000 7fced5c89700 1 Processor -- start 2022-07-21T01:48:46.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.711+0000 7fced5c89700 1 -- start start 2022-07-21T01:48:46.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fced5c89700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced005cf40 0x7fced00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fced5c89700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fced0062ec0 0x7fced0142b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fced5c89700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced00ad3e0 con 0x7fced0143070 2022-07-21T01:48:46.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fced5c89700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced00b3b10 con 0x7fced0062ec0 2022-07-21T01:48:46.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fced5c89700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced00ad680 con 0x7fced005cf40 2022-07-21T01:48:46.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecffff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fced0143070 0x7fced0146580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47042/0 (socket says 172.21.15.140:47042) 2022-07-21T01:48:46.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecffff700 1 -- 172.21.15.140:0/2116781448 learned_addr learned my addr 172.21.15.140:0/2116781448 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:46.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fceceffd700 1 --2- 172.21.15.140:0/2116781448 >> v2:172.21.15.152:3300/0 conn(0x7fced0062ec0 0x7fced0142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecf7fe700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced005cf40 0x7fced00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344048980 0 0) 0x7fced00ad3e0 con 0x7fced0143070 2022-07-21T01:48:46.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcebc003260 con 0x7fced0143070 2022-07-21T01:48:46.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 389070918 0 0) 0x7fcebc003260 con 0x7fced0143070 2022-07-21T01:48:46.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced005cf40 msgr2=0x7fced00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:46.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced005cf40 0x7fced00adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 >> v2:172.21.15.152:3300/0 conn(0x7fced0062ec0 msgr2=0x7fced0142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:46.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 --2- 172.21.15.140:0/2116781448 >> v2:172.21.15.152:3300/0 conn(0x7fced0062ec0 0x7fced0142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced0064810 con 0x7fced0143070 2022-07-21T01:48:46.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fcec8001660 con 0x7fced0143070 2022-07-21T01:48:46.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 >> v1:172.21.15.140:6789/0 conn(0x7fced0143070 legacy=0x7fced0146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcebc003750 0x7fcebc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcebc004500 0x7fcebc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced005cf40 0x7fcebc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced00b3b10 con 0x7fcebc003750 2022-07-21T01:48:46.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced00ad680 con 0x7fcebc004500 2022-07-21T01:48:46.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc00b750 con 0x7fced005cf40 2022-07-21T01:48:46.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecf7fe700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcebc003750 0x7fcebc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecffff700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced005cf40 0x7fcebc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fceceffd700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcebc004500 0x7fcebc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecf7fe700 1 -- 172.21.15.140:0/2116781448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced005cf40 msgr2=0x7fcebc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:46.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecf7fe700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced005cf40 0x7fcebc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecf7fe700 1 -- 172.21.15.140:0/2116781448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcebc004500 msgr2=0x7fcebc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:46.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecf7fe700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcebc004500 0x7fcebc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecf7fe700 1 -- 172.21.15.140:0/2116781448 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc003260 con 0x7fcebc003750 2022-07-21T01:48:46.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecf7fe700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcebc003750 0x7fcebc003b70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fcec400c000 tx=0x7fcec4006df0).ready entity=mon.0 client_cookie=182db1a15f1b57a3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:46.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcec400e830 con 0x7fcebc003750 2022-07-21T01:48:46.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec400e990 con 0x7fcebc003750 2022-07-21T01:48:46.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcece7fc700 1 -- 172.21.15.140:0/2116781448 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcec400ecf0 con 0x7fcebc003750 2022-07-21T01:48:46.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fced5c89700 1 -- 172.21.15.140:0/2116781448 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcebc003750 msgr2=0x7fcebc003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fced5c89700 1 --2- 172.21.15.140:0/2116781448 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcebc003750 0x7fcebc003b70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fcec400c000 tx=0x7fcec4006df0).stop 2022-07-21T01:48:46.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.715+0000 7fcecffff700 1 -- 172.21.15.140:0/2116781448 reap_dead start 2022-07-21T01:48:46.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 -- 172.21.15.140:0/2116781448 shutdown_connections 2022-07-21T01:48:46.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 -- 172.21.15.140:0/2116781448 >> 172.21.15.140:0/2116781448 conn(0x7fced00b7d30 msgr2=0x7fced00b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:46.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 -- 172.21.15.140:0/2116781448 shutdown_connections 2022-07-21T01:48:46.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 -- 172.21.15.140:0/2116781448 wait complete. 2022-07-21T01:48:46.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 Processor -- start 2022-07-21T01:48:46.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 -- start start 2022-07-21T01:48:46.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fced00af450 0x7fced00af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced009ba90 0x7fced009a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced01556c0 0x7fced0155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced00afdd0 con 0x7fced00af450 2022-07-21T01:48:46.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced00b0070 con 0x7fced009ba90 2022-07-21T01:48:46.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fced5c89700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced009cab0 con 0x7fced01556c0 2022-07-21T01:48:46.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fcecf7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fced00af450 0x7fced00af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fcecf7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fced00af450 0x7fced00af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32912/0 (socket says 172.21.15.140:32912) 2022-07-21T01:48:46.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fcecffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced01556c0 0x7fced0155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fcecf7fe700 1 -- 172.21.15.140:0/1178617460 learned_addr learned my addr 172.21.15.140:0/1178617460 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:46.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fceceffd700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced009ba90 0x7fced009a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fcecf7fe700 1 -- 172.21.15.140:0/1178617460 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced01556c0 msgr2=0x7fced0155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fcecf7fe700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced01556c0 0x7fced0155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fcecf7fe700 1 -- 172.21.15.140:0/1178617460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced009ba90 msgr2=0x7fced009a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fcecf7fe700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced009ba90 0x7fced009a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fcecf7fe700 1 -- 172.21.15.140:0/1178617460 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced009cf50 con 0x7fced00af450 2022-07-21T01:48:46.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fceceffd700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced009ba90 0x7fced009a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:46.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.719+0000 7fcecf7fe700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fced00af450 0x7fced00af830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fcec400c000 tx=0x7fcec4019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:46.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.723+0000 7fceccff9700 1 -- 172.21.15.140:0/1178617460 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcec401cd20 con 0x7fced00af450 2022-07-21T01:48:46.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.723+0000 7fceccff9700 1 -- 172.21.15.140:0/1178617460 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec4000e90 con 0x7fced00af450 2022-07-21T01:48:46.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.723+0000 7fced5c89700 1 -- 172.21.15.140:0/1178617460 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced009d140 con 0x7fced00af450 2022-07-21T01:48:46.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.723+0000 7fceccff9700 1 -- 172.21.15.140:0/1178617460 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcec40245e0 con 0x7fced00af450 2022-07-21T01:48:46.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.723+0000 7fced5c89700 1 -- 172.21.15.140:0/1178617460 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced0156250 con 0x7fced00af450 2022-07-21T01:48:46.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.723+0000 7fced5c89700 1 -- 172.21.15.140:0/1178617460 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced009cc70 con 0x7fced00af450 2022-07-21T01:48:46.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.723+0000 7fceccff9700 1 -- 172.21.15.140:0/1178617460 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcec402c3d0 con 0x7fced00af450 2022-07-21T01:48:46.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.723+0000 7fceccff9700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fceb0053a70 0x7fceb0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.723+0000 7fceccff9700 1 -- 172.21.15.140:0/1178617460 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcec406d620 con 0x7fced00af450 2022-07-21T01:48:46.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.723+0000 7fceceffd700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fceb0053a70 0x7fceb0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.723+0000 7fceceffd700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fceb0053a70 0x7fceb0055f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fcec000b000 tx=0x7fcec0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:46.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.731+0000 7fceccff9700 1 -- 172.21.15.140:0/1178617460 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcec40444e0 con 0x7fced00af450 2022-07-21T01:48:46.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.855+0000 7fced5c89700 1 -- 172.21.15.140:0/1178617460 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fced00574e0 con 0x7fced00af450 2022-07-21T01:48:46.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.855+0000 7fceccff9700 1 -- 172.21.15.140:0/1178617460 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcec400be60 con 0x7fced00af450 2022-07-21T01:48:46.857 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:48:46.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 -- 172.21.15.140:0/1178617460 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fceb0053a70 msgr2=0x7fceb0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fceb0053a70 0x7fceb0055f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fcec000b000 tx=0x7fcec0008040).stop 2022-07-21T01:48:46.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 -- 172.21.15.140:0/1178617460 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fced00af450 msgr2=0x7fced00af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fced00af450 0x7fced00af830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fcec400c000 tx=0x7fcec4019000).stop 2022-07-21T01:48:46.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 -- 172.21.15.140:0/1178617460 shutdown_connections 2022-07-21T01:48:46.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fceb0053a70 0x7fceb0055f30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fced01556c0 0x7fced0155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fced009ba90 0x7fced009a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 --2- 172.21.15.140:0/1178617460 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fced00af450 0x7fced00af830 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 -- 172.21.15.140:0/1178617460 >> 172.21.15.140:0/1178617460 conn(0x7fced00b7d30 msgr2=0x7fced00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:46.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 -- 172.21.15.140:0/1178617460 shutdown_connections 2022-07-21T01:48:46.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.859+0000 7fced5c89700 1 -- 172.21.15.140:0/1178617460 wait complete. 2022-07-21T01:48:46.869 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:48:46.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 Processor -- start 2022-07-21T01:48:46.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 -- start start 2022-07-21T01:48:46.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9af40b59d0 0x7f9af40b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9af40b14f0 0x7f9af4147050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:46.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af40afbc0 con 0x7f9af414ada0 2022-07-21T01:48:46.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af4148080 con 0x7f9af40b59d0 2022-07-21T01:48:46.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af40b00e0 con 0x7f9af40b14f0 2022-07-21T01:48:46.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af8f37700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9af40b59d0 0x7f9af40b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af8f37700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9af40b59d0 0x7f9af40b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32916/0 (socket says 172.21.15.140:32916) 2022-07-21T01:48:46.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af8f37700 1 -- 172.21.15.140:0/1461928409 learned_addr learned my addr 172.21.15.140:0/1461928409 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:46.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9738700 1 --2- 172.21.15.140:0/1461928409 >> v2:172.21.15.152:3300/0 conn(0x7f9af40b14f0 0x7f9af4147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:46.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af8f37700 1 -- 172.21.15.140:0/1461928409 >> v2:172.21.15.152:3300/0 conn(0x7f9af40b14f0 msgr2=0x7f9af4147050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:46.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af8f37700 1 --2- 172.21.15.140:0/1461928409 >> v2:172.21.15.152:3300/0 conn(0x7f9af40b14f0 0x7f9af4147050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af8f37700 1 -- 172.21.15.140:0/1461928409 >> v1:172.21.15.152:6789/0 conn(0x7f9af414ada0 legacy=0x7f9af405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af37fe700 1 -- 172.21.15.140:0/1461928409 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509812367 0 0) 0x7f9af40afbc0 con 0x7f9af414ada0 2022-07-21T01:48:46.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af8f37700 1 -- 172.21.15.140:0/1461928409 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af40b5db0 con 0x7f9af40b59d0 2022-07-21T01:48:46.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9738700 1 --2- 172.21.15.140:0/1461928409 >> v2:172.21.15.152:3300/0 conn(0x7f9af40b14f0 0x7f9af4147050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:46.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af8f37700 1 --2- 172.21.15.140:0/1461928409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9af40b59d0 0x7f9af40b07e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f9ae8002f70 tx=0x7f9ae8006eb0).ready entity=mon.0 client_cookie=4d636d7b3fc4ec88 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:46.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af37fe700 1 -- 172.21.15.140:0/1461928409 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ae800d610 con 0x7f9af40b59d0 2022-07-21T01:48:46.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af37fe700 1 -- 172.21.15.140:0/1461928409 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae8000e90 con 0x7f9af40b59d0 2022-07-21T01:48:46.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af37fe700 1 -- 172.21.15.140:0/1461928409 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ae800d960 con 0x7f9af40b59d0 2022-07-21T01:48:46.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 -- 172.21.15.140:0/1461928409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9af40b59d0 msgr2=0x7f9af40b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:46.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 --2- 172.21.15.140:0/1461928409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9af40b59d0 0x7f9af40b07e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f9ae8002f70 tx=0x7f9ae8006eb0).stop 2022-07-21T01:48:46.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 -- 172.21.15.140:0/1461928409 shutdown_connections 2022-07-21T01:48:46.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 --2- 172.21.15.140:0/1461928409 >> v2:172.21.15.152:3300/0 conn(0x7f9af40b14f0 0x7f9af4147050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 --2- 172.21.15.140:0/1461928409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9af40b59d0 0x7f9af40b07e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:46.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 -- 172.21.15.140:0/1461928409 >> 172.21.15.140:0/1461928409 conn(0x7f9af40634c0 msgr2=0x7f9af40b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:47.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 -- 172.21.15.140:0/1461928409 shutdown_connections 2022-07-21T01:48:47.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.987+0000 7f9af9f39700 1 -- 172.21.15.140:0/1461928409 wait complete. 2022-07-21T01:48:47.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af9f39700 1 Processor -- start 2022-07-21T01:48:47.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af9f39700 1 -- start start 2022-07-21T01:48:47.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af9f39700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af40b14f0 0x7f9af414c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af9f39700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9af40b59d0 0x7f9af41569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af9f39700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9af414ada0 0x7f9af4063c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af9f39700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af4156ea0 con 0x7f9af40b59d0 2022-07-21T01:48:47.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af9f39700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af40656c0 con 0x7f9af40b14f0 2022-07-21T01:48:47.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af9f39700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af4065960 con 0x7f9af414ada0 2022-07-21T01:48:47.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9af40b59d0 0x7f9af41569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9af40b59d0 0x7f9af41569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32922/0 (socket says 172.21.15.140:32922) 2022-07-21T01:48:47.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af3fff700 1 -- 172.21.15.140:0/1997251725 learned_addr learned my addr 172.21.15.140:0/1997251725 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:47.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af9738700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9af414ada0 0x7f9af4063c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af8f37700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af40b14f0 0x7f9af414c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af3fff700 1 -- 172.21.15.140:0/1997251725 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9af414ada0 msgr2=0x7f9af4063c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af3fff700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9af414ada0 0x7f9af4063c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af3fff700 1 -- 172.21.15.140:0/1997251725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af40b14f0 msgr2=0x7f9af414c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af3fff700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af40b14f0 0x7f9af414c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af3fff700 1 -- 172.21.15.140:0/1997251725 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae800b1e0 con 0x7f9af40b59d0 2022-07-21T01:48:47.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af8f37700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af40b14f0 0x7f9af414c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:47.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af3fff700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9af40b59d0 0x7f9af41569b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f9ae0000ed0 tx=0x7f9ae0007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:47.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af1ffb700 1 -- 172.21.15.140:0/1997251725 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ae0013450 con 0x7f9af40b59d0 2022-07-21T01:48:47.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af1ffb700 1 -- 172.21.15.140:0/1997251725 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae00135b0 con 0x7f9af40b59d0 2022-07-21T01:48:47.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af9f39700 1 -- 172.21.15.140:0/1997251725 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af4065c00 con 0x7f9af40b59d0 2022-07-21T01:48:47.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af1ffb700 1 -- 172.21.15.140:0/1997251725 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ae0013910 con 0x7f9af40b59d0 2022-07-21T01:48:47.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.991+0000 7f9af9f39700 1 -- 172.21.15.140:0/1997251725 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af40651f0 con 0x7f9af40b59d0 2022-07-21T01:48:47.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.995+0000 7f9af9f39700 1 -- 172.21.15.140:0/1997251725 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af414b2e0 con 0x7f9af40b59d0 2022-07-21T01:48:47.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.995+0000 7f9af1ffb700 1 -- 172.21.15.140:0/1997251725 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9ae001a060 con 0x7f9af40b59d0 2022-07-21T01:48:47.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.995+0000 7f9af1ffb700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9ad4053b30 0x7f9ad4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.995+0000 7f9af1ffb700 1 -- 172.21.15.140:0/1997251725 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9ae006b680 con 0x7f9af40b59d0 2022-07-21T01:48:47.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.995+0000 7f9af8f37700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9ad4053b30 0x7f9ad4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.995+0000 7f9af8f37700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9ad4053b30 0x7f9ad4055ff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f9ae801a000 tx=0x7f9ae8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:47.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:46.999+0000 7f9af1ffb700 1 -- 172.21.15.140:0/1997251725 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ae00425a0 con 0x7f9af40b59d0 2022-07-21T01:48:47.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.123+0000 7f9af9f39700 1 -- 172.21.15.140:0/1997251725 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9af40574e0 con 0x7f9af40b59d0 2022-07-21T01:48:47.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.123+0000 7f9af1ffb700 1 -- 172.21.15.140:0/1997251725 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9ae0018020 con 0x7f9af40b59d0 2022-07-21T01:48:47.125 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:47.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 -- 172.21.15.140:0/1997251725 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9ad4053b30 msgr2=0x7f9ad4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9ad4053b30 0x7f9ad4055ff0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f9ae801a000 tx=0x7f9ae8015000).stop 2022-07-21T01:48:47.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 -- 172.21.15.140:0/1997251725 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9af40b59d0 msgr2=0x7f9af41569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9af40b59d0 0x7f9af41569b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f9ae0000ed0 tx=0x7f9ae0007bf0).stop 2022-07-21T01:48:47.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 -- 172.21.15.140:0/1997251725 shutdown_connections 2022-07-21T01:48:47.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9ad4053b30 0x7f9ad4055ff0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9af414ada0 0x7f9af4063c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9af40b59d0 0x7f9af41569b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 --2- 172.21.15.140:0/1997251725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af40b14f0 0x7f9af414c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 -- 172.21.15.140:0/1997251725 >> 172.21.15.140:0/1997251725 conn(0x7f9af40634c0 msgr2=0x7f9af40b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:47.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 -- 172.21.15.140:0/1997251725 shutdown_connections 2022-07-21T01:48:47.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.127+0000 7f9af9f39700 1 -- 172.21.15.140:0/1997251725 wait complete. 2022-07-21T01:48:47.137 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:48:47.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 Processor -- start 2022-07-21T01:48:47.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 -- start start 2022-07-21T01:48:47.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1c9c0b0480 0x7f1c9c14a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1c9c14ab60 0x7f1c9c144320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9c0ac5e0 con 0x7f1c9c0b5b50 2022-07-21T01:48:47.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c9c0ad3c0 con 0x7f1c9c0b0480 2022-07-21T01:48:47.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c9c0b5f30 con 0x7f1c9c14ab60 2022-07-21T01:48:47.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c99d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1c9c0b0480 0x7f1c9c14a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c99d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1c9c0b0480 0x7f1c9c14a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32932/0 (socket says 172.21.15.140:32932) 2022-07-21T01:48:47.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c99d9b700 1 -- 172.21.15.140:0/2421627486 learned_addr learned my addr 172.21.15.140:0/2421627486 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:47.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c9ad9d700 1 --2- 172.21.15.140:0/2421627486 >> v2:172.21.15.152:3300/0 conn(0x7f1c9c14ab60 0x7f1c9c144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c99d9b700 1 -- 172.21.15.140:0/2421627486 >> v2:172.21.15.152:3300/0 conn(0x7f1c9c14ab60 msgr2=0x7f1c9c144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:47.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c99d9b700 1 --2- 172.21.15.140:0/2421627486 >> v2:172.21.15.152:3300/0 conn(0x7f1c9c14ab60 0x7f1c9c144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c99d9b700 1 -- 172.21.15.140:0/2421627486 >> v1:172.21.15.140:6789/0 conn(0x7f1c9c0b5b50 legacy=0x7f1c9c0accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c99d9b700 1 -- 172.21.15.140:0/2421627486 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c05cf70 con 0x7f1c9c0b0480 2022-07-21T01:48:47.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c99d9b700 1 --2- 172.21.15.140:0/2421627486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c9c0b0480 0x7f1c9c14a670 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f1c8c002f70 tx=0x7f1c8c006eb0).ready entity=mon.0 client_cookie=498576c60c5cf93f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:47.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c9959a700 1 -- 172.21.15.140:0/2421627486 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c8c00d610 con 0x7f1c9c0b0480 2022-07-21T01:48:47.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c9959a700 1 -- 172.21.15.140:0/2421627486 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c000e90 con 0x7f1c9c0b0480 2022-07-21T01:48:47.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1c9959a700 1 -- 172.21.15.140:0/2421627486 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c8c00d9a0 con 0x7f1c9c0b0480 2022-07-21T01:48:47.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/2421627486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c9c0b0480 msgr2=0x7f1c9c14a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 --2- 172.21.15.140:0/2421627486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c9c0b0480 0x7f1c9c14a670 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f1c8c002f70 tx=0x7f1c8c006eb0).stop 2022-07-21T01:48:47.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/2421627486 shutdown_connections 2022-07-21T01:48:47.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 --2- 172.21.15.140:0/2421627486 >> v2:172.21.15.152:3300/0 conn(0x7f1c9c14ab60 0x7f1c9c144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 --2- 172.21.15.140:0/2421627486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c9c0b0480 0x7f1c9c14a670 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/2421627486 >> 172.21.15.140:0/2421627486 conn(0x7f1c9c0635e0 msgr2=0x7f1c9c0b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/2421627486 shutdown_connections 2022-07-21T01:48:47.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.243+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/2421627486 wait complete. 2022-07-21T01:48:47.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1ca0a2a700 1 Processor -- start 2022-07-21T01:48:47.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1ca0a2a700 1 -- start start 2022-07-21T01:48:47.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1ca0a2a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c9c0b0480 0x7f1c9c0b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1ca0a2a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c9c0b5b50 0x7f1c9c1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1ca0a2a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c9c14ab60 0x7f1c9c15a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1ca0a2a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c0643f0 con 0x7f1c9c14ab60 2022-07-21T01:48:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1ca0a2a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c064150 con 0x7f1c9c0b5b50 2022-07-21T01:48:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1ca0a2a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c159dd0 con 0x7f1c9c0b0480 2022-07-21T01:48:47.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c9ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c9c14ab60 0x7f1c9c15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c9ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c9c14ab60 0x7f1c9c15a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32936/0 (socket says 172.21.15.140:32936) 2022-07-21T01:48:47.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c9ad9d700 1 -- 172.21.15.140:0/1645161318 learned_addr learned my addr 172.21.15.140:0/1645161318 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:47.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c99d9b700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c9c0b5b50 0x7f1c9c1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c9a59c700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c9c0b0480 0x7f1c9c0b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c9ad9d700 1 -- 172.21.15.140:0/1645161318 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c9c0b0480 msgr2=0x7f1c9c0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c9ad9d700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c9c0b0480 0x7f1c9c0b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c9ad9d700 1 -- 172.21.15.140:0/1645161318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c9c0b5b50 msgr2=0x7f1c9c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c9ad9d700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c9c0b5b50 0x7f1c9c1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c9ad9d700 1 -- 172.21.15.140:0/1645161318 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c00b1e0 con 0x7f1c9c14ab60 2022-07-21T01:48:47.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c9ad9d700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c9c14ab60 0x7f1c9c15a270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f1c9400a000 tx=0x7f1c94008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:47.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c937fe700 1 -- 172.21.15.140:0/1645161318 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c94011780 con 0x7f1c9c14ab60 2022-07-21T01:48:47.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c937fe700 1 -- 172.21.15.140:0/1645161318 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c940118e0 con 0x7f1c9c14ab60 2022-07-21T01:48:47.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1c937fe700 1 -- 172.21.15.140:0/1645161318 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c94011c40 con 0x7f1c9c14ab60 2022-07-21T01:48:47.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/1645161318 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c156ba0 con 0x7f1c9c14ab60 2022-07-21T01:48:47.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.247+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/1645161318 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c067710 con 0x7f1c9c14ab60 2022-07-21T01:48:47.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.251+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/1645161318 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9c067900 con 0x7f1c9c14ab60 2022-07-21T01:48:47.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.251+0000 7f1c937fe700 1 -- 172.21.15.140:0/1645161318 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1c94012420 con 0x7f1c9c14ab60 2022-07-21T01:48:47.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.251+0000 7f1c937fe700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c80053a70 0x7f1c80055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.251+0000 7f1c937fe700 1 -- 172.21.15.140:0/1645161318 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1c9406c030 con 0x7f1c9c14ab60 2022-07-21T01:48:47.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.251+0000 7f1c9a59c700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c80053a70 0x7f1c80055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.251+0000 7f1c9a59c700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c80053a70 0x7f1c80055f30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f1c8400b000 tx=0x7f1c84009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:47.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.259+0000 7f1c937fe700 1 -- 172.21.15.140:0/1645161318 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c94042ef0 con 0x7f1c9c14ab60 2022-07-21T01:48:47.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.375+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/1645161318 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c9c0ac810 con 0x7f1c9c14ab60 2022-07-21T01:48:47.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.375+0000 7f1c937fe700 1 -- 172.21.15.140:0/1645161318 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1c94046510 con 0x7f1c9c14ab60 2022-07-21T01:48:47.376 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:47.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/1645161318 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c80053a70 msgr2=0x7f1c80055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c80053a70 0x7f1c80055f30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f1c8400b000 tx=0x7f1c84009040).stop 2022-07-21T01:48:47.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/1645161318 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c9c14ab60 msgr2=0x7f1c9c15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c9c14ab60 0x7f1c9c15a270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f1c9400a000 tx=0x7f1c94008040).stop 2022-07-21T01:48:47.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/1645161318 shutdown_connections 2022-07-21T01:48:47.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c80053a70 0x7f1c80055f30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c9c14ab60 0x7f1c9c15a270 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c9c0b5b50 0x7f1c9c1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 --2- 172.21.15.140:0/1645161318 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c9c0b0480 0x7f1c9c0b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/1645161318 >> 172.21.15.140:0/1645161318 conn(0x7f1c9c0635e0 msgr2=0x7f1c9c145010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:47.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/1645161318 shutdown_connections 2022-07-21T01:48:47.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.379+0000 7f1ca0a2a700 1 -- 172.21.15.140:0/1645161318 wait complete. 2022-07-21T01:48:47.392 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:48:47.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.503+0000 7f526e4c7700 1 Processor -- start 2022-07-21T01:48:47.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.503+0000 7f526e4c7700 1 -- start start 2022-07-21T01:48:47.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.503+0000 7f526e4c7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f526805cda0 0x7f52680675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.503+0000 7f526e4c7700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52680b16f0 con 0x7f5268140860 2022-07-21T01:48:47.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.503+0000 7f526e4c7700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5268140d30 con 0x7f5268063270 2022-07-21T01:48:47.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.503+0000 7f526e4c7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52680624b0 con 0x7f526805cda0 2022-07-21T01:48:47.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.503+0000 7f52677fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f5268140860 0x7f5268061da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47090/0 (socket says 172.21.15.140:47090) 2022-07-21T01:48:47.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.503+0000 7f52677fe700 1 -- 172.21.15.140:0/528874392 learned_addr learned my addr 172.21.15.140:0/528874392 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:47.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5267fff700 1 --2- 172.21.15.140:0/528874392 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f526805cda0 0x7f52680675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5266ffd700 1 -- 172.21.15.140:0/528874392 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208528619 0 0) 0x7f52680b16f0 con 0x7f5268140860 2022-07-21T01:48:47.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5266ffd700 1 -- 172.21.15.140:0/528874392 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5254003260 con 0x7f5268140860 2022-07-21T01:48:47.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5266ffd700 1 -- 172.21.15.140:0/528874392 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846735100 0 0) 0x7f5268140d30 con 0x7f5268063270 2022-07-21T01:48:47.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5266ffd700 1 -- 172.21.15.140:0/528874392 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52680b16f0 con 0x7f5268063270 2022-07-21T01:48:47.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5267fff700 1 -- 172.21.15.140:0/528874392 >> v1:172.21.15.152:6789/0 conn(0x7f5268063270 legacy=0x7f5268141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5267fff700 1 -- 172.21.15.140:0/528874392 >> v1:172.21.15.140:6789/0 conn(0x7f5268140860 legacy=0x7f5268061da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5267fff700 1 -- 172.21.15.140:0/528874392 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5268140fd0 con 0x7f526805cda0 2022-07-21T01:48:47.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5267fff700 1 --2- 172.21.15.140:0/528874392 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f526805cda0 0x7f52680675f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f525c002f70 tx=0x7f525c006df0).ready entity=mon.2 client_cookie=6c9db9d29bb02601 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:47.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5266ffd700 1 -- 172.21.15.140:0/528874392 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f525c00d5c0 con 0x7f526805cda0 2022-07-21T01:48:47.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5266ffd700 1 -- 172.21.15.140:0/528874392 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c000e90 con 0x7f526805cda0 2022-07-21T01:48:47.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f5266ffd700 1 -- 172.21.15.140:0/528874392 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f525c00d910 con 0x7f526805cda0 2022-07-21T01:48:47.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f526e4c7700 1 -- 172.21.15.140:0/528874392 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f526805cda0 msgr2=0x7f52680675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f526e4c7700 1 --2- 172.21.15.140:0/528874392 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f526805cda0 0x7f52680675f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f525c002f70 tx=0x7f525c006df0).stop 2022-07-21T01:48:47.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f526e4c7700 1 -- 172.21.15.140:0/528874392 shutdown_connections 2022-07-21T01:48:47.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f526e4c7700 1 --2- 172.21.15.140:0/528874392 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f526805cda0 0x7f52680675f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f526e4c7700 1 -- 172.21.15.140:0/528874392 >> 172.21.15.140:0/528874392 conn(0x7f52680b7da0 msgr2=0x7f52680b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:47.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f526e4c7700 1 -- 172.21.15.140:0/528874392 shutdown_connections 2022-07-21T01:48:47.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.507+0000 7f526e4c7700 1 -- 172.21.15.140:0/528874392 wait complete. 2022-07-21T01:48:47.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526e4c7700 1 Processor -- start 2022-07-21T01:48:47.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526e4c7700 1 -- start start 2022-07-21T01:48:47.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526e4c7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f526805cda0 0x7f526809b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526e4c7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5268063270 0x7f526814f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526e4c7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5268140860 0x7f526814f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526e4c7700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5268099840 con 0x7f5268063270 2022-07-21T01:48:47.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526e4c7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5268098780 con 0x7f526805cda0 2022-07-21T01:48:47.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526e4c7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5268099a70 con 0x7f5268140860 2022-07-21T01:48:47.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52677fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5268063270 0x7f526814f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52677fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5268063270 0x7f526814f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32952/0 (socket says 172.21.15.140:32952) 2022-07-21T01:48:47.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52677fe700 1 -- 172.21.15.140:0/1839645435 learned_addr learned my addr 172.21.15.140:0/1839645435 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:47.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526ca64700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5268140860 0x7f526814f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f5267fff700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f526805cda0 0x7f526809b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52677fe700 1 -- 172.21.15.140:0/1839645435 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5268140860 msgr2=0x7f526814f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52677fe700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5268140860 0x7f526814f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52677fe700 1 -- 172.21.15.140:0/1839645435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f526805cda0 msgr2=0x7f526809b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52677fe700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f526805cda0 0x7f526809b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52677fe700 1 -- 172.21.15.140:0/1839645435 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c00b180 con 0x7f5268063270 2022-07-21T01:48:47.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526ca64700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5268140860 0x7f526814f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:47.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f5267fff700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f526805cda0 0x7f526809b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:47.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52677fe700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5268063270 0x7f526814f180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5258001110 tx=0x7f5258007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:47.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52657fa700 1 -- 172.21.15.140:0/1839645435 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f525800ce00 con 0x7f5268063270 2022-07-21T01:48:47.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52657fa700 1 -- 172.21.15.140:0/1839645435 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52580072e0 con 0x7f5268063270 2022-07-21T01:48:47.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526e4c7700 1 -- 172.21.15.140:0/1839645435 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5268150fa0 con 0x7f5268063270 2022-07-21T01:48:47.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f52657fa700 1 -- 172.21.15.140:0/1839645435 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5258005800 con 0x7f5268063270 2022-07-21T01:48:47.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.511+0000 7f526e4c7700 1 -- 172.21.15.140:0/1839645435 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526814fb60 con 0x7f5268063270 2022-07-21T01:48:47.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.515+0000 7f526e4c7700 1 -- 172.21.15.140:0/1839645435 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52680ad1a0 con 0x7f5268063270 2022-07-21T01:48:47.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.515+0000 7f52657fa700 1 -- 172.21.15.140:0/1839645435 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f525801a060 con 0x7f5268063270 2022-07-21T01:48:47.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.515+0000 7f52657fa700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5248053b00 0x7f5248055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.515+0000 7f52657fa700 1 -- 172.21.15.140:0/1839645435 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f525806b7a0 con 0x7f5268063270 2022-07-21T01:48:47.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.515+0000 7f5267fff700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5248053b00 0x7f5248055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.515+0000 7f5267fff700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5248053b00 0x7f5248055fc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f525c015000 tx=0x7f525c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:47.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.519+0000 7f52657fa700 1 -- 172.21.15.140:0/1839645435 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f52580425b0 con 0x7f5268063270 2022-07-21T01:48:47.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f526e4c7700 1 -- 172.21.15.140:0/1839645435 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5268141b80 con 0x7f5268063270 2022-07-21T01:48:47.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f52657fa700 1 -- 172.21.15.140:0/1839645435 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5258018020 con 0x7f5268063270 2022-07-21T01:48:47.644 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:47.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f526e4c7700 1 -- 172.21.15.140:0/1839645435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5248053b00 msgr2=0x7f5248055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f526e4c7700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5248053b00 0x7f5248055fc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f525c015000 tx=0x7f525c014000).stop 2022-07-21T01:48:47.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f526e4c7700 1 -- 172.21.15.140:0/1839645435 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5268063270 msgr2=0x7f526814f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f526e4c7700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5268063270 0x7f526814f180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5258001110 tx=0x7f5258007af0).stop 2022-07-21T01:48:47.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f526e4c7700 1 -- 172.21.15.140:0/1839645435 shutdown_connections 2022-07-21T01:48:47.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f526e4c7700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5248053b00 0x7f5248055fc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f526e4c7700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5268140860 0x7f526814f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f526e4c7700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5268063270 0x7f526814f180 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f526e4c7700 1 --2- 172.21.15.140:0/1839645435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f526805cda0 0x7f526809b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.643+0000 7f526e4c7700 1 -- 172.21.15.140:0/1839645435 >> 172.21.15.140:0/1839645435 conn(0x7f52680b7da0 msgr2=0x7f526813f740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:47.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.647+0000 7f526e4c7700 1 -- 172.21.15.140:0/1839645435 shutdown_connections 2022-07-21T01:48:47.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.647+0000 7f526e4c7700 1 -- 172.21.15.140:0/1839645435 wait complete. 2022-07-21T01:48:47.661 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:48:47.662 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:48:47.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0282467700 1 Processor -- start 2022-07-21T01:48:47.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0282467700 1 -- start start 2022-07-21T01:48:47.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0282467700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c0af8e0 0x7f027c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0282467700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027c147010 con 0x7f027c0b5650 2022-07-21T01:48:47.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0282467700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027c1472b0 con 0x7f027c0adff0 2022-07-21T01:48:47.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0282467700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c146d40 con 0x7f027c0af8e0 2022-07-21T01:48:47.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0281465700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f027c0b5650 0x7f027c147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47104/0 (socket says 172.21.15.140:47104) 2022-07-21T01:48:47.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0281465700 1 -- 172.21.15.140:0/3809104884 learned_addr learned my addr 172.21.15.140:0/3809104884 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:47.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0281c66700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c0af8e0 0x7f027c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52231833 0 0) 0x7f027c147010 con 0x7f027c0b5650 2022-07-21T01:48:47.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0260003260 con 0x7f027c0b5650 2022-07-21T01:48:47.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419615933 0 0) 0x7f027c1472b0 con 0x7f027c0adff0 2022-07-21T01:48:47.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f027c147010 con 0x7f027c0adff0 2022-07-21T01:48:47.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3336870387 0 0) 0x7f0260003260 con 0x7f027c0b5650 2022-07-21T01:48:47.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c0af8e0 msgr2=0x7f027c0b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:47.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c0af8e0 0x7f027c0b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 >> v1:172.21.15.152:6789/0 conn(0x7f027c0adff0 legacy=0x7f027c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c05c4a0 con 0x7f027c0b5650 2022-07-21T01:48:47.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f026c001660 con 0x7f027c0b5650 2022-07-21T01:48:47.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 >> v1:172.21.15.140:6789/0 conn(0x7f027c0b5650 legacy=0x7f027c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c0af8e0 0x7f0260003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0260003e00 0x7f0260004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02600049f0 0x7f0260004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c146d40 con 0x7f02600049f0 2022-07-21T01:48:47.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026000b750 con 0x7f0260003e00 2022-07-21T01:48:47.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026000b910 con 0x7f027c0af8e0 2022-07-21T01:48:47.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0281c66700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02600049f0 0x7f0260004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0280c64700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0260003e00 0x7f0260004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0281465700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c0af8e0 0x7f0260003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0281c66700 1 -- 172.21.15.140:0/3809104884 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c0af8e0 msgr2=0x7f0260003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:47.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0281c66700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c0af8e0 0x7f0260003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0281c66700 1 -- 172.21.15.140:0/3809104884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0260003e00 msgr2=0x7f0260004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:47.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0281c66700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0260003e00 0x7f0260004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.767+0000 7f0281c66700 1 -- 172.21.15.140:0/3809104884 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0260003260 con 0x7f02600049f0 2022-07-21T01:48:47.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0281c66700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02600049f0 0x7f0260004e70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f027800c000 tx=0x7f0278006df0).ready entity=mon.0 client_cookie=93174561ed3d96c0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:47.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f027800e800 con 0x7f02600049f0 2022-07-21T01:48:47.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027800e960 con 0x7f02600049f0 2022-07-21T01:48:47.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0273fff700 1 -- 172.21.15.140:0/3809104884 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f027800ecc0 con 0x7f02600049f0 2022-07-21T01:48:47.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0282467700 1 -- 172.21.15.140:0/3809104884 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02600049f0 msgr2=0x7f0260004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0282467700 1 --2- 172.21.15.140:0/3809104884 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02600049f0 0x7f0260004e70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f027800c000 tx=0x7f0278006df0).stop 2022-07-21T01:48:47.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0281c66700 1 -- 172.21.15.140:0/3809104884 reap_dead start 2022-07-21T01:48:47.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0282467700 1 -- 172.21.15.140:0/3809104884 shutdown_connections 2022-07-21T01:48:47.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0282467700 1 -- 172.21.15.140:0/3809104884 >> 172.21.15.140:0/3809104884 conn(0x7f027c0634e0 msgr2=0x7f027c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:47.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0282467700 1 -- 172.21.15.140:0/3809104884 shutdown_connections 2022-07-21T01:48:47.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0282467700 1 -- 172.21.15.140:0/3809104884 wait complete. 2022-07-21T01:48:47.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0282467700 1 Processor -- start 2022-07-21T01:48:47.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0282467700 1 -- start start 2022-07-21T01:48:47.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0282467700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c14abb0 0x7f027c1562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.771+0000 7f0282467700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f027c067640 0x7f027c067090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0282467700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f027c1569c0 0x7f027c156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0282467700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c157330 con 0x7f027c067640 2022-07-21T01:48:47.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0282467700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c111bb0 con 0x7f027c1569c0 2022-07-21T01:48:47.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0282467700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c14af90 con 0x7f027c14abb0 2022-07-21T01:48:47.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0280c64700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f027c067640 0x7f027c067090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0280c64700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f027c067640 0x7f027c067090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32972/0 (socket says 172.21.15.140:32972) 2022-07-21T01:48:47.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0280c64700 1 -- 172.21.15.140:0/2298607966 learned_addr learned my addr 172.21.15.140:0/2298607966 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:47.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0281465700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c14abb0 0x7f027c1562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0281c66700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f027c1569c0 0x7f027c156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0280c64700 1 -- 172.21.15.140:0/2298607966 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c14abb0 msgr2=0x7f027c1562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0280c64700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c14abb0 0x7f027c1562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0280c64700 1 -- 172.21.15.140:0/2298607966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f027c1569c0 msgr2=0x7f027c156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0280c64700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f027c1569c0 0x7f027c156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0280c64700 1 -- 172.21.15.140:0/2298607966 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c1113d0 con 0x7f027c067640 2022-07-21T01:48:47.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0281c66700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f027c1569c0 0x7f027c156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:47.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0280c64700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f027c067640 0x7f027c067090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f027c14b120 tx=0x7f0274007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:47.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f02727fc700 1 -- 172.21.15.140:0/2298607966 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0274010710 con 0x7f027c067640 2022-07-21T01:48:47.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f02727fc700 1 -- 172.21.15.140:0/2298607966 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0274010870 con 0x7f027c067640 2022-07-21T01:48:47.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f02727fc700 1 -- 172.21.15.140:0/2298607966 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0274010bd0 con 0x7f027c067640 2022-07-21T01:48:47.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0282467700 1 -- 172.21.15.140:0/2298607966 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027c111670 con 0x7f027c067640 2022-07-21T01:48:47.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0282467700 1 -- 172.21.15.140:0/2298607966 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027c159bd0 con 0x7f027c067640 2022-07-21T01:48:47.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0282467700 1 -- 172.21.15.140:0/2298607966 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f027c159910 con 0x7f027c067640 2022-07-21T01:48:47.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f02727fc700 1 -- 172.21.15.140:0/2298607966 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f027401b030 con 0x7f027c067640 2022-07-21T01:48:47.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f02727fc700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0264053a70 0x7f0264055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:47.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f02727fc700 1 -- 172.21.15.140:0/2298607966 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f027406bcc0 con 0x7f027c067640 2022-07-21T01:48:47.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.775+0000 7f0281465700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0264053a70 0x7f0264055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:47.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.779+0000 7f0281465700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0264053a70 0x7f0264055f30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f026c00c000 tx=0x7f026c0089e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:47.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.783+0000 7f02727fc700 1 -- 172.21.15.140:0/2298607966 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f027c159910 con 0x7f027c067640 2022-07-21T01:48:47.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.943+0000 7f0282467700 1 -- 172.21.15.140:0/2298607966 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f027c159910 con 0x7f027c067640 2022-07-21T01:48:47.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.943+0000 7f02727fc700 1 -- 172.21.15.140:0/2298607966 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7f027c159910 con 0x7f027c067640 2022-07-21T01:48:47.945 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:47.945 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/665899293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":665899293},{"type":"v1","addr":"172.21.15.140:6835","nonce":665899293}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:48:31.423028+0000","modified":"2022-07-21T01:48:32.427116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.152:6833/1276387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1276387778},{"type":"v1","addr":"172.21.15.152:6833","nonce":1276387778}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-07-21T01:48:47.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 -- 172.21.15.140:0/2298607966 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0264053a70 msgr2=0x7f0264055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0264053a70 0x7f0264055f30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f026c00c000 tx=0x7f026c0089e0).stop 2022-07-21T01:48:47.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 -- 172.21.15.140:0/2298607966 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f027c067640 msgr2=0x7f027c067090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:47.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f027c067640 0x7f027c067090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f027c14b120 tx=0x7f0274007dd0).stop 2022-07-21T01:48:47.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 -- 172.21.15.140:0/2298607966 shutdown_connections 2022-07-21T01:48:47.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0264053a70 0x7f0264055f30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f027c1569c0 0x7f027c156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f027c067640 0x7f027c067090 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 --2- 172.21.15.140:0/2298607966 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f027c14abb0 0x7f027c1562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:47.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 -- 172.21.15.140:0/2298607966 >> 172.21.15.140:0/2298607966 conn(0x7f027c0634e0 msgr2=0x7f027c0ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:47.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 -- 172.21.15.140:0/2298607966 shutdown_connections 2022-07-21T01:48:47.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:47.947+0000 7f0282467700 1 -- 172.21.15.140:0/2298607966 wait complete. 2022-07-21T01:48:47.950 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 5 2022-07-21T01:48:48.159 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:48:31.423028+0000', 'modified': '2022-07-21T01:48:32.427116+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.152:6833/1276387778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 1276387778}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 1276387778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:48:48.160 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:48:48.161 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:48:48.276 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":4282,"want_state":"up:active","state":"up:active","rank_uptime":16.843579194,"mdsmap_epoch":5,"osdmap_epoch":22,"osdmap_epoch_barrier":21,"uptime":20.191264652000001} 2022-07-21T01:48:48.277 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 4282, "mdsmap_epoch": 5, "osdmap_epoch": 22, "osdmap_epoch_barrier": 21, "rank_uptime": 16.843579194, "state": "up:active", "uptime": 20.191264652, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:48:48.277 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:48:48.278 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-07-21T01:48:48.278 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-07-21T01:48:48.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:48.279 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:48:48.289 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:48:48.307 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:48:48.307 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:48:48.320 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:48:48.321 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:48:48.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:48.323 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:48:48.329 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:48:48.329 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:48:48.372 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:48:48.389 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:48:48.390 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:48:48.404 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:48:48.405 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:48:48.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:48.408 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:48:48.448 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2022-07-21T01:48:48.449+0000 7fbe72913080 -1 init, newargv = 0x5556315561f0 newargc=9 2022-07-21T01:48:48.449 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22187]: starting ceph client 2022-07-21T01:48:48.458 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22187]: starting fuse 2022-07-21T01:48:49.562 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:48:49.581 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:48:49.581 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:48:49.639 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:48:49.641 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:48:49.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:49.642 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:48:49.648 INFO:teuthology.orchestra.run.smithi192.stdout:52 2022-07-21T01:48:49.649 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:48:49.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:49.649 DEBUG:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2022-07-21T01:48:49.650 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2022-07-21T01:48:49.650 DEBUG:teuthology.orchestra.run.smithi192:> import re 2022-07-21T01:48:49.651 DEBUG:teuthology.orchestra.run.smithi192:> import os 2022-07-21T01:48:49.651 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2022-07-21T01:48:49.652 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:48:49.652 DEBUG:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2022-07-21T01:48:49.653 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-07-21T01:48:49.653 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2022-07-21T01:48:49.653 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:48:49.654 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2022-07-21T01:48:49.654 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2022-07-21T01:48:49.654 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2022-07-21T01:48:49.655 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2022-07-21T01:48:49.655 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:48:49.655 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2022-07-21T01:48:49.656 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:48:49.656 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:48:49.656 DEBUG:teuthology.orchestra.run.smithi192:> return f 2022-07-21T01:48:49.657 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:48:49.657 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:48:49.657 DEBUG:teuthology.orchestra.run.smithi192:> print(find_socket("client.1")) 2022-07-21T01:48:49.658 DEBUG:teuthology.orchestra.run.smithi192:> ' 2022-07-21T01:48:49.732 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.22187.asok 2022-07-21T01:48:49.738 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22187.asok 2022-07-21T01:48:49.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:49.739 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22187.asok status 2022-07-21T01:48:49.828 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2022-07-21T01:48:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2022-07-21T01:48:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:48:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:48:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2022-07-21T01:48:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2022-07-21T01:48:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-07-21T01:48:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "22187", 2022-07-21T01:48:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2022-07-21T01:48:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:48:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2022-07-21T01:48:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2022-07-21T01:48:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 4641, 2022-07-21T01:48:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2022-07-21T01:48:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2022-07-21T01:48:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2022-07-21T01:48:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 4641 2022-07-21T01:48:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:48:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:48:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:48:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:48:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 617924264 2022-07-21T01:48:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: } 2022-07-21T01:48:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:48:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:48:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:48:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:48:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 617924264 2022-07-21T01:48:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:48:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.4641 172.21.15.192:0/617924264", 2022-07-21T01:48:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/617924264", 2022-07-21T01:48:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2022-07-21T01:48:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 5, 2022-07-21T01:48:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 22, 2022-07-21T01:48:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:48:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false, 2022-07-21T01:48:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2022-07-21T01:48:49.846 INFO:teuthology.orchestra.run.smithi192.stdout:} 2022-07-21T01:48:49.849 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:49.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.959+0000 7f4f70d2d700 1 Processor -- start 2022-07-21T01:48:49.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.959+0000 7f4f70d2d700 1 -- start start 2022-07-21T01:48:49.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f70d2d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4f6c061a50 0x7f4f6c061e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:49.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f70d2d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c0acba0 con 0x7f4f6c09aa30 2022-07-21T01:48:49.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f70d2d700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c0b1d30 con 0x7f4f6c05ce60 2022-07-21T01:48:49.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f70d2d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6c0b1fd0 con 0x7f4f6c061a50 2022-07-21T01:48:49.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f6b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4f6c061a50 0x7f4f6c061e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:49.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f6affd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4f6c09aa30 0x7f4f6c0b2470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47126/0 (socket says 172.21.15.140:47126) 2022-07-21T01:48:49.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f6affd700 1 -- 172.21.15.140:0/1540681708 learned_addr learned my addr 172.21.15.140:0/1540681708 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:49.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f6b7fe700 1 -- 172.21.15.140:0/1540681708 >> v1:172.21.15.152:6789/0 conn(0x7f4f6c05ce60 legacy=0x7f4f6c09b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:49.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f6b7fe700 1 -- 172.21.15.140:0/1540681708 >> v1:172.21.15.140:6789/0 conn(0x7f4f6c09aa30 legacy=0x7f4f6c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:49.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f6a7fc700 1 -- 172.21.15.140:0/1540681708 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679171710 0 0) 0x7f4f6c0acba0 con 0x7f4f6c09aa30 2022-07-21T01:48:49.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f6b7fe700 1 -- 172.21.15.140:0/1540681708 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c09b720 con 0x7f4f6c061a50 2022-07-21T01:48:49.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f6b7fe700 1 --2- 172.21.15.140:0/1540681708 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f6c061a50 0x7f4f6c061e30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f4f6c099800 tx=0x7f4f5c006df0).ready entity=mon.0 client_cookie=a1a49fdef767c4cf server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:49.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f6a7fc700 1 -- 172.21.15.140:0/1540681708 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f5c00d610 con 0x7f4f6c061a50 2022-07-21T01:48:49.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f6a7fc700 1 -- 172.21.15.140:0/1540681708 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c000e90 con 0x7f4f6c061a50 2022-07-21T01:48:49.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.963+0000 7f4f6a7fc700 1 -- 172.21.15.140:0/1540681708 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f5c00d960 con 0x7f4f6c061a50 2022-07-21T01:48:49.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.967+0000 7f4f70d2d700 1 -- 172.21.15.140:0/1540681708 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f6c061a50 msgr2=0x7f4f6c061e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:49.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.967+0000 7f4f70d2d700 1 --2- 172.21.15.140:0/1540681708 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f6c061a50 0x7f4f6c061e30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f4f6c099800 tx=0x7f4f5c006df0).stop 2022-07-21T01:48:49.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.967+0000 7f4f70d2d700 1 -- 172.21.15.140:0/1540681708 shutdown_connections 2022-07-21T01:48:49.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.967+0000 7f4f70d2d700 1 --2- 172.21.15.140:0/1540681708 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f6c061a50 0x7f4f6c061e30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:49.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.967+0000 7f4f70d2d700 1 -- 172.21.15.140:0/1540681708 >> 172.21.15.140:0/1540681708 conn(0x7f4f6c0b2ca0 msgr2=0x7f4f6c0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:49.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.967+0000 7f4f70d2d700 1 -- 172.21.15.140:0/1540681708 shutdown_connections 2022-07-21T01:48:49.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.967+0000 7f4f70d2d700 1 -- 172.21.15.140:0/1540681708 wait complete. 2022-07-21T01:48:49.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.967+0000 7f4f70d2d700 1 Processor -- start 2022-07-21T01:48:49.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.967+0000 7f4f70d2d700 1 -- start start 2022-07-21T01:48:49.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f70d2d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f6c05ce60 0x7f4f6c0af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:49.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f70d2d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f6c061a50 0x7f4f6c063df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:49.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f70d2d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f6c09aa30 0x7f4f6c0642e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:49.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f70d2d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c066db0 con 0x7f4f6c05ce60 2022-07-21T01:48:49.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f70d2d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c067050 con 0x7f4f6c09aa30 2022-07-21T01:48:49.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f70d2d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0667b0 con 0x7f4f6c061a50 2022-07-21T01:48:49.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f6b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f6c05ce60 0x7f4f6c0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:49.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f6b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f6c05ce60 0x7f4f6c0af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32984/0 (socket says 172.21.15.140:32984) 2022-07-21T01:48:49.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f6b7fe700 1 -- 172.21.15.140:0/3425896759 learned_addr learned my addr 172.21.15.140:0/3425896759 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:49.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f6affd700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f6c061a50 0x7f4f6c063df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:49.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f6bfff700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f6c09aa30 0x7f4f6c0642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:49.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f6b7fe700 1 -- 172.21.15.140:0/3425896759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f6c061a50 msgr2=0x7f4f6c063df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:49.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f6b7fe700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f6c061a50 0x7f4f6c063df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:49.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f6b7fe700 1 -- 172.21.15.140:0/3425896759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f6c09aa30 msgr2=0x7f4f6c0642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:49.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f6b7fe700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f6c09aa30 0x7f4f6c0642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:49.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f6b7fe700 1 -- 172.21.15.140:0/3425896759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c00b180 con 0x7f4f6c05ce60 2022-07-21T01:48:49.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f6b7fe700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f6c05ce60 0x7f4f6c0af270 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f4f6c099800 tx=0x7f4f5c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:49.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f68ff9700 1 -- 172.21.15.140:0/3425896759 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f5c00bec0 con 0x7f4f6c05ce60 2022-07-21T01:48:49.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f68ff9700 1 -- 172.21.15.140:0/3425896759 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c011b00 con 0x7f4f6c05ce60 2022-07-21T01:48:49.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f70d2d700 1 -- 172.21.15.140:0/3425896759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c067390 con 0x7f4f6c05ce60 2022-07-21T01:48:49.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f68ff9700 1 -- 172.21.15.140:0/3425896759 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f5c011e60 con 0x7f4f6c05ce60 2022-07-21T01:48:49.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.971+0000 7f4f70d2d700 1 -- 172.21.15.140:0/3425896759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c0647d0 con 0x7f4f6c05ce60 2022-07-21T01:48:49.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.975+0000 7f4f70d2d700 1 -- 172.21.15.140:0/3425896759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c05cbb0 con 0x7f4f6c05ce60 2022-07-21T01:48:49.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.975+0000 7f4f68ff9700 1 -- 172.21.15.140:0/3425896759 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4f5c025030 con 0x7f4f6c05ce60 2022-07-21T01:48:49.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.975+0000 7f4f68ff9700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f4c053a10 0x7f4f4c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:49.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.975+0000 7f4f68ff9700 1 -- 172.21.15.140:0/3425896759 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4f5c06d160 con 0x7f4f6c05ce60 2022-07-21T01:48:49.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.975+0000 7f4f6affd700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f4c053a10 0x7f4f4c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:49.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.975+0000 7f4f6affd700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f4c053a10 0x7f4f4c055ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f4f6000d000 tx=0x7f4f60007c00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:49.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:49.983+0000 7f4f68ff9700 1 -- 172.21.15.140:0/3425896759 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4f5c044080 con 0x7f4f6c05ce60 2022-07-21T01:48:50.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.111+0000 7f4f70d2d700 1 -- 172.21.15.140:0/3425896759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f6c0574e0 con 0x7f4f6c05ce60 2022-07-21T01:48:50.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.111+0000 7f4f68ff9700 1 -- 172.21.15.140:0/3425896759 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f4f5c023090 con 0x7f4f6c05ce60 2022-07-21T01:48:50.110 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:50.111 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:50.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.111+0000 7f4f70d2d700 1 -- 172.21.15.140:0/3425896759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f4c053a10 msgr2=0x7f4f4c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.111+0000 7f4f70d2d700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f4c053a10 0x7f4f4c055ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f4f6000d000 tx=0x7f4f60007c00).stop 2022-07-21T01:48:50.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.111+0000 7f4f70d2d700 1 -- 172.21.15.140:0/3425896759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f6c05ce60 msgr2=0x7f4f6c0af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.111+0000 7f4f70d2d700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f6c05ce60 0x7f4f6c0af270 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f4f6c099800 tx=0x7f4f5c015000).stop 2022-07-21T01:48:50.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.115+0000 7f4f70d2d700 1 -- 172.21.15.140:0/3425896759 shutdown_connections 2022-07-21T01:48:50.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.115+0000 7f4f70d2d700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4f4c053a10 0x7f4f4c055ed0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.115+0000 7f4f70d2d700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f6c09aa30 0x7f4f6c0642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.115+0000 7f4f70d2d700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f6c061a50 0x7f4f6c063df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.115+0000 7f4f70d2d700 1 --2- 172.21.15.140:0/3425896759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4f6c05ce60 0x7f4f6c0af270 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.115+0000 7f4f70d2d700 1 -- 172.21.15.140:0/3425896759 >> 172.21.15.140:0/3425896759 conn(0x7f4f6c0b2ca0 msgr2=0x7f4f6c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:50.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.115+0000 7f4f70d2d700 1 -- 172.21.15.140:0/3425896759 shutdown_connections 2022-07-21T01:48:50.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.115+0000 7f4f70d2d700 1 -- 172.21.15.140:0/3425896759 wait complete. 2022-07-21T01:48:50.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:48:29.344690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:48:30.427054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:48:50.127 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:48:50.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.235+0000 7efe43e67700 1 Processor -- start 2022-07-21T01:48:50.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.235+0000 7efe43e67700 1 -- start start 2022-07-21T01:48:50.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe3c0b5b50 0x7efe3c0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7efe3c05c120 0x7efe3c144320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c0ac5e0 con 0x7efe3c0b0480 2022-07-21T01:48:50.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c0b5f30 con 0x7efe3c05c120 2022-07-21T01:48:50.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c05cf70 con 0x7efe3c0b5b50 2022-07-21T01:48:50.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe42404700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7efe3c05c120 0x7efe3c144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe42404700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7efe3c05c120 0x7efe3c144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32994/0 (socket says 172.21.15.140:32994) 2022-07-21T01:48:50.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe42404700 1 -- 172.21.15.140:0/741230176 learned_addr learned my addr 172.21.15.140:0/741230176 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:50.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe41c03700 1 --2- 172.21.15.140:0/741230176 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe3c0b5b50 0x7efe3c0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe42404700 1 -- 172.21.15.140:0/741230176 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe3c0b5b50 msgr2=0x7efe3c0accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:50.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe42404700 1 --2- 172.21.15.140:0/741230176 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe3c0b5b50 0x7efe3c0accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe42404700 1 -- 172.21.15.140:0/741230176 >> v1:172.21.15.152:6789/0 conn(0x7efe3c0b0480 legacy=0x7efe3c14a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe42404700 1 -- 172.21.15.140:0/741230176 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c0ac810 con 0x7efe3c05c120 2022-07-21T01:48:50.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe42404700 1 --2- 172.21.15.140:0/741230176 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe3c05c120 0x7efe3c144320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7efe34002f70 tx=0x7efe34006eb0).ready entity=mon.0 client_cookie=6a12dfd85201f17c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:50.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe40c01700 1 -- 172.21.15.140:0/741230176 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe3400d570 con 0x7efe3c05c120 2022-07-21T01:48:50.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe40c01700 1 -- 172.21.15.140:0/741230176 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe34000e90 con 0x7efe3c05c120 2022-07-21T01:48:50.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe40c01700 1 -- 172.21.15.140:0/741230176 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe3400d900 con 0x7efe3c05c120 2022-07-21T01:48:50.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 -- 172.21.15.140:0/741230176 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe3c05c120 msgr2=0x7efe3c144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 --2- 172.21.15.140:0/741230176 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe3c05c120 0x7efe3c144320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7efe34002f70 tx=0x7efe34006eb0).stop 2022-07-21T01:48:50.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 -- 172.21.15.140:0/741230176 shutdown_connections 2022-07-21T01:48:50.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 --2- 172.21.15.140:0/741230176 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe3c05c120 0x7efe3c144320 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 --2- 172.21.15.140:0/741230176 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe3c0b5b50 0x7efe3c0accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 -- 172.21.15.140:0/741230176 >> 172.21.15.140:0/741230176 conn(0x7efe3c0635e0 msgr2=0x7efe3c0b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:50.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 -- 172.21.15.140:0/741230176 shutdown_connections 2022-07-21T01:48:50.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.239+0000 7efe43e67700 1 -- 172.21.15.140:0/741230176 wait complete. 2022-07-21T01:48:50.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe43e67700 1 Processor -- start 2022-07-21T01:48:50.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe43e67700 1 -- start start 2022-07-21T01:48:50.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe43e67700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe3c05c120 0x7efe3c0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe43e67700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe3c0b0480 0x7efe3c063a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe43e67700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe3c0b5b50 0x7efe3c156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe43e67700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c09ee90 con 0x7efe3c05c120 2022-07-21T01:48:50.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe43e67700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c064130 con 0x7efe3c0b0480 2022-07-21T01:48:50.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe43e67700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0643d0 con 0x7efe3c0b5b50 2022-07-21T01:48:50.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe41c03700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe3c05c120 0x7efe3c0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe41c03700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe3c05c120 0x7efe3c0ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:32996/0 (socket says 172.21.15.140:32996) 2022-07-21T01:48:50.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe41c03700 1 -- 172.21.15.140:0/3105155157 learned_addr learned my addr 172.21.15.140:0/3105155157 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:50.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe42404700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe3c0b5b50 0x7efe3c156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe41402700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe3c0b0480 0x7efe3c063a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe41c03700 1 -- 172.21.15.140:0/3105155157 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe3c0b5b50 msgr2=0x7efe3c156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe41c03700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe3c0b5b50 0x7efe3c156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe41c03700 1 -- 172.21.15.140:0/3105155157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe3c0b0480 msgr2=0x7efe3c063a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe41c03700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe3c0b0480 0x7efe3c063a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe41c03700 1 -- 172.21.15.140:0/3105155157 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3400b1e0 con 0x7efe3c05c120 2022-07-21T01:48:50.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe41c03700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe3c05c120 0x7efe3c0ae5a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7efe2c00c000 tx=0x7efe2c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:50.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe2affd700 1 -- 172.21.15.140:0/3105155157 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe2c011750 con 0x7efe3c05c120 2022-07-21T01:48:50.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe2affd700 1 -- 172.21.15.140:0/3105155157 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c0118b0 con 0x7efe3c05c120 2022-07-21T01:48:50.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe43e67700 1 -- 172.21.15.140:0/3105155157 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c064670 con 0x7efe3c05c120 2022-07-21T01:48:50.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.243+0000 7efe2affd700 1 -- 172.21.15.140:0/3105155157 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe2c011c10 con 0x7efe3c05c120 2022-07-21T01:48:50.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.247+0000 7efe43e67700 1 -- 172.21.15.140:0/3105155157 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c1569c0 con 0x7efe3c05c120 2022-07-21T01:48:50.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.247+0000 7efe43e67700 1 -- 172.21.15.140:0/3105155157 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c0a0ac0 con 0x7efe3c05c120 2022-07-21T01:48:50.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.247+0000 7efe2affd700 1 -- 172.21.15.140:0/3105155157 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efe2c011d70 con 0x7efe3c05c120 2022-07-21T01:48:50.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.247+0000 7efe2affd700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe1c053b30 0x7efe1c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.247+0000 7efe2affd700 1 -- 172.21.15.140:0/3105155157 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efe2c06c330 con 0x7efe3c05c120 2022-07-21T01:48:50.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.247+0000 7efe41402700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe1c053b30 0x7efe1c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.247+0000 7efe41402700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe1c053b30 0x7efe1c055ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7efe3800d000 tx=0x7efe38000f40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:50.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.255+0000 7efe2affd700 1 -- 172.21.15.140:0/3105155157 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efe3c0a0ac0 con 0x7efe3c05c120 2022-07-21T01:48:50.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.379+0000 7efe43e67700 1 -- 172.21.15.140:0/3105155157 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7efe3c0a0ac0 con 0x7efe3c05c120 2022-07-21T01:48:50.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.379+0000 7efe2affd700 1 -- 172.21.15.140:0/3105155157 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7efe3c0a0ac0 con 0x7efe3c05c120 2022-07-21T01:48:50.380 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:48:50.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 -- 172.21.15.140:0/3105155157 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe1c053b30 msgr2=0x7efe1c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe1c053b30 0x7efe1c055ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7efe3800d000 tx=0x7efe38000f40).stop 2022-07-21T01:48:50.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 -- 172.21.15.140:0/3105155157 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe3c05c120 msgr2=0x7efe3c0ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe3c05c120 0x7efe3c0ae5a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7efe2c00c000 tx=0x7efe2c006df0).stop 2022-07-21T01:48:50.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 -- 172.21.15.140:0/3105155157 shutdown_connections 2022-07-21T01:48:50.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe1c053b30 0x7efe1c055ff0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe3c0b5b50 0x7efe3c156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe3c0b0480 0x7efe3c063a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 --2- 172.21.15.140:0/3105155157 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe3c05c120 0x7efe3c0ae5a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 -- 172.21.15.140:0/3105155157 >> 172.21.15.140:0/3105155157 conn(0x7efe3c0635e0 msgr2=0x7efe3c1440f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:50.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 -- 172.21.15.140:0/3105155157 shutdown_connections 2022-07-21T01:48:50.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.383+0000 7efe43e67700 1 -- 172.21.15.140:0/3105155157 wait complete. 2022-07-21T01:48:50.395 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:48:50.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 Processor -- start 2022-07-21T01:48:50.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 -- start start 2022-07-21T01:48:50.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0c380b59d0 0x7f0c380b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0c3814ada0 0x7f0c380b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0c380b14f0 0x7f0c3805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c38148080 con 0x7f0c3814ada0 2022-07-21T01:48:50.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c380afbc0 con 0x7f0c380b14f0 2022-07-21T01:48:50.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c380b00e0 con 0x7f0c380b59d0 2022-07-21T01:48:50.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c2dd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0c3814ada0 0x7f0c380b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c2dd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0c3814ada0 0x7f0c380b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33006/0 (socket says 172.21.15.140:33006) 2022-07-21T01:48:50.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c2dd9b700 1 -- 172.21.15.140:0/960003492 learned_addr learned my addr 172.21.15.140:0/960003492 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:50.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3659c700 1 --2- 172.21.15.140:0/960003492 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0c380b59d0 0x7f0c380b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c36d9d700 1 --2- 172.21.15.140:0/960003492 >> v2:172.21.15.152:3300/0 conn(0x7f0c380b14f0 0x7f0c3805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c2dd9b700 1 -- 172.21.15.140:0/960003492 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0c380b59d0 msgr2=0x7f0c380b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:50.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c2dd9b700 1 --2- 172.21.15.140:0/960003492 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0c380b59d0 0x7f0c380b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c2dd9b700 1 -- 172.21.15.140:0/960003492 >> v2:172.21.15.152:3300/0 conn(0x7f0c380b14f0 msgr2=0x7f0c3805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:50.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c2dd9b700 1 --2- 172.21.15.140:0/960003492 >> v2:172.21.15.152:3300/0 conn(0x7f0c380b14f0 0x7f0c3805bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c2dd9b700 1 -- 172.21.15.140:0/960003492 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c380b5db0 con 0x7f0c3814ada0 2022-07-21T01:48:50.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c2dd9b700 1 --2- 172.21.15.140:0/960003492 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0c3814ada0 0x7f0c380b0cd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f0c20002f70 tx=0x7f0c20006eb0).ready entity=mon.0 client_cookie=2171fabefe42ddd1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:50.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c35d9b700 1 -- 172.21.15.140:0/960003492 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c2000d570 con 0x7f0c3814ada0 2022-07-21T01:48:50.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c35d9b700 1 -- 172.21.15.140:0/960003492 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c20000e90 con 0x7f0c3814ada0 2022-07-21T01:48:50.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c35d9b700 1 -- 172.21.15.140:0/960003492 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c2000d900 con 0x7f0c3814ada0 2022-07-21T01:48:50.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 -- 172.21.15.140:0/960003492 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0c3814ada0 msgr2=0x7f0c380b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 --2- 172.21.15.140:0/960003492 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0c3814ada0 0x7f0c380b0cd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f0c20002f70 tx=0x7f0c20006eb0).stop 2022-07-21T01:48:50.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 -- 172.21.15.140:0/960003492 shutdown_connections 2022-07-21T01:48:50.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 --2- 172.21.15.140:0/960003492 >> v2:172.21.15.152:3300/0 conn(0x7f0c380b14f0 0x7f0c3805bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 --2- 172.21.15.140:0/960003492 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0c3814ada0 0x7f0c380b0cd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 --2- 172.21.15.140:0/960003492 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0c380b59d0 0x7f0c380b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 -- 172.21.15.140:0/960003492 >> 172.21.15.140:0/960003492 conn(0x7f0c380634c0 msgr2=0x7f0c380b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:50.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 -- 172.21.15.140:0/960003492 shutdown_connections 2022-07-21T01:48:50.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.507+0000 7f0c3759e700 1 -- 172.21.15.140:0/960003492 wait complete. 2022-07-21T01:48:50.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c3759e700 1 Processor -- start 2022-07-21T01:48:50.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c3759e700 1 -- start start 2022-07-21T01:48:50.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c3759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0c380b14f0 0x7f0c38066fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c3759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c380b59d0 0x7f0c380639e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c3759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0c3814ada0 0x7f0c38063ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c3759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c38066790 con 0x7f0c3814ada0 2022-07-21T01:48:50.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c3759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3814a970 con 0x7f0c380b59d0 2022-07-21T01:48:50.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c3759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3814a6d0 con 0x7f0c380b14f0 2022-07-21T01:48:50.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c36d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0c3814ada0 0x7f0c38063ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c36d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0c3814ada0 0x7f0c38063ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33012/0 (socket says 172.21.15.140:33012) 2022-07-21T01:48:50.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c36d9d700 1 -- 172.21.15.140:0/746003771 learned_addr learned my addr 172.21.15.140:0/746003771 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:50.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c3659c700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0c380b14f0 0x7f0c38066fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c2dd9b700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c380b59d0 0x7f0c380639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c36d9d700 1 -- 172.21.15.140:0/746003771 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0c380b14f0 msgr2=0x7f0c38066fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c36d9d700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0c380b14f0 0x7f0c38066fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c36d9d700 1 -- 172.21.15.140:0/746003771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c380b59d0 msgr2=0x7f0c380639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c36d9d700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c380b59d0 0x7f0c380639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c36d9d700 1 -- 172.21.15.140:0/746003771 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2000b1e0 con 0x7f0c3814ada0 2022-07-21T01:48:50.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c36d9d700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0c3814ada0 0x7f0c38063ed0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f0c3000a000 tx=0x7f0c30008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:50.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c2ffff700 1 -- 172.21.15.140:0/746003771 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c30013620 con 0x7f0c3814ada0 2022-07-21T01:48:50.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c3759e700 1 -- 172.21.15.140:0/746003771 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c380675e0 con 0x7f0c3814ada0 2022-07-21T01:48:50.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c2ffff700 1 -- 172.21.15.140:0/746003771 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c30013780 con 0x7f0c3814ada0 2022-07-21T01:48:50.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c3759e700 1 -- 172.21.15.140:0/746003771 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c38149690 con 0x7f0c3814ada0 2022-07-21T01:48:50.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.511+0000 7f0c2ffff700 1 -- 172.21.15.140:0/746003771 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c30005c60 con 0x7f0c3814ada0 2022-07-21T01:48:50.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.515+0000 7f0c3759e700 1 -- 172.21.15.140:0/746003771 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c380b0340 con 0x7f0c3814ada0 2022-07-21T01:48:50.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.515+0000 7f0c2ffff700 1 -- 172.21.15.140:0/746003771 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0c300138f0 con 0x7f0c3814ada0 2022-07-21T01:48:50.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.515+0000 7f0c2ffff700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0c1c053a70 0x7f0c1c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.515+0000 7f0c2ffff700 1 -- 172.21.15.140:0/746003771 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0c3006c2c0 con 0x7f0c3814ada0 2022-07-21T01:48:50.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.515+0000 7f0c3659c700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0c1c053a70 0x7f0c1c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.515+0000 7f0c3659c700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0c1c053a70 0x7f0c1c055f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f0c2800b000 tx=0x7f0c28006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:50.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.519+0000 7f0c2ffff700 1 -- 172.21.15.140:0/746003771 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0c30042ff0 con 0x7f0c3814ada0 2022-07-21T01:48:50.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 -- 172.21.15.140:0/746003771 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0c380574e0 con 0x7f0c3814ada0 2022-07-21T01:48:50.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c2ffff700 1 -- 172.21.15.140:0/746003771 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0c30046610 con 0x7f0c3814ada0 2022-07-21T01:48:50.638 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:50.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 -- 172.21.15.140:0/746003771 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0c1c053a70 msgr2=0x7f0c1c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0c1c053a70 0x7f0c1c055f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f0c2800b000 tx=0x7f0c28006df0).stop 2022-07-21T01:48:50.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 -- 172.21.15.140:0/746003771 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0c3814ada0 msgr2=0x7f0c38063ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0c3814ada0 0x7f0c38063ed0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f0c3000a000 tx=0x7f0c30008040).stop 2022-07-21T01:48:50.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 -- 172.21.15.140:0/746003771 shutdown_connections 2022-07-21T01:48:50.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0c1c053a70 0x7f0c1c055f30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0c3814ada0 0x7f0c38063ed0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c380b59d0 0x7f0c380639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 --2- 172.21.15.140:0/746003771 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0c380b14f0 0x7f0c38066fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 -- 172.21.15.140:0/746003771 >> 172.21.15.140:0/746003771 conn(0x7f0c380634c0 msgr2=0x7f0c380adc30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:50.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 -- 172.21.15.140:0/746003771 shutdown_connections 2022-07-21T01:48:50.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.639+0000 7f0c3759e700 1 -- 172.21.15.140:0/746003771 wait complete. 2022-07-21T01:48:50.650 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:48:50.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.759+0000 7efe64812700 1 Processor -- start 2022-07-21T01:48:50.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.759+0000 7efe64812700 1 -- start start 2022-07-21T01:48:50.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.759+0000 7efe64812700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efe600af940 0x7efe600b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.759+0000 7efe64812700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6005cc70 con 0x7efe600b59f0 2022-07-21T01:48:50.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.759+0000 7efe64812700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe600b5dd0 con 0x7efe60147ab0 2022-07-21T01:48:50.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.759+0000 7efe64812700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6005c760 con 0x7efe600af940 2022-07-21T01:48:50.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5ed9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7efe600b59f0 0x7efe601473a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47164/0 (socket says 172.21.15.140:47164) 2022-07-21T01:48:50.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5ed9d700 1 -- 172.21.15.140:0/4215703229 learned_addr learned my addr 172.21.15.140:0/4215703229 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:50.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5e59c700 1 --2- 172.21.15.140:0/4215703229 >> v2:172.21.15.152:3300/0 conn(0x7efe600af940 0x7efe600b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061079974 0 0) 0x7efe6005cc70 con 0x7efe600b59f0 2022-07-21T01:48:50.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe40003260 con 0x7efe600b59f0 2022-07-21T01:48:50.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832757433 0 0) 0x7efe600b5dd0 con 0x7efe60147ab0 2022-07-21T01:48:50.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe6005cc70 con 0x7efe60147ab0 2022-07-21T01:48:50.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 884237861 0 0) 0x7efe40003260 con 0x7efe600b59f0 2022-07-21T01:48:50.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 >> v2:172.21.15.152:3300/0 conn(0x7efe600af940 msgr2=0x7efe600b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:50.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 --2- 172.21.15.140:0/4215703229 >> v2:172.21.15.152:3300/0 conn(0x7efe600af940 0x7efe600b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 >> v1:172.21.15.152:6789/0 conn(0x7efe60147ab0 legacy=0x7efe600b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6005c9d0 con 0x7efe600b59f0 2022-07-21T01:48:50.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7efe50001660 con 0x7efe600b59f0 2022-07-21T01:48:50.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 >> v1:172.21.15.140:6789/0 conn(0x7efe600b59f0 legacy=0x7efe601473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 --2- 172.21.15.140:0/4215703229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe400038e0 0x7efe40003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 --2- 172.21.15.140:0/4215703229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe40004500 0x7efe40004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 --2- 172.21.15.140:0/4215703229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe40004e70 0x7efe4000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6005c760 con 0x7efe40004500 2022-07-21T01:48:50.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4000bdd0 con 0x7efe40004e70 2022-07-21T01:48:50.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4000bf40 con 0x7efe400038e0 2022-07-21T01:48:50.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5e59c700 1 --2- 172.21.15.140:0/4215703229 >> v2:172.21.15.152:3300/0 conn(0x7efe600af940 0x7efe600b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:50.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5e59c700 1 --2- 172.21.15.140:0/4215703229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe40004500 0x7efe40004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5f59e700 1 --2- 172.21.15.140:0/4215703229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe40004e70 0x7efe4000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5ed9d700 1 --2- 172.21.15.140:0/4215703229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe400038e0 0x7efe40003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5e59c700 1 -- 172.21.15.140:0/4215703229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe400038e0 msgr2=0x7efe40003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:50.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5e59c700 1 --2- 172.21.15.140:0/4215703229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe400038e0 0x7efe40003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5e59c700 1 -- 172.21.15.140:0/4215703229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe40004e70 msgr2=0x7efe4000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:50.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5e59c700 1 --2- 172.21.15.140:0/4215703229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe40004e70 0x7efe4000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5f59e700 1 -- 172.21.15.140:0/4215703229 reap_dead start 2022-07-21T01:48:50.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5e59c700 1 -- 172.21.15.140:0/4215703229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe40003260 con 0x7efe40004500 2022-07-21T01:48:50.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5e59c700 1 --2- 172.21.15.140:0/4215703229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe40004500 0x7efe40004980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7efe4800c000 tx=0x7efe48006df0).ready entity=mon.0 client_cookie=c5789576c95a32cc server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:50.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe4800e830 con 0x7efe40004500 2022-07-21T01:48:50.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4800e990 con 0x7efe40004500 2022-07-21T01:48:50.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe5dd9b700 1 -- 172.21.15.140:0/4215703229 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe4800ecf0 con 0x7efe40004500 2022-07-21T01:48:50.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe64812700 1 -- 172.21.15.140:0/4215703229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe40004500 msgr2=0x7efe40004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe64812700 1 --2- 172.21.15.140:0/4215703229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe40004500 0x7efe40004980 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7efe4800c000 tx=0x7efe48006df0).stop 2022-07-21T01:48:50.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe64812700 1 -- 172.21.15.140:0/4215703229 shutdown_connections 2022-07-21T01:48:50.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe64812700 1 --2- 172.21.15.140:0/4215703229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe40004500 0x7efe40004980 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe64812700 1 -- 172.21.15.140:0/4215703229 >> 172.21.15.140:0/4215703229 conn(0x7efe60063480 msgr2=0x7efe600b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:50.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe64812700 1 -- 172.21.15.140:0/4215703229 shutdown_connections 2022-07-21T01:48:50.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.763+0000 7efe64812700 1 -- 172.21.15.140:0/4215703229 wait complete. 2022-07-21T01:48:50.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.767+0000 7efe64812700 1 Processor -- start 2022-07-21T01:48:50.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.767+0000 7efe64812700 1 -- start start 2022-07-21T01:48:50.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.767+0000 7efe64812700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe40004500 0x7efe601498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.767+0000 7efe64812700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe60144a40 0x7efe6014a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.767+0000 7efe64812700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe601482c0 0x7efe601103e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe64812700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6014ae00 con 0x7efe60144a40 2022-07-21T01:48:50.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe64812700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6014b050 con 0x7efe601482c0 2022-07-21T01:48:50.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe64812700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6010ff40 con 0x7efe40004500 2022-07-21T01:48:50.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe60144a40 0x7efe6014a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe60144a40 0x7efe6014a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33032/0 (socket says 172.21.15.140:33032) 2022-07-21T01:48:50.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5ed9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe40004500 0x7efe601498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5e59c700 1 -- 172.21.15.140:0/2873225086 learned_addr learned my addr 172.21.15.140:0/2873225086 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:50.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5f59e700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe601482c0 0x7efe601103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5e59c700 1 -- 172.21.15.140:0/2873225086 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe40004500 msgr2=0x7efe601498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5e59c700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe40004500 0x7efe601498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5e59c700 1 -- 172.21.15.140:0/2873225086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe601482c0 msgr2=0x7efe601103e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5e59c700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe601482c0 0x7efe601103e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5e59c700 1 -- 172.21.15.140:0/2873225086 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe60148770 con 0x7efe60144a40 2022-07-21T01:48:50.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5e59c700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe60144a40 0x7efe6014a910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7efe4800c000 tx=0x7efe48019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:50.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe57fff700 1 -- 172.21.15.140:0/2873225086 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe4801c9d0 con 0x7efe60144a40 2022-07-21T01:48:50.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe57fff700 1 -- 172.21.15.140:0/2873225086 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe48000e90 con 0x7efe60144a40 2022-07-21T01:48:50.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe64812700 1 -- 172.21.15.140:0/2873225086 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe601108d0 con 0x7efe60144a40 2022-07-21T01:48:50.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe57fff700 1 -- 172.21.15.140:0/2873225086 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe4801cd20 con 0x7efe60144a40 2022-07-21T01:48:50.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe64812700 1 -- 172.21.15.140:0/2873225086 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe60155520 con 0x7efe60144a40 2022-07-21T01:48:50.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe64812700 1 -- 172.21.15.140:0/2873225086 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6014b210 con 0x7efe60144a40 2022-07-21T01:48:50.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe57fff700 1 -- 172.21.15.140:0/2873225086 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efe480243f0 con 0x7efe60144a40 2022-07-21T01:48:50.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe57fff700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe44053a70 0x7efe44055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:50.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe57fff700 1 -- 172.21.15.140:0/2873225086 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efe4806d5a0 con 0x7efe60144a40 2022-07-21T01:48:50.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.771+0000 7efe5ed9d700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe44053a70 0x7efe44055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:50.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.775+0000 7efe5ed9d700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe44053a70 0x7efe44055f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7efe50005000 tx=0x7efe6005cb60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:50.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.775+0000 7efe57fff700 1 -- 172.21.15.140:0/2873225086 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efe48047dc0 con 0x7efe60144a40 2022-07-21T01:48:50.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.899+0000 7efe64812700 1 -- 172.21.15.140:0/2873225086 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe60146f00 con 0x7efe60144a40 2022-07-21T01:48:50.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe57fff700 1 -- 172.21.15.140:0/2873225086 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7efe480479b0 con 0x7efe60144a40 2022-07-21T01:48:50.901 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:50.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 -- 172.21.15.140:0/2873225086 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe44053a70 msgr2=0x7efe44055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe44053a70 0x7efe44055f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7efe50005000 tx=0x7efe6005cb60).stop 2022-07-21T01:48:50.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 -- 172.21.15.140:0/2873225086 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe60144a40 msgr2=0x7efe6014a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:50.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe60144a40 0x7efe6014a910 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7efe4800c000 tx=0x7efe48019000).stop 2022-07-21T01:48:50.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 -- 172.21.15.140:0/2873225086 shutdown_connections 2022-07-21T01:48:50.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efe44053a70 0x7efe44055f30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe601482c0 0x7efe601103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe60144a40 0x7efe6014a910 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 --2- 172.21.15.140:0/2873225086 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe40004500 0x7efe601498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:50.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 -- 172.21.15.140:0/2873225086 >> 172.21.15.140:0/2873225086 conn(0x7efe60063480 msgr2=0x7efe600b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:50.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 -- 172.21.15.140:0/2873225086 shutdown_connections 2022-07-21T01:48:50.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:50.903+0000 7efe64812700 1 -- 172.21.15.140:0/2873225086 wait complete. 2022-07-21T01:48:50.918 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:48:51.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57f1adf700 1 Processor -- start 2022-07-21T01:48:51.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57f1adf700 1 -- start start 2022-07-21T01:48:51.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57f1adf700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f57ec05cf40 0x7f57ec0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57f1adf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec0b1220 0x7f57ec14a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57f1adf700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ec1472b0 con 0x7f57ec147c90 2022-07-21T01:48:51.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57f1adf700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec0b0920 con 0x7f57ec05cf40 2022-07-21T01:48:51.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57f1adf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec147550 con 0x7f57ec0b1220 2022-07-21T01:48:51.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57eaffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f57ec147c90 0x7f57ec0b0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47186/0 (socket says 172.21.15.140:47186) 2022-07-21T01:48:51.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57eaffd700 1 -- 172.21.15.140:0/1995953965 learned_addr learned my addr 172.21.15.140:0/1995953965 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:51.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57eb7fe700 1 --2- 172.21.15.140:0/1995953965 >> v2:172.21.15.152:3300/0 conn(0x7f57ec05cf40 0x7f57ec0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ebfff700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec0b1220 0x7f57ec14a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516123651 0 0) 0x7f57ec1472b0 con 0x7f57ec147c90 2022-07-21T01:48:51.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57cc003260 con 0x7f57ec147c90 2022-07-21T01:48:51.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3466088051 0 0) 0x7f57cc003260 con 0x7f57ec147c90 2022-07-21T01:48:51.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec0b1220 msgr2=0x7f57ec14a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:51.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec0b1220 0x7f57ec14a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 >> v2:172.21.15.152:3300/0 conn(0x7f57ec05cf40 msgr2=0x7f57ec0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:51.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 --2- 172.21.15.140:0/1995953965 >> v2:172.21.15.152:3300/0 conn(0x7f57ec05cf40 0x7f57ec0afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec1477f0 con 0x7f57ec147c90 2022-07-21T01:48:51.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f57d4001660 con 0x7f57ec147c90 2022-07-21T01:48:51.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 >> v1:172.21.15.140:6789/0 conn(0x7f57ec147c90 legacy=0x7f57ec0b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec0b1220 0x7f57cc003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 0x7f57cc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57cc0049f0 0x7f57cc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0b0920 con 0x7f57cc003c80 2022-07-21T01:48:51.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec147550 con 0x7f57cc0049f0 2022-07-21T01:48:51.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc00b750 con 0x7f57ec0b1220 2022-07-21T01:48:51.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57eaffd700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 0x7f57cc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57ebfff700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57cc0049f0 0x7f57cc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57eb7fe700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec0b1220 0x7f57cc003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57eaffd700 1 -- 172.21.15.140:0/1995953965 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec0b1220 msgr2=0x7f57cc003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:51.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57eaffd700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec0b1220 0x7f57cc003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57eaffd700 1 -- 172.21.15.140:0/1995953965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57cc0049f0 msgr2=0x7f57cc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:51.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57eaffd700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57cc0049f0 0x7f57cc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.027+0000 7f57eaffd700 1 -- 172.21.15.140:0/1995953965 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc003260 con 0x7f57cc003c80 2022-07-21T01:48:51.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57eaffd700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 0x7f57cc004500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f57cc0054d0 tx=0x7f57ec0b22c0).ready entity=mon.0 client_cookie=5726107f2b736359 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:51.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d40071a0 con 0x7f57cc003c80 2022-07-21T01:48:51.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d4007300 con 0x7f57cc003c80 2022-07-21T01:48:51.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57ea7fc700 1 -- 172.21.15.140:0/1995953965 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d400cb70 con 0x7f57cc003c80 2022-07-21T01:48:51.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57f1adf700 1 -- 172.21.15.140:0/1995953965 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 msgr2=0x7f57cc004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57f1adf700 1 --2- 172.21.15.140:0/1995953965 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 0x7f57cc004500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f57cc0054d0 tx=0x7f57ec0b22c0).stop 2022-07-21T01:48:51.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57ebfff700 1 -- 172.21.15.140:0/1995953965 reap_dead start 2022-07-21T01:48:51.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57f1adf700 1 -- 172.21.15.140:0/1995953965 shutdown_connections 2022-07-21T01:48:51.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57f1adf700 1 -- 172.21.15.140:0/1995953965 >> 172.21.15.140:0/1995953965 conn(0x7f57ec0b5b70 msgr2=0x7f57ec0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:51.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57f1adf700 1 -- 172.21.15.140:0/1995953965 shutdown_connections 2022-07-21T01:48:51.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57f1adf700 1 -- 172.21.15.140:0/1995953965 wait complete. 2022-07-21T01:48:51.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57f1adf700 1 Processor -- start 2022-07-21T01:48:51.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.031+0000 7f57f1adf700 1 -- start start 2022-07-21T01:48:51.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57f1adf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 0x7f57ec141f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57f1adf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec140820 0x7f57ec098f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57f1adf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57ec099450 0x7f57ec099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57f1adf700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec1439b0 con 0x7f57cc003c80 2022-07-21T01:48:51.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57f1adf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec142cc0 con 0x7f57ec099450 2022-07-21T01:48:51.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57f1adf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec142a20 con 0x7f57ec140820 2022-07-21T01:48:51.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57eb7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 0x7f57ec141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57eb7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 0x7f57ec141f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33050/0 (socket says 172.21.15.140:33050) 2022-07-21T01:48:51.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57eb7fe700 1 -- 172.21.15.140:0/3314561797 learned_addr learned my addr 172.21.15.140:0/3314561797 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:51.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57eaffd700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec140820 0x7f57ec098f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57ebfff700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57ec099450 0x7f57ec099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57eb7fe700 1 -- 172.21.15.140:0/3314561797 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec140820 msgr2=0x7f57ec098f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57eb7fe700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec140820 0x7f57ec098f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57eb7fe700 1 -- 172.21.15.140:0/3314561797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57ec099450 msgr2=0x7f57ec099900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57eb7fe700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57ec099450 0x7f57ec099900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57eb7fe700 1 -- 172.21.15.140:0/3314561797 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec143e10 con 0x7f57cc003c80 2022-07-21T01:48:51.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57ebfff700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57ec099450 0x7f57ec099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:51.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57eb7fe700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 0x7f57ec141f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f57dc00b000 tx=0x7f57dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:51.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57e8ff9700 1 -- 172.21.15.140:0/3314561797 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57dc014680 con 0x7f57cc003c80 2022-07-21T01:48:51.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57e8ff9700 1 -- 172.21.15.140:0/3314561797 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc0147e0 con 0x7f57cc003c80 2022-07-21T01:48:51.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57f1adf700 1 -- 172.21.15.140:0/3314561797 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec1417c0 con 0x7f57cc003c80 2022-07-21T01:48:51.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57e8ff9700 1 -- 172.21.15.140:0/3314561797 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57dc014b40 con 0x7f57cc003c80 2022-07-21T01:48:51.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57f1adf700 1 -- 172.21.15.140:0/3314561797 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec067ac0 con 0x7f57cc003c80 2022-07-21T01:48:51.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57f1adf700 1 -- 172.21.15.140:0/3314561797 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ec143b70 con 0x7f57cc003c80 2022-07-21T01:48:51.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.035+0000 7f57e8ff9700 1 -- 172.21.15.140:0/3314561797 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f57dc024070 con 0x7f57cc003c80 2022-07-21T01:48:51.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.039+0000 7f57e8ff9700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57d0053b00 0x7f57d0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.039+0000 7f57e8ff9700 1 -- 172.21.15.140:0/3314561797 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f57dc06c790 con 0x7f57cc003c80 2022-07-21T01:48:51.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.039+0000 7f57eaffd700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57d0053b00 0x7f57d0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.039+0000 7f57eaffd700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57d0053b00 0x7f57d0055fc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f57d4019000 tx=0x7f57d4013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:51.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.047+0000 7f57e8ff9700 1 -- 172.21.15.140:0/3314561797 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57ec143b70 con 0x7f57cc003c80 2022-07-21T01:48:51.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.171+0000 7f57f1adf700 1 -- 172.21.15.140:0/3314561797 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57ec143b70 con 0x7f57cc003c80 2022-07-21T01:48:51.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.171+0000 7f57e8ff9700 1 -- 172.21.15.140:0/3314561797 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f57ec143b70 con 0x7f57cc003c80 2022-07-21T01:48:51.177 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:51.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.171+0000 7f57f1adf700 1 -- 172.21.15.140:0/3314561797 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57d0053b00 msgr2=0x7f57d0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.171+0000 7f57f1adf700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57d0053b00 0x7f57d0055fc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f57d4019000 tx=0x7f57d4013000).stop 2022-07-21T01:48:51.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.175+0000 7f57f1adf700 1 -- 172.21.15.140:0/3314561797 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 msgr2=0x7f57ec141f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.175+0000 7f57f1adf700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 0x7f57ec141f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f57dc00b000 tx=0x7f57dc008040).stop 2022-07-21T01:48:51.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.175+0000 7f57f1adf700 1 -- 172.21.15.140:0/3314561797 shutdown_connections 2022-07-21T01:48:51.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.175+0000 7f57f1adf700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57d0053b00 0x7f57d0055fc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.175+0000 7f57f1adf700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57ec099450 0x7f57ec099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.175+0000 7f57f1adf700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57ec140820 0x7f57ec098f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.175+0000 7f57f1adf700 1 --2- 172.21.15.140:0/3314561797 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc003c80 0x7f57ec141f00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.175+0000 7f57f1adf700 1 -- 172.21.15.140:0/3314561797 >> 172.21.15.140:0/3314561797 conn(0x7f57ec0b5b70 msgr2=0x7f57ec14c870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:51.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.175+0000 7f57f1adf700 1 -- 172.21.15.140:0/3314561797 shutdown_connections 2022-07-21T01:48:51.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.175+0000 7f57f1adf700 1 -- 172.21.15.140:0/3314561797 wait complete. 2022-07-21T01:48:51.193 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:48:51.194 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:48:51.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.303+0000 7f7a3234d700 1 Processor -- start 2022-07-21T01:48:51.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.303+0000 7f7a3234d700 1 -- start start 2022-07-21T01:48:51.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.303+0000 7f7a3234d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7a2c0b5650 0x7f7a2c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.303+0000 7f7a3234d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c0af8e0 0x7f7a2c05bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.303+0000 7f7a3234d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2c147010 con 0x7f7a2c0adff0 2022-07-21T01:48:51.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.303+0000 7f7a3234d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a2c1472b0 con 0x7f7a2c0b5650 2022-07-21T01:48:51.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.303+0000 7f7a3234d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c146d40 con 0x7f7a2c0af8e0 2022-07-21T01:48:51.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a30b4a700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f7a2c0adff0 0x7f7a2c147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47204/0 (socket says 172.21.15.140:47204) 2022-07-21T01:48:51.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a30b4a700 1 -- 172.21.15.140:0/2857632831 learned_addr learned my addr 172.21.15.140:0/2857632831 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:51.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a31b4c700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c0af8e0 0x7f7a2c05bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a3134b700 1 --2- 172.21.15.140:0/2857632831 >> v2:172.21.15.152:3300/0 conn(0x7f7a2c0b5650 0x7f7a2c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857428431 0 0) 0x7f7a2c147010 con 0x7f7a2c0adff0 2022-07-21T01:48:51.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a18003260 con 0x7f7a2c0adff0 2022-07-21T01:48:51.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3753465300 0 0) 0x7f7a18003260 con 0x7f7a2c0adff0 2022-07-21T01:48:51.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c0af8e0 msgr2=0x7f7a2c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:51.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c0af8e0 0x7f7a2c05bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 >> v2:172.21.15.152:3300/0 conn(0x7f7a2c0b5650 msgr2=0x7f7a2c0b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:51.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 --2- 172.21.15.140:0/2857632831 >> v2:172.21.15.152:3300/0 conn(0x7f7a2c0b5650 0x7f7a2c0b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c0acae0 con 0x7f7a2c0adff0 2022-07-21T01:48:51.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f7a28001b60 con 0x7f7a2c0adff0 2022-07-21T01:48:51.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 >> v1:172.21.15.140:6789/0 conn(0x7f7a2c0adff0 legacy=0x7f7a2c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a18003750 0x7f7a18003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 0x7f7a18004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c0af8e0 0x7f7a18004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c1472b0 con 0x7f7a18004500 2022-07-21T01:48:51.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c146d40 con 0x7f7a18003750 2022-07-21T01:48:51.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1800b750 con 0x7f7a2c0af8e0 2022-07-21T01:48:51.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a30b4a700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 0x7f7a18004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a31b4c700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c0af8e0 0x7f7a18004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a3134b700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a18003750 0x7f7a18003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a30b4a700 1 -- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c0af8e0 msgr2=0x7f7a18004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:51.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a30b4a700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c0af8e0 0x7f7a18004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a30b4a700 1 -- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a18003750 msgr2=0x7f7a18003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:51.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a30b4a700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a18003750 0x7f7a18003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a30b4a700 1 -- 172.21.15.140:0/2857632831 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a18003260 con 0x7f7a18004500 2022-07-21T01:48:51.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a3134b700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a18003750 0x7f7a18003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:51.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a30b4a700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 0x7f7a18004980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f7a28001870 tx=0x7f7a28007990).ready entity=mon.0 client_cookie=2b1f226be59de6c0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:51.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a2800cc70 con 0x7f7a18004500 2022-07-21T01:48:51.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2800cdd0 con 0x7f7a18004500 2022-07-21T01:48:51.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a23fff700 1 -- 172.21.15.140:0/2857632831 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a28005a50 con 0x7f7a18004500 2022-07-21T01:48:51.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a3234d700 1 -- 172.21.15.140:0/2857632831 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 msgr2=0x7f7a18004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a3234d700 1 --2- 172.21.15.140:0/2857632831 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 0x7f7a18004980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f7a28001870 tx=0x7f7a28007990).stop 2022-07-21T01:48:51.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a31b4c700 1 -- 172.21.15.140:0/2857632831 reap_dead start 2022-07-21T01:48:51.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a3234d700 1 -- 172.21.15.140:0/2857632831 shutdown_connections 2022-07-21T01:48:51.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a3234d700 1 -- 172.21.15.140:0/2857632831 >> 172.21.15.140:0/2857632831 conn(0x7f7a2c0634e0 msgr2=0x7f7a2c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:51.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a3234d700 1 -- 172.21.15.140:0/2857632831 shutdown_connections 2022-07-21T01:48:51.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.307+0000 7f7a3234d700 1 -- 172.21.15.140:0/2857632831 wait complete. 2022-07-21T01:48:51.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3234d700 1 Processor -- start 2022-07-21T01:48:51.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3234d700 1 -- start start 2022-07-21T01:48:51.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3234d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 0x7f7a2c1454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3234d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c145990 0x7f7a2c142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3234d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c1434c0 0x7f7a2c159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3234d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c14a440 con 0x7f7a18004500 2022-07-21T01:48:51.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3234d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c14a6e0 con 0x7f7a2c1434c0 2022-07-21T01:48:51.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3234d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c14a980 con 0x7f7a2c145990 2022-07-21T01:48:51.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3134b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 0x7f7a2c1454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3134b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 0x7f7a2c1454a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33070/0 (socket says 172.21.15.140:33070) 2022-07-21T01:48:51.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3134b700 1 -- 172.21.15.140:0/3015894736 learned_addr learned my addr 172.21.15.140:0/3015894736 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:51.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a30b4a700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c145990 0x7f7a2c142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a31b4c700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c1434c0 0x7f7a2c159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3134b700 1 -- 172.21.15.140:0/3015894736 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c145990 msgr2=0x7f7a2c142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3134b700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c145990 0x7f7a2c142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3134b700 1 -- 172.21.15.140:0/3015894736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c1434c0 msgr2=0x7f7a2c159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3134b700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c1434c0 0x7f7a2c159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3134b700 1 -- 172.21.15.140:0/3015894736 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c149850 con 0x7f7a18004500 2022-07-21T01:48:51.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a30b4a700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c145990 0x7f7a2c142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:51.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3134b700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 0x7f7a2c1454a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f7a1c00b000 tx=0x7f7a1c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:51.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a227fc700 1 -- 172.21.15.140:0/3015894736 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a1c017070 con 0x7f7a18004500 2022-07-21T01:48:51.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a227fc700 1 -- 172.21.15.140:0/3015894736 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c000e90 con 0x7f7a18004500 2022-07-21T01:48:51.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3234d700 1 -- 172.21.15.140:0/3015894736 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2c156110 con 0x7f7a18004500 2022-07-21T01:48:51.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a227fc700 1 -- 172.21.15.140:0/3015894736 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a1c012950 con 0x7f7a18004500 2022-07-21T01:48:51.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.311+0000 7f7a3234d700 1 -- 172.21.15.140:0/3015894736 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2c10e340 con 0x7f7a18004500 2022-07-21T01:48:51.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.315+0000 7f7a3234d700 1 -- 172.21.15.140:0/3015894736 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2c159730 con 0x7f7a18004500 2022-07-21T01:48:51.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.315+0000 7f7a227fc700 1 -- 172.21.15.140:0/3015894736 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7a1c012ab0 con 0x7f7a18004500 2022-07-21T01:48:51.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.315+0000 7f7a227fc700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a0c053a10 0x7f7a0c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:51.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.315+0000 7f7a30b4a700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a0c053a10 0x7f7a0c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:51.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.315+0000 7f7a227fc700 1 -- 172.21.15.140:0/3015894736 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7a1c06bfa0 con 0x7f7a18004500 2022-07-21T01:48:51.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.315+0000 7f7a30b4a700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a0c053a10 0x7f7a0c055ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f7a28018000 tx=0x7f7a28015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:51.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.319+0000 7f7a227fc700 1 -- 172.21.15.140:0/3015894736 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a1c042ec0 con 0x7f7a18004500 2022-07-21T01:48:51.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.475+0000 7f7a3234d700 1 -- 172.21.15.140:0/3015894736 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a2c05bd90 con 0x7f7a18004500 2022-07-21T01:48:51.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.475+0000 7f7a227fc700 1 -- 172.21.15.140:0/3015894736 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7f7a1c0464e0 con 0x7f7a18004500 2022-07-21T01:48:51.476 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:51.477 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/665899293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":665899293},{"type":"v1","addr":"172.21.15.140:6835","nonce":665899293}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:48:31.423028+0000","modified":"2022-07-21T01:48:32.427116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.152:6833/1276387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1276387778},{"type":"v1","addr":"172.21.15.152:6833","nonce":1276387778}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-07-21T01:48:51.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 -- 172.21.15.140:0/3015894736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a0c053a10 msgr2=0x7f7a0c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a0c053a10 0x7f7a0c055ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f7a28018000 tx=0x7f7a28015000).stop 2022-07-21T01:48:51.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 -- 172.21.15.140:0/3015894736 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 msgr2=0x7f7a2c1454a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:51.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 0x7f7a2c1454a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f7a1c00b000 tx=0x7f7a1c008040).stop 2022-07-21T01:48:51.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 -- 172.21.15.140:0/3015894736 shutdown_connections 2022-07-21T01:48:51.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a0c053a10 0x7f7a0c055ed0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a2c1434c0 0x7f7a2c159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a2c145990 0x7f7a2c142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 --2- 172.21.15.140:0/3015894736 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a18004500 0x7f7a2c1454a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:51.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 -- 172.21.15.140:0/3015894736 >> 172.21.15.140:0/3015894736 conn(0x7f7a2c0634e0 msgr2=0x7f7a2c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:51.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 -- 172.21.15.140:0/3015894736 shutdown_connections 2022-07-21T01:48:51.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:51.479+0000 7f7a3234d700 1 -- 172.21.15.140:0/3015894736 wait complete. 2022-07-21T01:48:51.485 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 5 2022-07-21T01:48:51.692 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:48:31.423028+0000', 'modified': '2022-07-21T01:48:32.427116+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.152:6833/1276387778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 1276387778}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 1276387778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:48:51.692 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:48:51.693 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:48:51.809 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":4282,"want_state":"up:active","state":"up:active","rank_uptime":20.376021866999999,"mdsmap_epoch":5,"osdmap_epoch":22,"osdmap_epoch_barrier":21,"uptime":23.723710920999999} 2022-07-21T01:48:51.810 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 4282, "mdsmap_epoch": 5, "osdmap_epoch": 22, "osdmap_epoch_barrier": 21, "rank_uptime": 20.376021867, "state": "up:active", "uptime": 23.723710921, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:48:51.810 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:48:51.811 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2022-07-21T01:48:51.811 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.2 at ubuntu@smithi195.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2... 2022-07-21T01:48:51.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:51.812 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:48:51.822 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2022-07-21T01:48:51.840 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:48:51.841 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:48:51.898 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:48:51.899 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:48:51.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:51.901 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2022-07-21T01:48:51.909 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:48:51.910 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.2 /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:48:51.914 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2022-07-21T01:48:51.935 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:48:51.936 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:48:51.993 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:48:51.997 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:48:51.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:51.998 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2022-07-21T01:48:52.004 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi195.stderr:2022-07-21T01:48:52.004+0000 7f66cab10080 -1 init, newargv = 0x55576dea1270 newargc=9 2022-07-21T01:48:52.005 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi195.stderr:ceph-fuse[22259]: starting ceph client 2022-07-21T01:48:52.013 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi195.stderr:ceph-fuse[22259]: starting fuse 2022-07-21T01:48:53.112 DEBUG:teuthology.orchestra.run.smithi195:> sudo modprobe fuse 2022-07-21T01:48:53.131 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:48:53.132 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:48:53.188 INFO:teuthology.orchestra.run.smithi195.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:48:53.189 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:48:53.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:53.192 DEBUG:teuthology.orchestra.run.smithi195:> ls /sys/fs/fuse/connections 2022-07-21T01:48:53.200 INFO:teuthology.orchestra.run.smithi195.stdout:52 2022-07-21T01:48:53.202 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:48:53.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:53.203 DEBUG:teuthology.orchestra.run.smithi195:> sudo python3 -c ' 2022-07-21T01:48:53.203 DEBUG:teuthology.orchestra.run.smithi195:> import glob 2022-07-21T01:48:53.204 DEBUG:teuthology.orchestra.run.smithi195:> import re 2022-07-21T01:48:53.204 DEBUG:teuthology.orchestra.run.smithi195:> import os 2022-07-21T01:48:53.205 DEBUG:teuthology.orchestra.run.smithi195:> import subprocess 2022-07-21T01:48:53.205 DEBUG:teuthology.orchestra.run.smithi195:> 2022-07-21T01:48:53.206 DEBUG:teuthology.orchestra.run.smithi195:> def find_socket(client_name): 2022-07-21T01:48:53.207 DEBUG:teuthology.orchestra.run.smithi195:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2022-07-21T01:48:53.207 DEBUG:teuthology.orchestra.run.smithi195:> files = glob.glob(asok_path) 2022-07-21T01:48:53.208 DEBUG:teuthology.orchestra.run.smithi195:> 2022-07-21T01:48:53.209 DEBUG:teuthology.orchestra.run.smithi195:> # Given a non-glob path, it better be there 2022-07-21T01:48:53.210 DEBUG:teuthology.orchestra.run.smithi195:> if "*" not in asok_path: 2022-07-21T01:48:53.211 DEBUG:teuthology.orchestra.run.smithi195:> assert(len(files) == 1) 2022-07-21T01:48:53.211 DEBUG:teuthology.orchestra.run.smithi195:> return files[0] 2022-07-21T01:48:53.212 DEBUG:teuthology.orchestra.run.smithi195:> 2022-07-21T01:48:53.212 DEBUG:teuthology.orchestra.run.smithi195:> for f in files: 2022-07-21T01:48:53.213 DEBUG:teuthology.orchestra.run.smithi195:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:48:53.213 DEBUG:teuthology.orchestra.run.smithi195:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:48:53.222 DEBUG:teuthology.orchestra.run.smithi195:> return f 2022-07-21T01:48:53.223 DEBUG:teuthology.orchestra.run.smithi195:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:48:53.223 DEBUG:teuthology.orchestra.run.smithi195:> 2022-07-21T01:48:53.224 DEBUG:teuthology.orchestra.run.smithi195:> print(find_socket("client.2")) 2022-07-21T01:48:53.225 DEBUG:teuthology.orchestra.run.smithi195:> ' 2022-07-21T01:48:53.288 INFO:teuthology.orchestra.run.smithi195.stdout:/var/run/ceph/ceph-client.2.22259.asok 2022-07-21T01:48:53.293 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.22259.asok 2022-07-21T01:48:53.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:53.294 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.22259.asok status 2022-07-21T01:48:53.397 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2022-07-21T01:48:53.397 INFO:teuthology.orchestra.run.smithi195.stdout: "metadata": { 2022-07-21T01:48:53.398 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:48:53.398 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:48:53.399 INFO:teuthology.orchestra.run.smithi195.stdout: "entity_id": "2", 2022-07-21T01:48:53.399 INFO:teuthology.orchestra.run.smithi195.stdout: "hostname": "smithi195", 2022-07-21T01:48:53.400 INFO:teuthology.orchestra.run.smithi195.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2022-07-21T01:48:53.400 INFO:teuthology.orchestra.run.smithi195.stdout: "pid": "22259", 2022-07-21T01:48:53.400 INFO:teuthology.orchestra.run.smithi195.stdout: "root": "/" 2022-07-21T01:48:53.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2022-07-21T01:48:53.401 INFO:teuthology.orchestra.run.smithi195.stdout: "dentry_count": 0, 2022-07-21T01:48:53.402 INFO:teuthology.orchestra.run.smithi195.stdout: "dentry_pinned_count": 0, 2022-07-21T01:48:53.402 INFO:teuthology.orchestra.run.smithi195.stdout: "id": 4699, 2022-07-21T01:48:53.403 INFO:teuthology.orchestra.run.smithi195.stdout: "inst": { 2022-07-21T01:48:53.403 INFO:teuthology.orchestra.run.smithi195.stdout: "name": { 2022-07-21T01:48:53.403 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "client", 2022-07-21T01:48:53.404 INFO:teuthology.orchestra.run.smithi195.stdout: "num": 4699 2022-07-21T01:48:53.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2022-07-21T01:48:53.405 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": { 2022-07-21T01:48:53.405 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "v1", 2022-07-21T01:48:53.405 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": "172.21.15.195:0", 2022-07-21T01:48:53.406 INFO:teuthology.orchestra.run.smithi195.stdout: "nonce": 1088368432 2022-07-21T01:48:53.406 INFO:teuthology.orchestra.run.smithi195.stdout: } 2022-07-21T01:48:53.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2022-07-21T01:48:53.407 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": { 2022-07-21T01:48:53.408 INFO:teuthology.orchestra.run.smithi195.stdout: "type": "v1", 2022-07-21T01:48:53.408 INFO:teuthology.orchestra.run.smithi195.stdout: "addr": "172.21.15.195:0", 2022-07-21T01:48:53.409 INFO:teuthology.orchestra.run.smithi195.stdout: "nonce": 1088368432 2022-07-21T01:48:53.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2022-07-21T01:48:53.409 INFO:teuthology.orchestra.run.smithi195.stdout: "inst_str": "client.4699 172.21.15.195:0/1088368432", 2022-07-21T01:48:53.410 INFO:teuthology.orchestra.run.smithi195.stdout: "addr_str": "172.21.15.195:0/1088368432", 2022-07-21T01:48:53.410 INFO:teuthology.orchestra.run.smithi195.stdout: "inode_count": 1, 2022-07-21T01:48:53.410 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_epoch": 5, 2022-07-21T01:48:53.411 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_epoch": 22, 2022-07-21T01:48:53.411 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:48:53.411 INFO:teuthology.orchestra.run.smithi195.stdout: "blacklisted": false, 2022-07-21T01:48:53.412 INFO:teuthology.orchestra.run.smithi195.stdout: "fs_name": "cephfs" 2022-07-21T01:48:53.412 INFO:teuthology.orchestra.run.smithi195.stdout:} 2022-07-21T01:48:53.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:53.414 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:48:53.415 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:48:53.423 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:48:53.425 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:48:53.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:53.426 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:48:53.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:53.450 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:48:53.451 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:48:53.466 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:48:53.467 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:48:53.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:53.469 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:48:53.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:53.539 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:48:53.540 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:48:53.549 INFO:teuthology.orchestra.run.smithi195.stdout:fuseblk 2022-07-21T01:48:53.552 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2022-07-21T01:48:53.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:48:53.553 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:48:53.619 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2022-07-21T01:48:53.713 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:53.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.823+0000 7f72ad522700 1 Processor -- start 2022-07-21T01:48:53.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.823+0000 7f72ad522700 1 -- start start 2022-07-21T01:48:53.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72ad522700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f72a805c3f0 0x7f72a814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:53.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72ad522700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a805d480 con 0x7f72a80b1e00 2022-07-21T01:48:53.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72ad522700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a80b21e0 con 0x7f72a80acef0 2022-07-21T01:48:53.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72ad522700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a805cf40 con 0x7f72a805c3f0 2022-07-21T01:48:53.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a6ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f72a80b1e00 0x7f72a80ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47224/0 (socket says 172.21.15.140:47224) 2022-07-21T01:48:53.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a6ffd700 1 -- 172.21.15.140:0/3984712267 learned_addr learned my addr 172.21.15.140:0/3984712267 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:53.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a67fc700 1 --2- 172.21.15.140:0/3984712267 >> v2:172.21.15.140:3300/0 conn(0x7f72a805c3f0 0x7f72a814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:53.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a5ffb700 1 -- 172.21.15.140:0/3984712267 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880991723 0 0) 0x7f72a805d480 con 0x7f72a80b1e00 2022-07-21T01:48:53.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a5ffb700 1 -- 172.21.15.140:0/3984712267 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f728c003260 con 0x7f72a80b1e00 2022-07-21T01:48:53.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a67fc700 1 -- 172.21.15.140:0/3984712267 >> v1:172.21.15.152:6789/0 conn(0x7f72a80acef0 legacy=0x7f72a8146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:53.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a67fc700 1 -- 172.21.15.140:0/3984712267 >> v1:172.21.15.140:6789/0 conn(0x7f72a80b1e00 legacy=0x7f72a80ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:53.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a67fc700 1 -- 172.21.15.140:0/3984712267 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a805d1e0 con 0x7f72a805c3f0 2022-07-21T01:48:53.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a67fc700 1 --2- 172.21.15.140:0/3984712267 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72a805c3f0 0x7f72a814a6d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f72a805f3e0 tx=0x7f729c006df0).ready entity=mon.0 client_cookie=385387578667a2de server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:53.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a5ffb700 1 -- 172.21.15.140:0/3984712267 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f729c00d5c0 con 0x7f72a805c3f0 2022-07-21T01:48:53.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a5ffb700 1 -- 172.21.15.140:0/3984712267 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c000e90 con 0x7f72a805c3f0 2022-07-21T01:48:53.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72a5ffb700 1 -- 172.21.15.140:0/3984712267 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f729c00d910 con 0x7f72a805c3f0 2022-07-21T01:48:53.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72ad522700 1 -- 172.21.15.140:0/3984712267 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72a805c3f0 msgr2=0x7f72a814a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:53.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72ad522700 1 --2- 172.21.15.140:0/3984712267 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72a805c3f0 0x7f72a814a6d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f72a805f3e0 tx=0x7f729c006df0).stop 2022-07-21T01:48:53.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72ad522700 1 -- 172.21.15.140:0/3984712267 shutdown_connections 2022-07-21T01:48:53.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72ad522700 1 --2- 172.21.15.140:0/3984712267 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72a805c3f0 0x7f72a814a6d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:53.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72ad522700 1 -- 172.21.15.140:0/3984712267 >> 172.21.15.140:0/3984712267 conn(0x7f72a80611c0 msgr2=0x7f72a8062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:53.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72ad522700 1 -- 172.21.15.140:0/3984712267 shutdown_connections 2022-07-21T01:48:53.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.827+0000 7f72ad522700 1 -- 172.21.15.140:0/3984712267 wait complete. 2022-07-21T01:48:53.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72ad522700 1 Processor -- start 2022-07-21T01:48:53.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72ad522700 1 -- start start 2022-07-21T01:48:53.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72ad522700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72a80acef0 0x7f72a814bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:53.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72ad522700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72a80b1e00 0x7f72a81442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:53.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72ad522700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80b50c0 0x7f72a81447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:53.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72ad522700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a80a2100 con 0x7f72a80acef0 2022-07-21T01:48:53.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72ad522700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a80a19c0 con 0x7f72a80b50c0 2022-07-21T01:48:53.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72ad522700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a80a1c60 con 0x7f72a80b1e00 2022-07-21T01:48:53.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72a6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72a80acef0 0x7f72a814bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:53.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72a6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72a80acef0 0x7f72a814bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33084/0 (socket says 172.21.15.140:33084) 2022-07-21T01:48:53.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72a6ffd700 1 -- 172.21.15.140:0/2253521510 learned_addr learned my addr 172.21.15.140:0/2253521510 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:53.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72a67fc700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72a80b1e00 0x7f72a81442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:53.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72a77fe700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80b50c0 0x7f72a81447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:53.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72a6ffd700 1 -- 172.21.15.140:0/2253521510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72a80b1e00 msgr2=0x7f72a81442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:53.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72a6ffd700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72a80b1e00 0x7f72a81442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:53.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72a6ffd700 1 -- 172.21.15.140:0/2253521510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80b50c0 msgr2=0x7f72a81447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:53.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72a6ffd700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80b50c0 0x7f72a81447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:53.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72a6ffd700 1 -- 172.21.15.140:0/2253521510 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c00b180 con 0x7f72a80acef0 2022-07-21T01:48:53.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72a6ffd700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72a80acef0 0x7f72a814bf30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f72a0001490 tx=0x7f72a0007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:53.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f7297fff700 1 -- 172.21.15.140:0/2253521510 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72a0005ba0 con 0x7f72a80acef0 2022-07-21T01:48:53.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f7297fff700 1 -- 172.21.15.140:0/2253521510 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a0005d00 con 0x7f72a80acef0 2022-07-21T01:48:53.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72ad522700 1 -- 172.21.15.140:0/2253521510 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a80a16a0 con 0x7f72a80acef0 2022-07-21T01:48:53.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f7297fff700 1 -- 172.21.15.140:0/2253521510 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72a00072c0 con 0x7f72a80acef0 2022-07-21T01:48:53.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.831+0000 7f72ad522700 1 -- 172.21.15.140:0/2253521510 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a80b5b90 con 0x7f72a80acef0 2022-07-21T01:48:53.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.835+0000 7f7297fff700 1 -- 172.21.15.140:0/2253521510 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f72a0010070 con 0x7f72a80acef0 2022-07-21T01:48:53.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.835+0000 7f72ad522700 1 -- 172.21.15.140:0/2253521510 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a80a14d0 con 0x7f72a80acef0 2022-07-21T01:48:53.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.835+0000 7f7297fff700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72840539e0 0x7f7284055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:53.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.835+0000 7f7297fff700 1 -- 172.21.15.140:0/2253521510 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f72a006b9b0 con 0x7f72a80acef0 2022-07-21T01:48:53.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.835+0000 7f72a67fc700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72840539e0 0x7f7284055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:53.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.835+0000 7f72a67fc700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72840539e0 0x7f7284055ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f729c01a000 tx=0x7f729c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:53.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.839+0000 7f7297fff700 1 -- 172.21.15.140:0/2253521510 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72a00426e0 con 0x7f72a80acef0 2022-07-21T01:48:53.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.967+0000 7f72ad522700 1 -- 172.21.15.140:0/2253521510 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72a80b04b0 con 0x7f72a80acef0 2022-07-21T01:48:53.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.967+0000 7f7297fff700 1 -- 172.21.15.140:0/2253521510 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f72a0045d00 con 0x7f72a80acef0 2022-07-21T01:48:53.967 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:53.968 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:53.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 -- 172.21.15.140:0/2253521510 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72840539e0 msgr2=0x7f7284055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:53.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72840539e0 0x7f7284055ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f729c01a000 tx=0x7f729c014000).stop 2022-07-21T01:48:53.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 -- 172.21.15.140:0/2253521510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72a80acef0 msgr2=0x7f72a814bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:53.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72a80acef0 0x7f72a814bf30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f72a0001490 tx=0x7f72a0007e50).stop 2022-07-21T01:48:53.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 -- 172.21.15.140:0/2253521510 shutdown_connections 2022-07-21T01:48:53.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72840539e0 0x7f7284055ea0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:53.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a80b50c0 0x7f72a81447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:53.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72a80b1e00 0x7f72a81442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:53.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 --2- 172.21.15.140:0/2253521510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72a80acef0 0x7f72a814bf30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:53.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 -- 172.21.15.140:0/2253521510 >> 172.21.15.140:0/2253521510 conn(0x7f72a805c3f0 msgr2=0x7f72a814c900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:53.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 -- 172.21.15.140:0/2253521510 shutdown_connections 2022-07-21T01:48:53.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:53.971+0000 7f72ad522700 1 -- 172.21.15.140:0/2253521510 wait complete. 2022-07-21T01:48:53.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:48:29.344690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:48:30.427054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:48:53.981 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:48:54.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f999b5e5700 1 Processor -- start 2022-07-21T01:48:54.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f999b5e5700 1 -- start start 2022-07-21T01:48:54.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f999b5e5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f999405cf40 0x7f99940ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f999b5e5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99940b3870 0x7f9994140f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f999b5e5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9994140aa0 con 0x7f9994062ec0 2022-07-21T01:48:54.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f999b5e5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9994144000 con 0x7f999405cf40 2022-07-21T01:48:54.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f999b5e5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99940b1ce0 con 0x7f99940b3870 2022-07-21T01:48:54.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9999b82700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f9994062ec0 0x7f99941442a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47242/0 (socket says 172.21.15.140:47242) 2022-07-21T01:48:54.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9999b82700 1 -- 172.21.15.140:0/309431199 learned_addr learned my addr 172.21.15.140:0/309431199 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:54.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9998b80700 1 --2- 172.21.15.140:0/309431199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99940b3870 0x7f9994140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9999381700 1 --2- 172.21.15.140:0/309431199 >> v2:172.21.15.152:3300/0 conn(0x7f999405cf40 0x7f99940ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9987fff700 1 -- 172.21.15.140:0/309431199 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211242842 0 0) 0x7f9994140aa0 con 0x7f9994062ec0 2022-07-21T01:48:54.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9987fff700 1 -- 172.21.15.140:0/309431199 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9978003260 con 0x7f9994062ec0 2022-07-21T01:48:54.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9998b80700 1 -- 172.21.15.140:0/309431199 >> v2:172.21.15.152:3300/0 conn(0x7f999405cf40 msgr2=0x7f99940ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:54.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9998b80700 1 --2- 172.21.15.140:0/309431199 >> v2:172.21.15.152:3300/0 conn(0x7f999405cf40 0x7f99940ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9998b80700 1 -- 172.21.15.140:0/309431199 >> v1:172.21.15.140:6789/0 conn(0x7f9994062ec0 legacy=0x7f99941442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9998b80700 1 -- 172.21.15.140:0/309431199 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99940b1e20 con 0x7f99940b3870 2022-07-21T01:48:54.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9999381700 1 --2- 172.21.15.140:0/309431199 >> v2:172.21.15.152:3300/0 conn(0x7f999405cf40 0x7f99940ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:54.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9998b80700 1 --2- 172.21.15.140:0/309431199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99940b3870 0x7f9994140f00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f9988002f70 tx=0x7f9988006df0).ready entity=mon.2 client_cookie=c02d1b1a21eb7921 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9987fff700 1 -- 172.21.15.140:0/309431199 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f998800d4a0 con 0x7f99940b3870 2022-07-21T01:48:54.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9987fff700 1 -- 172.21.15.140:0/309431199 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9988000e90 con 0x7f99940b3870 2022-07-21T01:48:54.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.091+0000 7f9987fff700 1 -- 172.21.15.140:0/309431199 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f998800d7f0 con 0x7f99940b3870 2022-07-21T01:48:54.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 -- 172.21.15.140:0/309431199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99940b3870 msgr2=0x7f9994140f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 --2- 172.21.15.140:0/309431199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99940b3870 0x7f9994140f00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f9988002f70 tx=0x7f9988006df0).stop 2022-07-21T01:48:54.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 -- 172.21.15.140:0/309431199 shutdown_connections 2022-07-21T01:48:54.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 --2- 172.21.15.140:0/309431199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99940b3870 0x7f9994140f00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 --2- 172.21.15.140:0/309431199 >> v2:172.21.15.152:3300/0 conn(0x7f999405cf40 0x7f99940ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 -- 172.21.15.140:0/309431199 >> 172.21.15.140:0/309431199 conn(0x7f99940b7d30 msgr2=0x7f99940b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:54.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 -- 172.21.15.140:0/309431199 shutdown_connections 2022-07-21T01:48:54.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 -- 172.21.15.140:0/309431199 wait complete. 2022-07-21T01:48:54.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 Processor -- start 2022-07-21T01:48:54.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 -- start start 2022-07-21T01:48:54.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f999405cf40 0x7f9994060f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9994062ec0 0x7f99940613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99940b3870 0x7f999409a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9994067580 con 0x7f999405cf40 2022-07-21T01:48:54.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99940618e0 con 0x7f99940b3870 2022-07-21T01:48:54.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f999b5e5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9994060410 con 0x7f9994062ec0 2022-07-21T01:48:54.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f9999381700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f999405cf40 0x7f9994060f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f9999381700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f999405cf40 0x7f9994060f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33098/0 (socket says 172.21.15.140:33098) 2022-07-21T01:48:54.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f9999381700 1 -- 172.21.15.140:0/2660981213 learned_addr learned my addr 172.21.15.140:0/2660981213 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:54.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f9998b80700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9994062ec0 0x7f99940613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.095+0000 7f9999b82700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99940b3870 0x7f999409a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f9999381700 1 -- 172.21.15.140:0/2660981213 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9994062ec0 msgr2=0x7f99940613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f9999381700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9994062ec0 0x7f99940613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f9999381700 1 -- 172.21.15.140:0/2660981213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99940b3870 msgr2=0x7f999409a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f9999381700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99940b3870 0x7f999409a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f9999381700 1 -- 172.21.15.140:0/2660981213 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999000b180 con 0x7f999405cf40 2022-07-21T01:48:54.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f9999381700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f999405cf40 0x7f9994060f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f9994146f50 tx=0x7f9990006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f99867fc700 1 -- 172.21.15.140:0/2660981213 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f999000c5f0 con 0x7f999405cf40 2022-07-21T01:48:54.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f99867fc700 1 -- 172.21.15.140:0/2660981213 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999000c750 con 0x7f999405cf40 2022-07-21T01:48:54.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f999b5e5700 1 -- 172.21.15.140:0/2660981213 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998800b180 con 0x7f999405cf40 2022-07-21T01:48:54.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f99867fc700 1 -- 172.21.15.140:0/2660981213 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f999000cab0 con 0x7f999405cf40 2022-07-21T01:48:54.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f999b5e5700 1 -- 172.21.15.140:0/2660981213 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99940679d0 con 0x7f999405cf40 2022-07-21T01:48:54.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f99867fc700 1 -- 172.21.15.140:0/2660981213 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f999000cc10 con 0x7f999405cf40 2022-07-21T01:48:54.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f999b5e5700 1 -- 172.21.15.140:0/2660981213 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999409a2b0 con 0x7f999405cf40 2022-07-21T01:48:54.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f99867fc700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f997c053950 0x7f997c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f99867fc700 1 -- 172.21.15.140:0/2660981213 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f999006c450 con 0x7f999405cf40 2022-07-21T01:48:54.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.099+0000 7f9998b80700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f997c053950 0x7f997c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.103+0000 7f9998b80700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f997c053950 0x7f997c055e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f998801a000 tx=0x7f9988014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.107+0000 7f99867fc700 1 -- 172.21.15.140:0/2660981213 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f99900432d0 con 0x7f999405cf40 2022-07-21T01:48:54.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.231+0000 7f999b5e5700 1 -- 172.21.15.140:0/2660981213 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f99940574e0 con 0x7f999405cf40 2022-07-21T01:48:54.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.231+0000 7f99867fc700 1 -- 172.21.15.140:0/2660981213 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f99900468f0 con 0x7f999405cf40 2022-07-21T01:48:54.232 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:48:54.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 -- 172.21.15.140:0/2660981213 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f997c053950 msgr2=0x7f997c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f997c053950 0x7f997c055e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f998801a000 tx=0x7f9988014000).stop 2022-07-21T01:48:54.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 -- 172.21.15.140:0/2660981213 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f999405cf40 msgr2=0x7f9994060f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f999405cf40 0x7f9994060f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f9994146f50 tx=0x7f9990006df0).stop 2022-07-21T01:48:54.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 -- 172.21.15.140:0/2660981213 shutdown_connections 2022-07-21T01:48:54.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f997c053950 0x7f997c055e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99940b3870 0x7f999409a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9994062ec0 0x7f99940613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 --2- 172.21.15.140:0/2660981213 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f999405cf40 0x7f9994060f00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 -- 172.21.15.140:0/2660981213 >> 172.21.15.140:0/2660981213 conn(0x7f99940b7d30 msgr2=0x7f99940b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:54.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 -- 172.21.15.140:0/2660981213 shutdown_connections 2022-07-21T01:48:54.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.235+0000 7f999b5e5700 1 -- 172.21.15.140:0/2660981213 wait complete. 2022-07-21T01:48:54.244 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:48:54.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0bfff700 1 Processor -- start 2022-07-21T01:48:54.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0bfff700 1 -- start start 2022-07-21T01:48:54.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fec0c0b14f0 0x7fec0c146750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0bfff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0c0afbc0 con 0x7fec0c0b59d0 2022-07-21T01:48:54.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0bfff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0c0b00e0 con 0x7fec0c14ada0 2022-07-21T01:48:54.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0bfff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec0c148080 con 0x7fec0c0b14f0 2022-07-21T01:48:54.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fec0c0b14f0 0x7fec0c146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0affd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fec0c0b59d0 0x7fec0c0b07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47252/0 (socket says 172.21.15.140:47252) 2022-07-21T01:48:54.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0affd700 1 -- 172.21.15.140:0/1341740101 learned_addr learned my addr 172.21.15.140:0/1341740101 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:54.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0b7fe700 1 -- 172.21.15.140:0/1341740101 >> v1:172.21.15.152:6789/0 conn(0x7fec0c14ada0 legacy=0x7fec0c146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0b7fe700 1 -- 172.21.15.140:0/1341740101 >> v1:172.21.15.140:6789/0 conn(0x7fec0c0b59d0 legacy=0x7fec0c0b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0b7fe700 1 -- 172.21.15.140:0/1341740101 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c0b5db0 con 0x7fec0c0b14f0 2022-07-21T01:48:54.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec0b7fe700 1 --2- 172.21.15.140:0/1341740101 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0c0b14f0 0x7fec0c146750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7febfc002f70 tx=0x7febfc006df0).ready entity=mon.0 client_cookie=43d9fd142985affa server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec09ffb700 1 -- 172.21.15.140:0/1341740101 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febfc00d610 con 0x7fec0c0b14f0 2022-07-21T01:48:54.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec09ffb700 1 -- 172.21.15.140:0/1341740101 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc000e90 con 0x7fec0c0b14f0 2022-07-21T01:48:54.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.351+0000 7fec09ffb700 1 -- 172.21.15.140:0/1341740101 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febfc00d9a0 con 0x7fec0c0b14f0 2022-07-21T01:48:54.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 -- 172.21.15.140:0/1341740101 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0c0b14f0 msgr2=0x7fec0c146750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 --2- 172.21.15.140:0/1341740101 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0c0b14f0 0x7fec0c146750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7febfc002f70 tx=0x7febfc006df0).stop 2022-07-21T01:48:54.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 -- 172.21.15.140:0/1341740101 shutdown_connections 2022-07-21T01:48:54.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 --2- 172.21.15.140:0/1341740101 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0c0b14f0 0x7fec0c146750 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 -- 172.21.15.140:0/1341740101 >> 172.21.15.140:0/1341740101 conn(0x7fec0c0634c0 msgr2=0x7fec0c0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:54.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 -- 172.21.15.140:0/1341740101 shutdown_connections 2022-07-21T01:48:54.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 -- 172.21.15.140:0/1341740101 wait complete. 2022-07-21T01:48:54.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 Processor -- start 2022-07-21T01:48:54.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 -- start start 2022-07-21T01:48:54.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec0c0b14f0 0x7fec0c111b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0c0b59d0 0x7fec0c143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec0c14ada0 0x7fec0c144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c10e480 con 0x7fec0c0b59d0 2022-07-21T01:48:54.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c10f3b0 con 0x7fec0c14ada0 2022-07-21T01:48:54.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c11b0d0 con 0x7fec0c0b14f0 2022-07-21T01:48:54.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0a7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0c0b59d0 0x7fec0c143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0a7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0c0b59d0 0x7fec0c143c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33114/0 (socket says 172.21.15.140:33114) 2022-07-21T01:48:54.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0a7fc700 1 -- 172.21.15.140:0/3735820359 learned_addr learned my addr 172.21.15.140:0/3735820359 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:54.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0affd700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec0c0b14f0 0x7fec0c111b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0b7fe700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec0c14ada0 0x7fec0c144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0a7fc700 1 -- 172.21.15.140:0/3735820359 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec0c0b14f0 msgr2=0x7fec0c111b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0a7fc700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec0c0b14f0 0x7fec0c111b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0a7fc700 1 -- 172.21.15.140:0/3735820359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec0c14ada0 msgr2=0x7fec0c144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0a7fc700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec0c14ada0 0x7fec0c144110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0a7fc700 1 -- 172.21.15.140:0/3735820359 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc00b180 con 0x7fec0c0b59d0 2022-07-21T01:48:54.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0b7fe700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec0c14ada0 0x7fec0c144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:54.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0affd700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec0c0b14f0 0x7fec0c111b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:54.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7fec0a7fc700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0c0b59d0 0x7fec0c143c20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fec0c111660 tx=0x7fec00007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.355+0000 7febfbfff700 1 -- 172.21.15.140:0/3735820359 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0000ce20 con 0x7fec0c0b59d0 2022-07-21T01:48:54.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.359+0000 7febfbfff700 1 -- 172.21.15.140:0/3735820359 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec00012400 con 0x7fec0c0b59d0 2022-07-21T01:48:54.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.359+0000 7fec0bfff700 1 -- 172.21.15.140:0/3735820359 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c10f150 con 0x7fec0c0b59d0 2022-07-21T01:48:54.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.359+0000 7febfbfff700 1 -- 172.21.15.140:0/3735820359 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec00012760 con 0x7fec0c0b59d0 2022-07-21T01:48:54.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.359+0000 7fec0bfff700 1 -- 172.21.15.140:0/3735820359 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c110300 con 0x7fec0c0b59d0 2022-07-21T01:48:54.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.359+0000 7fec0bfff700 1 -- 172.21.15.140:0/3735820359 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0c10e170 con 0x7fec0c0b59d0 2022-07-21T01:48:54.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.359+0000 7febfbfff700 1 -- 172.21.15.140:0/3735820359 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fec00019030 con 0x7fec0c0b59d0 2022-07-21T01:48:54.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.359+0000 7febfbfff700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 0x7febe8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.359+0000 7febfbfff700 1 -- 172.21.15.140:0/3735820359 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec0006b2c0 con 0x7fec0c0b59d0 2022-07-21T01:48:54.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.359+0000 7fec0affd700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 0x7febe8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.363+0000 7fec0affd700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 0x7febe8055fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fec04008000 tx=0x7fec04009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.367+0000 7febfbfff700 1 -- 172.21.15.140:0/3735820359 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fec00042180 con 0x7fec0c0b59d0 2022-07-21T01:48:54.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.491+0000 7fec0bfff700 1 -- 172.21.15.140:0/3735820359 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fec0c14b2e0 con 0x7fec0c0b59d0 2022-07-21T01:48:54.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.491+0000 7febfbfff700 1 -- 172.21.15.140:0/3735820359 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fec000457a0 con 0x7fec0c0b59d0 2022-07-21T01:48:54.492 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:54.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 -- 172.21.15.140:0/3735820359 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 msgr2=0x7febe8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 0x7febe8055fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fec04008000 tx=0x7fec04009040).stop 2022-07-21T01:48:54.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 -- 172.21.15.140:0/3735820359 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0c0b59d0 msgr2=0x7fec0c143c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0c0b59d0 0x7fec0c143c20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fec0c111660 tx=0x7fec00007c20).stop 2022-07-21T01:48:54.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 -- 172.21.15.140:0/3735820359 shutdown_connections 2022-07-21T01:48:54.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 0x7febe8055fc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec0c14ada0 0x7fec0c144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0c0b59d0 0x7fec0c143c20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 --2- 172.21.15.140:0/3735820359 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec0c0b14f0 0x7fec0c111b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 -- 172.21.15.140:0/3735820359 >> 172.21.15.140:0/3735820359 conn(0x7fec0c0634c0 msgr2=0x7fec0c0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:54.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 -- 172.21.15.140:0/3735820359 shutdown_connections 2022-07-21T01:48:54.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.495+0000 7fec0bfff700 1 -- 172.21.15.140:0/3735820359 wait complete. 2022-07-21T01:48:54.505 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:48:54.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.607+0000 7fadc7fff700 1 Processor -- start 2022-07-21T01:48:54.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.607+0000 7fadc7fff700 1 -- start start 2022-07-21T01:48:54.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fadc80b59f0 0x7fadc805c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fadc80af940 0x7fadc805c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc805d2f0 con 0x7fadc80addd0 2022-07-21T01:48:54.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc8146350 con 0x7fadc80b59f0 2022-07-21T01:48:54.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80b5dd0 con 0x7fadc80af940 2022-07-21T01:48:54.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc67fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fadc80af940 0x7fadc805c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc6ffd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fadc80b59f0 0x7fadc805c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc77fe700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fadc80addd0 0x7fadc8141cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:48384/0 (socket says 172.21.15.140:48384) 2022-07-21T01:48:54.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc77fe700 1 -- 172.21.15.140:0/612747544 learned_addr learned my addr 172.21.15.140:0/612747544 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:54.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc67fc700 1 -- 172.21.15.140:0/612747544 >> v2:172.21.15.152:3300/0 conn(0x7fadc80b59f0 msgr2=0x7fadc805c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:54.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc67fc700 1 --2- 172.21.15.140:0/612747544 >> v2:172.21.15.152:3300/0 conn(0x7fadc80b59f0 0x7fadc805c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc67fc700 1 -- 172.21.15.140:0/612747544 >> v1:172.21.15.152:6789/0 conn(0x7fadc80addd0 legacy=0x7fadc8141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc5ffb700 1 -- 172.21.15.140:0/612747544 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788800464 0 0) 0x7fadc805d2f0 con 0x7fadc80addd0 2022-07-21T01:48:54.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc67fc700 1 -- 172.21.15.140:0/612747544 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc81460b0 con 0x7fadc80af940 2022-07-21T01:48:54.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc6ffd700 1 --2- 172.21.15.140:0/612747544 >> v2:172.21.15.152:3300/0 conn(0x7fadc80b59f0 0x7fadc805c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:54.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc67fc700 1 --2- 172.21.15.140:0/612747544 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fadc80af940 0x7fadc805c5c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fadbc002f70 tx=0x7fadbc006df0).ready entity=mon.2 client_cookie=1f5895fbe45bb0a6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc5ffb700 1 -- 172.21.15.140:0/612747544 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadbc00d570 con 0x7fadc80af940 2022-07-21T01:48:54.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc5ffb700 1 -- 172.21.15.140:0/612747544 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc000e90 con 0x7fadc80af940 2022-07-21T01:48:54.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc5ffb700 1 -- 172.21.15.140:0/612747544 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadbc00d8c0 con 0x7fadc80af940 2022-07-21T01:48:54.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 -- 172.21.15.140:0/612747544 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fadc80af940 msgr2=0x7fadc805c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 --2- 172.21.15.140:0/612747544 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fadc80af940 0x7fadc805c5c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fadbc002f70 tx=0x7fadbc006df0).stop 2022-07-21T01:48:54.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 -- 172.21.15.140:0/612747544 shutdown_connections 2022-07-21T01:48:54.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 --2- 172.21.15.140:0/612747544 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fadc80af940 0x7fadc805c5c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 --2- 172.21.15.140:0/612747544 >> v2:172.21.15.152:3300/0 conn(0x7fadc80b59f0 0x7fadc805c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 -- 172.21.15.140:0/612747544 >> 172.21.15.140:0/612747544 conn(0x7fadc8063480 msgr2=0x7fadc80b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:54.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 -- 172.21.15.140:0/612747544 shutdown_connections 2022-07-21T01:48:54.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.611+0000 7fadc7fff700 1 -- 172.21.15.140:0/612747544 wait complete. 2022-07-21T01:48:54.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc7fff700 1 Processor -- start 2022-07-21T01:48:54.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc7fff700 1 -- start start 2022-07-21T01:48:54.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fadc80addd0 0x7fadc8067230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadc80af940 0x7fadc81125a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fadc80b59f0 0x7fadc8151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc7fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8114c20 con 0x7fadc80addd0 2022-07-21T01:48:54.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc7fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8066d90 con 0x7fadc80af940 2022-07-21T01:48:54.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8114ec0 con 0x7fadc80b59f0 2022-07-21T01:48:54.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fadc80addd0 0x7fadc8067230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fadc80addd0 0x7fadc8067230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33126/0 (socket says 172.21.15.140:33126) 2022-07-21T01:48:54.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc6ffd700 1 -- 172.21.15.140:0/1257278900 learned_addr learned my addr 172.21.15.140:0/1257278900 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:54.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc77fe700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fadc80b59f0 0x7fadc8151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc67fc700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadc80af940 0x7fadc81125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc6ffd700 1 -- 172.21.15.140:0/1257278900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fadc80b59f0 msgr2=0x7fadc8151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc6ffd700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fadc80b59f0 0x7fadc8151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc6ffd700 1 -- 172.21.15.140:0/1257278900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadc80af940 msgr2=0x7fadc81125a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc6ffd700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadc80af940 0x7fadc81125a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc6ffd700 1 -- 172.21.15.140:0/1257278900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc0009470 con 0x7fadc80addd0 2022-07-21T01:48:54.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc6ffd700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fadc80addd0 0x7fadc8067230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fadc000c000 tx=0x7fadc00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadb7fff700 1 -- 172.21.15.140:0/1257278900 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadc000c920 con 0x7fadc80addd0 2022-07-21T01:48:54.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadb7fff700 1 -- 172.21.15.140:0/1257278900 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc000ca80 con 0x7fadc80addd0 2022-07-21T01:48:54.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc7fff700 1 -- 172.21.15.140:0/1257278900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc00b180 con 0x7fadc80addd0 2022-07-21T01:48:54.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadb7fff700 1 -- 172.21.15.140:0/1257278900 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadc000cde0 con 0x7fadc80addd0 2022-07-21T01:48:54.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.615+0000 7fadc7fff700 1 -- 172.21.15.140:0/1257278900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc8145380 con 0x7fadc80addd0 2022-07-21T01:48:54.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.619+0000 7fadc7fff700 1 -- 172.21.15.140:0/1257278900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc8066f50 con 0x7fadc80addd0 2022-07-21T01:48:54.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.619+0000 7fadb7fff700 1 -- 172.21.15.140:0/1257278900 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fadc0022070 con 0x7fadc80addd0 2022-07-21T01:48:54.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.619+0000 7fadb7fff700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fada4053aa0 0x7fada4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.619+0000 7fadb7fff700 1 -- 172.21.15.140:0/1257278900 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fadc006c7c0 con 0x7fadc80addd0 2022-07-21T01:48:54.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.619+0000 7fadc67fc700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fada4053aa0 0x7fada4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.619+0000 7fadc67fc700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fada4053aa0 0x7fada4055f60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fadbc018000 tx=0x7fadbc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.623+0000 7fadb7fff700 1 -- 172.21.15.140:0/1257278900 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fadc0043640 con 0x7fadc80addd0 2022-07-21T01:48:54.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 -- 172.21.15.140:0/1257278900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fadc805cfe0 con 0x7fadc80addd0 2022-07-21T01:48:54.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadb7fff700 1 -- 172.21.15.140:0/1257278900 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fadc0046c60 con 0x7fadc80addd0 2022-07-21T01:48:54.751 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:54.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 -- 172.21.15.140:0/1257278900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fada4053aa0 msgr2=0x7fada4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fada4053aa0 0x7fada4055f60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fadbc018000 tx=0x7fadbc014000).stop 2022-07-21T01:48:54.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 -- 172.21.15.140:0/1257278900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fadc80addd0 msgr2=0x7fadc8067230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fadc80addd0 0x7fadc8067230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fadc000c000 tx=0x7fadc00091d0).stop 2022-07-21T01:48:54.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 -- 172.21.15.140:0/1257278900 shutdown_connections 2022-07-21T01:48:54.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fada4053aa0 0x7fada4055f60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fadc80b59f0 0x7fadc8151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadc80af940 0x7fadc81125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 --2- 172.21.15.140:0/1257278900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fadc80addd0 0x7fadc8067230 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 -- 172.21.15.140:0/1257278900 >> 172.21.15.140:0/1257278900 conn(0x7fadc8063480 msgr2=0x7fadc813edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:54.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 -- 172.21.15.140:0/1257278900 shutdown_connections 2022-07-21T01:48:54.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.751+0000 7fadc7fff700 1 -- 172.21.15.140:0/1257278900 wait complete. 2022-07-21T01:48:54.763 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:48:54.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.871+0000 7fe2d4c44700 1 Processor -- start 2022-07-21T01:48:54.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.871+0000 7fe2d4c44700 1 -- start start 2022-07-21T01:48:54.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.871+0000 7fe2d4c44700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 0x7fe2d0146680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.871+0000 7fe2d4c44700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe2d00988b0 0x7fe2d0146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.871+0000 7fe2d4c44700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d00afdb0 con 0x7fe2d01422c0 2022-07-21T01:48:54.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.871+0000 7fe2d4c44700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d00b0050 con 0x7fe2d00988b0 2022-07-21T01:48:54.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.871+0000 7fe2d4c44700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d005fcd0 con 0x7fe2d01462a0 2022-07-21T01:48:54.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.871+0000 7fe2ce59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 0x7fe2d0146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.871+0000 7fe2cdd9b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe2d00988b0 0x7fe2d0146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.871+0000 7fe2ce59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 0x7fe2d0146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:58096/0 (socket says 172.21.15.140:58096) 2022-07-21T01:48:54.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.871+0000 7fe2ce59c700 1 -- 172.21.15.140:0/56417189 learned_addr learned my addr 172.21.15.140:0/56417189 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:54.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2cdd9b700 1 -- 172.21.15.140:0/56417189 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 msgr2=0x7fe2d0146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:54.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2cdd9b700 1 --2- 172.21.15.140:0/56417189 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 0x7fe2d0146680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2cdd9b700 1 -- 172.21.15.140:0/56417189 >> v1:172.21.15.152:6789/0 conn(0x7fe2d01422c0 legacy=0x7fe2d0060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2cdd9b700 1 -- 172.21.15.140:0/56417189 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d00afae0 con 0x7fe2d00988b0 2022-07-21T01:48:54.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2cdd9b700 1 --2- 172.21.15.140:0/56417189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00988b0 0x7fe2d0146b70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fe2c8001490 tx=0x7fe2c4006df0).ready entity=mon.1 client_cookie=c2abd848355cb4fc server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2cd59a700 1 -- 172.21.15.140:0/56417189 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2c400d4a0 con 0x7fe2d00988b0 2022-07-21T01:48:54.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2cd59a700 1 -- 172.21.15.140:0/56417189 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c4000e90 con 0x7fe2d00988b0 2022-07-21T01:48:54.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2cd59a700 1 -- 172.21.15.140:0/56417189 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2c400d7f0 con 0x7fe2d00988b0 2022-07-21T01:48:54.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2d4c44700 1 -- 172.21.15.140:0/56417189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00988b0 msgr2=0x7fe2d0146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2d4c44700 1 --2- 172.21.15.140:0/56417189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00988b0 0x7fe2d0146b70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fe2c8001490 tx=0x7fe2c4006df0).stop 2022-07-21T01:48:54.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2d4c44700 1 -- 172.21.15.140:0/56417189 shutdown_connections 2022-07-21T01:48:54.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2d4c44700 1 --2- 172.21.15.140:0/56417189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00988b0 0x7fe2d0146b70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2d4c44700 1 --2- 172.21.15.140:0/56417189 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 0x7fe2d0146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2d4c44700 1 -- 172.21.15.140:0/56417189 >> 172.21.15.140:0/56417189 conn(0x7fe2d00b5f00 msgr2=0x7fe2d00b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:54.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2d4c44700 1 -- 172.21.15.140:0/56417189 shutdown_connections 2022-07-21T01:48:54.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2d4c44700 1 -- 172.21.15.140:0/56417189 wait complete. 2022-07-21T01:48:54.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.875+0000 7fe2d4c44700 1 Processor -- start 2022-07-21T01:48:54.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2d4c44700 1 -- start start 2022-07-21T01:48:54.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2d4c44700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00988b0 0x7fe2d0144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2d4c44700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2d01422c0 0x7fe2d009c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2d4c44700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 0x7fe2d009c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2d4c44700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d009de20 con 0x7fe2d01422c0 2022-07-21T01:48:54.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2d4c44700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d009f3d0 con 0x7fe2d00988b0 2022-07-21T01:48:54.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2d4c44700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d009f130 con 0x7fe2d01462a0 2022-07-21T01:48:54.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2cdd9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2d01422c0 0x7fe2d009c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2cdd9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2d01422c0 0x7fe2d009c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33142/0 (socket says 172.21.15.140:33142) 2022-07-21T01:48:54.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2cdd9b700 1 -- 172.21.15.140:0/3714044623 learned_addr learned my addr 172.21.15.140:0/3714044623 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:54.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2ced9d700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 0x7fe2d009c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2ce59c700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00988b0 0x7fe2d0144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2ced9d700 1 -- 172.21.15.140:0/3714044623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00988b0 msgr2=0x7fe2d0144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2ced9d700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00988b0 0x7fe2d0144cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2ced9d700 1 -- 172.21.15.140:0/3714044623 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2d01422c0 msgr2=0x7fe2d009c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:54.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2ced9d700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2d01422c0 0x7fe2d009c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:54.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2ced9d700 1 -- 172.21.15.140:0/3714044623 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c400b180 con 0x7fe2d01462a0 2022-07-21T01:48:54.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2cdd9b700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2d01422c0 0x7fe2d009c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:54.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2ced9d700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 0x7fe2d009c980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fe2c8001550 tx=0x7fe2c8007e50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2bf7fe700 1 -- 172.21.15.140:0/3714044623 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2c800e4a0 con 0x7fe2d01462a0 2022-07-21T01:48:54.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2bf7fe700 1 -- 172.21.15.140:0/3714044623 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c800e600 con 0x7fe2d01462a0 2022-07-21T01:48:54.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2d4c44700 1 -- 172.21.15.140:0/3714044623 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d009daa0 con 0x7fe2d01462a0 2022-07-21T01:48:54.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2bf7fe700 1 -- 172.21.15.140:0/3714044623 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe2c800e960 con 0x7fe2d01462a0 2022-07-21T01:48:54.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.879+0000 7fe2d4c44700 1 -- 172.21.15.140:0/3714044623 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d0140f10 con 0x7fe2d01462a0 2022-07-21T01:48:54.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.883+0000 7fe2d4c44700 1 -- 172.21.15.140:0/3714044623 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d00b0c70 con 0x7fe2d01462a0 2022-07-21T01:48:54.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.883+0000 7fe2bf7fe700 1 -- 172.21.15.140:0/3714044623 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe2c8019030 con 0x7fe2d01462a0 2022-07-21T01:48:54.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.883+0000 7fe2bf7fe700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe2ac053a70 0x7fe2ac055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:54.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.883+0000 7fe2bf7fe700 1 -- 172.21.15.140:0/3714044623 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe2c806bb00 con 0x7fe2d01462a0 2022-07-21T01:48:54.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.883+0000 7fe2ce59c700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe2ac053a70 0x7fe2ac055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:54.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.883+0000 7fe2ce59c700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe2ac053a70 0x7fe2ac055f30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fe2c0008000 tx=0x7fe2c0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:54.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:54.891+0000 7fe2bf7fe700 1 -- 172.21.15.140:0/3714044623 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe2d00b0c70 con 0x7fe2d01462a0 2022-07-21T01:48:55.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2d4c44700 1 -- 172.21.15.140:0/3714044623 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2d009dc90 con 0x7fe2d01462a0 2022-07-21T01:48:55.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2bf7fe700 1 -- 172.21.15.140:0/3714044623 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe2c8017070 con 0x7fe2d01462a0 2022-07-21T01:48:55.019 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:55.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2d4c44700 1 -- 172.21.15.140:0/3714044623 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe2ac053a70 msgr2=0x7fe2ac055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2d4c44700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe2ac053a70 0x7fe2ac055f30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fe2c0008000 tx=0x7fe2c0008040).stop 2022-07-21T01:48:55.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2d4c44700 1 -- 172.21.15.140:0/3714044623 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 msgr2=0x7fe2d009c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2d4c44700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 0x7fe2d009c980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fe2c8001550 tx=0x7fe2c8007e50).stop 2022-07-21T01:48:55.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2d4c44700 1 -- 172.21.15.140:0/3714044623 shutdown_connections 2022-07-21T01:48:55.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2d4c44700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe2ac053a70 0x7fe2ac055f30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2d4c44700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2d01462a0 0x7fe2d009c980 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2d4c44700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2d01422c0 0x7fe2d009c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2d4c44700 1 --2- 172.21.15.140:0/3714044623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2d00988b0 0x7fe2d0144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.019+0000 7fe2d4c44700 1 -- 172.21.15.140:0/3714044623 >> 172.21.15.140:0/3714044623 conn(0x7fe2d00b5f00 msgr2=0x7fe2d0143510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:55.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.023+0000 7fe2d4c44700 1 -- 172.21.15.140:0/3714044623 shutdown_connections 2022-07-21T01:48:55.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.023+0000 7fe2d4c44700 1 -- 172.21.15.140:0/3714044623 wait complete. 2022-07-21T01:48:55.032 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:55.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 Processor -- start 2022-07-21T01:48:55.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 -- start start 2022-07-21T01:48:55.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8e500b1e00 0x7f8e5014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e500acef0 0x7f8e50146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e5005d480 con 0x7f8e5005c3f0 2022-07-21T01:48:55.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e500b21e0 con 0x7f8e500b1e00 2022-07-21T01:48:55.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5005cf40 con 0x7f8e500acef0 2022-07-21T01:48:55.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e4ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8e500b1e00 0x7f8e5014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e477fe700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8e5005c3f0 0x7f8e500ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:48408/0 (socket says 172.21.15.140:48408) 2022-07-21T01:48:55.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e477fe700 1 -- 172.21.15.140:0/1933776980 learned_addr learned my addr 172.21.15.140:0/1933776980 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:55.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e4ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8e500b1e00 0x7f8e5014a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:38888/0 (socket says 172.21.15.140:38888) 2022-07-21T01:48:55.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e54d00700 1 --2- 172.21.15.140:0/1933776980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e500acef0 0x7f8e50146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e4ffff700 1 -- 172.21.15.140:0/1933776980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e500acef0 msgr2=0x7f8e50146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:55.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e4ffff700 1 --2- 172.21.15.140:0/1933776980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e500acef0 0x7f8e50146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e4ffff700 1 -- 172.21.15.140:0/1933776980 >> v1:172.21.15.152:6789/0 conn(0x7f8e5005c3f0 legacy=0x7f8e500ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e4ffff700 1 -- 172.21.15.140:0/1933776980 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5005d1e0 con 0x7f8e500b1e00 2022-07-21T01:48:55.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e4ffff700 1 --2- 172.21.15.140:0/1933776980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e500b1e00 0x7f8e5014a6d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f8e40002f70 tx=0x7f8e40006df0).ready entity=mon.1 client_cookie=63d3bb593414615e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e4f7fe700 1 -- 172.21.15.140:0/1933776980 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4000d570 con 0x7f8e500b1e00 2022-07-21T01:48:55.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e4f7fe700 1 -- 172.21.15.140:0/1933776980 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e40000e90 con 0x7f8e500b1e00 2022-07-21T01:48:55.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e4f7fe700 1 -- 172.21.15.140:0/1933776980 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4000d8c0 con 0x7f8e500b1e00 2022-07-21T01:48:55.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 -- 172.21.15.140:0/1933776980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e500b1e00 msgr2=0x7f8e5014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 --2- 172.21.15.140:0/1933776980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e500b1e00 0x7f8e5014a6d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f8e40002f70 tx=0x7f8e40006df0).stop 2022-07-21T01:48:55.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 -- 172.21.15.140:0/1933776980 shutdown_connections 2022-07-21T01:48:55.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 --2- 172.21.15.140:0/1933776980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e500acef0 0x7f8e50146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 --2- 172.21.15.140:0/1933776980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e500b1e00 0x7f8e5014a6d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 -- 172.21.15.140:0/1933776980 >> 172.21.15.140:0/1933776980 conn(0x7f8e500611c0 msgr2=0x7f8e50062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:55.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 -- 172.21.15.140:0/1933776980 shutdown_connections 2022-07-21T01:48:55.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.135+0000 7f8e56763700 1 -- 172.21.15.140:0/1933776980 wait complete. 2022-07-21T01:48:55.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e56763700 1 Processor -- start 2022-07-21T01:48:55.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e56763700 1 -- start start 2022-07-21T01:48:55.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e56763700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e500acef0 0x7f8e5009fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e56763700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e500b1e00 0x7f8e501425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e56763700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8e50142a90 0x7f8e50159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e56763700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5009e280 con 0x7f8e50142a90 2022-07-21T01:48:55.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e56763700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e50064500 con 0x7f8e500b1e00 2022-07-21T01:48:55.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e56763700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5009e4b0 con 0x7f8e500acef0 2022-07-21T01:48:55.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e54d00700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8e50142a90 0x7f8e50159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e4ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e500acef0 0x7f8e5009fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e54d00700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8e50142a90 0x7f8e50159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33158/0 (socket says 172.21.15.140:33158) 2022-07-21T01:48:55.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e54d00700 1 -- 172.21.15.140:0/1582697756 learned_addr learned my addr 172.21.15.140:0/1582697756 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:55.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e477fe700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e500b1e00 0x7f8e501425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e54d00700 1 -- 172.21.15.140:0/1582697756 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e500acef0 msgr2=0x7f8e5009fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e54d00700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e500acef0 0x7f8e5009fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e54d00700 1 -- 172.21.15.140:0/1582697756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e500b1e00 msgr2=0x7f8e501425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e54d00700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e500b1e00 0x7f8e501425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e54d00700 1 -- 172.21.15.140:0/1582697756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4000b180 con 0x7f8e50142a90 2022-07-21T01:48:55.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e54d00700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8e50142a90 0x7f8e50159e80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f8e48006e00 tx=0x7f8e480091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e4dffb700 1 -- 172.21.15.140:0/1582697756 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4800c640 con 0x7f8e50142a90 2022-07-21T01:48:55.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e4dffb700 1 -- 172.21.15.140:0/1582697756 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4800c7a0 con 0x7f8e50142a90 2022-07-21T01:48:55.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e56763700 1 -- 172.21.15.140:0/1582697756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5009e750 con 0x7f8e50142a90 2022-07-21T01:48:55.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e4dffb700 1 -- 172.21.15.140:0/1582697756 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e4800cb00 con 0x7f8e50142a90 2022-07-21T01:48:55.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.139+0000 7f8e56763700 1 -- 172.21.15.140:0/1582697756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e500a1ce0 con 0x7f8e50142a90 2022-07-21T01:48:55.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.143+0000 7f8e56763700 1 -- 172.21.15.140:0/1582697756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5009e8e0 con 0x7f8e50142a90 2022-07-21T01:48:55.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.143+0000 7f8e4dffb700 1 -- 172.21.15.140:0/1582697756 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8e48022070 con 0x7f8e50142a90 2022-07-21T01:48:55.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.143+0000 7f8e4dffb700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8e34053b30 0x7f8e34055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.143+0000 7f8e4dffb700 1 -- 172.21.15.140:0/1582697756 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8e4806c4c0 con 0x7f8e50142a90 2022-07-21T01:48:55.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.143+0000 7f8e4ffff700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8e34053b30 0x7f8e34055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.143+0000 7f8e4ffff700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8e34053b30 0x7f8e34055ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f8e4001a000 tx=0x7f8e40014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.147+0000 7f8e4dffb700 1 -- 172.21.15.140:0/1582697756 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8e480433e0 con 0x7f8e50142a90 2022-07-21T01:48:55.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.271+0000 7f8e56763700 1 -- 172.21.15.140:0/1582697756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e500b04b0 con 0x7f8e50142a90 2022-07-21T01:48:55.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e4dffb700 1 -- 172.21.15.140:0/1582697756 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f8e48046a00 con 0x7f8e50142a90 2022-07-21T01:48:55.274 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:55.274 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:55.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 -- 172.21.15.140:0/1582697756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8e34053b30 msgr2=0x7f8e34055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8e34053b30 0x7f8e34055ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f8e4001a000 tx=0x7f8e40014000).stop 2022-07-21T01:48:55.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 -- 172.21.15.140:0/1582697756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8e50142a90 msgr2=0x7f8e50159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8e50142a90 0x7f8e50159e80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f8e48006e00 tx=0x7f8e480091d0).stop 2022-07-21T01:48:55.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 -- 172.21.15.140:0/1582697756 shutdown_connections 2022-07-21T01:48:55.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8e34053b30 0x7f8e34055ff0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8e50142a90 0x7f8e50159e80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e500b1e00 0x7f8e501425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 --2- 172.21.15.140:0/1582697756 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e500acef0 0x7f8e5009fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 -- 172.21.15.140:0/1582697756 >> 172.21.15.140:0/1582697756 conn(0x7f8e5005c3f0 msgr2=0x7f8e50143360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:55.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 -- 172.21.15.140:0/1582697756 shutdown_connections 2022-07-21T01:48:55.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.275+0000 7f8e56763700 1 -- 172.21.15.140:0/1582697756 wait complete. 2022-07-21T01:48:55.287 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:48:29.344690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:48:30.427054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:48:55.288 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:48:55.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 Processor -- start 2022-07-21T01:48:55.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 -- start start 2022-07-21T01:48:55.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f61a00b59f0 0x7f61a005c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f61a00af940 0x7f61a005c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a005d2f0 con 0x7f61a00addd0 2022-07-21T01:48:55.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a0146350 con 0x7f61a00af940 2022-07-21T01:48:55.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a00b5dd0 con 0x7f61a00b59f0 2022-07-21T01:48:55.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a5cec700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f61a00af940 0x7f61a005c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a5cec700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f61a00af940 0x7f61a005c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33162/0 (socket says 172.21.15.140:33162) 2022-07-21T01:48:55.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a5cec700 1 -- 172.21.15.140:0/2583900787 learned_addr learned my addr 172.21.15.140:0/2583900787 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:55.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a64ed700 1 --2- 172.21.15.140:0/2583900787 >> v2:172.21.15.152:3300/0 conn(0x7f61a00b59f0 0x7f61a005c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a5cec700 1 -- 172.21.15.140:0/2583900787 >> v2:172.21.15.152:3300/0 conn(0x7f61a00b59f0 msgr2=0x7f61a005c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:55.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a5cec700 1 --2- 172.21.15.140:0/2583900787 >> v2:172.21.15.152:3300/0 conn(0x7f61a00b59f0 0x7f61a005c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a5cec700 1 -- 172.21.15.140:0/2583900787 >> v1:172.21.15.152:6789/0 conn(0x7f61a00addd0 legacy=0x7f61a0141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a5cec700 1 -- 172.21.15.140:0/2583900787 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a01460b0 con 0x7f61a00af940 2022-07-21T01:48:55.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a5cec700 1 --2- 172.21.15.140:0/2583900787 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61a00af940 0x7f61a005c5c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f619c002f70 tx=0x7f619c006eb0).ready entity=mon.0 client_cookie=1f8122f0b035cea2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a54eb700 1 -- 172.21.15.140:0/2583900787 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f619c00d5c0 con 0x7f61a00af940 2022-07-21T01:48:55.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a54eb700 1 -- 172.21.15.140:0/2583900787 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c000e90 con 0x7f61a00af940 2022-07-21T01:48:55.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a54eb700 1 -- 172.21.15.140:0/2583900787 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f619c00d950 con 0x7f61a00af940 2022-07-21T01:48:55.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 -- 172.21.15.140:0/2583900787 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61a00af940 msgr2=0x7f61a005c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 --2- 172.21.15.140:0/2583900787 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61a00af940 0x7f61a005c5c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f619c002f70 tx=0x7f619c006eb0).stop 2022-07-21T01:48:55.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 -- 172.21.15.140:0/2583900787 shutdown_connections 2022-07-21T01:48:55.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 --2- 172.21.15.140:0/2583900787 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61a00af940 0x7f61a005c5c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 --2- 172.21.15.140:0/2583900787 >> v2:172.21.15.152:3300/0 conn(0x7f61a00b59f0 0x7f61a005c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 -- 172.21.15.140:0/2583900787 >> 172.21.15.140:0/2583900787 conn(0x7f61a0063480 msgr2=0x7f61a00b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:55.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 -- 172.21.15.140:0/2583900787 shutdown_connections 2022-07-21T01:48:55.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.399+0000 7f61a74ef700 1 -- 172.21.15.140:0/2583900787 wait complete. 2022-07-21T01:48:55.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a74ef700 1 Processor -- start 2022-07-21T01:48:55.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a74ef700 1 -- start start 2022-07-21T01:48:55.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a74ef700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61a00addd0 0x7f61a0142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a74ef700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61a00af940 0x7f61a0064fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a74ef700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61a00b59f0 0x7f61a00654a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a74ef700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0066ed0 con 0x7f61a00b59f0 2022-07-21T01:48:55.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a74ef700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0067170 con 0x7f61a00af940 2022-07-21T01:48:55.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a74ef700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0067410 con 0x7f61a00addd0 2022-07-21T01:48:55.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a6cee700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61a00b59f0 0x7f61a00654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a6cee700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61a00b59f0 0x7f61a00654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33170/0 (socket says 172.21.15.140:33170) 2022-07-21T01:48:55.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a6cee700 1 -- 172.21.15.140:0/2940928136 learned_addr learned my addr 172.21.15.140:0/2940928136 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:55.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a64ed700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61a00addd0 0x7f61a0142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a5cec700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61a00af940 0x7f61a0064fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a6cee700 1 -- 172.21.15.140:0/2940928136 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61a00addd0 msgr2=0x7f61a0142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a6cee700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61a00addd0 0x7f61a0142b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a6cee700 1 -- 172.21.15.140:0/2940928136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61a00af940 msgr2=0x7f61a0064fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a6cee700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61a00af940 0x7f61a0064fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a6cee700 1 -- 172.21.15.140:0/2940928136 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c00b1e0 con 0x7f61a00b59f0 2022-07-21T01:48:55.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a5cec700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61a00af940 0x7f61a0064fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:55.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a6cee700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61a00b59f0 0x7f61a00654a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f61a0062640 tx=0x7f6198007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f618f7fe700 1 -- 172.21.15.140:0/2940928136 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f619800e4a0 con 0x7f61a00b59f0 2022-07-21T01:48:55.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f618f7fe700 1 -- 172.21.15.140:0/2940928136 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619800e600 con 0x7f61a00b59f0 2022-07-21T01:48:55.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a74ef700 1 -- 172.21.15.140:0/2940928136 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a0065e10 con 0x7f61a00b59f0 2022-07-21T01:48:55.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f618f7fe700 1 -- 172.21.15.140:0/2940928136 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f619800e960 con 0x7f61a00b59f0 2022-07-21T01:48:55.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.403+0000 7f61a74ef700 1 -- 172.21.15.140:0/2940928136 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a0143080 con 0x7f61a00b59f0 2022-07-21T01:48:55.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.407+0000 7f61a74ef700 1 -- 172.21.15.140:0/2940928136 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a005cfe0 con 0x7f61a00b59f0 2022-07-21T01:48:55.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.407+0000 7f618f7fe700 1 -- 172.21.15.140:0/2940928136 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6198019030 con 0x7f61a00b59f0 2022-07-21T01:48:55.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.407+0000 7f618f7fe700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6180053aa0 0x7f6180055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.407+0000 7f618f7fe700 1 -- 172.21.15.140:0/2940928136 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f619806b580 con 0x7f61a00b59f0 2022-07-21T01:48:55.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.407+0000 7f61a64ed700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6180053aa0 0x7f6180055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.407+0000 7f61a64ed700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6180053aa0 0x7f6180055f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f619000b000 tx=0x7f6190006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.415+0000 7f618f7fe700 1 -- 172.21.15.140:0/2940928136 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f61980423a0 con 0x7f61a00b59f0 2022-07-21T01:48:55.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.543+0000 7f61a74ef700 1 -- 172.21.15.140:0/2940928136 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f61a00574e0 con 0x7f61a00b59f0 2022-07-21T01:48:55.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f618f7fe700 1 -- 172.21.15.140:0/2940928136 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f61980459c0 con 0x7f61a00b59f0 2022-07-21T01:48:55.545 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:48:55.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 -- 172.21.15.140:0/2940928136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6180053aa0 msgr2=0x7f6180055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6180053aa0 0x7f6180055f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f619000b000 tx=0x7f6190006df0).stop 2022-07-21T01:48:55.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 -- 172.21.15.140:0/2940928136 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61a00b59f0 msgr2=0x7f61a00654a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61a00b59f0 0x7f61a00654a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f61a0062640 tx=0x7f6198007f40).stop 2022-07-21T01:48:55.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 -- 172.21.15.140:0/2940928136 shutdown_connections 2022-07-21T01:48:55.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6180053aa0 0x7f6180055f60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61a00b59f0 0x7f61a00654a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61a00af940 0x7f61a0064fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 --2- 172.21.15.140:0/2940928136 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61a00addd0 0x7f61a0142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 -- 172.21.15.140:0/2940928136 >> 172.21.15.140:0/2940928136 conn(0x7f61a0063480 msgr2=0x7f61a005c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:55.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 -- 172.21.15.140:0/2940928136 shutdown_connections 2022-07-21T01:48:55.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.547+0000 7f61a74ef700 1 -- 172.21.15.140:0/2940928136 wait complete. 2022-07-21T01:48:55.558 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:48:55.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.663+0000 7fb9ba7c4700 1 Processor -- start 2022-07-21T01:48:55.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.663+0000 7fb9ba7c4700 1 -- start start 2022-07-21T01:48:55.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9b40b5690 0x7fb9b4140730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9b4140c20 0x7fb9b41296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b405cc80 con 0x7fb9b40af030 2022-07-21T01:48:55.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b40af410 con 0x7fb9b40b5690 2022-07-21T01:48:55.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4144050 con 0x7fb9b4140c20 2022-07-21T01:48:55.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b9fc3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9b4140c20 0x7fb9b41296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b8fc1700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9b40b5690 0x7fb9b4140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b8fc1700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9b40b5690 0x7fb9b4140730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:38916/0 (socket says 172.21.15.140:38916) 2022-07-21T01:48:55.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b8fc1700 1 -- 172.21.15.140:0/2848317105 learned_addr learned my addr 172.21.15.140:0/2848317105 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:55.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b9fc3700 1 -- 172.21.15.140:0/2848317105 >> v2:172.21.15.152:3300/0 conn(0x7fb9b40b5690 msgr2=0x7fb9b4140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:55.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b9fc3700 1 --2- 172.21.15.140:0/2848317105 >> v2:172.21.15.152:3300/0 conn(0x7fb9b40b5690 0x7fb9b4140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b9fc3700 1 -- 172.21.15.140:0/2848317105 >> v1:172.21.15.152:6789/0 conn(0x7fb9b40af030 legacy=0x7fb9b405c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b9fc3700 1 -- 172.21.15.140:0/2848317105 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b405be30 con 0x7fb9b4140c20 2022-07-21T01:48:55.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b8fc1700 1 --2- 172.21.15.140:0/2848317105 >> v2:172.21.15.152:3300/0 conn(0x7fb9b40b5690 0x7fb9b4140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:55.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b9fc3700 1 --2- 172.21.15.140:0/2848317105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9b4140c20 0x7fb9b41296f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb9ac002f70 tx=0x7fb9ac006df0).ready entity=mon.2 client_cookie=51d9a8948a3012fb server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b3fff700 1 -- 172.21.15.140:0/2848317105 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9ac00d570 con 0x7fb9b4140c20 2022-07-21T01:48:55.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b3fff700 1 -- 172.21.15.140:0/2848317105 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac000e90 con 0x7fb9b4140c20 2022-07-21T01:48:55.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9b3fff700 1 -- 172.21.15.140:0/2848317105 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9ac00d8c0 con 0x7fb9b4140c20 2022-07-21T01:48:55.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/2848317105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9b4140c20 msgr2=0x7fb9b41296f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 --2- 172.21.15.140:0/2848317105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9b4140c20 0x7fb9b41296f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb9ac002f70 tx=0x7fb9ac006df0).stop 2022-07-21T01:48:55.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/2848317105 shutdown_connections 2022-07-21T01:48:55.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 --2- 172.21.15.140:0/2848317105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9b4140c20 0x7fb9b41296f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 --2- 172.21.15.140:0/2848317105 >> v2:172.21.15.152:3300/0 conn(0x7fb9b40b5690 0x7fb9b4140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/2848317105 >> 172.21.15.140:0/2848317105 conn(0x7fb9b40634c0 msgr2=0x7fb9b40b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:55.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/2848317105 shutdown_connections 2022-07-21T01:48:55.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.667+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/2848317105 wait complete. 2022-07-21T01:48:55.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9ba7c4700 1 Processor -- start 2022-07-21T01:48:55.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9ba7c4700 1 -- start start 2022-07-21T01:48:55.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9ba7c4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9b40af030 0x7fb9b40acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9ba7c4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b40b5690 0x7fb9b4130cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9ba7c4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9b4140c20 0x7fb9b41311c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9ba7c4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41316b0 con 0x7fb9b4140c20 2022-07-21T01:48:55.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9ba7c4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4067590 con 0x7fb9b40b5690 2022-07-21T01:48:55.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9ba7c4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4067830 con 0x7fb9b40af030 2022-07-21T01:48:55.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b9fc3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9b4140c20 0x7fb9b41311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b9fc3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9b4140c20 0x7fb9b41311c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33184/0 (socket says 172.21.15.140:33184) 2022-07-21T01:48:55.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b9fc3700 1 -- 172.21.15.140:0/1749615055 learned_addr learned my addr 172.21.15.140:0/1749615055 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:55.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b8fc1700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b40b5690 0x7fb9b4130cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b97c2700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9b40af030 0x7fb9b40acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b9fc3700 1 -- 172.21.15.140:0/1749615055 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9b40af030 msgr2=0x7fb9b40acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b9fc3700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9b40af030 0x7fb9b40acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b9fc3700 1 -- 172.21.15.140:0/1749615055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b40b5690 msgr2=0x7fb9b4130cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b9fc3700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b40b5690 0x7fb9b4130cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b9fc3700 1 -- 172.21.15.140:0/1749615055 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a8009530 con 0x7fb9b4140c20 2022-07-21T01:48:55.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b9fc3700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9b4140c20 0x7fb9b41311c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fb9ac002f70 tx=0x7fb9ac014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b27fc700 1 -- 172.21.15.140:0/1749615055 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9ac0139c0 con 0x7fb9b4140c20 2022-07-21T01:48:55.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b27fc700 1 -- 172.21.15.140:0/1749615055 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac013b20 con 0x7fb9b4140c20 2022-07-21T01:48:55.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9b27fc700 1 -- 172.21.15.140:0/1749615055 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9ac013e80 con 0x7fb9b4140c20 2022-07-21T01:48:55.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/1749615055 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac00b180 con 0x7fb9b4140c20 2022-07-21T01:48:55.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.671+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/1749615055 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b40ad1f0 con 0x7fb9b4140c20 2022-07-21T01:48:55.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.675+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/1749615055 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b405c0d0 con 0x7fb9b4140c20 2022-07-21T01:48:55.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.675+0000 7fb9b27fc700 1 -- 172.21.15.140:0/1749615055 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb9ac01c070 con 0x7fb9b4140c20 2022-07-21T01:48:55.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.675+0000 7fb9b27fc700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb994053aa0 0x7fb994055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.675+0000 7fb9b27fc700 1 -- 172.21.15.140:0/1749615055 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb9ac06c810 con 0x7fb9b4140c20 2022-07-21T01:48:55.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.675+0000 7fb9b97c2700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb994053aa0 0x7fb994055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.675+0000 7fb9b97c2700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb994053aa0 0x7fb994055f60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fb9a400d000 tx=0x7fb9a4007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.679+0000 7fb9b27fc700 1 -- 172.21.15.140:0/1749615055 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb9ac043730 con 0x7fb9b4140c20 2022-07-21T01:48:55.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.803+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/1749615055 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb9b40574e0 con 0x7fb9b4140c20 2022-07-21T01:48:55.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9b27fc700 1 -- 172.21.15.140:0/1749615055 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb9ac023020 con 0x7fb9b4140c20 2022-07-21T01:48:55.806 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:55.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/1749615055 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb994053aa0 msgr2=0x7fb994055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb994053aa0 0x7fb994055f60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fb9a400d000 tx=0x7fb9a4007b90).stop 2022-07-21T01:48:55.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/1749615055 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9b4140c20 msgr2=0x7fb9b41311c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9b4140c20 0x7fb9b41311c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fb9ac002f70 tx=0x7fb9ac014000).stop 2022-07-21T01:48:55.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/1749615055 shutdown_connections 2022-07-21T01:48:55.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb994053aa0 0x7fb994055f60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9b4140c20 0x7fb9b41311c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9b40b5690 0x7fb9b4130cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 --2- 172.21.15.140:0/1749615055 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9b40af030 0x7fb9b40acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/1749615055 >> 172.21.15.140:0/1749615055 conn(0x7fb9b40634c0 msgr2=0x7fb9b4126860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:55.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/1749615055 shutdown_connections 2022-07-21T01:48:55.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.807+0000 7fb9ba7c4700 1 -- 172.21.15.140:0/1749615055 wait complete. 2022-07-21T01:48:55.818 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:48:55.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.927+0000 7fdcd491f700 1 Processor -- start 2022-07-21T01:48:55.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.927+0000 7fdcd491f700 1 -- start start 2022-07-21T01:48:55.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdcd00b3cf0 0x7fdcd0142a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdcd0142f40 0x7fdcd01419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd00ba180 con 0x7fdcd00b9da0 2022-07-21T01:48:55.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd00ad5b0 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd01461b0 con 0x7fdcd0142f40 2022-07-21T01:48:55.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdccf7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fdcd00b9da0 0x7fdcd01468f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47334/0 (socket says 172.21.15.140:47334) 2022-07-21T01:48:55.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdccf7fe700 1 -- 172.21.15.140:0/2221145275 learned_addr learned my addr 172.21.15.140:0/2221145275 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:55.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcceffd700 1 --2- 172.21.15.140:0/2221145275 >> v2:172.21.15.140:3300/0 conn(0x7fdcd00b3cf0 0x7fdcd0142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdccffff700 1 --2- 172.21.15.140:0/2221145275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdcd0142f40 0x7fdcd01419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcce7fc700 1 -- 172.21.15.140:0/2221145275 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135017958 0 0) 0x7fdcd00ba180 con 0x7fdcd00b9da0 2022-07-21T01:48:55.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcce7fc700 1 -- 172.21.15.140:0/2221145275 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb0003260 con 0x7fdcd00b9da0 2022-07-21T01:48:55.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcceffd700 1 -- 172.21.15.140:0/2221145275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdcd0142f40 msgr2=0x7fdcd01419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:55.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcceffd700 1 --2- 172.21.15.140:0/2221145275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdcd0142f40 0x7fdcd01419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcceffd700 1 -- 172.21.15.140:0/2221145275 >> v1:172.21.15.140:6789/0 conn(0x7fdcd00b9da0 legacy=0x7fdcd01468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcceffd700 1 -- 172.21.15.140:0/2221145275 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0146450 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdccffff700 1 --2- 172.21.15.140:0/2221145275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdcd0142f40 0x7fdcd01419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:55.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcceffd700 1 --2- 172.21.15.140:0/2221145275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdcd00b3cf0 0x7fdcd0142a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fdcb8002f70 tx=0x7fdcb8006df0).ready entity=mon.0 client_cookie=14e5170044fe3ebb server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcce7fc700 1 -- 172.21.15.140:0/2221145275 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcb800d4a0 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcce7fc700 1 -- 172.21.15.140:0/2221145275 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb8000e90 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcce7fc700 1 -- 172.21.15.140:0/2221145275 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcb800d830 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 -- 172.21.15.140:0/2221145275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdcd00b3cf0 msgr2=0x7fdcd0142a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 --2- 172.21.15.140:0/2221145275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdcd00b3cf0 0x7fdcd0142a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fdcb8002f70 tx=0x7fdcb8006df0).stop 2022-07-21T01:48:55.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 -- 172.21.15.140:0/2221145275 shutdown_connections 2022-07-21T01:48:55.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 --2- 172.21.15.140:0/2221145275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdcd0142f40 0x7fdcd01419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 --2- 172.21.15.140:0/2221145275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdcd00b3cf0 0x7fdcd0142a50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 -- 172.21.15.140:0/2221145275 >> 172.21.15.140:0/2221145275 conn(0x7fdcd00678c0 msgr2=0x7fdcd00b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:55.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 -- 172.21.15.140:0/2221145275 shutdown_connections 2022-07-21T01:48:55.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.931+0000 7fdcd491f700 1 -- 172.21.15.140:0/2221145275 wait complete. 2022-07-21T01:48:55.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcd491f700 1 Processor -- start 2022-07-21T01:48:55.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcd491f700 1 -- start start 2022-07-21T01:48:55.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcd491f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdcd00b3cf0 0x7fdcd00af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcd491f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd00b9da0 0x7fdcd013da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcd491f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdcd0142f40 0x7fdcd0151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcd491f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd0152e90 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcd491f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd00b1b40 con 0x7fdcd00b9da0 2022-07-21T01:48:55.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcd491f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd00b1dd0 con 0x7fdcd0142f40 2022-07-21T01:48:55.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdccf7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdcd00b3cf0 0x7fdcd00af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdccffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdcd0142f40 0x7fdcd0151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcceffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd00b9da0 0x7fdcd013da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdccf7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdcd00b3cf0 0x7fdcd00af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33196/0 (socket says 172.21.15.140:33196) 2022-07-21T01:48:55.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdccf7fe700 1 -- 172.21.15.140:0/4062039878 learned_addr learned my addr 172.21.15.140:0/4062039878 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:55.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdccf7fe700 1 -- 172.21.15.140:0/4062039878 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdcd0142f40 msgr2=0x7fdcd0151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdccf7fe700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdcd0142f40 0x7fdcd0151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdccf7fe700 1 -- 172.21.15.140:0/4062039878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd00b9da0 msgr2=0x7fdcd013da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:55.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdccf7fe700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd00b9da0 0x7fdcd013da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:55.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdccf7fe700 1 -- 172.21.15.140:0/4062039878 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc0009470 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcceffd700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd00b9da0 0x7fdcd013da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:55.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdccf7fe700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdcd00b3cf0 0x7fdcd00af360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fdcc8001110 tx=0x7fdcc8007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcccff9700 1 -- 172.21.15.140:0/4062039878 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcc80070d0 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcccff9700 1 -- 172.21.15.140:0/4062039878 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc8007230 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcd491f700 1 -- 172.21.15.140:0/4062039878 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb800b180 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcccff9700 1 -- 172.21.15.140:0/4062039878 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcc8005610 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.935+0000 7fdcd491f700 1 -- 172.21.15.140:0/4062039878 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd00af850 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.939+0000 7fdcd491f700 1 -- 172.21.15.140:0/4062039878 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd00b1f90 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.939+0000 7fdcccff9700 1 -- 172.21.15.140:0/4062039878 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdcc8010070 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.939+0000 7fdcccff9700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdcb4053a70 0x7fdcb4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:55.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.939+0000 7fdcccff9700 1 -- 172.21.15.140:0/4062039878 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdcc806b530 con 0x7fdcd00b3cf0 2022-07-21T01:48:55.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.939+0000 7fdcceffd700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdcb4053a70 0x7fdcb4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:55.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.939+0000 7fdcceffd700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdcb4053a70 0x7fdcb4055f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fdcb801a000 tx=0x7fdcb8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:55.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:55.947+0000 7fdcccff9700 1 -- 172.21.15.140:0/4062039878 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdcc8042450 con 0x7fdcd00b3cf0 2022-07-21T01:48:56.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.067+0000 7fdcd491f700 1 -- 172.21.15.140:0/4062039878 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdcd0145c40 con 0x7fdcd00b3cf0 2022-07-21T01:48:56.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.067+0000 7fdcccff9700 1 -- 172.21.15.140:0/4062039878 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fdcc8045a70 con 0x7fdcd00b3cf0 2022-07-21T01:48:56.067 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:56.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 -- 172.21.15.140:0/4062039878 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdcb4053a70 msgr2=0x7fdcb4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdcb4053a70 0x7fdcb4055f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fdcb801a000 tx=0x7fdcb8014000).stop 2022-07-21T01:48:56.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 -- 172.21.15.140:0/4062039878 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdcd00b3cf0 msgr2=0x7fdcd00af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdcd00b3cf0 0x7fdcd00af360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fdcc8001110 tx=0x7fdcc8007aa0).stop 2022-07-21T01:48:56.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 -- 172.21.15.140:0/4062039878 shutdown_connections 2022-07-21T01:48:56.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdcb4053a70 0x7fdcb4055f30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdcd0142f40 0x7fdcd0151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcd00b9da0 0x7fdcd013da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 --2- 172.21.15.140:0/4062039878 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdcd00b3cf0 0x7fdcd00af360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 -- 172.21.15.140:0/4062039878 >> 172.21.15.140:0/4062039878 conn(0x7fdcd00678c0 msgr2=0x7fdcd00b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:56.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 -- 172.21.15.140:0/4062039878 shutdown_connections 2022-07-21T01:48:56.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.071+0000 7fdcd491f700 1 -- 172.21.15.140:0/4062039878 wait complete. 2022-07-21T01:48:56.080 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:48:56.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.183+0000 7f78f99c7700 1 Processor -- start 2022-07-21T01:48:56.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.183+0000 7f78f99c7700 1 -- start start 2022-07-21T01:48:56.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.183+0000 7f78f99c7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78f40b59d0 0x7f78f40b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.183+0000 7f78f99c7700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f40afbc0 con 0x7f78f40b14f0 2022-07-21T01:48:56.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.183+0000 7f78f99c7700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f40b00e0 con 0x7f78f414ada0 2022-07-21T01:48:56.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.183+0000 7f78f99c7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4148080 con 0x7f78f40b59d0 2022-07-21T01:48:56.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.183+0000 7f78f91c6700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f78f40b14f0 0x7f78f405c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47352/0 (socket says 172.21.15.140:47352) 2022-07-21T01:48:56.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.183+0000 7f78f91c6700 1 -- 172.21.15.140:0/795713980 learned_addr learned my addr 172.21.15.140:0/795713980 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:56.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.183+0000 7f78f89c5700 1 --2- 172.21.15.140:0/795713980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78f40b59d0 0x7f78f40b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f37fe700 1 -- 172.21.15.140:0/795713980 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001588995 0 0) 0x7f78f40afbc0 con 0x7f78f40b14f0 2022-07-21T01:48:56.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f37fe700 1 -- 172.21.15.140:0/795713980 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78d8003260 con 0x7f78f40b14f0 2022-07-21T01:48:56.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f37fe700 1 -- 172.21.15.140:0/795713980 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180584104 0 0) 0x7f78f40b00e0 con 0x7f78f414ada0 2022-07-21T01:48:56.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f89c5700 1 -- 172.21.15.140:0/795713980 >> v1:172.21.15.152:6789/0 conn(0x7f78f414ada0 legacy=0x7f78f405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f89c5700 1 -- 172.21.15.140:0/795713980 >> v1:172.21.15.140:6789/0 conn(0x7f78f40b14f0 legacy=0x7f78f405c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f89c5700 1 -- 172.21.15.140:0/795713980 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f40b5db0 con 0x7f78f40b59d0 2022-07-21T01:48:56.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f89c5700 1 --2- 172.21.15.140:0/795713980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78f40b59d0 0x7f78f40b07e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f78ec002f70 tx=0x7f78ec006df0).ready entity=mon.2 client_cookie=cbc2ede5261854b1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:56.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f37fe700 1 -- 172.21.15.140:0/795713980 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78ec00d5c0 con 0x7f78f40b59d0 2022-07-21T01:48:56.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f37fe700 1 -- 172.21.15.140:0/795713980 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec000e90 con 0x7f78f40b59d0 2022-07-21T01:48:56.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f37fe700 1 -- 172.21.15.140:0/795713980 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78ec00d910 con 0x7f78f40b59d0 2022-07-21T01:48:56.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f99c7700 1 -- 172.21.15.140:0/795713980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78f40b59d0 msgr2=0x7f78f40b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f99c7700 1 --2- 172.21.15.140:0/795713980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78f40b59d0 0x7f78f40b07e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f78ec002f70 tx=0x7f78ec006df0).stop 2022-07-21T01:48:56.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f99c7700 1 -- 172.21.15.140:0/795713980 shutdown_connections 2022-07-21T01:48:56.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f99c7700 1 --2- 172.21.15.140:0/795713980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78f40b59d0 0x7f78f40b07e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f99c7700 1 -- 172.21.15.140:0/795713980 >> 172.21.15.140:0/795713980 conn(0x7f78f40634c0 msgr2=0x7f78f40b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:56.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f99c7700 1 -- 172.21.15.140:0/795713980 shutdown_connections 2022-07-21T01:48:56.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f99c7700 1 -- 172.21.15.140:0/795713980 wait complete. 2022-07-21T01:48:56.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f99c7700 1 Processor -- start 2022-07-21T01:48:56.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.187+0000 7f78f99c7700 1 -- start start 2022-07-21T01:48:56.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f99c7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78f40b14f0 0x7f78f4145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f99c7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78f40b59d0 0x7f78f4143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f99c7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78f414ada0 0x7f78f4143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f99c7700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40679d0 con 0x7f78f40b14f0 2022-07-21T01:48:56.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f99c7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4111a80 con 0x7f78f414ada0 2022-07-21T01:48:56.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f99c7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4111d20 con 0x7f78f40b59d0 2022-07-21T01:48:56.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f89c5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78f40b14f0 0x7f78f4145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f89c5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78f40b14f0 0x7f78f4145120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33210/0 (socket says 172.21.15.140:33210) 2022-07-21T01:48:56.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f89c5700 1 -- 172.21.15.140:0/4126032665 learned_addr learned my addr 172.21.15.140:0/4126032665 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:56.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f3fff700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78f40b59d0 0x7f78f4143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f91c6700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78f414ada0 0x7f78f4143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f89c5700 1 -- 172.21.15.140:0/4126032665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78f40b59d0 msgr2=0x7f78f4143790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f89c5700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78f40b59d0 0x7f78f4143790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f89c5700 1 -- 172.21.15.140:0/4126032665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78f414ada0 msgr2=0x7f78f4143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f89c5700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78f414ada0 0x7f78f4143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f89c5700 1 -- 172.21.15.140:0/4126032665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec00b180 con 0x7f78f40b14f0 2022-07-21T01:48:56.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f89c5700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78f40b14f0 0x7f78f4145120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f78ec002f70 tx=0x7f78ec014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:56.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f1ffb700 1 -- 172.21.15.140:0/4126032665 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78ec013de0 con 0x7f78f40b14f0 2022-07-21T01:48:56.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f1ffb700 1 -- 172.21.15.140:0/4126032665 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec00de90 con 0x7f78f40b14f0 2022-07-21T01:48:56.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f99c7700 1 -- 172.21.15.140:0/4126032665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f4144170 con 0x7f78f40b14f0 2022-07-21T01:48:56.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f1ffb700 1 -- 172.21.15.140:0/4126032665 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78ec01e5f0 con 0x7f78f40b14f0 2022-07-21T01:48:56.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f99c7700 1 -- 172.21.15.140:0/4126032665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f410ece0 con 0x7f78f40b14f0 2022-07-21T01:48:56.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.191+0000 7f78f99c7700 1 -- 172.21.15.140:0/4126032665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f4063ce0 con 0x7f78f40b14f0 2022-07-21T01:48:56.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.195+0000 7f78f1ffb700 1 -- 172.21.15.140:0/4126032665 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f78ec01c070 con 0x7f78f40b14f0 2022-07-21T01:48:56.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.195+0000 7f78f1ffb700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78cc053aa0 0x7f78cc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.195+0000 7f78f1ffb700 1 -- 172.21.15.140:0/4126032665 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f78ec06c990 con 0x7f78f40b14f0 2022-07-21T01:48:56.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.195+0000 7f78f3fff700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78cc053aa0 0x7f78cc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.195+0000 7f78f3fff700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78cc053aa0 0x7f78cc055f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f78dc00d000 tx=0x7f78dc007f10).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:56.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.199+0000 7f78f1ffb700 1 -- 172.21.15.140:0/4126032665 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f78f4063ce0 con 0x7f78f40b14f0 2022-07-21T01:48:56.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 -- 172.21.15.140:0/4126032665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78f4063ce0 con 0x7f78f40b14f0 2022-07-21T01:48:56.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f1ffb700 1 -- 172.21.15.140:0/4126032665 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f78f4063ce0 con 0x7f78f40b14f0 2022-07-21T01:48:56.323 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:56.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 -- 172.21.15.140:0/4126032665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78cc053aa0 msgr2=0x7f78cc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78cc053aa0 0x7f78cc055f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f78dc00d000 tx=0x7f78dc007f10).stop 2022-07-21T01:48:56.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 -- 172.21.15.140:0/4126032665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78f40b14f0 msgr2=0x7f78f4145120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78f40b14f0 0x7f78f4145120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f78ec002f70 tx=0x7f78ec014000).stop 2022-07-21T01:48:56.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 -- 172.21.15.140:0/4126032665 shutdown_connections 2022-07-21T01:48:56.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78cc053aa0 0x7f78cc055f60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78f414ada0 0x7f78f4143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78f40b59d0 0x7f78f4143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 --2- 172.21.15.140:0/4126032665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78f40b14f0 0x7f78f4145120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 -- 172.21.15.140:0/4126032665 >> 172.21.15.140:0/4126032665 conn(0x7f78f40634c0 msgr2=0x7f78f40b4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:56.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 -- 172.21.15.140:0/4126032665 shutdown_connections 2022-07-21T01:48:56.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.323+0000 7f78f99c7700 1 -- 172.21.15.140:0/4126032665 wait complete. 2022-07-21T01:48:56.336 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:56.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.439+0000 7f29c21d4700 1 Processor -- start 2022-07-21T01:48:56.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.439+0000 7f29c21d4700 1 -- start start 2022-07-21T01:48:56.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.439+0000 7f29c21d4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f29bc0aebb0 0x7f29bc0b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.439+0000 7f29c21d4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f29bc0acae0 0x7f29bc13f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.439+0000 7f29c21d4700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29bc0aef90 con 0x7f29bc13fd60 2022-07-21T01:48:56.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.439+0000 7f29c21d4700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc0b0270 con 0x7f29bc0aebb0 2022-07-21T01:48:56.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.439+0000 7f29c21d4700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc0af200 con 0x7f29bc0acae0 2022-07-21T01:48:56.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c11d2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f29bc0aebb0 0x7f29bc0b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c11d2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f29bc0aebb0 0x7f29bc0b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33218/0 (socket says 172.21.15.140:33218) 2022-07-21T01:48:56.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c09d1700 1 --2- 172.21.15.140:0/1962263023 >> v2:172.21.15.152:3300/0 conn(0x7f29bc0acae0 0x7f29bc13f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c11d2700 1 -- 172.21.15.140:0/1962263023 learned_addr learned my addr 172.21.15.140:0/1962263023 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:56.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c11d2700 1 -- 172.21.15.140:0/1962263023 >> v2:172.21.15.152:3300/0 conn(0x7f29bc0acae0 msgr2=0x7f29bc13f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:56.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c11d2700 1 --2- 172.21.15.140:0/1962263023 >> v2:172.21.15.152:3300/0 conn(0x7f29bc0acae0 0x7f29bc13f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c11d2700 1 -- 172.21.15.140:0/1962263023 >> v1:172.21.15.152:6789/0 conn(0x7f29bc13fd60 legacy=0x7f29bc1423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c11d2700 1 -- 172.21.15.140:0/1962263023 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0af4a0 con 0x7f29bc0aebb0 2022-07-21T01:48:56.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c11d2700 1 --2- 172.21.15.140:0/1962263023 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0aebb0 0x7f29bc0b0690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f29b8002f70 tx=0x7f29b8006eb0).ready entity=mon.0 client_cookie=9e3be0e7e16828ea server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:56.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29abfff700 1 -- 172.21.15.140:0/1962263023 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29b800d5c0 con 0x7f29bc0aebb0 2022-07-21T01:48:56.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29abfff700 1 -- 172.21.15.140:0/1962263023 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b8000e90 con 0x7f29bc0aebb0 2022-07-21T01:48:56.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29abfff700 1 -- 172.21.15.140:0/1962263023 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29b800d950 con 0x7f29bc0aebb0 2022-07-21T01:48:56.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c21d4700 1 -- 172.21.15.140:0/1962263023 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0aebb0 msgr2=0x7f29bc0b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c21d4700 1 --2- 172.21.15.140:0/1962263023 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0aebb0 0x7f29bc0b0690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f29b8002f70 tx=0x7f29b8006eb0).stop 2022-07-21T01:48:56.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c21d4700 1 -- 172.21.15.140:0/1962263023 shutdown_connections 2022-07-21T01:48:56.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c21d4700 1 --2- 172.21.15.140:0/1962263023 >> v2:172.21.15.152:3300/0 conn(0x7f29bc0acae0 0x7f29bc13f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c21d4700 1 --2- 172.21.15.140:0/1962263023 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0aebb0 0x7f29bc0b0690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c21d4700 1 -- 172.21.15.140:0/1962263023 >> 172.21.15.140:0/1962263023 conn(0x7f29bc0634e0 msgr2=0x7f29bc0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:56.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c21d4700 1 -- 172.21.15.140:0/1962263023 shutdown_connections 2022-07-21T01:48:56.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c21d4700 1 -- 172.21.15.140:0/1962263023 wait complete. 2022-07-21T01:48:56.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c21d4700 1 Processor -- start 2022-07-21T01:48:56.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.443+0000 7f29c21d4700 1 -- start start 2022-07-21T01:48:56.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c21d4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29bc0acae0 0x7f29bc143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c21d4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29bc0aebb0 0x7f29bc064dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c21d4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc13fd60 0x7f29bc152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c21d4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc138b70 con 0x7f29bc13fd60 2022-07-21T01:48:56.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c21d4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc13acc0 con 0x7f29bc0aebb0 2022-07-21T01:48:56.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c21d4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc13af60 con 0x7f29bc0acae0 2022-07-21T01:48:56.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c19d3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc13fd60 0x7f29bc152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c19d3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc13fd60 0x7f29bc152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33226/0 (socket says 172.21.15.140:33226) 2022-07-21T01:48:56.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c19d3700 1 -- 172.21.15.140:0/2510970327 learned_addr learned my addr 172.21.15.140:0/2510970327 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:56.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c11d2700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29bc0acae0 0x7f29bc143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c09d1700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29bc0aebb0 0x7f29bc064dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c19d3700 1 -- 172.21.15.140:0/2510970327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29bc0acae0 msgr2=0x7f29bc143db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c19d3700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29bc0acae0 0x7f29bc143db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c19d3700 1 -- 172.21.15.140:0/2510970327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29bc0aebb0 msgr2=0x7f29bc064dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c19d3700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29bc0aebb0 0x7f29bc064dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c19d3700 1 -- 172.21.15.140:0/2510970327 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b800b1e0 con 0x7f29bc13fd60 2022-07-21T01:48:56.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c09d1700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29bc0aebb0 0x7f29bc064dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:56.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c19d3700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc13fd60 0x7f29bc152400 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f29bc060af0 tx=0x7f29b4007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:56.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29aa7fc700 1 -- 172.21.15.140:0/2510970327 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29b4006aa0 con 0x7f29bc13fd60 2022-07-21T01:48:56.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29aa7fc700 1 -- 172.21.15.140:0/2510970327 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4006c00 con 0x7f29bc13fd60 2022-07-21T01:48:56.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c21d4700 1 -- 172.21.15.140:0/2510970327 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc151e80 con 0x7f29bc13fd60 2022-07-21T01:48:56.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29aa7fc700 1 -- 172.21.15.140:0/2510970327 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29b4007030 con 0x7f29bc13fd60 2022-07-21T01:48:56.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c21d4700 1 -- 172.21.15.140:0/2510970327 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc13a740 con 0x7f29bc13fd60 2022-07-21T01:48:56.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29c21d4700 1 -- 172.21.15.140:0/2510970327 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29bc13ab30 con 0x7f29bc13fd60 2022-07-21T01:48:56.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.447+0000 7f29aa7fc700 1 -- 172.21.15.140:0/2510970327 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f29b4010030 con 0x7f29bc13fd60 2022-07-21T01:48:56.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.451+0000 7f29aa7fc700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053b00 0x7f299c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.451+0000 7f29aa7fc700 1 -- 172.21.15.140:0/2510970327 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f29b406bde0 con 0x7f29bc13fd60 2022-07-21T01:48:56.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.451+0000 7f29c11d2700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053b00 0x7f299c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.451+0000 7f29c11d2700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053b00 0x7f299c055fc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f29b801c000 tx=0x7f29b8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:56.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.455+0000 7f29aa7fc700 1 -- 172.21.15.140:0/2510970327 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f29b4042c60 con 0x7f29bc13fd60 2022-07-21T01:48:56.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.579+0000 7f29c21d4700 1 -- 172.21.15.140:0/2510970327 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29bc1554d0 con 0x7f29bc13fd60 2022-07-21T01:48:56.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29aa7fc700 1 -- 172.21.15.140:0/2510970327 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f29b4046280 con 0x7f29bc13fd60 2022-07-21T01:48:56.581 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:56.582 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:56.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 -- 172.21.15.140:0/2510970327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053b00 msgr2=0x7f299c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053b00 0x7f299c055fc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f29b801c000 tx=0x7f29b8015000).stop 2022-07-21T01:48:56.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 -- 172.21.15.140:0/2510970327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc13fd60 msgr2=0x7f29bc152400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc13fd60 0x7f29bc152400 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f29bc060af0 tx=0x7f29b4007f70).stop 2022-07-21T01:48:56.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 -- 172.21.15.140:0/2510970327 shutdown_connections 2022-07-21T01:48:56.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053b00 0x7f299c055fc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc13fd60 0x7f29bc152400 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29bc0aebb0 0x7f29bc064dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 --2- 172.21.15.140:0/2510970327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29bc0acae0 0x7f29bc143db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 -- 172.21.15.140:0/2510970327 >> 172.21.15.140:0/2510970327 conn(0x7f29bc0634e0 msgr2=0x7f29bc1422d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:56.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 -- 172.21.15.140:0/2510970327 shutdown_connections 2022-07-21T01:48:56.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.583+0000 7f29c21d4700 1 -- 172.21.15.140:0/2510970327 wait complete. 2022-07-21T01:48:56.595 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:48:29.344690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:48:30.427054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:48:56.596 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:48:56.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c91b07700 1 Processor -- start 2022-07-21T01:48:56.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c91b07700 1 -- start start 2022-07-21T01:48:56.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c91b07700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 0x7f2c8c061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c91b07700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2c8c145f60 0x7f2c8c1463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c91b07700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8c0ac340 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c91b07700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8c0acd20 con 0x7f2c8c145f60 2022-07-21T01:48:56.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c91b07700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c0ac5e0 con 0x7f2c8c0ab4a0 2022-07-21T01:48:56.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c8b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2c8c0b34d0 0x7f2c8c061100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47376/0 (socket says 172.21.15.140:47376) 2022-07-21T01:48:56.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c8b7fe700 1 -- 172.21.15.140:0/478515190 learned_addr learned my addr 172.21.15.140:0/478515190 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:56.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c8affd700 1 --2- 172.21.15.140:0/478515190 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 0x7f2c8c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c8bfff700 1 --2- 172.21.15.140:0/478515190 >> v2:172.21.15.152:3300/0 conn(0x7f2c8c145f60 0x7f2c8c1463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c8a7fc700 1 -- 172.21.15.140:0/478515190 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17440027 0 0) 0x7f2c8c0ac340 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c8a7fc700 1 -- 172.21.15.140:0/478515190 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c78003260 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c8affd700 1 -- 172.21.15.140:0/478515190 >> v2:172.21.15.152:3300/0 conn(0x7f2c8c145f60 msgr2=0x7f2c8c1463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:56.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c8affd700 1 --2- 172.21.15.140:0/478515190 >> v2:172.21.15.152:3300/0 conn(0x7f2c8c145f60 0x7f2c8c1463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c8affd700 1 -- 172.21.15.140:0/478515190 >> v1:172.21.15.140:6789/0 conn(0x7f2c8c0b34d0 legacy=0x7f2c8c061100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c8affd700 1 -- 172.21.15.140:0/478515190 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0ac880 con 0x7f2c8c0ab4a0 2022-07-21T01:48:56.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.703+0000 7f2c8affd700 1 --2- 172.21.15.140:0/478515190 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 0x7f2c8c061810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f2c80002f70 tx=0x7f2c80006df0).ready entity=mon.2 client_cookie=9bd9191b249cbfdb server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:56.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c8a7fc700 1 -- 172.21.15.140:0/478515190 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c8000d4a0 con 0x7f2c8c0ab4a0 2022-07-21T01:48:56.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c8a7fc700 1 -- 172.21.15.140:0/478515190 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c80000e90 con 0x7f2c8c0ab4a0 2022-07-21T01:48:56.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c8a7fc700 1 -- 172.21.15.140:0/478515190 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c8000d7f0 con 0x7f2c8c0ab4a0 2022-07-21T01:48:56.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c91b07700 1 -- 172.21.15.140:0/478515190 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 msgr2=0x7f2c8c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c91b07700 1 --2- 172.21.15.140:0/478515190 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 0x7f2c8c061810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f2c80002f70 tx=0x7f2c80006df0).stop 2022-07-21T01:48:56.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c91b07700 1 -- 172.21.15.140:0/478515190 shutdown_connections 2022-07-21T01:48:56.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c91b07700 1 --2- 172.21.15.140:0/478515190 >> v2:172.21.15.152:3300/0 conn(0x7f2c8c145f60 0x7f2c8c1463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c91b07700 1 --2- 172.21.15.140:0/478515190 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 0x7f2c8c061810 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c91b07700 1 -- 172.21.15.140:0/478515190 >> 172.21.15.140:0/478515190 conn(0x7f2c8c0b6c50 msgr2=0x7f2c8c0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:56.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c91b07700 1 -- 172.21.15.140:0/478515190 shutdown_connections 2022-07-21T01:48:56.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c91b07700 1 -- 172.21.15.140:0/478515190 wait complete. 2022-07-21T01:48:56.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c91b07700 1 Processor -- start 2022-07-21T01:48:56.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c91b07700 1 -- start start 2022-07-21T01:48:56.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.707+0000 7f2c91b07700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 0x7f2c8c09bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c91b07700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c8c0b34d0 0x7f2c8c149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c91b07700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c8c145f60 0x7f2c8c0af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c91b07700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c15a6a0 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c91b07700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c15a160 con 0x7f2c8c145f60 2022-07-21T01:48:56.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c91b07700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c15a400 con 0x7f2c8c0ab4a0 2022-07-21T01:48:56.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c8c0b34d0 0x7f2c8c149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c8c0b34d0 0x7f2c8c149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33238/0 (socket says 172.21.15.140:33238) 2022-07-21T01:48:56.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8affd700 1 -- 172.21.15.140:0/3239236185 learned_addr learned my addr 172.21.15.140:0/3239236185 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:56.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8b7fe700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 0x7f2c8c09bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8bfff700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c8c145f60 0x7f2c8c0af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8affd700 1 -- 172.21.15.140:0/3239236185 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 msgr2=0x7f2c8c09bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8affd700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 0x7f2c8c09bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8affd700 1 -- 172.21.15.140:0/3239236185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c8c145f60 msgr2=0x7f2c8c0af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8affd700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c8c145f60 0x7f2c8c0af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8affd700 1 -- 172.21.15.140:0/3239236185 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8000b180 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8bfff700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c8c145f60 0x7f2c8c0af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:56.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8b7fe700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 0x7f2c8c09bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:56.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c8affd700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c8c0b34d0 0x7f2c8c149ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f2c80002f70 tx=0x7f2c80014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:56.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c88ff9700 1 -- 172.21.15.140:0/3239236185 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c80012930 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c88ff9700 1 -- 172.21.15.140:0/3239236185 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c80012a90 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c91b07700 1 -- 172.21.15.140:0/3239236185 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8c0aea60 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c88ff9700 1 -- 172.21.15.140:0/3239236185 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c80012df0 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c91b07700 1 -- 172.21.15.140:0/3239236185 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8c0afe10 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c91b07700 1 -- 172.21.15.140:0/3239236185 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8c09bc70 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.711+0000 7f2c88ff9700 1 -- 172.21.15.140:0/3239236185 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2c8001c070 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.715+0000 7f2c88ff9700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2c6c053a70 0x7f2c6c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.715+0000 7f2c88ff9700 1 -- 172.21.15.140:0/3239236185 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2c8006c9d0 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.715+0000 7f2c8b7fe700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2c6c053a70 0x7f2c6c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.715+0000 7f2c8b7fe700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2c6c053a70 0x7f2c6c055f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f2c7c00d000 tx=0x7f2c7c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:56.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.719+0000 7f2c88ff9700 1 -- 172.21.15.140:0/3239236185 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2c80043850 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 -- 172.21.15.140:0/3239236185 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2c8c0574e0 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c88ff9700 1 -- 172.21.15.140:0/3239236185 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2c80019070 con 0x7f2c8c0b34d0 2022-07-21T01:48:56.846 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:48:56.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 -- 172.21.15.140:0/3239236185 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2c6c053a70 msgr2=0x7f2c6c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2c6c053a70 0x7f2c6c055f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f2c7c00d000 tx=0x7f2c7c009040).stop 2022-07-21T01:48:56.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 -- 172.21.15.140:0/3239236185 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c8c0b34d0 msgr2=0x7f2c8c149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c8c0b34d0 0x7f2c8c149ea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f2c80002f70 tx=0x7f2c80014000).stop 2022-07-21T01:48:56.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 -- 172.21.15.140:0/3239236185 shutdown_connections 2022-07-21T01:48:56.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2c6c053a70 0x7f2c6c055f30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c8c145f60 0x7f2c8c0af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c8c0b34d0 0x7f2c8c149ea0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 --2- 172.21.15.140:0/3239236185 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c8c0ab4a0 0x7f2c8c09bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 -- 172.21.15.140:0/3239236185 >> 172.21.15.140:0/3239236185 conn(0x7f2c8c0b6c50 msgr2=0x7f2c8c05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:56.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 -- 172.21.15.140:0/3239236185 shutdown_connections 2022-07-21T01:48:56.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.847+0000 7f2c91b07700 1 -- 172.21.15.140:0/3239236185 wait complete. 2022-07-21T01:48:56.859 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:48:56.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.967+0000 7f1c79091700 1 Processor -- start 2022-07-21T01:48:56.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c79091700 1 -- start start 2022-07-21T01:48:56.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c79091700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1c740b59d0 0x7f1c7405d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c79091700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c74128d70 0x7f1c7413fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c79091700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c740b5db0 con 0x7f1c740af180 2022-07-21T01:48:56.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c79091700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c740ad800 con 0x7f1c740b59d0 2022-07-21T01:48:56.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c79091700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740adaa0 con 0x7f1c74128d70 2022-07-21T01:48:56.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c73fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1c740b59d0 0x7f1c7405d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c737fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1c740af180 0x7f1c74128660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47390/0 (socket says 172.21.15.140:47390) 2022-07-21T01:48:56.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c73fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1c740b59d0 0x7f1c7405d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33244/0 (socket says 172.21.15.140:33244) 2022-07-21T01:48:56.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c737fe700 1 -- 172.21.15.140:0/160935533 learned_addr learned my addr 172.21.15.140:0/160935533 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:56.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c78890700 1 --2- 172.21.15.140:0/160935533 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c74128d70 0x7f1c7413fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c73fff700 1 -- 172.21.15.140:0/160935533 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c74128d70 msgr2=0x7f1c7413fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:56.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c73fff700 1 --2- 172.21.15.140:0/160935533 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c74128d70 0x7f1c7413fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c73fff700 1 -- 172.21.15.140:0/160935533 >> v1:172.21.15.140:6789/0 conn(0x7f1c740af180 legacy=0x7f1c74128660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c73fff700 1 -- 172.21.15.140:0/160935533 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740ad4f0 con 0x7f1c740b59d0 2022-07-21T01:48:56.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.971+0000 7f1c73fff700 1 --2- 172.21.15.140:0/160935533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c740b59d0 0x7f1c7405d1c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f1c64002f70 tx=0x7f1c64006e80).ready entity=mon.0 client_cookie=249e8ec238e332f0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:56.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c72ffd700 1 -- 172.21.15.140:0/160935533 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c6400b7d0 con 0x7f1c740b59d0 2022-07-21T01:48:56.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c72ffd700 1 -- 172.21.15.140:0/160935533 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6400b930 con 0x7f1c740b59d0 2022-07-21T01:48:56.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c72ffd700 1 -- 172.21.15.140:0/160935533 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c6400bc90 con 0x7f1c740b59d0 2022-07-21T01:48:56.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c79091700 1 -- 172.21.15.140:0/160935533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c740b59d0 msgr2=0x7f1c7405d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c79091700 1 --2- 172.21.15.140:0/160935533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c740b59d0 0x7f1c7405d1c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f1c64002f70 tx=0x7f1c64006e80).stop 2022-07-21T01:48:56.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c79091700 1 -- 172.21.15.140:0/160935533 shutdown_connections 2022-07-21T01:48:56.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c79091700 1 --2- 172.21.15.140:0/160935533 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c74128d70 0x7f1c7413fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c79091700 1 --2- 172.21.15.140:0/160935533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c740b59d0 0x7f1c7405d1c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c79091700 1 -- 172.21.15.140:0/160935533 >> 172.21.15.140:0/160935533 conn(0x7f1c740634c0 msgr2=0x7f1c740b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:56.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c79091700 1 -- 172.21.15.140:0/160935533 shutdown_connections 2022-07-21T01:48:56.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c79091700 1 -- 172.21.15.140:0/160935533 wait complete. 2022-07-21T01:48:56.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c79091700 1 Processor -- start 2022-07-21T01:48:56.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.975+0000 7f1c79091700 1 -- start start 2022-07-21T01:48:56.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c79091700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c740af180 0x7f1c74125220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c79091700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c740b59d0 0x7f1c7415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c79091700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c74128d70 0x7f1c74141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:56.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c79091700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c741597d0 con 0x7f1c74128d70 2022-07-21T01:48:56.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c79091700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7415a660 con 0x7f1c740b59d0 2022-07-21T01:48:56.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c79091700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c74159a30 con 0x7f1c740af180 2022-07-21T01:48:56.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c78890700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c74128d70 0x7f1c74141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c78890700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c74128d70 0x7f1c74141350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33254/0 (socket says 172.21.15.140:33254) 2022-07-21T01:48:56.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c78890700 1 -- 172.21.15.140:0/3145304417 learned_addr learned my addr 172.21.15.140:0/3145304417 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:56.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c73fff700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c740af180 0x7f1c74125220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c737fe700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c740b59d0 0x7f1c7415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:56.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c78890700 1 -- 172.21.15.140:0/3145304417 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c740af180 msgr2=0x7f1c74125220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:56.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c78890700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c740af180 0x7f1c74125220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:56.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c78890700 1 -- 172.21.15.140:0/3145304417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c740b59d0 msgr2=0x7f1c7415a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c78890700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c740b59d0 0x7f1c7415a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c78890700 1 -- 172.21.15.140:0/3145304417 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6400d410 con 0x7f1c74128d70 2022-07-21T01:48:57.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c78890700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c74128d70 0x7f1c74141350 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f1c7415a7f0 tx=0x7f1c6c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:57.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c717fa700 1 -- 172.21.15.140:0/3145304417 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c6c012600 con 0x7f1c74128d70 2022-07-21T01:48:57.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c717fa700 1 -- 172.21.15.140:0/3145304417 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c012760 con 0x7f1c74128d70 2022-07-21T01:48:57.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c79091700 1 -- 172.21.15.140:0/3145304417 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c74159cd0 con 0x7f1c74128d70 2022-07-21T01:48:57.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c717fa700 1 -- 172.21.15.140:0/3145304417 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c6c012ac0 con 0x7f1c74128d70 2022-07-21T01:48:57.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c79091700 1 -- 172.21.15.140:0/3145304417 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c74140bc0 con 0x7f1c74128d70 2022-07-21T01:48:57.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c79091700 1 -- 172.21.15.140:0/3145304417 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c74159e60 con 0x7f1c74128d70 2022-07-21T01:48:57.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c717fa700 1 -- 172.21.15.140:0/3145304417 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1c6c022070 con 0x7f1c74128d70 2022-07-21T01:48:57.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c717fa700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c54053a70 0x7f1c54055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.979+0000 7f1c717fa700 1 -- 172.21.15.140:0/3145304417 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1c6c06c460 con 0x7f1c74128d70 2022-07-21T01:48:57.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.983+0000 7f1c73fff700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c54053a70 0x7f1c54055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.983+0000 7f1c73fff700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c54053a70 0x7f1c54055f30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f1c6401b000 tx=0x7f1c64006e80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:57.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:56.987+0000 7f1c717fa700 1 -- 172.21.15.140:0/3145304417 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c6c043380 con 0x7f1c74128d70 2022-07-21T01:48:57.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.111+0000 7f1c79091700 1 -- 172.21.15.140:0/3145304417 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1c740574e0 con 0x7f1c74128d70 2022-07-21T01:48:57.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.111+0000 7f1c717fa700 1 -- 172.21.15.140:0/3145304417 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1c6c016020 con 0x7f1c74128d70 2022-07-21T01:48:57.110 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:57.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 -- 172.21.15.140:0/3145304417 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c54053a70 msgr2=0x7f1c54055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c54053a70 0x7f1c54055f30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f1c6401b000 tx=0x7f1c64006e80).stop 2022-07-21T01:48:57.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 -- 172.21.15.140:0/3145304417 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c74128d70 msgr2=0x7f1c74141350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c74128d70 0x7f1c74141350 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f1c7415a7f0 tx=0x7f1c6c0091d0).stop 2022-07-21T01:48:57.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 -- 172.21.15.140:0/3145304417 shutdown_connections 2022-07-21T01:48:57.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c54053a70 0x7f1c54055f30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c74128d70 0x7f1c74141350 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c740b59d0 0x7f1c7415a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 --2- 172.21.15.140:0/3145304417 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c740af180 0x7f1c74125220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 -- 172.21.15.140:0/3145304417 >> 172.21.15.140:0/3145304417 conn(0x7f1c740634c0 msgr2=0x7f1c7405c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:57.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 -- 172.21.15.140:0/3145304417 shutdown_connections 2022-07-21T01:48:57.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.115+0000 7f1c79091700 1 -- 172.21.15.140:0/3145304417 wait complete. 2022-07-21T01:48:57.129 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:48:57.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641ccf4700 1 Processor -- start 2022-07-21T01:48:57.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641ccf4700 1 -- start start 2022-07-21T01:48:57.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641ccf4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641ccf4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f64181437e0 0x7f641813f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641ccf4700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64180b3a50 con 0x7f641813fe10 2022-07-21T01:48:57.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641ccf4700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f641813d3b0 con 0x7f64181437e0 2022-07-21T01:48:57.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641ccf4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64180ac670 con 0x7f64180b3670 2022-07-21T01:48:57.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f6415d9b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f64181437e0 0x7f641813f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641659c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f6415d9b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f64181437e0 0x7f641813f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:38998/0 (socket says 172.21.15.140:38998) 2022-07-21T01:48:57.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641659c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180ac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:58220/0 (socket says 172.21.15.140:58220) 2022-07-21T01:48:57.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f6415d9b700 1 -- 172.21.15.140:0/1615277301 learned_addr learned my addr 172.21.15.140:0/1615277301 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:57.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f6415d9b700 1 -- 172.21.15.140:0/1615277301 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 msgr2=0x7f64180ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:57.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f6415d9b700 1 --2- 172.21.15.140:0/1615277301 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f6415d9b700 1 -- 172.21.15.140:0/1615277301 >> v1:172.21.15.152:6789/0 conn(0x7f641813fe10 legacy=0x7f6418099300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641559a700 1 -- 172.21.15.140:0/1615277301 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082297645 0 0) 0x7f64180b3a50 con 0x7f641813fe10 2022-07-21T01:48:57.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f6415d9b700 1 -- 172.21.15.140:0/1615277301 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64181430a0 con 0x7f64181437e0 2022-07-21T01:48:57.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641659c700 1 --2- 172.21.15.140:0/1615277301 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:57.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f6415d9b700 1 --2- 172.21.15.140:0/1615277301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64181437e0 0x7f641813f920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6408002f70 tx=0x7f6408006f40).ready entity=mon.1 client_cookie=c6cd4f0206ca38fd server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:57.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.239+0000 7f641559a700 1 -- 172.21.15.140:0/1615277301 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640800da80 con 0x7f64181437e0 2022-07-21T01:48:57.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641559a700 1 -- 172.21.15.140:0/1615277301 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6408000e90 con 0x7f64181437e0 2022-07-21T01:48:57.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641559a700 1 -- 172.21.15.140:0/1615277301 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640800ddd0 con 0x7f64181437e0 2022-07-21T01:48:57.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 -- 172.21.15.140:0/1615277301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64181437e0 msgr2=0x7f641813f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 --2- 172.21.15.140:0/1615277301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64181437e0 0x7f641813f920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6408002f70 tx=0x7f6408006f40).stop 2022-07-21T01:48:57.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 -- 172.21.15.140:0/1615277301 shutdown_connections 2022-07-21T01:48:57.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 --2- 172.21.15.140:0/1615277301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64181437e0 0x7f641813f920 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 --2- 172.21.15.140:0/1615277301 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 -- 172.21.15.140:0/1615277301 >> 172.21.15.140:0/1615277301 conn(0x7f64180b6c50 msgr2=0x7f64180b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:57.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 -- 172.21.15.140:0/1615277301 shutdown_connections 2022-07-21T01:48:57.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 -- 172.21.15.140:0/1615277301 wait complete. 2022-07-21T01:48:57.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 Processor -- start 2022-07-21T01:48:57.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 -- start start 2022-07-21T01:48:57.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180b1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f641813fe10 0x7f6418151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64181437e0 0x7f641813d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64180af0c0 con 0x7f641813fe10 2022-07-21T01:48:57.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6418063a40 con 0x7f64181437e0 2022-07-21T01:48:57.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.243+0000 7f641ccf4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6418063ce0 con 0x7f64180b3670 2022-07-21T01:48:57.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641659c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f6415d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f641813fe10 0x7f6418151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641659c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:58226/0 (socket says 172.21.15.140:58226) 2022-07-21T01:48:57.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641659c700 1 -- 172.21.15.140:0/1933541558 learned_addr learned my addr 172.21.15.140:0/1933541558 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:57.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f6416d9d700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64181437e0 0x7f641813d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641659c700 1 -- 172.21.15.140:0/1933541558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64181437e0 msgr2=0x7f641813d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641659c700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64181437e0 0x7f641813d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641659c700 1 -- 172.21.15.140:0/1933541558 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f641813fe10 msgr2=0x7f6418151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641659c700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f641813fe10 0x7f6418151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641659c700 1 -- 172.21.15.140:0/1933541558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640000b180 con 0x7f64180b3670 2022-07-21T01:48:57.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f6415d9b700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f641813fe10 0x7f6418151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:57.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641659c700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180b1960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f640000c000 tx=0x7f6400006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:57.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f640f7fe700 1 -- 172.21.15.140:0/1933541558 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640000c7e0 con 0x7f64180b3670 2022-07-21T01:48:57.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f640f7fe700 1 -- 172.21.15.140:0/1933541558 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640000c940 con 0x7f64180b3670 2022-07-21T01:48:57.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641ccf4700 1 -- 172.21.15.140:0/1933541558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640800b460 con 0x7f64180b3670 2022-07-21T01:48:57.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f640f7fe700 1 -- 172.21.15.140:0/1933541558 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640000cca0 con 0x7f64180b3670 2022-07-21T01:48:57.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641ccf4700 1 -- 172.21.15.140:0/1933541558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6418064220 con 0x7f64180b3670 2022-07-21T01:48:57.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f641ccf4700 1 -- 172.21.15.140:0/1933541558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64180643b0 con 0x7f64180b3670 2022-07-21T01:48:57.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.247+0000 7f640f7fe700 1 -- 172.21.15.140:0/1933541558 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f640001c460 con 0x7f64180b3670 2022-07-21T01:48:57.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.251+0000 7f640f7fe700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63fc053a70 0x7f63fc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.251+0000 7f640f7fe700 1 -- 172.21.15.140:0/1933541558 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f640006c4a0 con 0x7f64180b3670 2022-07-21T01:48:57.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.251+0000 7f6415d9b700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63fc053a70 0x7f63fc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.251+0000 7f6415d9b700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63fc053a70 0x7f63fc055f30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f640801e000 tx=0x7f6418062040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:57.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.259+0000 7f640f7fe700 1 -- 172.21.15.140:0/1933541558 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6400043360 con 0x7f64180b3670 2022-07-21T01:48:57.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 -- 172.21.15.140:0/1933541558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64180574e0 con 0x7f64180b3670 2022-07-21T01:48:57.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f640f7fe700 1 -- 172.21.15.140:0/1933541558 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7f640000ce00 con 0x7f64180b3670 2022-07-21T01:48:57.381 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:57.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 -- 172.21.15.140:0/1933541558 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63fc053a70 msgr2=0x7f63fc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63fc053a70 0x7f63fc055f30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f640801e000 tx=0x7f6418062040).stop 2022-07-21T01:48:57.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 -- 172.21.15.140:0/1933541558 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 msgr2=0x7f64180b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180b1960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f640000c000 tx=0x7f6400006df0).stop 2022-07-21T01:48:57.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 -- 172.21.15.140:0/1933541558 shutdown_connections 2022-07-21T01:48:57.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63fc053a70 0x7f63fc055f30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64181437e0 0x7f641813d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f641813fe10 0x7f6418151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 --2- 172.21.15.140:0/1933541558 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f64180b3670 0x7f64180b1960 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 -- 172.21.15.140:0/1933541558 >> 172.21.15.140:0/1933541558 conn(0x7f64180b6c50 msgr2=0x7f64180b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:57.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 -- 172.21.15.140:0/1933541558 shutdown_connections 2022-07-21T01:48:57.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.383+0000 7f641ccf4700 1 -- 172.21.15.140:0/1933541558 wait complete. 2022-07-21T01:48:57.395 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:48:57.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.503+0000 7f5cd4090700 1 Processor -- start 2022-07-21T01:48:57.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.503+0000 7f5cd4090700 1 -- start start 2022-07-21T01:48:57.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc1462a0 0x7f5ccc146680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5ccc0988b0 0x7f5ccc146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5ccc1422c0 0x7f5ccc142730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ccc0afdb0 con 0x7f5ccc1422c0 2022-07-21T01:48:57.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ccc0b0050 con 0x7f5ccc0988b0 2022-07-21T01:48:57.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc05fcd0 con 0x7f5ccc1462a0 2022-07-21T01:48:57.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd262d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5ccc1422c0 0x7f5ccc142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd262d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5ccc1422c0 0x7f5ccc142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33276/0 (socket says 172.21.15.140:33276) 2022-07-21T01:48:57.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd262d700 1 -- 172.21.15.140:0/1790562035 learned_addr learned my addr 172.21.15.140:0/1790562035 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:57.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd162b700 1 --2- 172.21.15.140:0/1790562035 >> v2:172.21.15.152:3300/0 conn(0x7f5ccc0988b0 0x7f5ccc146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd1e2c700 1 --2- 172.21.15.140:0/1790562035 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc1462a0 0x7f5ccc146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd262d700 1 -- 172.21.15.140:0/1790562035 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc1462a0 msgr2=0x7f5ccc146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:57.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd262d700 1 --2- 172.21.15.140:0/1790562035 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc1462a0 0x7f5ccc146680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd262d700 1 -- 172.21.15.140:0/1790562035 >> v2:172.21.15.152:3300/0 conn(0x7f5ccc0988b0 msgr2=0x7f5ccc146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:57.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd262d700 1 --2- 172.21.15.140:0/1790562035 >> v2:172.21.15.152:3300/0 conn(0x7f5ccc0988b0 0x7f5ccc146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd262d700 1 -- 172.21.15.140:0/1790562035 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc0afae0 con 0x7f5ccc1422c0 2022-07-21T01:48:57.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd262d700 1 --2- 172.21.15.140:0/1790562035 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc1422c0 0x7f5ccc142730 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f5cc40032f0 tx=0x7f5cc4009290).ready entity=mon.0 client_cookie=f1b3ad4a4fa10119 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:57.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd0e2a700 1 -- 172.21.15.140:0/1790562035 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cc400b870 con 0x7f5ccc1422c0 2022-07-21T01:48:57.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd0e2a700 1 -- 172.21.15.140:0/1790562035 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc400b9d0 con 0x7f5ccc1422c0 2022-07-21T01:48:57.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd0e2a700 1 -- 172.21.15.140:0/1790562035 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cc400bb40 con 0x7f5ccc1422c0 2022-07-21T01:48:57.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 -- 172.21.15.140:0/1790562035 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc1422c0 msgr2=0x7f5ccc142730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 --2- 172.21.15.140:0/1790562035 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc1422c0 0x7f5ccc142730 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f5cc40032f0 tx=0x7f5cc4009290).stop 2022-07-21T01:48:57.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 -- 172.21.15.140:0/1790562035 shutdown_connections 2022-07-21T01:48:57.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 --2- 172.21.15.140:0/1790562035 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc1422c0 0x7f5ccc142730 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 --2- 172.21.15.140:0/1790562035 >> v2:172.21.15.152:3300/0 conn(0x7f5ccc0988b0 0x7f5ccc146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 --2- 172.21.15.140:0/1790562035 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc1462a0 0x7f5ccc146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 -- 172.21.15.140:0/1790562035 >> 172.21.15.140:0/1790562035 conn(0x7f5ccc0b5f00 msgr2=0x7f5ccc0b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:57.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 -- 172.21.15.140:0/1790562035 shutdown_connections 2022-07-21T01:48:57.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.507+0000 7f5cd4090700 1 -- 172.21.15.140:0/1790562035 wait complete. 2022-07-21T01:48:57.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.511+0000 7f5cd4090700 1 Processor -- start 2022-07-21T01:48:57.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.511+0000 7f5cd4090700 1 -- start start 2022-07-21T01:48:57.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.511+0000 7f5cd4090700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc0988b0 0x7f5ccc0b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.511+0000 7f5cd4090700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc1422c0 0x7f5ccc144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.511+0000 7f5cd4090700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc1462a0 0x7f5ccc09dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.511+0000 7f5cd4090700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc146120 con 0x7f5ccc0988b0 2022-07-21T01:48:57.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.511+0000 7f5cd4090700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc144640 con 0x7f5ccc1422c0 2022-07-21T01:48:57.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.511+0000 7f5cd4090700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc09d8e0 con 0x7f5ccc1462a0 2022-07-21T01:48:57.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.511+0000 7f5cd1e2c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc0988b0 0x7f5ccc0b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.511+0000 7f5cd1e2c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc0988b0 0x7f5ccc0b8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33278/0 (socket says 172.21.15.140:33278) 2022-07-21T01:48:57.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.511+0000 7f5cd1e2c700 1 -- 172.21.15.140:0/3729164152 learned_addr learned my addr 172.21.15.140:0/3729164152 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:57.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cd262d700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc1462a0 0x7f5ccc09dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cd162b700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc1422c0 0x7f5ccc144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cd1e2c700 1 -- 172.21.15.140:0/3729164152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc1462a0 msgr2=0x7f5ccc09dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cd1e2c700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc1462a0 0x7f5ccc09dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cd1e2c700 1 -- 172.21.15.140:0/3729164152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc1422c0 msgr2=0x7f5ccc144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cd1e2c700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc1422c0 0x7f5ccc144be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cd1e2c700 1 -- 172.21.15.140:0/3729164152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc4009530 con 0x7f5ccc0988b0 2022-07-21T01:48:57.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cd1e2c700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc0988b0 0x7f5ccc0b8530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f5cbc0030b0 tx=0x7f5cbc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:57.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cbaffd700 1 -- 172.21.15.140:0/3729164152 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cbc011780 con 0x7f5ccc0988b0 2022-07-21T01:48:57.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cbaffd700 1 -- 172.21.15.140:0/3729164152 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc0118e0 con 0x7f5ccc0988b0 2022-07-21T01:48:57.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cd4090700 1 -- 172.21.15.140:0/3729164152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ccc1450d0 con 0x7f5ccc0988b0 2022-07-21T01:48:57.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cbaffd700 1 -- 172.21.15.140:0/3729164152 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cbc011c40 con 0x7f5ccc0988b0 2022-07-21T01:48:57.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cd4090700 1 -- 172.21.15.140:0/3729164152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ccc144340 con 0x7f5ccc0988b0 2022-07-21T01:48:57.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cd4090700 1 -- 172.21.15.140:0/3729164152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ccc09d720 con 0x7f5ccc0988b0 2022-07-21T01:48:57.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cbaffd700 1 -- 172.21.15.140:0/3729164152 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5cbc012420 con 0x7f5ccc0988b0 2022-07-21T01:48:57.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.515+0000 7f5cbaffd700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cac053950 0x7f5cac055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.519+0000 7f5cd162b700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cac053950 0x7f5cac055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.519+0000 7f5cbaffd700 1 -- 172.21.15.140:0/3729164152 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5cbc06bbc0 con 0x7f5ccc0988b0 2022-07-21T01:48:57.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.519+0000 7f5cd162b700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cac053950 0x7f5cac055e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f5cc800c000 tx=0x7f5cc8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:57.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.523+0000 7f5cbaffd700 1 -- 172.21.15.140:0/3729164152 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5cbc042a80 con 0x7f5ccc0988b0 2022-07-21T01:48:57.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.647+0000 7f5cd4090700 1 -- 172.21.15.140:0/3729164152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ccc0b3f60 con 0x7f5ccc0988b0 2022-07-21T01:48:57.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.647+0000 7f5cbaffd700 1 -- 172.21.15.140:0/3729164152 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5cbc0460a0 con 0x7f5ccc0988b0 2022-07-21T01:48:57.648 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:57.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 -- 172.21.15.140:0/3729164152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cac053950 msgr2=0x7f5cac055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cac053950 0x7f5cac055e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f5cc800c000 tx=0x7f5cc8008040).stop 2022-07-21T01:48:57.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 -- 172.21.15.140:0/3729164152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc0988b0 msgr2=0x7f5ccc0b8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc0988b0 0x7f5ccc0b8530 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f5cbc0030b0 tx=0x7f5cbc008040).stop 2022-07-21T01:48:57.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 -- 172.21.15.140:0/3729164152 shutdown_connections 2022-07-21T01:48:57.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cac053950 0x7f5cac055e10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc1462a0 0x7f5ccc09dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc1422c0 0x7f5ccc144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 --2- 172.21.15.140:0/3729164152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc0988b0 0x7f5ccc0b8530 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 -- 172.21.15.140:0/3729164152 >> 172.21.15.140:0/3729164152 conn(0x7f5ccc0b5f00 msgr2=0x7f5ccc060590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:57.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 -- 172.21.15.140:0/3729164152 shutdown_connections 2022-07-21T01:48:57.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.651+0000 7f5cd4090700 1 -- 172.21.15.140:0/3729164152 wait complete. 2022-07-21T01:48:57.665 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:48:57.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 Processor -- start 2022-07-21T01:48:57.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 -- start start 2022-07-21T01:48:57.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f145c0adff0 0x7f145c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f145c0af8e0 0x7f145c05bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145c147010 con 0x7f145c0b5650 2022-07-21T01:48:57.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f145c1472b0 con 0x7f145c0af8e0 2022-07-21T01:48:57.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145c146d40 con 0x7f145c0adff0 2022-07-21T01:48:57.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146113d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f145c0b5650 0x7f145c147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47432/0 (socket says 172.21.15.140:47432) 2022-07-21T01:48:57.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146113d700 1 -- 172.21.15.140:0/3743490480 learned_addr learned my addr 172.21.15.140:0/3743490480 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:57.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146093c700 1 --2- 172.21.15.140:0/3743490480 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f145c0adff0 0x7f145c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146193e700 1 --2- 172.21.15.140:0/3743490480 >> v2:172.21.15.152:3300/0 conn(0x7f145c0af8e0 0x7f145c05bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f144ffff700 1 -- 172.21.15.140:0/3743490480 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187926447 0 0) 0x7f145c147010 con 0x7f145c0b5650 2022-07-21T01:48:57.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f144ffff700 1 -- 172.21.15.140:0/3743490480 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1440003260 con 0x7f145c0b5650 2022-07-21T01:48:57.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146093c700 1 -- 172.21.15.140:0/3743490480 >> v2:172.21.15.152:3300/0 conn(0x7f145c0af8e0 msgr2=0x7f145c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:57.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146093c700 1 --2- 172.21.15.140:0/3743490480 >> v2:172.21.15.152:3300/0 conn(0x7f145c0af8e0 0x7f145c05bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146093c700 1 -- 172.21.15.140:0/3743490480 >> v1:172.21.15.140:6789/0 conn(0x7f145c0b5650 legacy=0x7f145c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146093c700 1 -- 172.21.15.140:0/3743490480 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c0acae0 con 0x7f145c0adff0 2022-07-21T01:48:57.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146093c700 1 --2- 172.21.15.140:0/3743490480 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f145c0adff0 0x7f145c0b5a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f14540032f0 tx=0x7f14540091a0).ready entity=mon.2 client_cookie=72f13280a69218ca server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:57.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f144ffff700 1 -- 172.21.15.140:0/3743490480 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f145400b600 con 0x7f145c0adff0 2022-07-21T01:48:57.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f144ffff700 1 -- 172.21.15.140:0/3743490480 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145400b760 con 0x7f145c0adff0 2022-07-21T01:48:57.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f144ffff700 1 -- 172.21.15.140:0/3743490480 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f145400b8d0 con 0x7f145c0adff0 2022-07-21T01:48:57.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 -- 172.21.15.140:0/3743490480 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f145c0adff0 msgr2=0x7f145c0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 --2- 172.21.15.140:0/3743490480 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f145c0adff0 0x7f145c0b5a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f14540032f0 tx=0x7f14540091a0).stop 2022-07-21T01:48:57.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 -- 172.21.15.140:0/3743490480 shutdown_connections 2022-07-21T01:48:57.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 --2- 172.21.15.140:0/3743490480 >> v2:172.21.15.152:3300/0 conn(0x7f145c0af8e0 0x7f145c05bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 --2- 172.21.15.140:0/3743490480 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f145c0adff0 0x7f145c0b5a30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 -- 172.21.15.140:0/3743490480 >> 172.21.15.140:0/3743490480 conn(0x7f145c0634e0 msgr2=0x7f145c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:57.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 -- 172.21.15.140:0/3743490480 shutdown_connections 2022-07-21T01:48:57.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.771+0000 7f146213f700 1 -- 172.21.15.140:0/3743490480 wait complete. 2022-07-21T01:48:57.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146213f700 1 Processor -- start 2022-07-21T01:48:57.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146213f700 1 -- start start 2022-07-21T01:48:57.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146213f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f145c0adff0 0x7f145c148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146213f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f145c0af8e0 0x7f145c144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146213f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f145c0b5650 0x7f145c1454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146213f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c149a10 con 0x7f145c0b5650 2022-07-21T01:48:57.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146213f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c156170 con 0x7f145c0af8e0 2022-07-21T01:48:57.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146213f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145c064ad0 con 0x7f145c0adff0 2022-07-21T01:48:57.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146193e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f145c0b5650 0x7f145c1454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146193e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f145c0b5650 0x7f145c1454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33294/0 (socket says 172.21.15.140:33294) 2022-07-21T01:48:57.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146193e700 1 -- 172.21.15.140:0/16371610 learned_addr learned my addr 172.21.15.140:0/16371610 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:57.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146113d700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f145c0adff0 0x7f145c148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146093c700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f145c0af8e0 0x7f145c144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146193e700 1 -- 172.21.15.140:0/16371610 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f145c0adff0 msgr2=0x7f145c148fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146193e700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f145c0adff0 0x7f145c148fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146193e700 1 -- 172.21.15.140:0/16371610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f145c0af8e0 msgr2=0x7f145c144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146193e700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f145c0af8e0 0x7f145c144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146193e700 1 -- 172.21.15.140:0/16371610 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454009420 con 0x7f145c0b5650 2022-07-21T01:48:57.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146193e700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f145c0b5650 0x7f145c1454e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f145800c000 tx=0x7f1458006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:57.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f144e7fc700 1 -- 172.21.15.140:0/16371610 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1458011750 con 0x7f145c0b5650 2022-07-21T01:48:57.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f144e7fc700 1 -- 172.21.15.140:0/16371610 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14580118b0 con 0x7f145c0b5650 2022-07-21T01:48:57.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146213f700 1 -- 172.21.15.140:0/16371610 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145c1459d0 con 0x7f145c0b5650 2022-07-21T01:48:57.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f144e7fc700 1 -- 172.21.15.140:0/16371610 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1458011c10 con 0x7f145c0b5650 2022-07-21T01:48:57.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.775+0000 7f146213f700 1 -- 172.21.15.140:0/16371610 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c10f660 con 0x7f145c0b5650 2022-07-21T01:48:57.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.779+0000 7f146213f700 1 -- 172.21.15.140:0/16371610 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145c064cd0 con 0x7f145c0b5650 2022-07-21T01:48:57.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.779+0000 7f144e7fc700 1 -- 172.21.15.140:0/16371610 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1458011d70 con 0x7f145c0b5650 2022-07-21T01:48:57.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.779+0000 7f144e7fc700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1444053a10 0x7f1444055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:57.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.779+0000 7f144e7fc700 1 -- 172.21.15.140:0/16371610 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f145806c1f0 con 0x7f145c0b5650 2022-07-21T01:48:57.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.779+0000 7f146113d700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1444053a10 0x7f1444055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:57.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.779+0000 7f146113d700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1444053a10 0x7f1444055ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f145000d000 tx=0x7f1450009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:57.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.783+0000 7f144e7fc700 1 -- 172.21.15.140:0/16371610 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1458043110 con 0x7f145c0b5650 2022-07-21T01:48:57.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.911+0000 7f146213f700 1 -- 172.21.15.140:0/16371610 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f145c064920 con 0x7f145c0b5650 2022-07-21T01:48:57.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.911+0000 7f144e7fc700 1 -- 172.21.15.140:0/16371610 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f1458046730 con 0x7f145c0b5650 2022-07-21T01:48:57.912 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:48:57.912 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:48:32.426720+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:48:29.344690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-07-21T01:48:30.427054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:48:57.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 -- 172.21.15.140:0/16371610 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1444053a10 msgr2=0x7f1444055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1444053a10 0x7f1444055ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f145000d000 tx=0x7f1450009040).stop 2022-07-21T01:48:57.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 -- 172.21.15.140:0/16371610 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f145c0b5650 msgr2=0x7f145c1454e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:57.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f145c0b5650 0x7f145c1454e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f145800c000 tx=0x7f1458006df0).stop 2022-07-21T01:48:57.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 -- 172.21.15.140:0/16371610 shutdown_connections 2022-07-21T01:48:57.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1444053a10 0x7f1444055ed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f145c0b5650 0x7f145c1454e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f145c0af8e0 0x7f145c144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 --2- 172.21.15.140:0/16371610 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f145c0adff0 0x7f145c148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:57.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 -- 172.21.15.140:0/16371610 >> 172.21.15.140:0/16371610 conn(0x7f145c0634e0 msgr2=0x7f145c0ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:57.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 -- 172.21.15.140:0/16371610 shutdown_connections 2022-07-21T01:48:57.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:57.915+0000 7f146213f700 1 -- 172.21.15.140:0/16371610 wait complete. 2022-07-21T01:48:57.925 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:48:29.344690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:48:30.427054+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:48:57.926 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:48:58.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.035+0000 7f59716cf700 1 Processor -- start 2022-07-21T01:48:58.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.035+0000 7f59716cf700 1 -- start start 2022-07-21T01:48:58.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f596c0b3670 0x7f596c0ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f596c1437e0 0x7f596c13f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c0b3a50 con 0x7f596c13fe10 2022-07-21T01:48:58.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c13d3b0 con 0x7f596c1437e0 2022-07-21T01:48:58.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c0ac670 con 0x7f596c0b3670 2022-07-21T01:48:58.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f596a7fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f596c1437e0 0x7f596c13f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f596b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f596c13fe10 0x7f596c099300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47450/0 (socket says 172.21.15.140:47450) 2022-07-21T01:48:58.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f596b7fe700 1 -- 172.21.15.140:0/1626765072 learned_addr learned my addr 172.21.15.140:0/1626765072 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:58.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f596affd700 1 --2- 172.21.15.140:0/1626765072 >> v2:172.21.15.152:3300/0 conn(0x7f596c0b3670 0x7f596c0ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f596a7fc700 1 -- 172.21.15.140:0/1626765072 >> v2:172.21.15.152:3300/0 conn(0x7f596c0b3670 msgr2=0x7f596c0ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:58.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f596a7fc700 1 --2- 172.21.15.140:0/1626765072 >> v2:172.21.15.152:3300/0 conn(0x7f596c0b3670 0x7f596c0ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f596a7fc700 1 -- 172.21.15.140:0/1626765072 >> v1:172.21.15.140:6789/0 conn(0x7f596c13fe10 legacy=0x7f596c099300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f596a7fc700 1 -- 172.21.15.140:0/1626765072 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c1430a0 con 0x7f596c1437e0 2022-07-21T01:48:58.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f596a7fc700 1 --2- 172.21.15.140:0/1626765072 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f596c1437e0 0x7f596c13f920 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f5960002f70 tx=0x7f5960006e80).ready entity=mon.0 client_cookie=f9403e94ec9c27e0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f5969ffb700 1 -- 172.21.15.140:0/1626765072 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f596000ba40 con 0x7f596c1437e0 2022-07-21T01:48:58.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f5969ffb700 1 -- 172.21.15.140:0/1626765072 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5960000e90 con 0x7f596c1437e0 2022-07-21T01:48:58.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f5969ffb700 1 -- 172.21.15.140:0/1626765072 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f596000bd90 con 0x7f596c1437e0 2022-07-21T01:48:58.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 -- 172.21.15.140:0/1626765072 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f596c1437e0 msgr2=0x7f596c13f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 --2- 172.21.15.140:0/1626765072 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f596c1437e0 0x7f596c13f920 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f5960002f70 tx=0x7f5960006e80).stop 2022-07-21T01:48:58.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 -- 172.21.15.140:0/1626765072 shutdown_connections 2022-07-21T01:48:58.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 --2- 172.21.15.140:0/1626765072 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f596c1437e0 0x7f596c13f920 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 --2- 172.21.15.140:0/1626765072 >> v2:172.21.15.152:3300/0 conn(0x7f596c0b3670 0x7f596c0ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 -- 172.21.15.140:0/1626765072 >> 172.21.15.140:0/1626765072 conn(0x7f596c0b6c50 msgr2=0x7f596c0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:58.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 -- 172.21.15.140:0/1626765072 shutdown_connections 2022-07-21T01:48:58.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 -- 172.21.15.140:0/1626765072 wait complete. 2022-07-21T01:48:58.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 Processor -- start 2022-07-21T01:48:58.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.039+0000 7f59716cf700 1 -- start start 2022-07-21T01:48:58.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f59716cf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f596c0b3670 0x7f596c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f59716cf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f596c13fe10 0x7f596c0af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f59716cf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f596c1437e0 0x7f596c0af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f59716cf700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0aee80 con 0x7f596c0b3670 2022-07-21T01:48:58.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f59716cf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c09b4a0 con 0x7f596c1437e0 2022-07-21T01:48:58.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f59716cf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c09b740 con 0x7f596c13fe10 2022-07-21T01:48:58.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f596c0b3670 0x7f596c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f596c0b3670 0x7f596c0b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33306/0 (socket says 172.21.15.140:33306) 2022-07-21T01:48:58.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596affd700 1 -- 172.21.15.140:0/3882936495 learned_addr learned my addr 172.21.15.140:0/3882936495 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:58.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596a7fc700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f596c13fe10 0x7f596c0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596b7fe700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f596c1437e0 0x7f596c0af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596affd700 1 -- 172.21.15.140:0/3882936495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f596c13fe10 msgr2=0x7f596c0af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596affd700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f596c13fe10 0x7f596c0af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596affd700 1 -- 172.21.15.140:0/3882936495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f596c1437e0 msgr2=0x7f596c0af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596affd700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f596c1437e0 0x7f596c0af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596affd700 1 -- 172.21.15.140:0/3882936495 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596000d410 con 0x7f596c0b3670 2022-07-21T01:48:58.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596b7fe700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f596c1437e0 0x7f596c0af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:48:58.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f596affd700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f596c0b3670 0x7f596c0b4db0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f595c0030b0 tx=0x7f595c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f595bfff700 1 -- 172.21.15.140:0/3882936495 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f595c011780 con 0x7f596c0b3670 2022-07-21T01:48:58.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f595bfff700 1 -- 172.21.15.140:0/3882936495 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c0118e0 con 0x7f596c0b3670 2022-07-21T01:48:58.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f59716cf700 1 -- 172.21.15.140:0/3882936495 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c09b9e0 con 0x7f596c0b3670 2022-07-21T01:48:58.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f595bfff700 1 -- 172.21.15.140:0/3882936495 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f595c011c40 con 0x7f596c0b3670 2022-07-21T01:48:58.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f59716cf700 1 -- 172.21.15.140:0/3882936495 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c0afc50 con 0x7f596c0b3670 2022-07-21T01:48:58.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.043+0000 7f595bfff700 1 -- 172.21.15.140:0/3882936495 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f595c012420 con 0x7f596c0b3670 2022-07-21T01:48:58.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.047+0000 7f59716cf700 1 -- 172.21.15.140:0/3882936495 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c09bb70 con 0x7f596c0b3670 2022-07-21T01:48:58.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.047+0000 7f595bfff700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59480539e0 0x7f5948055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.047+0000 7f595bfff700 1 -- 172.21.15.140:0/3882936495 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f595c06be60 con 0x7f596c0b3670 2022-07-21T01:48:58.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.047+0000 7f596a7fc700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59480539e0 0x7f5948055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.047+0000 7f596a7fc700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59480539e0 0x7f5948055ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f596001a000 tx=0x7f596c0aba60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.051+0000 7f595bfff700 1 -- 172.21.15.140:0/3882936495 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f595c042c10 con 0x7f596c0b3670 2022-07-21T01:48:58.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.175+0000 7f59716cf700 1 -- 172.21.15.140:0/3882936495 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f596c0574e0 con 0x7f596c0b3670 2022-07-21T01:48:58.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.175+0000 7f595bfff700 1 -- 172.21.15.140:0/3882936495 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f595c046230 con 0x7f596c0b3670 2022-07-21T01:48:58.175 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:48:58.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 -- 172.21.15.140:0/3882936495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59480539e0 msgr2=0x7f5948055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59480539e0 0x7f5948055ea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f596001a000 tx=0x7f596c0aba60).stop 2022-07-21T01:48:58.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 -- 172.21.15.140:0/3882936495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f596c0b3670 msgr2=0x7f596c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f596c0b3670 0x7f596c0b4db0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f595c0030b0 tx=0x7f595c008040).stop 2022-07-21T01:48:58.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 -- 172.21.15.140:0/3882936495 shutdown_connections 2022-07-21T01:48:58.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59480539e0 0x7f5948055ea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f596c1437e0 0x7f596c0af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f596c13fe10 0x7f596c0af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 --2- 172.21.15.140:0/3882936495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f596c0b3670 0x7f596c0b4db0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 -- 172.21.15.140:0/3882936495 >> 172.21.15.140:0/3882936495 conn(0x7f596c0b6c50 msgr2=0x7f596c13f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:58.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 -- 172.21.15.140:0/3882936495 shutdown_connections 2022-07-21T01:48:58.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.179+0000 7f59716cf700 1 -- 172.21.15.140:0/3882936495 wait complete. 2022-07-21T01:48:58.191 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:48:58.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f98dde700 1 Processor -- start 2022-07-21T01:48:58.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f98dde700 1 -- start start 2022-07-21T01:48:58.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f98dde700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7f940af180 0x7f7f9405d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f98dde700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7f94128d70 0x7f7f9413fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f98dde700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f940b5db0 con 0x7f7f940b59d0 2022-07-21T01:48:58.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f98dde700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f940ad800 con 0x7f7f94128d70 2022-07-21T01:48:58.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f98dde700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f940adaa0 con 0x7f7f940af180 2022-07-21T01:48:58.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f93fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7f94128d70 0x7f7f9413fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f93fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7f94128d70 0x7f7f9413fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33318/0 (socket says 172.21.15.140:33318) 2022-07-21T01:48:58.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f93fff700 1 -- 172.21.15.140:0/1130131858 learned_addr learned my addr 172.21.15.140:0/1130131858 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:58.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f92ffd700 1 --2- 172.21.15.140:0/1130131858 >> v2:172.21.15.152:3300/0 conn(0x7f7f940af180 0x7f7f9405d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f93fff700 1 -- 172.21.15.140:0/1130131858 >> v2:172.21.15.152:3300/0 conn(0x7f7f940af180 msgr2=0x7f7f9405d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:48:58.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f93fff700 1 --2- 172.21.15.140:0/1130131858 >> v2:172.21.15.152:3300/0 conn(0x7f7f940af180 0x7f7f9405d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f93fff700 1 -- 172.21.15.140:0/1130131858 >> v1:172.21.15.152:6789/0 conn(0x7f7f940b59d0 legacy=0x7f7f94128660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f93fff700 1 -- 172.21.15.140:0/1130131858 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f940ad4f0 con 0x7f7f94128d70 2022-07-21T01:48:58.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f93fff700 1 --2- 172.21.15.140:0/1130131858 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f94128d70 0x7f7f9413fc00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f7f8c0032f0 tx=0x7f7f8c00b530).ready entity=mon.0 client_cookie=6c3604536bd5b1e1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f927fc700 1 -- 172.21.15.140:0/1130131858 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8c00ddf0 con 0x7f7f94128d70 2022-07-21T01:48:58.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f927fc700 1 -- 172.21.15.140:0/1130131858 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c010470 con 0x7f7f94128d70 2022-07-21T01:48:58.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.307+0000 7f7f927fc700 1 -- 172.21.15.140:0/1130131858 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8c0107d0 con 0x7f7f94128d70 2022-07-21T01:48:58.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.311+0000 7f7f98dde700 1 -- 172.21.15.140:0/1130131858 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f94128d70 msgr2=0x7f7f9413fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.311+0000 7f7f98dde700 1 --2- 172.21.15.140:0/1130131858 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f94128d70 0x7f7f9413fc00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f7f8c0032f0 tx=0x7f7f8c00b530).stop 2022-07-21T01:48:58.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.311+0000 7f7f98dde700 1 -- 172.21.15.140:0/1130131858 shutdown_connections 2022-07-21T01:48:58.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.311+0000 7f7f98dde700 1 --2- 172.21.15.140:0/1130131858 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f94128d70 0x7f7f9413fc00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.311+0000 7f7f98dde700 1 --2- 172.21.15.140:0/1130131858 >> v2:172.21.15.152:3300/0 conn(0x7f7f940af180 0x7f7f9405d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.311+0000 7f7f98dde700 1 -- 172.21.15.140:0/1130131858 >> 172.21.15.140:0/1130131858 conn(0x7f7f940634c0 msgr2=0x7f7f940b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:58.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.311+0000 7f7f98dde700 1 -- 172.21.15.140:0/1130131858 shutdown_connections 2022-07-21T01:48:58.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.311+0000 7f7f98dde700 1 -- 172.21.15.140:0/1130131858 wait complete. 2022-07-21T01:48:58.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.311+0000 7f7f98dde700 1 Processor -- start 2022-07-21T01:48:58.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.311+0000 7f7f98dde700 1 -- start start 2022-07-21T01:48:58.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f98dde700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f940af180 0x7f7f94121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f98dde700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f940b59d0 0x7f7f94141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f98dde700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f94128d70 0x7f7f94141660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f98dde700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9413f730 con 0x7f7f94128d70 2022-07-21T01:48:58.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f98dde700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f94063cb0 con 0x7f7f940af180 2022-07-21T01:48:58.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f98dde700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f94140d40 con 0x7f7f940b59d0 2022-07-21T01:48:58.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f94128d70 0x7f7f94141660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f93fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f94128d70 0x7f7f94141660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33324/0 (socket says 172.21.15.140:33324) 2022-07-21T01:48:58.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f93fff700 1 -- 172.21.15.140:0/757807700 learned_addr learned my addr 172.21.15.140:0/757807700 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:58.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f92ffd700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f940b59d0 0x7f7f94141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f937fe700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f940af180 0x7f7f94121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f93fff700 1 -- 172.21.15.140:0/757807700 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f940b59d0 msgr2=0x7f7f94141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f93fff700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f940b59d0 0x7f7f94141170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f93fff700 1 -- 172.21.15.140:0/757807700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f940af180 msgr2=0x7f7f94121ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f93fff700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f940af180 0x7f7f94121ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f93fff700 1 -- 172.21.15.140:0/757807700 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c00b7d0 con 0x7f7f94128d70 2022-07-21T01:48:58.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f92ffd700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f940b59d0 0x7f7f94141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:58.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f937fe700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f940af180 0x7f7f94121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:48:58.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f93fff700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f94128d70 0x7f7f94141660 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f7f8c0032f0 tx=0x7f7f940ae150).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f90ff9700 1 -- 172.21.15.140:0/757807700 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8c01d070 con 0x7f7f94128d70 2022-07-21T01:48:58.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f90ff9700 1 -- 172.21.15.140:0/757807700 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c018800 con 0x7f7f94128d70 2022-07-21T01:48:58.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f98dde700 1 -- 172.21.15.140:0/757807700 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f941269c0 con 0x7f7f94128d70 2022-07-21T01:48:58.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f90ff9700 1 -- 172.21.15.140:0/757807700 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f8c018b60 con 0x7f7f94128d70 2022-07-21T01:48:58.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f98dde700 1 -- 172.21.15.140:0/757807700 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f94143b30 con 0x7f7f94128d70 2022-07-21T01:48:58.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.315+0000 7f7f98dde700 1 -- 172.21.15.140:0/757807700 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f940574e0 con 0x7f7f94128d70 2022-07-21T01:48:58.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.319+0000 7f7f90ff9700 1 -- 172.21.15.140:0/757807700 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7f8c018cc0 con 0x7f7f94128d70 2022-07-21T01:48:58.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.319+0000 7f7f90ff9700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f74053b00 0x7f7f74055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.319+0000 7f7f90ff9700 1 -- 172.21.15.140:0/757807700 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7f8c06d510 con 0x7f7f94128d70 2022-07-21T01:48:58.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.319+0000 7f7f937fe700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f74053b00 0x7f7f74055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.319+0000 7f7f937fe700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f74053b00 0x7f7f74055fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f7f8800d000 tx=0x7f7f88009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.323+0000 7f7f90ff9700 1 -- 172.21.15.140:0/757807700 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7f8c0444e0 con 0x7f7f94128d70 2022-07-21T01:48:58.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.447+0000 7f7f98dde700 1 -- 172.21.15.140:0/757807700 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7f9405db10 con 0x7f7f94128d70 2022-07-21T01:48:58.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f90ff9700 1 -- 172.21.15.140:0/757807700 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7f8c047b00 con 0x7f7f94128d70 2022-07-21T01:48:58.450 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:58.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 -- 172.21.15.140:0/757807700 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f74053b00 msgr2=0x7f7f74055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f74053b00 0x7f7f74055fc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f7f8800d000 tx=0x7f7f88009040).stop 2022-07-21T01:48:58.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 -- 172.21.15.140:0/757807700 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f94128d70 msgr2=0x7f7f94141660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f94128d70 0x7f7f94141660 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f7f8c0032f0 tx=0x7f7f940ae150).stop 2022-07-21T01:48:58.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 -- 172.21.15.140:0/757807700 shutdown_connections 2022-07-21T01:48:58.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f74053b00 0x7f7f74055fc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f94128d70 0x7f7f94141660 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f940b59d0 0x7f7f94141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 --2- 172.21.15.140:0/757807700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f940af180 0x7f7f94121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 -- 172.21.15.140:0/757807700 >> 172.21.15.140:0/757807700 conn(0x7f7f940634c0 msgr2=0x7f7f940b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:58.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 -- 172.21.15.140:0/757807700 shutdown_connections 2022-07-21T01:48:58.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.451+0000 7f7f98dde700 1 -- 172.21.15.140:0/757807700 wait complete. 2022-07-21T01:48:58.465 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:48:58.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.583+0000 7fe00f59e700 1 Processor -- start 2022-07-21T01:48:58.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.583+0000 7fe00f59e700 1 -- start start 2022-07-21T01:48:58.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00f59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe0080d9590 0x7fe0080db980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00f59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe008040da0 con 0x7fe008047ea0 2022-07-21T01:48:58.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00f59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe008048280 con 0x7fe00803e8a0 2022-07-21T01:48:58.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00f59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00803f210 con 0x7fe0080d9590 2022-07-21T01:48:58.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00ed9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe0080d9590 0x7fe0080db980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00e59c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe008047ea0 0x7fe008042bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47474/0 (socket says 172.21.15.140:47474) 2022-07-21T01:48:58.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00e59c700 1 -- 172.21.15.140:0/864632175 learned_addr learned my addr 172.21.15.140:0/864632175 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:58.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00ed9d700 1 -- 172.21.15.140:0/864632175 >> v1:172.21.15.152:6789/0 conn(0x7fe00803e8a0 legacy=0x7fe0080432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00ed9d700 1 -- 172.21.15.140:0/864632175 >> v1:172.21.15.140:6789/0 conn(0x7fe008047ea0 legacy=0x7fe008042bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00ed9d700 1 -- 172.21.15.140:0/864632175 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008041080 con 0x7fe0080d9590 2022-07-21T01:48:58.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00ed9d700 1 --2- 172.21.15.140:0/864632175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe0080d9590 0x7fe0080db980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fe010044d80 tx=0x7fe01005f750).ready entity=mon.0 client_cookie=a38605c759b53572 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00d59a700 1 -- 172.21.15.140:0/864632175 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe010084030 con 0x7fe0080d9590 2022-07-21T01:48:58.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00d59a700 1 -- 172.21.15.140:0/864632175 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01005af90 con 0x7fe0080d9590 2022-07-21T01:48:58.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00d59a700 1 -- 172.21.15.140:0/864632175 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe01006a910 con 0x7fe0080d9590 2022-07-21T01:48:58.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00f59e700 1 -- 172.21.15.140:0/864632175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe0080d9590 msgr2=0x7fe0080db980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00f59e700 1 --2- 172.21.15.140:0/864632175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe0080d9590 0x7fe0080db980 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fe010044d80 tx=0x7fe01005f750).stop 2022-07-21T01:48:58.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00f59e700 1 -- 172.21.15.140:0/864632175 shutdown_connections 2022-07-21T01:48:58.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00f59e700 1 --2- 172.21.15.140:0/864632175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe0080d9590 0x7fe0080db980 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00f59e700 1 -- 172.21.15.140:0/864632175 >> 172.21.15.140:0/864632175 conn(0x7fe00800e2a0 msgr2=0x7fe008047450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:58.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00f59e700 1 -- 172.21.15.140:0/864632175 shutdown_connections 2022-07-21T01:48:58.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.587+0000 7fe00f59e700 1 -- 172.21.15.140:0/864632175 wait complete. 2022-07-21T01:48:58.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00f59e700 1 Processor -- start 2022-07-21T01:48:58.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00f59e700 1 -- start start 2022-07-21T01:48:58.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00f59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe00803e8a0 0x7fe0080e4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00f59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe008047ea0 0x7fe0080c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe0080d9590 0x7fe0080e0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00f59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080c8f50 con 0x7fe00803e8a0 2022-07-21T01:48:58.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00f59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080e4630 con 0x7fe008047ea0 2022-07-21T01:48:58.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080d2200 con 0x7fe0080d9590 2022-07-21T01:48:58.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe00803e8a0 0x7fe0080e4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe00803e8a0 0x7fe0080e4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33334/0 (socket says 172.21.15.140:33334) 2022-07-21T01:48:58.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00e59c700 1 -- 172.21.15.140:0/3851906892 learned_addr learned my addr 172.21.15.140:0/3851906892 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:58.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00ed9d700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe0080d9590 0x7fe0080e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00dd9b700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe008047ea0 0x7fe0080c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00e59c700 1 -- 172.21.15.140:0/3851906892 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe0080d9590 msgr2=0x7fe0080e0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00e59c700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe0080d9590 0x7fe0080e0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00e59c700 1 -- 172.21.15.140:0/3851906892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe008047ea0 msgr2=0x7fe0080c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00e59c700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe008047ea0 0x7fe0080c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00e59c700 1 -- 172.21.15.140:0/3851906892 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0080e0710 con 0x7fe00803e8a0 2022-07-21T01:48:58.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00e59c700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe00803e8a0 0x7fe0080e4080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fe0080484b0 tx=0x7fe000007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fdfff7fe700 1 -- 172.21.15.140:0/3851906892 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0000070d0 con 0x7fe00803e8a0 2022-07-21T01:48:58.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fdfff7fe700 1 -- 172.21.15.140:0/3851906892 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe000007230 con 0x7fe00803e8a0 2022-07-21T01:48:58.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00f59e700 1 -- 172.21.15.140:0/3851906892 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0080cabe0 con 0x7fe00803e8a0 2022-07-21T01:48:58.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fdfff7fe700 1 -- 172.21.15.140:0/3851906892 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe000005610 con 0x7fe00803e8a0 2022-07-21T01:48:58.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.591+0000 7fe00f59e700 1 -- 172.21.15.140:0/3851906892 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0080d2da0 con 0x7fe00803e8a0 2022-07-21T01:48:58.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.595+0000 7fe00f59e700 1 -- 172.21.15.140:0/3851906892 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe008041320 con 0x7fe00803e8a0 2022-07-21T01:48:58.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.595+0000 7fdfff7fe700 1 -- 172.21.15.140:0/3851906892 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe000010070 con 0x7fe00803e8a0 2022-07-21T01:48:58.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.595+0000 7fdfff7fe700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdfec053b00 0x7fdfec055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.595+0000 7fdfff7fe700 1 -- 172.21.15.140:0/3851906892 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe00006b400 con 0x7fe00803e8a0 2022-07-21T01:48:58.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.595+0000 7fe00dd9b700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdfec053b00 0x7fdfec055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.595+0000 7fe00dd9b700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdfec053b00 0x7fdfec055fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fe00400d000 tx=0x7fe004009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.603+0000 7fdfff7fe700 1 -- 172.21.15.140:0/3851906892 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe000042320 con 0x7fe00803e8a0 2022-07-21T01:48:58.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.727+0000 7fe00f59e700 1 -- 172.21.15.140:0/3851906892 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe008005240 con 0x7fe00803e8a0 2022-07-21T01:48:58.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.727+0000 7fdfff7fe700 1 -- 172.21.15.140:0/3851906892 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe000045940 con 0x7fe00803e8a0 2022-07-21T01:48:58.728 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:58.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.727+0000 7fe00f59e700 1 -- 172.21.15.140:0/3851906892 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdfec053b00 msgr2=0x7fdfec055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.727+0000 7fe00f59e700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdfec053b00 0x7fdfec055fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fe00400d000 tx=0x7fe004009040).stop 2022-07-21T01:48:58.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.727+0000 7fe00f59e700 1 -- 172.21.15.140:0/3851906892 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe00803e8a0 msgr2=0x7fe0080e4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.727+0000 7fe00f59e700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe00803e8a0 0x7fe0080e4080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fe0080484b0 tx=0x7fe000007c40).stop 2022-07-21T01:48:58.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.727+0000 7fe00f59e700 1 -- 172.21.15.140:0/3851906892 shutdown_connections 2022-07-21T01:48:58.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.731+0000 7fe00f59e700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdfec053b00 0x7fdfec055fc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.731+0000 7fe00f59e700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe0080d9590 0x7fe0080e0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.731+0000 7fe00f59e700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe008047ea0 0x7fe0080c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.731+0000 7fe00f59e700 1 --2- 172.21.15.140:0/3851906892 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe00803e8a0 0x7fe0080e4080 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.731+0000 7fe00f59e700 1 -- 172.21.15.140:0/3851906892 >> 172.21.15.140:0/3851906892 conn(0x7fe00800e2a0 msgr2=0x7fe0080d0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:58.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.731+0000 7fe00f59e700 1 -- 172.21.15.140:0/3851906892 shutdown_connections 2022-07-21T01:48:58.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.731+0000 7fe00f59e700 1 -- 172.21.15.140:0/3851906892 wait complete. 2022-07-21T01:48:58.742 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:48:58.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.855+0000 7fa3ed64b700 1 Processor -- start 2022-07-21T01:48:58.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.855+0000 7fa3ed64b700 1 -- start start 2022-07-21T01:48:58.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.855+0000 7fa3ed64b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa3e8061fd0 0x7fa3e81465e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.855+0000 7fa3ed64b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e805cd40 con 0x7fa3e805c960 2022-07-21T01:48:58.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.855+0000 7fa3ed64b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e8060400 con 0x7fa3e80b2230 2022-07-21T01:48:58.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.855+0000 7fa3ed64b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e8061440 con 0x7fa3e8061fd0 2022-07-21T01:48:58.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.855+0000 7fa3ece4a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa3e8061fd0 0x7fa3e81465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.855+0000 7fa3ece4a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa3e8061fd0 0x7fa3e81465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33346/0 (socket says 172.21.15.140:33346) 2022-07-21T01:48:58.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.855+0000 7fa3ece4a700 1 -- 172.21.15.140:0/505644510 learned_addr learned my addr 172.21.15.140:0/505644510 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:58.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ece4a700 1 -- 172.21.15.140:0/505644510 >> v1:172.21.15.152:6789/0 conn(0x7fa3e80b2230 legacy=0x7fa3e8142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ece4a700 1 -- 172.21.15.140:0/505644510 >> v1:172.21.15.140:6789/0 conn(0x7fa3e805c960 legacy=0x7fa3e80618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3e6ffd700 1 -- 172.21.15.140:0/505644510 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57671598 0 0) 0x7fa3e805cd40 con 0x7fa3e805c960 2022-07-21T01:48:58.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ece4a700 1 -- 172.21.15.140:0/505644510 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e8060680 con 0x7fa3e8061fd0 2022-07-21T01:48:58.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ece4a700 1 --2- 172.21.15.140:0/505644510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3e8061fd0 0x7fa3e81465e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fa3e00032f0 tx=0x7fa3e0009290).ready entity=mon.0 client_cookie=1f64ba2e1e63fecc server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3e6ffd700 1 -- 172.21.15.140:0/505644510 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3e000b8a0 con 0x7fa3e8061fd0 2022-07-21T01:48:58.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3e6ffd700 1 -- 172.21.15.140:0/505644510 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e000ba00 con 0x7fa3e8061fd0 2022-07-21T01:48:58.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3e6ffd700 1 -- 172.21.15.140:0/505644510 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3e000bb70 con 0x7fa3e8061fd0 2022-07-21T01:48:58.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ed64b700 1 -- 172.21.15.140:0/505644510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3e8061fd0 msgr2=0x7fa3e81465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ed64b700 1 --2- 172.21.15.140:0/505644510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3e8061fd0 0x7fa3e81465e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fa3e00032f0 tx=0x7fa3e0009290).stop 2022-07-21T01:48:58.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ed64b700 1 -- 172.21.15.140:0/505644510 shutdown_connections 2022-07-21T01:48:58.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ed64b700 1 --2- 172.21.15.140:0/505644510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3e8061fd0 0x7fa3e81465e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ed64b700 1 -- 172.21.15.140:0/505644510 >> 172.21.15.140:0/505644510 conn(0x7fa3e80b61f0 msgr2=0x7fa3e80b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:58.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ed64b700 1 -- 172.21.15.140:0/505644510 shutdown_connections 2022-07-21T01:48:58.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ed64b700 1 -- 172.21.15.140:0/505644510 wait complete. 2022-07-21T01:48:58.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ed64b700 1 Processor -- start 2022-07-21T01:48:58.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.859+0000 7fa3ed64b700 1 -- start start 2022-07-21T01:48:58.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3ed64b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3e805c960 0x7fa3e809ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3ed64b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3e8061fd0 0x7fa3e8142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3ed64b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e80b2230 0x7fa3e8147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3ed64b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80ae070 con 0x7fa3e805c960 2022-07-21T01:48:58.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3ed64b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80adbf0 con 0x7fa3e80b2230 2022-07-21T01:48:58.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3ed64b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e8149d30 con 0x7fa3e8061fd0 2022-07-21T01:48:58.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3e805c960 0x7fa3e809ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3e805c960 0x7fa3e809ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33348/0 (socket says 172.21.15.140:33348) 2022-07-21T01:48:58.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e7fff700 1 -- 172.21.15.140:0/727237789 learned_addr learned my addr 172.21.15.140:0/727237789 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:58.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3ece4a700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e80b2230 0x7fa3e8147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e77fe700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3e8061fd0 0x7fa3e8142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e7fff700 1 -- 172.21.15.140:0/727237789 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3e8061fd0 msgr2=0x7fa3e8142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e7fff700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3e8061fd0 0x7fa3e8142080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e7fff700 1 -- 172.21.15.140:0/727237789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e80b2230 msgr2=0x7fa3e8147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:58.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e7fff700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e80b2230 0x7fa3e8147c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:58.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e7fff700 1 -- 172.21.15.140:0/727237789 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e0009530 con 0x7fa3e805c960 2022-07-21T01:48:58.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3ece4a700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e80b2230 0x7fa3e8147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:58.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e77fe700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3e8061fd0 0x7fa3e8142080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:48:58.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e7fff700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3e805c960 0x7fa3e809ab20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fa3e809be70 tx=0x7fa3d0007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e57fa700 1 -- 172.21.15.140:0/727237789 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d000ce10 con 0x7fa3e805c960 2022-07-21T01:48:58.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e57fa700 1 -- 172.21.15.140:0/727237789 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d000f470 con 0x7fa3e805c960 2022-07-21T01:48:58.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3ed64b700 1 -- 172.21.15.140:0/727237789 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e80ad950 con 0x7fa3e805c960 2022-07-21T01:48:58.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e57fa700 1 -- 172.21.15.140:0/727237789 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d000f7d0 con 0x7fa3e805c960 2022-07-21T01:48:58.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3ed64b700 1 -- 172.21.15.140:0/727237789 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e8148c70 con 0x7fa3e805c960 2022-07-21T01:48:58.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3ed64b700 1 -- 172.21.15.140:0/727237789 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e81479b0 con 0x7fa3e805c960 2022-07-21T01:48:58.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.863+0000 7fa3e57fa700 1 -- 172.21.15.140:0/727237789 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa3d0019030 con 0x7fa3e805c960 2022-07-21T01:48:58.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.867+0000 7fa3e57fa700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa3cc053980 0x7fa3cc055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:58.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.867+0000 7fa3e57fa700 1 -- 172.21.15.140:0/727237789 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa3d006b1f0 con 0x7fa3e805c960 2022-07-21T01:48:58.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.867+0000 7fa3e77fe700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa3cc053980 0x7fa3cc055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:58.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.867+0000 7fa3e77fe700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa3cc053980 0x7fa3cc055e40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fa3d8008000 tx=0x7fa3d8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:58.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.871+0000 7fa3e57fa700 1 -- 172.21.15.140:0/727237789 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa3d0042160 con 0x7fa3e805c960 2022-07-21T01:48:58.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 -- 172.21.15.140:0/727237789 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3e80b2840 con 0x7fa3e805c960 2022-07-21T01:48:58.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3e57fa700 1 -- 172.21.15.140:0/727237789 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa3d0045780 con 0x7fa3e805c960 2022-07-21T01:48:58.997 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:48:59.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 -- 172.21.15.140:0/727237789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa3cc053980 msgr2=0x7fa3cc055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:59.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa3cc053980 0x7fa3cc055e40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fa3d8008000 tx=0x7fa3d8009040).stop 2022-07-21T01:48:59.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 -- 172.21.15.140:0/727237789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3e805c960 msgr2=0x7fa3e809ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:59.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3e805c960 0x7fa3e809ab20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fa3e809be70 tx=0x7fa3d0007ac0).stop 2022-07-21T01:48:59.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 -- 172.21.15.140:0/727237789 shutdown_connections 2022-07-21T01:48:59.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa3cc053980 0x7fa3cc055e40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:59.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3e80b2230 0x7fa3e8147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:59.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3e8061fd0 0x7fa3e8142080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:59.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 --2- 172.21.15.140:0/727237789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3e805c960 0x7fa3e809ab20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:59.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 -- 172.21.15.140:0/727237789 >> 172.21.15.140:0/727237789 conn(0x7fa3e80b61f0 msgr2=0x7fa3e80b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:59.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 -- 172.21.15.140:0/727237789 shutdown_connections 2022-07-21T01:48:59.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:58.995+0000 7fa3ed64b700 1 -- 172.21.15.140:0/727237789 wait complete. 2022-07-21T01:48:59.019 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) 2022-07-21T01:48:59.021 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2022-07-21T01:48:59.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.131+0000 7f15ff59e700 1 Processor -- start 2022-07-21T01:48:59.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.131+0000 7f15ff59e700 1 -- start start 2022-07-21T01:48:59.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16000605a0 0x7f160013f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:59.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1600140940 con 0x7f16000b1ef0 2022-07-21T01:48:59.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16000b22d0 con 0x7f16000ab180 2022-07-21T01:48:59.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160005fbc0 con 0x7f16000605a0 2022-07-21T01:48:59.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fe59c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f16000b1ef0 0x7f16000abff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47504/0 (socket says 172.21.15.140:47504) 2022-07-21T01:48:59.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fe59c700 1 -- 172.21.15.140:0/3262336972 learned_addr learned my addr 172.21.15.140:0/3262336972 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:48:59.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fdd9b700 1 --2- 172.21.15.140:0/3262336972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16000605a0 0x7f160013f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:59.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fd59a700 1 -- 172.21.15.140:0/3262336972 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362609832 0 0) 0x7f1600140940 con 0x7f16000b1ef0 2022-07-21T01:48:59.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fd59a700 1 -- 172.21.15.140:0/3262336972 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15e4003260 con 0x7f16000b1ef0 2022-07-21T01:48:59.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fd59a700 1 -- 172.21.15.140:0/3262336972 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288329071 0 0) 0x7f16000b22d0 con 0x7f16000ab180 2022-07-21T01:48:59.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fd59a700 1 -- 172.21.15.140:0/3262336972 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1600140940 con 0x7f16000ab180 2022-07-21T01:48:59.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fdd9b700 1 -- 172.21.15.140:0/3262336972 >> v1:172.21.15.152:6789/0 conn(0x7f16000ab180 legacy=0x7f16000ab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:59.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fdd9b700 1 -- 172.21.15.140:0/3262336972 >> v1:172.21.15.140:6789/0 conn(0x7f16000b1ef0 legacy=0x7f16000abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:59.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fdd9b700 1 -- 172.21.15.140:0/3262336972 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160005fe60 con 0x7f16000605a0 2022-07-21T01:48:59.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fdd9b700 1 --2- 172.21.15.140:0/3262336972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16000605a0 0x7f160013f980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f15f80032f0 tx=0x7f15f80091d0).ready entity=mon.2 client_cookie=720d8daa676af005 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:59.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fd59a700 1 -- 172.21.15.140:0/3262336972 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15f800b640 con 0x7f16000605a0 2022-07-21T01:48:59.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fd59a700 1 -- 172.21.15.140:0/3262336972 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f800b7a0 con 0x7f16000605a0 2022-07-21T01:48:59.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15fd59a700 1 -- 172.21.15.140:0/3262336972 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15f800b910 con 0x7f16000605a0 2022-07-21T01:48:59.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 -- 172.21.15.140:0/3262336972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16000605a0 msgr2=0x7f160013f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:59.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 --2- 172.21.15.140:0/3262336972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16000605a0 0x7f160013f980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f15f80032f0 tx=0x7f15f80091d0).stop 2022-07-21T01:48:59.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 -- 172.21.15.140:0/3262336972 shutdown_connections 2022-07-21T01:48:59.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 --2- 172.21.15.140:0/3262336972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16000605a0 0x7f160013f980 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:59.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 -- 172.21.15.140:0/3262336972 >> 172.21.15.140:0/3262336972 conn(0x7f16000b2ca0 msgr2=0x7f16000b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:59.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 -- 172.21.15.140:0/3262336972 shutdown_connections 2022-07-21T01:48:59.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 -- 172.21.15.140:0/3262336972 wait complete. 2022-07-21T01:48:59.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 Processor -- start 2022-07-21T01:48:59.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.135+0000 7f15ff59e700 1 -- start start 2022-07-21T01:48:59.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ff59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16000605a0 0x7f1600136bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:59.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ff59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16000ab180 0x7f160009bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:59.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ff59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16000b1ef0 0x7f1600155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:59.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ff59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160009a2e0 con 0x7f16000ab180 2022-07-21T01:48:59.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ff59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16000ae8a0 con 0x7f16000b1ef0 2022-07-21T01:48:59.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ff59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16000afc90 con 0x7f16000605a0 2022-07-21T01:48:59.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15fdd9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16000ab180 0x7f160009bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:59.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15fdd9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16000ab180 0x7f160009bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33364/0 (socket says 172.21.15.140:33364) 2022-07-21T01:48:59.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15fdd9b700 1 -- 172.21.15.140:0/3323428177 learned_addr learned my addr 172.21.15.140:0/3323428177 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:48:59.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15fed9d700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16000b1ef0 0x7f1600155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:59.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15fe59c700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16000605a0 0x7f1600136bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:59.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15fdd9b700 1 -- 172.21.15.140:0/3323428177 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16000605a0 msgr2=0x7f1600136bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:59.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15fdd9b700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16000605a0 0x7f1600136bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:59.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15fdd9b700 1 -- 172.21.15.140:0/3323428177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16000b1ef0 msgr2=0x7f1600155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:59.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15fdd9b700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16000b1ef0 0x7f1600155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:59.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15fdd9b700 1 -- 172.21.15.140:0/3323428177 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f8009470 con 0x7f16000ab180 2022-07-21T01:48:59.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15fdd9b700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16000ab180 0x7f160009bc60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f15f80032f0 tx=0x7f15f8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:59.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ef7fe700 1 -- 172.21.15.140:0/3323428177 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15f800b910 con 0x7f16000ab180 2022-07-21T01:48:59.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ef7fe700 1 -- 172.21.15.140:0/3323428177 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f8011e00 con 0x7f16000ab180 2022-07-21T01:48:59.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ef7fe700 1 -- 172.21.15.140:0/3323428177 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15f800b640 con 0x7f16000ab180 2022-07-21T01:48:59.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ff59e700 1 -- 172.21.15.140:0/3323428177 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16000af780 con 0x7f16000ab180 2022-07-21T01:48:59.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ff59e700 1 -- 172.21.15.140:0/3323428177 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16000af380 con 0x7f16000ab180 2022-07-21T01:48:59.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ef7fe700 1 -- 172.21.15.140:0/3323428177 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f15f800f070 con 0x7f16000ab180 2022-07-21T01:48:59.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.139+0000 7f15ff59e700 1 -- 172.21.15.140:0/3323428177 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1600155f60 con 0x7f16000ab180 2022-07-21T01:48:59.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.143+0000 7f15ef7fe700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15dc053980 0x7f15dc055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:48:59.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.143+0000 7f15ef7fe700 1 -- 172.21.15.140:0/3323428177 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f15f806c520 con 0x7f16000ab180 2022-07-21T01:48:59.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.143+0000 7f15fe59c700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15dc053980 0x7f15dc055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:48:59.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.143+0000 7f15fe59c700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15dc053980 0x7f15dc055e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f15f400d000 tx=0x7f15f4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:48:59.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.147+0000 7f15ef7fe700 1 -- 172.21.15.140:0/3323428177 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f15f80434f0 con 0x7f16000ab180 2022-07-21T01:48:59.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.315+0000 7f15ff59e700 1 -- 172.21.15.140:0/3323428177 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f160009a120 con 0x7f16000ab180 2022-07-21T01:48:59.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.987+0000 7f15ef7fe700 1 -- 172.21.15.140:0/3323428177 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v42) v1 ==== 142+0+0 (secure 0 0 0) 0x7f15f80240d0 con 0x7f16000ab180 2022-07-21T01:48:59.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.991+0000 7f15ff59e700 1 -- 172.21.15.140:0/3323428177 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15dc053980 msgr2=0x7f15dc055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:59.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.991+0000 7f15ff59e700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15dc053980 0x7f15dc055e40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f15f400d000 tx=0x7f15f4009040).stop 2022-07-21T01:48:59.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.991+0000 7f15ff59e700 1 -- 172.21.15.140:0/3323428177 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16000ab180 msgr2=0x7f160009bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:48:59.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.991+0000 7f15ff59e700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16000ab180 0x7f160009bc60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f15f80032f0 tx=0x7f15f8016000).stop 2022-07-21T01:48:59.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.991+0000 7f15ff59e700 1 -- 172.21.15.140:0/3323428177 shutdown_connections 2022-07-21T01:48:59.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.991+0000 7f15ff59e700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15dc053980 0x7f15dc055e40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:59.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.991+0000 7f15ff59e700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16000b1ef0 0x7f1600155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:59.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.991+0000 7f15ff59e700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16000ab180 0x7f160009bc60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:59.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.991+0000 7f15ff59e700 1 --2- 172.21.15.140:0/3323428177 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16000605a0 0x7f1600136bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:48:59.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.991+0000 7f15ff59e700 1 -- 172.21.15.140:0/3323428177 >> 172.21.15.140:0/3323428177 conn(0x7f16000b2ca0 msgr2=0x7f16000b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:48:59.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.995+0000 7f15ff59e700 1 -- 172.21.15.140:0/3323428177 shutdown_connections 2022-07-21T01:48:59.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:48:59.995+0000 7f15ff59e700 1 -- 172.21.15.140:0/3323428177 wait complete. 2022-07-21T01:49:00.010 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:49:00.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.131+0000 7fafb82bf700 1 Processor -- start 2022-07-21T01:49:00.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.131+0000 7fafb82bf700 1 -- start start 2022-07-21T01:49:00.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fafb01462a0 0x7fafb0146680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:00.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fafb00988b0 0x7fafb0146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:00.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fafb01422c0 0x7fafb0142730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:00.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb00afdb0 con 0x7fafb01422c0 2022-07-21T01:49:00.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb00b0050 con 0x7fafb01462a0 2022-07-21T01:49:00.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb005fcd0 con 0x7fafb00988b0 2022-07-21T01:49:00.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb685c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fafb01422c0 0x7fafb0142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:00.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb685c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fafb01422c0 0x7fafb0142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33374/0 (socket says 172.21.15.140:33374) 2022-07-21T01:49:00.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb685c700 1 -- 172.21.15.140:0/3648039471 learned_addr learned my addr 172.21.15.140:0/3648039471 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:00.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb605b700 1 --2- 172.21.15.140:0/3648039471 >> v2:172.21.15.152:3300/0 conn(0x7fafb01462a0 0x7fafb0146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:00.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb585a700 1 --2- 172.21.15.140:0/3648039471 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fafb00988b0 0x7fafb0146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:00.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb685c700 1 -- 172.21.15.140:0/3648039471 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fafb00988b0 msgr2=0x7fafb0146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:00.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb685c700 1 --2- 172.21.15.140:0/3648039471 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fafb00988b0 0x7fafb0146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:00.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb685c700 1 -- 172.21.15.140:0/3648039471 >> v2:172.21.15.152:3300/0 conn(0x7fafb01462a0 msgr2=0x7fafb0146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:00.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb685c700 1 --2- 172.21.15.140:0/3648039471 >> v2:172.21.15.152:3300/0 conn(0x7fafb01462a0 0x7fafb0146680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:00.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb685c700 1 -- 172.21.15.140:0/3648039471 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb00afae0 con 0x7fafb01422c0 2022-07-21T01:49:00.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb685c700 1 --2- 172.21.15.140:0/3648039471 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fafb01422c0 0x7fafb0142730 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fafa80032f0 tx=0x7fafa8009290).ready entity=mon.0 client_cookie=d8db79c37ec3abf1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:00.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb5059700 1 -- 172.21.15.140:0/3648039471 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafa800b680 con 0x7fafb01422c0 2022-07-21T01:49:00.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb5059700 1 -- 172.21.15.140:0/3648039471 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa800b7e0 con 0x7fafb01422c0 2022-07-21T01:49:00.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb5059700 1 -- 172.21.15.140:0/3648039471 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafa800b950 con 0x7fafb01422c0 2022-07-21T01:49:00.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 -- 172.21.15.140:0/3648039471 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fafb01422c0 msgr2=0x7fafb0142730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:00.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 --2- 172.21.15.140:0/3648039471 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fafb01422c0 0x7fafb0142730 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fafa80032f0 tx=0x7fafa8009290).stop 2022-07-21T01:49:00.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 -- 172.21.15.140:0/3648039471 shutdown_connections 2022-07-21T01:49:00.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 --2- 172.21.15.140:0/3648039471 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fafb01422c0 0x7fafb0142730 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:00.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 --2- 172.21.15.140:0/3648039471 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fafb00988b0 0x7fafb0146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:00.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 --2- 172.21.15.140:0/3648039471 >> v2:172.21.15.152:3300/0 conn(0x7fafb01462a0 0x7fafb0146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:00.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 -- 172.21.15.140:0/3648039471 >> 172.21.15.140:0/3648039471 conn(0x7fafb00b5f00 msgr2=0x7fafb00b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:00.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 -- 172.21.15.140:0/3648039471 shutdown_connections 2022-07-21T01:49:00.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.135+0000 7fafb82bf700 1 -- 172.21.15.140:0/3648039471 wait complete. 2022-07-21T01:49:00.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb82bf700 1 Processor -- start 2022-07-21T01:49:00.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb82bf700 1 -- start start 2022-07-21T01:49:00.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb82bf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb00988b0 0x7fafb00990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:00.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb82bf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fafb01422c0 0x7fafb009c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:00.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb82bf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fafb01462a0 0x7fafb009cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:00.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb82bf700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00b19c0 con 0x7fafb01422c0 2022-07-21T01:49:00.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb82bf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00b1c60 con 0x7fafb00988b0 2022-07-21T01:49:00.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb82bf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00b1f00 con 0x7fafb01462a0 2022-07-21T01:49:00.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb585a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fafb01422c0 0x7fafb009c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:00.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb585a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fafb01422c0 0x7fafb009c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33378/0 (socket says 172.21.15.140:33378) 2022-07-21T01:49:00.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb585a700 1 -- 172.21.15.140:0/711937314 learned_addr learned my addr 172.21.15.140:0/711937314 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:00.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb605b700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb00988b0 0x7fafb00990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:00.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb685c700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fafb01462a0 0x7fafb009cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:00.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb605b700 1 -- 172.21.15.140:0/711937314 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fafb01462a0 msgr2=0x7fafb009cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:00.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb605b700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fafb01462a0 0x7fafb009cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:00.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb605b700 1 -- 172.21.15.140:0/711937314 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fafb01422c0 msgr2=0x7fafb009c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:00.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb605b700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fafb01422c0 0x7fafb009c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:00.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb605b700 1 -- 172.21.15.140:0/711937314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa8009530 con 0x7fafb00988b0 2022-07-21T01:49:00.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb585a700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fafb01422c0 0x7fafb009c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:00.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb605b700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb00988b0 0x7fafb00990e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fafa400c000 tx=0x7fafa4006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:00.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafa37fe700 1 -- 172.21.15.140:0/711937314 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafa4011750 con 0x7fafb00988b0 2022-07-21T01:49:00.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafa37fe700 1 -- 172.21.15.140:0/711937314 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa40118b0 con 0x7fafb00988b0 2022-07-21T01:49:00.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb82bf700 1 -- 172.21.15.140:0/711937314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb009d740 con 0x7fafb00988b0 2022-07-21T01:49:00.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafa37fe700 1 -- 172.21.15.140:0/711937314 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fafa4011c10 con 0x7fafb00988b0 2022-07-21T01:49:00.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.139+0000 7fafb82bf700 1 -- 172.21.15.140:0/711937314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb013f500 con 0x7fafb00988b0 2022-07-21T01:49:00.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.143+0000 7fafa37fe700 1 -- 172.21.15.140:0/711937314 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fafa4011d70 con 0x7fafb00988b0 2022-07-21T01:49:00.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.143+0000 7fafb82bf700 1 -- 172.21.15.140:0/711937314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb0099c00 con 0x7fafb00988b0 2022-07-21T01:49:00.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.143+0000 7fafa37fe700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf900539e0 0x7faf90055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:00.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.143+0000 7fafa37fe700 1 -- 172.21.15.140:0/711937314 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fafa406c370 con 0x7fafb00988b0 2022-07-21T01:49:00.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.143+0000 7fafb585a700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf900539e0 0x7faf90055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:00.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.143+0000 7fafb585a700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf900539e0 0x7faf90055ea0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fafac00b000 tx=0x7fafac006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:00.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.151+0000 7fafa37fe700 1 -- 172.21.15.140:0/711937314 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fafb0099c00 con 0x7fafb00988b0 2022-07-21T01:49:00.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.283+0000 7fafb82bf700 1 -- 172.21.15.140:0/711937314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fafb0099c00 con 0x7fafb00988b0 2022-07-21T01:49:00.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafa37fe700 1 -- 172.21.15.140:0/711937314 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7fafb0099c00 con 0x7fafb00988b0 2022-07-21T01:49:00.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 -- 172.21.15.140:0/711937314 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf900539e0 msgr2=0x7faf90055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:00.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf900539e0 0x7faf90055ea0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fafac00b000 tx=0x7fafac006df0).stop 2022-07-21T01:49:00.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 -- 172.21.15.140:0/711937314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb00988b0 msgr2=0x7fafb00990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:00.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb00988b0 0x7fafb00990e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fafa400c000 tx=0x7fafa4006df0).stop 2022-07-21T01:49:00.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 -- 172.21.15.140:0/711937314 shutdown_connections 2022-07-21T01:49:00.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf900539e0 0x7faf90055ea0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:00.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fafb01462a0 0x7fafb009cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:00.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fafb01422c0 0x7fafb009c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:00.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 --2- 172.21.15.140:0/711937314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafb00988b0 0x7fafb00990e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:00.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 -- 172.21.15.140:0/711937314 >> 172.21.15.140:0/711937314 conn(0x7fafb00b5f00 msgr2=0x7fafb0146680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:00.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 -- 172.21.15.140:0/711937314 shutdown_connections 2022-07-21T01:49:00.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:00.295+0000 7fafb82bf700 1 -- 172.21.15.140:0/711937314 wait complete. 2022-07-21T01:49:00.308 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:49:00.310 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:49:00.334 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:49:00.359 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:49:00.387 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:49:00.410 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:49:00.433 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:49:00.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.436 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.437 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:49:00.448 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:49:00.448 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:49:00.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.449 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.449 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:49:00.499 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:49:00.501 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:49:00.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.503 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.503 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:49:00.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:49:00.528 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-07-21T01:49:00.529 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-07-21T01:49:00.530 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:49:00.531 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:49:00.531 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:49:00.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.591 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.592 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:49:00.602 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:49:00.602 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:49:00.603 INFO:teuthology.orchestra.run:waiting for 900 2022-07-21T01:49:00.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:49:00.604 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:49:00.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.660 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.660 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:49:00.678 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:49:00.679 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:49:00.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.681 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.682 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:49:00.698 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:49:00.710 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:49:00.714 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:49:00.717 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:49:00.724 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:49:00.730 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:49:00.731 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:49:00.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.732 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.733 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:49:00.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:49:00.742 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': Transport endpoint is not connected 2022-07-21T01:49:00.743 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2022-07-21T01:49:00.744 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2022-07-21T01:49:00.745 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:49:00.746 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:49:00.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.792 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.793 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:49:00.840 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2022-07-21T01:49:00.841 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:49:00.842 INFO:teuthology.orchestra.run:waiting for 900 2022-07-21T01:49:00.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:49:00.843 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:49:00.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.861 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.861 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:49:00.870 INFO:teuthology.orchestra.run.smithi195.stdout:fuseblk 2022-07-21T01:49:00.871 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2022-07-21T01:49:00.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.873 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.873 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:49:00.880 INFO:teuthology.orchestra.run.smithi195.stdout:fuseblk 2022-07-21T01:49:00.881 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2022-07-21T01:49:00.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.883 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.883 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:49:00.931 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': Transport endpoint is not connected 2022-07-21T01:49:00.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:49:00.933 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.2 2022-07-21T01:49:00.934 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi195.front.sepia.ceph.com... 2022-07-21T01:49:00.935 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:49:00.936 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:49:00.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:00.962 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:00.962 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:49:00.970 INFO:teuthology.orchestra.run.smithi195.stdout:ext2/ext3 2022-07-21T01:49:00.971 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:49:00.972 INFO:teuthology.orchestra.run:waiting for 900 2022-07-21T01:49:00.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:49:00.973 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:49:01.041 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:49:01.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f3662dc2700 1 Processor -- start 2022-07-21T01:49:01.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f3662dc2700 1 -- start start 2022-07-21T01:49:01.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f3662dc2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f365c0acef0 0x7f365c14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f3662dc2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365c05d480 con 0x7f365c0b1e00 2022-07-21T01:49:01.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f3662dc2700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365c0b21e0 con 0x7f365c05c3f0 2022-07-21T01:49:01.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f3662dc2700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365c05cf40 con 0x7f365c0acef0 2022-07-21T01:49:01.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f3660b5e700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f365c0b1e00 0x7f365c0ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47532/0 (socket says 172.21.15.140:47532) 2022-07-21T01:49:01.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f3660b5e700 1 -- 172.21.15.140:0/1977498738 learned_addr learned my addr 172.21.15.140:0/1977498738 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:01.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f366135f700 1 --2- 172.21.15.140:0/1977498738 >> v2:172.21.15.152:3300/0 conn(0x7f365c0acef0 0x7f365c14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848915611 0 0) 0x7f365c05d480 con 0x7f365c0b1e00 2022-07-21T01:49:01.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3648003260 con 0x7f365c0b1e00 2022-07-21T01:49:01.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069970528 0 0) 0x7f365c0b21e0 con 0x7f365c05c3f0 2022-07-21T01:49:01.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f365c05d480 con 0x7f365c05c3f0 2022-07-21T01:49:01.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1990356329 0 0) 0x7f3648003260 con 0x7f365c0b1e00 2022-07-21T01:49:01.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 >> v2:172.21.15.152:3300/0 conn(0x7f365c0acef0 msgr2=0x7f365c14a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:01.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 --2- 172.21.15.140:0/1977498738 >> v2:172.21.15.152:3300/0 conn(0x7f365c0acef0 0x7f365c14a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 >> v1:172.21.15.152:6789/0 conn(0x7f365c05c3f0 legacy=0x7f365c146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c05d1e0 con 0x7f365c0b1e00 2022-07-21T01:49:01.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f3650001660 con 0x7f365c0b1e00 2022-07-21T01:49:01.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 >> v1:172.21.15.140:6789/0 conn(0x7f365c0b1e00 legacy=0x7f365c0ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 --2- 172.21.15.140:0/1977498738 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3648003930 0x7f3648003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 --2- 172.21.15.140:0/1977498738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3648004500 0x7f3648004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 --2- 172.21.15.140:0/1977498738 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3648004ee0 0x7f364800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c05cf40 con 0x7f3648004ee0 2022-07-21T01:49:01.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364800be80 con 0x7f3648004500 2022-07-21T01:49:01.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364800c040 con 0x7f3648003930 2022-07-21T01:49:01.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f366135f700 1 --2- 172.21.15.140:0/1977498738 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3648004ee0 0x7f364800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f3660b5e700 1 --2- 172.21.15.140:0/1977498738 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3648003930 0x7f3648003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365bfff700 1 --2- 172.21.15.140:0/1977498738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3648004500 0x7f3648004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f366135f700 1 -- 172.21.15.140:0/1977498738 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3648003930 msgr2=0x7f3648003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:01.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f366135f700 1 --2- 172.21.15.140:0/1977498738 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3648003930 0x7f3648003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f366135f700 1 -- 172.21.15.140:0/1977498738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3648004500 msgr2=0x7f3648004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:01.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f366135f700 1 --2- 172.21.15.140:0/1977498738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3648004500 0x7f3648004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f366135f700 1 -- 172.21.15.140:0/1977498738 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3648003260 con 0x7f3648004ee0 2022-07-21T01:49:01.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f366135f700 1 -- 172.21.15.140:0/1977498738 reap_dead start 2022-07-21T01:49:01.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f366135f700 1 --2- 172.21.15.140:0/1977498738 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3648004ee0 0x7f364800b750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f365400c000 tx=0x7f36540091d0).ready entity=mon.0 client_cookie=1f92f5a95434356 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:01.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f365400e7e0 con 0x7f3648004ee0 2022-07-21T01:49:01.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365400e940 con 0x7f3648004ee0 2022-07-21T01:49:01.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.155+0000 7f365b7fe700 1 -- 172.21.15.140:0/1977498738 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f365400eca0 con 0x7f3648004ee0 2022-07-21T01:49:01.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 -- 172.21.15.140:0/1977498738 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3648004ee0 msgr2=0x7f364800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 --2- 172.21.15.140:0/1977498738 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3648004ee0 0x7f364800b750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f365400c000 tx=0x7f36540091d0).stop 2022-07-21T01:49:01.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 -- 172.21.15.140:0/1977498738 shutdown_connections 2022-07-21T01:49:01.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 --2- 172.21.15.140:0/1977498738 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3648004ee0 0x7f364800b750 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 -- 172.21.15.140:0/1977498738 >> 172.21.15.140:0/1977498738 conn(0x7f365c0611c0 msgr2=0x7f365c062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:01.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 -- 172.21.15.140:0/1977498738 shutdown_connections 2022-07-21T01:49:01.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 -- 172.21.15.140:0/1977498738 wait complete. 2022-07-21T01:49:01.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 Processor -- start 2022-07-21T01:49:01.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 -- start start 2022-07-21T01:49:01.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f365c146940 0x7f365c066ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f365c066fa0 0x7f365c067420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365c0b2ec0 0x7f365c0b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c065aa0 con 0x7f365c146940 2022-07-21T01:49:01.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c067910 con 0x7f365c0b2ec0 2022-07-21T01:49:01.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3662dc2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365c1434f0 con 0x7f365c066fa0 2022-07-21T01:49:01.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3660b5e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f365c146940 0x7f365c066ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3660b5e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f365c146940 0x7f365c066ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33396/0 (socket says 172.21.15.140:33396) 2022-07-21T01:49:01.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f3660b5e700 1 -- 172.21.15.140:0/3817204470 learned_addr learned my addr 172.21.15.140:0/3817204470 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:01.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f365bfff700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f365c066fa0 0x7f365c067420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.159+0000 7f366135f700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365c0b2ec0 0x7f365c0b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3660b5e700 1 -- 172.21.15.140:0/3817204470 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f365c066fa0 msgr2=0x7f365c067420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3660b5e700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f365c066fa0 0x7f365c067420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3660b5e700 1 -- 172.21.15.140:0/3817204470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365c0b2ec0 msgr2=0x7f365c0b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3660b5e700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365c0b2ec0 0x7f365c0b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3660b5e700 1 -- 172.21.15.140:0/3817204470 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c146d20 con 0x7f365c146940 2022-07-21T01:49:01.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f366135f700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365c0b2ec0 0x7f365c0b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:01.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3660b5e700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f365c146940 0x7f365c066ab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f365c0b5f50 tx=0x7f365c0ac4d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:01.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3659ffb700 1 -- 172.21.15.140:0/3817204470 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36500116b0 con 0x7f365c146940 2022-07-21T01:49:01.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3659ffb700 1 -- 172.21.15.140:0/3817204470 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3650011810 con 0x7f365c146940 2022-07-21T01:49:01.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3662dc2700 1 -- 172.21.15.140:0/3817204470 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365c0b5930 con 0x7f365c146940 2022-07-21T01:49:01.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3659ffb700 1 -- 172.21.15.140:0/3817204470 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3650011b70 con 0x7f365c146940 2022-07-21T01:49:01.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3662dc2700 1 -- 172.21.15.140:0/3817204470 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365c0b4a40 con 0x7f365c146940 2022-07-21T01:49:01.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3659ffb700 1 -- 172.21.15.140:0/3817204470 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3650011cd0 con 0x7f365c146940 2022-07-21T01:49:01.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3662dc2700 1 -- 172.21.15.140:0/3817204470 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365c0b04b0 con 0x7f365c146940 2022-07-21T01:49:01.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3659ffb700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f363c053a70 0x7f363c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f3659ffb700 1 -- 172.21.15.140:0/3817204470 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f365006b9e0 con 0x7f365c146940 2022-07-21T01:49:01.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f365bfff700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f363c053a70 0x7f363c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.163+0000 7f365bfff700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f363c053a70 0x7f363c055f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f364c003000 tx=0x7f364c007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:01.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.171+0000 7f3659ffb700 1 -- 172.21.15.140:0/3817204470 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3650042860 con 0x7f365c146940 2022-07-21T01:49:01.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.295+0000 7f3662dc2700 1 -- 172.21.15.140:0/3817204470 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f365c0b4c60 con 0x7f365c146940 2022-07-21T01:49:01.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3659ffb700 1 -- 172.21.15.140:0/3817204470 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v22) v1 ==== 81+0+33741 (secure 0 0 0) 0x7f3650019020 con 0x7f365c146940 2022-07-21T01:49:01.298 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:01.299 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:49:01.299 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 22, 2022-07-21T01:49:01.299 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:49:01.300 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:49:01.300 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:48:32.426720+0000", 2022-07-21T01:49:01.300 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:49:01.301 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:49:01.301 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:49:01.302 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:49:01.302 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:49:01.303 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:49:01.303 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:49:01.303 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:49:01.304 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:49:01.305 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:49:01.305 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:49:01.306 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:49:01.306 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:49:01.307 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:49:01.307 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:49:01.307 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 4, 2022-07-21T01:49:01.308 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:49:01.308 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:49:01.308 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:49:01.309 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:49:01.309 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:49:01.309 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.309 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:49:01.310 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:49:01.310 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:49:01.310 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:49:01.311 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:49:01.311 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:49:01.311 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:49:01.311 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:49:01.312 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:49:01.312 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:49:01.312 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:49:01.313 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:49:01.313 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:49:01.314 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:49:01.314 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:49:01.315 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:49:01.315 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:49:01.316 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:49:01.316 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:49:01.317 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:49:01.318 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:49:01.318 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:49:01.318 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:49:01.319 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.319 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:49:01.320 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:49:01.320 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:49:01.320 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:49:01.321 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:49:01.321 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:49:01.322 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:49:01.323 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:49:01.323 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:49:01.323 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:49:01.324 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:49:01.324 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:49:01.324 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:49:01.325 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:49:01.325 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:49:01.326 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:49:01.326 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:49:01.326 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:49:01.327 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:49:01.327 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:49:01.327 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:49:01.328 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:49:01.328 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:49:01.329 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:49:01.329 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:49:01.329 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:49:01.329 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:49:01.330 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.330 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:49:01.331 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:49:01.331 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:49:01.332 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:49:01.332 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:49:01.333 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:49:01.333 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:49:01.334 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:49:01.334 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:49:01.335 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:49:01.335 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:49:01.335 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:49:01.336 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:49:01.336 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.336 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:49:01.336 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:49:01.337 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.337 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.337 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.337 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:49:01.338 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:49:01.338 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:49:01.338 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:49:01.338 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:49:01.339 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:49:01.339 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:49:01.339 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:49:01.339 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:49:01.339 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:49:01.340 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:49:01.340 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:49:01.340 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:49:01.340 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:49:01.341 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:49:01.341 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:49:01.341 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:49:01.341 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:49:01.342 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:49:01.342 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:49:01.342 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:49:01.342 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:49:01.343 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:49:01.343 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.343 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:49:01.343 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:49:01.343 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:49:01.344 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:49:01.344 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:49:01.344 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:49:01.344 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:49:01.345 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:49:01.345 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:49:01.345 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:49:01.345 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:49:01.346 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:49:01.346 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:49:01.346 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:49:01.346 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:49:01.346 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:49:01.347 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:49:01.347 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:49:01.347 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:49:01.347 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:49:01.348 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:49:01.348 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:49:01.348 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:49:01.348 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:49:01.349 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:49:01.349 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:49:01.349 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:49:01.349 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.349 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:49:01.350 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:49:01.350 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:49:01.350 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:49:01.350 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:49:01.351 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:49:01.351 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:49:01.351 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:49:01.351 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:49:01.352 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:49:01.352 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:49:01.352 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:49:01.352 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:49:01.353 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:49:01.353 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.353 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.353 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.353 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 3, 2022-07-21T01:49:01.354 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:49:01.354 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:29.344690+0000", 2022-07-21T01:49:01.354 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:49:01.354 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:49:01.355 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:49:01.355 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:49:01.355 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:49:01.356 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:49:01.356 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:49:01.356 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:49:01.356 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:49:01.357 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:49:01.357 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:49:01.357 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:49:01.357 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:49:01.357 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:49:01.358 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:49:01.358 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:49:01.359 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:49:01.359 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:49:01.359 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:49:01.359 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:49:01.359 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.360 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "21", 2022-07-21T01:49:01.360 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:49:01.360 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:49:01.360 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:49:01.361 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:49:01.361 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:49:01.361 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:49:01.361 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:49:01.362 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:49:01.362 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:49:01.362 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:49:01.362 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:49:01.363 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:49:01.363 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:49:01.363 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:49:01.363 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:49:01.364 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:49:01.364 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:49:01.364 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:49:01.364 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:49:01.365 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:49:01.365 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:49:01.365 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:49:01.365 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:49:01.365 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:49:01.366 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:49:01.366 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:49:01.366 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.366 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:49:01.367 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:49:01.367 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:49:01.367 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:49:01.367 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:49:01.368 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:49:01.368 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:49:01.368 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:49:01.368 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:49:01.368 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:49:01.369 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:49:01.369 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:49:01.369 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:49:01.370 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:49:01.370 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:49:01.370 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.370 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:49:01.371 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:49:01.371 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:49:01.371 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.371 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.372 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.372 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.372 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 4, 2022-07-21T01:49:01.372 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:49:01.373 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:30.427054+0000", 2022-07-21T01:49:01.373 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:49:01.373 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:49:01.373 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:49:01.373 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:49:01.374 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:49:01.374 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:49:01.374 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:49:01.375 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:49:01.375 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:49:01.375 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:49:01.375 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:49:01.376 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:49:01.376 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:49:01.376 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:49:01.376 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:49:01.376 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:49:01.377 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:49:01.377 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:49:01.377 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:49:01.377 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:49:01.378 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.378 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "22", 2022-07-21T01:49:01.378 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:49:01.378 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:49:01.379 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:49:01.379 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:49:01.379 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:49:01.379 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:49:01.380 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:49:01.380 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:49:01.380 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:49:01.380 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:49:01.380 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:49:01.381 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:49:01.381 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:49:01.381 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:49:01.381 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:49:01.382 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:49:01.382 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:49:01.382 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:49:01.382 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:49:01.383 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:49:01.383 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:49:01.383 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:49:01.383 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:49:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:49:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:49:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:49:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:49:01.385 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:49:01.385 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:49:01.385 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:49:01.385 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:49:01.386 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:49:01.386 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:49:01.386 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:49:01.387 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:49:01.387 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:49:01.387 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:49:01.387 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:49:01.387 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:49:01.388 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:49:01.388 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:49:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:49:01.390 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:49:01.390 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.390 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:49:01.390 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:49:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:49:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:49:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:49:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:49:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:49:01.392 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:49:01.392 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:49:01.392 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 20, 2022-07-21T01:49:01.392 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:49:01.393 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:49:01.393 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:49:01.393 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.393 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.394 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.394 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:49:01.394 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:49:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:49:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:49:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:49:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:49:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:49:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:49:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:49:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:49:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.401 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:49:01.401 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:49:01.401 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.401 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:49:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:49:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:49:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:49:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:49:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.405 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.405 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.405 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:49:01.405 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:49:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:49:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:49:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:49:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:49:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:49:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:49:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:49:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:49:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:49:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:49:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:49:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:49:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:49:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:49:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:49:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:49:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 20, 2022-07-21T01:49:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:49:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:49:01.412 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:49:01.412 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.412 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.412 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.413 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:49:01.413 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:49:01.413 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.413 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.413 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:49:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:49:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.415 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.415 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:49:01.415 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.415 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.416 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.416 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:49:01.416 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:49:01.417 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.417 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.417 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.418 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:49:01.418 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:49:01.419 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.419 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.420 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.420 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:49:01.420 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.420 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.421 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.421 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:49:01.422 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:49:01.423 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.423 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.423 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.423 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:49:01.424 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:49:01.424 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.424 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.424 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.425 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:49:01.425 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.425 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.426 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.427 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:49:01.427 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:49:01.428 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.428 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.429 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.429 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:49:01.430 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:49:01.431 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.431 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.432 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.432 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:49:01.433 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:49:01.433 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:49:01.433 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:49:01.434 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:49:01.434 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:49:01.435 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:49:01.435 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.436 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.437 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.437 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:49:01.438 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:49:01.438 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:49:01.439 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:49:01.439 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:49:01.439 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:49:01.440 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:49:01.440 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:49:01.441 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:49:01.441 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 16, 2022-07-21T01:49:01.442 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:49:01.443 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:49:01.443 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:49:01.444 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.444 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.444 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.445 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:49:01.445 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:49:01.446 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.446 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.447 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.448 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:49:01.448 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:49:01.449 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.449 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.450 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.450 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:49:01.450 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.451 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.451 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.452 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:49:01.452 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:49:01.453 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.453 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.454 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.454 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:49:01.455 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:49:01.455 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.456 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.456 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.457 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:49:01.457 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.457 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.458 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.458 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:49:01.459 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:49:01.459 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.460 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.460 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.461 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:49:01.461 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:49:01.462 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.462 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.463 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.463 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:49:01.464 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.464 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.465 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.465 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:49:01.465 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:49:01.466 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.466 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.466 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.467 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:49:01.467 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:49:01.467 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.468 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.468 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.468 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:49:01.468 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:49:01.469 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:49:01.469 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:49:01.470 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:49:01.470 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:49:01.470 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:49:01.471 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.471 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.471 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.472 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:49:01.472 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:49:01.472 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:49:01.473 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:49:01.473 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:49:01.473 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:49:01.473 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:49:01.474 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:49:01.474 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:49:01.474 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 20, 2022-07-21T01:49:01.474 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:49:01.475 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:49:01.475 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:49:01.475 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.475 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.476 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.476 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:49:01.476 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:49:01.477 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.477 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.477 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.477 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:49:01.478 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:49:01.478 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.478 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.479 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.479 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:49:01.480 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.480 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.480 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.481 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:49:01.481 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:49:01.481 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.481 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.482 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.482 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:49:01.482 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:49:01.483 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.483 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.483 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.483 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:49:01.484 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.484 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.484 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.484 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:49:01.485 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:49:01.485 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.485 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.485 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.486 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:49:01.486 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:49:01.486 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.487 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.487 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.487 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:49:01.487 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.487 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.488 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.488 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:49:01.488 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:49:01.489 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.489 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.489 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.489 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:49:01.489 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:49:01.490 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.490 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.490 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.491 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:49:01.491 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:49:01.491 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:49:01.492 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:49:01.492 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:49:01.492 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:49:01.493 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:49:01.493 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.493 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.494 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.494 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:49:01.494 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:49:01.494 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:49:01.495 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:49:01.495 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:49:01.495 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:49:01.495 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:49:01.496 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:49:01.496 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:49:01.496 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 20, 2022-07-21T01:49:01.497 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:49:01.497 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:49:01.497 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:49:01.497 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.498 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.498 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.498 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:49:01.498 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:49:01.499 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.499 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.499 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.499 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:49:01.500 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:49:01.500 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.500 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.501 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.501 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:49:01.501 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.501 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.502 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.502 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:49:01.502 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:49:01.502 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.503 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.503 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.503 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:49:01.504 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:49:01.504 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.504 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.504 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.505 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:49:01.505 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.505 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.506 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.506 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:49:01.507 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:49:01.507 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.508 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.508 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.508 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:49:01.509 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:49:01.509 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.509 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.510 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.510 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:49:01.510 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.511 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.511 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.511 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:49:01.512 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:49:01.512 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.512 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.513 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.513 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:49:01.513 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:49:01.514 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.514 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.515 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.515 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:49:01.515 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:49:01.515 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:49:01.516 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:49:01.516 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:49:01.516 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:49:01.517 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:49:01.517 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.517 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.518 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.518 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:49:01.518 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:49:01.518 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:49:01.519 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:49:01.519 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:49:01.519 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:49:01.519 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:49:01.520 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:49:01.520 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:49:01.520 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 19, 2022-07-21T01:49:01.520 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:49:01.521 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:49:01.521 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:49:01.521 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.521 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.522 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.522 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:49:01.522 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:49:01.523 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.523 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.523 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.523 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:49:01.524 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:49:01.524 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.524 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.524 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.524 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:49:01.525 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.525 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.525 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.525 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:49:01.526 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:49:01.526 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.526 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.527 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.527 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:49:01.527 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:49:01.527 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.528 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.528 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.528 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:49:01.528 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.529 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.529 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.529 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:49:01.529 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:49:01.530 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.530 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.530 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.531 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:49:01.531 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:49:01.531 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.531 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.531 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.532 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:49:01.532 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.532 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.532 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.533 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:49:01.533 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:49:01.533 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.534 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.534 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.534 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:49:01.534 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:49:01.535 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.535 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.535 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.535 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:49:01.536 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:49:01.536 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:49:01.536 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:49:01.536 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:49:01.537 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:49:01.537 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:49:01.537 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.537 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.538 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.538 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:49:01.538 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:49:01.538 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:49:01.539 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:49:01.539 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:49:01.539 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:49:01.539 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:49:01.540 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:49:01.540 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:49:01.540 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 20, 2022-07-21T01:49:01.540 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:49:01.541 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:49:01.541 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:49:01.541 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.541 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.542 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.542 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:49:01.543 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:49:01.543 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.543 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.543 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.543 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:49:01.544 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:49:01.544 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.544 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.545 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.545 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:49:01.545 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.546 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.546 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.546 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:49:01.546 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:49:01.547 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.547 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.547 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.547 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:49:01.548 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:49:01.548 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.548 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.548 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.549 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:49:01.549 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.549 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.549 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.550 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:49:01.550 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:49:01.550 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.550 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.551 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.551 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:49:01.551 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:49:01.552 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.552 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.552 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.552 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:49:01.553 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.553 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.553 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.553 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:49:01.554 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:49:01.554 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.554 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.554 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.554 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:49:01.555 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:49:01.555 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.555 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.555 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.556 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:49:01.556 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:49:01.556 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:49:01.556 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:49:01.557 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:49:01.557 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:49:01.557 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:49:01.558 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.558 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.558 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.559 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:49:01.559 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:49:01.559 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:49:01.560 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:49:01.560 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:49:01.560 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:49:01.560 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:49:01.561 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:49:01.561 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:49:01.561 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 16, 2022-07-21T01:49:01.562 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:49:01.562 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:49:01.562 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:49:01.562 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.563 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.563 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.563 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:49:01.563 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:49:01.564 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.564 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.564 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.565 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:49:01.565 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:49:01.565 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.565 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.566 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.566 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:49:01.566 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.567 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.567 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.567 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:49:01.568 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:49:01.568 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.568 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.568 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.569 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:49:01.569 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:49:01.569 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.570 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.570 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.570 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:49:01.571 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.571 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.571 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.571 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:49:01.572 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:49:01.572 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.572 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.572 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.573 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:49:01.573 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:49:01.573 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.574 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.574 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.574 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:49:01.574 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:49:01.575 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.575 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:49:01.575 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:49:01.576 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:49:01.576 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.578 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.578 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:49:01.579 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:49:01.579 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:49:01.579 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.580 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.580 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.580 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:49:01.581 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:49:01.581 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:49:01.581 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:49:01.581 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:49:01.582 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:49:01.582 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:49:01.582 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:01.583 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.583 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:49:01.584 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:49:01.585 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.585 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:49:01.586 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:49:01.586 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:49:01.587 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:49:01.587 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:49:01.588 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:49:01.588 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:49:01.588 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:49:01.589 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.589 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.590 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:49:01.590 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:49:01.590 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:49:01.591 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:49:01.591 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:49:01.591 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:49:01.591 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:49:01.592 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:49:01.592 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.592 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.593 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:49:01.593 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:49:01.593 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:49:01.594 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:49:01.594 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:49:01.594 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:49:01.595 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:49:01.595 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:49:01.595 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.595 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.596 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:49:01.596 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:49:01.596 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:49:01.597 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:49:01.597 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:49:01.597 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:49:01.597 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:49:01.598 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:49:01.598 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.598 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.599 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:49:01.599 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:49:01.599 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:49:01.599 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:49:01.599 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:49:01.600 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:49:01.600 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:49:01.600 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:49:01.600 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.601 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.601 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:49:01.601 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:49:01.601 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:49:01.602 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:49:01.602 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:49:01.602 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:49:01.602 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:49:01.603 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:49:01.603 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.603 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.603 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:49:01.604 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:49:01.604 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:49:01.604 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:49:01.604 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:49:01.605 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:49:01.605 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:49:01.605 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:49:01.605 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.606 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:01.606 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:49:01.606 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:49:01.606 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:49:01.607 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:49:01.607 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:49:01.607 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:49:01.607 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:49:01.608 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:49:01.608 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.608 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:49:01.609 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:49:01.609 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:49:01.609 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:49:01.609 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:49:01.610 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": {}, 2022-07-21T01:49:01.610 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:49:01.610 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:49:01.610 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:49:01.611 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:49:01.611 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:49:01.611 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:49:01.611 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:49:01.612 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:49:01.612 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:01.612 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:01.612 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:49:01.612 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:49:01.613 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:49:01.613 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:49:01.613 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:49:01.613 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:49:01.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 -- 172.21.15.140:0/3817204470 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f363c053a70 msgr2=0x7f363c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f363c053a70 0x7f363c055f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f364c003000 tx=0x7f364c007a00).stop 2022-07-21T01:49:01.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 -- 172.21.15.140:0/3817204470 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f365c146940 msgr2=0x7f365c066ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f365c146940 0x7f365c066ab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f365c0b5f50 tx=0x7f365c0ac4d0).stop 2022-07-21T01:49:01.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 -- 172.21.15.140:0/3817204470 shutdown_connections 2022-07-21T01:49:01.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f363c053a70 0x7f363c055f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f365c0b2ec0 0x7f365c0b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f365c066fa0 0x7f365c067420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 --2- 172.21.15.140:0/3817204470 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f365c146940 0x7f365c066ab0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 -- 172.21.15.140:0/3817204470 >> 172.21.15.140:0/3817204470 conn(0x7f3648004ee0 msgr2=0x7f365c0afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:01.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 -- 172.21.15.140:0/3817204470 shutdown_connections 2022-07-21T01:49:01.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.299+0000 7f3662dc2700 1 -- 172.21.15.140:0/3817204470 wait complete. 2022-07-21T01:49:01.620 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:49:01.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.735+0000 7fec1a74b700 1 Processor -- start 2022-07-21T01:49:01.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.735+0000 7fec1a74b700 1 -- start start 2022-07-21T01:49:01.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.735+0000 7fec1a74b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fec140b5650 0x7fec140b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.735+0000 7fec1a74b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec140af8e0 0x7fec1405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.735+0000 7fec1a74b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec14147010 con 0x7fec140adff0 2022-07-21T01:49:01.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.735+0000 7fec1a74b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec141472b0 con 0x7fec140b5650 2022-07-21T01:49:01.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.735+0000 7fec1a74b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec14146d40 con 0x7fec140af8e0 2022-07-21T01:49:01.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.735+0000 7fec19749700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fec140b5650 0x7fec140b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.735+0000 7fec18f48700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fec140adff0 0x7fec14147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:48664/0 (socket says 172.21.15.140:48664) 2022-07-21T01:49:01.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.735+0000 7fec18f48700 1 -- 172.21.15.140:0/2306944770 learned_addr learned my addr 172.21.15.140:0/2306944770 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:01.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.735+0000 7fec19f4a700 1 --2- 172.21.15.140:0/2306944770 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec140af8e0 0x7fec1405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec07fff700 1 -- 172.21.15.140:0/2306944770 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320616251 0 0) 0x7fec14147010 con 0x7fec140adff0 2022-07-21T01:49:01.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec07fff700 1 -- 172.21.15.140:0/2306944770 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febf8003260 con 0x7fec140adff0 2022-07-21T01:49:01.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec19f4a700 1 -- 172.21.15.140:0/2306944770 >> v2:172.21.15.140:3300/0 conn(0x7fec140b5650 msgr2=0x7fec140b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:01.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec19f4a700 1 --2- 172.21.15.140:0/2306944770 >> v2:172.21.15.140:3300/0 conn(0x7fec140b5650 0x7fec140b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec19f4a700 1 -- 172.21.15.140:0/2306944770 >> v1:172.21.15.152:6789/0 conn(0x7fec140adff0 legacy=0x7fec14147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec19f4a700 1 -- 172.21.15.140:0/2306944770 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec140acae0 con 0x7fec140af8e0 2022-07-21T01:49:01.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec19749700 1 --2- 172.21.15.140:0/2306944770 >> v2:172.21.15.140:3300/0 conn(0x7fec140b5650 0x7fec140b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:01.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec19f4a700 1 --2- 172.21.15.140:0/2306944770 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec140af8e0 0x7fec1405bd90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fec0c002f70 tx=0x7fec0c006df0).ready entity=mon.2 client_cookie=2bccd161785a1ae9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:01.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec07fff700 1 -- 172.21.15.140:0/2306944770 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0c00d4a0 con 0x7fec140af8e0 2022-07-21T01:49:01.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec07fff700 1 -- 172.21.15.140:0/2306944770 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c000e90 con 0x7fec140af8e0 2022-07-21T01:49:01.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec07fff700 1 -- 172.21.15.140:0/2306944770 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0c00d7f0 con 0x7fec140af8e0 2022-07-21T01:49:01.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec1a74b700 1 -- 172.21.15.140:0/2306944770 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec140af8e0 msgr2=0x7fec1405bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec1a74b700 1 --2- 172.21.15.140:0/2306944770 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec140af8e0 0x7fec1405bd90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fec0c002f70 tx=0x7fec0c006df0).stop 2022-07-21T01:49:01.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec1a74b700 1 -- 172.21.15.140:0/2306944770 shutdown_connections 2022-07-21T01:49:01.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec1a74b700 1 --2- 172.21.15.140:0/2306944770 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec140af8e0 0x7fec1405bd90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec1a74b700 1 --2- 172.21.15.140:0/2306944770 >> v2:172.21.15.140:3300/0 conn(0x7fec140b5650 0x7fec140b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec1a74b700 1 -- 172.21.15.140:0/2306944770 >> 172.21.15.140:0/2306944770 conn(0x7fec140634e0 msgr2=0x7fec140b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:01.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec1a74b700 1 -- 172.21.15.140:0/2306944770 shutdown_connections 2022-07-21T01:49:01.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.739+0000 7fec1a74b700 1 -- 172.21.15.140:0/2306944770 wait complete. 2022-07-21T01:49:01.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec1a74b700 1 Processor -- start 2022-07-21T01:49:01.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec1a74b700 1 -- start start 2022-07-21T01:49:01.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec1a74b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec140adff0 0x7fec141560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec1a74b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec140af8e0 0x7fec140b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec1a74b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec140b5650 0x7fec14066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec1a74b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec141116a0 con 0x7fec140b5650 2022-07-21T01:49:01.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec1a74b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec14111160 con 0x7fec140af8e0 2022-07-21T01:49:01.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec1a74b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec14111400 con 0x7fec140adff0 2022-07-21T01:49:01.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec19f4a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec140b5650 0x7fec14066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec19f4a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec140b5650 0x7fec14066cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33414/0 (socket says 172.21.15.140:33414) 2022-07-21T01:49:01.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec19f4a700 1 -- 172.21.15.140:0/3673885354 learned_addr learned my addr 172.21.15.140:0/3673885354 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:01.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec18f48700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec140af8e0 0x7fec140b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec19749700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec140adff0 0x7fec141560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec19f4a700 1 -- 172.21.15.140:0/3673885354 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec140adff0 msgr2=0x7fec141560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec19f4a700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec140adff0 0x7fec141560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec19f4a700 1 -- 172.21.15.140:0/3673885354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec140af8e0 msgr2=0x7fec140b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec19f4a700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec140af8e0 0x7fec140b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec19f4a700 1 -- 172.21.15.140:0/3673885354 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0800b180 con 0x7fec140b5650 2022-07-21T01:49:01.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec18f48700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec140af8e0 0x7fec140b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:01.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec19f4a700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec140b5650 0x7fec14066cf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fec0c002f70 tx=0x7fec0c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:01.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec067fc700 1 -- 172.21.15.140:0/3673885354 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0c016070 con 0x7fec140b5650 2022-07-21T01:49:01.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec067fc700 1 -- 172.21.15.140:0/3673885354 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c013850 con 0x7fec140b5650 2022-07-21T01:49:01.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec1a74b700 1 -- 172.21.15.140:0/3673885354 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c00b180 con 0x7fec140b5650 2022-07-21T01:49:01.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec067fc700 1 -- 172.21.15.140:0/3673885354 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0c013bb0 con 0x7fec140b5650 2022-07-21T01:49:01.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.743+0000 7fec1a74b700 1 -- 172.21.15.140:0/3673885354 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec140674b0 con 0x7fec140b5650 2022-07-21T01:49:01.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.747+0000 7fec1a74b700 1 -- 172.21.15.140:0/3673885354 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1405c280 con 0x7fec140b5650 2022-07-21T01:49:01.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.747+0000 7fec067fc700 1 -- 172.21.15.140:0/3673885354 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fec0c025030 con 0x7fec140b5650 2022-07-21T01:49:01.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.747+0000 7fec067fc700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febfc0539e0 0x7febfc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:01.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.747+0000 7fec067fc700 1 -- 172.21.15.140:0/3673885354 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec0c06ce50 con 0x7fec140b5650 2022-07-21T01:49:01.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.747+0000 7fec19749700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febfc0539e0 0x7febfc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:01.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.747+0000 7fec19749700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febfc0539e0 0x7febfc055ea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fec0800b000 tx=0x7fec08006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:01.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.755+0000 7fec067fc700 1 -- 172.21.15.140:0/3673885354 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fec0c043c60 con 0x7fec140b5650 2022-07-21T01:49:01.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.919+0000 7fec1a74b700 1 -- 172.21.15.140:0/3673885354 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec140574e0 con 0x7fec140b5650 2022-07-21T01:49:01.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.919+0000 7fec067fc700 1 -- 172.21.15.140:0/3673885354 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2137 (secure 0 0 0) 0x7fec0c023090 con 0x7fec140b5650 2022-07-21T01:49:01.921 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:01.921 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/665899293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":665899293},{"type":"v1","addr":"172.21.15.140:6835","nonce":665899293}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:48:31.423028+0000","modified":"2022-07-21T01:48:32.427116+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.152:6833/1276387778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1276387778},{"type":"v1","addr":"172.21.15.152:6833","nonce":1276387778}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-07-21T01:49:01.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 -- 172.21.15.140:0/3673885354 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febfc0539e0 msgr2=0x7febfc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febfc0539e0 0x7febfc055ea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fec0800b000 tx=0x7fec08006df0).stop 2022-07-21T01:49:01.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 -- 172.21.15.140:0/3673885354 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec140b5650 msgr2=0x7fec14066cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:01.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec140b5650 0x7fec14066cf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fec0c002f70 tx=0x7fec0c014000).stop 2022-07-21T01:49:01.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 -- 172.21.15.140:0/3673885354 shutdown_connections 2022-07-21T01:49:01.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febfc0539e0 0x7febfc055ea0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec140b5650 0x7fec14066cf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec140af8e0 0x7fec140b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 --2- 172.21.15.140:0/3673885354 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec140adff0 0x7fec141560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:01.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 -- 172.21.15.140:0/3673885354 >> 172.21.15.140:0/3673885354 conn(0x7fec140634e0 msgr2=0x7fec1414a250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:01.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 -- 172.21.15.140:0/3673885354 shutdown_connections 2022-07-21T01:49:01.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:01.923+0000 7fec1a74b700 1 -- 172.21.15.140:0/3673885354 wait complete. 2022-07-21T01:49:01.926 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 5 2022-07-21T01:49:01.935 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:49:02.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 Processor -- start 2022-07-21T01:49:02.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 -- start start 2022-07-21T01:49:02.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f49580b5990 0x7f4958146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:02.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49580af3e0 0x7f4958146560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:02.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49580b5d70 con 0x7f495805d210 2022-07-21T01:49:02.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49580ad050 con 0x7f49580b5990 2022-07-21T01:49:02.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49580b17e0 con 0x7f49580af3e0 2022-07-21T01:49:02.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495ce92700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f49580b5990 0x7f4958146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:02.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f4957fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49580af3e0 0x7f4958146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:02.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495d693700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f495805d210 0x7f495814a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:48680/0 (socket says 172.21.15.140:48680) 2022-07-21T01:49:02.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495ce92700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f49580b5990 0x7f4958146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33422/0 (socket says 172.21.15.140:33422) 2022-07-21T01:49:02.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495d693700 1 -- 172.21.15.140:0/137614278 learned_addr learned my addr 172.21.15.140:0/137614278 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:02.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f49577fe700 1 -- 172.21.15.140:0/137614278 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827893729 0 0) 0x7f49580b5d70 con 0x7f495805d210 2022-07-21T01:49:02.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495ce92700 1 -- 172.21.15.140:0/137614278 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49580af3e0 msgr2=0x7f4958146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:02.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495ce92700 1 --2- 172.21.15.140:0/137614278 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49580af3e0 0x7f4958146560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:02.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495ce92700 1 -- 172.21.15.140:0/137614278 >> v1:172.21.15.152:6789/0 conn(0x7f495805d210 legacy=0x7f495814a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:02.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495ce92700 1 -- 172.21.15.140:0/137614278 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49580b1010 con 0x7f49580b5990 2022-07-21T01:49:02.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495ce92700 1 --2- 172.21.15.140:0/137614278 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49580b5990 0x7f4958146070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f4948002f70 tx=0x7f4948006d90).ready entity=mon.0 client_cookie=3029b740c4d5f07e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:02.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f49577fe700 1 -- 172.21.15.140:0/137614278 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f494800d4a0 con 0x7f49580b5990 2022-07-21T01:49:02.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f49577fe700 1 -- 172.21.15.140:0/137614278 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4948000e90 con 0x7f49580b5990 2022-07-21T01:49:02.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f49577fe700 1 -- 172.21.15.140:0/137614278 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f494800d7f0 con 0x7f49580b5990 2022-07-21T01:49:02.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 -- 172.21.15.140:0/137614278 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49580b5990 msgr2=0x7f4958146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:02.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 --2- 172.21.15.140:0/137614278 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49580b5990 0x7f4958146070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f4948002f70 tx=0x7f4948006d90).stop 2022-07-21T01:49:02.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 -- 172.21.15.140:0/137614278 shutdown_connections 2022-07-21T01:49:02.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 --2- 172.21.15.140:0/137614278 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49580af3e0 0x7f4958146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:02.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 --2- 172.21.15.140:0/137614278 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49580b5990 0x7f4958146070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:02.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 -- 172.21.15.140:0/137614278 >> 172.21.15.140:0/137614278 conn(0x7f4958063480 msgr2=0x7f49580b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:02.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 -- 172.21.15.140:0/137614278 shutdown_connections 2022-07-21T01:49:02.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.039+0000 7f495de94700 1 -- 172.21.15.140:0/137614278 wait complete. 2022-07-21T01:49:02.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495de94700 1 Processor -- start 2022-07-21T01:49:02.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495de94700 1 -- start start 2022-07-21T01:49:02.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495de94700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f495805d210 0x7f49581116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:02.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495de94700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49580af3e0 0x7f49581440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:02.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495de94700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49580b5990 0x7f4958159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:02.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495de94700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4958111ef0 con 0x7f49580b5990 2022-07-21T01:49:02.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495de94700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4958159800 con 0x7f495805d210 2022-07-21T01:49:02.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495de94700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495814b000 con 0x7f49580af3e0 2022-07-21T01:49:02.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495d693700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49580b5990 0x7f4958159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:02.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495d693700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49580b5990 0x7f4958159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33428/0 (socket says 172.21.15.140:33428) 2022-07-21T01:49:02.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495d693700 1 -- 172.21.15.140:0/2386330123 learned_addr learned my addr 172.21.15.140:0/2386330123 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:02.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495ce92700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f495805d210 0x7f49581116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:02.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f4957fff700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49580af3e0 0x7f49581440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:02.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495d693700 1 -- 172.21.15.140:0/2386330123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49580af3e0 msgr2=0x7f49581440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:02.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495d693700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49580af3e0 0x7f49581440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:02.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495d693700 1 -- 172.21.15.140:0/2386330123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f495805d210 msgr2=0x7f49581116c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:02.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495d693700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f495805d210 0x7f49581116c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:02.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495d693700 1 -- 172.21.15.140:0/2386330123 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494800b120 con 0x7f49580b5990 2022-07-21T01:49:02.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f4957fff700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49580af3e0 0x7f49581440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:02.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495d693700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49580b5990 0x7f4958159c60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f495814b190 tx=0x7f4950007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:02.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f4955ffb700 1 -- 172.21.15.140:0/2386330123 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4950005ba0 con 0x7f49580b5990 2022-07-21T01:49:02.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495de94700 1 -- 172.21.15.140:0/2386330123 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495814ad60 con 0x7f49580b5990 2022-07-21T01:49:02.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f4955ffb700 1 -- 172.21.15.140:0/2386330123 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4950005d00 con 0x7f49580b5990 2022-07-21T01:49:02.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f4955ffb700 1 -- 172.21.15.140:0/2386330123 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49500072c0 con 0x7f49580b5990 2022-07-21T01:49:02.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495de94700 1 -- 172.21.15.140:0/2386330123 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495814a850 con 0x7f49580b5990 2022-07-21T01:49:02.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f495de94700 1 -- 172.21.15.140:0/2386330123 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495814ab60 con 0x7f49580b5990 2022-07-21T01:49:02.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.043+0000 7f4955ffb700 1 -- 172.21.15.140:0/2386330123 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4950010070 con 0x7f49580b5990 2022-07-21T01:49:02.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.047+0000 7f4955ffb700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f493c053b30 0x7f493c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:02.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.047+0000 7f4955ffb700 1 -- 172.21.15.140:0/2386330123 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f495006b930 con 0x7f49580b5990 2022-07-21T01:49:02.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.047+0000 7f495ce92700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f493c053b30 0x7f493c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:02.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.047+0000 7f495ce92700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f493c053b30 0x7f493c055ff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f494801a000 tx=0x7f4948014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:02.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.051+0000 7f4955ffb700 1 -- 172.21.15.140:0/2386330123 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f49500427b0 con 0x7f49580b5990 2022-07-21T01:49:02.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:02.211+0000 7f495de94700 1 -- 172.21.15.140:0/2386330123 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f49580afa60 con 0x7f49580b5990 2022-07-21T01:49:02.996 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:49:02.999+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:49:03.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.011+0000 7f4955ffb700 1 -- 172.21.15.140:0/2386330123 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v6) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4950045dd0 con 0x7f49580b5990 2022-07-21T01:49:03.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 -- 172.21.15.140:0/2386330123 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f493c053b30 msgr2=0x7f493c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f493c053b30 0x7f493c055ff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f494801a000 tx=0x7f4948014000).stop 2022-07-21T01:49:03.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 -- 172.21.15.140:0/2386330123 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49580b5990 msgr2=0x7f4958159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49580b5990 0x7f4958159c60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f495814b190 tx=0x7f4950007d40).stop 2022-07-21T01:49:03.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 -- 172.21.15.140:0/2386330123 shutdown_connections 2022-07-21T01:49:03.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f493c053b30 0x7f493c055ff0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49580b5990 0x7f4958159c60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49580af3e0 0x7f49581440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 --2- 172.21.15.140:0/2386330123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f495805d210 0x7f49581116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 -- 172.21.15.140:0/2386330123 >> 172.21.15.140:0/2386330123 conn(0x7f4958063480 msgr2=0x7f4958146200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:03.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 -- 172.21.15.140:0/2386330123 shutdown_connections 2022-07-21T01:49:03.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.015+0000 7f495de94700 1 -- 172.21.15.140:0/2386330123 wait complete. 2022-07-21T01:49:03.020 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:49:03.030 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:49:03.034 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:49:03.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.135+0000 7fc6db4ed700 1 Processor -- start 2022-07-21T01:49:03.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.135+0000 7fc6db4ed700 1 -- start start 2022-07-21T01:49:03.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.135+0000 7fc6db4ed700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc6d40b7b70 0x7fc6d40b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:03.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.135+0000 7fc6db4ed700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc6d4066160 0x7fc6d413df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:03.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.135+0000 7fc6db4ed700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d40b7f50 con 0x7fc6d40b31e0 2022-07-21T01:49:03.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.135+0000 7fc6db4ed700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d40b2800 con 0x7fc6d40b7b70 2022-07-21T01:49:03.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.135+0000 7fc6db4ed700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d40b2aa0 con 0x7fc6d4066160 2022-07-21T01:49:03.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6da4eb700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc6d40b7b70 0x7fc6d40b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:03.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6d9cea700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc6d40b31e0 0x7fc6d413d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:48692/0 (socket says 172.21.15.140:48692) 2022-07-21T01:49:03.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6d9cea700 1 -- 172.21.15.140:0/13985194 learned_addr learned my addr 172.21.15.140:0/13985194 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:03.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6dacec700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc6d4066160 0x7fc6d413df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:03.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6d94e9700 1 -- 172.21.15.140:0/13985194 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249265430 0 0) 0x7fc6d40b7f50 con 0x7fc6d40b31e0 2022-07-21T01:49:03.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6d94e9700 1 -- 172.21.15.140:0/13985194 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6bc003260 con 0x7fc6d40b31e0 2022-07-21T01:49:03.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6da4eb700 1 -- 172.21.15.140:0/13985194 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc6d4066160 msgr2=0x7fc6d413df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:03.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6da4eb700 1 --2- 172.21.15.140:0/13985194 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc6d4066160 0x7fc6d413df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6da4eb700 1 -- 172.21.15.140:0/13985194 >> v1:172.21.15.152:6789/0 conn(0x7fc6d40b31e0 legacy=0x7fc6d413d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6da4eb700 1 -- 172.21.15.140:0/13985194 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d40b2d40 con 0x7fc6d40b7b70 2022-07-21T01:49:03.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6da4eb700 1 --2- 172.21.15.140:0/13985194 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc6d40b7b70 0x7fc6d40b1ac0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fc6c4002f70 tx=0x7fc6c4006df0).ready entity=mon.0 client_cookie=98415b4c749ef155 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:03.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6d94e9700 1 -- 172.21.15.140:0/13985194 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6c400d4a0 con 0x7fc6d40b7b70 2022-07-21T01:49:03.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6d94e9700 1 -- 172.21.15.140:0/13985194 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c4000e90 con 0x7fc6d40b7b70 2022-07-21T01:49:03.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6d94e9700 1 -- 172.21.15.140:0/13985194 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6c400d7f0 con 0x7fc6d40b7b70 2022-07-21T01:49:03.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6db4ed700 1 -- 172.21.15.140:0/13985194 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc6d40b7b70 msgr2=0x7fc6d40b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6db4ed700 1 --2- 172.21.15.140:0/13985194 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc6d40b7b70 0x7fc6d40b1ac0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fc6c4002f70 tx=0x7fc6c4006df0).stop 2022-07-21T01:49:03.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6db4ed700 1 -- 172.21.15.140:0/13985194 shutdown_connections 2022-07-21T01:49:03.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6db4ed700 1 --2- 172.21.15.140:0/13985194 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc6d4066160 0x7fc6d413df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6db4ed700 1 --2- 172.21.15.140:0/13985194 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc6d40b7b70 0x7fc6d40b1ac0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6db4ed700 1 -- 172.21.15.140:0/13985194 >> 172.21.15.140:0/13985194 conn(0x7fc6d40656b0 msgr2=0x7fc6d40b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:03.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6db4ed700 1 -- 172.21.15.140:0/13985194 shutdown_connections 2022-07-21T01:49:03.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.139+0000 7fc6db4ed700 1 -- 172.21.15.140:0/13985194 wait complete. 2022-07-21T01:49:03.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6db4ed700 1 Processor -- start 2022-07-21T01:49:03.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6db4ed700 1 -- start start 2022-07-21T01:49:03.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6db4ed700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc6d4066160 0x7fc6d413d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:03.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6db4ed700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc6d40b31e0 0x7fc6d40aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:03.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6db4ed700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6d40b7b70 0x7fc6d4111130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:03.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6db4ed700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d414d900 con 0x7fc6d40b31e0 2022-07-21T01:49:03.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6db4ed700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4110520 con 0x7fc6d40b7b70 2022-07-21T01:49:03.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6db4ed700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4110950 con 0x7fc6d4066160 2022-07-21T01:49:03.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6d9cea700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc6d40b31e0 0x7fc6d40aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:03.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6d9cea700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc6d40b31e0 0x7fc6d40aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33440/0 (socket says 172.21.15.140:33440) 2022-07-21T01:49:03.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6d9cea700 1 -- 172.21.15.140:0/3188364928 learned_addr learned my addr 172.21.15.140:0/3188364928 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:03.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6dacec700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6d40b7b70 0x7fc6d4111130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:03.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6da4eb700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc6d4066160 0x7fc6d413d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:03.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6d9cea700 1 -- 172.21.15.140:0/3188364928 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc6d4066160 msgr2=0x7fc6d413d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6d9cea700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc6d4066160 0x7fc6d413d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6d9cea700 1 -- 172.21.15.140:0/3188364928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6d40b7b70 msgr2=0x7fc6d4111130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6d9cea700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6d40b7b70 0x7fc6d4111130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6d9cea700 1 -- 172.21.15.140:0/3188364928 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c400b180 con 0x7fc6d40b31e0 2022-07-21T01:49:03.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6d9cea700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc6d40b31e0 0x7fc6d40aeaf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fc6d414fc00 tx=0x7fc6d0007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:03.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6c37fe700 1 -- 172.21.15.140:0/3188364928 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6d000cea0 con 0x7fc6d40b31e0 2022-07-21T01:49:03.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6c37fe700 1 -- 172.21.15.140:0/3188364928 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d00070d0 con 0x7fc6d40b31e0 2022-07-21T01:49:03.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6db4ed700 1 -- 172.21.15.140:0/3188364928 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d4113640 con 0x7fc6d40b31e0 2022-07-21T01:49:03.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6c37fe700 1 -- 172.21.15.140:0/3188364928 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc6d0007430 con 0x7fc6d40b31e0 2022-07-21T01:49:03.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.143+0000 7fc6db4ed700 1 -- 172.21.15.140:0/3188364928 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d40afff0 con 0x7fc6d40b31e0 2022-07-21T01:49:03.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.147+0000 7fc6db4ed700 1 -- 172.21.15.140:0/3188364928 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d41117c0 con 0x7fc6d40b31e0 2022-07-21T01:49:03.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.147+0000 7fc6c37fe700 1 -- 172.21.15.140:0/3188364928 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc6d001a060 con 0x7fc6d40b31e0 2022-07-21T01:49:03.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.147+0000 7fc6c37fe700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc6b4053a70 0x7fc6b4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:03.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.147+0000 7fc6c37fe700 1 -- 172.21.15.140:0/3188364928 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc6d006b740 con 0x7fc6d40b31e0 2022-07-21T01:49:03.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.147+0000 7fc6da4eb700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc6b4053a70 0x7fc6b4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:03.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.147+0000 7fc6da4eb700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc6b4053a70 0x7fc6b4055f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fc6c401a000 tx=0x7fc6c4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:03.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.155+0000 7fc6c37fe700 1 -- 172.21.15.140:0/3188364928 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc6d0042500 con 0x7fc6d40b31e0 2022-07-21T01:49:03.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.311+0000 7fc6db4ed700 1 -- 172.21.15.140:0/3188364928 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6d40574e0 con 0x7fc6d40b31e0 2022-07-21T01:49:03.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.311+0000 7fc6c37fe700 1 -- 172.21.15.140:0/3188364928 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+1774 (secure 0 0 0) 0x7fc6d0018070 con 0x7fc6d40b31e0 2022-07-21T01:49:03.310 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:03.311 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/665899293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":665899293},{"type":"v1","addr":"172.21.15.140:6835","nonce":665899293}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:48:31.423028+0000","modified":"2022-07-21T01:49:03.001039+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-07-21T01:49:03.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.311+0000 7fc6db4ed700 1 -- 172.21.15.140:0/3188364928 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc6b4053a70 msgr2=0x7fc6b4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.315+0000 7fc6db4ed700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc6b4053a70 0x7fc6b4055f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fc6c401a000 tx=0x7fc6c4014000).stop 2022-07-21T01:49:03.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.315+0000 7fc6db4ed700 1 -- 172.21.15.140:0/3188364928 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc6d40b31e0 msgr2=0x7fc6d40aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.315+0000 7fc6db4ed700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc6d40b31e0 0x7fc6d40aeaf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fc6d414fc00 tx=0x7fc6d0007a80).stop 2022-07-21T01:49:03.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.315+0000 7fc6db4ed700 1 -- 172.21.15.140:0/3188364928 shutdown_connections 2022-07-21T01:49:03.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.315+0000 7fc6db4ed700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc6b4053a70 0x7fc6b4055f30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.315+0000 7fc6db4ed700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc6d40b7b70 0x7fc6d4111130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.315+0000 7fc6db4ed700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc6d40b31e0 0x7fc6d40aeaf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.315+0000 7fc6db4ed700 1 --2- 172.21.15.140:0/3188364928 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc6d4066160 0x7fc6d413d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.315+0000 7fc6db4ed700 1 -- 172.21.15.140:0/3188364928 >> 172.21.15.140:0/3188364928 conn(0x7fc6d40656b0 msgr2=0x7fc6d413dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:03.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.315+0000 7fc6db4ed700 1 -- 172.21.15.140:0/3188364928 shutdown_connections 2022-07-21T01:49:03.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.315+0000 7fc6db4ed700 1 -- 172.21.15.140:0/3188364928 wait complete. 2022-07-21T01:49:03.316 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 6 2022-07-21T01:49:03.323 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:49:03.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c0d336700 1 Processor -- start 2022-07-21T01:49:03.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c0d336700 1 -- start start 2022-07-21T01:49:03.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c0d336700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c080b6800 0x7f2c080aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:03.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c0d336700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c080ae280 con 0x7f2c080b3cb0 2022-07-21T01:49:03.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c0d336700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c080b6be0 con 0x7f2c080ad0e0 2022-07-21T01:49:03.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c0d336700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c080ae4b0 con 0x7f2c080b6800 2022-07-21T01:49:03.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c077fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2c080b3cb0 0x7f2c08145f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47594/0 (socket says 172.21.15.140:47594) 2022-07-21T01:49:03.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c077fe700 1 -- 172.21.15.140:0/2482242876 learned_addr learned my addr 172.21.15.140:0/2482242876 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:03.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c07fff700 1 --2- 172.21.15.140:0/2482242876 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c080b6800 0x7f2c080aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:03.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c06ffd700 1 -- 172.21.15.140:0/2482242876 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293789055 0 0) 0x7f2c080ae280 con 0x7f2c080b3cb0 2022-07-21T01:49:03.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c06ffd700 1 -- 172.21.15.140:0/2482242876 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2be8003260 con 0x7f2c080b3cb0 2022-07-21T01:49:03.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c06ffd700 1 -- 172.21.15.140:0/2482242876 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353158388 0 0) 0x7f2c080b6be0 con 0x7f2c080ad0e0 2022-07-21T01:49:03.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.431+0000 7f2c06ffd700 1 -- 172.21.15.140:0/2482242876 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c080ae280 con 0x7f2c080ad0e0 2022-07-21T01:49:03.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c07fff700 1 -- 172.21.15.140:0/2482242876 >> v1:172.21.15.152:6789/0 conn(0x7f2c080ad0e0 legacy=0x7f2c081418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c06ffd700 1 -- 172.21.15.140:0/2482242876 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4101683320 0 0) 0x7f2be8003260 con 0x7f2c080b3cb0 2022-07-21T01:49:03.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c07fff700 1 -- 172.21.15.140:0/2482242876 >> v1:172.21.15.140:6789/0 conn(0x7f2c080b3cb0 legacy=0x7f2c08145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c07fff700 1 -- 172.21.15.140:0/2482242876 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c080ae750 con 0x7f2c080b6800 2022-07-21T01:49:03.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c07fff700 1 --2- 172.21.15.140:0/2482242876 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c080b6800 0x7f2c080aebf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2bf0001110 tx=0x7f2bf8006df0).ready entity=mon.2 client_cookie=ac874b4a9aa5a132 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:03.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c06ffd700 1 -- 172.21.15.140:0/2482242876 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bf0001660 con 0x7f2c080b6800 2022-07-21T01:49:03.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c06ffd700 1 -- 172.21.15.140:0/2482242876 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf8000e90 con 0x7f2c080b6800 2022-07-21T01:49:03.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c06ffd700 1 -- 172.21.15.140:0/2482242876 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bf8012460 con 0x7f2c080b6800 2022-07-21T01:49:03.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c0d336700 1 -- 172.21.15.140:0/2482242876 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c080b6800 msgr2=0x7f2c080aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c0d336700 1 --2- 172.21.15.140:0/2482242876 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c080b6800 0x7f2c080aebf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f2bf0001110 tx=0x7f2bf8006df0).stop 2022-07-21T01:49:03.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c0d336700 1 -- 172.21.15.140:0/2482242876 shutdown_connections 2022-07-21T01:49:03.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c0d336700 1 --2- 172.21.15.140:0/2482242876 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c080b6800 0x7f2c080aebf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c0d336700 1 -- 172.21.15.140:0/2482242876 >> 172.21.15.140:0/2482242876 conn(0x7f2c080b9dd0 msgr2=0x7f2c080b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:03.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c0d336700 1 -- 172.21.15.140:0/2482242876 shutdown_connections 2022-07-21T01:49:03.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c0d336700 1 -- 172.21.15.140:0/2482242876 wait complete. 2022-07-21T01:49:03.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c0d336700 1 Processor -- start 2022-07-21T01:49:03.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.435+0000 7f2c0d336700 1 -- start start 2022-07-21T01:49:03.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c0d336700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c080ad0e0 0x7f2c080b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:03.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c0d336700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c080b3cb0 0x7f2c08112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:03.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c0d336700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c080b6800 0x7f2c08113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:03.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c0d336700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c08113880 con 0x7f2c080ad0e0 2022-07-21T01:49:03.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c0d336700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c081135f0 con 0x7f2c080b6800 2022-07-21T01:49:03.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c0d336700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c08115900 con 0x7f2c080b3cb0 2022-07-21T01:49:03.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c07fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c080ad0e0 0x7f2c080b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:03.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c07fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c080ad0e0 0x7f2c080b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33452/0 (socket says 172.21.15.140:33452) 2022-07-21T01:49:03.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c07fff700 1 -- 172.21.15.140:0/1138708993 learned_addr learned my addr 172.21.15.140:0/1138708993 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:03.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c0cb35700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c080b6800 0x7f2c08113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:03.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c077fe700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c080b3cb0 0x7f2c08112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:03.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c07fff700 1 -- 172.21.15.140:0/1138708993 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c080b3cb0 msgr2=0x7f2c08112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c07fff700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c080b3cb0 0x7f2c08112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c07fff700 1 -- 172.21.15.140:0/1138708993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c080b6800 msgr2=0x7f2c08113100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:03.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c07fff700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c080b6800 0x7f2c08113100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:03.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c07fff700 1 -- 172.21.15.140:0/1138708993 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf800b180 con 0x7f2c080ad0e0 2022-07-21T01:49:03.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c0cb35700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c080b6800 0x7f2c08113100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:03.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c07fff700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c080ad0e0 0x7f2c080b22e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f2bf0001110 tx=0x7f2bf8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:03.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c057fa700 1 -- 172.21.15.140:0/1138708993 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bf8016070 con 0x7f2c080ad0e0 2022-07-21T01:49:03.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c057fa700 1 -- 172.21.15.140:0/1138708993 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf8002f50 con 0x7f2c080ad0e0 2022-07-21T01:49:03.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c0d336700 1 -- 172.21.15.140:0/1138708993 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c08115ba0 con 0x7f2c080ad0e0 2022-07-21T01:49:03.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c057fa700 1 -- 172.21.15.140:0/1138708993 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bf8012460 con 0x7f2c080ad0e0 2022-07-21T01:49:03.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.439+0000 7f2c0d336700 1 -- 172.21.15.140:0/1138708993 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c080b3100 con 0x7f2c080ad0e0 2022-07-21T01:49:03.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.443+0000 7f2c0d336700 1 -- 172.21.15.140:0/1138708993 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c080b5530 con 0x7f2c080ad0e0 2022-07-21T01:49:03.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.443+0000 7f2c057fa700 1 -- 172.21.15.140:0/1138708993 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2bf800f030 con 0x7f2c080ad0e0 2022-07-21T01:49:03.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.443+0000 7f2c057fa700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bec053a70 0x7f2bec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:03.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.443+0000 7f2c057fa700 1 -- 172.21.15.140:0/1138708993 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2bf806c830 con 0x7f2c080ad0e0 2022-07-21T01:49:03.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.443+0000 7f2c077fe700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bec053a70 0x7f2bec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:03.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.443+0000 7f2c077fe700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bec053a70 0x7f2bec055f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f2bf000d000 tx=0x7f2bf0000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:03.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.447+0000 7f2c057fa700 1 -- 172.21.15.140:0/1138708993 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2bf8043700 con 0x7f2c080ad0e0 2022-07-21T01:49:03.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:03.607+0000 7f2c0d336700 1 -- 172.21.15.140:0/1138708993 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2c080b3100 con 0x7f2c080ad0e0 2022-07-21T01:49:04.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.011+0000 7f2c057fa700 1 -- 172.21.15.140:0/1138708993 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2c080b3100 con 0x7f2c080ad0e0 2022-07-21T01:49:04.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.015+0000 7f2c0d336700 1 -- 172.21.15.140:0/1138708993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bec053a70 msgr2=0x7f2bec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:04.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.015+0000 7f2c0d336700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bec053a70 0x7f2bec055f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f2bf000d000 tx=0x7f2bf0000bc0).stop 2022-07-21T01:49:04.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.015+0000 7f2c0d336700 1 -- 172.21.15.140:0/1138708993 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c080ad0e0 msgr2=0x7f2c080b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:04.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.015+0000 7f2c0d336700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c080ad0e0 0x7f2c080b22e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f2bf0001110 tx=0x7f2bf8014000).stop 2022-07-21T01:49:04.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.019+0000 7f2c0d336700 1 -- 172.21.15.140:0/1138708993 shutdown_connections 2022-07-21T01:49:04.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.019+0000 7f2c0d336700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bec053a70 0x7f2bec055f30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:04.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.019+0000 7f2c0d336700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c080b6800 0x7f2c08113100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:04.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.019+0000 7f2c0d336700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c080b3cb0 0x7f2c08112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:04.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.019+0000 7f2c0d336700 1 --2- 172.21.15.140:0/1138708993 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c080ad0e0 0x7f2c080b22e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:04.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.019+0000 7f2c0d336700 1 -- 172.21.15.140:0/1138708993 >> 172.21.15.140:0/1138708993 conn(0x7f2c080b9dd0 msgr2=0x7f2c081400d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:04.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.019+0000 7f2c0d336700 1 -- 172.21.15.140:0/1138708993 shutdown_connections 2022-07-21T01:49:04.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.019+0000 7f2c0d336700 1 -- 172.21.15.140:0/1138708993 wait complete. 2022-07-21T01:49:04.034 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:49:04.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1cee1f3700 1 Processor -- start 2022-07-21T01:49:04.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1cee1f3700 1 -- start start 2022-07-21T01:49:04.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1cee1f3700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1ce805cf40 0x7f1ce8062f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:04.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1cee1f3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b2be0 0x7f1ce813a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:04.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1cee1f3700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b1e00 con 0x7f1ce80b3880 2022-07-21T01:49:04.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1cee1f3700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b3070 con 0x7f1ce805cf40 2022-07-21T01:49:04.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1cee1f3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8144f40 con 0x7f1ce80b2be0 2022-07-21T01:49:04.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1ce6ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1ce80b3880 0x7f1ce80b24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47608/0 (socket says 172.21.15.140:47608) 2022-07-21T01:49:04.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1ce6ffd700 1 -- 172.21.15.140:0/2776202963 learned_addr learned my addr 172.21.15.140:0/2776202963 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:04.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1ce77fe700 1 --2- 172.21.15.140:0/2776202963 >> v2:172.21.15.152:3300/0 conn(0x7f1ce805cf40 0x7f1ce8062f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:04.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1ce7fff700 1 --2- 172.21.15.140:0/2776202963 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b2be0 0x7f1ce813a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:04.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1ce67fc700 1 -- 172.21.15.140:0/2776202963 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628086051 0 0) 0x7f1ce80b1e00 con 0x7f1ce80b3880 2022-07-21T01:49:04.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1ce67fc700 1 -- 172.21.15.140:0/2776202963 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc8003260 con 0x7f1ce80b3880 2022-07-21T01:49:04.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1ce7fff700 1 -- 172.21.15.140:0/2776202963 >> v2:172.21.15.152:3300/0 conn(0x7f1ce805cf40 msgr2=0x7f1ce8062f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:04.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1ce7fff700 1 --2- 172.21.15.140:0/2776202963 >> v2:172.21.15.152:3300/0 conn(0x7f1ce805cf40 0x7f1ce8062f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:04.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1ce7fff700 1 -- 172.21.15.140:0/2776202963 >> v1:172.21.15.140:6789/0 conn(0x7f1ce80b3880 legacy=0x7f1ce80b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:04.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1ce7fff700 1 -- 172.21.15.140:0/2776202963 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce80add50 con 0x7f1ce80b2be0 2022-07-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.155+0000 7f1ce7fff700 1 --2- 172.21.15.140:0/2776202963 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b2be0 0x7f1ce813a060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1ce00032f0 tx=0x7f1ce00091d0).ready entity=mon.2 client_cookie=eda9c94495160138 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1ce67fc700 1 -- 172.21.15.140:0/2776202963 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ce000b600 con 0x7f1ce80b2be0 2022-07-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1ce67fc700 1 -- 172.21.15.140:0/2776202963 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce000b760 con 0x7f1ce80b2be0 2022-07-21T01:49:04.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1ce67fc700 1 -- 172.21.15.140:0/2776202963 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ce000b8d0 con 0x7f1ce80b2be0 2022-07-21T01:49:04.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2776202963 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b2be0 msgr2=0x7f1ce813a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:04.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1cee1f3700 1 --2- 172.21.15.140:0/2776202963 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b2be0 0x7f1ce813a060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1ce00032f0 tx=0x7f1ce00091d0).stop 2022-07-21T01:49:04.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2776202963 shutdown_connections 2022-07-21T01:49:04.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1cee1f3700 1 --2- 172.21.15.140:0/2776202963 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b2be0 0x7f1ce813a060 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:04.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1cee1f3700 1 --2- 172.21.15.140:0/2776202963 >> v2:172.21.15.152:3300/0 conn(0x7f1ce805cf40 0x7f1ce8062f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:04.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2776202963 >> 172.21.15.140:0/2776202963 conn(0x7f1ce80b7d00 msgr2=0x7f1ce80b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:04.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2776202963 shutdown_connections 2022-07-21T01:49:04.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2776202963 wait complete. 2022-07-21T01:49:04.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1cee1f3700 1 Processor -- start 2022-07-21T01:49:04.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.159+0000 7f1cee1f3700 1 -- start start 2022-07-21T01:49:04.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1cee1f3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ce805cf40 0x7f1ce8146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:04.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1cee1f3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ce80b2be0 0x7f1ce8146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:04.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1cee1f3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b3880 0x7f1ce809c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:04.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1cee1f3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce809b5e0 con 0x7f1ce80b2be0 2022-07-21T01:49:04.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1cee1f3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8150600 con 0x7f1ce805cf40 2022-07-21T01:49:04.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1cee1f3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce809b340 con 0x7f1ce80b3880 2022-07-21T01:49:04.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ce80b2be0 0x7f1ce8146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:04.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ce80b2be0 0x7f1ce8146840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33466/0 (socket says 172.21.15.140:33466) 2022-07-21T01:49:04.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce6ffd700 1 -- 172.21.15.140:0/2200660424 learned_addr learned my addr 172.21.15.140:0/2200660424 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:04.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce7fff700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b3880 0x7f1ce809c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:04.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce77fe700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ce805cf40 0x7f1ce8146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:04.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce6ffd700 1 -- 172.21.15.140:0/2200660424 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b3880 msgr2=0x7f1ce809c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:04.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce6ffd700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b3880 0x7f1ce809c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:04.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce6ffd700 1 -- 172.21.15.140:0/2200660424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ce805cf40 msgr2=0x7f1ce8146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:04.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce6ffd700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ce805cf40 0x7f1ce8146350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:04.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce6ffd700 1 -- 172.21.15.140:0/2200660424 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce0009470 con 0x7f1ce80b2be0 2022-07-21T01:49:04.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce7fff700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b3880 0x7f1ce809c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:04.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce6ffd700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ce80b2be0 0x7f1ce8146840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f1cd0001290 tx=0x7f1cd0007910).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:04.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce4ff9700 1 -- 172.21.15.140:0/2200660424 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cd00070d0 con 0x7f1ce80b2be0 2022-07-21T01:49:04.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce4ff9700 1 -- 172.21.15.140:0/2200660424 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd0007230 con 0x7f1ce80b2be0 2022-07-21T01:49:04.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2200660424 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce8150890 con 0x7f1ce80b2be0 2022-07-21T01:49:04.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce4ff9700 1 -- 172.21.15.140:0/2200660424 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cd00073a0 con 0x7f1ce80b2be0 2022-07-21T01:49:04.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2200660424 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce809a960 con 0x7f1ce80b2be0 2022-07-21T01:49:04.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce4ff9700 1 -- 172.21.15.140:0/2200660424 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1cd001a060 con 0x7f1ce80b2be0 2022-07-21T01:49:04.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2200660424 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce8150a20 con 0x7f1ce80b2be0 2022-07-21T01:49:04.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce4ff9700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ccc053980 0x7f1ccc055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:04.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.163+0000 7f1ce4ff9700 1 -- 172.21.15.140:0/2200660424 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1cd006b250 con 0x7f1ce80b2be0 2022-07-21T01:49:04.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.167+0000 7f1ce77fe700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ccc053980 0x7f1ccc055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:04.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.167+0000 7f1ce77fe700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ccc053980 0x7f1ccc055e40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f1cd800b000 tx=0x7f1cd8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:04.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.171+0000 7f1ce4ff9700 1 -- 172.21.15.140:0/2200660424 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1cd0045af0 con 0x7f1ce80b2be0 2022-07-21T01:49:04.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:04.299+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2200660424 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ce80adff0 con 0x7f1ce80b2be0 2022-07-21T01:49:05.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.055+0000 7f1ce4ff9700 1 -- 172.21.15.140:0/2200660424 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 185+0+0 (secure 0 0 0) 0x7f1cd00180e0 con 0x7f1ce80b2be0 2022-07-21T01:49:05.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2200660424 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ccc053980 msgr2=0x7f1ccc055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:05.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ccc053980 0x7f1ccc055e40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f1cd800b000 tx=0x7f1cd8006df0).stop 2022-07-21T01:49:05.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2200660424 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ce80b2be0 msgr2=0x7f1ce8146840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:05.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ce80b2be0 0x7f1ce8146840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f1cd0001290 tx=0x7f1cd0007910).stop 2022-07-21T01:49:05.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2200660424 shutdown_connections 2022-07-21T01:49:05.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ccc053980 0x7f1ccc055e40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:05.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ce80b3880 0x7f1ce809c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:05.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ce80b2be0 0x7f1ce8146840 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:05.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 --2- 172.21.15.140:0/2200660424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ce805cf40 0x7f1ce8146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:05.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2200660424 >> 172.21.15.140:0/2200660424 conn(0x7f1ce80b7d00 msgr2=0x7f1ce813a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:05.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2200660424 shutdown_connections 2022-07-21T01:49:05.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.059+0000 7f1cee1f3700 1 -- 172.21.15.140:0/2200660424 wait complete. 2022-07-21T01:49:05.062 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:49:05.073 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:49:05.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.199+0000 7f2abccd0700 1 Processor -- start 2022-07-21T01:49:05.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.199+0000 7f2abccd0700 1 -- start start 2022-07-21T01:49:05.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2ab80b6840 0x7f2ab80b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:05.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2ab80b3cf0 0x7f2ab8146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:05.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab8066b10 con 0x7f2ab8146530 2022-07-21T01:49:05.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab80b6c20 con 0x7f2ab80b3cf0 2022-07-21T01:49:05.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab80aead0 con 0x7f2ab80b6840 2022-07-21T01:49:05.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab7fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2ab8146530 0x7f2ab805bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47622/0 (socket says 172.21.15.140:47622) 2022-07-21T01:49:05.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab7fff700 1 -- 172.21.15.140:0/2807756575 learned_addr learned my addr 172.21.15.140:0/2807756575 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:05.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab6ffd700 1 --2- 172.21.15.140:0/2807756575 >> v2:172.21.15.140:3300/0 conn(0x7f2ab80b3cf0 0x7f2ab8146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:05.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab77fe700 1 --2- 172.21.15.140:0/2807756575 >> v2:172.21.15.152:3300/0 conn(0x7f2ab80b6840 0x7f2ab80b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:05.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab67fc700 1 -- 172.21.15.140:0/2807756575 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973217153 0 0) 0x7f2ab8066b10 con 0x7f2ab8146530 2022-07-21T01:49:05.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab67fc700 1 -- 172.21.15.140:0/2807756575 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aa4003260 con 0x7f2ab8146530 2022-07-21T01:49:05.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab77fe700 1 -- 172.21.15.140:0/2807756575 >> v2:172.21.15.140:3300/0 conn(0x7f2ab80b3cf0 msgr2=0x7f2ab8146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:05.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab77fe700 1 --2- 172.21.15.140:0/2807756575 >> v2:172.21.15.140:3300/0 conn(0x7f2ab80b3cf0 0x7f2ab8146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:05.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab77fe700 1 -- 172.21.15.140:0/2807756575 >> v1:172.21.15.140:6789/0 conn(0x7f2ab8146530 legacy=0x7f2ab805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:05.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab77fe700 1 -- 172.21.15.140:0/2807756575 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab80aed70 con 0x7f2ab80b6840 2022-07-21T01:49:05.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab77fe700 1 --2- 172.21.15.140:0/2807756575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ab80b6840 0x7f2ab80b5580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2aa8002f70 tx=0x7f2aa8006df0).ready entity=mon.1 client_cookie=b671a05f61ec0270 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:05.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab67fc700 1 -- 172.21.15.140:0/2807756575 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2aa800d4f0 con 0x7f2ab80b6840 2022-07-21T01:49:05.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab67fc700 1 -- 172.21.15.140:0/2807756575 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa8000e90 con 0x7f2ab80b6840 2022-07-21T01:49:05.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2ab67fc700 1 -- 172.21.15.140:0/2807756575 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2aa800d840 con 0x7f2ab80b6840 2022-07-21T01:49:05.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 -- 172.21.15.140:0/2807756575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ab80b6840 msgr2=0x7f2ab80b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:05.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 --2- 172.21.15.140:0/2807756575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ab80b6840 0x7f2ab80b5580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2aa8002f70 tx=0x7f2aa8006df0).stop 2022-07-21T01:49:05.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 -- 172.21.15.140:0/2807756575 shutdown_connections 2022-07-21T01:49:05.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 --2- 172.21.15.140:0/2807756575 >> v2:172.21.15.140:3300/0 conn(0x7f2ab80b3cf0 0x7f2ab8146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:05.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 --2- 172.21.15.140:0/2807756575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ab80b6840 0x7f2ab80b5580 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:05.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 -- 172.21.15.140:0/2807756575 >> 172.21.15.140:0/2807756575 conn(0x7f2ab80b9e10 msgr2=0x7f2ab80b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:05.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 -- 172.21.15.140:0/2807756575 shutdown_connections 2022-07-21T01:49:05.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 -- 172.21.15.140:0/2807756575 wait complete. 2022-07-21T01:49:05.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 Processor -- start 2022-07-21T01:49:05.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.203+0000 7f2abccd0700 1 -- start start 2022-07-21T01:49:05.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2abccd0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ab80b3cf0 0x7f2ab80b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:05.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2abccd0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ab80b6840 0x7f2ab8112b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:05.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2abccd0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ab8146530 0x7f2ab80b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:05.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2abccd0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab8115530 con 0x7f2ab8146530 2022-07-21T01:49:05.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2abccd0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80b1280 con 0x7f2ab80b3cf0 2022-07-21T01:49:05.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2abccd0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80b1520 con 0x7f2ab80b6840 2022-07-21T01:49:05.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ab8146530 0x7f2ab80b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:05.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ab8146530 0x7f2ab80b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33484/0 (socket says 172.21.15.140:33484) 2022-07-21T01:49:05.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab7fff700 1 -- 172.21.15.140:0/2547859290 learned_addr learned my addr 172.21.15.140:0/2547859290 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:05.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab77fe700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ab80b3cf0 0x7f2ab80b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:05.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab6ffd700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ab80b6840 0x7f2ab8112b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:05.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab7fff700 1 -- 172.21.15.140:0/2547859290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ab80b6840 msgr2=0x7f2ab8112b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:05.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab7fff700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ab80b6840 0x7f2ab8112b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:05.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab7fff700 1 -- 172.21.15.140:0/2547859290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ab80b3cf0 msgr2=0x7f2ab80b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:05.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab7fff700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ab80b3cf0 0x7f2ab80b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:05.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab7fff700 1 -- 172.21.15.140:0/2547859290 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa800b180 con 0x7f2ab8146530 2022-07-21T01:49:05.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab7fff700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ab8146530 0x7f2ab80b19c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f2ab0001490 tx=0x7f2ab0007d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:05.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab4ff9700 1 -- 172.21.15.140:0/2547859290 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ab0005ba0 con 0x7f2ab8146530 2022-07-21T01:49:05.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab4ff9700 1 -- 172.21.15.140:0/2547859290 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab0005d00 con 0x7f2ab8146530 2022-07-21T01:49:05.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2abccd0700 1 -- 172.21.15.140:0/2547859290 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab8115220 con 0x7f2ab8146530 2022-07-21T01:49:05.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab4ff9700 1 -- 172.21.15.140:0/2547859290 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ab00072c0 con 0x7f2ab8146530 2022-07-21T01:49:05.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2abccd0700 1 -- 172.21.15.140:0/2547859290 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab8113030 con 0x7f2ab8146530 2022-07-21T01:49:05.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2abccd0700 1 -- 172.21.15.140:0/2547859290 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab80b16e0 con 0x7f2ab8146530 2022-07-21T01:49:05.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab4ff9700 1 -- 172.21.15.140:0/2547859290 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2ab0010070 con 0x7f2ab8146530 2022-07-21T01:49:05.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab4ff9700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2a980539e0 0x7f2a98055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:05.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab4ff9700 1 -- 172.21.15.140:0/2547859290 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f2ab006b6c0 con 0x7f2ab8146530 2022-07-21T01:49:05.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.207+0000 7f2ab77fe700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2a980539e0 0x7f2a98055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:05.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.211+0000 7f2ab77fe700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2a980539e0 0x7f2a98055ea0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f2aa801a000 tx=0x7f2aa8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:05.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.211+0000 7f2ab4ff9700 1 -- 172.21.15.140:0/2547859290 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2ab00426a0 con 0x7f2ab8146530 2022-07-21T01:49:05.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:05.339+0000 7f2abccd0700 1 -- 172.21.15.140:0/2547859290 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2ab80574e0 con 0x7f2ab8146530 2022-07-21T01:49:06.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.067+0000 7f2ab4ff9700 1 -- 172.21.15.140:0/2547859290 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2ab0045cc0 con 0x7f2ab8146530 2022-07-21T01:49:06.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.067+0000 7f2abccd0700 1 -- 172.21.15.140:0/2547859290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2a980539e0 msgr2=0x7f2a98055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:06.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.067+0000 7f2abccd0700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2a980539e0 0x7f2a98055ea0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f2aa801a000 tx=0x7f2aa8014000).stop 2022-07-21T01:49:06.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.067+0000 7f2abccd0700 1 -- 172.21.15.140:0/2547859290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ab8146530 msgr2=0x7f2ab80b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:06.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.067+0000 7f2abccd0700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ab8146530 0x7f2ab80b19c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f2ab0001490 tx=0x7f2ab0007d60).stop 2022-07-21T01:49:06.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.067+0000 7f2abccd0700 1 -- 172.21.15.140:0/2547859290 shutdown_connections 2022-07-21T01:49:06.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.067+0000 7f2abccd0700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2a980539e0 0x7f2a98055ea0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:06.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.067+0000 7f2abccd0700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ab8146530 0x7f2ab80b19c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:06.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.071+0000 7f2abccd0700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ab80b6840 0x7f2ab8112b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:06.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.071+0000 7f2abccd0700 1 --2- 172.21.15.140:0/2547859290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ab80b3cf0 0x7f2ab80b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:06.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.071+0000 7f2abccd0700 1 -- 172.21.15.140:0/2547859290 >> 172.21.15.140:0/2547859290 conn(0x7f2ab80b9e10 msgr2=0x7f2ab8140690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:06.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.071+0000 7f2abccd0700 1 -- 172.21.15.140:0/2547859290 shutdown_connections 2022-07-21T01:49:06.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.071+0000 7f2abccd0700 1 -- 172.21.15.140:0/2547859290 wait complete. 2022-07-21T01:49:06.072 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:49:06.084 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:49:06.085 INFO:tasks.ceph.mds.a:Stopping old one... 2022-07-21T01:49:06.087 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:49:06.087 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:49:06.088 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:49:06.089 INFO:tasks.ceph.mds.b:Stopping old one... 2022-07-21T01:49:06.089 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:49:06.092 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:49:06.102 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:49:06.103 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:49:06.105 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:49:06.105 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:49:06.107 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:49:06.107 DEBUG:teuthology.parallel:result is None 2022-07-21T01:49:06.110 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:49:06.111 DEBUG:teuthology.parallel:result is None 2022-07-21T01:49:06.111 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:49:06.176 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:49:06.184 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:49:06.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.215+0000 7f9018e75700 1 Processor -- start 2022-07-21T01:49:06.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.215+0000 7f9018e75700 1 -- start start 2022-07-21T01:49:06.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f90140b35c0 0x7f90140abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:06.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140ac470 0x7f9014061b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:06.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90140aba30 con 0x7f9014098ae0 2022-07-21T01:49:06.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90140b39a0 con 0x7f90140b35c0 2022-07-21T01:49:06.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901409c000 con 0x7f90140ac470 2022-07-21T01:49:06.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f900bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f9014098ae0 0x7f9014061440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47646/0 (socket says 172.21.15.140:47646) 2022-07-21T01:49:06.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f900bfff700 1 -- 172.21.15.140:0/3298480635 learned_addr learned my addr 172.21.15.140:0/3298480635 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:06.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f90137fe700 1 --2- 172.21.15.140:0/3298480635 >> v2:172.21.15.152:3300/0 conn(0x7f90140b35c0 0x7f90140abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:06.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9013fff700 1 --2- 172.21.15.140:0/3298480635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140ac470 0x7f9014061b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:06.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9012ffd700 1 -- 172.21.15.140:0/3298480635 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699734180 0 0) 0x7f90140aba30 con 0x7f9014098ae0 2022-07-21T01:49:06.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9012ffd700 1 -- 172.21.15.140:0/3298480635 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ff4003260 con 0x7f9014098ae0 2022-07-21T01:49:06.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9013fff700 1 -- 172.21.15.140:0/3298480635 >> v2:172.21.15.152:3300/0 conn(0x7f90140b35c0 msgr2=0x7f90140abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:06.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9013fff700 1 --2- 172.21.15.140:0/3298480635 >> v2:172.21.15.152:3300/0 conn(0x7f90140b35c0 0x7f90140abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:06.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9013fff700 1 -- 172.21.15.140:0/3298480635 >> v1:172.21.15.140:6789/0 conn(0x7f9014098ae0 legacy=0x7f9014061440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:06.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9013fff700 1 -- 172.21.15.140:0/3298480635 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90140acbc0 con 0x7f90140ac470 2022-07-21T01:49:06.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9013fff700 1 --2- 172.21.15.140:0/3298480635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140ac470 0x7f9014061b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f900c0032f0 tx=0x7f900c0091d0).ready entity=mon.2 client_cookie=795708479c670186 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:06.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9012ffd700 1 -- 172.21.15.140:0/3298480635 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f900c00b600 con 0x7f90140ac470 2022-07-21T01:49:06.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9012ffd700 1 -- 172.21.15.140:0/3298480635 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c00b760 con 0x7f90140ac470 2022-07-21T01:49:06.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9012ffd700 1 -- 172.21.15.140:0/3298480635 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f900c00b8d0 con 0x7f90140ac470 2022-07-21T01:49:06.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 -- 172.21.15.140:0/3298480635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140ac470 msgr2=0x7f9014061b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:06.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 --2- 172.21.15.140:0/3298480635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140ac470 0x7f9014061b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f900c0032f0 tx=0x7f900c0091d0).stop 2022-07-21T01:49:06.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 -- 172.21.15.140:0/3298480635 shutdown_connections 2022-07-21T01:49:06.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 --2- 172.21.15.140:0/3298480635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140ac470 0x7f9014061b50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:06.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 --2- 172.21.15.140:0/3298480635 >> v2:172.21.15.152:3300/0 conn(0x7f90140b35c0 0x7f90140abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:06.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 -- 172.21.15.140:0/3298480635 >> 172.21.15.140:0/3298480635 conn(0x7f90140b6b20 msgr2=0x7f90140b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:06.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 -- 172.21.15.140:0/3298480635 shutdown_connections 2022-07-21T01:49:06.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.219+0000 7f9018e75700 1 -- 172.21.15.140:0/3298480635 wait complete. 2022-07-21T01:49:06.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.223+0000 7f9018e75700 1 Processor -- start 2022-07-21T01:49:06.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.223+0000 7f9018e75700 1 -- start start 2022-07-21T01:49:06.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9018e75700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9014098ae0 0x7f90140ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:06.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9018e75700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90140ac470 0x7f9014065900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:06.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9018e75700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b35c0 0x7f9014066130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:06.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9018e75700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9014067910 con 0x7f90140b35c0 2022-07-21T01:49:06.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9018e75700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90140651c0 con 0x7f90140ac470 2022-07-21T01:49:06.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9018e75700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9014065460 con 0x7f9014098ae0 2022-07-21T01:49:06.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9013fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b35c0 0x7f9014066130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:06.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9013fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b35c0 0x7f9014066130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33508/0 (socket says 172.21.15.140:33508) 2022-07-21T01:49:06.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9013fff700 1 -- 172.21.15.140:0/2020420277 learned_addr learned my addr 172.21.15.140:0/2020420277 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:06.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f900bfff700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90140ac470 0x7f9014065900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:06.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f90137fe700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9014098ae0 0x7f90140ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:06.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9013fff700 1 -- 172.21.15.140:0/2020420277 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9014098ae0 msgr2=0x7f90140ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:06.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9013fff700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9014098ae0 0x7f90140ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:06.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9013fff700 1 -- 172.21.15.140:0/2020420277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90140ac470 msgr2=0x7f9014065900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:06.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9013fff700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90140ac470 0x7f9014065900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:06.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9013fff700 1 -- 172.21.15.140:0/2020420277 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c009470 con 0x7f90140b35c0 2022-07-21T01:49:06.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f900bfff700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90140ac470 0x7f9014065900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:06.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9013fff700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b35c0 0x7f9014066130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f900c0032f0 tx=0x7f900c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:06.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f90117fa700 1 -- 172.21.15.140:0/2020420277 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f900c00b8d0 con 0x7f90140b35c0 2022-07-21T01:49:06.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9018e75700 1 -- 172.21.15.140:0/2020420277 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9014065df0 con 0x7f90140b35c0 2022-07-21T01:49:06.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f90117fa700 1 -- 172.21.15.140:0/2020420277 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c01f3f0 con 0x7f90140b35c0 2022-07-21T01:49:06.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f90117fa700 1 -- 172.21.15.140:0/2020420277 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f900c00b600 con 0x7f90140b35c0 2022-07-21T01:49:06.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9018e75700 1 -- 172.21.15.140:0/2020420277 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9014066620 con 0x7f90140b35c0 2022-07-21T01:49:06.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.227+0000 7f9018e75700 1 -- 172.21.15.140:0/2020420277 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9014065620 con 0x7f90140b35c0 2022-07-21T01:49:06.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.231+0000 7f90117fa700 1 -- 172.21.15.140:0/2020420277 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f900c026030 con 0x7f90140b35c0 2022-07-21T01:49:06.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.231+0000 7f90117fa700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff8053aa0 0x7f8ff8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:06.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.231+0000 7f90117fa700 1 -- 172.21.15.140:0/2020420277 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f900c06c8b0 con 0x7f90140b35c0 2022-07-21T01:49:06.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.231+0000 7f90137fe700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff8053aa0 0x7f8ff8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:06.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.231+0000 7f90137fe700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff8053aa0 0x7f8ff8055f60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f900400a000 tx=0x7f9004006e80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:06.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.235+0000 7f90117fa700 1 -- 172.21.15.140:0/2020420277 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f900c043920 con 0x7f90140b35c0 2022-07-21T01:49:06.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:06.374+0000 7f9018e75700 1 -- 172.21.15.140:0/2020420277 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f90140ace60 con 0x7f90140b35c0 2022-07-21T01:49:07.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.074+0000 7f90117fa700 1 -- 172.21.15.140:0/2020420277 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7f900c024080 con 0x7f90140b35c0 2022-07-21T01:49:07.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 -- 172.21.15.140:0/2020420277 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff8053aa0 msgr2=0x7f8ff8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff8053aa0 0x7f8ff8055f60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f900400a000 tx=0x7f9004006e80).stop 2022-07-21T01:49:07.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 -- 172.21.15.140:0/2020420277 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b35c0 msgr2=0x7f9014066130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b35c0 0x7f9014066130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f900c0032f0 tx=0x7f900c016000).stop 2022-07-21T01:49:07.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 -- 172.21.15.140:0/2020420277 shutdown_connections 2022-07-21T01:49:07.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff8053aa0 0x7f8ff8055f60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b35c0 0x7f9014066130 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90140ac470 0x7f9014065900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 --2- 172.21.15.140:0/2020420277 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9014098ae0 0x7f90140ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 -- 172.21.15.140:0/2020420277 >> 172.21.15.140:0/2020420277 conn(0x7f90140b6b20 msgr2=0x7f90140b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:07.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 -- 172.21.15.140:0/2020420277 shutdown_connections 2022-07-21T01:49:07.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.078+0000 7f9018e75700 1 -- 172.21.15.140:0/2020420277 wait complete. 2022-07-21T01:49:07.085 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:49:07.093 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:49:07.094 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:49:07.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.194+0000 7f8a8c9f8700 1 Processor -- start 2022-07-21T01:49:07.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.194+0000 7f8a8c9f8700 1 -- start start 2022-07-21T01:49:07.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8a880b3990 0x7f8a8805d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a88142a50 0x7f8a880b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8a8813f9d0 0x7f8a8813fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a880b2b90 con 0x7f8a8813f9d0 2022-07-21T01:49:07.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a880b4410 con 0x7f8a880b3990 2022-07-21T01:49:07.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8805c090 con 0x7f8a88142a50 2022-07-21T01:49:07.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a87fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8a8813f9d0 0x7f8a8813fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a87fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8a8813f9d0 0x7f8a8813fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33516/0 (socket says 172.21.15.140:33516) 2022-07-21T01:49:07.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a87fff700 1 -- 172.21.15.140:0/2047269283 learned_addr learned my addr 172.21.15.140:0/2047269283 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:07.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a7ffff700 1 --2- 172.21.15.140:0/2047269283 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a88142a50 0x7f8a880b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a877fe700 1 --2- 172.21.15.140:0/2047269283 >> v2:172.21.15.152:3300/0 conn(0x7f8a880b3990 0x7f8a8805d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a87fff700 1 -- 172.21.15.140:0/2047269283 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a88142a50 msgr2=0x7f8a880b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:07.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a87fff700 1 --2- 172.21.15.140:0/2047269283 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a88142a50 0x7f8a880b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a87fff700 1 -- 172.21.15.140:0/2047269283 >> v2:172.21.15.152:3300/0 conn(0x7f8a880b3990 msgr2=0x7f8a8805d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:07.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a87fff700 1 --2- 172.21.15.140:0/2047269283 >> v2:172.21.15.152:3300/0 conn(0x7f8a880b3990 0x7f8a8805d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a87fff700 1 -- 172.21.15.140:0/2047269283 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a880b4170 con 0x7f8a8813f9d0 2022-07-21T01:49:07.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a87fff700 1 --2- 172.21.15.140:0/2047269283 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8a8813f9d0 0x7f8a8813fe40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f8a800032f0 tx=0x7f8a80009290).ready entity=mon.0 client_cookie=3fdab23f5e6ec9a7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:07.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a86ffd700 1 -- 172.21.15.140:0/2047269283 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8000b680 con 0x7f8a8813f9d0 2022-07-21T01:49:07.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a86ffd700 1 -- 172.21.15.140:0/2047269283 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8000b7e0 con 0x7f8a8813f9d0 2022-07-21T01:49:07.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a86ffd700 1 -- 172.21.15.140:0/2047269283 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8000b950 con 0x7f8a8813f9d0 2022-07-21T01:49:07.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/2047269283 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8a8813f9d0 msgr2=0x7f8a8813fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 --2- 172.21.15.140:0/2047269283 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8a8813f9d0 0x7f8a8813fe40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f8a800032f0 tx=0x7f8a80009290).stop 2022-07-21T01:49:07.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/2047269283 shutdown_connections 2022-07-21T01:49:07.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 --2- 172.21.15.140:0/2047269283 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8a8813f9d0 0x7f8a8813fe40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 --2- 172.21.15.140:0/2047269283 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a88142a50 0x7f8a880b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 --2- 172.21.15.140:0/2047269283 >> v2:172.21.15.152:3300/0 conn(0x7f8a880b3990 0x7f8a8805d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/2047269283 >> 172.21.15.140:0/2047269283 conn(0x7f8a88064be0 msgr2=0x7f8a88064fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:07.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/2047269283 shutdown_connections 2022-07-21T01:49:07.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.198+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/2047269283 wait complete. 2022-07-21T01:49:07.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a8c9f8700 1 Processor -- start 2022-07-21T01:49:07.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a8c9f8700 1 -- start start 2022-07-21T01:49:07.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a8c9f8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a880b3990 0x7f8a880aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a8c9f8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a8813f9d0 0x7f8a88146250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a8c9f8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8a88142a50 0x7f8a8813c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a8c9f8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880af000 con 0x7f8a88142a50 2022-07-21T01:49:07.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a8c9f8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a88067660 con 0x7f8a880b3990 2022-07-21T01:49:07.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a8c9f8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880678c0 con 0x7f8a8813f9d0 2022-07-21T01:49:07.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a87fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8a88142a50 0x7f8a8813c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a87fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8a88142a50 0x7f8a8813c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33522/0 (socket says 172.21.15.140:33522) 2022-07-21T01:49:07.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a87fff700 1 -- 172.21.15.140:0/4105929486 learned_addr learned my addr 172.21.15.140:0/4105929486 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:07.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a877fe700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a880b3990 0x7f8a880aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a7ffff700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a8813f9d0 0x7f8a88146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a87fff700 1 -- 172.21.15.140:0/4105929486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a8813f9d0 msgr2=0x7f8a88146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a87fff700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a8813f9d0 0x7f8a88146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a87fff700 1 -- 172.21.15.140:0/4105929486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a880b3990 msgr2=0x7f8a880aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a87fff700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a880b3990 0x7f8a880aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a87fff700 1 -- 172.21.15.140:0/4105929486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a80009530 con 0x7f8a88142a50 2022-07-21T01:49:07.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a87fff700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8a88142a50 0x7f8a8813c9a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f8a800032f0 tx=0x7f8a80017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:07.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a857fa700 1 -- 172.21.15.140:0/4105929486 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8000b680 con 0x7f8a88142a50 2022-07-21T01:49:07.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a857fa700 1 -- 172.21.15.140:0/4105929486 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8000bea0 con 0x7f8a88142a50 2022-07-21T01:49:07.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/4105929486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a88067b60 con 0x7f8a88142a50 2022-07-21T01:49:07.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a857fa700 1 -- 172.21.15.140:0/4105929486 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a8001b600 con 0x7f8a88142a50 2022-07-21T01:49:07.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.202+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/4105929486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a881553c0 con 0x7f8a88142a50 2022-07-21T01:49:07.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.206+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/4105929486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a880b05b0 con 0x7f8a88142a50 2022-07-21T01:49:07.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.206+0000 7f8a857fa700 1 -- 172.21.15.140:0/4105929486 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8a8000f070 con 0x7f8a88142a50 2022-07-21T01:49:07.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.206+0000 7f8a857fa700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8a6c053a70 0x7f8a6c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.206+0000 7f8a857fa700 1 -- 172.21.15.140:0/4105929486 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f8a8006d380 con 0x7f8a88142a50 2022-07-21T01:49:07.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.206+0000 7f8a877fe700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8a6c053a70 0x7f8a6c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.206+0000 7f8a877fe700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8a6c053a70 0x7f8a6c055f30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f8a7800b000 tx=0x7f8a78006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:07.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.214+0000 7f8a857fa700 1 -- 172.21.15.140:0/4105929486 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8a80044500 con 0x7f8a88142a50 2022-07-21T01:49:07.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.378+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/4105929486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8a880574e0 con 0x7f8a88142a50 2022-07-21T01:49:07.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.378+0000 7f8a857fa700 1 -- 172.21.15.140:0/4105929486 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+5627 (secure 0 0 0) 0x7f8a8001f020 con 0x7f8a88142a50 2022-07-21T01:49:07.380 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:07.380 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:49:07.380 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:49:07.381 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.381 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:49:07.381 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:49:07.382 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.382 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:49:07.382 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:49:07.382 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:49:07.383 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.383 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.383 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.383 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.383 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:49:07.384 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:49:07.384 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.384 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:49:07.385 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:49:07.385 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:49:07.385 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.390 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.391 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.392 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.392 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:49:07.393 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:49:07.394 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.394 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:49:07.394 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:49:07.395 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.396 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.396 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.396 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.397 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:49:07.397 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:49:07.398 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.398 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:49:07.399 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:49:07.399 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.400 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.400 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.401 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.401 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:49:07.401 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:49:07.402 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.402 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:49:07.403 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:49:07.403 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.404 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.405 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.406 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.406 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:49:07.407 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:49:07.407 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.407 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:49:07.408 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:49:07.408 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.408 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.408 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.409 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.409 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:49:07.409 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:49:07.409 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.410 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:49:07.410 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:49:07.410 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.410 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.411 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.411 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.411 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:49:07.411 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:49:07.412 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.412 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:49:07.412 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:49:07.412 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.413 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.413 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.413 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.414 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:49:07.414 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:49:07.414 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.414 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:49:07.415 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:49:07.415 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.415 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.415 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.415 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.416 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:49:07.416 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:49:07.416 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.416 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:49:07.417 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:49:07.417 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.417 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.417 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.418 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.418 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:49:07.418 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:49:07.418 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.419 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:49:07.419 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow r", 2022-07-21T01:49:07.419 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow rw", 2022-07-21T01:49:07.419 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rwx" 2022-07-21T01:49:07.420 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.420 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.420 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.420 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:49:07.420 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:49:07.421 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.421 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:49:07.421 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow r", 2022-07-21T01:49:07.421 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow rw", 2022-07-21T01:49:07.422 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rwx" 2022-07-21T01:49:07.422 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.422 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.422 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.423 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:49:07.423 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:49:07.423 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.423 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:49:07.424 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow r", 2022-07-21T01:49:07.424 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow rw", 2022-07-21T01:49:07.424 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rwx" 2022-07-21T01:49:07.424 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.425 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.425 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.425 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:49:07.425 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:49:07.425 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.426 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:49:07.426 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:49:07.426 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:49:07.426 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.427 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.427 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.427 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.427 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.bootstrap-mds", 2022-07-21T01:49:07.427 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBfsNhi9xXlBhAAKhUY9VBqIC6FKEEPs3tYxw==", 2022-07-21T01:49:07.428 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.428 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile bootstrap-mds" 2022-07-21T01:49:07.428 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.428 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.428 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.429 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.bootstrap-mgr", 2022-07-21T01:49:07.429 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBfsNhiQSjlBhAA+now8tpXa8OT98tz9RZ0qA==", 2022-07-21T01:49:07.429 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.429 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile bootstrap-mgr" 2022-07-21T01:49:07.430 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.430 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.430 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.430 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.bootstrap-osd", 2022-07-21T01:49:07.430 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBfsNhiXDjlBhAAh+U5LckW//H6fSaqIIP/5A==", 2022-07-21T01:49:07.431 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.431 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile bootstrap-osd" 2022-07-21T01:49:07.431 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.431 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.432 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.432 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.bootstrap-rbd", 2022-07-21T01:49:07.432 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBfsNhiJEjlBhAAHKAakle5TfNMz1VzknRRQw==", 2022-07-21T01:49:07.432 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.432 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile bootstrap-rbd" 2022-07-21T01:49:07.433 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.433 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.433 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.433 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.bootstrap-rbd-mirror", 2022-07-21T01:49:07.433 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBfsNhieVnlBhAAHk6JCc5htBwMZr3NoPSg2g==", 2022-07-21T01:49:07.434 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.434 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2022-07-21T01:49:07.434 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.434 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.435 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.435 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.bootstrap-rgw", 2022-07-21T01:49:07.435 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBfsNhif2rlBhAAZL/RvvISUkKTpmIeyVRfZQ==", 2022-07-21T01:49:07.435 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.435 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile bootstrap-rgw" 2022-07-21T01:49:07.436 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.436 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.436 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.436 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:49:07.437 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:49:07.437 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.438 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:49:07.438 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:49:07.438 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.438 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.439 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:07.439 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:07.439 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:49:07.439 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:49:07.439 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:49:07.440 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:49:07.440 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:49:07.440 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:49:07.440 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.440 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:07.441 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:49:07.441 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:49:07.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/4105929486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8a6c053a70 msgr2=0x7f8a6c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8a6c053a70 0x7f8a6c055f30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f8a7800b000 tx=0x7f8a78006df0).stop 2022-07-21T01:49:07.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/4105929486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8a88142a50 msgr2=0x7f8a8813c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8a88142a50 0x7f8a8813c9a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f8a800032f0 tx=0x7f8a80017000).stop 2022-07-21T01:49:07.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/4105929486 shutdown_connections 2022-07-21T01:49:07.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8a6c053a70 0x7f8a6c055f30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8a88142a50 0x7f8a8813c9a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a8813f9d0 0x7f8a88146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 --2- 172.21.15.140:0/4105929486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a880b3990 0x7f8a880aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/4105929486 >> 172.21.15.140:0/4105929486 conn(0x7f8a88064be0 msgr2=0x7f8a8805c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:07.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/4105929486 shutdown_connections 2022-07-21T01:49:07.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.382+0000 7f8a8c9f8700 1 -- 172.21.15.140:0/4105929486 wait complete. 2022-07-21T01:49:07.446 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2022-07-21T01:49:07.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4bffff700 1 Processor -- start 2022-07-21T01:49:07.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4bffff700 1 -- start start 2022-07-21T01:49:07.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4bffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc4c00af8c0 0x7fc4c00ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4bffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc4c0143a10 0x7fc4c0143e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4bffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c00b0b80 con 0x7fc4c00b5970 2022-07-21T01:49:07.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4bffff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c005d4d0 con 0x7fc4c00af8c0 2022-07-21T01:49:07.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4bffff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c00b5d50 con 0x7fc4c0143a10 2022-07-21T01:49:07.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4beffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc4c00b5970 0x7fc4c00ae100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47672/0 (socket says 172.21.15.140:47672) 2022-07-21T01:49:07.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4be7fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc4c00af8c0 0x7fc4c00ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4beffd700 1 -- 172.21.15.140:0/2360603940 learned_addr learned my addr 172.21.15.140:0/2360603940 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:07.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4bf7fe700 1 --2- 172.21.15.140:0/2360603940 >> v2:172.21.15.152:3300/0 conn(0x7fc4c0143a10 0x7fc4c0143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4be7fc700 1 -- 172.21.15.140:0/2360603940 >> v2:172.21.15.152:3300/0 conn(0x7fc4c0143a10 msgr2=0x7fc4c0143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:07.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4be7fc700 1 --2- 172.21.15.140:0/2360603940 >> v2:172.21.15.152:3300/0 conn(0x7fc4c0143a10 0x7fc4c0143e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4be7fc700 1 -- 172.21.15.140:0/2360603940 >> v1:172.21.15.140:6789/0 conn(0x7fc4c00b5970 legacy=0x7fc4c00ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4be7fc700 1 -- 172.21.15.140:0/2360603940 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c00b0db0 con 0x7fc4c00af8c0 2022-07-21T01:49:07.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4be7fc700 1 --2- 172.21.15.140:0/2360603940 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4c00af8c0 0x7fc4c00ae810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fc4b8001490 tx=0x7fc4b4006df0).ready entity=mon.0 client_cookie=c7bede72690441fa server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:07.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4bdffb700 1 -- 172.21.15.140:0/2360603940 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4b400d4a0 con 0x7fc4c00af8c0 2022-07-21T01:49:07.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4bdffb700 1 -- 172.21.15.140:0/2360603940 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b4000e90 con 0x7fc4c00af8c0 2022-07-21T01:49:07.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.554+0000 7fc4bdffb700 1 -- 172.21.15.140:0/2360603940 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4b400d7f0 con 0x7fc4c00af8c0 2022-07-21T01:49:07.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.558+0000 7fc4bffff700 1 -- 172.21.15.140:0/2360603940 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4c00af8c0 msgr2=0x7fc4c00ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.558+0000 7fc4bffff700 1 --2- 172.21.15.140:0/2360603940 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4c00af8c0 0x7fc4c00ae810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fc4b8001490 tx=0x7fc4b4006df0).stop 2022-07-21T01:49:07.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.558+0000 7fc4bffff700 1 -- 172.21.15.140:0/2360603940 shutdown_connections 2022-07-21T01:49:07.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.558+0000 7fc4bffff700 1 --2- 172.21.15.140:0/2360603940 >> v2:172.21.15.152:3300/0 conn(0x7fc4c0143a10 0x7fc4c0143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.558+0000 7fc4bffff700 1 --2- 172.21.15.140:0/2360603940 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4c00af8c0 0x7fc4c00ae810 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.558+0000 7fc4bffff700 1 -- 172.21.15.140:0/2360603940 >> 172.21.15.140:0/2360603940 conn(0x7fc4c0063490 msgr2=0x7fc4c00b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:07.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.558+0000 7fc4bffff700 1 -- 172.21.15.140:0/2360603940 shutdown_connections 2022-07-21T01:49:07.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.558+0000 7fc4bffff700 1 -- 172.21.15.140:0/2360603940 wait complete. 2022-07-21T01:49:07.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.558+0000 7fc4bffff700 1 Processor -- start 2022-07-21T01:49:07.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.558+0000 7fc4bffff700 1 -- start start 2022-07-21T01:49:07.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4c00af8c0 0x7fc4c0149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4c00b5970 0x7fc4c01404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4c0143a10 0x7fc4c0142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4bffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c01461f0 con 0x7fc4c00af8c0 2022-07-21T01:49:07.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4bffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c015a160 con 0x7fc4c00b5970 2022-07-21T01:49:07.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4bffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c015a3d0 con 0x7fc4c0143a10 2022-07-21T01:49:07.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4beffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4c00af8c0 0x7fc4c0149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4beffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4c00af8c0 0x7fc4c0149770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33534/0 (socket says 172.21.15.140:33534) 2022-07-21T01:49:07.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4beffd700 1 -- 172.21.15.140:0/235012319 learned_addr learned my addr 172.21.15.140:0/235012319 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:07.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4bf7fe700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4c0143a10 0x7fc4c0142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4be7fc700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4c00b5970 0x7fc4c01404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4beffd700 1 -- 172.21.15.140:0/235012319 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4c0143a10 msgr2=0x7fc4c0142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4beffd700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4c0143a10 0x7fc4c0142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4beffd700 1 -- 172.21.15.140:0/235012319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4c00b5970 msgr2=0x7fc4c01404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4beffd700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4c00b5970 0x7fc4c01404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4beffd700 1 -- 172.21.15.140:0/235012319 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b400b180 con 0x7fc4c00af8c0 2022-07-21T01:49:07.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4be7fc700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4c00b5970 0x7fc4c01404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:07.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4beffd700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4c00af8c0 0x7fc4c0149770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fc4c015a2f0 tx=0x7fc4b800cb20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:07.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4affff700 1 -- 172.21.15.140:0/235012319 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4b8005ba0 con 0x7fc4c00af8c0 2022-07-21T01:49:07.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4affff700 1 -- 172.21.15.140:0/235012319 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b8005d00 con 0x7fc4c00af8c0 2022-07-21T01:49:07.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4affff700 1 -- 172.21.15.140:0/235012319 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4b8007300 con 0x7fc4c00af8c0 2022-07-21T01:49:07.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4bffff700 1 -- 172.21.15.140:0/235012319 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c015a670 con 0x7fc4c00af8c0 2022-07-21T01:49:07.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.562+0000 7fc4bffff700 1 -- 172.21.15.140:0/235012319 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c01463b0 con 0x7fc4c00af8c0 2022-07-21T01:49:07.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.566+0000 7fc4bffff700 1 -- 172.21.15.140:0/235012319 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c01437c0 con 0x7fc4c00af8c0 2022-07-21T01:49:07.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.566+0000 7fc4affff700 1 -- 172.21.15.140:0/235012319 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc4b8010030 con 0x7fc4c00af8c0 2022-07-21T01:49:07.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.566+0000 7fc4affff700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc49c053a70 0x7fc49c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.566+0000 7fc4affff700 1 -- 172.21.15.140:0/235012319 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc4b806be20 con 0x7fc4c00af8c0 2022-07-21T01:49:07.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.566+0000 7fc4be7fc700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc49c053a70 0x7fc49c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.566+0000 7fc4be7fc700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc49c053a70 0x7fc49c055f30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc4b401a000 tx=0x7fc4b4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:07.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.570+0000 7fc4affff700 1 -- 172.21.15.140:0/235012319 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc4c01437c0 con 0x7fc4c00af8c0 2022-07-21T01:49:07.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.726+0000 7fc4bffff700 1 -- 172.21.15.140:0/235012319 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fc4c01437c0 con 0x7fc4c00af8c0 2022-07-21T01:49:07.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.734+0000 7fc4affff700 1 -- 172.21.15.140:0/235012319 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc4c01437c0 con 0x7fc4c00af8c0 2022-07-21T01:49:07.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 -- 172.21.15.140:0/235012319 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc49c053a70 msgr2=0x7fc49c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc49c053a70 0x7fc49c055f30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc4b401a000 tx=0x7fc4b4014000).stop 2022-07-21T01:49:07.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 -- 172.21.15.140:0/235012319 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4c00af8c0 msgr2=0x7fc4c0149770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4c00af8c0 0x7fc4c0149770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fc4c015a2f0 tx=0x7fc4b800cb20).stop 2022-07-21T01:49:07.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 -- 172.21.15.140:0/235012319 shutdown_connections 2022-07-21T01:49:07.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc49c053a70 0x7fc49c055f30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4c0143a10 0x7fc4c0142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4c00b5970 0x7fc4c01404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 --2- 172.21.15.140:0/235012319 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4c00af8c0 0x7fc4c0149770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 -- 172.21.15.140:0/235012319 >> 172.21.15.140:0/235012319 conn(0x7fc4c0063490 msgr2=0x7fc4c0147030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:07.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 -- 172.21.15.140:0/235012319 shutdown_connections 2022-07-21T01:49:07.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.738+0000 7fc4bffff700 1 -- 172.21.15.140:0/235012319 wait complete. 2022-07-21T01:49:07.747 INFO:teuthology.orchestra.run.smithi140.stderr:updated 2022-07-21T01:49:07.750 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2022-07-21T01:49:07.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.850+0000 7f2465790700 1 Processor -- start 2022-07-21T01:49:07.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.850+0000 7f2465790700 1 -- start start 2022-07-21T01:49:07.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.850+0000 7f2465790700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2460097a80 0x7f2460061080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.850+0000 7f2465790700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f24600afb80 0x7f246014b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.850+0000 7f2465790700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246014a750 con 0x7f2460060ca0 2022-07-21T01:49:07.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.850+0000 7f2465790700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2460148430 con 0x7f24600afb80 2022-07-21T01:49:07.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.850+0000 7f2465790700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24600b2840 con 0x7f2460097a80 2022-07-21T01:49:07.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.850+0000 7f245effd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2460060ca0 0x7f246014ab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47686/0 (socket says 172.21.15.140:47686) 2022-07-21T01:49:07.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.850+0000 7f245effd700 1 -- 172.21.15.140:0/1520691795 learned_addr learned my addr 172.21.15.140:0/1520691795 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:07.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245e7fc700 1 --2- 172.21.15.140:0/1520691795 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2460097a80 0x7f2460061080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245f7fe700 1 --2- 172.21.15.140:0/1520691795 >> v2:172.21.15.152:3300/0 conn(0x7f24600afb80 0x7f246014b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245dffb700 1 -- 172.21.15.140:0/1520691795 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498850585 0 0) 0x7f246014a750 con 0x7f2460060ca0 2022-07-21T01:49:07.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245dffb700 1 -- 172.21.15.140:0/1520691795 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2440003260 con 0x7f2460060ca0 2022-07-21T01:49:07.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245e7fc700 1 -- 172.21.15.140:0/1520691795 >> v2:172.21.15.152:3300/0 conn(0x7f24600afb80 msgr2=0x7f246014b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:07.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245e7fc700 1 --2- 172.21.15.140:0/1520691795 >> v2:172.21.15.152:3300/0 conn(0x7f24600afb80 0x7f246014b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245e7fc700 1 -- 172.21.15.140:0/1520691795 >> v1:172.21.15.140:6789/0 conn(0x7f2460060ca0 legacy=0x7f246014ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245e7fc700 1 -- 172.21.15.140:0/1520691795 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24600ac110 con 0x7f2460097a80 2022-07-21T01:49:07.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245e7fc700 1 --2- 172.21.15.140:0/1520691795 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2460097a80 0x7f2460061080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f24500032f0 tx=0x7f24500091d0).ready entity=mon.2 client_cookie=9e8e9d894fd9546e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:07.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245dffb700 1 -- 172.21.15.140:0/1520691795 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f245000b600 con 0x7f2460097a80 2022-07-21T01:49:07.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245dffb700 1 -- 172.21.15.140:0/1520691795 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245000b760 con 0x7f2460097a80 2022-07-21T01:49:07.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245dffb700 1 -- 172.21.15.140:0/1520691795 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f245000b8d0 con 0x7f2460097a80 2022-07-21T01:49:07.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 -- 172.21.15.140:0/1520691795 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2460097a80 msgr2=0x7f2460061080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 --2- 172.21.15.140:0/1520691795 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2460097a80 0x7f2460061080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f24500032f0 tx=0x7f24500091d0).stop 2022-07-21T01:49:07.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 -- 172.21.15.140:0/1520691795 shutdown_connections 2022-07-21T01:49:07.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 --2- 172.21.15.140:0/1520691795 >> v2:172.21.15.152:3300/0 conn(0x7f24600afb80 0x7f246014b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 --2- 172.21.15.140:0/1520691795 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2460097a80 0x7f2460061080 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 -- 172.21.15.140:0/1520691795 >> 172.21.15.140:0/1520691795 conn(0x7f24600b5cb0 msgr2=0x7f24600b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:07.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 -- 172.21.15.140:0/1520691795 shutdown_connections 2022-07-21T01:49:07.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 -- 172.21.15.140:0/1520691795 wait complete. 2022-07-21T01:49:07.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 Processor -- start 2022-07-21T01:49:07.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 -- start start 2022-07-21T01:49:07.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2460060ca0 0x7f2460148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2460097a80 0x7f24600aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24600afb80 0x7f24600af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24601563b0 con 0x7f2460097a80 2022-07-21T01:49:07.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246014caa0 con 0x7f2460060ca0 2022-07-21T01:49:07.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f2465790700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246014cd40 con 0x7f24600afb80 2022-07-21T01:49:07.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245e7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2460097a80 0x7f24600aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245e7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2460097a80 0x7f24600aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33546/0 (socket says 172.21.15.140:33546) 2022-07-21T01:49:07.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.854+0000 7f245e7fc700 1 -- 172.21.15.140:0/2452091134 learned_addr learned my addr 172.21.15.140:0/2452091134 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:07.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f245f7fe700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24600afb80 0x7f24600af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f245effd700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2460060ca0 0x7f2460148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f245e7fc700 1 -- 172.21.15.140:0/2452091134 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24600afb80 msgr2=0x7f24600af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f245e7fc700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24600afb80 0x7f24600af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f245e7fc700 1 -- 172.21.15.140:0/2452091134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2460060ca0 msgr2=0x7f2460148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:07.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f245e7fc700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2460060ca0 0x7f2460148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:07.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f245e7fc700 1 -- 172.21.15.140:0/2452091134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2450009470 con 0x7f2460097a80 2022-07-21T01:49:07.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f245e7fc700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2460097a80 0x7f24600aedb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f24500032f0 tx=0x7f2450016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:07.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f2457fff700 1 -- 172.21.15.140:0/2452091134 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f245000b8d0 con 0x7f2460097a80 2022-07-21T01:49:07.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f2457fff700 1 -- 172.21.15.140:0/2452091134 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245001f5e0 con 0x7f2460097a80 2022-07-21T01:49:07.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f2465790700 1 -- 172.21.15.140:0/2452091134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246014cfe0 con 0x7f2460097a80 2022-07-21T01:49:07.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f2457fff700 1 -- 172.21.15.140:0/2452091134 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f245000b600 con 0x7f2460097a80 2022-07-21T01:49:07.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f2465790700 1 -- 172.21.15.140:0/2452091134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24600adf60 con 0x7f2460097a80 2022-07-21T01:49:07.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f2457fff700 1 -- 172.21.15.140:0/2452091134 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2450026030 con 0x7f2460097a80 2022-07-21T01:49:07.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f2465790700 1 -- 172.21.15.140:0/2452091134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246014d1a0 con 0x7f2460097a80 2022-07-21T01:49:07.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f2457fff700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24440539e0 0x7f2444055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:07.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f245effd700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24440539e0 0x7f2444055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:07.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.858+0000 7f2457fff700 1 -- 172.21.15.140:0/2452091134 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f245006c820 con 0x7f2460097a80 2022-07-21T01:49:07.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.862+0000 7f245effd700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24440539e0 0x7f2444055ea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f244800b000 tx=0x7f2448009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:07.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:07.866+0000 7f2457fff700 1 -- 172.21.15.140:0/2452091134 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2450043a70 con 0x7f2460097a80 2022-07-21T01:49:08.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.030+0000 7f2465790700 1 -- 172.21.15.140:0/2452091134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f24600574e0 con 0x7f2460097a80 2022-07-21T01:49:08.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.038+0000 7f2457fff700 1 -- 172.21.15.140:0/2452091134 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2450024020 con 0x7f2460097a80 2022-07-21T01:49:08.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.038+0000 7f2465790700 1 -- 172.21.15.140:0/2452091134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24440539e0 msgr2=0x7f2444055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.038+0000 7f2465790700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24440539e0 0x7f2444055ea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f244800b000 tx=0x7f2448009040).stop 2022-07-21T01:49:08.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.038+0000 7f2465790700 1 -- 172.21.15.140:0/2452091134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2460097a80 msgr2=0x7f24600aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.038+0000 7f2465790700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2460097a80 0x7f24600aedb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f24500032f0 tx=0x7f2450016000).stop 2022-07-21T01:49:08.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.042+0000 7f2465790700 1 -- 172.21.15.140:0/2452091134 shutdown_connections 2022-07-21T01:49:08.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.042+0000 7f2465790700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24440539e0 0x7f2444055ea0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.042+0000 7f2465790700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24600afb80 0x7f24600af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.042+0000 7f2465790700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2460097a80 0x7f24600aedb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.042+0000 7f2465790700 1 --2- 172.21.15.140:0/2452091134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2460060ca0 0x7f2460148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.042+0000 7f2465790700 1 -- 172.21.15.140:0/2452091134 >> 172.21.15.140:0/2452091134 conn(0x7f24600b5cb0 msgr2=0x7f24600b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:08.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.042+0000 7f2465790700 1 -- 172.21.15.140:0/2452091134 shutdown_connections 2022-07-21T01:49:08.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.042+0000 7f2465790700 1 -- 172.21.15.140:0/2452091134 wait complete. 2022-07-21T01:49:08.045 INFO:teuthology.orchestra.run.smithi140.stderr:updated 2022-07-21T01:49:08.056 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2022-07-21T01:49:08.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a759e700 1 Processor -- start 2022-07-21T01:49:08.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a759e700 1 -- start start 2022-07-21T01:49:08.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a759e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd8a80b4a90 0x7fd8a80b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8a80b0270 0x7fd8a80acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a759e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a80b0bc0 con 0x7fd8a80ae350 2022-07-21T01:49:08.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a759e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a80b4e70 con 0x7fd8a80b4a90 2022-07-21T01:49:08.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80af470 con 0x7fd8a80b0270 2022-07-21T01:49:08.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a659c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd8a80b4a90 0x7fd8a80b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a659c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd8a80b4a90 0x7fd8a80b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33554/0 (socket says 172.21.15.140:33554) 2022-07-21T01:49:08.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a659c700 1 -- 172.21.15.140:0/3969274955 learned_addr learned my addr 172.21.15.140:0/3969274955 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:08.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a6d9d700 1 --1- 172.21.15.140:0/3969274955 >> v1:172.21.15.140:6789/0 conn(0x7fd8a80ae350 0x7fd8a8142ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47704/0 (socket says 172.21.15.140:47704) 2022-07-21T01:49:08.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.158+0000 7fd8a5d9b700 1 --2- 172.21.15.140:0/3969274955 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8a80b0270 0x7fd8a80acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a659c700 1 -- 172.21.15.140:0/3969274955 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8a80b0270 msgr2=0x7fd8a80acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:08.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a659c700 1 --2- 172.21.15.140:0/3969274955 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8a80b0270 0x7fd8a80acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a559a700 1 -- 172.21.15.140:0/3969274955 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38118173 0 0) 0x7fd8a0001490 con 0x7fd8a80ae350 2022-07-21T01:49:08.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a659c700 1 -- 172.21.15.140:0/3969274955 >> v1:172.21.15.140:6789/0 conn(0x7fd8a80ae350 legacy=0x7fd8a8142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a659c700 1 -- 172.21.15.140:0/3969274955 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80af710 con 0x7fd8a80b4a90 2022-07-21T01:49:08.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a659c700 1 --2- 172.21.15.140:0/3969274955 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8a80b4a90 0x7fd8a80b1760 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fd898002f70 tx=0x7fd898006eb0).ready entity=mon.0 client_cookie=de2833d1ca26e512 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:08.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a559a700 1 -- 172.21.15.140:0/3969274955 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd89800d5c0 con 0x7fd8a80b4a90 2022-07-21T01:49:08.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a559a700 1 -- 172.21.15.140:0/3969274955 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd898000e90 con 0x7fd8a80b4a90 2022-07-21T01:49:08.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a559a700 1 -- 172.21.15.140:0/3969274955 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd89800d910 con 0x7fd8a80b4a90 2022-07-21T01:49:08.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a759e700 1 -- 172.21.15.140:0/3969274955 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8a80b4a90 msgr2=0x7fd8a80b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a759e700 1 --2- 172.21.15.140:0/3969274955 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8a80b4a90 0x7fd8a80b1760 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fd898002f70 tx=0x7fd898006eb0).stop 2022-07-21T01:49:08.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a759e700 1 -- 172.21.15.140:0/3969274955 shutdown_connections 2022-07-21T01:49:08.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a759e700 1 --2- 172.21.15.140:0/3969274955 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8a80b0270 0x7fd8a80acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a759e700 1 --2- 172.21.15.140:0/3969274955 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8a80b4a90 0x7fd8a80b1760 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a759e700 1 -- 172.21.15.140:0/3969274955 >> 172.21.15.140:0/3969274955 conn(0x7fd8a8063150 msgr2=0x7fd8a8063530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:08.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a759e700 1 -- 172.21.15.140:0/3969274955 shutdown_connections 2022-07-21T01:49:08.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.162+0000 7fd8a759e700 1 -- 172.21.15.140:0/3969274955 wait complete. 2022-07-21T01:49:08.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a759e700 1 Processor -- start 2022-07-21T01:49:08.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a759e700 1 -- start start 2022-07-21T01:49:08.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a80ae350 0x7fd8a80b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8a80b0270 0x7fd8a8065030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8a80b4a90 0x7fd8a8065520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8140f50 con 0x7fd8a80b4a90 2022-07-21T01:49:08.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80aa960 con 0x7fd8a80ae350 2022-07-21T01:49:08.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8141da0 con 0x7fd8a80b0270 2022-07-21T01:49:08.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a6d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8a80b4a90 0x7fd8a8065520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a6d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8a80b4a90 0x7fd8a8065520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33564/0 (socket says 172.21.15.140:33564) 2022-07-21T01:49:08.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a6d9d700 1 -- 172.21.15.140:0/1593949070 learned_addr learned my addr 172.21.15.140:0/1593949070 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:08.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a5d9b700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8a80b0270 0x7fd8a8065030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a659c700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a80ae350 0x7fd8a80b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a6d9d700 1 -- 172.21.15.140:0/1593949070 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8a80b0270 msgr2=0x7fd8a8065030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a6d9d700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8a80b0270 0x7fd8a8065030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a6d9d700 1 -- 172.21.15.140:0/1593949070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a80ae350 msgr2=0x7fd8a80b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a6d9d700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a80ae350 0x7fd8a80b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a6d9d700 1 -- 172.21.15.140:0/1593949070 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89800b1e0 con 0x7fd8a80b4a90 2022-07-21T01:49:08.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a6d9d700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8a80b4a90 0x7fd8a8065520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fd8a0001250 tx=0x7fd8a0009e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:08.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8977fe700 1 -- 172.21.15.140:0/1593949070 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a0007cf0 con 0x7fd8a80b4a90 2022-07-21T01:49:08.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8977fe700 1 -- 172.21.15.140:0/1593949070 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a0007e50 con 0x7fd8a80b4a90 2022-07-21T01:49:08.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a759e700 1 -- 172.21.15.140:0/1593949070 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a81417a0 con 0x7fd8a80b4a90 2022-07-21T01:49:08.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8977fe700 1 -- 172.21.15.140:0/1593949070 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a0009300 con 0x7fd8a80b4a90 2022-07-21T01:49:08.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.166+0000 7fd8a759e700 1 -- 172.21.15.140:0/1593949070 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a813bae0 con 0x7fd8a80b4a90 2022-07-21T01:49:08.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.170+0000 7fd8977fe700 1 -- 172.21.15.140:0/1593949070 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd8a0012030 con 0x7fd8a80b4a90 2022-07-21T01:49:08.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.170+0000 7fd8a759e700 1 -- 172.21.15.140:0/1593949070 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a80aa7a0 con 0x7fd8a80b4a90 2022-07-21T01:49:08.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.170+0000 7fd8977fe700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd884053980 0x7fd884055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.170+0000 7fd8977fe700 1 -- 172.21.15.140:0/1593949070 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd8a006da80 con 0x7fd8a80b4a90 2022-07-21T01:49:08.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.170+0000 7fd8a659c700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd884053980 0x7fd884055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.170+0000 7fd8a659c700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd884053980 0x7fd884055e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fd89801c000 tx=0x7fd898015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:08.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.178+0000 7fd8977fe700 1 -- 172.21.15.140:0/1593949070 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd8a0044cd0 con 0x7fd8a80b4a90 2022-07-21T01:49:08.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.342+0000 7fd8a759e700 1 -- 172.21.15.140:0/1593949070 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fd8a80b0bc0 con 0x7fd8a80b4a90 2022-07-21T01:49:08.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.346+0000 7fd8977fe700 1 -- 172.21.15.140:0/1593949070 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd8a80b0bc0 con 0x7fd8a80b4a90 2022-07-21T01:49:08.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 -- 172.21.15.140:0/1593949070 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd884053980 msgr2=0x7fd884055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd884053980 0x7fd884055e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fd89801c000 tx=0x7fd898015000).stop 2022-07-21T01:49:08.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 -- 172.21.15.140:0/1593949070 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8a80b4a90 msgr2=0x7fd8a8065520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8a80b4a90 0x7fd8a8065520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fd8a0001250 tx=0x7fd8a0009e30).stop 2022-07-21T01:49:08.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 -- 172.21.15.140:0/1593949070 shutdown_connections 2022-07-21T01:49:08.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd884053980 0x7fd884055e40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8a80b4a90 0x7fd8a8065520 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8a80b0270 0x7fd8a8065030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 --2- 172.21.15.140:0/1593949070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8a80ae350 0x7fd8a80b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 -- 172.21.15.140:0/1593949070 >> 172.21.15.140:0/1593949070 conn(0x7fd8a8063150 msgr2=0x7fd8a8060d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:08.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 -- 172.21.15.140:0/1593949070 shutdown_connections 2022-07-21T01:49:08.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.349+0000 7fd8a759e700 1 -- 172.21.15.140:0/1593949070 wait complete. 2022-07-21T01:49:08.356 INFO:teuthology.orchestra.run.smithi140.stderr:updated 2022-07-21T01:49:08.365 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2022-07-21T01:49:08.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.461+0000 7f7793fff700 1 Processor -- start 2022-07-21T01:49:08.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.461+0000 7f7793fff700 1 -- start start 2022-07-21T01:49:08.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.461+0000 7f7793fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f77940b5970 0x7f77940ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.461+0000 7f7793fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77940b0b80 con 0x7f77940af8c0 2022-07-21T01:49:08.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.461+0000 7f7793fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779405d4d0 con 0x7f7794143a10 2022-07-21T01:49:08.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.461+0000 7f7793fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77940b5d50 con 0x7f77940b5970 2022-07-21T01:49:08.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.461+0000 7f7792ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f77940b5970 0x7f77940ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.461+0000 7f7792ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f77940b5970 0x7f77940ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33568/0 (socket says 172.21.15.140:33568) 2022-07-21T01:49:08.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.461+0000 7f7792ffd700 1 -- 172.21.15.140:0/2000934103 learned_addr learned my addr 172.21.15.140:0/2000934103 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:08.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7792ffd700 1 -- 172.21.15.140:0/2000934103 >> v1:172.21.15.152:6789/0 conn(0x7f7794143a10 legacy=0x7f7794143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7792ffd700 1 -- 172.21.15.140:0/2000934103 >> v1:172.21.15.140:6789/0 conn(0x7f77940af8c0 legacy=0x7f77940ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7792ffd700 1 -- 172.21.15.140:0/2000934103 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940b0db0 con 0x7f77940b5970 2022-07-21T01:49:08.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7792ffd700 1 --2- 172.21.15.140:0/2000934103 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f77940b5970 0x7f77940ae100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f778c0032f0 tx=0x7f778c009290).ready entity=mon.0 client_cookie=91c882bdbedf6601 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:08.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7791ffb700 1 -- 172.21.15.140:0/2000934103 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f778c00b8a0 con 0x7f77940b5970 2022-07-21T01:49:08.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7791ffb700 1 -- 172.21.15.140:0/2000934103 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c00ba00 con 0x7f77940b5970 2022-07-21T01:49:08.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7791ffb700 1 -- 172.21.15.140:0/2000934103 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f778c00bb70 con 0x7f77940b5970 2022-07-21T01:49:08.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7793fff700 1 -- 172.21.15.140:0/2000934103 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f77940b5970 msgr2=0x7f77940ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7793fff700 1 --2- 172.21.15.140:0/2000934103 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f77940b5970 0x7f77940ae100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f778c0032f0 tx=0x7f778c009290).stop 2022-07-21T01:49:08.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7793fff700 1 -- 172.21.15.140:0/2000934103 shutdown_connections 2022-07-21T01:49:08.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7793fff700 1 --2- 172.21.15.140:0/2000934103 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f77940b5970 0x7f77940ae100 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7793fff700 1 -- 172.21.15.140:0/2000934103 >> 172.21.15.140:0/2000934103 conn(0x7f7794063490 msgr2=0x7f77940b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:08.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7793fff700 1 -- 172.21.15.140:0/2000934103 shutdown_connections 2022-07-21T01:49:08.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.465+0000 7f7793fff700 1 -- 172.21.15.140:0/2000934103 wait complete. 2022-07-21T01:49:08.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7793fff700 1 Processor -- start 2022-07-21T01:49:08.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7793fff700 1 -- start start 2022-07-21T01:49:08.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7793fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77940af8c0 0x7f7794159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7793fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77940b5970 0x7f779415a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7793fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7794143a10 0x7f7794111840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7793fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7794157810 con 0x7f7794143a10 2022-07-21T01:49:08.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7793fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77941428d0 con 0x7f77940b5970 2022-07-21T01:49:08.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7793fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779410e420 con 0x7f77940af8c0 2022-07-21T01:49:08.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f77937fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7794143a10 0x7f7794111840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f77937fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7794143a10 0x7f7794111840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33578/0 (socket says 172.21.15.140:33578) 2022-07-21T01:49:08.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f77937fe700 1 -- 172.21.15.140:0/117494999 learned_addr learned my addr 172.21.15.140:0/117494999 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:08.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7792ffd700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77940af8c0 0x7f7794159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f77927fc700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77940b5970 0x7f779415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f77937fe700 1 -- 172.21.15.140:0/117494999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77940af8c0 msgr2=0x7f7794159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f77937fe700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77940af8c0 0x7f7794159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f77937fe700 1 -- 172.21.15.140:0/117494999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77940b5970 msgr2=0x7f779415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f77937fe700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77940b5970 0x7f779415a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f77937fe700 1 -- 172.21.15.140:0/117494999 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c009530 con 0x7f7794143a10 2022-07-21T01:49:08.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f77937fe700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7794143a10 0x7f7794111840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f7794111730 tx=0x7f7784007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:08.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7783fff700 1 -- 172.21.15.140:0/117494999 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7784007030 con 0x7f7794143a10 2022-07-21T01:49:08.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7783fff700 1 -- 172.21.15.140:0/117494999 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7784007190 con 0x7f7794143a10 2022-07-21T01:49:08.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7793fff700 1 -- 172.21.15.140:0/117494999 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7794063a80 con 0x7f7794143a10 2022-07-21T01:49:08.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7783fff700 1 -- 172.21.15.140:0/117494999 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7784006510 con 0x7f7794143a10 2022-07-21T01:49:08.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.469+0000 7f7793fff700 1 -- 172.21.15.140:0/117494999 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779415a9f0 con 0x7f7794143a10 2022-07-21T01:49:08.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.473+0000 7f7793fff700 1 -- 172.21.15.140:0/117494999 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7794063c10 con 0x7f7794143a10 2022-07-21T01:49:08.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.473+0000 7f7783fff700 1 -- 172.21.15.140:0/117494999 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7784010030 con 0x7f7794143a10 2022-07-21T01:49:08.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.473+0000 7f7783fff700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7770053a70 0x7f7770055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.473+0000 7f7783fff700 1 -- 172.21.15.140:0/117494999 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f778406b700 con 0x7f7794143a10 2022-07-21T01:49:08.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.473+0000 7f7792ffd700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7770053a70 0x7f7770055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.473+0000 7f7792ffd700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7770053a70 0x7f7770055f30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f778c01b000 tx=0x7f778c016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:08.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.481+0000 7f7783fff700 1 -- 172.21.15.140:0/117494999 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7784042950 con 0x7f7794143a10 2022-07-21T01:49:08.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.641+0000 7f7793fff700 1 -- 172.21.15.140:0/117494999 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f77940acae0 con 0x7f7794143a10 2022-07-21T01:49:08.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7783fff700 1 -- 172.21.15.140:0/117494999 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7784045f70 con 0x7f7794143a10 2022-07-21T01:49:08.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 -- 172.21.15.140:0/117494999 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7770053a70 msgr2=0x7f7770055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7770053a70 0x7f7770055f30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f778c01b000 tx=0x7f778c016000).stop 2022-07-21T01:49:08.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 -- 172.21.15.140:0/117494999 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7794143a10 msgr2=0x7f7794111840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7794143a10 0x7f7794111840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f7794111730 tx=0x7f7784007c20).stop 2022-07-21T01:49:08.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 -- 172.21.15.140:0/117494999 shutdown_connections 2022-07-21T01:49:08.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7770053a70 0x7f7770055f30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7794143a10 0x7f7794111840 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77940b5970 0x7f779415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 --2- 172.21.15.140:0/117494999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77940af8c0 0x7f7794159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 -- 172.21.15.140:0/117494999 >> 172.21.15.140:0/117494999 conn(0x7f7794063490 msgr2=0x7f77940b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:08.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 -- 172.21.15.140:0/117494999 shutdown_connections 2022-07-21T01:49:08.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.653+0000 7f7793fff700 1 -- 172.21.15.140:0/117494999 wait complete. 2022-07-21T01:49:08.663 INFO:teuthology.orchestra.run.smithi140.stderr:updated 2022-07-21T01:49:08.670 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2022-07-21T01:49:08.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.777+0000 7f5d98ce1700 1 Processor -- start 2022-07-21T01:49:08.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 -- start start 2022-07-21T01:49:08.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5d940b5970 0x7f5d940ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5d940af8c0 0x7f5d940ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d940b0b80 con 0x7f5d940aeae0 2022-07-21T01:49:08.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9405d4d0 con 0x7f5d940af8c0 2022-07-21T01:49:08.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d940b5d50 con 0x7f5d940b5970 2022-07-21T01:49:08.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d92ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5d940af8c0 0x7f5d940ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d93fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f5d940aeae0 0x7f5d9405bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47730/0 (socket says 172.21.15.140:47730) 2022-07-21T01:49:08.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d93fff700 1 -- 172.21.15.140:0/515325299 learned_addr learned my addr 172.21.15.140:0/515325299 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:08.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d937fe700 1 --2- 172.21.15.140:0/515325299 >> v2:172.21.15.152:3300/0 conn(0x7f5d940b5970 0x7f5d940ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d92ffd700 1 -- 172.21.15.140:0/515325299 >> v2:172.21.15.152:3300/0 conn(0x7f5d940b5970 msgr2=0x7f5d940ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:08.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d927fc700 1 -- 172.21.15.140:0/515325299 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871041861 0 0) 0x7f5d940b0b80 con 0x7f5d940aeae0 2022-07-21T01:49:08.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d92ffd700 1 --2- 172.21.15.140:0/515325299 >> v2:172.21.15.152:3300/0 conn(0x7f5d940b5970 0x7f5d940ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d92ffd700 1 -- 172.21.15.140:0/515325299 >> v1:172.21.15.140:6789/0 conn(0x7f5d940aeae0 legacy=0x7f5d9405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d92ffd700 1 -- 172.21.15.140:0/515325299 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940b0db0 con 0x7f5d940af8c0 2022-07-21T01:49:08.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d937fe700 1 --2- 172.21.15.140:0/515325299 >> v2:172.21.15.152:3300/0 conn(0x7f5d940b5970 0x7f5d940ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:49:08.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d92ffd700 1 --2- 172.21.15.140:0/515325299 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d940af8c0 0x7f5d940ae5f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f5d7c002f70 tx=0x7f5d7c006df0).ready entity=mon.0 client_cookie=c4b24b2d98055de7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:08.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d927fc700 1 -- 172.21.15.140:0/515325299 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d7c00d5c0 con 0x7f5d940af8c0 2022-07-21T01:49:08.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d927fc700 1 -- 172.21.15.140:0/515325299 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c000e90 con 0x7f5d940af8c0 2022-07-21T01:49:08.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d927fc700 1 -- 172.21.15.140:0/515325299 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d7c00d910 con 0x7f5d940af8c0 2022-07-21T01:49:08.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 -- 172.21.15.140:0/515325299 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d940af8c0 msgr2=0x7f5d940ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 --2- 172.21.15.140:0/515325299 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d940af8c0 0x7f5d940ae5f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f5d7c002f70 tx=0x7f5d7c006df0).stop 2022-07-21T01:49:08.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 -- 172.21.15.140:0/515325299 shutdown_connections 2022-07-21T01:49:08.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 --2- 172.21.15.140:0/515325299 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d940af8c0 0x7f5d940ae5f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 --2- 172.21.15.140:0/515325299 >> v2:172.21.15.152:3300/0 conn(0x7f5d940b5970 0x7f5d940ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 -- 172.21.15.140:0/515325299 >> 172.21.15.140:0/515325299 conn(0x7f5d94063490 msgr2=0x7f5d940b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:08.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 -- 172.21.15.140:0/515325299 shutdown_connections 2022-07-21T01:49:08.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.781+0000 7f5d98ce1700 1 -- 172.21.15.140:0/515325299 wait complete. 2022-07-21T01:49:08.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d98ce1700 1 Processor -- start 2022-07-21T01:49:08.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d98ce1700 1 -- start start 2022-07-21T01:49:08.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d98ce1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d940aeae0 0x7f5d94143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d98ce1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d940af8c0 0x7f5d9413bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d98ce1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d940b5970 0x7f5d94159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d98ce1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94146260 con 0x7f5d940af8c0 2022-07-21T01:49:08.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d98ce1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94145880 con 0x7f5d940aeae0 2022-07-21T01:49:08.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d98ce1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94145b20 con 0x7f5d940b5970 2022-07-21T01:49:08.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d92ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d940af8c0 0x7f5d9413bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d92ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d940af8c0 0x7f5d9413bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33590/0 (socket says 172.21.15.140:33590) 2022-07-21T01:49:08.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d92ffd700 1 -- 172.21.15.140:0/3834759264 learned_addr learned my addr 172.21.15.140:0/3834759264 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:08.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d93fff700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d940b5970 0x7f5d94159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d937fe700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d940aeae0 0x7f5d94143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d92ffd700 1 -- 172.21.15.140:0/3834759264 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d940b5970 msgr2=0x7f5d94159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d92ffd700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d940b5970 0x7f5d94159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d92ffd700 1 -- 172.21.15.140:0/3834759264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d940aeae0 msgr2=0x7f5d94143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d92ffd700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d940aeae0 0x7f5d94143dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d92ffd700 1 -- 172.21.15.140:0/3834759264 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c00b180 con 0x7f5d940af8c0 2022-07-21T01:49:08.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d92ffd700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d940af8c0 0x7f5d9413bf40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f5d7c002f70 tx=0x7f5d7c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:08.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d90ff9700 1 -- 172.21.15.140:0/3834759264 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d7c01e400 con 0x7f5d940af8c0 2022-07-21T01:49:08.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d90ff9700 1 -- 172.21.15.140:0/3834759264 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c01e560 con 0x7f5d940af8c0 2022-07-21T01:49:08.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d98ce1700 1 -- 172.21.15.140:0/3834759264 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d94145dc0 con 0x7f5d940af8c0 2022-07-21T01:49:08.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d90ff9700 1 -- 172.21.15.140:0/3834759264 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d7c01e8c0 con 0x7f5d940af8c0 2022-07-21T01:49:08.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d98ce1700 1 -- 172.21.15.140:0/3834759264 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9410f690 con 0x7f5d940af8c0 2022-07-21T01:49:08.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.785+0000 7f5d98ce1700 1 -- 172.21.15.140:0/3834759264 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d94145f50 con 0x7f5d940af8c0 2022-07-21T01:49:08.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.789+0000 7f5d90ff9700 1 -- 172.21.15.140:0/3834759264 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d7c025030 con 0x7f5d940af8c0 2022-07-21T01:49:08.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.789+0000 7f5d90ff9700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d78053a70 0x7f5d78055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:08.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.789+0000 7f5d90ff9700 1 -- 172.21.15.140:0/3834759264 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f5d7c06cc60 con 0x7f5d940af8c0 2022-07-21T01:49:08.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.789+0000 7f5d937fe700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d78053a70 0x7f5d78055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:08.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.789+0000 7f5d937fe700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d78053a70 0x7f5d78055f30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f5d8400b000 tx=0x7f5d84008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:08.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.797+0000 7f5d90ff9700 1 -- 172.21.15.140:0/3834759264 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5d7c043da0 con 0x7f5d940af8c0 2022-07-21T01:49:08.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.961+0000 7f5d98ce1700 1 -- 172.21.15.140:0/3834759264 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f5d940acae0 con 0x7f5d940af8c0 2022-07-21T01:49:08.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d90ff9700 1 -- 172.21.15.140:0/3834759264 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5d7c023020 con 0x7f5d940af8c0 2022-07-21T01:49:08.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 -- 172.21.15.140:0/3834759264 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d78053a70 msgr2=0x7f5d78055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d78053a70 0x7f5d78055f30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f5d8400b000 tx=0x7f5d84008040).stop 2022-07-21T01:49:08.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 -- 172.21.15.140:0/3834759264 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d940af8c0 msgr2=0x7f5d9413bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:08.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d940af8c0 0x7f5d9413bf40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f5d7c002f70 tx=0x7f5d7c014000).stop 2022-07-21T01:49:08.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 -- 172.21.15.140:0/3834759264 shutdown_connections 2022-07-21T01:49:08.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d78053a70 0x7f5d78055f30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d940b5970 0x7f5d94159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d940af8c0 0x7f5d9413bf40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 --2- 172.21.15.140:0/3834759264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d940aeae0 0x7f5d94143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:08.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 -- 172.21.15.140:0/3834759264 >> 172.21.15.140:0/3834759264 conn(0x7f5d94063490 msgr2=0x7f5d940b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:08.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 -- 172.21.15.140:0/3834759264 shutdown_connections 2022-07-21T01:49:08.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:08.969+0000 7f5d98ce1700 1 -- 172.21.15.140:0/3834759264 wait complete. 2022-07-21T01:49:08.983 INFO:teuthology.orchestra.run.smithi140.stderr:updated 2022-07-21T01:49:08.992 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2022-07-21T01:49:09.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f9cf0700 1 Processor -- start 2022-07-21T01:49:09.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f9cf0700 1 -- start start 2022-07-21T01:49:09.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f9cf0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f41f40af8c0 0x7f41f40ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f9cf0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f41f4143a10 0x7f41f4143e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f9cf0700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f40b0b80 con 0x7f41f40b5970 2022-07-21T01:49:09.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f9cf0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f405d4d0 con 0x7f41f4143a10 2022-07-21T01:49:09.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f9cf0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f40b5d50 con 0x7f41f40af8c0 2022-07-21T01:49:09.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f94ef700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f41f4143a10 0x7f41f4143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f94ef700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f41f4143a10 0x7f41f4143e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33600/0 (socket says 172.21.15.140:33600) 2022-07-21T01:49:09.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f94ef700 1 -- 172.21.15.140:0/4152868891 learned_addr learned my addr 172.21.15.140:0/4152868891 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:09.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f3fff700 1 --2- 172.21.15.140:0/4152868891 >> v2:172.21.15.152:3300/0 conn(0x7f41f40af8c0 0x7f41f40ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f94ef700 1 -- 172.21.15.140:0/4152868891 >> v2:172.21.15.152:3300/0 conn(0x7f41f40af8c0 msgr2=0x7f41f40ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:09.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f94ef700 1 --2- 172.21.15.140:0/4152868891 >> v2:172.21.15.152:3300/0 conn(0x7f41f40af8c0 0x7f41f40ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f94ef700 1 -- 172.21.15.140:0/4152868891 >> v1:172.21.15.152:6789/0 conn(0x7f41f40b5970 legacy=0x7f41f40ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f94ef700 1 -- 172.21.15.140:0/4152868891 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f40b0db0 con 0x7f41f4143a10 2022-07-21T01:49:09.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f94ef700 1 --2- 172.21.15.140:0/4152868891 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41f4143a10 0x7f41f4143e50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f41e4002f70 tx=0x7f41e4006eb0).ready entity=mon.0 client_cookie=4b9185324f3a6a90 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:09.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f37fe700 1 -- 172.21.15.140:0/4152868891 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41e400d5c0 con 0x7f41f4143a10 2022-07-21T01:49:09.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.113+0000 7f41f37fe700 1 -- 172.21.15.140:0/4152868891 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e4000e90 con 0x7f41f4143a10 2022-07-21T01:49:09.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.117+0000 7f41f37fe700 1 -- 172.21.15.140:0/4152868891 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41e400d950 con 0x7f41f4143a10 2022-07-21T01:49:09.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.117+0000 7f41f9cf0700 1 -- 172.21.15.140:0/4152868891 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41f4143a10 msgr2=0x7f41f4143e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.117+0000 7f41f9cf0700 1 --2- 172.21.15.140:0/4152868891 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41f4143a10 0x7f41f4143e50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f41e4002f70 tx=0x7f41e4006eb0).stop 2022-07-21T01:49:09.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.117+0000 7f41f9cf0700 1 -- 172.21.15.140:0/4152868891 shutdown_connections 2022-07-21T01:49:09.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.117+0000 7f41f9cf0700 1 --2- 172.21.15.140:0/4152868891 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41f4143a10 0x7f41f4143e50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.117+0000 7f41f9cf0700 1 --2- 172.21.15.140:0/4152868891 >> v2:172.21.15.152:3300/0 conn(0x7f41f40af8c0 0x7f41f40ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.117+0000 7f41f9cf0700 1 -- 172.21.15.140:0/4152868891 >> 172.21.15.140:0/4152868891 conn(0x7f41f4063490 msgr2=0x7f41f40b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:09.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.117+0000 7f41f9cf0700 1 -- 172.21.15.140:0/4152868891 shutdown_connections 2022-07-21T01:49:09.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.117+0000 7f41f9cf0700 1 -- 172.21.15.140:0/4152868891 wait complete. 2022-07-21T01:49:09.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.117+0000 7f41f9cf0700 1 Processor -- start 2022-07-21T01:49:09.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.117+0000 7f41f9cf0700 1 -- start start 2022-07-21T01:49:09.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f9cf0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41f40af8c0 0x7f41f41595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f9cf0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f41f40b5970 0x7f41f414bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f9cf0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41f4143a10 0x7f41f4142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f9cf0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f4144350 con 0x7f41f4143a10 2022-07-21T01:49:09.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f9cf0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f41461f0 con 0x7f41f40af8c0 2022-07-21T01:49:09.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f9cf0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f41450e0 con 0x7f41f40b5970 2022-07-21T01:49:09.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f94ef700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41f4143a10 0x7f41f4142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f94ef700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41f4143a10 0x7f41f4142630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33606/0 (socket says 172.21.15.140:33606) 2022-07-21T01:49:09.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f94ef700 1 -- 172.21.15.140:0/752739104 learned_addr learned my addr 172.21.15.140:0/752739104 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:09.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f3fff700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f41f40b5970 0x7f41f414bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f8cee700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41f40af8c0 0x7f41f41595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f94ef700 1 -- 172.21.15.140:0/752739104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f41f40b5970 msgr2=0x7f41f414bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f94ef700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f41f40b5970 0x7f41f414bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f94ef700 1 -- 172.21.15.140:0/752739104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41f40af8c0 msgr2=0x7f41f41595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f94ef700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41f40af8c0 0x7f41f41595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f94ef700 1 -- 172.21.15.140:0/752739104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e400b1e0 con 0x7f41f4143a10 2022-07-21T01:49:09.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f94ef700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41f4143a10 0x7f41f4142630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f41e4002f70 tx=0x7f41e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:09.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f1ffb700 1 -- 172.21.15.140:0/752739104 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41e40116e0 con 0x7f41f4143a10 2022-07-21T01:49:09.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f1ffb700 1 -- 172.21.15.140:0/752739104 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e4011840 con 0x7f41f4143a10 2022-07-21T01:49:09.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f9cf0700 1 -- 172.21.15.140:0/752739104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f4144ce0 con 0x7f41f4143a10 2022-07-21T01:49:09.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f1ffb700 1 -- 172.21.15.140:0/752739104 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f41e4011ba0 con 0x7f41f4143a10 2022-07-21T01:49:09.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f9cf0700 1 -- 172.21.15.140:0/752739104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f4140dd0 con 0x7f41f4143a10 2022-07-21T01:49:09.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.121+0000 7f41f9cf0700 1 -- 172.21.15.140:0/752739104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f40acae0 con 0x7f41f4143a10 2022-07-21T01:49:09.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.125+0000 7f41f1ffb700 1 -- 172.21.15.140:0/752739104 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f41e4025030 con 0x7f41f4143a10 2022-07-21T01:49:09.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.125+0000 7f41f1ffb700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41d8053b30 0x7f41d8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.125+0000 7f41f1ffb700 1 -- 172.21.15.140:0/752739104 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f41e406ccd0 con 0x7f41f4143a10 2022-07-21T01:49:09.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.125+0000 7f41f8cee700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41d8053b30 0x7f41d8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.125+0000 7f41f8cee700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41d8053b30 0x7f41d8055ff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f41dc00d000 tx=0x7f41dc007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:09.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.133+0000 7f41f1ffb700 1 -- 172.21.15.140:0/752739104 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f41e4043e10 con 0x7f41f4143a10 2022-07-21T01:49:09.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.297+0000 7f41f9cf0700 1 -- 172.21.15.140:0/752739104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f41f40574e0 con 0x7f41f4143a10 2022-07-21T01:49:09.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f1ffb700 1 -- 172.21.15.140:0/752739104 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v14) v1 ==== 97+0+0 (secure 0 0 0) 0x7f41e4023020 con 0x7f41f4143a10 2022-07-21T01:49:09.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 -- 172.21.15.140:0/752739104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41d8053b30 msgr2=0x7f41d8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41d8053b30 0x7f41d8055ff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f41dc00d000 tx=0x7f41dc007b90).stop 2022-07-21T01:49:09.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 -- 172.21.15.140:0/752739104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41f4143a10 msgr2=0x7f41f4142630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41f4143a10 0x7f41f4142630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f41e4002f70 tx=0x7f41e4015000).stop 2022-07-21T01:49:09.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 -- 172.21.15.140:0/752739104 shutdown_connections 2022-07-21T01:49:09.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41d8053b30 0x7f41d8055ff0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41f4143a10 0x7f41f4142630 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f41f40b5970 0x7f41f414bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 --2- 172.21.15.140:0/752739104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41f40af8c0 0x7f41f41595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 -- 172.21.15.140:0/752739104 >> 172.21.15.140:0/752739104 conn(0x7f41f4063490 msgr2=0x7f41f40b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:09.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 -- 172.21.15.140:0/752739104 shutdown_connections 2022-07-21T01:49:09.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.305+0000 7f41f9cf0700 1 -- 172.21.15.140:0/752739104 wait complete. 2022-07-21T01:49:09.320 INFO:teuthology.orchestra.run.smithi140.stderr:updated 2022-07-21T01:49:09.322 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:49:09.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f667f9700 1 Processor -- start 2022-07-21T01:49:09.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f667f9700 1 -- start start 2022-07-21T01:49:09.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f667f9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f600aea10 0x7f0f600b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f667f9700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f601433d0 con 0x7f0f600b0570 2022-07-21T01:49:09.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f667f9700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f60143670 con 0x7f0f601424e0 2022-07-21T01:49:09.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f667f9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f60143b10 con 0x7f0f600aea10 2022-07-21T01:49:09.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f657f7700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0f600b0570 0x7f0f60141dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47756/0 (socket says 172.21.15.140:47756) 2022-07-21T01:49:09.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f657f7700 1 -- 172.21.15.140:0/3460802553 learned_addr learned my addr 172.21.15.140:0/3460802553 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:09.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f65ff8700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f600aea10 0x7f0f600b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571456635 0 0) 0x7f0f601433d0 con 0x7f0f600b0570 2022-07-21T01:49:09.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f4c003260 con 0x7f0f600b0570 2022-07-21T01:49:09.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501285679 0 0) 0x7f0f60143670 con 0x7f0f601424e0 2022-07-21T01:49:09.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f601433d0 con 0x7f0f601424e0 2022-07-21T01:49:09.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1663230752 0 0) 0x7f0f4c003260 con 0x7f0f600b0570 2022-07-21T01:49:09.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f600aea10 msgr2=0x7f0f600b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:09.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f5ffff700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f600aea10 0x7f0f600b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 >> v1:172.21.15.152:6789/0 conn(0x7f0f601424e0 legacy=0x7f0f600ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f600b1010 con 0x7f0f600b0570 2022-07-21T01:49:09.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.433+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f0f500019e0 con 0x7f0f600b0570 2022-07-21T01:49:09.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 >> v1:172.21.15.140:6789/0 conn(0x7f0f600b0570 legacy=0x7f0f60141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f5ffff700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f4c003910 0x7f0f4c003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f5ffff700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f600aea10 0x7f0f4c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f5ffff700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f4c0049f0 0x7f0f4c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f60143b10 con 0x7f0f4c0049f0 2022-07-21T01:49:09.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c00b780 con 0x7f0f4c003910 2022-07-21T01:49:09.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c00b940 con 0x7f0f600aea10 2022-07-21T01:49:09.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f65ff8700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f4c0049f0 0x7f0f4c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f657f7700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f4c003910 0x7f0f4c003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f64ff6700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f600aea10 0x7f0f4c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f65ff8700 1 -- 172.21.15.140:0/3460802553 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f600aea10 msgr2=0x7f0f4c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:09.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f65ff8700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f600aea10 0x7f0f4c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f65ff8700 1 -- 172.21.15.140:0/3460802553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f4c003910 msgr2=0x7f0f4c003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:09.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f65ff8700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f4c003910 0x7f0f4c003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f65ff8700 1 -- 172.21.15.140:0/3460802553 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c003260 con 0x7f0f4c0049f0 2022-07-21T01:49:09.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f65ff8700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f4c0049f0 0x7f0f4c004e70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f0f5800c000 tx=0x7f0f58006df0).ready entity=mon.0 client_cookie=ab0b9d85e2f9d760 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:09.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f5800e5f0 con 0x7f0f4c0049f0 2022-07-21T01:49:09.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5800e750 con 0x7f0f4c0049f0 2022-07-21T01:49:09.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f5ffff700 1 -- 172.21.15.140:0/3460802553 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f5800eab0 con 0x7f0f4c0049f0 2022-07-21T01:49:09.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f667f9700 1 -- 172.21.15.140:0/3460802553 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f4c0049f0 msgr2=0x7f0f4c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f667f9700 1 --2- 172.21.15.140:0/3460802553 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f4c0049f0 0x7f0f4c004e70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f0f5800c000 tx=0x7f0f58006df0).stop 2022-07-21T01:49:09.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f65ff8700 1 -- 172.21.15.140:0/3460802553 reap_dead start 2022-07-21T01:49:09.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f667f9700 1 -- 172.21.15.140:0/3460802553 shutdown_connections 2022-07-21T01:49:09.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f667f9700 1 -- 172.21.15.140:0/3460802553 >> 172.21.15.140:0/3460802553 conn(0x7f0f600b9360 msgr2=0x7f0f600b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:09.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f667f9700 1 -- 172.21.15.140:0/3460802553 shutdown_connections 2022-07-21T01:49:09.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f667f9700 1 -- 172.21.15.140:0/3460802553 wait complete. 2022-07-21T01:49:09.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f667f9700 1 Processor -- start 2022-07-21T01:49:09.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.437+0000 7f0f667f9700 1 -- start start 2022-07-21T01:49:09.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f667f9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f4c0049f0 0x7f0f600b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f667f9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f600b2c40 0x7f0f60151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f667f9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f60152100 0x7f0f6013eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f667f9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f600b1780 con 0x7f0f60152100 2022-07-21T01:49:09.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f667f9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f600b1380 con 0x7f0f600b2c40 2022-07-21T01:49:09.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f667f9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f600b26a0 con 0x7f0f4c0049f0 2022-07-21T01:49:09.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f657f7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f4c0049f0 0x7f0f600b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f657f7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f4c0049f0 0x7f0f600b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:58584/0 (socket says 172.21.15.140:58584) 2022-07-21T01:49:09.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f657f7700 1 -- 172.21.15.140:0/194427316 learned_addr learned my addr 172.21.15.140:0/194427316 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:09.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f65ff8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f60152100 0x7f0f6013eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f64ff6700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f600b2c40 0x7f0f60151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f657f7700 1 -- 172.21.15.140:0/194427316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f600b2c40 msgr2=0x7f0f60151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f657f7700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f600b2c40 0x7f0f60151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f657f7700 1 -- 172.21.15.140:0/194427316 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f60152100 msgr2=0x7f0f6013eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f657f7700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f60152100 0x7f0f6013eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f657f7700 1 -- 172.21.15.140:0/194427316 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f60145cd0 con 0x7f0f4c0049f0 2022-07-21T01:49:09.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f657f7700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f4c0049f0 0x7f0f600b21b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0f5000d000 tx=0x7f0f50002150).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:09.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f5e7fc700 1 -- 172.21.15.140:0/194427316 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f50011780 con 0x7f0f4c0049f0 2022-07-21T01:49:09.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f5e7fc700 1 -- 172.21.15.140:0/194427316 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f500118e0 con 0x7f0f4c0049f0 2022-07-21T01:49:09.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f667f9700 1 -- 172.21.15.140:0/194427316 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f600b4f90 con 0x7f0f4c0049f0 2022-07-21T01:49:09.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f5e7fc700 1 -- 172.21.15.140:0/194427316 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f50011c40 con 0x7f0f4c0049f0 2022-07-21T01:49:09.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.441+0000 7f0f667f9700 1 -- 172.21.15.140:0/194427316 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f60152720 con 0x7f0f4c0049f0 2022-07-21T01:49:09.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.445+0000 7f0f667f9700 1 -- 172.21.15.140:0/194427316 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f600b14f0 con 0x7f0f4c0049f0 2022-07-21T01:49:09.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.445+0000 7f0f5e7fc700 1 -- 172.21.15.140:0/194427316 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0f5001b030 con 0x7f0f4c0049f0 2022-07-21T01:49:09.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.445+0000 7f0f5e7fc700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0f40053aa0 0x7f0f40055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.445+0000 7f0f5e7fc700 1 -- 172.21.15.140:0/194427316 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f0f5006b9c0 con 0x7f0f4c0049f0 2022-07-21T01:49:09.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.445+0000 7f0f64ff6700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0f40053aa0 0x7f0f40055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.445+0000 7f0f64ff6700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0f40053aa0 0x7f0f40055f60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f0f5400d000 tx=0x7f0f5400c040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:09.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.457+0000 7f0f5e7fc700 1 -- 172.21.15.140:0/194427316 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0f600b14f0 con 0x7f0f4c0049f0 2022-07-21T01:49:09.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.581+0000 7f0f667f9700 1 -- 172.21.15.140:0/194427316 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f600b14f0 con 0x7f0f4c0049f0 2022-07-21T01:49:09.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f5e7fc700 1 -- 172.21.15.140:0/194427316 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+13277 (secure 0 0 0) 0x7f0f600b14f0 con 0x7f0f4c0049f0 2022-07-21T01:49:09.587 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:09.588 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":26,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:49:07.064735+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:49:09.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 -- 172.21.15.140:0/194427316 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0f40053aa0 msgr2=0x7f0f40055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0f40053aa0 0x7f0f40055f60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f0f5400d000 tx=0x7f0f5400c040).stop 2022-07-21T01:49:09.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 -- 172.21.15.140:0/194427316 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f4c0049f0 msgr2=0x7f0f600b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f4c0049f0 0x7f0f600b21b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0f5000d000 tx=0x7f0f50002150).stop 2022-07-21T01:49:09.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 -- 172.21.15.140:0/194427316 shutdown_connections 2022-07-21T01:49:09.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0f40053aa0 0x7f0f40055f60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f60152100 0x7f0f6013eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f600b2c40 0x7f0f60151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 --2- 172.21.15.140:0/194427316 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f4c0049f0 0x7f0f600b21b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 -- 172.21.15.140:0/194427316 >> 172.21.15.140:0/194427316 conn(0x7f0f600b9360 msgr2=0x7f0f60141180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:09.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 -- 172.21.15.140:0/194427316 shutdown_connections 2022-07-21T01:49:09.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.585+0000 7f0f667f9700 1 -- 172.21.15.140:0/194427316 wait complete. 2022-07-21T01:49:09.601 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:49:09.602 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:49:09.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.713+0000 7fc03d496700 1 Processor -- start 2022-07-21T01:49:09.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.713+0000 7fc03d496700 1 -- start start 2022-07-21T01:49:09.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.713+0000 7fc03d496700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc0380b4d60 0x7fc038145d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.713+0000 7fc03d496700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0380607e0 con 0x7fc03809df40 2022-07-21T01:49:09.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.713+0000 7fc03d496700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc038060a80 con 0x7fc038060ee0 2022-07-21T01:49:09.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.713+0000 7fc03d496700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc038060d20 con 0x7fc0380b4d60 2022-07-21T01:49:09.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.713+0000 7fc036ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc03809df40 0x7fc038145680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47776/0 (socket says 172.21.15.140:47776) 2022-07-21T01:49:09.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.713+0000 7fc036ffd700 1 -- 172.21.15.140:0/3322957251 learned_addr learned my addr 172.21.15.140:0/3322957251 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:09.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.713+0000 7fc02e7fc700 1 --2- 172.21.15.140:0/3322957251 >> v2:172.21.15.152:3300/0 conn(0x7fc0380b4d60 0x7fc038145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc0367fc700 1 -- 172.21.15.140:0/3322957251 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731758438 0 0) 0x7fc0380607e0 con 0x7fc03809df40 2022-07-21T01:49:09.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc02e7fc700 1 -- 172.21.15.140:0/3322957251 >> v1:172.21.15.152:6789/0 conn(0x7fc038060ee0 legacy=0x7fc03809cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc02e7fc700 1 -- 172.21.15.140:0/3322957251 >> v1:172.21.15.140:6789/0 conn(0x7fc03809df40 legacy=0x7fc038145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc02e7fc700 1 -- 172.21.15.140:0/3322957251 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03809e320 con 0x7fc0380b4d60 2022-07-21T01:49:09.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc02e7fc700 1 --2- 172.21.15.140:0/3322957251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0380b4d60 0x7fc038145d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fc020002f70 tx=0x7fc020006df0).ready entity=mon.1 client_cookie=42bd1b6a0fb482f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:09.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc0367fc700 1 -- 172.21.15.140:0/3322957251 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc02000d610 con 0x7fc0380b4d60 2022-07-21T01:49:09.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc0367fc700 1 -- 172.21.15.140:0/3322957251 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc020000e90 con 0x7fc0380b4d60 2022-07-21T01:49:09.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc0367fc700 1 -- 172.21.15.140:0/3322957251 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc02000d960 con 0x7fc0380b4d60 2022-07-21T01:49:09.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc03d496700 1 -- 172.21.15.140:0/3322957251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0380b4d60 msgr2=0x7fc038145d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc03d496700 1 --2- 172.21.15.140:0/3322957251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0380b4d60 0x7fc038145d90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fc020002f70 tx=0x7fc020006df0).stop 2022-07-21T01:49:09.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc03d496700 1 -- 172.21.15.140:0/3322957251 shutdown_connections 2022-07-21T01:49:09.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc03d496700 1 --2- 172.21.15.140:0/3322957251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0380b4d60 0x7fc038145d90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc03d496700 1 -- 172.21.15.140:0/3322957251 >> 172.21.15.140:0/3322957251 conn(0x7fc0380b5f30 msgr2=0x7fc0380b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:09.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc03d496700 1 -- 172.21.15.140:0/3322957251 shutdown_connections 2022-07-21T01:49:09.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc03d496700 1 -- 172.21.15.140:0/3322957251 wait complete. 2022-07-21T01:49:09.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc03d496700 1 Processor -- start 2022-07-21T01:49:09.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.717+0000 7fc03d496700 1 -- start start 2022-07-21T01:49:09.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc03d496700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc038060ee0 0x7fc038149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc03d496700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03809df40 0x7fc0381412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc03d496700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0380b4d60 0x7fc0381417d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc03d496700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc038141cc0 con 0x7fc0380b4d60 2022-07-21T01:49:09.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc03d496700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03814a830 con 0x7fc038060ee0 2022-07-21T01:49:09.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc03d496700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380afc40 con 0x7fc03809df40 2022-07-21T01:49:09.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc0377fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0380b4d60 0x7fc0381417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc0377fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0380b4d60 0x7fc0381417d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33640/0 (socket says 172.21.15.140:33640) 2022-07-21T01:49:09.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc0377fe700 1 -- 172.21.15.140:0/2657155231 learned_addr learned my addr 172.21.15.140:0/2657155231 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:09.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc02e7fc700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03809df40 0x7fc0381412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc036ffd700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc038060ee0 0x7fc038149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc0377fe700 1 -- 172.21.15.140:0/2657155231 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03809df40 msgr2=0x7fc0381412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc0377fe700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03809df40 0x7fc0381412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc0377fe700 1 -- 172.21.15.140:0/2657155231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc038060ee0 msgr2=0x7fc038149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc0377fe700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc038060ee0 0x7fc038149a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc0377fe700 1 -- 172.21.15.140:0/2657155231 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02000b180 con 0x7fc0380b4d60 2022-07-21T01:49:09.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc02e7fc700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03809df40 0x7fc0381412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:09.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc0377fe700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0380b4d60 0x7fc0381417d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fc03805f350 tx=0x7fc030007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:09.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc034ff9700 1 -- 172.21.15.140:0/2657155231 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc03000f470 con 0x7fc0380b4d60 2022-07-21T01:49:09.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc034ff9700 1 -- 172.21.15.140:0/2657155231 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03000f5d0 con 0x7fc0380b4d60 2022-07-21T01:49:09.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc03d496700 1 -- 172.21.15.140:0/2657155231 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0380af050 con 0x7fc0380b4d60 2022-07-21T01:49:09.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc034ff9700 1 -- 172.21.15.140:0/2657155231 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc03000f930 con 0x7fc0380b4d60 2022-07-21T01:49:09.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.721+0000 7fc03d496700 1 -- 172.21.15.140:0/2657155231 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0380aedf0 con 0x7fc0380b4d60 2022-07-21T01:49:09.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.725+0000 7fc034ff9700 1 -- 172.21.15.140:0/2657155231 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc030019030 con 0x7fc0380b4d60 2022-07-21T01:49:09.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.725+0000 7fc03d496700 1 -- 172.21.15.140:0/2657155231 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0381556e0 con 0x7fc0380b4d60 2022-07-21T01:49:09.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.725+0000 7fc034ff9700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc01c053980 0x7fc01c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.725+0000 7fc036ffd700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc01c053980 0x7fc01c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.725+0000 7fc034ff9700 1 -- 172.21.15.140:0/2657155231 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc03006b1c0 con 0x7fc0380b4d60 2022-07-21T01:49:09.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.725+0000 7fc036ffd700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc01c053980 0x7fc01c055e40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fc02800d000 tx=0x7fc028001760).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:09.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.733+0000 7fc034ff9700 1 -- 172.21.15.140:0/2657155231 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc0381556e0 con 0x7fc0380b4d60 2022-07-21T01:49:09.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.857+0000 7fc03d496700 1 -- 172.21.15.140:0/2657155231 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc0381556e0 con 0x7fc0380b4d60 2022-07-21T01:49:09.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.857+0000 7fc034ff9700 1 -- 172.21.15.140:0/2657155231 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v26) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc0381556e0 con 0x7fc0380b4d60 2022-07-21T01:49:09.863 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:49:09.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 -- 172.21.15.140:0/2657155231 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc01c053980 msgr2=0x7fc01c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc01c053980 0x7fc01c055e40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fc02800d000 tx=0x7fc028001760).stop 2022-07-21T01:49:09.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 -- 172.21.15.140:0/2657155231 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0380b4d60 msgr2=0x7fc0381417d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0380b4d60 0x7fc0381417d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fc03805f350 tx=0x7fc030007f40).stop 2022-07-21T01:49:09.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 -- 172.21.15.140:0/2657155231 shutdown_connections 2022-07-21T01:49:09.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc01c053980 0x7fc01c055e40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0380b4d60 0x7fc0381417d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03809df40 0x7fc0381412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 --2- 172.21.15.140:0/2657155231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc038060ee0 0x7fc038149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 -- 172.21.15.140:0/2657155231 >> 172.21.15.140:0/2657155231 conn(0x7fc0380b5f30 msgr2=0x7fc038066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:09.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 -- 172.21.15.140:0/2657155231 shutdown_connections 2022-07-21T01:49:09.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.861+0000 7fc03d496700 1 -- 172.21.15.140:0/2657155231 wait complete. 2022-07-21T01:49:09.880 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:49:09.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688d096700 1 Processor -- start 2022-07-21T01:49:09.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688d096700 1 -- start start 2022-07-21T01:49:09.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688d096700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6888143940 0x7f688813f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:09.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688d096700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6888143ec0 con 0x7f68880ac650 2022-07-21T01:49:09.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688d096700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68880ac010 con 0x7f68880b2550 2022-07-21T01:49:09.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688d096700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6888140c10 con 0x7f6888143940 2022-07-21T01:49:09.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688659c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f68880ac650 0x7f688813efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47792/0 (socket says 172.21.15.140:47792) 2022-07-21T01:49:09.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688659c700 1 -- 172.21.15.140:0/3071396790 learned_addr learned my addr 172.21.15.140:0/3071396790 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:09.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688759e700 1 --2- 172.21.15.140:0/3071396790 >> v2:172.21.15.152:3300/0 conn(0x7f6888143940 0x7f688813f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:09.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f6885d9b700 1 -- 172.21.15.140:0/3071396790 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160773972 0 0) 0x7f6888143ec0 con 0x7f68880ac650 2022-07-21T01:49:09.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f6885d9b700 1 -- 172.21.15.140:0/3071396790 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6868003260 con 0x7f68880ac650 2022-07-21T01:49:09.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688759e700 1 -- 172.21.15.140:0/3071396790 >> v1:172.21.15.152:6789/0 conn(0x7f68880b2550 legacy=0x7f68880ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688759e700 1 -- 172.21.15.140:0/3071396790 >> v1:172.21.15.140:6789/0 conn(0x7f68880ac650 legacy=0x7f688813efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f6885d9b700 1 -- 172.21.15.140:0/3071396790 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2746232113 0 0) 0x7f6868003260 con 0x7f68880ac650 2022-07-21T01:49:09.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688759e700 1 -- 172.21.15.140:0/3071396790 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68880b2930 con 0x7f6888143940 2022-07-21T01:49:09.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f688759e700 1 --2- 172.21.15.140:0/3071396790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6888143940 0x7f688813f6c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f68800032f0 tx=0x7f68800091d0).ready entity=mon.1 client_cookie=41f88ee942fbd49e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:09.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f6885d9b700 1 -- 172.21.15.140:0/3071396790 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f688000b8b0 con 0x7f6888143940 2022-07-21T01:49:09.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f6885d9b700 1 -- 172.21.15.140:0/3071396790 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688000ba10 con 0x7f6888143940 2022-07-21T01:49:09.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.981+0000 7f6885d9b700 1 -- 172.21.15.140:0/3071396790 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f688000bd70 con 0x7f6888143940 2022-07-21T01:49:09.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 -- 172.21.15.140:0/3071396790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6888143940 msgr2=0x7f688813f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:09.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 --2- 172.21.15.140:0/3071396790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6888143940 0x7f688813f6c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f68800032f0 tx=0x7f68800091d0).stop 2022-07-21T01:49:09.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 -- 172.21.15.140:0/3071396790 shutdown_connections 2022-07-21T01:49:09.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 --2- 172.21.15.140:0/3071396790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6888143940 0x7f688813f6c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:09.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 -- 172.21.15.140:0/3071396790 >> 172.21.15.140:0/3071396790 conn(0x7f68880b36c0 msgr2=0x7f68880b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:09.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 -- 172.21.15.140:0/3071396790 shutdown_connections 2022-07-21T01:49:09.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 -- 172.21.15.140:0/3071396790 wait complete. 2022-07-21T01:49:10.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 Processor -- start 2022-07-21T01:49:10.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 -- start start 2022-07-21T01:49:10.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68880ac650 0x7f688809a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68880b2550 0x7f688813fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6888143940 0x7f688809af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688813cba0 con 0x7f68880b2550 2022-07-21T01:49:10.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68880996c0 con 0x7f68880ac650 2022-07-21T01:49:10.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688d096700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68880998f0 con 0x7f6888143940 2022-07-21T01:49:10.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68880b2550 0x7f688813fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68880b2550 0x7f688813fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33650/0 (socket says 172.21.15.140:33650) 2022-07-21T01:49:10.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.985+0000 7f688659c700 1 -- 172.21.15.140:0/1734100676 learned_addr learned my addr 172.21.15.140:0/1734100676 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:10.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f688759e700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6888143940 0x7f688809af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f6886d9d700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68880ac650 0x7f688809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f688659c700 1 -- 172.21.15.140:0/1734100676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6888143940 msgr2=0x7f688809af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f688659c700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6888143940 0x7f688809af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f688659c700 1 -- 172.21.15.140:0/1734100676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68880ac650 msgr2=0x7f688809a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f688659c700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68880ac650 0x7f688809a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f688659c700 1 -- 172.21.15.140:0/1734100676 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6880009470 con 0x7f68880b2550 2022-07-21T01:49:10.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f688659c700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68880b2550 0x7f688813fbc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f6870000bc0 tx=0x7f6870001800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:10.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f687ffff700 1 -- 172.21.15.140:0/1734100676 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6870007170 con 0x7f68880b2550 2022-07-21T01:49:10.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f687ffff700 1 -- 172.21.15.140:0/1734100676 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68700072d0 con 0x7f68880b2550 2022-07-21T01:49:10.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f688d096700 1 -- 172.21.15.140:0/1734100676 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6888099b90 con 0x7f68880b2550 2022-07-21T01:49:10.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f687ffff700 1 -- 172.21.15.140:0/1734100676 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6870004710 con 0x7f68880b2550 2022-07-21T01:49:10.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f688d096700 1 -- 172.21.15.140:0/1734100676 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6888151d80 con 0x7f68880b2550 2022-07-21T01:49:10.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f688d096700 1 -- 172.21.15.140:0/1734100676 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688805fcf0 con 0x7f68880b2550 2022-07-21T01:49:10.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f687ffff700 1 -- 172.21.15.140:0/1734100676 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6870004a80 con 0x7f68880b2550 2022-07-21T01:49:10.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.989+0000 7f687ffff700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f686c053a70 0x7f686c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.993+0000 7f687ffff700 1 -- 172.21.15.140:0/1734100676 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f687006b720 con 0x7f68880b2550 2022-07-21T01:49:10.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.993+0000 7f6886d9d700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f686c053a70 0x7f686c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.993+0000 7f6886d9d700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f686c053a70 0x7f686c055f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f687800b000 tx=0x7f6878009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:10.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:09.997+0000 7f687ffff700 1 -- 172.21.15.140:0/1734100676 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6870042970 con 0x7f68880b2550 2022-07-21T01:49:10.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.121+0000 7f688d096700 1 -- 172.21.15.140:0/1734100676 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f68880574e0 con 0x7f68880b2550 2022-07-21T01:49:10.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f687ffff700 1 -- 172.21.15.140:0/1734100676 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6870001940 con 0x7f68880b2550 2022-07-21T01:49:10.127 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:49:10.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 -- 172.21.15.140:0/1734100676 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f686c053a70 msgr2=0x7f686c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f686c053a70 0x7f686c055f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f687800b000 tx=0x7f6878009040).stop 2022-07-21T01:49:10.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 -- 172.21.15.140:0/1734100676 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68880b2550 msgr2=0x7f688813fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68880b2550 0x7f688813fbc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f6870000bc0 tx=0x7f6870001800).stop 2022-07-21T01:49:10.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 -- 172.21.15.140:0/1734100676 shutdown_connections 2022-07-21T01:49:10.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f686c053a70 0x7f686c055f30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6888143940 0x7f688809af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68880b2550 0x7f688813fbc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 --2- 172.21.15.140:0/1734100676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68880ac650 0x7f688809a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 -- 172.21.15.140:0/1734100676 >> 172.21.15.140:0/1734100676 conn(0x7f68880b36c0 msgr2=0x7f68880b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:10.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 -- 172.21.15.140:0/1734100676 shutdown_connections 2022-07-21T01:49:10.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.125+0000 7f688d096700 1 -- 172.21.15.140:0/1734100676 wait complete. 2022-07-21T01:49:10.140 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:49:10.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18e1c93700 1 Processor -- start 2022-07-21T01:49:10.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18e1c93700 1 -- start start 2022-07-21T01:49:10.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18e1c93700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f18dc0b0570 0x7f18dc0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18e1c93700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc1424e0 0x7f18dc0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18e1c93700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18dc1433d0 con 0x7f18dc0aea10 2022-07-21T01:49:10.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18e1c93700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc143b10 con 0x7f18dc0b0570 2022-07-21T01:49:10.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18e1c93700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc143670 con 0x7f18dc1424e0 2022-07-21T01:49:10.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18d3fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f18dc0aea10 0x7f18dc141dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47804/0 (socket says 172.21.15.140:47804) 2022-07-21T01:49:10.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18d3fff700 1 -- 172.21.15.140:0/2952866528 learned_addr learned my addr 172.21.15.140:0/2952866528 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:10.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18e1492700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc1424e0 0x7f18dc0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170657555 0 0) 0x7f18dc1433d0 con 0x7f18dc0aea10 2022-07-21T01:49:10.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18e0c91700 1 --2- 172.21.15.140:0/2952866528 >> v2:172.21.15.152:3300/0 conn(0x7f18dc0b0570 0x7f18dc0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.241+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c8003260 con 0x7f18dc0aea10 2022-07-21T01:49:10.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 745024644 0 0) 0x7f18c8003260 con 0x7f18dc0aea10 2022-07-21T01:49:10.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc1424e0 msgr2=0x7f18dc0ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:10.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc1424e0 0x7f18dc0ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 >> v2:172.21.15.152:3300/0 conn(0x7f18dc0b0570 msgr2=0x7f18dc0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:10.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 --2- 172.21.15.140:0/2952866528 >> v2:172.21.15.152:3300/0 conn(0x7f18dc0b0570 0x7f18dc0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0b1010 con 0x7f18dc0aea10 2022-07-21T01:49:10.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f18d8001660 con 0x7f18dc0aea10 2022-07-21T01:49:10.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 >> v1:172.21.15.140:6789/0 conn(0x7f18dc0aea10 legacy=0x7f18dc141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 0x7f18c8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18c8004500 0x7f18c8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc1424e0 0x7f18c8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc143b10 con 0x7f18c8003750 2022-07-21T01:49:10.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc143670 con 0x7f18c8004500 2022-07-21T01:49:10.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c800b750 con 0x7f18dc1424e0 2022-07-21T01:49:10.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e0c91700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 0x7f18c8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d3fff700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18c8004500 0x7f18c8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e1492700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc1424e0 0x7f18c8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e0c91700 1 -- 172.21.15.140:0/2952866528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc1424e0 msgr2=0x7f18c8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:10.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e0c91700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc1424e0 0x7f18c8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e0c91700 1 -- 172.21.15.140:0/2952866528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18c8004500 msgr2=0x7f18c8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:10.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e0c91700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18c8004500 0x7f18c8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e0c91700 1 -- 172.21.15.140:0/2952866528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8003260 con 0x7f18c8003750 2022-07-21T01:49:10.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e0c91700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 0x7f18c8003b70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f18cc00c000 tx=0x7f18cc006df0).ready entity=mon.0 client_cookie=9ac297f2d003af88 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:10.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18cc00e830 con 0x7f18c8003750 2022-07-21T01:49:10.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc00e990 con 0x7f18c8003750 2022-07-21T01:49:10.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18d37fe700 1 -- 172.21.15.140:0/2952866528 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18cc00ecf0 con 0x7f18c8003750 2022-07-21T01:49:10.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e1c93700 1 -- 172.21.15.140:0/2952866528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 msgr2=0x7f18c8003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e1c93700 1 --2- 172.21.15.140:0/2952866528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 0x7f18c8003b70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f18cc00c000 tx=0x7f18cc006df0).stop 2022-07-21T01:49:10.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e1492700 1 -- 172.21.15.140:0/2952866528 reap_dead start 2022-07-21T01:49:10.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e1c93700 1 -- 172.21.15.140:0/2952866528 shutdown_connections 2022-07-21T01:49:10.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e1c93700 1 -- 172.21.15.140:0/2952866528 >> 172.21.15.140:0/2952866528 conn(0x7f18dc0b9360 msgr2=0x7f18dc0b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:10.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e1c93700 1 -- 172.21.15.140:0/2952866528 shutdown_connections 2022-07-21T01:49:10.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.245+0000 7f18e1c93700 1 -- 172.21.15.140:0/2952866528 wait complete. 2022-07-21T01:49:10.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e1c93700 1 Processor -- start 2022-07-21T01:49:10.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e1c93700 1 -- start start 2022-07-21T01:49:10.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e1c93700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 0x7f18dc145050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e1c93700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc0b2720 0x7f18dc112ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e1c93700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18dc1133e0 0x7f18dc151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e1c93700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0b3d10 con 0x7f18c8003750 2022-07-21T01:49:10.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e0c91700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 0x7f18dc145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e1c93700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc1127f0 con 0x7f18dc1133e0 2022-07-21T01:49:10.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e1c93700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0b3b10 con 0x7f18dc0b2720 2022-07-21T01:49:10.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e0c91700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 0x7f18dc145050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33670/0 (socket says 172.21.15.140:33670) 2022-07-21T01:49:10.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e0c91700 1 -- 172.21.15.140:0/2145619313 learned_addr learned my addr 172.21.15.140:0/2145619313 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:10.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18d3fff700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc0b2720 0x7f18dc112ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e1492700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18dc1133e0 0x7f18dc151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e0c91700 1 -- 172.21.15.140:0/2145619313 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc0b2720 msgr2=0x7f18dc112ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e0c91700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc0b2720 0x7f18dc112ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e0c91700 1 -- 172.21.15.140:0/2145619313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18dc1133e0 msgr2=0x7f18dc151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e0c91700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18dc1133e0 0x7f18dc151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e0c91700 1 -- 172.21.15.140:0/2145619313 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0b3f40 con 0x7f18c8003750 2022-07-21T01:49:10.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.249+0000 7f18e1492700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18dc1133e0 0x7f18dc151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:10.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18e0c91700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 0x7f18dc145050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f18cc00c000 tx=0x7f18cc019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:10.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18d1ffb700 1 -- 172.21.15.140:0/2145619313 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18cc01b820 con 0x7f18c8003750 2022-07-21T01:49:10.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18d1ffb700 1 -- 172.21.15.140:0/2145619313 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc000e90 con 0x7f18c8003750 2022-07-21T01:49:10.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18e1c93700 1 -- 172.21.15.140:0/2145619313 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc112a50 con 0x7f18c8003750 2022-07-21T01:49:10.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18d1ffb700 1 -- 172.21.15.140:0/2145619313 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18cc01bb70 con 0x7f18c8003750 2022-07-21T01:49:10.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18e1c93700 1 -- 172.21.15.140:0/2145619313 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc1523e0 con 0x7f18c8003750 2022-07-21T01:49:10.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18e1c93700 1 -- 172.21.15.140:0/2145619313 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc146ba0 con 0x7f18c8003750 2022-07-21T01:49:10.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18d1ffb700 1 -- 172.21.15.140:0/2145619313 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f18cc01bcd0 con 0x7f18c8003750 2022-07-21T01:49:10.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18d1ffb700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18bc053950 0x7f18bc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18d1ffb700 1 -- 172.21.15.140:0/2145619313 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f18cc06cfa0 con 0x7f18c8003750 2022-07-21T01:49:10.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18d3fff700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18bc053950 0x7f18bc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.253+0000 7f18d3fff700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18bc053950 0x7f18bc055e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f18d8005000 tx=0x7f18d8001e50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:10.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.257+0000 7f18d1ffb700 1 -- 172.21.15.140:0/2145619313 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f18dc146ba0 con 0x7f18c8003750 2022-07-21T01:49:10.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.384+0000 7f18e1c93700 1 -- 172.21.15.140:0/2145619313 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f18dc146ba0 con 0x7f18c8003750 2022-07-21T01:49:10.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.384+0000 7f18d1ffb700 1 -- 172.21.15.140:0/2145619313 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+143 (secure 0 0 0) 0x7f18dc146ba0 con 0x7f18c8003750 2022-07-21T01:49:10.388 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:10.388 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:49:10.389 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:10.389 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:49:10.389 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:49:10.389 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:49:10.390 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:49:10.390 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:49:10.390 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:49:10.390 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:49:10.390 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:49:10.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 -- 172.21.15.140:0/2145619313 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18bc053950 msgr2=0x7f18bc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18bc053950 0x7f18bc055e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f18d8005000 tx=0x7f18d8001e50).stop 2022-07-21T01:49:10.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 -- 172.21.15.140:0/2145619313 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 msgr2=0x7f18dc145050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 0x7f18dc145050 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f18cc00c000 tx=0x7f18cc019000).stop 2022-07-21T01:49:10.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 -- 172.21.15.140:0/2145619313 shutdown_connections 2022-07-21T01:49:10.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18bc053950 0x7f18bc055e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18dc1133e0 0x7f18dc151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18dc0b2720 0x7f18dc112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 --2- 172.21.15.140:0/2145619313 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18c8003750 0x7f18dc145050 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 -- 172.21.15.140:0/2145619313 >> 172.21.15.140:0/2145619313 conn(0x7f18dc0b9360 msgr2=0x7f18dc141090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:10.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 -- 172.21.15.140:0/2145619313 shutdown_connections 2022-07-21T01:49:10.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.388+0000 7f18e1c93700 1 -- 172.21.15.140:0/2145619313 wait complete. 2022-07-21T01:49:10.405 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:49:10.406 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:49:10.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.508+0000 7faac2bd2700 1 Processor -- start 2022-07-21T01:49:10.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.508+0000 7faac2bd2700 1 -- start start 2022-07-21T01:49:10.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac2bd2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faabc0b1fa0 0x7faabc13fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac2bd2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc05bd90 con 0x7faabc0b3770 2022-07-21T01:49:10.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac2bd2700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc05c820 con 0x7faabc0b7bb0 2022-07-21T01:49:10.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac2bd2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc1467d0 con 0x7faabc0b1fa0 2022-07-21T01:49:10.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac13cf700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7faabc0b3770 0x7faabc05cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47828/0 (socket says 172.21.15.140:47828) 2022-07-21T01:49:10.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac13cf700 1 -- 172.21.15.140:0/479481311 learned_addr learned my addr 172.21.15.140:0/479481311 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:10.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac23d1700 1 --2- 172.21.15.140:0/479481311 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faabc0b1fa0 0x7faabc13fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac0bce700 1 -- 172.21.15.140:0/479481311 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445106519 0 0) 0x7faabc05bd90 con 0x7faabc0b3770 2022-07-21T01:49:10.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac0bce700 1 -- 172.21.15.140:0/479481311 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa4003260 con 0x7faabc0b3770 2022-07-21T01:49:10.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac0bce700 1 -- 172.21.15.140:0/479481311 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244514549 0 0) 0x7faabc05c820 con 0x7faabc0b7bb0 2022-07-21T01:49:10.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac0bce700 1 -- 172.21.15.140:0/479481311 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faabc05bd90 con 0x7faabc0b7bb0 2022-07-21T01:49:10.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac23d1700 1 -- 172.21.15.140:0/479481311 >> v1:172.21.15.152:6789/0 conn(0x7faabc0b7bb0 legacy=0x7faabc1460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac23d1700 1 -- 172.21.15.140:0/479481311 >> v1:172.21.15.140:6789/0 conn(0x7faabc0b3770 legacy=0x7faabc05cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac0bce700 1 -- 172.21.15.140:0/479481311 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 879633177 0 0) 0x7faaa4003260 con 0x7faabc0b3770 2022-07-21T01:49:10.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac23d1700 1 -- 172.21.15.140:0/479481311 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc0b7f90 con 0x7faabc0b1fa0 2022-07-21T01:49:10.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac23d1700 1 --2- 172.21.15.140:0/479481311 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faabc0b1fa0 0x7faabc13fc40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7faaac001110 tx=0x7faab40091d0).ready entity=mon.2 client_cookie=830eb726e9fbf69a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:10.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac0bce700 1 -- 172.21.15.140:0/479481311 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaac001680 con 0x7faabc0b1fa0 2022-07-21T01:49:10.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac0bce700 1 -- 172.21.15.140:0/479481311 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab400f510 con 0x7faabc0b1fa0 2022-07-21T01:49:10.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.512+0000 7faac0bce700 1 -- 172.21.15.140:0/479481311 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faab400f680 con 0x7faabc0b1fa0 2022-07-21T01:49:10.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 -- 172.21.15.140:0/479481311 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faabc0b1fa0 msgr2=0x7faabc13fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 --2- 172.21.15.140:0/479481311 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faabc0b1fa0 0x7faabc13fc40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7faaac001110 tx=0x7faab40091d0).stop 2022-07-21T01:49:10.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 -- 172.21.15.140:0/479481311 shutdown_connections 2022-07-21T01:49:10.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 --2- 172.21.15.140:0/479481311 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faabc0b1fa0 0x7faabc13fc40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 -- 172.21.15.140:0/479481311 >> 172.21.15.140:0/479481311 conn(0x7faabc065720 msgr2=0x7faabc0b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:10.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 -- 172.21.15.140:0/479481311 shutdown_connections 2022-07-21T01:49:10.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 -- 172.21.15.140:0/479481311 wait complete. 2022-07-21T01:49:10.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 Processor -- start 2022-07-21T01:49:10.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 -- start start 2022-07-21T01:49:10.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faabc0b1fa0 0x7faabc0b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc0b3770 0x7faabc13fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faabc0b7bb0 0x7faabc115df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0b0330 con 0x7faabc0b1fa0 2022-07-21T01:49:10.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0af520 con 0x7faabc0b3770 2022-07-21T01:49:10.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac2bd2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0b0560 con 0x7faabc0b7bb0 2022-07-21T01:49:10.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac1bd0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faabc0b1fa0 0x7faabc0b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac1bd0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faabc0b1fa0 0x7faabc0b6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33684/0 (socket says 172.21.15.140:33684) 2022-07-21T01:49:10.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac1bd0700 1 -- 172.21.15.140:0/2977917016 learned_addr learned my addr 172.21.15.140:0/2977917016 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:10.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac13cf700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc0b3770 0x7faabc13fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.516+0000 7faac23d1700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faabc0b7bb0 0x7faabc115df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faac1bd0700 1 -- 172.21.15.140:0/2977917016 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faabc0b7bb0 msgr2=0x7faabc115df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faac1bd0700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faabc0b7bb0 0x7faabc115df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faac1bd0700 1 -- 172.21.15.140:0/2977917016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc0b3770 msgr2=0x7faabc13fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:10.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faac1bd0700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc0b3770 0x7faabc13fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:10.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faac1bd0700 1 -- 172.21.15.140:0/2977917016 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab4009470 con 0x7faabc0b1fa0 2022-07-21T01:49:10.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faac1bd0700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faabc0b1fa0 0x7faabc0b6430 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7faab8001110 tx=0x7faab8007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:10.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faaaaffd700 1 -- 172.21.15.140:0/2977917016 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faab8012400 con 0x7faabc0b1fa0 2022-07-21T01:49:10.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faaaaffd700 1 -- 172.21.15.140:0/2977917016 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab8012560 con 0x7faabc0b1fa0 2022-07-21T01:49:10.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faac2bd2700 1 -- 172.21.15.140:0/2977917016 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faabc0af920 con 0x7faabc0b1fa0 2022-07-21T01:49:10.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faaaaffd700 1 -- 172.21.15.140:0/2977917016 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faab80128c0 con 0x7faabc0b1fa0 2022-07-21T01:49:10.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faac2bd2700 1 -- 172.21.15.140:0/2977917016 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc152ee0 con 0x7faabc0b1fa0 2022-07-21T01:49:10.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faac2bd2700 1 -- 172.21.15.140:0/2977917016 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faabc115be0 con 0x7faabc0b1fa0 2022-07-21T01:49:10.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faaaaffd700 1 -- 172.21.15.140:0/2977917016 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faab8019030 con 0x7faabc0b1fa0 2022-07-21T01:49:10.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faaaaffd700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa9c053b00 0x7faa9c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:10.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.520+0000 7faaaaffd700 1 -- 172.21.15.140:0/2977917016 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5253+0+0 (secure 0 0 0) 0x7faab806aeb0 con 0x7faabc0b1fa0 2022-07-21T01:49:10.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.524+0000 7faac13cf700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa9c053b00 0x7faa9c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:10.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.524+0000 7faac13cf700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa9c053b00 0x7faa9c055fc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7faaac00d000 tx=0x7faaac001770).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:10.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.532+0000 7faaaaffd700 1 -- 172.21.15.140:0/2977917016 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faabc115be0 con 0x7faabc0b1fa0 2022-07-21T01:49:10.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:10.656+0000 7faac2bd2700 1 -- 172.21.15.140:0/2977917016 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7faabc115be0 con 0x7faabc0b1fa0 2022-07-21T01:49:11.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.260+0000 7faaaaffd700 1 -- 172.21.15.140:0/2977917016 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v29) v1 ==== 133+0+0 (secure 0 0 0) 0x7faabc115be0 con 0x7faabc0b1fa0 2022-07-21T01:49:11.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.264+0000 7faac2bd2700 1 -- 172.21.15.140:0/2977917016 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa9c053b00 msgr2=0x7faa9c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:11.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.264+0000 7faac2bd2700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa9c053b00 0x7faa9c055fc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7faaac00d000 tx=0x7faaac001770).stop 2022-07-21T01:49:11.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.264+0000 7faac2bd2700 1 -- 172.21.15.140:0/2977917016 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faabc0b1fa0 msgr2=0x7faabc0b6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:11.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.264+0000 7faac2bd2700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faabc0b1fa0 0x7faabc0b6430 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7faab8001110 tx=0x7faab8007ae0).stop 2022-07-21T01:49:11.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.264+0000 7faac2bd2700 1 -- 172.21.15.140:0/2977917016 shutdown_connections 2022-07-21T01:49:11.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.264+0000 7faac2bd2700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa9c053b00 0x7faa9c055fc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:11.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.264+0000 7faac2bd2700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faabc0b7bb0 0x7faabc115df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:11.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.264+0000 7faac2bd2700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc0b3770 0x7faabc13fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:11.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.264+0000 7faac2bd2700 1 --2- 172.21.15.140:0/2977917016 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faabc0b1fa0 0x7faabc0b6430 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:11.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.264+0000 7faac2bd2700 1 -- 172.21.15.140:0/2977917016 >> 172.21.15.140:0/2977917016 conn(0x7faabc065720 msgr2=0x7faabc1410e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:11.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.268+0000 7faac2bd2700 1 -- 172.21.15.140:0/2977917016 shutdown_connections 2022-07-21T01:49:11.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.268+0000 7faac2bd2700 1 -- 172.21.15.140:0/2977917016 wait complete. 2022-07-21T01:49:11.274 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:49:11.280 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:49:11.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ff19700 1 Processor -- start 2022-07-21T01:49:11.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ff19700 1 -- start start 2022-07-21T01:49:11.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ff19700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5b880967e0 0x7f5b88140860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:11.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ff19700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5b880ac680 0x7f5b880b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:11.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ff19700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b8813fe80 con 0x7f5b880b2210 2022-07-21T01:49:11.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ff19700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b88140120 con 0x7f5b880967e0 2022-07-21T01:49:11.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ff19700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b881403c0 con 0x7f5b880ac680 2022-07-21T01:49:11.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8d4b4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5b880967e0 0x7f5b88140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:11.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8d4b4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5b880967e0 0x7f5b88140860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33696/0 (socket says 172.21.15.140:33696) 2022-07-21T01:49:11.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8d4b4700 1 -- 172.21.15.140:0/1815030779 learned_addr learned my addr 172.21.15.140:0/1815030779 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:11.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8dcb5700 1 --1- 172.21.15.140:0/1815030779 >> v1:172.21.15.140:6789/0 conn(0x7f5b880b2210 0x7f5b880ab0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47840/0 (socket says 172.21.15.140:47840) 2022-07-21T01:49:11.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8e4b6700 1 --2- 172.21.15.140:0/1815030779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5b880ac680 0x7f5b880b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:11.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8d4b4700 1 -- 172.21.15.140:0/1815030779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5b880ac680 msgr2=0x7f5b880b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:11.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8d4b4700 1 --2- 172.21.15.140:0/1815030779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5b880ac680 0x7f5b880b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:11.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8d4b4700 1 -- 172.21.15.140:0/1815030779 >> v1:172.21.15.140:6789/0 conn(0x7f5b880b2210 legacy=0x7f5b880ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:11.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ccb3700 1 -- 172.21.15.140:0/1815030779 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714618565 0 0) 0x7f5b8813fe80 con 0x7f5b880b2210 2022-07-21T01:49:11.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8d4b4700 1 -- 172.21.15.140:0/1815030779 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b880ab7c0 con 0x7f5b880967e0 2022-07-21T01:49:11.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8d4b4700 1 --2- 172.21.15.140:0/1815030779 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5b880967e0 0x7f5b88140860 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f5b78002f70 tx=0x7f5b78006eb0).ready entity=mon.0 client_cookie=a423803f1ce604c4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:11.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ccb3700 1 -- 172.21.15.140:0/1815030779 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b7800d570 con 0x7f5b880967e0 2022-07-21T01:49:11.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ccb3700 1 -- 172.21.15.140:0/1815030779 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b78000e90 con 0x7f5b880967e0 2022-07-21T01:49:11.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ccb3700 1 -- 172.21.15.140:0/1815030779 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b7800d8c0 con 0x7f5b880967e0 2022-07-21T01:49:11.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ff19700 1 -- 172.21.15.140:0/1815030779 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5b880967e0 msgr2=0x7f5b88140860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:11.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ff19700 1 --2- 172.21.15.140:0/1815030779 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5b880967e0 0x7f5b88140860 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f5b78002f70 tx=0x7f5b78006eb0).stop 2022-07-21T01:49:11.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ff19700 1 -- 172.21.15.140:0/1815030779 shutdown_connections 2022-07-21T01:49:11.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.396+0000 7f5b8ff19700 1 --2- 172.21.15.140:0/1815030779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5b880ac680 0x7f5b880b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:11.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 --2- 172.21.15.140:0/1815030779 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5b880967e0 0x7f5b88140860 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:11.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 -- 172.21.15.140:0/1815030779 >> 172.21.15.140:0/1815030779 conn(0x7f5b880b36c0 msgr2=0x7f5b880b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:11.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 -- 172.21.15.140:0/1815030779 shutdown_connections 2022-07-21T01:49:11.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 -- 172.21.15.140:0/1815030779 wait complete. 2022-07-21T01:49:11.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 Processor -- start 2022-07-21T01:49:11.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 -- start start 2022-07-21T01:49:11.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b880967e0 0x7f5b88146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:11.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5b880ac680 0x7f5b88141030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:11.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5b880b2210 0x7f5b88141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:11.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8813b900 con 0x7f5b880b2210 2022-07-21T01:49:11.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880adb10 con 0x7f5b880967e0 2022-07-21T01:49:11.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8ff19700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880aea50 con 0x7f5b880ac680 2022-07-21T01:49:11.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8e4b6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5b880b2210 0x7f5b88141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:11.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8e4b6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5b880b2210 0x7f5b88141520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33704/0 (socket says 172.21.15.140:33704) 2022-07-21T01:49:11.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8e4b6700 1 -- 172.21.15.140:0/2156382667 learned_addr learned my addr 172.21.15.140:0/2156382667 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:11.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8d4b4700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5b880ac680 0x7f5b88141030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:11.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8dcb5700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b880967e0 0x7f5b88146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:11.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8e4b6700 1 -- 172.21.15.140:0/2156382667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5b880ac680 msgr2=0x7f5b88141030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:11.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8e4b6700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5b880ac680 0x7f5b88141030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:11.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8e4b6700 1 -- 172.21.15.140:0/2156382667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b880967e0 msgr2=0x7f5b88146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:11.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8e4b6700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b880967e0 0x7f5b88146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:11.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.400+0000 7f5b8e4b6700 1 -- 172.21.15.140:0/2156382667 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7800b1e0 con 0x7f5b880b2210 2022-07-21T01:49:11.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.404+0000 7f5b8e4b6700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5b880b2210 0x7f5b88141520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f5b80006e00 tx=0x7f5b800091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:11.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.404+0000 7f5b7effd700 1 -- 172.21.15.140:0/2156382667 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b8000c850 con 0x7f5b880b2210 2022-07-21T01:49:11.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.404+0000 7f5b7effd700 1 -- 172.21.15.140:0/2156382667 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8000c9b0 con 0x7f5b880b2210 2022-07-21T01:49:11.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.404+0000 7f5b8ff19700 1 -- 172.21.15.140:0/2156382667 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b880aecf0 con 0x7f5b880b2210 2022-07-21T01:49:11.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.404+0000 7f5b7effd700 1 -- 172.21.15.140:0/2156382667 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b8000cd10 con 0x7f5b880b2210 2022-07-21T01:49:11.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.404+0000 7f5b8ff19700 1 -- 172.21.15.140:0/2156382667 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b88141a10 con 0x7f5b880b2210 2022-07-21T01:49:11.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.404+0000 7f5b8ff19700 1 -- 172.21.15.140:0/2156382667 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b880aee80 con 0x7f5b880b2210 2022-07-21T01:49:11.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.404+0000 7f5b7effd700 1 -- 172.21.15.140:0/2156382667 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5b800123f0 con 0x7f5b880b2210 2022-07-21T01:49:11.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.404+0000 7f5b7effd700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5b700539e0 0x7f5b70055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:11.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.404+0000 7f5b7effd700 1 -- 172.21.15.140:0/2156382667 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f5b8006c040 con 0x7f5b880b2210 2022-07-21T01:49:11.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.404+0000 7f5b8dcb5700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5b700539e0 0x7f5b70055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:11.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.408+0000 7f5b8dcb5700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5b700539e0 0x7f5b70055ea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f5b8400b000 tx=0x7f5b84007160).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:11.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.412+0000 7f5b7effd700 1 -- 172.21.15.140:0/2156382667 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5b800430d0 con 0x7f5b880b2210 2022-07-21T01:49:11.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:11.532+0000 7f5b8ff19700 1 -- 172.21.15.140:0/2156382667 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f5b880574e0 con 0x7f5b880b2210 2022-07-21T01:49:12.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.268+0000 7f5b7effd700 1 -- 172.21.15.140:0/2156382667 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v30) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5b800466f0 con 0x7f5b880b2210 2022-07-21T01:49:12.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 -- 172.21.15.140:0/2156382667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5b700539e0 msgr2=0x7f5b70055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5b700539e0 0x7f5b70055ea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f5b8400b000 tx=0x7f5b84007160).stop 2022-07-21T01:49:12.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 -- 172.21.15.140:0/2156382667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5b880b2210 msgr2=0x7f5b88141520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5b880b2210 0x7f5b88141520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f5b80006e00 tx=0x7f5b800091d0).stop 2022-07-21T01:49:12.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 -- 172.21.15.140:0/2156382667 shutdown_connections 2022-07-21T01:49:12.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5b700539e0 0x7f5b70055ea0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5b880b2210 0x7f5b88141520 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5b880ac680 0x7f5b88141030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 --2- 172.21.15.140:0/2156382667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b880967e0 0x7f5b88146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 -- 172.21.15.140:0/2156382667 >> 172.21.15.140:0/2156382667 conn(0x7f5b880b36c0 msgr2=0x7f5b8813f190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:12.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 -- 172.21.15.140:0/2156382667 shutdown_connections 2022-07-21T01:49:12.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.276+0000 7f5b8ff19700 1 -- 172.21.15.140:0/2156382667 wait complete. 2022-07-21T01:49:12.282 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:49:12.289 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:49:12.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.395+0000 7fd22906e700 1 Processor -- start 2022-07-21T01:49:12.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.395+0000 7fd22906e700 1 -- start start 2022-07-21T01:49:12.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.395+0000 7fd22906e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2240aabe0 0x7fd22413e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.395+0000 7fd22906e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd224061320 0x7fd22413ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22906e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2240ac790 con 0x7fd224060f40 2022-07-21T01:49:12.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22906e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22405fdf0 con 0x7fd2240aabe0 2022-07-21T01:49:12.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22906e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd224061860 con 0x7fd224061320 2022-07-21T01:49:12.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22259c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2240aabe0 0x7fd22413e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22259c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2240aabe0 0x7fd22413e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33710/0 (socket says 172.21.15.140:33710) 2022-07-21T01:49:12.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22259c700 1 -- 172.21.15.140:0/2395214375 learned_addr learned my addr 172.21.15.140:0/2395214375 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:12.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22359e700 1 --2- 172.21.15.140:0/2395214375 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd224061320 0x7fd22413ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22259c700 1 -- 172.21.15.140:0/2395214375 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd224061320 msgr2=0x7fd22413ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:12.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22259c700 1 --2- 172.21.15.140:0/2395214375 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd224061320 0x7fd22413ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22259c700 1 -- 172.21.15.140:0/2395214375 >> v1:172.21.15.140:6789/0 conn(0x7fd224060f40 legacy=0x7fd2240abd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd221d9b700 1 -- 172.21.15.140:0/2395214375 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774978939 0 0) 0x7fd2240ac790 con 0x7fd224060f40 2022-07-21T01:49:12.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22259c700 1 -- 172.21.15.140:0/2395214375 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22405f9a0 con 0x7fd2240aabe0 2022-07-21T01:49:12.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22259c700 1 --2- 172.21.15.140:0/2395214375 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2240aabe0 0x7fd22413e770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fd218002f70 tx=0x7fd218006e40).ready entity=mon.0 client_cookie=49dc7c2f2d4944f2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:12.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd221d9b700 1 -- 172.21.15.140:0/2395214375 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd21800bc90 con 0x7fd2240aabe0 2022-07-21T01:49:12.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd221d9b700 1 -- 172.21.15.140:0/2395214375 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd218000e90 con 0x7fd2240aabe0 2022-07-21T01:49:12.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd221d9b700 1 -- 172.21.15.140:0/2395214375 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2180153f0 con 0x7fd2240aabe0 2022-07-21T01:49:12.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22906e700 1 -- 172.21.15.140:0/2395214375 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2240aabe0 msgr2=0x7fd22413e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22906e700 1 --2- 172.21.15.140:0/2395214375 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2240aabe0 0x7fd22413e770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fd218002f70 tx=0x7fd218006e40).stop 2022-07-21T01:49:12.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22906e700 1 -- 172.21.15.140:0/2395214375 shutdown_connections 2022-07-21T01:49:12.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22906e700 1 --2- 172.21.15.140:0/2395214375 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd224061320 0x7fd22413ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22906e700 1 --2- 172.21.15.140:0/2395214375 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2240aabe0 0x7fd22413e770 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22906e700 1 -- 172.21.15.140:0/2395214375 >> 172.21.15.140:0/2395214375 conn(0x7fd2240b5910 msgr2=0x7fd2240b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:12.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22906e700 1 -- 172.21.15.140:0/2395214375 shutdown_connections 2022-07-21T01:49:12.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.399+0000 7fd22906e700 1 -- 172.21.15.140:0/2395214375 wait complete. 2022-07-21T01:49:12.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd22906e700 1 Processor -- start 2022-07-21T01:49:12.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd22906e700 1 -- start start 2022-07-21T01:49:12.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd22906e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd224060f40 0x7fd22414d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd22906e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd224061320 0x7fd2240acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd22906e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2240aabe0 0x7fd2240ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd22906e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240af760 con 0x7fd224060f40 2022-07-21T01:49:12.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd22906e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240add70 con 0x7fd224061320 2022-07-21T01:49:12.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd22906e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd224135590 con 0x7fd2240aabe0 2022-07-21T01:49:12.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd222d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd224060f40 0x7fd22414d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd222d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd224060f40 0x7fd22414d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33714/0 (socket says 172.21.15.140:33714) 2022-07-21T01:49:12.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd222d9d700 1 -- 172.21.15.140:0/3334718038 learned_addr learned my addr 172.21.15.140:0/3334718038 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:12.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd22359e700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2240aabe0 0x7fd2240ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd22259c700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd224061320 0x7fd2240acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd222d9d700 1 -- 172.21.15.140:0/3334718038 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2240aabe0 msgr2=0x7fd2240ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd222d9d700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2240aabe0 0x7fd2240ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd222d9d700 1 -- 172.21.15.140:0/3334718038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd224061320 msgr2=0x7fd2240acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd222d9d700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd224061320 0x7fd2240acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd222d9d700 1 -- 172.21.15.140:0/3334718038 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21800d4d0 con 0x7fd224060f40 2022-07-21T01:49:12.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd22259c700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd224061320 0x7fd2240acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:12.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.403+0000 7fd222d9d700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd224060f40 0x7fd22414d9d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7fd214000ed0 tx=0x7fd214007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:12.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.407+0000 7fd213fff700 1 -- 172.21.15.140:0/3334718038 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd214012400 con 0x7fd224060f40 2022-07-21T01:49:12.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.407+0000 7fd213fff700 1 -- 172.21.15.140:0/3334718038 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd214012560 con 0x7fd224060f40 2022-07-21T01:49:12.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.407+0000 7fd22906e700 1 -- 172.21.15.140:0/3334718038 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd224135830 con 0x7fd224060f40 2022-07-21T01:49:12.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.407+0000 7fd213fff700 1 -- 172.21.15.140:0/3334718038 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2140128c0 con 0x7fd224060f40 2022-07-21T01:49:12.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.407+0000 7fd22906e700 1 -- 172.21.15.140:0/3334718038 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22414dec0 con 0x7fd224060f40 2022-07-21T01:49:12.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.407+0000 7fd22906e700 1 -- 172.21.15.140:0/3334718038 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2240574e0 con 0x7fd224060f40 2022-07-21T01:49:12.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.407+0000 7fd213fff700 1 -- 172.21.15.140:0/3334718038 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd214019030 con 0x7fd224060f40 2022-07-21T01:49:12.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.407+0000 7fd213fff700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd200053a70 0x7fd200055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.407+0000 7fd213fff700 1 -- 172.21.15.140:0/3334718038 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fd21406b1d0 con 0x7fd224060f40 2022-07-21T01:49:12.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.407+0000 7fd22259c700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd200053a70 0x7fd200055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.407+0000 7fd22259c700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd200053a70 0x7fd200055f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fd218017000 tx=0x7fd218018000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:12.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.415+0000 7fd213fff700 1 -- 172.21.15.140:0/3334718038 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd214042210 con 0x7fd224060f40 2022-07-21T01:49:12.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.579+0000 7fd22906e700 1 -- 172.21.15.140:0/3334718038 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fd2240ae1a0 con 0x7fd224060f40 2022-07-21T01:49:12.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd213fff700 1 -- 172.21.15.140:0/3334718038 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v9) v1 ==== 187+0+0 (secure 0 0 0) 0x7fd214045830 con 0x7fd224060f40 2022-07-21T01:49:12.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 -- 172.21.15.140:0/3334718038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd200053a70 msgr2=0x7fd200055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd200053a70 0x7fd200055f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fd218017000 tx=0x7fd218018000).stop 2022-07-21T01:49:12.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 -- 172.21.15.140:0/3334718038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd224060f40 msgr2=0x7fd22414d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd224060f40 0x7fd22414d9d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7fd214000ed0 tx=0x7fd214007ac0).stop 2022-07-21T01:49:12.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 -- 172.21.15.140:0/3334718038 shutdown_connections 2022-07-21T01:49:12.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd200053a70 0x7fd200055f30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2240aabe0 0x7fd2240ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd224061320 0x7fd2240acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 --2- 172.21.15.140:0/3334718038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd224060f40 0x7fd22414d9d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 -- 172.21.15.140:0/3334718038 >> 172.21.15.140:0/3334718038 conn(0x7fd2240b5910 msgr2=0x7fd22413cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:12.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 -- 172.21.15.140:0/3334718038 shutdown_connections 2022-07-21T01:49:12.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.591+0000 7fd22906e700 1 -- 172.21.15.140:0/3334718038 wait complete. 2022-07-21T01:49:12.599 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 5 and data pool 6 2022-07-21T01:49:12.610 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:49:12.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 Processor -- start 2022-07-21T01:49:12.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 -- start start 2022-07-21T01:49:12.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f88200b5650 0x7f88200b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f88200adff0 0x7f8820147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8820147010 con 0x7f88200af8e0 2022-07-21T01:49:12.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88201472b0 con 0x7f88200adff0 2022-07-21T01:49:12.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8820146d40 con 0x7f88200b5650 2022-07-21T01:49:12.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8825787700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f88200adff0 0x7f8820147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8825787700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f88200adff0 0x7f8820147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33722/0 (socket says 172.21.15.140:33722) 2022-07-21T01:49:12.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8825787700 1 -- 172.21.15.140:0/25354979 learned_addr learned my addr 172.21.15.140:0/25354979 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:12.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8825f88700 1 --2- 172.21.15.140:0/25354979 >> v2:172.21.15.152:3300/0 conn(0x7f88200b5650 0x7f88200b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8825787700 1 -- 172.21.15.140:0/25354979 >> v2:172.21.15.152:3300/0 conn(0x7f88200b5650 msgr2=0x7f88200b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:12.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8825787700 1 --2- 172.21.15.140:0/25354979 >> v2:172.21.15.152:3300/0 conn(0x7f88200b5650 0x7f88200b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8825787700 1 -- 172.21.15.140:0/25354979 >> v1:172.21.15.152:6789/0 conn(0x7f88200af8e0 legacy=0x7f882005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8825787700 1 -- 172.21.15.140:0/25354979 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8820147c40 con 0x7f88200adff0 2022-07-21T01:49:12.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8825787700 1 --2- 172.21.15.140:0/25354979 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88200adff0 0x7f8820147750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f8810002f70 tx=0x7f8810006e50).ready entity=mon.0 client_cookie=303078d4bbd85fa7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:12.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8824f86700 1 -- 172.21.15.140:0/25354979 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f881000d610 con 0x7f88200adff0 2022-07-21T01:49:12.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8824f86700 1 -- 172.21.15.140:0/25354979 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8810000e90 con 0x7f88200adff0 2022-07-21T01:49:12.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8824f86700 1 -- 172.21.15.140:0/25354979 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f881000d9a0 con 0x7f88200adff0 2022-07-21T01:49:12.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 -- 172.21.15.140:0/25354979 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88200adff0 msgr2=0x7f8820147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 --2- 172.21.15.140:0/25354979 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88200adff0 0x7f8820147750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f8810002f70 tx=0x7f8810006e50).stop 2022-07-21T01:49:12.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 -- 172.21.15.140:0/25354979 shutdown_connections 2022-07-21T01:49:12.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 --2- 172.21.15.140:0/25354979 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88200adff0 0x7f8820147750 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 --2- 172.21.15.140:0/25354979 >> v2:172.21.15.152:3300/0 conn(0x7f88200b5650 0x7f88200b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 -- 172.21.15.140:0/25354979 >> 172.21.15.140:0/25354979 conn(0x7f88200634e0 msgr2=0x7f88200b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:12.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 -- 172.21.15.140:0/25354979 shutdown_connections 2022-07-21T01:49:12.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.699+0000 7f8826f8a700 1 -- 172.21.15.140:0/25354979 wait complete. 2022-07-21T01:49:12.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8826f8a700 1 Processor -- start 2022-07-21T01:49:12.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8826f8a700 1 -- start start 2022-07-21T01:49:12.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8826f8a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88200adff0 0x7f8820110020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8826f8a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88200af8e0 0x7f8820157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8826f8a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88200b5650 0x7f8820140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8826f8a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8820064e50 con 0x7f88200af8e0 2022-07-21T01:49:12.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8826f8a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882010e530 con 0x7f88200adff0 2022-07-21T01:49:12.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8826f8a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882010f410 con 0x7f88200b5650 2022-07-21T01:49:12.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8825787700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88200af8e0 0x7f8820157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8825787700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88200af8e0 0x7f8820157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33730/0 (socket says 172.21.15.140:33730) 2022-07-21T01:49:12.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8825787700 1 -- 172.21.15.140:0/690562947 learned_addr learned my addr 172.21.15.140:0/690562947 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:12.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8825f88700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88200adff0 0x7f8820110020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.703+0000 7f8826789700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88200b5650 0x7f8820140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8825787700 1 -- 172.21.15.140:0/690562947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88200b5650 msgr2=0x7f8820140fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8825787700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88200b5650 0x7f8820140fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8825787700 1 -- 172.21.15.140:0/690562947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88200adff0 msgr2=0x7f8820110020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8825787700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88200adff0 0x7f8820110020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8825787700 1 -- 172.21.15.140:0/690562947 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881000b1e0 con 0x7f88200af8e0 2022-07-21T01:49:12.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8825f88700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88200adff0 0x7f8820110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:12.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8826789700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88200b5650 0x7f8820140fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:12.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8825787700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88200af8e0 0x7f8820157020 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f8810002f70 tx=0x7f8810015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:12.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f880effd700 1 -- 172.21.15.140:0/690562947 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8810011900 con 0x7f88200af8e0 2022-07-21T01:49:12.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f880effd700 1 -- 172.21.15.140:0/690562947 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8810011a60 con 0x7f88200af8e0 2022-07-21T01:49:12.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8826f8a700 1 -- 172.21.15.140:0/690562947 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882010ef20 con 0x7f88200af8e0 2022-07-21T01:49:12.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f880effd700 1 -- 172.21.15.140:0/690562947 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8810011dc0 con 0x7f88200af8e0 2022-07-21T01:49:12.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8826f8a700 1 -- 172.21.15.140:0/690562947 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88200679b0 con 0x7f88200af8e0 2022-07-21T01:49:12.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8826f8a700 1 -- 172.21.15.140:0/690562947 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882010f550 con 0x7f88200af8e0 2022-07-21T01:49:12.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f880effd700 1 -- 172.21.15.140:0/690562947 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8810025030 con 0x7f88200af8e0 2022-07-21T01:49:12.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f880effd700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8800053980 0x7f8800055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f880effd700 1 -- 172.21.15.140:0/690562947 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f881006d0d0 con 0x7f88200af8e0 2022-07-21T01:49:12.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8825f88700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8800053980 0x7f8800055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.707+0000 7f8825f88700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8800053980 0x7f8800055e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f881c00c000 tx=0x7f881c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:12.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.711+0000 7f880effd700 1 -- 172.21.15.140:0/690562947 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8810043ff0 con 0x7f88200af8e0 2022-07-21T01:49:12.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.835+0000 7f8826f8a700 1 -- 172.21.15.140:0/690562947 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f882005c4a0 con 0x7f88200af8e0 2022-07-21T01:49:12.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.835+0000 7f880effd700 1 -- 172.21.15.140:0/690562947 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16147 (secure 0 0 0) 0x7f8810023070 con 0x7f88200af8e0 2022-07-21T01:49:12.839 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:12.840 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":31,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:49:12.584831+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:49:10.708637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-07-21T01:49:11.590708+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:49:12.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 -- 172.21.15.140:0/690562947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8800053980 msgr2=0x7f8800055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8800053980 0x7f8800055e40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f881c00c000 tx=0x7f881c008040).stop 2022-07-21T01:49:12.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 -- 172.21.15.140:0/690562947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88200af8e0 msgr2=0x7f8820157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88200af8e0 0x7f8820157020 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f8810002f70 tx=0x7f8810015000).stop 2022-07-21T01:49:12.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 -- 172.21.15.140:0/690562947 shutdown_connections 2022-07-21T01:49:12.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8800053980 0x7f8800055e40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88200b5650 0x7f8820140fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88200af8e0 0x7f8820157020 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 --2- 172.21.15.140:0/690562947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88200adff0 0x7f8820110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 -- 172.21.15.140:0/690562947 >> 172.21.15.140:0/690562947 conn(0x7f88200634e0 msgr2=0x7f88200afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:12.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 -- 172.21.15.140:0/690562947 shutdown_connections 2022-07-21T01:49:12.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.839+0000 7f8826f8a700 1 -- 172.21.15.140:0/690562947 wait complete. 2022-07-21T01:49:12.854 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:49:12.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.955+0000 7f8c2d089700 1 Processor -- start 2022-07-21T01:49:12.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.955+0000 7f8c2d089700 1 -- start start 2022-07-21T01:49:12.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.955+0000 7f8c2d089700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c280adff0 0x7f8c280b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.955+0000 7f8c2d089700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8c280af8e0 0x7f8c2805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.955+0000 7f8c2d089700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c28147010 con 0x7f8c280b5650 2022-07-21T01:49:12.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.955+0000 7f8c2d089700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c281472b0 con 0x7f8c280adff0 2022-07-21T01:49:12.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.955+0000 7f8c2d089700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c28146d40 con 0x7f8c280af8e0 2022-07-21T01:49:12.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.955+0000 7f8c1ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c280adff0 0x7f8c280b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.955+0000 7f8c1ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c280adff0 0x7f8c280b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33738/0 (socket says 172.21.15.140:33738) 2022-07-21T01:49:12.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.955+0000 7f8c1ffff700 1 -- 172.21.15.140:0/4198480203 learned_addr learned my addr 172.21.15.140:0/4198480203 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:12.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c2c888700 1 --2- 172.21.15.140:0/4198480203 >> v2:172.21.15.152:3300/0 conn(0x7f8c280af8e0 0x7f8c2805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c1ffff700 1 -- 172.21.15.140:0/4198480203 >> v2:172.21.15.152:3300/0 conn(0x7f8c280af8e0 msgr2=0x7f8c2805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:12.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c1ffff700 1 --2- 172.21.15.140:0/4198480203 >> v2:172.21.15.152:3300/0 conn(0x7f8c280af8e0 0x7f8c2805bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c1ffff700 1 -- 172.21.15.140:0/4198480203 >> v1:172.21.15.152:6789/0 conn(0x7f8c280b5650 legacy=0x7f8c28147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c1ffff700 1 -- 172.21.15.140:0/4198480203 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c280acae0 con 0x7f8c280adff0 2022-07-21T01:49:12.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c1ffff700 1 --2- 172.21.15.140:0/4198480203 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c280adff0 0x7f8c280b5a30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f8c10002f70 tx=0x7f8c10006e50).ready entity=mon.0 client_cookie=ff511e8cd018d3c9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:12.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c277fe700 1 -- 172.21.15.140:0/4198480203 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c1000d610 con 0x7f8c280adff0 2022-07-21T01:49:12.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c277fe700 1 -- 172.21.15.140:0/4198480203 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c10000e90 con 0x7f8c280adff0 2022-07-21T01:49:12.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c277fe700 1 -- 172.21.15.140:0/4198480203 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c1000d9a0 con 0x7f8c280adff0 2022-07-21T01:49:12.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c2d089700 1 -- 172.21.15.140:0/4198480203 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c280adff0 msgr2=0x7f8c280b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c2d089700 1 --2- 172.21.15.140:0/4198480203 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c280adff0 0x7f8c280b5a30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f8c10002f70 tx=0x7f8c10006e50).stop 2022-07-21T01:49:12.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c2d089700 1 -- 172.21.15.140:0/4198480203 shutdown_connections 2022-07-21T01:49:12.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c2d089700 1 --2- 172.21.15.140:0/4198480203 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c280adff0 0x7f8c280b5a30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c2d089700 1 --2- 172.21.15.140:0/4198480203 >> v2:172.21.15.152:3300/0 conn(0x7f8c280af8e0 0x7f8c2805bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c2d089700 1 -- 172.21.15.140:0/4198480203 >> 172.21.15.140:0/4198480203 conn(0x7f8c280634e0 msgr2=0x7f8c280b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:12.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c2d089700 1 -- 172.21.15.140:0/4198480203 shutdown_connections 2022-07-21T01:49:12.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c2d089700 1 -- 172.21.15.140:0/4198480203 wait complete. 2022-07-21T01:49:12.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c2d089700 1 Processor -- start 2022-07-21T01:49:12.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.959+0000 7f8c2d089700 1 -- start start 2022-07-21T01:49:12.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c2d089700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c280adff0 0x7f8c2810e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c2d089700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c280af8e0 0x7f8c28142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c2d089700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c280b5650 0x7f8c281433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c2d089700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c28065530 con 0x7f8c280adff0 2022-07-21T01:49:12.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c2d089700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c28064b10 con 0x7f8c280af8e0 2022-07-21T01:49:12.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c2d089700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c28111000 con 0x7f8c280b5650 2022-07-21T01:49:12.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c27fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c280adff0 0x7f8c2810e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c27fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c280adff0 0x7f8c2810e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33742/0 (socket says 172.21.15.140:33742) 2022-07-21T01:49:12.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c27fff700 1 -- 172.21.15.140:0/2589059909 learned_addr learned my addr 172.21.15.140:0/2589059909 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:12.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c2c888700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c280b5650 0x7f8c281433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c1ffff700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c280af8e0 0x7f8c28142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c27fff700 1 -- 172.21.15.140:0/2589059909 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c280b5650 msgr2=0x7f8c281433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c27fff700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c280b5650 0x7f8c281433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c27fff700 1 -- 172.21.15.140:0/2589059909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c280af8e0 msgr2=0x7f8c28142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:12.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c27fff700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c280af8e0 0x7f8c28142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:12.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c27fff700 1 -- 172.21.15.140:0/2589059909 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1000b1e0 con 0x7f8c280adff0 2022-07-21T01:49:12.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c27fff700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c280adff0 0x7f8c2810e7e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f8c28066150 tx=0x7f8c18007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:12.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c25ffb700 1 -- 172.21.15.140:0/2589059909 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c18012400 con 0x7f8c280adff0 2022-07-21T01:49:12.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c25ffb700 1 -- 172.21.15.140:0/2589059909 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c18012560 con 0x7f8c280adff0 2022-07-21T01:49:12.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c2d089700 1 -- 172.21.15.140:0/2589059909 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c28064d40 con 0x7f8c280adff0 2022-07-21T01:49:12.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c25ffb700 1 -- 172.21.15.140:0/2589059909 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c180128c0 con 0x7f8c280adff0 2022-07-21T01:49:12.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c2d089700 1 -- 172.21.15.140:0/2589059909 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c281438c0 con 0x7f8c280adff0 2022-07-21T01:49:12.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.963+0000 7f8c2d089700 1 -- 172.21.15.140:0/2589059909 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c28063d90 con 0x7f8c280adff0 2022-07-21T01:49:12.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.967+0000 7f8c25ffb700 1 -- 172.21.15.140:0/2589059909 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8c18019030 con 0x7f8c280adff0 2022-07-21T01:49:12.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.967+0000 7f8c25ffb700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c0c0539e0 0x7f8c0c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:12.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.967+0000 7f8c25ffb700 1 -- 172.21.15.140:0/2589059909 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8c1806b280 con 0x7f8c280adff0 2022-07-21T01:49:12.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.967+0000 7f8c1ffff700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c0c0539e0 0x7f8c0c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:12.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.967+0000 7f8c1ffff700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c0c0539e0 0x7f8c0c055ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8c1001a000 tx=0x7f8c10014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:12.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:12.971+0000 7f8c25ffb700 1 -- 172.21.15.140:0/2589059909 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c28063d90 con 0x7f8c280adff0 2022-07-21T01:49:13.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.091+0000 7f8c2d089700 1 -- 172.21.15.140:0/2589059909 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c28063d60 con 0x7f8c280adff0 2022-07-21T01:49:13.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c25ffb700 1 -- 172.21.15.140:0/2589059909 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16147 (secure 0 0 0) 0x7f8c18042090 con 0x7f8c280adff0 2022-07-21T01:49:13.098 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:13.099 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":31,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:49:12.584831+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:49:10.708637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-07-21T01:49:11.590708+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 -- 172.21.15.140:0/2589059909 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c0c0539e0 msgr2=0x7f8c0c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c0c0539e0 0x7f8c0c055ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f8c1001a000 tx=0x7f8c10014000).stop 2022-07-21T01:49:13.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 -- 172.21.15.140:0/2589059909 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c280adff0 msgr2=0x7f8c2810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c280adff0 0x7f8c2810e7e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f8c28066150 tx=0x7f8c18007c50).stop 2022-07-21T01:49:13.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 -- 172.21.15.140:0/2589059909 shutdown_connections 2022-07-21T01:49:13.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c0c0539e0 0x7f8c0c055ea0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c280b5650 0x7f8c281433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c280af8e0 0x7f8c28142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 --2- 172.21.15.140:0/2589059909 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c280adff0 0x7f8c2810e7e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 -- 172.21.15.140:0/2589059909 >> 172.21.15.140:0/2589059909 conn(0x7f8c280634e0 msgr2=0x7f8c281462d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:13.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 -- 172.21.15.140:0/2589059909 shutdown_connections 2022-07-21T01:49:13.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.095+0000 7f8c2d089700 1 -- 172.21.15.140:0/2589059909 wait complete. 2022-07-21T01:49:13.110 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:49:13.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f3790902700 1 Processor -- start 2022-07-21T01:49:13.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f3790902700 1 -- start start 2022-07-21T01:49:13.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f3790902700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f378c0b5b50 0x7f378c144060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f3790902700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c067c30 con 0x7f378c05c450 2022-07-21T01:49:13.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f3790902700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c0b5f30 con 0x7f378c13d610 2022-07-21T01:49:13.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f3790902700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378c067990 con 0x7f378c0b5b50 2022-07-21T01:49:13.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f3789d9b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f378c05c450 0x7f378c144550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47898/0 (socket says 172.21.15.140:47898) 2022-07-21T01:49:13.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f3789d9b700 1 -- 172.21.15.140:0/3194139616 learned_addr learned my addr 172.21.15.140:0/3194139616 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:13.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378a59c700 1 --2- 172.21.15.140:0/3194139616 >> v2:172.21.15.152:3300/0 conn(0x7f378c0b5b50 0x7f378c144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903921816 0 0) 0x7f378c067c30 con 0x7f378c05c450 2022-07-21T01:49:13.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3770003260 con 0x7f378c05c450 2022-07-21T01:49:13.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3485724218 0 0) 0x7f3770003260 con 0x7f378c05c450 2022-07-21T01:49:13.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 >> v2:172.21.15.152:3300/0 conn(0x7f378c0b5b50 msgr2=0x7f378c144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:13.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 --2- 172.21.15.140:0/3194139616 >> v2:172.21.15.152:3300/0 conn(0x7f378c0b5b50 0x7f378c144060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 >> v1:172.21.15.152:6789/0 conn(0x7f378c13d610 legacy=0x7f378c13dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c05cc60 con 0x7f378c05c450 2022-07-21T01:49:13.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f3780001660 con 0x7f378c05c450 2022-07-21T01:49:13.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 >> v1:172.21.15.140:6789/0 conn(0x7f378c05c450 legacy=0x7f378c144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3770003750 0x7f3770003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3770004500 0x7f3770004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770004e70 0x7f377000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c067990 con 0x7f3770004e70 2022-07-21T01:49:13.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377000bdd0 con 0x7f3770003750 2022-07-21T01:49:13.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377000bf60 con 0x7f3770004500 2022-07-21T01:49:13.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378a59c700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3770003750 0x7f3770003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378ad9d700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770004e70 0x7f377000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f3789d9b700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3770004500 0x7f3770004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378a59c700 1 -- 172.21.15.140:0/3194139616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3770004500 msgr2=0x7f3770004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:13.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378a59c700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3770004500 0x7f3770004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378a59c700 1 -- 172.21.15.140:0/3194139616 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770004e70 msgr2=0x7f377000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:13.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378a59c700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770004e70 0x7f377000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378a59c700 1 -- 172.21.15.140:0/3194139616 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3770003260 con 0x7f3770003750 2022-07-21T01:49:13.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378ad9d700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770004e70 0x7f377000b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:13.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378ad9d700 1 -- 172.21.15.140:0/3194139616 reap_dead start 2022-07-21T01:49:13.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.219+0000 7f378a59c700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3770003750 0x7f3770003b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f377c00c000 tx=0x7f377c006df0).ready entity=mon.1 client_cookie=c19d15bbd98a91c9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:13.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f377c00e830 con 0x7f3770003750 2022-07-21T01:49:13.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377c00e990 con 0x7f3770003750 2022-07-21T01:49:13.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f378959a700 1 -- 172.21.15.140:0/3194139616 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f377c00ecf0 con 0x7f3770003750 2022-07-21T01:49:13.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f3790902700 1 -- 172.21.15.140:0/3194139616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3770003750 msgr2=0x7f3770003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f3790902700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3770003750 0x7f3770003b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f377c00c000 tx=0x7f377c006df0).stop 2022-07-21T01:49:13.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f3790902700 1 -- 172.21.15.140:0/3194139616 shutdown_connections 2022-07-21T01:49:13.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f3790902700 1 --2- 172.21.15.140:0/3194139616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3770003750 0x7f3770003b70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f3790902700 1 -- 172.21.15.140:0/3194139616 >> 172.21.15.140:0/3194139616 conn(0x7f378c0635e0 msgr2=0x7f378c0b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:13.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f3790902700 1 -- 172.21.15.140:0/3194139616 shutdown_connections 2022-07-21T01:49:13.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f3790902700 1 -- 172.21.15.140:0/3194139616 wait complete. 2022-07-21T01:49:13.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f3790902700 1 Processor -- start 2022-07-21T01:49:13.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.223+0000 7f3790902700 1 -- start start 2022-07-21T01:49:13.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f3790902700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770003750 0x7f378c0b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f3790902700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f378c1449d0 0x7f378c139400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f3790902700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f378c155630 0x7f378c155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f3790902700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c1425f0 con 0x7f3770003750 2022-07-21T01:49:13.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f3790902700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0add80 con 0x7f378c155630 2022-07-21T01:49:13.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f3790902700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0ae060 con 0x7f378c1449d0 2022-07-21T01:49:13.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f378a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770003750 0x7f378c0b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f378a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770003750 0x7f378c0b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33762/0 (socket says 172.21.15.140:33762) 2022-07-21T01:49:13.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f378a59c700 1 -- 172.21.15.140:0/3574565669 learned_addr learned my addr 172.21.15.140:0/3574565669 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:13.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f3789d9b700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f378c1449d0 0x7f378c139400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f378ad9d700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f378c155630 0x7f378c155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f378a59c700 1 -- 172.21.15.140:0/3574565669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f378c1449d0 msgr2=0x7f378c139400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f378a59c700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f378c1449d0 0x7f378c139400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f378a59c700 1 -- 172.21.15.140:0/3574565669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f378c155630 msgr2=0x7f378c155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f378a59c700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f378c155630 0x7f378c155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f378a59c700 1 -- 172.21.15.140:0/3574565669 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c1434a0 con 0x7f3770003750 2022-07-21T01:49:13.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f378a59c700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770003750 0x7f378c0b4290 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f378c142730 tx=0x7f377c019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:13.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f377b7fe700 1 -- 172.21.15.140:0/3574565669 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f377c01ccf0 con 0x7f3770003750 2022-07-21T01:49:13.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f377b7fe700 1 -- 172.21.15.140:0/3574565669 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377c000e90 con 0x7f3770003750 2022-07-21T01:49:13.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f3790902700 1 -- 172.21.15.140:0/3574565669 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378c0adae0 con 0x7f3770003750 2022-07-21T01:49:13.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f377b7fe700 1 -- 172.21.15.140:0/3574565669 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f377c0245e0 con 0x7f3770003750 2022-07-21T01:49:13.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f3790902700 1 -- 172.21.15.140:0/3574565669 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c144e50 con 0x7f3770003750 2022-07-21T01:49:13.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f3790902700 1 -- 172.21.15.140:0/3574565669 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378c1422e0 con 0x7f3770003750 2022-07-21T01:49:13.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.227+0000 7f377b7fe700 1 -- 172.21.15.140:0/3574565669 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f377c02a020 con 0x7f3770003750 2022-07-21T01:49:13.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.231+0000 7f377b7fe700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3768053950 0x7f3768055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.231+0000 7f377b7fe700 1 -- 172.21.15.140:0/3574565669 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f377c06d440 con 0x7f3770003750 2022-07-21T01:49:13.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.231+0000 7f3789d9b700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3768053950 0x7f3768055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.231+0000 7f3789d9b700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3768053950 0x7f3768055e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f3780005000 tx=0x7f378c05cb50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:13.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.235+0000 7f377b7fe700 1 -- 172.21.15.140:0/3574565669 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f377c047d90 con 0x7f3770003750 2022-07-21T01:49:13.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.403+0000 7f3790902700 1 -- 172.21.15.140:0/3574565669 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f378c05d100 con 0x7f3770003750 2022-07-21T01:49:13.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.599+0000 7f377b7fe700 1 -- 172.21.15.140:0/3574565669 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v10) v1 ==== 118+0+0 (secure 0 0 0) 0x7f377c00be60 con 0x7f3770003750 2022-07-21T01:49:13.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.603+0000 7f3790902700 1 -- 172.21.15.140:0/3574565669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3768053950 msgr2=0x7f3768055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.603+0000 7f3790902700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3768053950 0x7f3768055e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f3780005000 tx=0x7f378c05cb50).stop 2022-07-21T01:49:13.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.603+0000 7f3790902700 1 -- 172.21.15.140:0/3574565669 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770003750 msgr2=0x7f378c0b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.603+0000 7f3790902700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770003750 0x7f378c0b4290 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f378c142730 tx=0x7f377c019000).stop 2022-07-21T01:49:13.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.607+0000 7f3790902700 1 -- 172.21.15.140:0/3574565669 shutdown_connections 2022-07-21T01:49:13.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.607+0000 7f3790902700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3768053950 0x7f3768055e10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.607+0000 7f3790902700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f378c155630 0x7f378c155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.607+0000 7f3790902700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f378c1449d0 0x7f378c139400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.607+0000 7f3790902700 1 --2- 172.21.15.140:0/3574565669 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3770003750 0x7f378c0b4290 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.607+0000 7f3790902700 1 -- 172.21.15.140:0/3574565669 >> 172.21.15.140:0/3574565669 conn(0x7f378c0635e0 msgr2=0x7f378c137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:13.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.607+0000 7f3790902700 1 -- 172.21.15.140:0/3574565669 shutdown_connections 2022-07-21T01:49:13.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.607+0000 7f3790902700 1 -- 172.21.15.140:0/3574565669 wait complete. 2022-07-21T01:49:13.624 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:49:13.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.731+0000 7f0e2bfff700 1 Processor -- start 2022-07-21T01:49:13.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.731+0000 7f0e2bfff700 1 -- start start 2022-07-21T01:49:13.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.731+0000 7f0e2bfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0e2c0b3ca0 0x7f0e2c0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.731+0000 7f0e2bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0e2c13cb70 0x7f0e2c143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.731+0000 7f0e2bfff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c142ba0 con 0x7f0e2c13c790 2022-07-21T01:49:13.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.731+0000 7f0e2bfff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c1429b0 con 0x7f0e2c13cb70 2022-07-21T01:49:13.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.731+0000 7f0e2bfff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c146330 con 0x7f0e2c0b3ca0 2022-07-21T01:49:13.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0e2c13cb70 0x7f0e2c143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0e2c13cb70 0x7f0e2c143000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33774/0 (socket says 172.21.15.140:33774) 2022-07-21T01:49:13.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2b7fe700 1 -- 172.21.15.140:0/3321609474 learned_addr learned my addr 172.21.15.140:0/3321609474 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:13.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2a7fc700 1 --2- 172.21.15.140:0/3321609474 >> v2:172.21.15.152:3300/0 conn(0x7f0e2c0b3ca0 0x7f0e2c0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2b7fe700 1 -- 172.21.15.140:0/3321609474 >> v2:172.21.15.152:3300/0 conn(0x7f0e2c0b3ca0 msgr2=0x7f0e2c0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:13.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2b7fe700 1 --2- 172.21.15.140:0/3321609474 >> v2:172.21.15.152:3300/0 conn(0x7f0e2c0b3ca0 0x7f0e2c0b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2b7fe700 1 -- 172.21.15.140:0/3321609474 >> v1:172.21.15.140:6789/0 conn(0x7f0e2c13c790 legacy=0x7f0e2c0b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2b7fe700 1 -- 172.21.15.140:0/3321609474 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0ba120 con 0x7f0e2c13cb70 2022-07-21T01:49:13.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2a7fc700 1 --2- 172.21.15.140:0/3321609474 >> v2:172.21.15.152:3300/0 conn(0x7f0e2c0b3ca0 0x7f0e2c0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:13.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2b7fe700 1 --2- 172.21.15.140:0/3321609474 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e2c13cb70 0x7f0e2c143000 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f0e1c002f70 tx=0x7f0e1c006eb0).ready entity=mon.0 client_cookie=6602905ecc9fc7b2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:13.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e29ffb700 1 -- 172.21.15.140:0/3321609474 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e1c00d610 con 0x7f0e2c13cb70 2022-07-21T01:49:13.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e29ffb700 1 -- 172.21.15.140:0/3321609474 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c000e90 con 0x7f0e2c13cb70 2022-07-21T01:49:13.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e29ffb700 1 -- 172.21.15.140:0/3321609474 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e1c00d9a0 con 0x7f0e2c13cb70 2022-07-21T01:49:13.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2bfff700 1 -- 172.21.15.140:0/3321609474 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e2c13cb70 msgr2=0x7f0e2c143000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2bfff700 1 --2- 172.21.15.140:0/3321609474 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e2c13cb70 0x7f0e2c143000 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f0e1c002f70 tx=0x7f0e1c006eb0).stop 2022-07-21T01:49:13.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2bfff700 1 -- 172.21.15.140:0/3321609474 shutdown_connections 2022-07-21T01:49:13.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2bfff700 1 --2- 172.21.15.140:0/3321609474 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e2c13cb70 0x7f0e2c143000 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2bfff700 1 --2- 172.21.15.140:0/3321609474 >> v2:172.21.15.152:3300/0 conn(0x7f0e2c0b3ca0 0x7f0e2c0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2bfff700 1 -- 172.21.15.140:0/3321609474 >> 172.21.15.140:0/3321609474 conn(0x7f0e2c0678b0 msgr2=0x7f0e2c0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:13.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2bfff700 1 -- 172.21.15.140:0/3321609474 shutdown_connections 2022-07-21T01:49:13.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.735+0000 7f0e2bfff700 1 -- 172.21.15.140:0/3321609474 wait complete. 2022-07-21T01:49:13.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2bfff700 1 Processor -- start 2022-07-21T01:49:13.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2bfff700 1 -- start start 2022-07-21T01:49:13.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e2c0b3ca0 0x7f0e2c13d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e2c13c790 0x7f0e2c140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e2c13cb70 0x7f0e2c0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2bfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c152cd0 con 0x7f0e2c13cb70 2022-07-21T01:49:13.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2bfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c141140 con 0x7f0e2c0b3ca0 2022-07-21T01:49:13.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c140e00 con 0x7f0e2c13c790 2022-07-21T01:49:13.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e2c13cb70 0x7f0e2c0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e2c13cb70 0x7f0e2c0b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33780/0 (socket says 172.21.15.140:33780) 2022-07-21T01:49:13.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2b7fe700 1 -- 172.21.15.140:0/1161850674 learned_addr learned my addr 172.21.15.140:0/1161850674 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:13.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2affd700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e2c0b3ca0 0x7f0e2c13d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2a7fc700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e2c13c790 0x7f0e2c140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2b7fe700 1 -- 172.21.15.140:0/1161850674 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e2c13c790 msgr2=0x7f0e2c140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2b7fe700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e2c13c790 0x7f0e2c140910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2b7fe700 1 -- 172.21.15.140:0/1161850674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e2c0b3ca0 msgr2=0x7f0e2c13d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2b7fe700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e2c0b3ca0 0x7f0e2c13d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2b7fe700 1 -- 172.21.15.140:0/1161850674 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c00b1e0 con 0x7f0e2c13cb70 2022-07-21T01:49:13.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2b7fe700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e2c13cb70 0x7f0e2c0b1280 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f0e1c002f70 tx=0x7f0e1c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:13.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e23fff700 1 -- 172.21.15.140:0/1161850674 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e1c00d290 con 0x7f0e2c13cb70 2022-07-21T01:49:13.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e23fff700 1 -- 172.21.15.140:0/1161850674 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c00d990 con 0x7f0e2c13cb70 2022-07-21T01:49:13.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2bfff700 1 -- 172.21.15.140:0/1161850674 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c152910 con 0x7f0e2c13cb70 2022-07-21T01:49:13.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e23fff700 1 -- 172.21.15.140:0/1161850674 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e1c011b80 con 0x7f0e2c13cb70 2022-07-21T01:49:13.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.739+0000 7f0e2bfff700 1 -- 172.21.15.140:0/1161850674 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c0b2f90 con 0x7f0e2c13cb70 2022-07-21T01:49:13.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.743+0000 7f0e2bfff700 1 -- 172.21.15.140:0/1161850674 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c152e40 con 0x7f0e2c13cb70 2022-07-21T01:49:13.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.743+0000 7f0e23fff700 1 -- 172.21.15.140:0/1161850674 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0e1c025030 con 0x7f0e2c13cb70 2022-07-21T01:49:13.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.743+0000 7f0e23fff700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e100539e0 0x7f0e10055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:13.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.743+0000 7f0e23fff700 1 -- 172.21.15.140:0/1161850674 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0e1c06d0c0 con 0x7f0e2c13cb70 2022-07-21T01:49:13.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.743+0000 7f0e2affd700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e100539e0 0x7f0e10055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:13.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.743+0000 7f0e2affd700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e100539e0 0x7f0e10055ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f0e2400d000 tx=0x7f0e24000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:13.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.751+0000 7f0e23fff700 1 -- 172.21.15.140:0/1161850674 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e2c152e40 con 0x7f0e2c13cb70 2022-07-21T01:49:13.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.907+0000 7f0e2bfff700 1 -- 172.21.15.140:0/1161850674 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e2c152e10 con 0x7f0e2c13cb70 2022-07-21T01:49:13.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.907+0000 7f0e23fff700 1 -- 172.21.15.140:0/1161850674 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f0e1c023070 con 0x7f0e2c13cb70 2022-07-21T01:49:13.916 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:13.916 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/838095894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":838095894},{"type":"v1","addr":"172.21.15.140:6835","nonce":838095894}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:49:12.585320+0000","modified":"2022-07-21T01:49:13.589194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1420650762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1420650762},{"type":"v1","addr":"172.21.15.152:6833","nonce":1420650762}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-07-21T01:49:13.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 -- 172.21.15.140:0/1161850674 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e100539e0 msgr2=0x7f0e10055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e100539e0 0x7f0e10055ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f0e2400d000 tx=0x7f0e24000bc0).stop 2022-07-21T01:49:13.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 -- 172.21.15.140:0/1161850674 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e2c13cb70 msgr2=0x7f0e2c0b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:13.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e2c13cb70 0x7f0e2c0b1280 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f0e1c002f70 tx=0x7f0e1c015000).stop 2022-07-21T01:49:13.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 -- 172.21.15.140:0/1161850674 shutdown_connections 2022-07-21T01:49:13.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e100539e0 0x7f0e10055ea0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e2c13cb70 0x7f0e2c0b1280 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e2c13c790 0x7f0e2c140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 --2- 172.21.15.140:0/1161850674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e2c0b3ca0 0x7f0e2c13d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:13.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 -- 172.21.15.140:0/1161850674 >> 172.21.15.140:0/1161850674 conn(0x7f0e2c0678b0 msgr2=0x7f0e2c142f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:13.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 -- 172.21.15.140:0/1161850674 shutdown_connections 2022-07-21T01:49:13.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:13.911+0000 7f0e2bfff700 1 -- 172.21.15.140:0/1161850674 wait complete. 2022-07-21T01:49:13.926 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 10 2022-07-21T01:49:13.928 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:49:14.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.027+0000 7fc20f59e700 1 Processor -- start 2022-07-21T01:49:14.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.027+0000 7fc20f59e700 1 -- start start 2022-07-21T01:49:14.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2100b13e0 0x7fc2100b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc2100af120 0x7fc210067260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2100b20f0 con 0x7fc2100b61d0 2022-07-21T01:49:14.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21013efd0 con 0x7fc2100af120 2022-07-21T01:49:14.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100b65b0 con 0x7fc2100b13e0 2022-07-21T01:49:14.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20e59c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc2100b61d0 0x7fc2100aea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47930/0 (socket says 172.21.15.140:47930) 2022-07-21T01:49:14.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20e59c700 1 -- 172.21.15.140:0/502399610 learned_addr learned my addr 172.21.15.140:0/502399610 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:14.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc205d9b700 1 --2- 172.21.15.140:0/502399610 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2100b13e0 0x7fc2100b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20ed9d700 1 --2- 172.21.15.140:0/502399610 >> v2:172.21.15.152:3300/0 conn(0x7fc2100af120 0x7fc210067260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc205d9b700 1 -- 172.21.15.140:0/502399610 >> v2:172.21.15.152:3300/0 conn(0x7fc2100af120 msgr2=0x7fc210067260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:14.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc205d9b700 1 --2- 172.21.15.140:0/502399610 >> v2:172.21.15.152:3300/0 conn(0x7fc2100af120 0x7fc210067260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc205d9b700 1 -- 172.21.15.140:0/502399610 >> v1:172.21.15.140:6789/0 conn(0x7fc2100b61d0 legacy=0x7fc2100aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc205d9b700 1 -- 172.21.15.140:0/502399610 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2100b1ad0 con 0x7fc2100b13e0 2022-07-21T01:49:14.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20dd9b700 1 -- 172.21.15.140:0/502399610 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676810373 0 0) 0x7fc2100b20f0 con 0x7fc2100b61d0 2022-07-21T01:49:14.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc205d9b700 1 --2- 172.21.15.140:0/502399610 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2100b13e0 0x7fc2100b0460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fc1f8002f70 tx=0x7fc1f8006df0).ready entity=mon.2 client_cookie=eb21674f8f894ecd server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20dd9b700 1 -- 172.21.15.140:0/502399610 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1f800d570 con 0x7fc2100b13e0 2022-07-21T01:49:14.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20dd9b700 1 -- 172.21.15.140:0/502399610 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f8000e90 con 0x7fc2100b13e0 2022-07-21T01:49:14.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20dd9b700 1 -- 172.21.15.140:0/502399610 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1f800d8c0 con 0x7fc2100b13e0 2022-07-21T01:49:14.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 -- 172.21.15.140:0/502399610 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2100b13e0 msgr2=0x7fc2100b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 --2- 172.21.15.140:0/502399610 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2100b13e0 0x7fc2100b0460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fc1f8002f70 tx=0x7fc1f8006df0).stop 2022-07-21T01:49:14.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 -- 172.21.15.140:0/502399610 shutdown_connections 2022-07-21T01:49:14.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 --2- 172.21.15.140:0/502399610 >> v2:172.21.15.152:3300/0 conn(0x7fc2100af120 0x7fc210067260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 --2- 172.21.15.140:0/502399610 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2100b13e0 0x7fc2100b0460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 -- 172.21.15.140:0/502399610 >> 172.21.15.140:0/502399610 conn(0x7fc210063cf0 msgr2=0x7fc2100b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:14.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 -- 172.21.15.140:0/502399610 shutdown_connections 2022-07-21T01:49:14.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.031+0000 7fc20f59e700 1 -- 172.21.15.140:0/502399610 wait complete. 2022-07-21T01:49:14.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.035+0000 7fc20f59e700 1 Processor -- start 2022-07-21T01:49:14.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.035+0000 7fc20f59e700 1 -- start start 2022-07-21T01:49:14.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.035+0000 7fc20f59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2100af120 0x7fc2101397c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.035+0000 7fc20f59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2100b13e0 0x7fc2100b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.035+0000 7fc20f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2100b61d0 0x7fc2100b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20f59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc210066480 con 0x7fc2100af120 2022-07-21T01:49:14.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20f59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100644e0 con 0x7fc2100b13e0 2022-07-21T01:49:14.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100651d0 con 0x7fc2100b61d0 2022-07-21T01:49:14.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2100af120 0x7fc2101397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2100af120 0x7fc2101397c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33790/0 (socket says 172.21.15.140:33790) 2022-07-21T01:49:14.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20e59c700 1 -- 172.21.15.140:0/2972678159 learned_addr learned my addr 172.21.15.140:0/2972678159 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:14.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20ed9d700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2100b61d0 0x7fc2100b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc205d9b700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2100b13e0 0x7fc2100b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20e59c700 1 -- 172.21.15.140:0/2972678159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2100b61d0 msgr2=0x7fc2100b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20e59c700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2100b61d0 0x7fc2100b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20e59c700 1 -- 172.21.15.140:0/2972678159 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2100b13e0 msgr2=0x7fc2100b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20e59c700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2100b13e0 0x7fc2100b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20e59c700 1 -- 172.21.15.140:0/2972678159 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f800b180 con 0x7fc2100af120 2022-07-21T01:49:14.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20e59c700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2100af120 0x7fc2101397c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fc200000ed0 tx=0x7fc200007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc207fff700 1 -- 172.21.15.140:0/2972678159 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2000070d0 con 0x7fc2100af120 2022-07-21T01:49:14.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc207fff700 1 -- 172.21.15.140:0/2972678159 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc200007230 con 0x7fc2100af120 2022-07-21T01:49:14.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20f59e700 1 -- 172.21.15.140:0/2972678159 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc210065470 con 0x7fc2100af120 2022-07-21T01:49:14.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc207fff700 1 -- 172.21.15.140:0/2972678159 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc200005610 con 0x7fc2100af120 2022-07-21T01:49:14.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20f59e700 1 -- 172.21.15.140:0/2972678159 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2100668b0 con 0x7fc2100af120 2022-07-21T01:49:14.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.039+0000 7fc20f59e700 1 -- 172.21.15.140:0/2972678159 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2100b1d70 con 0x7fc2100af120 2022-07-21T01:49:14.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.043+0000 7fc207fff700 1 -- 172.21.15.140:0/2972678159 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc20001a060 con 0x7fc2100af120 2022-07-21T01:49:14.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.043+0000 7fc207fff700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc1f4053aa0 0x7fc1f4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.043+0000 7fc207fff700 1 -- 172.21.15.140:0/2972678159 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc20006b220 con 0x7fc2100af120 2022-07-21T01:49:14.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.043+0000 7fc205d9b700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc1f4053aa0 0x7fc1f4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.043+0000 7fc205d9b700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc1f4053aa0 0x7fc1f4055f60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fc1f801a000 tx=0x7fc1f8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.047+0000 7fc207fff700 1 -- 172.21.15.140:0/2972678159 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc200042140 con 0x7fc2100af120 2022-07-21T01:49:14.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.171+0000 7fc20f59e700 1 -- 172.21.15.140:0/2972678159 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2100574e0 con 0x7fc2100af120 2022-07-21T01:49:14.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc207fff700 1 -- 172.21.15.140:0/2972678159 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16134 (secure 0 0 0) 0x7fc200018020 con 0x7fc2100af120 2022-07-21T01:49:14.178 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:14.179 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":32,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:49:13.588079+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:49:10.708637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-07-21T01:49:11.590708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:49:14.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 -- 172.21.15.140:0/2972678159 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc1f4053aa0 msgr2=0x7fc1f4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc1f4053aa0 0x7fc1f4055f60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fc1f801a000 tx=0x7fc1f8014000).stop 2022-07-21T01:49:14.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 -- 172.21.15.140:0/2972678159 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2100af120 msgr2=0x7fc2101397c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2100af120 0x7fc2101397c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fc200000ed0 tx=0x7fc200007ac0).stop 2022-07-21T01:49:14.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 -- 172.21.15.140:0/2972678159 shutdown_connections 2022-07-21T01:49:14.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc1f4053aa0 0x7fc1f4055f60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2100b61d0 0x7fc2100b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2100b13e0 0x7fc2100b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 --2- 172.21.15.140:0/2972678159 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2100af120 0x7fc2101397c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 -- 172.21.15.140:0/2972678159 >> 172.21.15.140:0/2972678159 conn(0x7fc210063cf0 msgr2=0x7fc2100b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:14.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 -- 172.21.15.140:0/2972678159 shutdown_connections 2022-07-21T01:49:14.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.175+0000 7fc20f59e700 1 -- 172.21.15.140:0/2972678159 wait complete. 2022-07-21T01:49:14.200 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:49:14.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629bfff700 1 Processor -- start 2022-07-21T01:49:14.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629bfff700 1 -- start start 2022-07-21T01:49:14.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629bfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f629c14f9d0 0x7f629c151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629bfff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c0b0b80 con 0x7f629c0b5970 2022-07-21T01:49:14.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629bfff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c05d4d0 con 0x7f629c0af8c0 2022-07-21T01:49:14.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629bfff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629c0b5d50 con 0x7f629c14f9d0 2022-07-21T01:49:14.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629affd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f629c0b5970 0x7f629c0ae100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47944/0 (socket says 172.21.15.140:47944) 2022-07-21T01:49:14.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629affd700 1 -- 172.21.15.140:0/3702514939 learned_addr learned my addr 172.21.15.140:0/3702514939 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:14.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629b7fe700 1 --2- 172.21.15.140:0/3702514939 >> v2:172.21.15.152:3300/0 conn(0x7f629c14f9d0 0x7f629c151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88293329 0 0) 0x7f629c0b0b80 con 0x7f629c0b5970 2022-07-21T01:49:14.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6280003260 con 0x7f629c0b5970 2022-07-21T01:49:14.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2901037623 0 0) 0x7f6280003260 con 0x7f629c0b5970 2022-07-21T01:49:14.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 >> v2:172.21.15.152:3300/0 conn(0x7f629c14f9d0 msgr2=0x7f629c151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:14.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 --2- 172.21.15.140:0/3702514939 >> v2:172.21.15.152:3300/0 conn(0x7f629c14f9d0 0x7f629c151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 >> v1:172.21.15.152:6789/0 conn(0x7f629c0af8c0 legacy=0x7f629c0ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c0b0db0 con 0x7f629c0b5970 2022-07-21T01:49:14.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f6290001660 con 0x7f629c0b5970 2022-07-21T01:49:14.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 >> v1:172.21.15.140:6789/0 conn(0x7f629c0b5970 legacy=0x7f629c0ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6280003750 0x7f6280003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6280004500 0x7f6280004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004e70 0x7f628000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0b5d50 con 0x7f6280004e70 2022-07-21T01:49:14.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628000bd20 con 0x7f6280004500 2022-07-21T01:49:14.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628000bee0 con 0x7f6280003750 2022-07-21T01:49:14.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629a7fc700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6280004500 0x7f6280004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629b7fe700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004e70 0x7f628000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629affd700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6280003750 0x7f6280003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629a7fc700 1 -- 172.21.15.140:0/3702514939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6280003750 msgr2=0x7f6280003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:14.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629a7fc700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6280003750 0x7f6280003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629a7fc700 1 -- 172.21.15.140:0/3702514939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004e70 msgr2=0x7f628000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:14.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629a7fc700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004e70 0x7f628000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629a7fc700 1 -- 172.21.15.140:0/3702514939 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6280003260 con 0x7f6280004500 2022-07-21T01:49:14.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629b7fe700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004e70 0x7f628000b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:14.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.315+0000 7f629b7fe700 1 -- 172.21.15.140:0/3702514939 reap_dead start 2022-07-21T01:49:14.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629a7fc700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6280004500 0x7f6280004980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f629c0b5f80 tx=0x7f628c007a00).ready entity=mon.1 client_cookie=860ad98e4c78e215 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f628c007d10 con 0x7f6280004500 2022-07-21T01:49:14.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628c007030 con 0x7f6280004500 2022-07-21T01:49:14.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f6299ffb700 1 -- 172.21.15.140:0/3702514939 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f628c007390 con 0x7f6280004500 2022-07-21T01:49:14.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 -- 172.21.15.140:0/3702514939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6280004500 msgr2=0x7f6280004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6280004500 0x7f6280004980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f629c0b5f80 tx=0x7f628c007a00).stop 2022-07-21T01:49:14.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 -- 172.21.15.140:0/3702514939 shutdown_connections 2022-07-21T01:49:14.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 --2- 172.21.15.140:0/3702514939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6280004500 0x7f6280004980 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 -- 172.21.15.140:0/3702514939 >> 172.21.15.140:0/3702514939 conn(0x7f629c063490 msgr2=0x7f629c0b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:14.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 -- 172.21.15.140:0/3702514939 shutdown_connections 2022-07-21T01:49:14.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 -- 172.21.15.140:0/3702514939 wait complete. 2022-07-21T01:49:14.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 Processor -- start 2022-07-21T01:49:14.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 -- start start 2022-07-21T01:49:14.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004500 0x7f629c111b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f629c063b80 0x7f629c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f629c1566d0 0x7f629c159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c10e790 con 0x7f6280004500 2022-07-21T01:49:14.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c156f00 con 0x7f629c1566d0 2022-07-21T01:49:14.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c10e450 con 0x7f629c063b80 2022-07-21T01:49:14.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004500 0x7f629c111b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004500 0x7f629c111b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33810/0 (socket says 172.21.15.140:33810) 2022-07-21T01:49:14.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629affd700 1 -- 172.21.15.140:0/2321298751 learned_addr learned my addr 172.21.15.140:0/2321298751 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:14.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629a7fc700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f629c063b80 0x7f629c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629b7fe700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f629c1566d0 0x7f629c159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629affd700 1 -- 172.21.15.140:0/2321298751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f629c063b80 msgr2=0x7f629c1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629affd700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f629c063b80 0x7f629c1561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629affd700 1 -- 172.21.15.140:0/2321298751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f629c1566d0 msgr2=0x7f629c159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629affd700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f629c1566d0 0x7f629c159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629affd700 1 -- 172.21.15.140:0/2321298751 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c13fac0 con 0x7f6280004500 2022-07-21T01:49:14.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629b7fe700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f629c1566d0 0x7f629c159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:49:14.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629affd700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004500 0x7f629c111b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f629000d000 tx=0x7f629c0b0f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f628bfff700 1 -- 172.21.15.140:0/2321298751 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6290011780 con 0x7f6280004500 2022-07-21T01:49:14.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f628bfff700 1 -- 172.21.15.140:0/2321298751 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62900118e0 con 0x7f6280004500 2022-07-21T01:49:14.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 -- 172.21.15.140:0/2321298751 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c156b80 con 0x7f6280004500 2022-07-21T01:49:14.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f628bfff700 1 -- 172.21.15.140:0/2321298751 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6290011c40 con 0x7f6280004500 2022-07-21T01:49:14.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.319+0000 7f629bfff700 1 -- 172.21.15.140:0/2321298751 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c159c80 con 0x7f6280004500 2022-07-21T01:49:14.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.323+0000 7f629bfff700 1 -- 172.21.15.140:0/2321298751 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629c0acae0 con 0x7f6280004500 2022-07-21T01:49:14.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.323+0000 7f628bfff700 1 -- 172.21.15.140:0/2321298751 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f629001b030 con 0x7f6280004500 2022-07-21T01:49:14.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.323+0000 7f628bfff700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6278053aa0 0x7f6278055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.323+0000 7f628bfff700 1 -- 172.21.15.140:0/2321298751 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f629006bb00 con 0x7f6280004500 2022-07-21T01:49:14.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.323+0000 7f629a7fc700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6278053aa0 0x7f6278055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.323+0000 7f629a7fc700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6278053aa0 0x7f6278055f60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f628c019000 tx=0x7f628c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.327+0000 7f628bfff700 1 -- 172.21.15.140:0/2321298751 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6290042980 con 0x7f6280004500 2022-07-21T01:49:14.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.491+0000 7f629bfff700 1 -- 172.21.15.140:0/2321298751 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f629c0574e0 con 0x7f6280004500 2022-07-21T01:49:14.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.499+0000 7f628bfff700 1 -- 172.21.15.140:0/2321298751 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6290019030 con 0x7f6280004500 2022-07-21T01:49:14.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 -- 172.21.15.140:0/2321298751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6278053aa0 msgr2=0x7f6278055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6278053aa0 0x7f6278055f60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f628c019000 tx=0x7f628c015000).stop 2022-07-21T01:49:14.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 -- 172.21.15.140:0/2321298751 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004500 msgr2=0x7f629c111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004500 0x7f629c111b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f629000d000 tx=0x7f629c0b0f40).stop 2022-07-21T01:49:14.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 -- 172.21.15.140:0/2321298751 shutdown_connections 2022-07-21T01:49:14.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6278053aa0 0x7f6278055f60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f629c1566d0 0x7f629c159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f629c063b80 0x7f629c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 --2- 172.21.15.140:0/2321298751 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6280004500 0x7f629c111b90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 -- 172.21.15.140:0/2321298751 >> 172.21.15.140:0/2321298751 conn(0x7f629c063490 msgr2=0x7f629c05bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:14.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 -- 172.21.15.140:0/2321298751 shutdown_connections 2022-07-21T01:49:14.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.503+0000 7f629bfff700 1 -- 172.21.15.140:0/2321298751 wait complete. 2022-07-21T01:49:14.510 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:49:14.522 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:49:14.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.631+0000 7fb633fff700 1 Processor -- start 2022-07-21T01:49:14.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.631+0000 7fb633fff700 1 -- start start 2022-07-21T01:49:14.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.631+0000 7fb633fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb6340b5630 0x7fb6340b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.631+0000 7fb633fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb63413c7c0 0x7fb63405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.631+0000 7fb633fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6340b0a10 con 0x7fb63413cc50 2022-07-21T01:49:14.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.631+0000 7fb633fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6340b0cb0 con 0x7fb6340b5630 2022-07-21T01:49:14.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.631+0000 7fb633fff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6340b0f50 con 0x7fb63413c7c0 2022-07-21T01:49:14.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.631+0000 7fb632ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb6340b5630 0x7fb6340b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.631+0000 7fb632ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb6340b5630 0x7fb6340b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33818/0 (socket says 172.21.15.140:33818) 2022-07-21T01:49:14.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.631+0000 7fb632ffd700 1 -- 172.21.15.140:0/3118943007 learned_addr learned my addr 172.21.15.140:0/3118943007 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:14.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.631+0000 7fb62a7fc700 1 --2- 172.21.15.140:0/3118943007 >> v2:172.21.15.152:3300/0 conn(0x7fb63413c7c0 0x7fb63405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb632ffd700 1 -- 172.21.15.140:0/3118943007 >> v2:172.21.15.152:3300/0 conn(0x7fb63413c7c0 msgr2=0x7fb63405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:14.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb632ffd700 1 --2- 172.21.15.140:0/3118943007 >> v2:172.21.15.152:3300/0 conn(0x7fb63413c7c0 0x7fb63405bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb632ffd700 1 -- 172.21.15.140:0/3118943007 >> v1:172.21.15.140:6789/0 conn(0x7fb63413cc50 legacy=0x7fb63405c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb632ffd700 1 -- 172.21.15.140:0/3118943007 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6340afa10 con 0x7fb6340b5630 2022-07-21T01:49:14.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb632ffd700 1 --2- 172.21.15.140:0/3118943007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6340b5630 0x7fb6340b5a10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fb624002f70 tx=0x7fb624006eb0).ready entity=mon.0 client_cookie=c1aca414d3e14293 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb6327fc700 1 -- 172.21.15.140:0/3118943007 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62400d610 con 0x7fb6340b5630 2022-07-21T01:49:14.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb6327fc700 1 -- 172.21.15.140:0/3118943007 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb624000e90 con 0x7fb6340b5630 2022-07-21T01:49:14.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb6327fc700 1 -- 172.21.15.140:0/3118943007 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62400d9a0 con 0x7fb6340b5630 2022-07-21T01:49:14.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb633fff700 1 -- 172.21.15.140:0/3118943007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6340b5630 msgr2=0x7fb6340b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb633fff700 1 --2- 172.21.15.140:0/3118943007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6340b5630 0x7fb6340b5a10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fb624002f70 tx=0x7fb624006eb0).stop 2022-07-21T01:49:14.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb633fff700 1 -- 172.21.15.140:0/3118943007 shutdown_connections 2022-07-21T01:49:14.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb633fff700 1 --2- 172.21.15.140:0/3118943007 >> v2:172.21.15.152:3300/0 conn(0x7fb63413c7c0 0x7fb63405bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb633fff700 1 --2- 172.21.15.140:0/3118943007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6340b5630 0x7fb6340b5a10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb633fff700 1 -- 172.21.15.140:0/3118943007 >> 172.21.15.140:0/3118943007 conn(0x7fb634063490 msgr2=0x7fb6340b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:14.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb633fff700 1 -- 172.21.15.140:0/3118943007 shutdown_connections 2022-07-21T01:49:14.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb633fff700 1 -- 172.21.15.140:0/3118943007 wait complete. 2022-07-21T01:49:14.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb633fff700 1 Processor -- start 2022-07-21T01:49:14.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.635+0000 7fb633fff700 1 -- start start 2022-07-21T01:49:14.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb633fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6340b5630 0x7fb634140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb633fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63413c7c0 0x7fb634144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb633fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb63413cc50 0x7fb63413d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb633fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb634145e50 con 0x7fb63413cc50 2022-07-21T01:49:14.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb633fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63413e9d0 con 0x7fb63413c7c0 2022-07-21T01:49:14.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb633fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63413ec70 con 0x7fb6340b5630 2022-07-21T01:49:14.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb6337fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb63413cc50 0x7fb63413d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb6337fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb63413cc50 0x7fb63413d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33828/0 (socket says 172.21.15.140:33828) 2022-07-21T01:49:14.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb6337fe700 1 -- 172.21.15.140:0/2579063724 learned_addr learned my addr 172.21.15.140:0/2579063724 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:14.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb62a7fc700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63413c7c0 0x7fb634144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb632ffd700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6340b5630 0x7fb634140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb6337fe700 1 -- 172.21.15.140:0/2579063724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6340b5630 msgr2=0x7fb634140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb6337fe700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6340b5630 0x7fb634140070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb6337fe700 1 -- 172.21.15.140:0/2579063724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63413c7c0 msgr2=0x7fb634144fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb6337fe700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63413c7c0 0x7fb634144fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb6337fe700 1 -- 172.21.15.140:0/2579063724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62400b1e0 con 0x7fb63413cc50 2022-07-21T01:49:14.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb6337fe700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb63413cc50 0x7fb63413d6d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fb6340b14d0 tx=0x7fb62c007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb630ff9700 1 -- 172.21.15.140:0/2579063724 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62c00cea0 con 0x7fb63413cc50 2022-07-21T01:49:14.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb630ff9700 1 -- 172.21.15.140:0/2579063724 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c006aa0 con 0x7fb63413cc50 2022-07-21T01:49:14.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb633fff700 1 -- 172.21.15.140:0/2579063724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63413ef10 con 0x7fb63413cc50 2022-07-21T01:49:14.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb630ff9700 1 -- 172.21.15.140:0/2579063724 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62c006e00 con 0x7fb63413cc50 2022-07-21T01:49:14.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb633fff700 1 -- 172.21.15.140:0/2579063724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63413d0b0 con 0x7fb63413cc50 2022-07-21T01:49:14.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.639+0000 7fb633fff700 1 -- 172.21.15.140:0/2579063724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb634063fe0 con 0x7fb63413cc50 2022-07-21T01:49:14.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.643+0000 7fb630ff9700 1 -- 172.21.15.140:0/2579063724 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb62c010030 con 0x7fb63413cc50 2022-07-21T01:49:14.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.643+0000 7fb630ff9700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb618053aa0 0x7fb618055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.643+0000 7fb630ff9700 1 -- 172.21.15.140:0/2579063724 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb62c06bc00 con 0x7fb63413cc50 2022-07-21T01:49:14.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.643+0000 7fb632ffd700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb618053aa0 0x7fb618055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.643+0000 7fb632ffd700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb618053aa0 0x7fb618055f60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fb62401c000 tx=0x7fb624015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.647+0000 7fb630ff9700 1 -- 172.21.15.140:0/2579063724 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb634063fe0 con 0x7fb63413cc50 2022-07-21T01:49:14.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.811+0000 7fb633fff700 1 -- 172.21.15.140:0/2579063724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fb634063fe0 con 0x7fb63413cc50 2022-07-21T01:49:14.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.819+0000 7fb630ff9700 1 -- 172.21.15.140:0/2579063724 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb634063fe0 con 0x7fb63413cc50 2022-07-21T01:49:14.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 -- 172.21.15.140:0/2579063724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb618053aa0 msgr2=0x7fb618055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb618053aa0 0x7fb618055f60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fb62401c000 tx=0x7fb624015000).stop 2022-07-21T01:49:14.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 -- 172.21.15.140:0/2579063724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb63413cc50 msgr2=0x7fb63413d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb63413cc50 0x7fb63413d6d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fb6340b14d0 tx=0x7fb62c007f70).stop 2022-07-21T01:49:14.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 -- 172.21.15.140:0/2579063724 shutdown_connections 2022-07-21T01:49:14.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb618053aa0 0x7fb618055f60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb63413cc50 0x7fb63413d6d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63413c7c0 0x7fb634144fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 --2- 172.21.15.140:0/2579063724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6340b5630 0x7fb634140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 -- 172.21.15.140:0/2579063724 >> 172.21.15.140:0/2579063724 conn(0x7fb634063490 msgr2=0x7fb63413b240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:14.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 -- 172.21.15.140:0/2579063724 shutdown_connections 2022-07-21T01:49:14.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.823+0000 7fb633fff700 1 -- 172.21.15.140:0/2579063724 wait complete. 2022-07-21T01:49:14.834 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:49:14.839 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:49:14.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1fb12700 1 Processor -- start 2022-07-21T01:49:14.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1fb12700 1 -- start start 2022-07-21T01:49:14.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1fb12700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faa1805cf40 0x7faa18146a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1fb12700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7faa180b5830 0x7faa18142c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1fb12700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa180b0130 con 0x7faa18143120 2022-07-21T01:49:14.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1fb12700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa181462d0 con 0x7faa1805cf40 2022-07-21T01:49:14.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1fb12700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa18146570 con 0x7faa180b5830 2022-07-21T01:49:14.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1d8ae700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faa1805cf40 0x7faa18146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1e0af700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7faa18143120 0x7faa180af2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:47982/0 (socket says 172.21.15.140:47982) 2022-07-21T01:49:14.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1e0af700 1 -- 172.21.15.140:0/2512610728 learned_addr learned my addr 172.21.15.140:0/2512610728 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:14.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1d8ae700 1 --2- 172.21.15.140:0/2512610728 >> v2:172.21.15.140:3300/0 conn(0x7faa1805cf40 0x7faa18146a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33832/0 (socket says 172.21.15.140:33832) 2022-07-21T01:49:14.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1d0ad700 1 --2- 172.21.15.140:0/2512610728 >> v2:172.21.15.152:3300/0 conn(0x7faa180b5830 0x7faa18142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1d8ae700 1 -- 172.21.15.140:0/2512610728 >> v2:172.21.15.152:3300/0 conn(0x7faa180b5830 msgr2=0x7faa18142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:14.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1d8ae700 1 --2- 172.21.15.140:0/2512610728 >> v2:172.21.15.152:3300/0 conn(0x7faa180b5830 0x7faa18142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1d8ae700 1 -- 172.21.15.140:0/2512610728 >> v1:172.21.15.140:6789/0 conn(0x7faa18143120 legacy=0x7faa180af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1c8ac700 1 -- 172.21.15.140:0/2512610728 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174132927 0 0) 0x7faa180b0130 con 0x7faa18143120 2022-07-21T01:49:14.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.955+0000 7faa1d8ae700 1 -- 172.21.15.140:0/2512610728 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa18061280 con 0x7faa1805cf40 2022-07-21T01:49:14.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1d8ae700 1 --2- 172.21.15.140:0/2512610728 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa1805cf40 0x7faa18146a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7faa0c002f70 tx=0x7faa0c006df0).ready entity=mon.0 client_cookie=8275fff218172351 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1c8ac700 1 -- 172.21.15.140:0/2512610728 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa0c00d570 con 0x7faa1805cf40 2022-07-21T01:49:14.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1c8ac700 1 -- 172.21.15.140:0/2512610728 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c000e90 con 0x7faa1805cf40 2022-07-21T01:49:14.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1c8ac700 1 -- 172.21.15.140:0/2512610728 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa0c00d8c0 con 0x7faa1805cf40 2022-07-21T01:49:14.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 -- 172.21.15.140:0/2512610728 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa1805cf40 msgr2=0x7faa18146a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 --2- 172.21.15.140:0/2512610728 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa1805cf40 0x7faa18146a10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7faa0c002f70 tx=0x7faa0c006df0).stop 2022-07-21T01:49:14.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 -- 172.21.15.140:0/2512610728 shutdown_connections 2022-07-21T01:49:14.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 --2- 172.21.15.140:0/2512610728 >> v2:172.21.15.152:3300/0 conn(0x7faa180b5830 0x7faa18142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 --2- 172.21.15.140:0/2512610728 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa1805cf40 0x7faa18146a10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 -- 172.21.15.140:0/2512610728 >> 172.21.15.140:0/2512610728 conn(0x7faa180b69e0 msgr2=0x7faa18066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:14.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 -- 172.21.15.140:0/2512610728 shutdown_connections 2022-07-21T01:49:14.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 -- 172.21.15.140:0/2512610728 wait complete. 2022-07-21T01:49:14.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 Processor -- start 2022-07-21T01:49:14.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 -- start start 2022-07-21T01:49:14.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa1805cf40 0x7faa1813c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faa180b5830 0x7faa1809d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa18143120 0x7faa180b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa180b23c0 con 0x7faa1805cf40 2022-07-21T01:49:14.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa180b1360 con 0x7faa18143120 2022-07-21T01:49:14.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1fb12700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1809dc90 con 0x7faa180b5830 2022-07-21T01:49:14.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1d8ae700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa1805cf40 0x7faa1813c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1d8ae700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa1805cf40 0x7faa1813c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33840/0 (socket says 172.21.15.140:33840) 2022-07-21T01:49:14.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1d8ae700 1 -- 172.21.15.140:0/3262819407 learned_addr learned my addr 172.21.15.140:0/3262819407 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:14.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1d0ad700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faa180b5830 0x7faa1809d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1e0af700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa18143120 0x7faa180b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1d8ae700 1 -- 172.21.15.140:0/3262819407 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faa180b5830 msgr2=0x7faa1809d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1d8ae700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faa180b5830 0x7faa1809d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1d8ae700 1 -- 172.21.15.140:0/3262819407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa18143120 msgr2=0x7faa180b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:14.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1d8ae700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa18143120 0x7faa180b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:14.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.959+0000 7faa1d8ae700 1 -- 172.21.15.140:0/3262819407 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c00b180 con 0x7faa1805cf40 2022-07-21T01:49:14.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.963+0000 7faa1d8ae700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa1805cf40 0x7faa1813c7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7faa0c002f70 tx=0x7faa0c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.963+0000 7faa0affd700 1 -- 172.21.15.140:0/3262819407 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa0c016070 con 0x7faa1805cf40 2022-07-21T01:49:14.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.963+0000 7faa0affd700 1 -- 172.21.15.140:0/3262819407 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c0139c0 con 0x7faa1805cf40 2022-07-21T01:49:14.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.963+0000 7faa1fb12700 1 -- 172.21.15.140:0/3262819407 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1809df30 con 0x7faa1805cf40 2022-07-21T01:49:14.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.963+0000 7faa0affd700 1 -- 172.21.15.140:0/3262819407 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa0c013d20 con 0x7faa1805cf40 2022-07-21T01:49:14.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.963+0000 7faa1fb12700 1 -- 172.21.15.140:0/3262819407 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa180b1060 con 0x7faa1805cf40 2022-07-21T01:49:14.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.963+0000 7faa1fb12700 1 -- 172.21.15.140:0/3262819407 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1809e0c0 con 0x7faa1805cf40 2022-07-21T01:49:14.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.963+0000 7faa0affd700 1 -- 172.21.15.140:0/3262819407 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faa0c025030 con 0x7faa1805cf40 2022-07-21T01:49:14.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.967+0000 7faa0affd700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa00053980 0x7faa00055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:14.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.967+0000 7faa0affd700 1 -- 172.21.15.140:0/3262819407 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faa0c06cf80 con 0x7faa1805cf40 2022-07-21T01:49:14.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.967+0000 7faa1d0ad700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa00053980 0x7faa00055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:14.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.967+0000 7faa1d0ad700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa00053980 0x7faa00055e40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7faa1400b000 tx=0x7faa14006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:14.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:14.971+0000 7faa0affd700 1 -- 172.21.15.140:0/3262819407 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faa1809e0c0 con 0x7faa1805cf40 2022-07-21T01:49:15.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.131+0000 7faa1fb12700 1 -- 172.21.15.140:0/3262819407 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7faa1809e0c0 con 0x7faa1805cf40 2022-07-21T01:49:15.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa0affd700 1 -- 172.21.15.140:0/3262819407 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7faa1809e0c0 con 0x7faa1805cf40 2022-07-21T01:49:15.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 -- 172.21.15.140:0/3262819407 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa00053980 msgr2=0x7faa00055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa00053980 0x7faa00055e40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7faa1400b000 tx=0x7faa14006df0).stop 2022-07-21T01:49:15.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 -- 172.21.15.140:0/3262819407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa1805cf40 msgr2=0x7faa1813c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa1805cf40 0x7faa1813c7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7faa0c002f70 tx=0x7faa0c014000).stop 2022-07-21T01:49:15.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 -- 172.21.15.140:0/3262819407 shutdown_connections 2022-07-21T01:49:15.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faa00053980 0x7faa00055e40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faa18143120 0x7faa180b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faa180b5830 0x7faa1809d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 --2- 172.21.15.140:0/3262819407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faa1805cf40 0x7faa1813c7a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 -- 172.21.15.140:0/3262819407 >> 172.21.15.140:0/3262819407 conn(0x7faa180b69e0 msgr2=0x7faa180af280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:15.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 -- 172.21.15.140:0/3262819407 shutdown_connections 2022-07-21T01:49:15.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.139+0000 7faa1fb12700 1 -- 172.21.15.140:0/3262819407 wait complete. 2022-07-21T01:49:15.155 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:49:15.158 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:49:15.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdbfff700 1 Processor -- start 2022-07-21T01:49:15.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdbfff700 1 -- start start 2022-07-21T01:49:15.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdbfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5fdc0b5650 0x7f5fdc0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdbfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5fdc0adff0 0x7f5fdc147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5fdc0af8e0 0x7f5fdc05bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdbfff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc147010 con 0x7f5fdc0b5650 2022-07-21T01:49:15.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdbfff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc1472b0 con 0x7f5fdc0adff0 2022-07-21T01:49:15.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdbfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc146d40 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdaffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5fdc0b5650 0x7f5fdc0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdaffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5fdc0b5650 0x7f5fdc0b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33846/0 (socket says 172.21.15.140:33846) 2022-07-21T01:49:15.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdaffd700 1 -- 172.21.15.140:0/2522140976 learned_addr learned my addr 172.21.15.140:0/2522140976 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:15.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdb7fe700 1 --2- 172.21.15.140:0/2522140976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5fdc0af8e0 0x7f5fdc05bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fda7fc700 1 --2- 172.21.15.140:0/2522140976 >> v2:172.21.15.152:3300/0 conn(0x7f5fdc0adff0 0x7f5fdc147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdaffd700 1 -- 172.21.15.140:0/2522140976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5fdc0af8e0 msgr2=0x7f5fdc05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:15.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdaffd700 1 --2- 172.21.15.140:0/2522140976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5fdc0af8e0 0x7f5fdc05bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdaffd700 1 -- 172.21.15.140:0/2522140976 >> v2:172.21.15.152:3300/0 conn(0x7f5fdc0adff0 msgr2=0x7f5fdc147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:15.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdaffd700 1 --2- 172.21.15.140:0/2522140976 >> v2:172.21.15.152:3300/0 conn(0x7f5fdc0adff0 0x7f5fdc147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.255+0000 7f5fdaffd700 1 -- 172.21.15.140:0/2522140976 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc147c40 con 0x7f5fdc0b5650 2022-07-21T01:49:15.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdaffd700 1 --2- 172.21.15.140:0/2522140976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5fdc0b5650 0x7f5fdc0b5a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f5fd40032f0 tx=0x7f5fd4009290).ready entity=mon.0 client_cookie=8225efa5889b4762 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:15.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fd9ffb700 1 -- 172.21.15.140:0/2522140976 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fd400b870 con 0x7f5fdc0b5650 2022-07-21T01:49:15.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fd9ffb700 1 -- 172.21.15.140:0/2522140976 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd400b9d0 con 0x7f5fdc0b5650 2022-07-21T01:49:15.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fd9ffb700 1 -- 172.21.15.140:0/2522140976 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fd400bb40 con 0x7f5fdc0b5650 2022-07-21T01:49:15.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 -- 172.21.15.140:0/2522140976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5fdc0b5650 msgr2=0x7f5fdc0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 --2- 172.21.15.140:0/2522140976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5fdc0b5650 0x7f5fdc0b5a30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f5fd40032f0 tx=0x7f5fd4009290).stop 2022-07-21T01:49:15.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 -- 172.21.15.140:0/2522140976 shutdown_connections 2022-07-21T01:49:15.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 --2- 172.21.15.140:0/2522140976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5fdc0af8e0 0x7f5fdc05bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 --2- 172.21.15.140:0/2522140976 >> v2:172.21.15.152:3300/0 conn(0x7f5fdc0adff0 0x7f5fdc147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 --2- 172.21.15.140:0/2522140976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5fdc0b5650 0x7f5fdc0b5a30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 -- 172.21.15.140:0/2522140976 >> 172.21.15.140:0/2522140976 conn(0x7f5fdc0634e0 msgr2=0x7f5fdc0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:15.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 -- 172.21.15.140:0/2522140976 shutdown_connections 2022-07-21T01:49:15.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 -- 172.21.15.140:0/2522140976 wait complete. 2022-07-21T01:49:15.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 Processor -- start 2022-07-21T01:49:15.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 -- start start 2022-07-21T01:49:15.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fdc0adff0 0x7f5fdc149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5fdc0af8e0 0x7f5fdc10ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5fdc0b5650 0x7f5fdc10f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc10f5e0 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc10e4c0 con 0x7f5fdc0adff0 2022-07-21T01:49:15.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdbfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc10e760 con 0x7f5fdc0b5650 2022-07-21T01:49:15.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fda7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5fdc0af8e0 0x7f5fdc10ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fda7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5fdc0af8e0 0x7f5fdc10ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33854/0 (socket says 172.21.15.140:33854) 2022-07-21T01:49:15.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fda7fc700 1 -- 172.21.15.140:0/4189630579 learned_addr learned my addr 172.21.15.140:0/4189630579 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:15.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdaffd700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fdc0adff0 0x7f5fdc149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.259+0000 7f5fdb7fe700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5fdc0b5650 0x7f5fdc10f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fda7fc700 1 -- 172.21.15.140:0/4189630579 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5fdc0b5650 msgr2=0x7f5fdc10f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fda7fc700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5fdc0b5650 0x7f5fdc10f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fda7fc700 1 -- 172.21.15.140:0/4189630579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fdc0adff0 msgr2=0x7f5fdc149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fda7fc700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fdc0adff0 0x7f5fdc149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fda7fc700 1 -- 172.21.15.140:0/4189630579 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd4009530 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fda7fc700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5fdc0af8e0 0x7f5fdc10ec00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f5fd000a000 tx=0x7f5fd0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:15.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fcbfff700 1 -- 172.21.15.140:0/4189630579 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fd0011780 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fcbfff700 1 -- 172.21.15.140:0/4189630579 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd00118e0 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fdbfff700 1 -- 172.21.15.140:0/4189630579 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fdc065bf0 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fcbfff700 1 -- 172.21.15.140:0/4189630579 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fd0011c40 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fdbfff700 1 -- 172.21.15.140:0/4189630579 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fdc10fb60 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fdbfff700 1 -- 172.21.15.140:0/4189630579 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fdc065a60 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fcbfff700 1 -- 172.21.15.140:0/4189630579 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5fd001c460 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fcbfff700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fb80539e0 0x7f5fb8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fcbfff700 1 -- 172.21.15.140:0/4189630579 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5fd006c380 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.263+0000 7f5fdaffd700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fb80539e0 0x7f5fb8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.267+0000 7f5fdaffd700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fb80539e0 0x7f5fb8055ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f5fd401b000 tx=0x7f5fd4016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:15.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.271+0000 7f5fcbfff700 1 -- 172.21.15.140:0/4189630579 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5fd00431c0 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 -- 172.21.15.140:0/4189630579 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fdc1101e0 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fcbfff700 1 -- 172.21.15.140:0/4189630579 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f5fdc1101e0 con 0x7f5fdc0af8e0 2022-07-21T01:49:15.435 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:15.435 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/838095894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":838095894},{"type":"v1","addr":"172.21.15.140:6835","nonce":838095894}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:49:12.585320+0000","modified":"2022-07-21T01:49:13.589194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1420650762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1420650762},{"type":"v1","addr":"172.21.15.152:6833","nonce":1420650762}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-07-21T01:49:15.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 -- 172.21.15.140:0/4189630579 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fb80539e0 msgr2=0x7f5fb8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fb80539e0 0x7f5fb8055ea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f5fd401b000 tx=0x7f5fd4016000).stop 2022-07-21T01:49:15.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 -- 172.21.15.140:0/4189630579 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5fdc0af8e0 msgr2=0x7f5fdc10ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5fdc0af8e0 0x7f5fdc10ec00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f5fd000a000 tx=0x7f5fd0008040).stop 2022-07-21T01:49:15.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 -- 172.21.15.140:0/4189630579 shutdown_connections 2022-07-21T01:49:15.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fb80539e0 0x7f5fb8055ea0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5fdc0b5650 0x7f5fdc10f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5fdc0af8e0 0x7f5fdc10ec00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 --2- 172.21.15.140:0/4189630579 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5fdc0adff0 0x7f5fdc149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 -- 172.21.15.140:0/4189630579 >> 172.21.15.140:0/4189630579 conn(0x7f5fdc0634e0 msgr2=0x7f5fdc0b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:15.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 -- 172.21.15.140:0/4189630579 shutdown_connections 2022-07-21T01:49:15.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.431+0000 7f5fdbfff700 1 -- 172.21.15.140:0/4189630579 wait complete. 2022-07-21T01:49:15.444 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 10 2022-07-21T01:49:15.648 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:49:12.585320+0000', 'modified': '2022-07-21T01:49:13.589194+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4894': {'gid': 4894, 'name': 'b', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/1420650762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 1420650762}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 1420650762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:49:15.649 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:49:15.650 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:49:15.766 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":4894,"want_state":"up:active","state":"up:active","rank_uptime":3.1677089660000002,"mdsmap_epoch":10,"osdmap_epoch":32,"osdmap_epoch_barrier":32,"uptime":9.5675134659999994} 2022-07-21T01:49:15.767 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 4894, "mdsmap_epoch": 10, "osdmap_epoch": 32, "osdmap_epoch_barrier": 32, "rank_uptime": 3.167708966, "state": "up:active", "uptime": 9.567513466, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:49:15.769 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:49:15.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.879+0000 7f1b14e80700 1 Processor -- start 2022-07-21T01:49:15.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.879+0000 7f1b14e80700 1 -- start start 2022-07-21T01:49:15.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b14e80700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b100b0570 0x7f1b100b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b14e80700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b101433d0 con 0x7f1b101424e0 2022-07-21T01:49:15.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b14e80700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b10143670 con 0x7f1b100aea10 2022-07-21T01:49:15.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b14e80700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b10143b10 con 0x7f1b100b0570 2022-07-21T01:49:15.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b0ffff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1b101424e0 0x7f1b100ad350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48010/0 (socket says 172.21.15.140:48010) 2022-07-21T01:49:15.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b0ffff700 1 -- 172.21.15.140:0/3494446652 learned_addr learned my addr 172.21.15.140:0/3494446652 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:15.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b0f7fe700 1 --2- 172.21.15.140:0/3494446652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b100b0570 0x7f1b100b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b0f7fe700 1 -- 172.21.15.140:0/3494446652 >> v1:172.21.15.152:6789/0 conn(0x7f1b100aea10 legacy=0x7f1b10141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b0f7fe700 1 -- 172.21.15.140:0/3494446652 >> v1:172.21.15.140:6789/0 conn(0x7f1b101424e0 legacy=0x7f1b100ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b0f7fe700 1 -- 172.21.15.140:0/3494446652 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b100b1010 con 0x7f1b100b0570 2022-07-21T01:49:15.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b0effd700 1 -- 172.21.15.140:0/3494446652 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293304391 0 0) 0x7f1b101433d0 con 0x7f1b101424e0 2022-07-21T01:49:15.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b0f7fe700 1 --2- 172.21.15.140:0/3494446652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b100b0570 0x7f1b100b0950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1b00002f70 tx=0x7f1b00006df0).ready entity=mon.2 client_cookie=2b15688ee8b215d4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:15.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b0effd700 1 -- 172.21.15.140:0/3494446652 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b0000d5c0 con 0x7f1b100b0570 2022-07-21T01:49:15.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b0effd700 1 -- 172.21.15.140:0/3494446652 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b00000e90 con 0x7f1b100b0570 2022-07-21T01:49:15.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b0effd700 1 -- 172.21.15.140:0/3494446652 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b0000d910 con 0x7f1b100b0570 2022-07-21T01:49:15.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b14e80700 1 -- 172.21.15.140:0/3494446652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b100b0570 msgr2=0x7f1b100b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b14e80700 1 --2- 172.21.15.140:0/3494446652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b100b0570 0x7f1b100b0950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1b00002f70 tx=0x7f1b00006df0).stop 2022-07-21T01:49:15.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b14e80700 1 -- 172.21.15.140:0/3494446652 shutdown_connections 2022-07-21T01:49:15.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b14e80700 1 --2- 172.21.15.140:0/3494446652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b100b0570 0x7f1b100b0950 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b14e80700 1 -- 172.21.15.140:0/3494446652 >> 172.21.15.140:0/3494446652 conn(0x7f1b100b9360 msgr2=0x7f1b100b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:15.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b14e80700 1 -- 172.21.15.140:0/3494446652 shutdown_connections 2022-07-21T01:49:15.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.883+0000 7f1b14e80700 1 -- 172.21.15.140:0/3494446652 wait complete. 2022-07-21T01:49:15.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b14e80700 1 Processor -- start 2022-07-21T01:49:15.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b14e80700 1 -- start start 2022-07-21T01:49:15.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b14e80700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b100aea10 0x7f1b100b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b14e80700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b100b0570 0x7f1b101130e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b14e80700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b101424e0 0x7f1b100b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b14e80700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100b3a40 con 0x7f1b100aea10 2022-07-21T01:49:15.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b14e80700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b100b4a80 con 0x7f1b100b0570 2022-07-21T01:49:15.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b14e80700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b10113b40 con 0x7f1b101424e0 2022-07-21T01:49:15.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b100aea10 0x7f1b100b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b100aea10 0x7f1b100b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33866/0 (socket says 172.21.15.140:33866) 2022-07-21T01:49:15.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b101424e0 0x7f1b100b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0f7fe700 1 -- 172.21.15.140:0/2289553890 learned_addr learned my addr 172.21.15.140:0/2289553890 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:15.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b07fff700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b100b0570 0x7f1b101130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0f7fe700 1 -- 172.21.15.140:0/2289553890 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b101424e0 msgr2=0x7f1b100b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0f7fe700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b101424e0 0x7f1b100b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0f7fe700 1 -- 172.21.15.140:0/2289553890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b100b0570 msgr2=0x7f1b101130e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:15.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0f7fe700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b100b0570 0x7f1b101130e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:15.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0f7fe700 1 -- 172.21.15.140:0/2289553890 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0000b180 con 0x7f1b100aea10 2022-07-21T01:49:15.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0f7fe700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b100aea10 0x7f1b100b4590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f1b00002f70 tx=0x7f1b00015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:15.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0d7fa700 1 -- 172.21.15.140:0/2289553890 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b0000de80 con 0x7f1b100aea10 2022-07-21T01:49:15.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0d7fa700 1 -- 172.21.15.140:0/2289553890 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b00011b40 con 0x7f1b100aea10 2022-07-21T01:49:15.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b14e80700 1 -- 172.21.15.140:0/2289553890 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b10152aa0 con 0x7f1b100aea10 2022-07-21T01:49:15.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b0d7fa700 1 -- 172.21.15.140:0/2289553890 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b00011ea0 con 0x7f1b100aea10 2022-07-21T01:49:15.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.887+0000 7f1b14e80700 1 -- 172.21.15.140:0/2289553890 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b100b4d50 con 0x7f1b100aea10 2022-07-21T01:49:15.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.891+0000 7f1b14e80700 1 -- 172.21.15.140:0/2289553890 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b100ace30 con 0x7f1b100aea10 2022-07-21T01:49:15.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.891+0000 7f1b0d7fa700 1 -- 172.21.15.140:0/2289553890 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1b00025030 con 0x7f1b100aea10 2022-07-21T01:49:15.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.891+0000 7f1b0d7fa700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1af4053b00 0x7f1af4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:15.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.891+0000 7f1b0d7fa700 1 -- 172.21.15.140:0/2289553890 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b0006d140 con 0x7f1b100aea10 2022-07-21T01:49:15.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.891+0000 7f1b07fff700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1af4053b00 0x7f1af4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:15.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.891+0000 7f1b07fff700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1af4053b00 0x7f1af4055fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f1af800d000 tx=0x7f1af8007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:15.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:15.899+0000 7f1b0d7fa700 1 -- 172.21.15.140:0/2289553890 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1b00044060 con 0x7f1b100aea10 2022-07-21T01:49:16.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.019+0000 7f1b14e80700 1 -- 172.21.15.140:0/2289553890 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b100574e0 con 0x7f1b100aea10 2022-07-21T01:49:16.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b0d7fa700 1 -- 172.21.15.140:0/2289553890 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f1b00023090 con 0x7f1b100aea10 2022-07-21T01:49:16.025 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:16.025 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":32,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:49:13.588079+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:49:10.708637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2022-07-21T01:49:11.590708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:49:16.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 -- 172.21.15.140:0/2289553890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1af4053b00 msgr2=0x7f1af4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1af4053b00 0x7f1af4055fc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f1af800d000 tx=0x7f1af8007b90).stop 2022-07-21T01:49:16.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 -- 172.21.15.140:0/2289553890 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b100aea10 msgr2=0x7f1b100b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b100aea10 0x7f1b100b4590 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f1b00002f70 tx=0x7f1b00015000).stop 2022-07-21T01:49:16.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 -- 172.21.15.140:0/2289553890 shutdown_connections 2022-07-21T01:49:16.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1af4053b00 0x7f1af4055fc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b101424e0 0x7f1b100b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b100b0570 0x7f1b101130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 --2- 172.21.15.140:0/2289553890 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b100aea10 0x7f1b100b4590 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 -- 172.21.15.140:0/2289553890 >> 172.21.15.140:0/2289553890 conn(0x7f1b100b9360 msgr2=0x7f1b100ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:16.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 -- 172.21.15.140:0/2289553890 shutdown_connections 2022-07-21T01:49:16.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.023+0000 7f1b14e80700 1 -- 172.21.15.140:0/2289553890 wait complete. 2022-07-21T01:49:16.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:49:10.708637+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:49:11.590708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:49:16.040 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:49:16.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a5ebb700 1 Processor -- start 2022-07-21T01:49:16.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a5ebb700 1 -- start start 2022-07-21T01:49:16.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a5ebb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2a00b9da0 0x7ff2a01468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a5ebb700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff2a01419a0 0x7ff2a0141e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a5ebb700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a00ba180 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a5ebb700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a00ad5b0 con 0x7ff2a01419a0 2022-07-21T01:49:16.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a5ebb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a01461b0 con 0x7ff2a00b9da0 2022-07-21T01:49:16.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff29ffff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff2a00b3cf0 0x7ff2a0142a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48022/0 (socket says 172.21.15.140:48022) 2022-07-21T01:49:16.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff29ffff700 1 -- 172.21.15.140:0/2506538607 learned_addr learned my addr 172.21.15.140:0/2506538607 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:16.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a56ba700 1 --2- 172.21.15.140:0/2506538607 >> v2:172.21.15.152:3300/0 conn(0x7ff2a01419a0 0x7ff2a0141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a4eb9700 1 --2- 172.21.15.140:0/2506538607 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2a00b9da0 0x7ff2a01468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff29f7fe700 1 -- 172.21.15.140:0/2506538607 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541506335 0 0) 0x7ff2a00ba180 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff29f7fe700 1 -- 172.21.15.140:0/2506538607 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff280003260 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a4eb9700 1 -- 172.21.15.140:0/2506538607 >> v2:172.21.15.152:3300/0 conn(0x7ff2a01419a0 msgr2=0x7ff2a0141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:16.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a4eb9700 1 --2- 172.21.15.140:0/2506538607 >> v2:172.21.15.152:3300/0 conn(0x7ff2a01419a0 0x7ff2a0141e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a4eb9700 1 -- 172.21.15.140:0/2506538607 >> v1:172.21.15.140:6789/0 conn(0x7ff2a00b3cf0 legacy=0x7ff2a0142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a4eb9700 1 -- 172.21.15.140:0/2506538607 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a0146450 con 0x7ff2a00b9da0 2022-07-21T01:49:16.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff2a4eb9700 1 --2- 172.21.15.140:0/2506538607 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2a00b9da0 0x7ff2a01468f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff290002f70 tx=0x7ff290006df0).ready entity=mon.2 client_cookie=1f673d57683be3a8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff29f7fe700 1 -- 172.21.15.140:0/2506538607 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff29000d4a0 con 0x7ff2a00b9da0 2022-07-21T01:49:16.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff29f7fe700 1 -- 172.21.15.140:0/2506538607 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff290000e90 con 0x7ff2a00b9da0 2022-07-21T01:49:16.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.147+0000 7ff29f7fe700 1 -- 172.21.15.140:0/2506538607 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff29000d7f0 con 0x7ff2a00b9da0 2022-07-21T01:49:16.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/2506538607 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2a00b9da0 msgr2=0x7ff2a01468f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 --2- 172.21.15.140:0/2506538607 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2a00b9da0 0x7ff2a01468f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7ff290002f70 tx=0x7ff290006df0).stop 2022-07-21T01:49:16.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/2506538607 shutdown_connections 2022-07-21T01:49:16.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 --2- 172.21.15.140:0/2506538607 >> v2:172.21.15.152:3300/0 conn(0x7ff2a01419a0 0x7ff2a0141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 --2- 172.21.15.140:0/2506538607 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2a00b9da0 0x7ff2a01468f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/2506538607 >> 172.21.15.140:0/2506538607 conn(0x7ff2a00678c0 msgr2=0x7ff2a00b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:16.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/2506538607 shutdown_connections 2022-07-21T01:49:16.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/2506538607 wait complete. 2022-07-21T01:49:16.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 Processor -- start 2022-07-21T01:49:16.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 -- start start 2022-07-21T01:49:16.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2a00b3cf0 0x7ff2a00b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2a00b9da0 0x7ff2a0151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2a01419a0 0x7ff2a00b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00b1310 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00b1070 con 0x7ff2a00b9da0 2022-07-21T01:49:16.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a5ebb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a0155310 con 0x7ff2a01419a0 2022-07-21T01:49:16.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a4eb9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2a00b3cf0 0x7ff2a00b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.151+0000 7ff2a4eb9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2a00b3cf0 0x7ff2a00b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33880/0 (socket says 172.21.15.140:33880) 2022-07-21T01:49:16.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff2a4eb9700 1 -- 172.21.15.140:0/339410356 learned_addr learned my addr 172.21.15.140:0/339410356 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:16.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff2a56ba700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2a01419a0 0x7ff2a00b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff29ffff700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2a00b9da0 0x7ff2a0151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff2a4eb9700 1 -- 172.21.15.140:0/339410356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2a01419a0 msgr2=0x7ff2a00b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff2a4eb9700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2a01419a0 0x7ff2a00b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff2a4eb9700 1 -- 172.21.15.140:0/339410356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2a00b9da0 msgr2=0x7ff2a0151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff2a4eb9700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2a00b9da0 0x7ff2a0151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff2a4eb9700 1 -- 172.21.15.140:0/339410356 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29000b180 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff2a4eb9700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2a00b3cf0 0x7ff2a00b6420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7ff290002f70 tx=0x7ff290014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff29dffb700 1 -- 172.21.15.140:0/339410356 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff290016070 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff29dffb700 1 -- 172.21.15.140:0/339410356 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2900138d0 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/339410356 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a01521a0 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff29dffb700 1 -- 172.21.15.140:0/339410356 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff290013c30 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/339410356 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a0114270 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/339410356 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a00574e0 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff29dffb700 1 -- 172.21.15.140:0/339410356 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff29000f030 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff29dffb700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff284053a70 0x7ff284055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff29dffb700 1 -- 172.21.15.140:0/339410356 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff29006ccf0 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.155+0000 7ff29ffff700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff284053a70 0x7ff284055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.159+0000 7ff29ffff700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff284053a70 0x7ff284055f30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7ff28800b000 tx=0x7ff288009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.163+0000 7ff29dffb700 1 -- 172.21.15.140:0/339410356 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff290043b70 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.287+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/339410356 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff2a00b9840 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.287+0000 7ff29dffb700 1 -- 172.21.15.140:0/339410356 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v32) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff290023020 con 0x7ff2a00b3cf0 2022-07-21T01:49:16.293 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:49:16.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/339410356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff284053a70 msgr2=0x7ff284055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff284053a70 0x7ff284055f30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7ff28800b000 tx=0x7ff288009040).stop 2022-07-21T01:49:16.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/339410356 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2a00b3cf0 msgr2=0x7ff2a00b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2a00b3cf0 0x7ff2a00b6420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7ff290002f70 tx=0x7ff290014000).stop 2022-07-21T01:49:16.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/339410356 shutdown_connections 2022-07-21T01:49:16.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff284053a70 0x7ff284055f30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2a01419a0 0x7ff2a00b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2a00b9da0 0x7ff2a0151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 --2- 172.21.15.140:0/339410356 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2a00b3cf0 0x7ff2a00b6420 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/339410356 >> 172.21.15.140:0/339410356 conn(0x7ff2a00678c0 msgr2=0x7ff2a00b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:16.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/339410356 shutdown_connections 2022-07-21T01:49:16.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.291+0000 7ff2a5ebb700 1 -- 172.21.15.140:0/339410356 wait complete. 2022-07-21T01:49:16.307 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:49:16.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d2446f700 1 Processor -- start 2022-07-21T01:49:16.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d2446f700 1 -- start start 2022-07-21T01:49:16.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d2446f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d1c1462a0 0x7f1d1c146680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d2446f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d1c0988b0 0x7f1d1c146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d2446f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1c0afdb0 con 0x7f1d1c1422c0 2022-07-21T01:49:16.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d2446f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c0b0050 con 0x7f1d1c0988b0 2022-07-21T01:49:16.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d2446f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c05fcd0 con 0x7f1d1c1462a0 2022-07-21T01:49:16.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d2220b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d1c1462a0 0x7f1d1c146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d21a0a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d1c0988b0 0x7f1d1c146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d21a0a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d1c0988b0 0x7f1d1c146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:39630/0 (socket says 172.21.15.140:39630) 2022-07-21T01:49:16.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d22a0c700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1d1c1422c0 0x7f1d1c060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:49152/0 (socket says 172.21.15.140:49152) 2022-07-21T01:49:16.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d21a0a700 1 -- 172.21.15.140:0/549434432 learned_addr learned my addr 172.21.15.140:0/549434432 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:16.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d2220b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d1c1462a0 0x7f1d1c146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:58850/0 (socket says 172.21.15.140:58850) 2022-07-21T01:49:16.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d21a0a700 1 -- 172.21.15.140:0/549434432 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d1c1462a0 msgr2=0x7f1d1c146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:16.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d21a0a700 1 --2- 172.21.15.140:0/549434432 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d1c1462a0 0x7f1d1c146680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d21a0a700 1 -- 172.21.15.140:0/549434432 >> v1:172.21.15.152:6789/0 conn(0x7f1d1c1422c0 legacy=0x7f1d1c060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d21a0a700 1 -- 172.21.15.140:0/549434432 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c0afae0 con 0x7f1d1c0988b0 2022-07-21T01:49:16.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d21a0a700 1 --2- 172.21.15.140:0/549434432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d1c0988b0 0x7f1d1c146b70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f1d0c002f70 tx=0x7f1d0c006eb0).ready entity=mon.1 client_cookie=77ac1dc52afe6e97 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d21209700 1 -- 172.21.15.140:0/549434432 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c00d570 con 0x7f1d1c0988b0 2022-07-21T01:49:16.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d21209700 1 -- 172.21.15.140:0/549434432 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c000e90 con 0x7f1d1c0988b0 2022-07-21T01:49:16.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.415+0000 7f1d21209700 1 -- 172.21.15.140:0/549434432 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c00d900 con 0x7f1d1c0988b0 2022-07-21T01:49:16.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 -- 172.21.15.140:0/549434432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d1c0988b0 msgr2=0x7f1d1c146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 --2- 172.21.15.140:0/549434432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d1c0988b0 0x7f1d1c146b70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f1d0c002f70 tx=0x7f1d0c006eb0).stop 2022-07-21T01:49:16.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 -- 172.21.15.140:0/549434432 shutdown_connections 2022-07-21T01:49:16.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 --2- 172.21.15.140:0/549434432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d1c0988b0 0x7f1d1c146b70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 --2- 172.21.15.140:0/549434432 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d1c1462a0 0x7f1d1c146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 -- 172.21.15.140:0/549434432 >> 172.21.15.140:0/549434432 conn(0x7f1d1c0b5f00 msgr2=0x7f1d1c0b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:16.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 -- 172.21.15.140:0/549434432 shutdown_connections 2022-07-21T01:49:16.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 -- 172.21.15.140:0/549434432 wait complete. 2022-07-21T01:49:16.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 Processor -- start 2022-07-21T01:49:16.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 -- start start 2022-07-21T01:49:16.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d1c0988b0 0x7f1d1c0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d1c1422c0 0x7f1d1c0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d1c1462a0 0x7f1d1c09b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c0b3840 con 0x7f1d1c1422c0 2022-07-21T01:49:16.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c0b25e0 con 0x7f1d1c1462a0 2022-07-21T01:49:16.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2446f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c0b2810 con 0x7f1d1c0988b0 2022-07-21T01:49:16.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d21a0a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d1c1422c0 0x7f1d1c0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d2220b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d1c0988b0 0x7f1d1c0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d21a0a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d1c1422c0 0x7f1d1c0b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33896/0 (socket says 172.21.15.140:33896) 2022-07-21T01:49:16.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d21a0a700 1 -- 172.21.15.140:0/3136081729 learned_addr learned my addr 172.21.15.140:0/3136081729 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:16.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.419+0000 7f1d22a0c700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d1c1462a0 0x7f1d1c09b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d21a0a700 1 -- 172.21.15.140:0/3136081729 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d1c0988b0 msgr2=0x7f1d1c0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d21a0a700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d1c0988b0 0x7f1d1c0b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d21a0a700 1 -- 172.21.15.140:0/3136081729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d1c1462a0 msgr2=0x7f1d1c09b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d21a0a700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d1c1462a0 0x7f1d1c09b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d21a0a700 1 -- 172.21.15.140:0/3136081729 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c00b1e0 con 0x7f1d1c1422c0 2022-07-21T01:49:16.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d22a0c700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d1c1462a0 0x7f1d1c09b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:16.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d21a0a700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d1c1422c0 0x7f1d1c0b2f20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f1d0c002f70 tx=0x7f1d0c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d0b7fe700 1 -- 172.21.15.140:0/3136081729 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c006b40 con 0x7f1d1c1422c0 2022-07-21T01:49:16.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d2446f700 1 -- 172.21.15.140:0/3136081729 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c0b1340 con 0x7f1d1c1422c0 2022-07-21T01:49:16.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d0b7fe700 1 -- 172.21.15.140:0/3136081729 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c00d8f0 con 0x7f1d1c1422c0 2022-07-21T01:49:16.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d0b7fe700 1 -- 172.21.15.140:0/3136081729 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0c011d70 con 0x7f1d1c1422c0 2022-07-21T01:49:16.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d2446f700 1 -- 172.21.15.140:0/3136081729 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1c099f50 con 0x7f1d1c1422c0 2022-07-21T01:49:16.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d2446f700 1 -- 172.21.15.140:0/3136081729 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1c0b29d0 con 0x7f1d1c1422c0 2022-07-21T01:49:16.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d0b7fe700 1 -- 172.21.15.140:0/3136081729 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d0c01a040 con 0x7f1d1c1422c0 2022-07-21T01:49:16.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d0b7fe700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1cfc053a70 0x7f1cfc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d0b7fe700 1 -- 172.21.15.140:0/3136081729 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1d0c06cd00 con 0x7f1d1c1422c0 2022-07-21T01:49:16.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d2220b700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1cfc053a70 0x7f1cfc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.423+0000 7f1d2220b700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1cfc053a70 0x7f1cfc055f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f1d1800b000 tx=0x7f1d18006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.431+0000 7f1d0b7fe700 1 -- 172.21.15.140:0/3136081729 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d0c070030 con 0x7f1d1c1422c0 2022-07-21T01:49:16.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.551+0000 7f1d2446f700 1 -- 172.21.15.140:0/3136081729 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1d1c0574e0 con 0x7f1d1c1422c0 2022-07-21T01:49:16.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d0b7fe700 1 -- 172.21.15.140:0/3136081729 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1d0c023020 con 0x7f1d1c1422c0 2022-07-21T01:49:16.557 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:49:16.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 -- 172.21.15.140:0/3136081729 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1cfc053a70 msgr2=0x7f1cfc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1cfc053a70 0x7f1cfc055f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f1d1800b000 tx=0x7f1d18006df0).stop 2022-07-21T01:49:16.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 -- 172.21.15.140:0/3136081729 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d1c1422c0 msgr2=0x7f1d1c0b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d1c1422c0 0x7f1d1c0b2f20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f1d0c002f70 tx=0x7f1d0c015000).stop 2022-07-21T01:49:16.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 -- 172.21.15.140:0/3136081729 shutdown_connections 2022-07-21T01:49:16.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1cfc053a70 0x7f1cfc055f30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d1c1462a0 0x7f1d1c09b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d1c1422c0 0x7f1d1c0b2f20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 --2- 172.21.15.140:0/3136081729 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d1c0988b0 0x7f1d1c0b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 -- 172.21.15.140:0/3136081729 >> 172.21.15.140:0/3136081729 conn(0x7f1d1c0b5f00 msgr2=0x7f1d1c140d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:16.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 -- 172.21.15.140:0/3136081729 shutdown_connections 2022-07-21T01:49:16.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.555+0000 7f1d2446f700 1 -- 172.21.15.140:0/3136081729 wait complete. 2022-07-21T01:49:16.571 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:49:16.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f592cb75700 1 Processor -- start 2022-07-21T01:49:16.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f592cb75700 1 -- start start 2022-07-21T01:49:16.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f592cb75700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f59280b9da0 0x7f59281468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f592cb75700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59281419a0 0x7f5928141e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f592cb75700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59280ba180 con 0x7f59280b3cf0 2022-07-21T01:49:16.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f592cb75700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59280ad5b0 con 0x7f59280b9da0 2022-07-21T01:49:16.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f592cb75700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59281461b0 con 0x7f59281419a0 2022-07-21T01:49:16.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f59277fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f59280b9da0 0x7f59281468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f5927fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59281419a0 0x7f5928141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f5926ffd700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f59280b3cf0 0x7f5928142a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:49164/0 (socket says 172.21.15.140:49164) 2022-07-21T01:49:16.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f5926ffd700 1 -- 172.21.15.140:0/2939022948 learned_addr learned my addr 172.21.15.140:0/2939022948 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:16.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f59277fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f59280b9da0 0x7f59281468f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:39642/0 (socket says 172.21.15.140:39642) 2022-07-21T01:49:16.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f59267fc700 1 -- 172.21.15.140:0/2939022948 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788075151 0 0) 0x7f59280ba180 con 0x7f59280b3cf0 2022-07-21T01:49:16.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f59267fc700 1 -- 172.21.15.140:0/2939022948 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5914003260 con 0x7f59280b3cf0 2022-07-21T01:49:16.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f59277fe700 1 -- 172.21.15.140:0/2939022948 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59281419a0 msgr2=0x7f5928141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:16.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f59277fe700 1 --2- 172.21.15.140:0/2939022948 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59281419a0 0x7f5928141e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f59277fe700 1 -- 172.21.15.140:0/2939022948 >> v1:172.21.15.152:6789/0 conn(0x7f59280b3cf0 legacy=0x7f5928142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f59277fe700 1 -- 172.21.15.140:0/2939022948 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5928146450 con 0x7f59280b9da0 2022-07-21T01:49:16.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f59277fe700 1 --2- 172.21.15.140:0/2939022948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59280b9da0 0x7f59281468f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f5918002f70 tx=0x7f5918006df0).ready entity=mon.1 client_cookie=a94feb8ad6a4b065 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f59267fc700 1 -- 172.21.15.140:0/2939022948 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f591800d4a0 con 0x7f59280b9da0 2022-07-21T01:49:16.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.679+0000 7f59267fc700 1 -- 172.21.15.140:0/2939022948 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5918000e90 con 0x7f59280b9da0 2022-07-21T01:49:16.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f59267fc700 1 -- 172.21.15.140:0/2939022948 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f591800d830 con 0x7f59280b9da0 2022-07-21T01:49:16.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 -- 172.21.15.140:0/2939022948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59280b9da0 msgr2=0x7f59281468f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 --2- 172.21.15.140:0/2939022948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59280b9da0 0x7f59281468f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f5918002f70 tx=0x7f5918006df0).stop 2022-07-21T01:49:16.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 -- 172.21.15.140:0/2939022948 shutdown_connections 2022-07-21T01:49:16.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 --2- 172.21.15.140:0/2939022948 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59281419a0 0x7f5928141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 --2- 172.21.15.140:0/2939022948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59280b9da0 0x7f59281468f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 -- 172.21.15.140:0/2939022948 >> 172.21.15.140:0/2939022948 conn(0x7f59280678c0 msgr2=0x7f59280b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:16.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 -- 172.21.15.140:0/2939022948 shutdown_connections 2022-07-21T01:49:16.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 -- 172.21.15.140:0/2939022948 wait complete. 2022-07-21T01:49:16.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 Processor -- start 2022-07-21T01:49:16.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 -- start start 2022-07-21T01:49:16.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59280b3cf0 0x7f5928152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59280b9da0 0x7f59281470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59281419a0 0x7f592813d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592813fbb0 con 0x7f59281419a0 2022-07-21T01:49:16.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592813fde0 con 0x7f59280b3cf0 2022-07-21T01:49:16.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f592cb75700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5928140080 con 0x7f59280b9da0 2022-07-21T01:49:16.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.683+0000 7f5927fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59281419a0 0x7f592813d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5926ffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59280b9da0 0x7f59281470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5927fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59281419a0 0x7f592813d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33912/0 (socket says 172.21.15.140:33912) 2022-07-21T01:49:16.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5927fff700 1 -- 172.21.15.140:0/654800635 learned_addr learned my addr 172.21.15.140:0/654800635 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:16.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f59277fe700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59280b3cf0 0x7f5928152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5927fff700 1 -- 172.21.15.140:0/654800635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59280b9da0 msgr2=0x7f59281470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5927fff700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59280b9da0 0x7f59281470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5927fff700 1 -- 172.21.15.140:0/654800635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59280b3cf0 msgr2=0x7f5928152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5927fff700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59280b3cf0 0x7f5928152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5927fff700 1 -- 172.21.15.140:0/654800635 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591800b180 con 0x7f59281419a0 2022-07-21T01:49:16.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5927fff700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59281419a0 0x7f592813d7f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f5920006e00 tx=0x7f59200091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5924ff9700 1 -- 172.21.15.140:0/654800635 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5920012600 con 0x7f59281419a0 2022-07-21T01:49:16.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5924ff9700 1 -- 172.21.15.140:0/654800635 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5920012760 con 0x7f59281419a0 2022-07-21T01:49:16.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f592cb75700 1 -- 172.21.15.140:0/654800635 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5928152c00 con 0x7f59281419a0 2022-07-21T01:49:16.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5924ff9700 1 -- 172.21.15.140:0/654800635 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5920012ac0 con 0x7f59281419a0 2022-07-21T01:49:16.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f592cb75700 1 -- 172.21.15.140:0/654800635 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59281411b0 con 0x7f59281419a0 2022-07-21T01:49:16.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5924ff9700 1 -- 172.21.15.140:0/654800635 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5920022070 con 0x7f59281419a0 2022-07-21T01:49:16.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5924ff9700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5908053aa0 0x7f5908055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f592cb75700 1 -- 172.21.15.140:0/654800635 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5928141460 con 0x7f59281419a0 2022-07-21T01:49:16.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f5924ff9700 1 -- 172.21.15.140:0/654800635 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f592006c410 con 0x7f59281419a0 2022-07-21T01:49:16.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f59277fe700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5908053aa0 0x7f5908055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.687+0000 7f59277fe700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5908053aa0 0x7f5908055f60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f591801a000 tx=0x7f5918014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.695+0000 7f5924ff9700 1 -- 172.21.15.140:0/654800635 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5920016080 con 0x7f59281419a0 2022-07-21T01:49:16.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.819+0000 7f592cb75700 1 -- 172.21.15.140:0/654800635 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59280574e0 con 0x7f59281419a0 2022-07-21T01:49:16.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.819+0000 7f5924ff9700 1 -- 172.21.15.140:0/654800635 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5920043290 con 0x7f59281419a0 2022-07-21T01:49:16.823 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:49:16.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.819+0000 7f592cb75700 1 -- 172.21.15.140:0/654800635 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5908053aa0 msgr2=0x7f5908055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.819+0000 7f592cb75700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5908053aa0 0x7f5908055f60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f591801a000 tx=0x7f5918014000).stop 2022-07-21T01:49:16.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.819+0000 7f592cb75700 1 -- 172.21.15.140:0/654800635 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59281419a0 msgr2=0x7f592813d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.819+0000 7f592cb75700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59281419a0 0x7f592813d7f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f5920006e00 tx=0x7f59200091d0).stop 2022-07-21T01:49:16.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.819+0000 7f592cb75700 1 -- 172.21.15.140:0/654800635 shutdown_connections 2022-07-21T01:49:16.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.819+0000 7f592cb75700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5908053aa0 0x7f5908055f60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.823+0000 7f592cb75700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59281419a0 0x7f592813d7f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.823+0000 7f592cb75700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59280b9da0 0x7f59281470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.823+0000 7f592cb75700 1 --2- 172.21.15.140:0/654800635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59280b3cf0 0x7f5928152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.823+0000 7f592cb75700 1 -- 172.21.15.140:0/654800635 >> 172.21.15.140:0/654800635 conn(0x7f59280678c0 msgr2=0x7f59280b9330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:16.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.823+0000 7f592cb75700 1 -- 172.21.15.140:0/654800635 shutdown_connections 2022-07-21T01:49:16.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.823+0000 7f592cb75700 1 -- 172.21.15.140:0/654800635 wait complete. 2022-07-21T01:49:16.835 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:49:16.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 Processor -- start 2022-07-21T01:49:16.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 -- start start 2022-07-21T01:49:16.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c3c0b1220 0x7f4c3c0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c1472b0 con 0x7f4c3c147c90 2022-07-21T01:49:16.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c147550 con 0x7f4c3c05cf40 2022-07-21T01:49:16.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0b0920 con 0x7f4c3c0b1220 2022-07-21T01:49:16.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c416d7700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4c3c147c90 0x7f4c3c14a570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48062/0 (socket says 172.21.15.140:48062) 2022-07-21T01:49:16.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c416d7700 1 -- 172.21.15.140:0/4207817676 learned_addr learned my addr 172.21.15.140:0/4207817676 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:16.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c426d9700 1 --2- 172.21.15.140:0/4207817676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c3c0b1220 0x7f4c3c0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c40ed6700 1 -- 172.21.15.140:0/4207817676 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342597196 0 0) 0x7f4c3c1472b0 con 0x7f4c3c147c90 2022-07-21T01:49:16.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c40ed6700 1 -- 172.21.15.140:0/4207817676 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c24003260 con 0x7f4c3c147c90 2022-07-21T01:49:16.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c40ed6700 1 -- 172.21.15.140:0/4207817676 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506818301 0 0) 0x7f4c3c147550 con 0x7f4c3c05cf40 2022-07-21T01:49:16.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c426d9700 1 -- 172.21.15.140:0/4207817676 >> v1:172.21.15.152:6789/0 conn(0x7f4c3c05cf40 legacy=0x7f4c3c0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c426d9700 1 -- 172.21.15.140:0/4207817676 >> v1:172.21.15.140:6789/0 conn(0x7f4c3c147c90 legacy=0x7f4c3c14a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c426d9700 1 -- 172.21.15.140:0/4207817676 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c1477f0 con 0x7f4c3c0b1220 2022-07-21T01:49:16.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c426d9700 1 --2- 172.21.15.140:0/4207817676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c3c0b1220 0x7f4c3c0b0430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f4c38001800 tx=0x7f4c340091d0).ready entity=mon.2 client_cookie=258359770ec06b2d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c40ed6700 1 -- 172.21.15.140:0/4207817676 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c38001660 con 0x7f4c3c0b1220 2022-07-21T01:49:16.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c40ed6700 1 -- 172.21.15.140:0/4207817676 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3400b580 con 0x7f4c3c0b1220 2022-07-21T01:49:16.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c40ed6700 1 -- 172.21.15.140:0/4207817676 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c3400b6f0 con 0x7f4c3c0b1220 2022-07-21T01:49:16.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 -- 172.21.15.140:0/4207817676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c3c0b1220 msgr2=0x7f4c3c0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 --2- 172.21.15.140:0/4207817676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c3c0b1220 0x7f4c3c0b0430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f4c38001800 tx=0x7f4c340091d0).stop 2022-07-21T01:49:16.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 -- 172.21.15.140:0/4207817676 shutdown_connections 2022-07-21T01:49:16.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 --2- 172.21.15.140:0/4207817676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c3c0b1220 0x7f4c3c0b0430 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 -- 172.21.15.140:0/4207817676 >> 172.21.15.140:0/4207817676 conn(0x7f4c3c0b5b70 msgr2=0x7f4c3c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:16.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 -- 172.21.15.140:0/4207817676 shutdown_connections 2022-07-21T01:49:16.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.939+0000 7f4c4413c700 1 -- 172.21.15.140:0/4207817676 wait complete. 2022-07-21T01:49:16.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c4413c700 1 Processor -- start 2022-07-21T01:49:16.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c4413c700 1 -- start start 2022-07-21T01:49:16.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c4413c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c3c05cf40 0x7f4c3c065890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c4413c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c3c0b1220 0x7f4c3c14a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c4413c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c3c147c90 0x7f4c3c066ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c4413c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0aedf0 con 0x7f4c3c05cf40 2022-07-21T01:49:16.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c4413c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c146b90 con 0x7f4c3c147c90 2022-07-21T01:49:16.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c4413c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0ab270 con 0x7f4c3c0b1220 2022-07-21T01:49:16.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c41ed8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c3c05cf40 0x7f4c3c065890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c426d9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c3c147c90 0x7f4c3c066ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c416d7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c3c0b1220 0x7f4c3c14a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c426d9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c3c147c90 0x7f4c3c066ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:39664/0 (socket says 172.21.15.140:39664) 2022-07-21T01:49:16.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c426d9700 1 -- 172.21.15.140:0/1026188167 learned_addr learned my addr 172.21.15.140:0/1026188167 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:16.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c41ed8700 1 -- 172.21.15.140:0/1026188167 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c3c0b1220 msgr2=0x7f4c3c14a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c41ed8700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c3c0b1220 0x7f4c3c14a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c41ed8700 1 -- 172.21.15.140:0/1026188167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c3c147c90 msgr2=0x7f4c3c066ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:16.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c41ed8700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c3c147c90 0x7f4c3c066ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:16.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c41ed8700 1 -- 172.21.15.140:0/1026188167 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34009470 con 0x7f4c3c05cf40 2022-07-21T01:49:16.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c426d9700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c3c147c90 0x7f4c3c066ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:16.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c41ed8700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c3c05cf40 0x7f4c3c065890 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f4c2c000ed0 tx=0x7f4c2c007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c2affd700 1 -- 172.21.15.140:0/1026188167 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c2c007e00 con 0x7f4c3c05cf40 2022-07-21T01:49:16.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c2affd700 1 -- 172.21.15.140:0/1026188167 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c012400 con 0x7f4c3c05cf40 2022-07-21T01:49:16.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c4413c700 1 -- 172.21.15.140:0/1026188167 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c0651f0 con 0x7f4c3c05cf40 2022-07-21T01:49:16.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c2affd700 1 -- 172.21.15.140:0/1026188167 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c2c012760 con 0x7f4c3c05cf40 2022-07-21T01:49:16.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.943+0000 7f4c4413c700 1 -- 172.21.15.140:0/1026188167 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c067160 con 0x7f4c3c05cf40 2022-07-21T01:49:16.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.947+0000 7f4c4413c700 1 -- 172.21.15.140:0/1026188167 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c0ab430 con 0x7f4c3c05cf40 2022-07-21T01:49:16.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.947+0000 7f4c2affd700 1 -- 172.21.15.140:0/1026188167 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4c2c019030 con 0x7f4c3c05cf40 2022-07-21T01:49:16.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.947+0000 7f4c2affd700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c1c053a70 0x7f4c1c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:16.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.947+0000 7f4c2affd700 1 -- 172.21.15.140:0/1026188167 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c2c06b0a0 con 0x7f4c3c05cf40 2022-07-21T01:49:16.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.947+0000 7f4c416d7700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c1c053a70 0x7f4c1c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:16.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.947+0000 7f4c416d7700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c1c053a70 0x7f4c1c055f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f4c3800d000 tx=0x7f4c38000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:16.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:16.951+0000 7f4c2affd700 1 -- 172.21.15.140:0/1026188167 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4c2c042070 con 0x7f4c3c05cf40 2022-07-21T01:49:17.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 -- 172.21.15.140:0/1026188167 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c3c146fa0 con 0x7f4c3c05cf40 2022-07-21T01:49:17.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c2affd700 1 -- 172.21.15.140:0/1026188167 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4c2c045690 con 0x7f4c3c05cf40 2022-07-21T01:49:17.078 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:49:17.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 -- 172.21.15.140:0/1026188167 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c1c053a70 msgr2=0x7f4c1c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:17.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c1c053a70 0x7f4c1c055f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f4c3800d000 tx=0x7f4c38000bc0).stop 2022-07-21T01:49:17.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 -- 172.21.15.140:0/1026188167 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c3c05cf40 msgr2=0x7f4c3c065890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:17.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c3c05cf40 0x7f4c3c065890 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f4c2c000ed0 tx=0x7f4c2c007b90).stop 2022-07-21T01:49:17.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 -- 172.21.15.140:0/1026188167 shutdown_connections 2022-07-21T01:49:17.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c1c053a70 0x7f4c1c055f30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c3c147c90 0x7f4c3c066ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c3c0b1220 0x7f4c3c14a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 --2- 172.21.15.140:0/1026188167 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c3c05cf40 0x7f4c3c065890 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 -- 172.21.15.140:0/1026188167 >> 172.21.15.140:0/1026188167 conn(0x7f4c3c0b5b70 msgr2=0x7f4c3c140410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:17.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 -- 172.21.15.140:0/1026188167 shutdown_connections 2022-07-21T01:49:17.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.075+0000 7f4c4413c700 1 -- 172.21.15.140:0/1026188167 wait complete. 2022-07-21T01:49:17.091 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:49:17.092 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:49:17.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.191+0000 7fdbd7fff700 1 Processor -- start 2022-07-21T01:49:17.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.191+0000 7fdbd7fff700 1 -- start start 2022-07-21T01:49:17.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.191+0000 7fdbd7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdbd80b5650 0x7fdbd80b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:17.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.191+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd80adff0 0x7fdbd8147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:17.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.191+0000 7fdbd7fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd8147010 con 0x7fdbd80af8e0 2022-07-21T01:49:17.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.191+0000 7fdbd7fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd81472b0 con 0x7fdbd80b5650 2022-07-21T01:49:17.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.191+0000 7fdbd7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd8146d40 con 0x7fdbd80adff0 2022-07-21T01:49:17.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd6ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdbd80b5650 0x7fdbd80b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:17.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd6ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdbd80b5650 0x7fdbd80b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33934/0 (socket says 172.21.15.140:33934) 2022-07-21T01:49:17.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd67fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd80adff0 0x7fdbd8147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:17.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd6ffd700 1 -- 172.21.15.140:0/4168561021 learned_addr learned my addr 172.21.15.140:0/4168561021 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:17.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd6ffd700 1 -- 172.21.15.140:0/4168561021 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd80adff0 msgr2=0x7fdbd8147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:17.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd6ffd700 1 --2- 172.21.15.140:0/4168561021 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd80adff0 0x7fdbd8147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd5ffb700 1 -- 172.21.15.140:0/4168561021 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226019080 0 0) 0x7fdbd8147010 con 0x7fdbd80af8e0 2022-07-21T01:49:17.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd6ffd700 1 -- 172.21.15.140:0/4168561021 >> v1:172.21.15.152:6789/0 conn(0x7fdbd80af8e0 legacy=0x7fdbd805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:17.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd6ffd700 1 -- 172.21.15.140:0/4168561021 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd8147c40 con 0x7fdbd80b5650 2022-07-21T01:49:17.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd6ffd700 1 --2- 172.21.15.140:0/4168561021 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd80b5650 0x7fdbd80b5a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fdbcc002f70 tx=0x7fdbcc006eb0).ready entity=mon.0 client_cookie=961a1cd6c050c0a9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:17.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd5ffb700 1 -- 172.21.15.140:0/4168561021 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbcc00d570 con 0x7fdbd80b5650 2022-07-21T01:49:17.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd5ffb700 1 -- 172.21.15.140:0/4168561021 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc000e90 con 0x7fdbd80b5650 2022-07-21T01:49:17.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd5ffb700 1 -- 172.21.15.140:0/4168561021 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbcc00d8c0 con 0x7fdbd80b5650 2022-07-21T01:49:17.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd7fff700 1 -- 172.21.15.140:0/4168561021 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd80b5650 msgr2=0x7fdbd80b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:17.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd7fff700 1 --2- 172.21.15.140:0/4168561021 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd80b5650 0x7fdbd80b5a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fdbcc002f70 tx=0x7fdbcc006eb0).stop 2022-07-21T01:49:17.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd7fff700 1 -- 172.21.15.140:0/4168561021 shutdown_connections 2022-07-21T01:49:17.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd7fff700 1 --2- 172.21.15.140:0/4168561021 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd80adff0 0x7fdbd8147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd7fff700 1 --2- 172.21.15.140:0/4168561021 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd80b5650 0x7fdbd80b5a30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd7fff700 1 -- 172.21.15.140:0/4168561021 >> 172.21.15.140:0/4168561021 conn(0x7fdbd80634e0 msgr2=0x7fdbd80b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:17.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd7fff700 1 -- 172.21.15.140:0/4168561021 shutdown_connections 2022-07-21T01:49:17.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.195+0000 7fdbd7fff700 1 -- 172.21.15.140:0/4168561021 wait complete. 2022-07-21T01:49:17.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd7fff700 1 Processor -- start 2022-07-21T01:49:17.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd7fff700 1 -- start start 2022-07-21T01:49:17.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd80adff0 0x7fdbd811b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:17.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd80af8e0 0x7fdbd8063990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:17.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbd80b5650 0x7fdbd814a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:17.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd7fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd815a200 con 0x7fdbd80adff0 2022-07-21T01:49:17.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd7fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd815a4a0 con 0x7fdbd80b5650 2022-07-21T01:49:17.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd81116b0 con 0x7fdbd80af8e0 2022-07-21T01:49:17.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd80adff0 0x7fdbd811b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:17.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd80adff0 0x7fdbd811b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33936/0 (socket says 172.21.15.140:33936) 2022-07-21T01:49:17.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd6ffd700 1 -- 172.21.15.140:0/2256367789 learned_addr learned my addr 172.21.15.140:0/2256367789 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:17.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd67fc700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd80af8e0 0x7fdbd8063990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:17.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd77fe700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbd80b5650 0x7fdbd814a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:17.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd6ffd700 1 -- 172.21.15.140:0/2256367789 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd80af8e0 msgr2=0x7fdbd8063990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:17.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd6ffd700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd80af8e0 0x7fdbd8063990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd6ffd700 1 -- 172.21.15.140:0/2256367789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbd80b5650 msgr2=0x7fdbd814a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:17.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd6ffd700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbd80b5650 0x7fdbd814a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd6ffd700 1 -- 172.21.15.140:0/2256367789 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc00b1e0 con 0x7fdbd80adff0 2022-07-21T01:49:17.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd77fe700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbd80b5650 0x7fdbd814a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:17.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd67fc700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd80af8e0 0x7fdbd8063990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:17.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd6ffd700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd80adff0 0x7fdbd811b0d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fdbcc002f70 tx=0x7fdbcc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:17.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbc7fff700 1 -- 172.21.15.140:0/2256367789 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbcc0118a0 con 0x7fdbd80adff0 2022-07-21T01:49:17.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbc7fff700 1 -- 172.21.15.140:0/2256367789 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc011a00 con 0x7fdbd80adff0 2022-07-21T01:49:17.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbc7fff700 1 -- 172.21.15.140:0/2256367789 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbcc011d60 con 0x7fdbd80adff0 2022-07-21T01:49:17.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd7fff700 1 -- 172.21.15.140:0/2256367789 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd8063e80 con 0x7fdbd80adff0 2022-07-21T01:49:17.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.199+0000 7fdbd7fff700 1 -- 172.21.15.140:0/2256367789 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd814a9c0 con 0x7fdbd80adff0 2022-07-21T01:49:17.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.203+0000 7fdbd7fff700 1 -- 172.21.15.140:0/2256367789 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd815a6a0 con 0x7fdbd80adff0 2022-07-21T01:49:17.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.203+0000 7fdbc7fff700 1 -- 172.21.15.140:0/2256367789 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdbcc01a040 con 0x7fdbd80adff0 2022-07-21T01:49:17.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.203+0000 7fdbc7fff700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb4053a70 0x7fdbb4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:17.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.203+0000 7fdbc7fff700 1 -- 172.21.15.140:0/2256367789 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdbcc06cd00 con 0x7fdbd80adff0 2022-07-21T01:49:17.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.203+0000 7fdbd67fc700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb4053a70 0x7fdbb4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:17.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.203+0000 7fdbd67fc700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb4053a70 0x7fdbb4055f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fdbc8006000 tx=0x7fdbc8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:17.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.207+0000 7fdbc7fff700 1 -- 172.21.15.140:0/2256367789 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdbcc070030 con 0x7fdbd80adff0 2022-07-21T01:49:17.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.363+0000 7fdbd7fff700 1 -- 172.21.15.140:0/2256367789 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbd805c4a0 con 0x7fdbd80adff0 2022-07-21T01:49:17.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbc7fff700 1 -- 172.21.15.140:0/2256367789 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2139 (secure 0 0 0) 0x7fdbcc023020 con 0x7fdbd80adff0 2022-07-21T01:49:17.371 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:17.371 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/838095894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":838095894},{"type":"v1","addr":"172.21.15.140:6835","nonce":838095894}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:49:12.585320+0000","modified":"2022-07-21T01:49:13.589194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1420650762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1420650762},{"type":"v1","addr":"172.21.15.152:6833","nonce":1420650762}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-07-21T01:49:17.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 -- 172.21.15.140:0/2256367789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb4053a70 msgr2=0x7fdbb4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:17.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb4053a70 0x7fdbb4055f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fdbc8006000 tx=0x7fdbc8008040).stop 2022-07-21T01:49:17.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 -- 172.21.15.140:0/2256367789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd80adff0 msgr2=0x7fdbd811b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:17.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd80adff0 0x7fdbd811b0d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fdbcc002f70 tx=0x7fdbcc015000).stop 2022-07-21T01:49:17.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 -- 172.21.15.140:0/2256367789 shutdown_connections 2022-07-21T01:49:17.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbb4053a70 0x7fdbb4055f30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbd80b5650 0x7fdbd814a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbd80af8e0 0x7fdbd8063990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 --2- 172.21.15.140:0/2256367789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbd80adff0 0x7fdbd811b0d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:17.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 -- 172.21.15.140:0/2256367789 >> 172.21.15.140:0/2256367789 conn(0x7fdbd80634e0 msgr2=0x7fdbd8145940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:17.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 -- 172.21.15.140:0/2256367789 shutdown_connections 2022-07-21T01:49:17.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:17.367+0000 7fdbd7fff700 1 -- 172.21.15.140:0/2256367789 wait complete. 2022-07-21T01:49:17.382 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 10 2022-07-21T01:49:17.587 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:49:12.585320+0000', 'modified': '2022-07-21T01:49:13.589194+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4894': {'gid': 4894, 'name': 'b', 'rank': 0, 'incarnation': 9, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/1420650762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 1420650762}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 1420650762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:49:17.588 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:49:17.589 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:49:17.704 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":4894,"want_state":"up:active","state":"up:active","rank_uptime":5.1041820080000004,"mdsmap_epoch":10,"osdmap_epoch":32,"osdmap_epoch_barrier":32,"uptime":11.503985418999999} 2022-07-21T01:49:17.705 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 4894, "mdsmap_epoch": 10, "osdmap_epoch": 32, "osdmap_epoch_barrier": 32, "rank_uptime": 5.104182008, "state": "up:active", "uptime": 11.503985419, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:49:17.706 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:49:17.707 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:49:17.708 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:49:17.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:17.710 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:49:17.721 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:49:17.780 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:49:17.781 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:49:17.795 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:49:17.797 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:49:17.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:17.799 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:49:17.811 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:49:17.812 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:49:17.857 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:49:17.874 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:49:17.875 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:49:17.933 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:49:17.936+0000 7f647b04b080 -1 init, newargv = 0x5576e9979490 newargc=9 2022-07-21T01:49:17.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22249]: starting ceph client 2022-07-21T01:49:17.939 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:49:17.940 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:49:17.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:17.942 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:49:17.943 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22249]: starting fuse 2022-07-21T01:49:17.950 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:49:17.951 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:49:17.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:17.952 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:49:17.953 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:49:17.953 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:49:17.954 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:49:17.955 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:49:17.955 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:49:17.956 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:49:17.957 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:49:17.957 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:49:17.958 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:49:17.959 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:49:17.960 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:49:17.961 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:49:17.961 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:49:17.962 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:49:17.962 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:49:17.963 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:49:17.966 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:49:17.967 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:49:17.968 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:49:17.969 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:49:17.970 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:49:17.973 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:49:18.041 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.22249.asok 2022-07-21T01:49:18.049 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22249.asok 2022-07-21T01:49:18.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:18.051 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22249.asok status 2022-07-21T01:49:18.147 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:49:18.147 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:49:18.148 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:49:18.148 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:49:18.148 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:49:18.148 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:49:18.149 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:49:18.149 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "22249", 2022-07-21T01:49:18.149 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:49:18.149 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:49:18.150 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:49:18.150 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:49:18.150 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 4841, 2022-07-21T01:49:18.150 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:49:18.151 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:49:18.151 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:49:18.151 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 4841 2022-07-21T01:49:18.152 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:49:18.152 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:49:18.152 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:49:18.152 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:49:18.153 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3363717656 2022-07-21T01:49:18.153 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:49:18.153 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:49:18.153 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:49:18.153 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:49:18.154 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:49:18.154 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3363717656 2022-07-21T01:49:18.154 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:49:18.154 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.4841 172.21.15.191:0/3363717656", 2022-07-21T01:49:18.155 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/3363717656", 2022-07-21T01:49:18.155 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:49:18.155 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 10, 2022-07-21T01:49:18.156 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 32, 2022-07-21T01:49:18.156 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:49:18.156 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:49:18.156 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:49:18.157 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:49:18.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:18.159 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:18.159 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:49:18.168 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:49:18.169 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:49:18.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:18.173 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:49:18.194 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_min_caps_per_client 2022-07-21T01:49:18.300 INFO:teuthology.orchestra.run.smithi140.stdout:{"mds_min_caps_per_client":"100"} 2022-07-21T01:49:18.301 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_min_caps_per_client": "100" } 2022-07-21T01:49:18.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:49:18.302 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:49:18.303 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:49:18.313 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:49:18.314 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:49:18.316 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-07-21T01:49:18.317 DEBUG:teuthology.orchestra.run.smithi191:> import sys 2022-07-21T01:49:18.318 DEBUG:teuthology.orchestra.run.smithi191:> import time 2022-07-21T01:49:18.318 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:49:18.319 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:49:18.320 DEBUG:teuthology.orchestra.run.smithi191:> n = 1000 2022-07-21T01:49:18.320 DEBUG:teuthology.orchestra.run.smithi191:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-07-21T01:49:18.321 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:49:18.322 DEBUG:teuthology.orchestra.run.smithi191:> if not os.path.exists(os.path.dirname(abs_path)): 2022-07-21T01:49:18.322 DEBUG:teuthology.orchestra.run.smithi191:> os.makedirs(os.path.dirname(abs_path)) 2022-07-21T01:49:18.323 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:49:18.324 DEBUG:teuthology.orchestra.run.smithi191:> for i in range(0, n): 2022-07-21T01:49:18.325 DEBUG:teuthology.orchestra.run.smithi191:> fname = "{0}_{1}".format(abs_path, i) 2022-07-21T01:49:18.326 DEBUG:teuthology.orchestra.run.smithi191:> with open(fname, '"'"'w'"'"') as f: 2022-07-21T01:49:18.329 DEBUG:teuthology.orchestra.run.smithi191:> f.write('"'"'content'"'"') 2022-07-21T01:49:18.330 DEBUG:teuthology.orchestra.run.smithi191:> if True: 2022-07-21T01:49:18.331 DEBUG:teuthology.orchestra.run.smithi191:> f.flush() 2022-07-21T01:49:18.332 DEBUG:teuthology.orchestra.run.smithi191:> os.fsync(f.fileno()) 2022-07-21T01:49:18.332 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:49:27.616 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:49:27.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.727+0000 7fb0fefa0700 1 Processor -- start 2022-07-21T01:49:27.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.727+0000 7fb0fefa0700 1 -- start start 2022-07-21T01:49:27.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb0f80b5650 0x7fb0f80b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:27.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb0f805bd90 0x7fb0f8142d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:27.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f80aea30 con 0x7fb0f80b1200 2022-07-21T01:49:27.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f8142970 con 0x7fb0f80b5650 2022-07-21T01:49:27.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f80aec60 con 0x7fb0f805bd90 2022-07-21T01:49:27.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fd79d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fb0f80b1200 0x7fb0f805c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48094/0 (socket says 172.21.15.140:48094) 2022-07-21T01:49:27.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fd79d700 1 -- 172.21.15.140:0/432000910 learned_addr learned my addr 172.21.15.140:0/432000910 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:27.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fdf9e700 1 --2- 172.21.15.140:0/432000910 >> v2:172.21.15.140:3300/0 conn(0x7fb0f80b5650 0x7fb0f80b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:27.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fe79f700 1 --2- 172.21.15.140:0/432000910 >> v2:172.21.15.152:3300/0 conn(0x7fb0f805bd90 0x7fb0f8142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:27.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fcf9c700 1 -- 172.21.15.140:0/432000910 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154243461 0 0) 0x7fb0f80aea30 con 0x7fb0f80b1200 2022-07-21T01:49:27.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fdf9e700 1 -- 172.21.15.140:0/432000910 >> v2:172.21.15.152:3300/0 conn(0x7fb0f805bd90 msgr2=0x7fb0f8142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:27.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fdf9e700 1 --2- 172.21.15.140:0/432000910 >> v2:172.21.15.152:3300/0 conn(0x7fb0f805bd90 0x7fb0f8142d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:27.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fdf9e700 1 -- 172.21.15.140:0/432000910 >> v1:172.21.15.140:6789/0 conn(0x7fb0f80b1200 legacy=0x7fb0f805c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:27.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fdf9e700 1 -- 172.21.15.140:0/432000910 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f80af180 con 0x7fb0f80b5650 2022-07-21T01:49:27.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fdf9e700 1 --2- 172.21.15.140:0/432000910 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb0f80b5650 0x7fb0f80b5a30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fb0ec002f70 tx=0x7fb0ec006df0).ready entity=mon.0 client_cookie=e8830e7e4652b9f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:27.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fcf9c700 1 -- 172.21.15.140:0/432000910 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0ec00d570 con 0x7fb0f80b5650 2022-07-21T01:49:27.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fcf9c700 1 -- 172.21.15.140:0/432000910 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ec000e90 con 0x7fb0f80b5650 2022-07-21T01:49:27.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fcf9c700 1 -- 172.21.15.140:0/432000910 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0ec00d8c0 con 0x7fb0f80b5650 2022-07-21T01:49:27.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 -- 172.21.15.140:0/432000910 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb0f80b5650 msgr2=0x7fb0f80b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:27.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 --2- 172.21.15.140:0/432000910 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb0f80b5650 0x7fb0f80b5a30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fb0ec002f70 tx=0x7fb0ec006df0).stop 2022-07-21T01:49:27.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 -- 172.21.15.140:0/432000910 shutdown_connections 2022-07-21T01:49:27.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 --2- 172.21.15.140:0/432000910 >> v2:172.21.15.152:3300/0 conn(0x7fb0f805bd90 0x7fb0f8142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:27.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 --2- 172.21.15.140:0/432000910 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb0f80b5650 0x7fb0f80b5a30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:27.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 -- 172.21.15.140:0/432000910 >> 172.21.15.140:0/432000910 conn(0x7fb0f80634e0 msgr2=0x7fb0f80b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:27.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 -- 172.21.15.140:0/432000910 shutdown_connections 2022-07-21T01:49:27.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 -- 172.21.15.140:0/432000910 wait complete. 2022-07-21T01:49:27.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.731+0000 7fb0fefa0700 1 Processor -- start 2022-07-21T01:49:27.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fefa0700 1 -- start start 2022-07-21T01:49:27.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fefa0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb0f805bd90 0x7fb0f8141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:27.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fefa0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0f80b1200 0x7fb0f8064f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:27.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fefa0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb0f80b5650 0x7fb0f813fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:27.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fefa0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f81480b0 con 0x7fb0f80b5650 2022-07-21T01:49:27.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fefa0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f813e520 con 0x7fb0f80b1200 2022-07-21T01:49:27.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fefa0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f80676e0 con 0x7fb0f805bd90 2022-07-21T01:49:27.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fe79f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb0f80b5650 0x7fb0f813fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:27.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fe79f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb0f80b5650 0x7fb0f813fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33958/0 (socket says 172.21.15.140:33958) 2022-07-21T01:49:27.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fe79f700 1 -- 172.21.15.140:0/2696091901 learned_addr learned my addr 172.21.15.140:0/2696091901 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:27.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fdf9e700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb0f805bd90 0x7fb0f8141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:27.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fd79d700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0f80b1200 0x7fb0f8064f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:27.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fe79f700 1 -- 172.21.15.140:0/2696091901 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb0f805bd90 msgr2=0x7fb0f8141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:27.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fe79f700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb0f805bd90 0x7fb0f8141a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:27.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fe79f700 1 -- 172.21.15.140:0/2696091901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0f80b1200 msgr2=0x7fb0f8064f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:27.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fe79f700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0f80b1200 0x7fb0f8064f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:27.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fe79f700 1 -- 172.21.15.140:0/2696091901 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec00b180 con 0x7fb0f80b5650 2022-07-21T01:49:27.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fe79f700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb0f80b5650 0x7fb0f813fb30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fb0f000c000 tx=0x7fb0f0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:27.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0eaffd700 1 -- 172.21.15.140:0/2696091901 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0f0016070 con 0x7fb0f80b5650 2022-07-21T01:49:27.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0eaffd700 1 -- 172.21.15.140:0/2696091901 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f0011750 con 0x7fb0f80b5650 2022-07-21T01:49:27.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0eaffd700 1 -- 172.21.15.140:0/2696091901 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb0f0011ab0 con 0x7fb0f80b5650 2022-07-21T01:49:27.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fefa0700 1 -- 172.21.15.140:0/2696091901 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f8153110 con 0x7fb0f80b5650 2022-07-21T01:49:27.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.735+0000 7fb0fefa0700 1 -- 172.21.15.140:0/2696091901 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f813e010 con 0x7fb0f80b5650 2022-07-21T01:49:27.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.739+0000 7fb0fefa0700 1 -- 172.21.15.140:0/2696091901 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f80678a0 con 0x7fb0f80b5650 2022-07-21T01:49:27.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.739+0000 7fb0eaffd700 1 -- 172.21.15.140:0/2696091901 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb0f0011c10 con 0x7fb0f80b5650 2022-07-21T01:49:27.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.739+0000 7fb0eaffd700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb0d8053b00 0x7fb0d8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:27.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.739+0000 7fb0eaffd700 1 -- 172.21.15.140:0/2696091901 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb0f006bef0 con 0x7fb0f80b5650 2022-07-21T01:49:27.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.739+0000 7fb0fdf9e700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb0d8053b00 0x7fb0d8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:27.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.739+0000 7fb0fdf9e700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb0d8053b00 0x7fb0d8055fc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fb0ec01a000 tx=0x7fb0ec014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:27.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.747+0000 7fb0eaffd700 1 -- 172.21.15.140:0/2696091901 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb0f0042e10 con 0x7fb0f80b5650 2022-07-21T01:49:27.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.907+0000 7fb0fefa0700 1 -- 172.21.15.140:0/2696091901 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0f80574e0 con 0x7fb0f80b5650 2022-07-21T01:49:27.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.907+0000 7fb0eaffd700 1 -- 172.21.15.140:0/2696091901 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2139 (secure 0 0 0) 0x7fb0f0046430 con 0x7fb0f80b5650 2022-07-21T01:49:27.913 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:27.913 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/838095894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":838095894},{"type":"v1","addr":"172.21.15.140:6835","nonce":838095894}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:49:12.585320+0000","modified":"2022-07-21T01:49:13.589194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1420650762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1420650762},{"type":"v1","addr":"172.21.15.152:6833","nonce":1420650762}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-07-21T01:49:27.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 -- 172.21.15.140:0/2696091901 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb0d8053b00 msgr2=0x7fb0d8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:27.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb0d8053b00 0x7fb0d8055fc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fb0ec01a000 tx=0x7fb0ec014000).stop 2022-07-21T01:49:27.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 -- 172.21.15.140:0/2696091901 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb0f80b5650 msgr2=0x7fb0f813fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:27.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb0f80b5650 0x7fb0f813fb30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fb0f000c000 tx=0x7fb0f0006df0).stop 2022-07-21T01:49:27.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 -- 172.21.15.140:0/2696091901 shutdown_connections 2022-07-21T01:49:27.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb0d8053b00 0x7fb0d8055fc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:27.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb0f80b5650 0x7fb0f813fb30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:27.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0f80b1200 0x7fb0f8064f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:27.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 --2- 172.21.15.140:0/2696091901 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb0f805bd90 0x7fb0f8141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:27.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 -- 172.21.15.140:0/2696091901 >> 172.21.15.140:0/2696091901 conn(0x7fb0f80634e0 msgr2=0x7fb0f81483a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:27.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 -- 172.21.15.140:0/2696091901 shutdown_connections 2022-07-21T01:49:27.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:27.911+0000 7fb0fefa0700 1 -- 172.21.15.140:0/2696091901 wait complete. 2022-07-21T01:49:27.920 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 10 2022-07-21T01:49:27.924 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_caps 20 2022-07-21T01:49:28.036 INFO:teuthology.orchestra.run.smithi152.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-07-21T01:49:28.037 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-07-21T01:49:28.039 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:49:28.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.147+0000 7f826ca60700 1 Processor -- start 2022-07-21T01:49:28.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.147+0000 7f826ca60700 1 -- start start 2022-07-21T01:49:28.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.147+0000 7f826ca60700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f82680b5650 0x7f82680b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.147+0000 7f826ca60700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82680adff0 0x7f8268147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.147+0000 7f826ca60700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8268147010 con 0x7f82680af8e0 2022-07-21T01:49:28.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.147+0000 7f826ca60700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82681472b0 con 0x7f82680b5650 2022-07-21T01:49:28.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.147+0000 7f826ca60700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8268146d40 con 0x7f82680adff0 2022-07-21T01:49:28.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.147+0000 7f82677fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f82680b5650 0x7f82680b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.147+0000 7f8267fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f82680af8e0 0x7f826805bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48112/0 (socket says 172.21.15.140:48112) 2022-07-21T01:49:28.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f8267fff700 1 -- 172.21.15.140:0/4087415000 learned_addr learned my addr 172.21.15.140:0/4087415000 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:28.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f8266ffd700 1 --2- 172.21.15.140:0/4087415000 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82680adff0 0x7f8268147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f82667fc700 1 -- 172.21.15.140:0/4087415000 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115731830 0 0) 0x7f8268147010 con 0x7f82680af8e0 2022-07-21T01:49:28.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f82667fc700 1 -- 172.21.15.140:0/4087415000 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8248003260 con 0x7f82680af8e0 2022-07-21T01:49:28.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f82677fe700 1 -- 172.21.15.140:0/4087415000 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82680adff0 msgr2=0x7f8268147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:28.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f82677fe700 1 --2- 172.21.15.140:0/4087415000 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82680adff0 0x7f8268147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f82677fe700 1 -- 172.21.15.140:0/4087415000 >> v1:172.21.15.140:6789/0 conn(0x7f82680af8e0 legacy=0x7f826805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f82677fe700 1 -- 172.21.15.140:0/4087415000 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268147c40 con 0x7f82680b5650 2022-07-21T01:49:28.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f8266ffd700 1 --2- 172.21.15.140:0/4087415000 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82680adff0 0x7f8268147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:49:28.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f82677fe700 1 --2- 172.21.15.140:0/4087415000 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82680b5650 0x7f82680b5a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f8250002f70 tx=0x7f8250006df0).ready entity=mon.0 client_cookie=899f497dcaa8cf7c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f82667fc700 1 -- 172.21.15.140:0/4087415000 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f825000d4a0 con 0x7f82680b5650 2022-07-21T01:49:28.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f82667fc700 1 -- 172.21.15.140:0/4087415000 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8250000e90 con 0x7f82680b5650 2022-07-21T01:49:28.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f82667fc700 1 -- 172.21.15.140:0/4087415000 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f825000d7f0 con 0x7f82680b5650 2022-07-21T01:49:28.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f826ca60700 1 -- 172.21.15.140:0/4087415000 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82680b5650 msgr2=0x7f82680b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f826ca60700 1 --2- 172.21.15.140:0/4087415000 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82680b5650 0x7f82680b5a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f8250002f70 tx=0x7f8250006df0).stop 2022-07-21T01:49:28.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f826ca60700 1 -- 172.21.15.140:0/4087415000 shutdown_connections 2022-07-21T01:49:28.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f826ca60700 1 --2- 172.21.15.140:0/4087415000 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82680adff0 0x7f8268147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f826ca60700 1 --2- 172.21.15.140:0/4087415000 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82680b5650 0x7f82680b5a30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f826ca60700 1 -- 172.21.15.140:0/4087415000 >> 172.21.15.140:0/4087415000 conn(0x7f82680634e0 msgr2=0x7f82680b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:28.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f826ca60700 1 -- 172.21.15.140:0/4087415000 shutdown_connections 2022-07-21T01:49:28.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f826ca60700 1 -- 172.21.15.140:0/4087415000 wait complete. 2022-07-21T01:49:28.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f826ca60700 1 Processor -- start 2022-07-21T01:49:28.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.151+0000 7f826ca60700 1 -- start start 2022-07-21T01:49:28.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f826ca60700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82680adff0 0x7f8268159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f826ca60700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82680af8e0 0x7f82680b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f826ca60700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8268110610 0x7f8268067040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f826ca60700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82680644f0 con 0x7f82680af8e0 2022-07-21T01:49:28.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f826ca60700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826810f7d0 con 0x7f82680adff0 2022-07-21T01:49:28.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f826ca60700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826810fa70 con 0x7f8268110610 2022-07-21T01:49:28.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8266ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82680af8e0 0x7f82680b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8266ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82680af8e0 0x7f82680b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33968/0 (socket says 172.21.15.140:33968) 2022-07-21T01:49:28.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8266ffd700 1 -- 172.21.15.140:0/2093730336 learned_addr learned my addr 172.21.15.140:0/2093730336 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:28.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f82677fe700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82680adff0 0x7f8268159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8267fff700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8268110610 0x7f8268067040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8266ffd700 1 -- 172.21.15.140:0/2093730336 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8268110610 msgr2=0x7f8268067040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8266ffd700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8268110610 0x7f8268067040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8266ffd700 1 -- 172.21.15.140:0/2093730336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82680adff0 msgr2=0x7f8268159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8266ffd700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82680adff0 0x7f8268159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8266ffd700 1 -- 172.21.15.140:0/2093730336 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258009420 con 0x7f82680af8e0 2022-07-21T01:49:28.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8266ffd700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82680af8e0 0x7f82680b0800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f825800c000 tx=0x7f82580091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8264ff9700 1 -- 172.21.15.140:0/2093730336 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f825800c850 con 0x7f82680af8e0 2022-07-21T01:49:28.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8264ff9700 1 -- 172.21.15.140:0/2093730336 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825800c9b0 con 0x7f82680af8e0 2022-07-21T01:49:28.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f826ca60700 1 -- 172.21.15.140:0/2093730336 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825000b180 con 0x7f82680af8e0 2022-07-21T01:49:28.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8264ff9700 1 -- 172.21.15.140:0/2093730336 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f825800cd10 con 0x7f82680af8e0 2022-07-21T01:49:28.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f826ca60700 1 -- 172.21.15.140:0/2093730336 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826810e900 con 0x7f82680af8e0 2022-07-21T01:49:28.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f826ca60700 1 -- 172.21.15.140:0/2093730336 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826810fc30 con 0x7f82680af8e0 2022-07-21T01:49:28.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8264ff9700 1 -- 172.21.15.140:0/2093730336 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f825801b400 con 0x7f82680af8e0 2022-07-21T01:49:28.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8264ff9700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f824c053a70 0x7f824c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.155+0000 7f8264ff9700 1 -- 172.21.15.140:0/2093730336 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f825806c890 con 0x7f82680af8e0 2022-07-21T01:49:28.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.159+0000 7f82677fe700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f824c053a70 0x7f824c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.159+0000 7f82677fe700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f824c053a70 0x7f824c055f30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f825001a000 tx=0x7f8250014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.163+0000 7f8264ff9700 1 -- 172.21.15.140:0/2093730336 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8258043750 con 0x7f82680af8e0 2022-07-21T01:49:28.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.327+0000 7f826ca60700 1 -- 172.21.15.140:0/2093730336 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f826805c4a0 con 0x7f82680af8e0 2022-07-21T01:49:28.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.327+0000 7f8264ff9700 1 -- 172.21.15.140:0/2093730336 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f8258046d70 con 0x7f82680af8e0 2022-07-21T01:49:28.334 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:28.335 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/838095894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":838095894},{"type":"v1","addr":"172.21.15.140:6835","nonce":838095894}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:49:12.585320+0000","modified":"2022-07-21T01:49:13.589194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1420650762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1420650762},{"type":"v1","addr":"172.21.15.152:6833","nonce":1420650762}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-07-21T01:49:28.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 -- 172.21.15.140:0/2093730336 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f824c053a70 msgr2=0x7f824c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f824c053a70 0x7f824c055f30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f825001a000 tx=0x7f8250014000).stop 2022-07-21T01:49:28.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 -- 172.21.15.140:0/2093730336 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82680af8e0 msgr2=0x7f82680b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82680af8e0 0x7f82680b0800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f825800c000 tx=0x7f82580091a0).stop 2022-07-21T01:49:28.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 -- 172.21.15.140:0/2093730336 shutdown_connections 2022-07-21T01:49:28.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f824c053a70 0x7f824c055f30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8268110610 0x7f8268067040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82680af8e0 0x7f82680b0800 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 --2- 172.21.15.140:0/2093730336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82680adff0 0x7f8268159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 -- 172.21.15.140:0/2093730336 >> 172.21.15.140:0/2093730336 conn(0x7f82680634e0 msgr2=0x7f82680ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:28.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 -- 172.21.15.140:0/2093730336 shutdown_connections 2022-07-21T01:49:28.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.331+0000 7f826ca60700 1 -- 172.21.15.140:0/2093730336 wait complete. 2022-07-21T01:49:28.346 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 10 2022-07-21T01:49:28.348 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_decay_threshold 400 2022-07-21T01:49:28.463 INFO:teuthology.orchestra.run.smithi152.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-07-21T01:49:28.464 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-07-21T01:49:28.465 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:49:28.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.571+0000 7feabe005700 1 Processor -- start 2022-07-21T01:49:28.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.571+0000 7feabe005700 1 -- start start 2022-07-21T01:49:28.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feabe005700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7feab80b1e00 0x7feab814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feabe005700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab805d480 con 0x7feab80acef0 2022-07-21T01:49:28.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feabe005700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab80b21e0 con 0x7feab805c3f0 2022-07-21T01:49:28.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feabe005700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab805cf40 con 0x7feab80b1e00 2022-07-21T01:49:28.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab7fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7feab80acef0 0x7feab8146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48126/0 (socket says 172.21.15.140:48126) 2022-07-21T01:49:28.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab77fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7feab80b1e00 0x7feab814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab7fff700 1 -- 172.21.15.140:0/1663247799 learned_addr learned my addr 172.21.15.140:0/1663247799 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:49:28.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab67fc700 1 -- 172.21.15.140:0/1663247799 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198404430 0 0) 0x7feab805d480 con 0x7feab80acef0 2022-07-21T01:49:28.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab67fc700 1 -- 172.21.15.140:0/1663247799 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea98003260 con 0x7feab80acef0 2022-07-21T01:49:28.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab67fc700 1 -- 172.21.15.140:0/1663247799 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 304046070 0 0) 0x7fea98003260 con 0x7feab80acef0 2022-07-21T01:49:28.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab77fe700 1 -- 172.21.15.140:0/1663247799 >> v1:172.21.15.152:6789/0 conn(0x7feab805c3f0 legacy=0x7feab80ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab77fe700 1 -- 172.21.15.140:0/1663247799 >> v1:172.21.15.140:6789/0 conn(0x7feab80acef0 legacy=0x7feab8146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab77fe700 1 -- 172.21.15.140:0/1663247799 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab805d1e0 con 0x7feab80b1e00 2022-07-21T01:49:28.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab77fe700 1 --2- 172.21.15.140:0/1663247799 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feab80b1e00 0x7feab814a6d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7feab0001800 tx=0x7feaa0006df0).ready entity=mon.0 client_cookie=ec34be5f44ad68e2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab67fc700 1 -- 172.21.15.140:0/1663247799 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feab0001660 con 0x7feab80b1e00 2022-07-21T01:49:28.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab67fc700 1 -- 172.21.15.140:0/1663247799 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa0000e90 con 0x7feab80b1e00 2022-07-21T01:49:28.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feab67fc700 1 -- 172.21.15.140:0/1663247799 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaa000d4f0 con 0x7feab80b1e00 2022-07-21T01:49:28.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feabe005700 1 -- 172.21.15.140:0/1663247799 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feab80b1e00 msgr2=0x7feab814a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feabe005700 1 --2- 172.21.15.140:0/1663247799 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feab80b1e00 0x7feab814a6d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7feab0001800 tx=0x7feaa0006df0).stop 2022-07-21T01:49:28.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feabe005700 1 -- 172.21.15.140:0/1663247799 shutdown_connections 2022-07-21T01:49:28.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feabe005700 1 --2- 172.21.15.140:0/1663247799 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feab80b1e00 0x7feab814a6d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feabe005700 1 -- 172.21.15.140:0/1663247799 >> 172.21.15.140:0/1663247799 conn(0x7feab80611c0 msgr2=0x7feab8062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:28.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feabe005700 1 -- 172.21.15.140:0/1663247799 shutdown_connections 2022-07-21T01:49:28.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.575+0000 7feabe005700 1 -- 172.21.15.140:0/1663247799 wait complete. 2022-07-21T01:49:28.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feabe005700 1 Processor -- start 2022-07-21T01:49:28.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feabe005700 1 -- start start 2022-07-21T01:49:28.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feabe005700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feab80acef0 0x7feab8065f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feabe005700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feab80b1e00 0x7feab8063f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feabe005700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7feab8064ef0 0x7feab8156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feabe005700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab8065830 con 0x7feab80b1e00 2022-07-21T01:49:28.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feabe005700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab8065470 con 0x7feab80acef0 2022-07-21T01:49:28.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feabe005700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab8065a90 con 0x7feab8064ef0 2022-07-21T01:49:28.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feab80b1e00 0x7feab8063f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feab80b1e00 0x7feab8063f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33984/0 (socket says 172.21.15.140:33984) 2022-07-21T01:49:28.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab6ffd700 1 -- 172.21.15.140:0/1020270252 learned_addr learned my addr 172.21.15.140:0/1020270252 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:28.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab77fe700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feab80acef0 0x7feab8065f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab7fff700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7feab8064ef0 0x7feab8156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab6ffd700 1 -- 172.21.15.140:0/1020270252 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7feab8064ef0 msgr2=0x7feab8156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab6ffd700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7feab8064ef0 0x7feab8156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab6ffd700 1 -- 172.21.15.140:0/1020270252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feab80acef0 msgr2=0x7feab8065f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab6ffd700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feab80acef0 0x7feab8065f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab6ffd700 1 -- 172.21.15.140:0/1020270252 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa000b180 con 0x7feab80b1e00 2022-07-21T01:49:28.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab6ffd700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feab80b1e00 0x7feab8063f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7feab805f3e0 tx=0x7feaa80014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab4ff9700 1 -- 172.21.15.140:0/1020270252 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaa800f470 con 0x7feab80b1e00 2022-07-21T01:49:28.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab4ff9700 1 -- 172.21.15.140:0/1020270252 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa800f5d0 con 0x7feab80b1e00 2022-07-21T01:49:28.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feabe005700 1 -- 172.21.15.140:0/1020270252 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab8064460 con 0x7feab80b1e00 2022-07-21T01:49:28.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feab4ff9700 1 -- 172.21.15.140:0/1020270252 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaa800f930 con 0x7feab80b1e00 2022-07-21T01:49:28.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.579+0000 7feabe005700 1 -- 172.21.15.140:0/1020270252 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab8067a60 con 0x7feab80b1e00 2022-07-21T01:49:28.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.583+0000 7feabe005700 1 -- 172.21.15.140:0/1020270252 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab80b04b0 con 0x7feab80b1e00 2022-07-21T01:49:28.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.583+0000 7feab4ff9700 1 -- 172.21.15.140:0/1020270252 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7feaa8019030 con 0x7feab80b1e00 2022-07-21T01:49:28.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.583+0000 7feab4ff9700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea9c053aa0 0x7fea9c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.583+0000 7feab4ff9700 1 -- 172.21.15.140:0/1020270252 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feaa806b410 con 0x7feab80b1e00 2022-07-21T01:49:28.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.583+0000 7feab77fe700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea9c053aa0 0x7fea9c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.583+0000 7feab77fe700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea9c053aa0 0x7fea9c055f60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7feaa001a000 tx=0x7feaa0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.587+0000 7feab4ff9700 1 -- 172.21.15.140:0/1020270252 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feaa8042290 con 0x7feab80b1e00 2022-07-21T01:49:28.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.747+0000 7feabe005700 1 -- 172.21.15.140:0/1020270252 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feab8067a60 con 0x7feab80b1e00 2022-07-21T01:49:28.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feab4ff9700 1 -- 172.21.15.140:0/1020270252 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2139 (secure 0 0 0) 0x7feab8067a60 con 0x7feab80b1e00 2022-07-21T01:49:28.753 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:49:28.753 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/838095894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":838095894},{"type":"v1","addr":"172.21.15.140:6835","nonce":838095894}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:49:12.585320+0000","modified":"2022-07-21T01:49:13.589194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1420650762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1420650762},{"type":"v1","addr":"172.21.15.152:6833","nonce":1420650762}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-07-21T01:49:28.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 -- 172.21.15.140:0/1020270252 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea9c053aa0 msgr2=0x7fea9c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea9c053aa0 0x7fea9c055f60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7feaa001a000 tx=0x7feaa0014000).stop 2022-07-21T01:49:28.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 -- 172.21.15.140:0/1020270252 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feab80b1e00 msgr2=0x7feab8063f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feab80b1e00 0x7feab8063f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7feab805f3e0 tx=0x7feaa80014a0).stop 2022-07-21T01:49:28.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 -- 172.21.15.140:0/1020270252 shutdown_connections 2022-07-21T01:49:28.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea9c053aa0 0x7fea9c055f60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7feab8064ef0 0x7feab8156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feab80b1e00 0x7feab8063f70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 --2- 172.21.15.140:0/1020270252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feab80acef0 0x7feab8065f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 -- 172.21.15.140:0/1020270252 >> 172.21.15.140:0/1020270252 conn(0x7feab805c3f0 msgr2=0x7feab80afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:28.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 -- 172.21.15.140:0/1020270252 shutdown_connections 2022-07-21T01:49:28.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.751+0000 7feabe005700 1 -- 172.21.15.140:0/1020270252 wait complete. 2022-07-21T01:49:28.758 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 10 2022-07-21T01:49:28.765 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b cache drop 2022-07-21T01:49:28.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.867+0000 7f59d6576700 1 Processor -- start 2022-07-21T01:49:28.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 -- start start 2022-07-21T01:49:28.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f59d00b57f0 0x7f59d00af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f59d00aecf0 0x7f59d0143d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d0065040 con 0x7f59d005c3b0 2022-07-21T01:49:28.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d005bf10 con 0x7f59d00b57f0 2022-07-21T01:49:28.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d0064d70 con 0x7f59d00aecf0 2022-07-21T01:49:28.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d5574700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f59d00b57f0 0x7f59d00af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d5574700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f59d00b57f0 0x7f59d00af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33990/0 (socket says 172.21.15.140:33990) 2022-07-21T01:49:28.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d5574700 1 -- 172.21.15.140:0/3355984911 learned_addr learned my addr 172.21.15.140:0/3355984911 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:28.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d5d75700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f59d005c3b0 0x7f59d013a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48138/0 (socket says 172.21.15.140:48138) 2022-07-21T01:49:28.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d4d73700 1 --2- 172.21.15.140:0/3355984911 >> v2:172.21.15.152:3300/0 conn(0x7f59d00aecf0 0x7f59d0143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d5574700 1 -- 172.21.15.140:0/3355984911 >> v2:172.21.15.152:3300/0 conn(0x7f59d00aecf0 msgr2=0x7f59d0143d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:28.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59c7fff700 1 -- 172.21.15.140:0/3355984911 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784542020 0 0) 0x7f59d0065040 con 0x7f59d005c3b0 2022-07-21T01:49:28.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d5574700 1 --2- 172.21.15.140:0/3355984911 >> v2:172.21.15.152:3300/0 conn(0x7f59d00aecf0 0x7f59d0143d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d5574700 1 -- 172.21.15.140:0/3355984911 >> v1:172.21.15.140:6789/0 conn(0x7f59d005c3b0 legacy=0x7f59d013a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d5574700 1 -- 172.21.15.140:0/3355984911 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d0144270 con 0x7f59d00b57f0 2022-07-21T01:49:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d5574700 1 --2- 172.21.15.140:0/3355984911 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59d00b57f0 0x7f59d00af270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f59cc002f70 tx=0x7f59cc006e50).ready entity=mon.0 client_cookie=e0714ac75a072ae1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59c7fff700 1 -- 172.21.15.140:0/3355984911 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59cc00d610 con 0x7f59d00b57f0 2022-07-21T01:49:28.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59c7fff700 1 -- 172.21.15.140:0/3355984911 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc000e90 con 0x7f59d00b57f0 2022-07-21T01:49:28.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59c7fff700 1 -- 172.21.15.140:0/3355984911 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59cc00d960 con 0x7f59d00b57f0 2022-07-21T01:49:28.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 -- 172.21.15.140:0/3355984911 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59d00b57f0 msgr2=0x7f59d00af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 --2- 172.21.15.140:0/3355984911 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59d00b57f0 0x7f59d00af270 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f59cc002f70 tx=0x7f59cc006e50).stop 2022-07-21T01:49:28.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 -- 172.21.15.140:0/3355984911 shutdown_connections 2022-07-21T01:49:28.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 --2- 172.21.15.140:0/3355984911 >> v2:172.21.15.152:3300/0 conn(0x7f59d00aecf0 0x7f59d0143d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 --2- 172.21.15.140:0/3355984911 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59d00b57f0 0x7f59d00af270 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 -- 172.21.15.140:0/3355984911 >> 172.21.15.140:0/3355984911 conn(0x7f59d0063480 msgr2=0x7f59d00b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:28.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 -- 172.21.15.140:0/3355984911 shutdown_connections 2022-07-21T01:49:28.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.871+0000 7f59d6576700 1 -- 172.21.15.140:0/3355984911 wait complete. 2022-07-21T01:49:28.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d6576700 1 Processor -- start 2022-07-21T01:49:28.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d6576700 1 -- start start 2022-07-21T01:49:28.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d6576700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59d005c3b0 0x7f59d01145d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d6576700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59d00aecf0 0x7f59d014fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d6576700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59d00b57f0 0x7f59d013e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d6576700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d0113130 con 0x7f59d00aecf0 2022-07-21T01:49:28.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d6576700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d01532a0 con 0x7f59d005c3b0 2022-07-21T01:49:28.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d6576700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d0110900 con 0x7f59d00b57f0 2022-07-21T01:49:28.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d4d73700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59d00aecf0 0x7f59d014fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d4d73700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59d00aecf0 0x7f59d014fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:33998/0 (socket says 172.21.15.140:33998) 2022-07-21T01:49:28.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d4d73700 1 -- 172.21.15.140:0/862798135 learned_addr learned my addr 172.21.15.140:0/862798135 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:28.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d5574700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59d005c3b0 0x7f59d01145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d5d75700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59d00b57f0 0x7f59d013e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d4d73700 1 -- 172.21.15.140:0/862798135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59d00b57f0 msgr2=0x7f59d013e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d4d73700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59d00b57f0 0x7f59d013e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d4d73700 1 -- 172.21.15.140:0/862798135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59d005c3b0 msgr2=0x7f59d01145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d4d73700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59d005c3b0 0x7f59d01145d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d4d73700 1 -- 172.21.15.140:0/862798135 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc00b1e0 con 0x7f59d00aecf0 2022-07-21T01:49:28.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d4d73700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59d00aecf0 0x7f59d014fc90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f59c000c000 tx=0x7f59c0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59c67fc700 1 -- 172.21.15.140:0/862798135 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59c0010780 con 0x7f59d00aecf0 2022-07-21T01:49:28.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59c67fc700 1 -- 172.21.15.140:0/862798135 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c00108e0 con 0x7f59d00aecf0 2022-07-21T01:49:28.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59d6576700 1 -- 172.21.15.140:0/862798135 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d0110ba0 con 0x7f59d00aecf0 2022-07-21T01:49:28.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.875+0000 7f59c67fc700 1 -- 172.21.15.140:0/862798135 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59c0010c40 con 0x7f59d00aecf0 2022-07-21T01:49:28.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.879+0000 7f59d6576700 1 -- 172.21.15.140:0/862798135 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d0110260 con 0x7f59d00aecf0 2022-07-21T01:49:28.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.879+0000 7f59c67fc700 1 -- 172.21.15.140:0/862798135 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f59c001c460 con 0x7f59d00aecf0 2022-07-21T01:49:28.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.879+0000 7f59c67fc700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59b8053a70 0x7f59b8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.879+0000 7f59c67fc700 1 -- 172.21.15.140:0/862798135 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f59c006c420 con 0x7f59d00aecf0 2022-07-21T01:49:28.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.879+0000 7f59d5574700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59b8053a70 0x7f59b8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.879+0000 7f59d5574700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59b8053a70 0x7f59b8055f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f59cc018000 tx=0x7f59cc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.879+0000 7f59e1a2f740 1 Processor -- start 2022-07-21T01:49:28.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.879+0000 7f59e1a2f740 1 -- start start 2022-07-21T01:49:28.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x273ac10 0x2734c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2735180 0x2735600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x26fdf80 0x26fe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x2672010 con 0x273ac10 2022-07-21T01:49:28.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x2652250 con 0x2735180 2022-07-21T01:49:28.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x2700400 con 0x26fdf80 2022-07-21T01:49:28.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d5574700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x273ac10 0x2734c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d5574700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x273ac10 0x2734c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34006/0 (socket says 172.21.15.140:34006) 2022-07-21T01:49:28.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d5574700 1 -- 172.21.15.140:0/3727015355 learned_addr learned my addr 172.21.15.140:0/3727015355 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:28.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d4d73700 1 --2- 172.21.15.140:0/3727015355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2735180 0x2735600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d5d75700 1 --2- 172.21.15.140:0/3727015355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x26fdf80 0x26fe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d5574700 1 -- 172.21.15.140:0/3727015355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x26fdf80 msgr2=0x26fe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:28.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d5574700 1 --2- 172.21.15.140:0/3727015355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x26fdf80 0x26fe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d5574700 1 -- 172.21.15.140:0/3727015355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2735180 msgr2=0x2735600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:28.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d5574700 1 --2- 172.21.15.140:0/3727015355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2735180 0x2735600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d5574700 1 -- 172.21.15.140:0/3727015355 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x26aa030 con 0x273ac10 2022-07-21T01:49:28.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d5d75700 1 --2- 172.21.15.140:0/3727015355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x26fdf80 0x26fe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:28.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d5574700 1 --2- 172.21.15.140:0/3727015355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x273ac10 0x2734c90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x2654ab0 tx=0x7f59cc01e000).ready entity=mon.0 client_cookie=6224620cfcb2abac server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d6576700 1 -- 172.21.15.140:0/3727015355 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59cc016040 con 0x273ac10 2022-07-21T01:49:28.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d6576700 1 -- 172.21.15.140:0/3727015355 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc00d8b0 con 0x273ac10 2022-07-21T01:49:28.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59d6576700 1 -- 172.21.15.140:0/3727015355 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59cc01f8d0 con 0x273ac10 2022-07-21T01:49:28.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 -- 172.21.15.140:0/3727015355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x273ac10 msgr2=0x2734c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/3727015355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x273ac10 0x2734c90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x2654ab0 tx=0x7f59cc01e000).stop 2022-07-21T01:49:28.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 -- 172.21.15.140:0/3727015355 shutdown_connections 2022-07-21T01:49:28.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/3727015355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x26fdf80 0x26fe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/3727015355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2735180 0x2735600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/3727015355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x273ac10 0x2734c90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 -- 172.21.15.140:0/3727015355 >> 172.21.15.140:0/3727015355 conn(0x26ff2e0 msgr2=0x273a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:28.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 -- 172.21.15.140:0/3727015355 shutdown_connections 2022-07-21T01:49:28.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 -- 172.21.15.140:0/3727015355 wait complete. 2022-07-21T01:49:28.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.883+0000 7f59e1a2f740 1 -- start start 2022-07-21T01:49:28.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 1 Processor -- start 2022-07-21T01:49:28.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2749290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x2702fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x273ac10 0x27034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x2749780 con 0x26fdf80 2022-07-21T01:49:28.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x27488b0 con 0x273ac10 2022-07-21T01:49:28.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x2748b50 con 0x2735180 2022-07-21T01:49:28.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 10 client.0 ll_register_callbacks cb 0x26b9610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-07-21T01:49:28.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d5574700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2749290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d5574700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2749290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34010/0 (socket says 172.21.15.140:34010) 2022-07-21T01:49:28.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d5574700 1 -- 172.21.15.140:0/1828311510 learned_addr learned my addr 172.21.15.140:0/1828311510 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:28.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d5d75700 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x273ac10 0x27034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d4d73700 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x2702fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d5574700 1 -- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 msgr2=0x2702fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:28.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d5574700 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x2702fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d5574700 1 -- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x273ac10 msgr2=0x27034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:28.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d5574700 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x273ac10 0x27034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d5574700 1 -- 172.21.15.140:0/1828311510 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d0144270 con 0x26fdf80 2022-07-21T01:49:28.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d5574700 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2749290 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x2654ab0 tx=0x7f59cc027000).ready entity=mon.0 client_cookie=fbc805fba2e5833f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59abfff700 10 client.0 ms_handle_connect on v2:172.21.15.140:3300/0 2022-07-21T01:49:28.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59abfff700 1 -- 172.21.15.140:0/1828311510 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59cc031070 con 0x26fdf80 2022-07-21T01:49:28.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59abfff700 1 -- 172.21.15.140:0/1828311510 <== mon.0 v2:172.21.15.140:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f59cc006aa0 con 0x26fdf80 2022-07-21T01:49:28.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 1 -- 172.21.15.140:0/1828311510 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2667fc0 con 0x26fdf80 2022-07-21T01:49:28.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59abfff700 1 -- 172.21.15.140:0/1828311510 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59cc016040 con 0x26fdf80 2022-07-21T01:49:28.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 10 client.5133 fetch_fsmap learned FSMap version 10 2022-07-21T01:49:28.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 1 -- 172.21.15.140:0/1828311510 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) v3 -- 0x26449b0 con 0x26fdf80 2022-07-21T01:49:28.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59abfff700 1 -- 172.21.15.140:0/1828311510 <== mon.0 v2:172.21.15.140:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2667fc0 con 0x26fdf80 2022-07-21T01:49:28.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59abfff700 1 -- 172.21.15.140:0/1828311510 <== mon.0 v2:172.21.15.140:3300/0 5 ==== fsmap(e 10) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f59cc012bc0 con 0x26fdf80 2022-07-21T01:49:28.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 10 client.5133 fetch_fsmap finished waiting for FSMap version 10 2022-07-21T01:49:28.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 10 client.5133 resolve_mds: resolved ID 'b' to GID 4894 2022-07-21T01:49:28.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 20 client.5133 populate_metadata read hostname 'smithi140' 2022-07-21T01:49:28.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:6832/1420650762,v1:172.21.15.152:6833/1420650762] conn(0x2704800 0x2745200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 4 client.5133 mds_command: new command op to 4894 tid=0[{"prefix": "get_command_descriptions"}] 2022-07-21T01:49:28.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59e1a2f740 1 -- 172.21.15.140:0/1828311510 --> [v2:172.21.15.152:6832/1420650762,v1:172.21.15.152:6833/1420650762] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2660040 con 0x2704800 2022-07-21T01:49:28.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.887+0000 7f59d4d73700 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:6832/1420650762,v1:172.21.15.152:6833/1420650762] conn(0x2704800 0x2745200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59d4d73700 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:6832/1420650762,v1:172.21.15.152:6833/1420650762] conn(0x2704800 0x2745200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=98341b7ea927299 server_cookie=b1e5359eb6b45efb in_seq=0 out_seq=0 2022-07-21T01:49:28.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59abfff700 10 client.5133 ms_handle_connect on v2:172.21.15.152:6832/1420650762 2022-07-21T01:49:28.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59abfff700 1 -- 172.21.15.140:0/1828311510 <== mds.0 v2:172.21.15.152:6832/1420650762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2660040 con 0x2704800 2022-07-21T01:49:28.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59abfff700 10 client.5133 handle_command_reply: tid=0 2022-07-21T01:49:28.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 1 client.5133 shutdown 2022-07-21T01:49:28.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 1 -- 172.21.15.140:0/1828311510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 msgr2=0x2749290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2749290 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x2654ab0 tx=0x7f59cc027000).stop 2022-07-21T01:49:28.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 1 -- 172.21.15.140:0/1828311510 shutdown_connections 2022-07-21T01:49:28.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:6832/1420650762,v1:172.21.15.152:6833/1420650762] conn(0x2704800 0x2745200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x273ac10 0x27034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59abfff700 0 client.5133 ms_handle_reset on v2:172.21.15.152:6832/1420650762 2022-07-21T01:49:28.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x2702fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/1828311510 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2749290 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 1 -- 172.21.15.140:0/1828311510 >> 172.21.15.140:0/1828311510 conn(0x26ff2e0 msgr2=0x273a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:28.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 1 -- 172.21.15.140:0/1828311510 shutdown_connections 2022-07-21T01:49:28.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 1 -- 172.21.15.140:0/1828311510 wait complete. 2022-07-21T01:49:28.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.891+0000 7f59e1a2f740 20 client.5133 trim_cache size 0 max 16384 2022-07-21T01:49:28.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 Processor -- start 2022-07-21T01:49:28.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 -- start start 2022-07-21T01:49:28.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2702c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2704800 0x2736bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x27370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x26be4c0 con 0x26fdf80 2022-07-21T01:49:28.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x26d04e0 con 0x2704800 2022-07-21T01:49:28.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x26786b0 con 0x2735180 2022-07-21T01:49:28.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d5574700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2702c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d5574700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2702c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34018/0 (socket says 172.21.15.140:34018) 2022-07-21T01:49:28.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d5574700 1 -- 172.21.15.140:0/65250201 learned_addr learned my addr 172.21.15.140:0/65250201 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:28.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d5d75700 1 --2- 172.21.15.140:0/65250201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x27370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d4d73700 1 --2- 172.21.15.140:0/65250201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2704800 0x2736bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d5574700 1 -- 172.21.15.140:0/65250201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 msgr2=0x27370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:28.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d5574700 1 --2- 172.21.15.140:0/65250201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x27370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d5574700 1 -- 172.21.15.140:0/65250201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2704800 msgr2=0x2736bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:28.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d5574700 1 --2- 172.21.15.140:0/65250201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2704800 0x2736bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d5574700 1 -- 172.21.15.140:0/65250201 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x26615b0 con 0x26fdf80 2022-07-21T01:49:28.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d5d75700 1 --2- 172.21.15.140:0/65250201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x27370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:28.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d4d73700 1 --2- 172.21.15.140:0/65250201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2704800 0x2736bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:49:28.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59d5574700 1 --2- 172.21.15.140:0/65250201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2702c00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x263a2d0 tx=0x7f59cc031000).ready entity=mon.0 client_cookie=ce8c353073c295e0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59b27fc700 1 -- 172.21.15.140:0/65250201 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59cc00d610 con 0x26fdf80 2022-07-21T01:49:28.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59b27fc700 1 -- 172.21.15.140:0/65250201 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc006aa0 con 0x26fdf80 2022-07-21T01:49:28.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59b27fc700 1 -- 172.21.15.140:0/65250201 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59cc012670 con 0x26fdf80 2022-07-21T01:49:28.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 -- 172.21.15.140:0/65250201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 msgr2=0x2702c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:49:28.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/65250201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2702c00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x263a2d0 tx=0x7f59cc031000).stop 2022-07-21T01:49:28.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 -- 172.21.15.140:0/65250201 shutdown_connections 2022-07-21T01:49:28.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/65250201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x27370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/65250201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2704800 0x2736bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/65250201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x2702c00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.907+0000 7f59e1a2f740 1 -- 172.21.15.140:0/65250201 >> 172.21.15.140:0/65250201 conn(0x26ff2e0 msgr2=0x26fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:49:28.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 -- 172.21.15.140:0/65250201 shutdown_connections 2022-07-21T01:49:28.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 -- 172.21.15.140:0/65250201 wait complete. 2022-07-21T01:49:28.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 -- start start 2022-07-21T01:49:28.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 Processor -- start 2022-07-21T01:49:28.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x279d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2704800 0x2700a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x273e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x2733070 con 0x26fdf80 2022-07-21T01:49:28.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x2732dd0 con 0x2704800 2022-07-21T01:49:28.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x27a0a50 con 0x2735180 2022-07-21T01:49:28.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 10 client.0 ll_register_callbacks cb 0x2701c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-07-21T01:49:28.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d5574700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x279d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d5574700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x279d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34026/0 (socket says 172.21.15.140:34026) 2022-07-21T01:49:28.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d4d73700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2704800 0x2700a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d5574700 1 -- 172.21.15.140:0/2994215744 learned_addr learned my addr 172.21.15.140:0/2994215744 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:49:28.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d5d75700 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x273e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d5574700 1 -- 172.21.15.140:0/2994215744 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 msgr2=0x273e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:28.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d5574700 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x273e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d5574700 1 -- 172.21.15.140:0/2994215744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2704800 msgr2=0x2700a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:49:28.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d5574700 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2704800 0x2700a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:49:28.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d5574700 1 -- 172.21.15.140:0/2994215744 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x26449b0 con 0x26fdf80 2022-07-21T01:49:28.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d5574700 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x279d4f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x263a2d0 tx=0x7f59cc02d040).ready entity=mon.0 client_cookie=c953dc059bd7c917 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:49:28.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.140:3300/0 2022-07-21T01:49:28.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59b17fa700 1 -- 172.21.15.140:0/2994215744 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59cc048070 con 0x26fdf80 2022-07-21T01:49:28.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59b17fa700 1 -- 172.21.15.140:0/2994215744 <== mon.0 v2:172.21.15.140:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f59cc036600 con 0x26fdf80 2022-07-21T01:49:28.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 -- 172.21.15.140:0/2994215744 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59cc00d8b0 con 0x26fdf80 2022-07-21T01:49:28.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59b17fa700 1 -- 172.21.15.140:0/2994215744 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59cc031070 con 0x26fdf80 2022-07-21T01:49:28.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 10 client.5139 fetch_fsmap learned FSMap version 10 2022-07-21T01:49:28.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 -- 172.21.15.140:0/2994215744 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) v3 -- 0x2666b90 con 0x26fdf80 2022-07-21T01:49:28.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59b17fa700 1 -- 172.21.15.140:0/2994215744 <== mon.0 v2:172.21.15.140:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc00d8b0 con 0x26fdf80 2022-07-21T01:49:28.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59b17fa700 1 -- 172.21.15.140:0/2994215744 <== mon.0 v2:172.21.15.140:3300/0 5 ==== fsmap(e 10) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f59cc042480 con 0x26fdf80 2022-07-21T01:49:28.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 10 client.5139 fetch_fsmap finished waiting for FSMap version 10 2022-07-21T01:49:28.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 10 client.5139 resolve_mds: resolved ID 'b' to GID 4894 2022-07-21T01:49:28.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 20 client.5139 populate_metadata read hostname 'smithi140' 2022-07-21T01:49:28.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.152:6832/1420650762,v1:172.21.15.152:6833/1420650762] conn(0x273ac10 0x273e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:49:28.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 4 client.5139 mds_command: new command op to 4894 tid=0[{"prefix": "cache drop"}] 2022-07-21T01:49:28.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59e1a2f740 1 -- 172.21.15.140:0/2994215744 --> [v2:172.21.15.152:6832/1420650762,v1:172.21.15.152:6833/1420650762] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x2555590 con 0x273ac10 2022-07-21T01:49:28.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.911+0000 7f59d4d73700 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.152:6832/1420650762,v1:172.21.15.152:6833/1420650762] conn(0x273ac10 0x273e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:49:28.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.915+0000 7f59d4d73700 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.152:6832/1420650762,v1:172.21.15.152:6833/1420650762] conn(0x273ac10 0x273e830 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=2eeaadeca6d6ff49 server_cookie=60b05c17321c0f12 in_seq=0 out_seq=0 2022-07-21T01:49:28.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:49:28.915+0000 7f59b17fa700 10 client.5139 ms_handle_connect on v2:172.21.15.152:6832/1420650762 2022-07-21T01:49:30.870 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:49:30.875 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:49:30.890 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:49:30.950 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:49:30.962 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:50:01.018 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:50:01.024 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:50:01.028 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:50:01.032 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:50:01.036 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:50:13.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.275+0000 7f59b17fa700 1 -- 172.21.15.140:0/2994215744 <== mds.0 v2:172.21.15.152:6832/1420650762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+380 (crc 0 0 0) 0x2555590 con 0x273ac10 2022-07-21T01:50:13.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.275+0000 7f59b17fa700 10 client.5139 handle_command_reply: tid=0 2022-07-21T01:50:13.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.275+0000 7f59e1a2f740 1 client.5139 shutdown 2022-07-21T01:50:13.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.275+0000 7f59e1a2f740 1 -- 172.21.15.140:0/2994215744 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 msgr2=0x279d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:13.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.275+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x279d4f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x263a2d0 tx=0x7f59cc02d040).stop 2022-07-21T01:50:13.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59e1a2f740 1 -- 172.21.15.140:0/2994215744 shutdown_connections 2022-07-21T01:50:13.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.152:6832/1420650762,v1:172.21.15.152:6833/1420650762] conn(0x273ac10 0x273e830 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:13.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x2735180 0x273e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:13.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59b17fa700 0 client.5139 ms_handle_reset on v2:172.21.15.152:6832/1420650762 2022-07-21T01:50:13.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x2704800 0x2700a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:13.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59e1a2f740 1 --2- 172.21.15.140:0/2994215744 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x26fdf80 0x279d4f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:13.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59e1a2f740 1 -- 172.21.15.140:0/2994215744 >> 172.21.15.140:0/2994215744 conn(0x26ff2e0 msgr2=0x2702c00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:13.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59e1a2f740 1 -- 172.21.15.140:0/2994215744 shutdown_connections 2022-07-21T01:50:13.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59e1a2f740 1 -- 172.21.15.140:0/2994215744 wait complete. 2022-07-21T01:50:13.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59e1a2f740 20 client.5139 trim_cache size 0 max 16384 2022-07-21T01:50:13.288 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:50:13.288 INFO:teuthology.orchestra.run.smithi140.stdout: "client_recall": { 2022-07-21T01:50:13.288 INFO:teuthology.orchestra.run.smithi140.stdout: "return_code": 0, 2022-07-21T01:50:13.288 INFO:teuthology.orchestra.run.smithi140.stdout: "message": "(0) Success", 2022-07-21T01:50:13.289 INFO:teuthology.orchestra.run.smithi140.stdout: "recalled": 920 2022-07-21T01:50:13.289 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:13.289 INFO:teuthology.orchestra.run.smithi140.stdout: "flush_journal": { 2022-07-21T01:50:13.289 INFO:teuthology.orchestra.run.smithi140.stdout: "return_code": 0, 2022-07-21T01:50:13.290 INFO:teuthology.orchestra.run.smithi140.stdout: "message": "" 2022-07-21T01:50:13.290 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:13.290 INFO:teuthology.orchestra.run.smithi140.stdout: "trim_cache": { 2022-07-21T01:50:13.290 INFO:teuthology.orchestra.run.smithi140.stdout: "trimmed": 900 2022-07-21T01:50:13.290 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:13.291 INFO:teuthology.orchestra.run.smithi140.stdout: "cache": { 2022-07-21T01:50:13.291 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": { 2022-07-21T01:50:13.291 INFO:teuthology.orchestra.run.smithi140.stdout: "items": 2541, 2022-07-21T01:50:13.291 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes": 432048 2022-07-21T01:50:13.292 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:13.292 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:13.292 INFO:teuthology.orchestra.run.smithi140.stdout: "duration": 44.361957894 2022-07-21T01:50:13.292 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:50:13.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 -- 172.21.15.140:0/862798135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59b8053a70 msgr2=0x7f59b8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:13.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59b8053a70 0x7f59b8055f30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f59cc018000 tx=0x7f59cc014000).stop 2022-07-21T01:50:13.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 -- 172.21.15.140:0/862798135 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59d00aecf0 msgr2=0x7f59d014fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:13.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59d00aecf0 0x7f59d014fc90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f59c000c000 tx=0x7f59c0006df0).stop 2022-07-21T01:50:13.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 -- 172.21.15.140:0/862798135 shutdown_connections 2022-07-21T01:50:13.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f59b8053a70 0x7f59b8055f30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:13.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59d00b57f0 0x7f59d013e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:13.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59d00aecf0 0x7f59d014fc90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:13.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 --2- 172.21.15.140:0/862798135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59d005c3b0 0x7f59d01145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:13.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 -- 172.21.15.140:0/862798135 >> 172.21.15.140:0/862798135 conn(0x7f59d0063480 msgr2=0x7f59d00b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:13.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 -- 172.21.15.140:0/862798135 shutdown_connections 2022-07-21T01:50:13.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.279+0000 7f59d6576700 1 -- 172.21.15.140:0/862798135 wait complete. 2022-07-21T01:50:13.307 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:50:13.308 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:50:13.329 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:50:13.350 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:50:13.376 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:50:13.399 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:50:13.423 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:50:13.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:13.425 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:13.427 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:13.436 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:50:13.438 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:50:13.439 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:50:13.440 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:50:13.441 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:13.458 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22249]: fuse finished with error 0 and tester_r 0 2022-07-21T01:50:13.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:13.468 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:13.468 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:13.519 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:50:13.519 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:50:13.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:50:13.606 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:13.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:13.617 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:13.618 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:50:13.625 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:50:13.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:50:13.628 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:50:13.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:50:13.631 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:50:13.679 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:50:13.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:50:13.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:13.681 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:13.681 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:50:13.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:50:13.694 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:50:13.694 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:50:13.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:50:13.696 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:50:13.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:50:13.744 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:50:13.744 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:50:13.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.859+0000 7f4c2afcc700 1 Processor -- start 2022-07-21T01:50:13.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.859+0000 7f4c2afcc700 1 -- start start 2022-07-21T01:50:13.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.859+0000 7f4c2afcc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b0460 0x7f4c240af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:13.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.859+0000 7f4c2afcc700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c240b20f0 con 0x7f4c240b13e0 2022-07-21T01:50:13.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.859+0000 7f4c2afcc700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c240b65b0 con 0x7f4c240b61d0 2022-07-21T01:50:13.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.859+0000 7f4c2afcc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2413efd0 con 0x7f4c240b0460 2022-07-21T01:50:13.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.859+0000 7f4c297c9700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4c240b13e0 0x7f4c24067260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48178/0 (socket says 172.21.15.140:48178) 2022-07-21T01:50:13.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.859+0000 7f4c297c9700 1 -- 172.21.15.140:0/958847451 learned_addr learned my addr 172.21.15.140:0/958847451 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:13.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.859+0000 7f4c2a7cb700 1 --2- 172.21.15.140:0/958847451 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b0460 0x7f4c240af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:13.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c28fc8700 1 -- 172.21.15.140:0/958847451 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814520339 0 0) 0x7f4c240b20f0 con 0x7f4c240b13e0 2022-07-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c28fc8700 1 -- 172.21.15.140:0/958847451 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c0c003260 con 0x7f4c240b13e0 2022-07-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c28fc8700 1 -- 172.21.15.140:0/958847451 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190102869 0 0) 0x7f4c240b65b0 con 0x7f4c240b61d0 2022-07-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c2a7cb700 1 -- 172.21.15.140:0/958847451 >> v1:172.21.15.152:6789/0 conn(0x7f4c240b61d0 legacy=0x7f4c240aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:13.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c2a7cb700 1 -- 172.21.15.140:0/958847451 >> v1:172.21.15.140:6789/0 conn(0x7f4c240b13e0 legacy=0x7f4c24067260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:13.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c2a7cb700 1 -- 172.21.15.140:0/958847451 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240b1ad0 con 0x7f4c240b0460 2022-07-21T01:50:13.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c2a7cb700 1 --2- 172.21.15.140:0/958847451 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b0460 0x7f4c240af120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f4c20001800 tx=0x7f4c1c00a370).ready entity=mon.2 client_cookie=2e9f4529fa6f2f43 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:13.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c28fc8700 1 -- 172.21.15.140:0/958847451 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c20001660 con 0x7f4c240b0460 2022-07-21T01:50:13.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c28fc8700 1 -- 172.21.15.140:0/958847451 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c00c960 con 0x7f4c240b0460 2022-07-21T01:50:13.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c28fc8700 1 -- 172.21.15.140:0/958847451 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c1c00ccc0 con 0x7f4c240b0460 2022-07-21T01:50:13.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c2afcc700 1 -- 172.21.15.140:0/958847451 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b0460 msgr2=0x7f4c240af120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:13.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c2afcc700 1 --2- 172.21.15.140:0/958847451 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b0460 0x7f4c240af120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f4c20001800 tx=0x7f4c1c00a370).stop 2022-07-21T01:50:13.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c2afcc700 1 -- 172.21.15.140:0/958847451 shutdown_connections 2022-07-21T01:50:13.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c2afcc700 1 --2- 172.21.15.140:0/958847451 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b0460 0x7f4c240af120 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:13.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c2afcc700 1 -- 172.21.15.140:0/958847451 >> 172.21.15.140:0/958847451 conn(0x7f4c24063cf0 msgr2=0x7f4c240b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:13.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c2afcc700 1 -- 172.21.15.140:0/958847451 shutdown_connections 2022-07-21T01:50:13.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.863+0000 7f4c2afcc700 1 -- 172.21.15.140:0/958847451 wait complete. 2022-07-21T01:50:13.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2afcc700 1 Processor -- start 2022-07-21T01:50:13.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2afcc700 1 -- start start 2022-07-21T01:50:13.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2afcc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c240b0460 0x7f4c24111910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:13.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2afcc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c240b13e0 0x7f4c2414d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:13.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2afcc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b61d0 0x7f4c2414db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:13.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2afcc700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24112640 con 0x7f4c240b0460 2022-07-21T01:50:13.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2afcc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2414e040 con 0x7f4c240b13e0 2022-07-21T01:50:13.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2afcc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24112100 con 0x7f4c240b61d0 2022-07-21T01:50:13.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c29fca700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c240b0460 0x7f4c24111910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:13.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c29fca700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c240b0460 0x7f4c24111910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34038/0 (socket says 172.21.15.140:34038) 2022-07-21T01:50:13.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c29fca700 1 -- 172.21.15.140:0/2847288762 learned_addr learned my addr 172.21.15.140:0/2847288762 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:13.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c297c9700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c240b13e0 0x7f4c2414d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:13.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2a7cb700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b61d0 0x7f4c2414db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:13.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c29fca700 1 -- 172.21.15.140:0/2847288762 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b61d0 msgr2=0x7f4c2414db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:13.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c29fca700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b61d0 0x7f4c2414db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:13.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c29fca700 1 -- 172.21.15.140:0/2847288762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c240b13e0 msgr2=0x7f4c2414d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:13.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c29fca700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c240b13e0 0x7f4c2414d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:13.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c29fca700 1 -- 172.21.15.140:0/2847288762 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c00a610 con 0x7f4c240b0460 2022-07-21T01:50:13.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2a7cb700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b61d0 0x7f4c2414db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:50:13.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c29fca700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c240b0460 0x7f4c24111910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f4c18000ed0 tx=0x7f4c18007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:13.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c16ffd700 1 -- 172.21.15.140:0/2847288762 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c1800ce80 con 0x7f4c240b0460 2022-07-21T01:50:13.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c16ffd700 1 -- 172.21.15.140:0/2847288762 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1800f470 con 0x7f4c240b0460 2022-07-21T01:50:13.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2afcc700 1 -- 172.21.15.140:0/2847288762 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c241123a0 con 0x7f4c240b0460 2022-07-21T01:50:13.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c16ffd700 1 -- 172.21.15.140:0/2847288762 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c1800f7d0 con 0x7f4c240b0460 2022-07-21T01:50:13.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.867+0000 7f4c2afcc700 1 -- 172.21.15.140:0/2847288762 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c24111e00 con 0x7f4c240b0460 2022-07-21T01:50:13.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.871+0000 7f4c2afcc700 1 -- 172.21.15.140:0/2847288762 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2414e1b0 con 0x7f4c240b0460 2022-07-21T01:50:13.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.871+0000 7f4c16ffd700 1 -- 172.21.15.140:0/2847288762 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4c18019030 con 0x7f4c240b0460 2022-07-21T01:50:13.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.871+0000 7f4c16ffd700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c04053aa0 0x7f4c04055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:13.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.871+0000 7f4c16ffd700 1 -- 172.21.15.140:0/2847288762 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c1806b340 con 0x7f4c240b0460 2022-07-21T01:50:13.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.871+0000 7f4c297c9700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c04053aa0 0x7f4c04055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:13.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.871+0000 7f4c297c9700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c04053aa0 0x7f4c04055f60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f4c2000b000 tx=0x7f4c20000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:13.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:13.875+0000 7f4c16ffd700 1 -- 172.21.15.140:0/2847288762 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4c18042260 con 0x7f4c240b0460 2022-07-21T01:50:14.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.003+0000 7f4c2afcc700 1 -- 172.21.15.140:0/2847288762 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4c240574e0 con 0x7f4c240b0460 2022-07-21T01:50:14.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.003+0000 7f4c16ffd700 1 -- 172.21.15.140:0/2847288762 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v32) v1 ==== 81+0+33741 (secure 0 0 0) 0x7f4c18045880 con 0x7f4c240b0460 2022-07-21T01:50:14.006 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:14.006 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:50:14.007 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 32, 2022-07-21T01:50:14.007 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:50:14.007 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:50:14.007 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:49:13.588079+0000", 2022-07-21T01:50:14.007 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:50:14.008 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:50:14.008 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:50:14.008 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:50:14.009 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:50:14.009 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:50:14.010 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:50:14.010 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:50:14.010 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:50:14.011 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:50:14.011 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:50:14.011 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:50:14.011 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:50:14.011 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:50:14.012 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:50:14.012 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 6, 2022-07-21T01:50:14.012 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:50:14.012 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:50:14.013 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:50:14.013 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:50:14.013 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:50:14.013 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.013 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:50:14.014 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:50:14.014 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:50:14.014 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:50:14.014 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:50:14.015 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:50:14.015 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:50:14.015 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:50:14.015 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:50:14.016 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:50:14.016 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:50:14.016 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:50:14.016 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:50:14.016 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:50:14.017 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:50:14.017 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:50:14.017 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:50:14.017 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:50:14.017 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:50:14.018 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:50:14.018 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:50:14.018 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:50:14.018 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:50:14.018 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.019 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:50:14.019 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:50:14.019 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:50:14.019 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:50:14.020 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:50:14.020 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:50:14.020 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:50:14.020 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:50:14.020 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:50:14.021 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:50:14.021 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:50:14.021 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:50:14.021 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:50:14.022 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:50:14.022 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:50:14.022 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:50:14.022 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:50:14.023 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:50:14.023 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:50:14.023 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:50:14.023 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:50:14.024 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:50:14.024 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:50:14.024 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:50:14.024 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:50:14.024 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:50:14.025 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:50:14.025 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.025 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:50:14.025 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:50:14.026 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:50:14.026 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:50:14.028 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:50:14.028 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:50:14.028 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:50:14.028 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:50:14.029 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:50:14.029 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:50:14.029 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:50:14.029 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:50:14.029 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:50:14.030 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.030 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:50:14.031 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:50:14.031 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.031 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.031 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.032 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:50:14.032 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:50:14.032 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:50:14.032 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:50:14.032 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:50:14.033 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:50:14.033 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:50:14.033 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:50:14.033 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:50:14.033 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:50:14.034 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:50:14.034 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:50:14.034 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:50:14.034 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:50:14.035 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:50:14.035 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:50:14.035 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:50:14.035 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:50:14.036 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:50:14.036 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:50:14.036 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:50:14.036 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:50:14.036 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:50:14.037 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.037 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:50:14.037 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:50:14.037 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:50:14.038 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:50:14.038 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:50:14.038 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:50:14.038 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:50:14.038 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:50:14.039 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:50:14.039 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:50:14.039 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:50:14.039 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:50:14.040 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:50:14.040 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:50:14.040 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:50:14.040 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:50:14.040 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:50:14.041 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:50:14.041 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:50:14.041 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:50:14.041 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:50:14.041 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:50:14.042 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:50:14.042 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:50:14.042 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:50:14.042 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:50:14.043 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:50:14.043 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.043 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:50:14.043 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:50:14.043 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:50:14.044 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:50:14.044 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:50:14.044 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:50:14.044 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:50:14.045 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:50:14.045 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:50:14.045 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:50:14.045 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:50:14.045 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:50:14.046 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:50:14.046 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:50:14.046 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.046 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.050 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.050 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 5, 2022-07-21T01:50:14.051 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:50:14.051 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:49:10.708637+0000", 2022-07-21T01:50:14.051 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:50:14.052 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:50:14.052 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:50:14.053 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:50:14.053 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:50:14.053 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:50:14.054 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:50:14.054 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:50:14.055 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:50:14.055 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:50:14.056 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:50:14.056 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:50:14.057 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:50:14.057 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:50:14.058 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:50:14.058 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:50:14.059 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:50:14.059 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:50:14.059 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:50:14.060 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:50:14.061 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.061 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "31", 2022-07-21T01:50:14.062 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:50:14.063 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:50:14.063 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:50:14.064 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:50:14.064 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:50:14.065 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:50:14.065 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:50:14.065 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:50:14.066 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:50:14.066 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:50:14.067 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:50:14.067 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:50:14.068 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:50:14.068 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:50:14.068 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:50:14.069 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:50:14.069 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:50:14.069 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:50:14.070 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:50:14.070 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:50:14.071 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:50:14.071 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:50:14.072 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:50:14.072 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:50:14.073 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:50:14.073 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:50:14.074 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.074 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:50:14.075 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:50:14.075 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:50:14.075 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:50:14.076 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:50:14.076 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:50:14.077 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:50:14.077 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:50:14.085 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:50:14.086 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:50:14.086 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:50:14.087 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:50:14.087 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:50:14.088 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:50:14.088 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:50:14.089 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.089 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:50:14.090 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:50:14.090 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:50:14.091 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.093 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.093 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.094 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.095 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 6, 2022-07-21T01:50:14.095 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:50:14.096 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:49:11.590708+0000", 2022-07-21T01:50:14.096 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:50:14.096 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:50:14.097 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:50:14.097 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:50:14.098 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:50:14.098 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:50:14.099 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:50:14.100 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:50:14.100 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:50:14.101 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:50:14.101 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:50:14.101 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:50:14.102 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:50:14.102 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:50:14.103 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:50:14.103 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:50:14.104 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:50:14.104 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:50:14.105 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:50:14.105 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:50:14.106 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.106 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "32", 2022-07-21T01:50:14.107 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:50:14.107 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:50:14.108 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:50:14.108 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:50:14.109 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:50:14.109 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:50:14.109 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:50:14.110 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:50:14.110 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:50:14.111 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:50:14.111 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:50:14.112 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:50:14.112 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:50:14.113 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:50:14.113 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:50:14.114 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:50:14.114 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:50:14.114 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:50:14.115 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:50:14.115 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:50:14.116 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:50:14.116 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:50:14.117 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:50:14.117 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:50:14.118 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:50:14.118 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:50:14.119 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.119 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:50:14.120 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:50:14.120 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:50:14.121 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:50:14.121 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:50:14.122 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:50:14.122 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:50:14.123 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:50:14.123 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:50:14.124 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:50:14.124 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:50:14.125 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:50:14.125 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:50:14.125 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:50:14.126 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:50:14.127 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.135 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.135 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.136 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:50:14.136 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:50:14.136 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.136 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:50:14.137 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:50:14.137 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:14.137 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:14.137 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:14.138 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:14.138 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:14.138 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:14.152 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:14.152 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:14.153 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:14.153 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:14.153 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:14.154 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.154 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.155 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.155 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:50:14.156 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:14.156 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.157 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.157 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.158 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:50:14.158 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:14.158 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.158 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.159 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.159 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:14.159 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.159 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.159 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.160 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:50:14.160 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:14.161 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.161 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.161 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.161 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:50:14.162 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:14.162 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.163 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.163 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.163 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:14.164 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.164 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.165 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.165 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:50:14.166 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:14.166 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.167 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.167 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.167 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:50:14.168 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:14.168 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.169 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.169 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.170 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:14.170 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.171 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.172 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.172 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:50:14.172 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:14.173 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.173 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.174 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.174 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:50:14.175 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:14.175 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.176 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.176 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.177 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:50:14.177 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:50:14.177 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:50:14.178 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:50:14.178 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:14.179 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:14.179 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:14.179 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.180 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.180 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.181 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:50:14.181 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:50:14.182 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:14.182 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:14.183 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:14.183 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:14.184 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:14.184 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:14.185 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:14.185 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:14.186 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:14.186 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:14.187 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:14.187 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.187 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.187 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.188 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:50:14.188 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:14.188 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.188 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.189 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.189 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:50:14.189 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:14.189 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.190 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.190 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.190 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:14.190 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.190 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.191 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.191 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:50:14.191 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:14.191 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.192 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.192 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.192 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:50:14.192 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:14.192 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.193 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.193 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.193 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:14.194 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.194 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.195 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.195 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:50:14.195 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:14.196 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.196 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.196 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.197 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:50:14.197 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:14.198 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.198 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.198 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.199 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:14.199 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.199 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.199 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.200 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:50:14.200 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:14.200 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.200 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.200 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.201 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:50:14.201 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:14.201 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.201 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.202 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.202 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:50:14.202 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:50:14.202 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:50:14.202 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:50:14.203 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:14.203 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:14.203 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:14.203 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.204 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.204 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.204 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:50:14.204 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:50:14.204 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:14.205 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:14.205 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:14.205 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:14.205 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:14.206 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:14.206 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:14.206 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 29, 2022-07-21T01:50:14.206 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:14.206 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:14.207 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:14.207 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.207 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.207 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.208 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:50:14.208 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:14.208 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.208 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.208 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.209 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:50:14.209 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:14.209 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.209 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.210 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.210 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:14.210 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.210 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.211 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.211 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:50:14.211 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:14.211 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.212 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.212 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.212 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:50:14.212 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:14.212 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.213 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.213 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.213 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:14.213 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.214 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.214 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.214 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:50:14.214 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:14.214 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.215 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.215 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.215 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:50:14.215 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:14.216 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.216 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.216 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.216 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:14.216 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.217 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.217 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.217 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:50:14.217 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:14.218 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.218 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.218 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.218 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:50:14.218 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:14.219 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.219 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.219 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.219 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:50:14.220 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:50:14.220 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:50:14.220 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:50:14.220 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:14.220 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:14.221 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:14.221 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.221 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.221 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.222 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:50:14.222 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:50:14.222 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:14.222 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:14.223 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:14.223 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:14.223 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:14.223 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:14.223 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:14.224 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:14.224 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:14.224 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:14.224 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:14.225 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.230 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.230 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.231 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:50:14.231 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:14.232 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.232 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.233 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.233 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:50:14.233 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:14.243 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.244 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.244 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.245 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:14.246 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.246 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.247 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.248 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:50:14.248 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:14.249 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.250 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.251 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.251 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:50:14.253 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:14.253 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.253 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.254 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.254 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:14.255 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.255 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.256 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.257 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:50:14.257 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:14.257 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.258 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.258 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.259 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:50:14.259 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:14.259 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.260 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.260 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.261 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:14.261 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.262 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.263 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.263 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:50:14.263 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:14.264 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.264 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.264 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.264 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:50:14.265 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:14.265 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.265 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.265 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.265 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:50:14.266 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:50:14.267 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:50:14.267 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:50:14.267 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:14.267 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:14.268 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:14.268 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.268 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.268 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.269 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:50:14.269 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:50:14.269 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:14.269 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:14.269 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:14.270 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:14.270 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:14.270 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:14.271 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:14.271 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:14.271 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:14.271 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:14.272 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:14.272 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.272 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.272 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.272 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:50:14.273 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:14.273 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.273 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.273 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.274 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:50:14.274 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:14.274 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.274 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.275 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.275 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:14.275 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.275 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.276 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.276 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:50:14.276 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:14.276 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.276 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.277 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.277 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:50:14.277 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:14.277 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.278 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.278 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.278 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:14.279 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.279 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.279 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.279 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:50:14.279 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:14.280 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.280 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.281 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.281 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:50:14.281 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:14.282 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.282 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.282 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.282 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:14.282 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.283 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.283 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.283 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:50:14.283 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:14.284 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.284 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.284 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.284 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:50:14.285 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:14.285 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.285 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.286 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.286 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:50:14.286 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:50:14.286 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:50:14.286 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:50:14.287 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:14.287 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:14.287 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:14.287 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.288 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.288 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.288 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:50:14.288 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:50:14.289 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:14.289 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:14.289 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:14.290 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:14.290 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:14.290 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:14.291 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:14.291 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:14.291 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:14.291 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:14.292 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:14.292 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.292 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.292 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.293 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:50:14.293 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:14.293 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.293 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.294 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.294 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:50:14.294 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:14.294 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.295 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.295 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.295 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:14.295 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.296 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.296 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.296 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:50:14.296 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:14.297 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.297 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.297 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.297 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:50:14.298 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:14.298 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.298 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.298 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.299 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:14.299 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.299 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.300 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.300 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:50:14.300 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:14.300 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.301 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.301 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.301 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:50:14.302 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:14.302 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.302 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.302 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.303 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:14.303 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.303 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.304 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.304 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:50:14.304 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:14.304 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.305 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.306 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.306 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:50:14.307 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:14.307 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.307 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.307 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.308 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:50:14.308 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:50:14.308 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:50:14.308 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:50:14.309 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:14.309 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:14.310 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:14.310 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.310 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.311 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.311 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:50:14.311 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:50:14.311 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:14.312 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:14.312 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:14.312 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:14.312 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:14.312 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:14.313 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:14.313 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 29, 2022-07-21T01:50:14.313 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:14.313 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:14.317 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:14.317 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.318 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.318 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.318 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:50:14.318 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:14.319 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.319 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.319 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.319 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:50:14.320 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:14.320 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.322 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.322 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.323 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:14.323 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.323 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.323 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.324 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:50:14.324 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:14.324 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.324 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.325 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.325 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:50:14.326 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:14.326 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.327 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.333 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.333 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:14.334 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.334 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.335 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.335 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:50:14.336 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:14.337 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.338 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.338 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.339 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:50:14.339 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:14.340 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.340 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.340 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.341 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:14.341 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.342 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.343 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.343 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:50:14.344 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:14.344 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.345 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.345 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.346 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:50:14.346 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:14.347 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.348 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.349 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.349 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:50:14.350 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:50:14.350 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:50:14.351 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:50:14.352 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:14.352 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:14.353 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:14.354 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.355 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.355 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.356 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:50:14.357 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:50:14.357 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:14.358 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:14.359 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:14.360 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:14.361 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:14.361 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:14.362 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:14.362 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:14.362 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:14.363 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:14.363 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:14.363 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.364 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.364 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.364 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:50:14.365 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:14.365 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.365 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.366 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.366 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:50:14.366 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:14.367 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.367 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.367 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.368 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:14.368 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.368 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.369 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.370 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:50:14.370 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:14.371 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.371 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.372 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.372 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:50:14.373 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:14.373 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.374 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.374 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.375 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:14.375 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.376 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.376 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.376 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:50:14.377 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:14.378 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.379 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.379 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.380 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:50:14.381 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:14.381 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.382 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.383 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.383 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:14.384 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:14.384 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.385 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:14.385 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:50:14.386 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:14.386 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.387 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.387 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:14.388 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:50:14.388 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:14.388 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.389 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.389 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.390 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:50:14.390 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:50:14.391 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:50:14.391 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:50:14.392 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:14.392 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:14.393 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:14.393 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:14.394 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.395 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:50:14.395 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:50:14.396 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.396 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:50:14.396 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:14.397 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:14.397 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:14.398 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:14.398 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:14.398 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:50:14.399 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:14.399 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.400 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.400 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:50:14.401 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:14.401 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:14.402 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:14.402 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:14.402 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:14.403 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:50:14.403 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:14.404 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.405 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.406 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:50:14.406 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:14.407 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:14.407 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:14.408 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:14.408 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:14.409 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:50:14.409 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:14.410 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.410 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.411 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:50:14.411 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:14.412 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:14.412 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:14.413 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:14.413 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:14.414 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:50:14.414 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:14.415 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.415 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.416 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:50:14.417 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:14.417 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:14.418 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:14.419 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:14.419 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:14.420 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:50:14.420 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:14.420 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.421 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.421 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:50:14.422 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:14.422 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:14.423 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:14.423 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:14.424 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:14.424 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:50:14.425 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:14.425 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.431 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.431 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:50:14.432 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:14.433 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:14.434 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:14.434 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:14.435 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:14.435 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:50:14.436 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:14.436 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.437 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:14.437 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:50:14.438 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:14.438 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:14.439 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:14.439 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:14.440 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:14.443 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:50:14.444 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:14.445 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.445 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:50:14.446 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:50:14.447 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:50:14.447 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:50:14.448 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:50:14.449 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": {}, 2022-07-21T01:50:14.449 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:50:14.450 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:50:14.450 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:50:14.451 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:50:14.451 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:50:14.452 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:50:14.452 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:50:14.453 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:50:14.453 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:14.454 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:14.455 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:50:14.456 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:50:14.456 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:50:14.457 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:50:14.457 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:50:14.458 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:50:14.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 -- 172.21.15.140:0/2847288762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c04053aa0 msgr2=0x7f4c04055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c04053aa0 0x7f4c04055f60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f4c2000b000 tx=0x7f4c20000bc0).stop 2022-07-21T01:50:14.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 -- 172.21.15.140:0/2847288762 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c240b0460 msgr2=0x7f4c24111910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c240b0460 0x7f4c24111910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f4c18000ed0 tx=0x7f4c18007c50).stop 2022-07-21T01:50:14.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 -- 172.21.15.140:0/2847288762 shutdown_connections 2022-07-21T01:50:14.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c04053aa0 0x7f4c04055f60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c240b61d0 0x7f4c2414db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c240b13e0 0x7f4c2414d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 --2- 172.21.15.140:0/2847288762 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c240b0460 0x7f4c24111910 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 -- 172.21.15.140:0/2847288762 >> 172.21.15.140:0/2847288762 conn(0x7f4c24063cf0 msgr2=0x7f4c240accc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:14.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 -- 172.21.15.140:0/2847288762 shutdown_connections 2022-07-21T01:50:14.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.007+0000 7f4c2afcc700 1 -- 172.21.15.140:0/2847288762 wait complete. 2022-07-21T01:50:14.478 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:50:14.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 Processor -- start 2022-07-21T01:50:14.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 -- start start 2022-07-21T01:50:14.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0dd40b21f0 0x7f0dd40b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0dd40ab8b0 0x7f0dd4146790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd405f970 con 0x7f0dd4146c80 2022-07-21T01:50:14.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd4060b90 con 0x7f0dd40b21f0 2022-07-21T01:50:14.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd414b640 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddaa93700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0dd4146c80 0x7f0dd40618e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48194/0 (socket says 172.21.15.140:48194) 2022-07-21T01:50:14.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddaa93700 1 -- 172.21.15.140:0/944447140 learned_addr learned my addr 172.21.15.140:0/944447140 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:14.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dda292700 1 --2- 172.21.15.140:0/944447140 >> v2:172.21.15.152:3300/0 conn(0x7f0dd40b21f0 0x7f0dd40b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dd9a91700 1 --2- 172.21.15.140:0/944447140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0dd40ab8b0 0x7f0dd4146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dd9290700 1 -- 172.21.15.140:0/944447140 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110667705 0 0) 0x7f0dd405f970 con 0x7f0dd4146c80 2022-07-21T01:50:14.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dd9290700 1 -- 172.21.15.140:0/944447140 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dc0003260 con 0x7f0dd4146c80 2022-07-21T01:50:14.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dd9290700 1 -- 172.21.15.140:0/944447140 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2318668151 0 0) 0x7f0dc0003260 con 0x7f0dd4146c80 2022-07-21T01:50:14.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dda292700 1 -- 172.21.15.140:0/944447140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0dd40ab8b0 msgr2=0x7f0dd4146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:14.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dda292700 1 --2- 172.21.15.140:0/944447140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0dd40ab8b0 0x7f0dd4146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dda292700 1 -- 172.21.15.140:0/944447140 >> v1:172.21.15.140:6789/0 conn(0x7f0dd4146c80 legacy=0x7f0dd40618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dda292700 1 -- 172.21.15.140:0/944447140 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd405fc10 con 0x7f0dd40b21f0 2022-07-21T01:50:14.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dda292700 1 --2- 172.21.15.140:0/944447140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd40b21f0 0x7f0dd40b25d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f0dc4001800 tx=0x7f0dcc0091d0).ready entity=mon.1 client_cookie=b4caaa64f7c9b6d5 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:14.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dd9290700 1 -- 172.21.15.140:0/944447140 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0dc4001660 con 0x7f0dd40b21f0 2022-07-21T01:50:14.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dd9290700 1 -- 172.21.15.140:0/944447140 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc00f510 con 0x7f0dd40b21f0 2022-07-21T01:50:14.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0dd9290700 1 -- 172.21.15.140:0/944447140 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0dcc00f680 con 0x7f0dd40b21f0 2022-07-21T01:50:14.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/944447140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd40b21f0 msgr2=0x7f0dd40b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 --2- 172.21.15.140:0/944447140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd40b21f0 0x7f0dd40b25d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f0dc4001800 tx=0x7f0dcc0091d0).stop 2022-07-21T01:50:14.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/944447140 shutdown_connections 2022-07-21T01:50:14.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 --2- 172.21.15.140:0/944447140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0dd40ab8b0 0x7f0dd4146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 --2- 172.21.15.140:0/944447140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd40b21f0 0x7f0dd40b25d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/944447140 >> 172.21.15.140:0/944447140 conn(0x7f0dd40b3690 msgr2=0x7f0dd40b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:14.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/944447140 shutdown_connections 2022-07-21T01:50:14.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.591+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/944447140 wait complete. 2022-07-21T01:50:14.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0ddc4f6700 1 Processor -- start 2022-07-21T01:50:14.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0ddc4f6700 1 -- start start 2022-07-21T01:50:14.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0ddc4f6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0dd40ab8b0 0x7f0dd40b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0ddc4f6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd40b21f0 0x7f0dd40b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0ddc4f6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0dd4146c80 0x7f0dd40ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0ddc4f6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40af960 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0ddc4f6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd409b0b0 con 0x7f0dd40b21f0 2022-07-21T01:50:14.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0ddc4f6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40afbc0 con 0x7f0dd4146c80 2022-07-21T01:50:14.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0dda292700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0dd40ab8b0 0x7f0dd40b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0dda292700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0dd40ab8b0 0x7f0dd40b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34052/0 (socket says 172.21.15.140:34052) 2022-07-21T01:50:14.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0dda292700 1 -- 172.21.15.140:0/390407848 learned_addr learned my addr 172.21.15.140:0/390407848 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:14.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0ddaa93700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0dd4146c80 0x7f0dd40ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.595+0000 7f0dd9a91700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd40b21f0 0x7f0dd40b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0dda292700 1 -- 172.21.15.140:0/390407848 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0dd4146c80 msgr2=0x7f0dd40ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0dda292700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0dd4146c80 0x7f0dd40ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0dda292700 1 -- 172.21.15.140:0/390407848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd40b21f0 msgr2=0x7f0dd40b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0dda292700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd40b21f0 0x7f0dd40b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0dda292700 1 -- 172.21.15.140:0/390407848 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc009470 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0dda292700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0dd40ab8b0 0x7f0dd40b0060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f0dc4001800 tx=0x7f0dcc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:14.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0dd37fe700 1 -- 172.21.15.140:0/390407848 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0dcc00f680 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0dd37fe700 1 -- 172.21.15.140:0/390407848 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc0032f0 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0dd37fe700 1 -- 172.21.15.140:0/390407848 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0dcc0205e0 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/390407848 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd409a9e0 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/390407848 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd409ad70 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/390407848 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd405feb0 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.599+0000 7f0dd37fe700 1 -- 172.21.15.140:0/390407848 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0dcc027070 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.603+0000 7f0dd37fe700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0db4053980 0x7f0db4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.603+0000 7f0dd37fe700 1 -- 172.21.15.140:0/390407848 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0dcc06d0a0 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.603+0000 7f0dd9a91700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0db4053980 0x7f0db4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.603+0000 7f0dd9a91700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0db4053980 0x7f0db4055e40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f0dc800b000 tx=0x7f0dc8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:14.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.607+0000 7f0dd37fe700 1 -- 172.21.15.140:0/390407848 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0dcc043eb0 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.771+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/390407848 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dd40574e0 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.771+0000 7f0dd37fe700 1 -- 172.21.15.140:0/390407848 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+2139 (secure 0 0 0) 0x7f0dcc025070 con 0x7f0dd40ab8b0 2022-07-21T01:50:14.774 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:14.775 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":10,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/838095894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":838095894},{"type":"v1","addr":"172.21.15.140:6835","nonce":838095894}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:49:12.585320+0000","modified":"2022-07-21T01:49:13.589194+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4894":{"gid":4894,"name":"b","rank":0,"incarnation":9,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/1420650762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1420650762},{"type":"v1","addr":"172.21.15.152:6833","nonce":1420650762}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-07-21T01:50:14.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/390407848 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0db4053980 msgr2=0x7f0db4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0db4053980 0x7f0db4055e40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f0dc800b000 tx=0x7f0dc8006df0).stop 2022-07-21T01:50:14.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/390407848 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0dd40ab8b0 msgr2=0x7f0dd40b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0dd40ab8b0 0x7f0dd40b0060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f0dc4001800 tx=0x7f0dcc016000).stop 2022-07-21T01:50:14.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/390407848 shutdown_connections 2022-07-21T01:50:14.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0db4053980 0x7f0db4055e40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0dd4146c80 0x7f0dd40ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dd40b21f0 0x7f0dd40b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 --2- 172.21.15.140:0/390407848 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0dd40ab8b0 0x7f0dd40b0060 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/390407848 >> 172.21.15.140:0/390407848 conn(0x7f0dd40b3690 msgr2=0x7f0dd40b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:14.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/390407848 shutdown_connections 2022-07-21T01:50:14.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.775+0000 7f0ddc4f6700 1 -- 172.21.15.140:0/390407848 wait complete. 2022-07-21T01:50:14.786 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 10 2022-07-21T01:50:14.793 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:50:14.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.895+0000 7f7baeb83700 1 Processor -- start 2022-07-21T01:50:14.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.895+0000 7f7baeb83700 1 -- start start 2022-07-21T01:50:14.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7ba80b35c0 0x7f7ba80abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7ba8098ae0 0x7f7ba8061440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba80aba30 con 0x7f7ba80ac470 2022-07-21T01:50:14.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba80b39a0 con 0x7f7ba80b35c0 2022-07-21T01:50:14.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba809c000 con 0x7f7ba8098ae0 2022-07-21T01:50:14.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7badb81700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7ba80b35c0 0x7f7ba80abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7bad380700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7ba8098ae0 0x7f7ba8061440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7badb81700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7ba80b35c0 0x7f7ba80abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34060/0 (socket says 172.21.15.140:34060) 2022-07-21T01:50:14.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7badb81700 1 -- 172.21.15.140:0/369353210 learned_addr learned my addr 172.21.15.140:0/369353210 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:14.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7bacb7f700 1 -- 172.21.15.140:0/369353210 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600247865 0 0) 0x7f7ba80aba30 con 0x7f7ba80ac470 2022-07-21T01:50:14.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7badb81700 1 -- 172.21.15.140:0/369353210 >> v2:172.21.15.152:3300/0 conn(0x7f7ba8098ae0 msgr2=0x7f7ba8061440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:14.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7badb81700 1 --2- 172.21.15.140:0/369353210 >> v2:172.21.15.152:3300/0 conn(0x7f7ba8098ae0 0x7f7ba8061440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7badb81700 1 -- 172.21.15.140:0/369353210 >> v1:172.21.15.152:6789/0 conn(0x7f7ba80ac470 legacy=0x7f7ba8061930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7badb81700 1 -- 172.21.15.140:0/369353210 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba80acbc0 con 0x7f7ba80b35c0 2022-07-21T01:50:14.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7badb81700 1 --2- 172.21.15.140:0/369353210 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ba80b35c0 0x7f7ba80abe60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f7ba80602b0 tx=0x7f7ba4006f40).ready entity=mon.0 client_cookie=ae8cea281090e7cc server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:14.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7bacb7f700 1 -- 172.21.15.140:0/369353210 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ba400da40 con 0x7f7ba80b35c0 2022-07-21T01:50:14.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7bacb7f700 1 -- 172.21.15.140:0/369353210 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba4000e90 con 0x7f7ba80b35c0 2022-07-21T01:50:14.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7bacb7f700 1 -- 172.21.15.140:0/369353210 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ba400dd90 con 0x7f7ba80b35c0 2022-07-21T01:50:14.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 -- 172.21.15.140:0/369353210 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ba80b35c0 msgr2=0x7f7ba80abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 --2- 172.21.15.140:0/369353210 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ba80b35c0 0x7f7ba80abe60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f7ba80602b0 tx=0x7f7ba4006f40).stop 2022-07-21T01:50:14.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 -- 172.21.15.140:0/369353210 shutdown_connections 2022-07-21T01:50:14.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 --2- 172.21.15.140:0/369353210 >> v2:172.21.15.152:3300/0 conn(0x7f7ba8098ae0 0x7f7ba8061440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 --2- 172.21.15.140:0/369353210 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ba80b35c0 0x7f7ba80abe60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 -- 172.21.15.140:0/369353210 >> 172.21.15.140:0/369353210 conn(0x7f7ba80b6b20 msgr2=0x7f7ba80b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:14.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 -- 172.21.15.140:0/369353210 shutdown_connections 2022-07-21T01:50:14.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.899+0000 7f7baeb83700 1 -- 172.21.15.140:0/369353210 wait complete. 2022-07-21T01:50:14.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7baeb83700 1 Processor -- start 2022-07-21T01:50:14.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7baeb83700 1 -- start start 2022-07-21T01:50:14.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7baeb83700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ba8098ae0 0x7f7ba80b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7baeb83700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ba80ac470 0x7f7ba813e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7baeb83700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ba80b35c0 0x7f7ba813ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7baeb83700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80662d0 con 0x7f7ba80ac470 2022-07-21T01:50:14.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7baeb83700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba813f210 con 0x7f7ba80b35c0 2022-07-21T01:50:14.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7baeb83700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba813f4b0 con 0x7f7ba8098ae0 2022-07-21T01:50:14.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7bad380700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ba80ac470 0x7f7ba813e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7bad380700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ba80ac470 0x7f7ba813e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34068/0 (socket says 172.21.15.140:34068) 2022-07-21T01:50:14.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7bad380700 1 -- 172.21.15.140:0/3699502478 learned_addr learned my addr 172.21.15.140:0/3699502478 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:14.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7badb81700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ba8098ae0 0x7f7ba80b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.903+0000 7f7bae382700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ba80b35c0 0x7f7ba813ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7bad380700 1 -- 172.21.15.140:0/3699502478 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ba8098ae0 msgr2=0x7f7ba80b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7bad380700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ba8098ae0 0x7f7ba80b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7bad380700 1 -- 172.21.15.140:0/3699502478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ba80b35c0 msgr2=0x7f7ba813ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:14.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7bad380700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ba80b35c0 0x7f7ba813ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:14.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7bad380700 1 -- 172.21.15.140:0/3699502478 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba400b460 con 0x7f7ba80ac470 2022-07-21T01:50:14.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7bad380700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ba80ac470 0x7f7ba813e730 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f7b9800c000 tx=0x7f7b98006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:14.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7b96ffd700 1 -- 172.21.15.140:0/3699502478 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b98010780 con 0x7f7ba80ac470 2022-07-21T01:50:14.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7b96ffd700 1 -- 172.21.15.140:0/3699502478 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b980108e0 con 0x7f7ba80ac470 2022-07-21T01:50:14.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7b96ffd700 1 -- 172.21.15.140:0/3699502478 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b98010c40 con 0x7f7ba80ac470 2022-07-21T01:50:14.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7baeb83700 1 -- 172.21.15.140:0/3699502478 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba80ad7e0 con 0x7f7ba80ac470 2022-07-21T01:50:14.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7baeb83700 1 -- 172.21.15.140:0/3699502478 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba80980d0 con 0x7f7ba80ac470 2022-07-21T01:50:14.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7baeb83700 1 -- 172.21.15.140:0/3699502478 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba8066440 con 0x7f7ba80ac470 2022-07-21T01:50:14.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.907+0000 7f7b96ffd700 1 -- 172.21.15.140:0/3699502478 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7b9801c460 con 0x7f7ba80ac470 2022-07-21T01:50:14.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.911+0000 7f7b96ffd700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b88053a70 0x7f7b88055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:14.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.911+0000 7f7b96ffd700 1 -- 172.21.15.140:0/3699502478 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7b9806c400 con 0x7f7ba80ac470 2022-07-21T01:50:14.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.911+0000 7f7badb81700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b88053a70 0x7f7b88055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:14.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.911+0000 7f7badb81700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b88053a70 0x7f7b88055f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f7ba401b000 tx=0x7f7ba80acd50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:14.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:14.915+0000 7f7b96ffd700 1 -- 172.21.15.140:0/3699502478 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7ba8066440 con 0x7f7ba80ac470 2022-07-21T01:50:15.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:15.079+0000 7f7baeb83700 1 -- 172.21.15.140:0/3699502478 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7ba8066410 con 0x7f7ba80ac470 2022-07-21T01:50:15.991 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:50:15.987+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:50:16.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.003+0000 7f7b96ffd700 1 -- 172.21.15.140:0/3699502478 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v11) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7b980431b0 con 0x7f7ba80ac470 2022-07-21T01:50:16.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.003+0000 7f7baeb83700 1 -- 172.21.15.140:0/3699502478 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b88053a70 msgr2=0x7f7b88055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.003+0000 7f7baeb83700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b88053a70 0x7f7b88055f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f7ba401b000 tx=0x7f7ba80acd50).stop 2022-07-21T01:50:16.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.003+0000 7f7baeb83700 1 -- 172.21.15.140:0/3699502478 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ba80ac470 msgr2=0x7f7ba813e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.003+0000 7f7baeb83700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ba80ac470 0x7f7ba813e730 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f7b9800c000 tx=0x7f7b98006df0).stop 2022-07-21T01:50:16.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.007+0000 7f7baeb83700 1 -- 172.21.15.140:0/3699502478 shutdown_connections 2022-07-21T01:50:16.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.007+0000 7f7baeb83700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b88053a70 0x7f7b88055f30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.007+0000 7f7baeb83700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ba80b35c0 0x7f7ba813ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.007+0000 7f7baeb83700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7ba80ac470 0x7f7ba813e730 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.007+0000 7f7baeb83700 1 --2- 172.21.15.140:0/3699502478 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7ba8098ae0 0x7f7ba80b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.007+0000 7f7baeb83700 1 -- 172.21.15.140:0/3699502478 >> 172.21.15.140:0/3699502478 conn(0x7f7ba80b6b20 msgr2=0x7f7ba813f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:16.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.007+0000 7f7baeb83700 1 -- 172.21.15.140:0/3699502478 shutdown_connections 2022-07-21T01:50:16.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.007+0000 7f7baeb83700 1 -- 172.21.15.140:0/3699502478 wait complete. 2022-07-21T01:50:16.015 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:50:16.024 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:50:16.039 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:50:16.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.131+0000 7f9a0759e700 1 Processor -- start 2022-07-21T01:50:16.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.131+0000 7f9a0759e700 1 -- start start 2022-07-21T01:50:16.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9a080b0570 0x7f9a080b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a081424e0 0x7f9a080ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a081433d0 con 0x7f9a080aea10 2022-07-21T01:50:16.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a08143b10 con 0x7f9a080b0570 2022-07-21T01:50:16.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08143670 con 0x7f9a081424e0 2022-07-21T01:50:16.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a06d9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a081424e0 0x7f9a080ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0659c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9a080b0570 0x7f9a080b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a06d9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a081424e0 0x7f9a080ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:59040/0 (socket says 172.21.15.140:59040) 2022-07-21T01:50:16.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a06d9d700 1 -- 172.21.15.140:0/959824746 learned_addr learned my addr 172.21.15.140:0/959824746 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:16.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a06d9d700 1 -- 172.21.15.140:0/959824746 >> v2:172.21.15.152:3300/0 conn(0x7f9a080b0570 msgr2=0x7f9a080b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:16.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a06d9d700 1 --2- 172.21.15.140:0/959824746 >> v2:172.21.15.152:3300/0 conn(0x7f9a080b0570 0x7f9a080b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a06d9d700 1 -- 172.21.15.140:0/959824746 >> v1:172.21.15.152:6789/0 conn(0x7f9a080aea10 legacy=0x7f9a08141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a06d9d700 1 -- 172.21.15.140:0/959824746 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a080b1010 con 0x7f9a081424e0 2022-07-21T01:50:16.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a06d9d700 1 --2- 172.21.15.140:0/959824746 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a081424e0 0x7f9a080ad350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f9a000032f0 tx=0x7f9a00009290).ready entity=mon.2 client_cookie=305390a53b2cf390 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:16.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0559a700 1 -- 172.21.15.140:0/959824746 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a0000b8a0 con 0x7f9a081424e0 2022-07-21T01:50:16.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0559a700 1 -- 172.21.15.140:0/959824746 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0000ba00 con 0x7f9a081424e0 2022-07-21T01:50:16.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0559a700 1 -- 172.21.15.140:0/959824746 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a0000bb70 con 0x7f9a081424e0 2022-07-21T01:50:16.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 -- 172.21.15.140:0/959824746 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a081424e0 msgr2=0x7f9a080ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 --2- 172.21.15.140:0/959824746 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a081424e0 0x7f9a080ad350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f9a000032f0 tx=0x7f9a00009290).stop 2022-07-21T01:50:16.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 -- 172.21.15.140:0/959824746 shutdown_connections 2022-07-21T01:50:16.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 --2- 172.21.15.140:0/959824746 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a081424e0 0x7f9a080ad350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 --2- 172.21.15.140:0/959824746 >> v2:172.21.15.152:3300/0 conn(0x7f9a080b0570 0x7f9a080b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 -- 172.21.15.140:0/959824746 >> 172.21.15.140:0/959824746 conn(0x7f9a080b9360 msgr2=0x7f9a080b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:16.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 -- 172.21.15.140:0/959824746 shutdown_connections 2022-07-21T01:50:16.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.135+0000 7f9a0759e700 1 -- 172.21.15.140:0/959824746 wait complete. 2022-07-21T01:50:16.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a0759e700 1 Processor -- start 2022-07-21T01:50:16.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a0759e700 1 -- start start 2022-07-21T01:50:16.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a0759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a080aea10 0x7f9a08112550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a0759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a080b0570 0x7f9a08114a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a0759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a081424e0 0x7f9a08140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a0759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08114f80 con 0x7f9a081424e0 2022-07-21T01:50:16.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a0759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081140b0 con 0x7f9a080aea10 2022-07-21T01:50:16.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a0759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08114350 con 0x7f9a080b0570 2022-07-21T01:50:16.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a06d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a081424e0 0x7f9a08140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a06d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a081424e0 0x7f9a08140240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34084/0 (socket says 172.21.15.140:34084) 2022-07-21T01:50:16.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a06d9d700 1 -- 172.21.15.140:0/2545369352 learned_addr learned my addr 172.21.15.140:0/2545369352 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:16.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a05d9b700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a080b0570 0x7f9a08114a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a0659c700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a080aea10 0x7f9a08112550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a06d9d700 1 -- 172.21.15.140:0/2545369352 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a080b0570 msgr2=0x7f9a08114a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a06d9d700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a080b0570 0x7f9a08114a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a06d9d700 1 -- 172.21.15.140:0/2545369352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a080aea10 msgr2=0x7f9a08112550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a06d9d700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a080aea10 0x7f9a08112550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a06d9d700 1 -- 172.21.15.140:0/2545369352 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00009530 con 0x7f9a081424e0 2022-07-21T01:50:16.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a06d9d700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a081424e0 0x7f9a08140240 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f9a000032f0 tx=0x7f9a00016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:16.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f99f77fe700 1 -- 172.21.15.140:0/2545369352 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a0000b8a0 con 0x7f9a081424e0 2022-07-21T01:50:16.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f99f77fe700 1 -- 172.21.15.140:0/2545369352 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a00021480 con 0x7f9a081424e0 2022-07-21T01:50:16.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a0759e700 1 -- 172.21.15.140:0/2545369352 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a081145f0 con 0x7f9a081424e0 2022-07-21T01:50:16.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f99f77fe700 1 -- 172.21.15.140:0/2545369352 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a000217e0 con 0x7f9a081424e0 2022-07-21T01:50:16.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.139+0000 7f9a0759e700 1 -- 172.21.15.140:0/2545369352 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a08140950 con 0x7f9a081424e0 2022-07-21T01:50:16.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.143+0000 7f9a0759e700 1 -- 172.21.15.140:0/2545369352 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a08114780 con 0x7f9a081424e0 2022-07-21T01:50:16.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.143+0000 7f99f77fe700 1 -- 172.21.15.140:0/2545369352 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9a00028070 con 0x7f9a081424e0 2022-07-21T01:50:16.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.143+0000 7f99f77fe700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f99e4053980 0x7f99e4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.143+0000 7f99f77fe700 1 -- 172.21.15.140:0/2545369352 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f9a0006cc60 con 0x7f9a081424e0 2022-07-21T01:50:16.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.143+0000 7f9a0659c700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f99e4053980 0x7f99e4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.143+0000 7f9a0659c700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f99e4053980 0x7f99e4055e40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f99fc00b000 tx=0x7f99fc006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:16.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.151+0000 7f99f77fe700 1 -- 172.21.15.140:0/2545369352 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a00043b30 con 0x7f9a081424e0 2022-07-21T01:50:16.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.307+0000 7f9a0759e700 1 -- 172.21.15.140:0/2545369352 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a080574e0 con 0x7f9a081424e0 2022-07-21T01:50:16.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.307+0000 7f99f77fe700 1 -- 172.21.15.140:0/2545369352 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+1776 (secure 0 0 0) 0x7f9a00015030 con 0x7f9a081424e0 2022-07-21T01:50:16.312 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:16.313 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/838095894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":838095894},{"type":"v1","addr":"172.21.15.140:6835","nonce":838095894}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8}],"filesystems":[{"mdsmap":{"epoch":11,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:49:12.585320+0000","modified":"2022-07-21T01:50:15.992391+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":33,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2022-07-21T01:50:16.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 -- 172.21.15.140:0/2545369352 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f99e4053980 msgr2=0x7f99e4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f99e4053980 0x7f99e4055e40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f99fc00b000 tx=0x7f99fc006df0).stop 2022-07-21T01:50:16.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 -- 172.21.15.140:0/2545369352 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a081424e0 msgr2=0x7f9a08140240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a081424e0 0x7f9a08140240 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f9a000032f0 tx=0x7f9a00016000).stop 2022-07-21T01:50:16.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 -- 172.21.15.140:0/2545369352 shutdown_connections 2022-07-21T01:50:16.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f99e4053980 0x7f99e4055e40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a081424e0 0x7f9a08140240 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a080b0570 0x7f9a08114a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 --2- 172.21.15.140:0/2545369352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a080aea10 0x7f9a08112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 -- 172.21.15.140:0/2545369352 >> 172.21.15.140:0/2545369352 conn(0x7f9a080b9360 msgr2=0x7f9a080b7260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:16.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 -- 172.21.15.140:0/2545369352 shutdown_connections 2022-07-21T01:50:16.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.311+0000 7f9a0759e700 1 -- 172.21.15.140:0/2545369352 wait complete. 2022-07-21T01:50:16.321 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 11 2022-07-21T01:50:16.327 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:50:16.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 Processor -- start 2022-07-21T01:50:16.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 -- start start 2022-07-21T01:50:16.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f386c0b5970 0x7f386c0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f386c0af8c0 0x7f386c0ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386c0b0b80 con 0x7f386c0aeae0 2022-07-21T01:50:16.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c05d4d0 con 0x7f386c0af8c0 2022-07-21T01:50:16.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0b5d50 con 0x7f386c0b5970 2022-07-21T01:50:16.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f3870ebf700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f386c0aeae0 0x7f386c05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48236/0 (socket says 172.21.15.140:48236) 2022-07-21T01:50:16.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f3870ebf700 1 -- 172.21.15.140:0/1552197535 learned_addr learned my addr 172.21.15.140:0/1552197535 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:16.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386b7fe700 1 --2- 172.21.15.140:0/1552197535 >> v2:172.21.15.140:3300/0 conn(0x7f386c0af8c0 0x7f386c0ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386bfff700 1 --2- 172.21.15.140:0/1552197535 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f386c0b5970 0x7f386c0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386affd700 1 -- 172.21.15.140:0/1552197535 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306150527 0 0) 0x7f386c0b0b80 con 0x7f386c0aeae0 2022-07-21T01:50:16.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386affd700 1 -- 172.21.15.140:0/1552197535 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3858003260 con 0x7f386c0aeae0 2022-07-21T01:50:16.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386b7fe700 1 -- 172.21.15.140:0/1552197535 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f386c0b5970 msgr2=0x7f386c0ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:16.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386b7fe700 1 --2- 172.21.15.140:0/1552197535 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f386c0b5970 0x7f386c0ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386b7fe700 1 -- 172.21.15.140:0/1552197535 >> v1:172.21.15.140:6789/0 conn(0x7f386c0aeae0 legacy=0x7f386c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386b7fe700 1 -- 172.21.15.140:0/1552197535 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c0b0db0 con 0x7f386c0af8c0 2022-07-21T01:50:16.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386b7fe700 1 --2- 172.21.15.140:0/1552197535 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f386c0af8c0 0x7f386c0ae5f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f3860002f70 tx=0x7f3860006df0).ready entity=mon.0 client_cookie=363ed45f54d6fd0c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:16.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386affd700 1 -- 172.21.15.140:0/1552197535 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f386000d4a0 con 0x7f386c0af8c0 2022-07-21T01:50:16.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386affd700 1 -- 172.21.15.140:0/1552197535 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3860000e90 con 0x7f386c0af8c0 2022-07-21T01:50:16.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f386affd700 1 -- 172.21.15.140:0/1552197535 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f386000d7f0 con 0x7f386c0af8c0 2022-07-21T01:50:16.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 -- 172.21.15.140:0/1552197535 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f386c0af8c0 msgr2=0x7f386c0ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 --2- 172.21.15.140:0/1552197535 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f386c0af8c0 0x7f386c0ae5f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f3860002f70 tx=0x7f3860006df0).stop 2022-07-21T01:50:16.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 -- 172.21.15.140:0/1552197535 shutdown_connections 2022-07-21T01:50:16.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 --2- 172.21.15.140:0/1552197535 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f386c0af8c0 0x7f386c0ae5f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 --2- 172.21.15.140:0/1552197535 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f386c0b5970 0x7f386c0ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 -- 172.21.15.140:0/1552197535 >> 172.21.15.140:0/1552197535 conn(0x7f386c063490 msgr2=0x7f386c0b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:16.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 -- 172.21.15.140:0/1552197535 shutdown_connections 2022-07-21T01:50:16.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.427+0000 7f38716c0700 1 -- 172.21.15.140:0/1552197535 wait complete. 2022-07-21T01:50:16.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38716c0700 1 Processor -- start 2022-07-21T01:50:16.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38716c0700 1 -- start start 2022-07-21T01:50:16.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38716c0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f386c0aeae0 0x7f386c140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38716c0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f386c0af8c0 0x7f386c10f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38716c0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f386c0b5970 0x7f386c111460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38716c0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c10f820 con 0x7f386c0aeae0 2022-07-21T01:50:16.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38716c0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c10fac0 con 0x7f386c0b5970 2022-07-21T01:50:16.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38716c0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c10fd60 con 0x7f386c0af8c0 2022-07-21T01:50:16.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f386bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f386c0aeae0 0x7f386c140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f386bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f386c0aeae0 0x7f386c140620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34094/0 (socket says 172.21.15.140:34094) 2022-07-21T01:50:16.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f386bfff700 1 -- 172.21.15.140:0/2598308189 learned_addr learned my addr 172.21.15.140:0/2598308189 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:16.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f386b7fe700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f386c0af8c0 0x7f386c10f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f3870ebf700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f386c0b5970 0x7f386c111460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f386bfff700 1 -- 172.21.15.140:0/2598308189 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f386c0af8c0 msgr2=0x7f386c10f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f386bfff700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f386c0af8c0 0x7f386c10f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f386bfff700 1 -- 172.21.15.140:0/2598308189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f386c0b5970 msgr2=0x7f386c111460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:16.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f386bfff700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f386c0b5970 0x7f386c111460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:16.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f386bfff700 1 -- 172.21.15.140:0/2598308189 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386000b180 con 0x7f386c0aeae0 2022-07-21T01:50:16.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f386bfff700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f386c0aeae0 0x7f386c140620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f386400c000 tx=0x7f38640091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:16.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38697fa700 1 -- 172.21.15.140:0/2598308189 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f386400c850 con 0x7f386c0aeae0 2022-07-21T01:50:16.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38697fa700 1 -- 172.21.15.140:0/2598308189 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386400c9b0 con 0x7f386c0aeae0 2022-07-21T01:50:16.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38716c0700 1 -- 172.21.15.140:0/2598308189 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c1414e0 con 0x7f386c0aeae0 2022-07-21T01:50:16.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38697fa700 1 -- 172.21.15.140:0/2598308189 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f386400cd10 con 0x7f386c0aeae0 2022-07-21T01:50:16.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.431+0000 7f38716c0700 1 -- 172.21.15.140:0/2598308189 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c10ec10 con 0x7f386c0aeae0 2022-07-21T01:50:16.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.435+0000 7f38716c0700 1 -- 172.21.15.140:0/2598308189 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c10f650 con 0x7f386c0aeae0 2022-07-21T01:50:16.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.435+0000 7f38697fa700 1 -- 172.21.15.140:0/2598308189 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3864022070 con 0x7f386c0aeae0 2022-07-21T01:50:16.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.435+0000 7f38697fa700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f384c053b00 0x7f384c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:16.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.435+0000 7f38697fa700 1 -- 172.21.15.140:0/2598308189 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f386406c6c0 con 0x7f386c0aeae0 2022-07-21T01:50:16.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.435+0000 7f386b7fe700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f384c053b00 0x7f384c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:16.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.435+0000 7f386b7fe700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f384c053b00 0x7f384c055fc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f3860018000 tx=0x7f3860014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:16.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.443+0000 7f38697fa700 1 -- 172.21.15.140:0/2598308189 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3864043590 con 0x7f386c0aeae0 2022-07-21T01:50:16.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:16.611+0000 7f38716c0700 1 -- 172.21.15.140:0/2598308189 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f386c111200 con 0x7f386c0aeae0 2022-07-21T01:50:17.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.003+0000 7f38697fa700 1 -- 172.21.15.140:0/2598308189 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3864046bb0 con 0x7f386c0aeae0 2022-07-21T01:50:17.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 -- 172.21.15.140:0/2598308189 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f384c053b00 msgr2=0x7f384c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:17.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f384c053b00 0x7f384c055fc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f3860018000 tx=0x7f3860014000).stop 2022-07-21T01:50:17.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 -- 172.21.15.140:0/2598308189 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f386c0aeae0 msgr2=0x7f386c140620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:17.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f386c0aeae0 0x7f386c140620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f386400c000 tx=0x7f38640091d0).stop 2022-07-21T01:50:17.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 -- 172.21.15.140:0/2598308189 shutdown_connections 2022-07-21T01:50:17.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f384c053b00 0x7f384c055fc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:17.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f386c0b5970 0x7f386c111460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:17.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f386c0af8c0 0x7f386c10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:17.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 --2- 172.21.15.140:0/2598308189 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f386c0aeae0 0x7f386c140620 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:17.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 -- 172.21.15.140:0/2598308189 >> 172.21.15.140:0/2598308189 conn(0x7f386c063490 msgr2=0x7f386c0b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:17.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 -- 172.21.15.140:0/2598308189 shutdown_connections 2022-07-21T01:50:17.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.007+0000 7f38716c0700 1 -- 172.21.15.140:0/2598308189 wait complete. 2022-07-21T01:50:17.024 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:50:17.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.139+0000 7f764ffff700 1 Processor -- start 2022-07-21T01:50:17.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.139+0000 7f764ffff700 1 -- start start 2022-07-21T01:50:17.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76500b6200 0x7f76500b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:17.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764ffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76500b17f0 con 0x7f76500b08e0 2022-07-21T01:50:17.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764ffff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76500b5190 con 0x7f76500b49f0 2022-07-21T01:50:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500b4640 con 0x7f76500b6200 2022-07-21T01:50:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764f7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f76500b08e0 0x7f7650146260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48250/0 (socket says 172.21.15.140:48250) 2022-07-21T01:50:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764f7fe700 1 -- 172.21.15.140:0/3283565661 learned_addr learned my addr 172.21.15.140:0/3283565661 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764effd700 1 --2- 172.21.15.140:0/3283565661 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76500b6200 0x7f76500b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:17.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764dffb700 1 -- 172.21.15.140:0/3283565661 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025331347 0 0) 0x7f76500b17f0 con 0x7f76500b08e0 2022-07-21T01:50:17.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764dffb700 1 -- 172.21.15.140:0/3283565661 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7634003260 con 0x7f76500b08e0 2022-07-21T01:50:17.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764dffb700 1 -- 172.21.15.140:0/3283565661 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782389290 0 0) 0x7f76500b5190 con 0x7f76500b49f0 2022-07-21T01:50:17.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764dffb700 1 -- 172.21.15.140:0/3283565661 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76500b17f0 con 0x7f76500b49f0 2022-07-21T01:50:17.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764effd700 1 -- 172.21.15.140:0/3283565661 >> v1:172.21.15.152:6789/0 conn(0x7f76500b49f0 legacy=0x7f76500b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:17.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764effd700 1 -- 172.21.15.140:0/3283565661 >> v1:172.21.15.140:6789/0 conn(0x7f76500b08e0 legacy=0x7f7650146260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:17.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764effd700 1 -- 172.21.15.140:0/3283565661 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500b65e0 con 0x7f76500b6200 2022-07-21T01:50:17.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764effd700 1 --2- 172.21.15.140:0/3283565661 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76500b6200 0x7f76500b4080 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f7644002f70 tx=0x7f7644006df0).ready entity=mon.2 client_cookie=852e63493c396241 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:17.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764dffb700 1 -- 172.21.15.140:0/3283565661 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f764400d5c0 con 0x7f76500b6200 2022-07-21T01:50:17.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764dffb700 1 -- 172.21.15.140:0/3283565661 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7644000e90 con 0x7f76500b6200 2022-07-21T01:50:17.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764dffb700 1 -- 172.21.15.140:0/3283565661 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f764400d910 con 0x7f76500b6200 2022-07-21T01:50:17.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764ffff700 1 -- 172.21.15.140:0/3283565661 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76500b6200 msgr2=0x7f76500b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:17.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764ffff700 1 --2- 172.21.15.140:0/3283565661 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76500b6200 0x7f76500b4080 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f7644002f70 tx=0x7f7644006df0).stop 2022-07-21T01:50:17.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764ffff700 1 -- 172.21.15.140:0/3283565661 shutdown_connections 2022-07-21T01:50:17.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764ffff700 1 --2- 172.21.15.140:0/3283565661 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76500b6200 0x7f76500b4080 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:17.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764ffff700 1 -- 172.21.15.140:0/3283565661 >> 172.21.15.140:0/3283565661 conn(0x7f7650063d10 msgr2=0x7f76500b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:17.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764ffff700 1 -- 172.21.15.140:0/3283565661 shutdown_connections 2022-07-21T01:50:17.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.143+0000 7f764ffff700 1 -- 172.21.15.140:0/3283565661 wait complete. 2022-07-21T01:50:17.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764ffff700 1 Processor -- start 2022-07-21T01:50:17.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764ffff700 1 -- start start 2022-07-21T01:50:17.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76500b08e0 0x7f7650066ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:17.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76500b49f0 0x7f76500af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:17.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76500b6200 0x7f76500af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:17.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650064760 con 0x7f76500b08e0 2022-07-21T01:50:17.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7650141280 con 0x7f76500b49f0 2022-07-21T01:50:17.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500644c0 con 0x7f76500b6200 2022-07-21T01:50:17.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76500b08e0 0x7f7650066ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:17.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76500b08e0 0x7f7650066ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34108/0 (socket says 172.21.15.140:34108) 2022-07-21T01:50:17.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764effd700 1 -- 172.21.15.140:0/2602736758 learned_addr learned my addr 172.21.15.140:0/2602736758 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:17.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764e7fc700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76500b49f0 0x7f76500af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:17.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.147+0000 7f764f7fe700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76500b6200 0x7f76500af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:17.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f764effd700 1 -- 172.21.15.140:0/2602736758 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76500b6200 msgr2=0x7f76500af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:17.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f764effd700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76500b6200 0x7f76500af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:17.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f764effd700 1 -- 172.21.15.140:0/2602736758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76500b49f0 msgr2=0x7f76500af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:17.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f764effd700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76500b49f0 0x7f76500af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:17.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f764effd700 1 -- 172.21.15.140:0/2602736758 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764400b180 con 0x7f76500b08e0 2022-07-21T01:50:17.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f764effd700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76500b08e0 0x7f7650066ee0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f7644002f70 tx=0x7f7644014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:17.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f763ffff700 1 -- 172.21.15.140:0/2602736758 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7644016070 con 0x7f76500b08e0 2022-07-21T01:50:17.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f763ffff700 1 -- 172.21.15.140:0/2602736758 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76440137d0 con 0x7f76500b08e0 2022-07-21T01:50:17.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f764ffff700 1 -- 172.21.15.140:0/2602736758 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76500afdd0 con 0x7f76500b08e0 2022-07-21T01:50:17.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f763ffff700 1 -- 172.21.15.140:0/2602736758 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7644013b30 con 0x7f76500b08e0 2022-07-21T01:50:17.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f764ffff700 1 -- 172.21.15.140:0/2602736758 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76500acf60 con 0x7f76500b08e0 2022-07-21T01:50:17.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f764ffff700 1 -- 172.21.15.140:0/2602736758 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7650142b80 con 0x7f76500b08e0 2022-07-21T01:50:17.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f763ffff700 1 -- 172.21.15.140:0/2602736758 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7644025030 con 0x7f76500b08e0 2022-07-21T01:50:17.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f763ffff700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f762c053950 0x7f762c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:17.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f763ffff700 1 -- 172.21.15.140:0/2602736758 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f764406cda0 con 0x7f76500b08e0 2022-07-21T01:50:17.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.151+0000 7f764e7fc700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f762c053950 0x7f762c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:17.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.155+0000 7f764e7fc700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f762c053950 0x7f762c055e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f764000d000 tx=0x7f7640007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:17.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.159+0000 7f763ffff700 1 -- 172.21.15.140:0/2602736758 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7644043c70 con 0x7f76500b08e0 2022-07-21T01:50:17.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:17.283+0000 7f764ffff700 1 -- 172.21.15.140:0/2602736758 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f76500574e0 con 0x7f76500b08e0 2022-07-21T01:50:18.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.011+0000 7f763ffff700 1 -- 172.21.15.140:0/2602736758 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) v1 ==== 185+0+0 (secure 0 0 0) 0x7f7644023020 con 0x7f76500b08e0 2022-07-21T01:50:18.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 -- 172.21.15.140:0/2602736758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f762c053950 msgr2=0x7f762c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:18.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f762c053950 0x7f762c055e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f764000d000 tx=0x7f7640007a20).stop 2022-07-21T01:50:18.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 -- 172.21.15.140:0/2602736758 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76500b08e0 msgr2=0x7f7650066ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:18.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76500b08e0 0x7f7650066ee0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f7644002f70 tx=0x7f7644014000).stop 2022-07-21T01:50:18.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 -- 172.21.15.140:0/2602736758 shutdown_connections 2022-07-21T01:50:18.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f762c053950 0x7f762c055e10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:18.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76500b6200 0x7f76500af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:18.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76500b49f0 0x7f76500af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:18.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 --2- 172.21.15.140:0/2602736758 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76500b08e0 0x7f7650066ee0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:18.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 -- 172.21.15.140:0/2602736758 >> 172.21.15.140:0/2602736758 conn(0x7f7650063d10 msgr2=0x7f76500ab920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:18.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 -- 172.21.15.140:0/2602736758 shutdown_connections 2022-07-21T01:50:18.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.015+0000 7f764ffff700 1 -- 172.21.15.140:0/2602736758 wait complete. 2022-07-21T01:50:18.026 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:50:18.033 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:50:18.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.143+0000 7f849814a700 1 Processor -- start 2022-07-21T01:50:18.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.143+0000 7f849814a700 1 -- start start 2022-07-21T01:50:18.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f849814a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 0x7f84900b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:18.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f849814a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84900ae670 0x7f84900aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:18.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f849814a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84900acae0 con 0x7f84900b6690 2022-07-21T01:50:18.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f849814a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8490147100 con 0x7f84900ae670 2022-07-21T01:50:18.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f849814a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84900acd10 con 0x7f84900b6a70 2022-07-21T01:50:18.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f84956e5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 0x7f84900b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:18.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f84966e7700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84900ae670 0x7f84900aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:18.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f84956e5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 0x7f84900b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:59080/0 (socket says 172.21.15.140:59080) 2022-07-21T01:50:18.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f84956e5700 1 -- 172.21.15.140:0/3468980305 learned_addr learned my addr 172.21.15.140:0/3468980305 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:18.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f84956e5700 1 -- 172.21.15.140:0/3468980305 >> v2:172.21.15.152:3300/0 conn(0x7f84900ae670 msgr2=0x7f84900aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:18.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f84956e5700 1 --2- 172.21.15.140:0/3468980305 >> v2:172.21.15.152:3300/0 conn(0x7f84900ae670 0x7f84900aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:18.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f84956e5700 1 -- 172.21.15.140:0/3468980305 >> v1:172.21.15.152:6789/0 conn(0x7f84900b6690 legacy=0x7f849013c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:18.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f84956e5700 1 -- 172.21.15.140:0/3468980305 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84900acfb0 con 0x7f84900b6a70 2022-07-21T01:50:18.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f84956e5700 1 --2- 172.21.15.140:0/3468980305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 0x7f84900b59a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f848c002f70 tx=0x7f848c006eb0).ready entity=mon.2 client_cookie=8cbe21767a17abaa server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:18.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f8494ee4700 1 -- 172.21.15.140:0/3468980305 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f848c00d570 con 0x7f84900b6a70 2022-07-21T01:50:18.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f8494ee4700 1 -- 172.21.15.140:0/3468980305 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c000e90 con 0x7f84900b6a70 2022-07-21T01:50:18.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f8494ee4700 1 -- 172.21.15.140:0/3468980305 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f848c00d900 con 0x7f84900b6a70 2022-07-21T01:50:18.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f849814a700 1 -- 172.21.15.140:0/3468980305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 msgr2=0x7f84900b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:18.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.147+0000 7f849814a700 1 --2- 172.21.15.140:0/3468980305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 0x7f84900b59a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f848c002f70 tx=0x7f848c006eb0).stop 2022-07-21T01:50:18.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 -- 172.21.15.140:0/3468980305 shutdown_connections 2022-07-21T01:50:18.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 --2- 172.21.15.140:0/3468980305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 0x7f84900b59a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:18.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 --2- 172.21.15.140:0/3468980305 >> v2:172.21.15.152:3300/0 conn(0x7f84900ae670 0x7f84900aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:18.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 -- 172.21.15.140:0/3468980305 >> 172.21.15.140:0/3468980305 conn(0x7f84900b9f70 msgr2=0x7f84900b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:18.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 -- 172.21.15.140:0/3468980305 shutdown_connections 2022-07-21T01:50:18.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 -- 172.21.15.140:0/3468980305 wait complete. 2022-07-21T01:50:18.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 Processor -- start 2022-07-21T01:50:18.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 -- start start 2022-07-21T01:50:18.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84900ae670 0x7f849013dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:18.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84900b6690 0x7f8490138d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:18.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 0x7f8490139280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:18.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84900b35a0 con 0x7f84900ae670 2022-07-21T01:50:18.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849013d810 con 0x7f84900b6690 2022-07-21T01:50:18.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f849814a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84900b3300 con 0x7f84900b6a70 2022-07-21T01:50:18.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f8495ee6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84900ae670 0x7f849013dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:18.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f8495ee6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84900ae670 0x7f849013dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34122/0 (socket says 172.21.15.140:34122) 2022-07-21T01:50:18.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f8495ee6700 1 -- 172.21.15.140:0/1260240589 learned_addr learned my addr 172.21.15.140:0/1260240589 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:18.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f84966e7700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 0x7f8490139280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:18.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.151+0000 7f84956e5700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84900b6690 0x7f8490138d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:18.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f8495ee6700 1 -- 172.21.15.140:0/1260240589 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 msgr2=0x7f8490139280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:18.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f8495ee6700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 0x7f8490139280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:18.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f8495ee6700 1 -- 172.21.15.140:0/1260240589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84900b6690 msgr2=0x7f8490138d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:18.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f8495ee6700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84900b6690 0x7f8490138d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:18.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f8495ee6700 1 -- 172.21.15.140:0/1260240589 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c00b1e0 con 0x7f84900ae670 2022-07-21T01:50:18.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f8495ee6700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84900ae670 0x7f849013dd20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f84900b3220 tx=0x7f8484007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:18.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f8482ffd700 1 -- 172.21.15.140:0/1260240589 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8484007030 con 0x7f84900ae670 2022-07-21T01:50:18.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f8482ffd700 1 -- 172.21.15.140:0/1260240589 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8484007190 con 0x7f84900ae670 2022-07-21T01:50:18.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f849814a700 1 -- 172.21.15.140:0/1260240589 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849013d500 con 0x7f84900ae670 2022-07-21T01:50:18.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f8482ffd700 1 -- 172.21.15.140:0/1260240589 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8484006510 con 0x7f84900ae670 2022-07-21T01:50:18.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f849814a700 1 -- 172.21.15.140:0/1260240589 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8490139770 con 0x7f84900ae670 2022-07-21T01:50:18.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f849814a700 1 -- 172.21.15.140:0/1260240589 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84900ad250 con 0x7f84900ae670 2022-07-21T01:50:18.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.155+0000 7f8482ffd700 1 -- 172.21.15.140:0/1260240589 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8484010030 con 0x7f84900ae670 2022-07-21T01:50:18.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.159+0000 7f8482ffd700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8470053a10 0x7f8470055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:18.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.159+0000 7f84956e5700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8470053a10 0x7f8470055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:18.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.159+0000 7f8482ffd700 1 -- 172.21.15.140:0/1260240589 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f848406b9f0 con 0x7f84900ae670 2022-07-21T01:50:18.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.159+0000 7f84956e5700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8470053a10 0x7f8470055ed0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f848c01c000 tx=0x7f848c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:18.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.163+0000 7f8482ffd700 1 -- 172.21.15.140:0/1260240589 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8484042960 con 0x7f84900ae670 2022-07-21T01:50:18.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:18.287+0000 7f849814a700 1 -- 172.21.15.140:0/1260240589 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84900574e0 con 0x7f84900ae670 2022-07-21T01:50:19.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f8482ffd700 1 -- 172.21.15.140:0/1260240589 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8484045f80 con 0x7f84900ae670 2022-07-21T01:50:19.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 -- 172.21.15.140:0/1260240589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8470053a10 msgr2=0x7f8470055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8470053a10 0x7f8470055ed0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f848c01c000 tx=0x7f848c015000).stop 2022-07-21T01:50:19.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 -- 172.21.15.140:0/1260240589 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84900ae670 msgr2=0x7f849013dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84900ae670 0x7f849013dd20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f84900b3220 tx=0x7f8484007c10).stop 2022-07-21T01:50:19.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 -- 172.21.15.140:0/1260240589 shutdown_connections 2022-07-21T01:50:19.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8470053a10 0x7f8470055ed0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84900b6a70 0x7f8490139280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84900b6690 0x7f8490138d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 --2- 172.21.15.140:0/1260240589 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84900ae670 0x7f849013dd20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 -- 172.21.15.140:0/1260240589 >> 172.21.15.140:0/1260240589 conn(0x7f84900b9f70 msgr2=0x7f84900b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:19.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 -- 172.21.15.140:0/1260240589 shutdown_connections 2022-07-21T01:50:19.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.027+0000 7f849814a700 1 -- 172.21.15.140:0/1260240589 wait complete. 2022-07-21T01:50:19.039 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:50:19.046 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-07-21T01:50:19.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f586db700 1 Processor -- start 2022-07-21T01:50:19.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f586db700 1 -- start start 2022-07-21T01:50:19.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f586db700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7f5005cda0 0x7f7f5005d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f586db700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7f50060bb0 0x7f7f50147760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f586db700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f50147090 con 0x7f7f500b11b0 2022-07-21T01:50:19.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f586db700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f50060160 con 0x7f7f50060bb0 2022-07-21T01:50:19.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f586db700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f50099080 con 0x7f7f5005cda0 2022-07-21T01:50:19.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f55c76700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f7f500b11b0 0x7f7f5005fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48278/0 (socket says 172.21.15.140:48278) 2022-07-21T01:50:19.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f56c78700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7f50060bb0 0x7f7f50147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f55c76700 1 -- 172.21.15.140:0/1943859903 learned_addr learned my addr 172.21.15.140:0/1943859903 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:19.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f56477700 1 --2- 172.21.15.140:0/1943859903 >> v2:172.21.15.152:3300/0 conn(0x7f7f5005cda0 0x7f7f5005d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f56c78700 1 -- 172.21.15.140:0/1943859903 >> v2:172.21.15.152:3300/0 conn(0x7f7f5005cda0 msgr2=0x7f7f5005d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:19.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f56c78700 1 --2- 172.21.15.140:0/1943859903 >> v2:172.21.15.152:3300/0 conn(0x7f7f5005cda0 0x7f7f5005d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f56c78700 1 -- 172.21.15.140:0/1943859903 >> v1:172.21.15.140:6789/0 conn(0x7f7f500b11b0 legacy=0x7f7f5005fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f56c78700 1 -- 172.21.15.140:0/1943859903 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f501472c0 con 0x7f7f50060bb0 2022-07-21T01:50:19.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f56c78700 1 --2- 172.21.15.140:0/1943859903 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f50060bb0 0x7f7f50147760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f7f48002f70 tx=0x7f7f48006df0).ready entity=mon.0 client_cookie=fa6aa98af79af62e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:19.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f55475700 1 -- 172.21.15.140:0/1943859903 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f4800d570 con 0x7f7f50060bb0 2022-07-21T01:50:19.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f55475700 1 -- 172.21.15.140:0/1943859903 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f48000e90 con 0x7f7f50060bb0 2022-07-21T01:50:19.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.155+0000 7f7f55475700 1 -- 172.21.15.140:0/1943859903 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f4800d8c0 con 0x7f7f50060bb0 2022-07-21T01:50:19.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.159+0000 7f7f586db700 1 -- 172.21.15.140:0/1943859903 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f50060bb0 msgr2=0x7f7f50147760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.159+0000 7f7f586db700 1 --2- 172.21.15.140:0/1943859903 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f50060bb0 0x7f7f50147760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f7f48002f70 tx=0x7f7f48006df0).stop 2022-07-21T01:50:19.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.159+0000 7f7f586db700 1 -- 172.21.15.140:0/1943859903 shutdown_connections 2022-07-21T01:50:19.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.159+0000 7f7f586db700 1 --2- 172.21.15.140:0/1943859903 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f50060bb0 0x7f7f50147760 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.159+0000 7f7f586db700 1 --2- 172.21.15.140:0/1943859903 >> v2:172.21.15.152:3300/0 conn(0x7f7f5005cda0 0x7f7f5005d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.159+0000 7f7f586db700 1 -- 172.21.15.140:0/1943859903 >> 172.21.15.140:0/1943859903 conn(0x7f7f500b5bb0 msgr2=0x7f7f500b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:19.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.159+0000 7f7f586db700 1 -- 172.21.15.140:0/1943859903 shutdown_connections 2022-07-21T01:50:19.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.159+0000 7f7f586db700 1 -- 172.21.15.140:0/1943859903 wait complete. 2022-07-21T01:50:19.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.159+0000 7f7f586db700 1 Processor -- start 2022-07-21T01:50:19.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.159+0000 7f7f586db700 1 -- start start 2022-07-21T01:50:19.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f586db700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f5005cda0 0x7f7f501469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f586db700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50060bb0 0x7f7f500ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f586db700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f500b11b0 0x7f7f50159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f586db700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f500ab6c0 con 0x7f7f5005cda0 2022-07-21T01:50:19.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f586db700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f50064680 con 0x7f7f50060bb0 2022-07-21T01:50:19.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f586db700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f500ab900 con 0x7f7f500b11b0 2022-07-21T01:50:19.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f56477700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f5005cda0 0x7f7f501469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f56477700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f5005cda0 0x7f7f501469f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34136/0 (socket says 172.21.15.140:34136) 2022-07-21T01:50:19.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f56477700 1 -- 172.21.15.140:0/2092167067 learned_addr learned my addr 172.21.15.140:0/2092167067 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:19.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f56c78700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f500b11b0 0x7f7f50159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f55c76700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50060bb0 0x7f7f500ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f56477700 1 -- 172.21.15.140:0/2092167067 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f500b11b0 msgr2=0x7f7f50159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f56477700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f500b11b0 0x7f7f50159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f56477700 1 -- 172.21.15.140:0/2092167067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50060bb0 msgr2=0x7f7f500ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f56477700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50060bb0 0x7f7f500ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f56477700 1 -- 172.21.15.140:0/2092167067 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4800b180 con 0x7f7f5005cda0 2022-07-21T01:50:19.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f56c78700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f500b11b0 0x7f7f50159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:50:19.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f55c76700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50060bb0 0x7f7f500ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:50:19.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f56477700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f5005cda0 0x7f7f501469f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f7f4400c000 tx=0x7f7f44006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:19.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f4f7fe700 1 -- 172.21.15.140:0/2092167067 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f44010780 con 0x7f7f5005cda0 2022-07-21T01:50:19.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f4f7fe700 1 -- 172.21.15.140:0/2092167067 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f440108e0 con 0x7f7f5005cda0 2022-07-21T01:50:19.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f586db700 1 -- 172.21.15.140:0/2092167067 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f500abba0 con 0x7f7f5005cda0 2022-07-21T01:50:19.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f4f7fe700 1 -- 172.21.15.140:0/2092167067 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7f44010c40 con 0x7f7f5005cda0 2022-07-21T01:50:19.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.163+0000 7f7f586db700 1 -- 172.21.15.140:0/2092167067 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f50064a80 con 0x7f7f5005cda0 2022-07-21T01:50:19.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.167+0000 7f7f4f7fe700 1 -- 172.21.15.140:0/2092167067 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7f4401c460 con 0x7f7f5005cda0 2022-07-21T01:50:19.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.167+0000 7f7f586db700 1 -- 172.21.15.140:0/2092167067 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f500574e0 con 0x7f7f5005cda0 2022-07-21T01:50:19.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.167+0000 7f7f4f7fe700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f30053aa0 0x7f7f30055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.167+0000 7f7f4f7fe700 1 -- 172.21.15.140:0/2092167067 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f7f4406c260 con 0x7f7f5005cda0 2022-07-21T01:50:19.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.167+0000 7f7f55c76700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f30053aa0 0x7f7f30055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.167+0000 7f7f55c76700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f30053aa0 0x7f7f30055f60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f7f40008000 tx=0x7f7f40009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:19.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.171+0000 7f7f4f7fe700 1 -- 172.21.15.140:0/2092167067 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7f440432f0 con 0x7f7f5005cda0 2022-07-21T01:50:19.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.287+0000 7f7f586db700 1 -- 172.21.15.140:0/2092167067 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7f500b24d0 con 0x7f7f5005cda0 2022-07-21T01:50:19.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.295+0000 7f7f4f7fe700 1 -- 172.21.15.140:0/2092167067 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7f44046910 con 0x7f7f5005cda0 2022-07-21T01:50:19.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.295+0000 7f7f586db700 1 -- 172.21.15.140:0/2092167067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f30053aa0 msgr2=0x7f7f30055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.295+0000 7f7f586db700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f30053aa0 0x7f7f30055f60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f7f40008000 tx=0x7f7f40009040).stop 2022-07-21T01:50:19.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.295+0000 7f7f586db700 1 -- 172.21.15.140:0/2092167067 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f5005cda0 msgr2=0x7f7f501469f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.295+0000 7f7f586db700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f5005cda0 0x7f7f501469f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f7f4400c000 tx=0x7f7f44006df0).stop 2022-07-21T01:50:19.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.299+0000 7f7f586db700 1 -- 172.21.15.140:0/2092167067 shutdown_connections 2022-07-21T01:50:19.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.299+0000 7f7f586db700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7f30053aa0 0x7f7f30055f60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.299+0000 7f7f586db700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f500b11b0 0x7f7f50159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.299+0000 7f7f586db700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f50060bb0 0x7f7f500ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.299+0000 7f7f586db700 1 --2- 172.21.15.140:0/2092167067 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7f5005cda0 0x7f7f501469f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.299+0000 7f7f586db700 1 -- 172.21.15.140:0/2092167067 >> 172.21.15.140:0/2092167067 conn(0x7f7f500b5bb0 msgr2=0x7f7f5009a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:19.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.299+0000 7f7f586db700 1 -- 172.21.15.140:0/2092167067 shutdown_connections 2022-07-21T01:50:19.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.299+0000 7f7f586db700 1 -- 172.21.15.140:0/2092167067 wait complete. 2022-07-21T01:50:19.331 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2022-07-21T01:50:19.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e8fc2700 1 Processor -- start 2022-07-21T01:50:19.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e8fc2700 1 -- start start 2022-07-21T01:50:19.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e8fc2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40ace90 0x7f10e40aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e8fc2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f10e40b13e0 0x7f10e41463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e8fc2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e40b5f10 con 0x7f10e40b5b30 2022-07-21T01:50:19.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e8fc2700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e40afc40 con 0x7f10e40b13e0 2022-07-21T01:50:19.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e8fc2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40afee0 con 0x7f10e40ace90 2022-07-21T01:50:19.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e259c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f10e40b5b30 0x7f10e40b0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48294/0 (socket says 172.21.15.140:48294) 2022-07-21T01:50:19.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e259c700 1 -- 172.21.15.140:0/4033497751 learned_addr learned my addr 172.21.15.140:0/4033497751 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:19.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e1d9b700 1 --2- 172.21.15.140:0/4033497751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40ace90 0x7f10e40aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e2d9d700 1 --2- 172.21.15.140:0/4033497751 >> v2:172.21.15.152:3300/0 conn(0x7f10e40b13e0 0x7f10e41463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e159a700 1 -- 172.21.15.140:0/4033497751 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550338806 0 0) 0x7f10e40b5f10 con 0x7f10e40b5b30 2022-07-21T01:50:19.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e159a700 1 -- 172.21.15.140:0/4033497751 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10c4003260 con 0x7f10e40b5b30 2022-07-21T01:50:19.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e1d9b700 1 -- 172.21.15.140:0/4033497751 >> v2:172.21.15.152:3300/0 conn(0x7f10e40b13e0 msgr2=0x7f10e41463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:19.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e1d9b700 1 --2- 172.21.15.140:0/4033497751 >> v2:172.21.15.152:3300/0 conn(0x7f10e40b13e0 0x7f10e41463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e1d9b700 1 -- 172.21.15.140:0/4033497751 >> v1:172.21.15.140:6789/0 conn(0x7f10e40b5b30 legacy=0x7f10e40b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.439+0000 7f10e1d9b700 1 -- 172.21.15.140:0/4033497751 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e40ad490 con 0x7f10e40ace90 2022-07-21T01:50:19.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e1d9b700 1 --2- 172.21.15.140:0/4033497751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40ace90 0x7f10e40aefb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f10d40032f0 tx=0x7f10d40091d0).ready entity=mon.2 client_cookie=660e16eeb5d05d5e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:19.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e159a700 1 -- 172.21.15.140:0/4033497751 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10d400b600 con 0x7f10e40ace90 2022-07-21T01:50:19.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e159a700 1 -- 172.21.15.140:0/4033497751 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d400b760 con 0x7f10e40ace90 2022-07-21T01:50:19.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e159a700 1 -- 172.21.15.140:0/4033497751 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10d400b8d0 con 0x7f10e40ace90 2022-07-21T01:50:19.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 -- 172.21.15.140:0/4033497751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40ace90 msgr2=0x7f10e40aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 --2- 172.21.15.140:0/4033497751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40ace90 0x7f10e40aefb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f10d40032f0 tx=0x7f10d40091d0).stop 2022-07-21T01:50:19.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 -- 172.21.15.140:0/4033497751 shutdown_connections 2022-07-21T01:50:19.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 --2- 172.21.15.140:0/4033497751 >> v2:172.21.15.152:3300/0 conn(0x7f10e40b13e0 0x7f10e41463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 --2- 172.21.15.140:0/4033497751 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40ace90 0x7f10e40aefb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 -- 172.21.15.140:0/4033497751 >> 172.21.15.140:0/4033497751 conn(0x7f10e40635b0 msgr2=0x7f10e40b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:19.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 -- 172.21.15.140:0/4033497751 shutdown_connections 2022-07-21T01:50:19.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 -- 172.21.15.140:0/4033497751 wait complete. 2022-07-21T01:50:19.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 Processor -- start 2022-07-21T01:50:19.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 -- start start 2022-07-21T01:50:19.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f10e40ace90 0x7f10e415a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40b13e0 0x7f10e410e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10e40b5b30 0x7f10e41461a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e4065580 con 0x7f10e40ace90 2022-07-21T01:50:19.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e4159b30 con 0x7f10e40b5b30 2022-07-21T01:50:19.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e8fc2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e4159dd0 con 0x7f10e40b13e0 2022-07-21T01:50:19.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.443+0000 7f10e259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f10e40ace90 0x7f10e415a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f10e40ace90 0x7f10e415a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34150/0 (socket says 172.21.15.140:34150) 2022-07-21T01:50:19.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e259c700 1 -- 172.21.15.140:0/3018585397 learned_addr learned my addr 172.21.15.140:0/3018585397 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:19.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e1d9b700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40b13e0 0x7f10e410e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e2d9d700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10e40b5b30 0x7f10e41461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e1d9b700 1 -- 172.21.15.140:0/3018585397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10e40b5b30 msgr2=0x7f10e41461a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e1d9b700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10e40b5b30 0x7f10e41461a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e1d9b700 1 -- 172.21.15.140:0/3018585397 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f10e40ace90 msgr2=0x7f10e415a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:19.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e1d9b700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f10e40ace90 0x7f10e415a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:19.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e1d9b700 1 -- 172.21.15.140:0/3018585397 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d4009470 con 0x7f10e40b13e0 2022-07-21T01:50:19.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e259c700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f10e40ace90 0x7f10e415a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:19.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e1d9b700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40b13e0 0x7f10e410e680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f10d40032f0 tx=0x7f10d4016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:19.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10db7fe700 1 -- 172.21.15.140:0/3018585397 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10d400b8d0 con 0x7f10e40b13e0 2022-07-21T01:50:19.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10db7fe700 1 -- 172.21.15.140:0/3018585397 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d4021630 con 0x7f10e40b13e0 2022-07-21T01:50:19.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e8fc2700 1 -- 172.21.15.140:0/3018585397 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e415a070 con 0x7f10e40b13e0 2022-07-21T01:50:19.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10db7fe700 1 -- 172.21.15.140:0/3018585397 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10d400b600 con 0x7f10e40b13e0 2022-07-21T01:50:19.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e8fc2700 1 -- 172.21.15.140:0/3018585397 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e40661c0 con 0x7f10e40b13e0 2022-07-21T01:50:19.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10db7fe700 1 -- 172.21.15.140:0/3018585397 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f10d4027020 con 0x7f10e40b13e0 2022-07-21T01:50:19.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.447+0000 7f10e8fc2700 1 -- 172.21.15.140:0/3018585397 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e415a200 con 0x7f10e40b13e0 2022-07-21T01:50:19.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.451+0000 7f10db7fe700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f10c8053b00 0x7f10c8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:19.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.451+0000 7f10db7fe700 1 -- 172.21.15.140:0/3018585397 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f10d406d270 con 0x7f10e40b13e0 2022-07-21T01:50:19.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.451+0000 7f10e259c700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f10c8053b00 0x7f10c8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:19.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.451+0000 7f10e259c700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f10c8053b00 0x7f10c8055fc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f10dc00d000 tx=0x7f10dc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:19.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.459+0000 7f10db7fe700 1 -- 172.21.15.140:0/3018585397 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f10d4044360 con 0x7f10e40b13e0 2022-07-21T01:50:19.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:19.623+0000 7f10e8fc2700 1 -- 172.21.15.140:0/3018585397 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7f10e40574e0 con 0x7f10e40b13e0 2022-07-21T01:50:20.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.023+0000 7f10db7fe700 1 -- 172.21.15.140:0/3018585397 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v95) v1 ==== 139+0+0 (secure 0 0 0) 0x7f10d40150d0 con 0x7f10e40b13e0 2022-07-21T01:50:20.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 -- 172.21.15.140:0/3018585397 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f10c8053b00 msgr2=0x7f10c8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:20.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f10c8053b00 0x7f10c8055fc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f10dc00d000 tx=0x7f10dc009040).stop 2022-07-21T01:50:20.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 -- 172.21.15.140:0/3018585397 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40b13e0 msgr2=0x7f10e410e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:20.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40b13e0 0x7f10e410e680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f10d40032f0 tx=0x7f10d4016000).stop 2022-07-21T01:50:20.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 -- 172.21.15.140:0/3018585397 shutdown_connections 2022-07-21T01:50:20.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f10c8053b00 0x7f10c8055fc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:20.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10e40b5b30 0x7f10e41461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:20.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10e40b13e0 0x7f10e410e680 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:20.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 --2- 172.21.15.140:0/3018585397 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f10e40ace90 0x7f10e415a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:20.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 -- 172.21.15.140:0/3018585397 >> 172.21.15.140:0/3018585397 conn(0x7f10e40635b0 msgr2=0x7f10e4148660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:20.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 -- 172.21.15.140:0/3018585397 shutdown_connections 2022-07-21T01:50:20.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.027+0000 7f10e8fc2700 1 -- 172.21.15.140:0/3018585397 wait complete. 2022-07-21T01:50:20.048 INFO:tasks.cephfs_test_runner:test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-07-21T01:50:20.050 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) 2022-07-21T01:50:20.051 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2022-07-21T01:50:20.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790f415700 1 Processor -- start 2022-07-21T01:50:20.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790f415700 1 -- start start 2022-07-21T01:50:20.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790f415700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f79080b5b30 0x7f79080aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:20.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790f415700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f79080b13e0 0x7f79081463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:20.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790f415700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79080b5f10 con 0x7f79080ace90 2022-07-21T01:50:20.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790f415700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79080afc40 con 0x7f79080b5b30 2022-07-21T01:50:20.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790f415700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79080afee0 con 0x7f79080b13e0 2022-07-21T01:50:20.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790d1b1700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f79080b5b30 0x7f79080aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:20.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790d1b1700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f79080b5b30 0x7f79080aefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34158/0 (socket says 172.21.15.140:34158) 2022-07-21T01:50:20.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790d1b1700 1 -- 172.21.15.140:0/1250637563 learned_addr learned my addr 172.21.15.140:0/1250637563 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:20.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790d9b2700 1 --2- 172.21.15.140:0/1250637563 >> v2:172.21.15.152:3300/0 conn(0x7f79080b13e0 0x7f79081463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:20.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790d1b1700 1 -- 172.21.15.140:0/1250637563 >> v2:172.21.15.152:3300/0 conn(0x7f79080b13e0 msgr2=0x7f79081463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:20.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790d1b1700 1 --2- 172.21.15.140:0/1250637563 >> v2:172.21.15.152:3300/0 conn(0x7f79080b13e0 0x7f79081463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:20.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790d1b1700 1 -- 172.21.15.140:0/1250637563 >> v1:172.21.15.140:6789/0 conn(0x7f79080ace90 legacy=0x7f79080b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:20.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790d1b1700 1 -- 172.21.15.140:0/1250637563 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79080ad490 con 0x7f79080b5b30 2022-07-21T01:50:20.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790d1b1700 1 --2- 172.21.15.140:0/1250637563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f79080b5b30 0x7f79080aefb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f78fc002f70 tx=0x7f78fc006eb0).ready entity=mon.0 client_cookie=70312bf417b64bdb server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:20.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f7907fff700 1 -- 172.21.15.140:0/1250637563 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78fc00d610 con 0x7f79080b5b30 2022-07-21T01:50:20.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f7907fff700 1 -- 172.21.15.140:0/1250637563 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc000e90 con 0x7f79080b5b30 2022-07-21T01:50:20.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f7907fff700 1 -- 172.21.15.140:0/1250637563 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78fc00d9a0 con 0x7f79080b5b30 2022-07-21T01:50:20.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790f415700 1 -- 172.21.15.140:0/1250637563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f79080b5b30 msgr2=0x7f79080aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:20.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.159+0000 7f790f415700 1 --2- 172.21.15.140:0/1250637563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f79080b5b30 0x7f79080aefb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f78fc002f70 tx=0x7f78fc006eb0).stop 2022-07-21T01:50:20.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.163+0000 7f790f415700 1 -- 172.21.15.140:0/1250637563 shutdown_connections 2022-07-21T01:50:20.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.163+0000 7f790f415700 1 --2- 172.21.15.140:0/1250637563 >> v2:172.21.15.152:3300/0 conn(0x7f79080b13e0 0x7f79081463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:20.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.163+0000 7f790f415700 1 --2- 172.21.15.140:0/1250637563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f79080b5b30 0x7f79080aefb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:20.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.163+0000 7f790f415700 1 -- 172.21.15.140:0/1250637563 >> 172.21.15.140:0/1250637563 conn(0x7f79080635b0 msgr2=0x7f79080b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:20.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.163+0000 7f790f415700 1 -- 172.21.15.140:0/1250637563 shutdown_connections 2022-07-21T01:50:20.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.163+0000 7f790f415700 1 -- 172.21.15.140:0/1250637563 wait complete. 2022-07-21T01:50:20.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.163+0000 7f790f415700 1 Processor -- start 2022-07-21T01:50:20.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.163+0000 7f790f415700 1 -- start start 2022-07-21T01:50:20.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790f415700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f79080ace90 0x7f7908066200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:20.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790f415700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79080b13e0 0x7f79080673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:20.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790f415700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f79080b5b30 0x7f7908143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:20.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790f415700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7908142300 con 0x7f79080ace90 2022-07-21T01:50:20.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790f415700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7908065730 con 0x7f79080b13e0 2022-07-21T01:50:20.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790f415700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79080651f0 con 0x7f79080b5b30 2022-07-21T01:50:20.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790d1b1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f79080ace90 0x7f7908066200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:20.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790d1b1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f79080ace90 0x7f7908066200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34164/0 (socket says 172.21.15.140:34164) 2022-07-21T01:50:20.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790d1b1700 1 -- 172.21.15.140:0/1969150124 learned_addr learned my addr 172.21.15.140:0/1969150124 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:20.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790c9b0700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79080b13e0 0x7f79080673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:20.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790d9b2700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f79080b5b30 0x7f7908143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:20.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790d1b1700 1 -- 172.21.15.140:0/1969150124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f79080b5b30 msgr2=0x7f7908143790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:20.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790d1b1700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f79080b5b30 0x7f7908143790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:20.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790d1b1700 1 -- 172.21.15.140:0/1969150124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79080b13e0 msgr2=0x7f79080673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:20.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790d1b1700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79080b13e0 0x7f79080673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:20.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790d1b1700 1 -- 172.21.15.140:0/1969150124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc00b1e0 con 0x7f79080ace90 2022-07-21T01:50:20.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790d1b1700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f79080ace90 0x7f7908066200 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f7908142440 tx=0x7f78fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:20.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f79067fc700 1 -- 172.21.15.140:0/1969150124 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78fc0117e0 con 0x7f79080ace90 2022-07-21T01:50:20.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f79067fc700 1 -- 172.21.15.140:0/1969150124 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc011940 con 0x7f79080ace90 2022-07-21T01:50:20.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790f415700 1 -- 172.21.15.140:0/1969150124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7908065490 con 0x7f79080ace90 2022-07-21T01:50:20.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f79067fc700 1 -- 172.21.15.140:0/1969150124 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78fc011ca0 con 0x7f79080ace90 2022-07-21T01:50:20.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790f415700 1 -- 172.21.15.140:0/1969150124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7908159d80 con 0x7f79080ace90 2022-07-21T01:50:20.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f790f415700 1 -- 172.21.15.140:0/1969150124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79080574e0 con 0x7f79080ace90 2022-07-21T01:50:20.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.167+0000 7f79067fc700 1 -- 172.21.15.140:0/1969150124 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f78fc025030 con 0x7f79080ace90 2022-07-21T01:50:20.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.171+0000 7f79067fc700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78e8053b00 0x7f78e8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:20.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.171+0000 7f79067fc700 1 -- 172.21.15.140:0/1969150124 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f78fc06ce20 con 0x7f79080ace90 2022-07-21T01:50:20.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.171+0000 7f790c9b0700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78e8053b00 0x7f78e8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:20.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.171+0000 7f790c9b0700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78e8053b00 0x7f78e8055fc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f790000d000 tx=0x7f7900007f10).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:20.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.175+0000 7f79067fc700 1 -- 172.21.15.140:0/1969150124 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f78fc044020 con 0x7f79080ace90 2022-07-21T01:50:20.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:20.347+0000 7f790f415700 1 -- 172.21.15.140:0/1969150124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f7908066790 con 0x7f79080ace90 2022-07-21T01:50:21.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.027+0000 7f79067fc700 1 -- 172.21.15.140:0/1969150124 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v96) v1 ==== 147+0+0 (secure 0 0 0) 0x7f78fc023020 con 0x7f79080ace90 2022-07-21T01:50:21.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 -- 172.21.15.140:0/1969150124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78e8053b00 msgr2=0x7f78e8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:21.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78e8053b00 0x7f78e8055fc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f790000d000 tx=0x7f7900007f10).stop 2022-07-21T01:50:21.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 -- 172.21.15.140:0/1969150124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f79080ace90 msgr2=0x7f7908066200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:21.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f79080ace90 0x7f7908066200 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f7908142440 tx=0x7f78fc015000).stop 2022-07-21T01:50:21.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 -- 172.21.15.140:0/1969150124 shutdown_connections 2022-07-21T01:50:21.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f78e8053b00 0x7f78e8055fc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f79080b5b30 0x7f7908143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79080b13e0 0x7f79080673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 --2- 172.21.15.140:0/1969150124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f79080ace90 0x7f7908066200 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 -- 172.21.15.140:0/1969150124 >> 172.21.15.140:0/1969150124 conn(0x7f79080635b0 msgr2=0x7f79080b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:21.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 -- 172.21.15.140:0/1969150124 shutdown_connections 2022-07-21T01:50:21.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.031+0000 7f790f415700 1 -- 172.21.15.140:0/1969150124 wait complete. 2022-07-21T01:50:21.046 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:50:21.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372359e700 1 Processor -- start 2022-07-21T01:50:21.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372359e700 1 -- start start 2022-07-21T01:50:21.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f372405bf30 0x7f372405d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372359e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3724143ea0 0x7f3724144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372359e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37240b4230 con 0x7f37240adfa0 2022-07-21T01:50:21.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372359e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37240677b0 con 0x7f3724143ea0 2022-07-21T01:50:21.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37240b2950 con 0x7f372405bf30 2022-07-21T01:50:21.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f3722d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3724143ea0 0x7f3724144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f3722d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3724143ea0 0x7f3724144330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34174/0 (socket says 172.21.15.140:34174) 2022-07-21T01:50:21.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f3721d9b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f37240adfa0 0x7f372413a550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48322/0 (socket says 172.21.15.140:48322) 2022-07-21T01:50:21.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f3722d9d700 1 -- 172.21.15.140:0/2322908327 learned_addr learned my addr 172.21.15.140:0/2322908327 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:21.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372259c700 1 --2- 172.21.15.140:0/2322908327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f372405bf30 0x7f372405d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f3722d9d700 1 -- 172.21.15.140:0/2322908327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f372405bf30 msgr2=0x7f372405d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:21.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f3722d9d700 1 --2- 172.21.15.140:0/2322908327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f372405bf30 0x7f372405d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f3722d9d700 1 -- 172.21.15.140:0/2322908327 >> v1:172.21.15.140:6789/0 conn(0x7f37240adfa0 legacy=0x7f372413a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:21.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372159a700 1 -- 172.21.15.140:0/2322908327 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642934997 0 0) 0x7f37240b4230 con 0x7f37240adfa0 2022-07-21T01:50:21.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f3722d9d700 1 -- 172.21.15.140:0/2322908327 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240b3850 con 0x7f3724143ea0 2022-07-21T01:50:21.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f3722d9d700 1 --2- 172.21.15.140:0/2322908327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3724143ea0 0x7f3724144330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f3714002f70 tx=0x7f3714006eb0).ready entity=mon.0 client_cookie=e5264b6255a8ad83 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:21.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372159a700 1 -- 172.21.15.140:0/2322908327 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f371400d570 con 0x7f3724143ea0 2022-07-21T01:50:21.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372159a700 1 -- 172.21.15.140:0/2322908327 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3714000e90 con 0x7f3724143ea0 2022-07-21T01:50:21.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.175+0000 7f372159a700 1 -- 172.21.15.140:0/2322908327 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f371400d8c0 con 0x7f3724143ea0 2022-07-21T01:50:21.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 -- 172.21.15.140:0/2322908327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3724143ea0 msgr2=0x7f3724144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:21.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 --2- 172.21.15.140:0/2322908327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3724143ea0 0x7f3724144330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f3714002f70 tx=0x7f3714006eb0).stop 2022-07-21T01:50:21.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 -- 172.21.15.140:0/2322908327 shutdown_connections 2022-07-21T01:50:21.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 --2- 172.21.15.140:0/2322908327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3724143ea0 0x7f3724144330 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 --2- 172.21.15.140:0/2322908327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f372405bf30 0x7f372405d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 -- 172.21.15.140:0/2322908327 >> 172.21.15.140:0/2322908327 conn(0x7f3724064b90 msgr2=0x7f3724064f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:21.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 -- 172.21.15.140:0/2322908327 shutdown_connections 2022-07-21T01:50:21.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 -- 172.21.15.140:0/2322908327 wait complete. 2022-07-21T01:50:21.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 Processor -- start 2022-07-21T01:50:21.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 -- start start 2022-07-21T01:50:21.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372405bf30 0x7f372414fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37240adfa0 0x7f37240b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3724143ea0 0x7f372413b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3724141e40 con 0x7f37240adfa0 2022-07-21T01:50:21.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37241437a0 con 0x7f372405bf30 2022-07-21T01:50:21.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3724143a40 con 0x7f3724143ea0 2022-07-21T01:50:21.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f3721d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37240adfa0 0x7f37240b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f3721d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37240adfa0 0x7f37240b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34180/0 (socket says 172.21.15.140:34180) 2022-07-21T01:50:21.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f372259c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372405bf30 0x7f372414fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f3721d9b700 1 -- 172.21.15.140:0/2735649563 learned_addr learned my addr 172.21.15.140:0/2735649563 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:21.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.179+0000 7f3722d9d700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3724143ea0 0x7f372413b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f3721d9b700 1 -- 172.21.15.140:0/2735649563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3724143ea0 msgr2=0x7f372413b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:21.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f3721d9b700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3724143ea0 0x7f372413b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f3721d9b700 1 -- 172.21.15.140:0/2735649563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372405bf30 msgr2=0x7f372414fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:21.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f3721d9b700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372405bf30 0x7f372414fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f3721d9b700 1 -- 172.21.15.140:0/2735649563 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371400b1e0 con 0x7f37240adfa0 2022-07-21T01:50:21.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f372259c700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372405bf30 0x7f372414fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:21.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f3721d9b700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37240adfa0 0x7f37240b5420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f370c000ed0 tx=0x7f370c007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:21.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f371b7fe700 1 -- 172.21.15.140:0/2735649563 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370c0070d0 con 0x7f37240adfa0 2022-07-21T01:50:21.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f371b7fe700 1 -- 172.21.15.140:0/2735649563 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c007230 con 0x7f37240adfa0 2022-07-21T01:50:21.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f372359e700 1 -- 172.21.15.140:0/2735649563 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3724143420 con 0x7f37240adfa0 2022-07-21T01:50:21.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f371b7fe700 1 -- 172.21.15.140:0/2735649563 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f370c005740 con 0x7f37240adfa0 2022-07-21T01:50:21.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f372359e700 1 -- 172.21.15.140:0/2735649563 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37240b5910 con 0x7f37240adfa0 2022-07-21T01:50:21.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f372359e700 1 -- 172.21.15.140:0/2735649563 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37240b3af0 con 0x7f37240adfa0 2022-07-21T01:50:21.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.183+0000 7f371b7fe700 1 -- 172.21.15.140:0/2735649563 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f370c01a060 con 0x7f37240adfa0 2022-07-21T01:50:21.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.187+0000 7f371b7fe700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36fc053b00 0x7f36fc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.187+0000 7f371b7fe700 1 -- 172.21.15.140:0/2735649563 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f370c06b530 con 0x7f37240adfa0 2022-07-21T01:50:21.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.187+0000 7f372259c700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36fc053b00 0x7f36fc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.187+0000 7f372259c700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36fc053b00 0x7f36fc055fc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f371c00b000 tx=0x7f371c0091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:21.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.191+0000 7f371b7fe700 1 -- 172.21.15.140:0/2735649563 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f370c042620 con 0x7f37240adfa0 2022-07-21T01:50:21.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.319+0000 7f372359e700 1 -- 172.21.15.140:0/2735649563 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f37240574e0 con 0x7f37240adfa0 2022-07-21T01:50:21.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.327+0000 7f371b7fe700 1 -- 172.21.15.140:0/2735649563 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f370c018020 con 0x7f37240adfa0 2022-07-21T01:50:21.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 -- 172.21.15.140:0/2735649563 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36fc053b00 msgr2=0x7f36fc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:21.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36fc053b00 0x7f36fc055fc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f371c00b000 tx=0x7f371c0091d0).stop 2022-07-21T01:50:21.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 -- 172.21.15.140:0/2735649563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37240adfa0 msgr2=0x7f37240b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:21.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37240adfa0 0x7f37240b5420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f370c000ed0 tx=0x7f370c007a90).stop 2022-07-21T01:50:21.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 -- 172.21.15.140:0/2735649563 shutdown_connections 2022-07-21T01:50:21.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36fc053b00 0x7f36fc055fc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3724143ea0 0x7f372413b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37240adfa0 0x7f37240b5420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 --2- 172.21.15.140:0/2735649563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f372405bf30 0x7f372414fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 -- 172.21.15.140:0/2735649563 >> 172.21.15.140:0/2735649563 conn(0x7f3724064b90 msgr2=0x7f372413dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:21.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 -- 172.21.15.140:0/2735649563 shutdown_connections 2022-07-21T01:50:21.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.331+0000 7f372359e700 1 -- 172.21.15.140:0/2735649563 wait complete. 2022-07-21T01:50:21.351 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:50:21.353 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:50:21.370 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:50:21.394 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:50:21.418 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:50:21.439 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:50:21.463 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:50:21.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:21.464 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:21.464 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:21.472 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:50:21.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:50:21.475 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:50:21.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:21.476 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:21.477 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:50:21.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:50:21.495 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:50:21.495 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:50:21.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:21.496 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:21.497 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:50:21.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:50:21.512 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:50:21.512 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:50:21.513 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:50:21.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 Processor -- start 2022-07-21T01:50:21.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 -- start start 2022-07-21T01:50:21.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1d2c0ac9d0 0x7f1d2c13e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 0x7f1d2c13ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d2c0b0080 0x7f1d2c0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2c0af260 con 0x7f1d2c0ac9d0 2022-07-21T01:50:21.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2c0acdb0 con 0x7f1d2c0b0080 2022-07-21T01:50:21.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0b1450 con 0x7f1d2c0aedc0 2022-07-21T01:50:21.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d3084b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1d2c0ac9d0 0x7f1d2c13e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d23fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 0x7f1d2c13ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d3104c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d2c0b0080 0x7f1d2c0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d23fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 0x7f1d2c13ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:59148/0 (socket says 172.21.15.140:59148) 2022-07-21T01:50:21.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d3104c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d2c0b0080 0x7f1d2c0b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:39926/0 (socket says 172.21.15.140:39926) 2022-07-21T01:50:21.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d23fff700 1 -- 172.21.15.140:0/2814452619 learned_addr learned my addr 172.21.15.140:0/2814452619 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:21.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d3104c700 1 -- 172.21.15.140:0/2814452619 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 msgr2=0x7f1d2c13ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:21.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d3104c700 1 --2- 172.21.15.140:0/2814452619 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 0x7f1d2c13ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d3104c700 1 -- 172.21.15.140:0/2814452619 >> v2:172.21.15.140:3300/0 conn(0x7f1d2c0ac9d0 msgr2=0x7f1d2c13e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:21.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d3104c700 1 --2- 172.21.15.140:0/2814452619 >> v2:172.21.15.140:3300/0 conn(0x7f1d2c0ac9d0 0x7f1d2c13e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d3104c700 1 -- 172.21.15.140:0/2814452619 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c0af450 con 0x7f1d2c0b0080 2022-07-21T01:50:21.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d3084b700 1 --2- 172.21.15.140:0/2814452619 >> v2:172.21.15.140:3300/0 conn(0x7f1d2c0ac9d0 0x7f1d2c13e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:21.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d3104c700 1 --2- 172.21.15.140:0/2814452619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2c0b0080 0x7f1d2c0b1b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f1d240032f0 tx=0x7f1d240091d0).ready entity=mon.1 client_cookie=92e4345cbcc51c1a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:21.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d237fe700 1 -- 172.21.15.140:0/2814452619 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d2400b600 con 0x7f1d2c0b0080 2022-07-21T01:50:21.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d237fe700 1 -- 172.21.15.140:0/2814452619 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2400b760 con 0x7f1d2c0b0080 2022-07-21T01:50:21.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d237fe700 1 -- 172.21.15.140:0/2814452619 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d2400b8d0 con 0x7f1d2c0b0080 2022-07-21T01:50:21.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 -- 172.21.15.140:0/2814452619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2c0b0080 msgr2=0x7f1d2c0b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:21.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 --2- 172.21.15.140:0/2814452619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2c0b0080 0x7f1d2c0b1b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f1d240032f0 tx=0x7f1d240091d0).stop 2022-07-21T01:50:21.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 -- 172.21.15.140:0/2814452619 shutdown_connections 2022-07-21T01:50:21.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 --2- 172.21.15.140:0/2814452619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2c0b0080 0x7f1d2c0b1b30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 --2- 172.21.15.140:0/2814452619 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 0x7f1d2c13ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 --2- 172.21.15.140:0/2814452619 >> v2:172.21.15.140:3300/0 conn(0x7f1d2c0ac9d0 0x7f1d2c13e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 -- 172.21.15.140:0/2814452619 >> 172.21.15.140:0/2814452619 conn(0x7f1d2c0611c0 msgr2=0x7f1d2c062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:21.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 -- 172.21.15.140:0/2814452619 shutdown_connections 2022-07-21T01:50:21.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.623+0000 7f1d32aaf700 1 -- 172.21.15.140:0/2814452619 wait complete. 2022-07-21T01:50:21.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d32aaf700 1 Processor -- start 2022-07-21T01:50:21.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d32aaf700 1 -- start start 2022-07-21T01:50:21.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d32aaf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 0x7f1d2c0a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d32aaf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2c0b0080 0x7f1d2c153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d32aaf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d2c0a4f80 0x7f1d2c153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d32aaf700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0a2bd0 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d32aaf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0a2e70 con 0x7f1d2c0b0080 2022-07-21T01:50:21.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d32aaf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c0a3110 con 0x7f1d2c0aedc0 2022-07-21T01:50:21.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d3104c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d2c0a4f80 0x7f1d2c153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d3104c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d2c0a4f80 0x7f1d2c153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34196/0 (socket says 172.21.15.140:34196) 2022-07-21T01:50:21.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d3104c700 1 -- 172.21.15.140:0/3110669076 learned_addr learned my addr 172.21.15.140:0/3110669076 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:21.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d23fff700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2c0b0080 0x7f1d2c153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d3084b700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 0x7f1d2c0a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d3104c700 1 -- 172.21.15.140:0/3110669076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 msgr2=0x7f1d2c0a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:21.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d3104c700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 0x7f1d2c0a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d3104c700 1 -- 172.21.15.140:0/3110669076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2c0b0080 msgr2=0x7f1d2c153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:21.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d3104c700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2c0b0080 0x7f1d2c153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:21.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d3104c700 1 -- 172.21.15.140:0/3110669076 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2800b180 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d3084b700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 0x7f1d2c0a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:21.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d3104c700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d2c0a4f80 0x7f1d2c153fc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f1d240032f0 tx=0x7f1d24016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:21.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d21ffb700 1 -- 172.21.15.140:0/3110669076 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d2400b600 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d21ffb700 1 -- 172.21.15.140:0/3110669076 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2401f5e0 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d32aaf700 1 -- 172.21.15.140:0/3110669076 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d24009470 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d21ffb700 1 -- 172.21.15.140:0/3110669076 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d2401f940 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.627+0000 7f1d32aaf700 1 -- 172.21.15.140:0/3110669076 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2c153520 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.631+0000 7f1d32aaf700 1 -- 172.21.15.140:0/3110669076 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2c0a4240 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.631+0000 7f1d21ffb700 1 -- 172.21.15.140:0/3110669076 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d24026030 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.631+0000 7f1d21ffb700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d0c0539e0 0x7f1d0c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:21.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.631+0000 7f1d21ffb700 1 -- 172.21.15.140:0/3110669076 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f1d2406c6d0 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.631+0000 7f1d3084b700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d0c0539e0 0x7f1d0c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:21.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.631+0000 7f1d3084b700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d0c0539e0 0x7f1d0c055ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f1d2800b000 tx=0x7f1d28008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:21.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.639+0000 7f1d21ffb700 1 -- 172.21.15.140:0/3110669076 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d2c0a4240 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.763+0000 7f1d32aaf700 1 -- 172.21.15.140:0/3110669076 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f1d2c0b2020 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.763+0000 7f1d21ffb700 1 -- 172.21.15.140:0/3110669076 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v35) v1 ==== 81+0+28925 (secure 0 0 0) 0x7f1d24024020 con 0x7f1d2c0a4f80 2022-07-21T01:50:21.768 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:21.769 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:50:21.769 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 35, 2022-07-21T01:50:21.769 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:50:21.770 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:50:21.770 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:50:19.018017+0000", 2022-07-21T01:50:21.771 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:50:21.771 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:50:21.772 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:50:21.772 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:50:21.773 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:50:21.773 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:50:21.774 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:50:21.774 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:50:21.774 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:50:21.775 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:50:21.775 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:50:21.776 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:50:21.776 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:50:21.776 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:50:21.777 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:50:21.777 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 6, 2022-07-21T01:50:21.778 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:50:21.778 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:50:21.778 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:50:21.779 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:50:21.779 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:50:21.780 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.780 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:50:21.781 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:50:21.781 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:50:21.781 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:50:21.782 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:50:21.782 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:50:21.783 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:50:21.783 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:50:21.784 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:50:21.784 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:50:21.784 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:50:21.785 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:50:21.785 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:50:21.786 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:50:21.786 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:50:21.786 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:50:21.787 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:50:21.787 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:50:21.787 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:50:21.788 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:50:21.788 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:50:21.788 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:50:21.789 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:50:21.789 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.790 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:50:21.790 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:50:21.790 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:50:21.791 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:50:21.791 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:50:21.791 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:50:21.791 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:50:21.792 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:50:21.792 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:50:21.792 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:50:21.793 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:50:21.793 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:50:21.793 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:50:21.794 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:50:21.794 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:50:21.794 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:50:21.795 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:50:21.795 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:50:21.795 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:50:21.796 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:50:21.796 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:50:21.797 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:50:21.797 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:50:21.797 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:50:21.798 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:50:21.798 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:50:21.799 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:50:21.799 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.799 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:50:21.800 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:50:21.800 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:50:21.801 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:50:21.801 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:50:21.801 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:50:21.802 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:50:21.802 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:50:21.803 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:50:21.803 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:50:21.803 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:50:21.804 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:50:21.804 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:50:21.804 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.805 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:50:21.805 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:50:21.805 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.806 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.806 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.807 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:50:21.807 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:50:21.807 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:50:21.808 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:50:21.808 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:50:21.808 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:50:21.809 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:50:21.809 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:50:21.809 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:50:21.809 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:50:21.809 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:50:21.810 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:50:21.810 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:50:21.811 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:50:21.811 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:50:21.812 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:50:21.812 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:50:21.813 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:50:21.813 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:50:21.813 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:50:21.814 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:50:21.814 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:50:21.815 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:50:21.815 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.816 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:50:21.816 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:50:21.817 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:50:21.817 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:50:21.817 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:50:21.818 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:50:21.818 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:50:21.819 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:50:21.819 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:50:21.820 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:50:21.820 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:50:21.821 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:50:21.821 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:50:21.822 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:50:21.822 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:50:21.822 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:50:21.823 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:50:21.823 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:50:21.824 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:50:21.824 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:50:21.825 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:50:21.825 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:50:21.826 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:50:21.826 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:50:21.826 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:50:21.827 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:50:21.827 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:50:21.828 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.828 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:50:21.829 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:50:21.829 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:50:21.830 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:50:21.830 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:50:21.831 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:50:21.831 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:50:21.831 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:50:21.832 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:50:21.832 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:50:21.833 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:50:21.833 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:50:21.834 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:50:21.834 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:50:21.834 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.835 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.835 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:50:21.836 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:50:21.836 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.836 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:50:21.837 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:50:21.837 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:21.838 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:21.838 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:21.838 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:21.838 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:21.839 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:21.839 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:21.839 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:21.839 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:21.840 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:21.840 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:21.840 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.840 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.841 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.841 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:50:21.841 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:21.841 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.842 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.842 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.843 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:50:21.843 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:21.844 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.844 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.904 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.904 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:21.904 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.905 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.905 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.905 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:50:21.905 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:21.905 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.906 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.906 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.906 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:50:21.906 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:21.907 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.907 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.907 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.907 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:21.907 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.908 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.908 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.908 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:50:21.908 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:21.909 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.909 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.909 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.909 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:50:21.909 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:21.910 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.910 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.910 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.910 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:21.911 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.911 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.911 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.911 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:50:21.911 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:21.912 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.912 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.912 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.912 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:50:21.913 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:50:21.913 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.913 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.913 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.914 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:50:21.914 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:50:21.915 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:50:21.915 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:50:21.915 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:21.915 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:21.915 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:21.916 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.916 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.916 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.916 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:50:21.917 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:50:21.917 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:21.917 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:21.917 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:21.918 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:21.918 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:21.918 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:21.918 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:21.919 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:21.919 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:21.919 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:21.919 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:21.920 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.920 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.920 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.920 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:50:21.921 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:21.921 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.921 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.921 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.921 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:50:21.922 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:21.922 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.922 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.922 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.923 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:21.923 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.923 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.923 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.924 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:50:21.924 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:21.924 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.925 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.925 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.925 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:50:21.925 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:21.926 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.926 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.926 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.926 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:21.926 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.927 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.927 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.927 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:50:21.927 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:21.928 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.928 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.928 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.928 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:50:21.929 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:21.929 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.929 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.929 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.929 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:21.930 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.931 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.931 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.931 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:50:21.931 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:21.932 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.932 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.932 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.932 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:50:21.933 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:50:21.933 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.933 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.933 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.933 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:50:21.934 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:50:21.934 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:50:21.934 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:50:21.934 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:21.935 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:21.935 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:21.936 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.936 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.936 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.936 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:50:21.937 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:50:21.937 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:21.937 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:21.937 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:21.937 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:21.938 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:21.938 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:21.938 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:21.938 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 29, 2022-07-21T01:50:21.939 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:21.939 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:21.939 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:21.939 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.939 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.940 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.940 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:50:21.940 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:21.940 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.940 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.941 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.941 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:50:21.941 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:21.941 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.942 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.942 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.942 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:21.942 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.942 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.943 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.943 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:50:21.943 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:21.943 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.944 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.944 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.944 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:50:21.944 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:21.944 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.945 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.945 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.946 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:21.946 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.946 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.946 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.947 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:50:21.947 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:21.947 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.947 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.947 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.948 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:50:21.948 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:21.948 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.948 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.949 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.949 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:21.949 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.949 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.949 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.950 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:50:21.950 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:21.950 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.950 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.951 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.951 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:50:21.951 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:50:21.952 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.952 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.952 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.953 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:50:21.953 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:50:21.953 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:50:21.953 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:50:21.954 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:21.954 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:21.954 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:21.954 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.954 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.955 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.955 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:50:21.955 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:50:21.955 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:21.956 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:21.956 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:21.957 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:21.957 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:21.957 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:21.957 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:21.957 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:21.958 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:21.958 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:21.958 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:21.958 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.959 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.959 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.959 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:50:21.959 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:21.959 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.960 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.960 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.960 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:50:21.960 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:21.961 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.961 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.967 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.967 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:21.967 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.967 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.968 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.968 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:50:21.968 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:21.968 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.969 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.969 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.969 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:50:21.969 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:21.969 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.970 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.970 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.970 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:21.970 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.970 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.971 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.971 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:50:21.971 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:21.971 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.972 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.972 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.972 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:50:21.972 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:21.972 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.973 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.973 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.973 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:21.973 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.974 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.974 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.974 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:50:21.974 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:21.974 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.975 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.975 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.975 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:50:21.975 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:50:21.976 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.976 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.976 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.976 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:50:21.977 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:50:21.977 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:50:21.978 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:50:21.978 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:21.978 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:21.978 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:21.979 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.979 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.979 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.979 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:50:21.980 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:50:21.980 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:21.980 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:21.980 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:21.980 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:21.981 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:21.981 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:21.981 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:21.981 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:21.981 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:21.982 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:21.982 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:21.982 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.982 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.982 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.983 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:50:21.983 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:21.983 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.983 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.984 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.984 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:50:21.984 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:21.984 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.985 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.985 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.985 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:21.985 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.986 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.986 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.986 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:50:21.986 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:21.986 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.987 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.987 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.988 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:50:21.988 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:21.988 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.988 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.988 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.989 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:21.989 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.989 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.989 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.989 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:50:21.990 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:21.990 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.990 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.990 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.991 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:50:21.991 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:21.991 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.991 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.991 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.992 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:21.992 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:21.992 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.992 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:21.993 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:50:21.993 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:21.993 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.993 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.993 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:21.994 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:50:21.994 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:50:21.994 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:21.995 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.995 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.995 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:50:21.995 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:50:21.995 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:50:21.996 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:50:21.996 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:21.996 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:21.996 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:21.996 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:21.997 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:21.997 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:21.997 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:50:21.997 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:50:21.998 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:21.999 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:21.999 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:21.999 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:21.999 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:22.000 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:22.000 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:22.000 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:22.000 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:22.001 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:22.001 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:22.001 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.001 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.001 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.002 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:50:22.002 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:22.002 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.002 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.003 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.003 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:50:22.003 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:22.003 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.003 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.004 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.004 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:22.004 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.004 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.005 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.005 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:50:22.005 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:22.005 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.005 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.006 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.006 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:50:22.006 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:22.006 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.007 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.007 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.007 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:22.007 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.007 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.008 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.008 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:50:22.009 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:22.009 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.009 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.009 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.010 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:50:22.010 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:22.010 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.010 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.011 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.011 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:22.011 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.011 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.012 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.012 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:50:22.012 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:22.012 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.012 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.013 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.013 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:50:22.013 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:50:22.013 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.014 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.014 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.014 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:50:22.014 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:50:22.015 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:50:22.015 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:50:22.015 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:22.015 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:22.015 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:22.016 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.016 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.016 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.016 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:50:22.016 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:50:22.017 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:22.017 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:22.017 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:22.017 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:22.018 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:22.018 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:22.018 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:22.018 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 29, 2022-07-21T01:50:22.019 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:22.019 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:22.020 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:22.020 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.020 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.020 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.020 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:50:22.021 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:22.021 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.021 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.021 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.022 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:50:22.022 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:22.022 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.022 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.023 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.023 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:22.023 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.024 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.024 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.024 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:50:22.024 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:22.024 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.025 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.025 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.025 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:50:22.025 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:22.026 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.026 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.026 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.026 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:22.026 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.027 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.027 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.027 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:50:22.027 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:22.027 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.028 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.028 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.028 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:50:22.028 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:22.028 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.029 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.029 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.029 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:22.029 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.030 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.030 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.031 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:50:22.031 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:22.031 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.031 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.031 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.032 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:50:22.032 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:50:22.032 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.032 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.033 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.033 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:50:22.033 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:50:22.033 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:50:22.033 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:50:22.034 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:22.034 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:22.034 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:22.034 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.035 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.035 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.035 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:50:22.035 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:50:22.036 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:50:22.036 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:50:22.036 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:50:22.036 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:50:22.036 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:50:22.037 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:50:22.037 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:50:22.037 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 30, 2022-07-21T01:50:22.037 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:50:22.038 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:50:22.038 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:50:22.038 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.038 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.038 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.039 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:50:22.039 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:22.039 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.039 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.040 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.040 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:50:22.040 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:22.041 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.041 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.041 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.041 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:50:22.042 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.043 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.045 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.045 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:50:22.046 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:22.046 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.047 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.047 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.047 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:50:22.048 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:22.048 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.049 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.049 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.050 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:50:22.050 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.051 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.051 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.052 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:50:22.052 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:22.052 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.053 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.053 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.054 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:50:22.054 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:22.055 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.055 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.056 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.056 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:50:22.057 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:50:22.057 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.058 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:50:22.058 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:50:22.058 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:22.059 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.059 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.060 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:50:22.060 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:50:22.061 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:50:22.061 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.062 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.065 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.065 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:50:22.066 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:50:22.066 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:50:22.067 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:50:22.067 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:50:22.068 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:50:22.068 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:50:22.068 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:22.069 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.069 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:50:22.070 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:50:22.070 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.071 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:50:22.071 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:22.071 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:22.073 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:22.073 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:22.073 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:22.075 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:50:22.075 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:22.076 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.076 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.076 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:50:22.077 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:22.077 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:22.077 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:22.078 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:22.079 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:22.079 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:50:22.080 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:22.080 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.081 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.081 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:50:22.081 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:22.082 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:22.082 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:22.083 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:22.083 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:22.083 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:50:22.084 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:22.084 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.085 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.085 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:50:22.086 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:22.086 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:22.087 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:22.087 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:22.088 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:22.088 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:50:22.088 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:22.089 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.089 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.090 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:50:22.090 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:22.091 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:22.091 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:22.092 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:22.092 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:22.093 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:50:22.093 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:22.093 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.094 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.094 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:50:22.095 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:22.095 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:22.096 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:22.096 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:22.098 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:22.098 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:50:22.098 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:22.099 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.099 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.099 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:50:22.100 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:22.100 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:22.100 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:22.100 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:22.101 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:22.101 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:50:22.101 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:22.102 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.102 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:22.102 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:50:22.103 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:50:22.103 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:50:22.103 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:50:22.103 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:50:22.104 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:50:22.104 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:50:22.104 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:50:22.104 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.105 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:50:22.105 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:50:22.105 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:50:22.105 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:50:22.106 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:50:22.106 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:50:22.106 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6833/1420650762": "2022-07-22T01:50:15.082054+0000", 2022-07-21T01:50:22.107 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6832/1420650762": "2022-07-22T01:50:15.082054+0000" 2022-07-21T01:50:22.107 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.108 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:50:22.108 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:50:22.108 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:50:22.108 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:50:22.109 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:50:22.109 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:50:22.109 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:50:22.110 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:50:22.110 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:22.110 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:22.110 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:50:22.111 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:50:22.111 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:50:22.111 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:50:22.111 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:50:22.112 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:50:22.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 -- 172.21.15.140:0/3110669076 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d0c0539e0 msgr2=0x7f1d0c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d0c0539e0 0x7f1d0c055ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f1d2800b000 tx=0x7f1d28008040).stop 2022-07-21T01:50:22.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 -- 172.21.15.140:0/3110669076 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d2c0a4f80 msgr2=0x7f1d2c153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d2c0a4f80 0x7f1d2c153fc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f1d240032f0 tx=0x7f1d24016000).stop 2022-07-21T01:50:22.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 -- 172.21.15.140:0/3110669076 shutdown_connections 2022-07-21T01:50:22.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d0c0539e0 0x7f1d0c055ea0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d2c0a4f80 0x7f1d2c153fc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d2c0b0080 0x7f1d2c153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 --2- 172.21.15.140:0/3110669076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d2c0aedc0 0x7f1d2c0a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 -- 172.21.15.140:0/3110669076 >> 172.21.15.140:0/3110669076 conn(0x7f1d2c0ac9d0 msgr2=0x7f1d2c0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:22.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 -- 172.21.15.140:0/3110669076 shutdown_connections 2022-07-21T01:50:22.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:21.767+0000 7f1d32aaf700 1 -- 172.21.15.140:0/3110669076 wait complete. 2022-07-21T01:50:22.118 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:50:22.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.227+0000 7ff829e65700 1 Processor -- start 2022-07-21T01:50:22.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.227+0000 7ff829e65700 1 -- start start 2022-07-21T01:50:22.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff82413c790 0x7ff8240b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:22.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8240b3ca0 0x7ff8240b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:22.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff824142ba0 con 0x7ff8240b9f00 2022-07-21T01:50:22.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8241429b0 con 0x7ff82413c790 2022-07-21T01:50:22.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff824146330 con 0x7ff8240b3ca0 2022-07-21T01:50:22.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff828e63700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff82413c790 0x7ff8240b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:22.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff828e63700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff82413c790 0x7ff8240b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34202/0 (socket says 172.21.15.140:34202) 2022-07-21T01:50:22.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff828e63700 1 -- 172.21.15.140:0/64506047 learned_addr learned my addr 172.21.15.140:0/64506047 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:22.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff823fff700 1 --2- 172.21.15.140:0/64506047 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8240b3ca0 0x7ff8240b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:22.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff8237fe700 1 -- 172.21.15.140:0/64506047 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027071624 0 0) 0x7ff824142ba0 con 0x7ff8240b9f00 2022-07-21T01:50:22.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff828e63700 1 -- 172.21.15.140:0/64506047 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8240b3ca0 msgr2=0x7ff8240b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:22.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff828e63700 1 --2- 172.21.15.140:0/64506047 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8240b3ca0 0x7ff8240b9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff828e63700 1 -- 172.21.15.140:0/64506047 >> v1:172.21.15.152:6789/0 conn(0x7ff8240b9f00 legacy=0x7ff824143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff828e63700 1 -- 172.21.15.140:0/64506047 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8240b4c00 con 0x7ff82413c790 2022-07-21T01:50:22.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff828e63700 1 --2- 172.21.15.140:0/64506047 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff82413c790 0x7ff8240b4710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7ff818002f70 tx=0x7ff818006eb0).ready entity=mon.0 client_cookie=a4fadd620fcfdabf server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:22.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff8237fe700 1 -- 172.21.15.140:0/64506047 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff81800d570 con 0x7ff82413c790 2022-07-21T01:50:22.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff8237fe700 1 -- 172.21.15.140:0/64506047 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff818000e90 con 0x7ff82413c790 2022-07-21T01:50:22.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff8237fe700 1 -- 172.21.15.140:0/64506047 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff81800d8c0 con 0x7ff82413c790 2022-07-21T01:50:22.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 -- 172.21.15.140:0/64506047 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff82413c790 msgr2=0x7ff8240b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 --2- 172.21.15.140:0/64506047 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff82413c790 0x7ff8240b4710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7ff818002f70 tx=0x7ff818006eb0).stop 2022-07-21T01:50:22.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 -- 172.21.15.140:0/64506047 shutdown_connections 2022-07-21T01:50:22.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 --2- 172.21.15.140:0/64506047 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8240b3ca0 0x7ff8240b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 --2- 172.21.15.140:0/64506047 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff82413c790 0x7ff8240b4710 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 -- 172.21.15.140:0/64506047 >> 172.21.15.140:0/64506047 conn(0x7ff8240678b0 msgr2=0x7ff8240b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:22.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 -- 172.21.15.140:0/64506047 shutdown_connections 2022-07-21T01:50:22.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.231+0000 7ff829e65700 1 -- 172.21.15.140:0/64506047 wait complete. 2022-07-21T01:50:22.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff829e65700 1 Processor -- start 2022-07-21T01:50:22.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff829e65700 1 -- start start 2022-07-21T01:50:22.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff829e65700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8240b3ca0 0x7ff824151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:22.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff829e65700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8240b9f00 0x7ff8240b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:22.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff829e65700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff82413c790 0x7ff824143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:22.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff829e65700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82413ab60 con 0x7ff8240b9f00 2022-07-21T01:50:22.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff829e65700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff824140200 con 0x7ff82413c790 2022-07-21T01:50:22.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff829e65700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82413ee20 con 0x7ff8240b3ca0 2022-07-21T01:50:22.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff823fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8240b9f00 0x7ff8240b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:22.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff828e63700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8240b3ca0 0x7ff824151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:22.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff823fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8240b9f00 0x7ff8240b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34210/0 (socket says 172.21.15.140:34210) 2022-07-21T01:50:22.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff823fff700 1 -- 172.21.15.140:0/3492914489 learned_addr learned my addr 172.21.15.140:0/3492914489 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:22.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff829664700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff82413c790 0x7ff824143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:22.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff823fff700 1 -- 172.21.15.140:0/3492914489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8240b3ca0 msgr2=0x7ff824151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff823fff700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8240b3ca0 0x7ff824151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff823fff700 1 -- 172.21.15.140:0/3492914489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff82413c790 msgr2=0x7ff824143260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff823fff700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff82413c790 0x7ff824143260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff823fff700 1 -- 172.21.15.140:0/3492914489 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81800b1e0 con 0x7ff8240b9f00 2022-07-21T01:50:22.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.235+0000 7ff828e63700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8240b3ca0 0x7ff824151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:22.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.239+0000 7ff823fff700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8240b9f00 0x7ff8240b1720 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ff81400c000 tx=0x7ff814006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:22.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.239+0000 7ff821ffb700 1 -- 172.21.15.140:0/3492914489 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff814010780 con 0x7ff8240b9f00 2022-07-21T01:50:22.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.239+0000 7ff821ffb700 1 -- 172.21.15.140:0/3492914489 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8140108e0 con 0x7ff8240b9f00 2022-07-21T01:50:22.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.239+0000 7ff821ffb700 1 -- 172.21.15.140:0/3492914489 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff814010c40 con 0x7ff8240b9f00 2022-07-21T01:50:22.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.239+0000 7ff829e65700 1 -- 172.21.15.140:0/3492914489 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82413fcc0 con 0x7ff8240b9f00 2022-07-21T01:50:22.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.239+0000 7ff829e65700 1 -- 172.21.15.140:0/3492914489 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82413f820 con 0x7ff8240b9f00 2022-07-21T01:50:22.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.239+0000 7ff829e65700 1 -- 172.21.15.140:0/3492914489 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff824146090 con 0x7ff8240b9f00 2022-07-21T01:50:22.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.239+0000 7ff821ffb700 1 -- 172.21.15.140:0/3492914489 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff81401c460 con 0x7ff8240b9f00 2022-07-21T01:50:22.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.239+0000 7ff821ffb700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff804053950 0x7ff804055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:22.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.239+0000 7ff821ffb700 1 -- 172.21.15.140:0/3492914489 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5339+0+0 (secure 0 0 0) 0x7ff81406c0d0 con 0x7ff8240b9f00 2022-07-21T01:50:22.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.239+0000 7ff828e63700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff804053950 0x7ff804055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:22.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.243+0000 7ff828e63700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff804053950 0x7ff804055e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7ff81801c000 tx=0x7ff818015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:22.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.247+0000 7ff821ffb700 1 -- 172.21.15.140:0/3492914489 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff8140431d0 con 0x7ff8240b9f00 2022-07-21T01:50:22.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 -- 172.21.15.140:0/3492914489 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8240574e0 con 0x7ff8240b9f00 2022-07-21T01:50:22.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff821ffb700 1 -- 172.21.15.140:0/3492914489 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+1211 (secure 0 0 0) 0x7ff8140467f0 con 0x7ff8240b9f00 2022-07-21T01:50:22.425 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:22.425 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/838095894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":838095894},{"type":"v1","addr":"172.21.15.140:6835","nonce":838095894}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1530597349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1530597349},{"type":"v1","addr":"172.21.15.152:6833","nonce":1530597349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":12}],"filesystems":[]} 2022-07-21T01:50:22.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 -- 172.21.15.140:0/3492914489 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff804053950 msgr2=0x7ff804055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff804053950 0x7ff804055e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7ff81801c000 tx=0x7ff818015000).stop 2022-07-21T01:50:22.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 -- 172.21.15.140:0/3492914489 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8240b9f00 msgr2=0x7ff8240b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8240b9f00 0x7ff8240b1720 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7ff81400c000 tx=0x7ff814006df0).stop 2022-07-21T01:50:22.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 -- 172.21.15.140:0/3492914489 shutdown_connections 2022-07-21T01:50:22.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff804053950 0x7ff804055e10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff82413c790 0x7ff824143260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8240b9f00 0x7ff8240b1720 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 --2- 172.21.15.140:0/3492914489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8240b3ca0 0x7ff824151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 -- 172.21.15.140:0/3492914489 >> 172.21.15.140:0/3492914489 conn(0x7ff8240678b0 msgr2=0x7ff824146730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:22.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 -- 172.21.15.140:0/3492914489 shutdown_connections 2022-07-21T01:50:22.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.423+0000 7ff829e65700 1 -- 172.21.15.140:0/3492914489 wait complete. 2022-07-21T01:50:22.434 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 12 2022-07-21T01:50:22.441 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:50:22.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 Processor -- start 2022-07-21T01:50:22.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 -- start start 2022-07-21T01:50:22.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f00b01424e0 0x7f00b00b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:22.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b01433d0 con 0x7f00b00aea10 2022-07-21T01:50:22.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b0143670 con 0x7f00b00b0570 2022-07-21T01:50:22.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b0143b10 con 0x7f00b01424e0 2022-07-21T01:50:22.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00af7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f00b01424e0 0x7f00b00b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:22.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00af7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f00b01424e0 0x7f00b00b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34216/0 (socket says 172.21.15.140:34216) 2022-07-21T01:50:22.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00af7fe700 1 -- 172.21.15.140:0/2567984998 learned_addr learned my addr 172.21.15.140:0/2567984998 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:22.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00af7fe700 1 -- 172.21.15.140:0/2567984998 >> v1:172.21.15.152:6789/0 conn(0x7f00b00b0570 legacy=0x7f00b0141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00af7fe700 1 -- 172.21.15.140:0/2567984998 >> v1:172.21.15.140:6789/0 conn(0x7f00b00aea10 legacy=0x7f00b00ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00af7fe700 1 -- 172.21.15.140:0/2567984998 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00b1010 con 0x7f00b01424e0 2022-07-21T01:50:22.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00af7fe700 1 --2- 172.21.15.140:0/2567984998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00b01424e0 0x7f00b00b0950 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f00a4002f70 tx=0x7f00a4006eb0).ready entity=mon.0 client_cookie=3b89b43f2d0b593a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:22.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00aeffd700 1 -- 172.21.15.140:0/2567984998 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00a400d660 con 0x7f00b01424e0 2022-07-21T01:50:22.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00aeffd700 1 -- 172.21.15.140:0/2567984998 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a4000e90 con 0x7f00b01424e0 2022-07-21T01:50:22.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00aeffd700 1 -- 172.21.15.140:0/2567984998 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00a400d9f0 con 0x7f00b01424e0 2022-07-21T01:50:22.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 -- 172.21.15.140:0/2567984998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00b01424e0 msgr2=0x7f00b00b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 --2- 172.21.15.140:0/2567984998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00b01424e0 0x7f00b00b0950 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f00a4002f70 tx=0x7f00a4006eb0).stop 2022-07-21T01:50:22.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 -- 172.21.15.140:0/2567984998 shutdown_connections 2022-07-21T01:50:22.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 --2- 172.21.15.140:0/2567984998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00b01424e0 0x7f00b00b0950 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 -- 172.21.15.140:0/2567984998 >> 172.21.15.140:0/2567984998 conn(0x7f00b00b9360 msgr2=0x7f00b00b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:22.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 -- 172.21.15.140:0/2567984998 shutdown_connections 2022-07-21T01:50:22.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.543+0000 7f00b552b700 1 -- 172.21.15.140:0/2567984998 wait complete. 2022-07-21T01:50:22.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b552b700 1 Processor -- start 2022-07-21T01:50:22.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b552b700 1 -- start start 2022-07-21T01:50:22.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b552b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00b00aea10 0x7f00b00b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:22.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b552b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00b0570 0x7f00b00b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:22.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b552b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00b01424e0 0x7f00b0152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:22.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b552b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b013da40 con 0x7f00b01424e0 2022-07-21T01:50:22.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b552b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b013dce0 con 0x7f00b00b0570 2022-07-21T01:50:22.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b552b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b013df80 con 0x7f00b00aea10 2022-07-21T01:50:22.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b4d2a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00b01424e0 0x7f00b0152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:22.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00affff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00b00aea10 0x7f00b00b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:22.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00affff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00b00aea10 0x7f00b00b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:59180/0 (socket says 172.21.15.140:59180) 2022-07-21T01:50:22.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00affff700 1 -- 172.21.15.140:0/3304946822 learned_addr learned my addr 172.21.15.140:0/3304946822 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:22.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00af7fe700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00b0570 0x7f00b00b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:22.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b4d2a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00b01424e0 0x7f00b0152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34222/0 (socket says 172.21.15.140:34222) 2022-07-21T01:50:22.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b4d2a700 1 -- 172.21.15.140:0/3304946822 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00b00aea10 msgr2=0x7f00b00b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b4d2a700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00b00aea10 0x7f00b00b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b4d2a700 1 -- 172.21.15.140:0/3304946822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00b0570 msgr2=0x7f00b00b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b4d2a700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00b0570 0x7f00b00b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b4d2a700 1 -- 172.21.15.140:0/3304946822 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a400b1e0 con 0x7f00b01424e0 2022-07-21T01:50:22.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00affff700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00b00aea10 0x7f00b00b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:22.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b4d2a700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00b01424e0 0x7f00b0152160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f00b0066710 tx=0x7f00a0007b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:22.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00ad7fa700 1 -- 172.21.15.140:0/3304946822 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00a000cd50 con 0x7f00b01424e0 2022-07-21T01:50:22.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00ad7fa700 1 -- 172.21.15.140:0/3304946822 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a0012400 con 0x7f00b01424e0 2022-07-21T01:50:22.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b552b700 1 -- 172.21.15.140:0/3304946822 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b00b3a10 con 0x7f00b01424e0 2022-07-21T01:50:22.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00ad7fa700 1 -- 172.21.15.140:0/3304946822 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00a0012760 con 0x7f00b01424e0 2022-07-21T01:50:22.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.547+0000 7f00b552b700 1 -- 172.21.15.140:0/3304946822 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b013e640 con 0x7f00b01424e0 2022-07-21T01:50:22.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.551+0000 7f00b552b700 1 -- 172.21.15.140:0/3304946822 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b013d890 con 0x7f00b01424e0 2022-07-21T01:50:22.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.551+0000 7f00ad7fa700 1 -- 172.21.15.140:0/3304946822 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f00a0019030 con 0x7f00b01424e0 2022-07-21T01:50:22.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.551+0000 7f00ad7fa700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0090053a70 0x7f0090055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:22.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.551+0000 7f00ad7fa700 1 -- 172.21.15.140:0/3304946822 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f00a006ade0 con 0x7f00b01424e0 2022-07-21T01:50:22.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.551+0000 7f00affff700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0090053a70 0x7f0090055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:22.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.551+0000 7f00affff700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0090053a70 0x7f0090055f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f00a800d000 tx=0x7f00a8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:22.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.559+0000 7f00ad7fa700 1 -- 172.21.15.140:0/3304946822 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f00a0041fe0 con 0x7f00b01424e0 2022-07-21T01:50:22.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.719+0000 7f00b552b700 1 -- 172.21.15.140:0/3304946822 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00b00b3ba0 con 0x7f00b01424e0 2022-07-21T01:50:22.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.719+0000 7f00ad7fa700 1 -- 172.21.15.140:0/3304946822 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+1211 (secure 0 0 0) 0x7f00a0017070 con 0x7f00b01424e0 2022-07-21T01:50:22.723 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:22.724 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/838095894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":838095894},{"type":"v1","addr":"172.21.15.140:6835","nonce":838095894}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":8},{"gid":5152,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1530597349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1530597349},{"type":"v1","addr":"172.21.15.152:6833","nonce":1530597349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":12}],"filesystems":[]} 2022-07-21T01:50:22.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 -- 172.21.15.140:0/3304946822 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0090053a70 msgr2=0x7f0090055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0090053a70 0x7f0090055f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f00a800d000 tx=0x7f00a8009040).stop 2022-07-21T01:50:22.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 -- 172.21.15.140:0/3304946822 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00b01424e0 msgr2=0x7f00b0152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:22.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00b01424e0 0x7f00b0152160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f00b0066710 tx=0x7f00a0007b30).stop 2022-07-21T01:50:22.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 -- 172.21.15.140:0/3304946822 shutdown_connections 2022-07-21T01:50:22.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0090053a70 0x7f0090055f30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00b01424e0 0x7f00b0152160 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00b00b0570 0x7f00b00b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 --2- 172.21.15.140:0/3304946822 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00b00aea10 0x7f00b00b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:22.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 -- 172.21.15.140:0/3304946822 >> 172.21.15.140:0/3304946822 conn(0x7f00b00b9360 msgr2=0x7f00b013f110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:22.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 -- 172.21.15.140:0/3304946822 shutdown_connections 2022-07-21T01:50:22.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:22.723+0000 7f00b552b700 1 -- 172.21.15.140:0/3304946822 wait complete. 2022-07-21T01:50:22.752 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 12 2022-07-21T01:50:22.753 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:50:22.757 INFO:tasks.ceph.mds.a:Stopping old one... 2022-07-21T01:50:22.759 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:50:22.759 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:50:22.760 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:50:22.761 INFO:tasks.ceph.mds.b:Stopping old one... 2022-07-21T01:50:22.796 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:50:22.797 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:50:22.799 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:50:22.800 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:50:22.821 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:50:22.822 DEBUG:teuthology.parallel:result is None 2022-07-21T01:50:22.823 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:50:22.827 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:50:22.894 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:50:22.895 DEBUG:teuthology.parallel:result is None 2022-07-21T01:50:22.897 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:50:22.902 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:50:22.955 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:50:23.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.031+0000 7fb45f2f0700 1 Processor -- start 2022-07-21T01:50:23.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.031+0000 7fb45f2f0700 1 -- start start 2022-07-21T01:50:23.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45f2f0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4580b6200 0x7fb4580b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:23.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45f2f0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb45813dfc0 0x7fb45813e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:23.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45f2f0700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4580b65e0 con 0x7fb45809b760 2022-07-21T01:50:23.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45f2f0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4580aaed0 con 0x7fb45813dfc0 2022-07-21T01:50:23.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45f2f0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4580b0350 con 0x7fb4580b6200 2022-07-21T01:50:23.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45eaef700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb45813dfc0 0x7fb45813e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:23.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45e2ee700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4580b6200 0x7fb4580b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:23.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45daed700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fb45809b760 0x7fb4580b0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48386/0 (socket says 172.21.15.140:48386) 2022-07-21T01:50:23.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45daed700 1 -- 172.21.15.140:0/2816994554 learned_addr learned my addr 172.21.15.140:0/2816994554 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:23.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45eaef700 1 -- 172.21.15.140:0/2816994554 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4580b6200 msgr2=0x7fb4580b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:23.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45eaef700 1 --2- 172.21.15.140:0/2816994554 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4580b6200 0x7fb4580b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:23.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45eaef700 1 -- 172.21.15.140:0/2816994554 >> v1:172.21.15.140:6789/0 conn(0x7fb45809b760 legacy=0x7fb4580b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:23.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45eaef700 1 -- 172.21.15.140:0/2816994554 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580ac9c0 con 0x7fb45813dfc0 2022-07-21T01:50:23.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45eaef700 1 --2- 172.21.15.140:0/2816994554 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb45813dfc0 0x7fb45813e450 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fb4500032f0 tx=0x7fb4500091d0).ready entity=mon.0 client_cookie=7baa66e571eaf5ef server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:23.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45d2ec700 1 -- 172.21.15.140:0/2816994554 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb45000b6c0 con 0x7fb45813dfc0 2022-07-21T01:50:23.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45d2ec700 1 -- 172.21.15.140:0/2816994554 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45000b820 con 0x7fb45813dfc0 2022-07-21T01:50:23.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.035+0000 7fb45d2ec700 1 -- 172.21.15.140:0/2816994554 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb45000b990 con 0x7fb45813dfc0 2022-07-21T01:50:23.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 -- 172.21.15.140:0/2816994554 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb45813dfc0 msgr2=0x7fb45813e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:23.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 --2- 172.21.15.140:0/2816994554 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb45813dfc0 0x7fb45813e450 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fb4500032f0 tx=0x7fb4500091d0).stop 2022-07-21T01:50:23.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 -- 172.21.15.140:0/2816994554 shutdown_connections 2022-07-21T01:50:23.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 --2- 172.21.15.140:0/2816994554 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb45813dfc0 0x7fb45813e450 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:23.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 --2- 172.21.15.140:0/2816994554 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4580b6200 0x7fb4580b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:23.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 -- 172.21.15.140:0/2816994554 >> 172.21.15.140:0/2816994554 conn(0x7fb458063d10 msgr2=0x7fb4580b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:23.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 -- 172.21.15.140:0/2816994554 shutdown_connections 2022-07-21T01:50:23.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 -- 172.21.15.140:0/2816994554 wait complete. 2022-07-21T01:50:23.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 Processor -- start 2022-07-21T01:50:23.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 -- start start 2022-07-21T01:50:23.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb45809b760 0x7fb4580adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:23.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4580b6200 0x7fb4580aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:23.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb45813dfc0 0x7fb458132460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:23.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4580ad7e0 con 0x7fb45813dfc0 2022-07-21T01:50:23.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb458066b40 con 0x7fb4580b6200 2022-07-21T01:50:23.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45814d7c0 con 0x7fb45809b760 2022-07-21T01:50:23.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45eaef700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb45813dfc0 0x7fb458132460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:23.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45eaef700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb45813dfc0 0x7fb458132460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34248/0 (socket says 172.21.15.140:34248) 2022-07-21T01:50:23.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45eaef700 1 -- 172.21.15.140:0/135703782 learned_addr learned my addr 172.21.15.140:0/135703782 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:23.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45e2ee700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb45809b760 0x7fb4580adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:23.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45daed700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4580b6200 0x7fb4580aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:23.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45eaef700 1 -- 172.21.15.140:0/135703782 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb45809b760 msgr2=0x7fb4580adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:23.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45eaef700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb45809b760 0x7fb4580adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:23.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45eaef700 1 -- 172.21.15.140:0/135703782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4580b6200 msgr2=0x7fb4580aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:23.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45eaef700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4580b6200 0x7fb4580aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:23.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45eaef700 1 -- 172.21.15.140:0/135703782 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb450009470 con 0x7fb45813dfc0 2022-07-21T01:50:23.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45eaef700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb45813dfc0 0x7fb458132460 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fb4500032f0 tx=0x7fb450016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:23.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb4477fe700 1 -- 172.21.15.140:0/135703782 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb450021440 con 0x7fb45813dfc0 2022-07-21T01:50:23.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb4477fe700 1 -- 172.21.15.140:0/135703782 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4500215a0 con 0x7fb45813dfc0 2022-07-21T01:50:23.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 -- 172.21.15.140:0/135703782 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4580ae840 con 0x7fb45813dfc0 2022-07-21T01:50:23.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb4477fe700 1 -- 172.21.15.140:0/135703782 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb450021900 con 0x7fb45813dfc0 2022-07-21T01:50:23.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.039+0000 7fb45f2f0700 1 -- 172.21.15.140:0/135703782 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4580b4270 con 0x7fb45813dfc0 2022-07-21T01:50:23.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.043+0000 7fb45f2f0700 1 -- 172.21.15.140:0/135703782 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4580ae9d0 con 0x7fb45813dfc0 2022-07-21T01:50:23.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.043+0000 7fb4477fe700 1 -- 172.21.15.140:0/135703782 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb450027020 con 0x7fb45813dfc0 2022-07-21T01:50:23.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.043+0000 7fb4477fe700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb438053980 0x7fb438055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:23.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.043+0000 7fb4477fe700 1 -- 172.21.15.140:0/135703782 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fb45006d040 con 0x7fb45813dfc0 2022-07-21T01:50:23.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.043+0000 7fb45e2ee700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb438053980 0x7fb438055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:23.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.043+0000 7fb45e2ee700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb438053980 0x7fb438055e40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fb45400b000 tx=0x7fb454008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:23.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.047+0000 7fb4477fe700 1 -- 172.21.15.140:0/135703782 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb450044240 con 0x7fb45813dfc0 2022-07-21T01:50:23.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:23.183+0000 7fb45f2f0700 1 -- 172.21.15.140:0/135703782 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fb45813cfd0 con 0x7fb45813dfc0 2022-07-21T01:50:24.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.047+0000 7fb4477fe700 1 -- 172.21.15.140:0/135703782 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v37) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb450015020 con 0x7fb45813dfc0 2022-07-21T01:50:24.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 -- 172.21.15.140:0/135703782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb438053980 msgr2=0x7fb438055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb438053980 0x7fb438055e40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fb45400b000 tx=0x7fb454008040).stop 2022-07-21T01:50:24.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 -- 172.21.15.140:0/135703782 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb45813dfc0 msgr2=0x7fb458132460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb45813dfc0 0x7fb458132460 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fb4500032f0 tx=0x7fb450016000).stop 2022-07-21T01:50:24.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 -- 172.21.15.140:0/135703782 shutdown_connections 2022-07-21T01:50:24.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb438053980 0x7fb438055e40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb45813dfc0 0x7fb458132460 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4580b6200 0x7fb4580aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 --2- 172.21.15.140:0/135703782 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb45809b760 0x7fb4580adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 -- 172.21.15.140:0/135703782 >> 172.21.15.140:0/135703782 conn(0x7fb458063d10 msgr2=0x7fb458131300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:24.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 -- 172.21.15.140:0/135703782 shutdown_connections 2022-07-21T01:50:24.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.051+0000 7fb45f2f0700 1 -- 172.21.15.140:0/135703782 wait complete. 2022-07-21T01:50:24.066 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:50:24.081 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:50:24.083 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:50:24.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1e0a0700 1 Processor -- start 2022-07-21T01:50:24.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1e0a0700 1 -- start start 2022-07-21T01:50:24.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1e0a0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1a180b3ca0 0x7f1a180b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1e0a0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1a1813cb70 0x7f1a18143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1e0a0700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a18142ba0 con 0x7f1a1813c790 2022-07-21T01:50:24.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1e0a0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a181429b0 con 0x7f1a180b3ca0 2022-07-21T01:50:24.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1e0a0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a18146330 con 0x7f1a1813cb70 2022-07-21T01:50:24.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1c89d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1a180b3ca0 0x7f1a180b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1d09e700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1a1813c790 0x7f1a180b9a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48400/0 (socket says 172.21.15.140:48400) 2022-07-21T01:50:24.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1d09e700 1 -- 172.21.15.140:0/2101403295 learned_addr learned my addr 172.21.15.140:0/2101403295 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:24.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1d89f700 1 --2- 172.21.15.140:0/2101403295 >> v2:172.21.15.152:3300/0 conn(0x7f1a1813cb70 0x7f1a18143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1c89d700 1 -- 172.21.15.140:0/2101403295 >> v2:172.21.15.152:3300/0 conn(0x7f1a1813cb70 msgr2=0x7f1a18143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:24.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1c89d700 1 --2- 172.21.15.140:0/2101403295 >> v2:172.21.15.152:3300/0 conn(0x7f1a1813cb70 0x7f1a18143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1c89d700 1 -- 172.21.15.140:0/2101403295 >> v1:172.21.15.140:6789/0 conn(0x7f1a1813c790 legacy=0x7f1a180b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1c89d700 1 -- 172.21.15.140:0/2101403295 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a180ba120 con 0x7f1a180b3ca0 2022-07-21T01:50:24.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1d89f700 1 --2- 172.21.15.140:0/2101403295 >> v2:172.21.15.152:3300/0 conn(0x7f1a1813cb70 0x7f1a18143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:24.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a1c89d700 1 --2- 172.21.15.140:0/2101403295 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a180b3ca0 0x7f1a180b4710 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f1a14002f70 tx=0x7f1a14006df0).ready entity=mon.0 client_cookie=f29c31200e4d5e3e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:24.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a0ffff700 1 -- 172.21.15.140:0/2101403295 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a1400d5c0 con 0x7f1a180b3ca0 2022-07-21T01:50:24.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a0ffff700 1 -- 172.21.15.140:0/2101403295 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a14000e90 con 0x7f1a180b3ca0 2022-07-21T01:50:24.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.203+0000 7f1a0ffff700 1 -- 172.21.15.140:0/2101403295 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a1400d910 con 0x7f1a180b3ca0 2022-07-21T01:50:24.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/2101403295 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a180b3ca0 msgr2=0x7f1a180b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 --2- 172.21.15.140:0/2101403295 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a180b3ca0 0x7f1a180b4710 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f1a14002f70 tx=0x7f1a14006df0).stop 2022-07-21T01:50:24.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/2101403295 shutdown_connections 2022-07-21T01:50:24.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 --2- 172.21.15.140:0/2101403295 >> v2:172.21.15.152:3300/0 conn(0x7f1a1813cb70 0x7f1a18143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 --2- 172.21.15.140:0/2101403295 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a180b3ca0 0x7f1a180b4710 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/2101403295 >> 172.21.15.140:0/2101403295 conn(0x7f1a180678b0 msgr2=0x7f1a180b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:24.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/2101403295 shutdown_connections 2022-07-21T01:50:24.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/2101403295 wait complete. 2022-07-21T01:50:24.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 Processor -- start 2022-07-21T01:50:24.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 -- start start 2022-07-21T01:50:24.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a180b3ca0 0x7f1a18155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a1813c790 0x7f1a1813b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a1813cb70 0x7f1a180ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a180b0bc0 con 0x7f1a180b3ca0 2022-07-21T01:50:24.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1813f2c0 con 0x7f1a1813c790 2022-07-21T01:50:24.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1e0a0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a180b0db0 con 0x7f1a1813cb70 2022-07-21T01:50:24.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1d09e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a180b3ca0 0x7f1a18155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1d09e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a180b3ca0 0x7f1a18155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34260/0 (socket says 172.21.15.140:34260) 2022-07-21T01:50:24.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1d09e700 1 -- 172.21.15.140:0/3201774006 learned_addr learned my addr 172.21.15.140:0/3201774006 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:24.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1c89d700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a1813c790 0x7f1a1813b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.207+0000 7f1a1d89f700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a1813cb70 0x7f1a180ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a1d09e700 1 -- 172.21.15.140:0/3201774006 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a1813cb70 msgr2=0x7f1a180ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a1d09e700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a1813cb70 0x7f1a180ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a1d09e700 1 -- 172.21.15.140:0/3201774006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a1813c790 msgr2=0x7f1a1813b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a1d09e700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a1813c790 0x7f1a1813b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a1d09e700 1 -- 172.21.15.140:0/3201774006 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1400b180 con 0x7f1a180b3ca0 2022-07-21T01:50:24.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a1d09e700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a180b3ca0 0x7f1a18155760 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f1a180608b0 tx=0x7f1a08007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:24.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a0e7fc700 1 -- 172.21.15.140:0/3201774006 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a080070d0 con 0x7f1a180b3ca0 2022-07-21T01:50:24.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a0e7fc700 1 -- 172.21.15.140:0/3201774006 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a08007230 con 0x7f1a180b3ca0 2022-07-21T01:50:24.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/3201774006 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1813e1f0 con 0x7f1a180b3ca0 2022-07-21T01:50:24.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a0e7fc700 1 -- 172.21.15.140:0/3201774006 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a08005610 con 0x7f1a180b3ca0 2022-07-21T01:50:24.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/3201774006 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a181400f0 con 0x7f1a180b3ca0 2022-07-21T01:50:24.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/3201774006 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a180b0f70 con 0x7f1a180b3ca0 2022-07-21T01:50:24.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a0e7fc700 1 -- 172.21.15.140:0/3201774006 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1a08010070 con 0x7f1a180b3ca0 2022-07-21T01:50:24.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a0e7fc700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a00053a10 0x7f1a00055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a0e7fc700 1 -- 172.21.15.140:0/3201774006 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f1a0806b0a0 con 0x7f1a180b3ca0 2022-07-21T01:50:24.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a1c89d700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a00053a10 0x7f1a00055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.211+0000 7f1a1c89d700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a00053a10 0x7f1a00055ed0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f1a1401a000 tx=0x7f1a14014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:24.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.223+0000 7f1a0e7fc700 1 -- 172.21.15.140:0/3201774006 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a080422f0 con 0x7f1a180b3ca0 2022-07-21T01:50:24.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.387+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/3201774006 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1a180574e0 con 0x7f1a180b3ca0 2022-07-21T01:50:24.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.387+0000 7f1a0e7fc700 1 -- 172.21.15.140:0/3201774006 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f1a08018030 con 0x7f1a180b3ca0 2022-07-21T01:50:24.394 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:24.395 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:50:24.398 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:50:24.399 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.400 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:50:24.401 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:50:24.406 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.407 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:50:24.407 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:50:24.408 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:50:24.408 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.409 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.409 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.409 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.410 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:50:24.410 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:50:24.411 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.411 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:50:24.412 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:50:24.412 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:50:24.413 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.413 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.413 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.414 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.415 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:50:24.415 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:50:24.416 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.416 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:50:24.417 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:50:24.417 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.418 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.418 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.418 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.419 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:50:24.419 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:50:24.419 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.420 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:50:24.420 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:50:24.420 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.420 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.421 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.421 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.422 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:50:24.422 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:50:24.422 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.431 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:50:24.432 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:50:24.433 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.433 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.433 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.442 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.442 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:50:24.443 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:50:24.443 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.443 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:50:24.443 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:50:24.444 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.444 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.444 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.444 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.445 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:50:24.445 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:50:24.445 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.445 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:50:24.445 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:50:24.446 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.446 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.446 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.446 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.452 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:50:24.453 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:50:24.453 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.453 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:50:24.453 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:50:24.454 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.454 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.455 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.455 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.456 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:50:24.456 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:50:24.456 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.457 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:50:24.458 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:50:24.458 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.459 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.459 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.460 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.460 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:50:24.461 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:50:24.461 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.461 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:50:24.462 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:50:24.462 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.463 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.463 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.464 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.464 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:50:24.465 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:50:24.465 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.466 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:50:24.466 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:50:24.467 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:50:24.467 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.467 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.468 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.469 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:50:24.469 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:50:24.470 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.470 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:50:24.471 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:50:24.471 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:50:24.471 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.472 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.472 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.473 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:50:24.473 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:50:24.474 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.474 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:50:24.474 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:50:24.475 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:50:24.475 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.476 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.477 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.477 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:50:24.478 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:50:24.479 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.480 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:50:24.480 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:50:24.480 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:50:24.481 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.481 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.482 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.483 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.483 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:50:24.483 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:50:24.484 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.484 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:50:24.485 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:50:24.485 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.488 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.488 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:24.489 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:24.489 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:50:24.490 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:50:24.490 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:50:24.491 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:50:24.491 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:50:24.492 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:50:24.493 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.500 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:24.501 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:50:24.501 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:50:24.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/3201774006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a00053a10 msgr2=0x7f1a00055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a00053a10 0x7f1a00055ed0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f1a1401a000 tx=0x7f1a14014000).stop 2022-07-21T01:50:24.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/3201774006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a180b3ca0 msgr2=0x7f1a18155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a180b3ca0 0x7f1a18155760 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f1a180608b0 tx=0x7f1a08007c00).stop 2022-07-21T01:50:24.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/3201774006 shutdown_connections 2022-07-21T01:50:24.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a00053a10 0x7f1a00055ed0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a1813cb70 0x7f1a180ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a1813c790 0x7f1a1813b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 --2- 172.21.15.140:0/3201774006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a180b3ca0 0x7f1a18155760 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/3201774006 >> 172.21.15.140:0/3201774006 conn(0x7f1a180678b0 msgr2=0x7f1a180b7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:24.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/3201774006 shutdown_connections 2022-07-21T01:50:24.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.391+0000 7f1a1e0a0700 1 -- 172.21.15.140:0/3201774006 wait complete. 2022-07-21T01:50:24.506 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:50:24.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.623+0000 7ff991ede700 1 Processor -- start 2022-07-21T01:50:24.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.623+0000 7ff991ede700 1 -- start start 2022-07-21T01:50:24.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff98c0b5650 0x7ff98c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c147010 con 0x7ff98c0adff0 2022-07-21T01:50:24.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c1472b0 con 0x7ff98c0af8e0 2022-07-21T01:50:24.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98c146d40 con 0x7ff98c0b5650 2022-07-21T01:50:24.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff990edc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff98c0b5650 0x7ff98c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff98bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff98c0adff0 0x7ff98c147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48414/0 (socket says 172.21.15.140:48414) 2022-07-21T01:50:24.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff98bfff700 1 -- 172.21.15.140:0/1293626613 learned_addr learned my addr 172.21.15.140:0/1293626613 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:24.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff98b7fe700 1 -- 172.21.15.140:0/1293626613 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202226497 0 0) 0x7ff98c147010 con 0x7ff98c0adff0 2022-07-21T01:50:24.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff98b7fe700 1 -- 172.21.15.140:0/1293626613 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff96c003260 con 0x7ff98c0adff0 2022-07-21T01:50:24.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff990edc700 1 -- 172.21.15.140:0/1293626613 >> v1:172.21.15.152:6789/0 conn(0x7ff98c0af8e0 legacy=0x7ff98c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff990edc700 1 -- 172.21.15.140:0/1293626613 >> v1:172.21.15.140:6789/0 conn(0x7ff98c0adff0 legacy=0x7ff98c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff990edc700 1 -- 172.21.15.140:0/1293626613 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c05c4a0 con 0x7ff98c0b5650 2022-07-21T01:50:24.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff990edc700 1 --2- 172.21.15.140:0/1293626613 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff98c0b5650 0x7ff98c0b5a30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7ff974002f70 tx=0x7ff974006df0).ready entity=mon.0 client_cookie=773005a286c58af9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:24.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff98b7fe700 1 -- 172.21.15.140:0/1293626613 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff97400d610 con 0x7ff98c0b5650 2022-07-21T01:50:24.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff98b7fe700 1 -- 172.21.15.140:0/1293626613 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff974000e90 con 0x7ff98c0b5650 2022-07-21T01:50:24.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff98b7fe700 1 -- 172.21.15.140:0/1293626613 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff97400d960 con 0x7ff98c0b5650 2022-07-21T01:50:24.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 -- 172.21.15.140:0/1293626613 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff98c0b5650 msgr2=0x7ff98c0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 --2- 172.21.15.140:0/1293626613 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff98c0b5650 0x7ff98c0b5a30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7ff974002f70 tx=0x7ff974006df0).stop 2022-07-21T01:50:24.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 -- 172.21.15.140:0/1293626613 shutdown_connections 2022-07-21T01:50:24.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 --2- 172.21.15.140:0/1293626613 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff98c0b5650 0x7ff98c0b5a30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 -- 172.21.15.140:0/1293626613 >> 172.21.15.140:0/1293626613 conn(0x7ff98c0634e0 msgr2=0x7ff98c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:24.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 -- 172.21.15.140:0/1293626613 shutdown_connections 2022-07-21T01:50:24.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 -- 172.21.15.140:0/1293626613 wait complete. 2022-07-21T01:50:24.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 Processor -- start 2022-07-21T01:50:24.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.627+0000 7ff991ede700 1 -- start start 2022-07-21T01:50:24.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff991ede700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff98c0adff0 0x7ff98c1118b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff991ede700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff98c0af8e0 0x7ff98c156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff991ede700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff98c0b5650 0x7ff98c156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff991ede700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c1492c0 con 0x7ff98c0b5650 2022-07-21T01:50:24.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff991ede700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c10e130 con 0x7ff98c0af8e0 2022-07-21T01:50:24.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff991ede700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c110f70 con 0x7ff98c0adff0 2022-07-21T01:50:24.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff9916dd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff98c0b5650 0x7ff98c156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff9916dd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff98c0b5650 0x7ff98c156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34278/0 (socket says 172.21.15.140:34278) 2022-07-21T01:50:24.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff9916dd700 1 -- 172.21.15.140:0/439641265 learned_addr learned my addr 172.21.15.140:0/439641265 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:24.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff98bfff700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff98c0af8e0 0x7ff98c156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff990edc700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff98c0adff0 0x7ff98c1118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff9916dd700 1 -- 172.21.15.140:0/439641265 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff98c0adff0 msgr2=0x7ff98c1118b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff9916dd700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff98c0adff0 0x7ff98c1118b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff9916dd700 1 -- 172.21.15.140:0/439641265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff98c0af8e0 msgr2=0x7ff98c156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff9916dd700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff98c0af8e0 0x7ff98c156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff9916dd700 1 -- 172.21.15.140:0/439641265 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97400b180 con 0x7ff98c0b5650 2022-07-21T01:50:24.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff990edc700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff98c0adff0 0x7ff98c1118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:24.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff9916dd700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff98c0b5650 0x7ff98c156970 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7ff98c0622d0 tx=0x7ff984007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:24.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff989ffb700 1 -- 172.21.15.140:0/439641265 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff984012400 con 0x7ff98c0b5650 2022-07-21T01:50:24.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff989ffb700 1 -- 172.21.15.140:0/439641265 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff984012560 con 0x7ff98c0b5650 2022-07-21T01:50:24.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff991ede700 1 -- 172.21.15.140:0/439641265 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98c1494f0 con 0x7ff98c0b5650 2022-07-21T01:50:24.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff989ffb700 1 -- 172.21.15.140:0/439641265 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff9840128c0 con 0x7ff98c0b5650 2022-07-21T01:50:24.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff991ede700 1 -- 172.21.15.140:0/439641265 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98c148370 con 0x7ff98c0b5650 2022-07-21T01:50:24.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.631+0000 7ff991ede700 1 -- 172.21.15.140:0/439641265 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98c0574e0 con 0x7ff98c0b5650 2022-07-21T01:50:24.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.635+0000 7ff989ffb700 1 -- 172.21.15.140:0/439641265 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff984019030 con 0x7ff98c0b5650 2022-07-21T01:50:24.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.635+0000 7ff989ffb700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff970053950 0x7ff970055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.635+0000 7ff990edc700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff970053950 0x7ff970055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.635+0000 7ff989ffb700 1 -- 172.21.15.140:0/439641265 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff98406afe0 con 0x7ff98c0b5650 2022-07-21T01:50:24.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.635+0000 7ff990edc700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff970053950 0x7ff970055e10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7ff974018000 tx=0x7ff974014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:24.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.643+0000 7ff989ffb700 1 -- 172.21.15.140:0/439641265 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff984042140 con 0x7ff98c0b5650 2022-07-21T01:50:24.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.767+0000 7ff991ede700 1 -- 172.21.15.140:0/439641265 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff98c067ea0 con 0x7ff98c0b5650 2022-07-21T01:50:24.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff989ffb700 1 -- 172.21.15.140:0/439641265 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+13277 (secure 0 0 0) 0x7ff984045760 con 0x7ff98c0b5650 2022-07-21T01:50:24.776 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:24.777 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":37,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:50:24.036827+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:50:24.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 -- 172.21.15.140:0/439641265 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff970053950 msgr2=0x7ff970055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff970053950 0x7ff970055e10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7ff974018000 tx=0x7ff974014000).stop 2022-07-21T01:50:24.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 -- 172.21.15.140:0/439641265 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff98c0b5650 msgr2=0x7ff98c156970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff98c0b5650 0x7ff98c156970 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7ff98c0622d0 tx=0x7ff984007bc0).stop 2022-07-21T01:50:24.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 -- 172.21.15.140:0/439641265 shutdown_connections 2022-07-21T01:50:24.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff970053950 0x7ff970055e10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff98c0b5650 0x7ff98c156970 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff98c0af8e0 0x7ff98c156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 --2- 172.21.15.140:0/439641265 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff98c0adff0 0x7ff98c1118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 -- 172.21.15.140:0/439641265 >> 172.21.15.140:0/439641265 conn(0x7ff98c0634e0 msgr2=0x7ff98c0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:24.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 -- 172.21.15.140:0/439641265 shutdown_connections 2022-07-21T01:50:24.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.771+0000 7ff991ede700 1 -- 172.21.15.140:0/439641265 wait complete. 2022-07-21T01:50:24.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:50:24.798 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:50:24.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea5d86f700 1 Processor -- start 2022-07-21T01:50:24.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea5d86f700 1 -- start start 2022-07-21T01:50:24.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea5d86f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea580b3ed0 0x7fea58143530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea5d86f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea580b6680 con 0x7fea58143a20 2022-07-21T01:50:24.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea5d86f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea581465c0 con 0x7fea580b62a0 2022-07-21T01:50:24.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea5d86f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea580b4af0 con 0x7fea580b3ed0 2022-07-21T01:50:24.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea577fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fea58143a20 0x7fea5813fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48432/0 (socket says 172.21.15.140:48432) 2022-07-21T01:50:24.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea577fe700 1 -- 172.21.15.140:0/260544461 learned_addr learned my addr 172.21.15.140:0/260544461 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:24.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447737395 0 0) 0x7fea580b6680 con 0x7fea58143a20 2022-07-21T01:50:24.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea567fc700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea580b3ed0 0x7fea58143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea3c003260 con 0x7fea58143a20 2022-07-21T01:50:24.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2687020976 0 0) 0x7fea3c003260 con 0x7fea58143a20 2022-07-21T01:50:24.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea580b3ed0 msgr2=0x7fea58143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:24.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea580b3ed0 0x7fea58143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 >> v1:172.21.15.152:6789/0 conn(0x7fea580b62a0 legacy=0x7fea580b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea58142df0 con 0x7fea58143a20 2022-07-21T01:50:24.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fea500019e0 con 0x7fea58143a20 2022-07-21T01:50:24.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 >> v1:172.21.15.140:6789/0 conn(0x7fea58143a20 legacy=0x7fea5813fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 0x7fea3c003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea580b3ed0 0x7fea3c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea3c0049f0 0x7fea3c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea580b4af0 con 0x7fea3c003780 2022-07-21T01:50:24.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c00b7b0 con 0x7fea3c0049f0 2022-07-21T01:50:24.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c00b970 con 0x7fea580b3ed0 2022-07-21T01:50:24.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea56ffd700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 0x7fea3c003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea567fc700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea580b3ed0 0x7fea3c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.915+0000 7fea577fe700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea3c0049f0 0x7fea3c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea56ffd700 1 -- 172.21.15.140:0/260544461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea580b3ed0 msgr2=0x7fea3c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:24.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea56ffd700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea580b3ed0 0x7fea3c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea56ffd700 1 -- 172.21.15.140:0/260544461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea3c0049f0 msgr2=0x7fea3c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:24.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea56ffd700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea3c0049f0 0x7fea3c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea56ffd700 1 -- 172.21.15.140:0/260544461 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c003260 con 0x7fea3c003780 2022-07-21T01:50:24.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea56ffd700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 0x7fea3c003ba0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fea58067ca0 tx=0x7fea48007a00).ready entity=mon.0 client_cookie=7fc94bfff215c352 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:24.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea48007d10 con 0x7fea3c003780 2022-07-21T01:50:24.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea48007030 con 0x7fea3c003780 2022-07-21T01:50:24.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea55ffb700 1 -- 172.21.15.140:0/260544461 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea48007390 con 0x7fea3c003780 2022-07-21T01:50:24.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea5d86f700 1 -- 172.21.15.140:0/260544461 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 msgr2=0x7fea3c003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea5d86f700 1 --2- 172.21.15.140:0/260544461 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 0x7fea3c003ba0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fea58067ca0 tx=0x7fea48007a00).stop 2022-07-21T01:50:24.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea577fe700 1 -- 172.21.15.140:0/260544461 reap_dead start 2022-07-21T01:50:24.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea5d86f700 1 -- 172.21.15.140:0/260544461 shutdown_connections 2022-07-21T01:50:24.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea5d86f700 1 -- 172.21.15.140:0/260544461 >> 172.21.15.140:0/260544461 conn(0x7fea58066f00 msgr2=0x7fea580672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:24.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea5d86f700 1 -- 172.21.15.140:0/260544461 shutdown_connections 2022-07-21T01:50:24.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea5d86f700 1 -- 172.21.15.140:0/260544461 wait complete. 2022-07-21T01:50:24.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea5d86f700 1 Processor -- start 2022-07-21T01:50:24.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.919+0000 7fea5d86f700 1 -- start start 2022-07-21T01:50:24.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea5d86f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 0x7fea58142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea5d86f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea58149ce0 0x7fea5814a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea5d86f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea5814a630 0x7fea581520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea5d86f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea58141a30 con 0x7fea3c003780 2022-07-21T01:50:24.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea5d86f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea58140850 con 0x7fea5814a630 2022-07-21T01:50:24.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea5d86f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea58140af0 con 0x7fea58149ce0 2022-07-21T01:50:24.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea56ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 0x7fea58142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea56ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 0x7fea58142390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34294/0 (socket says 172.21.15.140:34294) 2022-07-21T01:50:24.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea56ffd700 1 -- 172.21.15.140:0/3911774104 learned_addr learned my addr 172.21.15.140:0/3911774104 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:24.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea567fc700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea58149ce0 0x7fea5814a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea577fe700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea5814a630 0x7fea581520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea56ffd700 1 -- 172.21.15.140:0/3911774104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea58149ce0 msgr2=0x7fea5814a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea56ffd700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea58149ce0 0x7fea5814a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea56ffd700 1 -- 172.21.15.140:0/3911774104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea5814a630 msgr2=0x7fea581520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:24.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea56ffd700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea5814a630 0x7fea581520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:24.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea56ffd700 1 -- 172.21.15.140:0/3911774104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea58147d30 con 0x7fea3c003780 2022-07-21T01:50:24.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea56ffd700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 0x7fea58142390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fea58067ca0 tx=0x7fea48015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:24.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea47fff700 1 -- 172.21.15.140:0/3911774104 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea48007d10 con 0x7fea3c003780 2022-07-21T01:50:24.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea47fff700 1 -- 172.21.15.140:0/3911774104 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea48020640 con 0x7fea3c003780 2022-07-21T01:50:24.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea5d86f700 1 -- 172.21.15.140:0/3911774104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea58140da0 con 0x7fea3c003780 2022-07-21T01:50:24.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea47fff700 1 -- 172.21.15.140:0/3911774104 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea480209a0 con 0x7fea3c003780 2022-07-21T01:50:24.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.923+0000 7fea5d86f700 1 -- 172.21.15.140:0/3911774104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea58140630 con 0x7fea3c003780 2022-07-21T01:50:24.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.927+0000 7fea47fff700 1 -- 172.21.15.140:0/3911774104 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fea4801d070 con 0x7fea3c003780 2022-07-21T01:50:24.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.927+0000 7fea5d86f700 1 -- 172.21.15.140:0/3911774104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea58147a90 con 0x7fea3c003780 2022-07-21T01:50:24.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.927+0000 7fea47fff700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea34053b00 0x7fea34055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:24.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.927+0000 7fea47fff700 1 -- 172.21.15.140:0/3911774104 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fea48001e40 con 0x7fea3c003780 2022-07-21T01:50:24.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.927+0000 7fea567fc700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea34053b00 0x7fea34055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:24.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.927+0000 7fea567fc700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea34053b00 0x7fea34055fc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fea4c00b000 tx=0x7fea4c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:24.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:24.931+0000 7fea47fff700 1 -- 172.21.15.140:0/3911774104 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea58147a90 con 0x7fea3c003780 2022-07-21T01:50:25.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.055+0000 7fea5d86f700 1 -- 172.21.15.140:0/3911774104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fea58140f30 con 0x7fea3c003780 2022-07-21T01:50:25.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.055+0000 7fea47fff700 1 -- 172.21.15.140:0/3911774104 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v37) v1 ==== 110+0+10 (secure 0 0 0) 0x7fea48043bd0 con 0x7fea3c003780 2022-07-21T01:50:25.060 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:50:25.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 -- 172.21.15.140:0/3911774104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea34053b00 msgr2=0x7fea34055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea34053b00 0x7fea34055fc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fea4c00b000 tx=0x7fea4c006df0).stop 2022-07-21T01:50:25.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 -- 172.21.15.140:0/3911774104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 msgr2=0x7fea58142390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 0x7fea58142390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fea58067ca0 tx=0x7fea48015000).stop 2022-07-21T01:50:25.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 -- 172.21.15.140:0/3911774104 shutdown_connections 2022-07-21T01:50:25.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea34053b00 0x7fea34055fc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea5814a630 0x7fea581520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea58149ce0 0x7fea5814a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 --2- 172.21.15.140:0/3911774104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea3c003780 0x7fea58142390 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 -- 172.21.15.140:0/3911774104 >> 172.21.15.140:0/3911774104 conn(0x7fea58066f00 msgr2=0x7fea580b7360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:25.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 -- 172.21.15.140:0/3911774104 shutdown_connections 2022-07-21T01:50:25.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.059+0000 7fea5d86f700 1 -- 172.21.15.140:0/3911774104 wait complete. 2022-07-21T01:50:25.071 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:50:25.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.183+0000 7f60dd451700 1 Processor -- start 2022-07-21T01:50:25.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.183+0000 7f60dd451700 1 -- start start 2022-07-21T01:50:25.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.183+0000 7f60dd451700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f60d80b59d0 0x7f60d805d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.183+0000 7f60dd451700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f60d8128d70 0x7f60d813fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.183+0000 7f60dd451700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d80b5db0 con 0x7f60d80af180 2022-07-21T01:50:25.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.183+0000 7f60dd451700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d81291b0 con 0x7f60d80b59d0 2022-07-21T01:50:25.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.183+0000 7f60dd451700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d80ad800 con 0x7f60d8128d70 2022-07-21T01:50:25.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.183+0000 7f60d7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f60d80b59d0 0x7f60d805d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.183+0000 7f60d7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f60d80b59d0 0x7f60d805d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34304/0 (socket says 172.21.15.140:34304) 2022-07-21T01:50:25.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.183+0000 7f60dcc50700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f60d8128d70 0x7f60d813fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.183+0000 7f60d7fff700 1 -- 172.21.15.140:0/1591461002 learned_addr learned my addr 172.21.15.140:0/1591461002 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:25.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60d6ffd700 1 -- 172.21.15.140:0/1591461002 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595548826 0 0) 0x7f60d80b5db0 con 0x7f60d80af180 2022-07-21T01:50:25.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60d6ffd700 1 -- 172.21.15.140:0/1591461002 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60b8003260 con 0x7f60d80af180 2022-07-21T01:50:25.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60d7fff700 1 -- 172.21.15.140:0/1591461002 >> v2:172.21.15.152:3300/0 conn(0x7f60d8128d70 msgr2=0x7f60d813fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:25.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60d7fff700 1 --2- 172.21.15.140:0/1591461002 >> v2:172.21.15.152:3300/0 conn(0x7f60d8128d70 0x7f60d813fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60d7fff700 1 -- 172.21.15.140:0/1591461002 >> v1:172.21.15.152:6789/0 conn(0x7f60d80af180 legacy=0x7f60d8128660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60d7fff700 1 -- 172.21.15.140:0/1591461002 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d80adaa0 con 0x7f60d80b59d0 2022-07-21T01:50:25.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60dcc50700 1 --2- 172.21.15.140:0/1591461002 >> v2:172.21.15.152:3300/0 conn(0x7f60d8128d70 0x7f60d813fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:25.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60d7fff700 1 --2- 172.21.15.140:0/1591461002 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60d80b59d0 0x7f60d805d1c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f60c0002f70 tx=0x7f60c0006eb0).ready entity=mon.0 client_cookie=12a51d9d61e27cce server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:25.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60d6ffd700 1 -- 172.21.15.140:0/1591461002 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c000d610 con 0x7f60d80b59d0 2022-07-21T01:50:25.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60d6ffd700 1 -- 172.21.15.140:0/1591461002 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c0000e90 con 0x7f60d80b59d0 2022-07-21T01:50:25.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60d6ffd700 1 -- 172.21.15.140:0/1591461002 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c000d9a0 con 0x7f60d80b59d0 2022-07-21T01:50:25.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60dd451700 1 -- 172.21.15.140:0/1591461002 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60d80b59d0 msgr2=0x7f60d805d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60dd451700 1 --2- 172.21.15.140:0/1591461002 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60d80b59d0 0x7f60d805d1c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f60c0002f70 tx=0x7f60c0006eb0).stop 2022-07-21T01:50:25.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60dd451700 1 -- 172.21.15.140:0/1591461002 shutdown_connections 2022-07-21T01:50:25.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60dd451700 1 --2- 172.21.15.140:0/1591461002 >> v2:172.21.15.152:3300/0 conn(0x7f60d8128d70 0x7f60d813fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60dd451700 1 --2- 172.21.15.140:0/1591461002 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60d80b59d0 0x7f60d805d1c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60dd451700 1 -- 172.21.15.140:0/1591461002 >> 172.21.15.140:0/1591461002 conn(0x7f60d80634c0 msgr2=0x7f60d80b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:25.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60dd451700 1 -- 172.21.15.140:0/1591461002 shutdown_connections 2022-07-21T01:50:25.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.187+0000 7f60dd451700 1 -- 172.21.15.140:0/1591461002 wait complete. 2022-07-21T01:50:25.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dd451700 1 Processor -- start 2022-07-21T01:50:25.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dd451700 1 -- start start 2022-07-21T01:50:25.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dd451700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60d80af180 0x7f60d8067580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dd451700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60d80b59d0 0x7f60d8143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dd451700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d8128d70 0x7f60d805c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dd451700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8143aa0 con 0x7f60d80b59d0 2022-07-21T01:50:25.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dd451700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8063b20 con 0x7f60d8128d70 2022-07-21T01:50:25.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dd451700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8063dc0 con 0x7f60d80af180 2022-07-21T01:50:25.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60d80b59d0 0x7f60d8143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60d80b59d0 0x7f60d8143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34310/0 (socket says 172.21.15.140:34310) 2022-07-21T01:50:25.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d77fe700 1 -- 172.21.15.140:0/4054026438 learned_addr learned my addr 172.21.15.140:0/4054026438 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:25.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dcc50700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d8128d70 0x7f60d805c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d7fff700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60d80af180 0x7f60d8067580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d77fe700 1 -- 172.21.15.140:0/4054026438 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60d80af180 msgr2=0x7f60d8067580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d77fe700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60d80af180 0x7f60d8067580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d77fe700 1 -- 172.21.15.140:0/4054026438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d8128d70 msgr2=0x7f60d805c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d77fe700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d8128d70 0x7f60d805c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d77fe700 1 -- 172.21.15.140:0/4054026438 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d0009470 con 0x7f60d80b59d0 2022-07-21T01:50:25.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dcc50700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d8128d70 0x7f60d805c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:25.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d77fe700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60d80b59d0 0x7f60d8143c60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f60c8001110 tx=0x7f60c8007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:25.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d57fa700 1 -- 172.21.15.140:0/4054026438 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c800ce30 con 0x7f60d80b59d0 2022-07-21T01:50:25.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d57fa700 1 -- 172.21.15.140:0/4054026438 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c800f470 con 0x7f60d80b59d0 2022-07-21T01:50:25.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dd451700 1 -- 172.21.15.140:0/4054026438 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c000b1e0 con 0x7f60d80b59d0 2022-07-21T01:50:25.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60d57fa700 1 -- 172.21.15.140:0/4054026438 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c800f7d0 con 0x7f60d80b59d0 2022-07-21T01:50:25.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.191+0000 7f60dd451700 1 -- 172.21.15.140:0/4054026438 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d8064aa0 con 0x7f60d80b59d0 2022-07-21T01:50:25.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.195+0000 7f60dd451700 1 -- 172.21.15.140:0/4054026438 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d80ad4f0 con 0x7f60d80b59d0 2022-07-21T01:50:25.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.195+0000 7f60d57fa700 1 -- 172.21.15.140:0/4054026438 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f60c8019030 con 0x7f60d80b59d0 2022-07-21T01:50:25.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.195+0000 7f60d57fa700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053b00 0x7f60bc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.195+0000 7f60d7fff700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053b00 0x7f60bc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.195+0000 7f60d57fa700 1 -- 172.21.15.140:0/4054026438 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f60c806b130 con 0x7f60d80b59d0 2022-07-21T01:50:25.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.195+0000 7f60d7fff700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053b00 0x7f60bc055fc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f60c001a000 tx=0x7f60c0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:25.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.203+0000 7f60d57fa700 1 -- 172.21.15.140:0/4054026438 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f60c8042270 con 0x7f60d80b59d0 2022-07-21T01:50:25.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.327+0000 7f60dd451700 1 -- 172.21.15.140:0/4054026438 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f60d80574e0 con 0x7f60d80b59d0 2022-07-21T01:50:25.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.327+0000 7f60d57fa700 1 -- 172.21.15.140:0/4054026438 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7f60c8007b80 con 0x7f60d80b59d0 2022-07-21T01:50:25.332 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:50:25.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 -- 172.21.15.140:0/4054026438 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053b00 msgr2=0x7f60bc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053b00 0x7f60bc055fc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f60c001a000 tx=0x7f60c0015000).stop 2022-07-21T01:50:25.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 -- 172.21.15.140:0/4054026438 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60d80b59d0 msgr2=0x7f60d8143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60d80b59d0 0x7f60d8143c60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f60c8001110 tx=0x7f60c8007a60).stop 2022-07-21T01:50:25.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 -- 172.21.15.140:0/4054026438 shutdown_connections 2022-07-21T01:50:25.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053b00 0x7f60bc055fc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60d8128d70 0x7f60d805c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60d80b59d0 0x7f60d8143c60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 --2- 172.21.15.140:0/4054026438 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60d80af180 0x7f60d8067580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 -- 172.21.15.140:0/4054026438 >> 172.21.15.140:0/4054026438 conn(0x7f60d80634c0 msgr2=0x7f60d8127280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:25.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 -- 172.21.15.140:0/4054026438 shutdown_connections 2022-07-21T01:50:25.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.331+0000 7f60dd451700 1 -- 172.21.15.140:0/4054026438 wait complete. 2022-07-21T01:50:25.344 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:50:25.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.471+0000 7f2ed1ef2700 1 Processor -- start 2022-07-21T01:50:25.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.471+0000 7f2ed1ef2700 1 -- start start 2022-07-21T01:50:25.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.471+0000 7f2ed1ef2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2ecc0b5650 0x7f2ecc0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.471+0000 7f2ed1ef2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2ecc0b1200 0x7f2ecc05c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.471+0000 7f2ed1ef2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ecc05bd90 0x7f2ecc05c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.471+0000 7f2ed1ef2700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc142970 con 0x7f2ecc0b1200 2022-07-21T01:50:25.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.471+0000 7f2ed1ef2700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc0aea30 con 0x7f2ecc0b5650 2022-07-21T01:50:25.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.471+0000 7f2ed1ef2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0aec60 con 0x7f2ecc05bd90 2022-07-21T01:50:25.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.471+0000 7f2ecbfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2ecc0b1200 0x7f2ecc05c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.471+0000 7f2ecbfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2ecc0b1200 0x7f2ecc05c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34316/0 (socket says 172.21.15.140:34316) 2022-07-21T01:50:25.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.471+0000 7f2ecbfff700 1 -- 172.21.15.140:0/426816592 learned_addr learned my addr 172.21.15.140:0/426816592 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:25.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed16f1700 1 --2- 172.21.15.140:0/426816592 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ecc05bd90 0x7f2ecc05c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed0ef0700 1 --2- 172.21.15.140:0/426816592 >> v2:172.21.15.152:3300/0 conn(0x7f2ecc0b5650 0x7f2ecc0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ecbfff700 1 -- 172.21.15.140:0/426816592 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ecc05bd90 msgr2=0x7f2ecc05c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:25.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ecbfff700 1 --2- 172.21.15.140:0/426816592 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ecc05bd90 0x7f2ecc05c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ecbfff700 1 -- 172.21.15.140:0/426816592 >> v2:172.21.15.152:3300/0 conn(0x7f2ecc0b5650 msgr2=0x7f2ecc0b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:25.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ecbfff700 1 --2- 172.21.15.140:0/426816592 >> v2:172.21.15.152:3300/0 conn(0x7f2ecc0b5650 0x7f2ecc0b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ecbfff700 1 -- 172.21.15.140:0/426816592 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc0af180 con 0x7f2ecc0b1200 2022-07-21T01:50:25.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed0ef0700 1 --2- 172.21.15.140:0/426816592 >> v2:172.21.15.152:3300/0 conn(0x7f2ecc0b5650 0x7f2ecc0b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:25.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ecbfff700 1 --2- 172.21.15.140:0/426816592 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ecc0b1200 0x7f2ecc05c4f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f2eb40032f0 tx=0x7f2eb4009250).ready entity=mon.0 client_cookie=8db075edc15cfc08 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:25.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ecb7fe700 1 -- 172.21.15.140:0/426816592 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2eb4009ea0 con 0x7f2ecc0b1200 2022-07-21T01:50:25.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ecb7fe700 1 -- 172.21.15.140:0/426816592 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb400b870 con 0x7f2ecc0b1200 2022-07-21T01:50:25.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ecb7fe700 1 -- 172.21.15.140:0/426816592 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2eb400b9e0 con 0x7f2ecc0b1200 2022-07-21T01:50:25.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/426816592 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ecc0b1200 msgr2=0x7f2ecc05c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed1ef2700 1 --2- 172.21.15.140:0/426816592 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ecc0b1200 0x7f2ecc05c4f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f2eb40032f0 tx=0x7f2eb4009250).stop 2022-07-21T01:50:25.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/426816592 shutdown_connections 2022-07-21T01:50:25.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed1ef2700 1 --2- 172.21.15.140:0/426816592 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ecc05bd90 0x7f2ecc05c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed1ef2700 1 --2- 172.21.15.140:0/426816592 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ecc0b1200 0x7f2ecc05c4f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed1ef2700 1 --2- 172.21.15.140:0/426816592 >> v2:172.21.15.152:3300/0 conn(0x7f2ecc0b5650 0x7f2ecc0b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/426816592 >> 172.21.15.140:0/426816592 conn(0x7f2ecc0634e0 msgr2=0x7f2ecc0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:25.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/426816592 shutdown_connections 2022-07-21T01:50:25.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.475+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/426816592 wait complete. 2022-07-21T01:50:25.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed1ef2700 1 Processor -- start 2022-07-21T01:50:25.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed1ef2700 1 -- start start 2022-07-21T01:50:25.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed1ef2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ecc05bd90 0x7f2ecc064480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed1ef2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ecc0b1200 0x7f2ecc064970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed1ef2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ecc0b5650 0x7f2ecc1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed1ef2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc13ff70 con 0x7f2ecc05bd90 2022-07-21T01:50:25.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed1ef2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc152f30 con 0x7f2ecc0b5650 2022-07-21T01:50:25.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed1ef2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc142020 con 0x7f2ecc0b1200 2022-07-21T01:50:25.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed0ef0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ecc05bd90 0x7f2ecc064480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed0ef0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ecc05bd90 0x7f2ecc064480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34322/0 (socket says 172.21.15.140:34322) 2022-07-21T01:50:25.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed0ef0700 1 -- 172.21.15.140:0/2062213133 learned_addr learned my addr 172.21.15.140:0/2062213133 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:25.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed16f1700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ecc0b5650 0x7f2ecc1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ecbfff700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ecc0b1200 0x7f2ecc064970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed0ef0700 1 -- 172.21.15.140:0/2062213133 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ecc0b1200 msgr2=0x7f2ecc064970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed0ef0700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ecc0b1200 0x7f2ecc064970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed0ef0700 1 -- 172.21.15.140:0/2062213133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ecc0b5650 msgr2=0x7f2ecc1555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed0ef0700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ecc0b5650 0x7f2ecc1555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed0ef0700 1 -- 172.21.15.140:0/2062213133 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb40094d0 con 0x7f2ecc05bd90 2022-07-21T01:50:25.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ecbfff700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ecc0b1200 0x7f2ecc064970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:25.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed0ef0700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ecc05bd90 0x7f2ecc064480 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f2ecc064010 tx=0x7f2ebc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:25.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ec9ffb700 1 -- 172.21.15.140:0/2062213133 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ebc011780 con 0x7f2ecc05bd90 2022-07-21T01:50:25.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ec9ffb700 1 -- 172.21.15.140:0/2062213133 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc0118e0 con 0x7f2ecc05bd90 2022-07-21T01:50:25.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/2062213133 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ecc152a20 con 0x7f2ecc05bd90 2022-07-21T01:50:25.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ec9ffb700 1 -- 172.21.15.140:0/2062213133 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ebc011c40 con 0x7f2ecc05bd90 2022-07-21T01:50:25.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.479+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/2062213133 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc13d810 con 0x7f2ecc05bd90 2022-07-21T01:50:25.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.483+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/2062213133 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ecc0574e0 con 0x7f2ecc05bd90 2022-07-21T01:50:25.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.483+0000 7f2ec9ffb700 1 -- 172.21.15.140:0/2062213133 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2ebc01c460 con 0x7f2ecc05bd90 2022-07-21T01:50:25.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.483+0000 7f2ec9ffb700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2eb0053a70 0x7f2eb0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.483+0000 7f2ec9ffb700 1 -- 172.21.15.140:0/2062213133 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2ebc06be50 con 0x7f2ecc05bd90 2022-07-21T01:50:25.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.483+0000 7f2ecbfff700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2eb0053a70 0x7f2eb0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.483+0000 7f2ecbfff700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2eb0053a70 0x7f2eb0055f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f2eb4018000 tx=0x7f2eb4016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:25.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.491+0000 7f2ec9ffb700 1 -- 172.21.15.140:0/2062213133 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2ebc043040 con 0x7f2ecc05bd90 2022-07-21T01:50:25.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.615+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/2062213133 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2ecc067840 con 0x7f2ecc05bd90 2022-07-21T01:50:25.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.615+0000 7f2ec9ffb700 1 -- 172.21.15.140:0/2062213133 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) v1 ==== 84+0+143 (secure 0 0 0) 0x7f2ebc046660 con 0x7f2ecc05bd90 2022-07-21T01:50:25.618 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:25.618 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:50:25.619 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:25.619 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:50:25.620 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:50:25.620 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:25.620 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:50:25.620 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:50:25.620 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:50:25.621 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:25.621 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:50:25.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/2062213133 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2eb0053a70 msgr2=0x7f2eb0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2eb0053a70 0x7f2eb0055f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f2eb4018000 tx=0x7f2eb4016000).stop 2022-07-21T01:50:25.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/2062213133 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ecc05bd90 msgr2=0x7f2ecc064480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ecc05bd90 0x7f2ecc064480 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f2ecc064010 tx=0x7f2ebc008040).stop 2022-07-21T01:50:25.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/2062213133 shutdown_connections 2022-07-21T01:50:25.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2eb0053a70 0x7f2eb0055f30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ecc0b5650 0x7f2ecc1555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2ecc0b1200 0x7f2ecc064970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 --2- 172.21.15.140:0/2062213133 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2ecc05bd90 0x7f2ecc064480 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/2062213133 >> 172.21.15.140:0/2062213133 conn(0x7f2ecc0634e0 msgr2=0x7f2ecc0afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:25.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/2062213133 shutdown_connections 2022-07-21T01:50:25.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.619+0000 7f2ed1ef2700 1 -- 172.21.15.140:0/2062213133 wait complete. 2022-07-21T01:50:25.631 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:50:25.633 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:50:25.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.739+0000 7fcf6dacd700 1 Processor -- start 2022-07-21T01:50:25.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.739+0000 7fcf6dacd700 1 -- start start 2022-07-21T01:50:25.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.739+0000 7fcf6dacd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcf680b5b50 0x7fcf680accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.739+0000 7fcf6dacd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf680b0480 0x7fcf6814a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.739+0000 7fcf6dacd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcf6814ab60 0x7fcf68144320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.739+0000 7fcf6dacd700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf680ac5e0 con 0x7fcf6814ab60 2022-07-21T01:50:25.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.739+0000 7fcf6dacd700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf680b5f30 con 0x7fcf680b5b50 2022-07-21T01:50:25.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.739+0000 7fcf6dacd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6805cf70 con 0x7fcf680b0480 2022-07-21T01:50:25.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf67fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcf6814ab60 0x7fcf68144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf67fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcf6814ab60 0x7fcf68144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34334/0 (socket says 172.21.15.140:34334) 2022-07-21T01:50:25.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf67fff700 1 -- 172.21.15.140:0/680709096 learned_addr learned my addr 172.21.15.140:0/680709096 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:25.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf677fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcf680b5b50 0x7fcf680accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf66ffd700 1 --2- 172.21.15.140:0/680709096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf680b0480 0x7fcf6814a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf67fff700 1 -- 172.21.15.140:0/680709096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf680b0480 msgr2=0x7fcf6814a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:25.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf67fff700 1 --2- 172.21.15.140:0/680709096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf680b0480 0x7fcf6814a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf67fff700 1 -- 172.21.15.140:0/680709096 >> v2:172.21.15.152:3300/0 conn(0x7fcf680b5b50 msgr2=0x7fcf680accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:25.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf67fff700 1 --2- 172.21.15.140:0/680709096 >> v2:172.21.15.152:3300/0 conn(0x7fcf680b5b50 0x7fcf680accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf67fff700 1 -- 172.21.15.140:0/680709096 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf680ac810 con 0x7fcf6814ab60 2022-07-21T01:50:25.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf677fe700 1 --2- 172.21.15.140:0/680709096 >> v2:172.21.15.152:3300/0 conn(0x7fcf680b5b50 0x7fcf680accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:25.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf67fff700 1 --2- 172.21.15.140:0/680709096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf6814ab60 0x7fcf68144320 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fcf60002f70 tx=0x7fcf60006eb0).ready entity=mon.0 client_cookie=59f7a390ba49ddcc server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:25.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf667fc700 1 -- 172.21.15.140:0/680709096 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6000d570 con 0x7fcf6814ab60 2022-07-21T01:50:25.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf667fc700 1 -- 172.21.15.140:0/680709096 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf60000e90 con 0x7fcf6814ab60 2022-07-21T01:50:25.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf667fc700 1 -- 172.21.15.140:0/680709096 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6000d900 con 0x7fcf6814ab60 2022-07-21T01:50:25.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf6dacd700 1 -- 172.21.15.140:0/680709096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf6814ab60 msgr2=0x7fcf68144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf6dacd700 1 --2- 172.21.15.140:0/680709096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf6814ab60 0x7fcf68144320 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7fcf60002f70 tx=0x7fcf60006eb0).stop 2022-07-21T01:50:25.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf6dacd700 1 -- 172.21.15.140:0/680709096 shutdown_connections 2022-07-21T01:50:25.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf6dacd700 1 --2- 172.21.15.140:0/680709096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf6814ab60 0x7fcf68144320 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf6dacd700 1 --2- 172.21.15.140:0/680709096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf680b0480 0x7fcf6814a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf6dacd700 1 --2- 172.21.15.140:0/680709096 >> v2:172.21.15.152:3300/0 conn(0x7fcf680b5b50 0x7fcf680accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf6dacd700 1 -- 172.21.15.140:0/680709096 >> 172.21.15.140:0/680709096 conn(0x7fcf680635e0 msgr2=0x7fcf680b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:25.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf6dacd700 1 -- 172.21.15.140:0/680709096 shutdown_connections 2022-07-21T01:50:25.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.743+0000 7fcf6dacd700 1 -- 172.21.15.140:0/680709096 wait complete. 2022-07-21T01:50:25.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf6dacd700 1 Processor -- start 2022-07-21T01:50:25.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf6dacd700 1 -- start start 2022-07-21T01:50:25.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf6dacd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf680b0480 0x7fcf680b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf6dacd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf680b5b50 0x7fcf6809e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf6dacd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf6814ab60 0x7fcf6809ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf6dacd700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6809e080 con 0x7fcf680b0480 2022-07-21T01:50:25.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf6dacd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680a1b00 con 0x7fcf6814ab60 2022-07-21T01:50:25.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf6dacd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6814a0a0 con 0x7fcf680b5b50 2022-07-21T01:50:25.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf677fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf680b0480 0x7fcf680b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf677fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf680b0480 0x7fcf680b32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34336/0 (socket says 172.21.15.140:34336) 2022-07-21T01:50:25.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf677fe700 1 -- 172.21.15.140:0/2107327026 learned_addr learned my addr 172.21.15.140:0/2107327026 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:25.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf67fff700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf6814ab60 0x7fcf6809ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf66ffd700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf680b5b50 0x7fcf6809e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf677fe700 1 -- 172.21.15.140:0/2107327026 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf680b5b50 msgr2=0x7fcf6809e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf677fe700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf680b5b50 0x7fcf6809e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf677fe700 1 -- 172.21.15.140:0/2107327026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf6814ab60 msgr2=0x7fcf6809ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:25.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf677fe700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf6814ab60 0x7fcf6809ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:25.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf677fe700 1 -- 172.21.15.140:0/2107327026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c00b180 con 0x7fcf680b0480 2022-07-21T01:50:25.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf67fff700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf6814ab60 0x7fcf6809ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:25.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf677fe700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf680b0480 0x7fcf680b32b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fcf5c00c000 tx=0x7fcf5c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:25.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf64ff9700 1 -- 172.21.15.140:0/2107327026 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf5c00c890 con 0x7fcf680b0480 2022-07-21T01:50:25.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf64ff9700 1 -- 172.21.15.140:0/2107327026 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5c00c9f0 con 0x7fcf680b0480 2022-07-21T01:50:25.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf64ff9700 1 -- 172.21.15.140:0/2107327026 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf5c00cd50 con 0x7fcf680b0480 2022-07-21T01:50:25.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf6dacd700 1 -- 172.21.15.140:0/2107327026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6000b1e0 con 0x7fcf680b0480 2022-07-21T01:50:25.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.747+0000 7fcf6dacd700 1 -- 172.21.15.140:0/2107327026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6809ebc0 con 0x7fcf680b0480 2022-07-21T01:50:25.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.751+0000 7fcf6dacd700 1 -- 172.21.15.140:0/2107327026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf680a1940 con 0x7fcf680b0480 2022-07-21T01:50:25.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.751+0000 7fcf64ff9700 1 -- 172.21.15.140:0/2107327026 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcf5c01c460 con 0x7fcf680b0480 2022-07-21T01:50:25.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.751+0000 7fcf64ff9700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf48053a70 0x7fcf48055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:25.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.751+0000 7fcf64ff9700 1 -- 172.21.15.140:0/2107327026 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fcf5c06c180 con 0x7fcf680b0480 2022-07-21T01:50:25.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.751+0000 7fcf66ffd700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf48053a70 0x7fcf48055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:25.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.751+0000 7fcf66ffd700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf48053a70 0x7fcf48055f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fcf5800b000 tx=0x7fcf580091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:25.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.759+0000 7fcf64ff9700 1 -- 172.21.15.140:0/2107327026 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcf5c043370 con 0x7fcf680b0480 2022-07-21T01:50:25.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:25.883+0000 7fcf6dacd700 1 -- 172.21.15.140:0/2107327026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fcf6805d210 con 0x7fcf680b0480 2022-07-21T01:50:26.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.063+0000 7fcf64ff9700 1 -- 172.21.15.140:0/2107327026 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v38) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcf5c046990 con 0x7fcf680b0480 2022-07-21T01:50:26.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 -- 172.21.15.140:0/2107327026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf48053a70 msgr2=0x7fcf48055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:26.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf48053a70 0x7fcf48055f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fcf5800b000 tx=0x7fcf580091d0).stop 2022-07-21T01:50:26.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 -- 172.21.15.140:0/2107327026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf680b0480 msgr2=0x7fcf680b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:26.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf680b0480 0x7fcf680b32b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fcf5c00c000 tx=0x7fcf5c006df0).stop 2022-07-21T01:50:26.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 -- 172.21.15.140:0/2107327026 shutdown_connections 2022-07-21T01:50:26.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf48053a70 0x7fcf48055f30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:26.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf6814ab60 0x7fcf6809ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:26.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf680b5b50 0x7fcf6809e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:26.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 --2- 172.21.15.140:0/2107327026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf680b0480 0x7fcf680b32b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:26.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 -- 172.21.15.140:0/2107327026 >> 172.21.15.140:0/2107327026 conn(0x7fcf680635e0 msgr2=0x7fcf68143980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:26.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 -- 172.21.15.140:0/2107327026 shutdown_connections 2022-07-21T01:50:26.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.067+0000 7fcf6dacd700 1 -- 172.21.15.140:0/2107327026 wait complete. 2022-07-21T01:50:26.073 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:50:26.081 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:50:26.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.183+0000 7fa25b59e700 1 Processor -- start 2022-07-21T01:50:26.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.183+0000 7fa25b59e700 1 -- start start 2022-07-21T01:50:26.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.183+0000 7fa25b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa25c0b32a0 0x7fa25c1468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:26.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.183+0000 7fa25b59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c0b3f10 con 0x7fa25c05d160 2022-07-21T01:50:26.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.183+0000 7fa25b59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c0b2620 con 0x7fa25c0b2ec0 2022-07-21T01:50:26.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.183+0000 7fa25b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0b4ea0 con 0x7fa25c0b32a0 2022-07-21T01:50:26.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.183+0000 7fa253fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa25c05d160 0x7fa25c1461c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48492/0 (socket says 172.21.15.140:48492) 2022-07-21T01:50:26.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.183+0000 7fa253fff700 1 -- 172.21.15.140:0/1587592957 learned_addr learned my addr 172.21.15.140:0/1587592957 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:26.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25ad9d700 1 --2- 172.21.15.140:0/1587592957 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa25c0b32a0 0x7fa25c1468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:26.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa259d9b700 1 -- 172.21.15.140:0/1587592957 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990515543 0 0) 0x7fa25c0b3f10 con 0x7fa25c05d160 2022-07-21T01:50:26.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa259d9b700 1 -- 172.21.15.140:0/1587592957 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa23c003260 con 0x7fa25c05d160 2022-07-21T01:50:26.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa259d9b700 1 -- 172.21.15.140:0/1587592957 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557222272 0 0) 0x7fa25c0b2620 con 0x7fa25c0b2ec0 2022-07-21T01:50:26.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa259d9b700 1 -- 172.21.15.140:0/1587592957 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa25c0b3f10 con 0x7fa25c0b2ec0 2022-07-21T01:50:26.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa259d9b700 1 -- 172.21.15.140:0/1587592957 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3492134111 0 0) 0x7fa23c003260 con 0x7fa25c05d160 2022-07-21T01:50:26.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25ad9d700 1 -- 172.21.15.140:0/1587592957 >> v1:172.21.15.152:6789/0 conn(0x7fa25c0b2ec0 legacy=0x7fa25c142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:26.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25ad9d700 1 -- 172.21.15.140:0/1587592957 >> v1:172.21.15.140:6789/0 conn(0x7fa25c05d160 legacy=0x7fa25c1461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:26.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25ad9d700 1 -- 172.21.15.140:0/1587592957 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c0b28c0 con 0x7fa25c0b32a0 2022-07-21T01:50:26.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25ad9d700 1 --2- 172.21.15.140:0/1587592957 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa25c0b32a0 0x7fa25c1468d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fa244001110 tx=0x7fa2540091d0).ready entity=mon.2 client_cookie=88249005104687a3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:26.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa259d9b700 1 -- 172.21.15.140:0/1587592957 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa244001660 con 0x7fa25c0b32a0 2022-07-21T01:50:26.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa259d9b700 1 -- 172.21.15.140:0/1587592957 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400f510 con 0x7fa25c0b32a0 2022-07-21T01:50:26.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa259d9b700 1 -- 172.21.15.140:0/1587592957 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa25400f680 con 0x7fa25c0b32a0 2022-07-21T01:50:26.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25b59e700 1 -- 172.21.15.140:0/1587592957 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa25c0b32a0 msgr2=0x7fa25c1468d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:26.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25b59e700 1 --2- 172.21.15.140:0/1587592957 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa25c0b32a0 0x7fa25c1468d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fa244001110 tx=0x7fa2540091d0).stop 2022-07-21T01:50:26.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25b59e700 1 -- 172.21.15.140:0/1587592957 shutdown_connections 2022-07-21T01:50:26.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25b59e700 1 --2- 172.21.15.140:0/1587592957 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa25c0b32a0 0x7fa25c1468d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:26.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25b59e700 1 -- 172.21.15.140:0/1587592957 >> 172.21.15.140:0/1587592957 conn(0x7fa25c066e20 msgr2=0x7fa25c067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:26.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25b59e700 1 -- 172.21.15.140:0/1587592957 shutdown_connections 2022-07-21T01:50:26.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.187+0000 7fa25b59e700 1 -- 172.21.15.140:0/1587592957 wait complete. 2022-07-21T01:50:26.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25b59e700 1 Processor -- start 2022-07-21T01:50:26.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25b59e700 1 -- start start 2022-07-21T01:50:26.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa25c05d160 0x7fa25c141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:26.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa25c0b2ec0 0x7fa25c155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:26.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa25c0b32a0 0x7fa25c156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:26.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c114fe0 con 0x7fa25c0b2ec0 2022-07-21T01:50:26.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c1168f0 con 0x7fa25c05d160 2022-07-21T01:50:26.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c116650 con 0x7fa25c0b32a0 2022-07-21T01:50:26.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa253fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa25c0b2ec0 0x7fa25c155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:26.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25a59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa25c05d160 0x7fa25c141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:26.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25a59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa25c05d160 0x7fa25c141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:40090/0 (socket says 172.21.15.140:40090) 2022-07-21T01:50:26.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25a59c700 1 -- 172.21.15.140:0/2743066534 learned_addr learned my addr 172.21.15.140:0/2743066534 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:26.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25ad9d700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa25c0b32a0 0x7fa25c156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:26.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25a59c700 1 -- 172.21.15.140:0/2743066534 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa25c0b32a0 msgr2=0x7fa25c156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:26.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25a59c700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa25c0b32a0 0x7fa25c156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:26.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25a59c700 1 -- 172.21.15.140:0/2743066534 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa25c0b2ec0 msgr2=0x7fa25c155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:26.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25a59c700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa25c0b2ec0 0x7fa25c155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:26.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25a59c700 1 -- 172.21.15.140:0/2743066534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa254009470 con 0x7fa25c05d160 2022-07-21T01:50:26.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25a59c700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa25c05d160 0x7fa25c141250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fa24c001110 tx=0x7fa24c007ae0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:26.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa2537fe700 1 -- 172.21.15.140:0/2743066534 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa24c012400 con 0x7fa25c05d160 2022-07-21T01:50:26.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa2537fe700 1 -- 172.21.15.140:0/2743066534 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c012560 con 0x7fa25c05d160 2022-07-21T01:50:26.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25b59e700 1 -- 172.21.15.140:0/2743066534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c112a40 con 0x7fa25c05d160 2022-07-21T01:50:26.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa2537fe700 1 -- 172.21.15.140:0/2743066534 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa24c0128c0 con 0x7fa25c05d160 2022-07-21T01:50:26.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.191+0000 7fa25b59e700 1 -- 172.21.15.140:0/2743066534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c156570 con 0x7fa25c05d160 2022-07-21T01:50:26.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.195+0000 7fa2537fe700 1 -- 172.21.15.140:0/2743066534 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa24c019030 con 0x7fa25c05d160 2022-07-21T01:50:26.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.195+0000 7fa25b59e700 1 -- 172.21.15.140:0/2743066534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25c112bd0 con 0x7fa25c05d160 2022-07-21T01:50:26.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.195+0000 7fa2537fe700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa240053a70 0x7fa240055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:26.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.195+0000 7fa2537fe700 1 -- 172.21.15.140:0/2743066534 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fa24c06b2d0 con 0x7fa25c05d160 2022-07-21T01:50:26.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.195+0000 7fa253fff700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa240053a70 0x7fa240055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:26.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.195+0000 7fa253fff700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa240053a70 0x7fa240055f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fa24400d000 tx=0x7fa244000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:26.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.203+0000 7fa2537fe700 1 -- 172.21.15.140:0/2743066534 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa24c042230 con 0x7fa25c05d160 2022-07-21T01:50:26.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:26.335+0000 7fa25b59e700 1 -- 172.21.15.140:0/2743066534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fa25c0b4a00 con 0x7fa25c05d160 2022-07-21T01:50:27.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.011+0000 7fa2537fe700 1 -- 172.21.15.140:0/2743066534 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v39) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa24c045850 con 0x7fa25c05d160 2022-07-21T01:50:27.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 -- 172.21.15.140:0/2743066534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa240053a70 msgr2=0x7fa240055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa240053a70 0x7fa240055f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fa24400d000 tx=0x7fa244000bc0).stop 2022-07-21T01:50:27.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 -- 172.21.15.140:0/2743066534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa25c05d160 msgr2=0x7fa25c141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa25c05d160 0x7fa25c141250 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fa24c001110 tx=0x7fa24c007ae0).stop 2022-07-21T01:50:27.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 -- 172.21.15.140:0/2743066534 shutdown_connections 2022-07-21T01:50:27.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa240053a70 0x7fa240055f30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa25c0b32a0 0x7fa25c156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa25c0b2ec0 0x7fa25c155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 --2- 172.21.15.140:0/2743066534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa25c05d160 0x7fa25c141250 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 -- 172.21.15.140:0/2743066534 >> 172.21.15.140:0/2743066534 conn(0x7fa25c066e20 msgr2=0x7fa25c13e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:27.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 -- 172.21.15.140:0/2743066534 shutdown_connections 2022-07-21T01:50:27.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.019+0000 7fa25b59e700 1 -- 172.21.15.140:0/2743066534 wait complete. 2022-07-21T01:50:27.022 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:50:27.031 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:50:27.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 Processor -- start 2022-07-21T01:50:27.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 -- start start 2022-07-21T01:50:27.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f62580b5990 0x7f6258146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f625805d210 0x7f625814a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6258146c70 con 0x7f62580af3e0 2022-07-21T01:50:27.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62580ad050 con 0x7f625805d210 2022-07-21T01:50:27.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62580b17e0 con 0x7f62580b5990 2022-07-21T01:50:27.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625759e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f625805d210 0x7f625814a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625659c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f62580af3e0 0x7f6258146560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48504/0 (socket says 172.21.15.140:48504) 2022-07-21T01:50:27.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625659c700 1 -- 172.21.15.140:0/2382632341 learned_addr learned my addr 172.21.15.140:0/2382632341 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:27.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f6256d9d700 1 --2- 172.21.15.140:0/2382632341 >> v2:172.21.15.152:3300/0 conn(0x7f62580b5990 0x7f6258146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f6255d9b700 1 -- 172.21.15.140:0/2382632341 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868619577 0 0) 0x7f6258146c70 con 0x7f62580af3e0 2022-07-21T01:50:27.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f6255d9b700 1 -- 172.21.15.140:0/2382632341 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6238003260 con 0x7f62580af3e0 2022-07-21T01:50:27.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f6255d9b700 1 -- 172.21.15.140:0/2382632341 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1284723132 0 0) 0x7f6238003260 con 0x7f62580af3e0 2022-07-21T01:50:27.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625759e700 1 -- 172.21.15.140:0/2382632341 >> v2:172.21.15.152:3300/0 conn(0x7f62580b5990 msgr2=0x7f6258146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:27.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625759e700 1 --2- 172.21.15.140:0/2382632341 >> v2:172.21.15.152:3300/0 conn(0x7f62580b5990 0x7f6258146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625759e700 1 -- 172.21.15.140:0/2382632341 >> v1:172.21.15.140:6789/0 conn(0x7f62580af3e0 legacy=0x7f6258146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625759e700 1 -- 172.21.15.140:0/2382632341 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62580b5d70 con 0x7f625805d210 2022-07-21T01:50:27.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625759e700 1 --2- 172.21.15.140:0/2382632341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f625805d210 0x7f625814a360 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f6248001800 tx=0x7f62500091d0).ready entity=mon.0 client_cookie=4bbc200a28c7b52c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:27.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f6255d9b700 1 -- 172.21.15.140:0/2382632341 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6248001660 con 0x7f625805d210 2022-07-21T01:50:27.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f6255d9b700 1 -- 172.21.15.140:0/2382632341 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625000f510 con 0x7f625805d210 2022-07-21T01:50:27.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f6255d9b700 1 -- 172.21.15.140:0/2382632341 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f625000f680 con 0x7f625805d210 2022-07-21T01:50:27.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 -- 172.21.15.140:0/2382632341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f625805d210 msgr2=0x7f625814a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 --2- 172.21.15.140:0/2382632341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f625805d210 0x7f625814a360 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f6248001800 tx=0x7f62500091d0).stop 2022-07-21T01:50:27.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 -- 172.21.15.140:0/2382632341 shutdown_connections 2022-07-21T01:50:27.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 --2- 172.21.15.140:0/2382632341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f625805d210 0x7f625814a360 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 --2- 172.21.15.140:0/2382632341 >> v2:172.21.15.152:3300/0 conn(0x7f62580b5990 0x7f6258146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 -- 172.21.15.140:0/2382632341 >> 172.21.15.140:0/2382632341 conn(0x7f6258063480 msgr2=0x7f62580b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:27.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 -- 172.21.15.140:0/2382632341 shutdown_connections 2022-07-21T01:50:27.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.139+0000 7f625c83a700 1 -- 172.21.15.140:0/2382632341 wait complete. 2022-07-21T01:50:27.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625c83a700 1 Processor -- start 2022-07-21T01:50:27.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625c83a700 1 -- start start 2022-07-21T01:50:27.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625c83a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f625805d210 0x7f625810e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625c83a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62580af3e0 0x7f62581406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625c83a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62580b5990 0x7f6258159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625c83a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6258066880 con 0x7f62580af3e0 2022-07-21T01:50:27.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625c83a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6258111eb0 con 0x7f62580b5990 2022-07-21T01:50:27.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625c83a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625810f370 con 0x7f625805d210 2022-07-21T01:50:27.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62580af3e0 0x7f62581406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62580af3e0 0x7f62581406c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34366/0 (socket says 172.21.15.140:34366) 2022-07-21T01:50:27.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625659c700 1 -- 172.21.15.140:0/1345070831 learned_addr learned my addr 172.21.15.140:0/1345070831 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:27.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625759e700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62580b5990 0x7f6258159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f6256d9d700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f625805d210 0x7f625810e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625659c700 1 -- 172.21.15.140:0/1345070831 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f625805d210 msgr2=0x7f625810e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625659c700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f625805d210 0x7f625810e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625659c700 1 -- 172.21.15.140:0/1345070831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62580b5990 msgr2=0x7f6258159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625659c700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62580b5990 0x7f6258159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625659c700 1 -- 172.21.15.140:0/1345070831 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6250009470 con 0x7f62580af3e0 2022-07-21T01:50:27.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625659c700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62580af3e0 0x7f62581406c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f62480018c0 tx=0x7f6248001b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:27.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f624ffff700 1 -- 172.21.15.140:0/1345070831 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6248007170 con 0x7f62580af3e0 2022-07-21T01:50:27.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f624ffff700 1 -- 172.21.15.140:0/1345070831 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62480072d0 con 0x7f62580af3e0 2022-07-21T01:50:27.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625c83a700 1 -- 172.21.15.140:0/1345070831 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6258067400 con 0x7f62580af3e0 2022-07-21T01:50:27.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f624ffff700 1 -- 172.21.15.140:0/1345070831 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6248004920 con 0x7f62580af3e0 2022-07-21T01:50:27.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.143+0000 7f625c83a700 1 -- 172.21.15.140:0/1345070831 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6258066bf0 con 0x7f62580af3e0 2022-07-21T01:50:27.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.147+0000 7f625c83a700 1 -- 172.21.15.140:0/1345070831 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6258112070 con 0x7f62580af3e0 2022-07-21T01:50:27.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.147+0000 7f624ffff700 1 -- 172.21.15.140:0/1345070831 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f624801b070 con 0x7f62580af3e0 2022-07-21T01:50:27.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.147+0000 7f624ffff700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f623c053aa0 0x7f623c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.147+0000 7f624ffff700 1 -- 172.21.15.140:0/1345070831 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f624806b710 con 0x7f62580af3e0 2022-07-21T01:50:27.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.147+0000 7f6256d9d700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f623c053aa0 0x7f623c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.147+0000 7f6256d9d700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f623c053aa0 0x7f623c055f60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f6240006000 tx=0x7f6240006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:27.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.155+0000 7f624ffff700 1 -- 172.21.15.140:0/1345070831 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6248042600 con 0x7f62580af3e0 2022-07-21T01:50:27.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.319+0000 7f625c83a700 1 -- 172.21.15.140:0/1345070831 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f62580afa60 con 0x7f62580af3e0 2022-07-21T01:50:27.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.327+0000 7f624ffff700 1 -- 172.21.15.140:0/1345070831 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v15) v1 ==== 187+0+0 (secure 0 0 0) 0x7f6248019020 con 0x7f62580af3e0 2022-07-21T01:50:27.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 -- 172.21.15.140:0/1345070831 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f623c053aa0 msgr2=0x7f623c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f623c053aa0 0x7f623c055f60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f6240006000 tx=0x7f6240006d90).stop 2022-07-21T01:50:27.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 -- 172.21.15.140:0/1345070831 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62580af3e0 msgr2=0x7f62581406c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62580af3e0 0x7f62581406c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f62480018c0 tx=0x7f6248001b30).stop 2022-07-21T01:50:27.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 -- 172.21.15.140:0/1345070831 shutdown_connections 2022-07-21T01:50:27.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f623c053aa0 0x7f623c055f60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62580b5990 0x7f6258159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62580af3e0 0x7f62581406c0 secure :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0x7f62480018c0 tx=0x7f6248001b30).stop 2022-07-21T01:50:27.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 --2- 172.21.15.140:0/1345070831 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f625805d210 0x7f625810e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 -- 172.21.15.140:0/1345070831 >> 172.21.15.140:0/1345070831 conn(0x7f6258063480 msgr2=0x7f6258145e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:27.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 -- 172.21.15.140:0/1345070831 shutdown_connections 2022-07-21T01:50:27.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.335+0000 7f625c83a700 1 -- 172.21.15.140:0/1345070831 wait complete. 2022-07-21T01:50:27.346 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 7 and data pool 8 2022-07-21T01:50:27.350 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:50:27.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205ffff700 1 Processor -- start 2022-07-21T01:50:27.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205ffff700 1 -- start start 2022-07-21T01:50:27.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f20580d2380 0x7f205803d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205ffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2058047a20 con 0x7f2058047640 2022-07-21T01:50:27.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205ffff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2058042d40 con 0x7f2058042510 2022-07-21T01:50:27.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20580cc260 con 0x7f20580d2380 2022-07-21T01:50:27.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205effd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2058047640 0x7f205803a600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48528/0 (socket says 172.21.15.140:48528) 2022-07-21T01:50:27.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205effd700 1 -- 172.21.15.140:0/3943244237 learned_addr learned my addr 172.21.15.140:0/3943244237 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:27.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205f7fe700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f20580d2380 0x7f205803d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186384538 0 0) 0x7f2058047a20 con 0x7f2058047640 2022-07-21T01:50:27.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2044003260 con 0x7f2058047640 2022-07-21T01:50:27.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3468009259 0 0) 0x7f2044003260 con 0x7f2058047640 2022-07-21T01:50:27.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f20580d2380 msgr2=0x7f205803d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:27.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f20580d2380 0x7f205803d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 >> v1:172.21.15.152:6789/0 conn(0x7f2058042510 legacy=0x7f20580d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2058039ec0 con 0x7f2058047640 2022-07-21T01:50:27.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f2050001660 con 0x7f2058047640 2022-07-21T01:50:27.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 >> v1:172.21.15.140:6789/0 conn(0x7f2058047640 legacy=0x7f205803a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2044003780 0x7f2044003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2044004090 0x7f2044004560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2044004d10 0x7f204400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20580cc260 con 0x7f2044004d10 2022-07-21T01:50:27.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204400bc40 con 0x7f2044004090 2022-07-21T01:50:27.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204400bd80 con 0x7f2044003780 2022-07-21T01:50:27.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205f7fe700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2044004d10 0x7f204400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205effd700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2044003780 0x7f2044003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.447+0000 7f205e7fc700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2044004090 0x7f2044004560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205f7fe700 1 -- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2044003780 msgr2=0x7f2044003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:27.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205f7fe700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2044003780 0x7f2044003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205f7fe700 1 -- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2044004090 msgr2=0x7f2044004560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:27.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205f7fe700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2044004090 0x7f2044004560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205f7fe700 1 -- 172.21.15.140:0/3943244237 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2044003260 con 0x7f2044004d10 2022-07-21T01:50:27.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205effd700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2044003780 0x7f2044003b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:27.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205f7fe700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2044004d10 0x7f204400b750 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f2060065000 tx=0x7f206005f750).ready entity=mon.0 client_cookie=d0ab77850fbd01fa server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:27.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2060085070 con 0x7f2044004d10 2022-07-21T01:50:27.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20600447a0 con 0x7f2044004d10 2022-07-21T01:50:27.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205dffb700 1 -- 172.21.15.140:0/3943244237 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f206006a720 con 0x7f2044004d10 2022-07-21T01:50:27.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205ffff700 1 -- 172.21.15.140:0/3943244237 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2044004d10 msgr2=0x7f204400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205ffff700 1 --2- 172.21.15.140:0/3943244237 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2044004d10 0x7f204400b750 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f2060065000 tx=0x7f206005f750).stop 2022-07-21T01:50:27.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205f7fe700 1 -- 172.21.15.140:0/3943244237 reap_dead start 2022-07-21T01:50:27.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205ffff700 1 -- 172.21.15.140:0/3943244237 shutdown_connections 2022-07-21T01:50:27.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205ffff700 1 -- 172.21.15.140:0/3943244237 >> 172.21.15.140:0/3943244237 conn(0x7f205800da40 msgr2=0x7f2058046bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:27.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205ffff700 1 -- 172.21.15.140:0/3943244237 shutdown_connections 2022-07-21T01:50:27.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205ffff700 1 -- 172.21.15.140:0/3943244237 wait complete. 2022-07-21T01:50:27.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205ffff700 1 Processor -- start 2022-07-21T01:50:27.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205ffff700 1 -- start start 2022-07-21T01:50:27.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2044004d10 0x7f20580d4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.451+0000 7f205ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20580d3940 0x7f20580d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f20580c8290 0x7f20580c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20580d5110 con 0x7f20580c8290 2022-07-21T01:50:27.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20580d3340 con 0x7f20580d3940 2022-07-21T01:50:27.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20580cbc00 con 0x7f2044004d10 2022-07-21T01:50:27.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f20580c8290 0x7f20580c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f20580c8290 0x7f20580c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34398/0 (socket says 172.21.15.140:34398) 2022-07-21T01:50:27.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205f7fe700 1 -- 172.21.15.140:0/699669885 learned_addr learned my addr 172.21.15.140:0/699669885 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:27.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205effd700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2044004d10 0x7f20580d4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205e7fc700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20580d3940 0x7f20580d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205f7fe700 1 -- 172.21.15.140:0/699669885 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2044004d10 msgr2=0x7f20580d4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205f7fe700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2044004d10 0x7f20580d4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205f7fe700 1 -- 172.21.15.140:0/699669885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20580d3940 msgr2=0x7f20580d57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205f7fe700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20580d3940 0x7f20580d57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205f7fe700 1 -- 172.21.15.140:0/699669885 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20580cdbb0 con 0x7f20580c8290 2022-07-21T01:50:27.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205e7fc700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20580d3940 0x7f20580d57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:27.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205f7fe700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f20580c8290 0x7f20580c8710 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f205803a4e0 tx=0x7f2060082000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:27.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f204ffff700 1 -- 172.21.15.140:0/699669885 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2060091070 con 0x7f20580c8290 2022-07-21T01:50:27.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f204ffff700 1 -- 172.21.15.140:0/699669885 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206005dd00 con 0x7f20580c8290 2022-07-21T01:50:27.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205ffff700 1 -- 172.21.15.140:0/699669885 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20580cde50 con 0x7f20580c8290 2022-07-21T01:50:27.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f204ffff700 1 -- 172.21.15.140:0/699669885 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2060026300 con 0x7f20580c8290 2022-07-21T01:50:27.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205ffff700 1 -- 172.21.15.140:0/699669885 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20580d3540 con 0x7f20580c8290 2022-07-21T01:50:27.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f204ffff700 1 -- 172.21.15.140:0/699669885 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f206005d1e0 con 0x7f20580c8290 2022-07-21T01:50:27.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205ffff700 1 -- 172.21.15.140:0/699669885 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205803a160 con 0x7f20580c8290 2022-07-21T01:50:27.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f204ffff700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f203c053b30 0x7f203c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f204ffff700 1 -- 172.21.15.140:0/699669885 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f20600daa70 con 0x7f20580c8290 2022-07-21T01:50:27.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205effd700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f203c053b30 0x7f203c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.455+0000 7f205effd700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f203c053b30 0x7f203c055ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f205000d000 tx=0x7f2050007040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:27.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.463+0000 7f204ffff700 1 -- 172.21.15.140:0/699669885 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f20600b1890 con 0x7f20580c8290 2022-07-21T01:50:27.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.579+0000 7f205ffff700 1 -- 172.21.15.140:0/699669885 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2058005270 con 0x7f20580c8290 2022-07-21T01:50:27.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.579+0000 7f204ffff700 1 -- 172.21.15.140:0/699669885 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16160 (secure 0 0 0) 0x7f20600b4eb0 con 0x7f20580c8290 2022-07-21T01:50:27.584 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:27.584 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":40,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:50:27.322190+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:50:25.939192+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-07-21T01:50:26.391519+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:50:27.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 -- 172.21.15.140:0/699669885 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f203c053b30 msgr2=0x7f203c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f203c053b30 0x7f203c055ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f205000d000 tx=0x7f2050007040).stop 2022-07-21T01:50:27.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 -- 172.21.15.140:0/699669885 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f20580c8290 msgr2=0x7f20580c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f20580c8290 0x7f20580c8710 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f205803a4e0 tx=0x7f2060082000).stop 2022-07-21T01:50:27.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 -- 172.21.15.140:0/699669885 shutdown_connections 2022-07-21T01:50:27.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f203c053b30 0x7f203c055ff0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f20580c8290 0x7f20580c8710 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f20580d3940 0x7f20580d57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 --2- 172.21.15.140:0/699669885 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2044004d10 0x7f20580d4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 -- 172.21.15.140:0/699669885 >> 172.21.15.140:0/699669885 conn(0x7f205800da40 msgr2=0x7f205803ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:27.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 -- 172.21.15.140:0/699669885 shutdown_connections 2022-07-21T01:50:27.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.583+0000 7f205ffff700 1 -- 172.21.15.140:0/699669885 wait complete. 2022-07-21T01:50:27.597 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:50:27.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f352e5b8700 1 Processor -- start 2022-07-21T01:50:27.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f352e5b8700 1 -- start start 2022-07-21T01:50:27.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f352e5b8700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f35280b1e00 0x7f352814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f352e5b8700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352805d480 con 0x7f35280acef0 2022-07-21T01:50:27.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f352e5b8700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35280b21e0 con 0x7f352805c3f0 2022-07-21T01:50:27.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f352e5b8700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352805cf40 con 0x7f35280b1e00 2022-07-21T01:50:27.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f352cb55700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f35280acef0 0x7f3528146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48552/0 (socket says 172.21.15.140:48552) 2022-07-21T01:50:27.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f352cb55700 1 -- 172.21.15.140:0/2978254620 learned_addr learned my addr 172.21.15.140:0/2978254620 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:27.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3527fff700 1 --2- 172.21.15.140:0/2978254620 >> v2:172.21.15.152:3300/0 conn(0x7f35280b1e00 0x7f352814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3526ffd700 1 -- 172.21.15.140:0/2978254620 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616070387 0 0) 0x7f352805d480 con 0x7f35280acef0 2022-07-21T01:50:27.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3526ffd700 1 -- 172.21.15.140:0/2978254620 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3508003260 con 0x7f35280acef0 2022-07-21T01:50:27.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3526ffd700 1 -- 172.21.15.140:0/2978254620 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083464841 0 0) 0x7f35280b21e0 con 0x7f352805c3f0 2022-07-21T01:50:27.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3526ffd700 1 -- 172.21.15.140:0/2978254620 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f352805d480 con 0x7f352805c3f0 2022-07-21T01:50:27.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3527fff700 1 -- 172.21.15.140:0/2978254620 >> v1:172.21.15.152:6789/0 conn(0x7f352805c3f0 legacy=0x7f35280ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3527fff700 1 -- 172.21.15.140:0/2978254620 >> v1:172.21.15.140:6789/0 conn(0x7f35280acef0 legacy=0x7f3528146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3526ffd700 1 -- 172.21.15.140:0/2978254620 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 977091100 0 0) 0x7f3508003260 con 0x7f35280acef0 2022-07-21T01:50:27.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3527fff700 1 -- 172.21.15.140:0/2978254620 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352805d1e0 con 0x7f35280b1e00 2022-07-21T01:50:27.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3527fff700 1 --2- 172.21.15.140:0/2978254620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35280b1e00 0x7f352814a6d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f3518001110 tx=0x7f3510006df0).ready entity=mon.1 client_cookie=1eaf111af45c23ee server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:27.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3526ffd700 1 -- 172.21.15.140:0/2978254620 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3518001660 con 0x7f35280b1e00 2022-07-21T01:50:27.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3526ffd700 1 -- 172.21.15.140:0/2978254620 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3510000e90 con 0x7f35280b1e00 2022-07-21T01:50:27.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.699+0000 7f3526ffd700 1 -- 172.21.15.140:0/2978254620 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3510012460 con 0x7f35280b1e00 2022-07-21T01:50:27.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.703+0000 7f352e5b8700 1 -- 172.21.15.140:0/2978254620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35280b1e00 msgr2=0x7f352814a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.703+0000 7f352e5b8700 1 --2- 172.21.15.140:0/2978254620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35280b1e00 0x7f352814a6d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f3518001110 tx=0x7f3510006df0).stop 2022-07-21T01:50:27.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.703+0000 7f352e5b8700 1 -- 172.21.15.140:0/2978254620 shutdown_connections 2022-07-21T01:50:27.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.703+0000 7f352e5b8700 1 --2- 172.21.15.140:0/2978254620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35280b1e00 0x7f352814a6d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.703+0000 7f352e5b8700 1 -- 172.21.15.140:0/2978254620 >> 172.21.15.140:0/2978254620 conn(0x7f35280611c0 msgr2=0x7f3528062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:27.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.703+0000 7f352e5b8700 1 -- 172.21.15.140:0/2978254620 shutdown_connections 2022-07-21T01:50:27.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.703+0000 7f352e5b8700 1 -- 172.21.15.140:0/2978254620 wait complete. 2022-07-21T01:50:27.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.703+0000 7f352e5b8700 1 Processor -- start 2022-07-21T01:50:27.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.703+0000 7f352e5b8700 1 -- start start 2022-07-21T01:50:27.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352e5b8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f35280acef0 0x7f35281474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352e5b8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35280b1e00 0x7f3528142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352e5b8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35281435b0 0x7f3528142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352e5b8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352814a070 con 0x7f35281435b0 2022-07-21T01:50:27.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352e5b8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3528149e10 con 0x7f35280b1e00 2022-07-21T01:50:27.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352e5b8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3528143110 con 0x7f35280acef0 2022-07-21T01:50:27.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352cb55700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35281435b0 0x7f3528142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352cb55700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35281435b0 0x7f3528142810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34412/0 (socket says 172.21.15.140:34412) 2022-07-21T01:50:27.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352cb55700 1 -- 172.21.15.140:0/981679814 learned_addr learned my addr 172.21.15.140:0/981679814 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:27.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f35277fe700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35280b1e00 0x7f3528142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f3527fff700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f35280acef0 0x7f35281474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352cb55700 1 -- 172.21.15.140:0/981679814 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f35280acef0 msgr2=0x7f35281474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352cb55700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f35280acef0 0x7f35281474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352cb55700 1 -- 172.21.15.140:0/981679814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35280b1e00 msgr2=0x7f3528142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352cb55700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35280b1e00 0x7f3528142320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352cb55700 1 -- 172.21.15.140:0/981679814 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351000b180 con 0x7f35281435b0 2022-07-21T01:50:27.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f35277fe700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35280b1e00 0x7f3528142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:27.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f3527fff700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f35280acef0 0x7f35281474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:27.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352cb55700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35281435b0 0x7f3528142810 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f3518001840 tx=0x7f3518001680).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:27.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f35257fa700 1 -- 172.21.15.140:0/981679814 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f351800ce30 con 0x7f35281435b0 2022-07-21T01:50:27.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f35257fa700 1 -- 172.21.15.140:0/981679814 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3518007170 con 0x7f35281435b0 2022-07-21T01:50:27.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352e5b8700 1 -- 172.21.15.140:0/981679814 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3528142d00 con 0x7f35281435b0 2022-07-21T01:50:27.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f35257fa700 1 -- 172.21.15.140:0/981679814 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3518004730 con 0x7f35281435b0 2022-07-21T01:50:27.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.707+0000 7f352e5b8700 1 -- 172.21.15.140:0/981679814 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3528064e80 con 0x7f35281435b0 2022-07-21T01:50:27.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.711+0000 7f352e5b8700 1 -- 172.21.15.140:0/981679814 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3528143310 con 0x7f35281435b0 2022-07-21T01:50:27.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.711+0000 7f35257fa700 1 -- 172.21.15.140:0/981679814 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f351801b070 con 0x7f35281435b0 2022-07-21T01:50:27.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.711+0000 7f35257fa700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f350c0539e0 0x7f350c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.711+0000 7f35257fa700 1 -- 172.21.15.140:0/981679814 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f351806b620 con 0x7f35281435b0 2022-07-21T01:50:27.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.711+0000 7f3527fff700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f350c0539e0 0x7f350c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.711+0000 7f3527fff700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f350c0539e0 0x7f350c055ea0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f3510018000 tx=0x7f3510014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:27.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.715+0000 7f35257fa700 1 -- 172.21.15.140:0/981679814 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3528143310 con 0x7f35281435b0 2022-07-21T01:50:27.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.847+0000 7f352e5b8700 1 -- 172.21.15.140:0/981679814 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35280b04b0 con 0x7f35281435b0 2022-07-21T01:50:27.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.847+0000 7f35257fa700 1 -- 172.21.15.140:0/981679814 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16160 (secure 0 0 0) 0x7f3518045f70 con 0x7f35281435b0 2022-07-21T01:50:27.849 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:27.849 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":40,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:50:27.322190+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:50:25.939192+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-07-21T01:50:26.391519+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:50:27.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.847+0000 7f352e5b8700 1 -- 172.21.15.140:0/981679814 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f350c0539e0 msgr2=0x7f350c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.847+0000 7f352e5b8700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f350c0539e0 0x7f350c055ea0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f3510018000 tx=0x7f3510014000).stop 2022-07-21T01:50:27.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.851+0000 7f352e5b8700 1 -- 172.21.15.140:0/981679814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35281435b0 msgr2=0x7f3528142810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.851+0000 7f352e5b8700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35281435b0 0x7f3528142810 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f3518001840 tx=0x7f3518001680).stop 2022-07-21T01:50:27.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.851+0000 7f352e5b8700 1 -- 172.21.15.140:0/981679814 shutdown_connections 2022-07-21T01:50:27.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.851+0000 7f352e5b8700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f350c0539e0 0x7f350c055ea0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.851+0000 7f352e5b8700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35281435b0 0x7f3528142810 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.851+0000 7f352e5b8700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35280b1e00 0x7f3528142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.851+0000 7f352e5b8700 1 --2- 172.21.15.140:0/981679814 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f35280acef0 0x7f35281474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.851+0000 7f352e5b8700 1 -- 172.21.15.140:0/981679814 >> 172.21.15.140:0/981679814 conn(0x7f352805c3f0 msgr2=0x7f35280ade90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:27.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.851+0000 7f352e5b8700 1 -- 172.21.15.140:0/981679814 shutdown_connections 2022-07-21T01:50:27.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.851+0000 7f352e5b8700 1 -- 172.21.15.140:0/981679814 wait complete. 2022-07-21T01:50:27.869 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:50:27.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.975+0000 7ff3d74e2700 1 Processor -- start 2022-07-21T01:50:27.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.975+0000 7ff3d74e2700 1 -- start start 2022-07-21T01:50:27.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.975+0000 7ff3d74e2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3d009df40 0x7ff3d0145680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.975+0000 7ff3d74e2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3d00b4d60 0x7ff3d0145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.975+0000 7ff3d74e2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d00607e0 con 0x7ff3d0060ee0 2022-07-21T01:50:27.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.975+0000 7ff3d74e2700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d0060d20 con 0x7ff3d00b4d60 2022-07-21T01:50:27.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.975+0000 7ff3d74e2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d0060a80 con 0x7ff3d009df40 2022-07-21T01:50:27.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.975+0000 7ff3d4a7d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3d00b4d60 0x7ff3d0145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.975+0000 7ff3d4a7d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3d00b4d60 0x7ff3d0145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34418/0 (socket says 172.21.15.140:34418) 2022-07-21T01:50:27.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.975+0000 7ff3d4a7d700 1 -- 172.21.15.140:0/1180691023 learned_addr learned my addr 172.21.15.140:0/1180691023 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:27.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.975+0000 7ff3d527e700 1 --2- 172.21.15.140:0/1180691023 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3d009df40 0x7ff3d0145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d4a7d700 1 -- 172.21.15.140:0/1180691023 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3d009df40 msgr2=0x7ff3d0145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:27.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d4a7d700 1 --2- 172.21.15.140:0/1180691023 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3d009df40 0x7ff3d0145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d4a7d700 1 -- 172.21.15.140:0/1180691023 >> v1:172.21.15.140:6789/0 conn(0x7ff3d0060ee0 legacy=0x7ff3d0146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3c7fff700 1 -- 172.21.15.140:0/1180691023 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759305335 0 0) 0x7ff3d00607e0 con 0x7ff3d0060ee0 2022-07-21T01:50:27.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d4a7d700 1 -- 172.21.15.140:0/1180691023 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d009e320 con 0x7ff3d00b4d60 2022-07-21T01:50:27.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d4a7d700 1 --2- 172.21.15.140:0/1180691023 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3d00b4d60 0x7ff3d0145b70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7ff3cc002f70 tx=0x7ff3cc006eb0).ready entity=mon.0 client_cookie=287b6955da93e009 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:27.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3c7fff700 1 -- 172.21.15.140:0/1180691023 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3cc00d570 con 0x7ff3d00b4d60 2022-07-21T01:50:27.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3c7fff700 1 -- 172.21.15.140:0/1180691023 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3cc000e90 con 0x7ff3d00b4d60 2022-07-21T01:50:27.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3c7fff700 1 -- 172.21.15.140:0/1180691023 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3cc00d8c0 con 0x7ff3d00b4d60 2022-07-21T01:50:27.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d74e2700 1 -- 172.21.15.140:0/1180691023 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3d00b4d60 msgr2=0x7ff3d0145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d74e2700 1 --2- 172.21.15.140:0/1180691023 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3d00b4d60 0x7ff3d0145b70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7ff3cc002f70 tx=0x7ff3cc006eb0).stop 2022-07-21T01:50:27.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d74e2700 1 -- 172.21.15.140:0/1180691023 shutdown_connections 2022-07-21T01:50:27.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d74e2700 1 --2- 172.21.15.140:0/1180691023 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3d00b4d60 0x7ff3d0145b70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d74e2700 1 --2- 172.21.15.140:0/1180691023 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3d009df40 0x7ff3d0145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d74e2700 1 -- 172.21.15.140:0/1180691023 >> 172.21.15.140:0/1180691023 conn(0x7ff3d00b5f30 msgr2=0x7ff3d00b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:27.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d74e2700 1 -- 172.21.15.140:0/1180691023 shutdown_connections 2022-07-21T01:50:27.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d74e2700 1 -- 172.21.15.140:0/1180691023 wait complete. 2022-07-21T01:50:27.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d74e2700 1 Processor -- start 2022-07-21T01:50:27.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.979+0000 7ff3d74e2700 1 -- start start 2022-07-21T01:50:27.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d74e2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3d0060ee0 0x7ff3d01558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d74e2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3d009df40 0x7ff3d0152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d74e2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3d00b4d60 0x7ff3d00ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:27.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d74e2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d01564c0 con 0x7ff3d00b4d60 2022-07-21T01:50:27.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d74e2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d01443b0 con 0x7ff3d0060ee0 2022-07-21T01:50:27.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d74e2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d00b11d0 con 0x7ff3d009df40 2022-07-21T01:50:27.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d5a7f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3d00b4d60 0x7ff3d00ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d5a7f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3d00b4d60 0x7ff3d00ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34426/0 (socket says 172.21.15.140:34426) 2022-07-21T01:50:27.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d527e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3d0060ee0 0x7ff3d01558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d5a7f700 1 -- 172.21.15.140:0/672305102 learned_addr learned my addr 172.21.15.140:0/672305102 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:27.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d4a7d700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3d009df40 0x7ff3d0152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:27.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d5a7f700 1 -- 172.21.15.140:0/672305102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3d009df40 msgr2=0x7ff3d0152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:27.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d5a7f700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3d009df40 0x7ff3d0152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:27.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d5a7f700 1 -- 172.21.15.140:0/672305102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3d0060ee0 msgr2=0x7ff3d01558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d5a7f700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3d0060ee0 0x7ff3d01558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d5a7f700 1 -- 172.21.15.140:0/672305102 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3cc00b1e0 con 0x7ff3d00b4d60 2022-07-21T01:50:28.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d4a7d700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3d009df40 0x7ff3d0152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:28.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d5a7f700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3d00b4d60 0x7ff3d00ae450 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7ff3c8000f40 tx=0x7ff3c8007d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:28.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3c67fc700 1 -- 172.21.15.140:0/672305102 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3c800e4a0 con 0x7ff3d00b4d60 2022-07-21T01:50:28.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3c67fc700 1 -- 172.21.15.140:0/672305102 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c800e600 con 0x7ff3d00b4d60 2022-07-21T01:50:28.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d74e2700 1 -- 172.21.15.140:0/672305102 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d01561a0 con 0x7ff3d00b4d60 2022-07-21T01:50:28.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3c67fc700 1 -- 172.21.15.140:0/672305102 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3c800e960 con 0x7ff3d00b4d60 2022-07-21T01:50:28.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d74e2700 1 -- 172.21.15.140:0/672305102 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d0155f00 con 0x7ff3d00b4d60 2022-07-21T01:50:28.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3d74e2700 1 -- 172.21.15.140:0/672305102 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d0152ad0 con 0x7ff3d00b4d60 2022-07-21T01:50:28.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.983+0000 7ff3c67fc700 1 -- 172.21.15.140:0/672305102 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff3c8010070 con 0x7ff3d00b4d60 2022-07-21T01:50:28.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.987+0000 7ff3c67fc700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3b8053b00 0x7ff3b8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.987+0000 7ff3c67fc700 1 -- 172.21.15.140:0/672305102 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff3c806b240 con 0x7ff3d00b4d60 2022-07-21T01:50:28.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.987+0000 7ff3d527e700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3b8053b00 0x7ff3b8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.987+0000 7ff3d527e700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3b8053b00 0x7ff3b8055fc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7ff3c0006000 tx=0x7ff3c0006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:28.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:27.995+0000 7ff3c67fc700 1 -- 172.21.15.140:0/672305102 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff3c8042160 con 0x7ff3d00b4d60 2022-07-21T01:50:28.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.159+0000 7ff3d74e2700 1 -- 172.21.15.140:0/672305102 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff3d0146060 con 0x7ff3d00b4d60 2022-07-21T01:50:28.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.331+0000 7ff3c67fc700 1 -- 172.21.15.140:0/672305102 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff3c8045780 con 0x7ff3d00b4d60 2022-07-21T01:50:28.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.331+0000 7ff3d74e2700 1 -- 172.21.15.140:0/672305102 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3b8053b00 msgr2=0x7ff3b8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.331+0000 7ff3d74e2700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3b8053b00 0x7ff3b8055fc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7ff3c0006000 tx=0x7ff3c0006d90).stop 2022-07-21T01:50:28.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.331+0000 7ff3d74e2700 1 -- 172.21.15.140:0/672305102 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3d00b4d60 msgr2=0x7ff3d00ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.331+0000 7ff3d74e2700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3d00b4d60 0x7ff3d00ae450 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7ff3c8000f40 tx=0x7ff3c8007d60).stop 2022-07-21T01:50:28.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.331+0000 7ff3d74e2700 1 -- 172.21.15.140:0/672305102 shutdown_connections 2022-07-21T01:50:28.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.331+0000 7ff3d74e2700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3b8053b00 0x7ff3b8055fc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.331+0000 7ff3d74e2700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3d00b4d60 0x7ff3d00ae450 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.331+0000 7ff3d74e2700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3d009df40 0x7ff3d0152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.331+0000 7ff3d74e2700 1 --2- 172.21.15.140:0/672305102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3d0060ee0 0x7ff3d01558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.331+0000 7ff3d74e2700 1 -- 172.21.15.140:0/672305102 >> 172.21.15.140:0/672305102 conn(0x7ff3d00b5f30 msgr2=0x7ff3d00b67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:28.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.335+0000 7ff3d74e2700 1 -- 172.21.15.140:0/672305102 shutdown_connections 2022-07-21T01:50:28.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.335+0000 7ff3d74e2700 1 -- 172.21.15.140:0/672305102 wait complete. 2022-07-21T01:50:28.355 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:50:28.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31afd66700 1 Processor -- start 2022-07-21T01:50:28.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31afd66700 1 -- start start 2022-07-21T01:50:28.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31afd66700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f31a805c3f0 0x7f31a814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31afd66700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a805d480 con 0x7f31a80b1e00 2022-07-21T01:50:28.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31afd66700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a80b21e0 con 0x7f31a80acef0 2022-07-21T01:50:28.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31afd66700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a805cf40 con 0x7f31a805c3f0 2022-07-21T01:50:28.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31ad301700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f31a805c3f0 0x7f31a814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31adb02700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f31a80b1e00 0x7f31a80ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48576/0 (socket says 172.21.15.140:48576) 2022-07-21T01:50:28.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31adb02700 1 -- 172.21.15.140:0/91592253 learned_addr learned my addr 172.21.15.140:0/91592253 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:28.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31ad301700 1 -- 172.21.15.140:0/91592253 >> v1:172.21.15.152:6789/0 conn(0x7f31a80acef0 legacy=0x7f31a8146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31ad301700 1 -- 172.21.15.140:0/91592253 >> v1:172.21.15.140:6789/0 conn(0x7f31a80b1e00 legacy=0x7f31a80ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31acb00700 1 -- 172.21.15.140:0/91592253 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322518751 0 0) 0x7f31a805d480 con 0x7f31a80b1e00 2022-07-21T01:50:28.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31ad301700 1 -- 172.21.15.140:0/91592253 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a805d1e0 con 0x7f31a805c3f0 2022-07-21T01:50:28.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31ad301700 1 --2- 172.21.15.140:0/91592253 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31a805c3f0 0x7f31a814a6d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f31a40032f0 tx=0x7f31a40091d0).ready entity=mon.0 client_cookie=622062c86fea92b5 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:28.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31acb00700 1 -- 172.21.15.140:0/91592253 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a4009ea0 con 0x7f31a805c3f0 2022-07-21T01:50:28.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31acb00700 1 -- 172.21.15.140:0/91592253 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a400b8a0 con 0x7f31a805c3f0 2022-07-21T01:50:28.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.451+0000 7f31acb00700 1 -- 172.21.15.140:0/91592253 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a400ba10 con 0x7f31a805c3f0 2022-07-21T01:50:28.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 -- 172.21.15.140:0/91592253 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31a805c3f0 msgr2=0x7f31a814a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 --2- 172.21.15.140:0/91592253 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31a805c3f0 0x7f31a814a6d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f31a40032f0 tx=0x7f31a40091d0).stop 2022-07-21T01:50:28.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 -- 172.21.15.140:0/91592253 shutdown_connections 2022-07-21T01:50:28.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 --2- 172.21.15.140:0/91592253 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31a805c3f0 0x7f31a814a6d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 -- 172.21.15.140:0/91592253 >> 172.21.15.140:0/91592253 conn(0x7f31a80611c0 msgr2=0x7f31a8062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:28.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 -- 172.21.15.140:0/91592253 shutdown_connections 2022-07-21T01:50:28.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 -- 172.21.15.140:0/91592253 wait complete. 2022-07-21T01:50:28.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 Processor -- start 2022-07-21T01:50:28.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 -- start start 2022-07-21T01:50:28.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31a80acef0 0x7f31a814bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31a80b1e00 0x7f31a81565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31a80a1f20 0x7f31a8156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80b4480 con 0x7f31a80b1e00 2022-07-21T01:50:28.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80a1300 con 0x7f31a80acef0 2022-07-21T01:50:28.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31afd66700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80a1560 con 0x7f31a80a1f20 2022-07-21T01:50:28.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31ad301700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31a80b1e00 0x7f31a81565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31ad301700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31a80b1e00 0x7f31a81565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34438/0 (socket says 172.21.15.140:34438) 2022-07-21T01:50:28.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31ad301700 1 -- 172.21.15.140:0/82004037 learned_addr learned my addr 172.21.15.140:0/82004037 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:28.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31adb02700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31a80acef0 0x7f31a814bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31ae303700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31a80a1f20 0x7f31a8156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31ad301700 1 -- 172.21.15.140:0/82004037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31a80a1f20 msgr2=0x7f31a8156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31ad301700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31a80a1f20 0x7f31a8156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31ad301700 1 -- 172.21.15.140:0/82004037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31a80acef0 msgr2=0x7f31a814bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31ad301700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31a80acef0 0x7f31a814bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31ad301700 1 -- 172.21.15.140:0/82004037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a4009470 con 0x7f31a80b1e00 2022-07-21T01:50:28.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.455+0000 7f31ae303700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31a80a1f20 0x7f31a8156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:50:28.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.459+0000 7f31ad301700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31a80b1e00 0x7f31a81565a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f31a40032f0 tx=0x7f31a4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:28.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.459+0000 7f3196ffd700 1 -- 172.21.15.140:0/82004037 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a4009ea0 con 0x7f31a80b1e00 2022-07-21T01:50:28.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.459+0000 7f3196ffd700 1 -- 172.21.15.140:0/82004037 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a401f3f0 con 0x7f31a80b1e00 2022-07-21T01:50:28.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.459+0000 7f31afd66700 1 -- 172.21.15.140:0/82004037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a80a1a80 con 0x7f31a80b1e00 2022-07-21T01:50:28.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.459+0000 7f3196ffd700 1 -- 172.21.15.140:0/82004037 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a401f750 con 0x7f31a80b1e00 2022-07-21T01:50:28.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.459+0000 7f31afd66700 1 -- 172.21.15.140:0/82004037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a80b4fb0 con 0x7f31a80b1e00 2022-07-21T01:50:28.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.459+0000 7f31afd66700 1 -- 172.21.15.140:0/82004037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a80a1c10 con 0x7f31a80b1e00 2022-07-21T01:50:28.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.459+0000 7f3196ffd700 1 -- 172.21.15.140:0/82004037 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f31a4026030 con 0x7f31a80b1e00 2022-07-21T01:50:28.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.459+0000 7f3196ffd700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3188053a70 0x7f3188055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.459+0000 7f31adb02700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3188053a70 0x7f3188055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.459+0000 7f3196ffd700 1 -- 172.21.15.140:0/82004037 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f31a406caa0 con 0x7f31a80b1e00 2022-07-21T01:50:28.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.463+0000 7f31adb02700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3188053a70 0x7f3188055f30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f319800b000 tx=0x7f3198009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:28.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.467+0000 7f3196ffd700 1 -- 172.21.15.140:0/82004037 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f31a40439c0 con 0x7f31a80b1e00 2022-07-21T01:50:28.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.619+0000 7f31afd66700 1 -- 172.21.15.140:0/82004037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31a8069a60 con 0x7f31a80b1e00 2022-07-21T01:50:28.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.619+0000 7f3196ffd700 1 -- 172.21.15.140:0/82004037 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2141 (secure 0 0 0) 0x7f31a4024090 con 0x7f31a80b1e00 2022-07-21T01:50:28.622 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:28.623 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/2307860244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2307860244},{"type":"v1","addr":"172.21.15.152:6833","nonce":2307860244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:50:27.322541+0000","modified":"2022-07-21T01:50:28.325625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/207391195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":207391195},{"type":"v1","addr":"172.21.15.140:6835","nonce":207391195}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-07-21T01:50:28.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 -- 172.21.15.140:0/82004037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3188053a70 msgr2=0x7f3188055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3188053a70 0x7f3188055f30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f319800b000 tx=0x7f3198009040).stop 2022-07-21T01:50:28.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 -- 172.21.15.140:0/82004037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31a80b1e00 msgr2=0x7f31a81565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31a80b1e00 0x7f31a81565a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f31a40032f0 tx=0x7f31a4016000).stop 2022-07-21T01:50:28.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 -- 172.21.15.140:0/82004037 shutdown_connections 2022-07-21T01:50:28.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3188053a70 0x7f3188055f30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31a80a1f20 0x7f31a8156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31a80b1e00 0x7f31a81565a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 --2- 172.21.15.140:0/82004037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31a80acef0 0x7f31a814bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 -- 172.21.15.140:0/82004037 >> 172.21.15.140:0/82004037 conn(0x7f31a805c3f0 msgr2=0x7f31a80ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:28.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 -- 172.21.15.140:0/82004037 shutdown_connections 2022-07-21T01:50:28.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.623+0000 7f31afd66700 1 -- 172.21.15.140:0/82004037 wait complete. 2022-07-21T01:50:28.629 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 16 2022-07-21T01:50:28.635 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:50:28.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 Processor -- start 2022-07-21T01:50:28.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 -- start start 2022-07-21T01:50:28.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff59813c790 0x7ff5980b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff5980b3ca0 0x7ff5980b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff598142ba0 con 0x7ff5980b9f00 2022-07-21T01:50:28.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5981429b0 con 0x7ff59813c790 2022-07-21T01:50:28.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff598146330 con 0x7ff5980b3ca0 2022-07-21T01:50:28.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59d357700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff59813c790 0x7ff5980b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59d357700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff59813c790 0x7ff5980b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34446/0 (socket says 172.21.15.140:34446) 2022-07-21T01:50:28.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59d357700 1 -- 172.21.15.140:0/2696976998 learned_addr learned my addr 172.21.15.140:0/2696976998 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:28.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59cb56700 1 --2- 172.21.15.140:0/2696976998 >> v2:172.21.15.152:3300/0 conn(0x7ff5980b3ca0 0x7ff5980b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59d357700 1 -- 172.21.15.140:0/2696976998 >> v2:172.21.15.152:3300/0 conn(0x7ff5980b3ca0 msgr2=0x7ff5980b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:28.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59d357700 1 --2- 172.21.15.140:0/2696976998 >> v2:172.21.15.152:3300/0 conn(0x7ff5980b3ca0 0x7ff5980b9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59d357700 1 -- 172.21.15.140:0/2696976998 >> v1:172.21.15.152:6789/0 conn(0x7ff5980b9f00 legacy=0x7ff598143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59d357700 1 -- 172.21.15.140:0/2696976998 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980b4c00 con 0x7ff59813c790 2022-07-21T01:50:28.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59d357700 1 --2- 172.21.15.140:0/2696976998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff59813c790 0x7ff5980b4710 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7ff598146fe0 tx=0x7ff58c006eb0).ready entity=mon.0 client_cookie=5d19ec552d6f7750 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:28.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff58bfff700 1 -- 172.21.15.140:0/2696976998 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff58c00d610 con 0x7ff59813c790 2022-07-21T01:50:28.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff58bfff700 1 -- 172.21.15.140:0/2696976998 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c000e90 con 0x7ff59813c790 2022-07-21T01:50:28.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff58bfff700 1 -- 172.21.15.140:0/2696976998 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff58c00d9a0 con 0x7ff59813c790 2022-07-21T01:50:28.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 -- 172.21.15.140:0/2696976998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff59813c790 msgr2=0x7ff5980b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 --2- 172.21.15.140:0/2696976998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff59813c790 0x7ff5980b4710 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7ff598146fe0 tx=0x7ff58c006eb0).stop 2022-07-21T01:50:28.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 -- 172.21.15.140:0/2696976998 shutdown_connections 2022-07-21T01:50:28.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 --2- 172.21.15.140:0/2696976998 >> v2:172.21.15.152:3300/0 conn(0x7ff5980b3ca0 0x7ff5980b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 --2- 172.21.15.140:0/2696976998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff59813c790 0x7ff5980b4710 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 -- 172.21.15.140:0/2696976998 >> 172.21.15.140:0/2696976998 conn(0x7ff5980678b0 msgr2=0x7ff5980b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:28.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 -- 172.21.15.140:0/2696976998 shutdown_connections 2022-07-21T01:50:28.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.747+0000 7ff59e359700 1 -- 172.21.15.140:0/2696976998 wait complete. 2022-07-21T01:50:28.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.751+0000 7ff59e359700 1 Processor -- start 2022-07-21T01:50:28.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.751+0000 7ff59e359700 1 -- start start 2022-07-21T01:50:28.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.751+0000 7ff59e359700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5980b3ca0 0x7ff5980aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.751+0000 7ff59e359700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5980b9f00 0x7ff598139ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.751+0000 7ff59e359700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff59813c790 0x7ff598139fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.751+0000 7ff59e359700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980af8c0 con 0x7ff59813c790 2022-07-21T01:50:28.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.751+0000 7ff59e359700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980b1900 con 0x7ff5980b3ca0 2022-07-21T01:50:28.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.751+0000 7ff59e359700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5980b1ba0 con 0x7ff5980b9f00 2022-07-21T01:50:28.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59db58700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff59813c790 0x7ff598139fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59db58700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff59813c790 0x7ff598139fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34454/0 (socket says 172.21.15.140:34454) 2022-07-21T01:50:28.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59db58700 1 -- 172.21.15.140:0/4279401378 learned_addr learned my addr 172.21.15.140:0/4279401378 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:28.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59cb56700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5980b9f00 0x7ff598139ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59d357700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5980b3ca0 0x7ff5980aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59db58700 1 -- 172.21.15.140:0/4279401378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5980b9f00 msgr2=0x7ff598139ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59db58700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5980b9f00 0x7ff598139ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59db58700 1 -- 172.21.15.140:0/4279401378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5980b3ca0 msgr2=0x7ff5980aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59db58700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5980b3ca0 0x7ff5980aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59db58700 1 -- 172.21.15.140:0/4279401378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c00b1e0 con 0x7ff59813c790 2022-07-21T01:50:28.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59db58700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff59813c790 0x7ff598139fb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7ff5980608b0 tx=0x7ff594007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:28.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff58affd700 1 -- 172.21.15.140:0/4279401378 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff594013450 con 0x7ff59813c790 2022-07-21T01:50:28.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff58affd700 1 -- 172.21.15.140:0/4279401378 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5940135b0 con 0x7ff59813c790 2022-07-21T01:50:28.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59e359700 1 -- 172.21.15.140:0/4279401378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59813a4a0 con 0x7ff59813c790 2022-07-21T01:50:28.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff58affd700 1 -- 172.21.15.140:0/4279401378 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff594013910 con 0x7ff59813c790 2022-07-21T01:50:28.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59e359700 1 -- 172.21.15.140:0/4279401378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5980af480 con 0x7ff59813c790 2022-07-21T01:50:28.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59e359700 1 -- 172.21.15.140:0/4279401378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5980b1760 con 0x7ff59813c790 2022-07-21T01:50:28.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff58affd700 1 -- 172.21.15.140:0/4279401378 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff594010070 con 0x7ff59813c790 2022-07-21T01:50:28.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff58affd700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff574053b00 0x7ff574055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:28.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff58affd700 1 -- 172.21.15.140:0/4279401378 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff59406b500 con 0x7ff59813c790 2022-07-21T01:50:28.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.755+0000 7ff59d357700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff574053b00 0x7ff574055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:28.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.759+0000 7ff59d357700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff574053b00 0x7ff574055fc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7ff58c01b000 tx=0x7ff58c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:28.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.763+0000 7ff58affd700 1 -- 172.21.15.140:0/4279401378 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff5980b1760 con 0x7ff59813c790 2022-07-21T01:50:28.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.891+0000 7ff59e359700 1 -- 172.21.15.140:0/4279401378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5980b1760 con 0x7ff59813c790 2022-07-21T01:50:28.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.891+0000 7ff58affd700 1 -- 172.21.15.140:0/4279401378 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16147 (secure 0 0 0) 0x7ff5980b1760 con 0x7ff59813c790 2022-07-21T01:50:28.895 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:28.895 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":41,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:50:28.326191+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:50:25.939192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-07-21T01:50:26.391519+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:50:28.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 -- 172.21.15.140:0/4279401378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff574053b00 msgr2=0x7ff574055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff574053b00 0x7ff574055fc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7ff58c01b000 tx=0x7ff58c015000).stop 2022-07-21T01:50:28.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 -- 172.21.15.140:0/4279401378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff59813c790 msgr2=0x7ff598139fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:28.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff59813c790 0x7ff598139fb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7ff5980608b0 tx=0x7ff594007bc0).stop 2022-07-21T01:50:28.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 -- 172.21.15.140:0/4279401378 shutdown_connections 2022-07-21T01:50:28.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff574053b00 0x7ff574055fc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff59813c790 0x7ff598139fb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5980b9f00 0x7ff598139ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 --2- 172.21.15.140:0/4279401378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5980b3ca0 0x7ff5980aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:28.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 -- 172.21.15.140:0/4279401378 >> 172.21.15.140:0/4279401378 conn(0x7ff5980678b0 msgr2=0x7ff59813a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:28.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 -- 172.21.15.140:0/4279401378 shutdown_connections 2022-07-21T01:50:28.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:28.895+0000 7ff59e359700 1 -- 172.21.15.140:0/4279401378 wait complete. 2022-07-21T01:50:28.907 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:50:29.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.015+0000 7fdeb759a700 1 Processor -- start 2022-07-21T01:50:29.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.015+0000 7fdeb759a700 1 -- start start 2022-07-21T01:50:29.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdeb00b3700 0x7fdeb0098a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdeb0062230 0x7fdeb0095b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdeb00abd10 0x7fdeb0096070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb00b3ae0 con 0x7fdeb00abd10 2022-07-21T01:50:29.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb0098070 con 0x7fdeb0062230 2022-07-21T01:50:29.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb0098310 con 0x7fdeb00b3700 2022-07-21T01:50:29.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb6d99700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdeb00abd10 0x7fdeb0096070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb6d99700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdeb00abd10 0x7fdeb0096070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34462/0 (socket says 172.21.15.140:34462) 2022-07-21T01:50:29.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb6d99700 1 -- 172.21.15.140:0/1690192241 learned_addr learned my addr 172.21.15.140:0/1690192241 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:29.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb5d97700 1 --2- 172.21.15.140:0/1690192241 >> v2:172.21.15.152:3300/0 conn(0x7fdeb0062230 0x7fdeb0095b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb6598700 1 --2- 172.21.15.140:0/1690192241 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdeb00b3700 0x7fdeb0098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb6d99700 1 -- 172.21.15.140:0/1690192241 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdeb00b3700 msgr2=0x7fdeb0098a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:29.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb6d99700 1 --2- 172.21.15.140:0/1690192241 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdeb00b3700 0x7fdeb0098a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb6d99700 1 -- 172.21.15.140:0/1690192241 >> v2:172.21.15.152:3300/0 conn(0x7fdeb0062230 msgr2=0x7fdeb0095b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:29.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb6d99700 1 --2- 172.21.15.140:0/1690192241 >> v2:172.21.15.152:3300/0 conn(0x7fdeb0062230 0x7fdeb0095b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb6d99700 1 -- 172.21.15.140:0/1690192241 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb00985b0 con 0x7fdeb00abd10 2022-07-21T01:50:29.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb6d99700 1 --2- 172.21.15.140:0/1690192241 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdeb00abd10 0x7fdeb0096070 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fdea80032f0 tx=0x7fdea8009290).ready entity=mon.0 client_cookie=6ceda89619f93556 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb5596700 1 -- 172.21.15.140:0/1690192241 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdea800b680 con 0x7fdeb00abd10 2022-07-21T01:50:29.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb5596700 1 -- 172.21.15.140:0/1690192241 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea800b7e0 con 0x7fdeb00abd10 2022-07-21T01:50:29.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb5596700 1 -- 172.21.15.140:0/1690192241 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdea800b950 con 0x7fdeb00abd10 2022-07-21T01:50:29.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 -- 172.21.15.140:0/1690192241 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdeb00abd10 msgr2=0x7fdeb0096070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 --2- 172.21.15.140:0/1690192241 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdeb00abd10 0x7fdeb0096070 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fdea80032f0 tx=0x7fdea8009290).stop 2022-07-21T01:50:29.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 -- 172.21.15.140:0/1690192241 shutdown_connections 2022-07-21T01:50:29.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 --2- 172.21.15.140:0/1690192241 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdeb00abd10 0x7fdeb0096070 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 --2- 172.21.15.140:0/1690192241 >> v2:172.21.15.152:3300/0 conn(0x7fdeb0062230 0x7fdeb0095b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 --2- 172.21.15.140:0/1690192241 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdeb00b3700 0x7fdeb0098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 -- 172.21.15.140:0/1690192241 >> 172.21.15.140:0/1690192241 conn(0x7fdeb00b6ce0 msgr2=0x7fdeb00b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:29.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 -- 172.21.15.140:0/1690192241 shutdown_connections 2022-07-21T01:50:29.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.019+0000 7fdeb759a700 1 -- 172.21.15.140:0/1690192241 wait complete. 2022-07-21T01:50:29.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.023+0000 7fdeb759a700 1 Processor -- start 2022-07-21T01:50:29.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.023+0000 7fdeb759a700 1 -- start start 2022-07-21T01:50:29.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.023+0000 7fdeb759a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdeb0062230 0x7fdeb00af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.023+0000 7fdeb759a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdeb00abd10 0x7fdeb01443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.023+0000 7fdeb759a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdeb00b3700 0x7fdeb014f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.023+0000 7fdeb759a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb014edc0 con 0x7fdeb00abd10 2022-07-21T01:50:29.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.023+0000 7fdeb759a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb013e3c0 con 0x7fdeb0062230 2022-07-21T01:50:29.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.023+0000 7fdeb759a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb00ad7d0 con 0x7fdeb00b3700 2022-07-21T01:50:29.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb5d97700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdeb00abd10 0x7fdeb01443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb5d97700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdeb00abd10 0x7fdeb01443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34466/0 (socket says 172.21.15.140:34466) 2022-07-21T01:50:29.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb6d99700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdeb00b3700 0x7fdeb014f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb5d97700 1 -- 172.21.15.140:0/1070027572 learned_addr learned my addr 172.21.15.140:0/1070027572 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:29.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb6598700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdeb0062230 0x7fdeb00af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb5d97700 1 -- 172.21.15.140:0/1070027572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdeb00b3700 msgr2=0x7fdeb014f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb5d97700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdeb00b3700 0x7fdeb014f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb5d97700 1 -- 172.21.15.140:0/1070027572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdeb0062230 msgr2=0x7fdeb00af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb5d97700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdeb0062230 0x7fdeb00af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb5d97700 1 -- 172.21.15.140:0/1070027572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea8009530 con 0x7fdeb00abd10 2022-07-21T01:50:29.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb6d99700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdeb00b3700 0x7fdeb014f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:29.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb6598700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdeb0062230 0x7fdeb00af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:29.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb5d97700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdeb00abd10 0x7fdeb01443a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fdea000c000 tx=0x7fdea0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fde9f7fe700 1 -- 172.21.15.140:0/1070027572 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdea0015660 con 0x7fdeb00abd10 2022-07-21T01:50:29.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fde9f7fe700 1 -- 172.21.15.140:0/1070027572 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea00157c0 con 0x7fdeb00abd10 2022-07-21T01:50:29.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fde9f7fe700 1 -- 172.21.15.140:0/1070027572 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdea0015b20 con 0x7fdeb00abd10 2022-07-21T01:50:29.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb759a700 1 -- 172.21.15.140:0/1070027572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb00ada70 con 0x7fdeb00abd10 2022-07-21T01:50:29.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb759a700 1 -- 172.21.15.140:0/1070027572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb00b0580 con 0x7fdeb00abd10 2022-07-21T01:50:29.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb759a700 1 -- 172.21.15.140:0/1070027572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb00574e0 con 0x7fdeb00abd10 2022-07-21T01:50:29.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fde9f7fe700 1 -- 172.21.15.140:0/1070027572 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdea001c030 con 0x7fdeb00abd10 2022-07-21T01:50:29.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fde9f7fe700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde90053a70 0x7fde90055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fde9f7fe700 1 -- 172.21.15.140:0/1070027572 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdea006ded0 con 0x7fdeb00abd10 2022-07-21T01:50:29.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.027+0000 7fdeb6598700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde90053a70 0x7fde90055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.031+0000 7fdeb6598700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde90053a70 0x7fde90055f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fdeac00b000 tx=0x7fdeac006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.035+0000 7fde9f7fe700 1 -- 172.21.15.140:0/1070027572 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdea0044e00 con 0x7fdeb00abd10 2022-07-21T01:50:29.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.203+0000 7fdeb759a700 1 -- 172.21.15.140:0/1070027572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fdeb00ab450 con 0x7fdeb00abd10 2022-07-21T01:50:29.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.211+0000 7fde9f7fe700 1 -- 172.21.15.140:0/1070027572 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7fdea0048420 con 0x7fdeb00abd10 2022-07-21T01:50:29.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 -- 172.21.15.140:0/1070027572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde90053a70 msgr2=0x7fde90055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde90053a70 0x7fde90055f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fdeac00b000 tx=0x7fdeac006df0).stop 2022-07-21T01:50:29.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 -- 172.21.15.140:0/1070027572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdeb00abd10 msgr2=0x7fdeb01443a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdeb00abd10 0x7fdeb01443a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fdea000c000 tx=0x7fdea0006df0).stop 2022-07-21T01:50:29.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 -- 172.21.15.140:0/1070027572 shutdown_connections 2022-07-21T01:50:29.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde90053a70 0x7fde90055f30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdeb00b3700 0x7fdeb014f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdeb00abd10 0x7fdeb01443a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 --2- 172.21.15.140:0/1070027572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdeb0062230 0x7fdeb00af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 -- 172.21.15.140:0/1070027572 >> 172.21.15.140:0/1070027572 conn(0x7fdeb00b6ce0 msgr2=0x7fdeb00b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:29.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 -- 172.21.15.140:0/1070027572 shutdown_connections 2022-07-21T01:50:29.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.215+0000 7fdeb759a700 1 -- 172.21.15.140:0/1070027572 wait complete. 2022-07-21T01:50:29.223 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:50:29.230 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:50:29.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba7c33700 1 Processor -- start 2022-07-21T01:50:29.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba7c33700 1 -- start start 2022-07-21T01:50:29.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba7c33700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3ba005cf40 0x7f3ba0144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba7c33700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba00649d0 con 0x7f3ba0066cb0 2022-07-21T01:50:29.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba7c33700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba01442c0 con 0x7f3ba00b1c30 2022-07-21T01:50:29.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba7c33700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba0144560 con 0x7f3ba005cf40 2022-07-21T01:50:29.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba61d0700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f3ba0066cb0 0x7f3ba0141100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48622/0 (socket says 172.21.15.140:48622) 2022-07-21T01:50:29.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba61d0700 1 -- 172.21.15.140:0/4097560735 learned_addr learned my addr 172.21.15.140:0/4097560735 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:29.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba59cf700 1 --2- 172.21.15.140:0/4097560735 >> v2:172.21.15.140:3300/0 conn(0x7f3ba005cf40 0x7f3ba0144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba59cf700 1 -- 172.21.15.140:0/4097560735 >> v1:172.21.15.152:6789/0 conn(0x7f3ba00b1c30 legacy=0x7f3ba0061da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba59cf700 1 -- 172.21.15.140:0/4097560735 >> v1:172.21.15.140:6789/0 conn(0x7f3ba0066cb0 legacy=0x7f3ba0141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba59cf700 1 -- 172.21.15.140:0/4097560735 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba0144800 con 0x7f3ba005cf40 2022-07-21T01:50:29.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.339+0000 7f3ba59cf700 1 --2- 172.21.15.140:0/4097560735 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3ba005cf40 0x7f3ba0144ca0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f3b9c0032f0 tx=0x7f3b9c0091d0).ready entity=mon.0 client_cookie=f0d434023435a6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.343+0000 7f3ba49cd700 1 -- 172.21.15.140:0/4097560735 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b9c009ea0 con 0x7f3ba005cf40 2022-07-21T01:50:29.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.343+0000 7f3ba49cd700 1 -- 172.21.15.140:0/4097560735 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c00b8a0 con 0x7f3ba005cf40 2022-07-21T01:50:29.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.343+0000 7f3ba49cd700 1 -- 172.21.15.140:0/4097560735 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b9c00ba10 con 0x7f3ba005cf40 2022-07-21T01:50:29.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.343+0000 7f3ba7c33700 1 -- 172.21.15.140:0/4097560735 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3ba005cf40 msgr2=0x7f3ba0144ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.343+0000 7f3ba7c33700 1 --2- 172.21.15.140:0/4097560735 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3ba005cf40 0x7f3ba0144ca0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f3b9c0032f0 tx=0x7f3b9c0091d0).stop 2022-07-21T01:50:29.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.343+0000 7f3ba7c33700 1 -- 172.21.15.140:0/4097560735 shutdown_connections 2022-07-21T01:50:29.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.343+0000 7f3ba7c33700 1 --2- 172.21.15.140:0/4097560735 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3ba005cf40 0x7f3ba0144ca0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.343+0000 7f3ba7c33700 1 -- 172.21.15.140:0/4097560735 >> 172.21.15.140:0/4097560735 conn(0x7f3ba00b7d60 msgr2=0x7f3ba00b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:29.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.343+0000 7f3ba7c33700 1 -- 172.21.15.140:0/4097560735 shutdown_connections 2022-07-21T01:50:29.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.343+0000 7f3ba7c33700 1 -- 172.21.15.140:0/4097560735 wait complete. 2022-07-21T01:50:29.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba7c33700 1 Processor -- start 2022-07-21T01:50:29.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba7c33700 1 -- start start 2022-07-21T01:50:29.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba7c33700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3ba005cf40 0x7f3ba0098580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba7c33700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ba0066cb0 0x7f3ba014fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba7c33700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3ba00b1c30 0x7f3ba01502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba7c33700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba00af120 con 0x7f3ba005cf40 2022-07-21T01:50:29.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba7c33700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba0097e40 con 0x7f3ba0066cb0 2022-07-21T01:50:29.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba7c33700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba00980e0 con 0x7f3ba00b1c30 2022-07-21T01:50:29.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba59cf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3ba005cf40 0x7f3ba0098580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba59cf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3ba005cf40 0x7f3ba0098580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34478/0 (socket says 172.21.15.140:34478) 2022-07-21T01:50:29.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba59cf700 1 -- 172.21.15.140:0/202196131 learned_addr learned my addr 172.21.15.140:0/202196131 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:29.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba61d0700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3ba00b1c30 0x7f3ba01502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba51ce700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ba0066cb0 0x7f3ba014fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba59cf700 1 -- 172.21.15.140:0/202196131 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3ba00b1c30 msgr2=0x7f3ba01502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba59cf700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3ba00b1c30 0x7f3ba01502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba59cf700 1 -- 172.21.15.140:0/202196131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ba0066cb0 msgr2=0x7f3ba014fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba59cf700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ba0066cb0 0x7f3ba014fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba59cf700 1 -- 172.21.15.140:0/202196131 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c009470 con 0x7f3ba005cf40 2022-07-21T01:50:29.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba61d0700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3ba00b1c30 0x7f3ba01502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:29.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba59cf700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3ba005cf40 0x7f3ba0098580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f3b9c0032f0 tx=0x7f3b9c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3b8effd700 1 -- 172.21.15.140:0/202196131 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b9c009ea0 con 0x7f3ba005cf40 2022-07-21T01:50:29.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3b8effd700 1 -- 172.21.15.140:0/202196131 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c01f5e0 con 0x7f3ba005cf40 2022-07-21T01:50:29.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba7c33700 1 -- 172.21.15.140:0/202196131 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba00aee10 con 0x7f3ba005cf40 2022-07-21T01:50:29.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3b8effd700 1 -- 172.21.15.140:0/202196131 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b9c01f940 con 0x7f3ba005cf40 2022-07-21T01:50:29.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.347+0000 7f3ba7c33700 1 -- 172.21.15.140:0/202196131 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba0098a70 con 0x7f3ba005cf40 2022-07-21T01:50:29.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.351+0000 7f3ba7c33700 1 -- 172.21.15.140:0/202196131 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba00574e0 con 0x7f3ba005cf40 2022-07-21T01:50:29.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.351+0000 7f3b8effd700 1 -- 172.21.15.140:0/202196131 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3b9c026030 con 0x7f3ba005cf40 2022-07-21T01:50:29.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.351+0000 7f3b8effd700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b80053a70 0x7f3b80055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.351+0000 7f3b8effd700 1 -- 172.21.15.140:0/202196131 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3b9c06cc70 con 0x7f3ba005cf40 2022-07-21T01:50:29.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.351+0000 7f3ba51ce700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b80053a70 0x7f3b80055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.351+0000 7f3ba51ce700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b80053a70 0x7f3b80055f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f3b9000b000 tx=0x7f3b90009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.355+0000 7f3b8effd700 1 -- 172.21.15.140:0/202196131 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3b9c043b90 con 0x7f3ba005cf40 2022-07-21T01:50:29.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.523+0000 7f3ba7c33700 1 -- 172.21.15.140:0/202196131 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f3ba0069a60 con 0x7f3ba005cf40 2022-07-21T01:50:29.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.527+0000 7f3b8effd700 1 -- 172.21.15.140:0/202196131 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7f3b9c0240e0 con 0x7f3ba005cf40 2022-07-21T01:50:29.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 -- 172.21.15.140:0/202196131 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b80053a70 msgr2=0x7f3b80055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b80053a70 0x7f3b80055f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f3b9000b000 tx=0x7f3b90009040).stop 2022-07-21T01:50:29.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 -- 172.21.15.140:0/202196131 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3ba005cf40 msgr2=0x7f3ba0098580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3ba005cf40 0x7f3ba0098580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f3b9c0032f0 tx=0x7f3b9c016000).stop 2022-07-21T01:50:29.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 -- 172.21.15.140:0/202196131 shutdown_connections 2022-07-21T01:50:29.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b80053a70 0x7f3b80055f30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3ba00b1c30 0x7f3ba01502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3ba0066cb0 0x7f3ba014fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 --2- 172.21.15.140:0/202196131 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3ba005cf40 0x7f3ba0098580 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 -- 172.21.15.140:0/202196131 >> 172.21.15.140:0/202196131 conn(0x7f3ba00b7d60 msgr2=0x7f3ba00b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:29.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 -- 172.21.15.140:0/202196131 shutdown_connections 2022-07-21T01:50:29.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.531+0000 7f3ba7c33700 1 -- 172.21.15.140:0/202196131 wait complete. 2022-07-21T01:50:29.536 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:50:29.543 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:50:29.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f0814fc2700 1 Processor -- start 2022-07-21T01:50:29.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f0814fc2700 1 -- start start 2022-07-21T01:50:29.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f0814fc2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f081005c7c0 0x7f08100b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f0814fc2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0810147010 0x7f08100b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f0814fc2700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0810146390 con 0x7f08100ab250 2022-07-21T01:50:29.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f0814fc2700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0810146630 con 0x7f081005c7c0 2022-07-21T01:50:29.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f0814fc2700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08101468d0 con 0x7f0810147010 2022-07-21T01:50:29.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f080f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f081005c7c0 0x7f08100b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f080f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f081005c7c0 0x7f08100b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34486/0 (socket says 172.21.15.140:34486) 2022-07-21T01:50:29.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f080f7fe700 1 -- 172.21.15.140:0/1324042254 learned_addr learned my addr 172.21.15.140:0/1324042254 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:29.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f0807fff700 1 --2- 172.21.15.140:0/1324042254 >> v2:172.21.15.152:3300/0 conn(0x7f0810147010 0x7f08100b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f080f7fe700 1 -- 172.21.15.140:0/1324042254 >> v2:172.21.15.152:3300/0 conn(0x7f0810147010 msgr2=0x7f08100b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:29.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f080f7fe700 1 --2- 172.21.15.140:0/1324042254 >> v2:172.21.15.152:3300/0 conn(0x7f0810147010 0x7f08100b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f080f7fe700 1 -- 172.21.15.140:0/1324042254 >> v1:172.21.15.152:6789/0 conn(0x7f08100ab250 legacy=0x7f08101420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f080f7fe700 1 -- 172.21.15.140:0/1324042254 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0810146b70 con 0x7f081005c7c0 2022-07-21T01:50:29.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f0807fff700 1 --2- 172.21.15.140:0/1324042254 >> v2:172.21.15.152:3300/0 conn(0x7f0810147010 0x7f08100b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:29.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f080f7fe700 1 --2- 172.21.15.140:0/1324042254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f081005c7c0 0x7f08100b0bb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f0800002f70 tx=0x7f0800006eb0).ready entity=mon.0 client_cookie=58f87f48e0f887f6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.655+0000 7f080effd700 1 -- 172.21.15.140:0/1324042254 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f080000d610 con 0x7f081005c7c0 2022-07-21T01:50:29.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f080effd700 1 -- 172.21.15.140:0/1324042254 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0800000e90 con 0x7f081005c7c0 2022-07-21T01:50:29.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f080effd700 1 -- 172.21.15.140:0/1324042254 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f080000d9a0 con 0x7f081005c7c0 2022-07-21T01:50:29.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 -- 172.21.15.140:0/1324042254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f081005c7c0 msgr2=0x7f08100b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 --2- 172.21.15.140:0/1324042254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f081005c7c0 0x7f08100b0bb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f0800002f70 tx=0x7f0800006eb0).stop 2022-07-21T01:50:29.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 -- 172.21.15.140:0/1324042254 shutdown_connections 2022-07-21T01:50:29.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 --2- 172.21.15.140:0/1324042254 >> v2:172.21.15.152:3300/0 conn(0x7f0810147010 0x7f08100b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 --2- 172.21.15.140:0/1324042254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f081005c7c0 0x7f08100b0bb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 -- 172.21.15.140:0/1324042254 >> 172.21.15.140:0/1324042254 conn(0x7f08100b6210 msgr2=0x7f08100b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:29.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 -- 172.21.15.140:0/1324042254 shutdown_connections 2022-07-21T01:50:29.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 -- 172.21.15.140:0/1324042254 wait complete. 2022-07-21T01:50:29.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 Processor -- start 2022-07-21T01:50:29.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 -- start start 2022-07-21T01:50:29.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f081005c7c0 0x7f0810098720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08100ab250 0x7f081009b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0810147010 0x7f0810095cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0810097450 con 0x7f08100ab250 2022-07-21T01:50:29.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08101564b0 con 0x7f0810147010 2022-07-21T01:50:29.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0814fc2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0810096fb0 con 0x7f081005c7c0 2022-07-21T01:50:29.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0807fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08100ab250 0x7f081009b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f080f7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f081005c7c0 0x7f0810098720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0807fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08100ab250 0x7f081009b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34494/0 (socket says 172.21.15.140:34494) 2022-07-21T01:50:29.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f0807fff700 1 -- 172.21.15.140:0/3579193944 learned_addr learned my addr 172.21.15.140:0/3579193944 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:29.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.659+0000 7f080ffff700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0810147010 0x7f0810095cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f0807fff700 1 -- 172.21.15.140:0/3579193944 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f081005c7c0 msgr2=0x7f0810098720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f0807fff700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f081005c7c0 0x7f0810098720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f0807fff700 1 -- 172.21.15.140:0/3579193944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0810147010 msgr2=0x7f0810095cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f0807fff700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0810147010 0x7f0810095cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f0807fff700 1 -- 172.21.15.140:0/3579193944 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080000b1e0 con 0x7f08100ab250 2022-07-21T01:50:29.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f0807fff700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08100ab250 0x7f081009b4c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f07f8006940 tx=0x7f07f8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f080d7fa700 1 -- 172.21.15.140:0/3579193944 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07f8011780 con 0x7f08100ab250 2022-07-21T01:50:29.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f080d7fa700 1 -- 172.21.15.140:0/3579193944 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f80118e0 con 0x7f08100ab250 2022-07-21T01:50:29.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f0814fc2700 1 -- 172.21.15.140:0/3579193944 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081009b9b0 con 0x7f08100ab250 2022-07-21T01:50:29.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f080d7fa700 1 -- 172.21.15.140:0/3579193944 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07f8011c40 con 0x7f08100ab250 2022-07-21T01:50:29.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f0814fc2700 1 -- 172.21.15.140:0/3579193944 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08100961e0 con 0x7f08100ab250 2022-07-21T01:50:29.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f0814fc2700 1 -- 172.21.15.140:0/3579193944 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08101566b0 con 0x7f08100ab250 2022-07-21T01:50:29.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f080d7fa700 1 -- 172.21.15.140:0/3579193944 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f07f8012420 con 0x7f08100ab250 2022-07-21T01:50:29.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f080d7fa700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07f4053b00 0x7f07f4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f080d7fa700 1 -- 172.21.15.140:0/3579193944 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f07f806bc40 con 0x7f08100ab250 2022-07-21T01:50:29.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f080f7fe700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07f4053b00 0x7f07f4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.663+0000 7f080f7fe700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07f4053b00 0x7f07f4055fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f080001c000 tx=0x7f0800015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.671+0000 7f080d7fa700 1 -- 172.21.15.140:0/3579193944 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07f8042b00 con 0x7f08100ab250 2022-07-21T01:50:29.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.835+0000 7f0814fc2700 1 -- 172.21.15.140:0/3579193944 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f08100574e0 con 0x7f08100ab250 2022-07-21T01:50:29.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f080d7fa700 1 -- 172.21.15.140:0/3579193944 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7f07f8046120 con 0x7f08100ab250 2022-07-21T01:50:29.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 -- 172.21.15.140:0/3579193944 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07f4053b00 msgr2=0x7f07f4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07f4053b00 0x7f07f4055fc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f080001c000 tx=0x7f0800015000).stop 2022-07-21T01:50:29.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 -- 172.21.15.140:0/3579193944 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08100ab250 msgr2=0x7f081009b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08100ab250 0x7f081009b4c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f07f8006940 tx=0x7f07f8008040).stop 2022-07-21T01:50:29.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 -- 172.21.15.140:0/3579193944 shutdown_connections 2022-07-21T01:50:29.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07f4053b00 0x7f07f4055fc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0810147010 0x7f0810095cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08100ab250 0x7f081009b4c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 --2- 172.21.15.140:0/3579193944 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f081005c7c0 0x7f0810098720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 -- 172.21.15.140:0/3579193944 >> 172.21.15.140:0/3579193944 conn(0x7f08100b6210 msgr2=0x7f0810144300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:29.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 -- 172.21.15.140:0/3579193944 shutdown_connections 2022-07-21T01:50:29.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.843+0000 7f0814fc2700 1 -- 172.21.15.140:0/3579193944 wait complete. 2022-07-21T01:50:29.853 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:50:29.861 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:50:29.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 Processor -- start 2022-07-21T01:50:29.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 -- start start 2022-07-21T01:50:29.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbf80b5650 0x7fdbf80b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdbf80af8e0 0x7fdbf805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf8147010 con 0x7fdbf80adff0 2022-07-21T01:50:29.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf81472b0 con 0x7fdbf80af8e0 2022-07-21T01:50:29.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf8146d40 con 0x7fdbf80b5650 2022-07-21T01:50:29.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf77fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdbf80af8e0 0x7fdbf805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf77fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdbf80af8e0 0x7fdbf805bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34504/0 (socket says 172.21.15.140:34504) 2022-07-21T01:50:29.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf77fe700 1 -- 172.21.15.140:0/4040367108 learned_addr learned my addr 172.21.15.140:0/4040367108 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:29.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf6ffd700 1 --2- 172.21.15.140:0/4040367108 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbf80b5650 0x7fdbf80b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf77fe700 1 -- 172.21.15.140:0/4040367108 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbf80b5650 msgr2=0x7fdbf80b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:29.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf77fe700 1 --2- 172.21.15.140:0/4040367108 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbf80b5650 0x7fdbf80b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf77fe700 1 -- 172.21.15.140:0/4040367108 >> v1:172.21.15.152:6789/0 conn(0x7fdbf80adff0 legacy=0x7fdbf8147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf77fe700 1 -- 172.21.15.140:0/4040367108 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf80acae0 con 0x7fdbf80af8e0 2022-07-21T01:50:29.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf77fe700 1 --2- 172.21.15.140:0/4040367108 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbf80af8e0 0x7fdbf805bd90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fdbe8002f70 tx=0x7fdbe8006eb0).ready entity=mon.0 client_cookie=5b97122347f4cd4f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf5ffb700 1 -- 172.21.15.140:0/4040367108 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbe800d570 con 0x7fdbf80af8e0 2022-07-21T01:50:29.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf5ffb700 1 -- 172.21.15.140:0/4040367108 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe8000e90 con 0x7fdbf80af8e0 2022-07-21T01:50:29.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf5ffb700 1 -- 172.21.15.140:0/4040367108 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbe800d900 con 0x7fdbf80af8e0 2022-07-21T01:50:29.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 -- 172.21.15.140:0/4040367108 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbf80af8e0 msgr2=0x7fdbf805bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 --2- 172.21.15.140:0/4040367108 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbf80af8e0 0x7fdbf805bd90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fdbe8002f70 tx=0x7fdbe8006eb0).stop 2022-07-21T01:50:29.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 -- 172.21.15.140:0/4040367108 shutdown_connections 2022-07-21T01:50:29.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 --2- 172.21.15.140:0/4040367108 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbf80af8e0 0x7fdbf805bd90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 --2- 172.21.15.140:0/4040367108 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbf80b5650 0x7fdbf80b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 -- 172.21.15.140:0/4040367108 >> 172.21.15.140:0/4040367108 conn(0x7fdbf80634e0 msgr2=0x7fdbf80b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:29.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 -- 172.21.15.140:0/4040367108 shutdown_connections 2022-07-21T01:50:29.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.963+0000 7fdbf7fff700 1 -- 172.21.15.140:0/4040367108 wait complete. 2022-07-21T01:50:29.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf7fff700 1 Processor -- start 2022-07-21T01:50:29.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf7fff700 1 -- start start 2022-07-21T01:50:29.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbf80adff0 0x7fdbf811e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbf80af8e0 0x7fdbf8156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf80b5650 0x7fdbf8064920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf7fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf811e480 con 0x7fdbf80adff0 2022-07-21T01:50:29.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf7fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf81566d0 con 0x7fdbf80b5650 2022-07-21T01:50:29.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf8156970 con 0x7fdbf80af8e0 2022-07-21T01:50:29.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbf80adff0 0x7fdbf811e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbf80adff0 0x7fdbf811e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34506/0 (socket says 172.21.15.140:34506) 2022-07-21T01:50:29.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf6ffd700 1 -- 172.21.15.140:0/2870905308 learned_addr learned my addr 172.21.15.140:0/2870905308 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:29.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf77fe700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf80b5650 0x7fdbf8064920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf67fc700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbf80af8e0 0x7fdbf8156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf6ffd700 1 -- 172.21.15.140:0/2870905308 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbf80af8e0 msgr2=0x7fdbf8156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf6ffd700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbf80af8e0 0x7fdbf8156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf6ffd700 1 -- 172.21.15.140:0/2870905308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf80b5650 msgr2=0x7fdbf8064920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:29.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf6ffd700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf80b5650 0x7fdbf8064920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:29.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf6ffd700 1 -- 172.21.15.140:0/2870905308 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe800b1e0 con 0x7fdbf80adff0 2022-07-21T01:50:29.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf6ffd700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbf80adff0 0x7fdbf811e9e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fdbe000c000 tx=0x7fdbe0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbeffff700 1 -- 172.21.15.140:0/2870905308 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbe0011750 con 0x7fdbf80adff0 2022-07-21T01:50:29.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbeffff700 1 -- 172.21.15.140:0/2870905308 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe00118b0 con 0x7fdbf80adff0 2022-07-21T01:50:29.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf7fff700 1 -- 172.21.15.140:0/2870905308 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf811eed0 con 0x7fdbf80adff0 2022-07-21T01:50:29.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbeffff700 1 -- 172.21.15.140:0/2870905308 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbe0011c10 con 0x7fdbf80adff0 2022-07-21T01:50:29.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.967+0000 7fdbf7fff700 1 -- 172.21.15.140:0/2870905308 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf8157300 con 0x7fdbf80adff0 2022-07-21T01:50:29.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.971+0000 7fdbf7fff700 1 -- 172.21.15.140:0/2870905308 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf811e2d0 con 0x7fdbf80adff0 2022-07-21T01:50:29.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.971+0000 7fdbeffff700 1 -- 172.21.15.140:0/2870905308 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdbe0011d70 con 0x7fdbf80adff0 2022-07-21T01:50:29.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.971+0000 7fdbeffff700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbdc053a70 0x7fdbdc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:29.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.971+0000 7fdbeffff700 1 -- 172.21.15.140:0/2870905308 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdbe006c2d0 con 0x7fdbf80adff0 2022-07-21T01:50:29.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.971+0000 7fdbf67fc700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbdc053a70 0x7fdbdc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:29.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.971+0000 7fdbf67fc700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbdc053a70 0x7fdbdc055f30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fdbf000d000 tx=0x7fdbf0007f90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:29.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:29.975+0000 7fdbeffff700 1 -- 172.21.15.140:0/2870905308 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdbe0043190 con 0x7fdbf80adff0 2022-07-21T01:50:30.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.135+0000 7fdbf7fff700 1 -- 172.21.15.140:0/2870905308 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbf80574e0 con 0x7fdbf80adff0 2022-07-21T01:50:30.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.139+0000 7fdbeffff700 1 -- 172.21.15.140:0/2870905308 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2141 (secure 0 0 0) 0x7fdbe00467b0 con 0x7fdbf80adff0 2022-07-21T01:50:30.139 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:30.140 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/2307860244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2307860244},{"type":"v1","addr":"172.21.15.152:6833","nonce":2307860244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:50:27.322541+0000","modified":"2022-07-21T01:50:28.325625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/207391195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":207391195},{"type":"v1","addr":"172.21.15.140:6835","nonce":207391195}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-07-21T01:50:30.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.139+0000 7fdbf7fff700 1 -- 172.21.15.140:0/2870905308 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbdc053a70 msgr2=0x7fdbdc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.139+0000 7fdbf7fff700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbdc053a70 0x7fdbdc055f30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fdbf000d000 tx=0x7fdbf0007f90).stop 2022-07-21T01:50:30.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.139+0000 7fdbf7fff700 1 -- 172.21.15.140:0/2870905308 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbf80adff0 msgr2=0x7fdbf811e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.139+0000 7fdbf7fff700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbf80adff0 0x7fdbf811e9e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fdbe000c000 tx=0x7fdbe0006df0).stop 2022-07-21T01:50:30.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.143+0000 7fdbf7fff700 1 -- 172.21.15.140:0/2870905308 shutdown_connections 2022-07-21T01:50:30.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.143+0000 7fdbf7fff700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdbdc053a70 0x7fdbdc055f30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.143+0000 7fdbf7fff700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf80b5650 0x7fdbf8064920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.143+0000 7fdbf7fff700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbf80af8e0 0x7fdbf8156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.143+0000 7fdbf7fff700 1 --2- 172.21.15.140:0/2870905308 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbf80adff0 0x7fdbf811e9e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.143+0000 7fdbf7fff700 1 -- 172.21.15.140:0/2870905308 >> 172.21.15.140:0/2870905308 conn(0x7fdbf80634e0 msgr2=0x7fdbf80b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:30.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.143+0000 7fdbf7fff700 1 -- 172.21.15.140:0/2870905308 shutdown_connections 2022-07-21T01:50:30.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.143+0000 7fdbf7fff700 1 -- 172.21.15.140:0/2870905308 wait complete. 2022-07-21T01:50:30.151 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 16 2022-07-21T01:50:30.368 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:50:27.322541+0000', 'modified': '2022-07-21T01:50:28.325625+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5247': {'gid': 5247, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/207391195', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 207391195}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 207391195}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:50:30.369 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:50:30.370 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:50:30.487 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":5247,"want_state":"up:active","state":"up:active","rank_uptime":3.1547216950000001,"mdsmap_epoch":16,"osdmap_epoch":42,"osdmap_epoch_barrier":40,"uptime":7.5822837759999997} 2022-07-21T01:50:30.488 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 5247, "mdsmap_epoch": 16, "osdmap_epoch": 42, "osdmap_epoch_barrier": 40, "rank_uptime": 3.154721695, "state": "up:active", "uptime": 7.582283776, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:50:30.489 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:50:30.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36e0b19700 1 Processor -- start 2022-07-21T01:50:30.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36e0b19700 1 -- start start 2022-07-21T01:50:30.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36e0b19700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f36dc0b5650 0x7f36dc0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:30.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36e0b19700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36dc0adff0 0x7f36dc147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:30.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36e0b19700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36dc147010 con 0x7f36dc0af8e0 2022-07-21T01:50:30.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36e0b19700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36dc1472b0 con 0x7f36dc0b5650 2022-07-21T01:50:30.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36e0b19700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc146d40 con 0x7f36dc0adff0 2022-07-21T01:50:30.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36dbfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f36dc0af8e0 0x7f36dc05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48664/0 (socket says 172.21.15.140:48664) 2022-07-21T01:50:30.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36dbfff700 1 -- 172.21.15.140:0/1603224835 learned_addr learned my addr 172.21.15.140:0/1603224835 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:30.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36db7fe700 1 --2- 172.21.15.140:0/1603224835 >> v2:172.21.15.152:3300/0 conn(0x7f36dc0b5650 0x7f36dc0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:30.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36daffd700 1 --2- 172.21.15.140:0/1603224835 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36dc0adff0 0x7f36dc147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:30.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36da7fc700 1 -- 172.21.15.140:0/1603224835 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036611461 0 0) 0x7f36dc147010 con 0x7f36dc0af8e0 2022-07-21T01:50:30.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36da7fc700 1 -- 172.21.15.140:0/1603224835 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36c8003260 con 0x7f36dc0af8e0 2022-07-21T01:50:30.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36db7fe700 1 -- 172.21.15.140:0/1603224835 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36dc0adff0 msgr2=0x7f36dc147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:30.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36db7fe700 1 --2- 172.21.15.140:0/1603224835 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36dc0adff0 0x7f36dc147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36db7fe700 1 -- 172.21.15.140:0/1603224835 >> v1:172.21.15.140:6789/0 conn(0x7f36dc0af8e0 legacy=0x7f36dc05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36db7fe700 1 -- 172.21.15.140:0/1603224835 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc147c40 con 0x7f36dc0b5650 2022-07-21T01:50:30.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36daffd700 1 --2- 172.21.15.140:0/1603224835 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36dc0adff0 0x7f36dc147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:30.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.595+0000 7f36db7fe700 1 --2- 172.21.15.140:0/1603224835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36dc0b5650 0x7f36dc0b5a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f36d40032f0 tx=0x7f36d40091d0).ready entity=mon.1 client_cookie=d3f3c3ad819bc28d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:30.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.599+0000 7f36da7fc700 1 -- 172.21.15.140:0/1603224835 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36d400b600 con 0x7f36dc0b5650 2022-07-21T01:50:30.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.599+0000 7f36da7fc700 1 -- 172.21.15.140:0/1603224835 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d400b760 con 0x7f36dc0b5650 2022-07-21T01:50:30.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.599+0000 7f36da7fc700 1 -- 172.21.15.140:0/1603224835 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36d400b8d0 con 0x7f36dc0b5650 2022-07-21T01:50:30.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.599+0000 7f36e0b19700 1 -- 172.21.15.140:0/1603224835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36dc0b5650 msgr2=0x7f36dc0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.599+0000 7f36e0b19700 1 --2- 172.21.15.140:0/1603224835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36dc0b5650 0x7f36dc0b5a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f36d40032f0 tx=0x7f36d40091d0).stop 2022-07-21T01:50:30.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.599+0000 7f36e0b19700 1 -- 172.21.15.140:0/1603224835 shutdown_connections 2022-07-21T01:50:30.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.599+0000 7f36e0b19700 1 --2- 172.21.15.140:0/1603224835 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36dc0adff0 0x7f36dc147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.599+0000 7f36e0b19700 1 --2- 172.21.15.140:0/1603224835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36dc0b5650 0x7f36dc0b5a30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.599+0000 7f36e0b19700 1 -- 172.21.15.140:0/1603224835 >> 172.21.15.140:0/1603224835 conn(0x7f36dc0634e0 msgr2=0x7f36dc0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:30.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.599+0000 7f36e0b19700 1 -- 172.21.15.140:0/1603224835 shutdown_connections 2022-07-21T01:50:30.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.599+0000 7f36e0b19700 1 -- 172.21.15.140:0/1603224835 wait complete. 2022-07-21T01:50:30.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36e0b19700 1 Processor -- start 2022-07-21T01:50:30.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36e0b19700 1 -- start start 2022-07-21T01:50:30.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36e0b19700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36dc0adff0 0x7f36dc0657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:30.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36e0b19700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36dc0af8e0 0x7f36dc149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:30.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36e0b19700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36dc0b5650 0x7f36dc1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:30.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36e0b19700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc064df0 con 0x7f36dc0af8e0 2022-07-21T01:50:30.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36e0b19700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc065090 con 0x7f36dc0b5650 2022-07-21T01:50:30.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36e0b19700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36dc065330 con 0x7f36dc0adff0 2022-07-21T01:50:30.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36daffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36dc0af8e0 0x7f36dc149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:30.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36daffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36dc0af8e0 0x7f36dc149510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34524/0 (socket says 172.21.15.140:34524) 2022-07-21T01:50:30.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36daffd700 1 -- 172.21.15.140:0/907912010 learned_addr learned my addr 172.21.15.140:0/907912010 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:30.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36db7fe700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36dc0adff0 0x7f36dc0657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:30.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36dbfff700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36dc0b5650 0x7f36dc1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:30.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36daffd700 1 -- 172.21.15.140:0/907912010 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36dc0adff0 msgr2=0x7f36dc0657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36daffd700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36dc0adff0 0x7f36dc0657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36daffd700 1 -- 172.21.15.140:0/907912010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36dc0b5650 msgr2=0x7f36dc1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36daffd700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36dc0b5650 0x7f36dc1562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36daffd700 1 -- 172.21.15.140:0/907912010 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d000b120 con 0x7f36dc0af8e0 2022-07-21T01:50:30.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36daffd700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36dc0af8e0 0x7f36dc149510 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f36d000c000 tx=0x7f36d0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:30.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36d8ff9700 1 -- 172.21.15.140:0/907912010 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36d0012600 con 0x7f36dc0af8e0 2022-07-21T01:50:30.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36e0b19700 1 -- 172.21.15.140:0/907912010 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d4009470 con 0x7f36dc0af8e0 2022-07-21T01:50:30.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36d8ff9700 1 -- 172.21.15.140:0/907912010 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d0012760 con 0x7f36dc0af8e0 2022-07-21T01:50:30.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36d8ff9700 1 -- 172.21.15.140:0/907912010 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36d0012ac0 con 0x7f36dc0af8e0 2022-07-21T01:50:30.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.603+0000 7f36e0b19700 1 -- 172.21.15.140:0/907912010 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36dc111020 con 0x7f36dc0af8e0 2022-07-21T01:50:30.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.607+0000 7f36d8ff9700 1 -- 172.21.15.140:0/907912010 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f36d0022070 con 0x7f36dc0af8e0 2022-07-21T01:50:30.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.607+0000 7f36e0b19700 1 -- 172.21.15.140:0/907912010 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36dc0654f0 con 0x7f36dc0af8e0 2022-07-21T01:50:30.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.607+0000 7f36d8ff9700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36bc053a70 0x7f36bc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:30.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.607+0000 7f36d8ff9700 1 -- 172.21.15.140:0/907912010 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f36d006c3e0 con 0x7f36dc0af8e0 2022-07-21T01:50:30.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.607+0000 7f36db7fe700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36bc053a70 0x7f36bc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:30.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.607+0000 7f36db7fe700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36bc053a70 0x7f36bc055f30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f36d401b000 tx=0x7f36d4016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:30.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.615+0000 7f36d8ff9700 1 -- 172.21.15.140:0/907912010 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f36d0043170 con 0x7f36dc0af8e0 2022-07-21T01:50:30.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.743+0000 7f36e0b19700 1 -- 172.21.15.140:0/907912010 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36dc05c4a0 con 0x7f36dc0af8e0 2022-07-21T01:50:30.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.743+0000 7f36d8ff9700 1 -- 172.21.15.140:0/907912010 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f36d0046790 con 0x7f36dc0af8e0 2022-07-21T01:50:30.751 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:30.751 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":42,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:50:29.338430+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:50:25.939192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-07-21T01:50:26.391519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:50:30.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 -- 172.21.15.140:0/907912010 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36bc053a70 msgr2=0x7f36bc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36bc053a70 0x7f36bc055f30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f36d401b000 tx=0x7f36d4016000).stop 2022-07-21T01:50:30.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 -- 172.21.15.140:0/907912010 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36dc0af8e0 msgr2=0x7f36dc149510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36dc0af8e0 0x7f36dc149510 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f36d000c000 tx=0x7f36d0008040).stop 2022-07-21T01:50:30.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 -- 172.21.15.140:0/907912010 shutdown_connections 2022-07-21T01:50:30.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36bc053a70 0x7f36bc055f30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36dc0b5650 0x7f36dc1562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36dc0af8e0 0x7f36dc149510 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 --2- 172.21.15.140:0/907912010 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36dc0adff0 0x7f36dc0657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 -- 172.21.15.140:0/907912010 >> 172.21.15.140:0/907912010 conn(0x7f36dc0634e0 msgr2=0x7f36dc0b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:30.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 -- 172.21.15.140:0/907912010 shutdown_connections 2022-07-21T01:50:30.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.747+0000 7f36e0b19700 1 -- 172.21.15.140:0/907912010 wait complete. 2022-07-21T01:50:30.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:50:25.939192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:50:26.391519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:50:30.766 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:50:30.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11fd444700 1 Processor -- start 2022-07-21T01:50:30.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11fd444700 1 -- start start 2022-07-21T01:50:30.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11fd444700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f80aef20 0x7f11f80ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:30.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11fd444700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f80b5dd0 con 0x7f11f80b59f0 2022-07-21T01:50:30.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11fd444700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f80ad6e0 con 0x7f11f8140070 2022-07-21T01:50:30.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11fd444700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80ae1f0 con 0x7f11f80aef20 2022-07-21T01:50:30.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11f7fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f11f80b59f0 0x7f11f8139440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48674/0 (socket says 172.21.15.140:48674) 2022-07-21T01:50:30.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11f7fff700 1 -- 172.21.15.140:0/3562534127 learned_addr learned my addr 172.21.15.140:0/3562534127 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:30.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11fcc43700 1 --2- 172.21.15.140:0/3562534127 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f80aef20 0x7f11f80ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:30.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11f6ffd700 1 -- 172.21.15.140:0/3562534127 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251854015 0 0) 0x7f11f80b5dd0 con 0x7f11f80b59f0 2022-07-21T01:50:30.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11f6ffd700 1 -- 172.21.15.140:0/3562534127 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11d8003260 con 0x7f11f80b59f0 2022-07-21T01:50:30.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11fcc43700 1 -- 172.21.15.140:0/3562534127 >> v1:172.21.15.152:6789/0 conn(0x7f11f8140070 legacy=0x7f11f80acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11fcc43700 1 -- 172.21.15.140:0/3562534127 >> v1:172.21.15.140:6789/0 conn(0x7f11f80b59f0 legacy=0x7f11f8139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.879+0000 7f11fcc43700 1 -- 172.21.15.140:0/3562534127 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f8140500 con 0x7f11f80aef20 2022-07-21T01:50:30.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.883+0000 7f11fcc43700 1 --2- 172.21.15.140:0/3562534127 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f80aef20 0x7f11f80ad1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f11f00032f0 tx=0x7f11f00091d0).ready entity=mon.2 client_cookie=c7951d8aff3c0864 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:30.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.883+0000 7f11f6ffd700 1 -- 172.21.15.140:0/3562534127 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f000b6c0 con 0x7f11f80aef20 2022-07-21T01:50:30.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.883+0000 7f11f6ffd700 1 -- 172.21.15.140:0/3562534127 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f000b820 con 0x7f11f80aef20 2022-07-21T01:50:30.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.883+0000 7f11f6ffd700 1 -- 172.21.15.140:0/3562534127 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11f000b990 con 0x7f11f80aef20 2022-07-21T01:50:30.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.883+0000 7f11fd444700 1 -- 172.21.15.140:0/3562534127 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f80aef20 msgr2=0x7f11f80ad1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.883+0000 7f11fd444700 1 --2- 172.21.15.140:0/3562534127 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f80aef20 0x7f11f80ad1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f11f00032f0 tx=0x7f11f00091d0).stop 2022-07-21T01:50:30.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.883+0000 7f11fd444700 1 -- 172.21.15.140:0/3562534127 shutdown_connections 2022-07-21T01:50:30.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.883+0000 7f11fd444700 1 --2- 172.21.15.140:0/3562534127 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f80aef20 0x7f11f80ad1f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.883+0000 7f11fd444700 1 -- 172.21.15.140:0/3562534127 >> 172.21.15.140:0/3562534127 conn(0x7f11f8063480 msgr2=0x7f11f80b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:30.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.883+0000 7f11fd444700 1 -- 172.21.15.140:0/3562534127 shutdown_connections 2022-07-21T01:50:30.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.883+0000 7f11fd444700 1 -- 172.21.15.140:0/3562534127 wait complete. 2022-07-21T01:50:30.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fd444700 1 Processor -- start 2022-07-21T01:50:30.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fd444700 1 -- start start 2022-07-21T01:50:30.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fd444700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11f80aef20 0x7f11f813e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:30.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fd444700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11f80b59f0 0x7f11f8142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:30.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fd444700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f8066c50 0x7f11f81430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:30.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fd444700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8065bd0 con 0x7f11f80aef20 2022-07-21T01:50:30.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fd444700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8065e70 con 0x7f11f80b59f0 2022-07-21T01:50:30.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fd444700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8066110 con 0x7f11f8066c50 2022-07-21T01:50:30.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11f80aef20 0x7f11f813e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:30.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11f80aef20 0x7f11f813e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34534/0 (socket says 172.21.15.140:34534) 2022-07-21T01:50:30.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f7fff700 1 -- 172.21.15.140:0/1135066641 learned_addr learned my addr 172.21.15.140:0/1135066641 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:30.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f77fe700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11f80b59f0 0x7f11f8142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:30.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fcc43700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f8066c50 0x7f11f81430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:30.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f7fff700 1 -- 172.21.15.140:0/1135066641 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f8066c50 msgr2=0x7f11f81430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f7fff700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f8066c50 0x7f11f81430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f7fff700 1 -- 172.21.15.140:0/1135066641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11f80b59f0 msgr2=0x7f11f8142bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:30.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f7fff700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11f80b59f0 0x7f11f8142bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:30.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f7fff700 1 -- 172.21.15.140:0/1135066641 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0009470 con 0x7f11f80aef20 2022-07-21T01:50:30.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fcc43700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f8066c50 0x7f11f81430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:30.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f7fff700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11f80aef20 0x7f11f813e570 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f11e8001110 tx=0x7f11e8007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:30.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f57fa700 1 -- 172.21.15.140:0/1135066641 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e80070d0 con 0x7f11f80aef20 2022-07-21T01:50:30.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f57fa700 1 -- 172.21.15.140:0/1135066641 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e8007230 con 0x7f11f80aef20 2022-07-21T01:50:30.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11f57fa700 1 -- 172.21.15.140:0/1135066641 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11e8005610 con 0x7f11f80aef20 2022-07-21T01:50:30.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fd444700 1 -- 172.21.15.140:0/1135066641 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f80663b0 con 0x7f11f80aef20 2022-07-21T01:50:30.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.887+0000 7f11fd444700 1 -- 172.21.15.140:0/1135066641 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f8143590 con 0x7f11f80aef20 2022-07-21T01:50:30.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.891+0000 7f11fd444700 1 -- 172.21.15.140:0/1135066641 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f81424b0 con 0x7f11f80aef20 2022-07-21T01:50:30.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.891+0000 7f11f57fa700 1 -- 172.21.15.140:0/1135066641 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f11e8010070 con 0x7f11f80aef20 2022-07-21T01:50:30.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.891+0000 7f11f57fa700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11dc0539e0 0x7f11dc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:30.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.891+0000 7f11f57fa700 1 -- 172.21.15.140:0/1135066641 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f11e806b4f0 con 0x7f11f80aef20 2022-07-21T01:50:30.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.891+0000 7f11f77fe700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11dc0539e0 0x7f11dc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:30.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.891+0000 7f11f77fe700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11dc0539e0 0x7f11dc055ea0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f11e000d000 tx=0x7f11e0007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:30.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:30.899+0000 7f11f57fa700 1 -- 172.21.15.140:0/1135066641 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f11f81424b0 con 0x7f11f80aef20 2022-07-21T01:50:31.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.023+0000 7f11fd444700 1 -- 172.21.15.140:0/1135066641 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f11f81424b0 con 0x7f11f80aef20 2022-07-21T01:50:31.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.023+0000 7f11f57fa700 1 -- 172.21.15.140:0/1135066641 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v42) v1 ==== 110+0+10 (secure 0 0 0) 0x7f11f81424b0 con 0x7f11f80aef20 2022-07-21T01:50:31.027 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:50:31.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 -- 172.21.15.140:0/1135066641 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11dc0539e0 msgr2=0x7f11dc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11dc0539e0 0x7f11dc055ea0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f11e000d000 tx=0x7f11e0007b70).stop 2022-07-21T01:50:31.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 -- 172.21.15.140:0/1135066641 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11f80aef20 msgr2=0x7f11f813e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11f80aef20 0x7f11f813e570 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f11e8001110 tx=0x7f11e8007aa0).stop 2022-07-21T01:50:31.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 -- 172.21.15.140:0/1135066641 shutdown_connections 2022-07-21T01:50:31.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11dc0539e0 0x7f11dc055ea0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11f8066c50 0x7f11f81430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11f80b59f0 0x7f11f8142bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 --2- 172.21.15.140:0/1135066641 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11f80aef20 0x7f11f813e570 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 -- 172.21.15.140:0/1135066641 >> 172.21.15.140:0/1135066641 conn(0x7f11f8063480 msgr2=0x7f11f81390c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:31.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 -- 172.21.15.140:0/1135066641 shutdown_connections 2022-07-21T01:50:31.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.027+0000 7f11fd444700 1 -- 172.21.15.140:0/1135066641 wait complete. 2022-07-21T01:50:31.039 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:50:31.112 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:50:31.115 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:50:31.118 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:50:31.122 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:50:31.126 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:50:31.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e7fff700 1 Processor -- start 2022-07-21T01:50:31.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e7fff700 1 -- start start 2022-07-21T01:50:31.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80b32a0 0x7fd2e81468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e7fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e80b3f10 con 0x7fd2e805d160 2022-07-21T01:50:31.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e7fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e80b2620 con 0x7fd2e80b2ec0 2022-07-21T01:50:31.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80b4ea0 con 0x7fd2e80b32a0 2022-07-21T01:50:31.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e67fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd2e805d160 0x7fd2e81461c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48690/0 (socket says 172.21.15.140:48690) 2022-07-21T01:50:31.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e67fc700 1 -- 172.21.15.140:0/2521325523 learned_addr learned my addr 172.21.15.140:0/2521325523 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:31.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e77fe700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80b32a0 0x7fd2e81468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612537980 0 0) 0x7fd2e80b3f10 con 0x7fd2e805d160 2022-07-21T01:50:31.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2cc003260 con 0x7fd2e805d160 2022-07-21T01:50:31.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2186165576 0 0) 0x7fd2cc003260 con 0x7fd2e805d160 2022-07-21T01:50:31.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80b32a0 msgr2=0x7fd2e81468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:31.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80b32a0 0x7fd2e81468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 >> v1:172.21.15.152:6789/0 conn(0x7fd2e80b2ec0 legacy=0x7fd2e8142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80b28c0 con 0x7fd2e805d160 2022-07-21T01:50:31.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fd2dc001660 con 0x7fd2e805d160 2022-07-21T01:50:31.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 >> v1:172.21.15.140:6789/0 conn(0x7fd2e805d160 legacy=0x7fd2e81461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc003780 0x7fd2cc003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80b32a0 0x7fd2cc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2cc0049f0 0x7fd2cc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80b4ea0 con 0x7fd2cc0049f0 2022-07-21T01:50:31.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc00b750 con 0x7fd2cc003780 2022-07-21T01:50:31.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc00b8c0 con 0x7fd2e80b32a0 2022-07-21T01:50:31.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e6ffd700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc003780 0x7fd2cc003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e77fe700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2cc0049f0 0x7fd2cc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.139+0000 7fd2e67fc700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80b32a0 0x7fd2cc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e77fe700 1 -- 172.21.15.140:0/2521325523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80b32a0 msgr2=0x7fd2cc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:31.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e77fe700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80b32a0 0x7fd2cc004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e77fe700 1 -- 172.21.15.140:0/2521325523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc003780 msgr2=0x7fd2cc003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:31.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e77fe700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc003780 0x7fd2cc003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e77fe700 1 -- 172.21.15.140:0/2521325523 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc003260 con 0x7fd2cc0049f0 2022-07-21T01:50:31.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e77fe700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2cc0049f0 0x7fd2cc004e70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fd2d800c000 tx=0x7fd2d8006df0).ready entity=mon.0 client_cookie=f10cac49a9d73323 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d800e7d0 con 0x7fd2cc0049f0 2022-07-21T01:50:31.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d800e930 con 0x7fd2cc0049f0 2022-07-21T01:50:31.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e5ffb700 1 -- 172.21.15.140:0/2521325523 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2d800ec90 con 0x7fd2cc0049f0 2022-07-21T01:50:31.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 -- 172.21.15.140:0/2521325523 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2cc0049f0 msgr2=0x7fd2cc004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 --2- 172.21.15.140:0/2521325523 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2cc0049f0 0x7fd2cc004e70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7fd2d800c000 tx=0x7fd2d8006df0).stop 2022-07-21T01:50:31.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e77fe700 1 -- 172.21.15.140:0/2521325523 reap_dead start 2022-07-21T01:50:31.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 -- 172.21.15.140:0/2521325523 shutdown_connections 2022-07-21T01:50:31.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 -- 172.21.15.140:0/2521325523 >> 172.21.15.140:0/2521325523 conn(0x7fd2e8066e20 msgr2=0x7fd2e8067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:31.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 -- 172.21.15.140:0/2521325523 shutdown_connections 2022-07-21T01:50:31.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 -- 172.21.15.140:0/2521325523 wait complete. 2022-07-21T01:50:31.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 Processor -- start 2022-07-21T01:50:31.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 -- start start 2022-07-21T01:50:31.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2e80ace80 0x7fd2e8146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2e813d1c0 0x7fd2e8144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80adce0 0x7fd2e80ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e813efb0 con 0x7fd2e813d1c0 2022-07-21T01:50:31.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.143+0000 7fd2e7fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80ad260 con 0x7fd2e80ace80 2022-07-21T01:50:31.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e8146e40 con 0x7fd2e80adce0 2022-07-21T01:50:31.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e67fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2e813d1c0 0x7fd2e8144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e67fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2e813d1c0 0x7fd2e8144c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34556/0 (socket says 172.21.15.140:34556) 2022-07-21T01:50:31.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e67fc700 1 -- 172.21.15.140:0/1307650986 learned_addr learned my addr 172.21.15.140:0/1307650986 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:31.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e6ffd700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2e80ace80 0x7fd2e8146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e77fe700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80adce0 0x7fd2e80ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e67fc700 1 -- 172.21.15.140:0/1307650986 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80adce0 msgr2=0x7fd2e80ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e67fc700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80adce0 0x7fd2e80ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e67fc700 1 -- 172.21.15.140:0/1307650986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2e80ace80 msgr2=0x7fd2e8146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e67fc700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2e80ace80 0x7fd2e8146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e67fc700 1 -- 172.21.15.140:0/1307650986 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e8147060 con 0x7fd2e813d1c0 2022-07-21T01:50:31.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e77fe700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80adce0 0x7fd2e80ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:31.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e6ffd700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2e80ace80 0x7fd2e8146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:31.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e67fc700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2e813d1c0 0x7fd2e8144c90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fd2dc00d000 tx=0x7fd2dc001e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2d7fff700 1 -- 172.21.15.140:0/1307650986 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2dc011780 con 0x7fd2e813d1c0 2022-07-21T01:50:31.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2d7fff700 1 -- 172.21.15.140:0/1307650986 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc0118e0 con 0x7fd2e813d1c0 2022-07-21T01:50:31.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2d7fff700 1 -- 172.21.15.140:0/1307650986 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd2dc011c40 con 0x7fd2e813d1c0 2022-07-21T01:50:31.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e7fff700 1 -- 172.21.15.140:0/1307650986 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e8144550 con 0x7fd2e813d1c0 2022-07-21T01:50:31.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e7fff700 1 -- 172.21.15.140:0/1307650986 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e8155860 con 0x7fd2e813d1c0 2022-07-21T01:50:31.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e7fff700 1 -- 172.21.15.140:0/1307650986 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e80b4a00 con 0x7fd2e813d1c0 2022-07-21T01:50:31.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2d7fff700 1 -- 172.21.15.140:0/1307650986 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd2dc01b030 con 0x7fd2e813d1c0 2022-07-21T01:50:31.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2d7fff700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2c4053aa0 0x7fd2c4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2d7fff700 1 -- 172.21.15.140:0/1307650986 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd2dc06bb00 con 0x7fd2e813d1c0 2022-07-21T01:50:31.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e6ffd700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2c4053aa0 0x7fd2c4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.147+0000 7fd2e6ffd700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2c4053aa0 0x7fd2c4055f60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fd2e0003000 tx=0x7fd2e0007d60).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.155+0000 7fd2d7fff700 1 -- 172.21.15.140:0/1307650986 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd2dc042980 con 0x7fd2e813d1c0 2022-07-21T01:50:31.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.279+0000 7fd2e7fff700 1 -- 172.21.15.140:0/1307650986 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd2e80574e0 con 0x7fd2e813d1c0 2022-07-21T01:50:31.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.279+0000 7fd2d7fff700 1 -- 172.21.15.140:0/1307650986 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v42) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd2dc019030 con 0x7fd2e813d1c0 2022-07-21T01:50:31.281 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:50:31.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.279+0000 7fd2e7fff700 1 -- 172.21.15.140:0/1307650986 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2c4053aa0 msgr2=0x7fd2c4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.279+0000 7fd2e7fff700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2c4053aa0 0x7fd2c4055f60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fd2e0003000 tx=0x7fd2e0007d60).stop 2022-07-21T01:50:31.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.279+0000 7fd2e7fff700 1 -- 172.21.15.140:0/1307650986 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2e813d1c0 msgr2=0x7fd2e8144c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.279+0000 7fd2e7fff700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2e813d1c0 0x7fd2e8144c90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fd2dc00d000 tx=0x7fd2dc001e30).stop 2022-07-21T01:50:31.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.283+0000 7fd2e7fff700 1 -- 172.21.15.140:0/1307650986 shutdown_connections 2022-07-21T01:50:31.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.283+0000 7fd2e7fff700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd2c4053aa0 0x7fd2c4055f60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.283+0000 7fd2e7fff700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2e80adce0 0x7fd2e80ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.283+0000 7fd2e7fff700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2e813d1c0 0x7fd2e8144c90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.283+0000 7fd2e7fff700 1 --2- 172.21.15.140:0/1307650986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2e80ace80 0x7fd2e8146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.283+0000 7fd2e7fff700 1 -- 172.21.15.140:0/1307650986 >> 172.21.15.140:0/1307650986 conn(0x7fd2e8066e20 msgr2=0x7fd2e81435e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:31.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.283+0000 7fd2e7fff700 1 -- 172.21.15.140:0/1307650986 shutdown_connections 2022-07-21T01:50:31.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.283+0000 7fd2e7fff700 1 -- 172.21.15.140:0/1307650986 wait complete. 2022-07-21T01:50:31.294 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:50:31.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d759e700 1 Processor -- start 2022-07-21T01:50:31.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d759e700 1 -- start start 2022-07-21T01:50:31.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d80b1030 0x7f33d81431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d759e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d80ad380 con 0x7f33d80acfa0 2022-07-21T01:50:31.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d759e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d80af470 con 0x7f33d80b02e0 2022-07-21T01:50:31.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d813f030 con 0x7f33d80b1030 2022-07-21T01:50:31.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d659c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f33d80acfa0 0x7f33d8142ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48708/0 (socket says 172.21.15.140:48708) 2022-07-21T01:50:31.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d659c700 1 -- 172.21.15.140:0/2888489827 learned_addr learned my addr 172.21.15.140:0/2888489827 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:31.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33cdd9b700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d80b1030 0x7f33d81431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339355500 0 0) 0x7f33d80ad380 con 0x7f33d80acfa0 2022-07-21T01:50:31.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33b8004370 con 0x7f33d80acfa0 2022-07-21T01:50:31.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031861820 0 0) 0x7f33d80af470 con 0x7f33d80b02e0 2022-07-21T01:50:31.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33d80ad380 con 0x7f33d80b02e0 2022-07-21T01:50:31.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1686212024 0 0) 0x7f33b8004370 con 0x7f33d80acfa0 2022-07-21T01:50:31.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d80b1030 msgr2=0x7f33d81431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:31.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d80b1030 0x7f33d81431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 >> v1:172.21.15.152:6789/0 conn(0x7f33d80b02e0 legacy=0x7f33d80ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d80b1630 con 0x7f33d80acfa0 2022-07-21T01:50:31.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f33c8001680 con 0x7f33d80acfa0 2022-07-21T01:50:31.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33cdd9b700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d80b1030 0x7f33d81431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:31.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 >> v1:172.21.15.140:6789/0 conn(0x7f33d80acfa0 legacy=0x7f33d8142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33b8004a20 0x7f33b8004e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33b8005610 0x7f33b8005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d80b1030 0x7f33b8005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b800c860 con 0x7f33b8005610 2022-07-21T01:50:31.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b800c9a0 con 0x7f33b8004a20 2022-07-21T01:50:31.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b800cb60 con 0x7f33d80b1030 2022-07-21T01:50:31.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33cdd9b700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33b8005610 0x7f33b8005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d6d9d700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d80b1030 0x7f33b8005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33d659c700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33b8004a20 0x7f33b8004e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33cdd9b700 1 -- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d80b1030 msgr2=0x7f33b8005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:31.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33cdd9b700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d80b1030 0x7f33b8005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33cdd9b700 1 -- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33b8004a20 msgr2=0x7f33b8004e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:31.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33cdd9b700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33b8004a20 0x7f33b8004e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33cdd9b700 1 -- 172.21.15.140:0/2888489827 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c000b180 con 0x7f33b8005610 2022-07-21T01:50:31.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.403+0000 7f33cdd9b700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33b8005610 0x7f33b8005a90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f33c000c000 tx=0x7f33c0006df0).ready entity=mon.0 client_cookie=adee6cf348aed87d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33c000e800 con 0x7f33b8005610 2022-07-21T01:50:31.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c000e960 con 0x7f33b8005610 2022-07-21T01:50:31.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d5d9b700 1 -- 172.21.15.140:0/2888489827 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33c000ecc0 con 0x7f33b8005610 2022-07-21T01:50:31.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d759e700 1 -- 172.21.15.140:0/2888489827 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33b8005610 msgr2=0x7f33b8005a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d759e700 1 --2- 172.21.15.140:0/2888489827 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33b8005610 0x7f33b8005a90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f33c000c000 tx=0x7f33c0006df0).stop 2022-07-21T01:50:31.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d6d9d700 1 -- 172.21.15.140:0/2888489827 reap_dead start 2022-07-21T01:50:31.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d759e700 1 -- 172.21.15.140:0/2888489827 shutdown_connections 2022-07-21T01:50:31.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d759e700 1 -- 172.21.15.140:0/2888489827 >> 172.21.15.140:0/2888489827 conn(0x7f33d80632d0 msgr2=0x7f33d80636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:31.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d759e700 1 -- 172.21.15.140:0/2888489827 shutdown_connections 2022-07-21T01:50:31.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d759e700 1 -- 172.21.15.140:0/2888489827 wait complete. 2022-07-21T01:50:31.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d759e700 1 Processor -- start 2022-07-21T01:50:31.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.407+0000 7f33d759e700 1 -- start start 2022-07-21T01:50:31.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33b8005610 0x7f33d814d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33d8112710 0x7f33d810ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d8066490 0x7f33d8066910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8111c30 con 0x7f33b8005610 2022-07-21T01:50:31.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d814dcc0 con 0x7f33d8112710 2022-07-21T01:50:31.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d8066e00 con 0x7f33d8066490 2022-07-21T01:50:31.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33b8005610 0x7f33d814d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33b8005610 0x7f33d814d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34576/0 (socket says 172.21.15.140:34576) 2022-07-21T01:50:31.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d659c700 1 -- 172.21.15.140:0/1210117761 learned_addr learned my addr 172.21.15.140:0/1210117761 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:31.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d6d9d700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d8066490 0x7f33d8066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33cdd9b700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33d8112710 0x7f33d810ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d6d9d700 1 -- 172.21.15.140:0/1210117761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33d8112710 msgr2=0x7f33d810ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d6d9d700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33d8112710 0x7f33d810ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d6d9d700 1 -- 172.21.15.140:0/1210117761 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33b8005610 msgr2=0x7f33d814d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d6d9d700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33b8005610 0x7f33d814d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d6d9d700 1 -- 172.21.15.140:0/1210117761 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d814dee0 con 0x7f33d8066490 2022-07-21T01:50:31.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d6d9d700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d8066490 0x7f33d8066910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f33d0001210 tx=0x7f33d0009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33cffff700 1 -- 172.21.15.140:0/1210117761 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d0003210 con 0x7f33d8066490 2022-07-21T01:50:31.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33cffff700 1 -- 172.21.15.140:0/1210117761 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d0003370 con 0x7f33d8066490 2022-07-21T01:50:31.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d759e700 1 -- 172.21.15.140:0/1210117761 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d814e180 con 0x7f33d8066490 2022-07-21T01:50:31.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33cffff700 1 -- 172.21.15.140:0/1210117761 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33d00016e0 con 0x7f33d8066490 2022-07-21T01:50:31.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.411+0000 7f33d759e700 1 -- 172.21.15.140:0/1210117761 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d810f310 con 0x7f33d8066490 2022-07-21T01:50:31.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.415+0000 7f33d759e700 1 -- 172.21.15.140:0/1210117761 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d80574e0 con 0x7f33d8066490 2022-07-21T01:50:31.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.415+0000 7f33cffff700 1 -- 172.21.15.140:0/1210117761 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f33d001b060 con 0x7f33d8066490 2022-07-21T01:50:31.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.415+0000 7f33cffff700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f33bc053a70 0x7f33bc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.415+0000 7f33cffff700 1 -- 172.21.15.140:0/1210117761 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f33d006b8d0 con 0x7f33d8066490 2022-07-21T01:50:31.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.415+0000 7f33d659c700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f33bc053a70 0x7f33bc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.415+0000 7f33d659c700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f33bc053a70 0x7f33bc055f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f33c800d000 tx=0x7f33c8007040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.423+0000 7f33cffff700 1 -- 172.21.15.140:0/1210117761 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f33d00427f0 con 0x7f33d8066490 2022-07-21T01:50:31.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.551+0000 7f33d759e700 1 -- 172.21.15.140:0/1210117761 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33d8069a20 con 0x7f33d8066490 2022-07-21T01:50:31.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33cffff700 1 -- 172.21.15.140:0/1210117761 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+10 (secure 0 0 0) 0x7f33d0019020 con 0x7f33d8066490 2022-07-21T01:50:31.555 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:50:31.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 -- 172.21.15.140:0/1210117761 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f33bc053a70 msgr2=0x7f33bc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f33bc053a70 0x7f33bc055f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f33c800d000 tx=0x7f33c8007040).stop 2022-07-21T01:50:31.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 -- 172.21.15.140:0/1210117761 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d8066490 msgr2=0x7f33d8066910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d8066490 0x7f33d8066910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f33d0001210 tx=0x7f33d0009040).stop 2022-07-21T01:50:31.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 -- 172.21.15.140:0/1210117761 shutdown_connections 2022-07-21T01:50:31.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f33bc053a70 0x7f33bc055f30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33d8066490 0x7f33d8066910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33d8112710 0x7f33d810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 --2- 172.21.15.140:0/1210117761 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33b8005610 0x7f33d814d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 -- 172.21.15.140:0/1210117761 >> 172.21.15.140:0/1210117761 conn(0x7f33d80632d0 msgr2=0x7f33d80b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:31.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 -- 172.21.15.140:0/1210117761 shutdown_connections 2022-07-21T01:50:31.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.555+0000 7f33d759e700 1 -- 172.21.15.140:0/1210117761 wait complete. 2022-07-21T01:50:31.575 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:50:31.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.679+0000 7fdbc30db700 1 Processor -- start 2022-07-21T01:50:31.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.679+0000 7fdbc30db700 1 -- start start 2022-07-21T01:50:31.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdbbc1462a0 0x7fdbbc146680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdbbc146b70 0x7fdbbc060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbbc0afdb0 con 0x7fdbbc0988b0 2022-07-21T01:50:31.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc0b0050 con 0x7fdbbc1462a0 2022-07-21T01:50:31.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc05fcd0 con 0x7fdbbc146b70 2022-07-21T01:50:31.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc0e77700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdbbc1462a0 0x7fdbbc146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbbbfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fdbbc0988b0 0x7fdbbc1422c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48728/0 (socket says 172.21.15.140:48728) 2022-07-21T01:50:31.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbbbfff700 1 -- 172.21.15.140:0/1405011644 learned_addr learned my addr 172.21.15.140:0/1405011644 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:31.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc1678700 1 --2- 172.21.15.140:0/1405011644 >> v2:172.21.15.152:3300/0 conn(0x7fdbbc146b70 0x7fdbbc060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc0e77700 1 -- 172.21.15.140:0/1405011644 >> v2:172.21.15.152:3300/0 conn(0x7fdbbc146b70 msgr2=0x7fdbbc060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:31.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc0e77700 1 --2- 172.21.15.140:0/1405011644 >> v2:172.21.15.152:3300/0 conn(0x7fdbbc146b70 0x7fdbbc060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc0e77700 1 -- 172.21.15.140:0/1405011644 >> v1:172.21.15.140:6789/0 conn(0x7fdbbc0988b0 legacy=0x7fdbbc1422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbbb7fe700 1 -- 172.21.15.140:0/1405011644 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022167175 0 0) 0x7fdbbc0afdb0 con 0x7fdbbc0988b0 2022-07-21T01:50:31.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc0e77700 1 -- 172.21.15.140:0/1405011644 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc0afae0 con 0x7fdbbc1462a0 2022-07-21T01:50:31.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc0e77700 1 --2- 172.21.15.140:0/1405011644 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbbc1462a0 0x7fdbbc146680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fdbac002f70 tx=0x7fdbac006df0).ready entity=mon.0 client_cookie=20c1a14a8d8263ef server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbbb7fe700 1 -- 172.21.15.140:0/1405011644 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbac00d570 con 0x7fdbbc1462a0 2022-07-21T01:50:31.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbbb7fe700 1 -- 172.21.15.140:0/1405011644 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac000e90 con 0x7fdbbc1462a0 2022-07-21T01:50:31.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbbb7fe700 1 -- 172.21.15.140:0/1405011644 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdbac00d8c0 con 0x7fdbbc1462a0 2022-07-21T01:50:31.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 -- 172.21.15.140:0/1405011644 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbbc1462a0 msgr2=0x7fdbbc146680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 --2- 172.21.15.140:0/1405011644 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbbc1462a0 0x7fdbbc146680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fdbac002f70 tx=0x7fdbac006df0).stop 2022-07-21T01:50:31.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 -- 172.21.15.140:0/1405011644 shutdown_connections 2022-07-21T01:50:31.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 --2- 172.21.15.140:0/1405011644 >> v2:172.21.15.152:3300/0 conn(0x7fdbbc146b70 0x7fdbbc060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 --2- 172.21.15.140:0/1405011644 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbbc1462a0 0x7fdbbc146680 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 -- 172.21.15.140:0/1405011644 >> 172.21.15.140:0/1405011644 conn(0x7fdbbc0b5f00 msgr2=0x7fdbbc0b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:31.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 -- 172.21.15.140:0/1405011644 shutdown_connections 2022-07-21T01:50:31.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.683+0000 7fdbc30db700 1 -- 172.21.15.140:0/1405011644 wait complete. 2022-07-21T01:50:31.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbc30db700 1 Processor -- start 2022-07-21T01:50:31.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbc30db700 1 -- start start 2022-07-21T01:50:31.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbc30db700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbbc0988b0 0x7fdbbc1516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbc30db700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbbc1462a0 0x7fdbbc151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbc30db700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbbc146b70 0x7fdbbc098fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbc30db700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc1534a0 con 0x7fdbbc1462a0 2022-07-21T01:50:31.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbc30db700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc1531b0 con 0x7fdbbc0988b0 2022-07-21T01:50:31.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbc30db700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc1520b0 con 0x7fdbbc146b70 2022-07-21T01:50:31.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbbbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbbc1462a0 0x7fdbbc151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbc0e77700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbbc0988b0 0x7fdbbc1516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbbbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbbc1462a0 0x7fdbbc151bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34592/0 (socket says 172.21.15.140:34592) 2022-07-21T01:50:31.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbbbfff700 1 -- 172.21.15.140:0/2473783962 learned_addr learned my addr 172.21.15.140:0/2473783962 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:31.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbc1678700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbbc146b70 0x7fdbbc098fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbbbfff700 1 -- 172.21.15.140:0/2473783962 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbbc146b70 msgr2=0x7fdbbc098fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbbbfff700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbbc146b70 0x7fdbbc098fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbbbfff700 1 -- 172.21.15.140:0/2473783962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbbc0988b0 msgr2=0x7fdbbc1516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbbbfff700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbbc0988b0 0x7fdbbc1516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.687+0000 7fdbbbfff700 1 -- 172.21.15.140:0/2473783962 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac00b180 con 0x7fdbbc1462a0 2022-07-21T01:50:31.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.691+0000 7fdbbbfff700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbbc1462a0 0x7fdbbc151bc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fdbbc09e570 tx=0x7fdba4007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.691+0000 7fdbb9ffb700 1 -- 172.21.15.140:0/2473783962 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdba40070d0 con 0x7fdbbc1462a0 2022-07-21T01:50:31.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.691+0000 7fdbb9ffb700 1 -- 172.21.15.140:0/2473783962 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba4007230 con 0x7fdbbc1462a0 2022-07-21T01:50:31.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.691+0000 7fdbc30db700 1 -- 172.21.15.140:0/2473783962 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbbc152f10 con 0x7fdbbc1462a0 2022-07-21T01:50:31.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.691+0000 7fdbb9ffb700 1 -- 172.21.15.140:0/2473783962 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdba4005610 con 0x7fdbbc1462a0 2022-07-21T01:50:31.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.691+0000 7fdbc30db700 1 -- 172.21.15.140:0/2473783962 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc152cb0 con 0x7fdbbc1462a0 2022-07-21T01:50:31.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.691+0000 7fdbc30db700 1 -- 172.21.15.140:0/2473783962 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc0574e0 con 0x7fdbbc1462a0 2022-07-21T01:50:31.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.691+0000 7fdbb9ffb700 1 -- 172.21.15.140:0/2473783962 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdba401a060 con 0x7fdbbc1462a0 2022-07-21T01:50:31.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.691+0000 7fdbb9ffb700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdba0053950 0x7fdba0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.691+0000 7fdbb9ffb700 1 -- 172.21.15.140:0/2473783962 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdba406b270 con 0x7fdbbc1462a0 2022-07-21T01:50:31.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.691+0000 7fdbc0e77700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdba0053950 0x7fdba0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.695+0000 7fdbc0e77700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdba0053950 0x7fdba0055e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fdbac01a000 tx=0x7fdbac014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.699+0000 7fdbb9ffb700 1 -- 172.21.15.140:0/2473783962 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdba4042190 con 0x7fdbbc1462a0 2022-07-21T01:50:31.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.823+0000 7fdbc30db700 1 -- 172.21.15.140:0/2473783962 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdbbc067ea0 con 0x7fdbbc1462a0 2022-07-21T01:50:31.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbb9ffb700 1 -- 172.21.15.140:0/2473783962 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+10 (secure 0 0 0) 0x7fdba4018020 con 0x7fdbbc1462a0 2022-07-21T01:50:31.827 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:50:31.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 -- 172.21.15.140:0/2473783962 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdba0053950 msgr2=0x7fdba0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdba0053950 0x7fdba0055e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fdbac01a000 tx=0x7fdbac014000).stop 2022-07-21T01:50:31.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 -- 172.21.15.140:0/2473783962 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbbc1462a0 msgr2=0x7fdbbc151bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbbc1462a0 0x7fdbbc151bc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fdbbc09e570 tx=0x7fdba4007ac0).stop 2022-07-21T01:50:31.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 -- 172.21.15.140:0/2473783962 shutdown_connections 2022-07-21T01:50:31.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdba0053950 0x7fdba0055e10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbbc146b70 0x7fdbbc098fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdbbc1462a0 0x7fdbbc151bc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 --2- 172.21.15.140:0/2473783962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbbc0988b0 0x7fdbbc1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 -- 172.21.15.140:0/2473783962 >> 172.21.15.140:0/2473783962 conn(0x7fdbbc0b5f00 msgr2=0x7fdbbc13fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:31.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 -- 172.21.15.140:0/2473783962 shutdown_connections 2022-07-21T01:50:31.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.827+0000 7fdbc30db700 1 -- 172.21.15.140:0/2473783962 wait complete. 2022-07-21T01:50:31.846 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:50:31.847 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:50:31.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.951+0000 7f02a89ac700 1 Processor -- start 2022-07-21T01:50:31.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.951+0000 7f02a89ac700 1 -- start start 2022-07-21T01:50:31.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.951+0000 7f02a89ac700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02a4063d60 0x7f02a40b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.951+0000 7f02a89ac700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a4064140 con 0x7f02a40af150 2022-07-21T01:50:31.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.951+0000 7f02a89ac700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a40ad900 con 0x7f02a40b1440 2022-07-21T01:50:31.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.951+0000 7f02a89ac700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a4143800 con 0x7f02a4063d60 2022-07-21T01:50:31.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.951+0000 7f02a359e700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f02a40af150 0x7f02a40ad1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48742/0 (socket says 172.21.15.140:48742) 2022-07-21T01:50:31.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.951+0000 7f02a359e700 1 -- 172.21.15.140:0/1823085559 learned_addr learned my addr 172.21.15.140:0/1823085559 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:31.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a2d9d700 1 --2- 172.21.15.140:0/1823085559 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02a4063d60 0x7f02a40b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a259c700 1 -- 172.21.15.140:0/1823085559 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172173131 0 0) 0x7f02a4064140 con 0x7f02a40af150 2022-07-21T01:50:31.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a259c700 1 -- 172.21.15.140:0/1823085559 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0288003260 con 0x7f02a40af150 2022-07-21T01:50:31.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a2d9d700 1 -- 172.21.15.140:0/1823085559 >> v1:172.21.15.152:6789/0 conn(0x7f02a40b1440 legacy=0x7f02a40acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a259c700 1 -- 172.21.15.140:0/1823085559 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213744306 0 0) 0x7f02a40ad900 con 0x7f02a40b1440 2022-07-21T01:50:31.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a2d9d700 1 -- 172.21.15.140:0/1823085559 >> v1:172.21.15.140:6789/0 conn(0x7f02a40af150 legacy=0x7f02a40ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a2d9d700 1 -- 172.21.15.140:0/1823085559 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a4143aa0 con 0x7f02a4063d60 2022-07-21T01:50:31.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a2d9d700 1 --2- 172.21.15.140:0/1823085559 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02a4063d60 0x7f02a40b57e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f0294002f70 tx=0x7f0294006df0).ready entity=mon.2 client_cookie=1a1459a1f3c02b3b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a259c700 1 -- 172.21.15.140:0/1823085559 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f029400d570 con 0x7f02a4063d60 2022-07-21T01:50:31.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a259c700 1 -- 172.21.15.140:0/1823085559 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0294000e90 con 0x7f02a4063d60 2022-07-21T01:50:31.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a259c700 1 -- 172.21.15.140:0/1823085559 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f029400d8c0 con 0x7f02a4063d60 2022-07-21T01:50:31.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a89ac700 1 -- 172.21.15.140:0/1823085559 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02a4063d60 msgr2=0x7f02a40b57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a89ac700 1 --2- 172.21.15.140:0/1823085559 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02a4063d60 0x7f02a40b57e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f0294002f70 tx=0x7f0294006df0).stop 2022-07-21T01:50:31.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a89ac700 1 -- 172.21.15.140:0/1823085559 shutdown_connections 2022-07-21T01:50:31.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a89ac700 1 --2- 172.21.15.140:0/1823085559 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02a4063d60 0x7f02a40b57e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a89ac700 1 -- 172.21.15.140:0/1823085559 >> 172.21.15.140:0/1823085559 conn(0x7f02a40634e0 msgr2=0x7f02a40b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:31.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a89ac700 1 -- 172.21.15.140:0/1823085559 shutdown_connections 2022-07-21T01:50:31.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.955+0000 7f02a89ac700 1 -- 172.21.15.140:0/1823085559 wait complete. 2022-07-21T01:50:31.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a89ac700 1 Processor -- start 2022-07-21T01:50:31.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a89ac700 1 -- start start 2022-07-21T01:50:31.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a89ac700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02a4063d60 0x7f02a4066b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a89ac700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02a40af150 0x7f02a413dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a89ac700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02a40b1440 0x7f02a413e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a89ac700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40b5570 con 0x7f02a4063d60 2022-07-21T01:50:31.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a89ac700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a4152f00 con 0x7f02a40af150 2022-07-21T01:50:31.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a89ac700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a40b4ea0 con 0x7f02a40b1440 2022-07-21T01:50:31.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a2d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02a4063d60 0x7f02a4066b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a2d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02a4063d60 0x7f02a4066b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34600/0 (socket says 172.21.15.140:34600) 2022-07-21T01:50:31.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a2d9d700 1 -- 172.21.15.140:0/1473381272 learned_addr learned my addr 172.21.15.140:0/1473381272 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:31.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f029bfff700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02a40af150 0x7f02a413dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a359e700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02a40b1440 0x7f02a413e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a2d9d700 1 -- 172.21.15.140:0/1473381272 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02a40b1440 msgr2=0x7f02a413e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a2d9d700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02a40b1440 0x7f02a413e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a2d9d700 1 -- 172.21.15.140:0/1473381272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02a40af150 msgr2=0x7f02a413dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:31.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a2d9d700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02a40af150 0x7f02a413dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:31.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a2d9d700 1 -- 172.21.15.140:0/1473381272 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029400b180 con 0x7f02a4063d60 2022-07-21T01:50:31.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f029bfff700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02a40af150 0x7f02a413dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:31.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a2d9d700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02a4063d60 0x7f02a4066b00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f02a4153040 tx=0x7f0294014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a0d99700 1 -- 172.21.15.140:0/1473381272 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0294016070 con 0x7f02a4063d60 2022-07-21T01:50:31.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a0d99700 1 -- 172.21.15.140:0/1473381272 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0294013980 con 0x7f02a4063d60 2022-07-21T01:50:31.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a89ac700 1 -- 172.21.15.140:0/1473381272 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a4152ab0 con 0x7f02a4063d60 2022-07-21T01:50:31.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a0d99700 1 -- 172.21.15.140:0/1473381272 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0294013ce0 con 0x7f02a4063d60 2022-07-21T01:50:31.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.959+0000 7f02a89ac700 1 -- 172.21.15.140:0/1473381272 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a41526f0 con 0x7f02a4063d60 2022-07-21T01:50:31.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.963+0000 7f02a89ac700 1 -- 172.21.15.140:0/1473381272 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a40b1050 con 0x7f02a4063d60 2022-07-21T01:50:31.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.963+0000 7f02a0d99700 1 -- 172.21.15.140:0/1473381272 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f029400f030 con 0x7f02a4063d60 2022-07-21T01:50:31.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.963+0000 7f02a0d99700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f027c053a70 0x7f027c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:31.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.963+0000 7f02a0d99700 1 -- 172.21.15.140:0/1473381272 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f029406cc50 con 0x7f02a4063d60 2022-07-21T01:50:31.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.963+0000 7f029bfff700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f027c053a70 0x7f027c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:31.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.963+0000 7f029bfff700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f027c053a70 0x7f027c055f30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f028c00b000 tx=0x7f028c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:31.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:31.967+0000 7f02a0d99700 1 -- 172.21.15.140:0/1473381272 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0294043ad0 con 0x7f02a4063d60 2022-07-21T01:50:32.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.131+0000 7f02a89ac700 1 -- 172.21.15.140:0/1473381272 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02a40574e0 con 0x7f02a4063d60 2022-07-21T01:50:32.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.131+0000 7f02a0d99700 1 -- 172.21.15.140:0/1473381272 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2141 (secure 0 0 0) 0x7f0294023090 con 0x7f02a4063d60 2022-07-21T01:50:32.133 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:32.134 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/2307860244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2307860244},{"type":"v1","addr":"172.21.15.152:6833","nonce":2307860244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:50:27.322541+0000","modified":"2022-07-21T01:50:28.325625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/207391195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":207391195},{"type":"v1","addr":"172.21.15.140:6835","nonce":207391195}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-07-21T01:50:32.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 -- 172.21.15.140:0/1473381272 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f027c053a70 msgr2=0x7f027c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:32.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f027c053a70 0x7f027c055f30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f028c00b000 tx=0x7f028c009040).stop 2022-07-21T01:50:32.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 -- 172.21.15.140:0/1473381272 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02a4063d60 msgr2=0x7f02a4066b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:32.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02a4063d60 0x7f02a4066b00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f02a4153040 tx=0x7f0294014000).stop 2022-07-21T01:50:32.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 -- 172.21.15.140:0/1473381272 shutdown_connections 2022-07-21T01:50:32.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f027c053a70 0x7f027c055f30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:32.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02a40b1440 0x7f02a413e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:32.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02a40af150 0x7f02a413dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:32.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 --2- 172.21.15.140:0/1473381272 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02a4063d60 0x7f02a4066b00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:32.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 -- 172.21.15.140:0/1473381272 >> 172.21.15.140:0/1473381272 conn(0x7f02a40634e0 msgr2=0x7f02a405bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:32.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 -- 172.21.15.140:0/1473381272 shutdown_connections 2022-07-21T01:50:32.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:32.135+0000 7f02a89ac700 1 -- 172.21.15.140:0/1473381272 wait complete. 2022-07-21T01:50:32.142 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 16 2022-07-21T01:50:32.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:50:27.322541+0000', 'modified': '2022-07-21T01:50:28.325625+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5247': {'gid': 5247, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/207391195', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 207391195}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 207391195}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:50:32.348 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:50:32.348 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:50:32.464 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":5247,"want_state":"up:active","state":"up:active","rank_uptime":5.1307483539999996,"mdsmap_epoch":16,"osdmap_epoch":42,"osdmap_epoch_barrier":40,"uptime":9.5583102879999995} 2022-07-21T01:50:32.465 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 5247, "mdsmap_epoch": 16, "osdmap_epoch": 42, "osdmap_epoch_barrier": 40, "rank_uptime": 5.130748354, "state": "up:active", "uptime": 9.558310288, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:50:32.465 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:50:32.466 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:50:32.466 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:50:32.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:32.467 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:32.476 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:50:32.532 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:50:32.533 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:50:32.546 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:50:32.547 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:50:32.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:32.550 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:50:32.599 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:50:32.600 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:32.603 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:50:32.622 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:50:32.623 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:50:32.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:50:32.684+0000 7f0620ae2080 -1 init, newargv = 0x55c90f87e490 newargc=9 2022-07-21T01:50:32.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22347]: starting ceph client 2022-07-21T01:50:32.684 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:50:32.685 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:50:32.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:32.687 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:50:32.689 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22347]: starting fuse 2022-07-21T01:50:32.695 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:50:32.695 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:50:32.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:32.696 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:50:32.697 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:50:32.697 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:50:32.698 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:50:32.698 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:50:32.700 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:50:32.701 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:50:32.701 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:50:32.701 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:50:32.702 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:50:32.702 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:50:32.703 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:50:32.703 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:50:32.704 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:50:32.704 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:50:32.704 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:50:32.705 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:50:32.705 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:50:32.706 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:50:32.706 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:50:32.707 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:50:32.707 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:50:32.707 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:50:32.761 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.22347.asok 2022-07-21T01:50:32.766 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22347.asok 2022-07-21T01:50:32.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:32.767 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22347.asok status 2022-07-21T01:50:32.903 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:50:32.904 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:50:32.904 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:50:32.904 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:50:32.905 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:50:32.905 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:50:32.905 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:50:32.905 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "22347", 2022-07-21T01:50:32.906 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:50:32.906 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:50:32.906 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:50:32.906 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:50:32.907 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 5424, 2022-07-21T01:50:32.907 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:50:32.907 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:50:32.907 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:50:32.908 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 5424 2022-07-21T01:50:32.908 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:50:32.908 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:50:32.908 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:50:32.909 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:50:32.909 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 780301867 2022-07-21T01:50:32.909 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:50:32.909 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:50:32.910 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:50:32.910 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:50:32.910 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:50:32.910 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 780301867 2022-07-21T01:50:32.911 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:50:32.911 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.5424 172.21.15.191:0/780301867", 2022-07-21T01:50:32.911 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/780301867", 2022-07-21T01:50:32.911 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:50:32.912 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 16, 2022-07-21T01:50:32.912 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 42, 2022-07-21T01:50:32.912 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:50:32.912 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:50:32.913 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:50:32.913 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:50:32.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:32.918 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:32.918 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:32.928 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:50:32.929 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:50:32.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:32.931 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:32.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:32.991 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:32.992 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:33.000 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:50:33.003 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:50:33.004 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-07-21T01:50:33.005 DEBUG:teuthology.orchestra.run.smithi191:> import sys 2022-07-21T01:50:33.006 DEBUG:teuthology.orchestra.run.smithi191:> import time 2022-07-21T01:50:33.007 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:50:33.008 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:50:33.009 DEBUG:teuthology.orchestra.run.smithi191:> n = 1000 2022-07-21T01:50:33.009 DEBUG:teuthology.orchestra.run.smithi191:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-07-21T01:50:33.010 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:50:33.011 DEBUG:teuthology.orchestra.run.smithi191:> if not os.path.exists(os.path.dirname(abs_path)): 2022-07-21T01:50:33.012 DEBUG:teuthology.orchestra.run.smithi191:> os.makedirs(os.path.dirname(abs_path)) 2022-07-21T01:50:33.013 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:50:33.014 DEBUG:teuthology.orchestra.run.smithi191:> for i in range(0, n): 2022-07-21T01:50:33.015 DEBUG:teuthology.orchestra.run.smithi191:> fname = "{0}_{1}".format(abs_path, i) 2022-07-21T01:50:33.016 DEBUG:teuthology.orchestra.run.smithi191:> with open(fname, '"'"'w'"'"') as f: 2022-07-21T01:50:33.016 DEBUG:teuthology.orchestra.run.smithi191:> f.write('"'"'content'"'"') 2022-07-21T01:50:33.017 DEBUG:teuthology.orchestra.run.smithi191:> if True: 2022-07-21T01:50:33.018 DEBUG:teuthology.orchestra.run.smithi191:> f.flush() 2022-07-21T01:50:33.019 DEBUG:teuthology.orchestra.run.smithi191:> os.fsync(f.fileno()) 2022-07-21T01:50:33.020 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:50:42.127 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:50:42.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f50ef3700 1 Processor -- start 2022-07-21T01:50:42.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f50ef3700 1 -- start start 2022-07-21T01:50:42.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f50ef3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0b0570 0x7f3f4c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f50ef3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3f4c1424e0 0x7f3f4c0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f50ef3700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4c1433d0 con 0x7f3f4c0aea10 2022-07-21T01:50:42.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f50ef3700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4c143b10 con 0x7f3f4c1424e0 2022-07-21T01:50:42.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f50ef3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c143670 con 0x7f3f4c0b0570 2022-07-21T01:50:42.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f4bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3f4c1424e0 0x7f3f4c0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f42ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f3f4c0aea10 0x7f3f4c141dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48756/0 (socket says 172.21.15.140:48756) 2022-07-21T01:50:42.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f42ffd700 1 -- 172.21.15.140:0/1910346731 learned_addr learned my addr 172.21.15.140:0/1910346731 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:42.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f4b7fe700 1 --2- 172.21.15.140:0/1910346731 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0b0570 0x7f3f4c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f4affd700 1 -- 172.21.15.140:0/1910346731 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431257902 0 0) 0x7f3f4c1433d0 con 0x7f3f4c0aea10 2022-07-21T01:50:42.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.239+0000 7f3f4affd700 1 -- 172.21.15.140:0/1910346731 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c003260 con 0x7f3f4c0aea10 2022-07-21T01:50:42.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f4bfff700 1 -- 172.21.15.140:0/1910346731 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0b0570 msgr2=0x7f3f4c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:42.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f4bfff700 1 --2- 172.21.15.140:0/1910346731 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0b0570 0x7f3f4c0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f4affd700 1 -- 172.21.15.140:0/1910346731 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 820486046 0 0) 0x7f3f2c003260 con 0x7f3f4c0aea10 2022-07-21T01:50:42.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f4bfff700 1 -- 172.21.15.140:0/1910346731 >> v1:172.21.15.140:6789/0 conn(0x7f3f4c0aea10 legacy=0x7f3f4c141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f4bfff700 1 -- 172.21.15.140:0/1910346731 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c0b1010 con 0x7f3f4c1424e0 2022-07-21T01:50:42.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f4b7fe700 1 --2- 172.21.15.140:0/1910346731 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0b0570 0x7f3f4c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:42.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f4bfff700 1 --2- 172.21.15.140:0/1910346731 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f4c1424e0 0x7f3f4c0ad350 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f3f34001800 tx=0x7f3f440091d0).ready entity=mon.0 client_cookie=6b57b2c4e7d6740 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:42.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f4affd700 1 -- 172.21.15.140:0/1910346731 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f34001660 con 0x7f3f4c1424e0 2022-07-21T01:50:42.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f4affd700 1 -- 172.21.15.140:0/1910346731 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4400b580 con 0x7f3f4c1424e0 2022-07-21T01:50:42.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f4affd700 1 -- 172.21.15.140:0/1910346731 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f4400b6f0 con 0x7f3f4c1424e0 2022-07-21T01:50:42.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1910346731 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f4c1424e0 msgr2=0x7f3f4c0ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f50ef3700 1 --2- 172.21.15.140:0/1910346731 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f4c1424e0 0x7f3f4c0ad350 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f3f34001800 tx=0x7f3f440091d0).stop 2022-07-21T01:50:42.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1910346731 shutdown_connections 2022-07-21T01:50:42.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f50ef3700 1 --2- 172.21.15.140:0/1910346731 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f4c1424e0 0x7f3f4c0ad350 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f50ef3700 1 --2- 172.21.15.140:0/1910346731 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0b0570 0x7f3f4c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1910346731 >> 172.21.15.140:0/1910346731 conn(0x7f3f4c0b9360 msgr2=0x7f3f4c0b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:42.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1910346731 shutdown_connections 2022-07-21T01:50:42.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1910346731 wait complete. 2022-07-21T01:50:42.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f50ef3700 1 Processor -- start 2022-07-21T01:50:42.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.243+0000 7f3f50ef3700 1 -- start start 2022-07-21T01:50:42.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f50ef3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0aea10 0x7f3f4c112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f50ef3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f4c0b0570 0x7f3f4c13fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f50ef3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f4c1424e0 0x7f3f4c140370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f50ef3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c0b43d0 con 0x7f3f4c0b0570 2022-07-21T01:50:42.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f50ef3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c156220 con 0x7f3f4c1424e0 2022-07-21T01:50:42.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f50ef3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c155dc0 con 0x7f3f4c0aea10 2022-07-21T01:50:42.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f4c0b0570 0x7f3f4c13fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f42ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f4c0b0570 0x7f3f4c13fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34618/0 (socket says 172.21.15.140:34618) 2022-07-21T01:50:42.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f42ffd700 1 -- 172.21.15.140:0/1439399947 learned_addr learned my addr 172.21.15.140:0/1439399947 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:42.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f4b7fe700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0aea10 0x7f3f4c112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f4bfff700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f4c1424e0 0x7f3f4c140370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f4b7fe700 1 -- 172.21.15.140:0/1439399947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f4c1424e0 msgr2=0x7f3f4c140370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f4b7fe700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f4c1424e0 0x7f3f4c140370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f4b7fe700 1 -- 172.21.15.140:0/1439399947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f4c0b0570 msgr2=0x7f3f4c13fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f4b7fe700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f4c0b0570 0x7f3f4c13fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f4b7fe700 1 -- 172.21.15.140:0/1439399947 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f44009470 con 0x7f3f4c0aea10 2022-07-21T01:50:42.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f42ffd700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f4c0b0570 0x7f3f4c13fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:42.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f4b7fe700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0aea10 0x7f3f4c112b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f3f3c00a000 tx=0x7f3f3c008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:42.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f497fa700 1 -- 172.21.15.140:0/1439399947 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f3c011780 con 0x7f3f4c0aea10 2022-07-21T01:50:42.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f497fa700 1 -- 172.21.15.140:0/1439399947 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c0118e0 con 0x7f3f4c0aea10 2022-07-21T01:50:42.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1439399947 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c1135b0 con 0x7f3f4c0aea10 2022-07-21T01:50:42.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f497fa700 1 -- 172.21.15.140:0/1439399947 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f3c011c40 con 0x7f3f4c0aea10 2022-07-21T01:50:42.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1439399947 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4c113090 con 0x7f3f4c0aea10 2022-07-21T01:50:42.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.247+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1439399947 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4c155f80 con 0x7f3f4c0aea10 2022-07-21T01:50:42.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.251+0000 7f3f497fa700 1 -- 172.21.15.140:0/1439399947 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3f3c012420 con 0x7f3f4c0aea10 2022-07-21T01:50:42.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.251+0000 7f3f497fa700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f300539e0 0x7f3f30055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.251+0000 7f3f497fa700 1 -- 172.21.15.140:0/1439399947 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3f3c06bd20 con 0x7f3f4c0aea10 2022-07-21T01:50:42.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.251+0000 7f3f42ffd700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f300539e0 0x7f3f30055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.251+0000 7f3f42ffd700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f300539e0 0x7f3f30055ea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f3f3400d000 tx=0x7f3f34001b30).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:42.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.259+0000 7f3f497fa700 1 -- 172.21.15.140:0/1439399947 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3f4c155f80 con 0x7f3f4c0aea10 2022-07-21T01:50:42.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1439399947 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f4c155f50 con 0x7f3f4c0aea10 2022-07-21T01:50:42.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f497fa700 1 -- 172.21.15.140:0/1439399947 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2141 (secure 0 0 0) 0x7f3f3c042ad0 con 0x7f3f4c0aea10 2022-07-21T01:50:42.427 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:42.427 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/2307860244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2307860244},{"type":"v1","addr":"172.21.15.152:6833","nonce":2307860244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:50:27.322541+0000","modified":"2022-07-21T01:50:28.325625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/207391195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":207391195},{"type":"v1","addr":"172.21.15.140:6835","nonce":207391195}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-07-21T01:50:42.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1439399947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f300539e0 msgr2=0x7f3f30055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f300539e0 0x7f3f30055ea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f3f3400d000 tx=0x7f3f34001b30).stop 2022-07-21T01:50:42.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1439399947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0aea10 msgr2=0x7f3f4c112b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0aea10 0x7f3f4c112b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f3f3c00a000 tx=0x7f3f3c008040).stop 2022-07-21T01:50:42.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1439399947 shutdown_connections 2022-07-21T01:50:42.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f300539e0 0x7f3f30055ea0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f4c1424e0 0x7f3f4c140370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f4c0b0570 0x7f3f4c13fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 --2- 172.21.15.140:0/1439399947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f4c0aea10 0x7f3f4c112b20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1439399947 >> 172.21.15.140:0/1439399947 conn(0x7f3f4c0b9360 msgr2=0x7f3f4c0ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:42.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1439399947 shutdown_connections 2022-07-21T01:50:42.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.423+0000 7f3f50ef3700 1 -- 172.21.15.140:0/1439399947 wait complete. 2022-07-21T01:50:42.432 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 16 2022-07-21T01:50:42.437 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2022-07-21T01:50:42.541 INFO:teuthology.orchestra.run.smithi140.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-07-21T01:50:42.542 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-07-21T01:50:42.543 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:50:42.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf16380700 1 Processor -- start 2022-07-21T01:50:42.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf16380700 1 -- start start 2022-07-21T01:50:42.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf16380700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcf100b5650 0x7fcf100b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf16380700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1005bd90 0x7fcf10142d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf16380700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf100aea30 con 0x7fcf100b1200 2022-07-21T01:50:42.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf16380700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf10142970 con 0x7fcf100b5650 2022-07-21T01:50:42.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf16380700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100aec60 con 0x7fcf1005bd90 2022-07-21T01:50:42.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf1537e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcf100b5650 0x7fcf100b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf15b7f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1005bd90 0x7fcf10142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf14b7d700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fcf100b1200 0x7fcf1005c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:49884/0 (socket says 172.21.15.140:49884) 2022-07-21T01:50:42.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf15b7f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1005bd90 0x7fcf10142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:59586/0 (socket says 172.21.15.140:59586) 2022-07-21T01:50:42.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf14b7d700 1 -- 172.21.15.140:0/389015075 learned_addr learned my addr 172.21.15.140:0/389015075 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:42.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.639+0000 7fcf1537e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcf100b5650 0x7fcf100b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:40364/0 (socket says 172.21.15.140:40364) 2022-07-21T01:50:42.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf15b7f700 1 -- 172.21.15.140:0/389015075 >> v2:172.21.15.152:3300/0 conn(0x7fcf100b5650 msgr2=0x7fcf100b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:42.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf15b7f700 1 --2- 172.21.15.140:0/389015075 >> v2:172.21.15.152:3300/0 conn(0x7fcf100b5650 0x7fcf100b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf15b7f700 1 -- 172.21.15.140:0/389015075 >> v1:172.21.15.152:6789/0 conn(0x7fcf100b1200 legacy=0x7fcf1005c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf15b7f700 1 -- 172.21.15.140:0/389015075 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf100af180 con 0x7fcf1005bd90 2022-07-21T01:50:42.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf1537e700 1 --2- 172.21.15.140:0/389015075 >> v2:172.21.15.152:3300/0 conn(0x7fcf100b5650 0x7fcf100b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:42.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf15b7f700 1 --2- 172.21.15.140:0/389015075 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1005bd90 0x7fcf10142d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fcf00002f70 tx=0x7fcf00006df0).ready entity=mon.2 client_cookie=3a7e29eb77ec5fdf server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:42.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf0ffff700 1 -- 172.21.15.140:0/389015075 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf0000d570 con 0x7fcf1005bd90 2022-07-21T01:50:42.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf0ffff700 1 -- 172.21.15.140:0/389015075 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf00000e90 con 0x7fcf1005bd90 2022-07-21T01:50:42.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf0ffff700 1 -- 172.21.15.140:0/389015075 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf0000d8c0 con 0x7fcf1005bd90 2022-07-21T01:50:42.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf16380700 1 -- 172.21.15.140:0/389015075 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1005bd90 msgr2=0x7fcf10142d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf16380700 1 --2- 172.21.15.140:0/389015075 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1005bd90 0x7fcf10142d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fcf00002f70 tx=0x7fcf00006df0).stop 2022-07-21T01:50:42.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf16380700 1 -- 172.21.15.140:0/389015075 shutdown_connections 2022-07-21T01:50:42.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf16380700 1 --2- 172.21.15.140:0/389015075 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1005bd90 0x7fcf10142d60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf16380700 1 --2- 172.21.15.140:0/389015075 >> v2:172.21.15.152:3300/0 conn(0x7fcf100b5650 0x7fcf100b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf16380700 1 -- 172.21.15.140:0/389015075 >> 172.21.15.140:0/389015075 conn(0x7fcf100634e0 msgr2=0x7fcf100b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:42.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf16380700 1 -- 172.21.15.140:0/389015075 shutdown_connections 2022-07-21T01:50:42.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf16380700 1 -- 172.21.15.140:0/389015075 wait complete. 2022-07-21T01:50:42.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf16380700 1 Processor -- start 2022-07-21T01:50:42.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.643+0000 7fcf16380700 1 -- start start 2022-07-21T01:50:42.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf16380700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1005bd90 0x7fcf10140880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf16380700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf100b1200 0x7fcf100658e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf16380700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf100b5650 0x7fcf10063990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf16380700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100651a0 con 0x7fcf1005bd90 2022-07-21T01:50:42.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf16380700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10065440 con 0x7fcf100b1200 2022-07-21T01:50:42.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf16380700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf10065dd0 con 0x7fcf100b5650 2022-07-21T01:50:42.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf1537e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1005bd90 0x7fcf10140880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf15b7f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf100b5650 0x7fcf10063990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf1537e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1005bd90 0x7fcf10140880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34628/0 (socket says 172.21.15.140:34628) 2022-07-21T01:50:42.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf1537e700 1 -- 172.21.15.140:0/1182121219 learned_addr learned my addr 172.21.15.140:0/1182121219 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:42.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf14b7d700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf100b1200 0x7fcf100658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf1537e700 1 -- 172.21.15.140:0/1182121219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf100b5650 msgr2=0x7fcf10063990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf1537e700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf100b5650 0x7fcf10063990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf1537e700 1 -- 172.21.15.140:0/1182121219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf100b1200 msgr2=0x7fcf100658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf1537e700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf100b1200 0x7fcf100658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf1537e700 1 -- 172.21.15.140:0/1182121219 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf08009470 con 0x7fcf1005bd90 2022-07-21T01:50:42.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf15b7f700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf100b5650 0x7fcf10063990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:42.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf1537e700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1005bd90 0x7fcf10140880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fcf0800c000 tx=0x7fcf080091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:42.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf0e7fc700 1 -- 172.21.15.140:0/1182121219 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf0800c850 con 0x7fcf1005bd90 2022-07-21T01:50:42.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf0e7fc700 1 -- 172.21.15.140:0/1182121219 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0800c9b0 con 0x7fcf1005bd90 2022-07-21T01:50:42.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf16380700 1 -- 172.21.15.140:0/1182121219 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0000b180 con 0x7fcf1005bd90 2022-07-21T01:50:42.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf0e7fc700 1 -- 172.21.15.140:0/1182121219 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf0800cd10 con 0x7fcf1005bd90 2022-07-21T01:50:42.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.647+0000 7fcf16380700 1 -- 172.21.15.140:0/1182121219 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf10063e80 con 0x7fcf1005bd90 2022-07-21T01:50:42.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.651+0000 7fcf16380700 1 -- 172.21.15.140:0/1182121219 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf10064160 con 0x7fcf1005bd90 2022-07-21T01:50:42.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.651+0000 7fcf0e7fc700 1 -- 172.21.15.140:0/1182121219 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcf08022070 con 0x7fcf1005bd90 2022-07-21T01:50:42.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.651+0000 7fcf0e7fc700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcef4053a70 0x7fcef4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:42.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.651+0000 7fcf0e7fc700 1 -- 172.21.15.140:0/1182121219 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcf0806c6d0 con 0x7fcf1005bd90 2022-07-21T01:50:42.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.651+0000 7fcf14b7d700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcef4053a70 0x7fcef4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:42.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.651+0000 7fcf14b7d700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcef4053a70 0x7fcef4055f30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fcef800d000 tx=0x7fcef8007ad0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:42.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.655+0000 7fcf0e7fc700 1 -- 172.21.15.140:0/1182121219 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcf080435f0 con 0x7fcf1005bd90 2022-07-21T01:50:42.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 -- 172.21.15.140:0/1182121219 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf100574e0 con 0x7fcf1005bd90 2022-07-21T01:50:42.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf0e7fc700 1 -- 172.21.15.140:0/1182121219 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2141 (secure 0 0 0) 0x7fcf08046c10 con 0x7fcf1005bd90 2022-07-21T01:50:42.829 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:42.833 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/2307860244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2307860244},{"type":"v1","addr":"172.21.15.152:6833","nonce":2307860244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:50:27.322541+0000","modified":"2022-07-21T01:50:28.325625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/207391195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":207391195},{"type":"v1","addr":"172.21.15.140:6835","nonce":207391195}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-07-21T01:50:42.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 -- 172.21.15.140:0/1182121219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcef4053a70 msgr2=0x7fcef4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcef4053a70 0x7fcef4055f30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fcef800d000 tx=0x7fcef8007ad0).stop 2022-07-21T01:50:42.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 -- 172.21.15.140:0/1182121219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1005bd90 msgr2=0x7fcf10140880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:42.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1005bd90 0x7fcf10140880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fcf0800c000 tx=0x7fcf080091d0).stop 2022-07-21T01:50:42.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 -- 172.21.15.140:0/1182121219 shutdown_connections 2022-07-21T01:50:42.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcef4053a70 0x7fcef4055f30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf100b5650 0x7fcf10063990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf100b1200 0x7fcf100658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 --2- 172.21.15.140:0/1182121219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1005bd90 0x7fcf10140880 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:42.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 -- 172.21.15.140:0/1182121219 >> 172.21.15.140:0/1182121219 conn(0x7fcf100634e0 msgr2=0x7fcf10142220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:42.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 -- 172.21.15.140:0/1182121219 shutdown_connections 2022-07-21T01:50:42.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:42.827+0000 7fcf16380700 1 -- 172.21.15.140:0/1182121219 wait complete. 2022-07-21T01:50:42.838 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 16 2022-07-21T01:50:42.841 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2022-07-21T01:50:42.956 INFO:teuthology.orchestra.run.smithi140.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-07-21T01:50:42.956 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-07-21T01:50:42.957 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:50:42.984 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:50:43.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b1b521700 1 Processor -- start 2022-07-21T01:50:43.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b1b521700 1 -- start start 2022-07-21T01:50:43.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b1b521700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0b140adff0 0x7f0b140b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b1b521700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0b140af8e0 0x7f0b1405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b1b521700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b14147010 con 0x7f0b140b5650 2022-07-21T01:50:43.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b1b521700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b141472b0 con 0x7f0b140af8e0 2022-07-21T01:50:43.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b1b521700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14146d40 con 0x7f0b140adff0 2022-07-21T01:50:43.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b1ad20700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0b140af8e0 0x7f0b1405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b1ad20700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0b140af8e0 0x7f0b1405bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34640/0 (socket says 172.21.15.140:34640) 2022-07-21T01:50:43.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b1ad20700 1 -- 172.21.15.140:0/1985893233 learned_addr learned my addr 172.21.15.140:0/1985893233 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:43.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b1a51f700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0b140b5650 0x7f0b14147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48784/0 (socket says 172.21.15.140:48784) 2022-07-21T01:50:43.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.087+0000 7f0b19d1e700 1 --2- 172.21.15.140:0/1985893233 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0b140adff0 0x7f0b140b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1ad20700 1 -- 172.21.15.140:0/1985893233 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0b140adff0 msgr2=0x7f0b140b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1ad20700 1 --2- 172.21.15.140:0/1985893233 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0b140adff0 0x7f0b140b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1ad20700 1 -- 172.21.15.140:0/1985893233 >> v1:172.21.15.140:6789/0 conn(0x7f0b140b5650 legacy=0x7f0b14147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1951d700 1 -- 172.21.15.140:0/1985893233 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594926073 0 0) 0x7f0b14147010 con 0x7f0b140b5650 2022-07-21T01:50:43.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1ad20700 1 -- 172.21.15.140:0/1985893233 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140acae0 con 0x7f0b140af8e0 2022-07-21T01:50:43.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1ad20700 1 --2- 172.21.15.140:0/1985893233 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0b140af8e0 0x7f0b1405bd90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f0b0c002f70 tx=0x7f0b0c006eb0).ready entity=mon.0 client_cookie=b0c1a37e2ea892c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:43.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1951d700 1 -- 172.21.15.140:0/1985893233 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b0c00d570 con 0x7f0b140af8e0 2022-07-21T01:50:43.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1951d700 1 -- 172.21.15.140:0/1985893233 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c000e90 con 0x7f0b140af8e0 2022-07-21T01:50:43.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1951d700 1 -- 172.21.15.140:0/1985893233 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b0c00d8c0 con 0x7f0b140af8e0 2022-07-21T01:50:43.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 -- 172.21.15.140:0/1985893233 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0b140af8e0 msgr2=0x7f0b1405bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 --2- 172.21.15.140:0/1985893233 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0b140af8e0 0x7f0b1405bd90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f0b0c002f70 tx=0x7f0b0c006eb0).stop 2022-07-21T01:50:43.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 -- 172.21.15.140:0/1985893233 shutdown_connections 2022-07-21T01:50:43.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 --2- 172.21.15.140:0/1985893233 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0b140af8e0 0x7f0b1405bd90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 --2- 172.21.15.140:0/1985893233 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0b140adff0 0x7f0b140b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 -- 172.21.15.140:0/1985893233 >> 172.21.15.140:0/1985893233 conn(0x7f0b140634e0 msgr2=0x7f0b140b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:43.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 -- 172.21.15.140:0/1985893233 shutdown_connections 2022-07-21T01:50:43.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 -- 172.21.15.140:0/1985893233 wait complete. 2022-07-21T01:50:43.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 Processor -- start 2022-07-21T01:50:43.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 -- start start 2022-07-21T01:50:43.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0b140adff0 0x7f0b14159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0b140af8e0 0x7f0b1415a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b140b5650 0x7f0b14156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b141115e0 con 0x7f0b140adff0 2022-07-21T01:50:43.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b141459d0 con 0x7f0b140b5650 2022-07-21T01:50:43.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1b521700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14111810 con 0x7f0b140af8e0 2022-07-21T01:50:43.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1a51f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0b140adff0 0x7f0b14159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1a51f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0b140adff0 0x7f0b14159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34642/0 (socket says 172.21.15.140:34642) 2022-07-21T01:50:43.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1a51f700 1 -- 172.21.15.140:0/1405937254 learned_addr learned my addr 172.21.15.140:0/1405937254 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:43.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b19d1e700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0b140af8e0 0x7f0b1415a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1ad20700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b140b5650 0x7f0b14156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1a51f700 1 -- 172.21.15.140:0/1405937254 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0b140af8e0 msgr2=0x7f0b1415a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1a51f700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0b140af8e0 0x7f0b1415a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1a51f700 1 -- 172.21.15.140:0/1405937254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b140b5650 msgr2=0x7f0b14156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1a51f700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b140b5650 0x7f0b14156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1a51f700 1 -- 172.21.15.140:0/1405937254 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c00b1e0 con 0x7f0b140adff0 2022-07-21T01:50:43.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b1ad20700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b140b5650 0x7f0b14156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:43.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.091+0000 7f0b19d1e700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0b140af8e0 0x7f0b1415a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:43.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b1a51f700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0b140adff0 0x7f0b14159b60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f0b04000ed0 tx=0x7f0b04007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:43.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b0b7fe700 1 -- 172.21.15.140:0/1405937254 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b040070d0 con 0x7f0b140adff0 2022-07-21T01:50:43.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b0b7fe700 1 -- 172.21.15.140:0/1405937254 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b04007230 con 0x7f0b140adff0 2022-07-21T01:50:43.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b1b521700 1 -- 172.21.15.140:0/1405937254 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b14066f70 con 0x7f0b140adff0 2022-07-21T01:50:43.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b0b7fe700 1 -- 172.21.15.140:0/1405937254 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b04005550 con 0x7f0b140adff0 2022-07-21T01:50:43.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b1b521700 1 -- 172.21.15.140:0/1405937254 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b141113a0 con 0x7f0b140adff0 2022-07-21T01:50:43.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b1b521700 1 -- 172.21.15.140:0/1405937254 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b140574e0 con 0x7f0b140adff0 2022-07-21T01:50:43.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b0b7fe700 1 -- 172.21.15.140:0/1405937254 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0b0401a060 con 0x7f0b140adff0 2022-07-21T01:50:43.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b0b7fe700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0af4053aa0 0x7f0af4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b0b7fe700 1 -- 172.21.15.140:0/1405937254 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0b0406b2e0 con 0x7f0b140adff0 2022-07-21T01:50:43.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b19d1e700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0af4053aa0 0x7f0af4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.095+0000 7f0b19d1e700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0af4053aa0 0x7f0af4055f60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f0b1000b000 tx=0x7f0b10008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:43.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.103+0000 7f0b0b7fe700 1 -- 172.21.15.140:0/1405937254 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b04042200 con 0x7f0b140adff0 2022-07-21T01:50:43.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.263+0000 7f0b1b521700 1 -- 172.21.15.140:0/1405937254 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b14067ea0 con 0x7f0b140adff0 2022-07-21T01:50:43.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.263+0000 7f0b0b7fe700 1 -- 172.21.15.140:0/1405937254 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2141 (secure 0 0 0) 0x7f0b04018020 con 0x7f0b140adff0 2022-07-21T01:50:43.269 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:43.269 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/2307860244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2307860244},{"type":"v1","addr":"172.21.15.152:6833","nonce":2307860244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:50:27.322541+0000","modified":"2022-07-21T01:50:28.325625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/207391195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":207391195},{"type":"v1","addr":"172.21.15.140:6835","nonce":207391195}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-07-21T01:50:43.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.263+0000 7f0b1b521700 1 -- 172.21.15.140:0/1405937254 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0af4053aa0 msgr2=0x7f0af4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.263+0000 7f0b1b521700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0af4053aa0 0x7f0af4055f60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f0b1000b000 tx=0x7f0b10008040).stop 2022-07-21T01:50:43.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.263+0000 7f0b1b521700 1 -- 172.21.15.140:0/1405937254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0b140adff0 msgr2=0x7f0b14159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.263+0000 7f0b1b521700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0b140adff0 0x7f0b14159b60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f0b04000ed0 tx=0x7f0b04007c80).stop 2022-07-21T01:50:43.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.267+0000 7f0b1b521700 1 -- 172.21.15.140:0/1405937254 shutdown_connections 2022-07-21T01:50:43.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.267+0000 7f0b1b521700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0af4053aa0 0x7f0af4055f60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.267+0000 7f0b1b521700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b140b5650 0x7f0b14156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.267+0000 7f0b1b521700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0b140af8e0 0x7f0b1415a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.267+0000 7f0b1b521700 1 --2- 172.21.15.140:0/1405937254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0b140adff0 0x7f0b14159b60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.267+0000 7f0b1b521700 1 -- 172.21.15.140:0/1405937254 >> 172.21.15.140:0/1405937254 conn(0x7f0b140634e0 msgr2=0x7f0b141460c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:43.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.267+0000 7f0b1b521700 1 -- 172.21.15.140:0/1405937254 shutdown_connections 2022-07-21T01:50:43.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.267+0000 7f0b1b521700 1 -- 172.21.15.140:0/1405937254 wait complete. 2022-07-21T01:50:43.280 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 16 2022-07-21T01:50:43.283 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a cache drop 2022-07-21T01:50:43.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f68893f2700 1 Processor -- start 2022-07-21T01:50:43.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f68893f2700 1 -- start start 2022-07-21T01:50:43.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f68893f2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c048f90 0x7f687c0d51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f68893f2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c049370 con 0x7f687c0d1890 2022-07-21T01:50:43.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f68893f2700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c03bcb0 con 0x7f687c043b40 2022-07-21T01:50:43.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f68893f2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c03ca80 con 0x7f687c048f90 2022-07-21T01:50:43.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6888bf1700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f687c0d1890 0x7f687c03d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48796/0 (socket says 172.21.15.140:48796) 2022-07-21T01:50:43.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6888bf1700 1 -- 172.21.15.140:0/438503823 learned_addr learned my addr 172.21.15.140:0/438503823 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:43.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6883fff700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c048f90 0x7f687c0d51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846699473 0 0) 0x7f687c049370 con 0x7f687c0d1890 2022-07-21T01:50:43.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6868003260 con 0x7f687c0d1890 2022-07-21T01:50:43.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2742046499 0 0) 0x7f6868003260 con 0x7f687c0d1890 2022-07-21T01:50:43.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c048f90 msgr2=0x7f687c0d51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c048f90 0x7f687c0d51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 >> v1:172.21.15.152:6789/0 conn(0x7f687c043b40 legacy=0x7f687c0d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c03bf50 con 0x7f687c0d1890 2022-07-21T01:50:43.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f688405ad40 con 0x7f687c0d1890 2022-07-21T01:50:43.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 >> v1:172.21.15.140:6789/0 conn(0x7f687c0d1890 legacy=0x7f687c03d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c048f90 0x7f6868003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6868003c70 0x7f6868004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68680049f0 0x7f6868004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c03ca80 con 0x7f6868003c70 2022-07-21T01:50:43.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686800b750 con 0x7f68680049f0 2022-07-21T01:50:43.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686800b8c0 con 0x7f687c048f90 2022-07-21T01:50:43.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f68837fe700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6868003c70 0x7f6868004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6888bf1700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68680049f0 0x7f6868004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.387+0000 7f6883fff700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c048f90 0x7f6868003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68837fe700 1 -- 172.21.15.140:0/438503823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c048f90 msgr2=0x7f6868003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68837fe700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c048f90 0x7f6868003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68837fe700 1 -- 172.21.15.140:0/438503823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68680049f0 msgr2=0x7f6868004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68837fe700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68680049f0 0x7f6868004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68837fe700 1 -- 172.21.15.140:0/438503823 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6868003260 con 0x7f6868003c70 2022-07-21T01:50:43.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68837fe700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6868003c70 0x7f6868004500 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f6878001020 tx=0x7f68780079e0).ready entity=mon.0 client_cookie=a4692b0bd77009a8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:43.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6878007cf0 con 0x7f6868003c70 2022-07-21T01:50:43.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878007030 con 0x7f6868003c70 2022-07-21T01:50:43.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f6882ffd700 1 -- 172.21.15.140:0/438503823 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6878007390 con 0x7f6868003c70 2022-07-21T01:50:43.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68893f2700 1 -- 172.21.15.140:0/438503823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6868003c70 msgr2=0x7f6868004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68893f2700 1 --2- 172.21.15.140:0/438503823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6868003c70 0x7f6868004500 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f6878001020 tx=0x7f68780079e0).stop 2022-07-21T01:50:43.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f6888bf1700 1 -- 172.21.15.140:0/438503823 reap_dead start 2022-07-21T01:50:43.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68893f2700 1 -- 172.21.15.140:0/438503823 shutdown_connections 2022-07-21T01:50:43.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68893f2700 1 -- 172.21.15.140:0/438503823 >> 172.21.15.140:0/438503823 conn(0x7f687c00f350 msgr2=0x7f687c048540 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:43.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68893f2700 1 -- 172.21.15.140:0/438503823 shutdown_connections 2022-07-21T01:50:43.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68893f2700 1 -- 172.21.15.140:0/438503823 wait complete. 2022-07-21T01:50:43.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68893f2700 1 Processor -- start 2022-07-21T01:50:43.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.391+0000 7f68893f2700 1 -- start start 2022-07-21T01:50:43.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68893f2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6868003c70 0x7f687c046810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68893f2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f687c0e1cc0 0x7f687c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68893f2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c0e0d10 0x7f687c0d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68893f2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0d6420 con 0x7f687c0e1cc0 2022-07-21T01:50:43.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68893f2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0d66c0 con 0x7f6868003c70 2022-07-21T01:50:43.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68893f2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0cf380 con 0x7f687c0e0d10 2022-07-21T01:50:43.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f687c0e1cc0 0x7f687c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f687c0e1cc0 0x7f687c0ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34664/0 (socket says 172.21.15.140:34664) 2022-07-21T01:50:43.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68837fe700 1 -- 172.21.15.140:0/4092278342 learned_addr learned my addr 172.21.15.140:0/4092278342 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:43.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f6883fff700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6868003c70 0x7f687c046810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f6888bf1700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c0e0d10 0x7f687c0d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68837fe700 1 -- 172.21.15.140:0/4092278342 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c0e0d10 msgr2=0x7f687c0d6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68837fe700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c0e0d10 0x7f687c0d6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68837fe700 1 -- 172.21.15.140:0/4092278342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6868003c70 msgr2=0x7f687c046810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68837fe700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6868003c70 0x7f687c046810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68837fe700 1 -- 172.21.15.140:0/4092278342 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c0cf540 con 0x7f687c0e1cc0 2022-07-21T01:50:43.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f6888bf1700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c0e0d10 0x7f687c0d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:43.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68837fe700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f687c0e1cc0 0x7f687c0ce990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f6878001020 tx=0x7f6878016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:43.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68817fa700 1 -- 172.21.15.140:0/4092278342 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6878007cf0 con 0x7f687c0e1cc0 2022-07-21T01:50:43.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68817fa700 1 -- 172.21.15.140:0/4092278342 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878013810 con 0x7f687c0e1cc0 2022-07-21T01:50:43.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68893f2700 1 -- 172.21.15.140:0/4092278342 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c0d7050 con 0x7f687c0e1cc0 2022-07-21T01:50:43.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68817fa700 1 -- 172.21.15.140:0/4092278342 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6878013b70 con 0x7f687c0e1cc0 2022-07-21T01:50:43.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.395+0000 7f68893f2700 1 -- 172.21.15.140:0/4092278342 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c0d7550 con 0x7f687c0e1cc0 2022-07-21T01:50:43.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.399+0000 7f68817fa700 1 -- 172.21.15.140:0/4092278342 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6878010030 con 0x7f687c0e1cc0 2022-07-21T01:50:43.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.399+0000 7f68817fa700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6864053a70 0x7f6864055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.399+0000 7f68817fa700 1 -- 172.21.15.140:0/4092278342 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f687806d060 con 0x7f687c0e1cc0 2022-07-21T01:50:43.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.399+0000 7f6883fff700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6864053a70 0x7f6864055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.399+0000 7f6883fff700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6864053a70 0x7f6864055f30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f687000b000 tx=0x7f6870006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:43.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.399+0000 7f68948ab740 1 Processor -- start 2022-07-21T01:50:43.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.399+0000 7f68948ab740 1 -- start start 2022-07-21T01:50:43.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1ddbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1ddc180 0x1ddc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1da4f80 0x1da5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d19010 con 0x1de1c10 2022-07-21T01:50:43.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cf9250 con 0x1da4f80 2022-07-21T01:50:43.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x1da7400 con 0x1ddc180 2022-07-21T01:50:43.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f6883fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1ddbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f6883fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1ddbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34670/0 (socket says 172.21.15.140:34670) 2022-07-21T01:50:43.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f6883fff700 1 -- 172.21.15.140:0/243372719 learned_addr learned my addr 172.21.15.140:0/243372719 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:43.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f6888bf1700 1 --2- 172.21.15.140:0/243372719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1da4f80 0x1da5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68837fe700 1 --2- 172.21.15.140:0/243372719 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1ddc180 0x1ddc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f6883fff700 1 -- 172.21.15.140:0/243372719 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1ddc180 msgr2=0x1ddc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f6883fff700 1 --2- 172.21.15.140:0/243372719 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1ddc180 0x1ddc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f6883fff700 1 -- 172.21.15.140:0/243372719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1da4f80 msgr2=0x1da5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f6883fff700 1 --2- 172.21.15.140:0/243372719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1da4f80 0x1da5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f6883fff700 1 -- 172.21.15.140:0/243372719 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d51030 con 0x1de1c10 2022-07-21T01:50:43.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f6883fff700 1 --2- 172.21.15.140:0/243372719 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1ddbc90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x1cfbab0 tx=0x7f6870019000).ready entity=mon.0 client_cookie=c2b8f0f96cc86b33 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:43.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68893f2700 1 -- 172.21.15.140:0/243372719 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6870014830 con 0x1de1c10 2022-07-21T01:50:43.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68893f2700 1 -- 172.21.15.140:0/243372719 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6870015ac0 con 0x1de1c10 2022-07-21T01:50:43.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68893f2700 1 -- 172.21.15.140:0/243372719 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6870015c30 con 0x1de1c10 2022-07-21T01:50:43.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 -- 172.21.15.140:0/243372719 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 msgr2=0x1ddbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 --2- 172.21.15.140:0/243372719 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1ddbc90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x1cfbab0 tx=0x7f6870019000).stop 2022-07-21T01:50:43.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 -- 172.21.15.140:0/243372719 shutdown_connections 2022-07-21T01:50:43.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 --2- 172.21.15.140:0/243372719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1da4f80 0x1da5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 --2- 172.21.15.140:0/243372719 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1ddc180 0x1ddc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 --2- 172.21.15.140:0/243372719 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1ddbc90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 -- 172.21.15.140:0/243372719 >> 172.21.15.140:0/243372719 conn(0x1da62e0 msgr2=0x1de11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:43.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 -- 172.21.15.140:0/243372719 shutdown_connections 2022-07-21T01:50:43.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 -- 172.21.15.140:0/243372719 wait complete. 2022-07-21T01:50:43.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 -- start start 2022-07-21T01:50:43.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.403+0000 7f68948ab740 1 Processor -- start 2022-07-21T01:50:43.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1da4f80 0x1da8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1ddc180 0x1daa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1dd5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x1decfa0 con 0x1de1c10 2022-07-21T01:50:43.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ded240 con 0x1ddc180 2022-07-21T01:50:43.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x1dda370 con 0x1da4f80 2022-07-21T01:50:43.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 10 client.0 ll_register_callbacks cb 0x1d60610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-07-21T01:50:43.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6888bf1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1dd5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6888bf1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1dd5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34680/0 (socket says 172.21.15.140:34680) 2022-07-21T01:50:43.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6888bf1700 1 -- 172.21.15.140:0/3360417712 learned_addr learned my addr 172.21.15.140:0/3360417712 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:43.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68837fe700 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1ddc180 0x1daa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6883fff700 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1da4f80 0x1da8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6888bf1700 1 -- 172.21.15.140:0/3360417712 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1da4f80 msgr2=0x1da8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6888bf1700 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1da4f80 0x1da8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6888bf1700 1 -- 172.21.15.140:0/3360417712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1ddc180 msgr2=0x1daa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6888bf1700 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1ddc180 0x1daa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6888bf1700 1 -- 172.21.15.140:0/3360417712 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6870014990 con 0x1de1c10 2022-07-21T01:50:43.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6888bf1700 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1dd5aa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f688405da80 tx=0x7f6884063040).ready entity=mon.0 client_cookie=1bc5feced89ff2fa server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:43.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6862ffd700 10 client.0 ms_handle_connect on v2:172.21.15.140:3300/0 2022-07-21T01:50:43.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6862ffd700 1 -- 172.21.15.140:0/3360417712 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f688405dd00 con 0x1de1c10 2022-07-21T01:50:43.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 1 -- 172.21.15.140:0/3360417712 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d0efc0 con 0x1de1c10 2022-07-21T01:50:43.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6862ffd700 1 -- 172.21.15.140:0/3360417712 <== mon.0 v2:172.21.15.140:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6884080ba0 con 0x1de1c10 2022-07-21T01:50:43.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6862ffd700 1 -- 172.21.15.140:0/3360417712 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6884080da0 con 0x1de1c10 2022-07-21T01:50:43.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 10 client.5457 fetch_fsmap learned FSMap version 16 2022-07-21T01:50:43.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 1 -- 172.21.15.140:0/3360417712 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x1ceb9b0 con 0x1de1c10 2022-07-21T01:50:43.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6862ffd700 1 -- 172.21.15.140:0/3360417712 <== mon.0 v2:172.21.15.140:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d0efc0 con 0x1de1c10 2022-07-21T01:50:43.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6862ffd700 1 -- 172.21.15.140:0/3360417712 <== mon.0 v2:172.21.15.140:3300/0 5 ==== fsmap(e 16) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f688405f0c0 con 0x1de1c10 2022-07-21T01:50:43.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 10 client.5457 fetch_fsmap finished waiting for FSMap version 16 2022-07-21T01:50:43.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 10 client.5457 resolve_mds: resolved ID 'a' to GID 5247 2022-07-21T01:50:43.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 20 client.5457 populate_metadata read hostname 'smithi140' 2022-07-21T01:50:43.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.140:6834/207391195,v1:172.21.15.140:6835/207391195] conn(0x1dd82e0 0x1dd5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 4 client.5457 mds_command: new command op to 5247 tid=0[{"prefix": "get_command_descriptions"}] 2022-07-21T01:50:43.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f68948ab740 1 -- 172.21.15.140:0/3360417712 --> [v2:172.21.15.140:6834/207391195,v1:172.21.15.140:6835/207391195] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1d07040 con 0x1dd82e0 2022-07-21T01:50:43.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6883fff700 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.140:6834/207391195,v1:172.21.15.140:6835/207391195] conn(0x1dd82e0 0x1dd5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6862ffd700 10 client.5457 ms_handle_connect on v2:172.21.15.140:6834/207391195 2022-07-21T01:50:43.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.407+0000 7f6883fff700 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.140:6834/207391195,v1:172.21.15.140:6835/207391195] conn(0x1dd82e0 0x1dd5f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=764b74a2ce80e1b4 server_cookie=fe9dd4245d34eb3 in_seq=0 out_seq=0 2022-07-21T01:50:43.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f6862ffd700 1 -- 172.21.15.140:0/3360417712 <== mds.0 v2:172.21.15.140:6834/207391195 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1d07040 con 0x1dd82e0 2022-07-21T01:50:43.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f6862ffd700 10 client.5457 handle_command_reply: tid=0 2022-07-21T01:50:43.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 1 client.5457 shutdown 2022-07-21T01:50:43.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 1 -- 172.21.15.140:0/3360417712 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 msgr2=0x1dd5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1dd5aa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f688405da80 tx=0x7f6884063040).stop 2022-07-21T01:50:43.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 1 -- 172.21.15.140:0/3360417712 shutdown_connections 2022-07-21T01:50:43.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.140:6834/207391195,v1:172.21.15.140:6835/207391195] conn(0x1dd82e0 0x1dd5f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1de1c10 0x1dd5aa0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f6862ffd700 0 client.5457 ms_handle_reset on v2:172.21.15.140:6834/207391195 2022-07-21T01:50:43.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1ddc180 0x1daa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 1 --2- 172.21.15.140:0/3360417712 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1da4f80 0x1da8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 1 -- 172.21.15.140:0/3360417712 >> 172.21.15.140:0/3360417712 conn(0x1da62e0 msgr2=0x1de10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:43.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 1 -- 172.21.15.140:0/3360417712 shutdown_connections 2022-07-21T01:50:43.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 1 -- 172.21.15.140:0/3360417712 wait complete. 2022-07-21T01:50:43.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.411+0000 7f68948ab740 20 client.5457 trim_cache size 0 max 16384 2022-07-21T01:50:43.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.423+0000 7f68948ab740 1 Processor -- start 2022-07-21T01:50:43.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.423+0000 7f68948ab740 1 -- start start 2022-07-21T01:50:43.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1da4f80 0x1dd5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1da80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1de1c10 0x1ddb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x1dd7b30 con 0x1ddc180 2022-07-21T01:50:43.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ddef50 con 0x1da4f80 2022-07-21T01:50:43.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d45b70 con 0x1de1c10 2022-07-21T01:50:43.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1da80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1da80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34688/0 (socket says 172.21.15.140:34688) 2022-07-21T01:50:43.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68837fe700 1 -- 172.21.15.140:0/4167865907 learned_addr learned my addr 172.21.15.140:0/4167865907 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:43.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f6888bf1700 1 --2- 172.21.15.140:0/4167865907 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1de1c10 0x1ddb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f6883fff700 1 --2- 172.21.15.140:0/4167865907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1da4f80 0x1dd5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68837fe700 1 -- 172.21.15.140:0/4167865907 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1de1c10 msgr2=0x1ddb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68837fe700 1 --2- 172.21.15.140:0/4167865907 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1de1c10 0x1ddb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68837fe700 1 -- 172.21.15.140:0/4167865907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1da4f80 msgr2=0x1dd5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68837fe700 1 --2- 172.21.15.140:0/4167865907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1da4f80 0x1dd5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68837fe700 1 -- 172.21.15.140:0/4167865907 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1d085b0 con 0x1ddc180 2022-07-21T01:50:43.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f6883fff700 1 --2- 172.21.15.140:0/4167865907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1da4f80 0x1dd5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:43.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68837fe700 1 --2- 172.21.15.140:0/4167865907 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1da80d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x1da6c70 tx=0x7f6878001440).ready entity=mon.0 client_cookie=926b69c1ca2d39d0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:43.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f686d7fa700 1 -- 172.21.15.140:0/4167865907 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6878006e80 con 0x1ddc180 2022-07-21T01:50:43.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f686d7fa700 1 -- 172.21.15.140:0/4167865907 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878030040 con 0x1ddc180 2022-07-21T01:50:43.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f686d7fa700 1 -- 172.21.15.140:0/4167865907 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6878047ad0 con 0x1ddc180 2022-07-21T01:50:43.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 -- 172.21.15.140:0/4167865907 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 msgr2=0x1da80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:43.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 --2- 172.21.15.140:0/4167865907 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1da80d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x1da6c70 tx=0x7f6878001440).stop 2022-07-21T01:50:43.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 -- 172.21.15.140:0/4167865907 shutdown_connections 2022-07-21T01:50:43.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 --2- 172.21.15.140:0/4167865907 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1de1c10 0x1ddb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 --2- 172.21.15.140:0/4167865907 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1da80d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 --2- 172.21.15.140:0/4167865907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1da4f80 0x1dd5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 -- 172.21.15.140:0/4167865907 >> 172.21.15.140:0/4167865907 conn(0x1da62e0 msgr2=0x1ddde80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:43.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 -- 172.21.15.140:0/4167865907 shutdown_connections 2022-07-21T01:50:43.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.427+0000 7f68948ab740 1 -- 172.21.15.140:0/4167865907 wait complete. 2022-07-21T01:50:43.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68948ab740 1 -- start start 2022-07-21T01:50:43.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68948ab740 1 Processor -- start 2022-07-21T01:50:43.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68948ab740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1da4f80 0x1e48800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68948ab740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1e48cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68948ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1de1c10 0x1e4b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68948ab740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x1da5880 con 0x1ddc180 2022-07-21T01:50:43.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68948ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x1da5b20 con 0x1de1c10 2022-07-21T01:50:43.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68948ab740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e491e0 con 0x1da4f80 2022-07-21T01:50:43.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68948ab740 10 client.0 ll_register_callbacks cb 0x1daa000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-07-21T01:50:43.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1e48cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1e48cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34692/0 (socket says 172.21.15.140:34692) 2022-07-21T01:50:43.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68837fe700 1 -- 172.21.15.140:0/408772571 learned_addr learned my addr 172.21.15.140:0/408772571 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:43.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f6883fff700 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1da4f80 0x1e48800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f6888bf1700 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1de1c10 0x1e4b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68837fe700 1 -- 172.21.15.140:0/408772571 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1da4f80 msgr2=0x1e48800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68837fe700 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1da4f80 0x1e48800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68837fe700 1 -- 172.21.15.140:0/408772571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1de1c10 msgr2=0x1e4b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:43.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68837fe700 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1de1c10 0x1e4b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:43.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68837fe700 1 -- 172.21.15.140:0/408772571 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c0d7550 con 0x1ddc180 2022-07-21T01:50:43.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f6888bf1700 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1de1c10 0x1e4b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:43.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68837fe700 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1e48cf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x1da6c70 tx=0x7f6878006b30).ready entity=mon.0 client_cookie=90ad27b76d62c17d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:43.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f6863fff700 10 client.0 ms_handle_connect on v2:172.21.15.140:3300/0 2022-07-21T01:50:43.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f6863fff700 1 -- 172.21.15.140:0/408772571 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6878027560 con 0x1ddc180 2022-07-21T01:50:43.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f6863fff700 1 -- 172.21.15.140:0/408772571 <== mon.0 v2:172.21.15.140:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6878047020 con 0x1ddc180 2022-07-21T01:50:43.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.431+0000 7f68948ab740 1 -- 172.21.15.140:0/408772571 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6870015ac0 con 0x1ddc180 2022-07-21T01:50:43.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f6863fff700 1 -- 172.21.15.140:0/408772571 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68780471e0 con 0x1ddc180 2022-07-21T01:50:43.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f68948ab740 10 client.5463 fetch_fsmap learned FSMap version 16 2022-07-21T01:50:43.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f68948ab740 1 -- 172.21.15.140:0/408772571 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x1d0db90 con 0x1ddc180 2022-07-21T01:50:43.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f6863fff700 1 -- 172.21.15.140:0/408772571 <== mon.0 v2:172.21.15.140:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878043020 con 0x1ddc180 2022-07-21T01:50:43.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f6863fff700 1 -- 172.21.15.140:0/408772571 <== mon.0 v2:172.21.15.140:3300/0 5 ==== fsmap(e 16) v1 ==== 1247+0+0 (secure 0 0 0) 0x7f687803d060 con 0x1ddc180 2022-07-21T01:50:43.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f68948ab740 10 client.5463 fetch_fsmap finished waiting for FSMap version 16 2022-07-21T01:50:43.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f68948ab740 10 client.5463 resolve_mds: resolved ID 'a' to GID 5247 2022-07-21T01:50:43.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f68948ab740 20 client.5463 populate_metadata read hostname 'smithi140' 2022-07-21T01:50:43.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f68948ab740 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.140:6834/207391195,v1:172.21.15.140:6835/207391195] conn(0x1dd7220 0x1e4bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:43.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f68948ab740 4 client.5463 mds_command: new command op to 5247 tid=0[{"prefix": "cache drop"}] 2022-07-21T01:50:43.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f68948ab740 1 -- 172.21.15.140:0/408772571 --> [v2:172.21.15.140:6834/207391195,v1:172.21.15.140:6835/207391195] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x1bfc590 con 0x1dd7220 2022-07-21T01:50:43.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f6883fff700 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.140:6834/207391195,v1:172.21.15.140:6835/207391195] conn(0x1dd7220 0x1e4bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:43.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f6863fff700 10 client.5463 ms_handle_connect on v2:172.21.15.140:6834/207391195 2022-07-21T01:50:43.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:43.435+0000 7f6883fff700 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.140:6834/207391195,v1:172.21.15.140:6835/207391195] conn(0x1dd7220 0x1e4bc60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1c0fe78a40152bf9 server_cookie=b4d0cd8f9d8ea0a4 in_seq=0 out_seq=0 2022-07-21T01:50:54.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f6863fff700 1 -- 172.21.15.140:0/408772571 <== mds.0 v2:172.21.15.140:6834/207391195 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0x1bfc590 con 0x1dd7220 2022-07-21T01:50:54.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f6863fff700 10 client.5463 handle_command_reply: tid=0 2022-07-21T01:50:54.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 1 client.5463 shutdown 2022-07-21T01:50:54.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 1 -- 172.21.15.140:0/408772571 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 msgr2=0x1e48cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:54.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1e48cf0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x1da6c70 tx=0x7f6878006b30).stop 2022-07-21T01:50:54.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 1 -- 172.21.15.140:0/408772571 shutdown_connections 2022-07-21T01:50:54.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.140:6834/207391195,v1:172.21.15.140:6835/207391195] conn(0x1dd7220 0x1e4bc60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:54.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1de1c10 0x1e4b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:54.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1ddc180 0x1e48cf0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:54.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 1 --2- 172.21.15.140:0/408772571 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1da4f80 0x1e48800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:54.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f6863fff700 0 client.5463 ms_handle_reset on v2:172.21.15.140:6834/207391195 2022-07-21T01:50:54.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 1 -- 172.21.15.140:0/408772571 >> 172.21.15.140:0/408772571 conn(0x1da62e0 msgr2=0x1dddc70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:54.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 1 -- 172.21.15.140:0/408772571 shutdown_connections 2022-07-21T01:50:54.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 1 -- 172.21.15.140:0/408772571 wait complete. 2022-07-21T01:50:54.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68948ab740 20 client.5463 trim_cache size 0 max 16384 2022-07-21T01:50:54.628 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:50:54.628 INFO:teuthology.orchestra.run.smithi140.stdout: "client_recall": { 2022-07-21T01:50:54.629 INFO:teuthology.orchestra.run.smithi140.stdout: "return_code": 0, 2022-07-21T01:50:54.629 INFO:teuthology.orchestra.run.smithi140.stdout: "message": "(0) Success", 2022-07-21T01:50:54.630 INFO:teuthology.orchestra.run.smithi140.stdout: "recalled": 220 2022-07-21T01:50:54.630 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:54.631 INFO:teuthology.orchestra.run.smithi140.stdout: "flush_journal": { 2022-07-21T01:50:54.631 INFO:teuthology.orchestra.run.smithi140.stdout: "return_code": 0, 2022-07-21T01:50:54.631 INFO:teuthology.orchestra.run.smithi140.stdout: "message": "" 2022-07-21T01:50:54.632 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:54.632 INFO:teuthology.orchestra.run.smithi140.stdout: "trim_cache": { 2022-07-21T01:50:54.633 INFO:teuthology.orchestra.run.smithi140.stdout: "trimmed": 0 2022-07-21T01:50:54.633 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:54.639 INFO:teuthology.orchestra.run.smithi140.stdout: "cache": { 2022-07-21T01:50:54.639 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": { 2022-07-21T01:50:54.640 INFO:teuthology.orchestra.run.smithi140.stdout: "items": 20056, 2022-07-21T01:50:54.640 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes": 3103904 2022-07-21T01:50:54.640 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:50:54.641 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:50:54.641 INFO:teuthology.orchestra.run.smithi140.stdout: "duration": 11.184295497999999 2022-07-21T01:50:54.641 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:50:54.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 -- 172.21.15.140:0/4092278342 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6864053a70 msgr2=0x7f6864055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:54.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6864053a70 0x7f6864055f30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f687000b000 tx=0x7f6870006df0).stop 2022-07-21T01:50:54.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 -- 172.21.15.140:0/4092278342 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f687c0e1cc0 msgr2=0x7f687c0ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:54.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f687c0e1cc0 0x7f687c0ce990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f6878001020 tx=0x7f6878016000).stop 2022-07-21T01:50:54.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 -- 172.21.15.140:0/4092278342 shutdown_connections 2022-07-21T01:50:54.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6864053a70 0x7f6864055f30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:54.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f687c0e0d10 0x7f687c0d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:54.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f687c0e1cc0 0x7f687c0ce990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:54.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 --2- 172.21.15.140:0/4092278342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6868003c70 0x7f687c046810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:54.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 -- 172.21.15.140:0/4092278342 >> 172.21.15.140:0/4092278342 conn(0x7f687c00f350 msgr2=0x7f687c045070 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:54.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 -- 172.21.15.140:0/4092278342 shutdown_connections 2022-07-21T01:50:54.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.619+0000 7f68893f2700 1 -- 172.21.15.140:0/4092278342 wait complete. 2022-07-21T01:50:54.650 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-07-21T01:50:54.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:54.653 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:54.654 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:54.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:50:54.665 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-07-21T01:50:54.666 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-07-21T01:50:54.666 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:50:54.667 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:50:54.667 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:54.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:54.730 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:54.730 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:54.737 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:50:54.739 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:50:54.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:50:54.740 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:54.788 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:50:54.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.891+0000 7f9ecbfff700 1 Processor -- start 2022-07-21T01:50:54.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.891+0000 7f9ecbfff700 1 -- start start 2022-07-21T01:50:54.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9ecc061840 0x7f9ecc061cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:54.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ecc0b22d0 con 0x7f9ecc1427a0 2022-07-21T01:50:54.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ecc0ac0c0 con 0x7f9ecc0b1ef0 2022-07-21T01:50:54.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ecc1472a0 con 0x7f9ecc061840 2022-07-21T01:50:54.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecb7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9ecc061840 0x7f9ecc061cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:54.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ec27fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f9ecc1427a0 0x7f9ecc061130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48848/0 (socket says 172.21.15.140:48848) 2022-07-21T01:50:54.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ec27fc700 1 -- 172.21.15.140:0/889293482 learned_addr learned my addr 172.21.15.140:0/889293482 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:54.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecb7fe700 1 -- 172.21.15.140:0/889293482 >> v1:172.21.15.152:6789/0 conn(0x7f9ecc0b1ef0 legacy=0x7f9ecc142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:54.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecb7fe700 1 -- 172.21.15.140:0/889293482 >> v1:172.21.15.140:6789/0 conn(0x7f9ecc1427a0 legacy=0x7f9ecc061130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:54.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecb7fe700 1 -- 172.21.15.140:0/889293482 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc0ac360 con 0x7f9ecc061840 2022-07-21T01:50:54.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9eca7fc700 1 -- 172.21.15.140:0/889293482 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371027046 0 0) 0x7f9ecc0b22d0 con 0x7f9ecc1427a0 2022-07-21T01:50:54.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecb7fe700 1 --2- 172.21.15.140:0/889293482 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ecc061840 0x7f9ecc061cd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f9ec40032f0 tx=0x7f9ec40091d0).ready entity=mon.0 client_cookie=68daedf4d70e99f9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:54.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9eca7fc700 1 -- 172.21.15.140:0/889293482 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ec400b8a0 con 0x7f9ecc061840 2022-07-21T01:50:54.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9eca7fc700 1 -- 172.21.15.140:0/889293482 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec400ba00 con 0x7f9ecc061840 2022-07-21T01:50:54.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9eca7fc700 1 -- 172.21.15.140:0/889293482 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ec400bb70 con 0x7f9ecc061840 2022-07-21T01:50:54.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 -- 172.21.15.140:0/889293482 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ecc061840 msgr2=0x7f9ecc061cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:54.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 --2- 172.21.15.140:0/889293482 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ecc061840 0x7f9ecc061cd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f9ec40032f0 tx=0x7f9ec40091d0).stop 2022-07-21T01:50:54.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 -- 172.21.15.140:0/889293482 shutdown_connections 2022-07-21T01:50:54.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 --2- 172.21.15.140:0/889293482 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ecc061840 0x7f9ecc061cd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:54.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 -- 172.21.15.140:0/889293482 >> 172.21.15.140:0/889293482 conn(0x7f9ecc0b2ca0 msgr2=0x7f9ecc0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:54.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 -- 172.21.15.140:0/889293482 shutdown_connections 2022-07-21T01:50:54.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 -- 172.21.15.140:0/889293482 wait complete. 2022-07-21T01:50:54.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 Processor -- start 2022-07-21T01:50:54.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.895+0000 7f9ecbfff700 1 -- start start 2022-07-21T01:50:54.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ecc061840 0x7f9ecc0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:54.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecbfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ecc0b1ef0 0x7f9ecc155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:54.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ecc1427a0 0x7f9ecc099230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:54.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecbfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc0afa80 con 0x7f9ecc1427a0 2022-07-21T01:50:54.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecbfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc1562d0 con 0x7f9ecc061840 2022-07-21T01:50:54.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecbfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc156570 con 0x7f9ecc0b1ef0 2022-07-21T01:50:54.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecb7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ecc1427a0 0x7f9ecc099230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:54.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecb7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ecc1427a0 0x7f9ecc099230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34710/0 (socket says 172.21.15.140:34710) 2022-07-21T01:50:54.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecb7fe700 1 -- 172.21.15.140:0/3133009105 learned_addr learned my addr 172.21.15.140:0/3133009105 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:54.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ec27fc700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ecc0b1ef0 0x7f9ecc155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:54.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecaffd700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ecc061840 0x7f9ecc0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:54.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecb7fe700 1 -- 172.21.15.140:0/3133009105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ecc0b1ef0 msgr2=0x7f9ecc155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:54.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecb7fe700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ecc0b1ef0 0x7f9ecc155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:54.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecb7fe700 1 -- 172.21.15.140:0/3133009105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ecc061840 msgr2=0x7f9ecc0aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:54.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecb7fe700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ecc061840 0x7f9ecc0aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:54.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecb7fe700 1 -- 172.21.15.140:0/3133009105 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec4009470 con 0x7f9ecc1427a0 2022-07-21T01:50:54.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ec27fc700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ecc0b1ef0 0x7f9ecc155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:54.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.899+0000 7f9ecb7fe700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ecc1427a0 0x7f9ecc099230 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f9ec40032f0 tx=0x7f9ec4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:54.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.903+0000 7f9ec8ff9700 1 -- 172.21.15.140:0/3133009105 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ec400bb70 con 0x7f9ecc1427a0 2022-07-21T01:50:54.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.903+0000 7f9ec8ff9700 1 -- 172.21.15.140:0/3133009105 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec401f3f0 con 0x7f9ecc1427a0 2022-07-21T01:50:54.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.903+0000 7f9ec8ff9700 1 -- 172.21.15.140:0/3133009105 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ec401f750 con 0x7f9ecc1427a0 2022-07-21T01:50:54.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.903+0000 7f9ecbfff700 1 -- 172.21.15.140:0/3133009105 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ecc0ae690 con 0x7f9ecc1427a0 2022-07-21T01:50:54.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.903+0000 7f9ecbfff700 1 -- 172.21.15.140:0/3133009105 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc0ad640 con 0x7f9ecc1427a0 2022-07-21T01:50:54.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.903+0000 7f9ecbfff700 1 -- 172.21.15.140:0/3133009105 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ecc156120 con 0x7f9ecc1427a0 2022-07-21T01:50:54.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.903+0000 7f9ec8ff9700 1 -- 172.21.15.140:0/3133009105 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9ec4026030 con 0x7f9ecc1427a0 2022-07-21T01:50:54.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.903+0000 7f9ec8ff9700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb0053a70 0x7f9eb0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:54.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.903+0000 7f9ec8ff9700 1 -- 172.21.15.140:0/3133009105 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9ec406cc40 con 0x7f9ecc1427a0 2022-07-21T01:50:54.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.903+0000 7f9ecaffd700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb0053a70 0x7f9eb0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:54.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.907+0000 7f9ecaffd700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb0053a70 0x7f9eb0055f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f9ebc00d000 tx=0x7f9ebc007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:54.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:54.911+0000 7f9ec8ff9700 1 -- 172.21.15.140:0/3133009105 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ec4043b60 con 0x7f9ecc1427a0 2022-07-21T01:50:55.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.027+0000 7f9ecbfff700 1 -- 172.21.15.140:0/3133009105 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ecc0ac600 con 0x7f9ecc1427a0 2022-07-21T01:50:55.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.027+0000 7f9ec8ff9700 1 -- 172.21.15.140:0/3133009105 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16134 (secure 0 0 0) 0x7f9ec4024090 con 0x7f9ecc1427a0 2022-07-21T01:50:55.029 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:55.029 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":42,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:50:29.338430+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:50:25.939192+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2022-07-21T01:50:26.391519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:50:55.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 -- 172.21.15.140:0/3133009105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb0053a70 msgr2=0x7f9eb0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb0053a70 0x7f9eb0055f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f9ebc00d000 tx=0x7f9ebc007b80).stop 2022-07-21T01:50:55.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 -- 172.21.15.140:0/3133009105 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ecc1427a0 msgr2=0x7f9ecc099230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ecc1427a0 0x7f9ecc099230 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f9ec40032f0 tx=0x7f9ec4016000).stop 2022-07-21T01:50:55.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 -- 172.21.15.140:0/3133009105 shutdown_connections 2022-07-21T01:50:55.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb0053a70 0x7f9eb0055f30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ecc1427a0 0x7f9ecc099230 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ecc0b1ef0 0x7f9ecc155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 --2- 172.21.15.140:0/3133009105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ecc061840 0x7f9ecc0aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 -- 172.21.15.140:0/3133009105 >> 172.21.15.140:0/3133009105 conn(0x7f9ecc0b2ca0 msgr2=0x7f9ecc0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:55.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 -- 172.21.15.140:0/3133009105 shutdown_connections 2022-07-21T01:50:55.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.031+0000 7f9ecbfff700 1 -- 172.21.15.140:0/3133009105 wait complete. 2022-07-21T01:50:55.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:50:25.939192+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:50:26.391519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:50:55.043 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:50:55.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.147+0000 7f1f9d461700 1 Processor -- start 2022-07-21T01:50:55.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.147+0000 7f1f9d461700 1 -- start start 2022-07-21T01:50:55.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1f9809df40 0x7f1f98145680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1f980b4d60 0x7f1f98145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1f98060ee0 0x7f1f98146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f98060d20 con 0x7f1f980b4d60 2022-07-21T01:50:55.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f980607e0 con 0x7f1f9809df40 2022-07-21T01:50:55.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98060a80 con 0x7f1f98060ee0 2022-07-21T01:50:55.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f967fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1f980b4d60 0x7f1f98145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f967fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1f980b4d60 0x7f1f98145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34714/0 (socket says 172.21.15.140:34714) 2022-07-21T01:50:55.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f967fc700 1 -- 172.21.15.140:0/2903403971 learned_addr learned my addr 172.21.15.140:0/2903403971 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:55.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f977fe700 1 --2- 172.21.15.140:0/2903403971 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1f98060ee0 0x7f1f98146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f96ffd700 1 --2- 172.21.15.140:0/2903403971 >> v2:172.21.15.152:3300/0 conn(0x7f1f9809df40 0x7f1f98145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f967fc700 1 -- 172.21.15.140:0/2903403971 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1f98060ee0 msgr2=0x7f1f98146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:55.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f967fc700 1 --2- 172.21.15.140:0/2903403971 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1f98060ee0 0x7f1f98146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f967fc700 1 -- 172.21.15.140:0/2903403971 >> v2:172.21.15.152:3300/0 conn(0x7f1f9809df40 msgr2=0x7f1f98145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:55.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f967fc700 1 --2- 172.21.15.140:0/2903403971 >> v2:172.21.15.152:3300/0 conn(0x7f1f9809df40 0x7f1f98145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f967fc700 1 -- 172.21.15.140:0/2903403971 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9809e320 con 0x7f1f980b4d60 2022-07-21T01:50:55.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f977fe700 1 --2- 172.21.15.140:0/2903403971 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1f98060ee0 0x7f1f98146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:55.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f967fc700 1 --2- 172.21.15.140:0/2903403971 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1f980b4d60 0x7f1f98145b70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f1f88002f70 tx=0x7f1f88006eb0).ready entity=mon.0 client_cookie=a53d44641db7a797 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:55.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f95ffb700 1 -- 172.21.15.140:0/2903403971 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f8800d570 con 0x7f1f980b4d60 2022-07-21T01:50:55.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f95ffb700 1 -- 172.21.15.140:0/2903403971 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f88000e90 con 0x7f1f980b4d60 2022-07-21T01:50:55.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f95ffb700 1 -- 172.21.15.140:0/2903403971 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f8800d900 con 0x7f1f980b4d60 2022-07-21T01:50:55.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 -- 172.21.15.140:0/2903403971 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1f980b4d60 msgr2=0x7f1f98145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 --2- 172.21.15.140:0/2903403971 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1f980b4d60 0x7f1f98145b70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f1f88002f70 tx=0x7f1f88006eb0).stop 2022-07-21T01:50:55.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 -- 172.21.15.140:0/2903403971 shutdown_connections 2022-07-21T01:50:55.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 --2- 172.21.15.140:0/2903403971 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1f98060ee0 0x7f1f98146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 --2- 172.21.15.140:0/2903403971 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1f980b4d60 0x7f1f98145b70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 --2- 172.21.15.140:0/2903403971 >> v2:172.21.15.152:3300/0 conn(0x7f1f9809df40 0x7f1f98145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 -- 172.21.15.140:0/2903403971 >> 172.21.15.140:0/2903403971 conn(0x7f1f980b5f30 msgr2=0x7f1f980b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:55.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 -- 172.21.15.140:0/2903403971 shutdown_connections 2022-07-21T01:50:55.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.151+0000 7f1f9d461700 1 -- 172.21.15.140:0/2903403971 wait complete. 2022-07-21T01:50:55.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f9d461700 1 Processor -- start 2022-07-21T01:50:55.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f9d461700 1 -- start start 2022-07-21T01:50:55.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f9d461700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1f98060ee0 0x7f1f98146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f9d461700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f9809df40 0x7f1f98155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f9d461700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1f980b4d60 0x7f1f98090f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f9d461700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f980b0c40 con 0x7f1f980b4d60 2022-07-21T01:50:55.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f9d461700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f980b0560 con 0x7f1f9809df40 2022-07-21T01:50:55.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f9d461700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98090aa0 con 0x7f1f98060ee0 2022-07-21T01:50:55.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f977fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1f980b4d60 0x7f1f98090f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f967fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f9809df40 0x7f1f98155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f977fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1f980b4d60 0x7f1f98090f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34724/0 (socket says 172.21.15.140:34724) 2022-07-21T01:50:55.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f977fe700 1 -- 172.21.15.140:0/3608967933 learned_addr learned my addr 172.21.15.140:0/3608967933 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:55.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f96ffd700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1f98060ee0 0x7f1f98146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f977fe700 1 -- 172.21.15.140:0/3608967933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1f98060ee0 msgr2=0x7f1f98146aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f977fe700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1f98060ee0 0x7f1f98146aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f977fe700 1 -- 172.21.15.140:0/3608967933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f9809df40 msgr2=0x7f1f98155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f977fe700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f9809df40 0x7f1f98155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f977fe700 1 -- 172.21.15.140:0/3608967933 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f90009470 con 0x7f1f980b4d60 2022-07-21T01:50:55.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f977fe700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1f980b4d60 0x7f1f98090f00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f1f9000c000 tx=0x7f1f900091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:55.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f8ffff700 1 -- 172.21.15.140:0/3608967933 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f9000c850 con 0x7f1f980b4d60 2022-07-21T01:50:55.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f8ffff700 1 -- 172.21.15.140:0/3608967933 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9000c9b0 con 0x7f1f980b4d60 2022-07-21T01:50:55.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f9d461700 1 -- 172.21.15.140:0/3608967933 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8800b1e0 con 0x7f1f980b4d60 2022-07-21T01:50:55.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f8ffff700 1 -- 172.21.15.140:0/3608967933 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f9000cd10 con 0x7f1f980b4d60 2022-07-21T01:50:55.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f9d461700 1 -- 172.21.15.140:0/3608967933 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f980b01a0 con 0x7f1f980b4d60 2022-07-21T01:50:55.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.155+0000 7f1f9d461700 1 -- 172.21.15.140:0/3608967933 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f98090c10 con 0x7f1f980b4d60 2022-07-21T01:50:55.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.159+0000 7f1f8ffff700 1 -- 172.21.15.140:0/3608967933 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1f90022070 con 0x7f1f980b4d60 2022-07-21T01:50:55.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.159+0000 7f1f8ffff700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f7c053980 0x7f1f7c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.159+0000 7f1f96ffd700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f7c053980 0x7f1f7c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.159+0000 7f1f8ffff700 1 -- 172.21.15.140:0/3608967933 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1f9006c6f0 con 0x7f1f980b4d60 2022-07-21T01:50:55.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.159+0000 7f1f96ffd700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f7c053980 0x7f1f7c055e40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f1f80006000 tx=0x7f1f80006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:55.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.167+0000 7f1f8ffff700 1 -- 172.21.15.140:0/3608967933 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1f90043500 con 0x7f1f980b4d60 2022-07-21T01:50:55.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.287+0000 7f1f9d461700 1 -- 172.21.15.140:0/3608967933 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1f98146060 con 0x7f1f980b4d60 2022-07-21T01:50:55.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f8ffff700 1 -- 172.21.15.140:0/3608967933 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v42) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1f90016020 con 0x7f1f980b4d60 2022-07-21T01:50:55.290 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:50:55.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 -- 172.21.15.140:0/3608967933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f7c053980 msgr2=0x7f1f7c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f7c053980 0x7f1f7c055e40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f1f80006000 tx=0x7f1f80006d90).stop 2022-07-21T01:50:55.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 -- 172.21.15.140:0/3608967933 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1f980b4d60 msgr2=0x7f1f98090f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1f980b4d60 0x7f1f98090f00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f1f9000c000 tx=0x7f1f900091d0).stop 2022-07-21T01:50:55.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 -- 172.21.15.140:0/3608967933 shutdown_connections 2022-07-21T01:50:55.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f7c053980 0x7f1f7c055e40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1f980b4d60 0x7f1f98090f00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f9809df40 0x7f1f98155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 --2- 172.21.15.140:0/3608967933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1f98060ee0 0x7f1f98146aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 -- 172.21.15.140:0/3608967933 >> 172.21.15.140:0/3608967933 conn(0x7f1f980b5f30 msgr2=0x7f1f980b8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:55.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 -- 172.21.15.140:0/3608967933 shutdown_connections 2022-07-21T01:50:55.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.291+0000 7f1f9d461700 1 -- 172.21.15.140:0/3608967933 wait complete. 2022-07-21T01:50:55.310 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:50:55.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f88989fc700 1 Processor -- start 2022-07-21T01:50:55.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f88989fc700 1 -- start start 2022-07-21T01:50:55.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f88989fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f88940b23f0 0x7f889413fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f88989fc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8894097ad0 0x7f88941402b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f88989fc700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8894061080 con 0x7f889409a310 2022-07-21T01:50:55.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f88989fc700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88940b0ee0 con 0x7f88940b23f0 2022-07-21T01:50:55.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f88989fc700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88940b1180 con 0x7f8894097ad0 2022-07-21T01:50:55.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f8892d9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f889409a310 0x7f889409a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48878/0 (socket says 172.21.15.140:48878) 2022-07-21T01:50:55.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f8892d9d700 1 -- 172.21.15.140:0/4027555274 learned_addr learned my addr 172.21.15.140:0/4027555274 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:50:55.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f889259c700 1 --2- 172.21.15.140:0/4027555274 >> v2:172.21.15.140:3300/0 conn(0x7f88940b23f0 0x7f889413fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f8889d9b700 1 --2- 172.21.15.140:0/4027555274 >> v2:172.21.15.152:3300/0 conn(0x7f8894097ad0 0x7f88941402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.419+0000 7f8891d9b700 1 -- 172.21.15.140:0/4027555274 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973073907 0 0) 0x7f8894061080 con 0x7f889409a310 2022-07-21T01:50:55.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f8891d9b700 1 -- 172.21.15.140:0/4027555274 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8874003260 con 0x7f889409a310 2022-07-21T01:50:55.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f889259c700 1 -- 172.21.15.140:0/4027555274 >> v2:172.21.15.152:3300/0 conn(0x7f8894097ad0 msgr2=0x7f88941402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:55.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f8891d9b700 1 -- 172.21.15.140:0/4027555274 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1998154105 0 0) 0x7f8874003260 con 0x7f889409a310 2022-07-21T01:50:55.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f889259c700 1 --2- 172.21.15.140:0/4027555274 >> v2:172.21.15.152:3300/0 conn(0x7f8894097ad0 0x7f88941402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f889259c700 1 -- 172.21.15.140:0/4027555274 >> v1:172.21.15.140:6789/0 conn(0x7f889409a310 legacy=0x7f889409a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f889259c700 1 -- 172.21.15.140:0/4027555274 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88940b1420 con 0x7f88940b23f0 2022-07-21T01:50:55.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f889259c700 1 --2- 172.21.15.140:0/4027555274 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940b23f0 0x7f889413fdc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f888c001b80 tx=0x7f8884006df0).ready entity=mon.0 client_cookie=fe1963bbf163df1f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:55.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f8891d9b700 1 -- 172.21.15.140:0/4027555274 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f888c0019e0 con 0x7f88940b23f0 2022-07-21T01:50:55.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f8891d9b700 1 -- 172.21.15.140:0/4027555274 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8884000e90 con 0x7f88940b23f0 2022-07-21T01:50:55.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f8891d9b700 1 -- 172.21.15.140:0/4027555274 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8884012460 con 0x7f88940b23f0 2022-07-21T01:50:55.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f88989fc700 1 -- 172.21.15.140:0/4027555274 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940b23f0 msgr2=0x7f889413fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f88989fc700 1 --2- 172.21.15.140:0/4027555274 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940b23f0 0x7f889413fdc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f888c001b80 tx=0x7f8884006df0).stop 2022-07-21T01:50:55.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f88989fc700 1 -- 172.21.15.140:0/4027555274 shutdown_connections 2022-07-21T01:50:55.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f88989fc700 1 --2- 172.21.15.140:0/4027555274 >> v2:172.21.15.152:3300/0 conn(0x7f8894097ad0 0x7f88941402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f88989fc700 1 --2- 172.21.15.140:0/4027555274 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940b23f0 0x7f889413fdc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f88989fc700 1 -- 172.21.15.140:0/4027555274 >> 172.21.15.140:0/4027555274 conn(0x7f88940b5b70 msgr2=0x7f88940b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:55.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f88989fc700 1 -- 172.21.15.140:0/4027555274 shutdown_connections 2022-07-21T01:50:55.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f88989fc700 1 -- 172.21.15.140:0/4027555274 wait complete. 2022-07-21T01:50:55.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f88989fc700 1 Processor -- start 2022-07-21T01:50:55.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.423+0000 7f88989fc700 1 -- start start 2022-07-21T01:50:55.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f88989fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8894097ad0 0x7f8894067480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f88989fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f889409a310 0x7f8894146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f88989fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88940b23f0 0x7f8894066b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f88989fc700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8894067040 con 0x7f8894097ad0 2022-07-21T01:50:55.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f88989fc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8894152080 con 0x7f889409a310 2022-07-21T01:50:55.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f88989fc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88941522e0 con 0x7f88940b23f0 2022-07-21T01:50:55.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f889259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8894097ad0 0x7f8894067480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f889259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8894097ad0 0x7f8894067480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34734/0 (socket says 172.21.15.140:34734) 2022-07-21T01:50:55.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f889259c700 1 -- 172.21.15.140:0/2962217860 learned_addr learned my addr 172.21.15.140:0/2962217860 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:55.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f8892d9d700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88940b23f0 0x7f8894066b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f8889d9b700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f889409a310 0x7f8894146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f889259c700 1 -- 172.21.15.140:0/2962217860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88940b23f0 msgr2=0x7f8894066b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f889259c700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88940b23f0 0x7f8894066b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f889259c700 1 -- 172.21.15.140:0/2962217860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f889409a310 msgr2=0x7f8894146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f889259c700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f889409a310 0x7f8894146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f889259c700 1 -- 172.21.15.140:0/2962217860 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888400b180 con 0x7f8894097ad0 2022-07-21T01:50:55.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f889259c700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8894097ad0 0x7f8894067480 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f888c001b80 tx=0x7f8884014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:55.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f888bfff700 1 -- 172.21.15.140:0/2962217860 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f888400d920 con 0x7f8894097ad0 2022-07-21T01:50:55.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f888bfff700 1 -- 172.21.15.140:0/2962217860 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888400da80 con 0x7f8894097ad0 2022-07-21T01:50:55.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f88989fc700 1 -- 172.21.15.140:0/2962217860 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8894067ad0 con 0x7f8894097ad0 2022-07-21T01:50:55.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f888bfff700 1 -- 172.21.15.140:0/2962217860 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f888400dde0 con 0x7f8894097ad0 2022-07-21T01:50:55.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f88989fc700 1 -- 172.21.15.140:0/2962217860 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8894142e00 con 0x7f8894097ad0 2022-07-21T01:50:55.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.427+0000 7f88989fc700 1 -- 172.21.15.140:0/2962217860 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88941524a0 con 0x7f8894097ad0 2022-07-21T01:50:55.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.431+0000 7f888bfff700 1 -- 172.21.15.140:0/2962217860 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f888401c070 con 0x7f8894097ad0 2022-07-21T01:50:55.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.431+0000 7f888bfff700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8878053980 0x7f8878055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.431+0000 7f888bfff700 1 -- 172.21.15.140:0/2962217860 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f888406c9e0 con 0x7f8894097ad0 2022-07-21T01:50:55.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.431+0000 7f8889d9b700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8878053980 0x7f8878055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.431+0000 7f8889d9b700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8878053980 0x7f8878055e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f887c00b000 tx=0x7f887c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:55.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.435+0000 7f888bfff700 1 -- 172.21.15.140:0/2962217860 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8884043770 con 0x7f8894097ad0 2022-07-21T01:50:55.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.559+0000 7f88989fc700 1 -- 172.21.15.140:0/2962217860 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f88940b16c0 con 0x7f8894097ad0 2022-07-21T01:50:55.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f888bfff700 1 -- 172.21.15.140:0/2962217860 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v42) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8884023070 con 0x7f8894097ad0 2022-07-21T01:50:55.564 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:50:55.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f88989fc700 1 -- 172.21.15.140:0/2962217860 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8878053980 msgr2=0x7f8878055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f88989fc700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8878053980 0x7f8878055e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f887c00b000 tx=0x7f887c006df0).stop 2022-07-21T01:50:55.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f88989fc700 1 -- 172.21.15.140:0/2962217860 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8894097ad0 msgr2=0x7f8894067480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f88989fc700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8894097ad0 0x7f8894067480 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f888c001b80 tx=0x7f8884014000).stop 2022-07-21T01:50:55.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f88989fc700 1 -- 172.21.15.140:0/2962217860 shutdown_connections 2022-07-21T01:50:55.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f88989fc700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8878053980 0x7f8878055e40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f88989fc700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88940b23f0 0x7f8894066b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f88989fc700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f889409a310 0x7f8894146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f88989fc700 1 --2- 172.21.15.140:0/2962217860 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8894097ad0 0x7f8894067480 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f88989fc700 1 -- 172.21.15.140:0/2962217860 >> 172.21.15.140:0/2962217860 conn(0x7f88940b5b70 msgr2=0x7f889413c380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:55.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.563+0000 7f88989fc700 1 -- 172.21.15.140:0/2962217860 shutdown_connections 2022-07-21T01:50:55.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.567+0000 7f88989fc700 1 -- 172.21.15.140:0/2962217860 wait complete. 2022-07-21T01:50:55.585 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:50:55.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.699+0000 7f0fe5264700 1 Processor -- start 2022-07-21T01:50:55.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.699+0000 7f0fe5264700 1 -- start start 2022-07-21T01:50:55.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.699+0000 7f0fe5264700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0fe00acbe0 0x7f0fe00ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.699+0000 7f0fe5264700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0fe0141ee0 0x7f0fe0142340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.699+0000 7f0fe5264700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe0146260 con 0x7f0fe005f840 2022-07-21T01:50:55.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.699+0000 7f0fe5264700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe01470b0 con 0x7f0fe00acbe0 2022-07-21T01:50:55.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.699+0000 7f0fe5264700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe00ab760 con 0x7f0fe0141ee0 2022-07-21T01:50:55.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.699+0000 7f0fdffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0fe00acbe0 0x7f0fe00ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.699+0000 7f0fdffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0fe00acbe0 0x7f0fe00ab270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34742/0 (socket says 172.21.15.140:34742) 2022-07-21T01:50:55.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.699+0000 7f0fdffff700 1 -- 172.21.15.140:0/3015457361 learned_addr learned my addr 172.21.15.140:0/3015457361 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:55.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.699+0000 7f0fe4a63700 1 --2- 172.21.15.140:0/3015457361 >> v2:172.21.15.152:3300/0 conn(0x7f0fe0141ee0 0x7f0fe0142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fdffff700 1 -- 172.21.15.140:0/3015457361 >> v2:172.21.15.152:3300/0 conn(0x7f0fe0141ee0 msgr2=0x7f0fe0142340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:55.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fdffff700 1 --2- 172.21.15.140:0/3015457361 >> v2:172.21.15.152:3300/0 conn(0x7f0fe0141ee0 0x7f0fe0142340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fdffff700 1 -- 172.21.15.140:0/3015457361 >> v1:172.21.15.152:6789/0 conn(0x7f0fe005f840 legacy=0x7f0fe01469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fdffff700 1 -- 172.21.15.140:0/3015457361 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe0146500 con 0x7f0fe00acbe0 2022-07-21T01:50:55.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fdffff700 1 --2- 172.21.15.140:0/3015457361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fe00acbe0 0x7f0fe00ab270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f0fe00ab8a0 tx=0x7f0fd0006eb0).ready entity=mon.0 client_cookie=55db59b39050270d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:55.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fdeffd700 1 -- 172.21.15.140:0/3015457361 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd000d610 con 0x7f0fe00acbe0 2022-07-21T01:50:55.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fdeffd700 1 -- 172.21.15.140:0/3015457361 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0000e90 con 0x7f0fe00acbe0 2022-07-21T01:50:55.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fdeffd700 1 -- 172.21.15.140:0/3015457361 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd000d9a0 con 0x7f0fe00acbe0 2022-07-21T01:50:55.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fe5264700 1 -- 172.21.15.140:0/3015457361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fe00acbe0 msgr2=0x7f0fe00ab270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fe5264700 1 --2- 172.21.15.140:0/3015457361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fe00acbe0 0x7f0fe00ab270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f0fe00ab8a0 tx=0x7f0fd0006eb0).stop 2022-07-21T01:50:55.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fe5264700 1 -- 172.21.15.140:0/3015457361 shutdown_connections 2022-07-21T01:50:55.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fe5264700 1 --2- 172.21.15.140:0/3015457361 >> v2:172.21.15.152:3300/0 conn(0x7f0fe0141ee0 0x7f0fe0142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fe5264700 1 --2- 172.21.15.140:0/3015457361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fe00acbe0 0x7f0fe00ab270 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fe5264700 1 -- 172.21.15.140:0/3015457361 >> 172.21.15.140:0/3015457361 conn(0x7f0fe00b2af0 msgr2=0x7f0fe00b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:55.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fe5264700 1 -- 172.21.15.140:0/3015457361 shutdown_connections 2022-07-21T01:50:55.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.703+0000 7f0fe5264700 1 -- 172.21.15.140:0/3015457361 wait complete. 2022-07-21T01:50:55.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe5264700 1 Processor -- start 2022-07-21T01:50:55.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe5264700 1 -- start start 2022-07-21T01:50:55.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe5264700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fe005f840 0x7f0fe0151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe5264700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fe00acbe0 0x7f0fe0152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe5264700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fe0141ee0 0x7f0fe00ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe5264700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe0152d80 con 0x7f0fe0141ee0 2022-07-21T01:50:55.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe5264700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe0152ae0 con 0x7f0fe005f840 2022-07-21T01:50:55.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe5264700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe0061960 con 0x7f0fe00acbe0 2022-07-21T01:50:55.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe4a63700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fe0141ee0 0x7f0fe00ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fdffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fe005f840 0x7f0fe0151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe4a63700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fe0141ee0 0x7f0fe00ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34750/0 (socket says 172.21.15.140:34750) 2022-07-21T01:50:55.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe4a63700 1 -- 172.21.15.140:0/1147942469 learned_addr learned my addr 172.21.15.140:0/1147942469 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:55.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fdf7fe700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fe00acbe0 0x7f0fe0152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe4a63700 1 -- 172.21.15.140:0/1147942469 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fe00acbe0 msgr2=0x7f0fe0152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe4a63700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fe00acbe0 0x7f0fe0152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe4a63700 1 -- 172.21.15.140:0/1147942469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fe005f840 msgr2=0x7f0fe0151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe4a63700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fe005f840 0x7f0fe0151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe4a63700 1 -- 172.21.15.140:0/1147942469 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd000b1e0 con 0x7f0fe0141ee0 2022-07-21T01:50:55.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fdffff700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fe005f840 0x7f0fe0151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:50:55.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe4a63700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fe0141ee0 0x7f0fe00ae070 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f0fd8006e00 tx=0x7f0fd80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:55.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fdd7fa700 1 -- 172.21.15.140:0/1147942469 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd8012600 con 0x7f0fe0141ee0 2022-07-21T01:50:55.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fdd7fa700 1 -- 172.21.15.140:0/1147942469 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd8012760 con 0x7f0fe0141ee0 2022-07-21T01:50:55.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe5264700 1 -- 172.21.15.140:0/1147942469 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe0152760 con 0x7f0fe0141ee0 2022-07-21T01:50:55.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fdd7fa700 1 -- 172.21.15.140:0/1147942469 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd8012ac0 con 0x7f0fe0141ee0 2022-07-21T01:50:55.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.707+0000 7f0fe5264700 1 -- 172.21.15.140:0/1147942469 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe0096430 con 0x7f0fe0141ee0 2022-07-21T01:50:55.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.711+0000 7f0fdd7fa700 1 -- 172.21.15.140:0/1147942469 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0fd8022070 con 0x7f0fe0141ee0 2022-07-21T01:50:55.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.711+0000 7f0fdd7fa700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fc4053a10 0x7f0fc4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.711+0000 7f0fdd7fa700 1 -- 172.21.15.140:0/1147942469 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0fd806c550 con 0x7f0fe0141ee0 2022-07-21T01:50:55.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.711+0000 7f0fdffff700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fc4053a10 0x7f0fc4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.711+0000 7f0fe5264700 1 -- 172.21.15.140:0/1147942469 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe00acfc0 con 0x7f0fe0141ee0 2022-07-21T01:50:55.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.711+0000 7f0fdffff700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fc4053a10 0x7f0fc4055ed0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f0fd001b000 tx=0x7f0fd0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:55.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.715+0000 7f0fdd7fa700 1 -- 172.21.15.140:0/1147942469 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0fd8016080 con 0x7f0fe0141ee0 2022-07-21T01:50:55.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 -- 172.21.15.140:0/1147942469 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0fe00574e0 con 0x7f0fe0141ee0 2022-07-21T01:50:55.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fdd7fa700 1 -- 172.21.15.140:0/1147942469 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0fd8043480 con 0x7f0fe0141ee0 2022-07-21T01:50:55.840 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:50:55.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 -- 172.21.15.140:0/1147942469 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fc4053a10 msgr2=0x7f0fc4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fc4053a10 0x7f0fc4055ed0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f0fd001b000 tx=0x7f0fd0015000).stop 2022-07-21T01:50:55.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 -- 172.21.15.140:0/1147942469 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fe0141ee0 msgr2=0x7f0fe00ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fe0141ee0 0x7f0fe00ae070 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f0fd8006e00 tx=0x7f0fd80091d0).stop 2022-07-21T01:50:55.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 -- 172.21.15.140:0/1147942469 shutdown_connections 2022-07-21T01:50:55.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fc4053a10 0x7f0fc4055ed0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fe0141ee0 0x7f0fe00ae070 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fe00acbe0 0x7f0fe0152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 --2- 172.21.15.140:0/1147942469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fe005f840 0x7f0fe0151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 -- 172.21.15.140:0/1147942469 >> 172.21.15.140:0/1147942469 conn(0x7f0fe00b2af0 msgr2=0x7f0fe013f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:55.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 -- 172.21.15.140:0/1147942469 shutdown_connections 2022-07-21T01:50:55.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.839+0000 7f0fe5264700 1 -- 172.21.15.140:0/1147942469 wait complete. 2022-07-21T01:50:55.851 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:50:55.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.963+0000 7faca68d9700 1 Processor -- start 2022-07-21T01:50:55.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.963+0000 7faca68d9700 1 -- start start 2022-07-21T01:50:55.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faca00b5690 0x7faca0140730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faca0140c20 0x7faca01296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca005cc80 con 0x7faca00af030 2022-07-21T01:50:55.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca00af410 con 0x7faca0140c20 2022-07-21T01:50:55.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca0144050 con 0x7faca00b5690 2022-07-21T01:50:55.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca60d8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faca0140c20 0x7faca01296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca60d8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faca0140c20 0x7faca01296f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34758/0 (socket says 172.21.15.140:34758) 2022-07-21T01:50:55.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca60d8700 1 -- 172.21.15.140:0/3253727111 learned_addr learned my addr 172.21.15.140:0/3253727111 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:55.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca50d6700 1 --2- 172.21.15.140:0/3253727111 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faca00b5690 0x7faca0140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca60d8700 1 -- 172.21.15.140:0/3253727111 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faca00b5690 msgr2=0x7faca0140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:55.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca60d8700 1 --2- 172.21.15.140:0/3253727111 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faca00b5690 0x7faca0140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca60d8700 1 -- 172.21.15.140:0/3253727111 >> v1:172.21.15.152:6789/0 conn(0x7faca00af030 legacy=0x7faca005c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca60d8700 1 -- 172.21.15.140:0/3253727111 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca005be30 con 0x7faca0140c20 2022-07-21T01:50:55.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca60d8700 1 --2- 172.21.15.140:0/3253727111 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faca0140c20 0x7faca01296f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fac980032f0 tx=0x7fac98009290).ready entity=mon.0 client_cookie=e9ac3d3e2f242fbf server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:55.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca48d5700 1 -- 172.21.15.140:0/3253727111 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9800b8a0 con 0x7faca0140c20 2022-07-21T01:50:55.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca48d5700 1 -- 172.21.15.140:0/3253727111 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9800ba00 con 0x7faca0140c20 2022-07-21T01:50:55.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca48d5700 1 -- 172.21.15.140:0/3253727111 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9800bb70 con 0x7faca0140c20 2022-07-21T01:50:55.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 -- 172.21.15.140:0/3253727111 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faca0140c20 msgr2=0x7faca01296f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 --2- 172.21.15.140:0/3253727111 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faca0140c20 0x7faca01296f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fac980032f0 tx=0x7fac98009290).stop 2022-07-21T01:50:55.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 -- 172.21.15.140:0/3253727111 shutdown_connections 2022-07-21T01:50:55.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 --2- 172.21.15.140:0/3253727111 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faca0140c20 0x7faca01296f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 --2- 172.21.15.140:0/3253727111 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faca00b5690 0x7faca0140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 -- 172.21.15.140:0/3253727111 >> 172.21.15.140:0/3253727111 conn(0x7faca00634c0 msgr2=0x7faca00b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:55.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 -- 172.21.15.140:0/3253727111 shutdown_connections 2022-07-21T01:50:55.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.967+0000 7faca68d9700 1 -- 172.21.15.140:0/3253727111 wait complete. 2022-07-21T01:50:55.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.971+0000 7faca68d9700 1 Processor -- start 2022-07-21T01:50:55.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.971+0000 7faca68d9700 1 -- start start 2022-07-21T01:50:55.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.971+0000 7faca68d9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faca00af030 0x7faca01538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.971+0000 7faca68d9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca00b5690 0x7faca0065360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.971+0000 7faca68d9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faca0140c20 0x7faca014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:55.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.971+0000 7faca68d9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca012f160 con 0x7faca00af030 2022-07-21T01:50:55.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.971+0000 7faca68d9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca01531a0 con 0x7faca00b5690 2022-07-21T01:50:55.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.971+0000 7faca68d9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca0153440 con 0x7faca0140c20 2022-07-21T01:50:55.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca58d7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faca00af030 0x7faca01538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca50d6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca00b5690 0x7faca0065360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca58d7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faca00af030 0x7faca01538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34760/0 (socket says 172.21.15.140:34760) 2022-07-21T01:50:55.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca58d7700 1 -- 172.21.15.140:0/192155818 learned_addr learned my addr 172.21.15.140:0/192155818 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:55.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca60d8700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faca0140c20 0x7faca014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:55.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca58d7700 1 -- 172.21.15.140:0/192155818 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faca0140c20 msgr2=0x7faca014fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca58d7700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faca0140c20 0x7faca014fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca58d7700 1 -- 172.21.15.140:0/192155818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca00b5690 msgr2=0x7faca0065360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:55.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca58d7700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca00b5690 0x7faca0065360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:55.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca58d7700 1 -- 172.21.15.140:0/192155818 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98009530 con 0x7faca00af030 2022-07-21T01:50:55.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca58d7700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faca00af030 0x7faca01538e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7faca0062710 tx=0x7fac9c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:55.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7fac8effd700 1 -- 172.21.15.140:0/192155818 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9c012400 con 0x7faca00af030 2022-07-21T01:50:55.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7fac8effd700 1 -- 172.21.15.140:0/192155818 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c012560 con 0x7faca00af030 2022-07-21T01:50:55.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7fac8effd700 1 -- 172.21.15.140:0/192155818 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9c0128c0 con 0x7faca00af030 2022-07-21T01:50:55.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca68d9700 1 -- 172.21.15.140:0/192155818 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca012a940 con 0x7faca00af030 2022-07-21T01:50:55.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca68d9700 1 -- 172.21.15.140:0/192155818 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca0153ff0 con 0x7faca00af030 2022-07-21T01:50:55.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.975+0000 7faca68d9700 1 -- 172.21.15.140:0/192155818 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca005c0d0 con 0x7faca00af030 2022-07-21T01:50:55.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.979+0000 7fac8effd700 1 -- 172.21.15.140:0/192155818 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fac9c019030 con 0x7faca00af030 2022-07-21T01:50:56.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.979+0000 7fac8effd700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fac80053a70 0x7fac80055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:56.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.979+0000 7fac8effd700 1 -- 172.21.15.140:0/192155818 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fac9c06b370 con 0x7faca00af030 2022-07-21T01:50:56.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.979+0000 7faca50d6700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fac80053a70 0x7fac80055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:56.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.979+0000 7faca50d6700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fac80053a70 0x7fac80055f30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fac9000c000 tx=0x7fac90008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:56.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:55.987+0000 7fac8effd700 1 -- 172.21.15.140:0/192155818 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fac9c0420a0 con 0x7faca00af030 2022-07-21T01:50:56.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.111+0000 7faca68d9700 1 -- 172.21.15.140:0/192155818 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faca00574e0 con 0x7faca00af030 2022-07-21T01:50:56.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.111+0000 7fac8effd700 1 -- 172.21.15.140:0/192155818 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+10 (secure 0 0 0) 0x7fac9c0456c0 con 0x7faca00af030 2022-07-21T01:50:56.112 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:50:56.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 -- 172.21.15.140:0/192155818 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fac80053a70 msgr2=0x7fac80055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:56.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fac80053a70 0x7fac80055f30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fac9000c000 tx=0x7fac90008040).stop 2022-07-21T01:50:56.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 -- 172.21.15.140:0/192155818 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faca00af030 msgr2=0x7faca01538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:56.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faca00af030 0x7faca01538e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7faca0062710 tx=0x7fac9c007c50).stop 2022-07-21T01:50:56.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 -- 172.21.15.140:0/192155818 shutdown_connections 2022-07-21T01:50:56.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fac80053a70 0x7fac80055f30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faca0140c20 0x7faca014fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faca00b5690 0x7faca0065360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 --2- 172.21.15.140:0/192155818 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faca00af030 0x7faca01538e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 -- 172.21.15.140:0/192155818 >> 172.21.15.140:0/192155818 conn(0x7faca00634c0 msgr2=0x7faca0126800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:56.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 -- 172.21.15.140:0/192155818 shutdown_connections 2022-07-21T01:50:56.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.115+0000 7faca68d9700 1 -- 172.21.15.140:0/192155818 wait complete. 2022-07-21T01:50:56.131 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:50:56.132 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:50:56.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4385e2d700 1 Processor -- start 2022-07-21T01:50:56.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4385e2d700 1 -- start start 2022-07-21T01:50:56.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4385e2d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f43800b5650 0x7f43800b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:56.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4385e2d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f43800adff0 0x7f4380147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:56.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4385e2d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43800af8e0 0x7f438005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:56.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4385e2d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4380147010 con 0x7f43800b5650 2022-07-21T01:50:56.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4385e2d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43801472b0 con 0x7f43800adff0 2022-07-21T01:50:56.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4385e2d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4380146d40 con 0x7f43800af8e0 2022-07-21T01:50:56.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4384e2b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f43800b5650 0x7f43800b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:56.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4384e2b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f43800b5650 0x7f43800b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34768/0 (socket says 172.21.15.140:34768) 2022-07-21T01:50:56.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4384e2b700 1 -- 172.21.15.140:0/3909464166 learned_addr learned my addr 172.21.15.140:0/3909464166 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:56.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f438562c700 1 --2- 172.21.15.140:0/3909464166 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43800af8e0 0x7f438005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:56.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f437ffff700 1 --2- 172.21.15.140:0/3909464166 >> v2:172.21.15.152:3300/0 conn(0x7f43800adff0 0x7f4380147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:56.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4384e2b700 1 -- 172.21.15.140:0/3909464166 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43800af8e0 msgr2=0x7f438005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:56.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4384e2b700 1 --2- 172.21.15.140:0/3909464166 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43800af8e0 0x7f438005bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4384e2b700 1 -- 172.21.15.140:0/3909464166 >> v2:172.21.15.152:3300/0 conn(0x7f43800adff0 msgr2=0x7f4380147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:50:56.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4384e2b700 1 --2- 172.21.15.140:0/3909464166 >> v2:172.21.15.152:3300/0 conn(0x7f43800adff0 0x7f4380147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4384e2b700 1 -- 172.21.15.140:0/3909464166 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4380147c40 con 0x7f43800b5650 2022-07-21T01:50:56.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f4384e2b700 1 --2- 172.21.15.140:0/3909464166 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43800b5650 0x7f43800b5a30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f43780032f0 tx=0x7f4378009290).ready entity=mon.0 client_cookie=f05c3a472c7ef635 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:56.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f437f7fe700 1 -- 172.21.15.140:0/3909464166 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f437800b870 con 0x7f43800b5650 2022-07-21T01:50:56.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f437f7fe700 1 -- 172.21.15.140:0/3909464166 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437800b9d0 con 0x7f43800b5650 2022-07-21T01:50:56.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.235+0000 7f437f7fe700 1 -- 172.21.15.140:0/3909464166 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f437800bb40 con 0x7f43800b5650 2022-07-21T01:50:56.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 -- 172.21.15.140:0/3909464166 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43800b5650 msgr2=0x7f43800b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:56.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 --2- 172.21.15.140:0/3909464166 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43800b5650 0x7f43800b5a30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f43780032f0 tx=0x7f4378009290).stop 2022-07-21T01:50:56.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 -- 172.21.15.140:0/3909464166 shutdown_connections 2022-07-21T01:50:56.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 --2- 172.21.15.140:0/3909464166 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43800af8e0 0x7f438005bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 --2- 172.21.15.140:0/3909464166 >> v2:172.21.15.152:3300/0 conn(0x7f43800adff0 0x7f4380147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 --2- 172.21.15.140:0/3909464166 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43800b5650 0x7f43800b5a30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 -- 172.21.15.140:0/3909464166 >> 172.21.15.140:0/3909464166 conn(0x7f43800634e0 msgr2=0x7f43800b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:56.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 -- 172.21.15.140:0/3909464166 shutdown_connections 2022-07-21T01:50:56.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 -- 172.21.15.140:0/3909464166 wait complete. 2022-07-21T01:50:56.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 Processor -- start 2022-07-21T01:50:56.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 -- start start 2022-07-21T01:50:56.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43800adff0 0x7f4380149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:56.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43800af8e0 0x7f438010ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:56.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43800b5650 0x7f438010f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:56.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438010f5e0 con 0x7f43800af8e0 2022-07-21T01:50:56.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438010e4c0 con 0x7f43800b5650 2022-07-21T01:50:56.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4385e2d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438010e760 con 0x7f43800adff0 2022-07-21T01:50:56.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f437ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43800af8e0 0x7f438010ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:56.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f437ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43800af8e0 0x7f438010ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34776/0 (socket says 172.21.15.140:34776) 2022-07-21T01:50:56.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f437ffff700 1 -- 172.21.15.140:0/3200140586 learned_addr learned my addr 172.21.15.140:0/3200140586 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:50:56.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f4384e2b700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43800adff0 0x7f4380149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:56.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f438562c700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43800b5650 0x7f438010f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:56.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f437ffff700 1 -- 172.21.15.140:0/3200140586 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43800adff0 msgr2=0x7f4380149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:56.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f437ffff700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43800adff0 0x7f4380149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f437ffff700 1 -- 172.21.15.140:0/3200140586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43800b5650 msgr2=0x7f438010f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:56.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f437ffff700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43800b5650 0x7f438010f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f437ffff700 1 -- 172.21.15.140:0/3200140586 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4378009530 con 0x7f43800af8e0 2022-07-21T01:50:56.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.239+0000 7f438562c700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43800b5650 0x7f438010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:50:56.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.243+0000 7f437ffff700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43800af8e0 0x7f438010ec00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f437400a000 tx=0x7f4374008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:56.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.243+0000 7f437dffb700 1 -- 172.21.15.140:0/3200140586 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4374011850 con 0x7f43800af8e0 2022-07-21T01:50:56.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.243+0000 7f437dffb700 1 -- 172.21.15.140:0/3200140586 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43740119b0 con 0x7f43800af8e0 2022-07-21T01:50:56.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.243+0000 7f4385e2d700 1 -- 172.21.15.140:0/3200140586 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4380065bf0 con 0x7f43800af8e0 2022-07-21T01:50:56.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.243+0000 7f437dffb700 1 -- 172.21.15.140:0/3200140586 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4374011d10 con 0x7f43800af8e0 2022-07-21T01:50:56.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.243+0000 7f4385e2d700 1 -- 172.21.15.140:0/3200140586 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438010fb60 con 0x7f43800af8e0 2022-07-21T01:50:56.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.243+0000 7f4385e2d700 1 -- 172.21.15.140:0/3200140586 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4380065a60 con 0x7f43800af8e0 2022-07-21T01:50:56.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.243+0000 7f437dffb700 1 -- 172.21.15.140:0/3200140586 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f437401c460 con 0x7f43800af8e0 2022-07-21T01:50:56.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.243+0000 7f437dffb700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f43600539e0 0x7f4360055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:50:56.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.243+0000 7f437dffb700 1 -- 172.21.15.140:0/3200140586 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f437406c3c0 con 0x7f43800af8e0 2022-07-21T01:50:56.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.243+0000 7f4384e2b700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f43600539e0 0x7f4360055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:50:56.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.247+0000 7f4384e2b700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f43600539e0 0x7f4360055ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f437801b000 tx=0x7f4378016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:50:56.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.251+0000 7f437dffb700 1 -- 172.21.15.140:0/3200140586 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4374069e90 con 0x7f43800af8e0 2022-07-21T01:50:56.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.411+0000 7f4385e2d700 1 -- 172.21.15.140:0/3200140586 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43801101e0 con 0x7f43800af8e0 2022-07-21T01:50:56.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f437dffb700 1 -- 172.21.15.140:0/3200140586 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2141 (secure 0 0 0) 0x7f43801101e0 con 0x7f43800af8e0 2022-07-21T01:50:56.414 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:50:56.415 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/2307860244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2307860244},{"type":"v1","addr":"172.21.15.152:6833","nonce":2307860244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:50:27.322541+0000","modified":"2022-07-21T01:50:28.325625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/207391195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":207391195},{"type":"v1","addr":"172.21.15.140:6835","nonce":207391195}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-07-21T01:50:56.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 -- 172.21.15.140:0/3200140586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f43600539e0 msgr2=0x7f4360055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:56.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f43600539e0 0x7f4360055ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f437801b000 tx=0x7f4378016000).stop 2022-07-21T01:50:56.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 -- 172.21.15.140:0/3200140586 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43800af8e0 msgr2=0x7f438010ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:50:56.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43800af8e0 0x7f438010ec00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f437400a000 tx=0x7f4374008040).stop 2022-07-21T01:50:56.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 -- 172.21.15.140:0/3200140586 shutdown_connections 2022-07-21T01:50:56.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f43600539e0 0x7f4360055ea0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43800b5650 0x7f438010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43800af8e0 0x7f438010ec00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 --2- 172.21.15.140:0/3200140586 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43800adff0 0x7f4380149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:50:56.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 -- 172.21.15.140:0/3200140586 >> 172.21.15.140:0/3200140586 conn(0x7f43800634e0 msgr2=0x7f43800b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:50:56.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 -- 172.21.15.140:0/3200140586 shutdown_connections 2022-07-21T01:50:56.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:50:56.415+0000 7f4385e2d700 1 -- 172.21.15.140:0/3200140586 wait complete. 2022-07-21T01:50:56.419 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 16 2022-07-21T01:50:56.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:50:27.322541+0000', 'modified': '2022-07-21T01:50:28.325625+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5247': {'gid': 5247, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/207391195', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 207391195}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 207391195}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:50:56.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:50:56.629 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:50:56.743 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":5247,"want_state":"up:active","state":"up:active","rank_uptime":29.41069804,"mdsmap_epoch":16,"osdmap_epoch":42,"osdmap_epoch_barrier":40,"uptime":33.838262108000002} 2022-07-21T01:50:56.744 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 5247, "mdsmap_epoch": 16, "osdmap_epoch": 42, "osdmap_epoch_barrier": 40, "rank_uptime": 29.41069804, "state": "up:active", "uptime": 33.838262108, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:50:56.745 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:50:56.745 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:50:56.746 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:50:56.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:56.747 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:56.758 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:50:56.776 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:50:56.776 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:50:56.837 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:50:56.843 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:50:56.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:56.846 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:50:56.896 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:50:56.897 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:56.900 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:50:56.917 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:50:56.918 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:50:56.976 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:50:56.980+0000 7fb96b87c080 -1 init, newargv = 0x5600b2d65300 newargc=9ceph-fuse[22421]: starting ceph client 2022-07-21T01:50:56.976 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr: 2022-07-21T01:50:56.980 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:50:56.980 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:50:56.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:56.983 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:50:56.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22421]: starting fuse 2022-07-21T01:50:56.999 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:50:57.000 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:50:57.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:57.002 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:50:57.002 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:50:57.003 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:50:57.004 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:50:57.004 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:50:57.005 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:50:57.006 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:50:57.007 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:50:57.007 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:50:57.008 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:50:57.009 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:50:57.010 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:50:57.011 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:50:57.011 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:50:57.012 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:50:57.013 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:50:57.013 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:50:57.014 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:50:57.015 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:50:57.015 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:50:57.016 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:50:57.016 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:50:57.017 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:50:57.085 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.22421.asok 2022-07-21T01:50:57.089 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22421.asok 2022-07-21T01:50:57.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:57.091 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22421.asok status 2022-07-21T01:50:57.182 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:50:57.183 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:50:57.183 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:50:57.183 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:50:57.184 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:50:57.184 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:50:57.184 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:50:57.185 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "22421", 2022-07-21T01:50:57.185 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:50:57.185 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:50:57.185 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:50:57.185 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:50:57.186 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 5511, 2022-07-21T01:50:57.186 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:50:57.187 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:50:57.187 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:50:57.188 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 5511 2022-07-21T01:50:57.188 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:50:57.189 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:50:57.190 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:50:57.190 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:50:57.191 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3717043182 2022-07-21T01:50:57.191 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:50:57.192 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:50:57.192 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:50:57.193 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:50:57.193 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:50:57.194 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3717043182 2022-07-21T01:50:57.194 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:50:57.195 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.5511 172.21.15.191:0/3717043182", 2022-07-21T01:50:57.195 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/3717043182", 2022-07-21T01:50:57.196 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:50:57.196 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 16, 2022-07-21T01:50:57.196 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 42, 2022-07-21T01:50:57.197 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:50:57.197 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:50:57.198 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:50:57.198 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:50:57.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:57.200 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:50:57.201 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:50:57.209 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:50:57.211 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:50:57.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:50:57.213 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:51:01.180 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:51:01.184 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:51:01.188 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:51:01.191 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:51:01.280 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:51:31.327 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:51:31.330 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:51:31.333 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:51:31.335 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:51:31.338 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:51:42.904 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:51:42.905 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:51:42.926 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:51:42.948 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:51:42.970 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:51:42.991 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:51:43.012 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:51:43.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:51:43.014 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:51:43.014 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:51:43.022 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:51:43.024 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:51:43.025 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:51:43.025 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:51:43.026 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:51:43.082 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22421]: fuse finished with error 0 and tester_r 0 2022-07-21T01:51:43.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:51:43.084 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:51:43.085 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:51:43.092 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:51:43.093 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:51:43.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:51:43.213 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:51:43.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:51:43.221 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:51:43.222 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:51:43.229 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:51:43.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:51:43.231 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:51:43.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:51:43.232 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:51:43.238 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:51:43.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:51:43.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:51:43.240 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:51:43.241 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:51:43.249 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:51:43.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:51:43.251 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:51:43.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:51:43.252 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:51:43.259 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:51:43.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:51:43.260 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:51:43.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.371+0000 7fcadd311700 1 Processor -- start 2022-07-21T01:51:43.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.371+0000 7fcadd311700 1 -- start start 2022-07-21T01:51:43.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadd311700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcad80b5650 0x7fcad80b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadd311700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad80aea30 con 0x7fcad80b1200 2022-07-21T01:51:43.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadd311700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad80aec60 con 0x7fcad805bd90 2022-07-21T01:51:43.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadd311700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad8142970 con 0x7fcad80b5650 2022-07-21T01:51:43.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcacffff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fcad80b1200 0x7fcad805c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48928/0 (socket says 172.21.15.140:48928) 2022-07-21T01:51:43.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcacffff700 1 -- 172.21.15.140:0/1598185606 learned_addr learned my addr 172.21.15.140:0/1598185606 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:43.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad7fff700 1 --2- 172.21.15.140:0/1598185606 >> v2:172.21.15.152:3300/0 conn(0x7fcad80b5650 0x7fcad80b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112348443 0 0) 0x7fcad80aea30 con 0x7fcad80b1200 2022-07-21T01:51:43.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcabc003260 con 0x7fcad80b1200 2022-07-21T01:51:43.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4011044723 0 0) 0x7fcabc003260 con 0x7fcad80b1200 2022-07-21T01:51:43.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 >> v2:172.21.15.152:3300/0 conn(0x7fcad80b5650 msgr2=0x7fcad80b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:43.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 --2- 172.21.15.140:0/1598185606 >> v2:172.21.15.152:3300/0 conn(0x7fcad80b5650 0x7fcad80b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 >> v1:172.21.15.152:6789/0 conn(0x7fcad805bd90 legacy=0x7fcad8142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:43.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad80af180 con 0x7fcad80b1200 2022-07-21T01:51:43.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fcac00017e0 con 0x7fcad80b1200 2022-07-21T01:51:43.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 >> v1:172.21.15.140:6789/0 conn(0x7fcad80b1200 legacy=0x7fcad805c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:43.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 --2- 172.21.15.140:0/1598185606 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcabc003750 0x7fcabc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 --2- 172.21.15.140:0/1598185606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcabc004500 0x7fcabc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 --2- 172.21.15.140:0/1598185606 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcabc004e70 0x7fcabc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc00bcf0 con 0x7fcabc003750 2022-07-21T01:51:43.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc00be60 con 0x7fcabc004500 2022-07-21T01:51:43.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc00c020 con 0x7fcabc004e70 2022-07-21T01:51:43.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad7fff700 1 --2- 172.21.15.140:0/1598185606 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcabc003750 0x7fcabc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcacffff700 1 --2- 172.21.15.140:0/1598185606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcabc004500 0x7fcabc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadcb10700 1 --2- 172.21.15.140:0/1598185606 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcabc004e70 0x7fcabc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad7fff700 1 -- 172.21.15.140:0/1598185606 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcabc004e70 msgr2=0x7fcabc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:43.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad7fff700 1 --2- 172.21.15.140:0/1598185606 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcabc004e70 0x7fcabc00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad7fff700 1 -- 172.21.15.140:0/1598185606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcabc004500 msgr2=0x7fcabc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:43.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad7fff700 1 --2- 172.21.15.140:0/1598185606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcabc004500 0x7fcabc004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadcb10700 1 -- 172.21.15.140:0/1598185606 reap_dead start 2022-07-21T01:51:43.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad7fff700 1 -- 172.21.15.140:0/1598185606 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc003260 con 0x7fcabc003750 2022-07-21T01:51:43.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad7fff700 1 --2- 172.21.15.140:0/1598185606 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcabc003750 0x7fcabc003b70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fcac800c000 tx=0x7fcac8006df0).ready entity=mon.0 client_cookie=6c81aeb039e2466b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:43.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcac800e830 con 0x7fcabc003750 2022-07-21T01:51:43.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac800e990 con 0x7fcabc003750 2022-07-21T01:51:43.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcad77fe700 1 -- 172.21.15.140:0/1598185606 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcac800ecf0 con 0x7fcabc003750 2022-07-21T01:51:43.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadd311700 1 -- 172.21.15.140:0/1598185606 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcabc003750 msgr2=0x7fcabc003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:43.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadd311700 1 --2- 172.21.15.140:0/1598185606 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcabc003750 0x7fcabc003b70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fcac800c000 tx=0x7fcac8006df0).stop 2022-07-21T01:51:43.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadd311700 1 -- 172.21.15.140:0/1598185606 shutdown_connections 2022-07-21T01:51:43.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadd311700 1 --2- 172.21.15.140:0/1598185606 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcabc003750 0x7fcabc003b70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadd311700 1 -- 172.21.15.140:0/1598185606 >> 172.21.15.140:0/1598185606 conn(0x7fcad80634e0 msgr2=0x7fcad80b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:43.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadd311700 1 -- 172.21.15.140:0/1598185606 shutdown_connections 2022-07-21T01:51:43.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.375+0000 7fcadd311700 1 -- 172.21.15.140:0/1598185606 wait complete. 2022-07-21T01:51:43.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadd311700 1 Processor -- start 2022-07-21T01:51:43.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadd311700 1 -- start start 2022-07-21T01:51:43.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadd311700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcabc003750 0x7fcad8151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadd311700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcad813e800 0x7fcad813d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadd311700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcad8152790 0x7fcad8152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadd311700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad813f8f0 con 0x7fcad8152790 2022-07-21T01:51:43.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadd311700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad813d650 con 0x7fcabc003750 2022-07-21T01:51:43.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadd311700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad813ec60 con 0x7fcad813e800 2022-07-21T01:51:43.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadcb10700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcad8152790 0x7fcad8152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcad7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcabc003750 0x7fcad8151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcacffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcad813e800 0x7fcad813d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcad7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcabc003750 0x7fcad8151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:40532/0 (socket says 172.21.15.140:40532) 2022-07-21T01:51:43.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadcb10700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcad8152790 0x7fcad8152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34796/0 (socket says 172.21.15.140:34796) 2022-07-21T01:51:43.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcad7fff700 1 -- 172.21.15.140:0/366403492 learned_addr learned my addr 172.21.15.140:0/366403492 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:43.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadcb10700 1 -- 172.21.15.140:0/366403492 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcad813e800 msgr2=0x7fcad813d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:43.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadcb10700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcad813e800 0x7fcad813d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadcb10700 1 -- 172.21.15.140:0/366403492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcabc003750 msgr2=0x7fcad8151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:43.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadcb10700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcabc003750 0x7fcad8151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadcb10700 1 -- 172.21.15.140:0/366403492 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad813d250 con 0x7fcad8152790 2022-07-21T01:51:43.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadcb10700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcad8152790 0x7fcad8152c10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fcad813ef30 tx=0x7fcad0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:43.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcad5ffb700 1 -- 172.21.15.140:0/366403492 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcad0003030 con 0x7fcad8152790 2022-07-21T01:51:43.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcad5ffb700 1 -- 172.21.15.140:0/366403492 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad0003190 con 0x7fcad8152790 2022-07-21T01:51:43.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadd311700 1 -- 172.21.15.140:0/366403492 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad8064b10 con 0x7fcad8152790 2022-07-21T01:51:43.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcad5ffb700 1 -- 172.21.15.140:0/366403492 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcad0001580 con 0x7fcad8152790 2022-07-21T01:51:43.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.379+0000 7fcadd311700 1 -- 172.21.15.140:0/366403492 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad80b47d0 con 0x7fcad8152790 2022-07-21T01:51:43.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.383+0000 7fcadd311700 1 -- 172.21.15.140:0/366403492 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad813e060 con 0x7fcad8152790 2022-07-21T01:51:43.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.383+0000 7fcad5ffb700 1 -- 172.21.15.140:0/366403492 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcad001e070 con 0x7fcad8152790 2022-07-21T01:51:43.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.383+0000 7fcad5ffb700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcab0053980 0x7fcab0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.383+0000 7fcad5ffb700 1 -- 172.21.15.140:0/366403492 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcad006b1f0 con 0x7fcad8152790 2022-07-21T01:51:43.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.383+0000 7fcad7fff700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcab0053980 0x7fcab0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.383+0000 7fcad7fff700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcab0053980 0x7fcab0055e40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fcac801d000 tx=0x7fcac8019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:43.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.391+0000 7fcad5ffb700 1 -- 172.21.15.140:0/366403492 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcad0042110 con 0x7fcad8152790 2022-07-21T01:51:43.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.511+0000 7fcadd311700 1 -- 172.21.15.140:0/366403492 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fcad8064950 con 0x7fcad8152790 2022-07-21T01:51:43.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcad5ffb700 1 -- 172.21.15.140:0/366403492 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v42) v1 ==== 81+0+33741 (secure 0 0 0) 0x7fcad0045730 con 0x7fcad8152790 2022-07-21T01:51:43.515 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:43.515 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:51:43.515 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 42, 2022-07-21T01:51:43.515 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:51:43.516 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:51:43.516 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:50:29.338430+0000", 2022-07-21T01:51:43.516 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:51:43.517 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:51:43.517 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:51:43.517 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:51:43.518 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:51:43.518 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:51:43.519 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:51:43.519 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:51:43.520 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:51:43.520 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:51:43.520 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:51:43.521 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:51:43.521 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:51:43.522 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:51:43.523 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:51:43.523 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 8, 2022-07-21T01:51:43.524 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:51:43.524 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:51:43.524 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:51:43.524 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:51:43.525 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:51:43.525 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.525 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:51:43.525 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:51:43.526 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:51:43.526 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:51:43.526 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:51:43.527 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:51:43.527 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:51:43.528 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:51:43.528 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:51:43.528 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:51:43.529 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:51:43.529 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:51:43.529 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:51:43.529 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:51:43.530 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:51:43.530 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:51:43.530 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:51:43.531 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:51:43.531 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:51:43.531 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:51:43.531 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:51:43.532 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:51:43.532 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:51:43.532 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.533 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:51:43.533 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:51:43.533 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:51:43.533 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:51:43.534 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:51:43.534 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:51:43.534 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:51:43.535 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:51:43.535 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:51:43.535 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:51:43.535 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:51:43.536 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:51:43.536 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:51:43.536 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:51:43.536 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:51:43.537 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:51:43.537 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:51:43.537 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:51:43.538 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:51:43.538 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:51:43.538 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:51:43.539 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:51:43.539 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:51:43.539 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:51:43.539 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:51:43.540 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:51:43.540 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:51:43.540 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.541 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:51:43.541 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:51:43.541 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:51:43.541 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:51:43.542 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:51:43.542 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:51:43.542 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:51:43.542 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:51:43.543 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:51:43.543 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:51:43.543 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:51:43.543 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:51:43.543 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:51:43.544 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.544 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:51:43.544 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:51:43.544 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.545 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.545 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.545 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:51:43.545 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:51:43.546 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:51:43.546 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:51:43.546 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:51:43.546 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:51:43.547 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:51:43.547 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:51:43.547 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:51:43.547 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:51:43.548 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:51:43.548 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:51:43.548 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:51:43.548 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:51:43.548 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:51:43.549 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:51:43.549 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:51:43.549 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:51:43.549 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:51:43.550 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:51:43.550 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:51:43.550 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:51:43.550 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:51:43.550 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.551 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:51:43.551 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:51:43.551 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:51:43.551 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:51:43.552 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:51:43.552 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:51:43.552 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:51:43.552 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:51:43.552 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:51:43.553 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:51:43.553 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:51:43.553 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:51:43.553 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:51:43.554 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:51:43.554 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:51:43.554 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:51:43.554 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:51:43.555 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:51:43.555 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:51:43.555 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:51:43.555 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:51:43.556 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:51:43.556 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:51:43.556 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:51:43.556 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:51:43.556 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:51:43.557 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:51:43.557 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.557 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:51:43.557 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:51:43.558 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:51:43.558 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:51:43.558 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:51:43.558 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:51:43.558 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:51:43.559 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:51:43.559 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:51:43.559 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:51:43.559 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:51:43.560 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:51:43.560 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:51:43.560 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:51:43.560 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.560 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.561 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.561 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 7, 2022-07-21T01:51:43.561 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:51:43.561 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:50:25.939192+0000", 2022-07-21T01:51:43.562 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:51:43.562 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:51:43.562 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:51:43.562 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:51:43.562 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:51:43.563 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:51:43.563 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:51:43.563 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:51:43.564 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:51:43.564 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:51:43.564 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:51:43.564 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:51:43.565 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:51:43.565 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:51:43.565 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:51:43.565 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:51:43.566 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:51:43.566 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:51:43.566 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:51:43.566 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:51:43.567 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.567 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "41", 2022-07-21T01:51:43.567 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:51:43.567 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:51:43.568 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:51:43.568 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:51:43.568 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:51:43.568 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:51:43.569 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:51:43.569 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:51:43.569 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:51:43.569 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:51:43.570 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:51:43.570 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:51:43.570 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:51:43.570 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:51:43.571 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:51:43.571 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:51:43.571 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:51:43.571 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:51:43.572 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:51:43.572 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:51:43.572 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:51:43.572 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:51:43.573 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:51:43.573 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:51:43.573 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:51:43.573 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:51:43.573 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.574 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:51:43.574 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:51:43.574 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:51:43.574 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:51:43.575 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:51:43.575 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:51:43.575 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:51:43.575 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:51:43.576 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:51:43.576 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:51:43.576 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:51:43.577 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:51:43.577 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:51:43.577 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:51:43.578 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:51:43.578 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.579 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:51:43.579 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:51:43.580 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:51:43.580 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.581 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.581 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.582 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.582 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 8, 2022-07-21T01:51:43.583 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:51:43.583 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:50:26.391519+0000", 2022-07-21T01:51:43.584 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:51:43.584 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:51:43.585 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:51:43.585 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:51:43.585 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:51:43.586 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:51:43.586 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:51:43.587 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:51:43.587 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:51:43.587 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:51:43.588 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:51:43.588 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:51:43.589 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:51:43.589 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:51:43.589 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:51:43.590 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:51:43.590 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:51:43.590 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:51:43.591 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:51:43.591 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:51:43.591 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.591 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "42", 2022-07-21T01:51:43.592 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:51:43.592 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:51:43.592 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:51:43.592 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:51:43.593 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:51:43.593 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:51:43.593 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:51:43.593 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:51:43.594 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:51:43.594 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:51:43.594 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:51:43.594 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:51:43.595 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:51:43.595 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:51:43.595 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:51:43.595 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:51:43.595 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:51:43.596 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:51:43.596 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:51:43.596 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:51:43.596 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:51:43.597 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:51:43.597 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:51:43.597 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:51:43.597 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:51:43.598 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:51:43.598 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.598 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:51:43.598 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:51:43.598 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:51:43.599 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:51:43.599 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:51:43.599 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:51:43.599 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:51:43.600 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:51:43.600 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:51:43.600 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:51:43.600 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:51:43.601 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:51:43.601 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:51:43.601 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:51:43.601 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:51:43.601 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.602 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.602 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.602 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:51:43.602 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:51:43.603 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.603 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:51:43.603 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:51:43.603 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:43.604 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:43.604 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:43.604 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:43.604 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:43.604 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:43.605 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:43.605 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 39, 2022-07-21T01:51:43.605 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:43.605 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:43.606 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:43.606 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.606 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.606 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.607 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:51:43.607 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:43.607 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.607 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.608 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.608 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:51:43.608 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:43.608 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.609 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.609 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.609 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:43.610 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.610 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.610 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.610 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:51:43.610 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:43.611 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.611 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.611 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.611 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:51:43.612 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:43.612 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.612 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.612 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.612 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:43.613 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.613 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.613 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.613 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:51:43.614 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:43.614 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.614 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.614 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.615 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:51:43.615 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:43.615 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.616 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.616 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.616 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:43.616 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.616 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.617 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.617 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:51:43.617 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:43.618 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.618 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.618 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.618 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:51:43.619 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:43.619 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.619 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.620 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.620 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:51:43.620 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:51:43.620 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:51:43.621 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:51:43.621 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:43.621 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:43.621 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:43.622 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.622 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.622 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.622 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:51:43.623 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:51:43.623 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:43.623 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:43.624 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:43.624 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:43.624 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:43.624 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:43.625 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:43.625 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 40, 2022-07-21T01:51:43.625 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:43.625 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:43.626 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:43.626 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.626 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.626 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.627 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:51:43.627 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:43.627 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.627 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.628 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.628 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:51:43.628 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:43.628 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.629 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.629 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.629 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:43.629 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.630 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.630 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.630 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:51:43.630 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:43.631 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.631 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.631 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.631 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:51:43.632 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:43.632 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.632 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.632 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.633 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:43.633 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.633 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.633 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.633 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:51:43.634 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:43.634 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.634 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.634 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.635 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:51:43.635 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:43.635 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.635 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.635 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.636 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:43.636 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.636 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.636 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.637 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:51:43.637 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:43.637 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.637 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.638 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.638 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:51:43.638 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:43.638 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.639 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.639 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.639 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:51:43.639 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:51:43.640 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:51:43.640 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:51:43.640 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:43.640 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:43.640 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:43.641 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.641 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.641 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.641 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:51:43.642 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:51:43.642 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:43.642 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:43.642 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:43.643 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:43.643 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:43.643 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:43.643 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:43.643 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 29, 2022-07-21T01:51:43.644 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:43.644 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:43.644 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:43.644 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.645 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.645 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.645 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:51:43.645 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:43.645 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.646 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.646 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.646 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:51:43.646 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:43.647 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.647 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.647 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.647 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:43.648 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.648 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.648 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.648 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:51:43.649 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:43.649 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.649 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.649 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.650 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:51:43.650 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:43.650 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.650 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.651 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.651 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:43.651 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.651 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.652 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.652 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:51:43.652 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:43.652 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.653 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.653 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.653 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:51:43.653 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:43.653 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.654 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.654 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.654 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:43.654 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.655 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.655 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.655 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:51:43.655 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:43.656 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.656 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.656 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.656 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:51:43.656 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:43.657 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.657 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.657 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.657 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:51:43.658 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:51:43.658 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:51:43.658 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:51:43.658 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:43.658 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:43.659 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:43.659 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.659 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.659 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.660 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:51:43.660 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:51:43.660 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:43.660 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:43.660 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:43.661 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:43.661 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:43.661 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:43.661 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:43.662 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 39, 2022-07-21T01:51:43.662 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:43.662 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:43.662 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:43.663 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.663 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.663 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.663 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:51:43.664 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:43.664 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.664 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.664 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.665 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:51:43.665 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:43.665 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.666 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.666 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.666 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:43.666 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.667 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.667 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.667 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:51:43.667 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:43.667 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.668 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.668 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.668 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:51:43.668 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:43.669 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.669 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.669 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.669 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:43.670 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.670 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.670 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.670 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:51:43.670 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:43.671 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.671 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.671 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.671 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:51:43.672 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:43.672 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.672 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.672 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.672 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:43.673 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.673 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.673 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.673 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:51:43.674 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:43.674 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.674 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.674 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.675 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:51:43.675 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:43.675 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.675 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.675 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.676 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:51:43.676 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:51:43.676 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:51:43.676 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:51:43.677 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:43.677 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:43.677 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:43.677 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.677 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.678 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.678 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:51:43.678 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:51:43.678 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:43.679 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:43.679 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:43.679 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:43.679 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:43.680 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:43.680 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:43.680 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 39, 2022-07-21T01:51:43.680 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:43.680 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:43.681 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:43.681 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.681 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.681 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.681 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:51:43.682 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:43.682 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.682 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.682 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.683 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:51:43.683 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:43.683 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.683 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.684 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.684 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:43.684 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.684 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.684 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.685 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:51:43.685 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:43.685 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.685 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.686 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.686 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:51:43.686 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:43.687 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.687 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.687 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.687 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:43.687 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.688 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.688 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.688 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:51:43.688 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:43.689 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.689 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.689 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.689 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:51:43.690 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:43.690 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.690 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.690 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.691 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:43.691 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.691 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.691 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.692 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:51:43.692 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:43.692 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.692 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.692 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.693 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:51:43.693 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:43.693 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.693 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.694 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.694 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:51:43.694 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:51:43.694 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:51:43.694 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:51:43.695 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:43.695 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:43.695 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:43.695 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.696 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.696 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.696 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:51:43.696 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:51:43.696 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:43.697 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:43.697 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:43.697 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:43.697 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:43.698 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:43.698 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:43.698 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 39, 2022-07-21T01:51:43.698 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:43.699 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:43.699 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:43.699 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.699 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.700 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.700 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:51:43.700 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:43.700 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.701 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.701 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.701 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:51:43.701 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:43.702 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.702 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.702 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.703 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:43.703 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.703 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.703 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.704 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:51:43.704 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:43.704 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.704 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.705 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.705 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:51:43.705 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:43.705 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.706 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.706 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.706 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:43.706 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.706 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.707 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.707 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:51:43.707 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:43.707 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.708 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.708 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.708 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:51:43.708 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:43.708 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.709 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.709 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.709 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:43.709 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.709 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.710 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.710 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:51:43.710 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:43.710 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.711 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.711 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.711 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:51:43.711 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:43.712 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.712 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.712 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.712 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:51:43.712 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:51:43.713 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:51:43.713 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:51:43.713 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:43.713 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:43.714 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:43.714 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.714 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.714 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.715 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:51:43.715 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:51:43.715 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:43.715 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:43.715 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:43.716 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:43.716 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:43.716 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:43.716 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:43.717 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 39, 2022-07-21T01:51:43.717 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:43.717 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:43.717 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:43.718 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.718 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.718 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.718 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:51:43.718 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:43.719 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.719 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.719 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.719 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:51:43.720 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:43.720 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.720 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.720 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.720 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:43.721 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.721 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.721 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.721 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:51:43.722 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:43.722 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.722 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.722 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.722 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:51:43.723 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:43.723 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.723 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.724 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.724 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:43.724 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.724 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.725 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.725 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:51:43.725 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:43.725 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.725 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.726 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.726 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:51:43.726 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:43.726 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.726 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.727 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.727 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:43.727 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.727 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.728 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.728 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:51:43.728 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:43.728 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.728 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.729 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.729 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:51:43.729 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:43.729 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.730 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.730 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.730 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:51:43.730 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:51:43.731 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:51:43.731 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:51:43.731 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:43.731 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:43.732 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:43.732 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.732 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.732 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.732 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:51:43.733 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:51:43.733 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:43.733 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:43.734 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:43.734 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:43.734 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:43.734 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:43.735 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:43.735 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 38, 2022-07-21T01:51:43.735 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:43.735 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:43.736 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:43.736 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.736 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.736 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.737 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:51:43.737 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:43.737 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.737 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.738 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.738 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:51:43.738 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:43.739 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.739 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.739 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.739 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:43.740 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.740 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.740 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.740 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:51:43.741 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:43.741 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.741 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.741 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.742 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:51:43.742 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:43.742 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.742 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.743 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.743 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:43.743 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.744 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.744 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.744 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:51:43.744 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:43.745 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.745 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.745 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.745 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:51:43.746 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:43.746 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.746 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.746 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.747 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:43.747 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:43.747 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.748 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:43.748 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:51:43.748 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:43.748 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.749 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.749 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:43.749 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:51:43.749 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:43.750 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.750 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.750 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.751 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:51:43.751 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:51:43.751 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:51:43.751 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:51:43.752 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:43.752 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:43.752 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:43.752 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:43.753 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.753 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:51:43.753 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:51:43.754 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.754 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:51:43.754 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:43.754 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:43.755 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:43.755 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:43.755 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:43.756 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:51:43.756 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:43.756 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.756 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.757 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:51:43.757 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:43.757 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:43.757 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:43.758 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:43.758 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:43.758 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:51:43.758 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:43.759 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.759 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.759 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:51:43.760 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:43.760 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:43.760 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:43.760 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:43.761 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:43.761 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:51:43.761 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:43.761 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.762 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.762 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:51:43.762 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:43.762 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:43.763 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:43.763 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:43.763 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:43.763 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:51:43.764 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:43.764 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.764 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.765 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:51:43.765 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:43.765 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:43.765 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:43.766 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:43.766 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:43.766 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:51:43.766 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:43.767 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.767 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.767 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:51:43.767 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:43.768 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:43.768 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:43.768 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:43.768 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:43.769 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:51:43.769 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:43.769 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.769 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.770 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:51:43.770 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:43.770 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:43.770 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:43.771 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:43.771 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:43.771 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:51:43.771 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:43.772 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.772 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:43.772 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:51:43.773 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:43.773 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:43.773 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:43.773 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:43.774 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:43.774 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:51:43.774 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:43.774 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.775 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:51:43.775 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:51:43.775 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:51:43.775 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:51:43.776 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:51:43.776 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": {}, 2022-07-21T01:51:43.776 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:51:43.776 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:51:43.777 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:51:43.777 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:51:43.777 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:51:43.777 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:51:43.778 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:51:43.778 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:51:43.778 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:43.778 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:43.779 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:51:43.779 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:51:43.779 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:51:43.779 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:51:43.780 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:51:43.780 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:51:43.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 -- 172.21.15.140:0/366403492 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcab0053980 msgr2=0x7fcab0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:43.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcab0053980 0x7fcab0055e40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fcac801d000 tx=0x7fcac8019000).stop 2022-07-21T01:51:43.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 -- 172.21.15.140:0/366403492 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcad8152790 msgr2=0x7fcad8152c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:43.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcad8152790 0x7fcad8152c10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fcad813ef30 tx=0x7fcad0009040).stop 2022-07-21T01:51:43.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 -- 172.21.15.140:0/366403492 shutdown_connections 2022-07-21T01:51:43.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcab0053980 0x7fcab0055e40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcad8152790 0x7fcad8152c10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcad813e800 0x7fcad813d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 --2- 172.21.15.140:0/366403492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcabc003750 0x7fcad8151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 -- 172.21.15.140:0/366403492 >> 172.21.15.140:0/366403492 conn(0x7fcad80634e0 msgr2=0x7fcad80b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:43.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 -- 172.21.15.140:0/366403492 shutdown_connections 2022-07-21T01:51:43.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.515+0000 7fcadd311700 1 -- 172.21.15.140:0/366403492 wait complete. 2022-07-21T01:51:43.786 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:51:43.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.895+0000 7f75b6781700 1 Processor -- start 2022-07-21T01:51:43.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.895+0000 7f75b6781700 1 -- start start 2022-07-21T01:51:43.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75b00b1e00 0x7f75b014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f75b005c3f0 0x7f75b00ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f75b005bd90 0x7f75b00accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b005d480 con 0x7f75b005c3f0 2022-07-21T01:51:43.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b00ad1c0 con 0x7f75b005bd90 2022-07-21T01:51:43.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00b21e0 con 0x7f75b00b1e00 2022-07-21T01:51:43.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75af7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f75b005c3f0 0x7f75b00ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75af7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f75b005c3f0 0x7f75b00ac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34802/0 (socket says 172.21.15.140:34802) 2022-07-21T01:51:43.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75af7fe700 1 -- 172.21.15.140:0/2181236914 learned_addr learned my addr 172.21.15.140:0/2181236914 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:43.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75affff700 1 --2- 172.21.15.140:0/2181236914 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75b00b1e00 0x7f75b014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b4d1e700 1 --2- 172.21.15.140:0/2181236914 >> v2:172.21.15.152:3300/0 conn(0x7f75b005bd90 0x7f75b00accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75af7fe700 1 -- 172.21.15.140:0/2181236914 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75b00b1e00 msgr2=0x7f75b014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:43.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75af7fe700 1 --2- 172.21.15.140:0/2181236914 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75b00b1e00 0x7f75b014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75af7fe700 1 -- 172.21.15.140:0/2181236914 >> v2:172.21.15.152:3300/0 conn(0x7f75b005bd90 msgr2=0x7f75b00accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:43.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75af7fe700 1 --2- 172.21.15.140:0/2181236914 >> v2:172.21.15.152:3300/0 conn(0x7f75b005bd90 0x7f75b00accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75af7fe700 1 -- 172.21.15.140:0/2181236914 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b005cf40 con 0x7f75b005c3f0 2022-07-21T01:51:43.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75affff700 1 --2- 172.21.15.140:0/2181236914 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75b00b1e00 0x7f75b014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:51:43.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75af7fe700 1 --2- 172.21.15.140:0/2181236914 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f75b005c3f0 0x7f75b00ac7e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f7598002f70 tx=0x7f7598006eb0).ready entity=mon.0 client_cookie=4efc5df9f90b6fe2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:43.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75aeffd700 1 -- 172.21.15.140:0/2181236914 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f759800d570 con 0x7f75b005c3f0 2022-07-21T01:51:43.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75aeffd700 1 -- 172.21.15.140:0/2181236914 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598000e90 con 0x7f75b005c3f0 2022-07-21T01:51:43.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75aeffd700 1 -- 172.21.15.140:0/2181236914 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f759800d900 con 0x7f75b005c3f0 2022-07-21T01:51:43.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 -- 172.21.15.140:0/2181236914 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f75b005c3f0 msgr2=0x7f75b00ac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:43.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 --2- 172.21.15.140:0/2181236914 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f75b005c3f0 0x7f75b00ac7e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f7598002f70 tx=0x7f7598006eb0).stop 2022-07-21T01:51:43.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 -- 172.21.15.140:0/2181236914 shutdown_connections 2022-07-21T01:51:43.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 --2- 172.21.15.140:0/2181236914 >> v2:172.21.15.152:3300/0 conn(0x7f75b005bd90 0x7f75b00accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 --2- 172.21.15.140:0/2181236914 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f75b005c3f0 0x7f75b00ac7e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 --2- 172.21.15.140:0/2181236914 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75b00b1e00 0x7f75b014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 -- 172.21.15.140:0/2181236914 >> 172.21.15.140:0/2181236914 conn(0x7f75b00611c0 msgr2=0x7f75b0062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:43.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 -- 172.21.15.140:0/2181236914 shutdown_connections 2022-07-21T01:51:43.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 -- 172.21.15.140:0/2181236914 wait complete. 2022-07-21T01:51:43.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.899+0000 7f75b6781700 1 Processor -- start 2022-07-21T01:51:43.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75b6781700 1 -- start start 2022-07-21T01:51:43.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75b6781700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f75b005c3f0 0x7f75b01570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75b6781700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75b00b1e00 0x7f75b00a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75b6781700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75b0159c00 0x7f75b015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75b6781700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00b51c0 con 0x7f75b005c3f0 2022-07-21T01:51:43.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75b6781700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00b5400 con 0x7f75b00b1e00 2022-07-21T01:51:43.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75b6781700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00b55f0 con 0x7f75b0159c00 2022-07-21T01:51:43.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75affff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f75b005c3f0 0x7f75b01570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75affff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f75b005c3f0 0x7f75b01570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34806/0 (socket says 172.21.15.140:34806) 2022-07-21T01:51:43.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75affff700 1 -- 172.21.15.140:0/4074705756 learned_addr learned my addr 172.21.15.140:0/4074705756 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:43.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75af7fe700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75b00b1e00 0x7f75b00a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75b4d1e700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75b0159c00 0x7f75b015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75affff700 1 -- 172.21.15.140:0/4074705756 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75b0159c00 msgr2=0x7f75b015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:43.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75affff700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75b0159c00 0x7f75b015a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75affff700 1 -- 172.21.15.140:0/4074705756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75b00b1e00 msgr2=0x7f75b00a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:43.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75affff700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75b00b1e00 0x7f75b00a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:43.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75affff700 1 -- 172.21.15.140:0/4074705756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a8009470 con 0x7f75b005c3f0 2022-07-21T01:51:43.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75affff700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f75b005c3f0 0x7f75b01570b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f75a800c000 tx=0x7f75a80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:43.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75ad7fa700 1 -- 172.21.15.140:0/4074705756 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75a800c920 con 0x7f75b005c3f0 2022-07-21T01:51:43.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75ad7fa700 1 -- 172.21.15.140:0/4074705756 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a800ca80 con 0x7f75b005c3f0 2022-07-21T01:51:43.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75b6781700 1 -- 172.21.15.140:0/4074705756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759800b1e0 con 0x7f75b005c3f0 2022-07-21T01:51:43.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75ad7fa700 1 -- 172.21.15.140:0/4074705756 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75a800cde0 con 0x7f75b005c3f0 2022-07-21T01:51:43.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.903+0000 7f75b6781700 1 -- 172.21.15.140:0/4074705756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b00a12e0 con 0x7f75b005c3f0 2022-07-21T01:51:43.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.907+0000 7f75ad7fa700 1 -- 172.21.15.140:0/4074705756 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f75a801b400 con 0x7f75b005c3f0 2022-07-21T01:51:43.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.907+0000 7f75b6781700 1 -- 172.21.15.140:0/4074705756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b005d1e0 con 0x7f75b005c3f0 2022-07-21T01:51:43.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.907+0000 7f75ad7fa700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7594053aa0 0x7f7594055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:43.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.907+0000 7f75ad7fa700 1 -- 172.21.15.140:0/4074705756 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f75a806ca70 con 0x7f75b005c3f0 2022-07-21T01:51:43.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.907+0000 7f75af7fe700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7594053aa0 0x7f7594055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:43.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.907+0000 7f75af7fe700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7594053aa0 0x7f7594055f60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f759801c000 tx=0x7f7598015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:43.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:43.915+0000 7f75ad7fa700 1 -- 172.21.15.140:0/4074705756 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f75a8043820 con 0x7f75b005c3f0 2022-07-21T01:51:44.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.079+0000 7f75b6781700 1 -- 172.21.15.140:0/4074705756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75b00574e0 con 0x7f75b005c3f0 2022-07-21T01:51:44.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.079+0000 7f75ad7fa700 1 -- 172.21.15.140:0/4074705756 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2141 (secure 0 0 0) 0x7f75a8046e40 con 0x7f75b005c3f0 2022-07-21T01:51:44.079 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:44.079 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/2307860244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2307860244},{"type":"v1","addr":"172.21.15.152:6833","nonce":2307860244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:50:27.322541+0000","modified":"2022-07-21T01:50:28.325625+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5247":{"gid":5247,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/207391195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":207391195},{"type":"v1","addr":"172.21.15.140:6835","nonce":207391195}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-07-21T01:51:44.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.079+0000 7f75b6781700 1 -- 172.21.15.140:0/4074705756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7594053aa0 msgr2=0x7f7594055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:44.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.079+0000 7f75b6781700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7594053aa0 0x7f7594055f60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f759801c000 tx=0x7f7598015000).stop 2022-07-21T01:51:44.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.079+0000 7f75b6781700 1 -- 172.21.15.140:0/4074705756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f75b005c3f0 msgr2=0x7f75b01570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:44.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.079+0000 7f75b6781700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f75b005c3f0 0x7f75b01570b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f75a800c000 tx=0x7f75a80091d0).stop 2022-07-21T01:51:44.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.083+0000 7f75b6781700 1 -- 172.21.15.140:0/4074705756 shutdown_connections 2022-07-21T01:51:44.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.083+0000 7f75b6781700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7594053aa0 0x7f7594055f60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:44.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.083+0000 7f75b6781700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75b0159c00 0x7f75b015a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:44.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.083+0000 7f75b6781700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75b00b1e00 0x7f75b00a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:44.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.083+0000 7f75b6781700 1 --2- 172.21.15.140:0/4074705756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f75b005c3f0 0x7f75b01570b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:44.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.083+0000 7f75b6781700 1 -- 172.21.15.140:0/4074705756 >> 172.21.15.140:0/4074705756 conn(0x7f75b005bd90 msgr2=0x7f75b00ac700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:44.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.083+0000 7f75b6781700 1 -- 172.21.15.140:0/4074705756 shutdown_connections 2022-07-21T01:51:44.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.083+0000 7f75b6781700 1 -- 172.21.15.140:0/4074705756 wait complete. 2022-07-21T01:51:44.085 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 16 2022-07-21T01:51:44.092 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:51:44.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53de2e9700 1 Processor -- start 2022-07-21T01:51:44.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53de2e9700 1 -- start start 2022-07-21T01:51:44.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53de2e9700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f53d80b5990 0x7f53d8146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:44.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53de2e9700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d8146c70 con 0x7f53d80af3e0 2022-07-21T01:51:44.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53de2e9700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d80b5d70 con 0x7f53d805d210 2022-07-21T01:51:44.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53de2e9700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d80ad050 con 0x7f53d80b5990 2022-07-21T01:51:44.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53dcae6700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f53d80af3e0 0x7f53d8146560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48962/0 (socket says 172.21.15.140:48962) 2022-07-21T01:51:44.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53dcae6700 1 -- 172.21.15.140:0/84915460 learned_addr learned my addr 172.21.15.140:0/84915460 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:44.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53dd2e7700 1 --2- 172.21.15.140:0/84915460 >> v2:172.21.15.152:3300/0 conn(0x7f53d80b5990 0x7f53d8146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:44.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53d7fff700 1 -- 172.21.15.140:0/84915460 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485617033 0 0) 0x7f53d8146c70 con 0x7f53d80af3e0 2022-07-21T01:51:44.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53d7fff700 1 -- 172.21.15.140:0/84915460 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b8003260 con 0x7f53d80af3e0 2022-07-21T01:51:44.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53d7fff700 1 -- 172.21.15.140:0/84915460 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236038472 0 0) 0x7f53d80b5d70 con 0x7f53d805d210 2022-07-21T01:51:44.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53d7fff700 1 -- 172.21.15.140:0/84915460 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d8146c70 con 0x7f53d805d210 2022-07-21T01:51:44.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53dd2e7700 1 -- 172.21.15.140:0/84915460 >> v1:172.21.15.152:6789/0 conn(0x7f53d805d210 legacy=0x7f53d814a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:44.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53dd2e7700 1 -- 172.21.15.140:0/84915460 >> v1:172.21.15.140:6789/0 conn(0x7f53d80af3e0 legacy=0x7f53d8146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:44.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53dd2e7700 1 -- 172.21.15.140:0/84915460 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d80b1010 con 0x7f53d80b5990 2022-07-21T01:51:44.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53dd2e7700 1 --2- 172.21.15.140:0/84915460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d80b5990 0x7f53d8146070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f53c00032f0 tx=0x7f53c00091a0).ready entity=mon.1 client_cookie=65465ecf48dd04f6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:44.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53d7fff700 1 -- 172.21.15.140:0/84915460 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53c000b600 con 0x7f53d80b5990 2022-07-21T01:51:44.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53d7fff700 1 -- 172.21.15.140:0/84915460 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c000b760 con 0x7f53d80b5990 2022-07-21T01:51:44.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.199+0000 7f53d7fff700 1 -- 172.21.15.140:0/84915460 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53c000b8d0 con 0x7f53d80b5990 2022-07-21T01:51:44.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.203+0000 7f53de2e9700 1 -- 172.21.15.140:0/84915460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d80b5990 msgr2=0x7f53d8146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:44.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.203+0000 7f53de2e9700 1 --2- 172.21.15.140:0/84915460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d80b5990 0x7f53d8146070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f53c00032f0 tx=0x7f53c00091a0).stop 2022-07-21T01:51:44.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.203+0000 7f53de2e9700 1 -- 172.21.15.140:0/84915460 shutdown_connections 2022-07-21T01:51:44.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.203+0000 7f53de2e9700 1 --2- 172.21.15.140:0/84915460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d80b5990 0x7f53d8146070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:44.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.203+0000 7f53de2e9700 1 -- 172.21.15.140:0/84915460 >> 172.21.15.140:0/84915460 conn(0x7f53d8063480 msgr2=0x7f53d80b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:44.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.203+0000 7f53de2e9700 1 -- 172.21.15.140:0/84915460 shutdown_connections 2022-07-21T01:51:44.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.203+0000 7f53de2e9700 1 -- 172.21.15.140:0/84915460 wait complete. 2022-07-21T01:51:44.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.203+0000 7f53de2e9700 1 Processor -- start 2022-07-21T01:51:44.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.203+0000 7f53de2e9700 1 -- start start 2022-07-21T01:51:44.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53de2e9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f53d805d210 0x7f53d80b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:44.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53de2e9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f53d80af3e0 0x7f53d8156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:44.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53de2e9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d80b5990 0x7f53d8159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:44.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53de2e9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8156440 con 0x7f53d805d210 2022-07-21T01:51:44.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53de2e9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8066a70 con 0x7f53d80b5990 2022-07-21T01:51:44.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53de2e9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8156670 con 0x7f53d80af3e0 2022-07-21T01:51:44.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53dd2e7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f53d805d210 0x7f53d80b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:44.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53dd2e7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f53d805d210 0x7f53d80b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34820/0 (socket says 172.21.15.140:34820) 2022-07-21T01:51:44.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53dd2e7700 1 -- 172.21.15.140:0/4249886300 learned_addr learned my addr 172.21.15.140:0/4249886300 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:44.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53dcae6700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f53d80af3e0 0x7f53d8156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:44.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53ddae8700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d80b5990 0x7f53d8159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:44.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53dd2e7700 1 -- 172.21.15.140:0/4249886300 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f53d80af3e0 msgr2=0x7f53d8156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:44.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53dd2e7700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f53d80af3e0 0x7f53d8156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:44.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53dd2e7700 1 -- 172.21.15.140:0/4249886300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d80b5990 msgr2=0x7f53d8159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:44.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53dd2e7700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d80b5990 0x7f53d8159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:44.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53dd2e7700 1 -- 172.21.15.140:0/4249886300 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0009420 con 0x7f53d805d210 2022-07-21T01:51:44.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53dd2e7700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f53d805d210 0x7f53d80b4080 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f53c00032f0 tx=0x7f53c0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:44.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53d67fc700 1 -- 172.21.15.140:0/4249886300 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53c000b600 con 0x7f53d805d210 2022-07-21T01:51:44.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53d67fc700 1 -- 172.21.15.140:0/4249886300 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c0021670 con 0x7f53d805d210 2022-07-21T01:51:44.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53de2e9700 1 -- 172.21.15.140:0/4249886300 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d80b4570 con 0x7f53d805d210 2022-07-21T01:51:44.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53d67fc700 1 -- 172.21.15.140:0/4249886300 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f53c00219d0 con 0x7f53d805d210 2022-07-21T01:51:44.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.207+0000 7f53de2e9700 1 -- 172.21.15.140:0/4249886300 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d8157000 con 0x7f53d805d210 2022-07-21T01:51:44.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.211+0000 7f53de2e9700 1 -- 172.21.15.140:0/4249886300 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d81572b0 con 0x7f53d805d210 2022-07-21T01:51:44.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.211+0000 7f53d67fc700 1 -- 172.21.15.140:0/4249886300 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f53c000f070 con 0x7f53d805d210 2022-07-21T01:51:44.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.211+0000 7f53d67fc700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f53bc0539e0 0x7f53bc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:44.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.211+0000 7f53d67fc700 1 -- 172.21.15.140:0/4249886300 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f53c006d1e0 con 0x7f53d805d210 2022-07-21T01:51:44.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.211+0000 7f53dcae6700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f53bc0539e0 0x7f53bc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:44.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.211+0000 7f53dcae6700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f53bc0539e0 0x7f53bc055ea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f53c800d000 tx=0x7f53c8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:44.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.215+0000 7f53d67fc700 1 -- 172.21.15.140:0/4249886300 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f53c0044080 con 0x7f53d805d210 2022-07-21T01:51:44.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:44.375+0000 7f53de2e9700 1 -- 172.21.15.140:0/4249886300 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f53d80afa60 con 0x7f53d805d210 2022-07-21T01:51:45.020 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:51:45.023+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:51:45.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.039+0000 7f53d67fc700 1 -- 172.21.15.140:0/4249886300 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f53c0015020 con 0x7f53d805d210 2022-07-21T01:51:45.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 -- 172.21.15.140:0/4249886300 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f53bc0539e0 msgr2=0x7f53bc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f53bc0539e0 0x7f53bc055ea0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f53c800d000 tx=0x7f53c8009040).stop 2022-07-21T01:51:45.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 -- 172.21.15.140:0/4249886300 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f53d805d210 msgr2=0x7f53d80b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f53d805d210 0x7f53d80b4080 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f53c00032f0 tx=0x7f53c0016000).stop 2022-07-21T01:51:45.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 -- 172.21.15.140:0/4249886300 shutdown_connections 2022-07-21T01:51:45.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f53bc0539e0 0x7f53bc055ea0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f53d80b5990 0x7f53d8159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f53d80af3e0 0x7f53d8156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 --2- 172.21.15.140:0/4249886300 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f53d805d210 0x7f53d80b4080 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 -- 172.21.15.140:0/4249886300 >> 172.21.15.140:0/4249886300 conn(0x7f53d8063480 msgr2=0x7f53d81453f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:45.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 -- 172.21.15.140:0/4249886300 shutdown_connections 2022-07-21T01:51:45.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.043+0000 7f53de2e9700 1 -- 172.21.15.140:0/4249886300 wait complete. 2022-07-21T01:51:45.045 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:51:45.058 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:51:45.078 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:51:45.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.163+0000 7f7133371700 1 Processor -- start 2022-07-21T01:51:45.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.163+0000 7f7133371700 1 -- start start 2022-07-21T01:51:45.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f712c0b18d0 0x7f712c143710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f712c13ec50 0x7f712c13f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712c0afac0 con 0x7f712c0b61d0 2022-07-21T01:51:45.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712c0add80 con 0x7f712c0b18d0 2022-07-21T01:51:45.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712c0b65b0 con 0x7f712c13ec50 2022-07-21T01:51:45.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f713236f700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f712c0b61d0 0x7f712c143000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:48986/0 (socket says 172.21.15.140:48986) 2022-07-21T01:51:45.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f713236f700 1 -- 172.21.15.140:0/4255861332 learned_addr learned my addr 172.21.15.140:0/4255861332 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:45.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7131b6e700 1 --2- 172.21.15.140:0/4255861332 >> v2:172.21.15.140:3300/0 conn(0x7f712c0b18d0 0x7f712c143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7132b70700 1 --2- 172.21.15.140:0/4255861332 >> v2:172.21.15.152:3300/0 conn(0x7f712c13ec50 0x7f712c13f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f713136d700 1 -- 172.21.15.140:0/4255861332 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699070363 0 0) 0x7f712c0afac0 con 0x7f712c0b61d0 2022-07-21T01:51:45.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f713136d700 1 -- 172.21.15.140:0/4255861332 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7114003260 con 0x7f712c0b61d0 2022-07-21T01:51:45.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7131b6e700 1 -- 172.21.15.140:0/4255861332 >> v2:172.21.15.152:3300/0 conn(0x7f712c13ec50 msgr2=0x7f712c13f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:45.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7131b6e700 1 --2- 172.21.15.140:0/4255861332 >> v2:172.21.15.152:3300/0 conn(0x7f712c13ec50 0x7f712c13f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7131b6e700 1 -- 172.21.15.140:0/4255861332 >> v1:172.21.15.140:6789/0 conn(0x7f712c0b61d0 legacy=0x7f712c143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7131b6e700 1 -- 172.21.15.140:0/4255861332 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c05c3e0 con 0x7f712c0b18d0 2022-07-21T01:51:45.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7131b6e700 1 --2- 172.21.15.140:0/4255861332 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f712c0b18d0 0x7f712c143710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f711c002f70 tx=0x7f711c006df0).ready entity=mon.0 client_cookie=bdb82dd416b38b2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:45.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f713136d700 1 -- 172.21.15.140:0/4255861332 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f711c00d4f0 con 0x7f712c0b18d0 2022-07-21T01:51:45.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f713136d700 1 -- 172.21.15.140:0/4255861332 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711c000e90 con 0x7f712c0b18d0 2022-07-21T01:51:45.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f713136d700 1 -- 172.21.15.140:0/4255861332 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f711c00d840 con 0x7f712c0b18d0 2022-07-21T01:51:45.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 -- 172.21.15.140:0/4255861332 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f712c0b18d0 msgr2=0x7f712c143710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 --2- 172.21.15.140:0/4255861332 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f712c0b18d0 0x7f712c143710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f711c002f70 tx=0x7f711c006df0).stop 2022-07-21T01:51:45.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 -- 172.21.15.140:0/4255861332 shutdown_connections 2022-07-21T01:51:45.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 --2- 172.21.15.140:0/4255861332 >> v2:172.21.15.152:3300/0 conn(0x7f712c13ec50 0x7f712c13f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 --2- 172.21.15.140:0/4255861332 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f712c0b18d0 0x7f712c143710 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 -- 172.21.15.140:0/4255861332 >> 172.21.15.140:0/4255861332 conn(0x7f712c063cf0 msgr2=0x7f712c0b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:45.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 -- 172.21.15.140:0/4255861332 shutdown_connections 2022-07-21T01:51:45.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.167+0000 7f7133371700 1 -- 172.21.15.140:0/4255861332 wait complete. 2022-07-21T01:51:45.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7133371700 1 Processor -- start 2022-07-21T01:51:45.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7133371700 1 -- start start 2022-07-21T01:51:45.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7133371700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f712c0b18d0 0x7f712c066700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7133371700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f712c0b61d0 0x7f712c13d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7133371700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f712c13ec50 0x7f712c13dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7133371700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c066bf0 con 0x7f712c0b18d0 2022-07-21T01:51:45.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7133371700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0b5330 con 0x7f712c13ec50 2022-07-21T01:51:45.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7133371700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c066e70 con 0x7f712c0b61d0 2022-07-21T01:51:45.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f713236f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f712c0b18d0 0x7f712c066700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f713236f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f712c0b18d0 0x7f712c066700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34846/0 (socket says 172.21.15.140:34846) 2022-07-21T01:51:45.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f713236f700 1 -- 172.21.15.140:0/4162683813 learned_addr learned my addr 172.21.15.140:0/4162683813 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:45.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7131b6e700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f712c0b61d0 0x7f712c13d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7132b70700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f712c13ec50 0x7f712c13dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f713236f700 1 -- 172.21.15.140:0/4162683813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f712c0b61d0 msgr2=0x7f712c13d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f713236f700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f712c0b61d0 0x7f712c13d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f713236f700 1 -- 172.21.15.140:0/4162683813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f712c13ec50 msgr2=0x7f712c13dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f713236f700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f712c13ec50 0x7f712c13dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f713236f700 1 -- 172.21.15.140:0/4162683813 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c00b180 con 0x7f712c0b18d0 2022-07-21T01:51:45.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f713236f700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f712c0b18d0 0x7f712c066700 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f7124001490 tx=0x7f7124007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:45.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f711b7fe700 1 -- 172.21.15.140:0/4162683813 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7124005ba0 con 0x7f712c0b18d0 2022-07-21T01:51:45.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f711b7fe700 1 -- 172.21.15.140:0/4162683813 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7124005d00 con 0x7f712c0b18d0 2022-07-21T01:51:45.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7133371700 1 -- 172.21.15.140:0/4162683813 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c142b70 con 0x7f712c0b18d0 2022-07-21T01:51:45.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f711b7fe700 1 -- 172.21.15.140:0/4162683813 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f71240072c0 con 0x7f712c0b18d0 2022-07-21T01:51:45.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.171+0000 7f7133371700 1 -- 172.21.15.140:0/4162683813 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c13dfa0 con 0x7f712c0b18d0 2022-07-21T01:51:45.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.175+0000 7f7133371700 1 -- 172.21.15.140:0/4162683813 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c0b54f0 con 0x7f712c0b18d0 2022-07-21T01:51:45.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.175+0000 7f711b7fe700 1 -- 172.21.15.140:0/4162683813 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7124010070 con 0x7f712c0b18d0 2022-07-21T01:51:45.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.175+0000 7f711b7fe700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f710c053b00 0x7f710c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.175+0000 7f711b7fe700 1 -- 172.21.15.140:0/4162683813 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f712406bad0 con 0x7f712c0b18d0 2022-07-21T01:51:45.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.175+0000 7f7131b6e700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f710c053b00 0x7f710c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.175+0000 7f7131b6e700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f710c053b00 0x7f710c055fc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f711c01a000 tx=0x7f711c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:45.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.179+0000 7f711b7fe700 1 -- 172.21.15.140:0/4162683813 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7124042900 con 0x7f712c0b18d0 2022-07-21T01:51:45.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.339+0000 7f7133371700 1 -- 172.21.15.140:0/4162683813 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f712c0679b0 con 0x7f712c0b18d0 2022-07-21T01:51:45.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.339+0000 7f711b7fe700 1 -- 172.21.15.140:0/4162683813 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+1780 (secure 0 0 0) 0x7f7124045f20 con 0x7f712c0b18d0 2022-07-21T01:51:45.340 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:45.340 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/2307860244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2307860244},{"type":"v1","addr":"172.21.15.152:6833","nonce":2307860244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:50:27.322541+0000","modified":"2022-07-21T01:51:45.024818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2022-07-21T01:51:45.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 -- 172.21.15.140:0/4162683813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f710c053b00 msgr2=0x7f710c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f710c053b00 0x7f710c055fc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f711c01a000 tx=0x7f711c014000).stop 2022-07-21T01:51:45.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 -- 172.21.15.140:0/4162683813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f712c0b18d0 msgr2=0x7f712c066700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f712c0b18d0 0x7f712c066700 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f7124001490 tx=0x7f7124007e20).stop 2022-07-21T01:51:45.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 -- 172.21.15.140:0/4162683813 shutdown_connections 2022-07-21T01:51:45.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f710c053b00 0x7f710c055fc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f712c13ec50 0x7f712c13dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f712c0b61d0 0x7f712c13d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 --2- 172.21.15.140:0/4162683813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f712c0b18d0 0x7f712c066700 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 -- 172.21.15.140:0/4162683813 >> 172.21.15.140:0/4162683813 conn(0x7f712c063cf0 msgr2=0x7f712c13fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:45.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 -- 172.21.15.140:0/4162683813 shutdown_connections 2022-07-21T01:51:45.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.343+0000 7f7133371700 1 -- 172.21.15.140:0/4162683813 wait complete. 2022-07-21T01:51:45.347 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 17 2022-07-21T01:51:45.353 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:51:45.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.459+0000 7f1d52822700 1 Processor -- start 2022-07-21T01:51:45.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.459+0000 7f1d52822700 1 -- start start 2022-07-21T01:51:45.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.459+0000 7f1d52822700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1d4c0afec0 0x7f1d4c0b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.459+0000 7f1d52822700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d4c05bd90 0x7f1d4c099870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.459+0000 7f1d52822700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4c05c220 con 0x7f1d4c099d60 2022-07-21T01:51:45.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.459+0000 7f1d52822700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4c09ae60 con 0x7f1d4c0afec0 2022-07-21T01:51:45.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.459+0000 7f1d52822700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4c0b02a0 con 0x7f1d4c05bd90 2022-07-21T01:51:45.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1d4c0afec0 0x7f1d4c0b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d50dbf700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1d4c099d60 0x7f1d4c144250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49004/0 (socket says 172.21.15.140:49004) 2022-07-21T01:51:45.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d50dbf700 1 -- 172.21.15.140:0/1242043250 learned_addr learned my addr 172.21.15.140:0/1242043250 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:45.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4b7fe700 1 --2- 172.21.15.140:0/1242043250 >> v2:172.21.15.152:3300/0 conn(0x7f1d4c05bd90 0x7f1d4c099870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4affd700 1 -- 172.21.15.140:0/1242043250 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335254409 0 0) 0x7f1d4c05c220 con 0x7f1d4c099d60 2022-07-21T01:51:45.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4affd700 1 -- 172.21.15.140:0/1242043250 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d2c003260 con 0x7f1d4c099d60 2022-07-21T01:51:45.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4bfff700 1 -- 172.21.15.140:0/1242043250 >> v2:172.21.15.152:3300/0 conn(0x7f1d4c05bd90 msgr2=0x7f1d4c099870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:45.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4bfff700 1 --2- 172.21.15.140:0/1242043250 >> v2:172.21.15.152:3300/0 conn(0x7f1d4c05bd90 0x7f1d4c099870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4bfff700 1 -- 172.21.15.140:0/1242043250 >> v1:172.21.15.140:6789/0 conn(0x7f1d4c099d60 legacy=0x7f1d4c144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4bfff700 1 -- 172.21.15.140:0/1242043250 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c09a1b0 con 0x7f1d4c0afec0 2022-07-21T01:51:45.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4bfff700 1 --2- 172.21.15.140:0/1242043250 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d4c0afec0 0x7f1d4c0b26c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f1d34002f70 tx=0x7f1d34006df0).ready entity=mon.0 client_cookie=a153b25df67a1a32 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:45.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4affd700 1 -- 172.21.15.140:0/1242043250 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d3400d4f0 con 0x7f1d4c0afec0 2022-07-21T01:51:45.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4affd700 1 -- 172.21.15.140:0/1242043250 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34000e90 con 0x7f1d4c0afec0 2022-07-21T01:51:45.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d4affd700 1 -- 172.21.15.140:0/1242043250 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d3400d840 con 0x7f1d4c0afec0 2022-07-21T01:51:45.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 -- 172.21.15.140:0/1242043250 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d4c0afec0 msgr2=0x7f1d4c0b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 --2- 172.21.15.140:0/1242043250 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d4c0afec0 0x7f1d4c0b26c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f1d34002f70 tx=0x7f1d34006df0).stop 2022-07-21T01:51:45.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 -- 172.21.15.140:0/1242043250 shutdown_connections 2022-07-21T01:51:45.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 --2- 172.21.15.140:0/1242043250 >> v2:172.21.15.152:3300/0 conn(0x7f1d4c05bd90 0x7f1d4c099870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 --2- 172.21.15.140:0/1242043250 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d4c0afec0 0x7f1d4c0b26c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 -- 172.21.15.140:0/1242043250 >> 172.21.15.140:0/1242043250 conn(0x7f1d4c0b5cb0 msgr2=0x7f1d4c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:45.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 -- 172.21.15.140:0/1242043250 shutdown_connections 2022-07-21T01:51:45.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 -- 172.21.15.140:0/1242043250 wait complete. 2022-07-21T01:51:45.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 Processor -- start 2022-07-21T01:51:45.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 -- start start 2022-07-21T01:51:45.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d4c05bd90 0x7f1d4c0ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d4c099d60 0x7f1d4c067020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d4c0afec0 0x7f1d4c067510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c065c80 con 0x7f1d4c05bd90 2022-07-21T01:51:45.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c0ad170 con 0x7f1d4c099d60 2022-07-21T01:51:45.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.463+0000 7f1d52822700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c1536c0 con 0x7f1d4c0afec0 2022-07-21T01:51:45.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d4bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d4c05bd90 0x7f1d4c0ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d4bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d4c05bd90 0x7f1d4c0ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34862/0 (socket says 172.21.15.140:34862) 2022-07-21T01:51:45.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d4bfff700 1 -- 172.21.15.140:0/1637104890 learned_addr learned my addr 172.21.15.140:0/1637104890 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:45.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d4b7fe700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d4c099d60 0x7f1d4c067020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d50dbf700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d4c0afec0 0x7f1d4c067510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d4bfff700 1 -- 172.21.15.140:0/1637104890 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d4c0afec0 msgr2=0x7f1d4c067510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d4bfff700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d4c0afec0 0x7f1d4c067510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d4bfff700 1 -- 172.21.15.140:0/1637104890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d4c099d60 msgr2=0x7f1d4c067020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:45.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d4bfff700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d4c099d60 0x7f1d4c067020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:45.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d4bfff700 1 -- 172.21.15.140:0/1637104890 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3400b180 con 0x7f1d4c05bd90 2022-07-21T01:51:45.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d4bfff700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d4c05bd90 0x7f1d4c0ad660 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f1d34002f70 tx=0x7f1d34014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:45.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d497fa700 1 -- 172.21.15.140:0/1637104890 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d34013de0 con 0x7f1d4c05bd90 2022-07-21T01:51:45.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d497fa700 1 -- 172.21.15.140:0/1637104890 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3400de60 con 0x7f1d4c05bd90 2022-07-21T01:51:45.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d52822700 1 -- 172.21.15.140:0/1637104890 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4c153960 con 0x7f1d4c05bd90 2022-07-21T01:51:45.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d497fa700 1 -- 172.21.15.140:0/1637104890 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d3401e5f0 con 0x7f1d4c05bd90 2022-07-21T01:51:45.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d52822700 1 -- 172.21.15.140:0/1637104890 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c067a00 con 0x7f1d4c05bd90 2022-07-21T01:51:45.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d497fa700 1 -- 172.21.15.140:0/1637104890 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d3401c070 con 0x7f1d4c05bd90 2022-07-21T01:51:45.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d52822700 1 -- 172.21.15.140:0/1637104890 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4c0610d0 con 0x7f1d4c05bd90 2022-07-21T01:51:45.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d497fa700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d30053a10 0x7f1d30055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:45.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d4b7fe700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d30053a10 0x7f1d30055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:45.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.467+0000 7f1d497fa700 1 -- 172.21.15.140:0/1637104890 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1d3406ca00 con 0x7f1d4c05bd90 2022-07-21T01:51:45.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.471+0000 7f1d4b7fe700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d30053a10 0x7f1d30055ed0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f1d4400b000 tx=0x7f1d440091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:45.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.471+0000 7f1d497fa700 1 -- 172.21.15.140:0/1637104890 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d34043780 con 0x7f1d4c05bd90 2022-07-21T01:51:45.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:45.631+0000 7f1d52822700 1 -- 172.21.15.140:0/1637104890 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1d4c0574e0 con 0x7f1d4c05bd90 2022-07-21T01:51:46.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.039+0000 7f1d497fa700 1 -- 172.21.15.140:0/1637104890 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1d34023020 con 0x7f1d4c05bd90 2022-07-21T01:51:46.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 -- 172.21.15.140:0/1637104890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d30053a10 msgr2=0x7f1d30055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:46.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d30053a10 0x7f1d30055ed0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f1d4400b000 tx=0x7f1d440091d0).stop 2022-07-21T01:51:46.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 -- 172.21.15.140:0/1637104890 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d4c05bd90 msgr2=0x7f1d4c0ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:46.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d4c05bd90 0x7f1d4c0ad660 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f1d34002f70 tx=0x7f1d34014000).stop 2022-07-21T01:51:46.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 -- 172.21.15.140:0/1637104890 shutdown_connections 2022-07-21T01:51:46.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d30053a10 0x7f1d30055ed0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:46.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d4c0afec0 0x7f1d4c067510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:46.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d4c099d60 0x7f1d4c067020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:46.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 --2- 172.21.15.140:0/1637104890 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d4c05bd90 0x7f1d4c0ad660 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:46.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 -- 172.21.15.140:0/1637104890 >> 172.21.15.140:0/1637104890 conn(0x7f1d4c0b5cb0 msgr2=0x7f1d4c09a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:46.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 -- 172.21.15.140:0/1637104890 shutdown_connections 2022-07-21T01:51:46.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.043+0000 7f1d52822700 1 -- 172.21.15.140:0/1637104890 wait complete. 2022-07-21T01:51:46.059 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:51:46.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.171+0000 7f3d7e365700 1 Processor -- start 2022-07-21T01:51:46.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.171+0000 7f3d7e365700 1 -- start start 2022-07-21T01:51:46.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7e365700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3d780b35c0 0x7f3d780abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:46.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7e365700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3d78098ae0 0x7f3d78061440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:46.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7e365700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3d780ac470 0x7f3d78061930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:46.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7e365700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d780b39a0 con 0x7f3d780ac470 2022-07-21T01:51:46.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7e365700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7809c000 con 0x7f3d78098ae0 2022-07-21T01:51:46.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7e365700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780aba30 con 0x7f3d780b35c0 2022-07-21T01:51:46.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7db64700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3d780ac470 0x7f3d78061930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:46.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7db64700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3d780ac470 0x7f3d78061930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34872/0 (socket says 172.21.15.140:34872) 2022-07-21T01:51:46.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7db64700 1 -- 172.21.15.140:0/1364038475 learned_addr learned my addr 172.21.15.140:0/1364038475 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:46.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7cb62700 1 --2- 172.21.15.140:0/1364038475 >> v2:172.21.15.152:3300/0 conn(0x7f3d78098ae0 0x7f3d78061440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:46.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7d363700 1 --2- 172.21.15.140:0/1364038475 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3d780b35c0 0x7f3d780abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:46.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7cb62700 1 -- 172.21.15.140:0/1364038475 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3d780b35c0 msgr2=0x7f3d780abe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:46.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7cb62700 1 --2- 172.21.15.140:0/1364038475 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3d780b35c0 0x7f3d780abe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:46.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7cb62700 1 -- 172.21.15.140:0/1364038475 >> v2:172.21.15.140:3300/0 conn(0x7f3d780ac470 msgr2=0x7f3d78061930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:46.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7cb62700 1 --2- 172.21.15.140:0/1364038475 >> v2:172.21.15.140:3300/0 conn(0x7f3d780ac470 0x7f3d78061930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:46.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7cb62700 1 -- 172.21.15.140:0/1364038475 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d780acbc0 con 0x7f3d78098ae0 2022-07-21T01:51:46.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7db64700 1 --2- 172.21.15.140:0/1364038475 >> v2:172.21.15.140:3300/0 conn(0x7f3d780ac470 0x7f3d78061930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:51:46.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d7cb62700 1 --2- 172.21.15.140:0/1364038475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d78098ae0 0x7f3d78061440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f3d7809c140 tx=0x7f3d68006df0).ready entity=mon.1 client_cookie=a4f6d149fc751a7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:46.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d67fff700 1 -- 172.21.15.140:0/1364038475 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d6800d4a0 con 0x7f3d78098ae0 2022-07-21T01:51:46.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d67fff700 1 -- 172.21.15.140:0/1364038475 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d68000e90 con 0x7f3d78098ae0 2022-07-21T01:51:46.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.175+0000 7f3d67fff700 1 -- 172.21.15.140:0/1364038475 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d6800d7f0 con 0x7f3d78098ae0 2022-07-21T01:51:46.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.179+0000 7f3d7e365700 1 -- 172.21.15.140:0/1364038475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d78098ae0 msgr2=0x7f3d78061440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:46.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.179+0000 7f3d7e365700 1 --2- 172.21.15.140:0/1364038475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d78098ae0 0x7f3d78061440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f3d7809c140 tx=0x7f3d68006df0).stop 2022-07-21T01:51:46.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.179+0000 7f3d7e365700 1 -- 172.21.15.140:0/1364038475 shutdown_connections 2022-07-21T01:51:46.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.179+0000 7f3d7e365700 1 --2- 172.21.15.140:0/1364038475 >> v2:172.21.15.140:3300/0 conn(0x7f3d780ac470 0x7f3d78061930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:46.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.179+0000 7f3d7e365700 1 --2- 172.21.15.140:0/1364038475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d78098ae0 0x7f3d78061440 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:46.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.179+0000 7f3d7e365700 1 --2- 172.21.15.140:0/1364038475 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3d780b35c0 0x7f3d780abe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:46.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.179+0000 7f3d7e365700 1 -- 172.21.15.140:0/1364038475 >> 172.21.15.140:0/1364038475 conn(0x7f3d780b6b20 msgr2=0x7f3d780b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:46.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.179+0000 7f3d7e365700 1 -- 172.21.15.140:0/1364038475 shutdown_connections 2022-07-21T01:51:46.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.179+0000 7f3d7e365700 1 -- 172.21.15.140:0/1364038475 wait complete. 2022-07-21T01:51:46.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.179+0000 7f3d7e365700 1 Processor -- start 2022-07-21T01:51:46.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.179+0000 7f3d7e365700 1 -- start start 2022-07-21T01:51:46.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7e365700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d78098ae0 0x7f3d780b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:46.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7e365700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3d780ac470 0x7f3d780af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:46.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7e365700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3d780b35c0 0x7f3d7813d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:46.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7e365700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780af5f0 con 0x7f3d780ac470 2022-07-21T01:51:46.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7e365700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780b1050 con 0x7f3d78098ae0 2022-07-21T01:51:46.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7e365700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780b12c0 con 0x7f3d780b35c0 2022-07-21T01:51:46.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7cb62700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3d780ac470 0x7f3d780af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:46.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7cb62700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3d780ac470 0x7f3d780af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34876/0 (socket says 172.21.15.140:34876) 2022-07-21T01:51:46.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7cb62700 1 -- 172.21.15.140:0/969280798 learned_addr learned my addr 172.21.15.140:0/969280798 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:46.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7d363700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d78098ae0 0x7f3d780b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:46.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7db64700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3d780b35c0 0x7f3d7813d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:46.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7cb62700 1 -- 172.21.15.140:0/969280798 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3d780b35c0 msgr2=0x7f3d7813d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:46.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7cb62700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3d780b35c0 0x7f3d7813d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:46.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7cb62700 1 -- 172.21.15.140:0/969280798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d78098ae0 msgr2=0x7f3d780b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:46.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7cb62700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d78098ae0 0x7f3d780b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:46.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7cb62700 1 -- 172.21.15.140:0/969280798 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d70009530 con 0x7f3d780ac470 2022-07-21T01:51:46.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7cb62700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3d780ac470 0x7f3d780af100 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f3d780b1190 tx=0x7f3d68014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:46.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d667fc700 1 -- 172.21.15.140:0/969280798 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d68016070 con 0x7f3d780ac470 2022-07-21T01:51:46.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d667fc700 1 -- 172.21.15.140:0/969280798 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d68002f70 con 0x7f3d780ac470 2022-07-21T01:51:46.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7e365700 1 -- 172.21.15.140:0/969280798 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6800b180 con 0x7f3d780ac470 2022-07-21T01:51:46.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d667fc700 1 -- 172.21.15.140:0/969280798 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d68013da0 con 0x7f3d780ac470 2022-07-21T01:51:46.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7e365700 1 -- 172.21.15.140:0/969280798 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7813e660 con 0x7f3d780ac470 2022-07-21T01:51:46.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d667fc700 1 -- 172.21.15.140:0/969280798 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3d68025030 con 0x7f3d780ac470 2022-07-21T01:51:46.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.183+0000 7f3d7e365700 1 -- 172.21.15.140:0/969280798 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7813e3a0 con 0x7f3d780ac470 2022-07-21T01:51:46.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.187+0000 7f3d667fc700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d58053aa0 0x7f3d58055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:46.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.187+0000 7f3d667fc700 1 -- 172.21.15.140:0/969280798 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3d6806cd70 con 0x7f3d780ac470 2022-07-21T01:51:46.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.187+0000 7f3d7d363700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d58053aa0 0x7f3d58055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:46.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.187+0000 7f3d7d363700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d58053aa0 0x7f3d58055f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f3d7400b000 tx=0x7f3d74006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:46.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.191+0000 7f3d667fc700 1 -- 172.21.15.140:0/969280798 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3d7813e3a0 con 0x7f3d780ac470 2022-07-21T01:51:46.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:46.319+0000 7f3d7e365700 1 -- 172.21.15.140:0/969280798 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3d7813e3a0 con 0x7f3d780ac470 2022-07-21T01:51:47.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.123+0000 7f3d667fc700 1 -- 172.21.15.140:0/969280798 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 185+0+0 (secure 0 0 0) 0x7f3d7813e3a0 con 0x7f3d780ac470 2022-07-21T01:51:47.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 -- 172.21.15.140:0/969280798 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d58053aa0 msgr2=0x7f3d58055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:47.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d58053aa0 0x7f3d58055f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f3d7400b000 tx=0x7f3d74006df0).stop 2022-07-21T01:51:47.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 -- 172.21.15.140:0/969280798 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3d780ac470 msgr2=0x7f3d780af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:47.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3d780ac470 0x7f3d780af100 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f3d780b1190 tx=0x7f3d68014000).stop 2022-07-21T01:51:47.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 -- 172.21.15.140:0/969280798 shutdown_connections 2022-07-21T01:51:47.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d58053aa0 0x7f3d58055f60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:47.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3d780b35c0 0x7f3d7813d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:47.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3d780ac470 0x7f3d780af100 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:47.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 --2- 172.21.15.140:0/969280798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d78098ae0 0x7f3d780b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:47.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 -- 172.21.15.140:0/969280798 >> 172.21.15.140:0/969280798 conn(0x7f3d780b6b20 msgr2=0x7f3d7809af90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:47.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 -- 172.21.15.140:0/969280798 shutdown_connections 2022-07-21T01:51:47.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.127+0000 7f3d7e365700 1 -- 172.21.15.140:0/969280798 wait complete. 2022-07-21T01:51:47.145 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:51:47.147 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:51:47.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.259+0000 7f2d26bf6700 1 Processor -- start 2022-07-21T01:51:47.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 -- start start 2022-07-21T01:51:47.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2d200b6840 0x7f2d200b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:47.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2d200b3cf0 0x7f2d20146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d20066b10 con 0x7f2d20146530 2022-07-21T01:51:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d200b6c20 con 0x7f2d200b3cf0 2022-07-21T01:51:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d200aead0 con 0x7f2d200b6840 2022-07-21T01:51:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d253f3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2d200b3cf0 0x7f2d20146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:47.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d263f5700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2d20146530 0x7f2d2005bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49036/0 (socket says 172.21.15.140:49036) 2022-07-21T01:51:47.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d253f3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2d200b3cf0 0x7f2d20146040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34888/0 (socket says 172.21.15.140:34888) 2022-07-21T01:51:47.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d263f5700 1 -- 172.21.15.140:0/78200013 learned_addr learned my addr 172.21.15.140:0/78200013 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:47.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d25bf4700 1 --2- 172.21.15.140:0/78200013 >> v2:172.21.15.152:3300/0 conn(0x7f2d200b6840 0x7f2d200b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:47.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d253f3700 1 -- 172.21.15.140:0/78200013 >> v2:172.21.15.152:3300/0 conn(0x7f2d200b6840 msgr2=0x7f2d200b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:47.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d253f3700 1 --2- 172.21.15.140:0/78200013 >> v2:172.21.15.152:3300/0 conn(0x7f2d200b6840 0x7f2d200b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:47.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d253f3700 1 -- 172.21.15.140:0/78200013 >> v1:172.21.15.140:6789/0 conn(0x7f2d20146530 legacy=0x7f2d2005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:47.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d253f3700 1 -- 172.21.15.140:0/78200013 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d200aed70 con 0x7f2d200b3cf0 2022-07-21T01:51:47.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d253f3700 1 --2- 172.21.15.140:0/78200013 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d200b3cf0 0x7f2d20146040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f2d18001110 tx=0x7f2d1c0091d0).ready entity=mon.0 client_cookie=57b15f3f9d046767 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:47.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d24bf2700 1 -- 172.21.15.140:0/78200013 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d1c009ea0 con 0x7f2d200b3cf0 2022-07-21T01:51:47.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d24bf2700 1 -- 172.21.15.140:0/78200013 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c00b730 con 0x7f2d200b3cf0 2022-07-21T01:51:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d24bf2700 1 -- 172.21.15.140:0/78200013 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d1c00b8a0 con 0x7f2d200b3cf0 2022-07-21T01:51:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 -- 172.21.15.140:0/78200013 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d200b3cf0 msgr2=0x7f2d20146040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 --2- 172.21.15.140:0/78200013 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d200b3cf0 0x7f2d20146040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f2d18001110 tx=0x7f2d1c0091d0).stop 2022-07-21T01:51:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 -- 172.21.15.140:0/78200013 shutdown_connections 2022-07-21T01:51:47.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 --2- 172.21.15.140:0/78200013 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d200b3cf0 0x7f2d20146040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:47.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 --2- 172.21.15.140:0/78200013 >> v2:172.21.15.152:3300/0 conn(0x7f2d200b6840 0x7f2d200b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:47.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 -- 172.21.15.140:0/78200013 >> 172.21.15.140:0/78200013 conn(0x7f2d200b9e10 msgr2=0x7f2d200b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:47.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 -- 172.21.15.140:0/78200013 shutdown_connections 2022-07-21T01:51:47.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 -- 172.21.15.140:0/78200013 wait complete. 2022-07-21T01:51:47.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 Processor -- start 2022-07-21T01:51:47.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.263+0000 7f2d26bf6700 1 -- start start 2022-07-21T01:51:47.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d26bf6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d200b3cf0 0x7f2d200b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:47.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d26bf6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d200b6840 0x7f2d200b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:47.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d26bf6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d20146530 0x7f2d200b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:47.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d26bf6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200b25d0 con 0x7f2d20146530 2022-07-21T01:51:47.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d26bf6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200affa0 con 0x7f2d200b3cf0 2022-07-21T01:51:47.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d26bf6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200afa60 con 0x7f2d200b6840 2022-07-21T01:51:47.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d263f5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d20146530 0x7f2d200b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:47.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d263f5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d20146530 0x7f2d200b89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34896/0 (socket says 172.21.15.140:34896) 2022-07-21T01:51:47.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d263f5700 1 -- 172.21.15.140:0/3280761524 learned_addr learned my addr 172.21.15.140:0/3280761524 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:47.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d25bf4700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d200b3cf0 0x7f2d200b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:47.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d253f3700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d200b6840 0x7f2d200b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:47.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d263f5700 1 -- 172.21.15.140:0/3280761524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d200b6840 msgr2=0x7f2d200b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:47.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d263f5700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d200b6840 0x7f2d200b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:47.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d263f5700 1 -- 172.21.15.140:0/3280761524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d200b3cf0 msgr2=0x7f2d200b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:47.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d263f5700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d200b3cf0 0x7f2d200b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:47.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d263f5700 1 -- 172.21.15.140:0/3280761524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c009470 con 0x7f2d20146530 2022-07-21T01:51:47.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d253f3700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d200b6840 0x7f2d200b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:51:47.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d263f5700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d20146530 0x7f2d200b89b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f2d201551a0 tx=0x7f2d18007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:47.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d0effd700 1 -- 172.21.15.140:0/3280761524 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d180070d0 con 0x7f2d20146530 2022-07-21T01:51:47.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d0effd700 1 -- 172.21.15.140:0/3280761524 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d18007230 con 0x7f2d20146530 2022-07-21T01:51:47.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d26bf6700 1 -- 172.21.15.140:0/3280761524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d200afd00 con 0x7f2d20146530 2022-07-21T01:51:47.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d0effd700 1 -- 172.21.15.140:0/3280761524 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d180055f0 con 0x7f2d20146530 2022-07-21T01:51:47.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d26bf6700 1 -- 172.21.15.140:0/3280761524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d20153210 con 0x7f2d20146530 2022-07-21T01:51:47.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d26bf6700 1 -- 172.21.15.140:0/3280761524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d201551e0 con 0x7f2d20146530 2022-07-21T01:51:47.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d0effd700 1 -- 172.21.15.140:0/3280761524 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2d1801a060 con 0x7f2d20146530 2022-07-21T01:51:47.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d0effd700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d00053aa0 0x7f2d00055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:47.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d0effd700 1 -- 172.21.15.140:0/3280761524 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f2d1806b430 con 0x7f2d20146530 2022-07-21T01:51:47.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.267+0000 7f2d25bf4700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d00053aa0 0x7f2d00055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:47.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.271+0000 7f2d25bf4700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d00053aa0 0x7f2d00055f60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f2d1000c000 tx=0x7f2d10008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:47.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.271+0000 7f2d0effd700 1 -- 172.21.15.140:0/3280761524 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d180424b0 con 0x7f2d20146530 2022-07-21T01:51:47.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:47.395+0000 7f2d26bf6700 1 -- 172.21.15.140:0/3280761524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2d200b50e0 con 0x7f2d20146530 2022-07-21T01:51:48.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.131+0000 7f2d0effd700 1 -- 172.21.15.140:0/3280761524 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2d18018070 con 0x7f2d20146530 2022-07-21T01:51:48.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 -- 172.21.15.140:0/3280761524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d00053aa0 msgr2=0x7f2d00055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d00053aa0 0x7f2d00055f60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f2d1000c000 tx=0x7f2d10008040).stop 2022-07-21T01:51:48.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 -- 172.21.15.140:0/3280761524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d20146530 msgr2=0x7f2d200b89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d20146530 0x7f2d200b89b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f2d201551a0 tx=0x7f2d18007b50).stop 2022-07-21T01:51:48.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 -- 172.21.15.140:0/3280761524 shutdown_connections 2022-07-21T01:51:48.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d00053aa0 0x7f2d00055f60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d20146530 0x7f2d200b89b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d200b6840 0x7f2d200b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 --2- 172.21.15.140:0/3280761524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d200b3cf0 0x7f2d200b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 -- 172.21.15.140:0/3280761524 >> 172.21.15.140:0/3280761524 conn(0x7f2d200b9e10 msgr2=0x7f2d20141a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:48.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 -- 172.21.15.140:0/3280761524 shutdown_connections 2022-07-21T01:51:48.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.135+0000 7f2d26bf6700 1 -- 172.21.15.140:0/3280761524 wait complete. 2022-07-21T01:51:48.140 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:51:48.148 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-07-21T01:51:48.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.259+0000 7ff37d8c2700 1 Processor -- start 2022-07-21T01:51:48.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.259+0000 7ff37d8c2700 1 -- start start 2022-07-21T01:51:48.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.259+0000 7ff37d8c2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff3780b00f0 0x7ff3780b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.259+0000 7ff37d8c2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3780b3df0 0x7ff378141c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.259+0000 7ff37d8c2700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff378142dc0 con 0x7ff3780b9ea0 2022-07-21T01:51:48.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.259+0000 7ff37d8c2700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3780ba280 con 0x7ff3780b3df0 2022-07-21T01:51:48.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.259+0000 7ff37d8c2700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff378146280 con 0x7ff3780b00f0 2022-07-21T01:51:48.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.259+0000 7ff3777fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3780b3df0 0x7ff378141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff3777fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3780b3df0 0x7ff378141c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34904/0 (socket says 172.21.15.140:34904) 2022-07-21T01:51:48.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff3777fe700 1 -- 172.21.15.140:0/681818974 learned_addr learned my addr 172.21.15.140:0/681818974 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:48.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff3767fc700 1 --2- 172.21.15.140:0/681818974 >> v2:172.21.15.152:3300/0 conn(0x7ff3780b00f0 0x7ff3780b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff3777fe700 1 -- 172.21.15.140:0/681818974 >> v2:172.21.15.152:3300/0 conn(0x7ff3780b00f0 msgr2=0x7ff3780b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:48.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff3777fe700 1 --2- 172.21.15.140:0/681818974 >> v2:172.21.15.152:3300/0 conn(0x7ff3780b00f0 0x7ff3780b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff3777fe700 1 -- 172.21.15.140:0/681818974 >> v1:172.21.15.152:6789/0 conn(0x7ff3780b9ea0 legacy=0x7ff378143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff3777fe700 1 -- 172.21.15.140:0/681818974 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff378142ff0 con 0x7ff3780b3df0 2022-07-21T01:51:48.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff3777fe700 1 --2- 172.21.15.140:0/681818974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3780b3df0 0x7ff378141c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7ff368002f70 tx=0x7ff368006eb0).ready entity=mon.0 client_cookie=2c521378cfb13cbd server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:48.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff375ffb700 1 -- 172.21.15.140:0/681818974 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff36800d5c0 con 0x7ff3780b3df0 2022-07-21T01:51:48.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff375ffb700 1 -- 172.21.15.140:0/681818974 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff368000e90 con 0x7ff3780b3df0 2022-07-21T01:51:48.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff375ffb700 1 -- 172.21.15.140:0/681818974 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff36800d950 con 0x7ff3780b3df0 2022-07-21T01:51:48.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff37d8c2700 1 -- 172.21.15.140:0/681818974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3780b3df0 msgr2=0x7ff378141c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff37d8c2700 1 --2- 172.21.15.140:0/681818974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3780b3df0 0x7ff378141c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7ff368002f70 tx=0x7ff368006eb0).stop 2022-07-21T01:51:48.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff37d8c2700 1 -- 172.21.15.140:0/681818974 shutdown_connections 2022-07-21T01:51:48.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff37d8c2700 1 --2- 172.21.15.140:0/681818974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3780b3df0 0x7ff378141c80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff37d8c2700 1 --2- 172.21.15.140:0/681818974 >> v2:172.21.15.152:3300/0 conn(0x7ff3780b00f0 0x7ff3780b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff37d8c2700 1 -- 172.21.15.140:0/681818974 >> 172.21.15.140:0/681818974 conn(0x7ff378067960 msgr2=0x7ff3780b9450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:48.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff37d8c2700 1 -- 172.21.15.140:0/681818974 shutdown_connections 2022-07-21T01:51:48.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff37d8c2700 1 -- 172.21.15.140:0/681818974 wait complete. 2022-07-21T01:51:48.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff37d8c2700 1 Processor -- start 2022-07-21T01:51:48.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.263+0000 7ff37d8c2700 1 -- start start 2022-07-21T01:51:48.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff37d8c2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3780b00f0 0x7ff3780ae390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff37d8c2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3780b3df0 0x7ff3780acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff37d8c2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3780b9ea0 0x7ff3780b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff37d8c2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff378114e70 con 0x7ff3780b00f0 2022-07-21T01:51:48.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff37d8c2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780acfd0 con 0x7ff3780b9ea0 2022-07-21T01:51:48.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff37d8c2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff378114bd0 con 0x7ff3780b3df0 2022-07-21T01:51:48.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff376ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3780b00f0 0x7ff3780ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff376ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3780b00f0 0x7ff3780ae390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34906/0 (socket says 172.21.15.140:34906) 2022-07-21T01:51:48.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff376ffd700 1 -- 172.21.15.140:0/1610463212 learned_addr learned my addr 172.21.15.140:0/1610463212 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:48.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff3777fe700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3780b9ea0 0x7ff3780b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff3767fc700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3780b3df0 0x7ff3780acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff376ffd700 1 -- 172.21.15.140:0/1610463212 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3780b3df0 msgr2=0x7ff3780acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff376ffd700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3780b3df0 0x7ff3780acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff376ffd700 1 -- 172.21.15.140:0/1610463212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3780b9ea0 msgr2=0x7ff3780b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff376ffd700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3780b9ea0 0x7ff3780b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff376ffd700 1 -- 172.21.15.140:0/1610463212 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36800b1e0 con 0x7ff3780b00f0 2022-07-21T01:51:48.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff376ffd700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3780b00f0 0x7ff3780ae390 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7ff3780b1310 tx=0x7ff370001400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff367fff700 1 -- 172.21.15.140:0/1610463212 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff37000f470 con 0x7ff3780b00f0 2022-07-21T01:51:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff367fff700 1 -- 172.21.15.140:0/1610463212 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37000f5d0 con 0x7ff3780b00f0 2022-07-21T01:51:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff37d8c2700 1 -- 172.21.15.140:0/1610463212 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3781521c0 con 0x7ff3780b00f0 2022-07-21T01:51:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff367fff700 1 -- 172.21.15.140:0/1610463212 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff37000f930 con 0x7ff3780b00f0 2022-07-21T01:51:48.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff37d8c2700 1 -- 172.21.15.140:0/1610463212 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff378112cb0 con 0x7ff3780b00f0 2022-07-21T01:51:48.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff37d8c2700 1 -- 172.21.15.140:0/1610463212 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff378146520 con 0x7ff3780b00f0 2022-07-21T01:51:48.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff367fff700 1 -- 172.21.15.140:0/1610463212 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff370019030 con 0x7ff3780b00f0 2022-07-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff367fff700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff354053a70 0x7ff354055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff367fff700 1 -- 172.21.15.140:0/1610463212 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5339+0+0 (secure 0 0 0) 0x7ff37006b320 con 0x7ff3780b00f0 2022-07-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.267+0000 7ff3767fc700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff354053a70 0x7ff354055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.271+0000 7ff3767fc700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff354053a70 0x7ff354055f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7ff36c00b000 tx=0x7ff36c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:48.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.275+0000 7ff367fff700 1 -- 172.21.15.140:0/1610463212 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff370042420 con 0x7ff3780b00f0 2022-07-21T01:51:48.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.403+0000 7ff37d8c2700 1 -- 172.21.15.140:0/1610463212 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff3780574e0 con 0x7ff3780b00f0 2022-07-21T01:51:48.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.411+0000 7ff367fff700 1 -- 172.21.15.140:0/1610463212 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff370045a40 con 0x7ff3780b00f0 2022-07-21T01:51:48.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.411+0000 7ff37d8c2700 1 -- 172.21.15.140:0/1610463212 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff354053a70 msgr2=0x7ff354055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.411+0000 7ff37d8c2700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff354053a70 0x7ff354055f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7ff36c00b000 tx=0x7ff36c006df0).stop 2022-07-21T01:51:48.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.411+0000 7ff37d8c2700 1 -- 172.21.15.140:0/1610463212 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3780b00f0 msgr2=0x7ff3780ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.411+0000 7ff37d8c2700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3780b00f0 0x7ff3780ae390 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7ff3780b1310 tx=0x7ff370001400).stop 2022-07-21T01:51:48.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.415+0000 7ff37d8c2700 1 -- 172.21.15.140:0/1610463212 shutdown_connections 2022-07-21T01:51:48.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.415+0000 7ff37d8c2700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff354053a70 0x7ff354055f30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.415+0000 7ff37d8c2700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3780b9ea0 0x7ff3780b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.415+0000 7ff37d8c2700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3780b3df0 0x7ff3780acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.415+0000 7ff37d8c2700 1 --2- 172.21.15.140:0/1610463212 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3780b00f0 0x7ff3780ae390 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.415+0000 7ff37d8c2700 1 -- 172.21.15.140:0/1610463212 >> 172.21.15.140:0/1610463212 conn(0x7ff378067960 msgr2=0x7ff37813f060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:48.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.415+0000 7ff37d8c2700 1 -- 172.21.15.140:0/1610463212 shutdown_connections 2022-07-21T01:51:48.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.415+0000 7ff37d8c2700 1 -- 172.21.15.140:0/1610463212 wait complete. 2022-07-21T01:51:48.426 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2022-07-21T01:51:48.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.531+0000 7f328c9f0700 1 Processor -- start 2022-07-21T01:51:48.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.531+0000 7f328c9f0700 1 -- start start 2022-07-21T01:51:48.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f32880b0570 0x7f32880b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f32881424e0 0x7f32880ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32881433d0 con 0x7f32880aea10 2022-07-21T01:51:48.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3288143b10 con 0x7f32880b0570 2022-07-21T01:51:48.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3288143670 con 0x7f32881424e0 2022-07-21T01:51:48.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328659c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f32880aea10 0x7f3288141dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49060/0 (socket says 172.21.15.140:49060) 2022-07-21T01:51:48.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328659c700 1 -- 172.21.15.140:0/1039874262 learned_addr learned my addr 172.21.15.140:0/1039874262 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:48.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f3286d9d700 1 --2- 172.21.15.140:0/1039874262 >> v2:172.21.15.140:3300/0 conn(0x7f32880b0570 0x7f32880b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328759e700 1 --2- 172.21.15.140:0/1039874262 >> v2:172.21.15.152:3300/0 conn(0x7f32881424e0 0x7f32880ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f3285d9b700 1 -- 172.21.15.140:0/1039874262 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911272623 0 0) 0x7f32881433d0 con 0x7f32880aea10 2022-07-21T01:51:48.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f3285d9b700 1 -- 172.21.15.140:0/1039874262 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f326c003260 con 0x7f32880aea10 2022-07-21T01:51:48.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f3286d9d700 1 -- 172.21.15.140:0/1039874262 >> v2:172.21.15.152:3300/0 conn(0x7f32881424e0 msgr2=0x7f32880ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:48.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f3286d9d700 1 --2- 172.21.15.140:0/1039874262 >> v2:172.21.15.152:3300/0 conn(0x7f32881424e0 0x7f32880ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f3286d9d700 1 -- 172.21.15.140:0/1039874262 >> v1:172.21.15.140:6789/0 conn(0x7f32880aea10 legacy=0x7f3288141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f3286d9d700 1 -- 172.21.15.140:0/1039874262 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32880b1010 con 0x7f32880b0570 2022-07-21T01:51:48.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328759e700 1 --2- 172.21.15.140:0/1039874262 >> v2:172.21.15.152:3300/0 conn(0x7f32881424e0 0x7f32880ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:51:48.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f3286d9d700 1 --2- 172.21.15.140:0/1039874262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32880b0570 0x7f32880b0950 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f3278002f70 tx=0x7f3278006df0).ready entity=mon.0 client_cookie=2c688d4c5687ae77 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:48.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f3285d9b700 1 -- 172.21.15.140:0/1039874262 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f327800d570 con 0x7f32880b0570 2022-07-21T01:51:48.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f3285d9b700 1 -- 172.21.15.140:0/1039874262 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3278000e90 con 0x7f32880b0570 2022-07-21T01:51:48.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f3285d9b700 1 -- 172.21.15.140:0/1039874262 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f327800d8c0 con 0x7f32880b0570 2022-07-21T01:51:48.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 -- 172.21.15.140:0/1039874262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32880b0570 msgr2=0x7f32880b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 --2- 172.21.15.140:0/1039874262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32880b0570 0x7f32880b0950 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f3278002f70 tx=0x7f3278006df0).stop 2022-07-21T01:51:48.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 -- 172.21.15.140:0/1039874262 shutdown_connections 2022-07-21T01:51:48.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 --2- 172.21.15.140:0/1039874262 >> v2:172.21.15.152:3300/0 conn(0x7f32881424e0 0x7f32880ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 --2- 172.21.15.140:0/1039874262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32880b0570 0x7f32880b0950 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 -- 172.21.15.140:0/1039874262 >> 172.21.15.140:0/1039874262 conn(0x7f32880b9360 msgr2=0x7f32880b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:48.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 -- 172.21.15.140:0/1039874262 shutdown_connections 2022-07-21T01:51:48.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 -- 172.21.15.140:0/1039874262 wait complete. 2022-07-21T01:51:48.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.535+0000 7f328c9f0700 1 Processor -- start 2022-07-21T01:51:48.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328c9f0700 1 -- start start 2022-07-21T01:51:48.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328c9f0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32880aea10 0x7f32880b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328c9f0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f32880b0570 0x7f328813e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328c9f0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32881424e0 0x7f32880b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328c9f0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3288146dc0 con 0x7f32881424e0 2022-07-21T01:51:48.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328c9f0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32881456b0 con 0x7f32880aea10 2022-07-21T01:51:48.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328c9f0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32881458e0 con 0x7f32880b0570 2022-07-21T01:51:48.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32881424e0 0x7f32880b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328659c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f32880b0570 0x7f328813e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f3286d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32880aea10 0x7f32880b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32881424e0 0x7f32880b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34924/0 (socket says 172.21.15.140:34924) 2022-07-21T01:51:48.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328759e700 1 -- 172.21.15.140:0/1782947658 learned_addr learned my addr 172.21.15.140:0/1782947658 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:48.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328759e700 1 -- 172.21.15.140:0/1782947658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f32880b0570 msgr2=0x7f328813e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328759e700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f32880b0570 0x7f328813e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328759e700 1 -- 172.21.15.140:0/1782947658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32880aea10 msgr2=0x7f32880b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:48.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328759e700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32880aea10 0x7f32880b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:48.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328759e700 1 -- 172.21.15.140:0/1782947658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327800b180 con 0x7f32881424e0 2022-07-21T01:51:48.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328759e700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32881424e0 0x7f32880b33c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f328000a000 tx=0x7f3280008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:48.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f327ffff700 1 -- 172.21.15.140:0/1782947658 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3280013620 con 0x7f32881424e0 2022-07-21T01:51:48.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f327ffff700 1 -- 172.21.15.140:0/1782947658 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3280013780 con 0x7f32881424e0 2022-07-21T01:51:48.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328c9f0700 1 -- 172.21.15.140:0/1782947658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3288145b80 con 0x7f32881424e0 2022-07-21T01:51:48.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f327ffff700 1 -- 172.21.15.140:0/1782947658 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3280013ae0 con 0x7f32881424e0 2022-07-21T01:51:48.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328c9f0700 1 -- 172.21.15.140:0/1782947658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328813ea20 con 0x7f32881424e0 2022-07-21T01:51:48.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.539+0000 7f328c9f0700 1 -- 172.21.15.140:0/1782947658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3288145fd0 con 0x7f32881424e0 2022-07-21T01:51:48.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.543+0000 7f327ffff700 1 -- 172.21.15.140:0/1782947658 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3280013c40 con 0x7f32881424e0 2022-07-21T01:51:48.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.543+0000 7f327ffff700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32600539e0 0x7f3260055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:48.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.543+0000 7f327ffff700 1 -- 172.21.15.140:0/1782947658 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f328006be50 con 0x7f32881424e0 2022-07-21T01:51:48.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.543+0000 7f3286d9d700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32600539e0 0x7f3260055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:48.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.543+0000 7f3286d9d700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32600539e0 0x7f3260055ea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f327801a000 tx=0x7f3278014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:48.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.551+0000 7f327ffff700 1 -- 172.21.15.140:0/1782947658 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3280042f40 con 0x7f32881424e0 2022-07-21T01:51:48.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:48.715+0000 7f328c9f0700 1 -- 172.21.15.140:0/1782947658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f3288145d10 con 0x7f32881424e0 2022-07-21T01:51:49.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.131+0000 7f327ffff700 1 -- 172.21.15.140:0/1782947658 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v152) v1 ==== 144+0+0 (secure 0 0 0) 0x7f328000f040 con 0x7f32881424e0 2022-07-21T01:51:49.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 -- 172.21.15.140:0/1782947658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32600539e0 msgr2=0x7f3260055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:49.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32600539e0 0x7f3260055ea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f327801a000 tx=0x7f3278014000).stop 2022-07-21T01:51:49.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 -- 172.21.15.140:0/1782947658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32881424e0 msgr2=0x7f32880b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:49.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32881424e0 0x7f32880b33c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f328000a000 tx=0x7f3280008040).stop 2022-07-21T01:51:49.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 -- 172.21.15.140:0/1782947658 shutdown_connections 2022-07-21T01:51:49.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32600539e0 0x7f3260055ea0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:49.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f32881424e0 0x7f32880b33c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:49.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f32880b0570 0x7f328813e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:49.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 --2- 172.21.15.140:0/1782947658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f32880aea10 0x7f32880b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:49.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 -- 172.21.15.140:0/1782947658 >> 172.21.15.140:0/1782947658 conn(0x7f32880b9360 msgr2=0x7f3288141090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:49.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 -- 172.21.15.140:0/1782947658 shutdown_connections 2022-07-21T01:51:49.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.135+0000 7f328c9f0700 1 -- 172.21.15.140:0/1782947658 wait complete. 2022-07-21T01:51:49.164 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-07-21T01:51:49.165 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2022-07-21T01:51:49.166 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2022-07-21T01:51:49.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6299163700 1 Processor -- start 2022-07-21T01:51:49.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6299163700 1 -- start start 2022-07-21T01:51:49.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6299163700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f62940aefb0 0x7f62940b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:49.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6299163700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62940b18d0 con 0x7f62940b5b30 2022-07-21T01:51:49.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6299163700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62940b5f10 con 0x7f62940ace90 2022-07-21T01:51:49.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6299163700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62940af440 con 0x7f62940aefb0 2022-07-21T01:51:49.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6292d9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f62940b5b30 0x7f62940b0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49074/0 (socket says 172.21.15.140:49074) 2022-07-21T01:51:49.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6292d9d700 1 -- 172.21.15.140:0/3427543674 learned_addr learned my addr 172.21.15.140:0/3427543674 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:49.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f629359e700 1 --2- 172.21.15.140:0/3427543674 >> v2:172.21.15.152:3300/0 conn(0x7f62940aefb0 0x7f62940b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:49.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6291d9b700 1 -- 172.21.15.140:0/3427543674 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702069917 0 0) 0x7f62940b18d0 con 0x7f62940b5b30 2022-07-21T01:51:49.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6291d9b700 1 -- 172.21.15.140:0/3427543674 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6274003260 con 0x7f62940b5b30 2022-07-21T01:51:49.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f629359e700 1 -- 172.21.15.140:0/3427543674 >> v1:172.21.15.152:6789/0 conn(0x7f62940ace90 legacy=0x7f62941463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:49.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f629359e700 1 -- 172.21.15.140:0/3427543674 >> v1:172.21.15.140:6789/0 conn(0x7f62940b5b30 legacy=0x7f62940b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:49.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f629359e700 1 -- 172.21.15.140:0/3427543674 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62940afc40 con 0x7f62940aefb0 2022-07-21T01:51:49.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f629359e700 1 --2- 172.21.15.140:0/3427543674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62940aefb0 0x7f62940b13e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f6284002f70 tx=0x7f6284006df0).ready entity=mon.1 client_cookie=9142ae471c79e84 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:49.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6291d9b700 1 -- 172.21.15.140:0/3427543674 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f628400d610 con 0x7f62940aefb0 2022-07-21T01:51:49.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6291d9b700 1 -- 172.21.15.140:0/3427543674 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6284000e90 con 0x7f62940aefb0 2022-07-21T01:51:49.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.279+0000 7f6291d9b700 1 -- 172.21.15.140:0/3427543674 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f628400d960 con 0x7f62940aefb0 2022-07-21T01:51:49.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 -- 172.21.15.140:0/3427543674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62940aefb0 msgr2=0x7f62940b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:49.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 --2- 172.21.15.140:0/3427543674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62940aefb0 0x7f62940b13e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f6284002f70 tx=0x7f6284006df0).stop 2022-07-21T01:51:49.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 -- 172.21.15.140:0/3427543674 shutdown_connections 2022-07-21T01:51:49.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 --2- 172.21.15.140:0/3427543674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62940aefb0 0x7f62940b13e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:49.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 -- 172.21.15.140:0/3427543674 >> 172.21.15.140:0/3427543674 conn(0x7f62940635b0 msgr2=0x7f62940b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:49.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 -- 172.21.15.140:0/3427543674 shutdown_connections 2022-07-21T01:51:49.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 -- 172.21.15.140:0/3427543674 wait complete. 2022-07-21T01:51:49.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 Processor -- start 2022-07-21T01:51:49.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 -- start start 2022-07-21T01:51:49.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62940ace90 0x7f6294145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:49.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62940aefb0 0x7f6294063b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:49.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62940b5b30 0x7f6294149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:49.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6294149500 con 0x7f62940b5b30 2022-07-21T01:51:49.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62941497a0 con 0x7f62940ace90 2022-07-21T01:51:49.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6299163700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6294146160 con 0x7f62940aefb0 2022-07-21T01:51:49.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f629359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62940b5b30 0x7f6294149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:49.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f629359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62940b5b30 0x7f6294149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34938/0 (socket says 172.21.15.140:34938) 2022-07-21T01:51:49.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f629359e700 1 -- 172.21.15.140:0/1758057658 learned_addr learned my addr 172.21.15.140:0/1758057658 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:49.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f629259c700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62940aefb0 0x7f6294063b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:49.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f6292d9d700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62940ace90 0x7f6294145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:49.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f629359e700 1 -- 172.21.15.140:0/1758057658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62940aefb0 msgr2=0x7f6294063b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:49.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f629359e700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62940aefb0 0x7f6294063b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:49.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f629359e700 1 -- 172.21.15.140:0/1758057658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62940ace90 msgr2=0x7f6294145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:49.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f629359e700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62940ace90 0x7f6294145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:49.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f629359e700 1 -- 172.21.15.140:0/1758057658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628400b180 con 0x7f62940b5b30 2022-07-21T01:51:49.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f629259c700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62940aefb0 0x7f6294063b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:51:49.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f629359e700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62940b5b30 0x7f6294149c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f6284002f70 tx=0x7f6284014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:49.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.283+0000 7f628bfff700 1 -- 172.21.15.140:0/1758057658 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6284016070 con 0x7f62940b5b30 2022-07-21T01:51:49.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.287+0000 7f628bfff700 1 -- 172.21.15.140:0/1758057658 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628400d290 con 0x7f62940b5b30 2022-07-21T01:51:49.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.287+0000 7f6299163700 1 -- 172.21.15.140:0/1758057658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6294063ff0 con 0x7f62940b5b30 2022-07-21T01:51:49.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.287+0000 7f628bfff700 1 -- 172.21.15.140:0/1758057658 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f628400de30 con 0x7f62940b5b30 2022-07-21T01:51:49.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.287+0000 7f6299163700 1 -- 172.21.15.140:0/1758057658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629414a2d0 con 0x7f62940b5b30 2022-07-21T01:51:49.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.287+0000 7f6299163700 1 -- 172.21.15.140:0/1758057658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62940afee0 con 0x7f62940b5b30 2022-07-21T01:51:49.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.287+0000 7f628bfff700 1 -- 172.21.15.140:0/1758057658 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6284025030 con 0x7f62940b5b30 2022-07-21T01:51:49.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.287+0000 7f628bfff700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62780539e0 0x7f6278055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:49.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.287+0000 7f628bfff700 1 -- 172.21.15.140:0/1758057658 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f628406cd20 con 0x7f62940b5b30 2022-07-21T01:51:49.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.287+0000 7f6292d9d700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62780539e0 0x7f6278055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:49.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.287+0000 7f6292d9d700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62780539e0 0x7f6278055ea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f627c00b000 tx=0x7f627c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:49.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.295+0000 7f628bfff700 1 -- 172.21.15.140:0/1758057658 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6284043f60 con 0x7f62940b5b30 2022-07-21T01:51:49.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:49.459+0000 7f6299163700 1 -- 172.21.15.140:0/1758057658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f62940574e0 con 0x7f62940b5b30 2022-07-21T01:51:50.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.135+0000 7f628bfff700 1 -- 172.21.15.140:0/1758057658 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v153) v1 ==== 155+0+0 (secure 0 0 0) 0x7f6284023070 con 0x7f62940b5b30 2022-07-21T01:51:50.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 -- 172.21.15.140:0/1758057658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62780539e0 msgr2=0x7f6278055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:50.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62780539e0 0x7f6278055ea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f627c00b000 tx=0x7f627c009040).stop 2022-07-21T01:51:50.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 -- 172.21.15.140:0/1758057658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62940b5b30 msgr2=0x7f6294149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:50.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62940b5b30 0x7f6294149c40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f6284002f70 tx=0x7f6284014000).stop 2022-07-21T01:51:50.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 -- 172.21.15.140:0/1758057658 shutdown_connections 2022-07-21T01:51:50.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62780539e0 0x7f6278055ea0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62940b5b30 0x7f6294149c40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62940aefb0 0x7f6294063b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 --2- 172.21.15.140:0/1758057658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62940ace90 0x7f6294145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 -- 172.21.15.140:0/1758057658 >> 172.21.15.140:0/1758057658 conn(0x7f62940635b0 msgr2=0x7f629414c940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:50.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 -- 172.21.15.140:0/1758057658 shutdown_connections 2022-07-21T01:51:50.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.139+0000 7f6299163700 1 -- 172.21.15.140:0/1758057658 wait complete. 2022-07-21T01:51:50.154 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:51:50.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.271+0000 7fb1bde5c700 1 Processor -- start 2022-07-21T01:51:50.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.271+0000 7fb1bde5c700 1 -- start start 2022-07-21T01:51:50.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1b805cf40 0x7fb1b80afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb1b8147c90 0x7fb1b80b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb1b80b1220 0x7fb1b8099080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b81472b0 con 0x7fb1b8147c90 2022-07-21T01:51:50.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b8147550 con 0x7fb1b80b1220 2022-07-21T01:51:50.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b81477f0 con 0x7fb1b805cf40 2022-07-21T01:51:50.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b6ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb1b8147c90 0x7fb1b80b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b6ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb1b8147c90 0x7fb1b80b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34942/0 (socket says 172.21.15.140:34942) 2022-07-21T01:51:50.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b6ffd700 1 -- 172.21.15.140:0/4222839570 learned_addr learned my addr 172.21.15.140:0/4222839570 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:50.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b7fff700 1 --2- 172.21.15.140:0/4222839570 >> v2:172.21.15.152:3300/0 conn(0x7fb1b80b1220 0x7fb1b8099080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b77fe700 1 --2- 172.21.15.140:0/4222839570 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1b805cf40 0x7fb1b80afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b6ffd700 1 -- 172.21.15.140:0/4222839570 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1b805cf40 msgr2=0x7fb1b80afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:50.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b6ffd700 1 --2- 172.21.15.140:0/4222839570 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1b805cf40 0x7fb1b80afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b6ffd700 1 -- 172.21.15.140:0/4222839570 >> v2:172.21.15.152:3300/0 conn(0x7fb1b80b1220 msgr2=0x7fb1b8099080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:50.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b6ffd700 1 --2- 172.21.15.140:0/4222839570 >> v2:172.21.15.152:3300/0 conn(0x7fb1b80b1220 0x7fb1b8099080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b6ffd700 1 -- 172.21.15.140:0/4222839570 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b8146fa0 con 0x7fb1b8147c90 2022-07-21T01:51:50.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b6ffd700 1 --2- 172.21.15.140:0/4222839570 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1b8147c90 0x7fb1b80b0210 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fb1a80032f0 tx=0x7fb1a8009290).ready entity=mon.0 client_cookie=9b56bc6bc980261a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:50.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b67fc700 1 -- 172.21.15.140:0/4222839570 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1a800b680 con 0x7fb1b8147c90 2022-07-21T01:51:50.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b67fc700 1 -- 172.21.15.140:0/4222839570 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a800b7e0 con 0x7fb1b8147c90 2022-07-21T01:51:50.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1b67fc700 1 -- 172.21.15.140:0/4222839570 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1a800b950 con 0x7fb1b8147c90 2022-07-21T01:51:50.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 -- 172.21.15.140:0/4222839570 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1b8147c90 msgr2=0x7fb1b80b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:50.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 --2- 172.21.15.140:0/4222839570 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1b8147c90 0x7fb1b80b0210 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fb1a80032f0 tx=0x7fb1a8009290).stop 2022-07-21T01:51:50.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 -- 172.21.15.140:0/4222839570 shutdown_connections 2022-07-21T01:51:50.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 --2- 172.21.15.140:0/4222839570 >> v2:172.21.15.152:3300/0 conn(0x7fb1b80b1220 0x7fb1b8099080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 --2- 172.21.15.140:0/4222839570 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1b8147c90 0x7fb1b80b0210 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 --2- 172.21.15.140:0/4222839570 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1b805cf40 0x7fb1b80afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 -- 172.21.15.140:0/4222839570 >> 172.21.15.140:0/4222839570 conn(0x7fb1b80b5b70 msgr2=0x7fb1b80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:50.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 -- 172.21.15.140:0/4222839570 shutdown_connections 2022-07-21T01:51:50.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.275+0000 7fb1bde5c700 1 -- 172.21.15.140:0/4222839570 wait complete. 2022-07-21T01:51:50.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1bde5c700 1 Processor -- start 2022-07-21T01:51:50.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1bde5c700 1 -- start start 2022-07-21T01:51:50.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1bde5c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1b805cf40 0x7fb1b809a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1bde5c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1b80b1220 0x7fb1b8142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1bde5c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1b8147c90 0x7fb1b81431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1bde5c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b8064b10 con 0x7fb1b8147c90 2022-07-21T01:51:50.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1bde5c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b8142430 con 0x7fb1b80b1220 2022-07-21T01:51:50.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1bde5c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b81415c0 con 0x7fb1b805cf40 2022-07-21T01:51:50.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1b8147c90 0x7fb1b81431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1b8147c90 0x7fb1b81431e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34952/0 (socket says 172.21.15.140:34952) 2022-07-21T01:51:50.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b7fff700 1 -- 172.21.15.140:0/1615302900 learned_addr learned my addr 172.21.15.140:0/1615302900 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:50.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b6ffd700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1b80b1220 0x7fb1b8142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b77fe700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1b805cf40 0x7fb1b809a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b7fff700 1 -- 172.21.15.140:0/1615302900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1b805cf40 msgr2=0x7fb1b809a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:50.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b7fff700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1b805cf40 0x7fb1b809a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b7fff700 1 -- 172.21.15.140:0/1615302900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1b80b1220 msgr2=0x7fb1b8142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:50.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b7fff700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1b80b1220 0x7fb1b8142cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b7fff700 1 -- 172.21.15.140:0/1615302900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8009530 con 0x7fb1b8147c90 2022-07-21T01:51:50.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b77fe700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1b805cf40 0x7fb1b809a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:51:50.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b7fff700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1b8147c90 0x7fb1b81431e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fb1b000c000 tx=0x7fb1b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:50.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b4ff9700 1 -- 172.21.15.140:0/1615302900 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b0016070 con 0x7fb1b8147c90 2022-07-21T01:51:50.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1bde5c700 1 -- 172.21.15.140:0/1615302900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b8142660 con 0x7fb1b8147c90 2022-07-21T01:51:50.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b4ff9700 1 -- 172.21.15.140:0/1615302900 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b0010780 con 0x7fb1b8147c90 2022-07-21T01:51:50.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1b4ff9700 1 -- 172.21.15.140:0/1615302900 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b0010ae0 con 0x7fb1b8147c90 2022-07-21T01:51:50.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.279+0000 7fb1bde5c700 1 -- 172.21.15.140:0/1615302900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b81498f0 con 0x7fb1b8147c90 2022-07-21T01:51:50.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.283+0000 7fb1bde5c700 1 -- 172.21.15.140:0/1615302900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b8143870 con 0x7fb1b8147c90 2022-07-21T01:51:50.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.283+0000 7fb1b4ff9700 1 -- 172.21.15.140:0/1615302900 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb1b0010c40 con 0x7fb1b8147c90 2022-07-21T01:51:50.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.283+0000 7fb1b4ff9700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb198053a70 0x7fb198055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.283+0000 7fb1b4ff9700 1 -- 172.21.15.140:0/1615302900 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fb1b006bde0 con 0x7fb1b8147c90 2022-07-21T01:51:50.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.283+0000 7fb1b77fe700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb198053a70 0x7fb198055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.283+0000 7fb1b77fe700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb198053a70 0x7fb198055f30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fb1ac00b000 tx=0x7fb1ac008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:50.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.291+0000 7fb1b4ff9700 1 -- 172.21.15.140:0/1615302900 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1b0042e50 con 0x7fb1b8147c90 2022-07-21T01:51:50.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.411+0000 7fb1bde5c700 1 -- 172.21.15.140:0/1615302900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb1b805d320 con 0x7fb1b8147c90 2022-07-21T01:51:50.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1b4ff9700 1 -- 172.21.15.140:0/1615302900 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb1b0046470 con 0x7fb1b8147c90 2022-07-21T01:51:50.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1bde5c700 1 -- 172.21.15.140:0/1615302900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb198053a70 msgr2=0x7fb198055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:50.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1bde5c700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb198053a70 0x7fb198055f30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fb1ac00b000 tx=0x7fb1ac008040).stop 2022-07-21T01:51:50.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1bde5c700 1 -- 172.21.15.140:0/1615302900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1b8147c90 msgr2=0x7fb1b81431e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:50.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1bde5c700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1b8147c90 0x7fb1b81431e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fb1b000c000 tx=0x7fb1b0006df0).stop 2022-07-21T01:51:50.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1bde5c700 1 -- 172.21.15.140:0/1615302900 shutdown_connections 2022-07-21T01:51:50.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1bde5c700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb198053a70 0x7fb198055f30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1bde5c700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1b8147c90 0x7fb1b81431e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1bde5c700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1b80b1220 0x7fb1b8142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1bde5c700 1 --2- 172.21.15.140:0/1615302900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1b805cf40 0x7fb1b809a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1bde5c700 1 -- 172.21.15.140:0/1615302900 >> 172.21.15.140:0/1615302900 conn(0x7fb1b80b5b70 msgr2=0x7fb1b8144ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:50.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.419+0000 7fb1bde5c700 1 -- 172.21.15.140:0/1615302900 shutdown_connections 2022-07-21T01:51:50.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.423+0000 7fb1bde5c700 1 -- 172.21.15.140:0/1615302900 wait complete. 2022-07-21T01:51:50.436 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:51:50.437 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:51:50.457 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:51:50.481 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:51:50.505 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:51:50.530 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:51:50.555 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:51:50.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:51:50.557 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:51:50.564 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:51:50.571 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:51:50.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:51:50.573 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:51:50.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:51:50.574 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:51:50.575 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:51:50.582 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:51:50.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:51:50.584 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:51:50.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:51:50.585 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:51:50.585 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:51:50.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:51:50.595 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:51:50.596 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:51:50.596 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:51:50.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.703+0000 7f2c19350700 1 Processor -- start 2022-07-21T01:51:50.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 -- start start 2022-07-21T01:51:50.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2c140b9ef0 0x7f2c140b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2c140b2960 0x7f2c14142210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c140ba2d0 con 0x7f2c1405d160 2022-07-21T01:51:50.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c140b4560 con 0x7f2c140b2960 2022-07-21T01:51:50.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c140b4800 con 0x7f2c140b9ef0 2022-07-21T01:51:50.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c137fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2c140b2960 0x7f2c14142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c127fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2c1405d160 0x7f2c140b2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49104/0 (socket says 172.21.15.140:49104) 2022-07-21T01:51:50.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c127fc700 1 -- 172.21.15.140:0/1190354828 learned_addr learned my addr 172.21.15.140:0/1190354828 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:50.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c12ffd700 1 --2- 172.21.15.140:0/1190354828 >> v2:172.21.15.152:3300/0 conn(0x7f2c140b9ef0 0x7f2c140b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c11ffb700 1 -- 172.21.15.140:0/1190354828 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860472852 0 0) 0x7f2c140ba2d0 con 0x7f2c1405d160 2022-07-21T01:51:50.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c11ffb700 1 -- 172.21.15.140:0/1190354828 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bf4003260 con 0x7f2c1405d160 2022-07-21T01:51:50.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c137fe700 1 -- 172.21.15.140:0/1190354828 >> v2:172.21.15.152:3300/0 conn(0x7f2c140b9ef0 msgr2=0x7f2c140b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:50.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c137fe700 1 --2- 172.21.15.140:0/1190354828 >> v2:172.21.15.152:3300/0 conn(0x7f2c140b9ef0 0x7f2c140b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c137fe700 1 -- 172.21.15.140:0/1190354828 >> v1:172.21.15.140:6789/0 conn(0x7f2c1405d160 legacy=0x7f2c140b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:50.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c11ffb700 1 -- 172.21.15.140:0/1190354828 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2209144140 0 0) 0x7f2bf4003260 con 0x7f2c1405d160 2022-07-21T01:51:50.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c137fe700 1 -- 172.21.15.140:0/1190354828 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c140b1db0 con 0x7f2c140b2960 2022-07-21T01:51:50.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c12ffd700 1 --2- 172.21.15.140:0/1190354828 >> v2:172.21.15.152:3300/0 conn(0x7f2c140b9ef0 0x7f2c140b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:51:50.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c137fe700 1 --2- 172.21.15.140:0/1190354828 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c140b2960 0x7f2c14142210 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f2bfc001800 tx=0x7f2c0c006df0).ready entity=mon.0 client_cookie=e1d42c855a20a739 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:50.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c11ffb700 1 -- 172.21.15.140:0/1190354828 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bfc001660 con 0x7f2c140b2960 2022-07-21T01:51:50.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c11ffb700 1 -- 172.21.15.140:0/1190354828 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0c000e90 con 0x7f2c140b2960 2022-07-21T01:51:50.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c11ffb700 1 -- 172.21.15.140:0/1190354828 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c0c012460 con 0x7f2c140b2960 2022-07-21T01:51:50.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 -- 172.21.15.140:0/1190354828 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c140b2960 msgr2=0x7f2c14142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:50.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 --2- 172.21.15.140:0/1190354828 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c140b2960 0x7f2c14142210 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f2bfc001800 tx=0x7f2c0c006df0).stop 2022-07-21T01:51:50.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 -- 172.21.15.140:0/1190354828 shutdown_connections 2022-07-21T01:51:50.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 --2- 172.21.15.140:0/1190354828 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c140b2960 0x7f2c14142210 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 --2- 172.21.15.140:0/1190354828 >> v2:172.21.15.152:3300/0 conn(0x7f2c140b9ef0 0x7f2c140b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 -- 172.21.15.140:0/1190354828 >> 172.21.15.140:0/1190354828 conn(0x7f2c140679d0 msgr2=0x7f2c140b94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:50.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 -- 172.21.15.140:0/1190354828 shutdown_connections 2022-07-21T01:51:50.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.707+0000 7f2c19350700 1 -- 172.21.15.140:0/1190354828 wait complete. 2022-07-21T01:51:50.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c19350700 1 Processor -- start 2022-07-21T01:51:50.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c19350700 1 -- start start 2022-07-21T01:51:50.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c19350700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c1405d160 0x7f2c140b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c19350700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c140b2960 0x7f2c14146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c19350700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c140b9ef0 0x7f2c1413fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c19350700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c14155be0 con 0x7f2c1405d160 2022-07-21T01:51:50.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c19350700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c141560b0 con 0x7f2c140b2960 2022-07-21T01:51:50.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c19350700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c14155e10 con 0x7f2c140b9ef0 2022-07-21T01:51:50.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c12ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c1405d160 0x7f2c140b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c12ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c1405d160 0x7f2c140b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34962/0 (socket says 172.21.15.140:34962) 2022-07-21T01:51:50.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c12ffd700 1 -- 172.21.15.140:0/3825312918 learned_addr learned my addr 172.21.15.140:0/3825312918 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:50.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c127fc700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c140b2960 0x7f2c14146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c137fe700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c140b9ef0 0x7f2c1413fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c12ffd700 1 -- 172.21.15.140:0/3825312918 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c140b9ef0 msgr2=0x7f2c1413fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:50.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c12ffd700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c140b9ef0 0x7f2c1413fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c12ffd700 1 -- 172.21.15.140:0/3825312918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c140b2960 msgr2=0x7f2c14146310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:50.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c12ffd700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c140b2960 0x7f2c14146310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:50.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c12ffd700 1 -- 172.21.15.140:0/3825312918 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c04009470 con 0x7f2c1405d160 2022-07-21T01:51:50.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c12ffd700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c1405d160 0x7f2c140b7fa0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f2c0400c000 tx=0x7f2c040091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:50.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c0bfff700 1 -- 172.21.15.140:0/3825312918 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c0400c850 con 0x7f2c1405d160 2022-07-21T01:51:50.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c0bfff700 1 -- 172.21.15.140:0/3825312918 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0400c9b0 con 0x7f2c1405d160 2022-07-21T01:51:50.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c19350700 1 -- 172.21.15.140:0/3825312918 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c00b180 con 0x7f2c1405d160 2022-07-21T01:51:50.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c0bfff700 1 -- 172.21.15.140:0/3825312918 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c0400cd10 con 0x7f2c1405d160 2022-07-21T01:51:50.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.711+0000 7f2c19350700 1 -- 172.21.15.140:0/3825312918 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c14152d30 con 0x7f2c1405d160 2022-07-21T01:51:50.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.715+0000 7f2c19350700 1 -- 172.21.15.140:0/3825312918 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c140b3780 con 0x7f2c1405d160 2022-07-21T01:51:50.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.715+0000 7f2c0bfff700 1 -- 172.21.15.140:0/3825312918 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2c04022070 con 0x7f2c1405d160 2022-07-21T01:51:50.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.715+0000 7f2c0bfff700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bf80539e0 0x7f2bf8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:50.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.715+0000 7f2c0bfff700 1 -- 172.21.15.140:0/3825312918 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f2c0406c2b0 con 0x7f2c1405d160 2022-07-21T01:51:50.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.715+0000 7f2c127fc700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bf80539e0 0x7f2bf8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:50.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.715+0000 7f2c127fc700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bf80539e0 0x7f2bf8055ea0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f2bfc00d000 tx=0x7f2bfc001de0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:50.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.723+0000 7f2c0bfff700 1 -- 172.21.15.140:0/3825312918 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2c040434b0 con 0x7f2c1405d160 2022-07-21T01:51:50.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.851+0000 7f2c19350700 1 -- 172.21.15.140:0/3825312918 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2c140574e0 con 0x7f2c1405d160 2022-07-21T01:51:50.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.855+0000 7f2c0bfff700 1 -- 172.21.15.140:0/3825312918 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v45) v1 ==== 81+0+28923 (secure 0 0 0) 0x7f2c04046ad0 con 0x7f2c1405d160 2022-07-21T01:51:50.856 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:50.856 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:51:50.857 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 45, 2022-07-21T01:51:50.857 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:51:50.858 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:51:50.858 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:51:48.118980+0000", 2022-07-21T01:51:50.859 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:51:50.859 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:51:50.860 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:51:50.860 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:51:50.861 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:51:50.861 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:51:50.862 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:51:50.862 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:51:50.863 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:51:50.864 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:51:50.865 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:51:50.866 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:51:50.866 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:51:50.867 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:51:50.867 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:51:50.868 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 8, 2022-07-21T01:51:50.868 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:51:50.869 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:51:50.869 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:51:50.869 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:51:50.870 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:51:50.870 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.871 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:51:50.871 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:51:50.872 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:51:50.872 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:51:50.873 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:51:50.873 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:51:50.874 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:51:50.874 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:51:50.875 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:51:50.875 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:51:50.875 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:51:50.876 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:51:50.876 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:51:50.877 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:51:50.877 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:51:50.878 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:51:50.878 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:51:50.879 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:51:50.879 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:51:50.880 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:51:50.880 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:51:50.881 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:51:50.881 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:51:50.882 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.882 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:51:50.883 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:51:50.883 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:51:50.884 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:51:50.885 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:51:50.886 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:51:50.886 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:51:50.887 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:51:50.887 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:51:50.888 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:51:50.888 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:51:50.888 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:51:50.889 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:51:50.889 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:51:50.890 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:51:50.890 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:51:50.891 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:51:50.891 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:51:50.892 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:51:50.892 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:51:50.893 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:51:50.893 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:51:50.894 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:51:50.894 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:51:50.895 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:51:50.895 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:51:50.895 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:51:50.896 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.896 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:51:50.897 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:51:50.897 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:51:50.898 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:51:50.898 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:51:50.899 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:51:50.899 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:51:50.900 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:51:50.900 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:51:50.901 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:51:50.901 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:51:50.901 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:51:50.902 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:51:50.902 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.903 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:51:50.903 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:51:50.904 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:50.904 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.905 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.905 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:51:50.906 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:51:50.906 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:51:50.907 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:51:50.907 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:51:50.908 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:51:50.908 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:51:50.908 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:51:50.909 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:51:50.909 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:51:50.910 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:51:50.910 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:51:50.911 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:51:50.911 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:51:50.912 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:51:50.912 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:51:50.912 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:51:50.913 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:51:50.913 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:51:50.913 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:51:50.914 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:51:50.914 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:51:50.915 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:51:50.915 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.916 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:51:50.916 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:51:50.916 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:51:50.917 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:51:50.917 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:51:50.918 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:51:50.918 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:51:50.919 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:51:50.919 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:51:50.920 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:51:50.920 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:51:50.921 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:51:50.921 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:51:50.921 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:51:50.922 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:51:50.922 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:51:50.923 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:51:50.923 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:51:50.924 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:51:50.924 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:51:50.924 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:51:50.924 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:51:50.925 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:51:50.925 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:51:50.925 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:51:50.927 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:51:50.928 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:51:50.928 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.929 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:51:50.930 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:51:50.930 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:51:50.931 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:51:50.931 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:51:50.932 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:51:50.932 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:51:50.933 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:51:50.933 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:51:50.934 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:51:50.934 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:51:50.935 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:51:50.935 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:51:50.936 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:51:50.936 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:50.937 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:50.937 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:51:50.938 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:51:50.938 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.939 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:51:50.941 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:51:50.941 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:50.942 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:50.942 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:50.943 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:50.943 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:50.944 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:50.944 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:50.945 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 39, 2022-07-21T01:51:50.945 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:50.946 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:50.946 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:50.946 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:50.947 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.947 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:50.948 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:51:50.948 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:50.949 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.949 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.949 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:50.950 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:51:50.950 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:50.951 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:50.951 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:50.952 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.952 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:50.952 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:50.953 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.953 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:50.954 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:51:50.954 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:50.955 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.955 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.956 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:50.956 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:51:50.957 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:50.957 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:50.957 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:50.958 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.958 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:50.958 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:50.959 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.959 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:50.960 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:51:50.960 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:50.961 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.961 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.962 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:50.962 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:51:50.962 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:50.963 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:50.963 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:50.964 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.964 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:50.965 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:50.965 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.965 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:50.966 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:51:50.966 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:50.967 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.967 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.968 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:50.969 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:51:50.969 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:51:50.970 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:50.970 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:50.970 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.971 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:51:50.971 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:51:50.972 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:51:50.972 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:51:50.973 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:50.973 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:50.974 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:50.974 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:50.974 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.975 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.975 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:51:50.976 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:51:50.976 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:50.977 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:50.977 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:50.977 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:50.978 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:50.978 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:50.979 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:50.979 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 40, 2022-07-21T01:51:50.980 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:50.980 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:50.981 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:50.981 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:50.982 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.982 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:50.983 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:51:50.983 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:50.983 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.983 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.984 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:50.984 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:51:50.984 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:50.984 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:50.985 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:50.985 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.985 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:50.986 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:50.986 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.986 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:50.986 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:51:50.987 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:50.987 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.987 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.987 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:50.988 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:51:50.988 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:50.988 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:50.988 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:50.988 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.989 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:50.989 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:50.989 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.989 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:50.990 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:51:50.990 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:50.990 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.990 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.991 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:50.991 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:51:50.991 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:50.991 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:50.991 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:50.992 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.992 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:50.992 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:50.992 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.993 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:50.993 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:51:50.993 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:50.993 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.993 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.994 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:50.994 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:51:50.994 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:51:50.994 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:50.995 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:50.995 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.995 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:51:50.995 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:51:50.996 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:51:50.996 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:51:50.996 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:50.996 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:50.996 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:50.997 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:50.997 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:50.997 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:50.997 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:51:50.998 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:51:50.998 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:50.998 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:50.998 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:50.999 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:50.999 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:50.999 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:50.999 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:50.999 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 29, 2022-07-21T01:51:51.000 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:51.000 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:51.000 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:51.000 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.001 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.001 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.001 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:51:51.001 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:51.001 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.002 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.002 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.002 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:51:51.002 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:51.003 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.003 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.003 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.003 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:51.004 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.004 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.004 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.004 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:51:51.005 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:51.005 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.005 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.005 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.006 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:51:51.006 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:51.006 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.006 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.006 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.007 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:51.007 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.007 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.007 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.008 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:51:51.008 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:51.008 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.008 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.009 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.009 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:51:51.009 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:51.009 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.010 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.010 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.010 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:51.010 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.011 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.011 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.011 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:51:51.012 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:51.012 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.012 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.013 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.013 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:51:51.013 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:51:51.013 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.014 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.014 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.014 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:51:51.014 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:51:51.015 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:51:51.015 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:51:51.015 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:51.015 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:51.016 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:51.016 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.016 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.017 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.017 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:51:51.017 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:51:51.017 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:51.017 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:51.018 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:51.018 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:51.018 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:51.018 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:51.019 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:51.019 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 39, 2022-07-21T01:51:51.019 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:51.019 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:51.020 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:51.020 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.020 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.020 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.020 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:51:51.021 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:51.021 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.021 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.021 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.022 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:51:51.022 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:51.022 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.022 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.023 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.023 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:51.023 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.024 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.024 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.024 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:51:51.024 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:51.025 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.025 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.025 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.025 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:51:51.025 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:51.026 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.026 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.026 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.026 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:51.027 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.027 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.027 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.027 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:51:51.028 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:51.028 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.028 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.028 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.029 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:51:51.029 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:51.029 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.029 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.030 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.030 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:51.030 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.030 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.030 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.031 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:51:51.031 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:51.031 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.031 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.032 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.032 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:51:51.032 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:51:51.032 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.033 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.033 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.033 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:51:51.033 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:51:51.034 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:51:51.034 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:51:51.034 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:51.034 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:51.035 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:51.035 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.035 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.035 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.036 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:51:51.036 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:51:51.036 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:51.036 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:51.037 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:51.037 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:51.037 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:51.037 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:51.037 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:51.038 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 39, 2022-07-21T01:51:51.038 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:51.039 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:51.039 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:51.039 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.040 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.040 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.041 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:51:51.041 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:51.042 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.042 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.042 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.043 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:51:51.043 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:51.044 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.044 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.045 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.045 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:51.046 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.046 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.047 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.047 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:51:51.047 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:51.048 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.048 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.049 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.049 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:51:51.049 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:51.050 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.050 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.051 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.051 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:51.052 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.052 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.053 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.053 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:51:51.054 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:51.054 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.054 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.055 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.055 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:51:51.056 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:51.056 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.057 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.057 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.057 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:51.058 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.058 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.059 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.059 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:51:51.060 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:51.060 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.060 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.060 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.061 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:51:51.061 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:51:51.061 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.061 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.062 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.062 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:51:51.062 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:51:51.062 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:51:51.063 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:51:51.063 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:51.063 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:51.063 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:51.063 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.064 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.064 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.064 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:51:51.064 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:51:51.065 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:51.065 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:51.065 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:51.065 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:51.065 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:51.066 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:51.066 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:51.066 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 39, 2022-07-21T01:51:51.066 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:51.067 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:51.067 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:51.067 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.067 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.067 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.068 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:51:51.068 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:51.068 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.068 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.069 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.069 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:51:51.069 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:51.069 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.070 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.070 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.070 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:51.070 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.070 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.071 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.071 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:51:51.071 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:51.071 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.072 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.072 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.072 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:51:51.072 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:51.072 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.073 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.073 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.073 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:51.073 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.074 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.074 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.074 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:51:51.074 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:51.075 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.075 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.075 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.075 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:51:51.075 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:51.076 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.076 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.076 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.076 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:51.077 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.077 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.077 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.077 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:51:51.077 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:51.078 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.078 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.078 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.078 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:51:51.079 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:51:51.079 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.079 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.079 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.079 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:51:51.080 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:51:51.080 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:51:51.080 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:51:51.080 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:51.081 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:51.081 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:51.081 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.081 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.081 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.082 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:51:51.082 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:51:51.082 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:51.082 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:51.083 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:51.083 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:51.083 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:51.083 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:51.084 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:51.084 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 39, 2022-07-21T01:51:51.084 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:51.084 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:51.084 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:51.085 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.085 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.085 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.085 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:51:51.086 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:51.086 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.086 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.086 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.086 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:51:51.087 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:51.087 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.087 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.087 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.088 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:51.088 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.088 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.088 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.088 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:51:51.089 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:51.089 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.089 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.089 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.090 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:51:51.090 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:51.090 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.090 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.091 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.091 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:51.091 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.091 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.091 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.092 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:51:51.092 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:51.092 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.092 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.093 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.093 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:51:51.093 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:51.093 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.093 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.094 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.094 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:51.094 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.094 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.095 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.095 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:51:51.095 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:51.095 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.095 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.096 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.096 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:51:51.096 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:51:51.096 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.097 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.097 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.097 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:51:51.097 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:51:51.098 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:51:51.098 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:51:51.098 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:51.098 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:51.098 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:51.099 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.099 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.099 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.099 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:51:51.100 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:51:51.100 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:51:51.100 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:51:51.100 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:51:51.100 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:51:51.101 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:51:51.101 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:51:51.101 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:51:51.101 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 38, 2022-07-21T01:51:51.102 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:51:51.102 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:51:51.102 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:51:51.102 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.103 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.103 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.103 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:51:51.103 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:51.103 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.104 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.104 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.104 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:51:51.104 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:51.105 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.105 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.105 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.105 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:51:51.105 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.106 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.106 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.106 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:51:51.106 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:51.107 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.107 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.107 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.107 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:51:51.107 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:51.108 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.108 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.108 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.108 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:51:51.109 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.109 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.109 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.109 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:51:51.109 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:51.110 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.110 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.110 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.110 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:51:51.111 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:51.111 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.111 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.111 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.112 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:51:51.112 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:51:51.112 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.112 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:51:51.112 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:51:51.113 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:51.113 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.113 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.113 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:51:51.114 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:51:51.114 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:51:51.114 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.114 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.114 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.115 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:51:51.115 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:51:51.115 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:51:51.115 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:51:51.116 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:51:51.116 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:51:51.116 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:51:51.116 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:51.116 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.117 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:51:51.117 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:51:51.117 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.117 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:51:51.118 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:51.118 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:51.118 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:51.118 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:51.119 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:51.119 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:51:51.119 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:51.119 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.119 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.120 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:51:51.120 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:51.120 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:51.120 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:51.121 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:51.121 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:51.121 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:51:51.121 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:51.121 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.122 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.122 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:51:51.122 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:51.122 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:51.123 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:51.123 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:51.123 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:51.123 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:51:51.123 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:51.124 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.124 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.124 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:51:51.124 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:51.125 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:51.125 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:51.125 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:51.125 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:51.125 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:51:51.126 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:51.126 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.126 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.126 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:51:51.127 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:51.127 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:51.127 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:51.127 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:51.128 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:51.128 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:51:51.128 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:51.128 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.128 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.129 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:51:51.129 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:51.129 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:51.129 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:51.130 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:51.130 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:51.130 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:51:51.130 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:51.130 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.131 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.131 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:51:51.131 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:51.131 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:51.132 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:51.132 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:51.132 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:51.132 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:51:51.132 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:51.133 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.133 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:51.133 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:51:51.133 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:51:51.134 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:51:51.134 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:51:51.134 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:51:51.134 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:51:51.135 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:51:51.135 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:51:51.135 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.135 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:51:51.135 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:51:51.136 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:51:51.136 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:51:51.136 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:51:51.136 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:51:51.137 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6835/207391195": "2022-07-22T01:51:44.379724+0000", 2022-07-21T01:51:51.137 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6834/207391195": "2022-07-22T01:51:44.379724+0000" 2022-07-21T01:51:51.137 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.137 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:51:51.138 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:51:51.138 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:51:51.138 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:51:51.138 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:51:51.138 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:51:51.139 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:51:51.139 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:51:51.139 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:51.139 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:51.140 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:51:51.140 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:51:51.140 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:51:51.140 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:51:51.141 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:51:51.141 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:51:51.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.855+0000 7f2c19350700 1 -- 172.21.15.140:0/3825312918 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bf80539e0 msgr2=0x7f2bf8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.855+0000 7f2c19350700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bf80539e0 0x7f2bf8055ea0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f2bfc00d000 tx=0x7f2bfc001de0).stop 2022-07-21T01:51:51.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.855+0000 7f2c19350700 1 -- 172.21.15.140:0/3825312918 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c1405d160 msgr2=0x7f2c140b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.855+0000 7f2c19350700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c1405d160 0x7f2c140b7fa0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f2c0400c000 tx=0x7f2c040091d0).stop 2022-07-21T01:51:51.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.855+0000 7f2c19350700 1 -- 172.21.15.140:0/3825312918 shutdown_connections 2022-07-21T01:51:51.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.855+0000 7f2c19350700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2bf80539e0 0x7f2bf8055ea0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.855+0000 7f2c19350700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2c140b9ef0 0x7f2c1413fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.855+0000 7f2c19350700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c140b2960 0x7f2c14146310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.855+0000 7f2c19350700 1 --2- 172.21.15.140:0/3825312918 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2c1405d160 0x7f2c140b7fa0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.855+0000 7f2c19350700 1 -- 172.21.15.140:0/3825312918 >> 172.21.15.140:0/3825312918 conn(0x7f2c140679d0 msgr2=0x7f2c140b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:51.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.859+0000 7f2c19350700 1 -- 172.21.15.140:0/3825312918 shutdown_connections 2022-07-21T01:51:51.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:50.859+0000 7f2c19350700 1 -- 172.21.15.140:0/3825312918 wait complete. 2022-07-21T01:51:51.146 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:51:51.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.251+0000 7f702e958700 1 Processor -- start 2022-07-21T01:51:51.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.251+0000 7f702e958700 1 -- start start 2022-07-21T01:51:51.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70280ac260 0x7f70280ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f70280b3620 0x7f7028096890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70280b22d0 con 0x7f7028141200 2022-07-21T01:51:51.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70280b26d0 con 0x7f70280b3620 2022-07-21T01:51:51.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7028140ac0 con 0x7f70280ac260 2022-07-21T01:51:51.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f70277fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f70280b3620 0x7f7028096890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f7027fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70280ac260 0x7f70280ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f70277fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f70280b3620 0x7f7028096890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:40710/0 (socket says 172.21.15.140:40710) 2022-07-21T01:51:51.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f70277fe700 1 -- 172.21.15.140:0/3864688555 learned_addr learned my addr 172.21.15.140:0/3864688555 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:51.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f70277fe700 1 -- 172.21.15.140:0/3864688555 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70280ac260 msgr2=0x7f70280ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:51.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f70277fe700 1 --2- 172.21.15.140:0/3864688555 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70280ac260 0x7f70280ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f70277fe700 1 -- 172.21.15.140:0/3864688555 >> v1:172.21.15.152:6789/0 conn(0x7f7028141200 legacy=0x7f70281400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f70277fe700 1 -- 172.21.15.140:0/3864688555 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7028140d60 con 0x7f70280b3620 2022-07-21T01:51:51.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f70277fe700 1 --2- 172.21.15.140:0/3864688555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70280b3620 0x7f7028096890 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f7018002f70 tx=0x7f7018006e50).ready entity=mon.1 client_cookie=d4ed37d070253eab server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:51.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f7026ffd700 1 -- 172.21.15.140:0/3864688555 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f701800d5c0 con 0x7f70280b3620 2022-07-21T01:51:51.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f7026ffd700 1 -- 172.21.15.140:0/3864688555 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7018000e90 con 0x7f70280b3620 2022-07-21T01:51:51.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f7026ffd700 1 -- 172.21.15.140:0/3864688555 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f701800d950 con 0x7f70280b3620 2022-07-21T01:51:51.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 -- 172.21.15.140:0/3864688555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70280b3620 msgr2=0x7f7028096890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 --2- 172.21.15.140:0/3864688555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70280b3620 0x7f7028096890 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f7018002f70 tx=0x7f7018006e50).stop 2022-07-21T01:51:51.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 -- 172.21.15.140:0/3864688555 shutdown_connections 2022-07-21T01:51:51.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 --2- 172.21.15.140:0/3864688555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70280b3620 0x7f7028096890 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 --2- 172.21.15.140:0/3864688555 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70280ac260 0x7f70280ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 -- 172.21.15.140:0/3864688555 >> 172.21.15.140:0/3864688555 conn(0x7f70280b47d0 msgr2=0x7f702805bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:51.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 -- 172.21.15.140:0/3864688555 shutdown_connections 2022-07-21T01:51:51.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.255+0000 7f702e958700 1 -- 172.21.15.140:0/3864688555 wait complete. 2022-07-21T01:51:51.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f702e958700 1 Processor -- start 2022-07-21T01:51:51.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f702e958700 1 -- start start 2022-07-21T01:51:51.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f702e958700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70280ac260 0x7f7028093bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f702e958700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70280b3620 0x7f7028067020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f702e958700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7028141200 0x7f7028067510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f702e958700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7028094c20 con 0x7f70280b3620 2022-07-21T01:51:51.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f702e958700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702813d600 con 0x7f7028141200 2022-07-21T01:51:51.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f702e958700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7028060000 con 0x7f70280ac260 2022-07-21T01:51:51.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70277fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70280b3620 0x7f7028067020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70277fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70280b3620 0x7f7028067020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34980/0 (socket says 172.21.15.140:34980) 2022-07-21T01:51:51.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70277fe700 1 -- 172.21.15.140:0/3114178912 learned_addr learned my addr 172.21.15.140:0/3114178912 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:51.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f702cef5700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7028141200 0x7f7028067510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f7027fff700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70280ac260 0x7f7028093bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70277fe700 1 -- 172.21.15.140:0/3114178912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70280ac260 msgr2=0x7f7028093bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70277fe700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70280ac260 0x7f7028093bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70277fe700 1 -- 172.21.15.140:0/3114178912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7028141200 msgr2=0x7f7028067510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70277fe700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7028141200 0x7f7028067510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70277fe700 1 -- 172.21.15.140:0/3114178912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701800b1e0 con 0x7f70280b3620 2022-07-21T01:51:51.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f7027fff700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70280ac260 0x7f7028093bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:51:51.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70277fe700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70280b3620 0x7f7028067020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f7018002f70 tx=0x7f7018014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:51.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70257fa700 1 -- 172.21.15.140:0/3114178912 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7018013e90 con 0x7f70280b3620 2022-07-21T01:51:51.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70257fa700 1 -- 172.21.15.140:0/3114178912 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701801e400 con 0x7f70280b3620 2022-07-21T01:51:51.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f702e958700 1 -- 172.21.15.140:0/3114178912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70280610b0 con 0x7f70280b3620 2022-07-21T01:51:51.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f70257fa700 1 -- 172.21.15.140:0/3114178912 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f701801e760 con 0x7f70280b3620 2022-07-21T01:51:51.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.259+0000 7f702e958700 1 -- 172.21.15.140:0/3114178912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70280940b0 con 0x7f70280b3620 2022-07-21T01:51:51.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.263+0000 7f70257fa700 1 -- 172.21.15.140:0/3114178912 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f701801c070 con 0x7f70280b3620 2022-07-21T01:51:51.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.263+0000 7f702e958700 1 -- 172.21.15.140:0/3114178912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70281407d0 con 0x7f70280b3620 2022-07-21T01:51:51.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.263+0000 7f70257fa700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7008053aa0 0x7f7008055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.263+0000 7f70257fa700 1 -- 172.21.15.140:0/3114178912 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f701806c810 con 0x7f70280b3620 2022-07-21T01:51:51.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.263+0000 7f7027fff700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7008053aa0 0x7f7008055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.263+0000 7f7027fff700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7008053aa0 0x7f7008055f60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f701c00b000 tx=0x7f701c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:51.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.267+0000 7f70257fa700 1 -- 172.21.15.140:0/3114178912 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f70180438c0 con 0x7f70280b3620 2022-07-21T01:51:51.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.427+0000 7f702e958700 1 -- 172.21.15.140:0/3114178912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70280574e0 con 0x7f70280b3620 2022-07-21T01:51:51.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.427+0000 7f70257fa700 1 -- 172.21.15.140:0/3114178912 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+1215 (secure 0 0 0) 0x7f7018023020 con 0x7f70280b3620 2022-07-21T01:51:51.429 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:51.429 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/2307860244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2307860244},{"type":"v1","addr":"172.21.15.152:6833","nonce":2307860244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5538,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1294026792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1294026792},{"type":"v1","addr":"172.21.15.140:6835","nonce":1294026792}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":18}],"filesystems":[]} 2022-07-21T01:51:51.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 -- 172.21.15.140:0/3114178912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7008053aa0 msgr2=0x7f7008055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7008053aa0 0x7f7008055f60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f701c00b000 tx=0x7f701c008040).stop 2022-07-21T01:51:51.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 -- 172.21.15.140:0/3114178912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70280b3620 msgr2=0x7f7028067020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70280b3620 0x7f7028067020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f7018002f70 tx=0x7f7018014000).stop 2022-07-21T01:51:51.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 -- 172.21.15.140:0/3114178912 shutdown_connections 2022-07-21T01:51:51.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7008053aa0 0x7f7008055f60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7028141200 0x7f7028067510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70280b3620 0x7f7028067020 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 --2- 172.21.15.140:0/3114178912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70280ac260 0x7f7028093bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 -- 172.21.15.140:0/3114178912 >> 172.21.15.140:0/3114178912 conn(0x7f70280b47d0 msgr2=0x7f70280b61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:51.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 -- 172.21.15.140:0/3114178912 shutdown_connections 2022-07-21T01:51:51.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.431+0000 7f702e958700 1 -- 172.21.15.140:0/3114178912 wait complete. 2022-07-21T01:51:51.440 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 18 2022-07-21T01:51:51.442 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:51:51.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab0c9c1700 1 Processor -- start 2022-07-21T01:51:51.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab0c9c1700 1 -- start start 2022-07-21T01:51:51.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab0c9c1700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fab080b5650 0x7fab080b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab0c9c1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab080af8e0 0x7fab0805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab0c9c1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab08147010 con 0x7fab080adff0 2022-07-21T01:51:51.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab0c9c1700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab081472b0 con 0x7fab080b5650 2022-07-21T01:51:51.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab0c9c1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab08146d40 con 0x7fab080af8e0 2022-07-21T01:51:51.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab06ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fab080adff0 0x7fab08147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49130/0 (socket says 172.21.15.140:49130) 2022-07-21T01:51:51.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab06ffd700 1 -- 172.21.15.140:0/2608175870 learned_addr learned my addr 172.21.15.140:0/2608175870 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:51.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab077fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fab080b5650 0x7fab080b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab07fff700 1 --2- 172.21.15.140:0/2608175870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab080af8e0 0x7fab0805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab067fc700 1 -- 172.21.15.140:0/2608175870 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256044044 0 0) 0x7fab08147010 con 0x7fab080adff0 2022-07-21T01:51:51.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.551+0000 7fab067fc700 1 -- 172.21.15.140:0/2608175870 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaec003260 con 0x7fab080adff0 2022-07-21T01:51:51.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab067fc700 1 -- 172.21.15.140:0/2608175870 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 853592008 0 0) 0x7faaec003260 con 0x7fab080adff0 2022-07-21T01:51:51.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab077fe700 1 -- 172.21.15.140:0/2608175870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab080af8e0 msgr2=0x7fab0805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:51.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab077fe700 1 --2- 172.21.15.140:0/2608175870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab080af8e0 0x7fab0805bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab077fe700 1 -- 172.21.15.140:0/2608175870 >> v1:172.21.15.140:6789/0 conn(0x7fab080adff0 legacy=0x7fab08147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab077fe700 1 -- 172.21.15.140:0/2608175870 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab080acae0 con 0x7fab080b5650 2022-07-21T01:51:51.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab077fe700 1 --2- 172.21.15.140:0/2608175870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab080b5650 0x7fab080b5a30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7faaf0001d00 tx=0x7fab00006df0).ready entity=mon.0 client_cookie=fb3a563cf901f877 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:51.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab067fc700 1 -- 172.21.15.140:0/2608175870 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaf0001b60 con 0x7fab080b5650 2022-07-21T01:51:51.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab067fc700 1 -- 172.21.15.140:0/2608175870 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab00000e90 con 0x7fab080b5650 2022-07-21T01:51:51.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab067fc700 1 -- 172.21.15.140:0/2608175870 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab00012460 con 0x7fab080b5650 2022-07-21T01:51:51.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2608175870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab080b5650 msgr2=0x7fab080b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab0c9c1700 1 --2- 172.21.15.140:0/2608175870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab080b5650 0x7fab080b5a30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7faaf0001d00 tx=0x7fab00006df0).stop 2022-07-21T01:51:51.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2608175870 shutdown_connections 2022-07-21T01:51:51.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab0c9c1700 1 --2- 172.21.15.140:0/2608175870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab080af8e0 0x7fab0805bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab0c9c1700 1 --2- 172.21.15.140:0/2608175870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab080b5650 0x7fab080b5a30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2608175870 >> 172.21.15.140:0/2608175870 conn(0x7fab080634e0 msgr2=0x7fab080b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:51.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2608175870 shutdown_connections 2022-07-21T01:51:51.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2608175870 wait complete. 2022-07-21T01:51:51.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab0c9c1700 1 Processor -- start 2022-07-21T01:51:51.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.555+0000 7fab0c9c1700 1 -- start start 2022-07-21T01:51:51.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab0c9c1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab080adff0 0x7fab08149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab0c9c1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab080af8e0 0x7fab0815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab0c9c1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab080b5650 0x7fab0815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab0c9c1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab08064490 con 0x7fab080adff0 2022-07-21T01:51:51.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab0c9c1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab08110000 con 0x7fab080b5650 2022-07-21T01:51:51.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab0c9c1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0810fac0 con 0x7fab080af8e0 2022-07-21T01:51:51.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab077fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab080adff0 0x7fab08149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab077fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab080adff0 0x7fab08149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:34990/0 (socket says 172.21.15.140:34990) 2022-07-21T01:51:51.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab077fe700 1 -- 172.21.15.140:0/2226098463 learned_addr learned my addr 172.21.15.140:0/2226098463 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:51.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab06ffd700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab080af8e0 0x7fab0815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab07fff700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab080b5650 0x7fab0815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab077fe700 1 -- 172.21.15.140:0/2226098463 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab080af8e0 msgr2=0x7fab0815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab077fe700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab080af8e0 0x7fab0815a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab077fe700 1 -- 172.21.15.140:0/2226098463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab080b5650 msgr2=0x7fab0815a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab077fe700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab080b5650 0x7fab0815a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab077fe700 1 -- 172.21.15.140:0/2226098463 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0000b180 con 0x7fab080adff0 2022-07-21T01:51:51.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab077fe700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab080adff0 0x7fab08149170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fab08110190 tx=0x7fab00014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:51.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab04ff9700 1 -- 172.21.15.140:0/2226098463 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab00016070 con 0x7fab080adff0 2022-07-21T01:51:51.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab04ff9700 1 -- 172.21.15.140:0/2226098463 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab00013e40 con 0x7fab080adff0 2022-07-21T01:51:51.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2226098463 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0810fd60 con 0x7fab080adff0 2022-07-21T01:51:51.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab04ff9700 1 -- 172.21.15.140:0/2226098463 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0000dd70 con 0x7fab080adff0 2022-07-21T01:51:51.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2226098463 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab08149660 con 0x7fab080adff0 2022-07-21T01:51:51.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.559+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2226098463 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0805c280 con 0x7fab080adff0 2022-07-21T01:51:51.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.563+0000 7fab04ff9700 1 -- 172.21.15.140:0/2226098463 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fab0000f030 con 0x7fab080adff0 2022-07-21T01:51:51.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.563+0000 7fab04ff9700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faae0053b00 0x7faae0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.563+0000 7fab04ff9700 1 -- 172.21.15.140:0/2226098463 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fab0006c930 con 0x7fab080adff0 2022-07-21T01:51:51.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.563+0000 7fab06ffd700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faae0053b00 0x7faae0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.563+0000 7fab06ffd700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faae0053b00 0x7faae0055fc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7faaf000d000 tx=0x7faaf00031d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:51.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.567+0000 7fab04ff9700 1 -- 172.21.15.140:0/2226098463 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab00043be0 con 0x7fab080adff0 2022-07-21T01:51:51.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.731+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2226098463 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab080574e0 con 0x7fab080adff0 2022-07-21T01:51:51.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.731+0000 7fab04ff9700 1 -- 172.21.15.140:0/2226098463 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+1215 (secure 0 0 0) 0x7fab00047200 con 0x7fab080adff0 2022-07-21T01:51:51.732 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:51.733 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4946,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/2307860244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2307860244},{"type":"v1","addr":"172.21.15.152:6833","nonce":2307860244}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5538,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1294026792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1294026792},{"type":"v1","addr":"172.21.15.140:6835","nonce":1294026792}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":18}],"filesystems":[]} 2022-07-21T01:51:51.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.731+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2226098463 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faae0053b00 msgr2=0x7faae0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.731+0000 7fab0c9c1700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faae0053b00 0x7faae0055fc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7faaf000d000 tx=0x7faaf00031d0).stop 2022-07-21T01:51:51.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.731+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2226098463 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab080adff0 msgr2=0x7fab08149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.731+0000 7fab0c9c1700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab080adff0 0x7fab08149170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fab08110190 tx=0x7fab00014000).stop 2022-07-21T01:51:51.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.735+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2226098463 shutdown_connections 2022-07-21T01:51:51.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.735+0000 7fab0c9c1700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faae0053b00 0x7faae0055fc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.735+0000 7fab0c9c1700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab080b5650 0x7fab0815a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.735+0000 7fab0c9c1700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab080af8e0 0x7fab0815a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.735+0000 7fab0c9c1700 1 --2- 172.21.15.140:0/2226098463 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab080adff0 0x7fab08149170 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.735+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2226098463 >> 172.21.15.140:0/2226098463 conn(0x7fab080634e0 msgr2=0x7fab080ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:51.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.735+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2226098463 shutdown_connections 2022-07-21T01:51:51.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.735+0000 7fab0c9c1700 1 -- 172.21.15.140:0/2226098463 wait complete. 2022-07-21T01:51:51.739 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 18 2022-07-21T01:51:51.745 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:51:51.746 INFO:tasks.ceph.mds.a:Stopping old one... 2022-07-21T01:51:51.746 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:51:51.747 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:51:51.747 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:51:51.747 INFO:tasks.ceph.mds.b:Stopping old one... 2022-07-21T01:51:51.748 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:51:51.748 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:51:51.755 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:51:51.755 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:51:51.758 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:51:51.759 DEBUG:teuthology.parallel:result is None 2022-07-21T01:51:51.764 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:51:51.764 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:51:51.767 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:51:51.768 DEBUG:teuthology.parallel:result is None 2022-07-21T01:51:51.769 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:51:51.834 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:51:51.839 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:51:51.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e399d3700 1 Processor -- start 2022-07-21T01:51:51.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e399d3700 1 -- start start 2022-07-21T01:51:51.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e399d3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e340b11b0 0x7f5e3405d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e399d3700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5e34060bb0 0x7f5e34147760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e399d3700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e34147090 con 0x7f5e3405cda0 2022-07-21T01:51:51.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e399d3700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e34060160 con 0x7f5e34060bb0 2022-07-21T01:51:51.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e399d3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34099080 con 0x7f5e340b11b0 2022-07-21T01:51:51.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e327fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e340b11b0 0x7f5e3405d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e337fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5e34060bb0 0x7f5e34147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e32ffd700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f5e3405cda0 0x7f5e3405fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:50270/0 (socket says 172.21.15.140:50270) 2022-07-21T01:51:51.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e32ffd700 1 -- 172.21.15.140:0/1730007109 learned_addr learned my addr 172.21.15.140:0/1730007109 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:51.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e327fc700 1 -- 172.21.15.140:0/1730007109 >> v2:172.21.15.152:3300/0 conn(0x7f5e34060bb0 msgr2=0x7f5e34147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:51.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e327fc700 1 --2- 172.21.15.140:0/1730007109 >> v2:172.21.15.152:3300/0 conn(0x7f5e34060bb0 0x7f5e34147760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e327fc700 1 -- 172.21.15.140:0/1730007109 >> v1:172.21.15.152:6789/0 conn(0x7f5e3405cda0 legacy=0x7f5e3405fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e327fc700 1 -- 172.21.15.140:0/1730007109 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e341472c0 con 0x7f5e340b11b0 2022-07-21T01:51:51.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.863+0000 7f5e337fe700 1 --2- 172.21.15.140:0/1730007109 >> v2:172.21.15.152:3300/0 conn(0x7f5e34060bb0 0x7f5e34147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:51:51.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.867+0000 7f5e327fc700 1 --2- 172.21.15.140:0/1730007109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e340b11b0 0x7f5e3405d180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f5e24002f70 tx=0x7f5e24006df0).ready entity=mon.2 client_cookie=d167ddc32e844aa7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:51.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.871+0000 7f5e31ffb700 1 -- 172.21.15.140:0/1730007109 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e2400d570 con 0x7f5e340b11b0 2022-07-21T01:51:51.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.871+0000 7f5e31ffb700 1 -- 172.21.15.140:0/1730007109 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24000e90 con 0x7f5e340b11b0 2022-07-21T01:51:51.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.871+0000 7f5e31ffb700 1 -- 172.21.15.140:0/1730007109 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e2400d8c0 con 0x7f5e340b11b0 2022-07-21T01:51:51.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.871+0000 7f5e399d3700 1 -- 172.21.15.140:0/1730007109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e340b11b0 msgr2=0x7f5e3405d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:51.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.871+0000 7f5e399d3700 1 --2- 172.21.15.140:0/1730007109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e340b11b0 0x7f5e3405d180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f5e24002f70 tx=0x7f5e24006df0).stop 2022-07-21T01:51:51.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.871+0000 7f5e399d3700 1 -- 172.21.15.140:0/1730007109 shutdown_connections 2022-07-21T01:51:51.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.871+0000 7f5e399d3700 1 --2- 172.21.15.140:0/1730007109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e340b11b0 0x7f5e3405d180 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.871+0000 7f5e399d3700 1 --2- 172.21.15.140:0/1730007109 >> v2:172.21.15.152:3300/0 conn(0x7f5e34060bb0 0x7f5e34147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:51.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.871+0000 7f5e399d3700 1 -- 172.21.15.140:0/1730007109 >> 172.21.15.140:0/1730007109 conn(0x7f5e340b5bb0 msgr2=0x7f5e340b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:51.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.871+0000 7f5e399d3700 1 -- 172.21.15.140:0/1730007109 shutdown_connections 2022-07-21T01:51:51.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.871+0000 7f5e399d3700 1 -- 172.21.15.140:0/1730007109 wait complete. 2022-07-21T01:51:51.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e399d3700 1 Processor -- start 2022-07-21T01:51:51.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e399d3700 1 -- start start 2022-07-21T01:51:51.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e399d3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e3405cda0 0x7f5e340b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e399d3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e34060bb0 0x7f5e340adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e399d3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e340b11b0 0x7f5e340abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e399d3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340aad50 con 0x7f5e34060bb0 2022-07-21T01:51:51.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e399d3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340ab5a0 con 0x7f5e3405cda0 2022-07-21T01:51:51.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e399d3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340ab840 con 0x7f5e340b11b0 2022-07-21T01:51:51.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e327fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e34060bb0 0x7f5e340adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e327fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e34060bb0 0x7f5e340adc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35016/0 (socket says 172.21.15.140:35016) 2022-07-21T01:51:51.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e327fc700 1 -- 172.21.15.140:0/944011792 learned_addr learned my addr 172.21.15.140:0/944011792 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:51.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e327fc700 1 -- 172.21.15.140:0/944011792 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e340b11b0 msgr2=0x7f5e340abf90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-07-21T01:51:51.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e327fc700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e340b11b0 0x7f5e340abf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:51:51.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e327fc700 1 -- 172.21.15.140:0/944011792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e3405cda0 msgr2=0x7f5e340b0040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-07-21T01:51:51.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e327fc700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e3405cda0 0x7f5e340b0040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:51:51.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.875+0000 7f5e327fc700 1 -- 172.21.15.140:0/944011792 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c009470 con 0x7f5e34060bb0 2022-07-21T01:51:51.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e327fc700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e34060bb0 0x7f5e340adc20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f5e24002f70 tx=0x7f5e24014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:51.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e2bfff700 1 -- 172.21.15.140:0/944011792 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e2400dea0 con 0x7f5e34060bb0 2022-07-21T01:51:51.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e2bfff700 1 -- 172.21.15.140:0/944011792 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2401e400 con 0x7f5e34060bb0 2022-07-21T01:51:51.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e2bfff700 1 -- 172.21.15.140:0/944011792 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e2401e760 con 0x7f5e34060bb0 2022-07-21T01:51:51.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e399d3700 1 -- 172.21.15.140:0/944011792 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2400b180 con 0x7f5e34060bb0 2022-07-21T01:51:51.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e399d3700 1 -- 172.21.15.140:0/944011792 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e34146a80 con 0x7f5e34060bb0 2022-07-21T01:51:51.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e399d3700 1 -- 172.21.15.140:0/944011792 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e340aba00 con 0x7f5e34060bb0 2022-07-21T01:51:51.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e2bfff700 1 -- 172.21.15.140:0/944011792 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e24025030 con 0x7f5e34060bb0 2022-07-21T01:51:51.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e2bfff700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e18053b30 0x7f5e18055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:51.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e2bfff700 1 -- 172.21.15.140:0/944011792 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f5e2406cac0 con 0x7f5e34060bb0 2022-07-21T01:51:51.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e32ffd700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e18053b30 0x7f5e18055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:51.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.879+0000 7f5e32ffd700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e18053b30 0x7f5e18055ff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f5e34147640 tx=0x7f5e1c007790).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:51.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:51.887+0000 7f5e2bfff700 1 -- 172.21.15.140:0/944011792 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e24043cc0 con 0x7f5e34060bb0 2022-07-21T01:51:52.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.019+0000 7f5e399d3700 1 -- 172.21.15.140:0/944011792 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f5e340574e0 con 0x7f5e34060bb0 2022-07-21T01:51:52.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.895+0000 7f5e2bfff700 1 -- 172.21.15.140:0/944011792 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5e24023020 con 0x7f5e34060bb0 2022-07-21T01:51:52.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 -- 172.21.15.140:0/944011792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e18053b30 msgr2=0x7f5e18055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:52.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e18053b30 0x7f5e18055ff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f5e34147640 tx=0x7f5e1c007790).stop 2022-07-21T01:51:52.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 -- 172.21.15.140:0/944011792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e34060bb0 msgr2=0x7f5e340adc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:52.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e34060bb0 0x7f5e340adc20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f5e24002f70 tx=0x7f5e24014000).stop 2022-07-21T01:51:52.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 -- 172.21.15.140:0/944011792 shutdown_connections 2022-07-21T01:51:52.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e18053b30 0x7f5e18055ff0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:52.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e340b11b0 0x7f5e340abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:51:52.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e34060bb0 0x7f5e340adc20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:52.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 --2- 172.21.15.140:0/944011792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e3405cda0 0x7f5e340b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:51:52.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 -- 172.21.15.140:0/944011792 >> 172.21.15.140:0/944011792 conn(0x7f5e340b5bb0 msgr2=0x7f5e3414ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:52.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 -- 172.21.15.140:0/944011792 shutdown_connections 2022-07-21T01:51:52.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:52.899+0000 7f5e399d3700 1 -- 172.21.15.140:0/944011792 wait complete. 2022-07-21T01:51:52.904 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:51:52.913 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:51:52.914 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:51:53.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.023+0000 7fb392086700 1 Processor -- start 2022-07-21T01:51:53.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.023+0000 7fb392086700 1 -- start start 2022-07-21T01:51:53.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.023+0000 7fb392086700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb38c0b31e0 0x7fb38c13df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.023+0000 7fb392086700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb38c066160 0x7fb38c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.023+0000 7fb392086700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c0b7f50 con 0x7fb38c0b7b70 2022-07-21T01:51:53.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.023+0000 7fb392086700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38c0b2800 con 0x7fb38c0b31e0 2022-07-21T01:51:53.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.023+0000 7fb392086700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0b2aa0 con 0x7fb38c066160 2022-07-21T01:51:53.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.023+0000 7fb390883700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb38c0b31e0 0x7fb38c13df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.023+0000 7fb390883700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb38c0b31e0 0x7fb38c13df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35024/0 (socket says 172.21.15.140:35024) 2022-07-21T01:51:53.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.023+0000 7fb390883700 1 -- 172.21.15.140:0/3416808055 learned_addr learned my addr 172.21.15.140:0/3416808055 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:53.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.023+0000 7fb391885700 1 --2- 172.21.15.140:0/3416808055 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb38c066160 0x7fb38c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb383fff700 1 -- 172.21.15.140:0/3416808055 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535420262 0 0) 0x7fb38c0b7f50 con 0x7fb38c0b7b70 2022-07-21T01:51:53.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb390883700 1 -- 172.21.15.140:0/3416808055 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb38c066160 msgr2=0x7fb38c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:53.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb390883700 1 --2- 172.21.15.140:0/3416808055 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb38c066160 0x7fb38c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb390883700 1 -- 172.21.15.140:0/3416808055 >> v1:172.21.15.152:6789/0 conn(0x7fb38c0b7b70 legacy=0x7fb38c13d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb390883700 1 -- 172.21.15.140:0/3416808055 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c0b2d40 con 0x7fb38c0b31e0 2022-07-21T01:51:53.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb390883700 1 --2- 172.21.15.140:0/3416808055 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb38c0b31e0 0x7fb38c13df20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7fb37c002f70 tx=0x7fb37c006eb0).ready entity=mon.0 client_cookie=568bfd15a6fac9bc server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb383fff700 1 -- 172.21.15.140:0/3416808055 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb37c00d570 con 0x7fb38c0b31e0 2022-07-21T01:51:53.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb383fff700 1 -- 172.21.15.140:0/3416808055 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c000e90 con 0x7fb38c0b31e0 2022-07-21T01:51:53.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb383fff700 1 -- 172.21.15.140:0/3416808055 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb37c00d8c0 con 0x7fb38c0b31e0 2022-07-21T01:51:53.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb392086700 1 -- 172.21.15.140:0/3416808055 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb38c0b31e0 msgr2=0x7fb38c13df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb392086700 1 --2- 172.21.15.140:0/3416808055 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb38c0b31e0 0x7fb38c13df20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7fb37c002f70 tx=0x7fb37c006eb0).stop 2022-07-21T01:51:53.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb392086700 1 -- 172.21.15.140:0/3416808055 shutdown_connections 2022-07-21T01:51:53.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb392086700 1 --2- 172.21.15.140:0/3416808055 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb38c066160 0x7fb38c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb392086700 1 --2- 172.21.15.140:0/3416808055 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb38c0b31e0 0x7fb38c13df20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb392086700 1 -- 172.21.15.140:0/3416808055 >> 172.21.15.140:0/3416808055 conn(0x7fb38c0656b0 msgr2=0x7fb38c0b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:53.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb392086700 1 -- 172.21.15.140:0/3416808055 shutdown_connections 2022-07-21T01:51:53.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb392086700 1 -- 172.21.15.140:0/3416808055 wait complete. 2022-07-21T01:51:53.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb392086700 1 Processor -- start 2022-07-21T01:51:53.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.027+0000 7fb392086700 1 -- start start 2022-07-21T01:51:53.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb392086700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb38c066160 0x7fb38c152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb392086700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb38c0b31e0 0x7fb38c0afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb392086700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb38c0b7b70 0x7fb38c05f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb392086700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0af100 con 0x7fb38c0b31e0 2022-07-21T01:51:53.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb392086700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0ae000 con 0x7fb38c0b7b70 2022-07-21T01:51:53.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb392086700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0adc00 con 0x7fb38c066160 2022-07-21T01:51:53.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb390883700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb38c0b31e0 0x7fb38c0afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb390883700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb38c0b31e0 0x7fb38c0afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35030/0 (socket says 172.21.15.140:35030) 2022-07-21T01:51:53.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb391084700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb38c066160 0x7fb38c152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb390883700 1 -- 172.21.15.140:0/2549438168 learned_addr learned my addr 172.21.15.140:0/2549438168 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:53.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb391885700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb38c0b7b70 0x7fb38c05f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb390883700 1 -- 172.21.15.140:0/2549438168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb38c066160 msgr2=0x7fb38c152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb390883700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb38c066160 0x7fb38c152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb390883700 1 -- 172.21.15.140:0/2549438168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb38c0b7b70 msgr2=0x7fb38c05f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb390883700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb38c0b7b70 0x7fb38c05f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb390883700 1 -- 172.21.15.140:0/2549438168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c00b1e0 con 0x7fb38c0b31e0 2022-07-21T01:51:53.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb390883700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb38c0b31e0 0x7fb38c0afa70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fb37c002f70 tx=0x7fb37c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb3827fc700 1 -- 172.21.15.140:0/2549438168 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb37c0116e0 con 0x7fb38c0b31e0 2022-07-21T01:51:53.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb3827fc700 1 -- 172.21.15.140:0/2549438168 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c011840 con 0x7fb38c0b31e0 2022-07-21T01:51:53.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb392086700 1 -- 172.21.15.140:0/2549438168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c0aff60 con 0x7fb38c0b31e0 2022-07-21T01:51:53.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb3827fc700 1 -- 172.21.15.140:0/2549438168 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb37c011ba0 con 0x7fb38c0b31e0 2022-07-21T01:51:53.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.031+0000 7fb392086700 1 -- 172.21.15.140:0/2549438168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c05fd30 con 0x7fb38c0b31e0 2022-07-21T01:51:53.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.035+0000 7fb3827fc700 1 -- 172.21.15.140:0/2549438168 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb37c025030 con 0x7fb38c0b31e0 2022-07-21T01:51:53.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.035+0000 7fb392086700 1 -- 172.21.15.140:0/2549438168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c0addc0 con 0x7fb38c0b31e0 2022-07-21T01:51:53.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.035+0000 7fb3827fc700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb374053aa0 0x7fb374055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.035+0000 7fb3827fc700 1 -- 172.21.15.140:0/2549438168 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb37c06ccc0 con 0x7fb38c0b31e0 2022-07-21T01:51:53.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.035+0000 7fb391084700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb374053aa0 0x7fb374055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.035+0000 7fb391084700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb374053aa0 0x7fb374055f60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fb38800d000 tx=0x7fb388007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.043+0000 7fb3827fc700 1 -- 172.21.15.140:0/2549438168 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb37c043f10 con 0x7fb38c0b31e0 2022-07-21T01:51:53.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.203+0000 7fb392086700 1 -- 172.21.15.140:0/2549438168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb38c0574e0 con 0x7fb38c0b31e0 2022-07-21T01:51:53.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.203+0000 7fb3827fc700 1 -- 172.21.15.140:0/2549438168 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fb37c023020 con 0x7fb38c0b31e0 2022-07-21T01:51:53.204 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:53.204 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:51:53.204 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:51:53.205 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.205 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:51:53.205 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:51:53.205 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.206 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:51:53.206 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:51:53.206 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:51:53.206 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.206 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.207 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.207 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.207 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:51:53.207 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:51:53.208 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.208 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:51:53.208 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:51:53.208 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:51:53.208 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.209 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.209 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.209 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.209 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:51:53.210 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:51:53.210 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.210 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:51:53.210 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:51:53.211 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.211 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.211 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.211 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.212 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:51:53.212 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:51:53.212 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.212 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:51:53.212 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:51:53.213 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.213 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.213 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.213 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.214 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:51:53.214 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:51:53.214 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.214 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:51:53.214 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:51:53.223 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.223 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.224 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.224 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.224 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:51:53.225 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:51:53.225 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.226 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:51:53.226 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:51:53.227 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.227 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.228 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.228 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.229 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:51:53.229 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:51:53.229 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.230 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:51:53.230 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:51:53.231 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.231 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.232 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.232 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.233 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:51:53.233 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:51:53.234 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.234 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:51:53.234 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:51:53.235 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.235 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.236 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.236 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.236 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:51:53.237 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:51:53.237 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.237 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:51:53.238 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:51:53.238 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.239 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.239 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.240 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.240 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:51:53.241 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:51:53.241 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.242 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:51:53.242 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:51:53.243 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.243 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.244 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.244 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.245 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:51:53.245 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:51:53.246 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.246 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:51:53.247 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:51:53.247 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:51:53.247 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.248 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.248 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.249 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:51:53.249 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:51:53.250 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.250 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:51:53.251 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:51:53.251 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:51:53.252 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.252 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.253 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.254 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:51:53.254 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:51:53.255 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.255 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:51:53.255 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:51:53.256 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:51:53.256 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.257 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.257 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.258 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:51:53.259 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:51:53.259 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.260 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:51:53.261 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:51:53.261 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:51:53.262 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.263 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.263 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.264 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.264 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:51:53.265 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:51:53.266 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.267 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:51:53.267 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:51:53.268 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.268 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.269 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:53.270 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:53.270 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:51:53.271 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:51:53.272 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:51:53.273 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:51:53.273 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:51:53.273 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:51:53.274 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.274 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:53.275 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:51:53.275 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:51:53.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 -- 172.21.15.140:0/2549438168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb374053aa0 msgr2=0x7fb374055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb374053aa0 0x7fb374055f60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fb38800d000 tx=0x7fb388007b90).stop 2022-07-21T01:51:53.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 -- 172.21.15.140:0/2549438168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb38c0b31e0 msgr2=0x7fb38c0afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb38c0b31e0 0x7fb38c0afa70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fb37c002f70 tx=0x7fb37c015000).stop 2022-07-21T01:51:53.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 -- 172.21.15.140:0/2549438168 shutdown_connections 2022-07-21T01:51:53.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb374053aa0 0x7fb374055f60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb38c0b7b70 0x7fb38c05f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb38c0b31e0 0x7fb38c0afa70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 --2- 172.21.15.140:0/2549438168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb38c066160 0x7fb38c152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 -- 172.21.15.140:0/2549438168 >> 172.21.15.140:0/2549438168 conn(0x7fb38c0656b0 msgr2=0x7fb38c1426a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:53.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 -- 172.21.15.140:0/2549438168 shutdown_connections 2022-07-21T01:51:53.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.207+0000 7fb392086700 1 -- 172.21.15.140:0/2549438168 wait complete. 2022-07-21T01:51:53.283 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:51:53.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74deb72700 1 Processor -- start 2022-07-21T01:51:53.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74deb72700 1 -- start start 2022-07-21T01:51:53.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74deb72700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f74d80b1ef0 0x7f74d8142090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74deb72700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d8142580 0x7f74d805c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74deb72700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d80b22d0 con 0x7f74d805cb10 2022-07-21T01:51:53.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74deb72700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d81472a0 con 0x7f74d80b1ef0 2022-07-21T01:51:53.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74deb72700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d80ac0c0 con 0x7f74d8142580 2022-07-21T01:51:53.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74dd36f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d8142580 0x7f74d805c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74ddb70700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f74d80b1ef0 0x7f74d8142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74dd36f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d8142580 0x7f74d805c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:60000/0 (socket says 172.21.15.140:60000) 2022-07-21T01:51:53.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74dd36f700 1 -- 172.21.15.140:0/2860148953 learned_addr learned my addr 172.21.15.140:0/2860148953 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:53.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74de371700 1 --1- 172.21.15.140:0/2860148953 >> v1:172.21.15.152:6789/0 conn(0x7f74d805cb10 0x7f74d8061130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:50300/0 (socket says 172.21.15.140:50300) 2022-07-21T01:51:53.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74dd36f700 1 -- 172.21.15.140:0/2860148953 >> v2:172.21.15.152:3300/0 conn(0x7f74d80b1ef0 msgr2=0x7f74d8142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:53.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74dd36f700 1 --2- 172.21.15.140:0/2860148953 >> v2:172.21.15.152:3300/0 conn(0x7f74d80b1ef0 0x7f74d8142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74dd36f700 1 -- 172.21.15.140:0/2860148953 >> v1:172.21.15.152:6789/0 conn(0x7f74d805cb10 legacy=0x7f74d8061130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74dd36f700 1 -- 172.21.15.140:0/2860148953 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d80ac360 con 0x7f74d8142580 2022-07-21T01:51:53.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74ddb70700 1 --2- 172.21.15.140:0/2860148953 >> v2:172.21.15.152:3300/0 conn(0x7f74d80b1ef0 0x7f74d8142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:51:53.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.431+0000 7f74dd36f700 1 --2- 172.21.15.140:0/2860148953 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d8142580 0x7f74d805c620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f74d4002f70 tx=0x7f74d4006eb0).ready entity=mon.2 client_cookie=1cfb3bcc70394dfe server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74dcb6e700 1 -- 172.21.15.140:0/2860148953 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74d400d570 con 0x7f74d8142580 2022-07-21T01:51:53.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74dcb6e700 1 -- 172.21.15.140:0/2860148953 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d4000e90 con 0x7f74d8142580 2022-07-21T01:51:53.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74dcb6e700 1 -- 172.21.15.140:0/2860148953 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74d400d900 con 0x7f74d8142580 2022-07-21T01:51:53.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 -- 172.21.15.140:0/2860148953 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d8142580 msgr2=0x7f74d805c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 --2- 172.21.15.140:0/2860148953 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d8142580 0x7f74d805c620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f74d4002f70 tx=0x7f74d4006eb0).stop 2022-07-21T01:51:53.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 -- 172.21.15.140:0/2860148953 shutdown_connections 2022-07-21T01:51:53.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 --2- 172.21.15.140:0/2860148953 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d8142580 0x7f74d805c620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 --2- 172.21.15.140:0/2860148953 >> v2:172.21.15.152:3300/0 conn(0x7f74d80b1ef0 0x7f74d8142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 -- 172.21.15.140:0/2860148953 >> 172.21.15.140:0/2860148953 conn(0x7f74d80b2ca0 msgr2=0x7f74d80b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:53.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 -- 172.21.15.140:0/2860148953 shutdown_connections 2022-07-21T01:51:53.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 -- 172.21.15.140:0/2860148953 wait complete. 2022-07-21T01:51:53.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 Processor -- start 2022-07-21T01:51:53.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 -- start start 2022-07-21T01:51:53.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d805cb10 0x7f74d8142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d80b1ef0 0x7f74d809a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74d8142580 0x7f74d81558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d809af40 con 0x7f74d8142580 2022-07-21T01:51:53.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d8140290 con 0x7f74d805cb10 2022-07-21T01:51:53.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74deb72700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d809a840 con 0x7f74d80b1ef0 2022-07-21T01:51:53.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74de371700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74d8142580 0x7f74d81558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74dd36f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d80b1ef0 0x7f74d809a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74ddb70700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d805cb10 0x7f74d8142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74ddb70700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d805cb10 0x7f74d8142ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:40784/0 (socket says 172.21.15.140:40784) 2022-07-21T01:51:53.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74dd36f700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d80b1ef0 0x7f74d809a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:60004/0 (socket says 172.21.15.140:60004) 2022-07-21T01:51:53.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.435+0000 7f74ddb70700 1 -- 172.21.15.140:0/3438762832 learned_addr learned my addr 172.21.15.140:0/3438762832 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:53.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74ddb70700 1 -- 172.21.15.140:0/3438762832 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d80b1ef0 msgr2=0x7f74d809a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74ddb70700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d80b1ef0 0x7f74d809a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74ddb70700 1 -- 172.21.15.140:0/3438762832 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74d8142580 msgr2=0x7f74d81558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74ddb70700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74d8142580 0x7f74d81558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74ddb70700 1 -- 172.21.15.140:0/3438762832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c800b120 con 0x7f74d805cb10 2022-07-21T01:51:53.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74ddb70700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d805cb10 0x7f74d8142ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f74c800c000 tx=0x7f74c8008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74c6ffd700 1 -- 172.21.15.140:0/3438762832 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c8012600 con 0x7f74d805cb10 2022-07-21T01:51:53.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74c6ffd700 1 -- 172.21.15.140:0/3438762832 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c8012760 con 0x7f74d805cb10 2022-07-21T01:51:53.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74c6ffd700 1 -- 172.21.15.140:0/3438762832 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c8012ac0 con 0x7f74d805cb10 2022-07-21T01:51:53.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74deb72700 1 -- 172.21.15.140:0/3438762832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d400b1e0 con 0x7f74d805cb10 2022-07-21T01:51:53.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74deb72700 1 -- 172.21.15.140:0/3438762832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d8099eb0 con 0x7f74d805cb10 2022-07-21T01:51:53.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74deb72700 1 -- 172.21.15.140:0/3438762832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d80993d0 con 0x7f74d805cb10 2022-07-21T01:51:53.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74c6ffd700 1 -- 172.21.15.140:0/3438762832 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f74c8012c20 con 0x7f74d805cb10 2022-07-21T01:51:53.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74c6ffd700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f74b8053950 0x7f74b8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74c6ffd700 1 -- 172.21.15.140:0/3438762832 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f74c806c300 con 0x7f74d805cb10 2022-07-21T01:51:53.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.439+0000 7f74dd36f700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f74b8053950 0x7f74b8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.443+0000 7f74dd36f700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f74b8053950 0x7f74b8055e10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f74d401c000 tx=0x7f74d4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.447+0000 7f74c6ffd700 1 -- 172.21.15.140:0/3438762832 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74c80434b0 con 0x7f74d805cb10 2022-07-21T01:51:53.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.571+0000 7f74deb72700 1 -- 172.21.15.140:0/3438762832 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74d80ac600 con 0x7f74d805cb10 2022-07-21T01:51:53.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.571+0000 7f74c6ffd700 1 -- 172.21.15.140:0/3438762832 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+13277 (secure 0 0 0) 0x7f74c8046ad0 con 0x7f74d805cb10 2022-07-21T01:51:53.572 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:53.573 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":48,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:51:52.889248+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:51:53.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 -- 172.21.15.140:0/3438762832 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f74b8053950 msgr2=0x7f74b8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f74b8053950 0x7f74b8055e10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f74d401c000 tx=0x7f74d4015000).stop 2022-07-21T01:51:53.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 -- 172.21.15.140:0/3438762832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d805cb10 msgr2=0x7f74d8142ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d805cb10 0x7f74d8142ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f74c800c000 tx=0x7f74c8008040).stop 2022-07-21T01:51:53.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 -- 172.21.15.140:0/3438762832 shutdown_connections 2022-07-21T01:51:53.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f74b8053950 0x7f74b8055e10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74d8142580 0x7f74d81558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74d80b1ef0 0x7f74d809a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 --2- 172.21.15.140:0/3438762832 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74d805cb10 0x7f74d8142ce0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 -- 172.21.15.140:0/3438762832 >> 172.21.15.140:0/3438762832 conn(0x7f74d80b2ca0 msgr2=0x7f74d8061050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:53.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 -- 172.21.15.140:0/3438762832 shutdown_connections 2022-07-21T01:51:53.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.575+0000 7f74deb72700 1 -- 172.21.15.140:0/3438762832 wait complete. 2022-07-21T01:51:53.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:51:53.587 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:51:53.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef5759e700 1 Processor -- start 2022-07-21T01:51:53.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef5759e700 1 -- start start 2022-07-21T01:51:53.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef5759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef580aef30 0x7fef581445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef5759e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef58144a90 0x7fef58129340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef5759e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef580b5dd0 con 0x7fef580b59f0 2022-07-21T01:51:53.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef5759e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef580b1750 con 0x7fef58144a90 2022-07-21T01:51:53.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef5759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef58140ef0 con 0x7fef580aef30 2022-07-21T01:51:53.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef5659c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fef580b59f0 0x7fef581407e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49196/0 (socket says 172.21.15.140:49196) 2022-07-21T01:51:53.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef5659c700 1 -- 172.21.15.140:0/1499339408 learned_addr learned my addr 172.21.15.140:0/1499339408 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:53.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef56d9d700 1 --2- 172.21.15.140:0/1499339408 >> v2:172.21.15.152:3300/0 conn(0x7fef58144a90 0x7fef58129340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef4dd9b700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef580aef30 0x7fef581445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701058804 0 0) 0x7fef580b5dd0 con 0x7fef580b59f0 2022-07-21T01:51:53.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef38003260 con 0x7fef580b59f0 2022-07-21T01:51:53.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2594007879 0 0) 0x7fef38003260 con 0x7fef580b59f0 2022-07-21T01:51:53.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef580aef30 msgr2=0x7fef581445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:53.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef580aef30 0x7fef581445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 >> v2:172.21.15.152:3300/0 conn(0x7fef58144a90 msgr2=0x7fef58129340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:53.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 --2- 172.21.15.140:0/1499339408 >> v2:172.21.15.152:3300/0 conn(0x7fef58144a90 0x7fef58129340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580652a0 con 0x7fef580b59f0 2022-07-21T01:51:53.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fef48001660 con 0x7fef580b59f0 2022-07-21T01:51:53.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 >> v1:172.21.15.140:6789/0 conn(0x7fef580b59f0 legacy=0x7fef581407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 0x7fef38003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef580aef30 0x7fef38004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef380049f0 0x7fef38004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef580b1750 con 0x7fef38003750 2022-07-21T01:51:53.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef58140ef0 con 0x7fef380049f0 2022-07-21T01:51:53.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3800b750 con 0x7fef580aef30 2022-07-21T01:51:53.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.699+0000 7fef5659c700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 0x7fef38003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef4dd9b700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef580aef30 0x7fef38004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef56d9d700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef380049f0 0x7fef38004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5659c700 1 -- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef580aef30 msgr2=0x7fef38004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:53.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5659c700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef580aef30 0x7fef38004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5659c700 1 -- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef380049f0 msgr2=0x7fef38004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:53.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5659c700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef380049f0 0x7fef38004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5659c700 1 -- 172.21.15.140:0/1499339408 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef38003260 con 0x7fef38003750 2022-07-21T01:51:53.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef56d9d700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef380049f0 0x7fef38004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:51:53.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5659c700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 0x7fef38003b70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fef48001370 tx=0x7fef58065430).ready entity=mon.0 client_cookie=fe2217d4e8138b9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef480071a0 con 0x7fef38003750 2022-07-21T01:51:53.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef48007300 con 0x7fef38003750 2022-07-21T01:51:53.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef55d9b700 1 -- 172.21.15.140:0/1499339408 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef4800cbb0 con 0x7fef38003750 2022-07-21T01:51:53.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5759e700 1 -- 172.21.15.140:0/1499339408 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 msgr2=0x7fef38003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5759e700 1 --2- 172.21.15.140:0/1499339408 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 0x7fef38003b70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fef48001370 tx=0x7fef58065430).stop 2022-07-21T01:51:53.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef56d9d700 1 -- 172.21.15.140:0/1499339408 reap_dead start 2022-07-21T01:51:53.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5759e700 1 -- 172.21.15.140:0/1499339408 shutdown_connections 2022-07-21T01:51:53.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5759e700 1 -- 172.21.15.140:0/1499339408 >> 172.21.15.140:0/1499339408 conn(0x7fef58063480 msgr2=0x7fef580b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:53.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5759e700 1 -- 172.21.15.140:0/1499339408 shutdown_connections 2022-07-21T01:51:53.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.703+0000 7fef5759e700 1 -- 172.21.15.140:0/1499339408 wait complete. 2022-07-21T01:51:53.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5759e700 1 Processor -- start 2022-07-21T01:51:53.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5759e700 1 -- start start 2022-07-21T01:51:53.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 0x7fef5812fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef5812f1c0 0x7fef5812dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef5812e3d0 0x7fef5814fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5812e880 con 0x7fef38003750 2022-07-21T01:51:53.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581510e0 con 0x7fef5812e3d0 2022-07-21T01:51:53.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef58131950 con 0x7fef5812f1c0 2022-07-21T01:51:53.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 0x7fef5812fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 0x7fef5812fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35062/0 (socket says 172.21.15.140:35062) 2022-07-21T01:51:53.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5659c700 1 -- 172.21.15.140:0/2469373926 learned_addr learned my addr 172.21.15.140:0/2469373926 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:53.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef56d9d700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef5812e3d0 0x7fef5814fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef4dd9b700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef5812f1c0 0x7fef5812dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5659c700 1 -- 172.21.15.140:0/2469373926 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef5812f1c0 msgr2=0x7fef5812dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5659c700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef5812f1c0 0x7fef5812dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5659c700 1 -- 172.21.15.140:0/2469373926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef5812e3d0 msgr2=0x7fef5814fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5659c700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef5812e3d0 0x7fef5814fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5659c700 1 -- 172.21.15.140:0/2469373926 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58131bf0 con 0x7fef38003750 2022-07-21T01:51:53.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef4dd9b700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef5812f1c0 0x7fef5812dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:51:53.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5659c700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 0x7fef5812fc80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fef48001370 tx=0x7fef48013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef4ffff700 1 -- 172.21.15.140:0/2469373926 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef48007470 con 0x7fef38003750 2022-07-21T01:51:53.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef4ffff700 1 -- 172.21.15.140:0/2469373926 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4800c6d0 con 0x7fef38003750 2022-07-21T01:51:53.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5759e700 1 -- 172.21.15.140:0/2469373926 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef58131e90 con 0x7fef38003750 2022-07-21T01:51:53.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef4ffff700 1 -- 172.21.15.140:0/2469373926 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef48004920 con 0x7fef38003750 2022-07-21T01:51:53.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.707+0000 7fef5759e700 1 -- 172.21.15.140:0/2469373926 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5812d390 con 0x7fef38003750 2022-07-21T01:51:53.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.711+0000 7fef4ffff700 1 -- 172.21.15.140:0/2469373926 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fef48026070 con 0x7fef38003750 2022-07-21T01:51:53.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.711+0000 7fef5759e700 1 -- 172.21.15.140:0/2469373926 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef58143e60 con 0x7fef38003750 2022-07-21T01:51:53.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.711+0000 7fef4ffff700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef3c053aa0 0x7fef3c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.711+0000 7fef4dd9b700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef3c053aa0 0x7fef3c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.711+0000 7fef4ffff700 1 -- 172.21.15.140:0/2469373926 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fef4806c1b0 con 0x7fef38003750 2022-07-21T01:51:53.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.711+0000 7fef4dd9b700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef3c053aa0 0x7fef3c055f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fef4000b000 tx=0x7fef40008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.719+0000 7fef4ffff700 1 -- 172.21.15.140:0/2469373926 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fef48043400 con 0x7fef38003750 2022-07-21T01:51:53.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.843+0000 7fef5759e700 1 -- 172.21.15.140:0/2469373926 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fef580574e0 con 0x7fef38003750 2022-07-21T01:51:53.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef4ffff700 1 -- 172.21.15.140:0/2469373926 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v48) v1 ==== 110+0+10 (secure 0 0 0) 0x7fef48046a20 con 0x7fef38003750 2022-07-21T01:51:53.845 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:51:53.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 -- 172.21.15.140:0/2469373926 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef3c053aa0 msgr2=0x7fef3c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef3c053aa0 0x7fef3c055f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fef4000b000 tx=0x7fef40008040).stop 2022-07-21T01:51:53.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 -- 172.21.15.140:0/2469373926 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 msgr2=0x7fef5812fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 0x7fef5812fc80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fef48001370 tx=0x7fef48013000).stop 2022-07-21T01:51:53.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 -- 172.21.15.140:0/2469373926 shutdown_connections 2022-07-21T01:51:53.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef3c053aa0 0x7fef3c055f60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef5812e3d0 0x7fef5814fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef5812f1c0 0x7fef5812dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 --2- 172.21.15.140:0/2469373926 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef38003750 0x7fef5812fc80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 -- 172.21.15.140:0/2469373926 >> 172.21.15.140:0/2469373926 conn(0x7fef58063480 msgr2=0x7fef580b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:53.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 -- 172.21.15.140:0/2469373926 shutdown_connections 2022-07-21T01:51:53.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.847+0000 7fef5759e700 1 -- 172.21.15.140:0/2469373926 wait complete. 2022-07-21T01:51:53.863 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:51:53.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.971+0000 7f3b518a1700 1 Processor -- start 2022-07-21T01:51:53.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.971+0000 7f3b518a1700 1 -- start start 2022-07-21T01:51:53.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b518a1700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3b4c0af180 0x7f3b4c05d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b518a1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b4c0b5db0 con 0x7f3b4c0b59d0 2022-07-21T01:51:53.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b518a1700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b4c0ad800 con 0x7f3b4c128d70 2022-07-21T01:51:53.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b518a1700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4c0adaa0 con 0x7f3b4c0af180 2022-07-21T01:51:53.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b5089f700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f3b4c0b59d0 0x7f3b4c128660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49216/0 (socket says 172.21.15.140:49216) 2022-07-21T01:51:53.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b5089f700 1 -- 172.21.15.140:0/2541615721 learned_addr learned my addr 172.21.15.140:0/2541615721 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:53.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b43fff700 1 --2- 172.21.15.140:0/2541615721 >> v2:172.21.15.152:3300/0 conn(0x7f3b4c0af180 0x7f3b4c05d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b437fe700 1 -- 172.21.15.140:0/2541615721 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497505983 0 0) 0x7f3b4c0b5db0 con 0x7f3b4c0b59d0 2022-07-21T01:51:53.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b437fe700 1 -- 172.21.15.140:0/2541615721 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b38003260 con 0x7f3b4c0b59d0 2022-07-21T01:51:53.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b437fe700 1 -- 172.21.15.140:0/2541615721 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269754663 0 0) 0x7f3b4c0ad800 con 0x7f3b4c128d70 2022-07-21T01:51:53.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b437fe700 1 -- 172.21.15.140:0/2541615721 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b4c0b5db0 con 0x7f3b4c128d70 2022-07-21T01:51:53.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b437fe700 1 -- 172.21.15.140:0/2541615721 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1276548199 0 0) 0x7f3b38003260 con 0x7f3b4c0b59d0 2022-07-21T01:51:53.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b43fff700 1 -- 172.21.15.140:0/2541615721 >> v1:172.21.15.152:6789/0 conn(0x7f3b4c128d70 legacy=0x7f3b4c13fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b43fff700 1 -- 172.21.15.140:0/2541615721 >> v1:172.21.15.140:6789/0 conn(0x7f3b4c0b59d0 legacy=0x7f3b4c128660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b43fff700 1 -- 172.21.15.140:0/2541615721 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c0ad4f0 con 0x7f3b4c0af180 2022-07-21T01:51:53.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b43fff700 1 --2- 172.21.15.140:0/2541615721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b4c0af180 0x7f3b4c05d1c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f3b48001110 tx=0x7f3b3c006df0).ready entity=mon.1 client_cookie=cc41cb0b35eeb42f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b437fe700 1 -- 172.21.15.140:0/2541615721 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b48001660 con 0x7f3b4c0af180 2022-07-21T01:51:53.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b437fe700 1 -- 172.21.15.140:0/2541615721 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3c000e90 con 0x7f3b4c0af180 2022-07-21T01:51:53.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b437fe700 1 -- 172.21.15.140:0/2541615721 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b3c012460 con 0x7f3b4c0af180 2022-07-21T01:51:53.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b518a1700 1 -- 172.21.15.140:0/2541615721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b4c0af180 msgr2=0x7f3b4c05d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b518a1700 1 --2- 172.21.15.140:0/2541615721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b4c0af180 0x7f3b4c05d1c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f3b48001110 tx=0x7f3b3c006df0).stop 2022-07-21T01:51:53.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b518a1700 1 -- 172.21.15.140:0/2541615721 shutdown_connections 2022-07-21T01:51:53.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b518a1700 1 --2- 172.21.15.140:0/2541615721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b4c0af180 0x7f3b4c05d1c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b518a1700 1 -- 172.21.15.140:0/2541615721 >> 172.21.15.140:0/2541615721 conn(0x7f3b4c0634c0 msgr2=0x7f3b4c0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:53.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b518a1700 1 -- 172.21.15.140:0/2541615721 shutdown_connections 2022-07-21T01:51:53.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.975+0000 7f3b518a1700 1 -- 172.21.15.140:0/2541615721 wait complete. 2022-07-21T01:51:53.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b518a1700 1 Processor -- start 2022-07-21T01:51:53.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b518a1700 1 -- start start 2022-07-21T01:51:53.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b518a1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b4c0af180 0x7f3b4c138e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b518a1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b4c0b59d0 0x7f3b4c14f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b518a1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b4c128d70 0x7f3b4c14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b518a1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c064cb0 con 0x7f3b4c0af180 2022-07-21T01:51:53.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b518a1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c065960 con 0x7f3b4c128d70 2022-07-21T01:51:53.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b518a1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c065c00 con 0x7f3b4c0b59d0 2022-07-21T01:51:53.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b5089f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b4c0af180 0x7f3b4c138e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b5089f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b4c0af180 0x7f3b4c138e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35076/0 (socket says 172.21.15.140:35076) 2022-07-21T01:51:53.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b5089f700 1 -- 172.21.15.140:0/919309509 learned_addr learned my addr 172.21.15.140:0/919309509 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:53.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b43fff700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b4c0b59d0 0x7f3b4c14f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b510a0700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b4c128d70 0x7f3b4c14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b5089f700 1 -- 172.21.15.140:0/919309509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b4c0b59d0 msgr2=0x7f3b4c14f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b5089f700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b4c0b59d0 0x7f3b4c14f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b5089f700 1 -- 172.21.15.140:0/919309509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b4c128d70 msgr2=0x7f3b4c14f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:53.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b5089f700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b4c128d70 0x7f3b4c14f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:53.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b5089f700 1 -- 172.21.15.140:0/919309509 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c00b180 con 0x7f3b4c0af180 2022-07-21T01:51:53.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b5089f700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b4c0af180 0x7f3b4c138e60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f3b48001840 tx=0x7f3b48001720).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b41ffb700 1 -- 172.21.15.140:0/919309509 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b48007170 con 0x7f3b4c0af180 2022-07-21T01:51:53.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b41ffb700 1 -- 172.21.15.140:0/919309509 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b480072d0 con 0x7f3b4c0af180 2022-07-21T01:51:53.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.979+0000 7f3b518a1700 1 -- 172.21.15.140:0/919309509 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4c064930 con 0x7f3b4c0af180 2022-07-21T01:51:53.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.983+0000 7f3b41ffb700 1 -- 172.21.15.140:0/919309509 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b48004710 con 0x7f3b4c0af180 2022-07-21T01:51:53.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.983+0000 7f3b518a1700 1 -- 172.21.15.140:0/919309509 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4c067800 con 0x7f3b4c0af180 2022-07-21T01:51:53.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.983+0000 7f3b41ffb700 1 -- 172.21.15.140:0/919309509 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3b4801b070 con 0x7f3b4c0af180 2022-07-21T01:51:53.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.983+0000 7f3b518a1700 1 -- 172.21.15.140:0/919309509 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4c065dc0 con 0x7f3b4c0af180 2022-07-21T01:51:53.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.983+0000 7f3b41ffb700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b2c053b00 0x7f3b2c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:53.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.983+0000 7f3b41ffb700 1 -- 172.21.15.140:0/919309509 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3b4806b1f0 con 0x7f3b4c0af180 2022-07-21T01:51:53.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.983+0000 7f3b43fff700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b2c053b00 0x7f3b2c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:53.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.983+0000 7f3b43fff700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b2c053b00 0x7f3b2c055fc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f3b3c01a000 tx=0x7f3b3c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:53.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:53.991+0000 7f3b41ffb700 1 -- 172.21.15.140:0/919309509 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3b48042440 con 0x7f3b4c0af180 2022-07-21T01:51:54.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 -- 172.21.15.140:0/919309509 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3b4c0574e0 con 0x7f3b4c0af180 2022-07-21T01:51:54.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b41ffb700 1 -- 172.21.15.140:0/919309509 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v48) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3b48045a60 con 0x7f3b4c0af180 2022-07-21T01:51:54.115 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:51:54.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 -- 172.21.15.140:0/919309509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b2c053b00 msgr2=0x7f3b2c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b2c053b00 0x7f3b2c055fc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f3b3c01a000 tx=0x7f3b3c014000).stop 2022-07-21T01:51:54.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 -- 172.21.15.140:0/919309509 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b4c0af180 msgr2=0x7f3b4c138e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b4c0af180 0x7f3b4c138e60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f3b48001840 tx=0x7f3b48001720).stop 2022-07-21T01:51:54.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 -- 172.21.15.140:0/919309509 shutdown_connections 2022-07-21T01:51:54.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b2c053b00 0x7f3b2c055fc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b4c128d70 0x7f3b4c14f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b4c0b59d0 0x7f3b4c14f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 --2- 172.21.15.140:0/919309509 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b4c0af180 0x7f3b4c138e60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 -- 172.21.15.140:0/919309509 >> 172.21.15.140:0/919309509 conn(0x7f3b4c0634c0 msgr2=0x7f3b4c128430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:54.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 -- 172.21.15.140:0/919309509 shutdown_connections 2022-07-21T01:51:54.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.115+0000 7f3b518a1700 1 -- 172.21.15.140:0/919309509 wait complete. 2022-07-21T01:51:54.126 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:51:54.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.227+0000 7f3a4b59e700 1 Processor -- start 2022-07-21T01:51:54.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.227+0000 7f3a4b59e700 1 -- start start 2022-07-21T01:51:54.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3a4c0b5650 0x7f3a4c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a4c0af8e0 0x7f3a4c05bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4c147010 con 0x7f3a4c0adff0 2022-07-21T01:51:54.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4c1472b0 con 0x7f3a4c0b5650 2022-07-21T01:51:54.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c146d40 con 0x7f3a4c0af8e0 2022-07-21T01:51:54.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a49d9b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f3a4c0adff0 0x7f3a4c147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49230/0 (socket says 172.21.15.140:49230) 2022-07-21T01:51:54.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a49d9b700 1 -- 172.21.15.140:0/2369181624 learned_addr learned my addr 172.21.15.140:0/2369181624 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:54.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4a59c700 1 --2- 172.21.15.140:0/2369181624 >> v2:172.21.15.140:3300/0 conn(0x7f3a4c0b5650 0x7f3a4c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4ad9d700 1 --2- 172.21.15.140:0/2369181624 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a4c0af8e0 0x7f3a4c05bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4959a700 1 -- 172.21.15.140:0/2369181624 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987307885 0 0) 0x7f3a4c147010 con 0x7f3a4c0adff0 2022-07-21T01:51:54.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4959a700 1 -- 172.21.15.140:0/2369181624 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a2c003260 con 0x7f3a4c0adff0 2022-07-21T01:51:54.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4a59c700 1 -- 172.21.15.140:0/2369181624 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a4c0af8e0 msgr2=0x7f3a4c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:54.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4a59c700 1 --2- 172.21.15.140:0/2369181624 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a4c0af8e0 0x7f3a4c05bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4a59c700 1 -- 172.21.15.140:0/2369181624 >> v1:172.21.15.140:6789/0 conn(0x7f3a4c0adff0 legacy=0x7f3a4c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4a59c700 1 -- 172.21.15.140:0/2369181624 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c0acae0 con 0x7f3a4c0b5650 2022-07-21T01:51:54.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4a59c700 1 --2- 172.21.15.140:0/2369181624 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a4c0b5650 0x7f3a4c0b5a30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f3a3c002f70 tx=0x7f3a3c006df0).ready entity=mon.0 client_cookie=c36647bbd249079e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:54.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4959a700 1 -- 172.21.15.140:0/2369181624 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3c00d4a0 con 0x7f3a4c0b5650 2022-07-21T01:51:54.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4959a700 1 -- 172.21.15.140:0/2369181624 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c000e90 con 0x7f3a4c0b5650 2022-07-21T01:51:54.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4959a700 1 -- 172.21.15.140:0/2369181624 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a3c00d7f0 con 0x7f3a4c0b5650 2022-07-21T01:51:54.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 -- 172.21.15.140:0/2369181624 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a4c0b5650 msgr2=0x7f3a4c0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 --2- 172.21.15.140:0/2369181624 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a4c0b5650 0x7f3a4c0b5a30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f3a3c002f70 tx=0x7f3a3c006df0).stop 2022-07-21T01:51:54.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 -- 172.21.15.140:0/2369181624 shutdown_connections 2022-07-21T01:51:54.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 --2- 172.21.15.140:0/2369181624 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a4c0af8e0 0x7f3a4c05bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 --2- 172.21.15.140:0/2369181624 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a4c0b5650 0x7f3a4c0b5a30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 -- 172.21.15.140:0/2369181624 >> 172.21.15.140:0/2369181624 conn(0x7f3a4c0634e0 msgr2=0x7f3a4c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:54.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 -- 172.21.15.140:0/2369181624 shutdown_connections 2022-07-21T01:51:54.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 -- 172.21.15.140:0/2369181624 wait complete. 2022-07-21T01:51:54.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 Processor -- start 2022-07-21T01:51:54.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.231+0000 7f3a4b59e700 1 -- start start 2022-07-21T01:51:54.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a4c0adff0 0x7f3a4c065ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a4c0af8e0 0x7f3a4c10e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a4c0b5650 0x7f3a4c142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c1597e0 con 0x7f3a4c0b5650 2022-07-21T01:51:54.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c110f50 con 0x7f3a4c0adff0 2022-07-21T01:51:54.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c1111f0 con 0x7f3a4c0af8e0 2022-07-21T01:51:54.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a4c0b5650 0x7f3a4c142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a4c0b5650 0x7f3a4c142ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35094/0 (socket says 172.21.15.140:35094) 2022-07-21T01:51:54.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4ad9d700 1 -- 172.21.15.140:0/1733964494 learned_addr learned my addr 172.21.15.140:0/1733964494 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:54.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4a59c700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a4c0adff0 0x7f3a4c065ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a49d9b700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a4c0af8e0 0x7f3a4c10e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4ad9d700 1 -- 172.21.15.140:0/1733964494 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a4c0af8e0 msgr2=0x7f3a4c10e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4ad9d700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a4c0af8e0 0x7f3a4c10e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4ad9d700 1 -- 172.21.15.140:0/1733964494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a4c0adff0 msgr2=0x7f3a4c065ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4ad9d700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a4c0adff0 0x7f3a4c065ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4ad9d700 1 -- 172.21.15.140:0/1733964494 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c00b180 con 0x7f3a4c0b5650 2022-07-21T01:51:54.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4ad9d700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a4c0b5650 0x7f3a4c142ee0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3a4400c000 tx=0x7f3a44006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:54.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a437fe700 1 -- 172.21.15.140:0/1733964494 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a44011750 con 0x7f3a4c0b5650 2022-07-21T01:51:54.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a437fe700 1 -- 172.21.15.140:0/1733964494 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a440118b0 con 0x7f3a4c0b5650 2022-07-21T01:51:54.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4b59e700 1 -- 172.21.15.140:0/1733964494 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4c063a90 con 0x7f3a4c0b5650 2022-07-21T01:51:54.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a437fe700 1 -- 172.21.15.140:0/1733964494 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a44011c10 con 0x7f3a4c0b5650 2022-07-21T01:51:54.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4b59e700 1 -- 172.21.15.140:0/1733964494 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c0648e0 con 0x7f3a4c0b5650 2022-07-21T01:51:54.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.235+0000 7f3a4b59e700 1 -- 172.21.15.140:0/1733964494 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4c05c280 con 0x7f3a4c0b5650 2022-07-21T01:51:54.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.239+0000 7f3a437fe700 1 -- 172.21.15.140:0/1733964494 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3a44011d70 con 0x7f3a4c0b5650 2022-07-21T01:51:54.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.239+0000 7f3a437fe700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a300539e0 0x7f3a30055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.239+0000 7f3a437fe700 1 -- 172.21.15.140:0/1733964494 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3a4406c0a0 con 0x7f3a4c0b5650 2022-07-21T01:51:54.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.239+0000 7f3a4a59c700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a300539e0 0x7f3a30055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.239+0000 7f3a4a59c700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a300539e0 0x7f3a30055ea0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f3a3c01a000 tx=0x7f3a3c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:54.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.243+0000 7f3a437fe700 1 -- 172.21.15.140:0/1733964494 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3a440431e0 con 0x7f3a4c0b5650 2022-07-21T01:51:54.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 -- 172.21.15.140:0/1733964494 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3a4c0574e0 con 0x7f3a4c0b5650 2022-07-21T01:51:54.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a437fe700 1 -- 172.21.15.140:0/1733964494 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) v1 ==== 84+0+143 (secure 0 0 0) 0x7f3a44046800 con 0x7f3a4c0b5650 2022-07-21T01:51:54.375 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:54.376 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:51:54.376 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:54.377 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:51:54.377 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:51:54.378 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:51:54.378 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:51:54.378 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:51:54.380 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:51:54.380 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:51:54.380 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:51:54.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 -- 172.21.15.140:0/1733964494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a300539e0 msgr2=0x7f3a30055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a300539e0 0x7f3a30055ea0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f3a3c01a000 tx=0x7f3a3c014000).stop 2022-07-21T01:51:54.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 -- 172.21.15.140:0/1733964494 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a4c0b5650 msgr2=0x7f3a4c142ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a4c0b5650 0x7f3a4c142ee0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3a4400c000 tx=0x7f3a44006df0).stop 2022-07-21T01:51:54.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 -- 172.21.15.140:0/1733964494 shutdown_connections 2022-07-21T01:51:54.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3a300539e0 0x7f3a30055ea0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3a4c0b5650 0x7f3a4c142ee0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3a4c0af8e0 0x7f3a4c10e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 --2- 172.21.15.140:0/1733964494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a4c0adff0 0x7f3a4c065ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 -- 172.21.15.140:0/1733964494 >> 172.21.15.140:0/1733964494 conn(0x7f3a4c0634e0 msgr2=0x7f3a4c146400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:54.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 -- 172.21.15.140:0/1733964494 shutdown_connections 2022-07-21T01:51:54.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.375+0000 7f3a4b59e700 1 -- 172.21.15.140:0/1733964494 wait complete. 2022-07-21T01:51:54.387 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:51:54.388 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:51:54.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.495+0000 7fa9569cc700 1 Processor -- start 2022-07-21T01:51:54.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.495+0000 7fa9569cc700 1 -- start start 2022-07-21T01:51:54.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.495+0000 7fa9569cc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa9500b0cc0 0x7fa950143860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.495+0000 7fa9569cc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa95013f940 0x7fa95013fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.495+0000 7fa9569cc700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9500b5f30 con 0x7fa9500b5b50 2022-07-21T01:51:54.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.495+0000 7fa9569cc700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9500af1a0 con 0x7fa95013f940 2022-07-21T01:51:54.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.495+0000 7fa9569cc700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9500af440 con 0x7fa9500b0cc0 2022-07-21T01:51:54.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa954f69700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa95013f940 0x7fa95013fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa954f69700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa95013f940 0x7fa95013fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35102/0 (socket says 172.21.15.140:35102) 2022-07-21T01:51:54.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa954f69700 1 -- 172.21.15.140:0/391081472 learned_addr learned my addr 172.21.15.140:0/391081472 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:54.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa94ffff700 1 --1- 172.21.15.140:0/391081472 >> v1:172.21.15.140:6789/0 conn(0x7fa9500b5b50 0x7fa950143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49244/0 (socket says 172.21.15.140:49244) 2022-07-21T01:51:54.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa94f7fe700 1 --2- 172.21.15.140:0/391081472 >> v2:172.21.15.152:3300/0 conn(0x7fa9500b0cc0 0x7fa950143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa954f69700 1 -- 172.21.15.140:0/391081472 >> v2:172.21.15.152:3300/0 conn(0x7fa9500b0cc0 msgr2=0x7fa950143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:54.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa954f69700 1 --2- 172.21.15.140:0/391081472 >> v2:172.21.15.152:3300/0 conn(0x7fa9500b0cc0 0x7fa950143860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa954f69700 1 -- 172.21.15.140:0/391081472 >> v1:172.21.15.140:6789/0 conn(0x7fa9500b5b50 legacy=0x7fa950143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa954f69700 1 -- 172.21.15.140:0/391081472 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9500af6e0 con 0x7fa95013f940 2022-07-21T01:51:54.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa94f7fe700 1 --2- 172.21.15.140:0/391081472 >> v2:172.21.15.152:3300/0 conn(0x7fa9500b0cc0 0x7fa950143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:51:54.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa954f69700 1 --2- 172.21.15.140:0/391081472 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa95013f940 0x7fa95013fdd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fa9480032f0 tx=0x7fa948009290).ready entity=mon.0 client_cookie=f987bf7a183c995b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:54.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa94effd700 1 -- 172.21.15.140:0/391081472 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa94800b870 con 0x7fa95013f940 2022-07-21T01:51:54.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa94effd700 1 -- 172.21.15.140:0/391081472 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94800b9d0 con 0x7fa95013f940 2022-07-21T01:51:54.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa94effd700 1 -- 172.21.15.140:0/391081472 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa94800bb40 con 0x7fa95013f940 2022-07-21T01:51:54.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa9569cc700 1 -- 172.21.15.140:0/391081472 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa95013f940 msgr2=0x7fa95013fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa9569cc700 1 --2- 172.21.15.140:0/391081472 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa95013f940 0x7fa95013fdd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fa9480032f0 tx=0x7fa948009290).stop 2022-07-21T01:51:54.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa9569cc700 1 -- 172.21.15.140:0/391081472 shutdown_connections 2022-07-21T01:51:54.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa9569cc700 1 --2- 172.21.15.140:0/391081472 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa95013f940 0x7fa95013fdd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa9569cc700 1 --2- 172.21.15.140:0/391081472 >> v2:172.21.15.152:3300/0 conn(0x7fa9500b0cc0 0x7fa950143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa9569cc700 1 -- 172.21.15.140:0/391081472 >> 172.21.15.140:0/391081472 conn(0x7fa9500635e0 msgr2=0x7fa9500b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:54.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa9569cc700 1 -- 172.21.15.140:0/391081472 shutdown_connections 2022-07-21T01:51:54.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa9569cc700 1 -- 172.21.15.140:0/391081472 wait complete. 2022-07-21T01:51:54.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa9569cc700 1 Processor -- start 2022-07-21T01:51:54.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.499+0000 7fa9569cc700 1 -- start start 2022-07-21T01:51:54.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa9569cc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9500b0cc0 0x7fa950065c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa9569cc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9500b5b50 0x7fa95013b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa9569cc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95013f940 0x7fa95013bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa9569cc700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95005ce40 con 0x7fa9500b0cc0 2022-07-21T01:51:54.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa9569cc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9500b40c0 con 0x7fa95013f940 2022-07-21T01:51:54.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa9569cc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9500b4360 con 0x7fa9500b5b50 2022-07-21T01:51:54.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9500b0cc0 0x7fa950065c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9500b0cc0 0x7fa950065c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35104/0 (socket says 172.21.15.140:35104) 2022-07-21T01:51:54.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94ffff700 1 -- 172.21.15.140:0/2836800202 learned_addr learned my addr 172.21.15.140:0/2836800202 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:54.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94f7fe700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9500b5b50 0x7fa95013b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa954f69700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95013f940 0x7fa95013bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94ffff700 1 -- 172.21.15.140:0/2836800202 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9500b5b50 msgr2=0x7fa95013b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94ffff700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9500b5b50 0x7fa95013b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94ffff700 1 -- 172.21.15.140:0/2836800202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95013f940 msgr2=0x7fa95013bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:54.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94ffff700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95013f940 0x7fa95013bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:54.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94ffff700 1 -- 172.21.15.140:0/2836800202 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948009530 con 0x7fa9500b0cc0 2022-07-21T01:51:54.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94ffff700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9500b0cc0 0x7fa950065c30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fa9500b1130 tx=0x7fa938007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:54.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94d7fa700 1 -- 172.21.15.140:0/2836800202 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa93800ce30 con 0x7fa9500b0cc0 2022-07-21T01:51:54.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94d7fa700 1 -- 172.21.15.140:0/2836800202 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9380072e0 con 0x7fa9500b0cc0 2022-07-21T01:51:54.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa9569cc700 1 -- 172.21.15.140:0/2836800202 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa950063f90 con 0x7fa9500b0cc0 2022-07-21T01:51:54.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa94d7fa700 1 -- 172.21.15.140:0/2836800202 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa938005740 con 0x7fa9500b0cc0 2022-07-21T01:51:54.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa9569cc700 1 -- 172.21.15.140:0/2836800202 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa950135e70 con 0x7fa9500b0cc0 2022-07-21T01:51:54.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.503+0000 7fa9569cc700 1 -- 172.21.15.140:0/2836800202 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9500b4520 con 0x7fa9500b0cc0 2022-07-21T01:51:54.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.507+0000 7fa94d7fa700 1 -- 172.21.15.140:0/2836800202 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa938010070 con 0x7fa9500b0cc0 2022-07-21T01:51:54.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.507+0000 7fa94d7fa700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa934053980 0x7fa934055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:54.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.507+0000 7fa94d7fa700 1 -- 172.21.15.140:0/2836800202 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa93806b140 con 0x7fa9500b0cc0 2022-07-21T01:51:54.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.507+0000 7fa94f7fe700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa934053980 0x7fa934055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:54.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.507+0000 7fa94f7fe700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa934053980 0x7fa934055e40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fa94000b000 tx=0x7fa940008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:54.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.511+0000 7fa94d7fa700 1 -- 172.21.15.140:0/2836800202 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa938042390 con 0x7fa9500b0cc0 2022-07-21T01:51:54.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:54.639+0000 7fa9569cc700 1 -- 172.21.15.140:0/2836800202 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fa9500574e0 con 0x7fa9500b0cc0 2022-07-21T01:51:55.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.163+0000 7fa94d7fa700 1 -- 172.21.15.140:0/2836800202 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v49) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa938018070 con 0x7fa9500b0cc0 2022-07-21T01:51:55.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.167+0000 7fa9569cc700 1 -- 172.21.15.140:0/2836800202 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa934053980 msgr2=0x7fa934055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:55.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.167+0000 7fa9569cc700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa934053980 0x7fa934055e40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fa94000b000 tx=0x7fa940008040).stop 2022-07-21T01:51:55.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.167+0000 7fa9569cc700 1 -- 172.21.15.140:0/2836800202 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9500b0cc0 msgr2=0x7fa950065c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:55.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.167+0000 7fa9569cc700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9500b0cc0 0x7fa950065c30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fa9500b1130 tx=0x7fa938007c00).stop 2022-07-21T01:51:55.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.171+0000 7fa9569cc700 1 -- 172.21.15.140:0/2836800202 shutdown_connections 2022-07-21T01:51:55.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.171+0000 7fa9569cc700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa934053980 0x7fa934055e40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:55.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.171+0000 7fa9569cc700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa95013f940 0x7fa95013bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:55.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.171+0000 7fa9569cc700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9500b5b50 0x7fa95013b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:55.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.171+0000 7fa9569cc700 1 --2- 172.21.15.140:0/2836800202 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9500b0cc0 0x7fa950065c30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:55.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.171+0000 7fa9569cc700 1 -- 172.21.15.140:0/2836800202 >> 172.21.15.140:0/2836800202 conn(0x7fa9500635e0 msgr2=0x7fa9500acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:55.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.171+0000 7fa9569cc700 1 -- 172.21.15.140:0/2836800202 shutdown_connections 2022-07-21T01:51:55.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.171+0000 7fa9569cc700 1 -- 172.21.15.140:0/2836800202 wait complete. 2022-07-21T01:51:55.175 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:51:55.183 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:51:55.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae31754700 1 Processor -- start 2022-07-21T01:51:55.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae31754700 1 -- start start 2022-07-21T01:51:55.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae31754700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fae2c1462a0 0x7fae2c146680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:55.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae31754700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fae2c0988b0 0x7fae2c146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:55.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae31754700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c0afdb0 con 0x7fae2c1422c0 2022-07-21T01:51:55.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae31754700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2c0b0050 con 0x7fae2c1462a0 2022-07-21T01:51:55.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae31754700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c05fcd0 con 0x7fae2c0988b0 2022-07-21T01:51:55.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae2affd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fae2c1462a0 0x7fae2c146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:55.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae2affd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fae2c1462a0 0x7fae2c146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:40852/0 (socket says 172.21.15.140:40852) 2022-07-21T01:51:55.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae2affd700 1 -- 172.21.15.140:0/3468935563 learned_addr learned my addr 172.21.15.140:0/3468935563 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:55.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae2b7fe700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fae2c1422c0 0x7fae2c060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:50376/0 (socket says 172.21.15.140:50376) 2022-07-21T01:51:55.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae2a7fc700 1 --2- 172.21.15.140:0/3468935563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fae2c0988b0 0x7fae2c146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:55.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae2affd700 1 -- 172.21.15.140:0/3468935563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fae2c0988b0 msgr2=0x7fae2c146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:55.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae2affd700 1 --2- 172.21.15.140:0/3468935563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fae2c0988b0 0x7fae2c146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:55.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae2affd700 1 -- 172.21.15.140:0/3468935563 >> v1:172.21.15.152:6789/0 conn(0x7fae2c1422c0 legacy=0x7fae2c060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:55.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae2affd700 1 -- 172.21.15.140:0/3468935563 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c0afae0 con 0x7fae2c1462a0 2022-07-21T01:51:55.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae2a7fc700 1 --2- 172.21.15.140:0/3468935563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fae2c0988b0 0x7fae2c146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:51:55.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.295+0000 7fae2affd700 1 --2- 172.21.15.140:0/3468935563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae2c1462a0 0x7fae2c146680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fae240032f0 tx=0x7fae24009290).ready entity=mon.1 client_cookie=9a3f93939ee7ce8c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:55.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.299+0000 7fae29ffb700 1 -- 172.21.15.140:0/3468935563 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae2400b8a0 con 0x7fae2c1462a0 2022-07-21T01:51:55.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.299+0000 7fae29ffb700 1 -- 172.21.15.140:0/3468935563 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2400ba00 con 0x7fae2c1462a0 2022-07-21T01:51:55.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.299+0000 7fae29ffb700 1 -- 172.21.15.140:0/3468935563 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae2400bb70 con 0x7fae2c1462a0 2022-07-21T01:51:55.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.299+0000 7fae31754700 1 -- 172.21.15.140:0/3468935563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae2c1462a0 msgr2=0x7fae2c146680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:55.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.299+0000 7fae31754700 1 --2- 172.21.15.140:0/3468935563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae2c1462a0 0x7fae2c146680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fae240032f0 tx=0x7fae24009290).stop 2022-07-21T01:51:55.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.299+0000 7fae31754700 1 -- 172.21.15.140:0/3468935563 shutdown_connections 2022-07-21T01:51:55.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.299+0000 7fae31754700 1 --2- 172.21.15.140:0/3468935563 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fae2c0988b0 0x7fae2c146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:55.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.299+0000 7fae31754700 1 --2- 172.21.15.140:0/3468935563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae2c1462a0 0x7fae2c146680 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:55.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.299+0000 7fae31754700 1 -- 172.21.15.140:0/3468935563 >> 172.21.15.140:0/3468935563 conn(0x7fae2c0b5f00 msgr2=0x7fae2c0b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:55.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.299+0000 7fae31754700 1 -- 172.21.15.140:0/3468935563 shutdown_connections 2022-07-21T01:51:55.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.299+0000 7fae31754700 1 -- 172.21.15.140:0/3468935563 wait complete. 2022-07-21T01:51:55.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae31754700 1 Processor -- start 2022-07-21T01:51:55.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae31754700 1 -- start start 2022-07-21T01:51:55.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae31754700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fae2c0988b0 0x7fae2c09c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:55.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae31754700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fae2c1422c0 0x7fae2c155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:55.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae31754700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae2c1462a0 0x7fae2c155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:55.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae31754700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c145520 con 0x7fae2c0988b0 2022-07-21T01:51:55.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae31754700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c144440 con 0x7fae2c1462a0 2022-07-21T01:51:55.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae31754700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c1446e0 con 0x7fae2c1422c0 2022-07-21T01:51:55.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fae2c0988b0 0x7fae2c09c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:55.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fae2c0988b0 0x7fae2c09c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35120/0 (socket says 172.21.15.140:35120) 2022-07-21T01:51:55.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2affd700 1 -- 172.21.15.140:0/3719447476 learned_addr learned my addr 172.21.15.140:0/3719447476 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:55.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2b7fe700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae2c1462a0 0x7fae2c155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:55.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2a7fc700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fae2c1422c0 0x7fae2c155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:55.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2affd700 1 -- 172.21.15.140:0/3719447476 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fae2c1422c0 msgr2=0x7fae2c155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:55.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2affd700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fae2c1422c0 0x7fae2c155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:55.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2affd700 1 -- 172.21.15.140:0/3719447476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae2c1462a0 msgr2=0x7fae2c155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:55.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2affd700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae2c1462a0 0x7fae2c155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:55.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2affd700 1 -- 172.21.15.140:0/3719447476 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1400b180 con 0x7fae2c0988b0 2022-07-21T01:51:55.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2b7fe700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae2c1462a0 0x7fae2c155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:51:55.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae2affd700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fae2c0988b0 0x7fae2c09c7d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fae240032f0 tx=0x7fae24016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:55.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae23fff700 1 -- 172.21.15.140:0/3719447476 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae2400b8a0 con 0x7fae2c0988b0 2022-07-21T01:51:55.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae23fff700 1 -- 172.21.15.140:0/3719447476 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae24021440 con 0x7fae2c0988b0 2022-07-21T01:51:55.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae31754700 1 -- 172.21.15.140:0/3719447476 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae24009530 con 0x7fae2c0988b0 2022-07-21T01:51:55.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.303+0000 7fae23fff700 1 -- 172.21.15.140:0/3719447476 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae240217a0 con 0x7fae2c0988b0 2022-07-21T01:51:55.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.307+0000 7fae31754700 1 -- 172.21.15.140:0/3719447476 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2c09e670 con 0x7fae2c0988b0 2022-07-21T01:51:55.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.307+0000 7fae31754700 1 -- 172.21.15.140:0/3719447476 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2c0574e0 con 0x7fae2c0988b0 2022-07-21T01:51:55.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.307+0000 7fae23fff700 1 -- 172.21.15.140:0/3719447476 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fae24027020 con 0x7fae2c0988b0 2022-07-21T01:51:55.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.307+0000 7fae23fff700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fae10053a70 0x7fae10055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:55.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.307+0000 7fae23fff700 1 -- 172.21.15.140:0/3719447476 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fae2406cf70 con 0x7fae2c0988b0 2022-07-21T01:51:55.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.307+0000 7fae2a7fc700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fae10053a70 0x7fae10055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:55.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.311+0000 7fae2a7fc700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fae10053a70 0x7fae10055f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fae1400b000 tx=0x7fae14006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:55.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.315+0000 7fae23fff700 1 -- 172.21.15.140:0/3719447476 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fae24044060 con 0x7fae2c0988b0 2022-07-21T01:51:55.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:55.447+0000 7fae31754700 1 -- 172.21.15.140:0/3719447476 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fae2c069a60 con 0x7fae2c0988b0 2022-07-21T01:51:56.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.191+0000 7fae23fff700 1 -- 172.21.15.140:0/3719447476 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v50) v1 ==== 125+0+0 (secure 0 0 0) 0x7fae24015070 con 0x7fae2c0988b0 2022-07-21T01:51:56.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 -- 172.21.15.140:0/3719447476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fae10053a70 msgr2=0x7fae10055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fae10053a70 0x7fae10055f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fae1400b000 tx=0x7fae14006df0).stop 2022-07-21T01:51:56.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 -- 172.21.15.140:0/3719447476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fae2c0988b0 msgr2=0x7fae2c09c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fae2c0988b0 0x7fae2c09c7d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fae240032f0 tx=0x7fae24016000).stop 2022-07-21T01:51:56.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 -- 172.21.15.140:0/3719447476 shutdown_connections 2022-07-21T01:51:56.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fae10053a70 0x7fae10055f30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fae2c1462a0 0x7fae2c155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fae2c1422c0 0x7fae2c155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 --2- 172.21.15.140:0/3719447476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fae2c0988b0 0x7fae2c09c7d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 -- 172.21.15.140:0/3719447476 >> 172.21.15.140:0/3719447476 conn(0x7fae2c0b5f00 msgr2=0x7fae2c146680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:56.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 -- 172.21.15.140:0/3719447476 shutdown_connections 2022-07-21T01:51:56.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.195+0000 7fae31754700 1 -- 172.21.15.140:0/3719447476 wait complete. 2022-07-21T01:51:56.205 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:51:56.208 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:51:56.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.315+0000 7f30655cb700 1 Processor -- start 2022-07-21T01:51:56.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.315+0000 7f30655cb700 1 -- start start 2022-07-21T01:51:56.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f30600b6a70 0x7f30600b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f30600ae670 0x7f30600aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30600acae0 con 0x7f30600b6690 2022-07-21T01:51:56.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3060147100 con 0x7f30600b6a70 2022-07-21T01:51:56.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30600acd10 con 0x7f30600ae670 2022-07-21T01:51:56.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305e7fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f30600b6a70 0x7f30600b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305e7fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f30600b6a70 0x7f30600b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35128/0 (socket says 172.21.15.140:35128) 2022-07-21T01:51:56.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305e7fc700 1 -- 172.21.15.140:0/1749328880 learned_addr learned my addr 172.21.15.140:0/1749328880 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:56.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305f7fe700 1 --2- 172.21.15.140:0/1749328880 >> v2:172.21.15.152:3300/0 conn(0x7f30600ae670 0x7f30600aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305e7fc700 1 -- 172.21.15.140:0/1749328880 >> v2:172.21.15.152:3300/0 conn(0x7f30600ae670 msgr2=0x7f30600aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:56.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305e7fc700 1 --2- 172.21.15.140:0/1749328880 >> v2:172.21.15.152:3300/0 conn(0x7f30600ae670 0x7f30600aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305e7fc700 1 -- 172.21.15.140:0/1749328880 >> v1:172.21.15.140:6789/0 conn(0x7f30600b6690 legacy=0x7f306013c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305e7fc700 1 -- 172.21.15.140:0/1749328880 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30600acfb0 con 0x7f30600b6a70 2022-07-21T01:51:56.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305dffb700 1 -- 172.21.15.140:0/1749328880 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327953277 0 0) 0x7f30600acae0 con 0x7f30600b6690 2022-07-21T01:51:56.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305f7fe700 1 --2- 172.21.15.140:0/1749328880 >> v2:172.21.15.152:3300/0 conn(0x7f30600ae670 0x7f30600aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:51:56.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305e7fc700 1 --2- 172.21.15.140:0/1749328880 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30600b6a70 0x7f30600b59a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f3054002f70 tx=0x7f3054006eb0).ready entity=mon.0 client_cookie=87a1c0bb7a63ff5e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:56.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305dffb700 1 -- 172.21.15.140:0/1749328880 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f305400d610 con 0x7f30600b6a70 2022-07-21T01:51:56.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305dffb700 1 -- 172.21.15.140:0/1749328880 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3054000e90 con 0x7f30600b6a70 2022-07-21T01:51:56.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f305dffb700 1 -- 172.21.15.140:0/1749328880 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f305400d960 con 0x7f30600b6a70 2022-07-21T01:51:56.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 -- 172.21.15.140:0/1749328880 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30600b6a70 msgr2=0x7f30600b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 --2- 172.21.15.140:0/1749328880 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30600b6a70 0x7f30600b59a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f3054002f70 tx=0x7f3054006eb0).stop 2022-07-21T01:51:56.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 -- 172.21.15.140:0/1749328880 shutdown_connections 2022-07-21T01:51:56.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 --2- 172.21.15.140:0/1749328880 >> v2:172.21.15.152:3300/0 conn(0x7f30600ae670 0x7f30600aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 --2- 172.21.15.140:0/1749328880 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30600b6a70 0x7f30600b59a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 -- 172.21.15.140:0/1749328880 >> 172.21.15.140:0/1749328880 conn(0x7f30600b9f70 msgr2=0x7f30600b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:56.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.319+0000 7f30655cb700 1 -- 172.21.15.140:0/1749328880 shutdown_connections 2022-07-21T01:51:56.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f30655cb700 1 -- 172.21.15.140:0/1749328880 wait complete. 2022-07-21T01:51:56.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f30655cb700 1 Processor -- start 2022-07-21T01:51:56.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f30655cb700 1 -- start start 2022-07-21T01:51:56.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f30655cb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30600ae670 0x7f306013fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f30655cb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30600b6690 0x7f306013d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f30655cb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600b6a70 0x7f3060155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f30655cb700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30600b27c0 con 0x7f30600ae670 2022-07-21T01:51:56.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f30655cb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30600b3390 con 0x7f30600b6690 2022-07-21T01:51:56.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f30655cb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30600b30f0 con 0x7f30600b6a70 2022-07-21T01:51:56.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f305effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30600ae670 0x7f306013fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f305effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30600ae670 0x7f306013fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35132/0 (socket says 172.21.15.140:35132) 2022-07-21T01:51:56.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f305effd700 1 -- 172.21.15.140:0/947949524 learned_addr learned my addr 172.21.15.140:0/947949524 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:56.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f305f7fe700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600b6a70 0x7f3060155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.323+0000 7f305e7fc700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30600b6690 0x7f306013d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f305effd700 1 -- 172.21.15.140:0/947949524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600b6a70 msgr2=0x7f3060155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f305effd700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600b6a70 0x7f3060155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f305effd700 1 -- 172.21.15.140:0/947949524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30600b6690 msgr2=0x7f306013d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f305effd700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30600b6690 0x7f306013d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f305effd700 1 -- 172.21.15.140:0/947949524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305400b1e0 con 0x7f30600ae670 2022-07-21T01:51:56.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f305effd700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30600ae670 0x7f306013fd50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f3050000ed0 tx=0x7f3050007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:56.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f304ffff700 1 -- 172.21.15.140:0/947949524 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3050012400 con 0x7f30600ae670 2022-07-21T01:51:56.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f304ffff700 1 -- 172.21.15.140:0/947949524 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3050012560 con 0x7f30600ae670 2022-07-21T01:51:56.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f30655cb700 1 -- 172.21.15.140:0/947949524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306013dc80 con 0x7f30600ae670 2022-07-21T01:51:56.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f304ffff700 1 -- 172.21.15.140:0/947949524 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30500128c0 con 0x7f30600ae670 2022-07-21T01:51:56.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f30655cb700 1 -- 172.21.15.140:0/947949524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30600b1d60 con 0x7f30600ae670 2022-07-21T01:51:56.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f304ffff700 1 -- 172.21.15.140:0/947949524 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3050019030 con 0x7f30600ae670 2022-07-21T01:51:56.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f30655cb700 1 -- 172.21.15.140:0/947949524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306013de10 con 0x7f30600ae670 2022-07-21T01:51:56.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.327+0000 7f304ffff700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f303c053a70 0x7f303c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.331+0000 7f304ffff700 1 -- 172.21.15.140:0/947949524 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f305006b0e0 con 0x7f30600ae670 2022-07-21T01:51:56.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.331+0000 7f305e7fc700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f303c053a70 0x7f303c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.331+0000 7f305e7fc700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f303c053a70 0x7f303c055f30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f305401c000 tx=0x7f3054015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:56.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.335+0000 7f304ffff700 1 -- 172.21.15.140:0/947949524 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3050042120 con 0x7f30600ae670 2022-07-21T01:51:56.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.495+0000 7f30655cb700 1 -- 172.21.15.140:0/947949524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f30600ad250 con 0x7f30600ae670 2022-07-21T01:51:56.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.507+0000 7f304ffff700 1 -- 172.21.15.140:0/947949524 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v20) v1 ==== 188+0+0 (secure 0 0 0) 0x7f3050045740 con 0x7f30600ae670 2022-07-21T01:51:56.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.507+0000 7f30655cb700 1 -- 172.21.15.140:0/947949524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f303c053a70 msgr2=0x7f303c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.507+0000 7f30655cb700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f303c053a70 0x7f303c055f30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f305401c000 tx=0x7f3054015000).stop 2022-07-21T01:51:56.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.507+0000 7f30655cb700 1 -- 172.21.15.140:0/947949524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30600ae670 msgr2=0x7f306013fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.507+0000 7f30655cb700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30600ae670 0x7f306013fd50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f3050000ed0 tx=0x7f3050007ac0).stop 2022-07-21T01:51:56.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.511+0000 7f30655cb700 1 -- 172.21.15.140:0/947949524 shutdown_connections 2022-07-21T01:51:56.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.511+0000 7f30655cb700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f303c053a70 0x7f303c055f30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.511+0000 7f30655cb700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600b6a70 0x7f3060155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.511+0000 7f30655cb700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30600b6690 0x7f306013d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.511+0000 7f30655cb700 1 --2- 172.21.15.140:0/947949524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30600ae670 0x7f306013fd50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.511+0000 7f30655cb700 1 -- 172.21.15.140:0/947949524 >> 172.21.15.140:0/947949524 conn(0x7f30600b9f70 msgr2=0x7f30600b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:56.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.511+0000 7f30655cb700 1 -- 172.21.15.140:0/947949524 shutdown_connections 2022-07-21T01:51:56.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.511+0000 7f30655cb700 1 -- 172.21.15.140:0/947949524 wait complete. 2022-07-21T01:51:56.512 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 9 and data pool 10 2022-07-21T01:51:56.523 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:51:56.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4eb22700 1 Processor -- start 2022-07-21T01:51:56.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4eb22700 1 -- start start 2022-07-21T01:51:56.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4eb22700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1c48063d60 0x7f1c480b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4eb22700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1c480b1440 0x7f1c480acae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4eb22700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c48064140 con 0x7f1c480af150 2022-07-21T01:51:56.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4eb22700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c48143800 con 0x7f1c480b1440 2022-07-21T01:51:56.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4eb22700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c48143aa0 con 0x7f1c48063d60 2022-07-21T01:51:56.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4d31f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1c480b1440 0x7f1c480acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4e321700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1c480af150 0x7f1c480acfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49286/0 (socket says 172.21.15.140:49286) 2022-07-21T01:51:56.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4d31f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1c480b1440 0x7f1c480acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35140/0 (socket says 172.21.15.140:35140) 2022-07-21T01:51:56.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4e321700 1 -- 172.21.15.140:0/3488949315 learned_addr learned my addr 172.21.15.140:0/3488949315 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:56.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4db20700 1 --2- 172.21.15.140:0/3488949315 >> v2:172.21.15.152:3300/0 conn(0x7f1c48063d60 0x7f1c480b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4d31f700 1 -- 172.21.15.140:0/3488949315 >> v2:172.21.15.152:3300/0 conn(0x7f1c48063d60 msgr2=0x7f1c480b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:56.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4d31f700 1 --2- 172.21.15.140:0/3488949315 >> v2:172.21.15.152:3300/0 conn(0x7f1c48063d60 0x7f1c480b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4d31f700 1 -- 172.21.15.140:0/3488949315 >> v1:172.21.15.140:6789/0 conn(0x7f1c480af150 legacy=0x7f1c480acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4d31f700 1 -- 172.21.15.140:0/3488949315 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c480b1050 con 0x7f1c480b1440 2022-07-21T01:51:56.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.619+0000 7f1c4d31f700 1 --2- 172.21.15.140:0/3488949315 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c480b1440 0x7f1c480acae0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f1c3c002f70 tx=0x7f1c3c006e20).ready entity=mon.0 client_cookie=d316109bcee40a1b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:56.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4cb1e700 1 -- 172.21.15.140:0/3488949315 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3c00bbb0 con 0x7f1c480b1440 2022-07-21T01:51:56.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4cb1e700 1 -- 172.21.15.140:0/3488949315 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c000e90 con 0x7f1c480b1440 2022-07-21T01:51:56.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4cb1e700 1 -- 172.21.15.140:0/3488949315 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3c010470 con 0x7f1c480b1440 2022-07-21T01:51:56.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4eb22700 1 -- 172.21.15.140:0/3488949315 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c480b1440 msgr2=0x7f1c480acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4eb22700 1 --2- 172.21.15.140:0/3488949315 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c480b1440 0x7f1c480acae0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f1c3c002f70 tx=0x7f1c3c006e20).stop 2022-07-21T01:51:56.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4eb22700 1 -- 172.21.15.140:0/3488949315 shutdown_connections 2022-07-21T01:51:56.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4eb22700 1 --2- 172.21.15.140:0/3488949315 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c480b1440 0x7f1c480acae0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4eb22700 1 --2- 172.21.15.140:0/3488949315 >> v2:172.21.15.152:3300/0 conn(0x7f1c48063d60 0x7f1c480b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4eb22700 1 -- 172.21.15.140:0/3488949315 >> 172.21.15.140:0/3488949315 conn(0x7f1c480634e0 msgr2=0x7f1c480b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:56.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4eb22700 1 -- 172.21.15.140:0/3488949315 shutdown_connections 2022-07-21T01:51:56.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4eb22700 1 -- 172.21.15.140:0/3488949315 wait complete. 2022-07-21T01:51:56.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4eb22700 1 Processor -- start 2022-07-21T01:51:56.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.623+0000 7f1c4eb22700 1 -- start start 2022-07-21T01:51:56.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4eb22700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c48063d60 0x7f1c48065950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4eb22700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c480af150 0x7f1c480b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4eb22700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c480b1440 0x7f1c48066430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4eb22700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4813a510 con 0x7f1c48063d60 2022-07-21T01:51:56.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4eb22700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4814db20 con 0x7f1c480b1440 2022-07-21T01:51:56.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4eb22700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c48065030 con 0x7f1c480af150 2022-07-21T01:51:56.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4db20700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c48063d60 0x7f1c48065950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4db20700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c48063d60 0x7f1c48065950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35146/0 (socket says 172.21.15.140:35146) 2022-07-21T01:51:56.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4db20700 1 -- 172.21.15.140:0/2497932724 learned_addr learned my addr 172.21.15.140:0/2497932724 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:56.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4d31f700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c480af150 0x7f1c480b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4e321700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c480b1440 0x7f1c48066430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4db20700 1 -- 172.21.15.140:0/2497932724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c480af150 msgr2=0x7f1c480b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4db20700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c480af150 0x7f1c480b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4db20700 1 -- 172.21.15.140:0/2497932724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c480b1440 msgr2=0x7f1c48066430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4db20700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c480b1440 0x7f1c48066430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4db20700 1 -- 172.21.15.140:0/2497932724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c00d3b0 con 0x7f1c48063d60 2022-07-21T01:51:56.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4db20700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c48063d60 0x7f1c48065950 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f1c48067120 tx=0x7f1c44008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:56.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c3affd700 1 -- 172.21.15.140:0/2497932724 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c44011780 con 0x7f1c48063d60 2022-07-21T01:51:56.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c3affd700 1 -- 172.21.15.140:0/2497932724 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c440118e0 con 0x7f1c48063d60 2022-07-21T01:51:56.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4eb22700 1 -- 172.21.15.140:0/2497932724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c48151b70 con 0x7f1c48063d60 2022-07-21T01:51:56.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c3affd700 1 -- 172.21.15.140:0/2497932724 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c44011c40 con 0x7f1c48063d60 2022-07-21T01:51:56.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4eb22700 1 -- 172.21.15.140:0/2497932724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c48151e10 con 0x7f1c48063d60 2022-07-21T01:51:56.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.627+0000 7f1c4eb22700 1 -- 172.21.15.140:0/2497932724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c480574e0 con 0x7f1c48063d60 2022-07-21T01:51:56.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.631+0000 7f1c3affd700 1 -- 172.21.15.140:0/2497932724 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1c4401c460 con 0x7f1c48063d60 2022-07-21T01:51:56.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.631+0000 7f1c3affd700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c28053b00 0x7f1c28055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.631+0000 7f1c3affd700 1 -- 172.21.15.140:0/2497932724 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1c4406c150 con 0x7f1c48063d60 2022-07-21T01:51:56.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.631+0000 7f1c4d31f700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c28053b00 0x7f1c28055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.631+0000 7f1c4d31f700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c28053b00 0x7f1c28055fc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f1c3c018000 tx=0x7f1c3c006e20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:56.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.635+0000 7f1c3affd700 1 -- 172.21.15.140:0/2497932724 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c44046e60 con 0x7f1c48063d60 2022-07-21T01:51:56.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.759+0000 7f1c4eb22700 1 -- 172.21.15.140:0/2497932724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c48067ea0 con 0x7f1c48063d60 2022-07-21T01:51:56.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.759+0000 7f1c3affd700 1 -- 172.21.15.140:0/2497932724 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16162 (secure 0 0 0) 0x7f1c44042f70 con 0x7f1c48063d60 2022-07-21T01:51:56.760 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:56.761 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":51,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:51:56.500095+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:51:54.695165+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-07-21T01:51:55.500555+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:51:56.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 -- 172.21.15.140:0/2497932724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c28053b00 msgr2=0x7f1c28055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c28053b00 0x7f1c28055fc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f1c3c018000 tx=0x7f1c3c006e20).stop 2022-07-21T01:51:56.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 -- 172.21.15.140:0/2497932724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c48063d60 msgr2=0x7f1c48065950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c48063d60 0x7f1c48065950 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f1c48067120 tx=0x7f1c44008040).stop 2022-07-21T01:51:56.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 -- 172.21.15.140:0/2497932724 shutdown_connections 2022-07-21T01:51:56.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c28053b00 0x7f1c28055fc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c480b1440 0x7f1c48066430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c480af150 0x7f1c480b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 --2- 172.21.15.140:0/2497932724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c48063d60 0x7f1c48065950 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 -- 172.21.15.140:0/2497932724 >> 172.21.15.140:0/2497932724 conn(0x7f1c480634e0 msgr2=0x7f1c4813ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:56.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 -- 172.21.15.140:0/2497932724 shutdown_connections 2022-07-21T01:51:56.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.763+0000 7f1c4eb22700 1 -- 172.21.15.140:0/2497932724 wait complete. 2022-07-21T01:51:56.780 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:51:56.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a96cde700 1 Processor -- start 2022-07-21T01:51:56.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a96cde700 1 -- start start 2022-07-21T01:51:56.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a96cde700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900adff0 0x7f1a900b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a96cde700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1a900af8e0 0x7f1a9005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a96cde700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a90147010 con 0x7f1a900b5650 2022-07-21T01:51:56.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a96cde700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a901472b0 con 0x7f1a900af8e0 2022-07-21T01:51:56.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a96cde700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a90146d40 con 0x7f1a900adff0 2022-07-21T01:51:56.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a964dd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1a900af8e0 0x7f1a9005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a964dd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1a900af8e0 0x7f1a9005bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35158/0 (socket says 172.21.15.140:35158) 2022-07-21T01:51:56.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a964dd700 1 -- 172.21.15.140:0/3302529138 learned_addr learned my addr 172.21.15.140:0/3302529138 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:56.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a95cdc700 1 --1- 172.21.15.140:0/3302529138 >> v1:172.21.15.140:6789/0 conn(0x7f1a900b5650 0x7f1a90147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49302/0 (socket says 172.21.15.140:49302) 2022-07-21T01:51:56.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a954db700 1 --2- 172.21.15.140:0/3302529138 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900adff0 0x7f1a900b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a964dd700 1 -- 172.21.15.140:0/3302529138 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900adff0 msgr2=0x7f1a900b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:56.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a964dd700 1 --2- 172.21.15.140:0/3302529138 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900adff0 0x7f1a900b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a964dd700 1 -- 172.21.15.140:0/3302529138 >> v1:172.21.15.140:6789/0 conn(0x7f1a900b5650 legacy=0x7f1a90147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a94cda700 1 -- 172.21.15.140:0/3302529138 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083394081 0 0) 0x7f1a90147010 con 0x7f1a900b5650 2022-07-21T01:51:56.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a964dd700 1 -- 172.21.15.140:0/3302529138 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a900acae0 con 0x7f1a900af8e0 2022-07-21T01:51:56.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.887+0000 7f1a954db700 1 --2- 172.21.15.140:0/3302529138 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900adff0 0x7f1a900b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:51:56.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a964dd700 1 --2- 172.21.15.140:0/3302529138 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a900af8e0 0x7f1a9005bd90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f1a880032f0 tx=0x7f1a88009290).ready entity=mon.0 client_cookie=b91c38aff32d3df1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:56.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a94cda700 1 -- 172.21.15.140:0/3302529138 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8800b8a0 con 0x7f1a900af8e0 2022-07-21T01:51:56.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a94cda700 1 -- 172.21.15.140:0/3302529138 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8800ba00 con 0x7f1a900af8e0 2022-07-21T01:51:56.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a94cda700 1 -- 172.21.15.140:0/3302529138 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8800bb70 con 0x7f1a900af8e0 2022-07-21T01:51:56.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 -- 172.21.15.140:0/3302529138 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a900af8e0 msgr2=0x7f1a9005bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 --2- 172.21.15.140:0/3302529138 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a900af8e0 0x7f1a9005bd90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f1a880032f0 tx=0x7f1a88009290).stop 2022-07-21T01:51:56.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 -- 172.21.15.140:0/3302529138 shutdown_connections 2022-07-21T01:51:56.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 --2- 172.21.15.140:0/3302529138 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a900af8e0 0x7f1a9005bd90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 --2- 172.21.15.140:0/3302529138 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900adff0 0x7f1a900b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 -- 172.21.15.140:0/3302529138 >> 172.21.15.140:0/3302529138 conn(0x7f1a900634e0 msgr2=0x7f1a900b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:56.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 -- 172.21.15.140:0/3302529138 shutdown_connections 2022-07-21T01:51:56.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 -- 172.21.15.140:0/3302529138 wait complete. 2022-07-21T01:51:56.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 Processor -- start 2022-07-21T01:51:56.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 -- start start 2022-07-21T01:51:56.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a900adff0 0x7f1a90066750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900af8e0 0x7f1a90066c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a900b5650 0x7f1a90159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a900655f0 con 0x7f1a900adff0 2022-07-21T01:51:56.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901467f0 con 0x7f1a900b5650 2022-07-21T01:51:56.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a96cde700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a90156e50 con 0x7f1a900af8e0 2022-07-21T01:51:56.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a95cdc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a900adff0 0x7f1a90066750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a95cdc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a900adff0 0x7f1a90066750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35160/0 (socket says 172.21.15.140:35160) 2022-07-21T01:51:56.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a95cdc700 1 -- 172.21.15.140:0/713249715 learned_addr learned my addr 172.21.15.140:0/713249715 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:56.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a954db700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900af8e0 0x7f1a90066c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a964dd700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a900b5650 0x7f1a90159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a95cdc700 1 -- 172.21.15.140:0/713249715 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900af8e0 msgr2=0x7f1a90066c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a95cdc700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900af8e0 0x7f1a90066c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a95cdc700 1 -- 172.21.15.140:0/713249715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a900b5650 msgr2=0x7f1a90159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:56.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a95cdc700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a900b5650 0x7f1a90159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:56.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a95cdc700 1 -- 172.21.15.140:0/713249715 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a88009530 con 0x7f1a900adff0 2022-07-21T01:51:56.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a954db700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900af8e0 0x7f1a90066c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:51:56.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.891+0000 7f1a95cdc700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a900adff0 0x7f1a90066750 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f1a8c000ed0 tx=0x7f1a8c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:56.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.895+0000 7f1a7effd700 1 -- 172.21.15.140:0/713249715 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8c0070d0 con 0x7f1a900adff0 2022-07-21T01:51:56.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.895+0000 7f1a7effd700 1 -- 172.21.15.140:0/713249715 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c007230 con 0x7f1a900adff0 2022-07-21T01:51:56.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.895+0000 7f1a96cde700 1 -- 172.21.15.140:0/713249715 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a901570f0 con 0x7f1a900adff0 2022-07-21T01:51:56.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.895+0000 7f1a7effd700 1 -- 172.21.15.140:0/713249715 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8c005550 con 0x7f1a900adff0 2022-07-21T01:51:56.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.895+0000 7f1a96cde700 1 -- 172.21.15.140:0/713249715 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a900675f0 con 0x7f1a900adff0 2022-07-21T01:51:56.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.895+0000 7f1a96cde700 1 -- 172.21.15.140:0/713249715 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a900574e0 con 0x7f1a900adff0 2022-07-21T01:51:56.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.895+0000 7f1a7effd700 1 -- 172.21.15.140:0/713249715 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1a8c010070 con 0x7f1a900adff0 2022-07-21T01:51:56.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.895+0000 7f1a7effd700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a700539e0 0x7f1a70055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:56.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.895+0000 7f1a7effd700 1 -- 172.21.15.140:0/713249715 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1a8c06b650 con 0x7f1a900adff0 2022-07-21T01:51:56.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.895+0000 7f1a954db700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a700539e0 0x7f1a70055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:56.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.895+0000 7f1a954db700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a700539e0 0x7f1a70055ea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f1a80008000 tx=0x7f1a80008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:56.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:56.903+0000 7f1a7effd700 1 -- 172.21.15.140:0/713249715 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a8c042570 con 0x7f1a900adff0 2022-07-21T01:51:57.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.027+0000 7f1a96cde700 1 -- 172.21.15.140:0/713249715 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a900b2800 con 0x7f1a900adff0 2022-07-21T01:51:57.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a7effd700 1 -- 172.21.15.140:0/713249715 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16162 (secure 0 0 0) 0x7f1a8c045b90 con 0x7f1a900adff0 2022-07-21T01:51:57.029 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:57.030 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":51,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:51:56.500095+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:51:54.695165+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-07-21T01:51:55.500555+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:51:57.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 -- 172.21.15.140:0/713249715 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a700539e0 msgr2=0x7f1a70055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a700539e0 0x7f1a70055ea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f1a80008000 tx=0x7f1a80008040).stop 2022-07-21T01:51:57.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 -- 172.21.15.140:0/713249715 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a900adff0 msgr2=0x7f1a90066750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a900adff0 0x7f1a90066750 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f1a8c000ed0 tx=0x7f1a8c007c80).stop 2022-07-21T01:51:57.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 -- 172.21.15.140:0/713249715 shutdown_connections 2022-07-21T01:51:57.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a700539e0 0x7f1a70055ea0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a900b5650 0x7f1a90159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a900af8e0 0x7f1a90066c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 --2- 172.21.15.140:0/713249715 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a900adff0 0x7f1a90066750 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 -- 172.21.15.140:0/713249715 >> 172.21.15.140:0/713249715 conn(0x7f1a900634e0 msgr2=0x7f1a9014aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:57.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 -- 172.21.15.140:0/713249715 shutdown_connections 2022-07-21T01:51:57.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.031+0000 7f1a96cde700 1 -- 172.21.15.140:0/713249715 wait complete. 2022-07-21T01:51:57.043 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:51:57.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.155+0000 7f2e31b84700 1 Processor -- start 2022-07-21T01:51:57.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.155+0000 7f2e31b84700 1 -- start start 2022-07-21T01:51:57.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.155+0000 7f2e31b84700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e2c0b9da0 0x7f2e2c1468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.155+0000 7f2e31b84700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2e2c0b3cf0 0x7f2e2c142a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.155+0000 7f2e31b84700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2c0ba180 con 0x7f2e2c142f40 2022-07-21T01:51:57.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.155+0000 7f2e31b84700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c0ad5b0 con 0x7f2e2c0b3cf0 2022-07-21T01:51:57.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.155+0000 7f2e31b84700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c1461b0 con 0x7f2e2c0b9da0 2022-07-21T01:51:57.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.155+0000 7f2e31383700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2e2c142f40 0x7f2e2c05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49318/0 (socket says 172.21.15.140:49318) 2022-07-21T01:51:57.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.155+0000 7f2e31383700 1 -- 172.21.15.140:0/2044774991 learned_addr learned my addr 172.21.15.140:0/2044774991 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:57.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.155+0000 7f2e2bfff700 1 --2- 172.21.15.140:0/2044774991 >> v2:172.21.15.152:3300/0 conn(0x7f2e2c0b3cf0 0x7f2e2c142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.155+0000 7f2e30b82700 1 --2- 172.21.15.140:0/2044774991 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e2c0b9da0 0x7f2e2c1468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e2b7fe700 1 -- 172.21.15.140:0/2044774991 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577664652 0 0) 0x7f2e2c0ba180 con 0x7f2e2c142f40 2022-07-21T01:51:57.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e2b7fe700 1 -- 172.21.15.140:0/2044774991 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e0c003260 con 0x7f2e2c142f40 2022-07-21T01:51:57.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e2bfff700 1 -- 172.21.15.140:0/2044774991 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e2c0b9da0 msgr2=0x7f2e2c1468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:57.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e2bfff700 1 --2- 172.21.15.140:0/2044774991 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e2c0b9da0 0x7f2e2c1468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e2bfff700 1 -- 172.21.15.140:0/2044774991 >> v1:172.21.15.140:6789/0 conn(0x7f2e2c142f40 legacy=0x7f2e2c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e2bfff700 1 -- 172.21.15.140:0/2044774991 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c146450 con 0x7f2e2c0b3cf0 2022-07-21T01:51:57.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e30b82700 1 --2- 172.21.15.140:0/2044774991 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e2c0b9da0 0x7f2e2c1468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:51:57.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e2bfff700 1 --2- 172.21.15.140:0/2044774991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e2c0b3cf0 0x7f2e2c142a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f2e14002f70 tx=0x7f2e14006df0).ready entity=mon.1 client_cookie=f951afe35ea21423 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:57.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e2b7fe700 1 -- 172.21.15.140:0/2044774991 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e1400d4a0 con 0x7f2e2c0b3cf0 2022-07-21T01:51:57.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e2b7fe700 1 -- 172.21.15.140:0/2044774991 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e14000e90 con 0x7f2e2c0b3cf0 2022-07-21T01:51:57.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e2b7fe700 1 -- 172.21.15.140:0/2044774991 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e1400d7f0 con 0x7f2e2c0b3cf0 2022-07-21T01:51:57.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e31b84700 1 -- 172.21.15.140:0/2044774991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e2c0b3cf0 msgr2=0x7f2e2c142a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e31b84700 1 --2- 172.21.15.140:0/2044774991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e2c0b3cf0 0x7f2e2c142a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f2e14002f70 tx=0x7f2e14006df0).stop 2022-07-21T01:51:57.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e31b84700 1 -- 172.21.15.140:0/2044774991 shutdown_connections 2022-07-21T01:51:57.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e31b84700 1 --2- 172.21.15.140:0/2044774991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e2c0b3cf0 0x7f2e2c142a50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e31b84700 1 --2- 172.21.15.140:0/2044774991 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e2c0b9da0 0x7f2e2c1468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e31b84700 1 -- 172.21.15.140:0/2044774991 >> 172.21.15.140:0/2044774991 conn(0x7f2e2c0678c0 msgr2=0x7f2e2c0b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:57.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e31b84700 1 -- 172.21.15.140:0/2044774991 shutdown_connections 2022-07-21T01:51:57.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e31b84700 1 -- 172.21.15.140:0/2044774991 wait complete. 2022-07-21T01:51:57.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e31b84700 1 Processor -- start 2022-07-21T01:51:57.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.159+0000 7f2e31b84700 1 -- start start 2022-07-21T01:51:57.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31b84700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e2c0b3cf0 0x7f2e2c0b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31b84700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e2c0b9da0 0x7f2e2c0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31b84700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e2c142f40 0x7f2e2c0af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31b84700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c112930 con 0x7f2e2c142f40 2022-07-21T01:51:57.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31b84700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0b0ce0 con 0x7f2e2c0b9da0 2022-07-21T01:51:57.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31b84700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0b11a0 con 0x7f2e2c0b3cf0 2022-07-21T01:51:57.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31383700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e2c142f40 0x7f2e2c0af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31383700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e2c142f40 0x7f2e2c0af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35178/0 (socket says 172.21.15.140:35178) 2022-07-21T01:51:57.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31383700 1 -- 172.21.15.140:0/815379191 learned_addr learned my addr 172.21.15.140:0/815379191 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:57.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e30b82700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e2c0b3cf0 0x7f2e2c0b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e2bfff700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e2c0b9da0 0x7f2e2c0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31383700 1 -- 172.21.15.140:0/815379191 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e2c0b3cf0 msgr2=0x7f2e2c0b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31383700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e2c0b3cf0 0x7f2e2c0b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31383700 1 -- 172.21.15.140:0/815379191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e2c0b9da0 msgr2=0x7f2e2c0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31383700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e2c0b9da0 0x7f2e2c0b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31383700 1 -- 172.21.15.140:0/815379191 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c009470 con 0x7f2e2c142f40 2022-07-21T01:51:57.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31383700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e2c142f40 0x7f2e2c0af020 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f2e24001110 tx=0x7f2e24007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:57.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e29ffb700 1 -- 172.21.15.140:0/815379191 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e2400cda0 con 0x7f2e2c142f40 2022-07-21T01:51:57.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e29ffb700 1 -- 172.21.15.140:0/815379191 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e240070d0 con 0x7f2e2c142f40 2022-07-21T01:51:57.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31b84700 1 -- 172.21.15.140:0/815379191 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1400b180 con 0x7f2e2c142f40 2022-07-21T01:51:57.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e29ffb700 1 -- 172.21.15.140:0/815379191 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e24007430 con 0x7f2e2c142f40 2022-07-21T01:51:57.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.163+0000 7f2e31b84700 1 -- 172.21.15.140:0/815379191 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c113490 con 0x7f2e2c142f40 2022-07-21T01:51:57.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.167+0000 7f2e31b84700 1 -- 172.21.15.140:0/815379191 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2c0b1ed0 con 0x7f2e2c142f40 2022-07-21T01:51:57.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.167+0000 7f2e29ffb700 1 -- 172.21.15.140:0/815379191 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2e2401a060 con 0x7f2e2c142f40 2022-07-21T01:51:57.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.167+0000 7f2e29ffb700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2e10053a70 0x7f2e10055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.167+0000 7f2e29ffb700 1 -- 172.21.15.140:0/815379191 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2e2406b650 con 0x7f2e2c142f40 2022-07-21T01:51:57.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.167+0000 7f2e30b82700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2e10053a70 0x7f2e10055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.167+0000 7f2e30b82700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2e10053a70 0x7f2e10055f30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f2e1c00b000 tx=0x7f2e1c0091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:57.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.171+0000 7f2e29ffb700 1 -- 172.21.15.140:0/815379191 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2e2c0b1ed0 con 0x7f2e2c142f40 2022-07-21T01:51:57.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.339+0000 7f2e31b84700 1 -- 172.21.15.140:0/815379191 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2e2c0b1ed0 con 0x7f2e2c142f40 2022-07-21T01:51:57.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.515+0000 7f2e29ffb700 1 -- 172.21.15.140:0/815379191 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2e2c0b1ed0 con 0x7f2e2c142f40 2022-07-21T01:51:57.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 -- 172.21.15.140:0/815379191 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2e10053a70 msgr2=0x7f2e10055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2e10053a70 0x7f2e10055f30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f2e1c00b000 tx=0x7f2e1c0091d0).stop 2022-07-21T01:51:57.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 -- 172.21.15.140:0/815379191 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e2c142f40 msgr2=0x7f2e2c0af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e2c142f40 0x7f2e2c0af020 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f2e24001110 tx=0x7f2e24007a80).stop 2022-07-21T01:51:57.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 -- 172.21.15.140:0/815379191 shutdown_connections 2022-07-21T01:51:57.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2e10053a70 0x7f2e10055f30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e2c142f40 0x7f2e2c0af020 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e2c0b9da0 0x7f2e2c0b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 --2- 172.21.15.140:0/815379191 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e2c0b3cf0 0x7f2e2c0b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 -- 172.21.15.140:0/815379191 >> 172.21.15.140:0/815379191 conn(0x7f2e2c0678c0 msgr2=0x7f2e2c1470e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:57.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 -- 172.21.15.140:0/815379191 shutdown_connections 2022-07-21T01:51:57.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.519+0000 7f2e31b84700 1 -- 172.21.15.140:0/815379191 wait complete. 2022-07-21T01:51:57.534 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:51:57.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f403759e700 1 Processor -- start 2022-07-21T01:51:57.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f403759e700 1 -- start start 2022-07-21T01:51:57.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f403759e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f40380b5990 0x7f403813fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f403759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40380aecf0 0x7f4038140020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f403759e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f40380aff80 0x7f403805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f403759e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40380b5d70 con 0x7f40380aff80 2022-07-21T01:51:57.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f403759e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40380651c0 con 0x7f40380b5990 2022-07-21T01:51:57.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f403759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40380b0690 con 0x7f40380aecf0 2022-07-21T01:51:57.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f4036d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f40380aff80 0x7f403805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f4036d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f40380aff80 0x7f403805bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35186/0 (socket says 172.21.15.140:35186) 2022-07-21T01:51:57.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f4036d9d700 1 -- 172.21.15.140:0/1110675664 learned_addr learned my addr 172.21.15.140:0/1110675664 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:57.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f4035d9b700 1 --2- 172.21.15.140:0/1110675664 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40380aecf0 0x7f4038140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f403659c700 1 --2- 172.21.15.140:0/1110675664 >> v2:172.21.15.152:3300/0 conn(0x7f40380b5990 0x7f403813fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f4036d9d700 1 -- 172.21.15.140:0/1110675664 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40380aecf0 msgr2=0x7f4038140020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:57.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f4036d9d700 1 --2- 172.21.15.140:0/1110675664 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40380aecf0 0x7f4038140020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f4036d9d700 1 -- 172.21.15.140:0/1110675664 >> v2:172.21.15.152:3300/0 conn(0x7f40380b5990 msgr2=0x7f403813fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:57.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f4036d9d700 1 --2- 172.21.15.140:0/1110675664 >> v2:172.21.15.152:3300/0 conn(0x7f40380b5990 0x7f403813fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f4036d9d700 1 -- 172.21.15.140:0/1110675664 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40380b0930 con 0x7f40380aff80 2022-07-21T01:51:57.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f4035d9b700 1 --2- 172.21.15.140:0/1110675664 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40380aecf0 0x7f4038140020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:51:57.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.639+0000 7f4036d9d700 1 --2- 172.21.15.140:0/1110675664 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40380aff80 0x7f403805bd90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f4030002f70 tx=0x7f4030006eb0).ready entity=mon.0 client_cookie=5fd18f806f03a587 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:57.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403559a700 1 -- 172.21.15.140:0/1110675664 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403000d570 con 0x7f40380aff80 2022-07-21T01:51:57.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403559a700 1 -- 172.21.15.140:0/1110675664 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4030000e90 con 0x7f40380aff80 2022-07-21T01:51:57.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403559a700 1 -- 172.21.15.140:0/1110675664 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403000d900 con 0x7f40380aff80 2022-07-21T01:51:57.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403759e700 1 -- 172.21.15.140:0/1110675664 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40380aff80 msgr2=0x7f403805bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403759e700 1 --2- 172.21.15.140:0/1110675664 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40380aff80 0x7f403805bd90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f4030002f70 tx=0x7f4030006eb0).stop 2022-07-21T01:51:57.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403759e700 1 -- 172.21.15.140:0/1110675664 shutdown_connections 2022-07-21T01:51:57.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403759e700 1 --2- 172.21.15.140:0/1110675664 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40380aff80 0x7f403805bd90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403759e700 1 --2- 172.21.15.140:0/1110675664 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40380aecf0 0x7f4038140020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403759e700 1 --2- 172.21.15.140:0/1110675664 >> v2:172.21.15.152:3300/0 conn(0x7f40380b5990 0x7f403813fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403759e700 1 -- 172.21.15.140:0/1110675664 >> 172.21.15.140:0/1110675664 conn(0x7f40380634e0 msgr2=0x7f40380b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:57.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403759e700 1 -- 172.21.15.140:0/1110675664 shutdown_connections 2022-07-21T01:51:57.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403759e700 1 -- 172.21.15.140:0/1110675664 wait complete. 2022-07-21T01:51:57.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403759e700 1 Processor -- start 2022-07-21T01:51:57.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.643+0000 7f403759e700 1 -- start start 2022-07-21T01:51:57.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f403759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40380aecf0 0x7f40380ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f403759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40380aff80 0x7f4038150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f403759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40380b5990 0x7f403810e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f403759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4038066070 con 0x7f40380b5990 2022-07-21T01:51:57.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f403759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403810e130 con 0x7f40380aecf0 2022-07-21T01:51:57.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f403759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4038065d90 con 0x7f40380aff80 2022-07-21T01:51:57.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f4036d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40380b5990 0x7f403810e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f403659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40380aecf0 0x7f40380ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f4036d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40380b5990 0x7f403810e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35192/0 (socket says 172.21.15.140:35192) 2022-07-21T01:51:57.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f4036d9d700 1 -- 172.21.15.140:0/3091714658 learned_addr learned my addr 172.21.15.140:0/3091714658 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:57.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f4035d9b700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40380aff80 0x7f4038150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f4036d9d700 1 -- 172.21.15.140:0/3091714658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40380aff80 msgr2=0x7f4038150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f4036d9d700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40380aff80 0x7f4038150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f4036d9d700 1 -- 172.21.15.140:0/3091714658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40380aecf0 msgr2=0x7f40380ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f4036d9d700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40380aecf0 0x7f40380ad0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f4036d9d700 1 -- 172.21.15.140:0/3091714658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403000b1e0 con 0x7f40380b5990 2022-07-21T01:51:57.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f4036d9d700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40380b5990 0x7f403810e5d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f4030002f70 tx=0x7f4030015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:57.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f40277fe700 1 -- 172.21.15.140:0/3091714658 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40300116e0 con 0x7f40380b5990 2022-07-21T01:51:57.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f40277fe700 1 -- 172.21.15.140:0/3091714658 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4030011840 con 0x7f40380b5990 2022-07-21T01:51:57.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f403759e700 1 -- 172.21.15.140:0/3091714658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4038151480 con 0x7f40380b5990 2022-07-21T01:51:57.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f40277fe700 1 -- 172.21.15.140:0/3091714658 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4030011ba0 con 0x7f40380b5990 2022-07-21T01:51:57.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.647+0000 7f403759e700 1 -- 172.21.15.140:0/3091714658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4038143a30 con 0x7f40380b5990 2022-07-21T01:51:57.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.651+0000 7f403759e700 1 -- 172.21.15.140:0/3091714658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40380574e0 con 0x7f40380b5990 2022-07-21T01:51:57.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.651+0000 7f40277fe700 1 -- 172.21.15.140:0/3091714658 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f403001a040 con 0x7f40380b5990 2022-07-21T01:51:57.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.651+0000 7f40277fe700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4014053980 0x7f4014055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.651+0000 7f40277fe700 1 -- 172.21.15.140:0/3091714658 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f403006cd30 con 0x7f40380b5990 2022-07-21T01:51:57.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.651+0000 7f403659c700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4014053980 0x7f4014055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.651+0000 7f403659c700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4014053980 0x7f4014055e40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f402c00c000 tx=0x7f402c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:57.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.655+0000 7f40277fe700 1 -- 172.21.15.140:0/3091714658 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4030043bb0 con 0x7f40380b5990 2022-07-21T01:51:57.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.815+0000 7f403759e700 1 -- 172.21.15.140:0/3091714658 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4038067ea0 con 0x7f40380b5990 2022-07-21T01:51:57.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.815+0000 7f40277fe700 1 -- 172.21.15.140:0/3091714658 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2145 (secure 0 0 0) 0x7f4030023070 con 0x7f40380b5990 2022-07-21T01:51:57.815 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:57.815 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1915503334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1915503334},{"type":"v1","addr":"172.21.15.140:6835","nonce":1915503334}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:51:56.500368+0000","modified":"2022-07-21T01:51:57.504176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5204":{"gid":5204,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3908541795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3908541795},{"type":"v1","addr":"172.21.15.152:6833","nonce":3908541795}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-07-21T01:51:57.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 -- 172.21.15.140:0/3091714658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4014053980 msgr2=0x7f4014055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4014053980 0x7f4014055e40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f402c00c000 tx=0x7f402c008040).stop 2022-07-21T01:51:57.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 -- 172.21.15.140:0/3091714658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40380b5990 msgr2=0x7f403810e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40380b5990 0x7f403810e5d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f4030002f70 tx=0x7f4030015000).stop 2022-07-21T01:51:57.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 -- 172.21.15.140:0/3091714658 shutdown_connections 2022-07-21T01:51:57.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4014053980 0x7f4014055e40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40380b5990 0x7f403810e5d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40380aff80 0x7f4038150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 --2- 172.21.15.140:0/3091714658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40380aecf0 0x7f40380ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 -- 172.21.15.140:0/3091714658 >> 172.21.15.140:0/3091714658 conn(0x7f40380634e0 msgr2=0x7f40380ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:57.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 -- 172.21.15.140:0/3091714658 shutdown_connections 2022-07-21T01:51:57.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.819+0000 7f403759e700 1 -- 172.21.15.140:0/3091714658 wait complete. 2022-07-21T01:51:57.820 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 21 2022-07-21T01:51:57.828 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:51:57.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.931+0000 7f8156554700 1 Processor -- start 2022-07-21T01:51:57.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.931+0000 7f8156554700 1 -- start start 2022-07-21T01:51:57.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81500b1e00 0x7f815014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f815005c3f0 0x7f81500ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f815005bd90 0x7f81500accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f815005d480 con 0x7f815005bd90 2022-07-21T01:51:57.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81500ad1c0 con 0x7f815005c3f0 2022-07-21T01:51:57.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81500b21e0 con 0x7f81500b1e00 2022-07-21T01:51:57.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8154af1700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f815005bd90 0x7f81500accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8154af1700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f815005bd90 0x7f81500accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35200/0 (socket says 172.21.15.140:35200) 2022-07-21T01:51:57.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8154af1700 1 -- 172.21.15.140:0/2863236224 learned_addr learned my addr 172.21.15.140:0/2863236224 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:57.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f814f7fe700 1 --2- 172.21.15.140:0/2863236224 >> v2:172.21.15.152:3300/0 conn(0x7f815005c3f0 0x7f81500ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f814ffff700 1 --2- 172.21.15.140:0/2863236224 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81500b1e00 0x7f815014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8154af1700 1 -- 172.21.15.140:0/2863236224 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81500b1e00 msgr2=0x7f815014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:57.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8154af1700 1 --2- 172.21.15.140:0/2863236224 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81500b1e00 0x7f815014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8154af1700 1 -- 172.21.15.140:0/2863236224 >> v2:172.21.15.152:3300/0 conn(0x7f815005c3f0 msgr2=0x7f81500ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:57.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8154af1700 1 --2- 172.21.15.140:0/2863236224 >> v2:172.21.15.152:3300/0 conn(0x7f815005c3f0 0x7f81500ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8154af1700 1 -- 172.21.15.140:0/2863236224 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815005cf40 con 0x7f815005bd90 2022-07-21T01:51:57.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8154af1700 1 --2- 172.21.15.140:0/2863236224 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f815005bd90 0x7f81500accd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f81480032f0 tx=0x7f8148009290).ready entity=mon.0 client_cookie=dc0d4d18ea4f866f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:57.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f814effd700 1 -- 172.21.15.140:0/2863236224 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f814800b870 con 0x7f815005bd90 2022-07-21T01:51:57.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f814effd700 1 -- 172.21.15.140:0/2863236224 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814800b9d0 con 0x7f815005bd90 2022-07-21T01:51:57.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f814effd700 1 -- 172.21.15.140:0/2863236224 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f814800bb40 con 0x7f815005bd90 2022-07-21T01:51:57.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 -- 172.21.15.140:0/2863236224 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f815005bd90 msgr2=0x7f81500accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 --2- 172.21.15.140:0/2863236224 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f815005bd90 0x7f81500accd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f81480032f0 tx=0x7f8148009290).stop 2022-07-21T01:51:57.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 -- 172.21.15.140:0/2863236224 shutdown_connections 2022-07-21T01:51:57.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 --2- 172.21.15.140:0/2863236224 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f815005bd90 0x7f81500accd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 --2- 172.21.15.140:0/2863236224 >> v2:172.21.15.152:3300/0 conn(0x7f815005c3f0 0x7f81500ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 --2- 172.21.15.140:0/2863236224 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81500b1e00 0x7f815014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 -- 172.21.15.140:0/2863236224 >> 172.21.15.140:0/2863236224 conn(0x7f81500611c0 msgr2=0x7f8150062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:57.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 -- 172.21.15.140:0/2863236224 shutdown_connections 2022-07-21T01:51:57.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.935+0000 7f8156554700 1 -- 172.21.15.140:0/2863236224 wait complete. 2022-07-21T01:51:57.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f8156554700 1 Processor -- start 2022-07-21T01:51:57.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f8156554700 1 -- start start 2022-07-21T01:51:57.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f8156554700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f815005c3f0 0x7f8150063ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f8156554700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81500b1e00 0x7f81501471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f8156554700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81500a1d30 0x7f81501476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f8156554700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8150156f10 con 0x7f815005c3f0 2022-07-21T01:51:57.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f8156554700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8150157130 con 0x7f81500b1e00 2022-07-21T01:51:57.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f8156554700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81501573d0 con 0x7f81500a1d30 2022-07-21T01:51:57.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f814ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f815005c3f0 0x7f8150063ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f814ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f815005c3f0 0x7f8150063ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35202/0 (socket says 172.21.15.140:35202) 2022-07-21T01:51:57.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f814ffff700 1 -- 172.21.15.140:0/524741476 learned_addr learned my addr 172.21.15.140:0/524741476 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:57.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f814f7fe700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81500b1e00 0x7f81501471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.939+0000 7f8154af1700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81500a1d30 0x7f81501476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814ffff700 1 -- 172.21.15.140:0/524741476 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81500a1d30 msgr2=0x7f81501476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814ffff700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81500a1d30 0x7f81501476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814ffff700 1 -- 172.21.15.140:0/524741476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81500b1e00 msgr2=0x7f81501471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:57.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814ffff700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81500b1e00 0x7f81501471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:57.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814ffff700 1 -- 172.21.15.140:0/524741476 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8148009530 con 0x7f815005c3f0 2022-07-21T01:51:57.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814f7fe700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81500b1e00 0x7f81501471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:51:57.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f8154af1700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81500a1d30 0x7f81501476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:51:57.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814ffff700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f815005c3f0 0x7f8150063ca0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f81440030b0 tx=0x7f8144008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:57.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814d7fa700 1 -- 172.21.15.140:0/524741476 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8144011780 con 0x7f815005c3f0 2022-07-21T01:51:57.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814d7fa700 1 -- 172.21.15.140:0/524741476 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81440118e0 con 0x7f815005c3f0 2022-07-21T01:51:57.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f8156554700 1 -- 172.21.15.140:0/524741476 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815009e800 con 0x7f815005c3f0 2022-07-21T01:51:57.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814d7fa700 1 -- 172.21.15.140:0/524741476 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8144011c40 con 0x7f815005c3f0 2022-07-21T01:51:57.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f8156554700 1 -- 172.21.15.140:0/524741476 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815009e420 con 0x7f815005c3f0 2022-07-21T01:51:57.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f8156554700 1 -- 172.21.15.140:0/524741476 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815009f190 con 0x7f815005c3f0 2022-07-21T01:51:57.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814d7fa700 1 -- 172.21.15.140:0/524741476 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8144012420 con 0x7f815005c3f0 2022-07-21T01:51:57.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.943+0000 7f814d7fa700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8130053a10 0x7f8130055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:57.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.947+0000 7f814d7fa700 1 -- 172.21.15.140:0/524741476 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f814406bcd0 con 0x7f815005c3f0 2022-07-21T01:51:57.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.947+0000 7f814f7fe700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8130053a10 0x7f8130055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:57.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.947+0000 7f814f7fe700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8130053a10 0x7f8130055ed0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f814000b000 tx=0x7f8140008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:57.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:57.951+0000 7f814d7fa700 1 -- 172.21.15.140:0/524741476 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8144042b90 con 0x7f815005c3f0 2022-07-21T01:51:58.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.075+0000 7f8156554700 1 -- 172.21.15.140:0/524741476 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f815005d1e0 con 0x7f815005c3f0 2022-07-21T01:51:58.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f814d7fa700 1 -- 172.21.15.140:0/524741476 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16149 (secure 0 0 0) 0x7f81440461b0 con 0x7f815005c3f0 2022-07-21T01:51:58.077 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:58.077 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":52,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:51:57.503104+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:51:54.695165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-07-21T01:51:55.500555+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:51:58.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 -- 172.21.15.140:0/524741476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8130053a10 msgr2=0x7f8130055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8130053a10 0x7f8130055ed0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f814000b000 tx=0x7f8140008040).stop 2022-07-21T01:51:58.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 -- 172.21.15.140:0/524741476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f815005c3f0 msgr2=0x7f8150063ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f815005c3f0 0x7f8150063ca0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f81440030b0 tx=0x7f8144008040).stop 2022-07-21T01:51:58.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 -- 172.21.15.140:0/524741476 shutdown_connections 2022-07-21T01:51:58.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8130053a10 0x7f8130055ed0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81500a1d30 0x7f81501476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81500b1e00 0x7f81501471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 --2- 172.21.15.140:0/524741476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f815005c3f0 0x7f8150063ca0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 -- 172.21.15.140:0/524741476 >> 172.21.15.140:0/524741476 conn(0x7f815005bd90 msgr2=0x7f8150062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:58.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 -- 172.21.15.140:0/524741476 shutdown_connections 2022-07-21T01:51:58.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.079+0000 7f8156554700 1 -- 172.21.15.140:0/524741476 wait complete. 2022-07-21T01:51:58.091 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:51:58.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.199+0000 7f4bacbea700 1 Processor -- start 2022-07-21T01:51:58.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.199+0000 7f4bacbea700 1 -- start start 2022-07-21T01:51:58.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4ba80aba10 0x7f4ba813f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ba80b1b00 0x7f4ba80b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba80ac530 con 0x7f4ba80b3640 2022-07-21T01:51:58.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba80b3a20 con 0x7f4ba80aba10 2022-07-21T01:51:58.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80ac760 con 0x7f4ba80b1b00 2022-07-21T01:51:58.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4ba659c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4ba80b3640 0x7f4ba813eba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49356/0 (socket says 172.21.15.140:49356) 2022-07-21T01:51:58.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4ba659c700 1 -- 172.21.15.140:0/1093231344 learned_addr learned my addr 172.21.15.140:0/1093231344 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:58.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4b9ffff700 1 --2- 172.21.15.140:0/1093231344 >> v2:172.21.15.140:3300/0 conn(0x7f4ba80aba10 0x7f4ba813f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4ba6d9d700 1 --2- 172.21.15.140:0/1093231344 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ba80b1b00 0x7f4ba80b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4ba5d9b700 1 -- 172.21.15.140:0/1093231344 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315946816 0 0) 0x7f4ba80ac530 con 0x7f4ba80b3640 2022-07-21T01:51:58.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4ba5d9b700 1 -- 172.21.15.140:0/1093231344 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b88003260 con 0x7f4ba80b3640 2022-07-21T01:51:58.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4b9ffff700 1 -- 172.21.15.140:0/1093231344 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ba80b1b00 msgr2=0x7f4ba80b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:58.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4b9ffff700 1 --2- 172.21.15.140:0/1093231344 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ba80b1b00 0x7f4ba80b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4ba5d9b700 1 -- 172.21.15.140:0/1093231344 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1070732554 0 0) 0x7f4b88003260 con 0x7f4ba80b3640 2022-07-21T01:51:58.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4b9ffff700 1 -- 172.21.15.140:0/1093231344 >> v1:172.21.15.140:6789/0 conn(0x7f4ba80b3640 legacy=0x7f4ba813eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4b9ffff700 1 -- 172.21.15.140:0/1093231344 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba813f7a0 con 0x7f4ba80aba10 2022-07-21T01:51:58.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4b9ffff700 1 --2- 172.21.15.140:0/1093231344 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ba80aba10 0x7f4ba813f2b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f4b98001800 tx=0x7f4b90006df0).ready entity=mon.0 client_cookie=1481858034c1af55 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:58.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4ba5d9b700 1 -- 172.21.15.140:0/1093231344 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b98001660 con 0x7f4ba80aba10 2022-07-21T01:51:58.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4ba5d9b700 1 -- 172.21.15.140:0/1093231344 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b90000e90 con 0x7f4ba80aba10 2022-07-21T01:51:58.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4ba5d9b700 1 -- 172.21.15.140:0/1093231344 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b90012460 con 0x7f4ba80aba10 2022-07-21T01:51:58.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 -- 172.21.15.140:0/1093231344 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ba80aba10 msgr2=0x7f4ba813f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 --2- 172.21.15.140:0/1093231344 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ba80aba10 0x7f4ba813f2b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f4b98001800 tx=0x7f4b90006df0).stop 2022-07-21T01:51:58.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 -- 172.21.15.140:0/1093231344 shutdown_connections 2022-07-21T01:51:58.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 --2- 172.21.15.140:0/1093231344 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ba80b1b00 0x7f4ba80b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 --2- 172.21.15.140:0/1093231344 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ba80aba10 0x7f4ba813f2b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 -- 172.21.15.140:0/1093231344 >> 172.21.15.140:0/1093231344 conn(0x7f4ba80b6c20 msgr2=0x7f4ba80b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:58.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 -- 172.21.15.140:0/1093231344 shutdown_connections 2022-07-21T01:51:58.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.203+0000 7f4bacbea700 1 -- 172.21.15.140:0/1093231344 wait complete. 2022-07-21T01:51:58.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4bacbea700 1 Processor -- start 2022-07-21T01:51:58.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4bacbea700 1 -- start start 2022-07-21T01:51:58.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4bacbea700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ba80aba10 0x7f4ba80af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4bacbea700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ba80b1b00 0x7f4ba8141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4bacbea700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ba80b3640 0x7f4ba80991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4bacbea700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80af140 con 0x7f4ba80aba10 2022-07-21T01:51:58.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4bacbea700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80aeea0 con 0x7f4ba80b3640 2022-07-21T01:51:58.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4bacbea700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba809bb60 con 0x7f4ba80b1b00 2022-07-21T01:51:58.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4ba659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ba80aba10 0x7f4ba80af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4ba659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ba80aba10 0x7f4ba80af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35216/0 (socket says 172.21.15.140:35216) 2022-07-21T01:51:58.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4ba659c700 1 -- 172.21.15.140:0/201777605 learned_addr learned my addr 172.21.15.140:0/201777605 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:58.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4b9ffff700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ba80b1b00 0x7f4ba8141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4ba6d9d700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ba80b3640 0x7f4ba80991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4ba659c700 1 -- 172.21.15.140:0/201777605 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ba80b1b00 msgr2=0x7f4ba8141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4ba659c700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ba80b1b00 0x7f4ba8141b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4ba659c700 1 -- 172.21.15.140:0/201777605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ba80b3640 msgr2=0x7f4ba80991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4ba659c700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ba80b3640 0x7f4ba80991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4ba659c700 1 -- 172.21.15.140:0/201777605 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9000b180 con 0x7f4ba80aba10 2022-07-21T01:51:58.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.207+0000 7f4ba659c700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ba80aba10 0x7f4ba80af3d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f4b980018c0 tx=0x7f4b98001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:58.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.211+0000 7f4b9f7fe700 1 -- 172.21.15.140:0/201777605 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b98007170 con 0x7f4ba80aba10 2022-07-21T01:51:58.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.211+0000 7f4b9f7fe700 1 -- 172.21.15.140:0/201777605 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b980072d0 con 0x7f4ba80aba10 2022-07-21T01:51:58.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.211+0000 7f4bacbea700 1 -- 172.21.15.140:0/201777605 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba809be00 con 0x7f4ba80aba10 2022-07-21T01:51:58.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.211+0000 7f4b9f7fe700 1 -- 172.21.15.140:0/201777605 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b98004920 con 0x7f4ba80aba10 2022-07-21T01:51:58.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.211+0000 7f4bacbea700 1 -- 172.21.15.140:0/201777605 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba80aeac0 con 0x7f4ba80aba10 2022-07-21T01:51:58.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.211+0000 7f4bacbea700 1 -- 172.21.15.140:0/201777605 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba80aca00 con 0x7f4ba80aba10 2022-07-21T01:51:58.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.211+0000 7f4b9f7fe700 1 -- 172.21.15.140:0/201777605 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4b98004a80 con 0x7f4ba80aba10 2022-07-21T01:51:58.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.211+0000 7f4b9f7fe700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b8c053b00 0x7f4b8c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.211+0000 7f4b9f7fe700 1 -- 172.21.15.140:0/201777605 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4b9806b980 con 0x7f4ba80aba10 2022-07-21T01:51:58.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.211+0000 7f4b9ffff700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b8c053b00 0x7f4b8c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.215+0000 7f4b9ffff700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b8c053b00 0x7f4b8c055fc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f4b90018000 tx=0x7f4b90014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:58.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.219+0000 7f4b9f7fe700 1 -- 172.21.15.140:0/201777605 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4b980428a0 con 0x7f4ba80aba10 2022-07-21T01:51:58.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.379+0000 7f4bacbea700 1 -- 172.21.15.140:0/201777605 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4ba80574e0 con 0x7f4ba80aba10 2022-07-21T01:51:58.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.387+0000 7f4b9f7fe700 1 -- 172.21.15.140:0/201777605 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v21) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4b98045ec0 con 0x7f4ba80aba10 2022-07-21T01:51:58.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 -- 172.21.15.140:0/201777605 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b8c053b00 msgr2=0x7f4b8c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b8c053b00 0x7f4b8c055fc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f4b90018000 tx=0x7f4b90014000).stop 2022-07-21T01:51:58.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 -- 172.21.15.140:0/201777605 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ba80aba10 msgr2=0x7f4ba80af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ba80aba10 0x7f4ba80af3d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f4b980018c0 tx=0x7f4b98001e50).stop 2022-07-21T01:51:58.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 -- 172.21.15.140:0/201777605 shutdown_connections 2022-07-21T01:51:58.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b8c053b00 0x7f4b8c055fc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ba80b3640 0x7f4ba80991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ba80b1b00 0x7f4ba8141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 --2- 172.21.15.140:0/201777605 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ba80aba10 0x7f4ba80af3d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 -- 172.21.15.140:0/201777605 >> 172.21.15.140:0/201777605 conn(0x7f4ba80b6c20 msgr2=0x7f4ba8140430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:58.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 -- 172.21.15.140:0/201777605 shutdown_connections 2022-07-21T01:51:58.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.391+0000 7f4bacbea700 1 -- 172.21.15.140:0/201777605 wait complete. 2022-07-21T01:51:58.395 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:51:58.403 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:51:58.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.519+0000 7f5885d81700 1 Processor -- start 2022-07-21T01:51:58.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.519+0000 7f5885d81700 1 -- start start 2022-07-21T01:51:58.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.519+0000 7f5885d81700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f58800b3700 0x7f5880098a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.519+0000 7f5885d81700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58800abd10 0x7f5880096290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.519+0000 7f5885d81700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58800b3ae0 con 0x7f5880062230 2022-07-21T01:51:58.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.519+0000 7f5885d81700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5880098070 con 0x7f58800b3700 2022-07-21T01:51:58.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.519+0000 7f5885d81700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5880098310 con 0x7f58800abd10 2022-07-21T01:51:58.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.519+0000 7f587ffff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f5880062230 0x7f5880095b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49372/0 (socket says 172.21.15.140:49372) 2022-07-21T01:51:58.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.519+0000 7f587ffff700 1 -- 172.21.15.140:0/2783493569 learned_addr learned my addr 172.21.15.140:0/2783493569 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:58.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.519+0000 7f5885580700 1 --2- 172.21.15.140:0/2783493569 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58800abd10 0x7f5880096290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.519+0000 7f5884d7f700 1 --2- 172.21.15.140:0/2783493569 >> v2:172.21.15.152:3300/0 conn(0x7f58800b3700 0x7f5880098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f587f7fe700 1 -- 172.21.15.140:0/2783493569 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895618540 0 0) 0x7f58800b3ae0 con 0x7f5880062230 2022-07-21T01:51:58.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5884d7f700 1 -- 172.21.15.140:0/2783493569 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58800abd10 msgr2=0x7f5880096290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:58.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5884d7f700 1 --2- 172.21.15.140:0/2783493569 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58800abd10 0x7f5880096290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5884d7f700 1 -- 172.21.15.140:0/2783493569 >> v1:172.21.15.140:6789/0 conn(0x7f5880062230 legacy=0x7f5880095b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5884d7f700 1 -- 172.21.15.140:0/2783493569 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58800985b0 con 0x7f58800b3700 2022-07-21T01:51:58.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5884d7f700 1 --2- 172.21.15.140:0/2783493569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58800b3700 0x7f5880098a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f5874002f70 tx=0x7f5874006df0).ready entity=mon.1 client_cookie=71496ea452d9e580 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:58.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f587f7fe700 1 -- 172.21.15.140:0/2783493569 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587400d570 con 0x7f58800b3700 2022-07-21T01:51:58.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f587f7fe700 1 -- 172.21.15.140:0/2783493569 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5874000e90 con 0x7f58800b3700 2022-07-21T01:51:58.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f587f7fe700 1 -- 172.21.15.140:0/2783493569 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587400d8c0 con 0x7f58800b3700 2022-07-21T01:51:58.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5885d81700 1 -- 172.21.15.140:0/2783493569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58800b3700 msgr2=0x7f5880098a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5885d81700 1 --2- 172.21.15.140:0/2783493569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58800b3700 0x7f5880098a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f5874002f70 tx=0x7f5874006df0).stop 2022-07-21T01:51:58.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5885d81700 1 -- 172.21.15.140:0/2783493569 shutdown_connections 2022-07-21T01:51:58.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5885d81700 1 --2- 172.21.15.140:0/2783493569 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58800abd10 0x7f5880096290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5885d81700 1 --2- 172.21.15.140:0/2783493569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58800b3700 0x7f5880098a50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5885d81700 1 -- 172.21.15.140:0/2783493569 >> 172.21.15.140:0/2783493569 conn(0x7f58800b6ce0 msgr2=0x7f58800b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:58.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5885d81700 1 -- 172.21.15.140:0/2783493569 shutdown_connections 2022-07-21T01:51:58.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5885d81700 1 -- 172.21.15.140:0/2783493569 wait complete. 2022-07-21T01:51:58.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5885d81700 1 Processor -- start 2022-07-21T01:51:58.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.523+0000 7f5885d81700 1 -- start start 2022-07-21T01:51:58.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885d81700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5880062230 0x7f5880093a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885d81700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58800abd10 0x7f5880061250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885d81700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58800b3700 0x7f5880140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885d81700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588009c390 con 0x7f58800b3700 2022-07-21T01:51:58.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885d81700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58800955f0 con 0x7f5880062230 2022-07-21T01:51:58.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885d81700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588009a6e0 con 0x7f58800abd10 2022-07-21T01:51:58.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885580700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58800b3700 0x7f5880140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885580700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58800b3700 0x7f5880140080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35234/0 (socket says 172.21.15.140:35234) 2022-07-21T01:51:58.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885580700 1 -- 172.21.15.140:0/1643323175 learned_addr learned my addr 172.21.15.140:0/1643323175 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:58.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f587ffff700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58800abd10 0x7f5880061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5884d7f700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5880062230 0x7f5880093a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885580700 1 -- 172.21.15.140:0/1643323175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58800abd10 msgr2=0x7f5880061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885580700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58800abd10 0x7f5880061250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885580700 1 -- 172.21.15.140:0/1643323175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5880062230 msgr2=0x7f5880093a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885580700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5880062230 0x7f5880093a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885580700 1 -- 172.21.15.140:0/1643323175 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587400b180 con 0x7f58800b3700 2022-07-21T01:51:58.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885580700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58800b3700 0x7f5880140080 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f5878006e00 tx=0x7f58780091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:58.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f587dffb700 1 -- 172.21.15.140:0/1643323175 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587800c640 con 0x7f58800b3700 2022-07-21T01:51:58.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885d81700 1 -- 172.21.15.140:0/1643323175 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588009a440 con 0x7f58800b3700 2022-07-21T01:51:58.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885d81700 1 -- 172.21.15.140:0/1643323175 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5880093f90 con 0x7f58800b3700 2022-07-21T01:51:58.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f587dffb700 1 -- 172.21.15.140:0/1643323175 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587800c7a0 con 0x7f58800b3700 2022-07-21T01:51:58.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f587dffb700 1 -- 172.21.15.140:0/1643323175 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f587800cb00 con 0x7f58800b3700 2022-07-21T01:51:58.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f587dffb700 1 -- 172.21.15.140:0/1643323175 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5878022070 con 0x7f58800b3700 2022-07-21T01:51:58.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.527+0000 7f5885d81700 1 -- 172.21.15.140:0/1643323175 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5880061830 con 0x7f58800b3700 2022-07-21T01:51:58.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.531+0000 7f587dffb700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5860053aa0 0x7f5860055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.531+0000 7f587dffb700 1 -- 172.21.15.140:0/1643323175 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f587806c550 con 0x7f58800b3700 2022-07-21T01:51:58.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.531+0000 7f5884d7f700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5860053aa0 0x7f5860055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.531+0000 7f5884d7f700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5860053aa0 0x7f5860055f60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f587401a000 tx=0x7f5874014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:58.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.535+0000 7f587dffb700 1 -- 172.21.15.140:0/1643323175 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58780433d0 con 0x7f58800b3700 2022-07-21T01:51:58.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.703+0000 7f5885d81700 1 -- 172.21.15.140:0/1643323175 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f58800574e0 con 0x7f58800b3700 2022-07-21T01:51:58.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.711+0000 7f587dffb700 1 -- 172.21.15.140:0/1643323175 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7f58780160e0 con 0x7f58800b3700 2022-07-21T01:51:58.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 -- 172.21.15.140:0/1643323175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5860053aa0 msgr2=0x7f5860055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5860053aa0 0x7f5860055f60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f587401a000 tx=0x7f5874014000).stop 2022-07-21T01:51:58.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 -- 172.21.15.140:0/1643323175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58800b3700 msgr2=0x7f5880140080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58800b3700 0x7f5880140080 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f5878006e00 tx=0x7f58780091d0).stop 2022-07-21T01:51:58.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 -- 172.21.15.140:0/1643323175 shutdown_connections 2022-07-21T01:51:58.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5860053aa0 0x7f5860055f60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58800b3700 0x7f5880140080 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58800abd10 0x7f5880061250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 --2- 172.21.15.140:0/1643323175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5880062230 0x7f5880093a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 -- 172.21.15.140:0/1643323175 >> 172.21.15.140:0/1643323175 conn(0x7f58800b6ce0 msgr2=0x7f58800b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:58.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 -- 172.21.15.140:0/1643323175 shutdown_connections 2022-07-21T01:51:58.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.715+0000 7f5885d81700 1 -- 172.21.15.140:0/1643323175 wait complete. 2022-07-21T01:51:58.720 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:51:58.728 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:51:58.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf8a893700 1 Processor -- start 2022-07-21T01:51:58.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf8a893700 1 -- start start 2022-07-21T01:51:58.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf8a893700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcf8405cf40 0x7fcf84144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf8a893700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf840b1c30 0x7fcf84061da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf8a893700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf840649d0 con 0x7fcf84066cb0 2022-07-21T01:51:58.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf8a893700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf841442c0 con 0x7fcf8405cf40 2022-07-21T01:51:58.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf8a893700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84144560 con 0x7fcf840b1c30 2022-07-21T01:51:58.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf88e30700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fcf84066cb0 0x7fcf84062290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49388/0 (socket says 172.21.15.140:49388) 2022-07-21T01:51:58.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf88e30700 1 -- 172.21.15.140:0/3899380716 learned_addr learned my addr 172.21.15.140:0/3899380716 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:51:58.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf837fe700 1 --2- 172.21.15.140:0/3899380716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf840b1c30 0x7fcf84061da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf83fff700 1 --2- 172.21.15.140:0/3899380716 >> v2:172.21.15.152:3300/0 conn(0x7fcf8405cf40 0x7fcf84144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf82ffd700 1 -- 172.21.15.140:0/3899380716 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804512765 0 0) 0x7fcf840649d0 con 0x7fcf84066cb0 2022-07-21T01:51:58.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf82ffd700 1 -- 172.21.15.140:0/3899380716 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf70003260 con 0x7fcf84066cb0 2022-07-21T01:51:58.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf83fff700 1 -- 172.21.15.140:0/3899380716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf840b1c30 msgr2=0x7fcf84061da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:58.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf83fff700 1 --2- 172.21.15.140:0/3899380716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf840b1c30 0x7fcf84061da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf83fff700 1 -- 172.21.15.140:0/3899380716 >> v1:172.21.15.140:6789/0 conn(0x7fcf84066cb0 legacy=0x7fcf84062290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf82ffd700 1 -- 172.21.15.140:0/3899380716 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 556518578 0 0) 0x7fcf70003260 con 0x7fcf84066cb0 2022-07-21T01:51:58.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf83fff700 1 -- 172.21.15.140:0/3899380716 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf84144800 con 0x7fcf8405cf40 2022-07-21T01:51:58.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf83fff700 1 --2- 172.21.15.140:0/3899380716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf8405cf40 0x7fcf84144ca0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fcf7c001b80 tx=0x7fcf74006df0).ready entity=mon.1 client_cookie=f216da5fac5dc890 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:58.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.843+0000 7fcf82ffd700 1 -- 172.21.15.140:0/3899380716 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf7c0019e0 con 0x7fcf8405cf40 2022-07-21T01:51:58.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf82ffd700 1 -- 172.21.15.140:0/3899380716 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf74000e90 con 0x7fcf8405cf40 2022-07-21T01:51:58.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf82ffd700 1 -- 172.21.15.140:0/3899380716 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf74012460 con 0x7fcf8405cf40 2022-07-21T01:51:58.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf8a893700 1 -- 172.21.15.140:0/3899380716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf8405cf40 msgr2=0x7fcf84144ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf8a893700 1 --2- 172.21.15.140:0/3899380716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf8405cf40 0x7fcf84144ca0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fcf7c001b80 tx=0x7fcf74006df0).stop 2022-07-21T01:51:58.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf8a893700 1 -- 172.21.15.140:0/3899380716 shutdown_connections 2022-07-21T01:51:58.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf8a893700 1 --2- 172.21.15.140:0/3899380716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf840b1c30 0x7fcf84061da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf8a893700 1 --2- 172.21.15.140:0/3899380716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf8405cf40 0x7fcf84144ca0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf8a893700 1 -- 172.21.15.140:0/3899380716 >> 172.21.15.140:0/3899380716 conn(0x7fcf840b7d60 msgr2=0x7fcf840b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:58.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf8a893700 1 -- 172.21.15.140:0/3899380716 shutdown_connections 2022-07-21T01:51:58.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf8a893700 1 -- 172.21.15.140:0/3899380716 wait complete. 2022-07-21T01:51:58.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf8a893700 1 Processor -- start 2022-07-21T01:51:58.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.847+0000 7fcf8a893700 1 -- start start 2022-07-21T01:51:58.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf8a893700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf8405cf40 0x7fcf84153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf8a893700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf84066cb0 0x7fcf8409ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf8a893700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf840b1c30 0x7fcf840acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf8a893700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84154270 con 0x7fcf84066cb0 2022-07-21T01:51:58.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf8a893700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8413f810 con 0x7fcf840b1c30 2022-07-21T01:51:58.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf8a893700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84150180 con 0x7fcf8405cf40 2022-07-21T01:51:58.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf84066cb0 0x7fcf8409ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf84066cb0 0x7fcf8409ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35246/0 (socket says 172.21.15.140:35246) 2022-07-21T01:51:58.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf837fe700 1 -- 172.21.15.140:0/4262845459 learned_addr learned my addr 172.21.15.140:0/4262845459 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:58.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf88e30700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf840b1c30 0x7fcf840acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf83fff700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf8405cf40 0x7fcf84153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf837fe700 1 -- 172.21.15.140:0/4262845459 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf8405cf40 msgr2=0x7fcf84153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf837fe700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf8405cf40 0x7fcf84153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf837fe700 1 -- 172.21.15.140:0/4262845459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf840b1c30 msgr2=0x7fcf840acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:58.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf837fe700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf840b1c30 0x7fcf840acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:58.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf837fe700 1 -- 172.21.15.140:0/4262845459 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7400b180 con 0x7fcf84066cb0 2022-07-21T01:51:58.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf837fe700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf84066cb0 0x7fcf8409ba70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fcf7800c000 tx=0x7fcf78006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:58.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf817fa700 1 -- 172.21.15.140:0/4262845459 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf78011750 con 0x7fcf84066cb0 2022-07-21T01:51:58.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf817fa700 1 -- 172.21.15.140:0/4262845459 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf780118b0 con 0x7fcf84066cb0 2022-07-21T01:51:58.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf8a893700 1 -- 172.21.15.140:0/4262845459 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8414fe20 con 0x7fcf84066cb0 2022-07-21T01:51:58.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf817fa700 1 -- 172.21.15.140:0/4262845459 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf78011c10 con 0x7fcf84066cb0 2022-07-21T01:51:58.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf8a893700 1 -- 172.21.15.140:0/4262845459 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf840ad100 con 0x7fcf84066cb0 2022-07-21T01:51:58.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.851+0000 7fcf8a893700 1 -- 172.21.15.140:0/4262845459 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf841505d0 con 0x7fcf84066cb0 2022-07-21T01:51:58.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.855+0000 7fcf817fa700 1 -- 172.21.15.140:0/4262845459 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcf78011d70 con 0x7fcf84066cb0 2022-07-21T01:51:58.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.855+0000 7fcf817fa700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf64053a70 0x7fcf64055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:58.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.855+0000 7fcf817fa700 1 -- 172.21.15.140:0/4262845459 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcf7806be50 con 0x7fcf84066cb0 2022-07-21T01:51:58.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.855+0000 7fcf83fff700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf64053a70 0x7fcf64055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:58.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.855+0000 7fcf83fff700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf64053a70 0x7fcf64055f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fcf7401a000 tx=0x7fcf74014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:58.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:58.863+0000 7fcf817fa700 1 -- 172.21.15.140:0/4262845459 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcf78042d10 con 0x7fcf84066cb0 2022-07-21T01:51:59.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.027+0000 7fcf8a893700 1 -- 172.21.15.140:0/4262845459 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fcf840574e0 con 0x7fcf84066cb0 2022-07-21T01:51:59.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.031+0000 7fcf817fa700 1 -- 172.21.15.140:0/4262845459 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v23) v1 ==== 184+0+0 (secure 0 0 0) 0x7fcf78046330 con 0x7fcf84066cb0 2022-07-21T01:51:59.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 -- 172.21.15.140:0/4262845459 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf64053a70 msgr2=0x7fcf64055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf64053a70 0x7fcf64055f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fcf7401a000 tx=0x7fcf74014000).stop 2022-07-21T01:51:59.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 -- 172.21.15.140:0/4262845459 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf84066cb0 msgr2=0x7fcf8409ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf84066cb0 0x7fcf8409ba70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fcf7800c000 tx=0x7fcf78006d90).stop 2022-07-21T01:51:59.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 -- 172.21.15.140:0/4262845459 shutdown_connections 2022-07-21T01:51:59.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf64053a70 0x7fcf64055f30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf840b1c30 0x7fcf840acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf84066cb0 0x7fcf8409ba70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 --2- 172.21.15.140:0/4262845459 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf8405cf40 0x7fcf84153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 -- 172.21.15.140:0/4262845459 >> 172.21.15.140:0/4262845459 conn(0x7fcf840b7d60 msgr2=0x7fcf84146260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:59.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 -- 172.21.15.140:0/4262845459 shutdown_connections 2022-07-21T01:51:59.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.035+0000 7fcf8a893700 1 -- 172.21.15.140:0/4262845459 wait complete. 2022-07-21T01:51:59.037 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:51:59.047 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:51:59.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe05eae700 1 Processor -- start 2022-07-21T01:51:59.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe05eae700 1 -- start start 2022-07-21T01:51:59.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe05eae700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbe000b5650 0x7fbe000b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe05eae700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbe000af8e0 0x7fbe0005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe05eae700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe00147010 con 0x7fbe000adff0 2022-07-21T01:51:59.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe05eae700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe001472b0 con 0x7fbe000af8e0 2022-07-21T01:51:59.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe05eae700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe00146d40 con 0x7fbe000b5650 2022-07-21T01:51:59.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe056ad700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbe000af8e0 0x7fbe0005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe056ad700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbe000af8e0 0x7fbe0005bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35256/0 (socket says 172.21.15.140:35256) 2022-07-21T01:51:59.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe056ad700 1 -- 172.21.15.140:0/544148714 learned_addr learned my addr 172.21.15.140:0/544148714 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:59.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe04eac700 1 --2- 172.21.15.140:0/544148714 >> v2:172.21.15.152:3300/0 conn(0x7fbe000b5650 0x7fbe000b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe056ad700 1 -- 172.21.15.140:0/544148714 >> v2:172.21.15.152:3300/0 conn(0x7fbe000b5650 msgr2=0x7fbe000b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:59.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe056ad700 1 --2- 172.21.15.140:0/544148714 >> v2:172.21.15.152:3300/0 conn(0x7fbe000b5650 0x7fbe000b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe056ad700 1 -- 172.21.15.140:0/544148714 >> v1:172.21.15.152:6789/0 conn(0x7fbe000adff0 legacy=0x7fbe00147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe056ad700 1 -- 172.21.15.140:0/544148714 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe000acae0 con 0x7fbe000af8e0 2022-07-21T01:51:59.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbe056ad700 1 --2- 172.21.15.140:0/544148714 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe000af8e0 0x7fbe0005bd90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbdf80032f0 tx=0x7fbdf8009290).ready entity=mon.0 client_cookie=d29acbc7970966af server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:59.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbdff7fe700 1 -- 172.21.15.140:0/544148714 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdf800b870 con 0x7fbe000af8e0 2022-07-21T01:51:59.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbdff7fe700 1 -- 172.21.15.140:0/544148714 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf800b9d0 con 0x7fbe000af8e0 2022-07-21T01:51:59.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.155+0000 7fbdff7fe700 1 -- 172.21.15.140:0/544148714 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdf800bb40 con 0x7fbe000af8e0 2022-07-21T01:51:59.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 -- 172.21.15.140:0/544148714 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe000af8e0 msgr2=0x7fbe0005bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 --2- 172.21.15.140:0/544148714 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe000af8e0 0x7fbe0005bd90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbdf80032f0 tx=0x7fbdf8009290).stop 2022-07-21T01:51:59.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 -- 172.21.15.140:0/544148714 shutdown_connections 2022-07-21T01:51:59.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 --2- 172.21.15.140:0/544148714 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe000af8e0 0x7fbe0005bd90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 --2- 172.21.15.140:0/544148714 >> v2:172.21.15.152:3300/0 conn(0x7fbe000b5650 0x7fbe000b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 -- 172.21.15.140:0/544148714 >> 172.21.15.140:0/544148714 conn(0x7fbe000634e0 msgr2=0x7fbe000b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:59.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 -- 172.21.15.140:0/544148714 shutdown_connections 2022-07-21T01:51:59.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 -- 172.21.15.140:0/544148714 wait complete. 2022-07-21T01:51:59.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 Processor -- start 2022-07-21T01:51:59.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 -- start start 2022-07-21T01:51:59.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe000adff0 0x7fbe00159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe000af8e0 0x7fbe0015a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe000b5650 0x7fbe000659f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00144f00 con 0x7fbe000adff0 2022-07-21T01:51:59.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00067210 con 0x7fbe000af8e0 2022-07-21T01:51:59.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe05eae700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0011be70 con 0x7fbe000b5650 2022-07-21T01:51:59.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe04eac700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe000adff0 0x7fbe00159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe04eac700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe000adff0 0x7fbe00159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35258/0 (socket says 172.21.15.140:35258) 2022-07-21T01:51:59.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe04eac700 1 -- 172.21.15.140:0/3309516688 learned_addr learned my addr 172.21.15.140:0/3309516688 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:59.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe056ad700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe000b5650 0x7fbe000659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbdfffff700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe000af8e0 0x7fbe0015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe04eac700 1 -- 172.21.15.140:0/3309516688 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe000b5650 msgr2=0x7fbe000659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe04eac700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe000b5650 0x7fbe000659f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe04eac700 1 -- 172.21.15.140:0/3309516688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe000af8e0 msgr2=0x7fbe0015a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe04eac700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe000af8e0 0x7fbe0015a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.159+0000 7fbe04eac700 1 -- 172.21.15.140:0/3309516688 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf8009530 con 0x7fbe000adff0 2022-07-21T01:51:59.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbdfffff700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe000af8e0 0x7fbe0015a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:51:59.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbe04eac700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe000adff0 0x7fbe00159c50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fbdf00030b0 tx=0x7fbdf0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:59.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbdfdffb700 1 -- 172.21.15.140:0/3309516688 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdf0011780 con 0x7fbe000adff0 2022-07-21T01:51:59.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbdfdffb700 1 -- 172.21.15.140:0/3309516688 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf00118e0 con 0x7fbe000adff0 2022-07-21T01:51:59.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbe05eae700 1 -- 172.21.15.140:0/3309516688 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe000676f0 con 0x7fbe000adff0 2022-07-21T01:51:59.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbdfdffb700 1 -- 172.21.15.140:0/3309516688 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdf0011c40 con 0x7fbe000adff0 2022-07-21T01:51:59.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbe05eae700 1 -- 172.21.15.140:0/3309516688 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0011c530 con 0x7fbe000adff0 2022-07-21T01:51:59.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbdfdffb700 1 -- 172.21.15.140:0/3309516688 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbdf0012420 con 0x7fbe000adff0 2022-07-21T01:51:59.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbe05eae700 1 -- 172.21.15.140:0/3309516688 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe00067410 con 0x7fbe000adff0 2022-07-21T01:51:59.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbdfdffb700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbdd8053a70 0x7fbdd8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbdfdffb700 1 -- 172.21.15.140:0/3309516688 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbdf006bd40 con 0x7fbe000adff0 2022-07-21T01:51:59.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbdfffff700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbdd8053a70 0x7fbdd8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.163+0000 7fbdfffff700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbdd8053a70 0x7fbdd8055f30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fbde8008000 tx=0x7fbde8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:59.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.171+0000 7fbdfdffb700 1 -- 172.21.15.140:0/3309516688 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbdf0046630 con 0x7fbe000adff0 2022-07-21T01:51:59.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.331+0000 7fbe05eae700 1 -- 172.21.15.140:0/3309516688 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe0011c030 con 0x7fbe000adff0 2022-07-21T01:51:59.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbdfdffb700 1 -- 172.21.15.140:0/3309516688 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2145 (secure 0 0 0) 0x7fbdf0046220 con 0x7fbe000adff0 2022-07-21T01:51:59.337 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:59.337 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1915503334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1915503334},{"type":"v1","addr":"172.21.15.140:6835","nonce":1915503334}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:51:56.500368+0000","modified":"2022-07-21T01:51:57.504176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5204":{"gid":5204,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3908541795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3908541795},{"type":"v1","addr":"172.21.15.152:6833","nonce":3908541795}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-07-21T01:51:59.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 -- 172.21.15.140:0/3309516688 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbdd8053a70 msgr2=0x7fbdd8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbdd8053a70 0x7fbdd8055f30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fbde8008000 tx=0x7fbde8009040).stop 2022-07-21T01:51:59.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 -- 172.21.15.140:0/3309516688 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe000adff0 msgr2=0x7fbe00159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe000adff0 0x7fbe00159c50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fbdf00030b0 tx=0x7fbdf0008040).stop 2022-07-21T01:51:59.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 -- 172.21.15.140:0/3309516688 shutdown_connections 2022-07-21T01:51:59.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbdd8053a70 0x7fbdd8055f30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe000b5650 0x7fbe000659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe000af8e0 0x7fbe0015a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 --2- 172.21.15.140:0/3309516688 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbe000adff0 0x7fbe00159c50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 -- 172.21.15.140:0/3309516688 >> 172.21.15.140:0/3309516688 conn(0x7fbe000634e0 msgr2=0x7fbe000b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:59.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 -- 172.21.15.140:0/3309516688 shutdown_connections 2022-07-21T01:51:59.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.335+0000 7fbe05eae700 1 -- 172.21.15.140:0/3309516688 wait complete. 2022-07-21T01:51:59.342 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 21 2022-07-21T01:51:59.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:51:56.500368+0000', 'modified': '2022-07-21T01:51:57.504176+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5204}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5204': {'gid': 5204, 'name': 'b', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/3908541795', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 3908541795}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 3908541795}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:51:59.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:51:59.550 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:51:59.664 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":5204,"want_state":"up:active","state":"up:active","rank_uptime":3.1553286009999999,"mdsmap_epoch":21,"osdmap_epoch":53,"osdmap_epoch_barrier":52,"uptime":7.8143718450000001} 2022-07-21T01:51:59.665 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 5204, "mdsmap_epoch": 21, "osdmap_epoch": 53, "osdmap_epoch_barrier": 52, "rank_uptime": 3.155328601, "state": "up:active", "uptime": 7.814371845, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:51:59.666 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:51:59.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 Processor -- start 2022-07-21T01:51:59.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 -- start start 2022-07-21T01:51:59.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6b181424e0 0x7f6b180b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6b180aea10 0x7f6b180ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b181433d0 con 0x7f6b180b0570 2022-07-21T01:51:59.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b18143b10 con 0x7f6b181424e0 2022-07-21T01:51:59.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18143670 con 0x7f6b180aea10 2022-07-21T01:51:59.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b0effd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6b181424e0 0x7f6b180b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b17fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6b180aea10 0x7f6b180ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b0effd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6b181424e0 0x7f6b180b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:41008/0 (socket says 172.21.15.140:41008) 2022-07-21T01:51:59.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b0effd700 1 -- 172.21.15.140:0/2175366135 learned_addr learned my addr 172.21.15.140:0/2175366135 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:59.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b177fe700 1 --1- 172.21.15.140:0/2175366135 >> v1:172.21.15.152:6789/0 conn(0x7f6b180b0570 0x7f6b18141dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:50526/0 (socket says 172.21.15.140:50526) 2022-07-21T01:51:59.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b0effd700 1 -- 172.21.15.140:0/2175366135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6b180aea10 msgr2=0x7f6b180ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:51:59.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b0effd700 1 --2- 172.21.15.140:0/2175366135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6b180aea10 0x7f6b180ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b0effd700 1 -- 172.21.15.140:0/2175366135 >> v1:172.21.15.152:6789/0 conn(0x7f6b180b0570 legacy=0x7f6b18141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b0effd700 1 -- 172.21.15.140:0/2175366135 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b180b1010 con 0x7f6b181424e0 2022-07-21T01:51:59.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b0effd700 1 --2- 172.21.15.140:0/2175366135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b181424e0 0x7f6b180b0950 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f6b00002f70 tx=0x7f6b00006eb0).ready entity=mon.1 client_cookie=c837c021e7724db1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:59.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b16ffd700 1 -- 172.21.15.140:0/2175366135 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b0000d570 con 0x7f6b181424e0 2022-07-21T01:51:59.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b16ffd700 1 -- 172.21.15.140:0/2175366135 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b00000e90 con 0x7f6b181424e0 2022-07-21T01:51:59.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b16ffd700 1 -- 172.21.15.140:0/2175366135 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b0000d900 con 0x7f6b181424e0 2022-07-21T01:51:59.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/2175366135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b181424e0 msgr2=0x7f6b180b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 --2- 172.21.15.140:0/2175366135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b181424e0 0x7f6b180b0950 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f6b00002f70 tx=0x7f6b00006eb0).stop 2022-07-21T01:51:59.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/2175366135 shutdown_connections 2022-07-21T01:51:59.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 --2- 172.21.15.140:0/2175366135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6b180aea10 0x7f6b180ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 --2- 172.21.15.140:0/2175366135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b181424e0 0x7f6b180b0950 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/2175366135 >> 172.21.15.140:0/2175366135 conn(0x7f6b180b9360 msgr2=0x7f6b180b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:59.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/2175366135 shutdown_connections 2022-07-21T01:51:59.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.779+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/2175366135 wait complete. 2022-07-21T01:51:59.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b1cdcf700 1 Processor -- start 2022-07-21T01:51:59.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b1cdcf700 1 -- start start 2022-07-21T01:51:59.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b1cdcf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b180aea10 0x7f6b180b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b1cdcf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6b180b0570 0x7f6b18155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b1cdcf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6b181424e0 0x7f6b18112550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b1cdcf700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18155360 con 0x7f6b181424e0 2022-07-21T01:51:59.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b1cdcf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18155590 con 0x7f6b180aea10 2022-07-21T01:51:59.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b1cdcf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18155830 con 0x7f6b180b0570 2022-07-21T01:51:59.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b17fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6b181424e0 0x7f6b18112550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b17fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6b181424e0 0x7f6b18112550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35276/0 (socket says 172.21.15.140:35276) 2022-07-21T01:51:59.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b0effd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6b180b0570 0x7f6b18155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b17fff700 1 -- 172.21.15.140:0/1611395801 learned_addr learned my addr 172.21.15.140:0/1611395801 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:51:59.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b177fe700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b180aea10 0x7f6b180b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b17fff700 1 -- 172.21.15.140:0/1611395801 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6b180b0570 msgr2=0x7f6b18155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b17fff700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6b180b0570 0x7f6b18155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b17fff700 1 -- 172.21.15.140:0/1611395801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b180aea10 msgr2=0x7f6b180b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b17fff700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b180aea10 0x7f6b180b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b17fff700 1 -- 172.21.15.140:0/1611395801 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0000b1e0 con 0x7f6b181424e0 2022-07-21T01:51:59.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b17fff700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6b181424e0 0x7f6b18112550 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f6b1000c000 tx=0x7f6b100091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:59.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b157fa700 1 -- 172.21.15.140:0/1611395801 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b10012600 con 0x7f6b181424e0 2022-07-21T01:51:59.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b157fa700 1 -- 172.21.15.140:0/1611395801 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b10012760 con 0x7f6b181424e0 2022-07-21T01:51:59.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/1611395801 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b181561c0 con 0x7f6b181424e0 2022-07-21T01:51:59.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b157fa700 1 -- 172.21.15.140:0/1611395801 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b10012ac0 con 0x7f6b181424e0 2022-07-21T01:51:59.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.783+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/1611395801 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b18112a40 con 0x7f6b181424e0 2022-07-21T01:51:59.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.787+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/1611395801 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b181559f0 con 0x7f6b181424e0 2022-07-21T01:51:59.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.787+0000 7f6b157fa700 1 -- 172.21.15.140:0/1611395801 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6b10022070 con 0x7f6b181424e0 2022-07-21T01:51:59.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.787+0000 7f6b157fa700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6afc053950 0x7f6afc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:51:59.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.787+0000 7f6b157fa700 1 -- 172.21.15.140:0/1611395801 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6b1006c5f0 con 0x7f6b181424e0 2022-07-21T01:51:59.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.787+0000 7f6b177fe700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6afc053950 0x7f6afc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:51:59.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.787+0000 7f6b177fe700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6afc053950 0x7f6afc055e10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f6b0800d000 tx=0x7f6b08007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:51:59.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.791+0000 7f6b157fa700 1 -- 172.21.15.140:0/1611395801 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6b10043400 con 0x7f6b181424e0 2022-07-21T01:51:59.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.919+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/1611395801 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b180ace30 con 0x7f6b181424e0 2022-07-21T01:51:59.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.919+0000 7f6b157fa700 1 -- 172.21.15.140:0/1611395801 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16136 (secure 0 0 0) 0x7f6b10016020 con 0x7f6b181424e0 2022-07-21T01:51:59.919 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:51:59.920 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":53,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:51:58.506065+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:51:54.695165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-07-21T01:51:55.500555+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:51:59.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/1611395801 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6afc053950 msgr2=0x7f6afc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6afc053950 0x7f6afc055e10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f6b0800d000 tx=0x7f6b08007b80).stop 2022-07-21T01:51:59.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/1611395801 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6b181424e0 msgr2=0x7f6b18112550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:51:59.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6b181424e0 0x7f6b18112550 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f6b1000c000 tx=0x7f6b100091d0).stop 2022-07-21T01:51:59.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/1611395801 shutdown_connections 2022-07-21T01:51:59.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6afc053950 0x7f6afc055e10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6b181424e0 0x7f6b18112550 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6b180b0570 0x7f6b18155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 --2- 172.21.15.140:0/1611395801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b180aea10 0x7f6b180b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:51:59.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/1611395801 >> 172.21.15.140:0/1611395801 conn(0x7f6b180b9360 msgr2=0x7f6b181405e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:51:59.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/1611395801 shutdown_connections 2022-07-21T01:51:59.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:51:59.923+0000 7f6b1cdcf700 1 -- 172.21.15.140:0/1611395801 wait complete. 2022-07-21T01:51:59.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:51:54.695165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:51:55.500555+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:51:59.934 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:52:00.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1beae700 1 Processor -- start 2022-07-21T01:52:00.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1beae700 1 -- start start 2022-07-21T01:52:00.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1beae700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7efe140b0480 0x7efe1414a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1beae700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe1414ab60 0x7efe14144320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1beae700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe140ac5e0 con 0x7efe140b5b50 2022-07-21T01:52:00.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1beae700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe140ad3c0 con 0x7efe140b0480 2022-07-21T01:52:00.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1beae700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe140b5f30 con 0x7efe1414ab60 2022-07-21T01:52:00.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe19c4a700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7efe140b5b50 0x7efe140accb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49426/0 (socket says 172.21.15.140:49426) 2022-07-21T01:52:00.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe19c4a700 1 -- 172.21.15.140:0/43868812 learned_addr learned my addr 172.21.15.140:0/43868812 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:52:00.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe19449700 1 --2- 172.21.15.140:0/43868812 >> v2:172.21.15.140:3300/0 conn(0x7efe140b0480 0x7efe1414a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1a44b700 1 --2- 172.21.15.140:0/43868812 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe1414ab60 0x7efe14144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe18c48700 1 -- 172.21.15.140:0/43868812 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507106611 0 0) 0x7efe140ac5e0 con 0x7efe140b5b50 2022-07-21T01:52:00.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe18c48700 1 -- 172.21.15.140:0/43868812 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdfc003260 con 0x7efe140b5b50 2022-07-21T01:52:00.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1a44b700 1 -- 172.21.15.140:0/43868812 >> v2:172.21.15.140:3300/0 conn(0x7efe140b0480 msgr2=0x7efe1414a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:00.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1a44b700 1 --2- 172.21.15.140:0/43868812 >> v2:172.21.15.140:3300/0 conn(0x7efe140b0480 0x7efe1414a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1a44b700 1 -- 172.21.15.140:0/43868812 >> v1:172.21.15.140:6789/0 conn(0x7efe140b5b50 legacy=0x7efe140accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1a44b700 1 -- 172.21.15.140:0/43868812 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1405cf70 con 0x7efe1414ab60 2022-07-21T01:52:00.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe19449700 1 --2- 172.21.15.140:0/43868812 >> v2:172.21.15.140:3300/0 conn(0x7efe140b0480 0x7efe1414a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:52:00.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe1a44b700 1 --2- 172.21.15.140:0/43868812 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe1414ab60 0x7efe14144320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7efe0c0032f0 tx=0x7efe0c0091d0).ready entity=mon.2 client_cookie=3f5bf930e11adf74 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.047+0000 7efe18c48700 1 -- 172.21.15.140:0/43868812 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe0c00b600 con 0x7efe1414ab60 2022-07-21T01:52:00.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe18c48700 1 -- 172.21.15.140:0/43868812 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c00b760 con 0x7efe1414ab60 2022-07-21T01:52:00.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe18c48700 1 -- 172.21.15.140:0/43868812 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe0c00b8d0 con 0x7efe1414ab60 2022-07-21T01:52:00.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 -- 172.21.15.140:0/43868812 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe1414ab60 msgr2=0x7efe14144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 --2- 172.21.15.140:0/43868812 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe1414ab60 0x7efe14144320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7efe0c0032f0 tx=0x7efe0c0091d0).stop 2022-07-21T01:52:00.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 -- 172.21.15.140:0/43868812 shutdown_connections 2022-07-21T01:52:00.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 --2- 172.21.15.140:0/43868812 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe1414ab60 0x7efe14144320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 --2- 172.21.15.140:0/43868812 >> v2:172.21.15.140:3300/0 conn(0x7efe140b0480 0x7efe1414a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 -- 172.21.15.140:0/43868812 >> 172.21.15.140:0/43868812 conn(0x7efe140635e0 msgr2=0x7efe140b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:00.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 -- 172.21.15.140:0/43868812 shutdown_connections 2022-07-21T01:52:00.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 -- 172.21.15.140:0/43868812 wait complete. 2022-07-21T01:52:00.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 Processor -- start 2022-07-21T01:52:00.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 -- start start 2022-07-21T01:52:00.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe140b0480 0x7efe141496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe140b5b50 0x7efe14156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe1414ab60 0x7efe14156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe14157430 con 0x7efe1414ab60 2022-07-21T01:52:00.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe14063c60 con 0x7efe140b5b50 2022-07-21T01:52:00.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1beae700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe14063f00 con 0x7efe140b0480 2022-07-21T01:52:00.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1a44b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe1414ab60 0x7efe14156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.051+0000 7efe1a44b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe1414ab60 0x7efe14156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35290/0 (socket says 172.21.15.140:35290) 2022-07-21T01:52:00.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe1a44b700 1 -- 172.21.15.140:0/626939410 learned_addr learned my addr 172.21.15.140:0/626939410 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:00.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe19c4a700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe140b0480 0x7efe141496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe19449700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe140b5b50 0x7efe14156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe1a44b700 1 -- 172.21.15.140:0/626939410 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe140b0480 msgr2=0x7efe141496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe1a44b700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe140b0480 0x7efe141496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe1a44b700 1 -- 172.21.15.140:0/626939410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe140b5b50 msgr2=0x7efe14156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe1a44b700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe140b5b50 0x7efe14156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe1a44b700 1 -- 172.21.15.140:0/626939410 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0400b180 con 0x7efe1414ab60 2022-07-21T01:52:00.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe19449700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe140b5b50 0x7efe14156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:52:00.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe1a44b700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe1414ab60 0x7efe14156f40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7efe0c0032f0 tx=0x7efe0c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe02ffd700 1 -- 172.21.15.140:0/626939410 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe0c00b8d0 con 0x7efe1414ab60 2022-07-21T01:52:00.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe02ffd700 1 -- 172.21.15.140:0/626939410 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c021670 con 0x7efe1414ab60 2022-07-21T01:52:00.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe02ffd700 1 -- 172.21.15.140:0/626939410 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe0c00b600 con 0x7efe1414ab60 2022-07-21T01:52:00.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe1beae700 1 -- 172.21.15.140:0/626939410 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c009470 con 0x7efe1414ab60 2022-07-21T01:52:00.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe1beae700 1 -- 172.21.15.140:0/626939410 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe14147d80 con 0x7efe1414ab60 2022-07-21T01:52:00.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe1beae700 1 -- 172.21.15.140:0/626939410 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe14156680 con 0x7efe1414ab60 2022-07-21T01:52:00.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.055+0000 7efe02ffd700 1 -- 172.21.15.140:0/626939410 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efe0c028070 con 0x7efe1414ab60 2022-07-21T01:52:00.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.059+0000 7efe02ffd700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdf4053b00 0x7efdf4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.059+0000 7efe02ffd700 1 -- 172.21.15.140:0/626939410 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efe0c06cf80 con 0x7efe1414ab60 2022-07-21T01:52:00.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.059+0000 7efe19c4a700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdf4053b00 0x7efdf4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.059+0000 7efe19c4a700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdf4053b00 0x7efdf4055fc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7efe1000d000 tx=0x7efe10009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.063+0000 7efe02ffd700 1 -- 172.21.15.140:0/626939410 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efe14156680 con 0x7efe1414ab60 2022-07-21T01:52:00.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.191+0000 7efe1beae700 1 -- 172.21.15.140:0/626939410 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7efe14156680 con 0x7efe1414ab60 2022-07-21T01:52:00.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.191+0000 7efe02ffd700 1 -- 172.21.15.140:0/626939410 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v53) v1 ==== 110+0+10 (secure 0 0 0) 0x7efe14156680 con 0x7efe1414ab60 2022-07-21T01:52:00.191 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:52:00.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 -- 172.21.15.140:0/626939410 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdf4053b00 msgr2=0x7efdf4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdf4053b00 0x7efdf4055fc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7efe1000d000 tx=0x7efe10009040).stop 2022-07-21T01:52:00.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 -- 172.21.15.140:0/626939410 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe1414ab60 msgr2=0x7efe14156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe1414ab60 0x7efe14156f40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7efe0c0032f0 tx=0x7efe0c016000).stop 2022-07-21T01:52:00.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 -- 172.21.15.140:0/626939410 shutdown_connections 2022-07-21T01:52:00.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdf4053b00 0x7efdf4055fc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efe1414ab60 0x7efe14156f40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe140b5b50 0x7efe14156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 --2- 172.21.15.140:0/626939410 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efe140b0480 0x7efe141496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 -- 172.21.15.140:0/626939410 >> 172.21.15.140:0/626939410 conn(0x7efe140635e0 msgr2=0x7efe140b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:00.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 -- 172.21.15.140:0/626939410 shutdown_connections 2022-07-21T01:52:00.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.195+0000 7efe1beae700 1 -- 172.21.15.140:0/626939410 wait complete. 2022-07-21T01:52:00.205 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:52:00.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.315+0000 7f3bf7fff700 1 Processor -- start 2022-07-21T01:52:00.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.315+0000 7f3bf7fff700 1 -- start start 2022-07-21T01:52:00.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.315+0000 7f3bf7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3bf80b2ec0 0x7f3bf80b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.315+0000 7f3bf7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3bf805d160 0x7f3bf8142210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.315+0000 7f3bf7fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf80b3f10 con 0x7f3bf8142700 2022-07-21T01:52:00.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.315+0000 7f3bf7fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf8142b90 con 0x7f3bf805d160 2022-07-21T01:52:00.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.315+0000 7f3bf7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80b4ea0 con 0x7f3bf80b2ec0 2022-07-21T01:52:00.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.315+0000 7f3bf67fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3bf805d160 0x7f3bf8142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.315+0000 7f3bf67fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3bf805d160 0x7f3bf8142210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35296/0 (socket says 172.21.15.140:35296) 2022-07-21T01:52:00.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.315+0000 7f3bf67fc700 1 -- 172.21.15.140:0/286362018 learned_addr learned my addr 172.21.15.140:0/286362018 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:00.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf6ffd700 1 --2- 172.21.15.140:0/286362018 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3bf80b2ec0 0x7f3bf80b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf67fc700 1 -- 172.21.15.140:0/286362018 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3bf80b2ec0 msgr2=0x7f3bf80b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:00.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf67fc700 1 --2- 172.21.15.140:0/286362018 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3bf80b2ec0 0x7f3bf80b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf67fc700 1 -- 172.21.15.140:0/286362018 >> v1:172.21.15.152:6789/0 conn(0x7f3bf8142700 legacy=0x7f3bf81461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf67fc700 1 -- 172.21.15.140:0/286362018 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf80b3790 con 0x7f3bf805d160 2022-07-21T01:52:00.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf67fc700 1 --2- 172.21.15.140:0/286362018 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3bf805d160 0x7f3bf8142210 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f3bf80b4fe0 tx=0x7f3be0006eb0).ready entity=mon.0 client_cookie=e4bbde51c115b4d1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf5ffb700 1 -- 172.21.15.140:0/286362018 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be000d5c0 con 0x7f3bf805d160 2022-07-21T01:52:00.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf5ffb700 1 -- 172.21.15.140:0/286362018 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be0000e90 con 0x7f3bf805d160 2022-07-21T01:52:00.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf5ffb700 1 -- 172.21.15.140:0/286362018 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be000d950 con 0x7f3bf805d160 2022-07-21T01:52:00.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf7fff700 1 -- 172.21.15.140:0/286362018 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3bf805d160 msgr2=0x7f3bf8142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf7fff700 1 --2- 172.21.15.140:0/286362018 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3bf805d160 0x7f3bf8142210 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f3bf80b4fe0 tx=0x7f3be0006eb0).stop 2022-07-21T01:52:00.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf7fff700 1 -- 172.21.15.140:0/286362018 shutdown_connections 2022-07-21T01:52:00.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf7fff700 1 --2- 172.21.15.140:0/286362018 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3bf805d160 0x7f3bf8142210 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf7fff700 1 --2- 172.21.15.140:0/286362018 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3bf80b2ec0 0x7f3bf80b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf7fff700 1 -- 172.21.15.140:0/286362018 >> 172.21.15.140:0/286362018 conn(0x7f3bf8066e20 msgr2=0x7f3bf8067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:00.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf7fff700 1 -- 172.21.15.140:0/286362018 shutdown_connections 2022-07-21T01:52:00.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.319+0000 7f3bf7fff700 1 -- 172.21.15.140:0/286362018 wait complete. 2022-07-21T01:52:00.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf7fff700 1 Processor -- start 2022-07-21T01:52:00.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf7fff700 1 -- start start 2022-07-21T01:52:00.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bf805d160 0x7f3bf81444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3bf80b2ec0 0x7f3bf8155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3bf8142700 0x7f3bf8112800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf7fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80aded0 con 0x7f3bf8142700 2022-07-21T01:52:00.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf7fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80acd80 con 0x7f3bf805d160 2022-07-21T01:52:00.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80acae0 con 0x7f3bf80b2ec0 2022-07-21T01:52:00.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3bf8142700 0x7f3bf8112800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3bf8142700 0x7f3bf8112800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35304/0 (socket says 172.21.15.140:35304) 2022-07-21T01:52:00.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf77fe700 1 -- 172.21.15.140:0/2260099229 learned_addr learned my addr 172.21.15.140:0/2260099229 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:00.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf67fc700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3bf80b2ec0 0x7f3bf8155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf6ffd700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bf805d160 0x7f3bf81444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf77fe700 1 -- 172.21.15.140:0/2260099229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3bf80b2ec0 msgr2=0x7f3bf8155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf77fe700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3bf80b2ec0 0x7f3bf8155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf77fe700 1 -- 172.21.15.140:0/2260099229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bf805d160 msgr2=0x7f3bf81444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf77fe700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bf805d160 0x7f3bf81444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf77fe700 1 -- 172.21.15.140:0/2260099229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be000b1e0 con 0x7f3bf8142700 2022-07-21T01:52:00.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf77fe700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3bf8142700 0x7f3bf8112800 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f3bf80b6e70 tx=0x7f3bf0007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3beffff700 1 -- 172.21.15.140:0/2260099229 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3bf000f470 con 0x7f3bf8142700 2022-07-21T01:52:00.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3beffff700 1 -- 172.21.15.140:0/2260099229 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf000f5d0 con 0x7f3bf8142700 2022-07-21T01:52:00.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3bf7fff700 1 -- 172.21.15.140:0/2260099229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf80adc30 con 0x7f3bf8142700 2022-07-21T01:52:00.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.323+0000 7f3beffff700 1 -- 172.21.15.140:0/2260099229 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3bf000f930 con 0x7f3bf8142700 2022-07-21T01:52:00.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.327+0000 7f3bf7fff700 1 -- 172.21.15.140:0/2260099229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf8155fd0 con 0x7f3bf8142700 2022-07-21T01:52:00.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.327+0000 7f3bf7fff700 1 -- 172.21.15.140:0/2260099229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf80b2620 con 0x7f3bf8142700 2022-07-21T01:52:00.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.327+0000 7f3beffff700 1 -- 172.21.15.140:0/2260099229 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3bf0019030 con 0x7f3bf8142700 2022-07-21T01:52:00.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.327+0000 7f3beffff700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bdc053a70 0x7f3bdc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.327+0000 7f3beffff700 1 -- 172.21.15.140:0/2260099229 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3bf006b530 con 0x7f3bf8142700 2022-07-21T01:52:00.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.327+0000 7f3bf6ffd700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bdc053a70 0x7f3bdc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.331+0000 7f3bf6ffd700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bdc053a70 0x7f3bdc055f30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f3be800c000 tx=0x7f3be8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.335+0000 7f3beffff700 1 -- 172.21.15.140:0/2260099229 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3bf00423f0 con 0x7f3bf8142700 2022-07-21T01:52:00.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.455+0000 7f3bf7fff700 1 -- 172.21.15.140:0/2260099229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3bf80574e0 con 0x7f3bf8142700 2022-07-21T01:52:00.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3beffff700 1 -- 172.21.15.140:0/2260099229 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3bf0045a10 con 0x7f3bf8142700 2022-07-21T01:52:00.457 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:52:00.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 -- 172.21.15.140:0/2260099229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bdc053a70 msgr2=0x7f3bdc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bdc053a70 0x7f3bdc055f30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f3be800c000 tx=0x7f3be8008040).stop 2022-07-21T01:52:00.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 -- 172.21.15.140:0/2260099229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3bf8142700 msgr2=0x7f3bf8112800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3bf8142700 0x7f3bf8112800 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f3bf80b6e70 tx=0x7f3bf0007f40).stop 2022-07-21T01:52:00.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 -- 172.21.15.140:0/2260099229 shutdown_connections 2022-07-21T01:52:00.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bdc053a70 0x7f3bdc055f30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3bf8142700 0x7f3bf8112800 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3bf80b2ec0 0x7f3bf8155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 --2- 172.21.15.140:0/2260099229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3bf805d160 0x7f3bf81444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 -- 172.21.15.140:0/2260099229 >> 172.21.15.140:0/2260099229 conn(0x7f3bf8066e20 msgr2=0x7f3bf80b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:00.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 -- 172.21.15.140:0/2260099229 shutdown_connections 2022-07-21T01:52:00.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.459+0000 7f3bf7fff700 1 -- 172.21.15.140:0/2260099229 wait complete. 2022-07-21T01:52:00.470 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:52:00.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.579+0000 7f29c301f700 1 Processor -- start 2022-07-21T01:52:00.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.579+0000 7f29c301f700 1 -- start start 2022-07-21T01:52:00.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29c301f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f29bc0b4d60 0x7f29bc145d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29c301f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29bc0607e0 con 0x7f29bc09df40 2022-07-21T01:52:00.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29c301f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29bc060a80 con 0x7f29bc060ee0 2022-07-21T01:52:00.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29c301f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc060d20 con 0x7f29bc0b4d60 2022-07-21T01:52:00.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29bbfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f29bc0b4d60 0x7f29bc145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29c0dbb700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f29bc09df40 0x7f29bc145680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49454/0 (socket says 172.21.15.140:49454) 2022-07-21T01:52:00.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29c0dbb700 1 -- 172.21.15.140:0/1395547382 learned_addr learned my addr 172.21.15.140:0/1395547382 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:52:00.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29bbfff700 1 -- 172.21.15.140:0/1395547382 >> v1:172.21.15.152:6789/0 conn(0x7f29bc060ee0 legacy=0x7f29bc09cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29bbfff700 1 -- 172.21.15.140:0/1395547382 >> v1:172.21.15.140:6789/0 conn(0x7f29bc09df40 legacy=0x7f29bc145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29bbfff700 1 -- 172.21.15.140:0/1395547382 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc09e320 con 0x7f29bc0b4d60 2022-07-21T01:52:00.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29bbfff700 1 --2- 172.21.15.140:0/1395547382 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0b4d60 0x7f29bc145d90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f29ac002f70 tx=0x7f29ac006df0).ready entity=mon.0 client_cookie=bee719aff0f54de2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29bb7fe700 1 -- 172.21.15.140:0/1395547382 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29ac00d610 con 0x7f29bc0b4d60 2022-07-21T01:52:00.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29bb7fe700 1 -- 172.21.15.140:0/1395547382 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac000e90 con 0x7f29bc0b4d60 2022-07-21T01:52:00.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29bb7fe700 1 -- 172.21.15.140:0/1395547382 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29ac00d960 con 0x7f29bc0b4d60 2022-07-21T01:52:00.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29c301f700 1 -- 172.21.15.140:0/1395547382 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0b4d60 msgr2=0x7f29bc145d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29c301f700 1 --2- 172.21.15.140:0/1395547382 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0b4d60 0x7f29bc145d90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f29ac002f70 tx=0x7f29ac006df0).stop 2022-07-21T01:52:00.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29c301f700 1 -- 172.21.15.140:0/1395547382 shutdown_connections 2022-07-21T01:52:00.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29c301f700 1 --2- 172.21.15.140:0/1395547382 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0b4d60 0x7f29bc145d90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.583+0000 7f29c301f700 1 -- 172.21.15.140:0/1395547382 >> 172.21.15.140:0/1395547382 conn(0x7f29bc0b5f30 msgr2=0x7f29bc0b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:00.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.587+0000 7f29c301f700 1 -- 172.21.15.140:0/1395547382 shutdown_connections 2022-07-21T01:52:00.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.587+0000 7f29c301f700 1 -- 172.21.15.140:0/1395547382 wait complete. 2022-07-21T01:52:00.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.587+0000 7f29c301f700 1 Processor -- start 2022-07-21T01:52:00.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.587+0000 7f29c301f700 1 -- start start 2022-07-21T01:52:00.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c301f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29bc060ee0 0x7f29bc0afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c301f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29bc09df40 0x7f29bc05c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c301f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0b4d60 0x7f29bc152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c301f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0b0300 con 0x7f29bc0b4d60 2022-07-21T01:52:00.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c301f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0b05a0 con 0x7f29bc060ee0 2022-07-21T01:52:00.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c301f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0b1dd0 con 0x7f29bc09df40 2022-07-21T01:52:00.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c15bc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0b4d60 0x7f29bc152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c15bc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0b4d60 0x7f29bc152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35318/0 (socket says 172.21.15.140:35318) 2022-07-21T01:52:00.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c15bc700 1 -- 172.21.15.140:0/1414100584 learned_addr learned my addr 172.21.15.140:0/1414100584 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:00.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29bbfff700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29bc09df40 0x7f29bc05c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c0dbb700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29bc060ee0 0x7f29bc0afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c15bc700 1 -- 172.21.15.140:0/1414100584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29bc09df40 msgr2=0x7f29bc05c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c15bc700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29bc09df40 0x7f29bc05c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c15bc700 1 -- 172.21.15.140:0/1414100584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29bc060ee0 msgr2=0x7f29bc0afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c15bc700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29bc060ee0 0x7f29bc0afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c15bc700 1 -- 172.21.15.140:0/1414100584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac00b180 con 0x7f29bc0b4d60 2022-07-21T01:52:00.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c15bc700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0b4d60 0x7f29bc152110 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f29bc05f350 tx=0x7f29b40013d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29b9ffb700 1 -- 172.21.15.140:0/1414100584 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29b400f470 con 0x7f29bc0b4d60 2022-07-21T01:52:00.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29b9ffb700 1 -- 172.21.15.140:0/1414100584 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b400f5d0 con 0x7f29bc0b4d60 2022-07-21T01:52:00.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29b9ffb700 1 -- 172.21.15.140:0/1414100584 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29b400f930 con 0x7f29bc0b4d60 2022-07-21T01:52:00.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c301f700 1 -- 172.21.15.140:0/1414100584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc0b2050 con 0x7f29bc0b4d60 2022-07-21T01:52:00.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c301f700 1 -- 172.21.15.140:0/1414100584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc090da0 con 0x7f29bc0b4d60 2022-07-21T01:52:00.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.591+0000 7f29c301f700 1 -- 172.21.15.140:0/1414100584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29bc0b0760 con 0x7f29bc0b4d60 2022-07-21T01:52:00.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.595+0000 7f29b9ffb700 1 -- 172.21.15.140:0/1414100584 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f29b4019030 con 0x7f29bc0b4d60 2022-07-21T01:52:00.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.595+0000 7f29b9ffb700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053a70 0x7f299c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.595+0000 7f29b9ffb700 1 -- 172.21.15.140:0/1414100584 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f29b406b5f0 con 0x7f29bc0b4d60 2022-07-21T01:52:00.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.595+0000 7f29c0dbb700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053a70 0x7f299c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.595+0000 7f29c0dbb700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053a70 0x7f299c055f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f29b000d000 tx=0x7f29b0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.599+0000 7f29b9ffb700 1 -- 172.21.15.140:0/1414100584 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f29b4042410 con 0x7f29bc0b4d60 2022-07-21T01:52:00.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.723+0000 7f29c301f700 1 -- 172.21.15.140:0/1414100584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29bc0574e0 con 0x7f29bc0b4d60 2022-07-21T01:52:00.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29b9ffb700 1 -- 172.21.15.140:0/1414100584 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+10 (secure 0 0 0) 0x7f29b4045a30 con 0x7f29bc0b4d60 2022-07-21T01:52:00.725 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:52:00.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 -- 172.21.15.140:0/1414100584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053a70 msgr2=0x7f299c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053a70 0x7f299c055f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f29b000d000 tx=0x7f29b0009040).stop 2022-07-21T01:52:00.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 -- 172.21.15.140:0/1414100584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0b4d60 msgr2=0x7f29bc152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0b4d60 0x7f29bc152110 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f29bc05f350 tx=0x7f29b40013d0).stop 2022-07-21T01:52:00.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 -- 172.21.15.140:0/1414100584 shutdown_connections 2022-07-21T01:52:00.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f299c053a70 0x7f299c055f30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29bc0b4d60 0x7f29bc152110 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29bc09df40 0x7f29bc05c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 --2- 172.21.15.140:0/1414100584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29bc060ee0 0x7f29bc0afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 -- 172.21.15.140:0/1414100584 >> 172.21.15.140:0/1414100584 conn(0x7f29bc0b5f30 msgr2=0x7f29bc0b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:00.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 -- 172.21.15.140:0/1414100584 shutdown_connections 2022-07-21T01:52:00.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.727+0000 7f29c301f700 1 -- 172.21.15.140:0/1414100584 wait complete. 2022-07-21T01:52:00.738 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:52:00.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.847+0000 7f5a63fff700 1 Processor -- start 2022-07-21T01:52:00.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.847+0000 7f5a63fff700 1 -- start start 2022-07-21T01:52:00.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5a640b4b10 0x7f5a640af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5a640b0350 0x7f5a6413e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a640aabe0 con 0x7f5a640b0890 2022-07-21T01:52:00.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640b4ef0 con 0x7f5a640b4b10 2022-07-21T01:52:00.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640aae10 con 0x7f5a640b0350 2022-07-21T01:52:00.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a62ffd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5a640b4b10 0x7f5a640af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a637fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5a640b0350 0x7f5a6413e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a62ffd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5a640b4b10 0x7f5a640af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:41064/0 (socket says 172.21.15.140:41064) 2022-07-21T01:52:00.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a627fc700 1 --1- 172.21.15.140:0/1094284609 >> v1:172.21.15.152:6789/0 conn(0x7f5a640b0890 0x7f5a640ac1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:50582/0 (socket says 172.21.15.140:50582) 2022-07-21T01:52:00.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a62ffd700 1 -- 172.21.15.140:0/1094284609 learned_addr learned my addr 172.21.15.140:0/1094284609 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:00.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a62ffd700 1 -- 172.21.15.140:0/1094284609 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5a640b0350 msgr2=0x7f5a6413e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:00.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a62ffd700 1 --2- 172.21.15.140:0/1094284609 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5a640b0350 0x7f5a6413e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a62ffd700 1 -- 172.21.15.140:0/1094284609 >> v1:172.21.15.152:6789/0 conn(0x7f5a640b0890 legacy=0x7f5a640ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a62ffd700 1 -- 172.21.15.140:0/1094284609 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a640ab0b0 con 0x7f5a640b4b10 2022-07-21T01:52:00.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a62ffd700 1 --2- 172.21.15.140:0/1094284609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a640b4b10 0x7f5a640af8b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f5a54002f70 tx=0x7f5a54006eb0).ready entity=mon.1 client_cookie=ee19f7ea9d626fa7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a61ffb700 1 -- 172.21.15.140:0/1094284609 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a5400d570 con 0x7f5a640b4b10 2022-07-21T01:52:00.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a61ffb700 1 -- 172.21.15.140:0/1094284609 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a54000e90 con 0x7f5a640b4b10 2022-07-21T01:52:00.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a61ffb700 1 -- 172.21.15.140:0/1094284609 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a5400d900 con 0x7f5a640b4b10 2022-07-21T01:52:00.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 -- 172.21.15.140:0/1094284609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a640b4b10 msgr2=0x7f5a640af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 --2- 172.21.15.140:0/1094284609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a640b4b10 0x7f5a640af8b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f5a54002f70 tx=0x7f5a54006eb0).stop 2022-07-21T01:52:00.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 -- 172.21.15.140:0/1094284609 shutdown_connections 2022-07-21T01:52:00.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 --2- 172.21.15.140:0/1094284609 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5a640b0350 0x7f5a6413e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 --2- 172.21.15.140:0/1094284609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a640b4b10 0x7f5a640af8b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 -- 172.21.15.140:0/1094284609 >> 172.21.15.140:0/1094284609 conn(0x7f5a640632b0 msgr2=0x7f5a64063690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:00.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 -- 172.21.15.140:0/1094284609 shutdown_connections 2022-07-21T01:52:00.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.851+0000 7f5a63fff700 1 -- 172.21.15.140:0/1094284609 wait complete. 2022-07-21T01:52:00.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a63fff700 1 Processor -- start 2022-07-21T01:52:00.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a63fff700 1 -- start start 2022-07-21T01:52:00.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a63fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5a640b0350 0x7f5a6413ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a63fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a640b0890 0x7f5a64143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a63fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5a640b4b10 0x7f5a64143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a63fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640ad020 con 0x7f5a640b4b10 2022-07-21T01:52:00.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a63fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640ad2c0 con 0x7f5a640b0890 2022-07-21T01:52:00.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a63fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640ad560 con 0x7f5a640b0350 2022-07-21T01:52:00.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a637fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5a640b4b10 0x7f5a64143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a637fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5a640b4b10 0x7f5a64143740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35332/0 (socket says 172.21.15.140:35332) 2022-07-21T01:52:00.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a637fe700 1 -- 172.21.15.140:0/3771116710 learned_addr learned my addr 172.21.15.140:0/3771116710 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:00.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a627fc700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a640b0890 0x7f5a64143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.855+0000 7f5a62ffd700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5a640b0350 0x7f5a6413ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a637fe700 1 -- 172.21.15.140:0/3771116710 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5a640b0350 msgr2=0x7f5a6413ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a637fe700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5a640b0350 0x7f5a6413ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a637fe700 1 -- 172.21.15.140:0/3771116710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a640b0890 msgr2=0x7f5a64143250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a637fe700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a640b0890 0x7f5a64143250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a637fe700 1 -- 172.21.15.140:0/3771116710 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5400b1e0 con 0x7f5a640b4b10 2022-07-21T01:52:00.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a627fc700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a640b0890 0x7f5a64143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:52:00.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a637fe700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5a640b4b10 0x7f5a64143740 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f5a5c006e00 tx=0x7f5a5c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a53fff700 1 -- 172.21.15.140:0/3771116710 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a5c00c640 con 0x7f5a640b4b10 2022-07-21T01:52:00.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a53fff700 1 -- 172.21.15.140:0/3771116710 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c00c7a0 con 0x7f5a640b4b10 2022-07-21T01:52:00.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a63fff700 1 -- 172.21.15.140:0/3771116710 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a64143c30 con 0x7f5a640b4b10 2022-07-21T01:52:00.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a53fff700 1 -- 172.21.15.140:0/3771116710 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a5c00cb00 con 0x7f5a640b4b10 2022-07-21T01:52:00.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a63fff700 1 -- 172.21.15.140:0/3771116710 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a640ada00 con 0x7f5a640b4b10 2022-07-21T01:52:00.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a63fff700 1 -- 172.21.15.140:0/3771116710 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a640ad720 con 0x7f5a640b4b10 2022-07-21T01:52:00.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a53fff700 1 -- 172.21.15.140:0/3771116710 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5a5c00cc60 con 0x7f5a640b4b10 2022-07-21T01:52:00.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a53fff700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a400539e0 0x7f5a40055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:00.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a53fff700 1 -- 172.21.15.140:0/3771116710 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5a5c06c0f0 con 0x7f5a640b4b10 2022-07-21T01:52:00.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.859+0000 7f5a62ffd700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a400539e0 0x7f5a40055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:00.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.863+0000 7f5a62ffd700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a400539e0 0x7f5a40055ea0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f5a5401c000 tx=0x7f5a54015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:00.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.867+0000 7f5a53fff700 1 -- 172.21.15.140:0/3771116710 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5a5c043010 con 0x7f5a640b4b10 2022-07-21T01:52:00.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.991+0000 7f5a63fff700 1 -- 172.21.15.140:0/3771116710 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a640afcc0 con 0x7f5a640b4b10 2022-07-21T01:52:00.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.991+0000 7f5a53fff700 1 -- 172.21.15.140:0/3771116710 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5a5c046630 con 0x7f5a640b4b10 2022-07-21T01:52:00.992 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:52:00.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 -- 172.21.15.140:0/3771116710 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a400539e0 msgr2=0x7f5a40055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a400539e0 0x7f5a40055ea0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f5a5401c000 tx=0x7f5a54015000).stop 2022-07-21T01:52:00.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 -- 172.21.15.140:0/3771116710 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5a640b4b10 msgr2=0x7f5a64143740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:00.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5a640b4b10 0x7f5a64143740 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f5a5c006e00 tx=0x7f5a5c0091d0).stop 2022-07-21T01:52:00.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 -- 172.21.15.140:0/3771116710 shutdown_connections 2022-07-21T01:52:00.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a400539e0 0x7f5a40055ea0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5a640b4b10 0x7f5a64143740 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a640b0890 0x7f5a64143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 --2- 172.21.15.140:0/3771116710 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5a640b0350 0x7f5a6413ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:00.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 -- 172.21.15.140:0/3771116710 >> 172.21.15.140:0/3771116710 conn(0x7f5a640632b0 msgr2=0x7f5a640b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:00.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 -- 172.21.15.140:0/3771116710 shutdown_connections 2022-07-21T01:52:00.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:00.995+0000 7f5a63fff700 1 -- 172.21.15.140:0/3771116710 wait complete. 2022-07-21T01:52:01.009 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:52:01.010 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:52:01.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1728137700 1 Processor -- start 2022-07-21T01:52:01.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1728137700 1 -- start start 2022-07-21T01:52:01.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1728137700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1720061d80 0x7f17200b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:01.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1728137700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17200621a0 con 0x7f17200b3110 2022-07-21T01:52:01.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1728137700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17200b39e0 con 0x7f172013f940 2022-07-21T01:52:01.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1728137700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17200acd90 con 0x7f1720061d80 2022-07-21T01:52:01.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1725ed3700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f17200b3110 0x7f17201414b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49482/0 (socket says 172.21.15.140:49482) 2022-07-21T01:52:01.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1725ed3700 1 -- 172.21.15.140:0/1423308562 learned_addr learned my addr 172.21.15.140:0/1423308562 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:52:01.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f17256d2700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1720061d80 0x7f17200b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:01.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940807 0 0) 0x7f17200621a0 con 0x7f17200b3110 2022-07-21T01:52:01.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1708003260 con 0x7f17200b3110 2022-07-21T01:52:01.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347201338 0 0) 0x7f17200b39e0 con 0x7f172013f940 2022-07-21T01:52:01.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17200621a0 con 0x7f172013f940 2022-07-21T01:52:01.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1814277752 0 0) 0x7f1708003260 con 0x7f17200b3110 2022-07-21T01:52:01.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1720061d80 msgr2=0x7f17200b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:01.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1720061d80 0x7f17200b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:01.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 >> v1:172.21.15.152:6789/0 conn(0x7f172013f940 legacy=0x7f172013fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:01.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17200acaf0 con 0x7f17200b3110 2022-07-21T01:52:01.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f1710001660 con 0x7f17200b3110 2022-07-21T01:52:01.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 >> v1:172.21.15.140:6789/0 conn(0x7f17200b3110 legacy=0x7f17201414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:01.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1708003910 0x7f1708003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:01.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708004500 0x7f1708004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:01.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1708004e70 0x7f170800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:01.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17200acd90 con 0x7f1708003910 2022-07-21T01:52:01.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170800bd70 con 0x7f1708004500 2022-07-21T01:52:01.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170800bf30 con 0x7f1708004e70 2022-07-21T01:52:01.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f17256d2700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1720061d80 0x7f17200b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:52:01.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f17266d4700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1708004e70 0x7f170800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:01.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f1725ed3700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1708003910 0x7f1708003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:01.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.111+0000 7f17256d2700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708004500 0x7f1708004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:01.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f17266d4700 1 -- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708004500 msgr2=0x7f1708004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:01.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f17266d4700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708004500 0x7f1708004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:01.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f17266d4700 1 -- 172.21.15.140:0/1423308562 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1708003910 msgr2=0x7f1708003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:01.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f17266d4700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1708003910 0x7f1708003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:01.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f17266d4700 1 -- 172.21.15.140:0/1423308562 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1708003260 con 0x7f1708004e70 2022-07-21T01:52:01.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1725ed3700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1708003910 0x7f1708003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:52:01.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f17266d4700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1708004e70 0x7f170800b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f1718001490 tx=0x7f1718007d60).ready entity=mon.2 client_cookie=803e1044d446107 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:01.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17180017b0 con 0x7f1708004e70 2022-07-21T01:52:01.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1718006890 con 0x7f1708004e70 2022-07-21T01:52:01.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1724ed1700 1 -- 172.21.15.140:0/1423308562 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1718006bf0 con 0x7f1708004e70 2022-07-21T01:52:01.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1728137700 1 -- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1708004e70 msgr2=0x7f170800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:01.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1728137700 1 --2- 172.21.15.140:0/1423308562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1708004e70 0x7f170800b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f1718001490 tx=0x7f1718007d60).stop 2022-07-21T01:52:01.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f17266d4700 1 -- 172.21.15.140:0/1423308562 reap_dead start 2022-07-21T01:52:01.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1728137700 1 -- 172.21.15.140:0/1423308562 shutdown_connections 2022-07-21T01:52:01.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1728137700 1 -- 172.21.15.140:0/1423308562 >> 172.21.15.140:0/1423308562 conn(0x7f17200b47e0 msgr2=0x7f172005bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:01.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1728137700 1 -- 172.21.15.140:0/1423308562 shutdown_connections 2022-07-21T01:52:01.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1728137700 1 -- 172.21.15.140:0/1423308562 wait complete. 2022-07-21T01:52:01.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1728137700 1 Processor -- start 2022-07-21T01:52:01.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.115+0000 7f1728137700 1 -- start start 2022-07-21T01:52:01.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f1728137700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708004e70 0x7f1720150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:01.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f1728137700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1720065180 0x7f172009b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:01.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f1728137700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1720066b10 0x7f1720066f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:01.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f1728137700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1720096f00 con 0x7f1720065180 2022-07-21T01:52:01.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f1728137700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172009b990 con 0x7f1708004e70 2022-07-21T01:52:01.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f1728137700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172009bc30 con 0x7f1720066b10 2022-07-21T01:52:01.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f17256d2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1720065180 0x7f172009b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:01.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f17256d2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1720065180 0x7f172009b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35352/0 (socket says 172.21.15.140:35352) 2022-07-21T01:52:01.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f17256d2700 1 -- 172.21.15.140:0/4283123744 learned_addr learned my addr 172.21.15.140:0/4283123744 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:01.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f17266d4700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1720066b10 0x7f1720066f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:01.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f1725ed3700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708004e70 0x7f1720150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:01.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f17256d2700 1 -- 172.21.15.140:0/4283123744 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1720066b10 msgr2=0x7f1720066f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:01.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f17256d2700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1720066b10 0x7f1720066f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:01.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f17256d2700 1 -- 172.21.15.140:0/4283123744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708004e70 msgr2=0x7f1720150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:01.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f17256d2700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708004e70 0x7f1720150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:01.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f17256d2700 1 -- 172.21.15.140:0/4283123744 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172009bed0 con 0x7f1720065180 2022-07-21T01:52:01.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f1725ed3700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708004e70 0x7f1720150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:52:01.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f17256d2700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1720065180 0x7f172009b130 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f171c000b60 tx=0x7f171c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:01.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f170effd700 1 -- 172.21.15.140:0/4283123744 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171c00f760 con 0x7f1720065180 2022-07-21T01:52:01.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f170effd700 1 -- 172.21.15.140:0/4283123744 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c00f8c0 con 0x7f1720065180 2022-07-21T01:52:01.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f170effd700 1 -- 172.21.15.140:0/4283123744 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171c00fa30 con 0x7f1720065180 2022-07-21T01:52:01.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f1728137700 1 -- 172.21.15.140:0/4283123744 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1720150c40 con 0x7f1720065180 2022-07-21T01:52:01.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f1728137700 1 -- 172.21.15.140:0/4283123744 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172009b620 con 0x7f1720065180 2022-07-21T01:52:01.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f1728137700 1 -- 172.21.15.140:0/4283123744 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17200b2190 con 0x7f1720065180 2022-07-21T01:52:01.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f170effd700 1 -- 172.21.15.140:0/4283123744 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f171c00fb90 con 0x7f1720065180 2022-07-21T01:52:01.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.119+0000 7f170effd700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1700053950 0x7f1700055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:01.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.123+0000 7f170effd700 1 -- 172.21.15.140:0/4283123744 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f171c06c5c0 con 0x7f1720065180 2022-07-21T01:52:01.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.123+0000 7f1725ed3700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1700053950 0x7f1700055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:01.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.123+0000 7f1725ed3700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1700053950 0x7f1700055e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f1710005000 tx=0x7f1710009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:01.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.127+0000 7f170effd700 1 -- 172.21.15.140:0/4283123744 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f171c043440 con 0x7f1720065180 2022-07-21T01:52:01.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 -- 172.21.15.140:0/4283123744 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f172009c0c0 con 0x7f1720065180 2022-07-21T01:52:01.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f170effd700 1 -- 172.21.15.140:0/4283123744 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2145 (secure 0 0 0) 0x7f171c006a60 con 0x7f1720065180 2022-07-21T01:52:01.290 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:52:01.290 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1915503334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1915503334},{"type":"v1","addr":"172.21.15.140:6835","nonce":1915503334}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:51:56.500368+0000","modified":"2022-07-21T01:51:57.504176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5204":{"gid":5204,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3908541795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3908541795},{"type":"v1","addr":"172.21.15.152:6833","nonce":3908541795}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-07-21T01:52:01.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 -- 172.21.15.140:0/4283123744 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1700053950 msgr2=0x7f1700055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:01.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1700053950 0x7f1700055e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f1710005000 tx=0x7f1710009040).stop 2022-07-21T01:52:01.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 -- 172.21.15.140:0/4283123744 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1720065180 msgr2=0x7f172009b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:01.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1720065180 0x7f172009b130 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f171c000b60 tx=0x7f171c006df0).stop 2022-07-21T01:52:01.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 -- 172.21.15.140:0/4283123744 shutdown_connections 2022-07-21T01:52:01.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1700053950 0x7f1700055e10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:01.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1720066b10 0x7f1720066f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:01.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1720065180 0x7f172009b130 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:01.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 --2- 172.21.15.140:0/4283123744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1708004e70 0x7f1720150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:01.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 -- 172.21.15.140:0/4283123744 >> 172.21.15.140:0/4283123744 conn(0x7f17200b47e0 msgr2=0x7f1720146450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:01.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 -- 172.21.15.140:0/4283123744 shutdown_connections 2022-07-21T01:52:01.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:01.291+0000 7f1728137700 1 -- 172.21.15.140:0/4283123744 wait complete. 2022-07-21T01:52:01.295 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 21 2022-07-21T01:52:01.389 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:52:01.393 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:52:01.397 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:52:01.400 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:52:01.404 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:52:01.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:51:56.500368+0000', 'modified': '2022-07-21T01:51:57.504176+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5204}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5204': {'gid': 5204, 'name': 'b', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/3908541795', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 3908541795}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 3908541795}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:52:01.510 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:52:01.511 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:52:01.624 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":5204,"want_state":"up:active","state":"up:active","rank_uptime":5.1152433159999999,"mdsmap_epoch":21,"osdmap_epoch":53,"osdmap_epoch_barrier":52,"uptime":9.7742770179999994} 2022-07-21T01:52:01.625 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 5204, "mdsmap_epoch": 21, "osdmap_epoch": 53, "osdmap_epoch_barrier": 52, "rank_uptime": 5.115243316, "state": "up:active", "uptime": 9.774277018, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:52:01.625 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:52:01.626 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:52:01.626 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:52:01.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:01.627 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:01.636 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:52:01.650 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:52:01.651 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:52:01.663 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:52:01.664 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:52:01.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:01.666 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:52:01.715 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:52:01.716 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:01.721 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:52:01.740 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:52:01.741 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:52:01.794 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:52:01.800+0000 7fe541efe080 -1 init, newargv = 0x55ef9af193a0 newargc=9 2022-07-21T01:52:01.795 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22505]: starting ceph client 2022-07-21T01:52:01.795 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:52:01.796 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:52:01.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:01.799 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:52:01.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22505]: starting fuse 2022-07-21T01:52:02.905 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:52:02.925 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:52:02.926 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:52:02.983 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:52:02.984 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:52:02.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:02.986 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:52:02.991 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:52:02.992 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:52:02.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:02.993 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:52:02.994 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:52:02.995 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:52:02.996 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:52:02.996 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:52:02.997 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:52:02.998 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:52:02.999 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:52:03.000 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:52:03.000 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:52:03.001 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:52:03.002 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:52:03.003 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:52:03.003 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:52:03.004 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:52:03.005 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:52:03.006 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:52:03.007 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:52:03.007 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:52:03.008 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:52:03.009 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:52:03.010 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:52:03.010 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:52:03.055 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.22505.asok 2022-07-21T01:52:03.060 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22505.asok 2022-07-21T01:52:03.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:03.061 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22505.asok status 2022-07-21T01:52:03.197 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:52:03.198 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:52:03.198 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:52:03.199 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:52:03.200 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:52:03.200 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:52:03.201 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:52:03.201 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "22505", 2022-07-21T01:52:03.202 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:52:03.202 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:52:03.203 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:52:03.203 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:52:03.203 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 5327, 2022-07-21T01:52:03.204 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:52:03.204 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:52:03.205 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:52:03.205 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 5327 2022-07-21T01:52:03.206 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:52:03.206 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:52:03.207 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:52:03.207 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:52:03.207 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 2046008460 2022-07-21T01:52:03.208 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:52:03.208 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:52:03.209 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:52:03.209 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:52:03.210 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:52:03.210 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 2046008460 2022-07-21T01:52:03.211 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:52:03.211 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.5327 172.21.15.191:0/2046008460", 2022-07-21T01:52:03.212 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/2046008460", 2022-07-21T01:52:03.212 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:52:03.212 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 21, 2022-07-21T01:52:03.213 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 53, 2022-07-21T01:52:03.213 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:52:03.214 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:52:03.214 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:52:03.214 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:52:03.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:03.218 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:52:03.218 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:03.226 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:52:03.228 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:52:03.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:03.229 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:03.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:03.249 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:52:03.249 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:03.309 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:52:03.310 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:52:03.310 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-07-21T01:52:03.311 DEBUG:teuthology.orchestra.run.smithi191:> import sys 2022-07-21T01:52:03.312 DEBUG:teuthology.orchestra.run.smithi191:> import time 2022-07-21T01:52:03.312 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:52:03.313 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:52:03.313 DEBUG:teuthology.orchestra.run.smithi191:> n = 1000 2022-07-21T01:52:03.314 DEBUG:teuthology.orchestra.run.smithi191:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-07-21T01:52:03.314 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:52:03.315 DEBUG:teuthology.orchestra.run.smithi191:> if not os.path.exists(os.path.dirname(abs_path)): 2022-07-21T01:52:03.315 DEBUG:teuthology.orchestra.run.smithi191:> os.makedirs(os.path.dirname(abs_path)) 2022-07-21T01:52:03.316 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:52:03.316 DEBUG:teuthology.orchestra.run.smithi191:> for i in range(0, n): 2022-07-21T01:52:03.316 DEBUG:teuthology.orchestra.run.smithi191:> fname = "{0}_{1}".format(abs_path, i) 2022-07-21T01:52:03.317 DEBUG:teuthology.orchestra.run.smithi191:> with open(fname, '"'"'w'"'"') as f: 2022-07-21T01:52:03.317 DEBUG:teuthology.orchestra.run.smithi191:> f.write('"'"'content'"'"') 2022-07-21T01:52:03.318 DEBUG:teuthology.orchestra.run.smithi191:> if True: 2022-07-21T01:52:03.318 DEBUG:teuthology.orchestra.run.smithi191:> f.flush() 2022-07-21T01:52:03.319 DEBUG:teuthology.orchestra.run.smithi191:> os.fsync(f.fileno()) 2022-07-21T01:52:03.319 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:52:12.803 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:52:12.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.915+0000 7f5925ec1700 1 Processor -- start 2022-07-21T01:52:12.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.915+0000 7f5925ec1700 1 -- start start 2022-07-21T01:52:12.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.915+0000 7f5925ec1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59200b3990 0x7f592005d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:12.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.915+0000 7f5925ec1700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5920142a50 0x7f59200b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:12.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.915+0000 7f5925ec1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59200b4410 con 0x7f592013f9d0 2022-07-21T01:52:12.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.915+0000 7f5925ec1700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59200b2b90 con 0x7f5920142a50 2022-07-21T01:52:12.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.915+0000 7f5925ec1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5920140550 con 0x7f59200b3990 2022-07-21T01:52:12.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.915+0000 7f591ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5920142a50 0x7f59200b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:12.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.915+0000 7f591ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5920142a50 0x7f59200b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35358/0 (socket says 172.21.15.140:35358) 2022-07-21T01:52:12.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.915+0000 7f59256c0700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f592013f9d0 0x7f592013fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49506/0 (socket says 172.21.15.140:49506) 2022-07-21T01:52:12.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.915+0000 7f591ffff700 1 -- 172.21.15.140:0/3343669332 learned_addr learned my addr 172.21.15.140:0/3343669332 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:12.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5924ebf700 1 --2- 172.21.15.140:0/3343669332 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59200b3990 0x7f592005d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:12.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f591ffff700 1 -- 172.21.15.140:0/3343669332 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59200b3990 msgr2=0x7f592005d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:12.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f591ffff700 1 --2- 172.21.15.140:0/3343669332 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59200b3990 0x7f592005d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:12.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f591ffff700 1 -- 172.21.15.140:0/3343669332 >> v1:172.21.15.140:6789/0 conn(0x7f592013f9d0 legacy=0x7f592013fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:12.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f591ffff700 1 -- 172.21.15.140:0/3343669332 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592005c090 con 0x7f5920142a50 2022-07-21T01:52:12.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5924ebf700 1 --2- 172.21.15.140:0/3343669332 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59200b3990 0x7f592005d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:52:12.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f591ffff700 1 --2- 172.21.15.140:0/3343669332 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5920142a50 0x7f59200b1f40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f5914002f70 tx=0x7f5914006eb0).ready entity=mon.0 client_cookie=b70015a7711dc1d2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:12.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f591f7fe700 1 -- 172.21.15.140:0/3343669332 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f591400d5c0 con 0x7f5920142a50 2022-07-21T01:52:12.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f591f7fe700 1 -- 172.21.15.140:0/3343669332 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914000e90 con 0x7f5920142a50 2022-07-21T01:52:12.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f591f7fe700 1 -- 172.21.15.140:0/3343669332 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f591400d950 con 0x7f5920142a50 2022-07-21T01:52:12.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5925ec1700 1 -- 172.21.15.140:0/3343669332 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5920142a50 msgr2=0x7f59200b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:12.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5925ec1700 1 --2- 172.21.15.140:0/3343669332 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5920142a50 0x7f59200b1f40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f5914002f70 tx=0x7f5914006eb0).stop 2022-07-21T01:52:12.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5925ec1700 1 -- 172.21.15.140:0/3343669332 shutdown_connections 2022-07-21T01:52:12.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5925ec1700 1 --2- 172.21.15.140:0/3343669332 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5920142a50 0x7f59200b1f40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:12.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5925ec1700 1 --2- 172.21.15.140:0/3343669332 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59200b3990 0x7f592005d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:12.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5925ec1700 1 -- 172.21.15.140:0/3343669332 >> 172.21.15.140:0/3343669332 conn(0x7f5920064be0 msgr2=0x7f5920064fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:12.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5925ec1700 1 -- 172.21.15.140:0/3343669332 shutdown_connections 2022-07-21T01:52:12.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5925ec1700 1 -- 172.21.15.140:0/3343669332 wait complete. 2022-07-21T01:52:12.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5925ec1700 1 Processor -- start 2022-07-21T01:52:12.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.919+0000 7f5925ec1700 1 -- start start 2022-07-21T01:52:12.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f5925ec1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59200b3990 0x7f5920147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:12.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f5925ec1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f592013f9d0 0x7f5920151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:12.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f5925ec1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5920142a50 0x7f5920155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:12.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f5925ec1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5920152f10 con 0x7f592013f9d0 2022-07-21T01:52:12.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f5925ec1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59201428f0 con 0x7f59200b3990 2022-07-21T01:52:12.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f5925ec1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5920142420 con 0x7f5920142a50 2022-07-21T01:52:12.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f592013f9d0 0x7f5920151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:12.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f592013f9d0 0x7f5920151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35364/0 (socket says 172.21.15.140:35364) 2022-07-21T01:52:12.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591ffff700 1 -- 172.21.15.140:0/3024303112 learned_addr learned my addr 172.21.15.140:0/3024303112 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:12.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f59256c0700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5920142a50 0x7f5920155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:12.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f5924ebf700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59200b3990 0x7f5920147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:12.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591ffff700 1 -- 172.21.15.140:0/3024303112 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5920142a50 msgr2=0x7f5920155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:12.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591ffff700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5920142a50 0x7f5920155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:12.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591ffff700 1 -- 172.21.15.140:0/3024303112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59200b3990 msgr2=0x7f5920147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:12.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591ffff700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59200b3990 0x7f5920147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:12.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591ffff700 1 -- 172.21.15.140:0/3024303112 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591400b1e0 con 0x7f592013f9d0 2022-07-21T01:52:12.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f59256c0700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5920142a50 0x7f5920155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:52:12.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f5924ebf700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59200b3990 0x7f5920147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:52:12.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591ffff700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f592013f9d0 0x7f5920151c90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f59200b06b0 tx=0x7f5914015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:12.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591dffb700 1 -- 172.21.15.140:0/3024303112 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5914011a40 con 0x7f592013f9d0 2022-07-21T01:52:12.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591dffb700 1 -- 172.21.15.140:0/3024303112 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914011ba0 con 0x7f592013f9d0 2022-07-21T01:52:12.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f5925ec1700 1 -- 172.21.15.140:0/3024303112 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59201553e0 con 0x7f592013f9d0 2022-07-21T01:52:12.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f591dffb700 1 -- 172.21.15.140:0/3024303112 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f591401e460 con 0x7f592013f9d0 2022-07-21T01:52:12.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.923+0000 7f5925ec1700 1 -- 172.21.15.140:0/3024303112 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5920152180 con 0x7f592013f9d0 2022-07-21T01:52:12.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.927+0000 7f5925ec1700 1 -- 172.21.15.140:0/3024303112 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59200b3d70 con 0x7f592013f9d0 2022-07-21T01:52:12.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.927+0000 7f591dffb700 1 -- 172.21.15.140:0/3024303112 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5914025030 con 0x7f592013f9d0 2022-07-21T01:52:12.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.927+0000 7f591dffb700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5900053a70 0x7f5900055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:12.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.927+0000 7f591dffb700 1 -- 172.21.15.140:0/3024303112 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f591406d310 con 0x7f592013f9d0 2022-07-21T01:52:12.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.927+0000 7f5924ebf700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5900053a70 0x7f5900055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:12.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.927+0000 7f5924ebf700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5900053a70 0x7f5900055f30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f591800b000 tx=0x7f5918008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:12.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:12.931+0000 7f591dffb700 1 -- 172.21.15.140:0/3024303112 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5914044230 con 0x7f592013f9d0 2022-07-21T01:52:13.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.095+0000 7f5925ec1700 1 -- 172.21.15.140:0/3024303112 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5920155570 con 0x7f592013f9d0 2022-07-21T01:52:13.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f591dffb700 1 -- 172.21.15.140:0/3024303112 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2145 (secure 0 0 0) 0x7f5914023020 con 0x7f592013f9d0 2022-07-21T01:52:13.097 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:52:13.098 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1915503334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1915503334},{"type":"v1","addr":"172.21.15.140:6835","nonce":1915503334}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:51:56.500368+0000","modified":"2022-07-21T01:51:57.504176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5204":{"gid":5204,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3908541795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3908541795},{"type":"v1","addr":"172.21.15.152:6833","nonce":3908541795}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-07-21T01:52:13.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 -- 172.21.15.140:0/3024303112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5900053a70 msgr2=0x7f5900055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5900053a70 0x7f5900055f30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f591800b000 tx=0x7f5918008040).stop 2022-07-21T01:52:13.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 -- 172.21.15.140:0/3024303112 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f592013f9d0 msgr2=0x7f5920151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f592013f9d0 0x7f5920151c90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f59200b06b0 tx=0x7f5914015000).stop 2022-07-21T01:52:13.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 -- 172.21.15.140:0/3024303112 shutdown_connections 2022-07-21T01:52:13.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5900053a70 0x7f5900055f30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5920142a50 0x7f5920155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f592013f9d0 0x7f5920151c90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 --2- 172.21.15.140:0/3024303112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59200b3990 0x7f5920147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 -- 172.21.15.140:0/3024303112 >> 172.21.15.140:0/3024303112 conn(0x7f5920064be0 msgr2=0x7f5920146400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:13.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 -- 172.21.15.140:0/3024303112 shutdown_connections 2022-07-21T01:52:13.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.099+0000 7f5925ec1700 1 -- 172.21.15.140:0/3024303112 wait complete. 2022-07-21T01:52:13.107 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 21 2022-07-21T01:52:13.109 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_caps 20 2022-07-21T01:52:13.222 INFO:teuthology.orchestra.run.smithi152.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-07-21T01:52:13.223 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-07-21T01:52:13.224 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:52:13.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.331+0000 7f014a4ba700 1 Processor -- start 2022-07-21T01:52:13.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.331+0000 7f014a4ba700 1 -- start start 2022-07-21T01:52:13.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f01440b1e00 0x7f01440ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f01440ae5c0 0x7f0144143200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01440aff40 con 0x7f01440afa30 2022-07-21T01:52:13.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01440b0bc0 con 0x7f01440ae5c0 2022-07-21T01:52:13.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01440b21e0 con 0x7f01440b1e00 2022-07-21T01:52:13.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0148a57700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f01440ae5c0 0x7f0144143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0148a57700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f01440ae5c0 0x7f0144143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35372/0 (socket says 172.21.15.140:35372) 2022-07-21T01:52:13.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0148a57700 1 -- 172.21.15.140:0/546155238 learned_addr learned my addr 172.21.15.140:0/546155238 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:13.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0143fff700 1 --2- 172.21.15.140:0/546155238 >> v2:172.21.15.152:3300/0 conn(0x7f01440b1e00 0x7f01440ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0148a57700 1 -- 172.21.15.140:0/546155238 >> v2:172.21.15.152:3300/0 conn(0x7f01440b1e00 msgr2=0x7f01440ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:13.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0148a57700 1 --2- 172.21.15.140:0/546155238 >> v2:172.21.15.152:3300/0 conn(0x7f01440b1e00 0x7f01440ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0148a57700 1 -- 172.21.15.140:0/546155238 >> v1:172.21.15.152:6789/0 conn(0x7f01440afa30 legacy=0x7f01440adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0148a57700 1 -- 172.21.15.140:0/546155238 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01440af510 con 0x7f01440ae5c0 2022-07-21T01:52:13.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0148a57700 1 --2- 172.21.15.140:0/546155238 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01440ae5c0 0x7f0144143200 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f013c002f70 tx=0x7f013c006eb0).ready entity=mon.0 client_cookie=a305285ac326ae11 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:13.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0142ffd700 1 -- 172.21.15.140:0/546155238 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f013c00d5c0 con 0x7f01440ae5c0 2022-07-21T01:52:13.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0142ffd700 1 -- 172.21.15.140:0/546155238 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c000e90 con 0x7f01440ae5c0 2022-07-21T01:52:13.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f0142ffd700 1 -- 172.21.15.140:0/546155238 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f013c00d950 con 0x7f01440ae5c0 2022-07-21T01:52:13.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 -- 172.21.15.140:0/546155238 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01440ae5c0 msgr2=0x7f0144143200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 --2- 172.21.15.140:0/546155238 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01440ae5c0 0x7f0144143200 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f013c002f70 tx=0x7f013c006eb0).stop 2022-07-21T01:52:13.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 -- 172.21.15.140:0/546155238 shutdown_connections 2022-07-21T01:52:13.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 --2- 172.21.15.140:0/546155238 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01440ae5c0 0x7f0144143200 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 --2- 172.21.15.140:0/546155238 >> v2:172.21.15.152:3300/0 conn(0x7f01440b1e00 0x7f01440ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 -- 172.21.15.140:0/546155238 >> 172.21.15.140:0/546155238 conn(0x7f01440611c0 msgr2=0x7f0144062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:13.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 -- 172.21.15.140:0/546155238 shutdown_connections 2022-07-21T01:52:13.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.335+0000 7f014a4ba700 1 -- 172.21.15.140:0/546155238 wait complete. 2022-07-21T01:52:13.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f014a4ba700 1 Processor -- start 2022-07-21T01:52:13.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f014a4ba700 1 -- start start 2022-07-21T01:52:13.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f014a4ba700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01440afa30 0x7f0144155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f014a4ba700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01440b1e00 0x7f0144144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f014a4ba700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01440a2f10 0x7f01440aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f014a4ba700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01441567a0 con 0x7f01440afa30 2022-07-21T01:52:13.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f014a4ba700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01440a4760 con 0x7f01440b1e00 2022-07-21T01:52:13.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f014a4ba700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0144146e50 con 0x7f01440a2f10 2022-07-21T01:52:13.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f0143fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01440afa30 0x7f0144155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f0143fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01440afa30 0x7f0144155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35376/0 (socket says 172.21.15.140:35376) 2022-07-21T01:52:13.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f0143fff700 1 -- 172.21.15.140:0/4292863943 learned_addr learned my addr 172.21.15.140:0/4292863943 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:13.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f01437fe700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01440b1e00 0x7f0144144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f0148a57700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01440a2f10 0x7f01440aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f0143fff700 1 -- 172.21.15.140:0/4292863943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01440a2f10 msgr2=0x7f01440aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f0143fff700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01440a2f10 0x7f01440aaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f0143fff700 1 -- 172.21.15.140:0/4292863943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01440b1e00 msgr2=0x7f0144144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f0143fff700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01440b1e00 0x7f0144144660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f0143fff700 1 -- 172.21.15.140:0/4292863943 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c00b1e0 con 0x7f01440afa30 2022-07-21T01:52:13.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f0143fff700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01440afa30 0x7f0144155800 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f01440a48a0 tx=0x7f01340091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:13.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f01417fa700 1 -- 172.21.15.140:0/4292863943 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f013400c7f0 con 0x7f01440afa30 2022-07-21T01:52:13.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f01417fa700 1 -- 172.21.15.140:0/4292863943 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013400c950 con 0x7f01440afa30 2022-07-21T01:52:13.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f014a4ba700 1 -- 172.21.15.140:0/4292863943 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01441470f0 con 0x7f01440afa30 2022-07-21T01:52:13.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f01417fa700 1 -- 172.21.15.140:0/4292863943 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f013400ccb0 con 0x7f01440afa30 2022-07-21T01:52:13.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.339+0000 7f014a4ba700 1 -- 172.21.15.140:0/4292863943 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0144148010 con 0x7f01440afa30 2022-07-21T01:52:13.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.343+0000 7f014a4ba700 1 -- 172.21.15.140:0/4292863943 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01440b0170 con 0x7f01440afa30 2022-07-21T01:52:13.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.343+0000 7f01417fa700 1 -- 172.21.15.140:0/4292863943 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0134022070 con 0x7f01440afa30 2022-07-21T01:52:13.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.343+0000 7f01417fa700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0128053b00 0x7f0128055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.343+0000 7f01417fa700 1 -- 172.21.15.140:0/4292863943 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f013406c590 con 0x7f01440afa30 2022-07-21T01:52:13.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.343+0000 7f01437fe700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0128053b00 0x7f0128055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.343+0000 7f01437fe700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0128053b00 0x7f0128055fc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f012c00d000 tx=0x7f012c007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:13.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.347+0000 7f01417fa700 1 -- 172.21.15.140:0/4292863943 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f01340434b0 con 0x7f01440afa30 2022-07-21T01:52:13.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.511+0000 7f014a4ba700 1 -- 172.21.15.140:0/4292863943 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01440574e0 con 0x7f01440afa30 2022-07-21T01:52:13.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.511+0000 7f01417fa700 1 -- 172.21.15.140:0/4292863943 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2145 (secure 0 0 0) 0x7f0134046ad0 con 0x7f01440afa30 2022-07-21T01:52:13.510 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:52:13.510 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1915503334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1915503334},{"type":"v1","addr":"172.21.15.140:6835","nonce":1915503334}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:51:56.500368+0000","modified":"2022-07-21T01:51:57.504176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5204":{"gid":5204,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3908541795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3908541795},{"type":"v1","addr":"172.21.15.152:6833","nonce":3908541795}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-07-21T01:52:13.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 -- 172.21.15.140:0/4292863943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0128053b00 msgr2=0x7f0128055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0128053b00 0x7f0128055fc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f012c00d000 tx=0x7f012c007b90).stop 2022-07-21T01:52:13.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 -- 172.21.15.140:0/4292863943 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01440afa30 msgr2=0x7f0144155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01440afa30 0x7f0144155800 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f01440a48a0 tx=0x7f01340091d0).stop 2022-07-21T01:52:13.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 -- 172.21.15.140:0/4292863943 shutdown_connections 2022-07-21T01:52:13.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0128053b00 0x7f0128055fc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01440a2f10 0x7f01440aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01440b1e00 0x7f0144144660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 --2- 172.21.15.140:0/4292863943 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01440afa30 0x7f0144155800 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 -- 172.21.15.140:0/4292863943 >> 172.21.15.140:0/4292863943 conn(0x7f01440ae5c0 msgr2=0x7f01440b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:13.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 -- 172.21.15.140:0/4292863943 shutdown_connections 2022-07-21T01:52:13.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.515+0000 7f014a4ba700 1 -- 172.21.15.140:0/4292863943 wait complete. 2022-07-21T01:52:13.517 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 21 2022-07-21T01:52:13.528 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_decay_threshold 400 2022-07-21T01:52:13.640 INFO:teuthology.orchestra.run.smithi152.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-07-21T01:52:13.641 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-07-21T01:52:13.641 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:52:13.659 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:52:13.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.767+0000 7f1fa7fff700 1 Processor -- start 2022-07-21T01:52:13.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.767+0000 7f1fa7fff700 1 -- start start 2022-07-21T01:52:13.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1fa80b0570 0x7f1fa80b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fa80aea10 0x7f1fa8141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa8142750 con 0x7f1fa81422c0 2022-07-21T01:52:13.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa8143b10 con 0x7f1fa80b0570 2022-07-21T01:52:13.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa81433d0 con 0x7f1fa80aea10 2022-07-21T01:52:13.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa6ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1fa80b0570 0x7f1fa80b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa77fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1fa81422c0 0x7f1fa80ad350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49532/0 (socket says 172.21.15.140:49532) 2022-07-21T01:52:13.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa77fe700 1 -- 172.21.15.140:0/1601039439 learned_addr learned my addr 172.21.15.140:0/1601039439 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:52:13.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa67fc700 1 --2- 172.21.15.140:0/1601039439 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fa80aea10 0x7f1fa8141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa6ffd700 1 -- 172.21.15.140:0/1601039439 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fa80aea10 msgr2=0x7f1fa8141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:13.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa6ffd700 1 --2- 172.21.15.140:0/1601039439 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fa80aea10 0x7f1fa8141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa6ffd700 1 -- 172.21.15.140:0/1601039439 >> v1:172.21.15.140:6789/0 conn(0x7f1fa81422c0 legacy=0x7f1fa80ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa6ffd700 1 -- 172.21.15.140:0/1601039439 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8143670 con 0x7f1fa80b0570 2022-07-21T01:52:13.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa5ffb700 1 -- 172.21.15.140:0/1601039439 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084819081 0 0) 0x7f1fa8142750 con 0x7f1fa81422c0 2022-07-21T01:52:13.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa6ffd700 1 --2- 172.21.15.140:0/1601039439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fa80b0570 0x7f1fa80b0950 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f1f98002f70 tx=0x7f1f98006df0).ready entity=mon.0 client_cookie=52f3917c9ddc1f2c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:13.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa5ffb700 1 -- 172.21.15.140:0/1601039439 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f9800d570 con 0x7f1fa80b0570 2022-07-21T01:52:13.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa5ffb700 1 -- 172.21.15.140:0/1601039439 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f98000e90 con 0x7f1fa80b0570 2022-07-21T01:52:13.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa5ffb700 1 -- 172.21.15.140:0/1601039439 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f9800d8c0 con 0x7f1fa80b0570 2022-07-21T01:52:13.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1601039439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fa80b0570 msgr2=0x7f1fa80b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 --2- 172.21.15.140:0/1601039439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fa80b0570 0x7f1fa80b0950 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f1f98002f70 tx=0x7f1f98006df0).stop 2022-07-21T01:52:13.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1601039439 shutdown_connections 2022-07-21T01:52:13.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 --2- 172.21.15.140:0/1601039439 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fa80aea10 0x7f1fa8141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 --2- 172.21.15.140:0/1601039439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fa80b0570 0x7f1fa80b0950 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1601039439 >> 172.21.15.140:0/1601039439 conn(0x7f1fa80b9360 msgr2=0x7f1fa80b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:13.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1601039439 shutdown_connections 2022-07-21T01:52:13.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.771+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1601039439 wait complete. 2022-07-21T01:52:13.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa7fff700 1 Processor -- start 2022-07-21T01:52:13.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa7fff700 1 -- start start 2022-07-21T01:52:13.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fa80aea10 0x7f1fa80b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa80b0570 0x7f1fa80b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fa81422c0 0x7f1fa8112a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa7fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa813fb50 con 0x7f1fa81422c0 2022-07-21T01:52:13.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa7fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8115300 con 0x7f1fa80b0570 2022-07-21T01:52:13.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa81155a0 con 0x7f1fa80aea10 2022-07-21T01:52:13.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fa81422c0 0x7f1fa8112a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fa81422c0 0x7f1fa8112a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35394/0 (socket says 172.21.15.140:35394) 2022-07-21T01:52:13.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa77fe700 1 -- 172.21.15.140:0/1280967608 learned_addr learned my addr 172.21.15.140:0/1280967608 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:13.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa6ffd700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fa80aea10 0x7f1fa80b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa67fc700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa80b0570 0x7f1fa80b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa77fe700 1 -- 172.21.15.140:0/1280967608 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fa80aea10 msgr2=0x7f1fa80b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa77fe700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fa80aea10 0x7f1fa80b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa77fe700 1 -- 172.21.15.140:0/1280967608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa80b0570 msgr2=0x7f1fa80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa77fe700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa80b0570 0x7f1fa80b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa77fe700 1 -- 172.21.15.140:0/1280967608 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9800b180 con 0x7f1fa81422c0 2022-07-21T01:52:13.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1fa77fe700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fa81422c0 0x7f1fa8112a80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f1fa0001250 tx=0x7f1fa0001380).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:13.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.775+0000 7f1f97fff700 1 -- 172.21.15.140:0/1280967608 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fa0005ba0 con 0x7f1fa81422c0 2022-07-21T01:52:13.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.779+0000 7f1f97fff700 1 -- 172.21.15.140:0/1280967608 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0005d00 con 0x7f1fa81422c0 2022-07-21T01:52:13.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.779+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1280967608 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa8115840 con 0x7f1fa81422c0 2022-07-21T01:52:13.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.779+0000 7f1f97fff700 1 -- 172.21.15.140:0/1280967608 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fa0004a90 con 0x7f1fa81422c0 2022-07-21T01:52:13.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.779+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1280967608 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa80b4c90 con 0x7f1fa81422c0 2022-07-21T01:52:13.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.779+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1280967608 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa81159d0 con 0x7f1fa81422c0 2022-07-21T01:52:13.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.779+0000 7f1f97fff700 1 -- 172.21.15.140:0/1280967608 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1fa0004bf0 con 0x7f1fa81422c0 2022-07-21T01:52:13.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.779+0000 7f1f97fff700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f84053b00 0x7f1f84055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:13.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.779+0000 7f1f97fff700 1 -- 172.21.15.140:0/1280967608 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1fa006ba00 con 0x7f1fa81422c0 2022-07-21T01:52:13.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.779+0000 7f1fa6ffd700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f84053b00 0x7f1f84055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:13.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.783+0000 7f1fa6ffd700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f84053b00 0x7f1f84055fc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f1f9801c000 tx=0x7f1f98015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:13.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.787+0000 7f1f97fff700 1 -- 172.21.15.140:0/1280967608 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1fa0042920 con 0x7f1fa81422c0 2022-07-21T01:52:13.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.943+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1280967608 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fa80b1010 con 0x7f1fa81422c0 2022-07-21T01:52:13.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1f97fff700 1 -- 172.21.15.140:0/1280967608 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2145 (secure 0 0 0) 0x7f1fa0045f40 con 0x7f1fa81422c0 2022-07-21T01:52:13.945 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:52:13.945 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1915503334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1915503334},{"type":"v1","addr":"172.21.15.140:6835","nonce":1915503334}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:51:56.500368+0000","modified":"2022-07-21T01:51:57.504176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5204":{"gid":5204,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3908541795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3908541795},{"type":"v1","addr":"172.21.15.152:6833","nonce":3908541795}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-07-21T01:52:13.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1280967608 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f84053b00 msgr2=0x7f1f84055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f84053b00 0x7f1f84055fc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f1f9801c000 tx=0x7f1f98015000).stop 2022-07-21T01:52:13.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1280967608 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fa81422c0 msgr2=0x7f1fa8112a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:13.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fa81422c0 0x7f1fa8112a80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f1fa0001250 tx=0x7f1fa0001380).stop 2022-07-21T01:52:13.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1280967608 shutdown_connections 2022-07-21T01:52:13.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f84053b00 0x7f1f84055fc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fa81422c0 0x7f1fa8112a80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fa80b0570 0x7f1fa80b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 --2- 172.21.15.140:0/1280967608 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fa80aea10 0x7f1fa80b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:13.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1280967608 >> 172.21.15.140:0/1280967608 conn(0x7f1fa80b9360 msgr2=0x7f1fa80b7500 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:13.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1280967608 shutdown_connections 2022-07-21T01:52:13.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:13.947+0000 7f1fa7fff700 1 -- 172.21.15.140:0/1280967608 wait complete. 2022-07-21T01:52:13.950 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 21 2022-07-21T01:52:13.958 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b cache drop 5 2022-07-21T01:52:14.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc3fff700 1 Processor -- start 2022-07-21T01:52:14.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc3fff700 1 -- start start 2022-07-21T01:52:14.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc3fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7effc40b1d40 0x7effc40ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc3fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc40ac5e0 con 0x7effc413f7d0 2022-07-21T01:52:14.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc3fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc40b2120 con 0x7effc40ac7e0 2022-07-21T01:52:14.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc3fff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc4060830 con 0x7effc40b1d40 2022-07-21T01:52:14.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc27fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7effc413f7d0 0x7effc40ab7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49546/0 (socket says 172.21.15.140:49546) 2022-07-21T01:52:14.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc27fc700 1 -- 172.21.15.140:0/668685407 learned_addr learned my addr 172.21.15.140:0/668685407 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:52:14.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc2ffd700 1 --2- 172.21.15.140:0/668685407 >> v2:172.21.15.152:3300/0 conn(0x7effc40b1d40 0x7effc40ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365141711 0 0) 0x7effc40ac5e0 con 0x7effc413f7d0 2022-07-21T01:52:14.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa4003260 con 0x7effc413f7d0 2022-07-21T01:52:14.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926909833 0 0) 0x7effa4003260 con 0x7effc413f7d0 2022-07-21T01:52:14.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 >> v2:172.21.15.152:3300/0 conn(0x7effc40b1d40 msgr2=0x7effc40ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:14.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 --2- 172.21.15.140:0/668685407 >> v2:172.21.15.152:3300/0 conn(0x7effc40b1d40 0x7effc40ab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 >> v1:172.21.15.152:6789/0 conn(0x7effc40ac7e0 legacy=0x7effc413e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:14.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc4060ad0 con 0x7effc413f7d0 2022-07-21T01:52:14.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7effac001660 con 0x7effc413f7d0 2022-07-21T01:52:14.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 >> v1:172.21.15.140:6789/0 conn(0x7effc413f7d0 legacy=0x7effc40ab7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:14.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 --2- 172.21.15.140:0/668685407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effa4003750 0x7effa4003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 --2- 172.21.15.140:0/668685407 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effa40040b0 0x7effa4004580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 --2- 172.21.15.140:0/668685407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effa4004d30 0x7effa400d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.071+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc4060830 con 0x7effa4003750 2022-07-21T01:52:14.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa400ded0 con 0x7effa4004d30 2022-07-21T01:52:14.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc2ffd700 1 --2- 172.21.15.140:0/668685407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effa4003750 0x7effa4003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa400e090 con 0x7effa40040b0 2022-07-21T01:52:14.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc27fc700 1 --2- 172.21.15.140:0/668685407 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effa40040b0 0x7effa4004580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc37fe700 1 --2- 172.21.15.140:0/668685407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effa4004d30 0x7effa400d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc2ffd700 1 -- 172.21.15.140:0/668685407 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effa40040b0 msgr2=0x7effa4004580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:14.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc2ffd700 1 --2- 172.21.15.140:0/668685407 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effa40040b0 0x7effa4004580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc2ffd700 1 -- 172.21.15.140:0/668685407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effa4004d30 msgr2=0x7effa400d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:14.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc2ffd700 1 --2- 172.21.15.140:0/668685407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effa4004d30 0x7effa400d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc2ffd700 1 -- 172.21.15.140:0/668685407 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa4003260 con 0x7effa4003750 2022-07-21T01:52:14.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc37fe700 1 -- 172.21.15.140:0/668685407 reap_dead start 2022-07-21T01:52:14.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc2ffd700 1 --2- 172.21.15.140:0/668685407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effa4003750 0x7effa4003b30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7effb400c000 tx=0x7effb4006d90).ready entity=mon.0 client_cookie=a709d88252a5d41b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:14.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb4013450 con 0x7effa4003750 2022-07-21T01:52:14.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb40135b0 con 0x7effa4003750 2022-07-21T01:52:14.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc1ffb700 1 -- 172.21.15.140:0/668685407 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb4013910 con 0x7effa4003750 2022-07-21T01:52:14.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 -- 172.21.15.140:0/668685407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effa4003750 msgr2=0x7effa4003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:14.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 --2- 172.21.15.140:0/668685407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effa4003750 0x7effa4003b30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7effb400c000 tx=0x7effb4006d90).stop 2022-07-21T01:52:14.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 -- 172.21.15.140:0/668685407 shutdown_connections 2022-07-21T01:52:14.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 --2- 172.21.15.140:0/668685407 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effa4003750 0x7effa4003b30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 -- 172.21.15.140:0/668685407 >> 172.21.15.140:0/668685407 conn(0x7effc40b2af0 msgr2=0x7effc40b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:14.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 -- 172.21.15.140:0/668685407 shutdown_connections 2022-07-21T01:52:14.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 -- 172.21.15.140:0/668685407 wait complete. 2022-07-21T01:52:14.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 Processor -- start 2022-07-21T01:52:14.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 -- start start 2022-07-21T01:52:14.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effa4003750 0x7effc4063a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effc409b3f0 0x7effc40986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effc409a170 0x7effc409a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc409c230 con 0x7effc409b3f0 2022-07-21T01:52:14.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc409b900 con 0x7effa4003750 2022-07-21T01:52:14.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.075+0000 7effc3fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc4156c60 con 0x7effc409a170 2022-07-21T01:52:14.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc27fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effc409b3f0 0x7effc40986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc27fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effc409b3f0 0x7effc40986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35412/0 (socket says 172.21.15.140:35412) 2022-07-21T01:52:14.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc27fc700 1 -- 172.21.15.140:0/2676737913 learned_addr learned my addr 172.21.15.140:0/2676737913 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:14.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc2ffd700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effa4003750 0x7effc4063a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc37fe700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effc409a170 0x7effc409a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc27fc700 1 -- 172.21.15.140:0/2676737913 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effc409a170 msgr2=0x7effc409a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:14.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc27fc700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effc409a170 0x7effc409a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc27fc700 1 -- 172.21.15.140:0/2676737913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effa4003750 msgr2=0x7effc4063a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:14.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc27fc700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effa4003750 0x7effc4063a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc27fc700 1 -- 172.21.15.140:0/2676737913 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc4156960 con 0x7effc409b3f0 2022-07-21T01:52:14.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc27fc700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effc409b3f0 0x7effc40986f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7effac009000 tx=0x7effc40615d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:14.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effbbfff700 1 -- 172.21.15.140:0/2676737913 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effac010510 con 0x7effc409b3f0 2022-07-21T01:52:14.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effbbfff700 1 -- 172.21.15.140:0/2676737913 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac00e030 con 0x7effc409b3f0 2022-07-21T01:52:14.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc3fff700 1 -- 172.21.15.140:0/2676737913 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc409aae0 con 0x7effc409b3f0 2022-07-21T01:52:14.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effbbfff700 1 -- 172.21.15.140:0/2676737913 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effac010860 con 0x7effc409b3f0 2022-07-21T01:52:14.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effc3fff700 1 -- 172.21.15.140:0/2676737913 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc4095980 con 0x7effc409b3f0 2022-07-21T01:52:14.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.079+0000 7effbbfff700 1 -- 172.21.15.140:0/2676737913 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7effac0109c0 con 0x7effc409b3f0 2022-07-21T01:52:14.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effbbfff700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7effa8053a70 0x7effa8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effbbfff700 1 -- 172.21.15.140:0/2676737913 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7effac06c300 con 0x7effc409b3f0 2022-07-21T01:52:14.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effc2ffd700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7effa8053a70 0x7effa8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effc2ffd700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7effa8053a70 0x7effa8055f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7effb401b000 tx=0x7effb4017000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:14.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effd3a45740 1 Processor -- start 2022-07-21T01:52:14.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effd3a45740 1 -- start start 2022-07-21T01:52:14.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effd3a45740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe24c30 0xe26740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effd3a45740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe241b0 0xe27330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effd3a45740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe1f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effd3a45740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0xd95210 con 0xe25010 2022-07-21T01:52:14.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effd3a45740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0xd75450 con 0xe24c30 2022-07-21T01:52:14.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.083+0000 7effd3a45740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0xe21430 con 0xe241b0 2022-07-21T01:52:14.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe1f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe1f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35422/0 (socket says 172.21.15.140:35422) 2022-07-21T01:52:14.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc37fe700 1 -- 172.21.15.140:0/2262209860 learned_addr learned my addr 172.21.15.140:0/2262209860 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:14.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc2ffd700 1 --2- 172.21.15.140:0/2262209860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe24c30 0xe26740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc27fc700 1 --2- 172.21.15.140:0/2262209860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe241b0 0xe27330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc37fe700 1 -- 172.21.15.140:0/2262209860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe241b0 msgr2=0xe27330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:14.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc37fe700 1 --2- 172.21.15.140:0/2262209860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe241b0 0xe27330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc37fe700 1 -- 172.21.15.140:0/2262209860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe24c30 msgr2=0xe26740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:14.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc37fe700 1 --2- 172.21.15.140:0/2262209860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe24c30 0xe26740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc37fe700 1 -- 172.21.15.140:0/2262209860 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xdccad0 con 0xe25010 2022-07-21T01:52:14.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc37fe700 1 --2- 172.21.15.140:0/2262209860 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe1f680 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0xe24590 tx=0x7effbc009040).ready entity=mon.0 client_cookie=b3665db43d8d38e3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:14.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc3fff700 1 -- 172.21.15.140:0/2262209860 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effbc0032c0 con 0xe25010 2022-07-21T01:52:14.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc3fff700 1 -- 172.21.15.140:0/2262209860 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc003420 con 0xe25010 2022-07-21T01:52:14.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effc3fff700 1 -- 172.21.15.140:0/2262209860 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effbc001770 con 0xe25010 2022-07-21T01:52:14.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effd3a45740 1 -- 172.21.15.140:0/2262209860 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 msgr2=0xe1f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:14.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effd3a45740 1 --2- 172.21.15.140:0/2262209860 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe1f680 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0xe24590 tx=0x7effbc009040).stop 2022-07-21T01:52:14.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effd3a45740 1 -- 172.21.15.140:0/2262209860 shutdown_connections 2022-07-21T01:52:14.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effd3a45740 1 --2- 172.21.15.140:0/2262209860 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe1f680 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effd3a45740 1 --2- 172.21.15.140:0/2262209860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe241b0 0xe27330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effd3a45740 1 --2- 172.21.15.140:0/2262209860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe24c30 0xe26740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effd3a45740 1 -- 172.21.15.140:0/2262209860 >> 172.21.15.140:0/2262209860 conn(0xdac8f0 msgr2=0xe22f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:14.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effd3a45740 1 -- 172.21.15.140:0/2262209860 shutdown_connections 2022-07-21T01:52:14.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effd3a45740 1 -- 172.21.15.140:0/2262209860 wait complete. 2022-07-21T01:52:14.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effd3a45740 1 -- start start 2022-07-21T01:52:14.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.087+0000 7effd3a45740 1 Processor -- start 2022-07-21T01:52:14.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe241b0 0xe1fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe24c30 0xe2bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe25010 0xe5a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0xe2d0c0 con 0xe24c30 2022-07-21T01:52:14.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0xe2e9a0 con 0xe241b0 2022-07-21T01:52:14.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0xe2e700 con 0xe25010 2022-07-21T01:52:14.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc27fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe24c30 0xe2bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 10 client.0 ll_register_callbacks cb 0xddc0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-07-21T01:52:14.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc27fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe24c30 0xe2bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35426/0 (socket says 172.21.15.140:35426) 2022-07-21T01:52:14.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc27fc700 1 -- 172.21.15.140:0/1871027434 learned_addr learned my addr 172.21.15.140:0/1871027434 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:14.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc2ffd700 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe241b0 0xe1fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc37fe700 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe25010 0xe5a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc27fc700 1 -- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe25010 msgr2=0xe5a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:14.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc27fc700 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe25010 0xe5a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc27fc700 1 -- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe241b0 msgr2=0xe1fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:14.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc27fc700 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe241b0 0xe1fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc27fc700 1 -- 172.21.15.140:0/1871027434 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc00f040 con 0xe24c30 2022-07-21T01:52:14.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc27fc700 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe24c30 0xe2bae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7effac027000 tx=0x7effac00a000).ready entity=mon.0 client_cookie=2615a12d13bb485f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:14.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effa1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.140:3300/0 2022-07-21T01:52:14.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effa1ffb700 1 -- 172.21.15.140:0/1871027434 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effac046550 con 0xe24c30 2022-07-21T01:52:14.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effa1ffb700 1 -- 172.21.15.140:0/1871027434 <== mon.0 v2:172.21.15.140:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7effac06c0d0 con 0xe24c30 2022-07-21T01:52:14.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 1 -- 172.21.15.140:0/1871027434 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xd8b1c0 con 0xe24c30 2022-07-21T01:52:14.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effa1ffb700 1 -- 172.21.15.140:0/1871027434 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7effac042bc0 con 0xe24c30 2022-07-21T01:52:14.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 10 client.5823 fetch_fsmap learned FSMap version 21 2022-07-21T01:52:14.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 1 -- 172.21.15.140:0/1871027434 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0xd847b0 con 0xe24c30 2022-07-21T01:52:14.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effa1ffb700 1 -- 172.21.15.140:0/1871027434 <== mon.0 v2:172.21.15.140:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xd8b1c0 con 0xe24c30 2022-07-21T01:52:14.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effa1ffb700 1 -- 172.21.15.140:0/1871027434 <== mon.0 v2:172.21.15.140:3300/0 5 ==== fsmap(e 21) v1 ==== 1247+0+0 (secure 0 0 0) 0x7effac029040 con 0xe24c30 2022-07-21T01:52:14.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 10 client.5823 fetch_fsmap finished waiting for FSMap version 21 2022-07-21T01:52:14.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 10 client.5823 resolve_mds: resolved ID 'b' to GID 5204 2022-07-21T01:52:14.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 20 client.5823 populate_metadata read hostname 'smithi140' 2022-07-21T01:52:14.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:6832/3908541795,v1:172.21.15.152:6833/3908541795] conn(0xe20310 0xe5aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 4 client.5823 mds_command: new command op to 5204 tid=0[{"prefix": "get_command_descriptions"}] 2022-07-21T01:52:14.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effd3a45740 1 -- 172.21.15.140:0/1871027434 --> [v2:172.21.15.152:6832/3908541795,v1:172.21.15.152:6833/3908541795] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xd83240 con 0xe20310 2022-07-21T01:52:14.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc2ffd700 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:6832/3908541795,v1:172.21.15.152:6833/3908541795] conn(0xe20310 0xe5aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effa1ffb700 10 client.5823 ms_handle_connect on v2:172.21.15.152:6832/3908541795 2022-07-21T01:52:14.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.091+0000 7effc2ffd700 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:6832/3908541795,v1:172.21.15.152:6833/3908541795] conn(0xe20310 0xe5aef0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=bcd2972bd83e0598 server_cookie=b478babdf27b0f1 in_seq=0 out_seq=0 2022-07-21T01:52:14.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effa1ffb700 1 -- 172.21.15.140:0/1871027434 <== mds.0 v2:172.21.15.152:6832/3908541795 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0xd83240 con 0xe20310 2022-07-21T01:52:14.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effa1ffb700 10 client.5823 handle_command_reply: tid=0 2022-07-21T01:52:14.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 1 client.5823 shutdown 2022-07-21T01:52:14.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 1 -- 172.21.15.140:0/1871027434 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe24c30 msgr2=0xe2bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:14.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe24c30 0xe2bae0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7effac027000 tx=0x7effac00a000).stop 2022-07-21T01:52:14.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 1 -- 172.21.15.140:0/1871027434 shutdown_connections 2022-07-21T01:52:14.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:6832/3908541795,v1:172.21.15.152:6833/3908541795] conn(0xe20310 0xe5aef0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe25010 0xe5a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effa1ffb700 0 client.5823 ms_handle_reset on v2:172.21.15.152:6832/3908541795 2022-07-21T01:52:14.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe24c30 0xe2bae0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 1 --2- 172.21.15.140:0/1871027434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe241b0 0xe1fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 1 -- 172.21.15.140:0/1871027434 >> 172.21.15.140:0/1871027434 conn(0xdac8f0 msgr2=0xe26660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:14.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 1 -- 172.21.15.140:0/1871027434 shutdown_connections 2022-07-21T01:52:14.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 1 -- 172.21.15.140:0/1871027434 wait complete. 2022-07-21T01:52:14.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.095+0000 7effd3a45740 20 client.5823 trim_cache size 0 max 16384 2022-07-21T01:52:14.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 Processor -- start 2022-07-21T01:52:14.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 -- start start 2022-07-21T01:52:14.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe241b0 0xe26660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe24c30 0xe2da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe2def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0xe23cc0 con 0xe25010 2022-07-21T01:52:14.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0xd964f0 con 0xe241b0 2022-07-21T01:52:14.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0xda6f10 con 0xe24c30 2022-07-21T01:52:14.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe2def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe2def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35436/0 (socket says 172.21.15.140:35436) 2022-07-21T01:52:14.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc37fe700 1 -- 172.21.15.140:0/3419020803 learned_addr learned my addr 172.21.15.140:0/3419020803 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:14.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc27fc700 1 --2- 172.21.15.140:0/3419020803 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe24c30 0xe2da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc2ffd700 1 --2- 172.21.15.140:0/3419020803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe241b0 0xe26660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc37fe700 1 -- 172.21.15.140:0/3419020803 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe24c30 msgr2=0xe2da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:14.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc37fe700 1 --2- 172.21.15.140:0/3419020803 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe24c30 0xe2da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc37fe700 1 -- 172.21.15.140:0/3419020803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe241b0 msgr2=0xe26660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:14.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc37fe700 1 --2- 172.21.15.140:0/3419020803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe241b0 0xe26660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc37fe700 1 -- 172.21.15.140:0/3419020803 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xd8d3a0 con 0xe25010 2022-07-21T01:52:14.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc2ffd700 1 --2- 172.21.15.140:0/3419020803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe241b0 0xe26660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:52:14.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effc37fe700 1 --2- 172.21.15.140:0/3419020803 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe2def0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7effbc01f000 tx=0xe23790).ready entity=mon.0 client_cookie=a9fe8450614a2c55 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:14.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effa3fff700 1 -- 172.21.15.140:0/3419020803 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effbc003220 con 0xe25010 2022-07-21T01:52:14.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effa3fff700 1 -- 172.21.15.140:0/3419020803 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc0014a0 con 0xe25010 2022-07-21T01:52:14.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effa3fff700 1 -- 172.21.15.140:0/3419020803 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effbc001800 con 0xe25010 2022-07-21T01:52:14.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 -- 172.21.15.140:0/3419020803 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 msgr2=0xe2def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:14.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 --2- 172.21.15.140:0/3419020803 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe2def0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7effbc01f000 tx=0xe23790).stop 2022-07-21T01:52:14.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 -- 172.21.15.140:0/3419020803 shutdown_connections 2022-07-21T01:52:14.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 --2- 172.21.15.140:0/3419020803 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe25010 0xe2def0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 --2- 172.21.15.140:0/3419020803 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe24c30 0xe2da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 --2- 172.21.15.140:0/3419020803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe241b0 0xe26660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 -- 172.21.15.140:0/3419020803 >> 172.21.15.140:0/3419020803 conn(0xdac8f0 msgr2=0xe59420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:14.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 -- 172.21.15.140:0/3419020803 shutdown_connections 2022-07-21T01:52:14.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.111+0000 7effd3a45740 1 -- 172.21.15.140:0/3419020803 wait complete. 2022-07-21T01:52:14.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 -- start start 2022-07-21T01:52:14.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 Processor -- start 2022-07-21T01:52:14.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe241b0 0xe2b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe24c30 0xeb92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe25010 0xeb97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0xe22690 con 0xe241b0 2022-07-21T01:52:14.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0xe2df10 con 0xe25010 2022-07-21T01:52:14.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0xebd4e0 con 0xe24c30 2022-07-21T01:52:14.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 10 client.0 ll_register_callbacks cb 0xe25810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-07-21T01:52:14.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc2ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe241b0 0xe2b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc2ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe241b0 0xe2b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35438/0 (socket says 172.21.15.140:35438) 2022-07-21T01:52:14.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc2ffd700 1 -- 172.21.15.140:0/2379962498 learned_addr learned my addr 172.21.15.140:0/2379962498 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:14.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc27fc700 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe24c30 0xeb92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc37fe700 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe25010 0xeb97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc2ffd700 1 -- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe24c30 msgr2=0xeb92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:14.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc2ffd700 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe24c30 0xeb92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc2ffd700 1 -- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe25010 msgr2=0xeb97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:14.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc2ffd700 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe25010 0xeb97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:14.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc2ffd700 1 -- 172.21.15.140:0/2379962498 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xdccad0 con 0xe241b0 2022-07-21T01:52:14.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc27fc700 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe24c30 0xeb92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:52:14.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc2ffd700 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe241b0 0xe2b070 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7effb4037000 tx=0x7effb4022040).ready entity=mon.0 client_cookie=8aaf4de3cb8a480f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:14.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effa2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.140:3300/0 2022-07-21T01:52:14.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effa2ffd700 1 -- 172.21.15.140:0/2379962498 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb4031040 con 0xe241b0 2022-07-21T01:52:14.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effa2ffd700 1 -- 172.21.15.140:0/2379962498 <== mon.0 v2:172.21.15.140:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7effb402e870 con 0xe241b0 2022-07-21T01:52:14.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 -- 172.21.15.140:0/2379962498 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effbc003420 con 0xe241b0 2022-07-21T01:52:14.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effa2ffd700 1 -- 172.21.15.140:0/2379962498 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7effb402ea30 con 0xe241b0 2022-07-21T01:52:14.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 10 client.5829 fetch_fsmap learned FSMap version 21 2022-07-21T01:52:14.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 -- 172.21.15.140:0/2379962498 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0xd89d90 con 0xe241b0 2022-07-21T01:52:14.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effa2ffd700 1 -- 172.21.15.140:0/2379962498 <== mon.0 v2:172.21.15.140:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc003420 con 0xe241b0 2022-07-21T01:52:14.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effa2ffd700 1 -- 172.21.15.140:0/2379962498 <== mon.0 v2:172.21.15.140:3300/0 5 ==== fsmap(e 21) v1 ==== 1247+0+0 (secure 0 0 0) 0x7effb401d040 con 0xe241b0 2022-07-21T01:52:14.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 10 client.5829 fetch_fsmap finished waiting for FSMap version 21 2022-07-21T01:52:14.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 10 client.5829 resolve_mds: resolved ID 'b' to GID 5204 2022-07-21T01:52:14.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 20 client.5829 populate_metadata read hostname 'smithi140' 2022-07-21T01:52:14.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:6832/3908541795,v1:172.21.15.152:6833/3908541795] conn(0xe2bbb0 0xe69760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:14.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 4 client.5829 mds_command: new command op to 5204 tid=0[{"prefix": "cache drop", "timeout": 5}] 2022-07-21T01:52:14.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effd3a45740 1 -- 172.21.15.140:0/2379962498 --> [v2:172.21.15.152:6832/3908541795,v1:172.21.15.152:6833/3908541795] -- command(tid 0: {"prefix": "cache drop", "timeout": 5}) v1 -- 0xc785e0 con 0xe2bbb0 2022-07-21T01:52:14.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.115+0000 7effc27fc700 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:6832/3908541795,v1:172.21.15.152:6833/3908541795] conn(0xe2bbb0 0xe69760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:14.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.119+0000 7effa2ffd700 10 client.5829 ms_handle_connect on v2:172.21.15.152:6832/3908541795 2022-07-21T01:52:14.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:14.119+0000 7effc27fc700 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:6832/3908541795,v1:172.21.15.152:6833/3908541795] conn(0xe2bbb0 0xe69760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a5496735d70aecb2 server_cookie=b252a379558d31ba in_seq=0 out_seq=0 2022-07-21T01:52:19.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effa2ffd700 1 -- 172.21.15.140:0/2379962498 <== mds.0 v2:172.21.15.152:6832/3908541795 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+404 (crc 0 0 0) 0xc785e0 con 0xe2bbb0 2022-07-21T01:52:19.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effa2ffd700 10 client.5829 handle_command_reply: tid=0 2022-07-21T01:52:19.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effd3a45740 1 client.5829 shutdown 2022-07-21T01:52:19.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effd3a45740 1 -- 172.21.15.140:0/2379962498 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe241b0 msgr2=0xe2b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effd3a45740 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe241b0 0xe2b070 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7effb4037000 tx=0x7effb4022040).stop 2022-07-21T01:52:19.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effd3a45740 1 -- 172.21.15.140:0/2379962498 shutdown_connections 2022-07-21T01:52:19.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effd3a45740 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:6832/3908541795,v1:172.21.15.152:6833/3908541795] conn(0xe2bbb0 0xe69760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effd3a45740 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0xe25010 0xeb97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effd3a45740 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0xe24c30 0xeb92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effd3a45740 1 --2- 172.21.15.140:0/2379962498 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0xe241b0 0xe2b070 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effd3a45740 1 -- 172.21.15.140:0/2379962498 >> 172.21.15.140:0/2379962498 conn(0xdac8f0 msgr2=0xe22d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:19.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effa2ffd700 0 client.5829 ms_handle_reset on v2:172.21.15.152:6832/3908541795 2022-07-21T01:52:19.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effd3a45740 1 -- 172.21.15.140:0/2379962498 shutdown_connections 2022-07-21T01:52:19.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.287+0000 7effd3a45740 1 -- 172.21.15.140:0/2379962498 wait complete. 2022-07-21T01:52:19.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effd3a45740 20 client.5829 trim_cache size 0 max 16384 2022-07-21T01:52:19.293 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:52:19.293 INFO:teuthology.orchestra.run.smithi140.stdout: "client_recall": { 2022-07-21T01:52:19.293 INFO:teuthology.orchestra.run.smithi140.stdout: "return_code": -110, 2022-07-21T01:52:19.294 INFO:teuthology.orchestra.run.smithi140.stdout: "message": "(110) Connection timed out", 2022-07-21T01:52:19.294 INFO:teuthology.orchestra.run.smithi140.stdout: "recalled": 120 2022-07-21T01:52:19.294 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:52:19.294 INFO:teuthology.orchestra.run.smithi140.stdout: "flush_journal": { 2022-07-21T01:52:19.294 INFO:teuthology.orchestra.run.smithi140.stdout: "return_code": 0, 2022-07-21T01:52:19.295 INFO:teuthology.orchestra.run.smithi140.stdout: "message": "" 2022-07-21T01:52:19.295 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:52:19.295 INFO:teuthology.orchestra.run.smithi140.stdout: "trim_cache": { 2022-07-21T01:52:19.295 INFO:teuthology.orchestra.run.smithi140.stdout: "trimmed": 0 2022-07-21T01:52:19.296 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:52:19.296 INFO:teuthology.orchestra.run.smithi140.stdout: "cache": { 2022-07-21T01:52:19.296 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": { 2022-07-21T01:52:19.296 INFO:teuthology.orchestra.run.smithi140.stdout: "items": 20102, 2022-07-21T01:52:19.297 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes": 3106848 2022-07-21T01:52:19.297 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:52:19.297 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:52:19.297 INFO:teuthology.orchestra.run.smithi140.stdout: "duration": 5.1705619079999998 2022-07-21T01:52:19.297 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:52:19.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 -- 172.21.15.140:0/2676737913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7effa8053a70 msgr2=0x7effa8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7effa8053a70 0x7effa8055f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7effb401b000 tx=0x7effb4017000).stop 2022-07-21T01:52:19.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 -- 172.21.15.140:0/2676737913 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effc409b3f0 msgr2=0x7effc40986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effc409b3f0 0x7effc40986f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7effac009000 tx=0x7effc40615d0).stop 2022-07-21T01:52:19.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 -- 172.21.15.140:0/2676737913 shutdown_connections 2022-07-21T01:52:19.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7effa8053a70 0x7effa8055f30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effc409a170 0x7effc409a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7effc409b3f0 0x7effc40986f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 --2- 172.21.15.140:0/2676737913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effa4003750 0x7effc4063a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 -- 172.21.15.140:0/2676737913 >> 172.21.15.140:0/2676737913 conn(0x7effc40b2af0 msgr2=0x7effc413ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:19.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 -- 172.21.15.140:0/2676737913 shutdown_connections 2022-07-21T01:52:19.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.291+0000 7effc3fff700 1 -- 172.21.15.140:0/2676737913 wait complete. 2022-07-21T01:52:19.305 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-07-21T01:52:19.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:19.306 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:52:19.306 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:19.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:52:19.316 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-07-21T01:52:19.317 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-07-21T01:52:19.317 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:52:19.318 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:52:19.318 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:19.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:19.338 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:52:19.339 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:19.345 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:52:19.346 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:52:19.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:52:19.347 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:19.394 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:52:19.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f50792be700 1 Processor -- start 2022-07-21T01:52:19.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f50792be700 1 -- start start 2022-07-21T01:52:19.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f50792be700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f50740b1ef0 0x7f50740605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f50792be700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50740ab180 0x7f50740ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f50792be700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5074140940 con 0x7f507413f980 2022-07-21T01:52:19.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f50792be700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50740b22d0 con 0x7f50740b1ef0 2022-07-21T01:52:19.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f50792be700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507405fbc0 con 0x7f50740ab180 2022-07-21T01:52:19.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f5073fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f50740b1ef0 0x7f50740605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f50737fe700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f507413f980 0x7f50740abff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:50706/0 (socket says 172.21.15.140:50706) 2022-07-21T01:52:19.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f5073fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f50740b1ef0 0x7f50740605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:41188/0 (socket says 172.21.15.140:41188) 2022-07-21T01:52:19.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f50737fe700 1 -- 172.21.15.140:0/3914803256 learned_addr learned my addr 172.21.15.140:0/3914803256 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:52:19.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.503+0000 7f5078abd700 1 --2- 172.21.15.140:0/3914803256 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50740ab180 0x7f50740ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f5072ffd700 1 -- 172.21.15.140:0/3914803256 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790167898 0 0) 0x7f5074140940 con 0x7f507413f980 2022-07-21T01:52:19.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f5072ffd700 1 -- 172.21.15.140:0/3914803256 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5060003260 con 0x7f507413f980 2022-07-21T01:52:19.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f5078abd700 1 -- 172.21.15.140:0/3914803256 >> v2:172.21.15.152:3300/0 conn(0x7f50740b1ef0 msgr2=0x7f50740605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:19.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f5078abd700 1 --2- 172.21.15.140:0/3914803256 >> v2:172.21.15.152:3300/0 conn(0x7f50740b1ef0 0x7f50740605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f5078abd700 1 -- 172.21.15.140:0/3914803256 >> v1:172.21.15.152:6789/0 conn(0x7f507413f980 legacy=0x7f50740abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f5078abd700 1 -- 172.21.15.140:0/3914803256 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507405fe60 con 0x7f50740ab180 2022-07-21T01:52:19.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f5073fff700 1 --2- 172.21.15.140:0/3914803256 >> v2:172.21.15.152:3300/0 conn(0x7f50740b1ef0 0x7f50740605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:52:19.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f5078abd700 1 --2- 172.21.15.140:0/3914803256 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50740ab180 0x7f50740ab5d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f506c0032f0 tx=0x7f506c0091d0).ready entity=mon.2 client_cookie=697d27e00996db33 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:19.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f5072ffd700 1 -- 172.21.15.140:0/3914803256 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f506c00b600 con 0x7f50740ab180 2022-07-21T01:52:19.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f5072ffd700 1 -- 172.21.15.140:0/3914803256 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c00b760 con 0x7f50740ab180 2022-07-21T01:52:19.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f5072ffd700 1 -- 172.21.15.140:0/3914803256 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f506c00b8d0 con 0x7f50740ab180 2022-07-21T01:52:19.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f50792be700 1 -- 172.21.15.140:0/3914803256 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50740ab180 msgr2=0x7f50740ab5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f50792be700 1 --2- 172.21.15.140:0/3914803256 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50740ab180 0x7f50740ab5d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f506c0032f0 tx=0x7f506c0091d0).stop 2022-07-21T01:52:19.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f50792be700 1 -- 172.21.15.140:0/3914803256 shutdown_connections 2022-07-21T01:52:19.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f50792be700 1 --2- 172.21.15.140:0/3914803256 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50740ab180 0x7f50740ab5d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f50792be700 1 --2- 172.21.15.140:0/3914803256 >> v2:172.21.15.152:3300/0 conn(0x7f50740b1ef0 0x7f50740605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f50792be700 1 -- 172.21.15.140:0/3914803256 >> 172.21.15.140:0/3914803256 conn(0x7f50740b2ca0 msgr2=0x7f50740b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:19.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f50792be700 1 -- 172.21.15.140:0/3914803256 shutdown_connections 2022-07-21T01:52:19.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f50792be700 1 -- 172.21.15.140:0/3914803256 wait complete. 2022-07-21T01:52:19.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.507+0000 7f50792be700 1 Processor -- start 2022-07-21T01:52:19.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50792be700 1 -- start start 2022-07-21T01:52:19.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50792be700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50740ab180 0x7f50740ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50792be700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50740b1ef0 0x7f5074099c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50792be700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f507413f980 0x7f50740ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50792be700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507412f120 con 0x7f50740ab180 2022-07-21T01:52:19.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50792be700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50740b0030 con 0x7f507413f980 2022-07-21T01:52:19.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50792be700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507413d090 con 0x7f50740b1ef0 2022-07-21T01:52:19.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f5073fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50740ab180 0x7f50740ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f5073fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50740ab180 0x7f50740ad640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35452/0 (socket says 172.21.15.140:35452) 2022-07-21T01:52:19.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f5073fff700 1 -- 172.21.15.140:0/2771988850 learned_addr learned my addr 172.21.15.140:0/2771988850 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:19.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50737fe700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50740b1ef0 0x7f5074099c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f5078abd700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f507413f980 0x7f50740ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f5073fff700 1 -- 172.21.15.140:0/2771988850 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50740b1ef0 msgr2=0x7f5074099c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f5073fff700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50740b1ef0 0x7f5074099c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f5073fff700 1 -- 172.21.15.140:0/2771988850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f507413f980 msgr2=0x7f50740ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f5073fff700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f507413f980 0x7f50740ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f5073fff700 1 -- 172.21.15.140:0/2771988850 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506800b180 con 0x7f50740ab180 2022-07-21T01:52:19.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f5073fff700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50740ab180 0x7f50740ad640 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f506800c000 tx=0x7f5068006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:19.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50717fa700 1 -- 172.21.15.140:0/2771988850 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f506800c7e0 con 0x7f50740ab180 2022-07-21T01:52:19.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50717fa700 1 -- 172.21.15.140:0/2771988850 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506800c940 con 0x7f50740ab180 2022-07-21T01:52:19.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50792be700 1 -- 172.21.15.140:0/2771988850 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506c009470 con 0x7f50740ab180 2022-07-21T01:52:19.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50717fa700 1 -- 172.21.15.140:0/2771988850 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f506800cca0 con 0x7f50740ab180 2022-07-21T01:52:19.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.511+0000 7f50792be700 1 -- 172.21.15.140:0/2771988850 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5074142940 con 0x7f50740ab180 2022-07-21T01:52:19.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.515+0000 7f50792be700 1 -- 172.21.15.140:0/2771988850 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50740ae7a0 con 0x7f50740ab180 2022-07-21T01:52:19.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.515+0000 7f50717fa700 1 -- 172.21.15.140:0/2771988850 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f506801c460 con 0x7f50740ab180 2022-07-21T01:52:19.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.515+0000 7f50717fa700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5054053950 0x7f5054055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.515+0000 7f50737fe700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5054053950 0x7f5054055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.515+0000 7f50717fa700 1 -- 172.21.15.140:0/2771988850 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f506806c590 con 0x7f50740ab180 2022-07-21T01:52:19.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.515+0000 7f50737fe700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5054053950 0x7f5054055e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f506400d000 tx=0x7f5064009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:19.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.519+0000 7f50717fa700 1 -- 172.21.15.140:0/2771988850 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5068043340 con 0x7f50740ab180 2022-07-21T01:52:19.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.647+0000 7f50792be700 1 -- 172.21.15.140:0/2771988850 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f507409a2d0 con 0x7f50740ab180 2022-07-21T01:52:19.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.651+0000 7f50717fa700 1 -- 172.21.15.140:0/2771988850 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16136 (secure 0 0 0) 0x7f506800ce00 con 0x7f50740ab180 2022-07-21T01:52:19.650 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:52:19.650 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":53,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:51:58.506065+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:51:54.695165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2022-07-21T01:51:55.500555+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:52:19.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 -- 172.21.15.140:0/2771988850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5054053950 msgr2=0x7f5054055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5054053950 0x7f5054055e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f506400d000 tx=0x7f5064009040).stop 2022-07-21T01:52:19.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 -- 172.21.15.140:0/2771988850 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50740ab180 msgr2=0x7f50740ad640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50740ab180 0x7f50740ad640 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f506800c000 tx=0x7f5068006df0).stop 2022-07-21T01:52:19.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 -- 172.21.15.140:0/2771988850 shutdown_connections 2022-07-21T01:52:19.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5054053950 0x7f5054055e10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f507413f980 0x7f50740ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50740b1ef0 0x7f5074099c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 --2- 172.21.15.140:0/2771988850 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50740ab180 0x7f50740ad640 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 -- 172.21.15.140:0/2771988850 >> 172.21.15.140:0/2771988850 conn(0x7f50740b2ca0 msgr2=0x7f507405c600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:19.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 -- 172.21.15.140:0/2771988850 shutdown_connections 2022-07-21T01:52:19.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.655+0000 7f50792be700 1 -- 172.21.15.140:0/2771988850 wait complete. 2022-07-21T01:52:19.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:51:54.695165+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:51:55.500555+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:52:19.672 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:52:19.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.775+0000 7f4371730700 1 Processor -- start 2022-07-21T01:52:19.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.775+0000 7f4371730700 1 -- start start 2022-07-21T01:52:19.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f436c0b59f0 0x7f436c05c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f436c0addd0 0x7f436c142d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436c05d2f0 con 0x7f436c0af940 2022-07-21T01:52:19.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436c05ccd0 con 0x7f436c0b59f0 2022-07-21T01:52:19.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c146350 con 0x7f436c0addd0 2022-07-21T01:52:19.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4370f2f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f436c0addd0 0x7f436c142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f436bfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f436c0b59f0 0x7f436c05c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f436b7fe700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f436c0af940 0x7f436c05c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:50722/0 (socket says 172.21.15.140:50722) 2022-07-21T01:52:19.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f436b7fe700 1 -- 172.21.15.140:0/2510453492 learned_addr learned my addr 172.21.15.140:0/2510453492 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:52:19.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f436affd700 1 -- 172.21.15.140:0/2510453492 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100005332 0 0) 0x7f436c05d2f0 con 0x7f436c0af940 2022-07-21T01:52:19.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f436affd700 1 -- 172.21.15.140:0/2510453492 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f434c003260 con 0x7f436c0af940 2022-07-21T01:52:19.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4370f2f700 1 -- 172.21.15.140:0/2510453492 >> v2:172.21.15.152:3300/0 conn(0x7f436c0b59f0 msgr2=0x7f436c05c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:19.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4370f2f700 1 --2- 172.21.15.140:0/2510453492 >> v2:172.21.15.152:3300/0 conn(0x7f436c0b59f0 0x7f436c05c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4370f2f700 1 -- 172.21.15.140:0/2510453492 >> v1:172.21.15.152:6789/0 conn(0x7f436c0af940 legacy=0x7f436c05c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4370f2f700 1 -- 172.21.15.140:0/2510453492 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c0b5dd0 con 0x7f436c0addd0 2022-07-21T01:52:19.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4370f2f700 1 --2- 172.21.15.140:0/2510453492 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f436c0addd0 0x7f436c142d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f43640032f0 tx=0x7f43640091d0).ready entity=mon.2 client_cookie=3253dd1ef6657ee2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:19.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f436affd700 1 -- 172.21.15.140:0/2510453492 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f436400b600 con 0x7f436c0addd0 2022-07-21T01:52:19.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f436affd700 1 -- 172.21.15.140:0/2510453492 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436400b760 con 0x7f436c0addd0 2022-07-21T01:52:19.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f436affd700 1 -- 172.21.15.140:0/2510453492 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f436400b8d0 con 0x7f436c0addd0 2022-07-21T01:52:19.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 -- 172.21.15.140:0/2510453492 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f436c0addd0 msgr2=0x7f436c142d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 --2- 172.21.15.140:0/2510453492 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f436c0addd0 0x7f436c142d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f43640032f0 tx=0x7f43640091d0).stop 2022-07-21T01:52:19.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 -- 172.21.15.140:0/2510453492 shutdown_connections 2022-07-21T01:52:19.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 --2- 172.21.15.140:0/2510453492 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f436c0addd0 0x7f436c142d00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 --2- 172.21.15.140:0/2510453492 >> v2:172.21.15.152:3300/0 conn(0x7f436c0b59f0 0x7f436c05c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 -- 172.21.15.140:0/2510453492 >> 172.21.15.140:0/2510453492 conn(0x7f436c063480 msgr2=0x7f436c0b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:19.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 -- 172.21.15.140:0/2510453492 shutdown_connections 2022-07-21T01:52:19.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.779+0000 7f4371730700 1 -- 172.21.15.140:0/2510453492 wait complete. 2022-07-21T01:52:19.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4371730700 1 Processor -- start 2022-07-21T01:52:19.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4371730700 1 -- start start 2022-07-21T01:52:19.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4371730700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f436c0addd0 0x7f436c151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4371730700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f436c0af940 0x7f436c0b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4371730700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f436c0b59f0 0x7f436c13e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4371730700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c1552c0 con 0x7f436c0b59f0 2022-07-21T01:52:19.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4371730700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c066cc0 con 0x7f436c0af940 2022-07-21T01:52:19.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4371730700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c1437d0 con 0x7f436c0addd0 2022-07-21T01:52:19.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4370f2f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f436c0b59f0 0x7f436c13e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4370f2f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f436c0b59f0 0x7f436c13e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35470/0 (socket says 172.21.15.140:35470) 2022-07-21T01:52:19.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4370f2f700 1 -- 172.21.15.140:0/2934853646 learned_addr learned my addr 172.21.15.140:0/2934853646 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:19.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f436b7fe700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f436c0af940 0x7f436c0b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f436bfff700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f436c0addd0 0x7f436c151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4370f2f700 1 -- 172.21.15.140:0/2934853646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f436c0addd0 msgr2=0x7f436c151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4370f2f700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f436c0addd0 0x7f436c151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4370f2f700 1 -- 172.21.15.140:0/2934853646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f436c0af940 msgr2=0x7f436c0b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4370f2f700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f436c0af940 0x7f436c0b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4370f2f700 1 -- 172.21.15.140:0/2934853646 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4364009470 con 0x7f436c0b59f0 2022-07-21T01:52:19.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4370f2f700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f436c0b59f0 0x7f436c13e750 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f43640032f0 tx=0x7f4364016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:19.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f43697fa700 1 -- 172.21.15.140:0/2934853646 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f436400b600 con 0x7f436c0b59f0 2022-07-21T01:52:19.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f43697fa700 1 -- 172.21.15.140:0/2934853646 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436400b8d0 con 0x7f436c0b59f0 2022-07-21T01:52:19.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4371730700 1 -- 172.21.15.140:0/2934853646 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c063fd0 con 0x7f436c0b59f0 2022-07-21T01:52:19.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f43697fa700 1 -- 172.21.15.140:0/2934853646 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f436401f5e0 con 0x7f436c0b59f0 2022-07-21T01:52:19.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.783+0000 7f4371730700 1 -- 172.21.15.140:0/2934853646 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c0648e0 con 0x7f436c0b59f0 2022-07-21T01:52:19.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.787+0000 7f4371730700 1 -- 172.21.15.140:0/2934853646 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c1460b0 con 0x7f436c0b59f0 2022-07-21T01:52:19.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.787+0000 7f43697fa700 1 -- 172.21.15.140:0/2934853646 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4364026030 con 0x7f436c0b59f0 2022-07-21T01:52:19.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.787+0000 7f43697fa700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4350053a70 0x7f4350055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:19.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.787+0000 7f43697fa700 1 -- 172.21.15.140:0/2934853646 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f436406caa0 con 0x7f436c0b59f0 2022-07-21T01:52:19.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.787+0000 7f436bfff700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4350053a70 0x7f4350055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:19.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.787+0000 7f436bfff700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4350053a70 0x7f4350055f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f435c00b000 tx=0x7f435c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:19.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.795+0000 7f43697fa700 1 -- 172.21.15.140:0/2934853646 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f43640438c0 con 0x7f436c0b59f0 2022-07-21T01:52:19.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 -- 172.21.15.140:0/2934853646 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f436c0574e0 con 0x7f436c0b59f0 2022-07-21T01:52:19.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f43697fa700 1 -- 172.21.15.140:0/2934853646 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v53) v1 ==== 110+0+10 (secure 0 0 0) 0x7f43640240b0 con 0x7f436c0b59f0 2022-07-21T01:52:19.921 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:52:19.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 -- 172.21.15.140:0/2934853646 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4350053a70 msgr2=0x7f4350055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4350053a70 0x7f4350055f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f435c00b000 tx=0x7f435c006df0).stop 2022-07-21T01:52:19.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 -- 172.21.15.140:0/2934853646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f436c0b59f0 msgr2=0x7f436c13e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:19.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f436c0b59f0 0x7f436c13e750 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f43640032f0 tx=0x7f4364016000).stop 2022-07-21T01:52:19.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 -- 172.21.15.140:0/2934853646 shutdown_connections 2022-07-21T01:52:19.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4350053a70 0x7f4350055f30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f436c0b59f0 0x7f436c13e750 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f436c0af940 0x7f436c0b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 --2- 172.21.15.140:0/2934853646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f436c0addd0 0x7f436c151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:19.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 -- 172.21.15.140:0/2934853646 >> 172.21.15.140:0/2934853646 conn(0x7f436c063480 msgr2=0x7f436c0b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:19.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 -- 172.21.15.140:0/2934853646 shutdown_connections 2022-07-21T01:52:19.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:19.923+0000 7f4371730700 1 -- 172.21.15.140:0/2934853646 wait complete. 2022-07-21T01:52:19.934 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:52:20.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d976b700 1 Processor -- start 2022-07-21T01:52:20.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d976b700 1 -- start start 2022-07-21T01:52:20.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d976b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f39d40ab0c0 0x7f39d4142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d976b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d40b4d70 0x7f39d40b5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d976b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d40ac840 con 0x7f39d40ac460 2022-07-21T01:52:20.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d976b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d4142fb0 con 0x7f39d40ab0c0 2022-07-21T01:52:20.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d976b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4143240 con 0x7f39d40b4d70 2022-07-21T01:52:20.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d3fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f39d40ac460 0x7f39d413c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49620/0 (socket says 172.21.15.140:49620) 2022-07-21T01:52:20.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d3fff700 1 -- 172.21.15.140:0/1583790602 learned_addr learned my addr 172.21.15.140:0/1583790602 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:52:20.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d37fe700 1 --2- 172.21.15.140:0/1583790602 >> v2:172.21.15.152:3300/0 conn(0x7f39d40ab0c0 0x7f39d4142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d8f6a700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d40b4d70 0x7f39d40b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318566359 0 0) 0x7f39d40ac840 con 0x7f39d40ac460 2022-07-21T01:52:20.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39c0003260 con 0x7f39d40ac460 2022-07-21T01:52:20.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2224085696 0 0) 0x7f39c0003260 con 0x7f39d40ac460 2022-07-21T01:52:20.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d40b4d70 msgr2=0x7f39d40b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:20.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d40b4d70 0x7f39d40b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 >> v2:172.21.15.152:3300/0 conn(0x7f39d40ab0c0 msgr2=0x7f39d4142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:20.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 --2- 172.21.15.140:0/1583790602 >> v2:172.21.15.152:3300/0 conn(0x7f39d40ab0c0 0x7f39d4142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d41434e0 con 0x7f39d40ac460 2022-07-21T01:52:20.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f39c4001660 con 0x7f39d40ac460 2022-07-21T01:52:20.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d37fe700 1 --2- 172.21.15.140:0/1583790602 >> v2:172.21.15.152:3300/0 conn(0x7f39d40ab0c0 0x7f39d4142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:52:20.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 >> v1:172.21.15.140:6789/0 conn(0x7f39d40ac460 legacy=0x7f39d413c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d40b4d70 0x7f39c0003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 0x7f39c0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39c00049f0 0x7f39c0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4142fb0 con 0x7f39c0003c80 2022-07-21T01:52:20.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4143240 con 0x7f39c00049f0 2022-07-21T01:52:20.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c000b780 con 0x7f39d40b4d70 2022-07-21T01:52:20.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d37fe700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 0x7f39c0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d3fff700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d40b4d70 0x7f39c0003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d8f6a700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39c00049f0 0x7f39c0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d37fe700 1 -- 172.21.15.140:0/1583790602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d40b4d70 msgr2=0x7f39c0003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:20.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d37fe700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d40b4d70 0x7f39c0003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d37fe700 1 -- 172.21.15.140:0/1583790602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39c00049f0 msgr2=0x7f39c0004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:20.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d37fe700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39c00049f0 0x7f39c0004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d37fe700 1 -- 172.21.15.140:0/1583790602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c0003260 con 0x7f39c0003c80 2022-07-21T01:52:20.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d37fe700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 0x7f39c0004500 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f39c800a000 tx=0x7f39c8008040).ready entity=mon.0 client_cookie=f9527692c487515f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c8013670 con 0x7f39c0003c80 2022-07-21T01:52:20.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c80137d0 con 0x7f39c0003c80 2022-07-21T01:52:20.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.043+0000 7f39d2ffd700 1 -- 172.21.15.140:0/1583790602 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c8013b30 con 0x7f39c0003c80 2022-07-21T01:52:20.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 -- 172.21.15.140:0/1583790602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 msgr2=0x7f39c0004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 --2- 172.21.15.140:0/1583790602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 0x7f39c0004500 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f39c800a000 tx=0x7f39c8008040).stop 2022-07-21T01:52:20.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d8f6a700 1 -- 172.21.15.140:0/1583790602 reap_dead start 2022-07-21T01:52:20.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 -- 172.21.15.140:0/1583790602 shutdown_connections 2022-07-21T01:52:20.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 -- 172.21.15.140:0/1583790602 >> 172.21.15.140:0/1583790602 conn(0x7f39d40632b0 msgr2=0x7f39d4063690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:20.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 -- 172.21.15.140:0/1583790602 shutdown_connections 2022-07-21T01:52:20.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 -- 172.21.15.140:0/1583790602 wait complete. 2022-07-21T01:52:20.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 Processor -- start 2022-07-21T01:52:20.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 -- start start 2022-07-21T01:52:20.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 0x7f39d4144310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d4146a00 0x7f39d41452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39d40acfe0 0x7f39d41423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40661d0 con 0x7f39c0003c80 2022-07-21T01:52:20.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4066400 con 0x7f39d40acfe0 2022-07-21T01:52:20.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d976b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40666a0 con 0x7f39d4146a00 2022-07-21T01:52:20.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.047+0000 7f39d3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 0x7f39d4144310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 0x7f39d4144310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35486/0 (socket says 172.21.15.140:35486) 2022-07-21T01:52:20.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d3fff700 1 -- 172.21.15.140:0/1291414939 learned_addr learned my addr 172.21.15.140:0/1291414939 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:20.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d37fe700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d4146a00 0x7f39d41452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d8f6a700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39d40acfe0 0x7f39d41423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d3fff700 1 -- 172.21.15.140:0/1291414939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d4146a00 msgr2=0x7f39d41452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d3fff700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d4146a00 0x7f39d41452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d3fff700 1 -- 172.21.15.140:0/1291414939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39d40acfe0 msgr2=0x7f39d41423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d3fff700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39d40acfe0 0x7f39d41423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d3fff700 1 -- 172.21.15.140:0/1291414939 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d4151c50 con 0x7f39c0003c80 2022-07-21T01:52:20.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d8f6a700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39d40acfe0 0x7f39d41423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:52:20.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d3fff700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 0x7f39d4144310 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f39c400b000 tx=0x7f39c40014c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d17fa700 1 -- 172.21.15.140:0/1291414939 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c4010710 con 0x7f39c0003c80 2022-07-21T01:52:20.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d17fa700 1 -- 172.21.15.140:0/1291414939 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c4010870 con 0x7f39c0003c80 2022-07-21T01:52:20.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d976b700 1 -- 172.21.15.140:0/1291414939 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d41428a0 con 0x7f39c0003c80 2022-07-21T01:52:20.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d17fa700 1 -- 172.21.15.140:0/1291414939 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f39c4010bd0 con 0x7f39c0003c80 2022-07-21T01:52:20.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d976b700 1 -- 172.21.15.140:0/1291414939 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d41523f0 con 0x7f39c0003c80 2022-07-21T01:52:20.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d976b700 1 -- 172.21.15.140:0/1291414939 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d40ad3c0 con 0x7f39c0003c80 2022-07-21T01:52:20.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d17fa700 1 -- 172.21.15.140:0/1291414939 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f39c4010d30 con 0x7f39c0003c80 2022-07-21T01:52:20.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.051+0000 7f39d17fa700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f39b4053950 0x7f39b4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.055+0000 7f39d17fa700 1 -- 172.21.15.140:0/1291414939 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f39c406c4b0 con 0x7f39c0003c80 2022-07-21T01:52:20.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.055+0000 7f39d37fe700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f39b4053950 0x7f39b4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.055+0000 7f39d37fe700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f39b4053950 0x7f39b4055e10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f39c8019000 tx=0x7f39c8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.059+0000 7f39d17fa700 1 -- 172.21.15.140:0/1291414939 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f39c40433d0 con 0x7f39c0003c80 2022-07-21T01:52:20.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 -- 172.21.15.140:0/1291414939 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f39d4151e40 con 0x7f39c0003c80 2022-07-21T01:52:20.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d17fa700 1 -- 172.21.15.140:0/1291414939 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7f39c40469f0 con 0x7f39c0003c80 2022-07-21T01:52:20.186 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:52:20.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 -- 172.21.15.140:0/1291414939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f39b4053950 msgr2=0x7f39b4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f39b4053950 0x7f39b4055e10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f39c8019000 tx=0x7f39c8008040).stop 2022-07-21T01:52:20.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 -- 172.21.15.140:0/1291414939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 msgr2=0x7f39d4144310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 0x7f39d4144310 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f39c400b000 tx=0x7f39c40014c0).stop 2022-07-21T01:52:20.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 -- 172.21.15.140:0/1291414939 shutdown_connections 2022-07-21T01:52:20.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f39b4053950 0x7f39b4055e10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39d40acfe0 0x7f39d41423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39d4146a00 0x7f39d41452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 --2- 172.21.15.140:0/1291414939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39c0003c80 0x7f39d4144310 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 -- 172.21.15.140:0/1291414939 >> 172.21.15.140:0/1291414939 conn(0x7f39d40632b0 msgr2=0x7f39d40b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:20.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 -- 172.21.15.140:0/1291414939 shutdown_connections 2022-07-21T01:52:20.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.187+0000 7f39d976b700 1 -- 172.21.15.140:0/1291414939 wait complete. 2022-07-21T01:52:20.198 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:52:20.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b60e02700 1 Processor -- start 2022-07-21T01:52:20.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b60e02700 1 -- start start 2022-07-21T01:52:20.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b60e02700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b5c0b5b50 0x7f1b5c0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b60e02700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1b5c0b0480 0x7f1b5c14a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b60e02700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1b5c14ab60 0x7f1b5c144320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b60e02700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c0ac5e0 con 0x7f1b5c0b0480 2022-07-21T01:52:20.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b60e02700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c0b5f30 con 0x7f1b5c14ab60 2022-07-21T01:52:20.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b60e02700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c05cf70 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b59d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1b5c0b0480 0x7f1b5c14a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b59d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1b5c0b0480 0x7f1b5c14a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35496/0 (socket says 172.21.15.140:35496) 2022-07-21T01:52:20.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b59d9b700 1 -- 172.21.15.140:0/3239488401 learned_addr learned my addr 172.21.15.140:0/3239488401 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:20.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b5a59c700 1 --2- 172.21.15.140:0/3239488401 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b5c0b5b50 0x7f1b5c0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b5ad9d700 1 --2- 172.21.15.140:0/3239488401 >> v2:172.21.15.152:3300/0 conn(0x7f1b5c14ab60 0x7f1b5c144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b59d9b700 1 -- 172.21.15.140:0/3239488401 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b5c0b5b50 msgr2=0x7f1b5c0accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:20.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b59d9b700 1 --2- 172.21.15.140:0/3239488401 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b5c0b5b50 0x7f1b5c0accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b59d9b700 1 -- 172.21.15.140:0/3239488401 >> v2:172.21.15.152:3300/0 conn(0x7f1b5c14ab60 msgr2=0x7f1b5c144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:20.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b59d9b700 1 --2- 172.21.15.140:0/3239488401 >> v2:172.21.15.152:3300/0 conn(0x7f1b5c14ab60 0x7f1b5c144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b59d9b700 1 -- 172.21.15.140:0/3239488401 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c0ac810 con 0x7f1b5c0b0480 2022-07-21T01:52:20.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b59d9b700 1 --2- 172.21.15.140:0/3239488401 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b5c0b0480 0x7f1b5c14a670 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f1b44002f70 tx=0x7f1b44006eb0).ready entity=mon.0 client_cookie=f52bfc3de0ce718b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b5959a700 1 -- 172.21.15.140:0/3239488401 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b4400d570 con 0x7f1b5c0b0480 2022-07-21T01:52:20.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b5959a700 1 -- 172.21.15.140:0/3239488401 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b44000e90 con 0x7f1b5c0b0480 2022-07-21T01:52:20.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.303+0000 7f1b5959a700 1 -- 172.21.15.140:0/3239488401 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b4400d900 con 0x7f1b5c0b0480 2022-07-21T01:52:20.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 -- 172.21.15.140:0/3239488401 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b5c0b0480 msgr2=0x7f1b5c14a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 --2- 172.21.15.140:0/3239488401 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b5c0b0480 0x7f1b5c14a670 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f1b44002f70 tx=0x7f1b44006eb0).stop 2022-07-21T01:52:20.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 -- 172.21.15.140:0/3239488401 shutdown_connections 2022-07-21T01:52:20.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 --2- 172.21.15.140:0/3239488401 >> v2:172.21.15.152:3300/0 conn(0x7f1b5c14ab60 0x7f1b5c144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 --2- 172.21.15.140:0/3239488401 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b5c0b0480 0x7f1b5c14a670 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 --2- 172.21.15.140:0/3239488401 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b5c0b5b50 0x7f1b5c0accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 -- 172.21.15.140:0/3239488401 >> 172.21.15.140:0/3239488401 conn(0x7f1b5c0635e0 msgr2=0x7f1b5c0b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:20.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 -- 172.21.15.140:0/3239488401 shutdown_connections 2022-07-21T01:52:20.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 -- 172.21.15.140:0/3239488401 wait complete. 2022-07-21T01:52:20.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 Processor -- start 2022-07-21T01:52:20.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 -- start start 2022-07-21T01:52:20.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b5c0b0480 0x7f1b5c0b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b5c0b5b50 0x7f1b5c159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b5c14ab60 0x7f1b5c15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0a16a0 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0a18d0 con 0x7f1b5c0b0480 2022-07-21T01:52:20.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b60e02700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0b43f0 con 0x7f1b5c14ab60 2022-07-21T01:52:20.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b59d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b5c0b5b50 0x7f1b5c159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b59d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b5c0b5b50 0x7f1b5c159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35502/0 (socket says 172.21.15.140:35502) 2022-07-21T01:52:20.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b59d9b700 1 -- 172.21.15.140:0/3116097701 learned_addr learned my addr 172.21.15.140:0/3116097701 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:20.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.307+0000 7f1b5a59c700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b5c0b0480 0x7f1b5c0b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b5ad9d700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b5c14ab60 0x7f1b5c15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b59d9b700 1 -- 172.21.15.140:0/3116097701 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b5c14ab60 msgr2=0x7f1b5c15a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b59d9b700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b5c14ab60 0x7f1b5c15a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b59d9b700 1 -- 172.21.15.140:0/3116097701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b5c0b0480 msgr2=0x7f1b5c0b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b59d9b700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b5c0b0480 0x7f1b5c0b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b59d9b700 1 -- 172.21.15.140:0/3116097701 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4400b1e0 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b59d9b700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b5c0b5b50 0x7f1b5c159f10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f1b44002f70 tx=0x7f1b44015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b537fe700 1 -- 172.21.15.140:0/3116097701 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b440117e0 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b537fe700 1 -- 172.21.15.140:0/3116097701 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b44011940 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b60e02700 1 -- 172.21.15.140:0/3116097701 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c15a8f0 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b537fe700 1 -- 172.21.15.140:0/3116097701 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b44011ca0 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b60e02700 1 -- 172.21.15.140:0/3116097701 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c14a200 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b60e02700 1 -- 172.21.15.140:0/3116097701 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c0a1a90 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b537fe700 1 -- 172.21.15.140:0/3116097701 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1b44025030 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b537fe700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b400539e0 0x7f1b40055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.311+0000 7f1b537fe700 1 -- 172.21.15.140:0/3116097701 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1b4406cfb0 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.315+0000 7f1b5a59c700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b400539e0 0x7f1b40055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.315+0000 7f1b5a59c700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b400539e0 0x7f1b40055ea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f1b4c00b000 tx=0x7f1b4c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.323+0000 7f1b537fe700 1 -- 172.21.15.140:0/3116097701 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1b44043ed0 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.443+0000 7f1b60e02700 1 -- 172.21.15.140:0/3116097701 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1b5c05d210 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.443+0000 7f1b537fe700 1 -- 172.21.15.140:0/3116097701 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1b44023020 con 0x7f1b5c0b5b50 2022-07-21T01:52:20.444 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:52:20.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 -- 172.21.15.140:0/3116097701 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b400539e0 msgr2=0x7f1b40055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b400539e0 0x7f1b40055ea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f1b4c00b000 tx=0x7f1b4c006df0).stop 2022-07-21T01:52:20.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 -- 172.21.15.140:0/3116097701 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b5c0b5b50 msgr2=0x7f1b5c159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b5c0b5b50 0x7f1b5c159f10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f1b44002f70 tx=0x7f1b44015000).stop 2022-07-21T01:52:20.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 -- 172.21.15.140:0/3116097701 shutdown_connections 2022-07-21T01:52:20.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1b400539e0 0x7f1b40055ea0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b5c14ab60 0x7f1b5c15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1b5c0b5b50 0x7f1b5c159f10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 --2- 172.21.15.140:0/3116097701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b5c0b0480 0x7f1b5c0b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 -- 172.21.15.140:0/3116097701 >> 172.21.15.140:0/3116097701 conn(0x7f1b5c0635e0 msgr2=0x7f1b5c1440f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:20.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 -- 172.21.15.140:0/3116097701 shutdown_connections 2022-07-21T01:52:20.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.447+0000 7f1b60e02700 1 -- 172.21.15.140:0/3116097701 wait complete. 2022-07-21T01:52:20.456 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:52:20.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 Processor -- start 2022-07-21T01:52:20.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 -- start start 2022-07-21T01:52:20.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecc80b59d0 0x7fecc80b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fecc80b14f0 0x7fecc8147050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc80afbc0 con 0x7fecc814ada0 2022-07-21T01:52:20.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc8148080 con 0x7fecc80b14f0 2022-07-21T01:52:20.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc80b00e0 con 0x7fecc80b59d0 2022-07-21T01:52:20.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecccc04700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fecc80b14f0 0x7fecc8147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecccc04700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fecc80b14f0 0x7fecc8147050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35512/0 (socket says 172.21.15.140:35512) 2022-07-21T01:52:20.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecccc04700 1 -- 172.21.15.140:0/2966247937 learned_addr learned my addr 172.21.15.140:0/2966247937 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:20.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecc7fff700 1 --2- 172.21.15.140:0/2966247937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecc80b59d0 0x7fecc80b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecccc04700 1 -- 172.21.15.140:0/2966247937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecc80b59d0 msgr2=0x7fecc80b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:20.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecccc04700 1 --2- 172.21.15.140:0/2966247937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecc80b59d0 0x7fecc80b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecccc04700 1 -- 172.21.15.140:0/2966247937 >> v1:172.21.15.152:6789/0 conn(0x7fecc814ada0 legacy=0x7fecc805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecccc04700 1 -- 172.21.15.140:0/2966247937 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc80b5db0 con 0x7fecc80b14f0 2022-07-21T01:52:20.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecccc04700 1 --2- 172.21.15.140:0/2966247937 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecc80b14f0 0x7fecc8147050 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fecc00032f0 tx=0x7fecc0009290).ready entity=mon.0 client_cookie=e03eb22ae74ef7f0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecc6ffd700 1 -- 172.21.15.140:0/2966247937 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecc000b8a0 con 0x7fecc80b14f0 2022-07-21T01:52:20.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecc6ffd700 1 -- 172.21.15.140:0/2966247937 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc000ba00 con 0x7fecc80b14f0 2022-07-21T01:52:20.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7fecc6ffd700 1 -- 172.21.15.140:0/2966247937 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecc000bb70 con 0x7fecc80b14f0 2022-07-21T01:52:20.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 -- 172.21.15.140:0/2966247937 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecc80b14f0 msgr2=0x7fecc8147050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 --2- 172.21.15.140:0/2966247937 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecc80b14f0 0x7fecc8147050 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fecc00032f0 tx=0x7fecc0009290).stop 2022-07-21T01:52:20.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 -- 172.21.15.140:0/2966247937 shutdown_connections 2022-07-21T01:52:20.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 --2- 172.21.15.140:0/2966247937 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecc80b14f0 0x7fecc8147050 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 --2- 172.21.15.140:0/2966247937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecc80b59d0 0x7fecc80b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 -- 172.21.15.140:0/2966247937 >> 172.21.15.140:0/2966247937 conn(0x7fecc80634c0 msgr2=0x7fecc80b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:20.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.571+0000 7feccd405700 1 -- 172.21.15.140:0/2966247937 shutdown_connections 2022-07-21T01:52:20.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7feccd405700 1 -- 172.21.15.140:0/2966247937 wait complete. 2022-07-21T01:52:20.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7feccd405700 1 Processor -- start 2022-07-21T01:52:20.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7feccd405700 1 -- start start 2022-07-21T01:52:20.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7feccd405700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecc80b14f0 0x7fecc8110f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7feccd405700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc80b59d0 0x7fecc814b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7feccd405700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecc814ada0 0x7fecc8063f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7feccd405700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc81118b0 con 0x7fecc80b14f0 2022-07-21T01:52:20.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7feccd405700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc81573b0 con 0x7fecc80b59d0 2022-07-21T01:52:20.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7feccd405700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc81107d0 con 0x7fecc814ada0 2022-07-21T01:52:20.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecc80b14f0 0x7fecc8110f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecc80b14f0 0x7fecc8110f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35514/0 (socket says 172.21.15.140:35514) 2022-07-21T01:52:20.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc7fff700 1 -- 172.21.15.140:0/1897361104 learned_addr learned my addr 172.21.15.140:0/1897361104 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:20.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecccc04700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecc814ada0 0x7fecc8063f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc77fe700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc80b59d0 0x7fecc814b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc7fff700 1 -- 172.21.15.140:0/1897361104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecc814ada0 msgr2=0x7fecc8063f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc7fff700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecc814ada0 0x7fecc8063f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc7fff700 1 -- 172.21.15.140:0/1897361104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc80b59d0 msgr2=0x7fecc814b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc7fff700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc80b59d0 0x7fecc814b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc7fff700 1 -- 172.21.15.140:0/1897361104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc0009530 con 0x7fecc80b14f0 2022-07-21T01:52:20.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc77fe700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc80b59d0 0x7fecc814b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:52:20.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc7fff700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecc80b14f0 0x7fecc8110f10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fecb800c000 tx=0x7fecb8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc57fa700 1 -- 172.21.15.140:0/1897361104 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecb8010780 con 0x7fecc80b14f0 2022-07-21T01:52:20.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc57fa700 1 -- 172.21.15.140:0/1897361104 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb80108e0 con 0x7fecc80b14f0 2022-07-21T01:52:20.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7feccd405700 1 -- 172.21.15.140:0/1897361104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc8110a70 con 0x7fecc80b14f0 2022-07-21T01:52:20.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7fecc57fa700 1 -- 172.21.15.140:0/1897361104 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecb8010c40 con 0x7fecc80b14f0 2022-07-21T01:52:20.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.575+0000 7feccd405700 1 -- 172.21.15.140:0/1897361104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc80b4730 con 0x7fecc80b14f0 2022-07-21T01:52:20.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.579+0000 7feccd405700 1 -- 172.21.15.140:0/1897361104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc8110630 con 0x7fecc80b14f0 2022-07-21T01:52:20.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.579+0000 7fecc57fa700 1 -- 172.21.15.140:0/1897361104 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fecb801c460 con 0x7fecc80b14f0 2022-07-21T01:52:20.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.579+0000 7fecc57fa700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feca8053a70 0x7feca8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.579+0000 7fecc57fa700 1 -- 172.21.15.140:0/1897361104 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fecb806c360 con 0x7fecc80b14f0 2022-07-21T01:52:20.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.579+0000 7fecc77fe700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feca8053a70 0x7feca8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.579+0000 7fecc77fe700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feca8053a70 0x7feca8055f30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fecbc00b000 tx=0x7fecbc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.583+0000 7fecc57fa700 1 -- 172.21.15.140:0/1897361104 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fecc8110630 con 0x7fecc80b14f0 2022-07-21T01:52:20.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.703+0000 7feccd405700 1 -- 172.21.15.140:0/1897361104 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fecc8110630 con 0x7fecc80b14f0 2022-07-21T01:52:20.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.703+0000 7fecc57fa700 1 -- 172.21.15.140:0/1897361104 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+10 (secure 0 0 0) 0x7fecc8110630 con 0x7fecc80b14f0 2022-07-21T01:52:20.704 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:52:20.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 -- 172.21.15.140:0/1897361104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feca8053a70 msgr2=0x7feca8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feca8053a70 0x7feca8055f30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fecbc00b000 tx=0x7fecbc009040).stop 2022-07-21T01:52:20.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 -- 172.21.15.140:0/1897361104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecc80b14f0 msgr2=0x7fecc8110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecc80b14f0 0x7fecc8110f10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fecb800c000 tx=0x7fecb8006df0).stop 2022-07-21T01:52:20.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 -- 172.21.15.140:0/1897361104 shutdown_connections 2022-07-21T01:52:20.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feca8053a70 0x7feca8055f30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecc814ada0 0x7fecc8063f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecc80b59d0 0x7fecc814b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 --2- 172.21.15.140:0/1897361104 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecc80b14f0 0x7fecc8110f10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 -- 172.21.15.140:0/1897361104 >> 172.21.15.140:0/1897361104 conn(0x7fecc80634c0 msgr2=0x7fecc814a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:20.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 -- 172.21.15.140:0/1897361104 shutdown_connections 2022-07-21T01:52:20.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.707+0000 7feccd405700 1 -- 172.21.15.140:0/1897361104 wait complete. 2022-07-21T01:52:20.717 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:52:20.718 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:52:20.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 Processor -- start 2022-07-21T01:52:20.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 -- start start 2022-07-21T01:52:20.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5d680b1e00 0x7f5d6814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d6805c3f0 0x7f5d680ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5d6805bd90 0x7f5d680accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6805d480 con 0x7f5d680b1e00 2022-07-21T01:52:20.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d680ad1c0 con 0x7f5d6805bd90 2022-07-21T01:52:20.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680b21e0 con 0x7f5d6805c3f0 2022-07-21T01:52:20.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6659c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5d680b1e00 0x7f5d6814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6659c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5d680b1e00 0x7f5d6814a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35522/0 (socket says 172.21.15.140:35522) 2022-07-21T01:52:20.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6659c700 1 -- 172.21.15.140:0/2480420156 learned_addr learned my addr 172.21.15.140:0/2480420156 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:20.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d5dd9b700 1 --2- 172.21.15.140:0/2480420156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d6805c3f0 0x7f5d680ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d66d9d700 1 --2- 172.21.15.140:0/2480420156 >> v2:172.21.15.152:3300/0 conn(0x7f5d6805bd90 0x7f5d680accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6659c700 1 -- 172.21.15.140:0/2480420156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d6805c3f0 msgr2=0x7f5d680ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:20.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6659c700 1 --2- 172.21.15.140:0/2480420156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d6805c3f0 0x7f5d680ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6659c700 1 -- 172.21.15.140:0/2480420156 >> v2:172.21.15.152:3300/0 conn(0x7f5d6805bd90 msgr2=0x7f5d680accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:52:20.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6659c700 1 --2- 172.21.15.140:0/2480420156 >> v2:172.21.15.152:3300/0 conn(0x7f5d6805bd90 0x7f5d680accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6659c700 1 -- 172.21.15.140:0/2480420156 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6805cf40 con 0x7f5d680b1e00 2022-07-21T01:52:20.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6659c700 1 --2- 172.21.15.140:0/2480420156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d680b1e00 0x7f5d6814a6d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f5d58002f70 tx=0x7f5d58006eb0).ready entity=mon.0 client_cookie=c61f92436cfbec2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d65d9b700 1 -- 172.21.15.140:0/2480420156 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d5800d570 con 0x7f5d680b1e00 2022-07-21T01:52:20.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d65d9b700 1 -- 172.21.15.140:0/2480420156 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58000e90 con 0x7f5d680b1e00 2022-07-21T01:52:20.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d65d9b700 1 -- 172.21.15.140:0/2480420156 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d5800d900 con 0x7f5d680b1e00 2022-07-21T01:52:20.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2480420156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d680b1e00 msgr2=0x7f5d6814a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 --2- 172.21.15.140:0/2480420156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d680b1e00 0x7f5d6814a6d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f5d58002f70 tx=0x7f5d58006eb0).stop 2022-07-21T01:52:20.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2480420156 shutdown_connections 2022-07-21T01:52:20.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 --2- 172.21.15.140:0/2480420156 >> v2:172.21.15.152:3300/0 conn(0x7f5d6805bd90 0x7f5d680accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 --2- 172.21.15.140:0/2480420156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d6805c3f0 0x7f5d680ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 --2- 172.21.15.140:0/2480420156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d680b1e00 0x7f5d6814a6d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2480420156 >> 172.21.15.140:0/2480420156 conn(0x7f5d680611c0 msgr2=0x7f5d68062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:20.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.827+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2480420156 shutdown_connections 2022-07-21T01:52:20.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.831+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2480420156 wait complete. 2022-07-21T01:52:20.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.831+0000 7f5d6cfc6700 1 Processor -- start 2022-07-21T01:52:20.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.831+0000 7f5d6cfc6700 1 -- start start 2022-07-21T01:52:20.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6cfc6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d6805c3f0 0x7f5d68143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6cfc6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d680b1e00 0x7f5d68142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6cfc6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d6814a3a0 0x7f5d68142f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6cfc6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6815ac80 con 0x7f5d6805c3f0 2022-07-21T01:52:20.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6cfc6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6815a9e0 con 0x7f5d680b1e00 2022-07-21T01:52:20.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6cfc6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6815a710 con 0x7f5d6814a3a0 2022-07-21T01:52:20.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d6805c3f0 0x7f5d68143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d6805c3f0 0x7f5d68143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35528/0 (socket says 172.21.15.140:35528) 2022-07-21T01:52:20.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6659c700 1 -- 172.21.15.140:0/2380683211 learned_addr learned my addr 172.21.15.140:0/2380683211 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:52:20.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d5dd9b700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d680b1e00 0x7f5d68142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d66d9d700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d6814a3a0 0x7f5d68142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6659c700 1 -- 172.21.15.140:0/2380683211 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d6814a3a0 msgr2=0x7f5d68142f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6659c700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d6814a3a0 0x7f5d68142f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6659c700 1 -- 172.21.15.140:0/2380683211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d680b1e00 msgr2=0x7f5d68142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:20.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6659c700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d680b1e00 0x7f5d68142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:20.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6659c700 1 -- 172.21.15.140:0/2380683211 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5800b1e0 con 0x7f5d6805c3f0 2022-07-21T01:52:20.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6659c700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d6805c3f0 0x7f5d68143eb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f5d58002f70 tx=0x7f5d58015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d5ffff700 1 -- 172.21.15.140:0/2380683211 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d58011610 con 0x7f5d6805c3f0 2022-07-21T01:52:20.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d5ffff700 1 -- 172.21.15.140:0/2380683211 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58011770 con 0x7f5d6805c3f0 2022-07-21T01:52:20.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2380683211 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d681433f0 con 0x7f5d6805c3f0 2022-07-21T01:52:20.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d5ffff700 1 -- 172.21.15.140:0/2380683211 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d58011ad0 con 0x7f5d6805c3f0 2022-07-21T01:52:20.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2380683211 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d680a0480 con 0x7f5d6805c3f0 2022-07-21T01:52:20.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.835+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2380683211 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6809e080 con 0x7f5d6805c3f0 2022-07-21T01:52:20.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.839+0000 7f5d5ffff700 1 -- 172.21.15.140:0/2380683211 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d5801a040 con 0x7f5d6805c3f0 2022-07-21T01:52:20.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.839+0000 7f5d5ffff700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d4c053b00 0x7f5d4c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:52:20.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.839+0000 7f5d5ffff700 1 -- 172.21.15.140:0/2380683211 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5d5806cc70 con 0x7f5d6805c3f0 2022-07-21T01:52:20.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.839+0000 7f5d5dd9b700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d4c053b00 0x7f5d4c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:52:20.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.839+0000 7f5d5dd9b700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d4c053b00 0x7f5d4c055fc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f5d5000b000 tx=0x7f5d50006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:52:20.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:20.843+0000 7f5d5ffff700 1 -- 172.21.15.140:0/2380683211 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5d58043a80 con 0x7f5d6805c3f0 2022-07-21T01:52:21.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2380683211 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d680574e0 con 0x7f5d6805c3f0 2022-07-21T01:52:21.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d5ffff700 1 -- 172.21.15.140:0/2380683211 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2145 (secure 0 0 0) 0x7f5d58023020 con 0x7f5d6805c3f0 2022-07-21T01:52:21.010 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:52:21.010 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1915503334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1915503334},{"type":"v1","addr":"172.21.15.140:6835","nonce":1915503334}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:51:56.500368+0000","modified":"2022-07-21T01:51:57.504176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5204":{"gid":5204,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3908541795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3908541795},{"type":"v1","addr":"172.21.15.152:6833","nonce":3908541795}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-07-21T01:52:21.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2380683211 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d4c053b00 msgr2=0x7f5d4c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:21.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d4c053b00 0x7f5d4c055fc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f5d5000b000 tx=0x7f5d50006df0).stop 2022-07-21T01:52:21.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2380683211 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d6805c3f0 msgr2=0x7f5d68143eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:52:21.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d6805c3f0 0x7f5d68143eb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f5d58002f70 tx=0x7f5d58015000).stop 2022-07-21T01:52:21.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2380683211 shutdown_connections 2022-07-21T01:52:21.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d4c053b00 0x7f5d4c055fc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:21.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d6814a3a0 0x7f5d68142f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:21.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d680b1e00 0x7f5d68142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:21.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 --2- 172.21.15.140:0/2380683211 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d6805c3f0 0x7f5d68143eb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:52:21.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2380683211 >> 172.21.15.140:0/2380683211 conn(0x7f5d6805bd90 msgr2=0x7f5d68144fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:52:21.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2380683211 shutdown_connections 2022-07-21T01:52:21.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:52:21.011+0000 7f5d6cfc6700 1 -- 172.21.15.140:0/2380683211 wait complete. 2022-07-21T01:52:21.016 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 21 2022-07-21T01:52:21.230 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:51:56.500368+0000', 'modified': '2022-07-21T01:51:57.504176+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5204}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5204': {'gid': 5204, 'name': 'b', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/3908541795', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 3908541795}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 3908541795}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:52:21.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:52:21.232 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:52:21.348 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":5204,"want_state":"up:active","state":"up:active","rank_uptime":24.839398018000001,"mdsmap_epoch":21,"osdmap_epoch":53,"osdmap_epoch_barrier":52,"uptime":29.498434271000001} 2022-07-21T01:52:21.349 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 5204, "mdsmap_epoch": 21, "osdmap_epoch": 53, "osdmap_epoch_barrier": 52, "rank_uptime": 24.839398018, "state": "up:active", "uptime": 29.498434271, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:52:21.350 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:52:21.350 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:52:21.350 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:52:21.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:21.351 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:21.360 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:52:21.378 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:52:21.379 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:52:21.433 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:52:21.434 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:52:21.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:21.436 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:52:21.442 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:52:21.444 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:21.488 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:52:21.507 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:52:21.508 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:52:21.565 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:52:21.572+0000 7fac770ab080 -1 init, newargv = 0x564036d38220 newargc=9 2022-07-21T01:52:21.565 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22584]: starting ceph client 2022-07-21T01:52:21.567 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:52:21.568 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:52:21.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:21.570 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:52:21.573 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22584]: starting fuse 2022-07-21T01:52:21.577 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:52:21.577 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:52:21.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:21.579 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:52:21.579 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:52:21.580 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:52:21.580 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:52:21.580 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:52:21.581 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:52:21.581 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:52:21.582 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:52:21.582 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:52:21.583 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:52:21.583 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:52:21.584 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:52:21.584 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:52:21.585 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:52:21.585 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:52:21.586 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:52:21.586 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:52:21.587 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:52:21.587 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:52:21.588 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:52:21.588 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:52:21.589 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:52:21.589 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:52:21.665 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.22584.asok 2022-07-21T01:52:21.671 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22584.asok 2022-07-21T01:52:21.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:21.672 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22584.asok status 2022-07-21T01:52:21.763 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:52:21.764 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:52:21.764 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:52:21.764 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:52:21.764 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:52:21.765 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:52:21.765 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:52:21.765 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "22584", 2022-07-21T01:52:21.765 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:52:21.766 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:52:21.766 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:52:21.766 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:52:21.766 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 5396, 2022-07-21T01:52:21.767 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:52:21.767 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:52:21.767 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:52:21.767 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 5396 2022-07-21T01:52:21.768 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:52:21.768 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:52:21.768 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:52:21.768 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:52:21.769 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3954774492 2022-07-21T01:52:21.769 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:52:21.769 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:52:21.769 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:52:21.770 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:52:21.770 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:52:21.770 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3954774492 2022-07-21T01:52:21.770 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:52:21.771 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.5396 172.21.15.191:0/3954774492", 2022-07-21T01:52:21.771 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/3954774492", 2022-07-21T01:52:21.771 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:52:21.771 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 21, 2022-07-21T01:52:21.772 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 53, 2022-07-21T01:52:21.772 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:52:21.772 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:52:21.772 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:52:21.773 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:52:21.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:21.775 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:52:21.775 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:21.783 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:52:21.785 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:52:21.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:52:21.785 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:52:31.455 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:52:31.460 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:52:31.463 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:52:31.467 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:52:31.470 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:53:01.517 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:53:01.521 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:53:01.524 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:53:01.527 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:53:01.531 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:53:16.867 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:53:16.869 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:16.892 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:16.914 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:16.937 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:16.961 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:16.984 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:53:16.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:16.986 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:16.987 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:16.996 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:53:16.998 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:53:16.999 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:53:17.000 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:53:17.001 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:17.057 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22584]: fuse finished with error 0 and tester_r 0 2022-07-21T01:53:17.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:17.060 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:17.061 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:17.068 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:53:17.070 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:53:17.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:53:17.201 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:17.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:17.243 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:17.243 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:53:17.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:53:17.254 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:53:17.255 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:53:17.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:53:17.257 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:53:17.302 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:53:17.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:53:17.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:17.304 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:17.305 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:53:17.312 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:53:17.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:53:17.314 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:53:17.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:53:17.315 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:53:17.323 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:53:17.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:53:17.325 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:53:17.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 Processor -- start 2022-07-21T01:53:17.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 -- start start 2022-07-21T01:53:17.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5c100acae0 0x7f5c1013f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:17.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5c1013fd60 0x7f5c1013dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:17.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c100aef90 con 0x7f5c100aebb0 2022-07-21T01:53:17.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c100b0270 con 0x7f5c100acae0 2022-07-21T01:53:17.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c100af200 con 0x7f5c1013fd60 2022-07-21T01:53:17.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0dd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5c100acae0 0x7f5c1013f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:17.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0dd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5c100acae0 0x7f5c1013f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35538/0 (socket says 172.21.15.140:35538) 2022-07-21T01:53:17.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0dd9b700 1 -- 172.21.15.140:0/837153763 learned_addr learned my addr 172.21.15.140:0/837153763 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:17.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0e59c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f5c100aebb0 0x7f5c100b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49682/0 (socket says 172.21.15.140:49682) 2022-07-21T01:53:17.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0dd9b700 1 -- 172.21.15.140:0/837153763 >> v2:172.21.15.152:3300/0 conn(0x7f5c1013fd60 msgr2=0x7f5c1013dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:17.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0ed9d700 1 --2- 172.21.15.140:0/837153763 >> v2:172.21.15.152:3300/0 conn(0x7f5c1013fd60 0x7f5c1013dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:17.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0d59a700 1 -- 172.21.15.140:0/837153763 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346328594 0 0) 0x7f5c100aef90 con 0x7f5c100aebb0 2022-07-21T01:53:17.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0dd9b700 1 --2- 172.21.15.140:0/837153763 >> v2:172.21.15.152:3300/0 conn(0x7f5c1013fd60 0x7f5c1013dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:17.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0dd9b700 1 -- 172.21.15.140:0/837153763 >> v1:172.21.15.140:6789/0 conn(0x7f5c100aebb0 legacy=0x7f5c100b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:17.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0dd9b700 1 -- 172.21.15.140:0/837153763 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c100af4a0 con 0x7f5c100acae0 2022-07-21T01:53:17.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0ed9d700 1 --2- 172.21.15.140:0/837153763 >> v2:172.21.15.152:3300/0 conn(0x7f5c1013fd60 0x7f5c1013dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:53:17.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0dd9b700 1 --2- 172.21.15.140:0/837153763 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c100acae0 0x7f5c1013f870 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f5c04002f70 tx=0x7f5c04006e50).ready entity=mon.0 client_cookie=1760fa6ce22e8424 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:17.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0d59a700 1 -- 172.21.15.140:0/837153763 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c04010810 con 0x7f5c100acae0 2022-07-21T01:53:17.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0d59a700 1 -- 172.21.15.140:0/837153763 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c04000e90 con 0x7f5c100acae0 2022-07-21T01:53:17.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0d59a700 1 -- 172.21.15.140:0/837153763 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c04010b60 con 0x7f5c100acae0 2022-07-21T01:53:17.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 -- 172.21.15.140:0/837153763 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c100acae0 msgr2=0x7f5c1013f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:17.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 --2- 172.21.15.140:0/837153763 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c100acae0 0x7f5c1013f870 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f5c04002f70 tx=0x7f5c04006e50).stop 2022-07-21T01:53:17.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 -- 172.21.15.140:0/837153763 shutdown_connections 2022-07-21T01:53:17.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 --2- 172.21.15.140:0/837153763 >> v2:172.21.15.152:3300/0 conn(0x7f5c1013fd60 0x7f5c1013dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:17.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 --2- 172.21.15.140:0/837153763 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c100acae0 0x7f5c1013f870 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:17.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 -- 172.21.15.140:0/837153763 >> 172.21.15.140:0/837153763 conn(0x7f5c100634e0 msgr2=0x7f5c100b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:17.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 -- 172.21.15.140:0/837153763 shutdown_connections 2022-07-21T01:53:17.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.443+0000 7f5c0f59e700 1 -- 172.21.15.140:0/837153763 wait complete. 2022-07-21T01:53:17.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0f59e700 1 Processor -- start 2022-07-21T01:53:17.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0f59e700 1 -- start start 2022-07-21T01:53:17.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c100acae0 0x7f5c1013cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:17.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0f59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c100aebb0 0x7f5c101367a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:17.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0f59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c1013fd60 0x7f5c10136c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:17.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0f59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10064760 con 0x7f5c1013fd60 2022-07-21T01:53:17.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0f59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10063d80 con 0x7f5c100aebb0 2022-07-21T01:53:17.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10064020 con 0x7f5c100acae0 2022-07-21T01:53:17.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c1013fd60 0x7f5c10136c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:17.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c1013fd60 0x7f5c10136c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35546/0 (socket says 172.21.15.140:35546) 2022-07-21T01:53:17.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0e59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c100acae0 0x7f5c1013cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:17.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0ed9d700 1 -- 172.21.15.140:0/2137885045 learned_addr learned my addr 172.21.15.140:0/2137885045 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:17.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0dd9b700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c100aebb0 0x7f5c101367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:17.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0ed9d700 1 -- 172.21.15.140:0/2137885045 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c100acae0 msgr2=0x7f5c1013cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:17.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0ed9d700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c100acae0 0x7f5c1013cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:17.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0ed9d700 1 -- 172.21.15.140:0/2137885045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c100aebb0 msgr2=0x7f5c101367a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:17.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0ed9d700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c100aebb0 0x7f5c101367a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:17.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.447+0000 7f5c0ed9d700 1 -- 172.21.15.140:0/2137885045 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0400c2f0 con 0x7f5c1013fd60 2022-07-21T01:53:17.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.451+0000 7f5c0ed9d700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c1013fd60 0x7f5c10136c90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f5c08007000 tx=0x7f5c08008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:17.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.451+0000 7f5bff7fe700 1 -- 172.21.15.140:0/2137885045 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0800e510 con 0x7f5c1013fd60 2022-07-21T01:53:17.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.451+0000 7f5bff7fe700 1 -- 172.21.15.140:0/2137885045 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0800e670 con 0x7f5c1013fd60 2022-07-21T01:53:17.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.451+0000 7f5c0f59e700 1 -- 172.21.15.140:0/2137885045 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c100642c0 con 0x7f5c1013fd60 2022-07-21T01:53:17.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.451+0000 7f5bff7fe700 1 -- 172.21.15.140:0/2137885045 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0800e9d0 con 0x7f5c1013fd60 2022-07-21T01:53:17.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.451+0000 7f5c0f59e700 1 -- 172.21.15.140:0/2137885045 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c10137180 con 0x7f5c1013fd60 2022-07-21T01:53:17.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.451+0000 7f5c0f59e700 1 -- 172.21.15.140:0/2137885045 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c10063ba0 con 0x7f5c1013fd60 2022-07-21T01:53:17.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.451+0000 7f5bff7fe700 1 -- 172.21.15.140:0/2137885045 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5c08019030 con 0x7f5c1013fd60 2022-07-21T01:53:17.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.451+0000 7f5bff7fe700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bec053a70 0x7f5bec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:17.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.451+0000 7f5bff7fe700 1 -- 172.21.15.140:0/2137885045 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5c0806bcd0 con 0x7f5c1013fd60 2022-07-21T01:53:17.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.451+0000 7f5c0e59c700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bec053a70 0x7f5bec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:17.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.455+0000 7f5c0e59c700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bec053a70 0x7f5bec055f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f5c0000d000 tx=0x7f5c00007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:17.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.459+0000 7f5bff7fe700 1 -- 172.21.15.140:0/2137885045 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c08042af0 con 0x7f5c1013fd60 2022-07-21T01:53:17.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.583+0000 7f5c0f59e700 1 -- 172.21.15.140:0/2137885045 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5c10064450 con 0x7f5c1013fd60 2022-07-21T01:53:17.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5bff7fe700 1 -- 172.21.15.140:0/2137885045 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v53) v1 ==== 81+0+33743 (secure 0 0 0) 0x7f5c08046110 con 0x7f5c1013fd60 2022-07-21T01:53:17.586 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:17.587 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:53:17.589 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 53, 2022-07-21T01:53:17.589 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:53:17.590 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:53:17.590 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:51:58.506065+0000", 2022-07-21T01:53:17.591 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:53:17.592 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:53:17.592 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:53:17.593 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:53:17.593 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:53:17.594 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:53:17.594 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:53:17.595 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:53:17.595 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:53:17.596 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:17.597 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:53:17.597 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:53:17.598 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:53:17.599 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:53:17.599 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:53:17.600 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 10, 2022-07-21T01:53:17.601 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:53:17.601 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:53:17.602 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:53:17.603 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:53:17.603 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:53:17.604 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.604 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:53:17.605 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:53:17.605 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:53:17.606 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:53:17.606 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:53:17.607 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:53:17.607 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:53:17.608 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:53:17.609 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:53:17.609 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:53:17.610 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:53:17.610 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:53:17.611 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:53:17.612 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:53:17.612 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:53:17.613 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:53:17.613 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:53:17.613 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:53:17.614 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:53:17.614 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:53:17.614 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:53:17.615 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:53:17.615 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:53:17.615 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.615 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:53:17.616 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:53:17.616 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:53:17.616 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:53:17.616 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:53:17.617 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:53:17.617 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:53:17.617 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:53:17.617 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:53:17.618 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:53:17.618 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:53:17.618 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:53:17.618 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:53:17.619 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:53:17.619 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:53:17.619 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:53:17.619 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:53:17.620 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:53:17.620 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:53:17.620 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:53:17.621 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:53:17.621 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:53:17.621 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:53:17.621 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:53:17.622 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:53:17.622 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:53:17.622 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:53:17.623 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.623 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:53:17.623 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:53:17.623 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:53:17.623 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:53:17.624 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:53:17.624 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:53:17.624 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:53:17.625 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:53:17.625 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:53:17.625 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:53:17.626 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:53:17.626 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:53:17.626 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:53:17.626 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.627 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:53:17.627 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:53:17.627 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.627 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.628 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.628 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:53:17.628 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:53:17.628 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:53:17.629 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:53:17.629 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:53:17.629 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:53:17.630 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:53:17.630 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:53:17.630 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:53:17.630 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:53:17.631 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:53:17.631 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:53:17.631 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:53:17.632 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:53:17.632 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:53:17.632 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:53:17.633 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:53:17.633 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:53:17.633 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:53:17.633 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:53:17.634 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:53:17.634 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:53:17.634 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:53:17.635 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.635 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:53:17.635 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:53:17.635 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:53:17.636 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:53:17.636 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:53:17.636 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:53:17.636 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:53:17.637 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:53:17.637 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:53:17.637 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:53:17.638 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:53:17.638 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:53:17.638 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:53:17.638 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:53:17.639 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:53:17.639 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:53:17.639 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:53:17.639 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:53:17.640 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:53:17.640 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:53:17.640 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:53:17.641 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:53:17.641 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:53:17.641 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:53:17.641 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:53:17.642 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:53:17.642 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:53:17.642 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.643 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:53:17.643 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:53:17.643 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:53:17.644 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:53:17.644 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:53:17.644 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:53:17.645 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:53:17.645 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:53:17.645 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:53:17.645 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:53:17.646 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:53:17.646 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:53:17.646 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:53:17.647 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:53:17.647 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.647 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.647 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.648 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 9, 2022-07-21T01:53:17.648 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:53:17.648 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:51:54.695165+0000", 2022-07-21T01:53:17.648 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:53:17.649 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:53:17.649 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:53:17.649 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:53:17.649 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:53:17.650 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:53:17.650 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:53:17.650 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:53:17.651 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:53:17.651 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:53:17.651 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:53:17.651 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:53:17.652 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:53:17.652 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:53:17.652 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:53:17.653 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:53:17.653 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:53:17.653 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:53:17.654 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:53:17.654 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:53:17.654 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.655 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "52", 2022-07-21T01:53:17.655 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:53:17.655 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:53:17.655 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:53:17.656 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:53:17.656 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:53:17.656 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:53:17.656 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:53:17.657 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:53:17.657 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:53:17.657 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:53:17.657 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:53:17.658 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:53:17.658 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:53:17.658 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:53:17.659 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:53:17.659 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:53:17.659 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:53:17.659 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:53:17.660 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:53:17.660 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:53:17.660 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:53:17.660 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:53:17.661 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:53:17.661 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:53:17.661 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:53:17.661 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:53:17.662 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.662 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:53:17.662 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:53:17.662 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:53:17.663 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:53:17.663 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:53:17.663 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:53:17.664 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:53:17.664 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:53:17.664 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:53:17.665 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:53:17.665 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:53:17.665 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:53:17.665 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:53:17.666 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:53:17.666 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:53:17.666 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.667 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:53:17.667 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:53:17.667 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:53:17.667 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.668 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.668 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.668 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.668 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 10, 2022-07-21T01:53:17.669 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:53:17.669 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:51:55.500555+0000", 2022-07-21T01:53:17.669 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:53:17.669 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:53:17.670 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:53:17.670 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:53:17.670 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:53:17.671 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:53:17.671 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:53:17.671 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:53:17.671 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:53:17.672 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:53:17.672 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:53:17.672 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:53:17.672 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:53:17.673 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:53:17.673 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:53:17.673 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:53:17.674 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:53:17.674 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:53:17.674 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:53:17.675 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:53:17.675 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.675 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "53", 2022-07-21T01:53:17.676 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:53:17.676 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:53:17.676 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:53:17.677 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:53:17.677 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:53:17.677 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:53:17.677 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:53:17.678 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:53:17.678 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:53:17.678 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:53:17.679 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:53:17.679 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:53:17.679 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:53:17.679 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:53:17.680 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:53:17.680 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:53:17.680 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:53:17.680 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:53:17.681 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:53:17.681 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:53:17.681 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:53:17.681 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:53:17.682 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:53:17.682 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:53:17.682 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:53:17.683 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:53:17.683 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.683 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:53:17.683 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:53:17.684 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:53:17.684 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:53:17.684 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:53:17.684 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:53:17.685 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:53:17.685 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:53:17.685 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:53:17.686 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:53:17.686 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:53:17.686 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:53:17.687 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:53:17.687 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:53:17.687 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:53:17.688 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.688 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.688 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.689 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:17.689 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:53:17.689 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.689 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:53:17.690 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:53:17.690 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:17.690 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:17.691 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:17.691 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:17.691 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:17.691 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:17.692 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:17.692 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 50, 2022-07-21T01:53:17.692 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:17.693 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:17.693 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:17.693 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.693 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.694 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.694 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:53:17.694 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:17.695 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.695 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.695 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.696 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:53:17.696 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:17.696 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.697 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.697 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.697 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:17.698 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.698 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.698 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.698 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:53:17.699 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:17.699 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.699 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.700 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.700 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:53:17.700 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:17.700 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.701 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.701 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.701 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:17.701 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.702 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.702 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.702 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:53:17.703 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:17.703 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.703 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.703 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.704 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:53:17.704 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:17.704 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.705 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.705 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.705 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:17.705 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.706 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.706 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.706 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:53:17.707 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:17.707 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.707 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.708 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.708 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:53:17.708 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:17.708 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.708 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.709 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.709 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:53:17.709 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:53:17.710 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:53:17.710 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:53:17.710 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:17.710 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:17.711 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:17.711 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.711 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.711 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.712 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:53:17.712 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:53:17.712 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:17.712 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:17.713 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:17.713 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:17.713 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:17.713 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:17.714 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:17.714 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 50, 2022-07-21T01:53:17.714 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:17.715 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:17.715 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:17.715 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.715 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.716 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.716 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:53:17.716 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:17.717 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.717 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.717 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.718 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:53:17.718 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:17.718 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.719 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.719 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.719 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:17.719 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.720 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.720 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.720 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:53:17.720 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:17.721 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.721 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.721 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.721 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:53:17.722 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:17.722 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.722 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.723 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.723 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:17.723 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.723 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.724 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.724 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:53:17.724 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:17.724 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.725 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.725 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.725 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:53:17.726 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:17.726 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.726 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.726 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.727 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:17.727 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.727 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.728 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.728 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:53:17.728 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:17.729 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.729 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.729 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.730 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:53:17.730 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:17.730 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.730 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.731 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.731 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:53:17.731 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:53:17.731 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:53:17.732 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:53:17.732 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:17.732 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:17.732 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:17.733 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.733 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.733 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.733 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:53:17.733 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:53:17.734 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:17.734 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:17.734 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:17.735 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:17.735 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:17.735 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:17.735 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:17.736 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 51, 2022-07-21T01:53:17.736 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:17.736 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:17.737 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:17.737 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.737 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.738 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.738 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:53:17.739 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:17.739 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.739 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.740 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.740 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:53:17.740 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:17.741 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.741 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.741 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.741 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:17.742 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.742 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.742 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.742 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:53:17.743 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:17.743 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.743 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.743 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.744 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:53:17.744 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:17.744 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.745 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.745 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.745 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:17.745 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.746 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.746 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.746 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:53:17.747 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:17.747 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.747 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.748 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.748 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:53:17.748 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:17.749 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.749 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.749 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.750 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:17.750 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.750 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.750 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.751 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:53:17.751 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:17.751 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.751 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.752 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.752 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:53:17.752 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:17.753 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.753 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.753 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.753 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:53:17.754 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:53:17.754 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:53:17.754 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:53:17.755 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:17.755 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:17.755 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:17.755 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.756 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.756 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.756 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:53:17.756 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:53:17.757 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:17.757 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:17.757 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:17.758 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:17.758 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:17.758 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:17.759 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:17.760 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 49, 2022-07-21T01:53:17.760 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:17.760 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:17.760 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:17.761 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.761 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.761 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.762 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:53:17.762 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:17.763 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.763 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.763 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.763 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:53:17.764 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:17.764 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.765 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.765 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.765 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:17.766 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.766 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.766 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.767 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:53:17.767 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:17.767 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.767 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.768 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.768 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:53:17.768 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:17.768 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.769 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.769 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.769 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:17.770 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.770 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.770 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.771 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:53:17.771 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:17.771 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.772 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.772 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.772 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:53:17.773 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:17.773 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.773 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.773 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.774 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:17.774 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.774 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.775 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.775 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:53:17.775 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:17.775 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.776 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.776 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.776 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:53:17.777 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:17.777 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.777 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.777 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.778 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:53:17.778 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:53:17.778 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:53:17.779 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:53:17.779 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:17.779 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:17.780 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:17.780 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.780 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.781 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.781 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:53:17.781 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:53:17.781 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:17.782 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:17.782 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:17.782 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:17.783 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:17.783 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:17.783 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:17.783 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 50, 2022-07-21T01:53:17.784 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:17.784 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:17.784 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:17.785 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.785 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.785 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.785 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:53:17.786 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:17.786 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.786 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.786 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.787 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:53:17.787 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:17.787 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.788 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.788 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.788 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:17.788 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.789 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.789 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.789 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:53:17.789 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:17.790 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.790 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.790 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.791 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:53:17.791 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:17.791 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.791 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.792 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.792 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:17.792 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.793 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.793 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.793 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:53:17.793 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:17.794 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.794 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.794 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.795 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:53:17.795 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:17.795 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.795 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.796 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.796 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:17.796 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.797 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.797 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.797 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:53:17.797 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:17.798 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.798 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.798 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.798 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:53:17.799 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:17.799 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.799 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.800 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.800 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:53:17.800 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:53:17.800 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:53:17.801 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:53:17.801 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:17.801 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:17.802 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:17.802 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.802 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.802 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.803 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:53:17.803 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:53:17.803 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:17.803 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:17.804 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:17.804 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:17.804 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:17.805 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:17.805 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:17.805 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 50, 2022-07-21T01:53:17.805 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:17.806 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:17.806 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:17.806 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.807 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.807 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.807 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:53:17.807 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:17.808 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.808 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.808 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.809 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:53:17.809 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:17.809 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.810 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.810 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.810 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:17.811 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.811 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.811 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.811 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:53:17.812 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:17.812 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.812 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.813 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.813 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:53:17.813 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:17.813 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.814 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.814 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.814 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:17.815 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.815 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.815 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.815 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:53:17.816 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:17.816 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.816 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.817 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.817 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:53:17.817 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:17.817 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.818 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.818 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.818 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:17.818 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.819 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.819 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.819 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:53:17.819 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:17.820 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.820 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.820 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.820 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:53:17.821 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:17.821 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.821 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.822 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.822 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:53:17.822 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:53:17.823 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:53:17.823 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:53:17.823 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:17.823 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:17.824 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:17.824 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.824 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.824 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.825 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:53:17.825 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:53:17.825 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:17.825 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:17.826 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:17.826 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:17.826 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:17.827 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:17.827 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:17.827 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 50, 2022-07-21T01:53:17.827 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:17.828 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:17.828 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:17.828 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.829 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.829 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.829 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:53:17.829 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:17.830 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.830 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.830 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.831 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:53:17.831 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:17.831 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.831 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.832 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.832 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:17.832 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.833 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.833 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.833 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:53:17.833 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:17.834 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.834 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.834 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.834 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:53:17.835 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:17.835 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.835 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.836 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.836 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:17.836 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.836 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.837 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.837 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:53:17.837 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:17.837 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.838 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.838 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.838 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:53:17.839 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:17.839 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.839 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.839 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.840 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:17.840 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.840 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.841 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.841 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:53:17.841 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:17.842 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.842 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.842 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.842 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:53:17.843 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:17.843 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.843 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.843 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.844 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:53:17.844 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:53:17.844 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:53:17.844 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:53:17.845 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:17.845 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:17.845 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:17.846 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.846 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.846 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.847 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:53:17.847 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:53:17.847 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:17.848 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:17.848 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:17.848 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:17.848 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:17.849 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:17.849 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:17.849 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 49, 2022-07-21T01:53:17.850 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:17.850 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:17.850 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:17.851 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.851 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.851 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.851 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:53:17.852 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:17.852 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.852 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.852 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.853 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:53:17.853 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:17.853 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.853 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.854 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.854 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:17.854 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.854 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.855 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.855 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:53:17.855 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:17.856 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.856 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.857 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.857 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:53:17.857 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:17.858 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.858 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.858 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.859 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:17.859 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.859 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.859 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.860 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:53:17.860 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:17.860 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.860 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.861 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.861 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:53:17.861 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:17.862 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.862 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.862 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.862 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:17.863 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:17.863 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.863 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:17.864 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:53:17.864 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:17.864 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.864 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.865 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:17.865 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:53:17.865 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:17.866 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.866 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.866 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.867 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:53:17.867 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:53:17.867 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:53:17.868 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:53:17.868 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:17.868 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:17.869 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:17.869 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:17.869 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.870 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:17.870 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:53:17.870 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.871 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:53:17.871 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:17.871 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:17.871 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:17.872 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:17.872 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:17.872 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:53:17.873 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:17.873 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.873 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.873 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:53:17.874 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:17.874 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:17.874 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:17.875 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:17.875 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:17.875 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:53:17.875 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:17.876 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.876 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.876 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:53:17.877 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:17.877 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:17.877 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:17.878 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:17.878 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:17.878 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:53:17.879 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:17.879 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.879 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.880 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:53:17.880 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:17.880 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:17.880 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:17.881 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:17.881 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:17.881 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:53:17.882 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:17.882 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.882 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.882 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:53:17.883 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:17.883 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:17.883 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:17.883 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:17.884 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:17.884 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:53:17.884 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:17.885 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.885 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.885 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:53:17.885 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:17.886 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:17.886 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:17.886 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:17.887 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:17.887 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:53:17.887 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:17.887 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.888 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.888 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:53:17.888 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:17.889 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:17.889 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:17.890 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:17.890 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:17.890 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:53:17.891 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:17.891 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.891 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:17.891 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:53:17.892 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:17.892 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:17.892 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:17.892 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:17.893 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:17.893 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:53:17.893 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:17.894 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.894 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:17.894 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:53:17.895 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:53:17.895 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:53:17.895 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:53:17.896 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": {}, 2022-07-21T01:53:17.896 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:53:17.896 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:53:17.897 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:53:17.897 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:53:17.897 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:53:17.898 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:53:17.898 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:53:17.898 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:53:17.899 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:17.899 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:17.899 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:53:17.900 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:53:17.900 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:53:17.900 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:53:17.901 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:53:17.901 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:53:17.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5c0f59e700 1 -- 172.21.15.140:0/2137885045 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bec053a70 msgr2=0x7f5bec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:17.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5c0f59e700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bec053a70 0x7f5bec055f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f5c0000d000 tx=0x7f5c00007a00).stop 2022-07-21T01:53:17.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5c0f59e700 1 -- 172.21.15.140:0/2137885045 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c1013fd60 msgr2=0x7f5c10136c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:17.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5c0f59e700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c1013fd60 0x7f5c10136c90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f5c08007000 tx=0x7f5c08008040).stop 2022-07-21T01:53:17.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5c0f59e700 1 -- 172.21.15.140:0/2137885045 shutdown_connections 2022-07-21T01:53:17.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5c0f59e700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bec053a70 0x7f5bec055f30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:17.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5c0f59e700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c1013fd60 0x7f5c10136c90 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:17.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5c0f59e700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c100aebb0 0x7f5c101367a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:17.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5c0f59e700 1 --2- 172.21.15.140:0/2137885045 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c100acae0 0x7f5c1013cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:17.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5c0f59e700 1 -- 172.21.15.140:0/2137885045 >> 172.21.15.140:0/2137885045 conn(0x7f5c100634e0 msgr2=0x7f5c10138ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:17.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.587+0000 7f5c0f59e700 1 -- 172.21.15.140:0/2137885045 shutdown_connections 2022-07-21T01:53:17.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:17.591+0000 7f5c0f59e700 1 -- 172.21.15.140:0/2137885045 wait complete. 2022-07-21T01:53:17.999 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:53:18.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9c17f9700 1 Processor -- start 2022-07-21T01:53:18.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9c17f9700 1 -- start start 2022-07-21T01:53:18.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9c17f9700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa9bc140810 0x7fa9bc13a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9c17f9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9bc0b20c0 0x7fa9bc05d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9c17f9700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9bc0b4410 con 0x7fa9bc13a5b0 2022-07-21T01:53:18.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9c17f9700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc141ac0 con 0x7fa9bc140810 2022-07-21T01:53:18.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9c17f9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b4170 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9bb7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa9bc140810 0x7fa9bc13a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9c0ff8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9bc0b20c0 0x7fa9bc05d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9bb7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa9bc140810 0x7fa9bc13a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:41292/0 (socket says 172.21.15.140:41292) 2022-07-21T01:53:18.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9bb7fe700 1 -- 172.21.15.140:0/1552745937 learned_addr learned my addr 172.21.15.140:0/1552745937 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:18.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9bb7fe700 1 -- 172.21.15.140:0/1552745937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9bc0b20c0 msgr2=0x7fa9bc05d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:18.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9bb7fe700 1 --2- 172.21.15.140:0/1552745937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9bc0b20c0 0x7fa9bc05d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9bb7fe700 1 -- 172.21.15.140:0/1552745937 >> v1:172.21.15.152:6789/0 conn(0x7fa9bc13a5b0 legacy=0x7fa9bc0b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:18.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9bb7fe700 1 -- 172.21.15.140:0/1552745937 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc05bfc0 con 0x7fa9bc140810 2022-07-21T01:53:18.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9bb7fe700 1 --2- 172.21.15.140:0/1552745937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9bc140810 0x7fa9bc13a990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fa9bc0ae0f0 tx=0x7fa9b4009290).ready entity=mon.1 client_cookie=efd4afea29835fc8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:18.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9baffd700 1 -- 172.21.15.140:0/1552745937 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b400b860 con 0x7fa9bc140810 2022-07-21T01:53:18.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9baffd700 1 -- 172.21.15.140:0/1552745937 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b400b9c0 con 0x7fa9bc140810 2022-07-21T01:53:18.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.115+0000 7fa9baffd700 1 -- 172.21.15.140:0/1552745937 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9b400bb30 con 0x7fa9bc140810 2022-07-21T01:53:18.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 -- 172.21.15.140:0/1552745937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9bc140810 msgr2=0x7fa9bc13a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:18.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 --2- 172.21.15.140:0/1552745937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9bc140810 0x7fa9bc13a990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fa9bc0ae0f0 tx=0x7fa9b4009290).stop 2022-07-21T01:53:18.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 -- 172.21.15.140:0/1552745937 shutdown_connections 2022-07-21T01:53:18.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 --2- 172.21.15.140:0/1552745937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9bc0b20c0 0x7fa9bc05d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 --2- 172.21.15.140:0/1552745937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9bc140810 0x7fa9bc13a990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 -- 172.21.15.140:0/1552745937 >> 172.21.15.140:0/1552745937 conn(0x7fa9bc064be0 msgr2=0x7fa9bc064fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:18.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 -- 172.21.15.140:0/1552745937 shutdown_connections 2022-07-21T01:53:18.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 -- 172.21.15.140:0/1552745937 wait complete. 2022-07-21T01:53:18.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 Processor -- start 2022-07-21T01:53:18.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 -- start start 2022-07-21T01:53:18.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9bc0b20c0 0x7fa9bc138e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9bc13a5b0 0x7fa9bc145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9bc140810 0x7fa9bc145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc145110 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc13d1d0 con 0x7fa9bc140810 2022-07-21T01:53:18.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c17f9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc13d470 con 0x7fa9bc13a5b0 2022-07-21T01:53:18.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9bbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9bc0b20c0 0x7fa9bc138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9c0ff8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9bc140810 0x7fa9bc145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9bb7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9bc13a5b0 0x7fa9bc145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9bbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9bc0b20c0 0x7fa9bc138e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35556/0 (socket says 172.21.15.140:35556) 2022-07-21T01:53:18.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.119+0000 7fa9bbfff700 1 -- 172.21.15.140:0/2506708007 learned_addr learned my addr 172.21.15.140:0/2506708007 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:18.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9bbfff700 1 -- 172.21.15.140:0/2506708007 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9bc13a5b0 msgr2=0x7fa9bc145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:18.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9bbfff700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9bc13a5b0 0x7fa9bc145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9bbfff700 1 -- 172.21.15.140:0/2506708007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9bc140810 msgr2=0x7fa9bc145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:18.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9bbfff700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9bc140810 0x7fa9bc145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9bbfff700 1 -- 172.21.15.140:0/2506708007 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b4009530 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9c0ff8700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9bc140810 0x7fa9bc145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:18.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9bbfff700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9bc0b20c0 0x7fa9bc138e50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fa9bc05c260 tx=0x7fa9ac007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:18.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9b97fa700 1 -- 172.21.15.140:0/2506708007 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ac007030 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9b97fa700 1 -- 172.21.15.140:0/2506708007 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac007190 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9c17f9700 1 -- 172.21.15.140:0/2506708007 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9bc060ab0 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9b97fa700 1 -- 172.21.15.140:0/2506708007 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ac006510 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9c17f9700 1 -- 172.21.15.140:0/2506708007 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc13cbe0 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9c17f9700 1 -- 172.21.15.140:0/2506708007 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9bc144f50 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.123+0000 7fa9b97fa700 1 -- 172.21.15.140:0/2506708007 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa9ac010030 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.127+0000 7fa9b97fa700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa994053a70 0x7fa994055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.127+0000 7fa9b97fa700 1 -- 172.21.15.140:0/2506708007 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa9ac06baa0 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.127+0000 7fa9bb7fe700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa994053a70 0x7fa994055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.127+0000 7fa9bb7fe700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa994053a70 0x7fa994055f30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fa9b401b000 tx=0x7fa9b4016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:18.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.131+0000 7fa9b97fa700 1 -- 172.21.15.140:0/2506708007 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa9ac0429c0 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.291+0000 7fa9c17f9700 1 -- 172.21.15.140:0/2506708007 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9bc0b3d60 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9b97fa700 1 -- 172.21.15.140:0/2506708007 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+2145 (secure 0 0 0) 0x7fa9ac045fe0 con 0x7fa9bc0b20c0 2022-07-21T01:53:18.292 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:18.292 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":21,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1915503334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1915503334},{"type":"v1","addr":"172.21.15.140:6835","nonce":1915503334}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:51:56.500368+0000","modified":"2022-07-21T01:51:57.504176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5204},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5204":{"gid":5204,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/3908541795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3908541795},{"type":"v1","addr":"172.21.15.152:6833","nonce":3908541795}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-07-21T01:53:18.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 -- 172.21.15.140:0/2506708007 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa994053a70 msgr2=0x7fa994055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:18.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa994053a70 0x7fa994055f30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fa9b401b000 tx=0x7fa9b4016000).stop 2022-07-21T01:53:18.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 -- 172.21.15.140:0/2506708007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9bc0b20c0 msgr2=0x7fa9bc138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:18.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9bc0b20c0 0x7fa9bc138e50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fa9bc05c260 tx=0x7fa9ac007c10).stop 2022-07-21T01:53:18.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 -- 172.21.15.140:0/2506708007 shutdown_connections 2022-07-21T01:53:18.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa994053a70 0x7fa994055f30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9bc140810 0x7fa9bc145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa9bc13a5b0 0x7fa9bc145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 --2- 172.21.15.140:0/2506708007 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa9bc0b20c0 0x7fa9bc138e50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 -- 172.21.15.140:0/2506708007 >> 172.21.15.140:0/2506708007 conn(0x7fa9bc064be0 msgr2=0x7fa9bc0671e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:18.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 -- 172.21.15.140:0/2506708007 shutdown_connections 2022-07-21T01:53:18.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.295+0000 7fa9c17f9700 1 -- 172.21.15.140:0/2506708007 wait complete. 2022-07-21T01:53:18.299 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 21 2022-07-21T01:53:18.304 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:53:18.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd857838700 1 Processor -- start 2022-07-21T01:53:18.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd857838700 1 -- start start 2022-07-21T01:53:18.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd857838700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd8500b7cc0 0x7fd850143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd857838700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd85005d250 0x7fd85013fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd857838700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8500b80a0 con 0x7fd8500b2620 2022-07-21T01:53:18.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd857838700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85005c320 con 0x7fd85005d250 2022-07-21T01:53:18.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd857838700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85005c660 con 0x7fd8500b7cc0 2022-07-21T01:53:18.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd855dd5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd85005d250 0x7fd85013fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd855dd5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd85005d250 0x7fd85013fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35564/0 (socket says 172.21.15.140:35564) 2022-07-21T01:53:18.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd855dd5700 1 -- 172.21.15.140:0/2007022258 learned_addr learned my addr 172.21.15.140:0/2007022258 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:18.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd8555d4700 1 --2- 172.21.15.140:0/2007022258 >> v2:172.21.15.152:3300/0 conn(0x7fd8500b7cc0 0x7fd850143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd855dd5700 1 -- 172.21.15.140:0/2007022258 >> v2:172.21.15.152:3300/0 conn(0x7fd8500b7cc0 msgr2=0x7fd850143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:18.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd855dd5700 1 --2- 172.21.15.140:0/2007022258 >> v2:172.21.15.152:3300/0 conn(0x7fd8500b7cc0 0x7fd850143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd855dd5700 1 -- 172.21.15.140:0/2007022258 >> v1:172.21.15.152:6789/0 conn(0x7fd8500b2620 legacy=0x7fd850066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:18.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd855dd5700 1 -- 172.21.15.140:0/2007022258 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85005bfa0 con 0x7fd85005d250 2022-07-21T01:53:18.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd855dd5700 1 --2- 172.21.15.140:0/2007022258 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85005d250 0x7fd85013fd50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fd8480032f0 tx=0x7fd848009290).ready entity=mon.0 client_cookie=f41a6740927ba6ab server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:18.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd84ffff700 1 -- 172.21.15.140:0/2007022258 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd84800b870 con 0x7fd85005d250 2022-07-21T01:53:18.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd84ffff700 1 -- 172.21.15.140:0/2007022258 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84800b9d0 con 0x7fd85005d250 2022-07-21T01:53:18.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd84ffff700 1 -- 172.21.15.140:0/2007022258 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd84800bb40 con 0x7fd85005d250 2022-07-21T01:53:18.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd857838700 1 -- 172.21.15.140:0/2007022258 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85005d250 msgr2=0x7fd85013fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:18.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.415+0000 7fd857838700 1 --2- 172.21.15.140:0/2007022258 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85005d250 0x7fd85013fd50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fd8480032f0 tx=0x7fd848009290).stop 2022-07-21T01:53:18.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 -- 172.21.15.140:0/2007022258 shutdown_connections 2022-07-21T01:53:18.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 --2- 172.21.15.140:0/2007022258 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85005d250 0x7fd85013fd50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 --2- 172.21.15.140:0/2007022258 >> v2:172.21.15.152:3300/0 conn(0x7fd8500b7cc0 0x7fd850143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 -- 172.21.15.140:0/2007022258 >> 172.21.15.140:0/2007022258 conn(0x7fd850065760 msgr2=0x7fd8500b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:18.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 -- 172.21.15.140:0/2007022258 shutdown_connections 2022-07-21T01:53:18.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 -- 172.21.15.140:0/2007022258 wait complete. 2022-07-21T01:53:18.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 Processor -- start 2022-07-21T01:53:18.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 -- start start 2022-07-21T01:53:18.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85005d250 0x7fd8501520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8500b7cc0 0x7fd8500ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd85013de80 0x7fd850155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500b1460 con 0x7fd85005d250 2022-07-21T01:53:18.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8501525a0 con 0x7fd85013de80 2022-07-21T01:53:18.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd857838700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85013e390 con 0x7fd8500b7cc0 2022-07-21T01:53:18.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd8555d4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85005d250 0x7fd8501520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd8555d4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85005d250 0x7fd8501520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35572/0 (socket says 172.21.15.140:35572) 2022-07-21T01:53:18.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd8555d4700 1 -- 172.21.15.140:0/3001070737 learned_addr learned my addr 172.21.15.140:0/3001070737 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:18.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd855dd5700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd85013de80 0x7fd850155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd854dd3700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8500b7cc0 0x7fd8500ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd8555d4700 1 -- 172.21.15.140:0/3001070737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8500b7cc0 msgr2=0x7fd8500ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:18.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd8555d4700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8500b7cc0 0x7fd8500ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd8555d4700 1 -- 172.21.15.140:0/3001070737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd85013de80 msgr2=0x7fd850155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:18.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd8555d4700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd85013de80 0x7fd850155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:18.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd8555d4700 1 -- 172.21.15.140:0/3001070737 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848009530 con 0x7fd85005d250 2022-07-21T01:53:18.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd8555d4700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85005d250 0x7fd8501520b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fd850152730 tx=0x7fd840006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:18.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.419+0000 7fd84e7fc700 1 -- 172.21.15.140:0/3001070737 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd840010780 con 0x7fd85005d250 2022-07-21T01:53:18.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.423+0000 7fd84e7fc700 1 -- 172.21.15.140:0/3001070737 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8400108e0 con 0x7fd85005d250 2022-07-21T01:53:18.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.423+0000 7fd857838700 1 -- 172.21.15.140:0/3001070737 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85013e630 con 0x7fd85005d250 2022-07-21T01:53:18.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.423+0000 7fd84e7fc700 1 -- 172.21.15.140:0/3001070737 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd840010c40 con 0x7fd85005d250 2022-07-21T01:53:18.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.423+0000 7fd857838700 1 -- 172.21.15.140:0/3001070737 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8500aeac0 con 0x7fd85005d250 2022-07-21T01:53:18.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.423+0000 7fd84e7fc700 1 -- 172.21.15.140:0/3001070737 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd84001c460 con 0x7fd85005d250 2022-07-21T01:53:18.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.423+0000 7fd857838700 1 -- 172.21.15.140:0/3001070737 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8500574e0 con 0x7fd85005d250 2022-07-21T01:53:18.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.423+0000 7fd84e7fc700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd830053a70 0x7fd830055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:18.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.423+0000 7fd854dd3700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd830053a70 0x7fd830055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:18.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.423+0000 7fd84e7fc700 1 -- 172.21.15.140:0/3001070737 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd84006c2e0 con 0x7fd85005d250 2022-07-21T01:53:18.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.423+0000 7fd854dd3700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd830053a70 0x7fd830055f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fd84400d000 tx=0x7fd844007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:18.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.427+0000 7fd84e7fc700 1 -- 172.21.15.140:0/3001070737 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd840043090 con 0x7fd85005d250 2022-07-21T01:53:18.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:18.591+0000 7fd857838700 1 -- 172.21.15.140:0/3001070737 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd850141890 con 0x7fd85005d250 2022-07-21T01:53:19.055 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:53:19.059+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:53:19.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.063+0000 7fd84e7fc700 1 -- 172.21.15.140:0/3001070737 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd8400466b0 con 0x7fd85005d250 2022-07-21T01:53:19.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 -- 172.21.15.140:0/3001070737 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd830053a70 msgr2=0x7fd830055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd830053a70 0x7fd830055f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fd84400d000 tx=0x7fd844007b90).stop 2022-07-21T01:53:19.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 -- 172.21.15.140:0/3001070737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85005d250 msgr2=0x7fd8501520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85005d250 0x7fd8501520b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fd850152730 tx=0x7fd840006df0).stop 2022-07-21T01:53:19.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 -- 172.21.15.140:0/3001070737 shutdown_connections 2022-07-21T01:53:19.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd830053a70 0x7fd830055f30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd85013de80 0x7fd850155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8500b7cc0 0x7fd8500ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 --2- 172.21.15.140:0/3001070737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85005d250 0x7fd8501520b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 -- 172.21.15.140:0/3001070737 >> 172.21.15.140:0/3001070737 conn(0x7fd850065760 msgr2=0x7fd8500b5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:19.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 -- 172.21.15.140:0/3001070737 shutdown_connections 2022-07-21T01:53:19.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.071+0000 7fd857838700 1 -- 172.21.15.140:0/3001070737 wait complete. 2022-07-21T01:53:19.071 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:53:19.082 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:53:19.100 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:53:19.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 Processor -- start 2022-07-21T01:53:19.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 -- start start 2022-07-21T01:53:19.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb7c80b20b0 0x7fb7c8147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7c81480a0 0x7fb7c8146590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c80ae440 con 0x7fb7c8146a80 2022-07-21T01:53:19.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c80ad7c0 con 0x7fb7c80b20b0 2022-07-21T01:53:19.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80ada60 con 0x7fb7c81480a0 2022-07-21T01:53:19.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c6d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb7c80b20b0 0x7fb7c8147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c759e700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fb7c8146a80 0x7fb7c805bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49728/0 (socket says 172.21.15.140:49728) 2022-07-21T01:53:19.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c6d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb7c80b20b0 0x7fb7c8147bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35578/0 (socket says 172.21.15.140:35578) 2022-07-21T01:53:19.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c759e700 1 -- 172.21.15.140:0/2263132213 learned_addr learned my addr 172.21.15.140:0/2263132213 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:19.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c659c700 1 --2- 172.21.15.140:0/2263132213 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7c81480a0 0x7fb7c8146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c6d9d700 1 -- 172.21.15.140:0/2263132213 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7c81480a0 msgr2=0x7fb7c8146590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:19.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c6d9d700 1 --2- 172.21.15.140:0/2263132213 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7c81480a0 0x7fb7c8146590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c6d9d700 1 -- 172.21.15.140:0/2263132213 >> v1:172.21.15.140:6789/0 conn(0x7fb7c8146a80 legacy=0x7fb7c805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c6d9d700 1 -- 172.21.15.140:0/2263132213 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c80add00 con 0x7fb7c80b20b0 2022-07-21T01:53:19.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c659c700 1 --2- 172.21.15.140:0/2263132213 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7c81480a0 0x7fb7c8146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:19.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c6d9d700 1 --2- 172.21.15.140:0/2263132213 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7c80b20b0 0x7fb7c8147bb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fb7b0002f70 tx=0x7fb7b0006df0).ready entity=mon.0 client_cookie=b1f695211f99c246 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:19.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c5d9b700 1 -- 172.21.15.140:0/2263132213 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7b000d570 con 0x7fb7c80b20b0 2022-07-21T01:53:19.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c5d9b700 1 -- 172.21.15.140:0/2263132213 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b0000e90 con 0x7fb7c80b20b0 2022-07-21T01:53:19.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7c5d9b700 1 -- 172.21.15.140:0/2263132213 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7b000d8c0 con 0x7fb7c80b20b0 2022-07-21T01:53:19.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 -- 172.21.15.140:0/2263132213 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7c80b20b0 msgr2=0x7fb7c8147bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 --2- 172.21.15.140:0/2263132213 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7c80b20b0 0x7fb7c8147bb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fb7b0002f70 tx=0x7fb7b0006df0).stop 2022-07-21T01:53:19.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 -- 172.21.15.140:0/2263132213 shutdown_connections 2022-07-21T01:53:19.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 --2- 172.21.15.140:0/2263132213 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7c81480a0 0x7fb7c8146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 --2- 172.21.15.140:0/2263132213 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7c80b20b0 0x7fb7c8147bb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 -- 172.21.15.140:0/2263132213 >> 172.21.15.140:0/2263132213 conn(0x7fb7c80b72c0 msgr2=0x7fb7c80b76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:19.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.191+0000 7fb7cd012700 1 -- 172.21.15.140:0/2263132213 shutdown_connections 2022-07-21T01:53:19.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7cd012700 1 -- 172.21.15.140:0/2263132213 wait complete. 2022-07-21T01:53:19.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7cd012700 1 Processor -- start 2022-07-21T01:53:19.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7cd012700 1 -- start start 2022-07-21T01:53:19.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7cd012700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7c80b20b0 0x7fb7c80b6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7cd012700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7c8146a80 0x7fb7c8156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7cd012700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7c81480a0 0x7fb7c8156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7cd012700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80aff50 con 0x7fb7c81480a0 2022-07-21T01:53:19.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7cd012700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c8156850 con 0x7fb7c80b20b0 2022-07-21T01:53:19.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7cd012700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c80b0140 con 0x7fb7c8146a80 2022-07-21T01:53:19.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7c759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7c81480a0 0x7fb7c8156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7c6d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7c80b20b0 0x7fb7c80b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7c659c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7c8146a80 0x7fb7c8156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7c759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7c81480a0 0x7fb7c8156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35588/0 (socket says 172.21.15.140:35588) 2022-07-21T01:53:19.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7c759e700 1 -- 172.21.15.140:0/3569472216 learned_addr learned my addr 172.21.15.140:0/3569472216 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:19.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7c759e700 1 -- 172.21.15.140:0/3569472216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7c8146a80 msgr2=0x7fb7c8156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7c759e700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7c8146a80 0x7fb7c8156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7c759e700 1 -- 172.21.15.140:0/3569472216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7c80b20b0 msgr2=0x7fb7c80b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7c759e700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7c80b20b0 0x7fb7c80b6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7c759e700 1 -- 172.21.15.140:0/3569472216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b000b180 con 0x7fb7c81480a0 2022-07-21T01:53:19.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.195+0000 7fb7c759e700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7c81480a0 0x7fb7c8156be0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fb7c80619f0 tx=0x7fb7b8007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:19.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.199+0000 7fb7bffff700 1 -- 172.21.15.140:0/3569472216 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7b80070d0 con 0x7fb7c81480a0 2022-07-21T01:53:19.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.199+0000 7fb7bffff700 1 -- 172.21.15.140:0/3569472216 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b8007230 con 0x7fb7c81480a0 2022-07-21T01:53:19.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.199+0000 7fb7cd012700 1 -- 172.21.15.140:0/3569472216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c8148aa0 con 0x7fb7c81480a0 2022-07-21T01:53:19.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.199+0000 7fb7bffff700 1 -- 172.21.15.140:0/3569472216 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7b8005610 con 0x7fb7c81480a0 2022-07-21T01:53:19.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.199+0000 7fb7cd012700 1 -- 172.21.15.140:0/3569472216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c8149a50 con 0x7fb7c81480a0 2022-07-21T01:53:19.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.199+0000 7fb7cd012700 1 -- 172.21.15.140:0/3569472216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c80b0340 con 0x7fb7c81480a0 2022-07-21T01:53:19.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.199+0000 7fb7bffff700 1 -- 172.21.15.140:0/3569472216 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb7b8010070 con 0x7fb7c81480a0 2022-07-21T01:53:19.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.199+0000 7fb7bffff700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb7ac053a70 0x7fb7ac055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.199+0000 7fb7bffff700 1 -- 172.21.15.140:0/3569472216 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb7b806b470 con 0x7fb7c81480a0 2022-07-21T01:53:19.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.199+0000 7fb7c6d9d700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb7ac053a70 0x7fb7ac055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.199+0000 7fb7c6d9d700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb7ac053a70 0x7fb7ac055f30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fb7b001c000 tx=0x7fb7b0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:19.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.207+0000 7fb7bffff700 1 -- 172.21.15.140:0/3569472216 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb7b8042230 con 0x7fb7c81480a0 2022-07-21T01:53:19.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.363+0000 7fb7cd012700 1 -- 172.21.15.140:0/3569472216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7c80adfa0 con 0x7fb7c81480a0 2022-07-21T01:53:19.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.363+0000 7fb7bffff700 1 -- 172.21.15.140:0/3569472216 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+1781 (secure 0 0 0) 0x7fb7b8045850 con 0x7fb7c81480a0 2022-07-21T01:53:19.362 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:19.362 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":22,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1915503334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1915503334},{"type":"v1","addr":"172.21.15.140:6835","nonce":1915503334}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:51:56.500368+0000","modified":"2022-07-21T01:53:19.061477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2022-07-21T01:53:19.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 -- 172.21.15.140:0/3569472216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb7ac053a70 msgr2=0x7fb7ac055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb7ac053a70 0x7fb7ac055f30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fb7b001c000 tx=0x7fb7b0015000).stop 2022-07-21T01:53:19.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 -- 172.21.15.140:0/3569472216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7c81480a0 msgr2=0x7fb7c8156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7c81480a0 0x7fb7c8156be0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7fb7c80619f0 tx=0x7fb7b8007bc0).stop 2022-07-21T01:53:19.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 -- 172.21.15.140:0/3569472216 shutdown_connections 2022-07-21T01:53:19.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb7ac053a70 0x7fb7ac055f30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7c81480a0 0x7fb7c8156be0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7c8146a80 0x7fb7c8156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 --2- 172.21.15.140:0/3569472216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7c80b20b0 0x7fb7c80b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 -- 172.21.15.140:0/3569472216 >> 172.21.15.140:0/3569472216 conn(0x7fb7c80b72c0 msgr2=0x7fb7c8065ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:19.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 -- 172.21.15.140:0/3569472216 shutdown_connections 2022-07-21T01:53:19.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.367+0000 7fb7cd012700 1 -- 172.21.15.140:0/3569472216 wait complete. 2022-07-21T01:53:19.368 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 22 2022-07-21T01:53:19.374 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:53:19.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b7fff700 1 Processor -- start 2022-07-21T01:53:19.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b7fff700 1 -- start start 2022-07-21T01:53:19.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f85b80b1060 0x7f85b813f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85b80acae0 0x7f85b813fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b7fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b80ada70 con 0x7f85b80b5970 2022-07-21T01:53:19.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b7fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b80b5d50 con 0x7f85b80b1060 2022-07-21T01:53:19.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80af8c0 con 0x7f85b80acae0 2022-07-21T01:53:19.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b67fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f85b80b1060 0x7f85b813f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b67fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f85b80b1060 0x7f85b813f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35594/0 (socket says 172.21.15.140:35594) 2022-07-21T01:53:19.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b67fc700 1 -- 172.21.15.140:0/504886497 learned_addr learned my addr 172.21.15.140:0/504886497 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:19.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b77fe700 1 --2- 172.21.15.140:0/504886497 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85b80acae0 0x7f85b813fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b67fc700 1 -- 172.21.15.140:0/504886497 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85b80acae0 msgr2=0x7f85b813fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:19.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b67fc700 1 --2- 172.21.15.140:0/504886497 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85b80acae0 0x7f85b813fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b67fc700 1 -- 172.21.15.140:0/504886497 >> v1:172.21.15.140:6789/0 conn(0x7f85b80b5970 legacy=0x7f85b813c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b5ffb700 1 -- 172.21.15.140:0/504886497 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216191444 0 0) 0x7f85b80ada70 con 0x7f85b80b5970 2022-07-21T01:53:19.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b67fc700 1 -- 172.21.15.140:0/504886497 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b80adcd0 con 0x7f85b80b1060 2022-07-21T01:53:19.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.483+0000 7f85b67fc700 1 --2- 172.21.15.140:0/504886497 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85b80b1060 0x7f85b813f8c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f85a0002f70 tx=0x7f85a0006eb0).ready entity=mon.0 client_cookie=d313a1e78f89ac6b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:19.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b5ffb700 1 -- 172.21.15.140:0/504886497 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85a000d5c0 con 0x7f85b80b1060 2022-07-21T01:53:19.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b5ffb700 1 -- 172.21.15.140:0/504886497 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0000e90 con 0x7f85b80b1060 2022-07-21T01:53:19.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b5ffb700 1 -- 172.21.15.140:0/504886497 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85a000d910 con 0x7f85b80b1060 2022-07-21T01:53:19.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b7fff700 1 -- 172.21.15.140:0/504886497 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85b80b1060 msgr2=0x7f85b813f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b7fff700 1 --2- 172.21.15.140:0/504886497 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85b80b1060 0x7f85b813f8c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f85a0002f70 tx=0x7f85a0006eb0).stop 2022-07-21T01:53:19.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b7fff700 1 -- 172.21.15.140:0/504886497 shutdown_connections 2022-07-21T01:53:19.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b7fff700 1 --2- 172.21.15.140:0/504886497 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85b80acae0 0x7f85b813fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b7fff700 1 --2- 172.21.15.140:0/504886497 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85b80b1060 0x7f85b813f8c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b7fff700 1 -- 172.21.15.140:0/504886497 >> 172.21.15.140:0/504886497 conn(0x7f85b8063490 msgr2=0x7f85b80b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:19.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b7fff700 1 -- 172.21.15.140:0/504886497 shutdown_connections 2022-07-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b7fff700 1 -- 172.21.15.140:0/504886497 wait complete. 2022-07-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b7fff700 1 Processor -- start 2022-07-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.487+0000 7f85b7fff700 1 -- start start 2022-07-21T01:53:19.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85b80acae0 0x7f85b813cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85b80b1060 0x7f85b81383b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85b80b5970 0x7f85b81388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b7fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8066500 con 0x7f85b80b5970 2022-07-21T01:53:19.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b7fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8064370 con 0x7f85b80b1060 2022-07-21T01:53:19.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b7fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8138d90 con 0x7f85b80acae0 2022-07-21T01:53:19.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85b80b5970 0x7f85b81388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85b80b5970 0x7f85b81388a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35602/0 (socket says 172.21.15.140:35602) 2022-07-21T01:53:19.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b77fe700 1 -- 172.21.15.140:0/4087792779 learned_addr learned my addr 172.21.15.140:0/4087792779 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:19.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b67fc700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85b80b1060 0x7f85b81383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b6ffd700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85b80acae0 0x7f85b813cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b77fe700 1 -- 172.21.15.140:0/4087792779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85b80acae0 msgr2=0x7f85b813cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b77fe700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85b80acae0 0x7f85b813cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b77fe700 1 -- 172.21.15.140:0/4087792779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85b80b1060 msgr2=0x7f85b81383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:19.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b77fe700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85b80b1060 0x7f85b81383b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:19.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b77fe700 1 -- 172.21.15.140:0/4087792779 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a000b1e0 con 0x7f85b80b5970 2022-07-21T01:53:19.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b77fe700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85b80b5970 0x7f85b81388a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f85b000a000 tx=0x7f85b0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:19.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85affff700 1 -- 172.21.15.140:0/4087792779 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85b0009b00 con 0x7f85b80b5970 2022-07-21T01:53:19.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85affff700 1 -- 172.21.15.140:0/4087792779 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b0009c60 con 0x7f85b80b5970 2022-07-21T01:53:19.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85affff700 1 -- 172.21.15.140:0/4087792779 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85b001a470 con 0x7f85b80b5970 2022-07-21T01:53:19.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b7fff700 1 -- 172.21.15.140:0/4087792779 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b8063e60 con 0x7f85b80b5970 2022-07-21T01:53:19.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.491+0000 7f85b7fff700 1 -- 172.21.15.140:0/4087792779 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b8152100 con 0x7f85b80b5970 2022-07-21T01:53:19.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.495+0000 7f85b7fff700 1 -- 172.21.15.140:0/4087792779 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b80646f0 con 0x7f85b80b5970 2022-07-21T01:53:19.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.495+0000 7f85affff700 1 -- 172.21.15.140:0/4087792779 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f85b001a5d0 con 0x7f85b80b5970 2022-07-21T01:53:19.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.495+0000 7f85affff700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8590053b30 0x7f8590055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:19.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.495+0000 7f85affff700 1 -- 172.21.15.140:0/4087792779 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f85b006e9f0 con 0x7f85b80b5970 2022-07-21T01:53:19.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.495+0000 7f85b6ffd700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8590053b30 0x7f8590055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:19.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.495+0000 7f85b6ffd700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8590053b30 0x7f8590055ff0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f85a800d000 tx=0x7f85a8007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:19.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.503+0000 7f85affff700 1 -- 172.21.15.140:0/4087792779 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f85b0070e60 con 0x7f85b80b5970 2022-07-21T01:53:19.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:19.667+0000 7f85b7fff700 1 -- 172.21.15.140:0/4087792779 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f85b80af4c0 con 0x7f85b80b5970 2022-07-21T01:53:20.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.087+0000 7f85affff700 1 -- 172.21.15.140:0/4087792779 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f85b0045860 con 0x7f85b80b5970 2022-07-21T01:53:20.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 -- 172.21.15.140:0/4087792779 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8590053b30 msgr2=0x7f8590055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:20.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8590053b30 0x7f8590055ff0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f85a800d000 tx=0x7f85a8007b90).stop 2022-07-21T01:53:20.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 -- 172.21.15.140:0/4087792779 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85b80b5970 msgr2=0x7f85b81388a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:20.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85b80b5970 0x7f85b81388a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f85b000a000 tx=0x7f85b0008040).stop 2022-07-21T01:53:20.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 -- 172.21.15.140:0/4087792779 shutdown_connections 2022-07-21T01:53:20.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8590053b30 0x7f8590055ff0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:20.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85b80b5970 0x7f85b81388a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:20.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85b80b1060 0x7f85b81383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:20.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 --2- 172.21.15.140:0/4087792779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85b80acae0 0x7f85b813cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:20.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 -- 172.21.15.140:0/4087792779 >> 172.21.15.140:0/4087792779 conn(0x7f85b8063490 msgr2=0x7f85b80b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:20.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 -- 172.21.15.140:0/4087792779 shutdown_connections 2022-07-21T01:53:20.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.091+0000 7f85b7fff700 1 -- 172.21.15.140:0/4087792779 wait complete. 2022-07-21T01:53:20.102 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:53:20.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.219+0000 7fe815c24700 1 Processor -- start 2022-07-21T01:53:20.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.219+0000 7fe815c24700 1 -- start start 2022-07-21T01:53:20.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815c24700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe8100b1090 0x7fe81013c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:20.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815c24700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe8100b1b00 0x7fe81013cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:20.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815c24700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81005d060 con 0x7fe810142a10 2022-07-21T01:53:20.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815c24700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81005ca80 con 0x7fe8100b1b00 2022-07-21T01:53:20.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815c24700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe810067450 con 0x7fe8100b1090 2022-07-21T01:53:20.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815423700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe8100b1b00 0x7fe81013cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:20.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe80ffff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe810142a10 0x7fe810146280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49752/0 (socket says 172.21.15.140:49752) 2022-07-21T01:53:20.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815423700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe8100b1b00 0x7fe81013cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35610/0 (socket says 172.21.15.140:35610) 2022-07-21T01:53:20.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe80ffff700 1 -- 172.21.15.140:0/2108860822 learned_addr learned my addr 172.21.15.140:0/2108860822 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:20.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe814c22700 1 --2- 172.21.15.140:0/2108860822 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe8100b1090 0x7fe81013c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:20.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815423700 1 -- 172.21.15.140:0/2108860822 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe8100b1090 msgr2=0x7fe81013c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:20.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815423700 1 --2- 172.21.15.140:0/2108860822 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe8100b1090 0x7fe81013c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:20.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815423700 1 -- 172.21.15.140:0/2108860822 >> v1:172.21.15.140:6789/0 conn(0x7fe810142a10 legacy=0x7fe810146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:20.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815423700 1 -- 172.21.15.140:0/2108860822 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8100676f0 con 0x7fe8100b1b00 2022-07-21T01:53:20.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe814c22700 1 --2- 172.21.15.140:0/2108860822 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe8100b1090 0x7fe81013c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:20.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe815423700 1 --2- 172.21.15.140:0/2108860822 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe8100b1b00 0x7fe81013cb00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fe8080032f0 tx=0x7fe8080091d0).ready entity=mon.0 client_cookie=c9b88e4040273de7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:20.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe80f7fe700 1 -- 172.21.15.140:0/2108860822 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe808009ea0 con 0x7fe8100b1b00 2022-07-21T01:53:20.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe80f7fe700 1 -- 172.21.15.140:0/2108860822 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80800b8a0 con 0x7fe8100b1b00 2022-07-21T01:53:20.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.223+0000 7fe80f7fe700 1 -- 172.21.15.140:0/2108860822 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe80800ba10 con 0x7fe8100b1b00 2022-07-21T01:53:20.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.227+0000 7fe815c24700 1 -- 172.21.15.140:0/2108860822 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe8100b1b00 msgr2=0x7fe81013cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:20.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.227+0000 7fe815c24700 1 --2- 172.21.15.140:0/2108860822 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe8100b1b00 0x7fe81013cb00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fe8080032f0 tx=0x7fe8080091d0).stop 2022-07-21T01:53:20.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.227+0000 7fe815c24700 1 -- 172.21.15.140:0/2108860822 shutdown_connections 2022-07-21T01:53:20.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.227+0000 7fe815c24700 1 --2- 172.21.15.140:0/2108860822 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe8100b1b00 0x7fe81013cb00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:20.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.227+0000 7fe815c24700 1 --2- 172.21.15.140:0/2108860822 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe8100b1090 0x7fe81013c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:20.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.227+0000 7fe815c24700 1 -- 172.21.15.140:0/2108860822 >> 172.21.15.140:0/2108860822 conn(0x7fe810064c00 msgr2=0x7fe810064fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:20.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.227+0000 7fe815c24700 1 -- 172.21.15.140:0/2108860822 shutdown_connections 2022-07-21T01:53:20.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.227+0000 7fe815c24700 1 -- 172.21.15.140:0/2108860822 wait complete. 2022-07-21T01:53:20.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.227+0000 7fe815c24700 1 Processor -- start 2022-07-21T01:53:20.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.227+0000 7fe815c24700 1 -- start start 2022-07-21T01:53:20.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe815c24700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe8100b1090 0x7fe810111540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:20.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe815c24700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8100b1b00 0x7fe810111c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:20.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe815c24700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe810155210 0x7fe810145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:20.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe815c24700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8101556c0 con 0x7fe8100b1090 2022-07-21T01:53:20.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe815c24700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe810151ae0 con 0x7fe8100b1b00 2022-07-21T01:53:20.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe815c24700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8101122a0 con 0x7fe810155210 2022-07-21T01:53:20.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe814c22700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe8100b1090 0x7fe810111540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:20.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe814c22700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe8100b1090 0x7fe810111540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35612/0 (socket says 172.21.15.140:35612) 2022-07-21T01:53:20.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe814c22700 1 -- 172.21.15.140:0/726194550 learned_addr learned my addr 172.21.15.140:0/726194550 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:20.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe815423700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe810155210 0x7fe810145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:20.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe80ffff700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8100b1b00 0x7fe810111c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:20.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe814c22700 1 -- 172.21.15.140:0/726194550 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe810155210 msgr2=0x7fe810145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:20.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe814c22700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe810155210 0x7fe810145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:20.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe814c22700 1 -- 172.21.15.140:0/726194550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8100b1b00 msgr2=0x7fe810111c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:20.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe814c22700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8100b1b00 0x7fe810111c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:20.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe814c22700 1 -- 172.21.15.140:0/726194550 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe808009470 con 0x7fe8100b1090 2022-07-21T01:53:20.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe80ffff700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8100b1b00 0x7fe810111c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:20.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe814c22700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe8100b1090 0x7fe810111540 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fe810112ad0 tx=0x7fe800008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:20.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe80dffb700 1 -- 172.21.15.140:0/726194550 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe800011780 con 0x7fe8100b1090 2022-07-21T01:53:20.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe80dffb700 1 -- 172.21.15.140:0/726194550 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8000118e0 con 0x7fe8100b1090 2022-07-21T01:53:20.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe815c24700 1 -- 172.21.15.140:0/726194550 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe810113ad0 con 0x7fe8100b1090 2022-07-21T01:53:20.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe80dffb700 1 -- 172.21.15.140:0/726194550 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe800011c40 con 0x7fe8100b1090 2022-07-21T01:53:20.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.231+0000 7fe815c24700 1 -- 172.21.15.140:0/726194550 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe810112540 con 0x7fe8100b1090 2022-07-21T01:53:20.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.235+0000 7fe815c24700 1 -- 172.21.15.140:0/726194550 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8101120f0 con 0x7fe8100b1090 2022-07-21T01:53:20.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.235+0000 7fe80dffb700 1 -- 172.21.15.140:0/726194550 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe800012420 con 0x7fe8100b1090 2022-07-21T01:53:20.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.235+0000 7fe80dffb700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe7f0053aa0 0x7fe7f0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:20.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.235+0000 7fe80dffb700 1 -- 172.21.15.140:0/726194550 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe80006bd90 con 0x7fe8100b1090 2022-07-21T01:53:20.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.235+0000 7fe80ffff700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe7f0053aa0 0x7fe7f0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:20.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.235+0000 7fe80ffff700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe7f0053aa0 0x7fe7f0055f60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fe80400b000 tx=0x7fe804009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:20.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.243+0000 7fe80dffb700 1 -- 172.21.15.140:0/726194550 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe800042c00 con 0x7fe8100b1090 2022-07-21T01:53:20.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:20.367+0000 7fe815c24700 1 -- 172.21.15.140:0/726194550 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe810113c60 con 0x7fe8100b1090 2022-07-21T01:53:21.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.095+0000 7fe80dffb700 1 -- 172.21.15.140:0/726194550 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 185+0+0 (secure 0 0 0) 0x7fe800046220 con 0x7fe8100b1090 2022-07-21T01:53:21.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 -- 172.21.15.140:0/726194550 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe7f0053aa0 msgr2=0x7fe7f0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:21.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe7f0053aa0 0x7fe7f0055f60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fe80400b000 tx=0x7fe804009040).stop 2022-07-21T01:53:21.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 -- 172.21.15.140:0/726194550 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe8100b1090 msgr2=0x7fe810111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:21.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe8100b1090 0x7fe810111540 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fe810112ad0 tx=0x7fe800008040).stop 2022-07-21T01:53:21.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 -- 172.21.15.140:0/726194550 shutdown_connections 2022-07-21T01:53:21.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe7f0053aa0 0x7fe7f0055f60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:21.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe810155210 0x7fe810145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:21.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8100b1b00 0x7fe810111c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:21.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 --2- 172.21.15.140:0/726194550 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe8100b1090 0x7fe810111540 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:21.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 -- 172.21.15.140:0/726194550 >> 172.21.15.140:0/726194550 conn(0x7fe810064c00 msgr2=0x7fe81013ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:21.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 -- 172.21.15.140:0/726194550 shutdown_connections 2022-07-21T01:53:21.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.099+0000 7fe815c24700 1 -- 172.21.15.140:0/726194550 wait complete. 2022-07-21T01:53:21.105 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:53:21.113 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:53:21.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.227+0000 7f0bf9060700 1 Processor -- start 2022-07-21T01:53:21.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.227+0000 7f0bf9060700 1 -- start start 2022-07-21T01:53:21.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bf4142d80 0x7f0bf40ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:21.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0bf40b1490 0x7f0bf40aec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:21.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf40afd20 con 0x7f0bf40b5990 2022-07-21T01:53:21.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf40affc0 con 0x7f0bf40b1490 2022-07-21T01:53:21.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40ad2d0 con 0x7f0bf4142d80 2022-07-21T01:53:21.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf359e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0bf40b1490 0x7f0bf40aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:21.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf2d9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0bf40b5990 0x7f0bf413c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49766/0 (socket says 172.21.15.140:49766) 2022-07-21T01:53:21.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf2d9d700 1 -- 172.21.15.140:0/3806547059 learned_addr learned my addr 172.21.15.140:0/3806547059 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:21.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf259c700 1 --2- 172.21.15.140:0/3806547059 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bf4142d80 0x7f0bf40ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:21.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf359e700 1 -- 172.21.15.140:0/3806547059 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bf4142d80 msgr2=0x7f0bf40ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:21.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf359e700 1 --2- 172.21.15.140:0/3806547059 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bf4142d80 0x7f0bf40ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:21.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf359e700 1 -- 172.21.15.140:0/3806547059 >> v1:172.21.15.140:6789/0 conn(0x7f0bf40b5990 legacy=0x7f0bf413c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:21.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf359e700 1 -- 172.21.15.140:0/3806547059 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf40b5d70 con 0x7f0bf40b1490 2022-07-21T01:53:21.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf359e700 1 --2- 172.21.15.140:0/3806547059 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bf40b1490 0x7f0bf40aec50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f0bdc001110 tx=0x7f0bec0091d0).ready entity=mon.0 client_cookie=2dcdc7b70aaf122e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:21.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf1d9b700 1 -- 172.21.15.140:0/3806547059 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bec00b580 con 0x7f0bf40b1490 2022-07-21T01:53:21.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf1d9b700 1 -- 172.21.15.140:0/3806547059 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec00b6e0 con 0x7f0bf40b1490 2022-07-21T01:53:21.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf1d9b700 1 -- 172.21.15.140:0/3806547059 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bec00b850 con 0x7f0bf40b1490 2022-07-21T01:53:21.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 -- 172.21.15.140:0/3806547059 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bf40b1490 msgr2=0x7f0bf40aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:21.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 --2- 172.21.15.140:0/3806547059 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bf40b1490 0x7f0bf40aec50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f0bdc001110 tx=0x7f0bec0091d0).stop 2022-07-21T01:53:21.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 -- 172.21.15.140:0/3806547059 shutdown_connections 2022-07-21T01:53:21.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 --2- 172.21.15.140:0/3806547059 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bf40b1490 0x7f0bf40aec50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:21.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 --2- 172.21.15.140:0/3806547059 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bf4142d80 0x7f0bf40ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:21.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 -- 172.21.15.140:0/3806547059 >> 172.21.15.140:0/3806547059 conn(0x7f0bf40635b0 msgr2=0x7f0bf40b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:21.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 -- 172.21.15.140:0/3806547059 shutdown_connections 2022-07-21T01:53:21.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.231+0000 7f0bf9060700 1 -- 172.21.15.140:0/3806547059 wait complete. 2022-07-21T01:53:21.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.235+0000 7f0bf9060700 1 Processor -- start 2022-07-21T01:53:21.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.235+0000 7f0bf9060700 1 -- start start 2022-07-21T01:53:21.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.235+0000 7f0bf9060700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bf40b1490 0x7f0bf4146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:21.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.235+0000 7f0bf9060700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf40b5990 0x7f0bf413ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:21.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.235+0000 7f0bf9060700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bf4142d80 0x7f0bf413b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:21.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.235+0000 7f0bf9060700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4067790 con 0x7f0bf4142d80 2022-07-21T01:53:21.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.235+0000 7f0bf9060700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4066550 con 0x7f0bf40b5990 2022-07-21T01:53:21.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.235+0000 7f0bf9060700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40667f0 con 0x7f0bf40b1490 2022-07-21T01:53:21.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bf4142d80 0x7f0bf413b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:21.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bf4142d80 0x7f0bf413b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35630/0 (socket says 172.21.15.140:35630) 2022-07-21T01:53:21.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf359e700 1 -- 172.21.15.140:0/1527588782 learned_addr learned my addr 172.21.15.140:0/1527588782 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:21.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf259c700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf40b5990 0x7f0bf413ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:21.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf2d9d700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bf40b1490 0x7f0bf4146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:21.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf359e700 1 -- 172.21.15.140:0/1527588782 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bf40b1490 msgr2=0x7f0bf4146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:21.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf359e700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bf40b1490 0x7f0bf4146350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:21.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf359e700 1 -- 172.21.15.140:0/1527588782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf40b5990 msgr2=0x7f0bf413ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:21.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf359e700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf40b5990 0x7f0bf413ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:21.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf359e700 1 -- 172.21.15.140:0/1527588782 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec009470 con 0x7f0bf4142d80 2022-07-21T01:53:21.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf359e700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bf4142d80 0x7f0bf413b240 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f0bdc001110 tx=0x7f0bec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:21.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bebfff700 1 -- 172.21.15.140:0/1527588782 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bec011b60 con 0x7f0bf4142d80 2022-07-21T01:53:21.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bebfff700 1 -- 172.21.15.140:0/1527588782 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec011cc0 con 0x7f0bf4142d80 2022-07-21T01:53:21.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf9060700 1 -- 172.21.15.140:0/1527588782 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf4146de0 con 0x7f0bf4142d80 2022-07-21T01:53:21.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bebfff700 1 -- 172.21.15.140:0/1527588782 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bec01f5e0 con 0x7f0bf4142d80 2022-07-21T01:53:21.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf9060700 1 -- 172.21.15.140:0/1527588782 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf413b730 con 0x7f0bf4142d80 2022-07-21T01:53:21.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bf9060700 1 -- 172.21.15.140:0/1527588782 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf4066b90 con 0x7f0bf4142d80 2022-07-21T01:53:21.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.239+0000 7f0bebfff700 1 -- 172.21.15.140:0/1527588782 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0bec026030 con 0x7f0bf4142d80 2022-07-21T01:53:21.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.243+0000 7f0bebfff700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bd8053b00 0x7f0bd8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:21.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.243+0000 7f0bebfff700 1 -- 172.21.15.140:0/1527588782 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f0bec06c800 con 0x7f0bf4142d80 2022-07-21T01:53:21.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.243+0000 7f0bf2d9d700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bd8053b00 0x7f0bd8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:21.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.243+0000 7f0bf2d9d700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bd8053b00 0x7f0bd8055fc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f0bdc00d000 tx=0x7f0bdc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:21.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.247+0000 7f0bebfff700 1 -- 172.21.15.140:0/1527588782 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0bec043880 con 0x7f0bf4142d80 2022-07-21T01:53:21.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:21.379+0000 7f0bf9060700 1 -- 172.21.15.140:0/1527588782 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0bf40574e0 con 0x7f0bf4142d80 2022-07-21T01:53:22.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.107+0000 7f0bebfff700 1 -- 172.21.15.140:0/1527588782 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0bec0240d0 con 0x7f0bf4142d80 2022-07-21T01:53:22.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 -- 172.21.15.140:0/1527588782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bd8053b00 msgr2=0x7f0bd8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bd8053b00 0x7f0bd8055fc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f0bdc00d000 tx=0x7f0bdc009040).stop 2022-07-21T01:53:22.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 -- 172.21.15.140:0/1527588782 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bf4142d80 msgr2=0x7f0bf413b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bf4142d80 0x7f0bf413b240 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f0bdc001110 tx=0x7f0bec016000).stop 2022-07-21T01:53:22.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 -- 172.21.15.140:0/1527588782 shutdown_connections 2022-07-21T01:53:22.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bd8053b00 0x7f0bd8055fc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bf4142d80 0x7f0bf413b240 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf40b5990 0x7f0bf413ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 --2- 172.21.15.140:0/1527588782 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bf40b1490 0x7f0bf4146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 -- 172.21.15.140:0/1527588782 >> 172.21.15.140:0/1527588782 conn(0x7f0bf40635b0 msgr2=0x7f0bf40b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:22.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 -- 172.21.15.140:0/1527588782 shutdown_connections 2022-07-21T01:53:22.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.115+0000 7f0bf9060700 1 -- 172.21.15.140:0/1527588782 wait complete. 2022-07-21T01:53:22.116 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:53:22.122 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-07-21T01:53:22.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2dbfff700 1 Processor -- start 2022-07-21T01:53:22.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2dbfff700 1 -- start start 2022-07-21T01:53:22.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2dbfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff2dc0acae0 0x7ff2dc14d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2dbfff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc0b46e0 con 0x7ff2dc0b5860 2022-07-21T01:53:22.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2dbfff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc0b4b10 con 0x7ff2dc0b4f80 2022-07-21T01:53:22.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2dbfff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2dc0ad990 con 0x7ff2dc0acae0 2022-07-21T01:53:22.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2daffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff2dc0b5860 0x7ff2dc146060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49780/0 (socket says 172.21.15.140:49780) 2022-07-21T01:53:22.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2daffd700 1 -- 172.21.15.140:0/1171038347 learned_addr learned my addr 172.21.15.140:0/1171038347 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:22.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2db7fe700 1 --2- 172.21.15.140:0/1171038347 >> v2:172.21.15.152:3300/0 conn(0x7ff2dc0acae0 0x7ff2dc14d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027569638 0 0) 0x7ff2dc0b46e0 con 0x7ff2dc0b5860 2022-07-21T01:53:22.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c0003260 con 0x7ff2dc0b5860 2022-07-21T01:53:22.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2991811679 0 0) 0x7ff2c0003260 con 0x7ff2dc0b5860 2022-07-21T01:53:22.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 >> v2:172.21.15.152:3300/0 conn(0x7ff2dc0acae0 msgr2=0x7ff2dc14d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:22.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 --2- 172.21.15.140:0/1171038347 >> v2:172.21.15.152:3300/0 conn(0x7ff2dc0acae0 0x7ff2dc14d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 >> v1:172.21.15.152:6789/0 conn(0x7ff2dc0b4f80 legacy=0x7ff2dc146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc0ad3d0 con 0x7ff2dc0b5860 2022-07-21T01:53:22.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7ff2d40019e0 con 0x7ff2dc0b5860 2022-07-21T01:53:22.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 >> v1:172.21.15.140:6789/0 conn(0x7ff2dc0b5860 legacy=0x7ff2dc146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 --2- 172.21.15.140:0/1171038347 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2c0003750 0x7ff2c0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 --2- 172.21.15.140:0/1171038347 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 0x7ff2c0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 --2- 172.21.15.140:0/1171038347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2c0004e70 0x7ff2c000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0ad990 con 0x7ff2c0004500 2022-07-21T01:53:22.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c000bdd0 con 0x7ff2c0004e70 2022-07-21T01:53:22.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c000bf60 con 0x7ff2c0003750 2022-07-21T01:53:22.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.231+0000 7ff2da7fc700 1 --2- 172.21.15.140:0/1171038347 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 0x7ff2c0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2daffd700 1 --2- 172.21.15.140:0/1171038347 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2c0003750 0x7ff2c0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2db7fe700 1 --2- 172.21.15.140:0/1171038347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2c0004e70 0x7ff2c000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2da7fc700 1 -- 172.21.15.140:0/1171038347 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2c0003750 msgr2=0x7ff2c0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:22.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2da7fc700 1 --2- 172.21.15.140:0/1171038347 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2c0003750 0x7ff2c0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2da7fc700 1 -- 172.21.15.140:0/1171038347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2c0004e70 msgr2=0x7ff2c000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:22.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2da7fc700 1 --2- 172.21.15.140:0/1171038347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2c0004e70 0x7ff2c000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2db7fe700 1 -- 172.21.15.140:0/1171038347 reap_dead start 2022-07-21T01:53:22.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2da7fc700 1 -- 172.21.15.140:0/1171038347 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c0003260 con 0x7ff2c0004500 2022-07-21T01:53:22.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2da7fc700 1 --2- 172.21.15.140:0/1171038347 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 0x7ff2c0004980 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7ff2dc0b3be0 tx=0x7ff2d0007a00).ready entity=mon.0 client_cookie=21cf533ddd872e41 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:22.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2d0007d10 con 0x7ff2c0004500 2022-07-21T01:53:22.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d0007030 con 0x7ff2c0004500 2022-07-21T01:53:22.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2d9ffb700 1 -- 172.21.15.140:0/1171038347 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2d0007390 con 0x7ff2c0004500 2022-07-21T01:53:22.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 -- 172.21.15.140:0/1171038347 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 msgr2=0x7ff2c0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 --2- 172.21.15.140:0/1171038347 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 0x7ff2c0004980 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7ff2dc0b3be0 tx=0x7ff2d0007a00).stop 2022-07-21T01:53:22.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 -- 172.21.15.140:0/1171038347 shutdown_connections 2022-07-21T01:53:22.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 --2- 172.21.15.140:0/1171038347 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 0x7ff2c0004980 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 -- 172.21.15.140:0/1171038347 >> 172.21.15.140:0/1171038347 conn(0x7ff2dc0b5e10 msgr2=0x7ff2dc0b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:22.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 -- 172.21.15.140:0/1171038347 shutdown_connections 2022-07-21T01:53:22.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 -- 172.21.15.140:0/1171038347 wait complete. 2022-07-21T01:53:22.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 Processor -- start 2022-07-21T01:53:22.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 -- start start 2022-07-21T01:53:22.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 0x7ff2dc0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2dc0b8080 0x7ff2dc13d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2dc13d5f0 0x7ff2dc151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0b18f0 con 0x7ff2c0004500 2022-07-21T01:53:22.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0b1b20 con 0x7ff2dc0b8080 2022-07-21T01:53:22.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2dbfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0af5b0 con 0x7ff2dc13d5f0 2022-07-21T01:53:22.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2daffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 0x7ff2dc0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2db7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2dc13d5f0 0x7ff2dc151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2daffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 0x7ff2dc0afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35646/0 (socket says 172.21.15.140:35646) 2022-07-21T01:53:22.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2daffd700 1 -- 172.21.15.140:0/2000640654 learned_addr learned my addr 172.21.15.140:0/2000640654 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:22.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.235+0000 7ff2da7fc700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2dc0b8080 0x7ff2dc13d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2daffd700 1 -- 172.21.15.140:0/2000640654 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2dc13d5f0 msgr2=0x7ff2dc151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2daffd700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2dc13d5f0 0x7ff2dc151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2daffd700 1 -- 172.21.15.140:0/2000640654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2dc0b8080 msgr2=0x7ff2dc13d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2daffd700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2dc0b8080 0x7ff2dc13d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2daffd700 1 -- 172.21.15.140:0/2000640654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc13f310 con 0x7ff2c0004500 2022-07-21T01:53:22.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2da7fc700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2dc0b8080 0x7ff2dc13d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:22.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2daffd700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 0x7ff2dc0afa50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7ff2d4001c40 tx=0x7ff2d4002190).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:22.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2cbfff700 1 -- 172.21.15.140:0/2000640654 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2d400f820 con 0x7ff2c0004500 2022-07-21T01:53:22.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2cbfff700 1 -- 172.21.15.140:0/2000640654 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d400f980 con 0x7ff2c0004500 2022-07-21T01:53:22.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2dbfff700 1 -- 172.21.15.140:0/2000640654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc13f710 con 0x7ff2c0004500 2022-07-21T01:53:22.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2cbfff700 1 -- 172.21.15.140:0/2000640654 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2d400fce0 con 0x7ff2c0004500 2022-07-21T01:53:22.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2dbfff700 1 -- 172.21.15.140:0/2000640654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc0b36a0 con 0x7ff2c0004500 2022-07-21T01:53:22.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2dbfff700 1 -- 172.21.15.140:0/2000640654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2dc13f4a0 con 0x7ff2c0004500 2022-07-21T01:53:22.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.239+0000 7ff2cbfff700 1 -- 172.21.15.140:0/2000640654 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff2d401b030 con 0x7ff2c0004500 2022-07-21T01:53:22.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.243+0000 7ff2cbfff700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff2b8053b00 0x7ff2b8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.243+0000 7ff2cbfff700 1 -- 172.21.15.140:0/2000640654 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5339+0+0 (secure 0 0 0) 0x7ff2d406ba10 con 0x7ff2c0004500 2022-07-21T01:53:22.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.243+0000 7ff2da7fc700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff2b8053b00 0x7ff2b8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.243+0000 7ff2da7fc700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff2b8053b00 0x7ff2b8055fc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7ff2d001b000 tx=0x7ff2d0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:22.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.247+0000 7ff2cbfff700 1 -- 172.21.15.140:0/2000640654 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff2d4042c10 con 0x7ff2c0004500 2022-07-21T01:53:22.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.379+0000 7ff2dbfff700 1 -- 172.21.15.140:0/2000640654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff2dc0af770 con 0x7ff2c0004500 2022-07-21T01:53:22.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2cbfff700 1 -- 172.21.15.140:0/2000640654 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff2d4019070 con 0x7ff2c0004500 2022-07-21T01:53:22.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 -- 172.21.15.140:0/2000640654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff2b8053b00 msgr2=0x7ff2b8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff2b8053b00 0x7ff2b8055fc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7ff2d001b000 tx=0x7ff2d0015000).stop 2022-07-21T01:53:22.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 -- 172.21.15.140:0/2000640654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 msgr2=0x7ff2dc0afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 0x7ff2dc0afa50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7ff2d4001c40 tx=0x7ff2d4002190).stop 2022-07-21T01:53:22.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 -- 172.21.15.140:0/2000640654 shutdown_connections 2022-07-21T01:53:22.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff2b8053b00 0x7ff2b8055fc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff2dc13d5f0 0x7ff2dc151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff2dc0b8080 0x7ff2dc13d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 --2- 172.21.15.140:0/2000640654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff2c0004500 0x7ff2dc0afa50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 -- 172.21.15.140:0/2000640654 >> 172.21.15.140:0/2000640654 conn(0x7ff2dc0b5e10 msgr2=0x7ff2dc141590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:22.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 -- 172.21.15.140:0/2000640654 shutdown_connections 2022-07-21T01:53:22.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.387+0000 7ff2dbfff700 1 -- 172.21.15.140:0/2000640654 wait complete. 2022-07-21T01:53:22.398 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2022-07-21T01:53:22.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdeca34b700 1 Processor -- start 2022-07-21T01:53:22.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdeca34b700 1 -- start start 2022-07-21T01:53:22.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdeca34b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdec40afa10 0x7fdec40b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdeca34b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdec40b3de0 0x7fdec41452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdeca34b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec40b42a0 con 0x7fdec40608b0 2022-07-21T01:53:22.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdeca34b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec4060c90 con 0x7fdec40afa10 2022-07-21T01:53:22.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdeca34b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40af410 con 0x7fdec40b3de0 2022-07-21T01:53:22.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec37fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdec40afa10 0x7fdec40b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec88e8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdec40b3de0 0x7fdec41452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec37fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdec40afa10 0x7fdec40b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:41396/0 (socket says 172.21.15.140:41396) 2022-07-21T01:53:22.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec3fff700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdec40608b0 0x7fdec405bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:50914/0 (socket says 172.21.15.140:50914) 2022-07-21T01:53:22.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec37fe700 1 -- 172.21.15.140:0/3803209290 learned_addr learned my addr 172.21.15.140:0/3803209290 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:22.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec88e8700 1 -- 172.21.15.140:0/3803209290 >> v2:172.21.15.152:3300/0 conn(0x7fdec40afa10 msgr2=0x7fdec40b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:22.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec88e8700 1 --2- 172.21.15.140:0/3803209290 >> v2:172.21.15.152:3300/0 conn(0x7fdec40afa10 0x7fdec40b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec88e8700 1 -- 172.21.15.140:0/3803209290 >> v1:172.21.15.152:6789/0 conn(0x7fdec40608b0 legacy=0x7fdec405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec2ffd700 1 -- 172.21.15.140:0/3803209290 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947647187 0 0) 0x7fdec40b42a0 con 0x7fdec40608b0 2022-07-21T01:53:22.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec88e8700 1 -- 172.21.15.140:0/3803209290 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec40af6b0 con 0x7fdec40b3de0 2022-07-21T01:53:22.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec37fe700 1 --2- 172.21.15.140:0/3803209290 >> v2:172.21.15.152:3300/0 conn(0x7fdec40afa10 0x7fdec40b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:22.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec88e8700 1 --2- 172.21.15.140:0/3803209290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdec40b3de0 0x7fdec41452d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fdebc002f70 tx=0x7fdebc006df0).ready entity=mon.2 client_cookie=37833f8d357683a5 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:22.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec2ffd700 1 -- 172.21.15.140:0/3803209290 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdebc00d570 con 0x7fdec40b3de0 2022-07-21T01:53:22.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec2ffd700 1 -- 172.21.15.140:0/3803209290 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc000e90 con 0x7fdec40b3de0 2022-07-21T01:53:22.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.503+0000 7fdec2ffd700 1 -- 172.21.15.140:0/3803209290 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdebc00d8c0 con 0x7fdec40b3de0 2022-07-21T01:53:22.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 -- 172.21.15.140:0/3803209290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdec40b3de0 msgr2=0x7fdec41452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 --2- 172.21.15.140:0/3803209290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdec40b3de0 0x7fdec41452d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fdebc002f70 tx=0x7fdebc006df0).stop 2022-07-21T01:53:22.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 -- 172.21.15.140:0/3803209290 shutdown_connections 2022-07-21T01:53:22.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 --2- 172.21.15.140:0/3803209290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdec40b3de0 0x7fdec41452d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 --2- 172.21.15.140:0/3803209290 >> v2:172.21.15.152:3300/0 conn(0x7fdec40afa10 0x7fdec40b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 -- 172.21.15.140:0/3803209290 >> 172.21.15.140:0/3803209290 conn(0x7fdec40b5ee0 msgr2=0x7fdec40b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:22.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 -- 172.21.15.140:0/3803209290 shutdown_connections 2022-07-21T01:53:22.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 -- 172.21.15.140:0/3803209290 wait complete. 2022-07-21T01:53:22.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 Processor -- start 2022-07-21T01:53:22.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 -- start start 2022-07-21T01:53:22.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdec40608b0 0x7fdec409d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdec40afa10 0x7fdec4152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdec40b3de0 0x7fdec4091970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4093560 con 0x7fdec40afa10 2022-07-21T01:53:22.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4092d40 con 0x7fdec40b3de0 2022-07-21T01:53:22.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdeca34b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4151bf0 con 0x7fdec40608b0 2022-07-21T01:53:22.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdec37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdec40afa10 0x7fdec4152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdec37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdec40afa10 0x7fdec4152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35662/0 (socket says 172.21.15.140:35662) 2022-07-21T01:53:22.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdec37fe700 1 -- 172.21.15.140:0/4033682315 learned_addr learned my addr 172.21.15.140:0/4033682315 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:22.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdec3fff700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdec40608b0 0x7fdec409d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.507+0000 7fdec88e8700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdec40b3de0 0x7fdec4091970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec37fe700 1 -- 172.21.15.140:0/4033682315 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdec40608b0 msgr2=0x7fdec409d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec37fe700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdec40608b0 0x7fdec409d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec37fe700 1 -- 172.21.15.140:0/4033682315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdec40b3de0 msgr2=0x7fdec4091970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:22.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec37fe700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdec40b3de0 0x7fdec4091970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:22.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec37fe700 1 -- 172.21.15.140:0/4033682315 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb4009530 con 0x7fdec40afa10 2022-07-21T01:53:22.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec37fe700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdec40afa10 0x7fdec4152800 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fdeb400c000 tx=0x7fdeb4009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:22.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec17fa700 1 -- 172.21.15.140:0/4033682315 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdeb400c920 con 0x7fdec40afa10 2022-07-21T01:53:22.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec17fa700 1 -- 172.21.15.140:0/4033682315 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb400ca80 con 0x7fdec40afa10 2022-07-21T01:53:22.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdeca34b700 1 -- 172.21.15.140:0/4033682315 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc00b180 con 0x7fdec40afa10 2022-07-21T01:53:22.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec17fa700 1 -- 172.21.15.140:0/4033682315 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdeb400cde0 con 0x7fdec40afa10 2022-07-21T01:53:22.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdeca34b700 1 -- 172.21.15.140:0/4033682315 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec41520c0 con 0x7fdec40afa10 2022-07-21T01:53:22.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdeca34b700 1 -- 172.21.15.140:0/4033682315 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec4152500 con 0x7fdec40afa10 2022-07-21T01:53:22.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec17fa700 1 -- 172.21.15.140:0/4033682315 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdeb401b400 con 0x7fdec40afa10 2022-07-21T01:53:22.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec17fa700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde9c053b00 0x7fde9c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:22.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec17fa700 1 -- 172.21.15.140:0/4033682315 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fdeb406c6a0 con 0x7fdec40afa10 2022-07-21T01:53:22.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.511+0000 7fdec3fff700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde9c053b00 0x7fde9c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:22.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.515+0000 7fdec3fff700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde9c053b00 0x7fde9c055fc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fdeac00d000 tx=0x7fdeac009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:22.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.515+0000 7fdec17fa700 1 -- 172.21.15.140:0/4033682315 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdeb4043840 con 0x7fdec40afa10 2022-07-21T01:53:22.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:22.683+0000 7fdeca34b700 1 -- 172.21.15.140:0/4033682315 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7fdec40574e0 con 0x7fdec40afa10 2022-07-21T01:53:23.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.107+0000 7fdec17fa700 1 -- 172.21.15.140:0/4033682315 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v211) v1 ==== 152+0+0 (secure 0 0 0) 0x7fdeb4046e60 con 0x7fdec40afa10 2022-07-21T01:53:23.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 -- 172.21.15.140:0/4033682315 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde9c053b00 msgr2=0x7fde9c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:23.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde9c053b00 0x7fde9c055fc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fdeac00d000 tx=0x7fdeac009040).stop 2022-07-21T01:53:23.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 -- 172.21.15.140:0/4033682315 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdec40afa10 msgr2=0x7fdec4152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:23.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdec40afa10 0x7fdec4152800 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fdeb400c000 tx=0x7fdeb4009290).stop 2022-07-21T01:53:23.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 -- 172.21.15.140:0/4033682315 shutdown_connections 2022-07-21T01:53:23.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fde9c053b00 0x7fde9c055fc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:23.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdec40b3de0 0x7fdec4091970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:23.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdec40afa10 0x7fdec4152800 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:23.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 --2- 172.21.15.140:0/4033682315 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdec40608b0 0x7fdec409d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:23.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 -- 172.21.15.140:0/4033682315 >> 172.21.15.140:0/4033682315 conn(0x7fdec40b5ee0 msgr2=0x7fdec40b8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:23.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 -- 172.21.15.140:0/4033682315 shutdown_connections 2022-07-21T01:53:23.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.111+0000 7fdeca34b700 1 -- 172.21.15.140:0/4033682315 wait complete. 2022-07-21T01:53:23.125 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-07-21T01:53:23.125 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2022-07-21T01:53:23.126 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2022-07-21T01:53:23.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.231+0000 7f2d8484b700 1 Processor -- start 2022-07-21T01:53:23.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.231+0000 7f2d8484b700 1 -- start start 2022-07-21T01:53:23.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.231+0000 7f2d8484b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2d800b7b70 0x7f2d800b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:23.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.231+0000 7f2d8484b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d80066160 0x7f2d8013df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:23.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.231+0000 7f2d8484b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d800b7f50 con 0x7f2d800b31e0 2022-07-21T01:53:23.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.231+0000 7f2d8484b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d800b2800 con 0x7f2d800b7b70 2022-07-21T01:53:23.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.231+0000 7f2d8484b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800b2aa0 con 0x7f2d80066160 2022-07-21T01:53:23.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.231+0000 7f2d7ed9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2d800b7b70 0x7f2d800b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:23.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.231+0000 7f2d7e59c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2d800b31e0 0x7f2d8013d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49814/0 (socket says 172.21.15.140:49814) 2022-07-21T01:53:23.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.231+0000 7f2d7e59c700 1 -- 172.21.15.140:0/3674617600 learned_addr learned my addr 172.21.15.140:0/3674617600 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:23.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d7f59e700 1 --2- 172.21.15.140:0/3674617600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d80066160 0x7f2d8013df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:23.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d7dd9b700 1 -- 172.21.15.140:0/3674617600 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761515390 0 0) 0x7f2d800b7f50 con 0x7f2d800b31e0 2022-07-21T01:53:23.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d7dd9b700 1 -- 172.21.15.140:0/3674617600 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d64003260 con 0x7f2d800b31e0 2022-07-21T01:53:23.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d7ed9d700 1 -- 172.21.15.140:0/3674617600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d80066160 msgr2=0x7f2d8013df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:23.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d7ed9d700 1 --2- 172.21.15.140:0/3674617600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d80066160 0x7f2d8013df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:23.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d7ed9d700 1 -- 172.21.15.140:0/3674617600 >> v1:172.21.15.140:6789/0 conn(0x7f2d800b31e0 legacy=0x7f2d8013d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:23.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d7ed9d700 1 -- 172.21.15.140:0/3674617600 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d800b2d40 con 0x7f2d800b7b70 2022-07-21T01:53:23.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d7ed9d700 1 --2- 172.21.15.140:0/3674617600 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d800b7b70 0x7f2d800b1ac0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f2d70002f70 tx=0x7f2d70006df0).ready entity=mon.0 client_cookie=e200476f78a7f8a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:23.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d7dd9b700 1 -- 172.21.15.140:0/3674617600 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d7000d4a0 con 0x7f2d800b7b70 2022-07-21T01:53:23.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d7dd9b700 1 -- 172.21.15.140:0/3674617600 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d70000e90 con 0x7f2d800b7b70 2022-07-21T01:53:23.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d7dd9b700 1 -- 172.21.15.140:0/3674617600 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d7000d7f0 con 0x7f2d800b7b70 2022-07-21T01:53:23.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d8484b700 1 -- 172.21.15.140:0/3674617600 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d800b7b70 msgr2=0x7f2d800b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:23.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d8484b700 1 --2- 172.21.15.140:0/3674617600 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d800b7b70 0x7f2d800b1ac0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f2d70002f70 tx=0x7f2d70006df0).stop 2022-07-21T01:53:23.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d8484b700 1 -- 172.21.15.140:0/3674617600 shutdown_connections 2022-07-21T01:53:23.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d8484b700 1 --2- 172.21.15.140:0/3674617600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d80066160 0x7f2d8013df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:23.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d8484b700 1 --2- 172.21.15.140:0/3674617600 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d800b7b70 0x7f2d800b1ac0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:23.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d8484b700 1 -- 172.21.15.140:0/3674617600 >> 172.21.15.140:0/3674617600 conn(0x7f2d800656b0 msgr2=0x7f2d800b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:23.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d8484b700 1 -- 172.21.15.140:0/3674617600 shutdown_connections 2022-07-21T01:53:23.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d8484b700 1 -- 172.21.15.140:0/3674617600 wait complete. 2022-07-21T01:53:23.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d8484b700 1 Processor -- start 2022-07-21T01:53:23.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.235+0000 7f2d8484b700 1 -- start start 2022-07-21T01:53:23.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d8484b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d80066160 0x7f2d8014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:23.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d8484b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d800b31e0 0x7f2d801500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:23.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d8484b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d800b7b70 0x7f2d80113ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:23.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d8484b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d801508b0 con 0x7f2d800b31e0 2022-07-21T01:53:23.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d8484b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80150b50 con 0x7f2d800b7b70 2022-07-21T01:53:23.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d8484b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d801103b0 con 0x7f2d80066160 2022-07-21T01:53:23.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d7e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d800b31e0 0x7f2d801500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:23.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d7e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d800b31e0 0x7f2d801500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35676/0 (socket says 172.21.15.140:35676) 2022-07-21T01:53:23.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d7e59c700 1 -- 172.21.15.140:0/697338979 learned_addr learned my addr 172.21.15.140:0/697338979 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:23.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d7ed9d700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d80066160 0x7f2d8014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:23.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d7f59e700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d800b7b70 0x7f2d80113ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:23.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d7e59c700 1 -- 172.21.15.140:0/697338979 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d80066160 msgr2=0x7f2d8014fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:23.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d7e59c700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d80066160 0x7f2d8014fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:23.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d7e59c700 1 -- 172.21.15.140:0/697338979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d800b7b70 msgr2=0x7f2d80113ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:23.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d7e59c700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d800b7b70 0x7f2d80113ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:23.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d7e59c700 1 -- 172.21.15.140:0/697338979 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7000b180 con 0x7f2d800b31e0 2022-07-21T01:53:23.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d7e59c700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d800b31e0 0x7f2d801500f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f2d74001110 tx=0x7f2d74007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:23.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d6ffff700 1 -- 172.21.15.140:0/697338979 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d74007110 con 0x7f2d800b31e0 2022-07-21T01:53:23.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d8484b700 1 -- 172.21.15.140:0/697338979 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d801505e0 con 0x7f2d800b31e0 2022-07-21T01:53:23.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d6ffff700 1 -- 172.21.15.140:0/697338979 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d74007270 con 0x7f2d800b31e0 2022-07-21T01:53:23.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d6ffff700 1 -- 172.21.15.140:0/697338979 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d74005030 con 0x7f2d800b31e0 2022-07-21T01:53:23.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d8484b700 1 -- 172.21.15.140:0/697338979 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d80110bb0 con 0x7f2d800b31e0 2022-07-21T01:53:23.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.239+0000 7f2d8484b700 1 -- 172.21.15.140:0/697338979 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d80110570 con 0x7f2d800b31e0 2022-07-21T01:53:23.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.243+0000 7f2d6ffff700 1 -- 172.21.15.140:0/697338979 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2d7401b070 con 0x7f2d800b31e0 2022-07-21T01:53:23.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.243+0000 7f2d6ffff700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d5c053aa0 0x7f2d5c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:23.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.243+0000 7f2d6ffff700 1 -- 172.21.15.140:0/697338979 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f2d7406b120 con 0x7f2d800b31e0 2022-07-21T01:53:23.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.243+0000 7f2d7ed9d700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d5c053aa0 0x7f2d5c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:23.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.243+0000 7f2d7ed9d700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d5c053aa0 0x7f2d5c055f60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f2d7001a000 tx=0x7f2d70014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:23.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.247+0000 7f2d6ffff700 1 -- 172.21.15.140:0/697338979 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d74068e90 con 0x7f2d800b31e0 2022-07-21T01:53:23.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:23.423+0000 7f2d8484b700 1 -- 172.21.15.140:0/697338979 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f2d800574e0 con 0x7f2d800b31e0 2022-07-21T01:53:24.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.111+0000 7f2d6ffff700 1 -- 172.21.15.140:0/697338979 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v212) v1 ==== 150+0+0 (secure 0 0 0) 0x7f2d74047070 con 0x7f2d800b31e0 2022-07-21T01:53:24.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.115+0000 7f2d8484b700 1 -- 172.21.15.140:0/697338979 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d5c053aa0 msgr2=0x7f2d5c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.115+0000 7f2d8484b700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d5c053aa0 0x7f2d5c055f60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f2d7001a000 tx=0x7f2d70014000).stop 2022-07-21T01:53:24.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.115+0000 7f2d8484b700 1 -- 172.21.15.140:0/697338979 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d800b31e0 msgr2=0x7f2d801500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.115+0000 7f2d8484b700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d800b31e0 0x7f2d801500f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f2d74001110 tx=0x7f2d74007a40).stop 2022-07-21T01:53:24.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.115+0000 7f2d8484b700 1 -- 172.21.15.140:0/697338979 shutdown_connections 2022-07-21T01:53:24.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.115+0000 7f2d8484b700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d5c053aa0 0x7f2d5c055f60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.115+0000 7f2d8484b700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d800b7b70 0x7f2d80113ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.115+0000 7f2d8484b700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d800b31e0 0x7f2d801500f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.115+0000 7f2d8484b700 1 --2- 172.21.15.140:0/697338979 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d80066160 0x7f2d8014fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.115+0000 7f2d8484b700 1 -- 172.21.15.140:0/697338979 >> 172.21.15.140:0/697338979 conn(0x7f2d800656b0 msgr2=0x7f2d8013fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:24.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.115+0000 7f2d8484b700 1 -- 172.21.15.140:0/697338979 shutdown_connections 2022-07-21T01:53:24.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.119+0000 7f2d8484b700 1 -- 172.21.15.140:0/697338979 wait complete. 2022-07-21T01:53:24.130 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:53:24.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.239+0000 7f071228c700 1 Processor -- start 2022-07-21T01:53:24.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.239+0000 7f071228c700 1 -- start start 2022-07-21T01:53:24.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f070c0b5920 0x7f070c13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:24.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f070c0b4ee0 0x7f070c142530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:24.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f070c0b5d00 con 0x7f070c0ab990 2022-07-21T01:53:24.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070c13f090 con 0x7f070c0b4ee0 2022-07-21T01:53:24.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070c13f330 con 0x7f070c0b5920 2022-07-21T01:53:24.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f0710a89700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f070c0b4ee0 0x7f070c142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:24.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f0710a89700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f070c0b4ee0 0x7f070c142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35682/0 (socket says 172.21.15.140:35682) 2022-07-21T01:53:24.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f0710a89700 1 -- 172.21.15.140:0/4118087432 learned_addr learned my addr 172.21.15.140:0/4118087432 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:24.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071128a700 1 --2- 172.21.15.140:0/4118087432 >> v2:172.21.15.152:3300/0 conn(0x7f070c0b5920 0x7f070c13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:24.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f0710a89700 1 -- 172.21.15.140:0/4118087432 >> v2:172.21.15.152:3300/0 conn(0x7f070c0b5920 msgr2=0x7f070c13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:24.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f0710a89700 1 --2- 172.21.15.140:0/4118087432 >> v2:172.21.15.152:3300/0 conn(0x7f070c0b5920 0x7f070c13fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f0710a89700 1 -- 172.21.15.140:0/4118087432 >> v1:172.21.15.152:6789/0 conn(0x7f070c0ab990 legacy=0x7f070c13e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f0710a89700 1 -- 172.21.15.140:0/4118087432 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c13f5d0 con 0x7f070c0b4ee0 2022-07-21T01:53:24.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071128a700 1 --2- 172.21.15.140:0/4118087432 >> v2:172.21.15.152:3300/0 conn(0x7f070c0b5920 0x7f070c13fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:24.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f0710a89700 1 --2- 172.21.15.140:0/4118087432 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f070c0b4ee0 0x7f070c142530 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f07080032f0 tx=0x7f070800a430).ready entity=mon.0 client_cookie=3caf01d29886418a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:24.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f0703fff700 1 -- 172.21.15.140:0/4118087432 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f070800cca0 con 0x7f070c0b4ee0 2022-07-21T01:53:24.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f0703fff700 1 -- 172.21.15.140:0/4118087432 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070800ce00 con 0x7f070c0b4ee0 2022-07-21T01:53:24.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f0703fff700 1 -- 172.21.15.140:0/4118087432 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0708010600 con 0x7f070c0b4ee0 2022-07-21T01:53:24.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 -- 172.21.15.140:0/4118087432 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f070c0b4ee0 msgr2=0x7f070c142530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 --2- 172.21.15.140:0/4118087432 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f070c0b4ee0 0x7f070c142530 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f07080032f0 tx=0x7f070800a430).stop 2022-07-21T01:53:24.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 -- 172.21.15.140:0/4118087432 shutdown_connections 2022-07-21T01:53:24.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 --2- 172.21.15.140:0/4118087432 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f070c0b4ee0 0x7f070c142530 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 --2- 172.21.15.140:0/4118087432 >> v2:172.21.15.152:3300/0 conn(0x7f070c0b5920 0x7f070c13fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 -- 172.21.15.140:0/4118087432 >> 172.21.15.140:0/4118087432 conn(0x7f070c0632b0 msgr2=0x7f070c063690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:24.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 -- 172.21.15.140:0/4118087432 shutdown_connections 2022-07-21T01:53:24.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 -- 172.21.15.140:0/4118087432 wait complete. 2022-07-21T01:53:24.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 Processor -- start 2022-07-21T01:53:24.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.243+0000 7f071228c700 1 -- start start 2022-07-21T01:53:24.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071228c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f070c0ab990 0x7f070c13d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:24.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071228c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f070c0b4ee0 0x7f070c09aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:24.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071228c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f070c0b5920 0x7f070c0ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:24.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071228c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c067240 con 0x7f070c0b4ee0 2022-07-21T01:53:24.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071228c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c064640 con 0x7f070c0b5920 2022-07-21T01:53:24.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071228c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0648e0 con 0x7f070c0ab990 2022-07-21T01:53:24.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f0710a89700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f070c0b4ee0 0x7f070c09aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:24.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f0710a89700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f070c0b4ee0 0x7f070c09aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35690/0 (socket says 172.21.15.140:35690) 2022-07-21T01:53:24.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f0710a89700 1 -- 172.21.15.140:0/1276549602 learned_addr learned my addr 172.21.15.140:0/1276549602 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:24.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f0711a8b700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f070c0b5920 0x7f070c0ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:24.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071128a700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f070c0ab990 0x7f070c13d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:24.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f0710a89700 1 -- 172.21.15.140:0/1276549602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f070c0ab990 msgr2=0x7f070c13d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f0710a89700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f070c0ab990 0x7f070c13d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f0710a89700 1 -- 172.21.15.140:0/1276549602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f070c0b5920 msgr2=0x7f070c0ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f0710a89700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f070c0b5920 0x7f070c0ad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f0710a89700 1 -- 172.21.15.140:0/1276549602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070800a6d0 con 0x7f070c0b4ee0 2022-07-21T01:53:24.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f0710a89700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f070c0b4ee0 0x7f070c09aba0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f07080032f0 tx=0x7f070c0b4a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:24.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f07027fc700 1 -- 172.21.15.140:0/1276549602 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f070801a670 con 0x7f070c0b4ee0 2022-07-21T01:53:24.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f07027fc700 1 -- 172.21.15.140:0/1276549602 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070801a7d0 con 0x7f070c0b4ee0 2022-07-21T01:53:24.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071228c700 1 -- 172.21.15.140:0/1276549602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070c09b090 con 0x7f070c0b4ee0 2022-07-21T01:53:24.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f07027fc700 1 -- 172.21.15.140:0/1276549602 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f070801ab30 con 0x7f070c0b4ee0 2022-07-21T01:53:24.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071228c700 1 -- 172.21.15.140:0/1276549602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070c13b3f0 con 0x7f070c0b4ee0 2022-07-21T01:53:24.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f07027fc700 1 -- 172.21.15.140:0/1276549602 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f070800e040 con 0x7f070c0b4ee0 2022-07-21T01:53:24.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f07027fc700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f06ec053a70 0x7f06ec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:24.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f07027fc700 1 -- 172.21.15.140:0/1276549602 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f070806ddc0 con 0x7f070c0b4ee0 2022-07-21T01:53:24.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071128a700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f06ec053a70 0x7f06ec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:24.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071228c700 1 -- 172.21.15.140:0/1276549602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070c064c60 con 0x7f070c0b4ee0 2022-07-21T01:53:24.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.247+0000 7f071128a700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f06ec053a70 0x7f06ec055f30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f06fc00c000 tx=0x7f06fc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:24.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.255+0000 7f07027fc700 1 -- 172.21.15.140:0/1276549602 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0708044fc0 con 0x7f070c0b4ee0 2022-07-21T01:53:24.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.379+0000 7f071228c700 1 -- 172.21.15.140:0/1276549602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f070c0574e0 con 0x7f070c0b4ee0 2022-07-21T01:53:24.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.387+0000 7f07027fc700 1 -- 172.21.15.140:0/1276549602 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f07080485e0 con 0x7f070c0b4ee0 2022-07-21T01:53:24.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 -- 172.21.15.140:0/1276549602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f06ec053a70 msgr2=0x7f06ec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f06ec053a70 0x7f06ec055f30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f06fc00c000 tx=0x7f06fc008040).stop 2022-07-21T01:53:24.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 -- 172.21.15.140:0/1276549602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f070c0b4ee0 msgr2=0x7f070c09aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f070c0b4ee0 0x7f070c09aba0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f07080032f0 tx=0x7f070c0b4a60).stop 2022-07-21T01:53:24.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 -- 172.21.15.140:0/1276549602 shutdown_connections 2022-07-21T01:53:24.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f06ec053a70 0x7f06ec055f30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f070c0b5920 0x7f070c0ad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f070c0b4ee0 0x7f070c09aba0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 --2- 172.21.15.140:0/1276549602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f070c0ab990 0x7f070c13d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 -- 172.21.15.140:0/1276549602 >> 172.21.15.140:0/1276549602 conn(0x7f070c0632b0 msgr2=0x7f070c0aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:24.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 -- 172.21.15.140:0/1276549602 shutdown_connections 2022-07-21T01:53:24.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.391+0000 7f071228c700 1 -- 172.21.15.140:0/1276549602 wait complete. 2022-07-21T01:53:24.401 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:53:24.402 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:24.420 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:24.440 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:24.461 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:24.480 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:24.501 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:53:24.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:24.502 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:24.502 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:24.509 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:53:24.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:53:24.511 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:53:24.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:24.512 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:24.512 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:53:24.520 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:53:24.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:53:24.521 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:53:24.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:24.523 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:24.523 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:53:24.530 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:53:24.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:53:24.532 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:53:24.533 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:53:24.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 Processor -- start 2022-07-21T01:53:24.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 -- start start 2022-07-21T01:53:24.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f99a00b3ca0 0x7f99a00b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:24.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a0142ba0 con 0x7f99a013c790 2022-07-21T01:53:24.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a00ba120 con 0x7f99a013cb70 2022-07-21T01:53:24.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a01429b0 con 0x7f99a00b3ca0 2022-07-21T01:53:24.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999effd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f99a013c790 0x7f99a00b9a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49842/0 (socket says 172.21.15.140:49842) 2022-07-21T01:53:24.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999effd700 1 -- 172.21.15.140:0/2842034275 learned_addr learned my addr 172.21.15.140:0/2842034275 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:24.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999e7fc700 1 --2- 172.21.15.140:0/2842034275 >> v2:172.21.15.140:3300/0 conn(0x7f99a00b3ca0 0x7f99a00b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:24.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999dffb700 1 -- 172.21.15.140:0/2842034275 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315615960 0 0) 0x7f99a0142ba0 con 0x7f99a013c790 2022-07-21T01:53:24.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999dffb700 1 -- 172.21.15.140:0/2842034275 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9984003260 con 0x7f99a013c790 2022-07-21T01:53:24.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999e7fc700 1 -- 172.21.15.140:0/2842034275 >> v1:172.21.15.152:6789/0 conn(0x7f99a013cb70 legacy=0x7f99a0143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999e7fc700 1 -- 172.21.15.140:0/2842034275 >> v1:172.21.15.140:6789/0 conn(0x7f99a013c790 legacy=0x7f99a00b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999dffb700 1 -- 172.21.15.140:0/2842034275 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3083366999 0 0) 0x7f9984003260 con 0x7f99a013c790 2022-07-21T01:53:24.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999e7fc700 1 -- 172.21.15.140:0/2842034275 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a00b4c00 con 0x7f99a00b3ca0 2022-07-21T01:53:24.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999e7fc700 1 --2- 172.21.15.140:0/2842034275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99a00b3ca0 0x7f99a00b4710 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f9990001b80 tx=0x7f9988006df0).ready entity=mon.0 client_cookie=c30fcfc1f9e5aca7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:24.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999dffb700 1 -- 172.21.15.140:0/2842034275 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99900019e0 con 0x7f99a00b3ca0 2022-07-21T01:53:24.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999dffb700 1 -- 172.21.15.140:0/2842034275 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9988000e90 con 0x7f99a00b3ca0 2022-07-21T01:53:24.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999dffb700 1 -- 172.21.15.140:0/2842034275 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f998800d4f0 con 0x7f99a00b3ca0 2022-07-21T01:53:24.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 -- 172.21.15.140:0/2842034275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99a00b3ca0 msgr2=0x7f99a00b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 --2- 172.21.15.140:0/2842034275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99a00b3ca0 0x7f99a00b4710 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f9990001b80 tx=0x7f9988006df0).stop 2022-07-21T01:53:24.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 -- 172.21.15.140:0/2842034275 shutdown_connections 2022-07-21T01:53:24.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 --2- 172.21.15.140:0/2842034275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99a00b3ca0 0x7f99a00b4710 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 -- 172.21.15.140:0/2842034275 >> 172.21.15.140:0/2842034275 conn(0x7f99a00678b0 msgr2=0x7f99a00b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:24.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 -- 172.21.15.140:0/2842034275 shutdown_connections 2022-07-21T01:53:24.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.643+0000 7f999ffff700 1 -- 172.21.15.140:0/2842034275 wait complete. 2022-07-21T01:53:24.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999ffff700 1 Processor -- start 2022-07-21T01:53:24.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999ffff700 1 -- start start 2022-07-21T01:53:24.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a00b3ca0 0x7f99a00af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:24.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99a013c790 0x7f99a0155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:24.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99a013cb70 0x7f99a013fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:24.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a013ec70 con 0x7f99a013cb70 2022-07-21T01:53:24.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a013f750 con 0x7f99a00b3ca0 2022-07-21T01:53:24.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a013e2d0 con 0x7f99a013c790 2022-07-21T01:53:24.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99a013cb70 0x7f99a013fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:24.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99a013cb70 0x7f99a013fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35704/0 (socket says 172.21.15.140:35704) 2022-07-21T01:53:24.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999f7fe700 1 -- 172.21.15.140:0/1635234524 learned_addr learned my addr 172.21.15.140:0/1635234524 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:24.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999effd700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a00b3ca0 0x7f99a00af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:24.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.647+0000 7f999e7fc700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99a013c790 0x7f99a0155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:24.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f999f7fe700 1 -- 172.21.15.140:0/1635234524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99a013c790 msgr2=0x7f99a0155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f999f7fe700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99a013c790 0x7f99a0155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f999f7fe700 1 -- 172.21.15.140:0/1635234524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a00b3ca0 msgr2=0x7f99a00af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f999f7fe700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a00b3ca0 0x7f99a00af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f999f7fe700 1 -- 172.21.15.140:0/1635234524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998800b180 con 0x7f99a013cb70 2022-07-21T01:53:24.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f999f7fe700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99a013cb70 0x7f99a013fb80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f99a013e7f0 tx=0x7f9998007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:24.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f9997fff700 1 -- 172.21.15.140:0/1635234524 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9998007030 con 0x7f99a013cb70 2022-07-21T01:53:24.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f9997fff700 1 -- 172.21.15.140:0/1635234524 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9998007190 con 0x7f99a013cb70 2022-07-21T01:53:24.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f999ffff700 1 -- 172.21.15.140:0/1635234524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a013d270 con 0x7f99a013cb70 2022-07-21T01:53:24.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f9997fff700 1 -- 172.21.15.140:0/1635234524 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9998006510 con 0x7f99a013cb70 2022-07-21T01:53:24.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f999ffff700 1 -- 172.21.15.140:0/1635234524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a00b2070 con 0x7f99a013cb70 2022-07-21T01:53:24.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f999ffff700 1 -- 172.21.15.140:0/1635234524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a00574e0 con 0x7f99a013cb70 2022-07-21T01:53:24.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f9997fff700 1 -- 172.21.15.140:0/1635234524 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9998010030 con 0x7f99a013cb70 2022-07-21T01:53:24.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f9997fff700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9978053a10 0x7f9978055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:24.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f9997fff700 1 -- 172.21.15.140:0/1635234524 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f999806b790 con 0x7f99a013cb70 2022-07-21T01:53:24.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.651+0000 7f999effd700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9978053a10 0x7f9978055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:24.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.655+0000 7f999effd700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9978053a10 0x7f9978055ed0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f999000d000 tx=0x7f9990002210).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:24.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.659+0000 7f9997fff700 1 -- 172.21.15.140:0/1635234524 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9998042990 con 0x7f99a013cb70 2022-07-21T01:53:24.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.783+0000 7f999ffff700 1 -- 172.21.15.140:0/1635234524 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f99a00b3760 con 0x7f99a013cb70 2022-07-21T01:53:24.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.783+0000 7f9997fff700 1 -- 172.21.15.140:0/1635234524 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) v1 ==== 81+0+28926 (secure 0 0 0) 0x7f9998045fb0 con 0x7f99a013cb70 2022-07-21T01:53:24.782 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:24.782 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:53:24.782 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 56, 2022-07-21T01:53:24.783 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:53:24.783 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:53:24.783 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:53:22.093887+0000", 2022-07-21T01:53:24.783 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:53:24.784 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:53:24.784 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:53:24.784 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:53:24.784 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:53:24.784 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:53:24.785 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:53:24.785 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:53:24.785 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:53:24.785 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:24.785 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:53:24.786 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:53:24.786 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:53:24.786 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:53:24.786 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:53:24.787 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 10, 2022-07-21T01:53:24.787 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:53:24.787 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:53:24.787 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:53:24.787 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:53:24.788 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:53:24.788 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.788 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:53:24.788 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:53:24.789 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:53:24.789 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:53:24.789 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:53:24.789 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:53:24.789 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:53:24.790 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:53:24.790 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:53:24.790 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:53:24.790 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:53:24.791 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:53:24.791 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:53:24.791 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:53:24.791 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:53:24.791 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:53:24.792 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:53:24.792 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:53:24.792 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:53:24.792 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:53:24.793 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:53:24.793 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:53:24.793 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:53:24.793 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.793 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:53:24.794 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:53:24.794 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:53:24.794 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:53:24.794 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:53:24.794 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:53:24.795 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:53:24.795 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:53:24.795 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:53:24.795 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:53:24.796 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:53:24.796 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:53:24.796 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:53:24.796 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:53:24.796 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:53:24.797 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:53:24.797 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:53:24.797 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:53:24.797 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:53:24.798 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:53:24.798 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:53:24.798 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:53:24.798 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:53:24.798 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:53:24.799 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:53:24.799 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:53:24.799 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:53:24.799 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.799 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:53:24.800 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:53:24.800 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:53:24.800 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:53:24.800 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:53:24.801 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:53:24.801 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:53:24.801 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:53:24.801 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:53:24.801 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:53:24.802 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:53:24.802 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:53:24.802 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:53:24.802 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.802 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:53:24.803 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:53:24.803 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.803 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.803 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.804 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:53:24.804 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:53:24.804 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:53:24.804 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:53:24.804 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:53:24.805 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:53:24.805 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:53:24.805 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:53:24.805 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:53:24.805 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:53:24.806 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:53:24.806 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:53:24.806 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:53:24.806 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:53:24.807 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:53:24.807 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:53:24.807 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:53:24.807 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:53:24.807 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:53:24.808 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:53:24.808 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:53:24.808 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:53:24.808 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:53:24.809 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.809 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:53:24.809 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:53:24.809 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:53:24.809 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:53:24.810 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:53:24.810 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:53:24.810 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:53:24.810 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:53:24.811 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:53:24.811 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:53:24.811 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:53:24.811 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:53:24.811 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:53:24.812 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:53:24.812 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:53:24.812 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:53:24.812 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:53:24.813 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:53:24.813 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:53:24.813 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:53:24.813 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:53:24.813 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:53:24.814 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:53:24.814 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:53:24.814 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:53:24.814 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:53:24.814 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:53:24.815 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.815 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:53:24.815 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:53:24.815 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:53:24.816 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:53:24.816 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:53:24.816 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:53:24.816 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:53:24.816 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:53:24.817 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:53:24.817 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:53:24.817 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:53:24.817 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:53:24.817 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:53:24.818 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:53:24.818 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.818 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.818 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:24.819 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:53:24.819 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.819 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:53:24.819 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:53:24.819 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:24.820 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:24.820 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:24.820 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:24.820 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:24.820 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:24.821 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:24.821 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 50, 2022-07-21T01:53:24.821 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:24.821 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:24.822 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:24.822 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.822 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.822 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.822 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:53:24.823 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:24.823 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.823 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.823 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.824 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:53:24.824 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:24.824 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.824 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.824 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.825 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:24.825 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.825 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.825 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.825 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:53:24.826 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:24.826 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.826 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.826 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.827 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:53:24.827 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:24.827 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.827 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.827 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.828 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:24.828 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.828 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.828 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.828 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:53:24.829 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:24.829 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.829 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.829 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.829 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:53:24.830 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:24.830 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.830 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.830 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.831 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:24.831 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.831 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.831 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.831 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:53:24.832 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:24.832 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.832 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.833 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.833 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:53:24.833 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:24.833 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.834 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.834 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.834 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:53:24.834 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:53:24.834 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:53:24.835 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:53:24.835 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:24.835 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:24.835 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:24.836 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.836 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.836 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.836 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:53:24.836 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:53:24.837 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:24.837 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:24.837 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:24.837 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:24.838 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:24.838 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:24.838 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:24.838 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 50, 2022-07-21T01:53:24.838 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:24.839 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:24.839 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:24.839 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.839 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.840 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.840 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:53:24.840 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:24.841 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.841 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.841 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.841 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:53:24.841 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:24.842 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.842 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.842 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.842 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:24.843 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.843 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.843 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.843 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:53:24.843 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:24.844 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.844 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.844 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.844 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:53:24.844 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:24.845 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.845 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.845 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.845 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:24.846 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.846 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.846 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.846 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:53:24.846 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:24.847 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.847 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.847 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.847 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:53:24.848 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:24.848 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.848 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.848 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.848 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:24.849 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.849 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.849 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.849 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:53:24.849 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:24.850 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.850 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.850 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.850 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:53:24.851 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:24.851 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.851 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.851 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.851 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:53:24.852 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:53:24.852 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:53:24.852 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:53:24.852 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:24.853 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:24.853 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:24.853 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.853 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.853 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.854 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:53:24.854 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:53:24.854 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:24.854 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:24.854 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:24.855 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:24.855 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:24.855 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:24.855 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:24.856 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 51, 2022-07-21T01:53:24.856 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:24.856 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:24.856 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:24.856 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.857 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.857 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.857 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:53:24.857 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:24.857 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.858 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.858 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.858 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:53:24.858 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:24.859 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.859 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.859 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.859 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:24.859 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.860 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.860 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.860 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:53:24.860 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:24.860 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.861 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.861 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.861 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:53:24.861 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:24.861 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.862 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.862 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.862 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:24.862 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.863 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.863 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.863 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:53:24.863 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:24.863 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.864 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.864 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.864 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:53:24.864 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:24.865 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.865 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.865 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.865 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:24.865 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.866 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.866 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.866 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:53:24.866 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:24.866 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.867 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.867 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.867 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:53:24.867 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:24.868 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.868 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.868 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.868 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:53:24.868 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:53:24.869 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:53:24.869 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:53:24.869 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:24.869 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:24.869 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:24.870 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.870 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.870 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.870 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:53:24.871 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:53:24.871 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:24.871 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:24.871 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:24.871 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:24.872 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:24.872 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:24.872 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:24.872 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 49, 2022-07-21T01:53:24.873 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:24.873 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:24.873 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:24.873 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.873 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.874 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.874 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:53:24.874 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:24.874 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.875 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.875 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.875 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:53:24.875 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:24.875 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.876 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.876 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.876 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:24.876 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.877 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.877 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.877 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:53:24.878 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:24.878 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.878 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.878 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.879 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:53:24.879 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:24.879 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.879 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.880 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.880 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:24.880 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.880 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.881 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.881 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:53:24.881 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:24.881 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.881 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.882 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.882 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:53:24.882 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:24.882 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.883 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.883 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.883 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:24.883 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.883 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.884 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.884 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:53:24.884 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:24.884 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.884 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.885 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.885 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:53:24.885 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:24.885 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.886 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.886 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.886 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:53:24.886 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:53:24.886 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:53:24.887 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:53:24.887 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:24.887 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:24.887 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:24.887 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.888 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.888 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.888 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:53:24.888 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:53:24.889 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:24.889 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:24.889 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:24.889 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:24.889 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:24.890 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:24.890 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:24.890 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 50, 2022-07-21T01:53:24.890 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:24.890 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:24.891 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:24.891 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.891 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.891 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.892 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:53:24.892 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:24.892 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.892 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.892 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.893 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:53:24.893 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:24.893 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.893 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.894 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.894 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:24.894 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.894 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.894 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.895 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:53:24.895 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:24.895 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.895 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.896 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.896 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:53:24.896 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:24.896 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.896 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.897 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.897 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:24.897 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.897 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.897 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.898 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:53:24.898 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:24.898 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.898 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.899 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.899 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:53:24.899 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:24.899 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.899 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.900 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.900 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:24.900 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.900 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.901 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.901 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:53:24.901 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:24.901 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.901 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.902 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.902 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:53:24.902 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:24.902 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.903 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.903 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.903 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:53:24.903 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:53:24.903 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:53:24.904 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:53:24.904 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:24.904 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:24.904 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:24.904 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.905 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.905 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.905 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:53:24.905 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:53:24.906 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:24.906 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:24.906 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:24.906 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:24.906 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:24.907 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:24.907 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:24.907 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 50, 2022-07-21T01:53:24.907 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:24.907 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:24.908 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:24.908 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.908 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.908 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.909 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:53:24.909 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:24.909 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.909 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.909 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.910 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:53:24.910 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:24.910 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.910 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.911 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.911 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:24.911 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.911 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.911 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.912 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:53:24.912 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:24.912 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.912 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.912 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.913 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:53:24.913 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:24.913 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.913 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.914 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.914 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:24.914 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.914 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.914 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.915 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:53:24.915 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:24.915 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.915 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.915 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.916 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:53:24.916 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:24.916 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.916 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.917 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.917 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:24.917 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.917 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.917 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.918 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:53:24.918 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:24.918 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.918 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.918 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.919 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:53:24.919 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:24.919 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.919 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.920 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.920 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:53:24.920 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:53:24.920 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:53:24.920 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:53:24.921 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:24.921 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:24.921 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:24.921 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.921 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.922 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.922 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:53:24.922 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:53:24.922 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:24.923 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:24.923 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:24.923 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:24.923 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:24.923 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:24.924 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:24.924 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 50, 2022-07-21T01:53:24.924 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:24.924 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:24.925 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:24.925 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.925 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.925 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.925 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:53:24.926 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:24.926 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.926 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.926 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.926 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:53:24.927 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:24.927 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.927 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.927 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.928 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:24.928 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.928 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.928 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.928 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:53:24.929 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:24.929 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.929 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.929 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.929 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:53:24.930 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:24.930 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.930 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.930 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.931 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:24.931 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.931 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.931 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.931 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:53:24.932 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:24.932 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.932 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.933 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.933 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:53:24.933 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:24.933 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.934 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.934 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.934 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:24.934 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.934 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.935 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.935 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:53:24.935 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:24.935 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.936 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.936 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.936 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:53:24.936 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:24.936 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.937 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.937 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.937 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:53:24.937 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:53:24.938 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:53:24.938 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:53:24.938 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:24.938 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:24.938 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:24.939 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.939 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.939 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.939 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:53:24.940 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:53:24.940 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:24.940 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:24.941 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:24.941 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:24.941 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:24.941 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:24.942 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:24.942 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 49, 2022-07-21T01:53:24.942 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:24.942 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:24.943 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:24.943 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.943 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.943 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.944 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:53:24.944 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:24.944 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.944 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.944 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.945 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:53:24.945 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:24.945 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.946 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.947 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.947 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:24.947 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.947 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.948 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.948 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:53:24.948 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:24.948 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.948 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.949 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.949 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:53:24.949 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:24.949 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.949 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.950 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.950 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:24.950 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.950 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.951 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.951 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:53:24.951 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:24.951 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.951 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.952 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.952 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:53:24.952 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:24.952 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.953 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.953 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.953 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:24.953 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:24.953 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.954 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:24.954 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:53:24.954 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:24.954 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.954 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.955 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:24.955 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:53:24.955 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:24.955 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.956 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.956 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.956 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:53:24.956 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:53:24.956 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:53:24.957 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:53:24.957 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:24.957 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:24.957 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:24.957 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:24.958 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.959 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:24.959 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:53:24.959 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.960 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:53:24.960 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:24.960 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:24.960 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:24.961 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:24.961 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:24.961 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:53:24.961 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:24.962 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.962 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.962 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:53:24.962 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:24.963 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:24.963 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:24.963 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:24.963 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:24.964 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:53:24.964 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:24.964 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.964 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.965 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:53:24.965 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:24.965 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:24.965 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:24.966 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:24.966 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:24.966 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:53:24.966 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:24.967 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.967 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.967 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:53:24.967 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:24.968 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:24.968 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:24.968 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:24.968 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:24.969 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:53:24.969 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:24.969 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.969 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.970 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:53:24.970 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:24.970 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:24.970 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:24.971 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:24.971 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:24.971 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:53:24.971 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:24.972 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.972 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.972 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:53:24.972 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:24.973 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:24.973 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:24.973 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:24.973 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:24.973 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:53:24.974 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:24.974 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.974 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.974 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:53:24.975 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:24.975 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:24.975 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:24.975 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:24.976 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:24.976 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:53:24.976 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:24.976 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.977 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:24.977 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:53:24.977 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:24.977 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:24.977 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:24.978 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:24.978 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:24.978 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:53:24.979 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:24.979 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.979 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:24.979 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:53:24.980 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:53:24.980 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:53:24.980 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:53:24.980 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:53:24.980 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6833/3908541795": "2022-07-22T01:53:18.595217+0000", 2022-07-21T01:53:24.981 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6832/3908541795": "2022-07-22T01:53:18.595217+0000" 2022-07-21T01:53:24.981 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.981 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:53:24.981 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:53:24.982 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:53:24.982 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:53:24.982 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:53:24.982 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:53:24.983 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:53:24.983 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:53:24.983 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:24.983 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:24.983 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:53:24.984 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:53:24.984 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:53:24.984 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:53:24.984 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:53:24.985 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:53:24.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 -- 172.21.15.140:0/1635234524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9978053a10 msgr2=0x7f9978055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9978053a10 0x7f9978055ed0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f999000d000 tx=0x7f9990002210).stop 2022-07-21T01:53:24.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 -- 172.21.15.140:0/1635234524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99a013cb70 msgr2=0x7f99a013fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:24.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99a013cb70 0x7f99a013fb80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f99a013e7f0 tx=0x7f9998007c20).stop 2022-07-21T01:53:24.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 -- 172.21.15.140:0/1635234524 shutdown_connections 2022-07-21T01:53:24.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9978053a10 0x7f9978055ed0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99a013cb70 0x7f99a013fb80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99a013c790 0x7f99a0155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 --2- 172.21.15.140:0/1635234524 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99a00b3ca0 0x7f99a00af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:24.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 -- 172.21.15.140:0/1635234524 >> 172.21.15.140:0/1635234524 conn(0x7f99a00678b0 msgr2=0x7f99a00b95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:24.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 -- 172.21.15.140:0/1635234524 shutdown_connections 2022-07-21T01:53:24.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:24.787+0000 7f999ffff700 1 -- 172.21.15.140:0/1635234524 wait complete. 2022-07-21T01:53:24.991 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:53:25.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.107+0000 7ff12ffff700 1 Processor -- start 2022-07-21T01:53:25.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.107+0000 7ff12ffff700 1 -- start start 2022-07-21T01:53:25.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.107+0000 7ff12ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff1300af8e0 0x7ff1300b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.107+0000 7ff12ffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff130147010 con 0x7ff1300adff0 2022-07-21T01:53:25.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.107+0000 7ff12ffff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1301472b0 con 0x7ff1300b5650 2022-07-21T01:53:25.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.107+0000 7ff12ffff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff130146d40 con 0x7ff1300af8e0 2022-07-21T01:53:25.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.111+0000 7ff12e7fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff1300adff0 0x7ff13005bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49856/0 (socket says 172.21.15.140:49856) 2022-07-21T01:53:25.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.111+0000 7ff12e7fc700 1 -- 172.21.15.140:0/3584102430 learned_addr learned my addr 172.21.15.140:0/3584102430 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:25.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.111+0000 7ff12f7fe700 1 --2- 172.21.15.140:0/3584102430 >> v2:172.21.15.152:3300/0 conn(0x7ff1300af8e0 0x7ff1300b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.111+0000 7ff12f7fe700 1 -- 172.21.15.140:0/3584102430 >> v1:172.21.15.152:6789/0 conn(0x7ff1300b5650 legacy=0x7ff130147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.111+0000 7ff12f7fe700 1 -- 172.21.15.140:0/3584102430 >> v1:172.21.15.140:6789/0 conn(0x7ff1300adff0 legacy=0x7ff13005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.111+0000 7ff12f7fe700 1 -- 172.21.15.140:0/3584102430 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13005c4a0 con 0x7ff1300af8e0 2022-07-21T01:53:25.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.111+0000 7ff12f7fe700 1 --2- 172.21.15.140:0/3584102430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1300af8e0 0x7ff1300b5a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7ff124002f70 tx=0x7ff124006df0).ready entity=mon.1 client_cookie=b7e67e911b6792bd server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:25.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.115+0000 7ff12dffb700 1 -- 172.21.15.140:0/3584102430 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff12400d610 con 0x7ff1300af8e0 2022-07-21T01:53:25.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.115+0000 7ff12dffb700 1 -- 172.21.15.140:0/3584102430 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff124000e90 con 0x7ff1300af8e0 2022-07-21T01:53:25.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.115+0000 7ff12dffb700 1 -- 172.21.15.140:0/3584102430 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff12400d960 con 0x7ff1300af8e0 2022-07-21T01:53:25.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.115+0000 7ff12ffff700 1 -- 172.21.15.140:0/3584102430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1300af8e0 msgr2=0x7ff1300b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.115+0000 7ff12ffff700 1 --2- 172.21.15.140:0/3584102430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1300af8e0 0x7ff1300b5a30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7ff124002f70 tx=0x7ff124006df0).stop 2022-07-21T01:53:25.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.115+0000 7ff12ffff700 1 -- 172.21.15.140:0/3584102430 shutdown_connections 2022-07-21T01:53:25.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.115+0000 7ff12ffff700 1 --2- 172.21.15.140:0/3584102430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1300af8e0 0x7ff1300b5a30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.115+0000 7ff12ffff700 1 -- 172.21.15.140:0/3584102430 >> 172.21.15.140:0/3584102430 conn(0x7ff1300634e0 msgr2=0x7ff1300b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:25.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.115+0000 7ff12ffff700 1 -- 172.21.15.140:0/3584102430 shutdown_connections 2022-07-21T01:53:25.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.115+0000 7ff12ffff700 1 -- 172.21.15.140:0/3584102430 wait complete. 2022-07-21T01:53:25.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12ffff700 1 Processor -- start 2022-07-21T01:53:25.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12ffff700 1 -- start start 2022-07-21T01:53:25.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff1300adff0 0x7ff130063d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1300af8e0 0x7ff1300668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff1300b5650 0x7ff130066dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130110a10 con 0x7ff1300adff0 2022-07-21T01:53:25.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130110c00 con 0x7ff1300af8e0 2022-07-21T01:53:25.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff130111850 con 0x7ff1300b5650 2022-07-21T01:53:25.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff1300adff0 0x7ff130063d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12e7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1300af8e0 0x7ff1300668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff1300adff0 0x7ff130063d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35716/0 (socket says 172.21.15.140:35716) 2022-07-21T01:53:25.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12effd700 1 -- 172.21.15.140:0/286335933 learned_addr learned my addr 172.21.15.140:0/286335933 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:25.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12f7fe700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff1300b5650 0x7ff130066dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12effd700 1 -- 172.21.15.140:0/286335933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff1300b5650 msgr2=0x7ff130066dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12effd700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff1300b5650 0x7ff130066dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12effd700 1 -- 172.21.15.140:0/286335933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1300af8e0 msgr2=0x7ff1300668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.119+0000 7ff12effd700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1300af8e0 0x7ff1300668d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.123+0000 7ff12effd700 1 -- 172.21.15.140:0/286335933 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12400b180 con 0x7ff1300adff0 2022-07-21T01:53:25.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.123+0000 7ff12effd700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff1300adff0 0x7ff130063d70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7ff1300622d0 tx=0x7ff128001400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:25.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.123+0000 7ff11ffff700 1 -- 172.21.15.140:0/286335933 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff12800f470 con 0x7ff1300adff0 2022-07-21T01:53:25.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.123+0000 7ff11ffff700 1 -- 172.21.15.140:0/286335933 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12800f5d0 con 0x7ff1300adff0 2022-07-21T01:53:25.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.123+0000 7ff11ffff700 1 -- 172.21.15.140:0/286335933 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff12800f930 con 0x7ff1300adff0 2022-07-21T01:53:25.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.123+0000 7ff12ffff700 1 -- 172.21.15.140:0/286335933 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff130111af0 con 0x7ff1300adff0 2022-07-21T01:53:25.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.123+0000 7ff12ffff700 1 -- 172.21.15.140:0/286335933 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff130064260 con 0x7ff1300adff0 2022-07-21T01:53:25.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.123+0000 7ff12ffff700 1 -- 172.21.15.140:0/286335933 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1300574e0 con 0x7ff1300adff0 2022-07-21T01:53:25.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.131+0000 7ff11ffff700 1 -- 172.21.15.140:0/286335933 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff128019030 con 0x7ff1300adff0 2022-07-21T01:53:25.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.131+0000 7ff11ffff700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff10c053b30 0x7ff10c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.131+0000 7ff11ffff700 1 -- 172.21.15.140:0/286335933 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5339+0+0 (secure 0 0 0) 0x7ff12806b7b0 con 0x7ff1300adff0 2022-07-21T01:53:25.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.131+0000 7ff12e7fc700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff10c053b30 0x7ff10c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.131+0000 7ff11ffff700 1 -- 172.21.15.140:0/286335933 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff1280428a0 con 0x7ff1300adff0 2022-07-21T01:53:25.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.131+0000 7ff12e7fc700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff10c053b30 0x7ff10c055ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7ff12000d000 tx=0x7ff120009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:25.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.291+0000 7ff12ffff700 1 -- 172.21.15.140:0/286335933 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff130067ea0 con 0x7ff1300adff0 2022-07-21T01:53:25.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.291+0000 7ff11ffff700 1 -- 172.21.15.140:0/286335933 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+1215 (secure 0 0 0) 0x7ff128045ec0 con 0x7ff1300adff0 2022-07-21T01:53:25.291 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:25.291 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1915503334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1915503334},{"type":"v1","addr":"172.21.15.140:6835","nonce":1915503334}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19},{"gid":5411,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1393141587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1393141587},{"type":"v1","addr":"172.21.15.152:6833","nonce":1393141587}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[]} 2022-07-21T01:53:25.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 -- 172.21.15.140:0/286335933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff10c053b30 msgr2=0x7ff10c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff10c053b30 0x7ff10c055ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7ff12000d000 tx=0x7ff120009040).stop 2022-07-21T01:53:25.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 -- 172.21.15.140:0/286335933 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff1300adff0 msgr2=0x7ff130063d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff1300adff0 0x7ff130063d70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7ff1300622d0 tx=0x7ff128001400).stop 2022-07-21T01:53:25.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 -- 172.21.15.140:0/286335933 shutdown_connections 2022-07-21T01:53:25.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff10c053b30 0x7ff10c055ff0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff1300b5650 0x7ff130066dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff1300af8e0 0x7ff1300668d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 --2- 172.21.15.140:0/286335933 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff1300adff0 0x7ff130063d70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 -- 172.21.15.140:0/286335933 >> 172.21.15.140:0/286335933 conn(0x7ff1300634e0 msgr2=0x7ff130069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:25.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 -- 172.21.15.140:0/286335933 shutdown_connections 2022-07-21T01:53:25.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.295+0000 7ff12ffff700 1 -- 172.21.15.140:0/286335933 wait complete. 2022-07-21T01:53:25.299 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 23 2022-07-21T01:53:25.302 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:53:25.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.403+0000 7f185ee19700 1 Processor -- start 2022-07-21T01:53:25.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.403+0000 7f185ee19700 1 -- start start 2022-07-21T01:53:25.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.403+0000 7f185ee19700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f18580adff0 0x7f18580b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.403+0000 7f185ee19700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18580af8e0 0x7f185805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.403+0000 7f185ee19700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1858147010 con 0x7f18580b5650 2022-07-21T01:53:25.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.403+0000 7f185ee19700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18581472b0 con 0x7f18580adff0 2022-07-21T01:53:25.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.403+0000 7f185ee19700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1858146d40 con 0x7f18580af8e0 2022-07-21T01:53:25.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185d616700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f18580adff0 0x7f18580b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185d616700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f18580adff0 0x7f18580b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35724/0 (socket says 172.21.15.140:35724) 2022-07-21T01:53:25.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185d616700 1 -- 172.21.15.140:0/1399116980 learned_addr learned my addr 172.21.15.140:0/1399116980 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:25.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185e618700 1 --2- 172.21.15.140:0/1399116980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18580af8e0 0x7f185805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185d616700 1 -- 172.21.15.140:0/1399116980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18580af8e0 msgr2=0x7f185805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:25.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185d616700 1 --2- 172.21.15.140:0/1399116980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18580af8e0 0x7f185805bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185d616700 1 -- 172.21.15.140:0/1399116980 >> v1:172.21.15.140:6789/0 conn(0x7f18580b5650 legacy=0x7f1858147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185d616700 1 -- 172.21.15.140:0/1399116980 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18580acae0 con 0x7f18580adff0 2022-07-21T01:53:25.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185d616700 1 --2- 172.21.15.140:0/1399116980 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18580adff0 0x7f18580b5a30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f18540032f0 tx=0x7f1854009250).ready entity=mon.0 client_cookie=5e81bf5b366e8a7d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:25.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ce15700 1 -- 172.21.15.140:0/1399116980 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f185400b8a0 con 0x7f18580adff0 2022-07-21T01:53:25.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ce15700 1 -- 172.21.15.140:0/1399116980 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185400ba00 con 0x7f18580adff0 2022-07-21T01:53:25.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ce15700 1 -- 172.21.15.140:0/1399116980 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f185400bb70 con 0x7f18580adff0 2022-07-21T01:53:25.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 -- 172.21.15.140:0/1399116980 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18580adff0 msgr2=0x7f18580b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 --2- 172.21.15.140:0/1399116980 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18580adff0 0x7f18580b5a30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f18540032f0 tx=0x7f1854009250).stop 2022-07-21T01:53:25.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 -- 172.21.15.140:0/1399116980 shutdown_connections 2022-07-21T01:53:25.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 --2- 172.21.15.140:0/1399116980 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18580af8e0 0x7f185805bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 --2- 172.21.15.140:0/1399116980 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18580adff0 0x7f18580b5a30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 -- 172.21.15.140:0/1399116980 >> 172.21.15.140:0/1399116980 conn(0x7f18580634e0 msgr2=0x7f18580b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:25.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 -- 172.21.15.140:0/1399116980 shutdown_connections 2022-07-21T01:53:25.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 -- 172.21.15.140:0/1399116980 wait complete. 2022-07-21T01:53:25.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 Processor -- start 2022-07-21T01:53:25.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 -- start start 2022-07-21T01:53:25.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18580adff0 0x7f18580674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18580af8e0 0x7f185810e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18580b5650 0x7f1858111090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1858111580 con 0x7f18580af8e0 2022-07-21T01:53:25.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1858066310 con 0x7f18580b5650 2022-07-21T01:53:25.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185ee19700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1858063d20 con 0x7f18580adff0 2022-07-21T01:53:25.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185d616700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18580af8e0 0x7f185810e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185d616700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18580af8e0 0x7f185810e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35730/0 (socket says 172.21.15.140:35730) 2022-07-21T01:53:25.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185d616700 1 -- 172.21.15.140:0/1992456654 learned_addr learned my addr 172.21.15.140:0/1992456654 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:25.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185de17700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18580adff0 0x7f18580674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.407+0000 7f185e618700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18580b5650 0x7f1858111090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f185d616700 1 -- 172.21.15.140:0/1992456654 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18580adff0 msgr2=0x7f18580674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f185d616700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18580adff0 0x7f18580674e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f185d616700 1 -- 172.21.15.140:0/1992456654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18580b5650 msgr2=0x7f1858111090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f185d616700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18580b5650 0x7f1858111090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f185d616700 1 -- 172.21.15.140:0/1992456654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540094d0 con 0x7f18580af8e0 2022-07-21T01:53:25.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f185d616700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18580af8e0 0x7f185810e3d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f18540032f0 tx=0x7f1854016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:25.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f184effd700 1 -- 172.21.15.140:0/1992456654 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1854006590 con 0x7f18580af8e0 2022-07-21T01:53:25.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f184effd700 1 -- 172.21.15.140:0/1992456654 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18540066f0 con 0x7f18580af8e0 2022-07-21T01:53:25.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f184effd700 1 -- 172.21.15.140:0/1992456654 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1854021480 con 0x7f18580af8e0 2022-07-21T01:53:25.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f185ee19700 1 -- 172.21.15.140:0/1992456654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1858111cc0 con 0x7f18580af8e0 2022-07-21T01:53:25.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f185ee19700 1 -- 172.21.15.140:0/1992456654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1858064360 con 0x7f18580af8e0 2022-07-21T01:53:25.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f185ee19700 1 -- 172.21.15.140:0/1992456654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1858063ee0 con 0x7f18580af8e0 2022-07-21T01:53:25.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f184effd700 1 -- 172.21.15.140:0/1992456654 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f185400f070 con 0x7f18580af8e0 2022-07-21T01:53:25.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f184effd700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1838053a70 0x7f1838055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f184effd700 1 -- 172.21.15.140:0/1992456654 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f185406cda0 con 0x7f18580af8e0 2022-07-21T01:53:25.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.411+0000 7f185de17700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1838053a70 0x7f1838055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.415+0000 7f185de17700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1838053a70 0x7f1838055f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f184800d000 tx=0x7f1848009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:25.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.419+0000 7f184effd700 1 -- 172.21.15.140:0/1992456654 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1854044050 con 0x7f18580af8e0 2022-07-21T01:53:25.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.583+0000 7f185ee19700 1 -- 172.21.15.140:0/1992456654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18580574e0 con 0x7f18580af8e0 2022-07-21T01:53:25.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.583+0000 7f184effd700 1 -- 172.21.15.140:0/1992456654 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+1215 (secure 0 0 0) 0x7f1854015020 con 0x7f18580af8e0 2022-07-21T01:53:25.582 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:25.582 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5201,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1915503334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1915503334},{"type":"v1","addr":"172.21.15.140:6835","nonce":1915503334}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19},{"gid":5411,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/1393141587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1393141587},{"type":"v1","addr":"172.21.15.152:6833","nonce":1393141587}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":23}],"filesystems":[]} 2022-07-21T01:53:25.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 -- 172.21.15.140:0/1992456654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1838053a70 msgr2=0x7f1838055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1838053a70 0x7f1838055f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f184800d000 tx=0x7f1848009040).stop 2022-07-21T01:53:25.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 -- 172.21.15.140:0/1992456654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18580af8e0 msgr2=0x7f185810e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18580af8e0 0x7f185810e3d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f18540032f0 tx=0x7f1854016000).stop 2022-07-21T01:53:25.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 -- 172.21.15.140:0/1992456654 shutdown_connections 2022-07-21T01:53:25.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1838053a70 0x7f1838055f30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18580b5650 0x7f1858111090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18580af8e0 0x7f185810e3d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 --2- 172.21.15.140:0/1992456654 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18580adff0 0x7f18580674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 -- 172.21.15.140:0/1992456654 >> 172.21.15.140:0/1992456654 conn(0x7f18580634e0 msgr2=0x7f1858146350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:25.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 -- 172.21.15.140:0/1992456654 shutdown_connections 2022-07-21T01:53:25.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.587+0000 7f185ee19700 1 -- 172.21.15.140:0/1992456654 wait complete. 2022-07-21T01:53:25.588 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 23 2022-07-21T01:53:25.595 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:53:25.595 INFO:tasks.ceph.mds.a:Stopping old one... 2022-07-21T01:53:25.596 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:53:25.597 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:53:25.597 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:53:25.598 INFO:tasks.ceph.mds.b:Stopping old one... 2022-07-21T01:53:25.599 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:53:25.599 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:53:25.611 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:53:25.612 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:53:25.615 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:53:25.615 DEBUG:teuthology.parallel:result is None 2022-07-21T01:53:25.617 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:53:25.618 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:53:25.621 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:53:25.622 DEBUG:teuthology.parallel:result is None 2022-07-21T01:53:25.623 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:53:25.675 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:53:25.695 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:53:25.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.727+0000 7f86f897a700 1 Processor -- start 2022-07-21T01:53:25.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.727+0000 7f86f897a700 1 -- start start 2022-07-21T01:53:25.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.727+0000 7f86f897a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f86f40b51e0 0x7f86f41407d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.727+0000 7f86f897a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f40b4440 con 0x7f86f40b6060 2022-07-21T01:53:25.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.727+0000 7f86f897a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f4143fd0 con 0x7f86f4140cc0 2022-07-21T01:53:25.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.727+0000 7f86f897a700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f4144270 con 0x7f86f40b51e0 2022-07-21T01:53:25.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f259c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f86f40b51e0 0x7f86f41407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f2d9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f86f40b6060 0x7f86f4144710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49896/0 (socket says 172.21.15.140:49896) 2022-07-21T01:53:25.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f2d9d700 1 -- 172.21.15.140:0/2322866905 learned_addr learned my addr 172.21.15.140:0/2322866905 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:25.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f1d9b700 1 -- 172.21.15.140:0/2322866905 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415207357 0 0) 0x7f86f4143fd0 con 0x7f86f4140cc0 2022-07-21T01:53:25.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f1d9b700 1 -- 172.21.15.140:0/2322866905 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d4003260 con 0x7f86f4140cc0 2022-07-21T01:53:25.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f1d9b700 1 -- 172.21.15.140:0/2322866905 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767799628 0 0) 0x7f86f40b4440 con 0x7f86f40b6060 2022-07-21T01:53:25.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f1d9b700 1 -- 172.21.15.140:0/2322866905 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f4143fd0 con 0x7f86f40b6060 2022-07-21T01:53:25.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f259c700 1 -- 172.21.15.140:0/2322866905 >> v1:172.21.15.152:6789/0 conn(0x7f86f4140cc0 legacy=0x7f86f4141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f259c700 1 -- 172.21.15.140:0/2322866905 >> v1:172.21.15.140:6789/0 conn(0x7f86f40b6060 legacy=0x7f86f4144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f259c700 1 -- 172.21.15.140:0/2322866905 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f4144e20 con 0x7f86f40b51e0 2022-07-21T01:53:25.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f259c700 1 --2- 172.21.15.140:0/2322866905 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86f40b51e0 0x7f86f41407d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f86dc002f70 tx=0x7f86dc006df0).ready entity=mon.0 client_cookie=32108b1beff840bf server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:25.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f1d9b700 1 -- 172.21.15.140:0/2322866905 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86dc00d540 con 0x7f86f40b51e0 2022-07-21T01:53:25.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f1d9b700 1 -- 172.21.15.140:0/2322866905 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc000e90 con 0x7f86f40b51e0 2022-07-21T01:53:25.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f1d9b700 1 -- 172.21.15.140:0/2322866905 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86dc00d890 con 0x7f86f40b51e0 2022-07-21T01:53:25.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f897a700 1 -- 172.21.15.140:0/2322866905 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86f40b51e0 msgr2=0x7f86f41407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f897a700 1 --2- 172.21.15.140:0/2322866905 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86f40b51e0 0x7f86f41407d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f86dc002f70 tx=0x7f86dc006df0).stop 2022-07-21T01:53:25.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f897a700 1 -- 172.21.15.140:0/2322866905 shutdown_connections 2022-07-21T01:53:25.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f897a700 1 --2- 172.21.15.140:0/2322866905 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86f40b51e0 0x7f86f41407d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f897a700 1 -- 172.21.15.140:0/2322866905 >> 172.21.15.140:0/2322866905 conn(0x7f86f4063d10 msgr2=0x7f86f40b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:25.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f897a700 1 -- 172.21.15.140:0/2322866905 shutdown_connections 2022-07-21T01:53:25.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f897a700 1 -- 172.21.15.140:0/2322866905 wait complete. 2022-07-21T01:53:25.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f897a700 1 Processor -- start 2022-07-21T01:53:25.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.731+0000 7f86f897a700 1 -- start start 2022-07-21T01:53:25.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f897a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86f40b51e0 0x7f86f41452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f897a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86f40b6060 0x7f86f4152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f897a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86f4140cc0 0x7f86f4152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f897a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f4145a80 con 0x7f86f40b51e0 2022-07-21T01:53:25.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f897a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f4152c00 con 0x7f86f4140cc0 2022-07-21T01:53:25.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f897a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f40aace0 con 0x7f86f40b6060 2022-07-21T01:53:25.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f2d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86f40b51e0 0x7f86f41452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f2d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86f40b51e0 0x7f86f41452e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35756/0 (socket says 172.21.15.140:35756) 2022-07-21T01:53:25.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f2d9d700 1 -- 172.21.15.140:0/265504294 learned_addr learned my addr 172.21.15.140:0/265504294 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:25.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f359e700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86f4140cc0 0x7f86f4152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f259c700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86f40b6060 0x7f86f4152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f2d9d700 1 -- 172.21.15.140:0/265504294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86f40b6060 msgr2=0x7f86f4152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f2d9d700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86f40b6060 0x7f86f4152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f2d9d700 1 -- 172.21.15.140:0/265504294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86f4140cc0 msgr2=0x7f86f4152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:25.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f2d9d700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86f4140cc0 0x7f86f4152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:25.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f2d9d700 1 -- 172.21.15.140:0/265504294 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc00b180 con 0x7f86f40b51e0 2022-07-21T01:53:25.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f2d9d700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86f40b51e0 0x7f86f41452e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f86e4001490 tx=0x7f86e4007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:25.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86ebfff700 1 -- 172.21.15.140:0/265504294 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86e4005c70 con 0x7f86f40b51e0 2022-07-21T01:53:25.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86ebfff700 1 -- 172.21.15.140:0/265504294 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4005dd0 con 0x7f86f40b51e0 2022-07-21T01:53:25.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f897a700 1 -- 172.21.15.140:0/265504294 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f4147070 con 0x7f86f40b51e0 2022-07-21T01:53:25.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86ebfff700 1 -- 172.21.15.140:0/265504294 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86e40072c0 con 0x7f86f40b51e0 2022-07-21T01:53:25.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.735+0000 7f86f897a700 1 -- 172.21.15.140:0/265504294 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f4141e30 con 0x7f86f40b51e0 2022-07-21T01:53:25.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.739+0000 7f86ebfff700 1 -- 172.21.15.140:0/265504294 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f86e401a070 con 0x7f86f40b51e0 2022-07-21T01:53:25.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.739+0000 7f86f897a700 1 -- 172.21.15.140:0/265504294 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f40ab140 con 0x7f86f40b51e0 2022-07-21T01:53:25.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.739+0000 7f86ebfff700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d8053980 0x7f86d8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:25.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.739+0000 7f86ebfff700 1 -- 172.21.15.140:0/265504294 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f86e406b2a0 con 0x7f86f40b51e0 2022-07-21T01:53:25.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.739+0000 7f86f259c700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d8053980 0x7f86d8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:25.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.739+0000 7f86f259c700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d8053980 0x7f86d8055e40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f86dc01a000 tx=0x7f86dc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:25.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.743+0000 7f86ebfff700 1 -- 172.21.15.140:0/265504294 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f86e4042400 con 0x7f86f40b51e0 2022-07-21T01:53:25.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:25.879+0000 7f86f897a700 1 -- 172.21.15.140:0/265504294 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f86f40574e0 con 0x7f86f40b51e0 2022-07-21T01:53:26.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.731+0000 7f86ebfff700 1 -- 172.21.15.140:0/265504294 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7f86e4045a20 con 0x7f86f40b51e0 2022-07-21T01:53:26.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 -- 172.21.15.140:0/265504294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d8053980 msgr2=0x7f86d8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:26.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d8053980 0x7f86d8055e40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f86dc01a000 tx=0x7f86dc014000).stop 2022-07-21T01:53:26.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 -- 172.21.15.140:0/265504294 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86f40b51e0 msgr2=0x7f86f41452e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:26.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86f40b51e0 0x7f86f41452e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f86e4001490 tx=0x7f86e4007e50).stop 2022-07-21T01:53:26.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 -- 172.21.15.140:0/265504294 shutdown_connections 2022-07-21T01:53:26.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86d8053980 0x7f86d8055e40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:26.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86f4140cc0 0x7f86f4152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:26.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86f40b6060 0x7f86f4152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:26.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 --2- 172.21.15.140:0/265504294 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86f40b51e0 0x7f86f41452e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:26.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 -- 172.21.15.140:0/265504294 >> 172.21.15.140:0/265504294 conn(0x7f86f4063d10 msgr2=0x7f86f40af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:26.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 -- 172.21.15.140:0/265504294 shutdown_connections 2022-07-21T01:53:26.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.735+0000 7f86f897a700 1 -- 172.21.15.140:0/265504294 wait complete. 2022-07-21T01:53:26.735 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:53:26.746 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:53:26.747 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:53:26.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.855+0000 7fd9b6689700 1 Processor -- start 2022-07-21T01:53:26.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.855+0000 7fd9b6689700 1 -- start start 2022-07-21T01:53:26.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.855+0000 7fd9b6689700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 0x7fd9b0065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:26.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.855+0000 7fd9b6689700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd9b00b2b20 0x7fd9b00b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:26.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.855+0000 7fd9b6689700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b00b26f0 con 0x7fd9b00b7b70 2022-07-21T01:53:26.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.855+0000 7fd9b6689700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b00b2460 con 0x7fd9b00b2b20 2022-07-21T01:53:26.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.855+0000 7fd9b6689700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b014ab30 con 0x7fd9b0061270 2022-07-21T01:53:26.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b4e86700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 0x7fd9b0065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:26.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b5687700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd9b00b7b70 0x7fd9b014a420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:51026/0 (socket says 172.21.15.140:51026) 2022-07-21T01:53:26.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b5687700 1 -- 172.21.15.140:0/2681773102 learned_addr learned my addr 172.21.15.140:0/2681773102 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:26.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b5e88700 1 --2- 172.21.15.140:0/2681773102 >> v2:172.21.15.152:3300/0 conn(0x7fd9b00b2b20 0x7fd9b00b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:26.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9a7fff700 1 -- 172.21.15.140:0/2681773102 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191700663 0 0) 0x7fd9b00b26f0 con 0x7fd9b00b7b70 2022-07-21T01:53:26.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9a7fff700 1 -- 172.21.15.140:0/2681773102 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd994003260 con 0x7fd9b00b7b70 2022-07-21T01:53:26.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b5e88700 1 -- 172.21.15.140:0/2681773102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 msgr2=0x7fd9b0065bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:26.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b5e88700 1 --2- 172.21.15.140:0/2681773102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 0x7fd9b0065bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:26.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b5e88700 1 -- 172.21.15.140:0/2681773102 >> v1:172.21.15.152:6789/0 conn(0x7fd9b00b7b70 legacy=0x7fd9b014a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:26.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b5e88700 1 -- 172.21.15.140:0/2681773102 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b00b7f50 con 0x7fd9b00b2b20 2022-07-21T01:53:26.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b4e86700 1 --2- 172.21.15.140:0/2681773102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 0x7fd9b0065bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:26.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b5e88700 1 --2- 172.21.15.140:0/2681773102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9b00b2b20 0x7fd9b00b0ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fd9ac002f70 tx=0x7fd9ac006df0).ready entity=mon.1 client_cookie=680335d4357abe37 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:26.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9a7fff700 1 -- 172.21.15.140:0/2681773102 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9ac00d4a0 con 0x7fd9b00b2b20 2022-07-21T01:53:26.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9a7fff700 1 -- 172.21.15.140:0/2681773102 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac000e90 con 0x7fd9b00b2b20 2022-07-21T01:53:26.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9a7fff700 1 -- 172.21.15.140:0/2681773102 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9ac00d7f0 con 0x7fd9b00b2b20 2022-07-21T01:53:26.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b6689700 1 -- 172.21.15.140:0/2681773102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9b00b2b20 msgr2=0x7fd9b00b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:26.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b6689700 1 --2- 172.21.15.140:0/2681773102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9b00b2b20 0x7fd9b00b0ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fd9ac002f70 tx=0x7fd9ac006df0).stop 2022-07-21T01:53:26.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b6689700 1 -- 172.21.15.140:0/2681773102 shutdown_connections 2022-07-21T01:53:26.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b6689700 1 --2- 172.21.15.140:0/2681773102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 0x7fd9b0065bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:26.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b6689700 1 --2- 172.21.15.140:0/2681773102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9b00b2b20 0x7fd9b00b0ef0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:26.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b6689700 1 -- 172.21.15.140:0/2681773102 >> 172.21.15.140:0/2681773102 conn(0x7fd9b00656b0 msgr2=0x7fd9b00b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:26.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b6689700 1 -- 172.21.15.140:0/2681773102 shutdown_connections 2022-07-21T01:53:26.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.859+0000 7fd9b6689700 1 -- 172.21.15.140:0/2681773102 wait complete. 2022-07-21T01:53:26.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b6689700 1 Processor -- start 2022-07-21T01:53:26.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b6689700 1 -- start start 2022-07-21T01:53:26.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b6689700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 0x7fd9b0156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:26.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b6689700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd9b00b2b20 0x7fd9b00b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:26.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b6689700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9b00b7b70 0x7fd9b00b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:26.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b6689700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0156f30 con 0x7fd9b00b2b20 2022-07-21T01:53:26.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b6689700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b00b65d0 con 0x7fd9b00b7b70 2022-07-21T01:53:26.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b6689700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b00b4f50 con 0x7fd9b0061270 2022-07-21T01:53:26.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b4e86700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd9b00b2b20 0x7fd9b00b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:26.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b5e88700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9b00b7b70 0x7fd9b00b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:26.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b5687700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 0x7fd9b0156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:26.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b4e86700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd9b00b2b20 0x7fd9b00b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35774/0 (socket says 172.21.15.140:35774) 2022-07-21T01:53:26.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b4e86700 1 -- 172.21.15.140:0/600163837 learned_addr learned my addr 172.21.15.140:0/600163837 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:26.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b5687700 1 -- 172.21.15.140:0/600163837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9b00b7b70 msgr2=0x7fd9b00b7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:26.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b5687700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9b00b7b70 0x7fd9b00b7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:26.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b5687700 1 -- 172.21.15.140:0/600163837 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd9b00b2b20 msgr2=0x7fd9b00b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:26.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b5687700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd9b00b2b20 0x7fd9b00b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:26.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b5687700 1 -- 172.21.15.140:0/600163837 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a000b180 con 0x7fd9b0061270 2022-07-21T01:53:26.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b5687700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 0x7fd9b0156a40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fd9a8001490 tx=0x7fd9a8007d60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:26.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9a67fc700 1 -- 172.21.15.140:0/600163837 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9a800ce70 con 0x7fd9b0061270 2022-07-21T01:53:26.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9a67fc700 1 -- 172.21.15.140:0/600163837 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a8005ba0 con 0x7fd9b0061270 2022-07-21T01:53:26.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9a67fc700 1 -- 172.21.15.140:0/600163837 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9a80070d0 con 0x7fd9b0061270 2022-07-21T01:53:26.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b6689700 1 -- 172.21.15.140:0/600163837 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ac00b180 con 0x7fd9b0061270 2022-07-21T01:53:26.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.863+0000 7fd9b6689700 1 -- 172.21.15.140:0/600163837 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b00b6250 con 0x7fd9b0061270 2022-07-21T01:53:26.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.867+0000 7fd9b6689700 1 -- 172.21.15.140:0/600163837 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b00b53f0 con 0x7fd9b0061270 2022-07-21T01:53:26.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.867+0000 7fd9a67fc700 1 -- 172.21.15.140:0/600163837 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd9a8010070 con 0x7fd9b0061270 2022-07-21T01:53:26.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.867+0000 7fd9a67fc700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd998053a10 0x7fd998055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:26.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.867+0000 7fd9a67fc700 1 -- 172.21.15.140:0/600163837 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd9a806b330 con 0x7fd9b0061270 2022-07-21T01:53:26.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.867+0000 7fd9b4e86700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd998053a10 0x7fd998055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:26.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.867+0000 7fd9b4e86700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd998053a10 0x7fd998055ed0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fd9a000b000 tx=0x7fd9a0006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:26.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:26.875+0000 7fd9a67fc700 1 -- 172.21.15.140:0/600163837 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd9b00b53f0 con 0x7fd9b0061270 2022-07-21T01:53:27.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.039+0000 7fd9b6689700 1 -- 172.21.15.140:0/600163837 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd9b00b53f0 con 0x7fd9b0061270 2022-07-21T01:53:27.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9a67fc700 1 -- 172.21.15.140:0/600163837 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fd9b00b53f0 con 0x7fd9b0061270 2022-07-21T01:53:27.040 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:27.040 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:53:27.040 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:53:27.040 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.041 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:53:27.041 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:53:27.041 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.042 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:53:27.042 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:53:27.042 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:53:27.042 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.043 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.043 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.043 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.044 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:53:27.044 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:53:27.045 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.045 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:53:27.045 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:53:27.046 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:53:27.046 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.046 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.046 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.047 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.047 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:53:27.047 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:53:27.048 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.048 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:53:27.048 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:53:27.049 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.049 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.050 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.050 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.051 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:53:27.051 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:53:27.052 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.052 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:53:27.053 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:53:27.053 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.054 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.054 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.055 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.055 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:53:27.056 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:53:27.056 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.057 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:53:27.057 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:53:27.057 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.058 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.058 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.058 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.059 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:53:27.059 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:53:27.060 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.060 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:53:27.061 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:53:27.061 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.061 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.061 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.062 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.062 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:53:27.062 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:53:27.062 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.063 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:53:27.063 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:53:27.063 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.063 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.064 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.064 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.064 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:53:27.064 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:53:27.064 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.065 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:53:27.065 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:53:27.065 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.065 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.066 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.066 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.066 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:53:27.066 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:53:27.067 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.067 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:53:27.067 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:53:27.067 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.067 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.068 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.068 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.068 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:53:27.069 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:53:27.069 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.069 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:53:27.069 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:53:27.070 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.070 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.070 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.071 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.071 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:53:27.071 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:53:27.071 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.072 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:53:27.072 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:53:27.072 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:53:27.072 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.073 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.073 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.073 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:53:27.073 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:53:27.074 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.074 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:53:27.075 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:53:27.075 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:53:27.075 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.075 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.075 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.076 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:53:27.076 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:53:27.076 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.076 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:53:27.077 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:53:27.077 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:53:27.077 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.077 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.078 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.078 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:53:27.078 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:53:27.078 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.079 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:53:27.079 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:53:27.079 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:53:27.079 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.079 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.080 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.080 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.080 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:53:27.081 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:53:27.081 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.081 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:53:27.081 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:53:27.082 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.082 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.082 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:27.082 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:27.083 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:53:27.083 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:53:27.083 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:53:27.084 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:53:27.084 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:53:27.084 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:53:27.084 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.085 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:27.085 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:27.085 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:53:27.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 -- 172.21.15.140:0/600163837 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd998053a10 msgr2=0x7fd998055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd998053a10 0x7fd998055ed0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fd9a000b000 tx=0x7fd9a0006df0).stop 2022-07-21T01:53:27.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 -- 172.21.15.140:0/600163837 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 msgr2=0x7fd9b0156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 0x7fd9b0156a40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fd9a8001490 tx=0x7fd9a8007d60).stop 2022-07-21T01:53:27.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 -- 172.21.15.140:0/600163837 shutdown_connections 2022-07-21T01:53:27.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd998053a10 0x7fd998055ed0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd9b00b7b70 0x7fd9b00b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd9b00b2b20 0x7fd9b00b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 --2- 172.21.15.140:0/600163837 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9b0061270 0x7fd9b0156a40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 -- 172.21.15.140:0/600163837 >> 172.21.15.140:0/600163837 conn(0x7fd9b00656b0 msgr2=0x7fd9b00b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:27.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 -- 172.21.15.140:0/600163837 shutdown_connections 2022-07-21T01:53:27.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.043+0000 7fd9b6689700 1 -- 172.21.15.140:0/600163837 wait complete. 2022-07-21T01:53:27.091 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:53:27.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 Processor -- start 2022-07-21T01:53:27.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 -- start start 2022-07-21T01:53:27.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5100aebb0 0x7fd5100b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd51013f870 0x7fd51013dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5100aef90 con 0x7fd5100acae0 2022-07-21T01:53:27.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5100b5d60 con 0x7fd51013f870 2022-07-21T01:53:27.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100b0270 con 0x7fd5100aebb0 2022-07-21T01:53:27.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50f7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5100aebb0 0x7fd5100b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd51013f870 0x7fd51013dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd51013f870 0x7fd51013dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:41520/0 (socket says 172.21.15.140:41520) 2022-07-21T01:53:27.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50f7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5100aebb0 0x7fd5100b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:60742/0 (socket says 172.21.15.140:60742) 2022-07-21T01:53:27.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50ffff700 1 -- 172.21.15.140:0/726592050 learned_addr learned my addr 172.21.15.140:0/726592050 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:27.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50ffff700 1 -- 172.21.15.140:0/726592050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5100aebb0 msgr2=0x7fd5100b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:27.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50ffff700 1 --2- 172.21.15.140:0/726592050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5100aebb0 0x7fd5100b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50ffff700 1 -- 172.21.15.140:0/726592050 >> v1:172.21.15.152:6789/0 conn(0x7fd5100acae0 legacy=0x7fd5100b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50ffff700 1 -- 172.21.15.140:0/726592050 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100af200 con 0x7fd51013f870 2022-07-21T01:53:27.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50ffff700 1 --2- 172.21.15.140:0/726592050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51013f870 0x7fd51013dc80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fd5100adfe0 tx=0x7fd508009290).ready entity=mon.1 client_cookie=8d50b43d2591ad21 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:27.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50e7fc700 1 -- 172.21.15.140:0/726592050 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd50800b860 con 0x7fd51013f870 2022-07-21T01:53:27.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50e7fc700 1 -- 172.21.15.140:0/726592050 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50800b9c0 con 0x7fd51013f870 2022-07-21T01:53:27.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd50e7fc700 1 -- 172.21.15.140:0/726592050 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd50800bb30 con 0x7fd51013f870 2022-07-21T01:53:27.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 -- 172.21.15.140:0/726592050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51013f870 msgr2=0x7fd51013dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 --2- 172.21.15.140:0/726592050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51013f870 0x7fd51013dc80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fd5100adfe0 tx=0x7fd508009290).stop 2022-07-21T01:53:27.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 -- 172.21.15.140:0/726592050 shutdown_connections 2022-07-21T01:53:27.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 --2- 172.21.15.140:0/726592050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd51013f870 0x7fd51013dc80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 --2- 172.21.15.140:0/726592050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5100aebb0 0x7fd5100b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 -- 172.21.15.140:0/726592050 >> 172.21.15.140:0/726592050 conn(0x7fd5100634e0 msgr2=0x7fd5100b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:27.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 -- 172.21.15.140:0/726592050 shutdown_connections 2022-07-21T01:53:27.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.199+0000 7fd514ef0700 1 -- 172.21.15.140:0/726592050 wait complete. 2022-07-21T01:53:27.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd514ef0700 1 Processor -- start 2022-07-21T01:53:27.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd514ef0700 1 -- start start 2022-07-21T01:53:27.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd514ef0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5100acae0 0x7fd51013aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd514ef0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5100aebb0 0x7fd510150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd514ef0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd51013f870 0x7fd51013cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd514ef0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51013b390 con 0x7fd5100acae0 2022-07-21T01:53:27.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd514ef0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd510142350 con 0x7fd5100aebb0 2022-07-21T01:53:27.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd514ef0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd510142680 con 0x7fd51013f870 2022-07-21T01:53:27.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5100acae0 0x7fd51013aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5100aebb0 0x7fd510150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd51013f870 0x7fd51013cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5100acae0 0x7fd51013aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35786/0 (socket says 172.21.15.140:35786) 2022-07-21T01:53:27.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5100aebb0 0x7fd510150990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:41528/0 (socket says 172.21.15.140:41528) 2022-07-21T01:53:27.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50f7fe700 1 -- 172.21.15.140:0/3292927912 learned_addr learned my addr 172.21.15.140:0/3292927912 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:27.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50f7fe700 1 -- 172.21.15.140:0/3292927912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd51013f870 msgr2=0x7fd51013cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50f7fe700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd51013f870 0x7fd51013cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50f7fe700 1 -- 172.21.15.140:0/3292927912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5100aebb0 msgr2=0x7fd510150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50f7fe700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5100aebb0 0x7fd510150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50f7fe700 1 -- 172.21.15.140:0/3292927912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd508009530 con 0x7fd5100acae0 2022-07-21T01:53:27.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50effd700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5100aebb0 0x7fd510150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:27.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50f7fe700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5100acae0 0x7fd51013aea0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fd50000c000 tx=0x7fd500006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:27.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50cff9700 1 -- 172.21.15.140:0/3292927912 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd500010780 con 0x7fd5100acae0 2022-07-21T01:53:27.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50cff9700 1 -- 172.21.15.140:0/3292927912 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5000108e0 con 0x7fd5100acae0 2022-07-21T01:53:27.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd514ef0700 1 -- 172.21.15.140:0/3292927912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51013b610 con 0x7fd5100acae0 2022-07-21T01:53:27.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd50cff9700 1 -- 172.21.15.140:0/3292927912 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd500010c40 con 0x7fd5100acae0 2022-07-21T01:53:27.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.203+0000 7fd514ef0700 1 -- 172.21.15.140:0/3292927912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51013a6c0 con 0x7fd5100acae0 2022-07-21T01:53:27.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.207+0000 7fd514ef0700 1 -- 172.21.15.140:0/3292927912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5100af4a0 con 0x7fd5100acae0 2022-07-21T01:53:27.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.207+0000 7fd50cff9700 1 -- 172.21.15.140:0/3292927912 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd50001c460 con 0x7fd5100acae0 2022-07-21T01:53:27.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.207+0000 7fd50cff9700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd4f0053a10 0x7fd4f0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.207+0000 7fd50cff9700 1 -- 172.21.15.140:0/3292927912 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd50006bfc0 con 0x7fd5100acae0 2022-07-21T01:53:27.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.207+0000 7fd50effd700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd4f0053a10 0x7fd4f0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.207+0000 7fd50effd700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd4f0053a10 0x7fd4f0055ed0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fd50400d000 tx=0x7fd504009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:27.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.215+0000 7fd50cff9700 1 -- 172.21.15.140:0/3292927912 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd500043110 con 0x7fd5100acae0 2022-07-21T01:53:27.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.339+0000 7fd514ef0700 1 -- 172.21.15.140:0/3292927912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5100574e0 con 0x7fd5100acae0 2022-07-21T01:53:27.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd50cff9700 1 -- 172.21.15.140:0/3292927912 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+13278 (secure 0 0 0) 0x7fd500046730 con 0x7fd5100acae0 2022-07-21T01:53:27.339 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:27.339 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":59,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:53:26.720977+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:53:27.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 -- 172.21.15.140:0/3292927912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd4f0053a10 msgr2=0x7fd4f0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd4f0053a10 0x7fd4f0055ed0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fd50400d000 tx=0x7fd504009040).stop 2022-07-21T01:53:27.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 -- 172.21.15.140:0/3292927912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5100acae0 msgr2=0x7fd51013aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5100acae0 0x7fd51013aea0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fd50000c000 tx=0x7fd500006df0).stop 2022-07-21T01:53:27.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 -- 172.21.15.140:0/3292927912 shutdown_connections 2022-07-21T01:53:27.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd4f0053a10 0x7fd4f0055ed0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd51013f870 0x7fd51013cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5100aebb0 0x7fd510150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 --2- 172.21.15.140:0/3292927912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5100acae0 0x7fd51013aea0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 -- 172.21.15.140:0/3292927912 >> 172.21.15.140:0/3292927912 conn(0x7fd5100634e0 msgr2=0x7fd5100b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:27.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 -- 172.21.15.140:0/3292927912 shutdown_connections 2022-07-21T01:53:27.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.343+0000 7fd514ef0700 1 -- 172.21.15.140:0/3292927912 wait complete. 2022-07-21T01:53:27.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:53:27.352 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:53:27.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.463+0000 7f0356579700 1 Processor -- start 2022-07-21T01:53:27.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.463+0000 7f0356579700 1 -- start start 2022-07-21T01:53:27.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.463+0000 7f0356579700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f03500b9da0 0x7f03501468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.463+0000 7f0356579700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 0x7f0350142a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.463+0000 7f0356579700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03500ba180 con 0x7f0350142f40 2022-07-21T01:53:27.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.463+0000 7f0356579700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03500ad5b0 con 0x7f03500b9da0 2022-07-21T01:53:27.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.463+0000 7f0356579700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03501461b0 con 0x7f03500b3cf0 2022-07-21T01:53:27.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0355d78700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0350142f40 0x7f035005bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49944/0 (socket says 172.21.15.140:49944) 2022-07-21T01:53:27.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0355d78700 1 -- 172.21.15.140:0/2116363600 learned_addr learned my addr 172.21.15.140:0/2116363600 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:27.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0354d76700 1 --2- 172.21.15.140:0/2116363600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 0x7f0350142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0355577700 1 --2- 172.21.15.140:0/2116363600 >> v2:172.21.15.152:3300/0 conn(0x7f03500b9da0 0x7f03501468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0343fff700 1 -- 172.21.15.140:0/2116363600 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810661795 0 0) 0x7f03500ba180 con 0x7f0350142f40 2022-07-21T01:53:27.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0343fff700 1 -- 172.21.15.140:0/2116363600 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0334003260 con 0x7f0350142f40 2022-07-21T01:53:27.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0354d76700 1 -- 172.21.15.140:0/2116363600 >> v2:172.21.15.152:3300/0 conn(0x7f03500b9da0 msgr2=0x7f03501468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:27.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0354d76700 1 --2- 172.21.15.140:0/2116363600 >> v2:172.21.15.152:3300/0 conn(0x7f03500b9da0 0x7f03501468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0354d76700 1 -- 172.21.15.140:0/2116363600 >> v1:172.21.15.140:6789/0 conn(0x7f0350142f40 legacy=0x7f035005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0354d76700 1 -- 172.21.15.140:0/2116363600 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0350146450 con 0x7f03500b3cf0 2022-07-21T01:53:27.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0354d76700 1 --2- 172.21.15.140:0/2116363600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 0x7f0350142a50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f034c002f70 tx=0x7f034c006df0).ready entity=mon.2 client_cookie=46a2e17fc47abc68 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:27.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0343fff700 1 -- 172.21.15.140:0/2116363600 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f034c00d4a0 con 0x7f03500b3cf0 2022-07-21T01:53:27.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0343fff700 1 -- 172.21.15.140:0/2116363600 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c000e90 con 0x7f03500b3cf0 2022-07-21T01:53:27.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0343fff700 1 -- 172.21.15.140:0/2116363600 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f034c00d7f0 con 0x7f03500b3cf0 2022-07-21T01:53:27.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 -- 172.21.15.140:0/2116363600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 msgr2=0x7f0350142a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 --2- 172.21.15.140:0/2116363600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 0x7f0350142a50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f034c002f70 tx=0x7f034c006df0).stop 2022-07-21T01:53:27.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 -- 172.21.15.140:0/2116363600 shutdown_connections 2022-07-21T01:53:27.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 --2- 172.21.15.140:0/2116363600 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 0x7f0350142a50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 --2- 172.21.15.140:0/2116363600 >> v2:172.21.15.152:3300/0 conn(0x7f03500b9da0 0x7f03501468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 -- 172.21.15.140:0/2116363600 >> 172.21.15.140:0/2116363600 conn(0x7f03500678c0 msgr2=0x7f03500b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:27.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 -- 172.21.15.140:0/2116363600 shutdown_connections 2022-07-21T01:53:27.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 -- 172.21.15.140:0/2116363600 wait complete. 2022-07-21T01:53:27.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 Processor -- start 2022-07-21T01:53:27.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 -- start start 2022-07-21T01:53:27.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 0x7f03500af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03500b9da0 0x7f035013d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.467+0000 7f0356579700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0350142f40 0x7f035013dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0356579700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03500b0c20 con 0x7f0350142f40 2022-07-21T01:53:27.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0356579700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03500b0e50 con 0x7f03500b9da0 2022-07-21T01:53:27.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0356579700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03500b10f0 con 0x7f03500b3cf0 2022-07-21T01:53:27.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0355d78700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0350142f40 0x7f035013dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0355577700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 0x7f03500af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0355577700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 0x7f03500af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:60762/0 (socket says 172.21.15.140:60762) 2022-07-21T01:53:27.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0355577700 1 -- 172.21.15.140:0/2304103522 learned_addr learned my addr 172.21.15.140:0/2304103522 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:27.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0354d76700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03500b9da0 0x7f035013d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0355577700 1 -- 172.21.15.140:0/2304103522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03500b9da0 msgr2=0x7f035013d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0355577700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03500b9da0 0x7f035013d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0355577700 1 -- 172.21.15.140:0/2304103522 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0350142f40 msgr2=0x7f035013dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0355577700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0350142f40 0x7f035013dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0355577700 1 -- 172.21.15.140:0/2304103522 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c00b180 con 0x7f03500b3cf0 2022-07-21T01:53:27.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0355577700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 0x7f03500af020 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f03500b0ac0 tx=0x7f0344006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:27.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f03427fc700 1 -- 172.21.15.140:0/2304103522 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0344010780 con 0x7f03500b3cf0 2022-07-21T01:53:27.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f03427fc700 1 -- 172.21.15.140:0/2304103522 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03440108e0 con 0x7f03500b3cf0 2022-07-21T01:53:27.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0356579700 1 -- 172.21.15.140:0/2304103522 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03500b1390 con 0x7f03500b3cf0 2022-07-21T01:53:27.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f03427fc700 1 -- 172.21.15.140:0/2304103522 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0344010c40 con 0x7f03500b3cf0 2022-07-21T01:53:27.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0356579700 1 -- 172.21.15.140:0/2304103522 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0350141550 con 0x7f03500b3cf0 2022-07-21T01:53:27.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.471+0000 7f0356579700 1 -- 172.21.15.140:0/2304103522 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03500b1520 con 0x7f03500b3cf0 2022-07-21T01:53:27.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.475+0000 7f03427fc700 1 -- 172.21.15.140:0/2304103522 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f034401c460 con 0x7f03500b3cf0 2022-07-21T01:53:27.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.475+0000 7f03427fc700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0338053a70 0x7f0338055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.475+0000 7f03427fc700 1 -- 172.21.15.140:0/2304103522 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f034406c020 con 0x7f03500b3cf0 2022-07-21T01:53:27.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.475+0000 7f0354d76700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0338053a70 0x7f0338055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.475+0000 7f0354d76700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0338053a70 0x7f0338055f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f034c01a000 tx=0x7f034c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:27.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.483+0000 7f03427fc700 1 -- 172.21.15.140:0/2304103522 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0344043170 con 0x7f03500b3cf0 2022-07-21T01:53:27.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.607+0000 7f0356579700 1 -- 172.21.15.140:0/2304103522 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f03500574e0 con 0x7f03500b3cf0 2022-07-21T01:53:27.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f03427fc700 1 -- 172.21.15.140:0/2304103522 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v59) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0344046790 con 0x7f03500b3cf0 2022-07-21T01:53:27.607 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:53:27.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 -- 172.21.15.140:0/2304103522 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0338053a70 msgr2=0x7f0338055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0338053a70 0x7f0338055f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f034c01a000 tx=0x7f034c014000).stop 2022-07-21T01:53:27.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 -- 172.21.15.140:0/2304103522 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 msgr2=0x7f03500af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 0x7f03500af020 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f03500b0ac0 tx=0x7f0344006df0).stop 2022-07-21T01:53:27.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 -- 172.21.15.140:0/2304103522 shutdown_connections 2022-07-21T01:53:27.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0338053a70 0x7f0338055f30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0350142f40 0x7f035013dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03500b9da0 0x7f035013d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 --2- 172.21.15.140:0/2304103522 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f03500b3cf0 0x7f03500af020 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 -- 172.21.15.140:0/2304103522 >> 172.21.15.140:0/2304103522 conn(0x7f03500678c0 msgr2=0x7f03500b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:27.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 -- 172.21.15.140:0/2304103522 shutdown_connections 2022-07-21T01:53:27.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.611+0000 7f0356579700 1 -- 172.21.15.140:0/2304103522 wait complete. 2022-07-21T01:53:27.620 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:53:27.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.735+0000 7f8f756db700 1 Processor -- start 2022-07-21T01:53:27.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.735+0000 7f8f756db700 1 -- start start 2022-07-21T01:53:27.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.735+0000 7f8f756db700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8f700988b0 0x7f8f701422c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.735+0000 7f8f756db700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 0x7f8f70060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.735+0000 7f8f756db700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f700afdb0 con 0x7f8f701462a0 2022-07-21T01:53:27.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.735+0000 7f8f756db700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f700b0050 con 0x7f8f700988b0 2022-07-21T01:53:27.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.735+0000 7f8f756db700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7005fcd0 con 0x7f8f701427b0 2022-07-21T01:53:27.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.735+0000 7f8f6effd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f8f701462a0 0x7f8f70146680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49954/0 (socket says 172.21.15.140:49954) 2022-07-21T01:53:27.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.735+0000 7f8f6effd700 1 -- 172.21.15.140:0/3784140672 learned_addr learned my addr 172.21.15.140:0/3784140672 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:27.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6f7fe700 1 --2- 172.21.15.140:0/3784140672 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 0x7f8f70060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6e7fc700 1 --2- 172.21.15.140:0/3784140672 >> v2:172.21.15.152:3300/0 conn(0x7f8f700988b0 0x7f8f701422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6dffb700 1 -- 172.21.15.140:0/3784140672 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678437534 0 0) 0x7f8f700afdb0 con 0x7f8f701462a0 2022-07-21T01:53:27.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6dffb700 1 -- 172.21.15.140:0/3784140672 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f50003260 con 0x7f8f701462a0 2022-07-21T01:53:27.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6f7fe700 1 -- 172.21.15.140:0/3784140672 >> v2:172.21.15.152:3300/0 conn(0x7f8f700988b0 msgr2=0x7f8f701422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:27.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6f7fe700 1 --2- 172.21.15.140:0/3784140672 >> v2:172.21.15.152:3300/0 conn(0x7f8f700988b0 0x7f8f701422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6f7fe700 1 -- 172.21.15.140:0/3784140672 >> v1:172.21.15.140:6789/0 conn(0x7f8f701462a0 legacy=0x7f8f70146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6f7fe700 1 -- 172.21.15.140:0/3784140672 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f700afae0 con 0x7f8f701427b0 2022-07-21T01:53:27.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6f7fe700 1 --2- 172.21.15.140:0/3784140672 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 0x7f8f70060170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f8f680032f0 tx=0x7f8f680091d0).ready entity=mon.2 client_cookie=e8dcc5e44918a0e6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:27.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6dffb700 1 -- 172.21.15.140:0/3784140672 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f6800b600 con 0x7f8f701427b0 2022-07-21T01:53:27.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6dffb700 1 -- 172.21.15.140:0/3784140672 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6800b760 con 0x7f8f701427b0 2022-07-21T01:53:27.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f6dffb700 1 -- 172.21.15.140:0/3784140672 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f6800b8d0 con 0x7f8f701427b0 2022-07-21T01:53:27.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f756db700 1 -- 172.21.15.140:0/3784140672 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 msgr2=0x7f8f70060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f756db700 1 --2- 172.21.15.140:0/3784140672 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 0x7f8f70060170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f8f680032f0 tx=0x7f8f680091d0).stop 2022-07-21T01:53:27.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f756db700 1 -- 172.21.15.140:0/3784140672 shutdown_connections 2022-07-21T01:53:27.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f756db700 1 --2- 172.21.15.140:0/3784140672 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 0x7f8f70060170 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f756db700 1 --2- 172.21.15.140:0/3784140672 >> v2:172.21.15.152:3300/0 conn(0x7f8f700988b0 0x7f8f701422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f756db700 1 -- 172.21.15.140:0/3784140672 >> 172.21.15.140:0/3784140672 conn(0x7f8f700b5f00 msgr2=0x7f8f700b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:27.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f756db700 1 -- 172.21.15.140:0/3784140672 shutdown_connections 2022-07-21T01:53:27.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f756db700 1 -- 172.21.15.140:0/3784140672 wait complete. 2022-07-21T01:53:27.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f756db700 1 Processor -- start 2022-07-21T01:53:27.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.739+0000 7f8f756db700 1 -- start start 2022-07-21T01:53:27.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f756db700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f700988b0 0x7f8f70140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f756db700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 0x7f8f701555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f756db700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f701462a0 0x7f8f7009ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f756db700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f700b2500 con 0x7f8f701462a0 2022-07-21T01:53:27.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f756db700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f700b2770 con 0x7f8f700988b0 2022-07-21T01:53:27.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f756db700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f701490d0 con 0x7f8f701427b0 2022-07-21T01:53:27.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f6f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f701462a0 0x7f8f7009ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f6f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f701462a0 0x7f8f7009ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35816/0 (socket says 172.21.15.140:35816) 2022-07-21T01:53:27.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f6f7fe700 1 -- 172.21.15.140:0/3501742000 learned_addr learned my addr 172.21.15.140:0/3501742000 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:27.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f6e7fc700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 0x7f8f701555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f6effd700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f700988b0 0x7f8f70140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f6e7fc700 1 -- 172.21.15.140:0/3501742000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f700988b0 msgr2=0x7f8f70140c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f6e7fc700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f700988b0 0x7f8f70140c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f6e7fc700 1 -- 172.21.15.140:0/3501742000 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f701462a0 msgr2=0x7f8f7009ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f6e7fc700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f701462a0 0x7f8f7009ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f6e7fc700 1 -- 172.21.15.140:0/3501742000 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f68009470 con 0x7f8f701427b0 2022-07-21T01:53:27.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f6e7fc700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 0x7f8f701555d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f8f6000c000 tx=0x7f8f60006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:27.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f67fff700 1 -- 172.21.15.140:0/3501742000 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f60016070 con 0x7f8f701427b0 2022-07-21T01:53:27.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f67fff700 1 -- 172.21.15.140:0/3501742000 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f60011750 con 0x7f8f701427b0 2022-07-21T01:53:27.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f756db700 1 -- 172.21.15.140:0/3501742000 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f700b1570 con 0x7f8f701427b0 2022-07-21T01:53:27.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f67fff700 1 -- 172.21.15.140:0/3501742000 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f60011ab0 con 0x7f8f701427b0 2022-07-21T01:53:27.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f756db700 1 -- 172.21.15.140:0/3501742000 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f700b0a40 con 0x7f8f701427b0 2022-07-21T01:53:27.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.743+0000 7f8f756db700 1 -- 172.21.15.140:0/3501742000 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f700b1880 con 0x7f8f701427b0 2022-07-21T01:53:27.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.747+0000 7f8f67fff700 1 -- 172.21.15.140:0/3501742000 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8f60011c10 con 0x7f8f701427b0 2022-07-21T01:53:27.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.747+0000 7f8f67fff700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f54053950 0x7f8f54055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.747+0000 7f8f6effd700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f54053950 0x7f8f54055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:27.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.747+0000 7f8f67fff700 1 -- 172.21.15.140:0/3501742000 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f8f6006bbf0 con 0x7f8f701427b0 2022-07-21T01:53:27.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.747+0000 7f8f6effd700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f54053950 0x7f8f54055e10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f8f5800d000 tx=0x7f8f58009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:27.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.755+0000 7f8f67fff700 1 -- 172.21.15.140:0/3501742000 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f60042d30 con 0x7f8f701427b0 2022-07-21T01:53:27.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.879+0000 7f8f756db700 1 -- 172.21.15.140:0/3501742000 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8f700574e0 con 0x7f8f701427b0 2022-07-21T01:53:27.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.879+0000 7f8f67fff700 1 -- 172.21.15.140:0/3501742000 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8f60046350 con 0x7f8f701427b0 2022-07-21T01:53:27.877 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:53:27.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 -- 172.21.15.140:0/3501742000 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f54053950 msgr2=0x7f8f54055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f54053950 0x7f8f54055e10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f8f5800d000 tx=0x7f8f58009040).stop 2022-07-21T01:53:27.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 -- 172.21.15.140:0/3501742000 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 msgr2=0x7f8f701555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:27.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 0x7f8f701555d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f8f6000c000 tx=0x7f8f60006df0).stop 2022-07-21T01:53:27.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 -- 172.21.15.140:0/3501742000 shutdown_connections 2022-07-21T01:53:27.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f54053950 0x7f8f54055e10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f701462a0 0x7f8f7009ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f701427b0 0x7f8f701555d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 --2- 172.21.15.140:0/3501742000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f700988b0 0x7f8f70140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:27.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 -- 172.21.15.140:0/3501742000 >> 172.21.15.140:0/3501742000 conn(0x7f8f700b5f00 msgr2=0x7f8f700b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:27.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 -- 172.21.15.140:0/3501742000 shutdown_connections 2022-07-21T01:53:27.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.883+0000 7f8f756db700 1 -- 172.21.15.140:0/3501742000 wait complete. 2022-07-21T01:53:27.890 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:53:27.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.995+0000 7f638df5f700 1 Processor -- start 2022-07-21T01:53:27.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.995+0000 7f638df5f700 1 -- start start 2022-07-21T01:53:27.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f638df5f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f63880b13e0 0x7f63880b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:27.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f638df5f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63880b20f0 con 0x7f63880af120 2022-07-21T01:53:27.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f638df5f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63880b65b0 con 0x7f63880b61d0 2022-07-21T01:53:28.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f638df5f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638813efd0 con 0x7f63880b13e0 2022-07-21T01:53:28.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f638d75e700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f63880af120 0x7f6388067260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49972/0 (socket says 172.21.15.140:49972) 2022-07-21T01:53:28.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f638d75e700 1 -- 172.21.15.140:0/4179684323 learned_addr learned my addr 172.21.15.140:0/4179684323 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:28.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f6387fff700 1 --2- 172.21.15.140:0/4179684323 >> v2:172.21.15.152:3300/0 conn(0x7f63880b13e0 0x7f63880b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f63877fe700 1 -- 172.21.15.140:0/4179684323 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830495883 0 0) 0x7f63880b20f0 con 0x7f63880af120 2022-07-21T01:53:28.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f63877fe700 1 -- 172.21.15.140:0/4179684323 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f636c004370 con 0x7f63880af120 2022-07-21T01:53:28.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f6387fff700 1 -- 172.21.15.140:0/4179684323 >> v1:172.21.15.152:6789/0 conn(0x7f63880b61d0 legacy=0x7f63880aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:28.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f6387fff700 1 -- 172.21.15.140:0/4179684323 >> v1:172.21.15.140:6789/0 conn(0x7f63880af120 legacy=0x7f6388067260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:28.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f6387fff700 1 -- 172.21.15.140:0/4179684323 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63880b1ad0 con 0x7f63880b13e0 2022-07-21T01:53:28.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f6387fff700 1 --2- 172.21.15.140:0/4179684323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880b13e0 0x7f63880b0460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f6374002f70 tx=0x7f6374006df0).ready entity=mon.1 client_cookie=fd70ab12ea200298 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:28.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f63877fe700 1 -- 172.21.15.140:0/4179684323 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637400d610 con 0x7f63880b13e0 2022-07-21T01:53:28.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f63877fe700 1 -- 172.21.15.140:0/4179684323 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6374000e90 con 0x7f63880b13e0 2022-07-21T01:53:28.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f63877fe700 1 -- 172.21.15.140:0/4179684323 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637400d960 con 0x7f63880b13e0 2022-07-21T01:53:28.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f638df5f700 1 -- 172.21.15.140:0/4179684323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880b13e0 msgr2=0x7f63880b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:28.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:27.999+0000 7f638df5f700 1 --2- 172.21.15.140:0/4179684323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880b13e0 0x7f63880b0460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f6374002f70 tx=0x7f6374006df0).stop 2022-07-21T01:53:28.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 -- 172.21.15.140:0/4179684323 shutdown_connections 2022-07-21T01:53:28.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 --2- 172.21.15.140:0/4179684323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880b13e0 0x7f63880b0460 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 -- 172.21.15.140:0/4179684323 >> 172.21.15.140:0/4179684323 conn(0x7f6388063cf0 msgr2=0x7f63880b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:28.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 -- 172.21.15.140:0/4179684323 shutdown_connections 2022-07-21T01:53:28.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 -- 172.21.15.140:0/4179684323 wait complete. 2022-07-21T01:53:28.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 Processor -- start 2022-07-21T01:53:28.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 -- start start 2022-07-21T01:53:28.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880af120 0x7f638814f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:28.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63880b13e0 0x7f638814fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:28.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63880b61d0 0x7f63881118d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:28.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6388150680 con 0x7f63880b13e0 2022-07-21T01:53:28.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6388150920 con 0x7f63880af120 2022-07-21T01:53:28.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638df5f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63881503b0 con 0x7f63880b61d0 2022-07-21T01:53:28.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f6387fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63880b13e0 0x7f638814fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f6387fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63880b13e0 0x7f638814fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35830/0 (socket says 172.21.15.140:35830) 2022-07-21T01:53:28.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f6387fff700 1 -- 172.21.15.140:0/1045138335 learned_addr learned my addr 172.21.15.140:0/1045138335 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:28.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638d75e700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63880b61d0 0x7f63881118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f638cf5d700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880af120 0x7f638814f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f6387fff700 1 -- 172.21.15.140:0/1045138335 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63880b61d0 msgr2=0x7f63881118d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:28.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f6387fff700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63880b61d0 0x7f63881118d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f6387fff700 1 -- 172.21.15.140:0/1045138335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880af120 msgr2=0x7f638814f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:28.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f6387fff700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880af120 0x7f638814f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f6387fff700 1 -- 172.21.15.140:0/1045138335 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637400b180 con 0x7f63880b13e0 2022-07-21T01:53:28.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f6387fff700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63880b13e0 0x7f638814fec0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f6374002f70 tx=0x7f6374014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:28.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.003+0000 7f6385ffb700 1 -- 172.21.15.140:0/1045138335 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6374016070 con 0x7f63880b13e0 2022-07-21T01:53:28.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.007+0000 7f6385ffb700 1 -- 172.21.15.140:0/1045138335 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637400de90 con 0x7f63880b13e0 2022-07-21T01:53:28.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.007+0000 7f638df5f700 1 -- 172.21.15.140:0/1045138335 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6388065be0 con 0x7f63880b13e0 2022-07-21T01:53:28.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.007+0000 7f6385ffb700 1 -- 172.21.15.140:0/1045138335 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6374013da0 con 0x7f63880b13e0 2022-07-21T01:53:28.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.007+0000 7f638df5f700 1 -- 172.21.15.140:0/1045138335 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638810eb90 con 0x7f63880b13e0 2022-07-21T01:53:28.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.007+0000 7f638df5f700 1 -- 172.21.15.140:0/1045138335 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63880b1d70 con 0x7f63880b13e0 2022-07-21T01:53:28.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.007+0000 7f6385ffb700 1 -- 172.21.15.140:0/1045138335 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6374024030 con 0x7f63880b13e0 2022-07-21T01:53:28.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.007+0000 7f6385ffb700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6368053aa0 0x7f6368055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:28.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.007+0000 7f6385ffb700 1 -- 172.21.15.140:0/1045138335 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f637406c6e0 con 0x7f63880b13e0 2022-07-21T01:53:28.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.007+0000 7f638cf5d700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6368053aa0 0x7f6368055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.011+0000 7f638cf5d700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6368053aa0 0x7f6368055f60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f637c00d000 tx=0x7f637c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:28.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.015+0000 7f6385ffb700 1 -- 172.21.15.140:0/1045138335 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6374043930 con 0x7f63880b13e0 2022-07-21T01:53:28.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.143+0000 7f638df5f700 1 -- 172.21.15.140:0/1045138335 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f63880574e0 con 0x7f63880b13e0 2022-07-21T01:53:28.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.143+0000 7f6385ffb700 1 -- 172.21.15.140:0/1045138335 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) v1 ==== 84+0+143 (secure 0 0 0) 0x7f6374022070 con 0x7f63880b13e0 2022-07-21T01:53:28.140 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:28.141 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:53:28.141 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:28.141 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:53:28.142 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:53:28.142 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:28.142 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:28.143 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:53:28.143 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:53:28.143 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:28.143 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:53:28.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.143+0000 7f638df5f700 1 -- 172.21.15.140:0/1045138335 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6368053aa0 msgr2=0x7f6368055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:28.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.147+0000 7f638df5f700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6368053aa0 0x7f6368055f60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f637c00d000 tx=0x7f637c009040).stop 2022-07-21T01:53:28.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.147+0000 7f638df5f700 1 -- 172.21.15.140:0/1045138335 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63880b13e0 msgr2=0x7f638814fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:28.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.147+0000 7f638df5f700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63880b13e0 0x7f638814fec0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f6374002f70 tx=0x7f6374014000).stop 2022-07-21T01:53:28.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.147+0000 7f638df5f700 1 -- 172.21.15.140:0/1045138335 shutdown_connections 2022-07-21T01:53:28.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.147+0000 7f638df5f700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6368053aa0 0x7f6368055f60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.147+0000 7f638df5f700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63880b61d0 0x7f63881118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.147+0000 7f638df5f700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63880b13e0 0x7f638814fec0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.147+0000 7f638df5f700 1 --2- 172.21.15.140:0/1045138335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63880af120 0x7f638814f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.147+0000 7f638df5f700 1 -- 172.21.15.140:0/1045138335 >> 172.21.15.140:0/1045138335 conn(0x7f6388063cf0 msgr2=0x7f63880acf40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:28.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.147+0000 7f638df5f700 1 -- 172.21.15.140:0/1045138335 shutdown_connections 2022-07-21T01:53:28.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.147+0000 7f638df5f700 1 -- 172.21.15.140:0/1045138335 wait complete. 2022-07-21T01:53:28.153 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:53:28.154 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:53:28.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.263+0000 7f9b8502d700 1 Processor -- start 2022-07-21T01:53:28.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.263+0000 7f9b8502d700 1 -- start start 2022-07-21T01:53:28.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8502d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9b8013fec0 0x7f9b800b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:28.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8502d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9b801402a0 0x7f9b80146940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:28.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8502d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b8005f840 0x7f9b800aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:28.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8502d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b800b2260 con 0x7f9b801402a0 2022-07-21T01:53:28.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8502d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b800b2010 con 0x7f9b8013fec0 2022-07-21T01:53:28.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8502d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80146200 con 0x7f9b8005f840 2022-07-21T01:53:28.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b777fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9b801402a0 0x7f9b80146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b777fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9b801402a0 0x7f9b80146940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35838/0 (socket says 172.21.15.140:35838) 2022-07-21T01:53:28.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b777fe700 1 -- 172.21.15.140:0/3544310999 learned_addr learned my addr 172.21.15.140:0/3544310999 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:28.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8482c700 1 --2- 172.21.15.140:0/3544310999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b8005f840 0x7f9b800aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b7ffff700 1 --2- 172.21.15.140:0/3544310999 >> v2:172.21.15.152:3300/0 conn(0x7f9b8013fec0 0x7f9b800b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8482c700 1 -- 172.21.15.140:0/3544310999 >> v2:172.21.15.152:3300/0 conn(0x7f9b8013fec0 msgr2=0x7f9b800b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:28.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8482c700 1 --2- 172.21.15.140:0/3544310999 >> v2:172.21.15.152:3300/0 conn(0x7f9b8013fec0 0x7f9b800b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8482c700 1 -- 172.21.15.140:0/3544310999 >> v2:172.21.15.140:3300/0 conn(0x7f9b801402a0 msgr2=0x7f9b80146940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:28.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8482c700 1 --2- 172.21.15.140:0/3544310999 >> v2:172.21.15.140:3300/0 conn(0x7f9b801402a0 0x7f9b80146940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8482c700 1 -- 172.21.15.140:0/3544310999 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b801464a0 con 0x7f9b8005f840 2022-07-21T01:53:28.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b7ffff700 1 --2- 172.21.15.140:0/3544310999 >> v2:172.21.15.152:3300/0 conn(0x7f9b8013fec0 0x7f9b800b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:28.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b777fe700 1 --2- 172.21.15.140:0/3544310999 >> v2:172.21.15.140:3300/0 conn(0x7f9b801402a0 0x7f9b80146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:28.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b8482c700 1 --2- 172.21.15.140:0/3544310999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b8005f840 0x7f9b800aca40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9b780032f0 tx=0x7f9b780091d0).ready entity=mon.2 client_cookie=de482f24b1010431 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:28.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b7f7fe700 1 -- 172.21.15.140:0/3544310999 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b7800b600 con 0x7f9b8005f840 2022-07-21T01:53:28.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b7f7fe700 1 -- 172.21.15.140:0/3544310999 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7800b760 con 0x7f9b8005f840 2022-07-21T01:53:28.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.267+0000 7f9b7f7fe700 1 -- 172.21.15.140:0/3544310999 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b7800b8d0 con 0x7f9b8005f840 2022-07-21T01:53:28.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 -- 172.21.15.140:0/3544310999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b8005f840 msgr2=0x7f9b800aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:28.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 --2- 172.21.15.140:0/3544310999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b8005f840 0x7f9b800aca40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9b780032f0 tx=0x7f9b780091d0).stop 2022-07-21T01:53:28.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 -- 172.21.15.140:0/3544310999 shutdown_connections 2022-07-21T01:53:28.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 --2- 172.21.15.140:0/3544310999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b8005f840 0x7f9b800aca40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 --2- 172.21.15.140:0/3544310999 >> v2:172.21.15.140:3300/0 conn(0x7f9b801402a0 0x7f9b80146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 --2- 172.21.15.140:0/3544310999 >> v2:172.21.15.152:3300/0 conn(0x7f9b8013fec0 0x7f9b800b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 -- 172.21.15.140:0/3544310999 >> 172.21.15.140:0/3544310999 conn(0x7f9b800b2af0 msgr2=0x7f9b800b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:28.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 -- 172.21.15.140:0/3544310999 shutdown_connections 2022-07-21T01:53:28.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 -- 172.21.15.140:0/3544310999 wait complete. 2022-07-21T01:53:28.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 Processor -- start 2022-07-21T01:53:28.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 -- start start 2022-07-21T01:53:28.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8005f840 0x7f9b80152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:28.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9b8013fec0 0x7f9b800ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:28.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b801402a0 0x7f9b800aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:28.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80151e80 con 0x7f9b8013fec0 2022-07-21T01:53:28.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80152120 con 0x7f9b8005f840 2022-07-21T01:53:28.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8502d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b801523c0 con 0x7f9b801402a0 2022-07-21T01:53:28.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b777fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9b8013fec0 0x7f9b800ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b777fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9b8013fec0 0x7f9b800ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35846/0 (socket says 172.21.15.140:35846) 2022-07-21T01:53:28.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b777fe700 1 -- 172.21.15.140:0/2345007325 learned_addr learned my addr 172.21.15.140:0/2345007325 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:28.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b8482c700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b801402a0 0x7f9b800aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.271+0000 7f9b7ffff700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8005f840 0x7f9b80152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b777fe700 1 -- 172.21.15.140:0/2345007325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b801402a0 msgr2=0x7f9b800aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:28.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b777fe700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b801402a0 0x7f9b800aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b777fe700 1 -- 172.21.15.140:0/2345007325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8005f840 msgr2=0x7f9b80152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:28.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b777fe700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8005f840 0x7f9b80152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:28.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b777fe700 1 -- 172.21.15.140:0/2345007325 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7000b120 con 0x7f9b8013fec0 2022-07-21T01:53:28.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b8482c700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b801402a0 0x7f9b800aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:53:28.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b777fe700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9b8013fec0 0x7f9b800ae920 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f9b6800c000 tx=0x7f9b68006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:28.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b7dffb700 1 -- 172.21.15.140:0/2345007325 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b68012600 con 0x7f9b8013fec0 2022-07-21T01:53:28.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b7dffb700 1 -- 172.21.15.140:0/2345007325 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b68012760 con 0x7f9b8013fec0 2022-07-21T01:53:28.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b7dffb700 1 -- 172.21.15.140:0/2345007325 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b68012ac0 con 0x7f9b8013fec0 2022-07-21T01:53:28.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b8502d700 1 -- 172.21.15.140:0/2345007325 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6800b1e0 con 0x7f9b8013fec0 2022-07-21T01:53:28.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b8502d700 1 -- 172.21.15.140:0/2345007325 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b78009470 con 0x7f9b8013fec0 2022-07-21T01:53:28.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b8502d700 1 -- 172.21.15.140:0/2345007325 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b800574e0 con 0x7f9b8013fec0 2022-07-21T01:53:28.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b7dffb700 1 -- 172.21.15.140:0/2345007325 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9b68012c20 con 0x7f9b8013fec0 2022-07-21T01:53:28.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b7dffb700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9b640539e0 0x7f9b64055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:28.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b7dffb700 1 -- 172.21.15.140:0/2345007325 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9b6806bd60 con 0x7f9b8013fec0 2022-07-21T01:53:28.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b7ffff700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9b640539e0 0x7f9b64055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:28.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.275+0000 7f9b7ffff700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9b640539e0 0x7f9b64055ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f9b70008000 tx=0x7f9b70008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:28.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.279+0000 7f9b7dffb700 1 -- 172.21.15.140:0/2345007325 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9b68042f10 con 0x7f9b8013fec0 2022-07-21T01:53:28.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:28.403+0000 7f9b8502d700 1 -- 172.21.15.140:0/2345007325 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f9b80152580 con 0x7f9b8013fec0 2022-07-21T01:53:29.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.143+0000 7f9b7dffb700 1 -- 172.21.15.140:0/2345007325 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9b68046530 con 0x7f9b8013fec0 2022-07-21T01:53:29.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 -- 172.21.15.140:0/2345007325 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9b640539e0 msgr2=0x7f9b64055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:29.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9b640539e0 0x7f9b64055ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f9b70008000 tx=0x7f9b70008040).stop 2022-07-21T01:53:29.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 -- 172.21.15.140:0/2345007325 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9b8013fec0 msgr2=0x7f9b800ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:29.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9b8013fec0 0x7f9b800ae920 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f9b6800c000 tx=0x7f9b68006eb0).stop 2022-07-21T01:53:29.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 -- 172.21.15.140:0/2345007325 shutdown_connections 2022-07-21T01:53:29.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9b640539e0 0x7f9b64055ea0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:29.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b801402a0 0x7f9b800aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:29.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9b8013fec0 0x7f9b800ae920 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:29.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 --2- 172.21.15.140:0/2345007325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b8005f840 0x7f9b80152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:29.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 -- 172.21.15.140:0/2345007325 >> 172.21.15.140:0/2345007325 conn(0x7f9b800b2af0 msgr2=0x7f9b8013e710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:29.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 -- 172.21.15.140:0/2345007325 shutdown_connections 2022-07-21T01:53:29.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.151+0000 7f9b8502d700 1 -- 172.21.15.140:0/2345007325 wait complete. 2022-07-21T01:53:29.150 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:53:29.161 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:53:29.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1393fff700 1 Processor -- start 2022-07-21T01:53:29.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1393fff700 1 -- start start 2022-07-21T01:53:29.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1393fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f13940b0b90 0x7f13940aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:29.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1393fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f13940ac9f0 0x7f1394140520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:29.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1393fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13940aced0 con 0x7f13940ac610 2022-07-21T01:53:29.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1393fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1394143c80 con 0x7f13940b0b90 2022-07-21T01:53:29.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1393fff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1394143f20 con 0x7f13940ac9f0 2022-07-21T01:53:29.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f13927fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f13940b0b90 0x7f13940aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:29.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1392ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f13940ac610 0x7f139405fad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:49996/0 (socket says 172.21.15.140:49996) 2022-07-21T01:53:29.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1392ffd700 1 -- 172.21.15.140:0/168623723 learned_addr learned my addr 172.21.15.140:0/168623723 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:29.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f13937fe700 1 --2- 172.21.15.140:0/168623723 >> v2:172.21.15.152:3300/0 conn(0x7f13940ac9f0 0x7f1394140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:29.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f13927fc700 1 -- 172.21.15.140:0/168623723 >> v2:172.21.15.152:3300/0 conn(0x7f13940ac9f0 msgr2=0x7f1394140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:29.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f13927fc700 1 --2- 172.21.15.140:0/168623723 >> v2:172.21.15.152:3300/0 conn(0x7f13940ac9f0 0x7f1394140520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:29.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f13927fc700 1 -- 172.21.15.140:0/168623723 >> v1:172.21.15.140:6789/0 conn(0x7f13940ac610 legacy=0x7f139405fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:29.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1391ffb700 1 -- 172.21.15.140:0/168623723 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692512439 0 0) 0x7f13940aced0 con 0x7f13940ac610 2022-07-21T01:53:29.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f13927fc700 1 -- 172.21.15.140:0/168623723 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394098b20 con 0x7f13940b0b90 2022-07-21T01:53:29.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f13927fc700 1 --2- 172.21.15.140:0/168623723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f13940b0b90 0x7f13940aba70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f1388002f70 tx=0x7f1388006df0).ready entity=mon.0 client_cookie=f57502ac5434d947 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:29.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1391ffb700 1 -- 172.21.15.140:0/168623723 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f138800d570 con 0x7f13940b0b90 2022-07-21T01:53:29.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1391ffb700 1 -- 172.21.15.140:0/168623723 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1388000e90 con 0x7f13940b0b90 2022-07-21T01:53:29.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.279+0000 7f1391ffb700 1 -- 172.21.15.140:0/168623723 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f138800d8c0 con 0x7f13940b0b90 2022-07-21T01:53:29.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 -- 172.21.15.140:0/168623723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f13940b0b90 msgr2=0x7f13940aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:29.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 --2- 172.21.15.140:0/168623723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f13940b0b90 0x7f13940aba70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f1388002f70 tx=0x7f1388006df0).stop 2022-07-21T01:53:29.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 -- 172.21.15.140:0/168623723 shutdown_connections 2022-07-21T01:53:29.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 --2- 172.21.15.140:0/168623723 >> v2:172.21.15.152:3300/0 conn(0x7f13940ac9f0 0x7f1394140520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:29.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 --2- 172.21.15.140:0/168623723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f13940b0b90 0x7f13940aba70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:29.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 -- 172.21.15.140:0/168623723 >> 172.21.15.140:0/168623723 conn(0x7f13940b61f0 msgr2=0x7f13940b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:29.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 -- 172.21.15.140:0/168623723 shutdown_connections 2022-07-21T01:53:29.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 -- 172.21.15.140:0/168623723 wait complete. 2022-07-21T01:53:29.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 Processor -- start 2022-07-21T01:53:29.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 -- start start 2022-07-21T01:53:29.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f13940ac610 0x7f139409a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:29.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940ac9f0 0x7f13940aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:29.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f13940b0b90 0x7f1394151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:29.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394067b60 con 0x7f13940ac610 2022-07-21T01:53:29.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394066910 con 0x7f13940ac9f0 2022-07-21T01:53:29.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1393fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1394066bb0 con 0x7f13940b0b90 2022-07-21T01:53:29.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f13940ac610 0x7f139409a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:29.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f13940ac610 0x7f139409a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35856/0 (socket says 172.21.15.140:35856) 2022-07-21T01:53:29.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1392ffd700 1 -- 172.21.15.140:0/1699810009 learned_addr learned my addr 172.21.15.140:0/1699810009 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:29.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f13927fc700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940ac9f0 0x7f13940aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:29.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f13937fe700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f13940b0b90 0x7f1394151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:29.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1392ffd700 1 -- 172.21.15.140:0/1699810009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f13940b0b90 msgr2=0x7f1394151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:29.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1392ffd700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f13940b0b90 0x7f1394151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:29.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1392ffd700 1 -- 172.21.15.140:0/1699810009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940ac9f0 msgr2=0x7f13940aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:29.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1392ffd700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940ac9f0 0x7f13940aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:29.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1392ffd700 1 -- 172.21.15.140:0/1699810009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138800b180 con 0x7f13940ac610 2022-07-21T01:53:29.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.283+0000 7f1392ffd700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f13940ac610 0x7f139409a110 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f13940af110 tx=0x7f138c007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:29.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.287+0000 7f1383fff700 1 -- 172.21.15.140:0/1699810009 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f138c005ba0 con 0x7f13940ac610 2022-07-21T01:53:29.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.287+0000 7f1383fff700 1 -- 172.21.15.140:0/1699810009 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138c005d00 con 0x7f13940ac610 2022-07-21T01:53:29.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.287+0000 7f1393fff700 1 -- 172.21.15.140:0/1699810009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1394066e50 con 0x7f13940ac610 2022-07-21T01:53:29.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.287+0000 7f1383fff700 1 -- 172.21.15.140:0/1699810009 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f138c007300 con 0x7f13940ac610 2022-07-21T01:53:29.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.287+0000 7f1393fff700 1 -- 172.21.15.140:0/1699810009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1394066610 con 0x7f13940ac610 2022-07-21T01:53:29.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.287+0000 7f1393fff700 1 -- 172.21.15.140:0/1699810009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1394067ec0 con 0x7f13940ac610 2022-07-21T01:53:29.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.287+0000 7f1383fff700 1 -- 172.21.15.140:0/1699810009 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f138c010030 con 0x7f13940ac610 2022-07-21T01:53:29.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.287+0000 7f1383fff700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1370053aa0 0x7f1370055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:29.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.287+0000 7f13927fc700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1370053aa0 0x7f1370055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:29.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.287+0000 7f1383fff700 1 -- 172.21.15.140:0/1699810009 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f138c06bd80 con 0x7f13940ac610 2022-07-21T01:53:29.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.291+0000 7f13927fc700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1370053aa0 0x7f1370055f60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f1388018000 tx=0x7f1388014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:29.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.299+0000 7f1383fff700 1 -- 172.21.15.140:0/1699810009 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f138c042e70 con 0x7f13940ac610 2022-07-21T01:53:29.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:29.423+0000 7f1393fff700 1 -- 172.21.15.140:0/1699810009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f139409bf80 con 0x7f13940ac610 2022-07-21T01:53:30.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.159+0000 7f1383fff700 1 -- 172.21.15.140:0/1699810009 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v61) v1 ==== 125+0+0 (secure 0 0 0) 0x7f138c06d070 con 0x7f13940ac610 2022-07-21T01:53:30.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.163+0000 7f1393fff700 1 -- 172.21.15.140:0/1699810009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1370053aa0 msgr2=0x7f1370055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.163+0000 7f1393fff700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1370053aa0 0x7f1370055f60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f1388018000 tx=0x7f1388014000).stop 2022-07-21T01:53:30.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.163+0000 7f1393fff700 1 -- 172.21.15.140:0/1699810009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f13940ac610 msgr2=0x7f139409a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.163+0000 7f1393fff700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f13940ac610 0x7f139409a110 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f13940af110 tx=0x7f138c007e40).stop 2022-07-21T01:53:30.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.167+0000 7f1393fff700 1 -- 172.21.15.140:0/1699810009 shutdown_connections 2022-07-21T01:53:30.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.167+0000 7f1393fff700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1370053aa0 0x7f1370055f60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.167+0000 7f1393fff700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f13940b0b90 0x7f1394151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.167+0000 7f1393fff700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13940ac9f0 0x7f13940aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.167+0000 7f1393fff700 1 --2- 172.21.15.140:0/1699810009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f13940ac610 0x7f139409a110 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.167+0000 7f1393fff700 1 -- 172.21.15.140:0/1699810009 >> 172.21.15.140:0/1699810009 conn(0x7f13940b61f0 msgr2=0x7f139413f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:30.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.167+0000 7f1393fff700 1 -- 172.21.15.140:0/1699810009 shutdown_connections 2022-07-21T01:53:30.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.167+0000 7f1393fff700 1 -- 172.21.15.140:0/1699810009 wait complete. 2022-07-21T01:53:30.165 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:53:30.177 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:53:30.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.287+0000 7f6ae639f700 1 Processor -- start 2022-07-21T01:53:30.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.287+0000 7f6ae639f700 1 -- start start 2022-07-21T01:53:30.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.287+0000 7f6ae639f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6ae00b4040 0x7f6ae005ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.287+0000 7f6ae639f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae00b4420 con 0x7f6ae00acae0 2022-07-21T01:53:30.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.287+0000 7f6ae639f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae005c320 con 0x7f6ae0060090 2022-07-21T01:53:30.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.287+0000 7f6ae639f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae005f950 con 0x7f6ae00b4040 2022-07-21T01:53:30.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6ae493c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f6ae00acae0 0x7f6ae00acf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50014/0 (socket says 172.21.15.140:50014) 2022-07-21T01:53:30.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6ae493c700 1 -- 172.21.15.140:0/1615619750 learned_addr learned my addr 172.21.15.140:0/1615619750 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:30.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6adffff700 1 --2- 172.21.15.140:0/1615619750 >> v2:172.21.15.152:3300/0 conn(0x7f6ae00b4040 0x7f6ae005ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6adeffd700 1 -- 172.21.15.140:0/1615619750 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437577797 0 0) 0x7f6ae00b4420 con 0x7f6ae00acae0 2022-07-21T01:53:30.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6adeffd700 1 -- 172.21.15.140:0/1615619750 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ac0003260 con 0x7f6ae00acae0 2022-07-21T01:53:30.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6adffff700 1 -- 172.21.15.140:0/1615619750 >> v1:172.21.15.152:6789/0 conn(0x7f6ae0060090 legacy=0x7f6ae01462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6adffff700 1 -- 172.21.15.140:0/1615619750 >> v1:172.21.15.140:6789/0 conn(0x7f6ae00acae0 legacy=0x7f6ae00acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6adffff700 1 -- 172.21.15.140:0/1615619750 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae005fbf0 con 0x7f6ae00b4040 2022-07-21T01:53:30.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6adffff700 1 --2- 172.21.15.140:0/1615619750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ae00b4040 0x7f6ae005ced0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f6ac8002f70 tx=0x7f6ac8006d90).ready entity=mon.1 client_cookie=a44fde8e5dfbb4c7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:30.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6adeffd700 1 -- 172.21.15.140:0/1615619750 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac800d4a0 con 0x7f6ae00b4040 2022-07-21T01:53:30.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6adeffd700 1 -- 172.21.15.140:0/1615619750 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac8000e90 con 0x7f6ae00b4040 2022-07-21T01:53:30.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6adeffd700 1 -- 172.21.15.140:0/1615619750 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac800d7f0 con 0x7f6ae00b4040 2022-07-21T01:53:30.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6ae639f700 1 -- 172.21.15.140:0/1615619750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ae00b4040 msgr2=0x7f6ae005ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6ae639f700 1 --2- 172.21.15.140:0/1615619750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ae00b4040 0x7f6ae005ced0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f6ac8002f70 tx=0x7f6ac8006d90).stop 2022-07-21T01:53:30.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6ae639f700 1 -- 172.21.15.140:0/1615619750 shutdown_connections 2022-07-21T01:53:30.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6ae639f700 1 --2- 172.21.15.140:0/1615619750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ae00b4040 0x7f6ae005ced0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6ae639f700 1 -- 172.21.15.140:0/1615619750 >> 172.21.15.140:0/1615619750 conn(0x7f6ae0064d20 msgr2=0x7f6ae0065100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:30.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6ae639f700 1 -- 172.21.15.140:0/1615619750 shutdown_connections 2022-07-21T01:53:30.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.291+0000 7f6ae639f700 1 -- 172.21.15.140:0/1615619750 wait complete. 2022-07-21T01:53:30.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6ae639f700 1 Processor -- start 2022-07-21T01:53:30.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6ae639f700 1 -- start start 2022-07-21T01:53:30.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6ae639f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6ae0060090 0x7f6ae0145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6ae639f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae00acae0 0x7f6ae01400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6ae639f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ae00b4040 0x7f6ae0140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6ae639f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae00b0280 con 0x7f6ae0060090 2022-07-21T01:53:30.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6ae639f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae00b1550 con 0x7f6ae00b4040 2022-07-21T01:53:30.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6ae639f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae00b04b0 con 0x7f6ae00acae0 2022-07-21T01:53:30.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6adffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6ae0060090 0x7f6ae0145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6adffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6ae0060090 0x7f6ae0145d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35870/0 (socket says 172.21.15.140:35870) 2022-07-21T01:53:30.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6adffff700 1 -- 172.21.15.140:0/450492903 learned_addr learned my addr 172.21.15.140:0/450492903 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:30.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6adf7fe700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae00acae0 0x7f6ae01400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6ae493c700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ae00b4040 0x7f6ae0140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6adffff700 1 -- 172.21.15.140:0/450492903 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae00acae0 msgr2=0x7f6ae01400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6adffff700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae00acae0 0x7f6ae01400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6adffff700 1 -- 172.21.15.140:0/450492903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ae00b4040 msgr2=0x7f6ae0140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6adffff700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ae00b4040 0x7f6ae0140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6adffff700 1 -- 172.21.15.140:0/450492903 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac800b120 con 0x7f6ae0060090 2022-07-21T01:53:30.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6adffff700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6ae0060090 0x7f6ae0145d40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f6ac8002f70 tx=0x7f6ac8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:30.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6add7fa700 1 -- 172.21.15.140:0/450492903 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac8011880 con 0x7f6ae0060090 2022-07-21T01:53:30.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6add7fa700 1 -- 172.21.15.140:0/450492903 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac80119e0 con 0x7f6ae0060090 2022-07-21T01:53:30.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.295+0000 7f6ae639f700 1 -- 172.21.15.140:0/450492903 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae00b12b0 con 0x7f6ae0060090 2022-07-21T01:53:30.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.299+0000 7f6add7fa700 1 -- 172.21.15.140:0/450492903 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ac8011d40 con 0x7f6ae0060090 2022-07-21T01:53:30.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.299+0000 7f6ae639f700 1 -- 172.21.15.140:0/450492903 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae0140a80 con 0x7f6ae0060090 2022-07-21T01:53:30.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.299+0000 7f6ae639f700 1 -- 172.21.15.140:0/450492903 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae01469b0 con 0x7f6ae0060090 2022-07-21T01:53:30.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.299+0000 7f6add7fa700 1 -- 172.21.15.140:0/450492903 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6ac800f070 con 0x7f6ae0060090 2022-07-21T01:53:30.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.299+0000 7f6add7fa700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ac4053b00 0x7f6ac4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.299+0000 7f6add7fa700 1 -- 172.21.15.140:0/450492903 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f6ac806d300 con 0x7f6ae0060090 2022-07-21T01:53:30.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.299+0000 7f6adf7fe700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ac4053b00 0x7f6ac4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.299+0000 7f6adf7fe700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ac4053b00 0x7f6ac4055fc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f6ad000d000 tx=0x7f6ad0007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:30.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.307+0000 7f6add7fa700 1 -- 172.21.15.140:0/450492903 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6ac8044180 con 0x7f6ae0060090 2022-07-21T01:53:30.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.471+0000 7f6ae639f700 1 -- 172.21.15.140:0/450492903 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f6ae00574e0 con 0x7f6ae0060090 2022-07-21T01:53:30.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6add7fa700 1 -- 172.21.15.140:0/450492903 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v25) v1 ==== 189+0+0 (secure 0 0 0) 0x7f6ac8006920 con 0x7f6ae0060090 2022-07-21T01:53:30.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 -- 172.21.15.140:0/450492903 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ac4053b00 msgr2=0x7f6ac4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ac4053b00 0x7f6ac4055fc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f6ad000d000 tx=0x7f6ad0007b90).stop 2022-07-21T01:53:30.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 -- 172.21.15.140:0/450492903 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6ae0060090 msgr2=0x7f6ae0145d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6ae0060090 0x7f6ae0145d40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f6ac8002f70 tx=0x7f6ac8014000).stop 2022-07-21T01:53:30.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 -- 172.21.15.140:0/450492903 shutdown_connections 2022-07-21T01:53:30.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ac4053b00 0x7f6ac4055fc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ae00b4040 0x7f6ae0140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae00acae0 0x7f6ae01400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 --2- 172.21.15.140:0/450492903 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6ae0060090 0x7f6ae0145d40 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 -- 172.21.15.140:0/450492903 >> 172.21.15.140:0/450492903 conn(0x7f6ae0064d20 msgr2=0x7f6ae00604f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:30.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 -- 172.21.15.140:0/450492903 shutdown_connections 2022-07-21T01:53:30.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.483+0000 7f6ae639f700 1 -- 172.21.15.140:0/450492903 wait complete. 2022-07-21T01:53:30.486 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 11 and data pool 12 2022-07-21T01:53:30.497 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:53:30.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.595+0000 7f36a1c1b700 1 Processor -- start 2022-07-21T01:53:30.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.595+0000 7f36a1c1b700 1 -- start start 2022-07-21T01:53:30.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.595+0000 7f36a1c1b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b61d0 0x7f369c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.595+0000 7f36a1c1b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f369c0af120 0x7f369c067260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.595+0000 7f36a1c1b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f369c0b20f0 con 0x7f369c0b13e0 2022-07-21T01:53:30.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.595+0000 7f36a1c1b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369c13efd0 con 0x7f369c0af120 2022-07-21T01:53:30.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.595+0000 7f36a1c1b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b65b0 con 0x7f369c0b61d0 2022-07-21T01:53:30.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.595+0000 7f36a0c19700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b61d0 0x7f369c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.595+0000 7f36a141a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f369c0af120 0x7f369c067260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.595+0000 7f36a0c19700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b61d0 0x7f369c0b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:60840/0 (socket says 172.21.15.140:60840) 2022-07-21T01:53:30.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.595+0000 7f36a0c19700 1 -- 172.21.15.140:0/1546504314 learned_addr learned my addr 172.21.15.140:0/1546504314 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:30.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a0c19700 1 -- 172.21.15.140:0/1546504314 >> v2:172.21.15.152:3300/0 conn(0x7f369c0af120 msgr2=0x7f369c067260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:30.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a0c19700 1 --2- 172.21.15.140:0/1546504314 >> v2:172.21.15.152:3300/0 conn(0x7f369c0af120 0x7f369c067260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a0c19700 1 -- 172.21.15.140:0/1546504314 >> v1:172.21.15.152:6789/0 conn(0x7f369c0b13e0 legacy=0x7f369c0aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a0c19700 1 -- 172.21.15.140:0/1546504314 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c0b1ad0 con 0x7f369c0b61d0 2022-07-21T01:53:30.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a0c19700 1 --2- 172.21.15.140:0/1546504314 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b61d0 0x7f369c0b0460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3690002f70 tx=0x7f3690006e40).ready entity=mon.2 client_cookie=4b33ed90cf94f101 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:30.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f369b7fe700 1 -- 172.21.15.140:0/1546504314 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f369000bc90 con 0x7f369c0b61d0 2022-07-21T01:53:30.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f369b7fe700 1 -- 172.21.15.140:0/1546504314 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3690000e90 con 0x7f369c0b61d0 2022-07-21T01:53:30.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f369b7fe700 1 -- 172.21.15.140:0/1546504314 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36900153f0 con 0x7f369c0b61d0 2022-07-21T01:53:30.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a1c1b700 1 -- 172.21.15.140:0/1546504314 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b61d0 msgr2=0x7f369c0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a1c1b700 1 --2- 172.21.15.140:0/1546504314 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b61d0 0x7f369c0b0460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f3690002f70 tx=0x7f3690006e40).stop 2022-07-21T01:53:30.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a1c1b700 1 -- 172.21.15.140:0/1546504314 shutdown_connections 2022-07-21T01:53:30.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a1c1b700 1 --2- 172.21.15.140:0/1546504314 >> v2:172.21.15.152:3300/0 conn(0x7f369c0af120 0x7f369c067260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a1c1b700 1 --2- 172.21.15.140:0/1546504314 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b61d0 0x7f369c0b0460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a1c1b700 1 -- 172.21.15.140:0/1546504314 >> 172.21.15.140:0/1546504314 conn(0x7f369c063cf0 msgr2=0x7f369c0b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:30.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a1c1b700 1 -- 172.21.15.140:0/1546504314 shutdown_connections 2022-07-21T01:53:30.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.599+0000 7f36a1c1b700 1 -- 172.21.15.140:0/1546504314 wait complete. 2022-07-21T01:53:30.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a1c1b700 1 Processor -- start 2022-07-21T01:53:30.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a1c1b700 1 -- start start 2022-07-21T01:53:30.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a1c1b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f369c0af120 0x7f369c14db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a1c1b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b13e0 0x7f369c141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a1c1b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f369c0b61d0 0x7f369c111f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a1c1b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c110390 con 0x7f369c0af120 2022-07-21T01:53:30.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a1c1b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c110860 con 0x7f369c0b61d0 2022-07-21T01:53:30.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a1c1b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c1105c0 con 0x7f369c0b13e0 2022-07-21T01:53:30.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a0c19700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f369c0af120 0x7f369c14db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a0c19700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f369c0af120 0x7f369c14db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35884/0 (socket says 172.21.15.140:35884) 2022-07-21T01:53:30.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a0c19700 1 -- 172.21.15.140:0/3555225618 learned_addr learned my addr 172.21.15.140:0/3555225618 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:30.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f369bfff700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b13e0 0x7f369c141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a141a700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f369c0b61d0 0x7f369c111f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a0c19700 1 -- 172.21.15.140:0/3555225618 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b13e0 msgr2=0x7f369c141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a0c19700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b13e0 0x7f369c141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a0c19700 1 -- 172.21.15.140:0/3555225618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f369c0b61d0 msgr2=0x7f369c111f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a0c19700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f369c0b61d0 0x7f369c111f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a0c19700 1 -- 172.21.15.140:0/3555225618 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369000d4d0 con 0x7f369c0af120 2022-07-21T01:53:30.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a141a700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f369c0b61d0 0x7f369c111f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:30.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a0c19700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f369c0af120 0x7f369c14db90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f3690006fc0 tx=0x7f3690018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:30.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f3699ffb700 1 -- 172.21.15.140:0/3555225618 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3690015a20 con 0x7f369c0af120 2022-07-21T01:53:30.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f3699ffb700 1 -- 172.21.15.140:0/3555225618 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3690015b80 con 0x7f369c0af120 2022-07-21T01:53:30.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a1c1b700 1 -- 172.21.15.140:0/3555225618 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369c10ee80 con 0x7f369c0af120 2022-07-21T01:53:30.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f3699ffb700 1 -- 172.21.15.140:0/3555225618 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f369001e400 con 0x7f369c0af120 2022-07-21T01:53:30.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.603+0000 7f36a1c1b700 1 -- 172.21.15.140:0/3555225618 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c067010 con 0x7f369c0af120 2022-07-21T01:53:30.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.607+0000 7f3699ffb700 1 -- 172.21.15.140:0/3555225618 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f369001e560 con 0x7f369c0af120 2022-07-21T01:53:30.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.607+0000 7f36a1c1b700 1 -- 172.21.15.140:0/3555225618 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369c0b1d70 con 0x7f369c0af120 2022-07-21T01:53:30.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.607+0000 7f3699ffb700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f367c053a70 0x7f367c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.607+0000 7f369bfff700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f367c053a70 0x7f367c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.607+0000 7f3699ffb700 1 -- 172.21.15.140:0/3555225618 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f369006f9e0 con 0x7f369c0af120 2022-07-21T01:53:30.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.607+0000 7f369bfff700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f367c053a70 0x7f367c055f30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f368800d000 tx=0x7f3688007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:30.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.615+0000 7f3699ffb700 1 -- 172.21.15.140:0/3555225618 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3690046790 con 0x7f369c0af120 2022-07-21T01:53:30.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.739+0000 7f36a1c1b700 1 -- 172.21.15.140:0/3555225618 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f369c0574e0 con 0x7f369c0af120 2022-07-21T01:53:30.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.743+0000 7f3699ffb700 1 -- 172.21.15.140:0/3555225618 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16150 (secure 0 0 0) 0x7f3690049db0 con 0x7f369c0af120 2022-07-21T01:53:30.755 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:30.756 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":62,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:53:30.475817+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:53:28.457127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-07-21T01:53:29.483080+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:53:30.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.743+0000 7f36a1c1b700 1 -- 172.21.15.140:0/3555225618 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f367c053a70 msgr2=0x7f367c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.743+0000 7f36a1c1b700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f367c053a70 0x7f367c055f30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f368800d000 tx=0x7f3688007b90).stop 2022-07-21T01:53:30.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.743+0000 7f36a1c1b700 1 -- 172.21.15.140:0/3555225618 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f369c0af120 msgr2=0x7f369c14db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.743+0000 7f36a1c1b700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f369c0af120 0x7f369c14db90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f3690006fc0 tx=0x7f3690018000).stop 2022-07-21T01:53:30.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.743+0000 7f36a1c1b700 1 -- 172.21.15.140:0/3555225618 shutdown_connections 2022-07-21T01:53:30.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.743+0000 7f36a1c1b700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f367c053a70 0x7f367c055f30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.743+0000 7f36a1c1b700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f369c0b61d0 0x7f369c111f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.743+0000 7f36a1c1b700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f369c0b13e0 0x7f369c141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.743+0000 7f36a1c1b700 1 --2- 172.21.15.140:0/3555225618 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f369c0af120 0x7f369c14db90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.743+0000 7f36a1c1b700 1 -- 172.21.15.140:0/3555225618 >> 172.21.15.140:0/3555225618 conn(0x7f369c063cf0 msgr2=0x7f369c1425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:30.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.747+0000 7f36a1c1b700 1 -- 172.21.15.140:0/3555225618 shutdown_connections 2022-07-21T01:53:30.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.747+0000 7f36a1c1b700 1 -- 172.21.15.140:0/3555225618 wait complete. 2022-07-21T01:53:30.766 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:53:30.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce830e5700 1 Processor -- start 2022-07-21T01:53:30.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce830e5700 1 -- start start 2022-07-21T01:53:30.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce830e5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fce7c0b5650 0x7fce7c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce830e5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fce7c0adff0 0x7fce7c147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce830e5700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c147010 con 0x7fce7c0af8e0 2022-07-21T01:53:30.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce830e5700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c1472b0 con 0x7fce7c0b5650 2022-07-21T01:53:30.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce830e5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c146d40 con 0x7fce7c0adff0 2022-07-21T01:53:30.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce820e3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fce7c0b5650 0x7fce7c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce820e3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fce7c0b5650 0x7fce7c0b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35892/0 (socket says 172.21.15.140:35892) 2022-07-21T01:53:30.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce820e3700 1 -- 172.21.15.140:0/2293123150 learned_addr learned my addr 172.21.15.140:0/2293123150 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:30.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce818e2700 1 --2- 172.21.15.140:0/2293123150 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fce7c0adff0 0x7fce7c147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce820e3700 1 -- 172.21.15.140:0/2293123150 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fce7c0adff0 msgr2=0x7fce7c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:30.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce820e3700 1 --2- 172.21.15.140:0/2293123150 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fce7c0adff0 0x7fce7c147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce820e3700 1 -- 172.21.15.140:0/2293123150 >> v1:172.21.15.152:6789/0 conn(0x7fce7c0af8e0 legacy=0x7fce7c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.871+0000 7fce820e3700 1 -- 172.21.15.140:0/2293123150 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c147c40 con 0x7fce7c0b5650 2022-07-21T01:53:30.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce820e3700 1 --2- 172.21.15.140:0/2293123150 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fce7c0b5650 0x7fce7c0b5a30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fce740032f0 tx=0x7fce74009290).ready entity=mon.0 client_cookie=c0d59420888cc92f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:30.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce810e1700 1 -- 172.21.15.140:0/2293123150 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7400b8a0 con 0x7fce7c0b5650 2022-07-21T01:53:30.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce810e1700 1 -- 172.21.15.140:0/2293123150 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7400ba00 con 0x7fce7c0b5650 2022-07-21T01:53:30.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce810e1700 1 -- 172.21.15.140:0/2293123150 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7400bb70 con 0x7fce7c0b5650 2022-07-21T01:53:30.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce830e5700 1 -- 172.21.15.140:0/2293123150 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fce7c0b5650 msgr2=0x7fce7c0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce830e5700 1 --2- 172.21.15.140:0/2293123150 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fce7c0b5650 0x7fce7c0b5a30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fce740032f0 tx=0x7fce74009290).stop 2022-07-21T01:53:30.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce830e5700 1 -- 172.21.15.140:0/2293123150 shutdown_connections 2022-07-21T01:53:30.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce830e5700 1 --2- 172.21.15.140:0/2293123150 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fce7c0adff0 0x7fce7c147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce830e5700 1 --2- 172.21.15.140:0/2293123150 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fce7c0b5650 0x7fce7c0b5a30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce830e5700 1 -- 172.21.15.140:0/2293123150 >> 172.21.15.140:0/2293123150 conn(0x7fce7c0634e0 msgr2=0x7fce7c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:30.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce830e5700 1 -- 172.21.15.140:0/2293123150 shutdown_connections 2022-07-21T01:53:30.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.875+0000 7fce830e5700 1 -- 172.21.15.140:0/2293123150 wait complete. 2022-07-21T01:53:30.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce830e5700 1 Processor -- start 2022-07-21T01:53:30.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce830e5700 1 -- start start 2022-07-21T01:53:30.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce830e5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fce7c0adff0 0x7fce7c11df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce830e5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce7c0af8e0 0x7fce7c11e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce830e5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fce7c0b5650 0x7fce7c1562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce830e5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c11cf50 con 0x7fce7c0adff0 2022-07-21T01:53:30.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce830e5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c067a50 con 0x7fce7c0af8e0 2022-07-21T01:53:30.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce830e5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c1451e0 con 0x7fce7c0b5650 2022-07-21T01:53:30.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce820e3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fce7c0adff0 0x7fce7c11df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce820e3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fce7c0adff0 0x7fce7c11df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35898/0 (socket says 172.21.15.140:35898) 2022-07-21T01:53:30.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce820e3700 1 -- 172.21.15.140:0/2169312823 learned_addr learned my addr 172.21.15.140:0/2169312823 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:30.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce818e2700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce7c0af8e0 0x7fce7c11e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce828e4700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fce7c0b5650 0x7fce7c1562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce820e3700 1 -- 172.21.15.140:0/2169312823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fce7c0b5650 msgr2=0x7fce7c1562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce820e3700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fce7c0b5650 0x7fce7c1562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce820e3700 1 -- 172.21.15.140:0/2169312823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce7c0af8e0 msgr2=0x7fce7c11e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:30.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce820e3700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce7c0af8e0 0x7fce7c11e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:30.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce820e3700 1 -- 172.21.15.140:0/2169312823 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74009530 con 0x7fce7c0adff0 2022-07-21T01:53:30.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce820e3700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fce7c0adff0 0x7fce7c11df90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fce7c145320 tx=0x7fce74016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:30.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce6b7fe700 1 -- 172.21.15.140:0/2169312823 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7400b8a0 con 0x7fce7c0adff0 2022-07-21T01:53:30.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce6b7fe700 1 -- 172.21.15.140:0/2169312823 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce74021440 con 0x7fce7c0adff0 2022-07-21T01:53:30.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce830e5700 1 -- 172.21.15.140:0/2169312823 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c157030 con 0x7fce7c0adff0 2022-07-21T01:53:30.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce6b7fe700 1 -- 172.21.15.140:0/2169312823 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce740217a0 con 0x7fce7c0adff0 2022-07-21T01:53:30.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.879+0000 7fce830e5700 1 -- 172.21.15.140:0/2169312823 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c067610 con 0x7fce7c0adff0 2022-07-21T01:53:30.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.883+0000 7fce6b7fe700 1 -- 172.21.15.140:0/2169312823 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fce74027020 con 0x7fce7c0adff0 2022-07-21T01:53:30.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.883+0000 7fce830e5700 1 -- 172.21.15.140:0/2169312823 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c05c4a0 con 0x7fce7c0adff0 2022-07-21T01:53:30.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.883+0000 7fce6b7fe700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fce5c0539e0 0x7fce5c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:30.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.883+0000 7fce6b7fe700 1 -- 172.21.15.140:0/2169312823 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fce7406d110 con 0x7fce7c0adff0 2022-07-21T01:53:30.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.883+0000 7fce818e2700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fce5c0539e0 0x7fce5c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:30.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.883+0000 7fce818e2700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fce5c0539e0 0x7fce5c055ea0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fce6c006000 tx=0x7fce6c006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:30.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:30.887+0000 7fce6b7fe700 1 -- 172.21.15.140:0/2169312823 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fce74044030 con 0x7fce7c0adff0 2022-07-21T01:53:31.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.015+0000 7fce830e5700 1 -- 172.21.15.140:0/2169312823 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce7c0574e0 con 0x7fce7c0adff0 2022-07-21T01:53:31.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce6b7fe700 1 -- 172.21.15.140:0/2169312823 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16150 (secure 0 0 0) 0x7fce74047650 con 0x7fce7c0adff0 2022-07-21T01:53:31.015 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:31.015 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":62,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:53:30.475817+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:53:28.457127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-07-21T01:53:29.483080+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 -- 172.21.15.140:0/2169312823 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fce5c0539e0 msgr2=0x7fce5c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fce5c0539e0 0x7fce5c055ea0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fce6c006000 tx=0x7fce6c006d90).stop 2022-07-21T01:53:31.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 -- 172.21.15.140:0/2169312823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fce7c0adff0 msgr2=0x7fce7c11df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fce7c0adff0 0x7fce7c11df90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fce7c145320 tx=0x7fce74016000).stop 2022-07-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 -- 172.21.15.140:0/2169312823 shutdown_connections 2022-07-21T01:53:31.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fce5c0539e0 0x7fce5c055ea0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fce7c0b5650 0x7fce7c1562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce7c0af8e0 0x7fce7c11e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 --2- 172.21.15.140:0/2169312823 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fce7c0adff0 0x7fce7c11df90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 -- 172.21.15.140:0/2169312823 >> 172.21.15.140:0/2169312823 conn(0x7fce7c0634e0 msgr2=0x7fce7c0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 -- 172.21.15.140:0/2169312823 shutdown_connections 2022-07-21T01:53:31.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.019+0000 7fce830e5700 1 -- 172.21.15.140:0/2169312823 wait complete. 2022-07-21T01:53:31.028 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:53:31.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.135+0000 7fe55fa2c700 1 Processor -- start 2022-07-21T01:53:31.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.135+0000 7fe55fa2c700 1 -- start start 2022-07-21T01:53:31.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.135+0000 7fe55fa2c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe55809df40 0x7fe558145680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.135+0000 7fe55fa2c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5580607e0 con 0x7fe558060ee0 2022-07-21T01:53:31.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.135+0000 7fe55fa2c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe558060a80 con 0x7fe5580b4d60 2022-07-21T01:53:31.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.135+0000 7fe55fa2c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe558060d20 con 0x7fe55809df40 2022-07-21T01:53:31.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55dfc9700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe558060ee0 0x7fe55809cba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50056/0 (socket says 172.21.15.140:50056) 2022-07-21T01:53:31.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55dfc9700 1 -- 172.21.15.140:0/3676673755 learned_addr learned my addr 172.21.15.140:0/3676673755 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:31.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55d7c8700 1 --2- 172.21.15.140:0/3676673755 >> v2:172.21.15.152:3300/0 conn(0x7fe55809df40 0x7fe558145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe547fff700 1 -- 172.21.15.140:0/3676673755 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419868684 0 0) 0x7fe5580607e0 con 0x7fe558060ee0 2022-07-21T01:53:31.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe547fff700 1 -- 172.21.15.140:0/3676673755 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe540003260 con 0x7fe558060ee0 2022-07-21T01:53:31.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55d7c8700 1 -- 172.21.15.140:0/3676673755 >> v1:172.21.15.152:6789/0 conn(0x7fe5580b4d60 legacy=0x7fe558145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55d7c8700 1 -- 172.21.15.140:0/3676673755 >> v1:172.21.15.140:6789/0 conn(0x7fe558060ee0 legacy=0x7fe55809cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55d7c8700 1 -- 172.21.15.140:0/3676673755 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55809e320 con 0x7fe55809df40 2022-07-21T01:53:31.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55d7c8700 1 --2- 172.21.15.140:0/3676673755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe55809df40 0x7fe558145680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fe548002f70 tx=0x7fe548006d90).ready entity=mon.1 client_cookie=297a3074841e4d05 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:31.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe547fff700 1 -- 172.21.15.140:0/3676673755 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe54800d4a0 con 0x7fe55809df40 2022-07-21T01:53:31.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe547fff700 1 -- 172.21.15.140:0/3676673755 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe548000e90 con 0x7fe55809df40 2022-07-21T01:53:31.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe547fff700 1 -- 172.21.15.140:0/3676673755 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe54800d7f0 con 0x7fe55809df40 2022-07-21T01:53:31.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55fa2c700 1 -- 172.21.15.140:0/3676673755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe55809df40 msgr2=0x7fe558145680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55fa2c700 1 --2- 172.21.15.140:0/3676673755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe55809df40 0x7fe558145680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fe548002f70 tx=0x7fe548006d90).stop 2022-07-21T01:53:31.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55fa2c700 1 -- 172.21.15.140:0/3676673755 shutdown_connections 2022-07-21T01:53:31.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55fa2c700 1 --2- 172.21.15.140:0/3676673755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe55809df40 0x7fe558145680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55fa2c700 1 -- 172.21.15.140:0/3676673755 >> 172.21.15.140:0/3676673755 conn(0x7fe5580b5f30 msgr2=0x7fe5580b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:31.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55fa2c700 1 -- 172.21.15.140:0/3676673755 shutdown_connections 2022-07-21T01:53:31.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.139+0000 7fe55fa2c700 1 -- 172.21.15.140:0/3676673755 wait complete. 2022-07-21T01:53:31.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55fa2c700 1 Processor -- start 2022-07-21T01:53:31.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55fa2c700 1 -- start start 2022-07-21T01:53:31.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55fa2c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe558060ee0 0x7fe558155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55fa2c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe55809df40 0x7fe5580b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55fa2c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5580b4d60 0x7fe558090250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55fa2c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55808fe60 con 0x7fe55809df40 2022-07-21T01:53:31.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55fa2c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580b2730 con 0x7fe558060ee0 2022-07-21T01:53:31.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55fa2c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580b0300 con 0x7fe5580b4d60 2022-07-21T01:53:31.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55cfc7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe55809df40 0x7fe5580b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55cfc7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe55809df40 0x7fe5580b8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35914/0 (socket says 172.21.15.140:35914) 2022-07-21T01:53:31.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55cfc7700 1 -- 172.21.15.140:0/1972931515 learned_addr learned my addr 172.21.15.140:0/1972931515 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:31.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55d7c8700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe558060ee0 0x7fe558155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.143+0000 7fe55dfc9700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5580b4d60 0x7fe558090250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe55cfc7700 1 -- 172.21.15.140:0/1972931515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5580b4d60 msgr2=0x7fe558090250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe55cfc7700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5580b4d60 0x7fe558090250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe55cfc7700 1 -- 172.21.15.140:0/1972931515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe558060ee0 msgr2=0x7fe558155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe55cfc7700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe558060ee0 0x7fe558155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe55cfc7700 1 -- 172.21.15.140:0/1972931515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54800b120 con 0x7fe55809df40 2022-07-21T01:53:31.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe55d7c8700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe558060ee0 0x7fe558155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:31.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe55cfc7700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe55809df40 0x7fe5580b8a90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fe55805f350 tx=0x7fe554007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:31.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe5467fc700 1 -- 172.21.15.140:0/1972931515 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe55400ccd0 con 0x7fe55809df40 2022-07-21T01:53:31.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe5467fc700 1 -- 172.21.15.140:0/1972931515 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55400ce30 con 0x7fe55809df40 2022-07-21T01:53:31.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe55fa2c700 1 -- 172.21.15.140:0/1972931515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5580b0670 con 0x7fe55809df40 2022-07-21T01:53:31.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe5467fc700 1 -- 172.21.15.140:0/1972931515 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5540125f0 con 0x7fe55809df40 2022-07-21T01:53:31.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe55fa2c700 1 -- 172.21.15.140:0/1972931515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5580b1850 con 0x7fe55809df40 2022-07-21T01:53:31.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe55fa2c700 1 -- 172.21.15.140:0/1972931515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5580b04c0 con 0x7fe55809df40 2022-07-21T01:53:31.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe5467fc700 1 -- 172.21.15.140:0/1972931515 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe554019030 con 0x7fe55809df40 2022-07-21T01:53:31.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe5467fc700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe538053980 0x7fe538055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe5467fc700 1 -- 172.21.15.140:0/1972931515 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe55406b190 con 0x7fe55809df40 2022-07-21T01:53:31.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.147+0000 7fe55d7c8700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe538053980 0x7fe538055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.151+0000 7fe55d7c8700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe538053980 0x7fe538055e40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fe54801a000 tx=0x7fe548014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:31.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.155+0000 7fe5467fc700 1 -- 172.21.15.140:0/1972931515 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe554042010 con 0x7fe55809df40 2022-07-21T01:53:31.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.319+0000 7fe55fa2c700 1 -- 172.21.15.140:0/1972931515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe5580b2570 con 0x7fe55809df40 2022-07-21T01:53:31.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.483+0000 7fe5467fc700 1 -- 172.21.15.140:0/1972931515 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe554045630 con 0x7fe55809df40 2022-07-21T01:53:31.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.483+0000 7fe55fa2c700 1 -- 172.21.15.140:0/1972931515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe538053980 msgr2=0x7fe538055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.483+0000 7fe55fa2c700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe538053980 0x7fe538055e40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7fe54801a000 tx=0x7fe548014000).stop 2022-07-21T01:53:31.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.483+0000 7fe55fa2c700 1 -- 172.21.15.140:0/1972931515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe55809df40 msgr2=0x7fe5580b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.483+0000 7fe55fa2c700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe55809df40 0x7fe5580b8a90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fe55805f350 tx=0x7fe554007c20).stop 2022-07-21T01:53:31.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.483+0000 7fe55fa2c700 1 -- 172.21.15.140:0/1972931515 shutdown_connections 2022-07-21T01:53:31.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.483+0000 7fe55fa2c700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe538053980 0x7fe538055e40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.483+0000 7fe55fa2c700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5580b4d60 0x7fe558090250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.483+0000 7fe55fa2c700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe55809df40 0x7fe5580b8a90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.487+0000 7fe55fa2c700 1 --2- 172.21.15.140:0/1972931515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe558060ee0 0x7fe558155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.487+0000 7fe55fa2c700 1 -- 172.21.15.140:0/1972931515 >> 172.21.15.140:0/1972931515 conn(0x7fe5580b5f30 msgr2=0x7fe5580b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:31.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.487+0000 7fe55fa2c700 1 -- 172.21.15.140:0/1972931515 shutdown_connections 2022-07-21T01:53:31.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.487+0000 7fe55fa2c700 1 -- 172.21.15.140:0/1972931515 wait complete. 2022-07-21T01:53:31.500 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:53:31.594 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:53:31.597 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:53:31.601 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:53:31.605 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:53:31.609 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:53:31.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.611+0000 7fcb23fff700 1 Processor -- start 2022-07-21T01:53:31.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.611+0000 7fcb23fff700 1 -- start start 2022-07-21T01:53:31.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.611+0000 7fcb23fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb240b61d0 0x7fcb241412a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.611+0000 7fcb23fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcb24060670 0x7fcb24143e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.611+0000 7fcb23fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb240b65b0 con 0x7fcb2405fb00 2022-07-21T01:53:31.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.611+0000 7fcb23fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb240b0f10 con 0x7fcb24060670 2022-07-21T01:53:31.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.611+0000 7fcb23fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb24141790 con 0x7fcb240b61d0 2022-07-21T01:53:31.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.611+0000 7fcb1bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcb24060670 0x7fcb24143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb1bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcb24060670 0x7fcb24143e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35920/0 (socket says 172.21.15.140:35920) 2022-07-21T01:53:31.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb1bfff700 1 -- 172.21.15.140:0/1623011925 learned_addr learned my addr 172.21.15.140:0/1623011925 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:31.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb22ffd700 1 --2- 172.21.15.140:0/1623011925 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb240b61d0 0x7fcb241412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb1bfff700 1 -- 172.21.15.140:0/1623011925 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb240b61d0 msgr2=0x7fcb241412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:31.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb1bfff700 1 --2- 172.21.15.140:0/1623011925 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb240b61d0 0x7fcb241412a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb1bfff700 1 -- 172.21.15.140:0/1623011925 >> v1:172.21.15.152:6789/0 conn(0x7fcb2405fb00 legacy=0x7fcb2413feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb1bfff700 1 -- 172.21.15.140:0/1623011925 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb24140b60 con 0x7fcb24060670 2022-07-21T01:53:31.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb1bfff700 1 --2- 172.21.15.140:0/1623011925 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb24060670 0x7fcb24143e70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fcb0c002f70 tx=0x7fcb0c006eb0).ready entity=mon.0 client_cookie=c71d2a7a7b2e53a4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:31.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb227fc700 1 -- 172.21.15.140:0/1623011925 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb0c00d570 con 0x7fcb24060670 2022-07-21T01:53:31.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb227fc700 1 -- 172.21.15.140:0/1623011925 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c000e90 con 0x7fcb24060670 2022-07-21T01:53:31.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb227fc700 1 -- 172.21.15.140:0/1623011925 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb0c00d900 con 0x7fcb24060670 2022-07-21T01:53:31.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb23fff700 1 -- 172.21.15.140:0/1623011925 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb24060670 msgr2=0x7fcb24143e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb23fff700 1 --2- 172.21.15.140:0/1623011925 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb24060670 0x7fcb24143e70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fcb0c002f70 tx=0x7fcb0c006eb0).stop 2022-07-21T01:53:31.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb23fff700 1 -- 172.21.15.140:0/1623011925 shutdown_connections 2022-07-21T01:53:31.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb23fff700 1 --2- 172.21.15.140:0/1623011925 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb24060670 0x7fcb24143e70 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb23fff700 1 --2- 172.21.15.140:0/1623011925 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb240b61d0 0x7fcb241412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb23fff700 1 -- 172.21.15.140:0/1623011925 >> 172.21.15.140:0/1623011925 conn(0x7fcb24063cf0 msgr2=0x7fcb240b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:31.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb23fff700 1 -- 172.21.15.140:0/1623011925 shutdown_connections 2022-07-21T01:53:31.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb23fff700 1 -- 172.21.15.140:0/1623011925 wait complete. 2022-07-21T01:53:31.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb23fff700 1 Processor -- start 2022-07-21T01:53:31.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.615+0000 7fcb23fff700 1 -- start start 2022-07-21T01:53:31.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb23fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb2405fb00 0x7fcb24141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb23fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb24060670 0x7fcb2410f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb23fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240b61d0 0x7fcb240667f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb23fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb241112c0 con 0x7fcb24060670 2022-07-21T01:53:31.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb23fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2410fb40 con 0x7fcb240b61d0 2022-07-21T01:53:31.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb23fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2410fd60 con 0x7fcb2405fb00 2022-07-21T01:53:31.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb1bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb24060670 0x7fcb2410f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb1bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb24060670 0x7fcb2410f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35928/0 (socket says 172.21.15.140:35928) 2022-07-21T01:53:31.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb1bfff700 1 -- 172.21.15.140:0/2225844361 learned_addr learned my addr 172.21.15.140:0/2225844361 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:31.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb22ffd700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb2405fb00 0x7fcb24141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb237fe700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240b61d0 0x7fcb240667f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb1bfff700 1 -- 172.21.15.140:0/2225844361 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb2405fb00 msgr2=0x7fcb24141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb1bfff700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb2405fb00 0x7fcb24141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb1bfff700 1 -- 172.21.15.140:0/2225844361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240b61d0 msgr2=0x7fcb240667f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb1bfff700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240b61d0 0x7fcb240667f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb1bfff700 1 -- 172.21.15.140:0/2225844361 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c00b1e0 con 0x7fcb24060670 2022-07-21T01:53:31.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb237fe700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240b61d0 0x7fcb240667f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:31.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb22ffd700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb2405fb00 0x7fcb24141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:31.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb1bfff700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb24060670 0x7fcb2410f650 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fcb0c002f70 tx=0x7fcb0c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:31.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb20ff9700 1 -- 172.21.15.140:0/2225844361 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb0c011860 con 0x7fcb24060670 2022-07-21T01:53:31.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb20ff9700 1 -- 172.21.15.140:0/2225844361 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c0119c0 con 0x7fcb24060670 2022-07-21T01:53:31.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb23fff700 1 -- 172.21.15.140:0/2225844361 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb24110000 con 0x7fcb24060670 2022-07-21T01:53:31.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb20ff9700 1 -- 172.21.15.140:0/2225844361 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb0c011d20 con 0x7fcb24060670 2022-07-21T01:53:31.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb23fff700 1 -- 172.21.15.140:0/2225844361 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb24066ce0 con 0x7fcb24060670 2022-07-21T01:53:31.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.619+0000 7fcb23fff700 1 -- 172.21.15.140:0/2225844361 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb24110760 con 0x7fcb24060670 2022-07-21T01:53:31.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.623+0000 7fcb20ff9700 1 -- 172.21.15.140:0/2225844361 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcb0c025030 con 0x7fcb24060670 2022-07-21T01:53:31.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.623+0000 7fcb20ff9700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb08053a70 0x7fcb08055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.623+0000 7fcb20ff9700 1 -- 172.21.15.140:0/2225844361 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcb0c06d210 con 0x7fcb24060670 2022-07-21T01:53:31.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.623+0000 7fcb22ffd700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb08053a70 0x7fcb08055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.623+0000 7fcb22ffd700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb08053a70 0x7fcb08055f30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fcb1400b000 tx=0x7fcb14008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:31.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.627+0000 7fcb20ff9700 1 -- 172.21.15.140:0/2225844361 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcb24110760 con 0x7fcb24060670 2022-07-21T01:53:31.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.787+0000 7fcb23fff700 1 -- 172.21.15.140:0/2225844361 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb24110760 con 0x7fcb24060670 2022-07-21T01:53:31.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb20ff9700 1 -- 172.21.15.140:0/2225844361 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2137 (secure 0 0 0) 0x7fcb24110760 con 0x7fcb24060670 2022-07-21T01:53:31.788 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:31.788 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":26,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/689662045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":689662045},{"type":"v1","addr":"172.21.15.140:6835","nonce":689662045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:53:30.476186+0000","modified":"2022-07-21T01:53:31.479177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5899":{"gid":5899,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/74479549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":74479549},{"type":"v1","addr":"172.21.15.152:6833","nonce":74479549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-07-21T01:53:31.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 -- 172.21.15.140:0/2225844361 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb08053a70 msgr2=0x7fcb08055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb08053a70 0x7fcb08055f30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7fcb1400b000 tx=0x7fcb14008040).stop 2022-07-21T01:53:31.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 -- 172.21.15.140:0/2225844361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb24060670 msgr2=0x7fcb2410f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb24060670 0x7fcb2410f650 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fcb0c002f70 tx=0x7fcb0c015000).stop 2022-07-21T01:53:31.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 -- 172.21.15.140:0/2225844361 shutdown_connections 2022-07-21T01:53:31.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcb08053a70 0x7fcb08055f30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcb240b61d0 0x7fcb240667f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcb24060670 0x7fcb2410f650 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 --2- 172.21.15.140:0/2225844361 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcb2405fb00 0x7fcb24141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 -- 172.21.15.140:0/2225844361 >> 172.21.15.140:0/2225844361 conn(0x7fcb24063cf0 msgr2=0x7fcb24065830 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:31.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 -- 172.21.15.140:0/2225844361 shutdown_connections 2022-07-21T01:53:31.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.791+0000 7fcb23fff700 1 -- 172.21.15.140:0/2225844361 wait complete. 2022-07-21T01:53:31.796 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 26 2022-07-21T01:53:31.800 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:53:31.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba88420700 1 Processor -- start 2022-07-21T01:53:31.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba88420700 1 -- start start 2022-07-21T01:53:31.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba88420700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba800b1e00 0x7fba8014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba88420700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8005d480 con 0x7fba800acef0 2022-07-21T01:53:31.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba88420700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba800b21e0 con 0x7fba8005c3f0 2022-07-21T01:53:31.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba88420700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8005cf40 con 0x7fba800b1e00 2022-07-21T01:53:31.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba869bd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fba800acef0 0x7fba80146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50084/0 (socket says 172.21.15.140:50084) 2022-07-21T01:53:31.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba869bd700 1 -- 172.21.15.140:0/926158704 learned_addr learned my addr 172.21.15.140:0/926158704 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:31.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba861bc700 1 --2- 172.21.15.140:0/926158704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba800b1e00 0x7fba8014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba851ba700 1 -- 172.21.15.140:0/926158704 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130840507 0 0) 0x7fba8005d480 con 0x7fba800acef0 2022-07-21T01:53:31.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba851ba700 1 -- 172.21.15.140:0/926158704 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba68003260 con 0x7fba800acef0 2022-07-21T01:53:31.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba851ba700 1 -- 172.21.15.140:0/926158704 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655216055 0 0) 0x7fba800b21e0 con 0x7fba8005c3f0 2022-07-21T01:53:31.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba851ba700 1 -- 172.21.15.140:0/926158704 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba8005d480 con 0x7fba8005c3f0 2022-07-21T01:53:31.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba861bc700 1 -- 172.21.15.140:0/926158704 >> v1:172.21.15.152:6789/0 conn(0x7fba8005c3f0 legacy=0x7fba800ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba861bc700 1 -- 172.21.15.140:0/926158704 >> v1:172.21.15.140:6789/0 conn(0x7fba800acef0 legacy=0x7fba80146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba861bc700 1 -- 172.21.15.140:0/926158704 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8005d1e0 con 0x7fba800b1e00 2022-07-21T01:53:31.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba861bc700 1 --2- 172.21.15.140:0/926158704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba800b1e00 0x7fba8014a6d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fba70002f70 tx=0x7fba70006df0).ready entity=mon.2 client_cookie=4ee97a0448eeea58 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:31.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba851ba700 1 -- 172.21.15.140:0/926158704 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba7000d5c0 con 0x7fba800b1e00 2022-07-21T01:53:31.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.907+0000 7fba851ba700 1 -- 172.21.15.140:0/926158704 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70000e90 con 0x7fba800b1e00 2022-07-21T01:53:31.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba851ba700 1 -- 172.21.15.140:0/926158704 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba7000d910 con 0x7fba800b1e00 2022-07-21T01:53:31.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 -- 172.21.15.140:0/926158704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba800b1e00 msgr2=0x7fba8014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 --2- 172.21.15.140:0/926158704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba800b1e00 0x7fba8014a6d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fba70002f70 tx=0x7fba70006df0).stop 2022-07-21T01:53:31.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 -- 172.21.15.140:0/926158704 shutdown_connections 2022-07-21T01:53:31.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 --2- 172.21.15.140:0/926158704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba800b1e00 0x7fba8014a6d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 -- 172.21.15.140:0/926158704 >> 172.21.15.140:0/926158704 conn(0x7fba800611c0 msgr2=0x7fba80062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:31.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 -- 172.21.15.140:0/926158704 shutdown_connections 2022-07-21T01:53:31.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 -- 172.21.15.140:0/926158704 wait complete. 2022-07-21T01:53:31.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 Processor -- start 2022-07-21T01:53:31.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 -- start start 2022-07-21T01:53:31.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba800acef0 0x7fba80146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba800b1e00 0x7fba8013d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba8014a7b0 0x7fba8013d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba80149e80 con 0x7fba800b1e00 2022-07-21T01:53:31.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8014a120 con 0x7fba800acef0 2022-07-21T01:53:31.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba88420700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8014a3c0 con 0x7fba8014a7b0 2022-07-21T01:53:31.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba859bb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba800b1e00 0x7fba8013d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba859bb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba800b1e00 0x7fba8013d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35942/0 (socket says 172.21.15.140:35942) 2022-07-21T01:53:31.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba859bb700 1 -- 172.21.15.140:0/2971260931 learned_addr learned my addr 172.21.15.140:0/2971260931 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:31.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba861bc700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba800acef0 0x7fba80146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba869bd700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba8014a7b0 0x7fba8013d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba859bb700 1 -- 172.21.15.140:0/2971260931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba8014a7b0 msgr2=0x7fba8013d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.911+0000 7fba859bb700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba8014a7b0 0x7fba8013d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba859bb700 1 -- 172.21.15.140:0/2971260931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba800acef0 msgr2=0x7fba80146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:31.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba859bb700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba800acef0 0x7fba80146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:31.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba859bb700 1 -- 172.21.15.140:0/2971260931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7000b180 con 0x7fba800b1e00 2022-07-21T01:53:31.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba859bb700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba800b1e00 0x7fba8013d240 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fba78001490 tx=0x7fba78007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:31.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba6f7fe700 1 -- 172.21.15.140:0/2971260931 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba78002240 con 0x7fba800b1e00 2022-07-21T01:53:31.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba6f7fe700 1 -- 172.21.15.140:0/2971260931 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba780023a0 con 0x7fba800b1e00 2022-07-21T01:53:31.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba6f7fe700 1 -- 172.21.15.140:0/2971260931 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba78001650 con 0x7fba800b1e00 2022-07-21T01:53:31.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba88420700 1 -- 172.21.15.140:0/2971260931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba80147110 con 0x7fba800b1e00 2022-07-21T01:53:31.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba88420700 1 -- 172.21.15.140:0/2971260931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba80147f60 con 0x7fba800b1e00 2022-07-21T01:53:31.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba88420700 1 -- 172.21.15.140:0/2971260931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba80064500 con 0x7fba800b1e00 2022-07-21T01:53:31.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba6f7fe700 1 -- 172.21.15.140:0/2971260931 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fba78010030 con 0x7fba800b1e00 2022-07-21T01:53:31.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.915+0000 7fba6f7fe700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba600539e0 0x7fba60055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:31.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.919+0000 7fba6f7fe700 1 -- 172.21.15.140:0/2971260931 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fba7806bf10 con 0x7fba800b1e00 2022-07-21T01:53:31.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.919+0000 7fba861bc700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba600539e0 0x7fba60055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:31.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.919+0000 7fba861bc700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba600539e0 0x7fba60055ea0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fba7001a000 tx=0x7fba70014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:31.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:31.923+0000 7fba6f7fe700 1 -- 172.21.15.140:0/2971260931 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fba78042d90 con 0x7fba800b1e00 2022-07-21T01:53:32.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.047+0000 7fba88420700 1 -- 172.21.15.140:0/2971260931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba8014a580 con 0x7fba800b1e00 2022-07-21T01:53:32.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.047+0000 7fba6f7fe700 1 -- 172.21.15.140:0/2971260931 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16137 (secure 0 0 0) 0x7fba780463b0 con 0x7fba800b1e00 2022-07-21T01:53:32.046 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:32.046 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":63,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:53:31.479929+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:53:28.457127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-07-21T01:53:29.483080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:53:32.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 -- 172.21.15.140:0/2971260931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba600539e0 msgr2=0x7fba60055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba600539e0 0x7fba60055ea0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fba7001a000 tx=0x7fba70014000).stop 2022-07-21T01:53:32.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 -- 172.21.15.140:0/2971260931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba800b1e00 msgr2=0x7fba8013d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba800b1e00 0x7fba8013d240 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fba78001490 tx=0x7fba78007e30).stop 2022-07-21T01:53:32.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 -- 172.21.15.140:0/2971260931 shutdown_connections 2022-07-21T01:53:32.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba600539e0 0x7fba60055ea0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba8014a7b0 0x7fba8013d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba800b1e00 0x7fba8013d240 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 --2- 172.21.15.140:0/2971260931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba800acef0 0x7fba80146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 -- 172.21.15.140:0/2971260931 >> 172.21.15.140:0/2971260931 conn(0x7fba8005c3f0 msgr2=0x7fba8014b810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:32.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 -- 172.21.15.140:0/2971260931 shutdown_connections 2022-07-21T01:53:32.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.051+0000 7fba88420700 1 -- 172.21.15.140:0/2971260931 wait complete. 2022-07-21T01:53:32.064 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:53:32.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c3fff700 1 Processor -- start 2022-07-21T01:53:32.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c3fff700 1 -- start start 2022-07-21T01:53:32.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c3fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc7c40ad0e0 0x7fc7c40aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c3fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c40ae280 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c3fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c40b6be0 con 0x7fc7c40b6800 2022-07-21T01:53:32.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c3fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c40ae4b0 con 0x7fc7c40ad0e0 2022-07-21T01:53:32.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c27fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc7c40b3cb0 0x7fc7c41418f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50096/0 (socket says 172.21.15.140:50096) 2022-07-21T01:53:32.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c37fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc7c40ad0e0 0x7fc7c40aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c27fc700 1 -- 172.21.15.140:0/1937247506 learned_addr learned my addr 172.21.15.140:0/1937247506 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:32.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c1ffb700 1 -- 172.21.15.140:0/1937247506 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476652953 0 0) 0x7fc7c40ae280 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c1ffb700 1 -- 172.21.15.140:0/1937247506 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a8003260 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c37fe700 1 -- 172.21.15.140:0/1937247506 >> v1:172.21.15.152:6789/0 conn(0x7fc7c40b6800 legacy=0x7fc7c4145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c37fe700 1 -- 172.21.15.140:0/1937247506 >> v1:172.21.15.140:6789/0 conn(0x7fc7c40b3cb0 legacy=0x7fc7c41418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c1ffb700 1 -- 172.21.15.140:0/1937247506 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871930647 0 0) 0x7fc7c40b6be0 con 0x7fc7c40b6800 2022-07-21T01:53:32.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c37fe700 1 -- 172.21.15.140:0/1937247506 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c40ae750 con 0x7fc7c40ad0e0 2022-07-21T01:53:32.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c37fe700 1 --2- 172.21.15.140:0/1937247506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7c40ad0e0 0x7fc7c40aebf0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fc7bc0032f0 tx=0x7fc7bc0091d0).ready entity=mon.0 client_cookie=290aecb3f449356e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:32.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c1ffb700 1 -- 172.21.15.140:0/1937247506 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7bc00b6c0 con 0x7fc7c40ad0e0 2022-07-21T01:53:32.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c1ffb700 1 -- 172.21.15.140:0/1937247506 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc00b820 con 0x7fc7c40ad0e0 2022-07-21T01:53:32.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c1ffb700 1 -- 172.21.15.140:0/1937247506 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7bc00b990 con 0x7fc7c40ad0e0 2022-07-21T01:53:32.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c3fff700 1 -- 172.21.15.140:0/1937247506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7c40ad0e0 msgr2=0x7fc7c40aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.187+0000 7fc7c3fff700 1 --2- 172.21.15.140:0/1937247506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7c40ad0e0 0x7fc7c40aebf0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7fc7bc0032f0 tx=0x7fc7bc0091d0).stop 2022-07-21T01:53:32.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 -- 172.21.15.140:0/1937247506 shutdown_connections 2022-07-21T01:53:32.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 --2- 172.21.15.140:0/1937247506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7c40ad0e0 0x7fc7c40aebf0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 -- 172.21.15.140:0/1937247506 >> 172.21.15.140:0/1937247506 conn(0x7fc7c40b9dd0 msgr2=0x7fc7c40b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:32.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 -- 172.21.15.140:0/1937247506 shutdown_connections 2022-07-21T01:53:32.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 -- 172.21.15.140:0/1937247506 wait complete. 2022-07-21T01:53:32.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 Processor -- start 2022-07-21T01:53:32.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 -- start start 2022-07-21T01:53:32.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7c40ad0e0 0x7fc7c40af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7c40b3cb0 0x7fc7c41465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7c40b6800 0x7fc7c4155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40b19e0 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c40b1c50 con 0x7fc7c40b6800 2022-07-21T01:53:32.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c3fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c41454c0 con 0x7fc7c40ad0e0 2022-07-21T01:53:32.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c27fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7c40b3cb0 0x7fc7c41465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c27fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7c40b3cb0 0x7fc7c41465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35954/0 (socket says 172.21.15.140:35954) 2022-07-21T01:53:32.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.191+0000 7fc7c27fc700 1 -- 172.21.15.140:0/878369423 learned_addr learned my addr 172.21.15.140:0/878369423 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:32.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c2ffd700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7c40ad0e0 0x7fc7c40af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c37fe700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7c40b6800 0x7fc7c4155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c27fc700 1 -- 172.21.15.140:0/878369423 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7c40ad0e0 msgr2=0x7fc7c40af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c27fc700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7c40ad0e0 0x7fc7c40af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c27fc700 1 -- 172.21.15.140:0/878369423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7c40b6800 msgr2=0x7fc7c4155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c27fc700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7c40b6800 0x7fc7c4155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c27fc700 1 -- 172.21.15.140:0/878369423 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc009470 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c2ffd700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7c40ad0e0 0x7fc7c40af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:32.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c27fc700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7c40b3cb0 0x7fc7c41465f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fc7b8001110 tx=0x7fc7b8007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:32.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7b3fff700 1 -- 172.21.15.140:0/878369423 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7b80070d0 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7b3fff700 1 -- 172.21.15.140:0/878369423 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b8007230 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c3fff700 1 -- 172.21.15.140:0/878369423 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c4145960 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7b3fff700 1 -- 172.21.15.140:0/878369423 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7b8005610 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c3fff700 1 -- 172.21.15.140:0/878369423 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c4146d00 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c3fff700 1 -- 172.21.15.140:0/878369423 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c40b1e20 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7b3fff700 1 -- 172.21.15.140:0/878369423 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc7b8010070 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7b3fff700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc7a0053b00 0x7fc7a0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7b3fff700 1 -- 172.21.15.140:0/878369423 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc7b806b4b0 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.195+0000 7fc7c2ffd700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc7a0053b00 0x7fc7a0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.199+0000 7fc7c2ffd700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc7a0053b00 0x7fc7a0055fc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fc7b400d000 tx=0x7fc7b4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:32.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.203+0000 7fc7b3fff700 1 -- 172.21.15.140:0/878369423 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc7b80423d0 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.367+0000 7fc7c3fff700 1 -- 172.21.15.140:0/878369423 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc7c4145300 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.375+0000 7fc7b3fff700 1 -- 172.21.15.140:0/878369423 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v24) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc7b80459f0 con 0x7fc7c40b3cb0 2022-07-21T01:53:32.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 -- 172.21.15.140:0/878369423 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc7a0053b00 msgr2=0x7fc7a0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc7a0053b00 0x7fc7a0055fc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fc7b400d000 tx=0x7fc7b4009040).stop 2022-07-21T01:53:32.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 -- 172.21.15.140:0/878369423 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7c40b3cb0 msgr2=0x7fc7c41465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7c40b3cb0 0x7fc7c41465f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fc7b8001110 tx=0x7fc7b8007a60).stop 2022-07-21T01:53:32.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 -- 172.21.15.140:0/878369423 shutdown_connections 2022-07-21T01:53:32.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc7a0053b00 0x7fc7a0055fc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7c40b6800 0x7fc7c4155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7c40b3cb0 0x7fc7c41465f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 --2- 172.21.15.140:0/878369423 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7c40ad0e0 0x7fc7c40af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 -- 172.21.15.140:0/878369423 >> 172.21.15.140:0/878369423 conn(0x7fc7c40b9dd0 msgr2=0x7fc7c40b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:32.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 -- 172.21.15.140:0/878369423 shutdown_connections 2022-07-21T01:53:32.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.379+0000 7fc7c3fff700 1 -- 172.21.15.140:0/878369423 wait complete. 2022-07-21T01:53:32.381 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:53:32.389 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:53:32.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.503+0000 7fa7dffff700 1 Processor -- start 2022-07-21T01:53:32.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.503+0000 7fa7dffff700 1 -- start start 2022-07-21T01:53:32.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.503+0000 7fa7dffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa7e00b6800 0x7fa7e00aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.503+0000 7fa7dffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa7e00b3cb0 0x7fa7e0145f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.503+0000 7fa7dffff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e00ae280 con 0x7fa7e00ad0e0 2022-07-21T01:53:32.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.503+0000 7fa7dffff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e00b6be0 con 0x7fa7e00b3cb0 2022-07-21T01:53:32.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.503+0000 7fa7dffff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e00ae4b0 con 0x7fa7e00b6800 2022-07-21T01:53:32.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7de7fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa7e00b3cb0 0x7fa7e0145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7de7fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa7e00b3cb0 0x7fa7e0145f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35964/0 (socket says 172.21.15.140:35964) 2022-07-21T01:53:32.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7deffd700 1 --2- 172.21.15.140:0/2074625054 >> v2:172.21.15.152:3300/0 conn(0x7fa7e00b6800 0x7fa7e00aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7de7fc700 1 -- 172.21.15.140:0/2074625054 learned_addr learned my addr 172.21.15.140:0/2074625054 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:32.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7de7fc700 1 -- 172.21.15.140:0/2074625054 >> v2:172.21.15.152:3300/0 conn(0x7fa7e00b6800 msgr2=0x7fa7e00aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:32.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7de7fc700 1 --2- 172.21.15.140:0/2074625054 >> v2:172.21.15.152:3300/0 conn(0x7fa7e00b6800 0x7fa7e00aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7de7fc700 1 -- 172.21.15.140:0/2074625054 >> v1:172.21.15.152:6789/0 conn(0x7fa7e00ad0e0 legacy=0x7fa7e005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7ddffb700 1 -- 172.21.15.140:0/2074625054 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115284946 0 0) 0x7fa7e00ae280 con 0x7fa7e00ad0e0 2022-07-21T01:53:32.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7de7fc700 1 -- 172.21.15.140:0/2074625054 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00ae750 con 0x7fa7e00b3cb0 2022-07-21T01:53:32.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7de7fc700 1 --2- 172.21.15.140:0/2074625054 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7e00b3cb0 0x7fa7e0145f40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fa7c8002f70 tx=0x7fa7c8006eb0).ready entity=mon.0 client_cookie=c9d4df2d45bfe986 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:32.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7ddffb700 1 -- 172.21.15.140:0/2074625054 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7c800d5c0 con 0x7fa7e00b3cb0 2022-07-21T01:53:32.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7ddffb700 1 -- 172.21.15.140:0/2074625054 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c8000e90 con 0x7fa7e00b3cb0 2022-07-21T01:53:32.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7ddffb700 1 -- 172.21.15.140:0/2074625054 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7c800d910 con 0x7fa7e00b3cb0 2022-07-21T01:53:32.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7dffff700 1 -- 172.21.15.140:0/2074625054 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7e00b3cb0 msgr2=0x7fa7e0145f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7dffff700 1 --2- 172.21.15.140:0/2074625054 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7e00b3cb0 0x7fa7e0145f40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fa7c8002f70 tx=0x7fa7c8006eb0).stop 2022-07-21T01:53:32.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7dffff700 1 -- 172.21.15.140:0/2074625054 shutdown_connections 2022-07-21T01:53:32.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7dffff700 1 --2- 172.21.15.140:0/2074625054 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7e00b3cb0 0x7fa7e0145f40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7dffff700 1 --2- 172.21.15.140:0/2074625054 >> v2:172.21.15.152:3300/0 conn(0x7fa7e00b6800 0x7fa7e00aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7dffff700 1 -- 172.21.15.140:0/2074625054 >> 172.21.15.140:0/2074625054 conn(0x7fa7e00b9dd0 msgr2=0x7fa7e00b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:32.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7dffff700 1 -- 172.21.15.140:0/2074625054 shutdown_connections 2022-07-21T01:53:32.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.507+0000 7fa7dffff700 1 -- 172.21.15.140:0/2074625054 wait complete. 2022-07-21T01:53:32.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7dffff700 1 Processor -- start 2022-07-21T01:53:32.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7dffff700 1 -- start start 2022-07-21T01:53:32.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7dffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7e00ad0e0 0x7fa7e0149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7dffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7e00b3cb0 0x7fa7e0144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7dffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7e00b6800 0x7fa7e013e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7dffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00af1e0 con 0x7fa7e00b6800 2022-07-21T01:53:32.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7dffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00af480 con 0x7fa7e00ad0e0 2022-07-21T01:53:32.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7dffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0151e00 con 0x7fa7e00b3cb0 2022-07-21T01:53:32.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7df7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7e00b6800 0x7fa7e013e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7deffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7e00ad0e0 0x7fa7e0149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7de7fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7e00b3cb0 0x7fa7e0144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7deffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7e00ad0e0 0x7fa7e0149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:41706/0 (socket says 172.21.15.140:41706) 2022-07-21T01:53:32.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7deffd700 1 -- 172.21.15.140:0/1681659443 learned_addr learned my addr 172.21.15.140:0/1681659443 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:32.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7df7fe700 1 -- 172.21.15.140:0/1681659443 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7e00b3cb0 msgr2=0x7fa7e0144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7df7fe700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7e00b3cb0 0x7fa7e0144360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7df7fe700 1 -- 172.21.15.140:0/1681659443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7e00ad0e0 msgr2=0x7fa7e0149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7df7fe700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7e00ad0e0 0x7fa7e0149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7df7fe700 1 -- 172.21.15.140:0/1681659443 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c800b1e0 con 0x7fa7e00b6800 2022-07-21T01:53:32.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7df7fe700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7e00b6800 0x7fa7e013e4e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fa7d0000f40 tx=0x7fa7d0007cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:32.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.511+0000 7fa7d7fff700 1 -- 172.21.15.140:0/1681659443 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7d000f470 con 0x7fa7e00b6800 2022-07-21T01:53:32.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.515+0000 7fa7d7fff700 1 -- 172.21.15.140:0/1681659443 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d000f5d0 con 0x7fa7e00b6800 2022-07-21T01:53:32.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.515+0000 7fa7dffff700 1 -- 172.21.15.140:0/1681659443 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e0144850 con 0x7fa7e00b6800 2022-07-21T01:53:32.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.515+0000 7fa7d7fff700 1 -- 172.21.15.140:0/1681659443 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7d000f970 con 0x7fa7e00b6800 2022-07-21T01:53:32.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.515+0000 7fa7dffff700 1 -- 172.21.15.140:0/1681659443 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e0145dd0 con 0x7fa7e00b6800 2022-07-21T01:53:32.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.515+0000 7fa7d7fff700 1 -- 172.21.15.140:0/1681659443 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa7d0019030 con 0x7fa7e00b6800 2022-07-21T01:53:32.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.515+0000 7fa7d7fff700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7c4053b00 0x7fa7c4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.515+0000 7fa7d7fff700 1 -- 172.21.15.140:0/1681659443 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa7d006b3f0 con 0x7fa7e00b6800 2022-07-21T01:53:32.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.515+0000 7fa7deffd700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7c4053b00 0x7fa7c4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.515+0000 7fa7dffff700 1 -- 172.21.15.140:0/1681659443 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e01449e0 con 0x7fa7e00b6800 2022-07-21T01:53:32.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.515+0000 7fa7deffd700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7c4053b00 0x7fa7c4055fc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fa7d800b000 tx=0x7fa7d8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:32.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.523+0000 7fa7d7fff700 1 -- 172.21.15.140:0/1681659443 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa7d00422f0 con 0x7fa7e00b6800 2022-07-21T01:53:32.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.687+0000 7fa7dffff700 1 -- 172.21.15.140:0/1681659443 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa7e00b5530 con 0x7fa7e00b6800 2022-07-21T01:53:32.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.695+0000 7fa7d7fff700 1 -- 172.21.15.140:0/1681659443 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa7d0021070 con 0x7fa7e00b6800 2022-07-21T01:53:32.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 -- 172.21.15.140:0/1681659443 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7c4053b00 msgr2=0x7fa7c4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7c4053b00 0x7fa7c4055fc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fa7d800b000 tx=0x7fa7d8006df0).stop 2022-07-21T01:53:32.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 -- 172.21.15.140:0/1681659443 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7e00b6800 msgr2=0x7fa7e013e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7e00b6800 0x7fa7e013e4e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fa7d0000f40 tx=0x7fa7d0007cb0).stop 2022-07-21T01:53:32.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 -- 172.21.15.140:0/1681659443 shutdown_connections 2022-07-21T01:53:32.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7c4053b00 0x7fa7c4055fc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7e00b6800 0x7fa7e013e4e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7e00b3cb0 0x7fa7e0144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 --2- 172.21.15.140:0/1681659443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7e00ad0e0 0x7fa7e0149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 -- 172.21.15.140:0/1681659443 >> 172.21.15.140:0/1681659443 conn(0x7fa7e00b9dd0 msgr2=0x7fa7e00b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:32.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 -- 172.21.15.140:0/1681659443 shutdown_connections 2022-07-21T01:53:32.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.699+0000 7fa7dffff700 1 -- 172.21.15.140:0/1681659443 wait complete. 2022-07-21T01:53:32.702 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:53:32.709 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:53:32.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f983b352700 1 Processor -- start 2022-07-21T01:53:32.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f983b352700 1 -- start start 2022-07-21T01:53:32.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f983b352700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f98340b1dd0 0x7f9834063c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f983b352700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f98340aed00 0x7f9834143320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f983b352700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98340ae8a0 con 0x7f98340af160 2022-07-21T01:53:32.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f983b352700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98340ad9d0 con 0x7f98340aed00 2022-07-21T01:53:32.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f983b352700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98340b1310 con 0x7f98340b1dd0 2022-07-21T01:53:32.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f98390ee700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f98340b1dd0 0x7f9834063c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f98390ee700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f98340b1dd0 0x7f9834063c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:60936/0 (socket says 172.21.15.140:60936) 2022-07-21T01:53:32.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f98388ed700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f98340aed00 0x7f9834143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f98390ee700 1 -- 172.21.15.140:0/3440684372 learned_addr learned my addr 172.21.15.140:0/3440684372 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:32.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f98398ef700 1 --1- 172.21.15.140:0/3440684372 >> v1:172.21.15.152:6789/0 conn(0x7f98340af160 0x7f9834143810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:51238/0 (socket says 172.21.15.140:51238) 2022-07-21T01:53:32.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f98388ed700 1 -- 172.21.15.140:0/3440684372 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f98340b1dd0 msgr2=0x7f9834063c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:32.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f98388ed700 1 --2- 172.21.15.140:0/3440684372 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f98340b1dd0 0x7f9834063c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f98388ed700 1 -- 172.21.15.140:0/3440684372 >> v1:172.21.15.152:6789/0 conn(0x7f98340af160 legacy=0x7f9834143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f98388ed700 1 -- 172.21.15.140:0/3440684372 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340b21b0 con 0x7f98340aed00 2022-07-21T01:53:32.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f9823fff700 1 -- 172.21.15.140:0/3440684372 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319939185 0 0) 0x7f98340ae8a0 con 0x7f98340af160 2022-07-21T01:53:32.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f98388ed700 1 --2- 172.21.15.140:0/3440684372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98340aed00 0x7f9834143320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f9824002f70 tx=0x7f9824006df0).ready entity=mon.1 client_cookie=fb13bbfbace959a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:32.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f9823fff700 1 -- 172.21.15.140:0/3440684372 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f982400d570 con 0x7f98340aed00 2022-07-21T01:53:32.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f9823fff700 1 -- 172.21.15.140:0/3440684372 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9824000e90 con 0x7f98340aed00 2022-07-21T01:53:32.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.827+0000 7f9823fff700 1 -- 172.21.15.140:0/3440684372 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f982400d8c0 con 0x7f98340aed00 2022-07-21T01:53:32.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.831+0000 7f983b352700 1 -- 172.21.15.140:0/3440684372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98340aed00 msgr2=0x7f9834143320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.831+0000 7f983b352700 1 --2- 172.21.15.140:0/3440684372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98340aed00 0x7f9834143320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f9824002f70 tx=0x7f9824006df0).stop 2022-07-21T01:53:32.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.831+0000 7f983b352700 1 -- 172.21.15.140:0/3440684372 shutdown_connections 2022-07-21T01:53:32.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.831+0000 7f983b352700 1 --2- 172.21.15.140:0/3440684372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98340aed00 0x7f9834143320 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.831+0000 7f983b352700 1 --2- 172.21.15.140:0/3440684372 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f98340b1dd0 0x7f9834063c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.831+0000 7f983b352700 1 -- 172.21.15.140:0/3440684372 >> 172.21.15.140:0/3440684372 conn(0x7f98340613a0 msgr2=0x7f9834062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:32.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.831+0000 7f983b352700 1 -- 172.21.15.140:0/3440684372 shutdown_connections 2022-07-21T01:53:32.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.831+0000 7f983b352700 1 -- 172.21.15.140:0/3440684372 wait complete. 2022-07-21T01:53:32.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.831+0000 7f983b352700 1 Processor -- start 2022-07-21T01:53:32.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.831+0000 7f983b352700 1 -- start start 2022-07-21T01:53:32.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f983b352700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f98340aed00 0x7f98341420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f983b352700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98340af160 0x7f98341425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f983b352700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f98340b1dd0 0x7f9834155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f983b352700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9834141af0 con 0x7f98340aed00 2022-07-21T01:53:32.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f983b352700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983413d3a0 con 0x7f98340af160 2022-07-21T01:53:32.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f983b352700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983413d640 con 0x7f98340b1dd0 2022-07-21T01:53:32.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98390ee700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f98340aed00 0x7f98341420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98390ee700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f98340aed00 0x7f98341420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35982/0 (socket says 172.21.15.140:35982) 2022-07-21T01:53:32.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98390ee700 1 -- 172.21.15.140:0/2453276155 learned_addr learned my addr 172.21.15.140:0/2453276155 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:32.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98388ed700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98340af160 0x7f98341425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98398ef700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f98340b1dd0 0x7f9834155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98390ee700 1 -- 172.21.15.140:0/2453276155 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f98340b1dd0 msgr2=0x7f9834155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98390ee700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f98340b1dd0 0x7f9834155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98390ee700 1 -- 172.21.15.140:0/2453276155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98340af160 msgr2=0x7f98341425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:32.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98390ee700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98340af160 0x7f98341425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:32.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98390ee700 1 -- 172.21.15.140:0/2453276155 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982400b180 con 0x7f98340aed00 2022-07-21T01:53:32.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98390ee700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f98340aed00 0x7f98341420b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f982c006ec0 tx=0x7f982c009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:32.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98227fc700 1 -- 172.21.15.140:0/2453276155 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f982c00c640 con 0x7f98340aed00 2022-07-21T01:53:32.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98227fc700 1 -- 172.21.15.140:0/2453276155 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c00c7a0 con 0x7f98340aed00 2022-07-21T01:53:32.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f983b352700 1 -- 172.21.15.140:0/2453276155 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9834141770 con 0x7f98340aed00 2022-07-21T01:53:32.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f98227fc700 1 -- 172.21.15.140:0/2453276155 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f982c00cb00 con 0x7f98340aed00 2022-07-21T01:53:32.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f983b352700 1 -- 172.21.15.140:0/2453276155 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983413e430 con 0x7f98340aed00 2022-07-21T01:53:32.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.835+0000 7f983b352700 1 -- 172.21.15.140:0/2453276155 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9834141960 con 0x7f98340aed00 2022-07-21T01:53:32.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.839+0000 7f98227fc700 1 -- 172.21.15.140:0/2453276155 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f982c00cc60 con 0x7f98340aed00 2022-07-21T01:53:32.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.839+0000 7f98227fc700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98140539e0 0x7f9814055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:32.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.839+0000 7f98227fc700 1 -- 172.21.15.140:0/2453276155 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f982c06c0c0 con 0x7f98340aed00 2022-07-21T01:53:32.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.839+0000 7f98388ed700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98140539e0 0x7f9814055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:32.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.839+0000 7f98388ed700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98140539e0 0x7f9814055ea0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f982401a000 tx=0x7f9824014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:32.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:32.843+0000 7f98227fc700 1 -- 172.21.15.140:0/2453276155 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f982c042f80 con 0x7f98340aed00 2022-07-21T01:53:33.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.011+0000 7f983b352700 1 -- 172.21.15.140:0/2453276155 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f983413d800 con 0x7f98340aed00 2022-07-21T01:53:33.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.023+0000 7f98227fc700 1 -- 172.21.15.140:0/2453276155 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7f982c0465a0 con 0x7f98340aed00 2022-07-21T01:53:33.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.023+0000 7f983b352700 1 -- 172.21.15.140:0/2453276155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98140539e0 msgr2=0x7f9814055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.023+0000 7f983b352700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98140539e0 0x7f9814055ea0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f982401a000 tx=0x7f9824014000).stop 2022-07-21T01:53:33.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.023+0000 7f983b352700 1 -- 172.21.15.140:0/2453276155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f98340aed00 msgr2=0x7f98341420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.023+0000 7f983b352700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f98340aed00 0x7f98341420b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f982c006ec0 tx=0x7f982c009290).stop 2022-07-21T01:53:33.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.023+0000 7f983b352700 1 -- 172.21.15.140:0/2453276155 shutdown_connections 2022-07-21T01:53:33.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.023+0000 7f983b352700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98140539e0 0x7f9814055ea0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.023+0000 7f983b352700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f98340b1dd0 0x7f9834155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.023+0000 7f983b352700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98340af160 0x7f98341425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.023+0000 7f983b352700 1 --2- 172.21.15.140:0/2453276155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f98340aed00 0x7f98341420b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.023+0000 7f983b352700 1 -- 172.21.15.140:0/2453276155 >> 172.21.15.140:0/2453276155 conn(0x7f98340613a0 msgr2=0x7f98340b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:33.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.027+0000 7f983b352700 1 -- 172.21.15.140:0/2453276155 shutdown_connections 2022-07-21T01:53:33.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.027+0000 7f983b352700 1 -- 172.21.15.140:0/2453276155 wait complete. 2022-07-21T01:53:33.028 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:53:33.037 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:53:33.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.139+0000 7f3e1b59e700 1 Processor -- start 2022-07-21T01:53:33.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.139+0000 7f3e1b59e700 1 -- start start 2022-07-21T01:53:33.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e1c0b61d0 0x7f3e1c143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3e1c13ec50 0x7f3e1c13f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e1c0afac0 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e1c0add80 con 0x7f3e1c13ec50 2022-07-21T01:53:33.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c0b65b0 con 0x7f3e1c0b61d0 2022-07-21T01:53:33.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1ad9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3e1c13ec50 0x7f3e1c13f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1a59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e1c0b61d0 0x7f3e1c143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1ad9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3e1c13ec50 0x7f3e1c13f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35992/0 (socket says 172.21.15.140:35992) 2022-07-21T01:53:33.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1ad9d700 1 -- 172.21.15.140:0/3928910077 learned_addr learned my addr 172.21.15.140:0/3928910077 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:33.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e13fff700 1 --1- 172.21.15.140:0/3928910077 >> v1:172.21.15.140:6789/0 conn(0x7f3e1c0b18d0 0x7f3e1c1434f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50136/0 (socket says 172.21.15.140:50136) 2022-07-21T01:53:33.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1ad9d700 1 -- 172.21.15.140:0/3928910077 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e1c0b61d0 msgr2=0x7f3e1c143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:33.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1ad9d700 1 --2- 172.21.15.140:0/3928910077 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e1c0b61d0 0x7f3e1c143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1ad9d700 1 -- 172.21.15.140:0/3928910077 >> v1:172.21.15.140:6789/0 conn(0x7f3e1c0b18d0 legacy=0x7f3e1c1434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e19d9b700 1 -- 172.21.15.140:0/3928910077 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888539463 0 0) 0x7f3e1c0afac0 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1ad9d700 1 -- 172.21.15.140:0/3928910077 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c05c3e0 con 0x7f3e1c13ec50 2022-07-21T01:53:33.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1ad9d700 1 --2- 172.21.15.140:0/3928910077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e1c13ec50 0x7f3e1c13f080 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f3e140032f0 tx=0x7f3e14009290).ready entity=mon.0 client_cookie=408d7c7e8f9c00e1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:33.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e19d9b700 1 -- 172.21.15.140:0/3928910077 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e1400b8a0 con 0x7f3e1c13ec50 2022-07-21T01:53:33.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e19d9b700 1 -- 172.21.15.140:0/3928910077 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1400ba00 con 0x7f3e1c13ec50 2022-07-21T01:53:33.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e19d9b700 1 -- 172.21.15.140:0/3928910077 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e1400bb70 con 0x7f3e1c13ec50 2022-07-21T01:53:33.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 -- 172.21.15.140:0/3928910077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e1c13ec50 msgr2=0x7f3e1c13f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 --2- 172.21.15.140:0/3928910077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e1c13ec50 0x7f3e1c13f080 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f3e140032f0 tx=0x7f3e14009290).stop 2022-07-21T01:53:33.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 -- 172.21.15.140:0/3928910077 shutdown_connections 2022-07-21T01:53:33.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 --2- 172.21.15.140:0/3928910077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e1c13ec50 0x7f3e1c13f080 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 --2- 172.21.15.140:0/3928910077 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e1c0b61d0 0x7f3e1c143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 -- 172.21.15.140:0/3928910077 >> 172.21.15.140:0/3928910077 conn(0x7f3e1c063cf0 msgr2=0x7f3e1c0b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:33.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 -- 172.21.15.140:0/3928910077 shutdown_connections 2022-07-21T01:53:33.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.143+0000 7f3e1b59e700 1 -- 172.21.15.140:0/3928910077 wait complete. 2022-07-21T01:53:33.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.147+0000 7f3e1b59e700 1 Processor -- start 2022-07-21T01:53:33.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.147+0000 7f3e1b59e700 1 -- start start 2022-07-21T01:53:33.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.147+0000 7f3e1b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e1c0b18d0 0x7f3e1c066620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.147+0000 7f3e1b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e1c0b61d0 0x7f3e1c13b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.147+0000 7f3e1b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e1c13ec50 0x7f3e1c064210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.147+0000 7f3e1b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c13bd50 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.147+0000 7f3e1b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c13c2f0 con 0x7f3e1c13ec50 2022-07-21T01:53:33.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.147+0000 7f3e1b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c0603d0 con 0x7f3e1c0b61d0 2022-07-21T01:53:33.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e1c0b18d0 0x7f3e1c066620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e1c0b18d0 0x7f3e1c066620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:35994/0 (socket says 172.21.15.140:35994) 2022-07-21T01:53:33.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1a59c700 1 -- 172.21.15.140:0/2662551716 learned_addr learned my addr 172.21.15.140:0/2662551716 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:33.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1ad9d700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e1c13ec50 0x7f3e1c064210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e13fff700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e1c0b61d0 0x7f3e1c13b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1a59c700 1 -- 172.21.15.140:0/2662551716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e1c0b61d0 msgr2=0x7f3e1c13b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1a59c700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e1c0b61d0 0x7f3e1c13b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1a59c700 1 -- 172.21.15.140:0/2662551716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e1c13ec50 msgr2=0x7f3e1c064210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1a59c700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e1c13ec50 0x7f3e1c064210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1a59c700 1 -- 172.21.15.140:0/2662551716 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e14009530 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1a59c700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e1c0b18d0 0x7f3e1c066620 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f3e0c00c000 tx=0x7f3e0c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:33.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e137fe700 1 -- 172.21.15.140:0/2662551716 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e0c010780 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e137fe700 1 -- 172.21.15.140:0/2662551716 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c0108e0 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1b59e700 1 -- 172.21.15.140:0/2662551716 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1c13b3c0 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e137fe700 1 -- 172.21.15.140:0/2662551716 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e0c010c40 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1b59e700 1 -- 172.21.15.140:0/2662551716 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1c065540 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e1b59e700 1 -- 172.21.15.140:0/2662551716 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1c067120 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e137fe700 1 -- 172.21.15.140:0/2662551716 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3e0c01c460 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e137fe700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e00053b00 0x7f3e00055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.151+0000 7f3e13fff700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e00053b00 0x7f3e00055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.155+0000 7f3e137fe700 1 -- 172.21.15.140:0/2662551716 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3e0c06c290 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.155+0000 7f3e13fff700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e00053b00 0x7f3e00055fc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f3e0400d000 tx=0x7f3e04007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:33.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.159+0000 7f3e137fe700 1 -- 172.21.15.140:0/2662551716 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3e1c067120 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 -- 172.21.15.140:0/2662551716 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e1c067120 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e137fe700 1 -- 172.21.15.140:0/2662551716 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f3e1c067120 con 0x7f3e1c0b18d0 2022-07-21T01:53:33.316 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:33.316 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":26,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/689662045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":689662045},{"type":"v1","addr":"172.21.15.140:6835","nonce":689662045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:53:30.476186+0000","modified":"2022-07-21T01:53:31.479177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5899":{"gid":5899,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/74479549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":74479549},{"type":"v1","addr":"172.21.15.152:6833","nonce":74479549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-07-21T01:53:33.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 -- 172.21.15.140:0/2662551716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e00053b00 msgr2=0x7f3e00055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e00053b00 0x7f3e00055fc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f3e0400d000 tx=0x7f3e04007a00).stop 2022-07-21T01:53:33.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 -- 172.21.15.140:0/2662551716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e1c0b18d0 msgr2=0x7f3e1c066620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e1c0b18d0 0x7f3e1c066620 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f3e0c00c000 tx=0x7f3e0c006df0).stop 2022-07-21T01:53:33.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 -- 172.21.15.140:0/2662551716 shutdown_connections 2022-07-21T01:53:33.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e00053b00 0x7f3e00055fc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e1c13ec50 0x7f3e1c064210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e1c0b61d0 0x7f3e1c13b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 --2- 172.21.15.140:0/2662551716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e1c0b18d0 0x7f3e1c066620 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 -- 172.21.15.140:0/2662551716 >> 172.21.15.140:0/2662551716 conn(0x7f3e1c063cf0 msgr2=0x7f3e1c0b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:33.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 -- 172.21.15.140:0/2662551716 shutdown_connections 2022-07-21T01:53:33.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.319+0000 7f3e1b59e700 1 -- 172.21.15.140:0/2662551716 wait complete. 2022-07-21T01:53:33.322 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 26 2022-07-21T01:53:33.533 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:53:30.476186+0000', 'modified': '2022-07-21T01:53:31.479177+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5899': {'gid': 5899, 'name': 'b', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/74479549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 74479549}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 74479549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:53:33.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:53:33.535 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:53:33.652 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":5899,"want_state":"up:active","state":"up:active","rank_uptime":3.1677250899999998,"mdsmap_epoch":26,"osdmap_epoch":63,"osdmap_epoch_barrier":62,"uptime":7.9450465289999999} 2022-07-21T01:53:33.653 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 5899, "mdsmap_epoch": 26, "osdmap_epoch": 63, "osdmap_epoch_barrier": 62, "rank_uptime": 3.16772509, "state": "up:active", "uptime": 7.945046529, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:53:33.654 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:53:33.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.767+0000 7fad6e1b1700 1 Processor -- start 2022-07-21T01:53:33.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.767+0000 7fad6e1b1700 1 -- start start 2022-07-21T01:53:33.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad6e1b1700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fad680acef0 0x7fad6814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad6e1b1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6805d480 con 0x7fad6805c3f0 2022-07-21T01:53:33.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad6e1b1700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad680b21e0 con 0x7fad680b1e00 2022-07-21T01:53:33.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad6e1b1700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad6805cf40 con 0x7fad680acef0 2022-07-21T01:53:33.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad66ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fad6805c3f0 0x7fad68146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50148/0 (socket says 172.21.15.140:50148) 2022-07-21T01:53:33.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad66ffd700 1 -- 172.21.15.140:0/2926420838 learned_addr learned my addr 172.21.15.140:0/2926420838 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:33.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad67fff700 1 --2- 172.21.15.140:0/2926420838 >> v2:172.21.15.152:3300/0 conn(0x7fad680acef0 0x7fad6814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803330790 0 0) 0x7fad6805d480 con 0x7fad6805c3f0 2022-07-21T01:53:33.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad54003260 con 0x7fad6805c3f0 2022-07-21T01:53:33.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1221904467 0 0) 0x7fad54003260 con 0x7fad6805c3f0 2022-07-21T01:53:33.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 >> v2:172.21.15.152:3300/0 conn(0x7fad680acef0 msgr2=0x7fad6814a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:33.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 --2- 172.21.15.140:0/2926420838 >> v2:172.21.15.152:3300/0 conn(0x7fad680acef0 0x7fad6814a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 >> v1:172.21.15.152:6789/0 conn(0x7fad680b1e00 legacy=0x7fad680ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6805d1e0 con 0x7fad6805c3f0 2022-07-21T01:53:33.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad67fff700 1 --2- 172.21.15.140:0/2926420838 >> v2:172.21.15.152:3300/0 conn(0x7fad680acef0 0x7fad6814a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:33.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fad58001660 con 0x7fad6805c3f0 2022-07-21T01:53:33.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 >> v1:172.21.15.140:6789/0 conn(0x7fad6805c3f0 legacy=0x7fad68146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad54003750 0x7fad54003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad540040b0 0x7fad54004580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad54004dd0 0x7fad5400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6805cf40 con 0x7fad540040b0 2022-07-21T01:53:33.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad66ffd700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad540040b0 0x7fad54004580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5400bd50 con 0x7fad54003750 2022-07-21T01:53:33.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5400bf10 con 0x7fad54004dd0 2022-07-21T01:53:33.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad677fe700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad54003750 0x7fad54003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad67fff700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad54004dd0 0x7fad5400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad66ffd700 1 -- 172.21.15.140:0/2926420838 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad54004dd0 msgr2=0x7fad5400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:33.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad66ffd700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad54004dd0 0x7fad5400b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad66ffd700 1 -- 172.21.15.140:0/2926420838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad54003750 msgr2=0x7fad54003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:33.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad66ffd700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad54003750 0x7fad54003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad66ffd700 1 -- 172.21.15.140:0/2926420838 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad54003260 con 0x7fad540040b0 2022-07-21T01:53:33.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad67fff700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad54004dd0 0x7fad5400b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:53:33.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad67fff700 1 -- 172.21.15.140:0/2926420838 reap_dead start 2022-07-21T01:53:33.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad66ffd700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad540040b0 0x7fad54004580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fad58001370 tx=0x7fad680ac1a0).ready entity=mon.0 client_cookie=4d66feac1418f7b6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:33.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad58005440 con 0x7fad540040b0 2022-07-21T01:53:33.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad58008170 con 0x7fad540040b0 2022-07-21T01:53:33.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.771+0000 7fad667fc700 1 -- 172.21.15.140:0/2926420838 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad580084d0 con 0x7fad540040b0 2022-07-21T01:53:33.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.775+0000 7fad6e1b1700 1 -- 172.21.15.140:0/2926420838 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad540040b0 msgr2=0x7fad54004580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.775+0000 7fad6e1b1700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad540040b0 0x7fad54004580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fad58001370 tx=0x7fad680ac1a0).stop 2022-07-21T01:53:33.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.775+0000 7fad6e1b1700 1 -- 172.21.15.140:0/2926420838 shutdown_connections 2022-07-21T01:53:33.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.775+0000 7fad6e1b1700 1 --2- 172.21.15.140:0/2926420838 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad540040b0 0x7fad54004580 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.775+0000 7fad6e1b1700 1 -- 172.21.15.140:0/2926420838 >> 172.21.15.140:0/2926420838 conn(0x7fad680611c0 msgr2=0x7fad68062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:33.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.775+0000 7fad6e1b1700 1 -- 172.21.15.140:0/2926420838 shutdown_connections 2022-07-21T01:53:33.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.775+0000 7fad6e1b1700 1 -- 172.21.15.140:0/2926420838 wait complete. 2022-07-21T01:53:33.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.775+0000 7fad6e1b1700 1 Processor -- start 2022-07-21T01:53:33.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.775+0000 7fad6e1b1700 1 -- start start 2022-07-21T01:53:33.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad6e1b1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad6809fb80 0x7fad6809ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad6e1b1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad681573f0 0x7fad680a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad6e1b1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad6809f320 0x7fad6809e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad6e1b1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad680a0cb0 con 0x7fad6809f320 2022-07-21T01:53:33.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad6e1b1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad68063a80 con 0x7fad681573f0 2022-07-21T01:53:33.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad6e1b1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad68063f50 con 0x7fad6809fb80 2022-07-21T01:53:33.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad67fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad6809f320 0x7fad6809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad677fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad6809fb80 0x7fad6809ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad67fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad6809f320 0x7fad6809e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36018/0 (socket says 172.21.15.140:36018) 2022-07-21T01:53:33.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad66ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad681573f0 0x7fad680a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad67fff700 1 -- 172.21.15.140:0/4098329026 learned_addr learned my addr 172.21.15.140:0/4098329026 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:33.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad67fff700 1 -- 172.21.15.140:0/4098329026 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad6809fb80 msgr2=0x7fad6809ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad67fff700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad6809fb80 0x7fad6809ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad67fff700 1 -- 172.21.15.140:0/4098329026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad681573f0 msgr2=0x7fad680a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad67fff700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad681573f0 0x7fad680a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad67fff700 1 -- 172.21.15.140:0/4098329026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad68156f60 con 0x7fad6809f320 2022-07-21T01:53:33.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad67fff700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad6809f320 0x7fad6809e080 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fad68156690 tx=0x7fad60008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:33.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad64ff9700 1 -- 172.21.15.140:0/4098329026 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad60019070 con 0x7fad6809f320 2022-07-21T01:53:33.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad64ff9700 1 -- 172.21.15.140:0/4098329026 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad60014810 con 0x7fad6809f320 2022-07-21T01:53:33.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad64ff9700 1 -- 172.21.15.140:0/4098329026 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad60014b70 con 0x7fad6809f320 2022-07-21T01:53:33.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad6e1b1700 1 -- 172.21.15.140:0/4098329026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad68063cb0 con 0x7fad6809f320 2022-07-21T01:53:33.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad6e1b1700 1 -- 172.21.15.140:0/4098329026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6809e960 con 0x7fad6809f320 2022-07-21T01:53:33.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.779+0000 7fad6e1b1700 1 -- 172.21.15.140:0/4098329026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad680a0450 con 0x7fad6809f320 2022-07-21T01:53:33.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.783+0000 7fad64ff9700 1 -- 172.21.15.140:0/4098329026 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fad60012040 con 0x7fad6809f320 2022-07-21T01:53:33.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.783+0000 7fad64ff9700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fad48053b00 0x7fad48055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:33.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.783+0000 7fad64ff9700 1 -- 172.21.15.140:0/4098329026 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fad6006c1c0 con 0x7fad6809f320 2022-07-21T01:53:33.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.783+0000 7fad677fe700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fad48053b00 0x7fad48055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:33.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.783+0000 7fad677fe700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fad48053b00 0x7fad48055fc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fad5c00d000 tx=0x7fad5c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:33.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.787+0000 7fad64ff9700 1 -- 172.21.15.140:0/4098329026 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fad60042f50 con 0x7fad6809f320 2022-07-21T01:53:33.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.915+0000 7fad6e1b1700 1 -- 172.21.15.140:0/4098329026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad680b04b0 con 0x7fad6809f320 2022-07-21T01:53:33.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad64ff9700 1 -- 172.21.15.140:0/4098329026 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16137 (secure 0 0 0) 0x7fad60046570 con 0x7fad6809f320 2022-07-21T01:53:33.915 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:33.916 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":63,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:53:31.479929+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:53:28.457127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2022-07-21T01:53:29.483080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:53:33.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 -- 172.21.15.140:0/4098329026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fad48053b00 msgr2=0x7fad48055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fad48053b00 0x7fad48055fc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fad5c00d000 tx=0x7fad5c009040).stop 2022-07-21T01:53:33.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 -- 172.21.15.140:0/4098329026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad6809f320 msgr2=0x7fad6809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:33.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad6809f320 0x7fad6809e080 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fad68156690 tx=0x7fad60008040).stop 2022-07-21T01:53:33.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 -- 172.21.15.140:0/4098329026 shutdown_connections 2022-07-21T01:53:33.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fad48053b00 0x7fad48055fc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fad6809f320 0x7fad6809e080 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad681573f0 0x7fad680a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 --2- 172.21.15.140:0/4098329026 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad6809fb80 0x7fad6809ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:33.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 -- 172.21.15.140:0/4098329026 >> 172.21.15.140:0/4098329026 conn(0x7fad540040b0 msgr2=0x7fad680ada90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:33.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 -- 172.21.15.140:0/4098329026 shutdown_connections 2022-07-21T01:53:33.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:33.919+0000 7fad6e1b1700 1 -- 172.21.15.140:0/4098329026 wait complete. 2022-07-21T01:53:33.932 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:53:28.457127+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:53:29.483080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:53:33.933 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:53:34.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59b391700 1 Processor -- start 2022-07-21T01:53:34.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59b391700 1 -- start start 2022-07-21T01:53:34.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59b391700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd5940b1a00 0x7fd5940b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59b391700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd594095880 0x7fd5940ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59b391700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59405f840 con 0x7fd5940ab680 2022-07-21T01:53:34.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59b391700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5940acab0 con 0x7fd594095880 2022-07-21T01:53:34.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59b391700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd594098a80 con 0x7fd5940b1a00 2022-07-21T01:53:34.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59ab90700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd5940ab680 0x7fd59413eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50168/0 (socket says 172.21.15.140:50168) 2022-07-21T01:53:34.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59ab90700 1 -- 172.21.15.140:0/3129250542 learned_addr learned my addr 172.21.15.140:0/3129250542 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:34.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd599b8e700 1 --2- 172.21.15.140:0/3129250542 >> v2:172.21.15.140:3300/0 conn(0x7fd594095880 0x7fd5940ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59a38f700 1 --2- 172.21.15.140:0/3129250542 >> v2:172.21.15.152:3300/0 conn(0x7fd5940b1a00 0x7fd5940b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59938d700 1 -- 172.21.15.140:0/3129250542 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443079914 0 0) 0x7fd59405f840 con 0x7fd5940ab680 2022-07-21T01:53:34.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59938d700 1 -- 172.21.15.140:0/3129250542 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd574003260 con 0x7fd5940ab680 2022-07-21T01:53:34.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd599b8e700 1 -- 172.21.15.140:0/3129250542 >> v2:172.21.15.152:3300/0 conn(0x7fd5940b1a00 msgr2=0x7fd5940b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:34.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd599b8e700 1 --2- 172.21.15.140:0/3129250542 >> v2:172.21.15.152:3300/0 conn(0x7fd5940b1a00 0x7fd5940b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd599b8e700 1 -- 172.21.15.140:0/3129250542 >> v1:172.21.15.140:6789/0 conn(0x7fd5940ab680 legacy=0x7fd59413eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd599b8e700 1 -- 172.21.15.140:0/3129250542 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd594143870 con 0x7fd594095880 2022-07-21T01:53:34.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd599b8e700 1 --2- 172.21.15.140:0/3129250542 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd594095880 0x7fd5940ab190 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fd5940625b0 tx=0x7fd5840091d0).ready entity=mon.0 client_cookie=20b4fe7f128d328f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59938d700 1 -- 172.21.15.140:0/3129250542 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd58400b5a0 con 0x7fd594095880 2022-07-21T01:53:34.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59938d700 1 -- 172.21.15.140:0/3129250542 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58400b700 con 0x7fd594095880 2022-07-21T01:53:34.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59938d700 1 -- 172.21.15.140:0/3129250542 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd58400b870 con 0x7fd594095880 2022-07-21T01:53:34.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59b391700 1 -- 172.21.15.140:0/3129250542 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd594095880 msgr2=0x7fd5940ab190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.039+0000 7fd59b391700 1 --2- 172.21.15.140:0/3129250542 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd594095880 0x7fd5940ab190 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fd5940625b0 tx=0x7fd5840091d0).stop 2022-07-21T01:53:34.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 -- 172.21.15.140:0/3129250542 shutdown_connections 2022-07-21T01:53:34.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 --2- 172.21.15.140:0/3129250542 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd594095880 0x7fd5940ab190 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 --2- 172.21.15.140:0/3129250542 >> v2:172.21.15.152:3300/0 conn(0x7fd5940b1a00 0x7fd5940b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 -- 172.21.15.140:0/3129250542 >> 172.21.15.140:0/3129250542 conn(0x7fd5940b2af0 msgr2=0x7fd5940b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:34.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 -- 172.21.15.140:0/3129250542 shutdown_connections 2022-07-21T01:53:34.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 -- 172.21.15.140:0/3129250542 wait complete. 2022-07-21T01:53:34.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 Processor -- start 2022-07-21T01:53:34.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 -- start start 2022-07-21T01:53:34.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd594095880 0x7fd5941397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5940ab680 0x7fd594096650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5940b1a00 0x7fd59409b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59413a310 con 0x7fd5940b1a00 2022-07-21T01:53:34.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940b0020 con 0x7fd594095880 2022-07-21T01:53:34.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59b391700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940b04c0 con 0x7fd5940ab680 2022-07-21T01:53:34.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59ab90700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5940b1a00 0x7fd59409b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59a38f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd594095880 0x7fd5941397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59a38f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd594095880 0x7fd5941397b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:41768/0 (socket says 172.21.15.140:41768) 2022-07-21T01:53:34.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59a38f700 1 -- 172.21.15.140:0/364503808 learned_addr learned my addr 172.21.15.140:0/364503808 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:34.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd599b8e700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5940ab680 0x7fd594096650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59ab90700 1 -- 172.21.15.140:0/364503808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5940ab680 msgr2=0x7fd594096650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59ab90700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5940ab680 0x7fd594096650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59ab90700 1 -- 172.21.15.140:0/364503808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd594095880 msgr2=0x7fd5941397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59ab90700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd594095880 0x7fd5941397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59ab90700 1 -- 172.21.15.140:0/364503808 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd584009470 con 0x7fd5940b1a00 2022-07-21T01:53:34.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.043+0000 7fd59a38f700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd594095880 0x7fd5941397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:34.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd59ab90700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5940b1a00 0x7fd59409b840 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fd57c001110 tx=0x7fd57c0079c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd5937fe700 1 -- 172.21.15.140:0/364503808 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd57c00cdb0 con 0x7fd5940b1a00 2022-07-21T01:53:34.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd5937fe700 1 -- 172.21.15.140:0/364503808 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c0070d0 con 0x7fd5940b1a00 2022-07-21T01:53:34.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd59b391700 1 -- 172.21.15.140:0/364503808 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd594139f90 con 0x7fd5940b1a00 2022-07-21T01:53:34.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd5937fe700 1 -- 172.21.15.140:0/364503808 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd57c007430 con 0x7fd5940b1a00 2022-07-21T01:53:34.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd59b391700 1 -- 172.21.15.140:0/364503808 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd594140270 con 0x7fd5940b1a00 2022-07-21T01:53:34.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd59b391700 1 -- 172.21.15.140:0/364503808 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59409c070 con 0x7fd5940b1a00 2022-07-21T01:53:34.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd5937fe700 1 -- 172.21.15.140:0/364503808 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd57c01a060 con 0x7fd5940b1a00 2022-07-21T01:53:34.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd5937fe700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd5780539e0 0x7fd578055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd59a38f700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd5780539e0 0x7fd578055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd5937fe700 1 -- 172.21.15.140:0/364503808 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd57c06b240 con 0x7fd5940b1a00 2022-07-21T01:53:34.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.047+0000 7fd59a38f700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd5780539e0 0x7fd578055ea0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fd58c006000 tx=0x7fd58c006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.051+0000 7fd5937fe700 1 -- 172.21.15.140:0/364503808 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd57c042050 con 0x7fd5940b1a00 2022-07-21T01:53:34.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.175+0000 7fd59b391700 1 -- 172.21.15.140:0/364503808 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd59413a510 con 0x7fd5940b1a00 2022-07-21T01:53:34.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd5937fe700 1 -- 172.21.15.140:0/364503808 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v63) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd57c018020 con 0x7fd5940b1a00 2022-07-21T01:53:34.182 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:53:34.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 -- 172.21.15.140:0/364503808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd5780539e0 msgr2=0x7fd578055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd5780539e0 0x7fd578055ea0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fd58c006000 tx=0x7fd58c006d90).stop 2022-07-21T01:53:34.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 -- 172.21.15.140:0/364503808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5940b1a00 msgr2=0x7fd59409b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5940b1a00 0x7fd59409b840 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fd57c001110 tx=0x7fd57c0079c0).stop 2022-07-21T01:53:34.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 -- 172.21.15.140:0/364503808 shutdown_connections 2022-07-21T01:53:34.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd5780539e0 0x7fd578055ea0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5940b1a00 0x7fd59409b840 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5940ab680 0x7fd594096650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 --2- 172.21.15.140:0/364503808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd594095880 0x7fd5941397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 -- 172.21.15.140:0/364503808 >> 172.21.15.140:0/364503808 conn(0x7fd5940b2af0 msgr2=0x7fd5940b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:34.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 -- 172.21.15.140:0/364503808 shutdown_connections 2022-07-21T01:53:34.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.179+0000 7fd59b391700 1 -- 172.21.15.140:0/364503808 wait complete. 2022-07-21T01:53:34.191 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:53:34.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.295+0000 7fc4fe3f0700 1 Processor -- start 2022-07-21T01:53:34.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.295+0000 7fc4fe3f0700 1 -- start start 2022-07-21T01:53:34.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc4f805d0e0 0x7fc4f8146920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc4f80b5960 0x7fc4f8142b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f805d4c0 con 0x7fc4f8143060 2022-07-21T01:53:34.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f81461e0 con 0x7fc4f80b5960 2022-07-21T01:53:34.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f8146480 con 0x7fc4f805d0e0 2022-07-21T01:53:34.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f77fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc4f80b5960 0x7fc4f8142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fc98d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc4f8143060 0x7fc4f809b6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50182/0 (socket says 172.21.15.140:50182) 2022-07-21T01:53:34.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f77fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc4f80b5960 0x7fc4f8142b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36036/0 (socket says 172.21.15.140:36036) 2022-07-21T01:53:34.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f77fe700 1 -- 172.21.15.140:0/2305467183 learned_addr learned my addr 172.21.15.140:0/2305467183 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:34.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f7fff700 1 --2- 172.21.15.140:0/2305467183 >> v2:172.21.15.152:3300/0 conn(0x7fc4f805d0e0 0x7fc4f8146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f77fe700 1 -- 172.21.15.140:0/2305467183 >> v2:172.21.15.152:3300/0 conn(0x7fc4f805d0e0 msgr2=0x7fc4f8146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:34.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f77fe700 1 --2- 172.21.15.140:0/2305467183 >> v2:172.21.15.152:3300/0 conn(0x7fc4f805d0e0 0x7fc4f8146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f77fe700 1 -- 172.21.15.140:0/2305467183 >> v1:172.21.15.140:6789/0 conn(0x7fc4f8143060 legacy=0x7fc4f809b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f6ffd700 1 -- 172.21.15.140:0/2305467183 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51014484 0 0) 0x7fc4f805d4c0 con 0x7fc4f8143060 2022-07-21T01:53:34.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f77fe700 1 -- 172.21.15.140:0/2305467183 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f80b01d0 con 0x7fc4f80b5960 2022-07-21T01:53:34.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f77fe700 1 --2- 172.21.15.140:0/2305467183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4f80b5960 0x7fc4f8142b70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fc4e0002f70 tx=0x7fc4e0006eb0).ready entity=mon.0 client_cookie=f62879d1de0d7b56 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f6ffd700 1 -- 172.21.15.140:0/2305467183 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4e000d610 con 0x7fc4f80b5960 2022-07-21T01:53:34.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f6ffd700 1 -- 172.21.15.140:0/2305467183 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e0000e90 con 0x7fc4f80b5960 2022-07-21T01:53:34.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4f6ffd700 1 -- 172.21.15.140:0/2305467183 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4e000d960 con 0x7fc4f80b5960 2022-07-21T01:53:34.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/2305467183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4f80b5960 msgr2=0x7fc4f8142b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 --2- 172.21.15.140:0/2305467183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4f80b5960 0x7fc4f8142b70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fc4e0002f70 tx=0x7fc4e0006eb0).stop 2022-07-21T01:53:34.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/2305467183 shutdown_connections 2022-07-21T01:53:34.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 --2- 172.21.15.140:0/2305467183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4f80b5960 0x7fc4f8142b70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 --2- 172.21.15.140:0/2305467183 >> v2:172.21.15.152:3300/0 conn(0x7fc4f805d0e0 0x7fc4f8146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/2305467183 >> 172.21.15.140:0/2305467183 conn(0x7fc4f80b6a20 msgr2=0x7fc4f8066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:34.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/2305467183 shutdown_connections 2022-07-21T01:53:34.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/2305467183 wait complete. 2022-07-21T01:53:34.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 Processor -- start 2022-07-21T01:53:34.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.299+0000 7fc4fe3f0700 1 -- start start 2022-07-21T01:53:34.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4fe3f0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4f805d0e0 0x7fc4f8142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4fe3f0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4f80b5960 0x7fc4f8149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4fe3f0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4f8143060 0x7fc4f8147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4fe3f0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80b3350 con 0x7fc4f805d0e0 2022-07-21T01:53:34.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4fe3f0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f809ea60 con 0x7fc4f80b5960 2022-07-21T01:53:34.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4fe3f0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80b3540 con 0x7fc4f8143060 2022-07-21T01:53:34.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4f805d0e0 0x7fc4f8142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4f805d0e0 0x7fc4f8142920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36042/0 (socket says 172.21.15.140:36042) 2022-07-21T01:53:34.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f7fff700 1 -- 172.21.15.140:0/663856106 learned_addr learned my addr 172.21.15.140:0/663856106 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:34.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4fc98d700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4f8143060 0x7fc4f8147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f77fe700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4f80b5960 0x7fc4f8149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f7fff700 1 -- 172.21.15.140:0/663856106 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4f8143060 msgr2=0x7fc4f8147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f7fff700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4f8143060 0x7fc4f8147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f7fff700 1 -- 172.21.15.140:0/663856106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4f80b5960 msgr2=0x7fc4f8149630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f7fff700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4f80b5960 0x7fc4f8149630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f7fff700 1 -- 172.21.15.140:0/663856106 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e000b1e0 con 0x7fc4f805d0e0 2022-07-21T01:53:34.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f7fff700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4f805d0e0 0x7fc4f8142920 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fc4e80030b0 tx=0x7fc4e8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f57fa700 1 -- 172.21.15.140:0/663856106 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4e8011780 con 0x7fc4f805d0e0 2022-07-21T01:53:34.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f57fa700 1 -- 172.21.15.140:0/663856106 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e80118e0 con 0x7fc4f805d0e0 2022-07-21T01:53:34.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/663856106 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f8148300 con 0x7fc4f805d0e0 2022-07-21T01:53:34.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4f57fa700 1 -- 172.21.15.140:0/663856106 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc4e8011c40 con 0x7fc4f805d0e0 2022-07-21T01:53:34.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/663856106 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f8147900 con 0x7fc4f805d0e0 2022-07-21T01:53:34.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.303+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/663856106 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f81484e0 con 0x7fc4f805d0e0 2022-07-21T01:53:34.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.307+0000 7fc4f57fa700 1 -- 172.21.15.140:0/663856106 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc4e8012420 con 0x7fc4f805d0e0 2022-07-21T01:53:34.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.307+0000 7fc4f57fa700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc4dc0539e0 0x7fc4dc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.307+0000 7fc4f77fe700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc4dc0539e0 0x7fc4dc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.307+0000 7fc4f57fa700 1 -- 172.21.15.140:0/663856106 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc4e806bde0 con 0x7fc4f805d0e0 2022-07-21T01:53:34.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.307+0000 7fc4f77fe700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc4dc0539e0 0x7fc4dc055ea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fc4e001a000 tx=0x7fc4e0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.315+0000 7fc4f57fa700 1 -- 172.21.15.140:0/663856106 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc4f81484e0 con 0x7fc4f805d0e0 2022-07-21T01:53:34.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.439+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/663856106 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc4f81484e0 con 0x7fc4f805d0e0 2022-07-21T01:53:34.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4f57fa700 1 -- 172.21.15.140:0/663856106 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v63) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc4f81484e0 con 0x7fc4f805d0e0 2022-07-21T01:53:34.441 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:53:34.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/663856106 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc4dc0539e0 msgr2=0x7fc4dc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc4dc0539e0 0x7fc4dc055ea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fc4e001a000 tx=0x7fc4e0015000).stop 2022-07-21T01:53:34.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/663856106 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4f805d0e0 msgr2=0x7fc4f8142920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4f805d0e0 0x7fc4f8142920 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7fc4e80030b0 tx=0x7fc4e8008040).stop 2022-07-21T01:53:34.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/663856106 shutdown_connections 2022-07-21T01:53:34.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc4dc0539e0 0x7fc4dc055ea0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4f8143060 0x7fc4f8147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4f80b5960 0x7fc4f8149630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 --2- 172.21.15.140:0/663856106 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4f805d0e0 0x7fc4f8142920 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/663856106 >> 172.21.15.140:0/663856106 conn(0x7fc4f80b6a20 msgr2=0x7fc4f80b7730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:34.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/663856106 shutdown_connections 2022-07-21T01:53:34.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.443+0000 7fc4fe3f0700 1 -- 172.21.15.140:0/663856106 wait complete. 2022-07-21T01:53:34.451 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:53:34.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f468a0700 1 Processor -- start 2022-07-21T01:53:34.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f468a0700 1 -- start start 2022-07-21T01:53:34.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f468a0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f400b59f0 0x7f9f401473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f468a0700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4005cc70 con 0x7f9f400b1400 2022-07-21T01:53:34.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f468a0700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f400b5dd0 con 0x7f9f400af940 2022-07-21T01:53:34.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f468a0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400b1070 con 0x7f9f400b59f0 2022-07-21T01:53:34.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4609f700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f9f400b1400 0x7f9f4014aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50200/0 (socket says 172.21.15.140:50200) 2022-07-21T01:53:34.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4609f700 1 -- 172.21.15.140:0/3597056766 learned_addr learned my addr 172.21.15.140:0/3597056766 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:34.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4589e700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f400b59f0 0x7f9f401473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483314905 0 0) 0x7f9f4005cc70 con 0x7f9f400b1400 2022-07-21T01:53:34.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f28003260 con 0x7f9f400b1400 2022-07-21T01:53:34.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256514346 0 0) 0x7f9f400b5dd0 con 0x7f9f400af940 2022-07-21T01:53:34.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f4005cc70 con 0x7f9f400af940 2022-07-21T01:53:34.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3142342932 0 0) 0x7f9f28003260 con 0x7f9f400b1400 2022-07-21T01:53:34.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f400b59f0 msgr2=0x7f9f401473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:34.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f400b59f0 0x7f9f401473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 >> v1:172.21.15.152:6789/0 conn(0x7f9f400af940 legacy=0x7f9f4014a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4005c760 con 0x7f9f400b1400 2022-07-21T01:53:34.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f9f38001a00 con 0x7f9f400b1400 2022-07-21T01:53:34.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 >> v1:172.21.15.140:6789/0 conn(0x7f9f400b1400 legacy=0x7f9f4014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f400b59f0 0x7f9f28003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f28003e20 0x7f9f28004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f280049f0 0x7f9f28004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400b1070 con 0x7f9f28003e20 2022-07-21T01:53:34.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2800b800 con 0x7f9f280049f0 2022-07-21T01:53:34.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2800b9c0 con 0x7f9f400b59f0 2022-07-21T01:53:34.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4509d700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f28003e20 0x7f9f28004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4589e700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f400b59f0 0x7f9f28003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.567+0000 7f9f4609f700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f280049f0 0x7f9f28004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 -- 172.21.15.140:0/3597056766 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f400b59f0 msgr2=0x7f9f28003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:34.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f400b59f0 0x7f9f28003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 -- 172.21.15.140:0/3597056766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f280049f0 msgr2=0x7f9f28004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:34.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f280049f0 0x7f9f28004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 -- 172.21.15.140:0/3597056766 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f28003260 con 0x7f9f28003e20 2022-07-21T01:53:34.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f28003e20 0x7f9f28004500 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f9f30001110 tx=0x7f9f30007a00).ready entity=mon.0 client_cookie=adbfe588ebf03404 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f30007d10 con 0x7f9f28003e20 2022-07-21T01:53:34.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f30007030 con 0x7f9f28003e20 2022-07-21T01:53:34.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4489c700 1 -- 172.21.15.140:0/3597056766 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f30007390 con 0x7f9f28003e20 2022-07-21T01:53:34.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 -- 172.21.15.140:0/3597056766 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f28003e20 msgr2=0x7f9f28004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 --2- 172.21.15.140:0/3597056766 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f28003e20 0x7f9f28004500 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f9f30001110 tx=0x7f9f30007a00).stop 2022-07-21T01:53:34.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4609f700 1 -- 172.21.15.140:0/3597056766 reap_dead start 2022-07-21T01:53:34.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 -- 172.21.15.140:0/3597056766 shutdown_connections 2022-07-21T01:53:34.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 -- 172.21.15.140:0/3597056766 >> 172.21.15.140:0/3597056766 conn(0x7f9f40063480 msgr2=0x7f9f400b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:34.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 -- 172.21.15.140:0/3597056766 shutdown_connections 2022-07-21T01:53:34.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 -- 172.21.15.140:0/3597056766 wait complete. 2022-07-21T01:53:34.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 Processor -- start 2022-07-21T01:53:34.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 -- start start 2022-07-21T01:53:34.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f28003e20 0x7f9f40159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f400673d0 0x7f9f40159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f40111070 0x7f9f40111520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f40067190 con 0x7f9f400673d0 2022-07-21T01:53:34.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f40111e00 con 0x7f9f28003e20 2022-07-21T01:53:34.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f468a0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f401120a0 con 0x7f9f40111070 2022-07-21T01:53:34.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f400673d0 0x7f9f40159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f400673d0 0x7f9f40159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36064/0 (socket says 172.21.15.140:36064) 2022-07-21T01:53:34.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 -- 172.21.15.140:0/677736601 learned_addr learned my addr 172.21.15.140:0/677736601 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:34.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4609f700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f40111070 0x7f9f40111520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4589e700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f28003e20 0x7f9f40159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 -- 172.21.15.140:0/677736601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f40111070 msgr2=0x7f9f40111520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f40111070 0x7f9f40111520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 -- 172.21.15.140:0/677736601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f28003e20 msgr2=0x7f9f40159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f28003e20 0x7f9f40159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 -- 172.21.15.140:0/677736601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f400664f0 con 0x7f9f400673d0 2022-07-21T01:53:34.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4609f700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f40111070 0x7f9f40111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:34.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4589e700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f28003e20 0x7f9f40159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:34.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.571+0000 7f9f4509d700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f400673d0 0x7f9f40159f20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f9f30001110 tx=0x7f9f30016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.575+0000 7f9f2effd700 1 -- 172.21.15.140:0/677736601 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f3001f620 con 0x7f9f400673d0 2022-07-21T01:53:34.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.575+0000 7f9f2effd700 1 -- 172.21.15.140:0/677736601 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3001f780 con 0x7f9f400673d0 2022-07-21T01:53:34.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.575+0000 7f9f468a0700 1 -- 172.21.15.140:0/677736601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f40111a10 con 0x7f9f400673d0 2022-07-21T01:53:34.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.575+0000 7f9f2effd700 1 -- 172.21.15.140:0/677736601 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f3001fae0 con 0x7f9f400673d0 2022-07-21T01:53:34.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.575+0000 7f9f468a0700 1 -- 172.21.15.140:0/677736601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4015a960 con 0x7f9f400673d0 2022-07-21T01:53:34.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.575+0000 7f9f2effd700 1 -- 172.21.15.140:0/677736601 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9f30025020 con 0x7f9f400673d0 2022-07-21T01:53:34.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.575+0000 7f9f468a0700 1 -- 172.21.15.140:0/677736601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4005c9d0 con 0x7f9f400673d0 2022-07-21T01:53:34.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.575+0000 7f9f2effd700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9f20053aa0 0x7f9f20055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.575+0000 7f9f2effd700 1 -- 172.21.15.140:0/677736601 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9f3006d0e0 con 0x7f9f400673d0 2022-07-21T01:53:34.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.575+0000 7f9f4589e700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9f20053aa0 0x7f9f20055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.579+0000 7f9f4589e700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9f20053aa0 0x7f9f20055f60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f9f3c00b000 tx=0x7f9f3c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.579+0000 7f9f2effd700 1 -- 172.21.15.140:0/677736601 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9f30044000 con 0x7f9f400673d0 2022-07-21T01:53:34.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 -- 172.21.15.140:0/677736601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f4005c9d0 con 0x7f9f400673d0 2022-07-21T01:53:34.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f2effd700 1 -- 172.21.15.140:0/677736601 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9f30047620 con 0x7f9f400673d0 2022-07-21T01:53:34.704 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:53:34.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 -- 172.21.15.140:0/677736601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9f20053aa0 msgr2=0x7f9f20055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9f20053aa0 0x7f9f20055f60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f9f3c00b000 tx=0x7f9f3c008040).stop 2022-07-21T01:53:34.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 -- 172.21.15.140:0/677736601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f400673d0 msgr2=0x7f9f40159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f400673d0 0x7f9f40159f20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f9f30001110 tx=0x7f9f30016000).stop 2022-07-21T01:53:34.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 -- 172.21.15.140:0/677736601 shutdown_connections 2022-07-21T01:53:34.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9f20053aa0 0x7f9f20055f60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9f40111070 0x7f9f40111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9f400673d0 0x7f9f40159f20 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 --2- 172.21.15.140:0/677736601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f28003e20 0x7f9f40159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 -- 172.21.15.140:0/677736601 >> 172.21.15.140:0/677736601 conn(0x7f9f40063480 msgr2=0x7f9f400b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:34.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.707+0000 7f9f468a0700 1 -- 172.21.15.140:0/677736601 shutdown_connections 2022-07-21T01:53:34.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.711+0000 7f9f468a0700 1 -- 172.21.15.140:0/677736601 wait complete. 2022-07-21T01:53:34.717 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:53:34.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d66187700 1 Processor -- start 2022-07-21T01:53:34.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d66187700 1 -- start start 2022-07-21T01:53:34.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d66187700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5d6005d0c0 0x7f5d600b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d66187700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d600adf70 con 0x7f5d600af920 2022-07-21T01:53:34.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d66187700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d600b5db0 con 0x7f5d600b59d0 2022-07-21T01:53:34.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d66187700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d60146300 con 0x7f5d6005d0c0 2022-07-21T01:53:34.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d64984700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f5d600af920 0x7f5d60141900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50216/0 (socket says 172.21.15.140:50216) 2022-07-21T01:53:34.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d64984700 1 -- 172.21.15.140:0/3377431339 learned_addr learned my addr 172.21.15.140:0/3377431339 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:34.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d65986700 1 --2- 172.21.15.140:0/3377431339 >> v2:172.21.15.152:3300/0 conn(0x7f5d6005d0c0 0x7f5d600b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d5ffff700 1 -- 172.21.15.140:0/3377431339 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434868428 0 0) 0x7f5d600adf70 con 0x7f5d600af920 2022-07-21T01:53:34.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d5ffff700 1 -- 172.21.15.140:0/3377431339 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d48003260 con 0x7f5d600af920 2022-07-21T01:53:34.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d65986700 1 -- 172.21.15.140:0/3377431339 >> v1:172.21.15.152:6789/0 conn(0x7f5d600b59d0 legacy=0x7f5d600ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d65986700 1 -- 172.21.15.140:0/3377431339 >> v1:172.21.15.140:6789/0 conn(0x7f5d600af920 legacy=0x7f5d60141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d5ffff700 1 -- 172.21.15.140:0/3377431339 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3791745978 0 0) 0x7f5d48003260 con 0x7f5d600af920 2022-07-21T01:53:34.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d65986700 1 -- 172.21.15.140:0/3377431339 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d60146060 con 0x7f5d6005d0c0 2022-07-21T01:53:34.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d65986700 1 --2- 172.21.15.140:0/3377431339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d6005d0c0 0x7f5d600b0040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f5d4c001b80 tx=0x7f5d58006df0).ready entity=mon.1 client_cookie=e0d98ebe52f235b2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d5ffff700 1 -- 172.21.15.140:0/3377431339 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d4c0019e0 con 0x7f5d6005d0c0 2022-07-21T01:53:34.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d5ffff700 1 -- 172.21.15.140:0/3377431339 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58000e90 con 0x7f5d6005d0c0 2022-07-21T01:53:34.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.823+0000 7f5d5ffff700 1 -- 172.21.15.140:0/3377431339 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d5800d4f0 con 0x7f5d6005d0c0 2022-07-21T01:53:34.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 -- 172.21.15.140:0/3377431339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d6005d0c0 msgr2=0x7f5d600b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 --2- 172.21.15.140:0/3377431339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d6005d0c0 0x7f5d600b0040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f5d4c001b80 tx=0x7f5d58006df0).stop 2022-07-21T01:53:34.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 -- 172.21.15.140:0/3377431339 shutdown_connections 2022-07-21T01:53:34.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 --2- 172.21.15.140:0/3377431339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d6005d0c0 0x7f5d600b0040 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 -- 172.21.15.140:0/3377431339 >> 172.21.15.140:0/3377431339 conn(0x7f5d600634c0 msgr2=0x7f5d600b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:34.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 -- 172.21.15.140:0/3377431339 shutdown_connections 2022-07-21T01:53:34.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 -- 172.21.15.140:0/3377431339 wait complete. 2022-07-21T01:53:34.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 Processor -- start 2022-07-21T01:53:34.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 -- start start 2022-07-21T01:53:34.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d6005d0c0 0x7f5d60065df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d600af920 0x7f5d600b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d600b59d0 0x7f5d60144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60144540 con 0x7f5d600af920 2022-07-21T01:53:34.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60065410 con 0x7f5d6005d0c0 2022-07-21T01:53:34.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d66187700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600656b0 con 0x7f5d600b59d0 2022-07-21T01:53:34.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d64984700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d600af920 0x7f5d600b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d64984700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d600af920 0x7f5d600b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36078/0 (socket says 172.21.15.140:36078) 2022-07-21T01:53:34.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d64984700 1 -- 172.21.15.140:0/3308508381 learned_addr learned my addr 172.21.15.140:0/3308508381 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:34.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d65185700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d6005d0c0 0x7f5d60065df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d65986700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d600b59d0 0x7f5d60144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d64984700 1 -- 172.21.15.140:0/3308508381 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d600b59d0 msgr2=0x7f5d60144050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d64984700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d600b59d0 0x7f5d60144050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d64984700 1 -- 172.21.15.140:0/3308508381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d6005d0c0 msgr2=0x7f5d60065df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d64984700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d6005d0c0 0x7f5d60065df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d64984700 1 -- 172.21.15.140:0/3308508381 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5800b180 con 0x7f5d600af920 2022-07-21T01:53:34.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d65185700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d6005d0c0 0x7f5d60065df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:34.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.827+0000 7f5d64984700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d600af920 0x7f5d600b4180 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f5d4c001c40 tx=0x7f5d4c002100).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.831+0000 7f5d5e7fc700 1 -- 172.21.15.140:0/3308508381 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d4c005ba0 con 0x7f5d600af920 2022-07-21T01:53:34.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.831+0000 7f5d5e7fc700 1 -- 172.21.15.140:0/3308508381 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c005d00 con 0x7f5d600af920 2022-07-21T01:53:34.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.831+0000 7f5d66187700 1 -- 172.21.15.140:0/3308508381 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d60065950 con 0x7f5d600af920 2022-07-21T01:53:34.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.831+0000 7f5d5e7fc700 1 -- 172.21.15.140:0/3308508381 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d4c007300 con 0x7f5d600af920 2022-07-21T01:53:34.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.831+0000 7f5d66187700 1 -- 172.21.15.140:0/3308508381 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6013ffe0 con 0x7f5d600af920 2022-07-21T01:53:34.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.831+0000 7f5d66187700 1 -- 172.21.15.140:0/3308508381 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d600af2d0 con 0x7f5d600af920 2022-07-21T01:53:34.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.831+0000 7f5d5e7fc700 1 -- 172.21.15.140:0/3308508381 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5d4c010030 con 0x7f5d600af920 2022-07-21T01:53:34.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.831+0000 7f5d5e7fc700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d3c053950 0x7f5d3c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:34.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.831+0000 7f5d5e7fc700 1 -- 172.21.15.140:0/3308508381 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5d4c06c0b0 con 0x7f5d600af920 2022-07-21T01:53:34.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.831+0000 7f5d65185700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d3c053950 0x7f5d3c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:34.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.831+0000 7f5d65185700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d3c053950 0x7f5d3c055e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f5d5000b000 tx=0x7f5d50009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:34.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.839+0000 7f5d5e7fc700 1 -- 172.21.15.140:0/3308508381 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5d4c042f30 con 0x7f5d600af920 2022-07-21T01:53:34.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.963+0000 7f5d66187700 1 -- 172.21.15.140:0/3308508381 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d600574e0 con 0x7f5d600af920 2022-07-21T01:53:34.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.963+0000 7f5d5e7fc700 1 -- 172.21.15.140:0/3308508381 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5d4c046550 con 0x7f5d600af920 2022-07-21T01:53:34.961 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:53:34.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.963+0000 7f5d66187700 1 -- 172.21.15.140:0/3308508381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d3c053950 msgr2=0x7f5d3c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.963+0000 7f5d66187700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d3c053950 0x7f5d3c055e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f5d5000b000 tx=0x7f5d50009040).stop 2022-07-21T01:53:34.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.963+0000 7f5d66187700 1 -- 172.21.15.140:0/3308508381 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d600af920 msgr2=0x7f5d600b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:34.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.963+0000 7f5d66187700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d600af920 0x7f5d600b4180 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f5d4c001c40 tx=0x7f5d4c002100).stop 2022-07-21T01:53:34.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.967+0000 7f5d66187700 1 -- 172.21.15.140:0/3308508381 shutdown_connections 2022-07-21T01:53:34.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.967+0000 7f5d66187700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5d3c053950 0x7f5d3c055e10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.967+0000 7f5d66187700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5d600b59d0 0x7f5d60144050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.967+0000 7f5d66187700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5d600af920 0x7f5d600b4180 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.967+0000 7f5d66187700 1 --2- 172.21.15.140:0/3308508381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5d6005d0c0 0x7f5d60065df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:34.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.967+0000 7f5d66187700 1 -- 172.21.15.140:0/3308508381 >> 172.21.15.140:0/3308508381 conn(0x7f5d600634c0 msgr2=0x7f5d6005c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:34.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.967+0000 7f5d66187700 1 -- 172.21.15.140:0/3308508381 shutdown_connections 2022-07-21T01:53:34.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:34.967+0000 7f5d66187700 1 -- 172.21.15.140:0/3308508381 wait complete. 2022-07-21T01:53:34.973 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:53:34.974 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:53:35.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439c960700 1 Processor -- start 2022-07-21T01:53:35.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439c960700 1 -- start start 2022-07-21T01:53:35.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439c960700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43980b5650 0x7f43980b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:35.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439c960700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f43980af8e0 0x7f439805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:35.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439c960700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4398147010 con 0x7f43980adff0 2022-07-21T01:53:35.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439c960700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43981472b0 con 0x7f43980af8e0 2022-07-21T01:53:35.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439c960700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4398146d40 con 0x7f43980b5650 2022-07-21T01:53:35.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439659c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f43980adff0 0x7f4398147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50232/0 (socket says 172.21.15.140:50232) 2022-07-21T01:53:35.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439659c700 1 -- 172.21.15.140:0/4077506041 learned_addr learned my addr 172.21.15.140:0/4077506041 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:35.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439759e700 1 --2- 172.21.15.140:0/4077506041 >> v2:172.21.15.152:3300/0 conn(0x7f43980af8e0 0x7f439805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:35.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4396d9d700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43980b5650 0x7f43980b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:35.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053236358 0 0) 0x7f4398147010 con 0x7f43980adff0 2022-07-21T01:53:35.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f437c003260 con 0x7f43980adff0 2022-07-21T01:53:35.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 55077746 0 0) 0x7f437c003260 con 0x7f43980adff0 2022-07-21T01:53:35.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43980b5650 msgr2=0x7f43980b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:35.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43980b5650 0x7f43980b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:35.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 >> v2:172.21.15.152:3300/0 conn(0x7f43980af8e0 msgr2=0x7f439805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:35.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 --2- 172.21.15.140:0/4077506041 >> v2:172.21.15.152:3300/0 conn(0x7f43980af8e0 0x7f439805bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:35.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43980acae0 con 0x7f43980adff0 2022-07-21T01:53:35.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f438c0017e0 con 0x7f43980adff0 2022-07-21T01:53:35.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 >> v1:172.21.15.140:6789/0 conn(0x7f43980adff0 legacy=0x7f4398147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:35.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f437c003750 0x7f437c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:35.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c004500 0x7f437c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:35.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43980b5650 0x7f437c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:35.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43981472b0 con 0x7f437c003750 2022-07-21T01:53:35.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4398146d40 con 0x7f437c004500 2022-07-21T01:53:35.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437c00b750 con 0x7f43980b5650 2022-07-21T01:53:35.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4396d9d700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f437c003750 0x7f437c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:35.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439659c700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c004500 0x7f437c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:35.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439759e700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43980b5650 0x7f437c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:35.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4396d9d700 1 -- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43980b5650 msgr2=0x7f437c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:35.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4396d9d700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43980b5650 0x7f437c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:35.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4396d9d700 1 -- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c004500 msgr2=0x7f437c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:35.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4396d9d700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c004500 0x7f437c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:35.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4396d9d700 1 -- 172.21.15.140:0/4077506041 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437c003260 con 0x7f437c003750 2022-07-21T01:53:35.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f439659c700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f437c004500 0x7f437c004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:35.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4396d9d700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f437c003750 0x7f437c003b70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f438800c000 tx=0x7f4388006df0).ready entity=mon.0 client_cookie=2d5a501ed31bb7c8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:35.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f438800e830 con 0x7f437c003750 2022-07-21T01:53:35.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438800e990 con 0x7f437c003750 2022-07-21T01:53:35.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.079+0000 7f4395d9b700 1 -- 172.21.15.140:0/4077506041 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f438800ecf0 con 0x7f437c003750 2022-07-21T01:53:35.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 -- 172.21.15.140:0/4077506041 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f437c003750 msgr2=0x7f437c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:35.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 --2- 172.21.15.140:0/4077506041 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f437c003750 0x7f437c003b70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f438800c000 tx=0x7f4388006df0).stop 2022-07-21T01:53:35.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439759e700 1 -- 172.21.15.140:0/4077506041 reap_dead start 2022-07-21T01:53:35.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 -- 172.21.15.140:0/4077506041 shutdown_connections 2022-07-21T01:53:35.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 -- 172.21.15.140:0/4077506041 >> 172.21.15.140:0/4077506041 conn(0x7f43980634e0 msgr2=0x7f43980b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:35.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 -- 172.21.15.140:0/4077506041 shutdown_connections 2022-07-21T01:53:35.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 -- 172.21.15.140:0/4077506041 wait complete. 2022-07-21T01:53:35.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 Processor -- start 2022-07-21T01:53:35.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 -- start start 2022-07-21T01:53:35.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4398066c50 0x7f4398064100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:35.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43981574e0 0x7f43980645f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:35.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4398111920 0x7f4398159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:35.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4398065e50 con 0x7f4398111920 2022-07-21T01:53:35.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4398067380 con 0x7f4398066c50 2022-07-21T01:53:35.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439c960700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4398067620 con 0x7f43981574e0 2022-07-21T01:53:35.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4398111920 0x7f4398159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:35.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4398111920 0x7f4398159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36098/0 (socket says 172.21.15.140:36098) 2022-07-21T01:53:35.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439759e700 1 -- 172.21.15.140:0/3806015795 learned_addr learned my addr 172.21.15.140:0/3806015795 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:35.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f4396d9d700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4398066c50 0x7f4398064100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:35.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.083+0000 7f439659c700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43981574e0 0x7f43980645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:35.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f439759e700 1 -- 172.21.15.140:0/3806015795 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43981574e0 msgr2=0x7f43980645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:35.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f439759e700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43981574e0 0x7f43980645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:35.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f439759e700 1 -- 172.21.15.140:0/3806015795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4398066c50 msgr2=0x7f4398064100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:35.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f439759e700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4398066c50 0x7f4398064100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:35.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f439759e700 1 -- 172.21.15.140:0/3806015795 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4398157040 con 0x7f4398111920 2022-07-21T01:53:35.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f439759e700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4398111920 0x7f4398159b70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f439000b000 tx=0x7f43900091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:35.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f4387fff700 1 -- 172.21.15.140:0/3806015795 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4390019070 con 0x7f4398111920 2022-07-21T01:53:35.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f4387fff700 1 -- 172.21.15.140:0/3806015795 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4390014720 con 0x7f4398111920 2022-07-21T01:53:35.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f439c960700 1 -- 172.21.15.140:0/3806015795 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4398110bb0 con 0x7f4398111920 2022-07-21T01:53:35.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f4387fff700 1 -- 172.21.15.140:0/3806015795 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4390014a80 con 0x7f4398111920 2022-07-21T01:53:35.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f439c960700 1 -- 172.21.15.140:0/3806015795 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4398064ae0 con 0x7f4398111920 2022-07-21T01:53:35.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f439c960700 1 -- 172.21.15.140:0/3806015795 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43981571d0 con 0x7f4398111920 2022-07-21T01:53:35.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.087+0000 7f4387fff700 1 -- 172.21.15.140:0/3806015795 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4390014be0 con 0x7f4398111920 2022-07-21T01:53:35.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.091+0000 7f4387fff700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4374053b30 0x7f4374055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:35.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.091+0000 7f4387fff700 1 -- 172.21.15.140:0/3806015795 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f439006c500 con 0x7f4398111920 2022-07-21T01:53:35.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.091+0000 7f4396d9d700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4374053b30 0x7f4374055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:35.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.091+0000 7f4396d9d700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4374053b30 0x7f4374055ff0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f438801d000 tx=0x7f4388019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:35.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.095+0000 7f4387fff700 1 -- 172.21.15.140:0/3806015795 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4390043420 con 0x7f4398111920 2022-07-21T01:53:35.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.255+0000 7f439c960700 1 -- 172.21.15.140:0/3806015795 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43980b5a30 con 0x7f4398111920 2022-07-21T01:53:35.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.255+0000 7f4387fff700 1 -- 172.21.15.140:0/3806015795 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f4390046a40 con 0x7f4398111920 2022-07-21T01:53:35.253 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:35.254 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":26,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/689662045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":689662045},{"type":"v1","addr":"172.21.15.140:6835","nonce":689662045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:53:30.476186+0000","modified":"2022-07-21T01:53:31.479177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5899":{"gid":5899,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/74479549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":74479549},{"type":"v1","addr":"172.21.15.152:6833","nonce":74479549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-07-21T01:53:35.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 -- 172.21.15.140:0/3806015795 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4374053b30 msgr2=0x7f4374055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:35.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4374053b30 0x7f4374055ff0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f438801d000 tx=0x7f4388019000).stop 2022-07-21T01:53:35.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 -- 172.21.15.140:0/3806015795 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4398111920 msgr2=0x7f4398159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:35.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4398111920 0x7f4398159b70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f439000b000 tx=0x7f43900091d0).stop 2022-07-21T01:53:35.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 -- 172.21.15.140:0/3806015795 shutdown_connections 2022-07-21T01:53:35.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4374053b30 0x7f4374055ff0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:35.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4398111920 0x7f4398159b70 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:35.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43981574e0 0x7f43980645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:35.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 --2- 172.21.15.140:0/3806015795 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4398066c50 0x7f4398064100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:35.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 -- 172.21.15.140:0/3806015795 >> 172.21.15.140:0/3806015795 conn(0x7f43980634e0 msgr2=0x7f439805c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:35.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 -- 172.21.15.140:0/3806015795 shutdown_connections 2022-07-21T01:53:35.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:35.259+0000 7f439c960700 1 -- 172.21.15.140:0/3806015795 wait complete. 2022-07-21T01:53:35.262 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 26 2022-07-21T01:53:35.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:53:30.476186+0000', 'modified': '2022-07-21T01:53:31.479177+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5899': {'gid': 5899, 'name': 'b', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/74479549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 74479549}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 74479549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:53:35.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:53:35.469 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:53:35.587 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":5899,"want_state":"up:active","state":"up:active","rank_uptime":5.1041334469999997,"mdsmap_epoch":26,"osdmap_epoch":63,"osdmap_epoch_barrier":62,"uptime":9.8814547590000004} 2022-07-21T01:53:35.588 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 5899, "mdsmap_epoch": 26, "osdmap_epoch": 63, "osdmap_epoch_barrier": 62, "rank_uptime": 5.104133447, "state": "up:active", "uptime": 9.881454759, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:53:35.588 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:53:35.589 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:53:35.589 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:53:35.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:35.591 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:35.600 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:53:35.659 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:53:35.661 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:53:35.674 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:53:35.675 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:53:35.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:35.678 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:53:35.725 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:53:35.725 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:35.728 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:53:35.748 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:53:35.748 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:53:35.765 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:53:35.767 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:53:35.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:35.769 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:53:35.805 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:53:35.812+0000 7fa9290d8080 -1 init, newargv = 0x55a050191380 newargc=9ceph-fuse[22680]: starting ceph client 2022-07-21T01:53:35.805 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr: 2022-07-21T01:53:35.813 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22680]: starting fuse 2022-07-21T01:53:36.915 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:53:36.934 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:53:36.934 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:53:36.945 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:53:36.946 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:53:36.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:36.947 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:53:36.953 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:53:36.953 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:53:36.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:36.954 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:53:36.955 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:53:36.955 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:53:36.956 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:53:36.956 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:53:36.957 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:53:36.957 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:53:36.958 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:53:36.958 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:53:36.959 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:53:36.959 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:53:36.960 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:53:36.960 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:53:36.961 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:53:36.961 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:53:36.962 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:53:36.962 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:53:36.963 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:53:36.963 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:53:36.964 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:53:36.964 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:53:36.965 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:53:36.965 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:53:37.037 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.22680.asok 2022-07-21T01:53:37.043 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22680.asok 2022-07-21T01:53:37.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:37.044 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22680.asok status 2022-07-21T01:53:37.135 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:53:37.136 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:53:37.137 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:53:37.137 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:53:37.137 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:53:37.138 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:53:37.138 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:53:37.139 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "22680", 2022-07-21T01:53:37.139 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:53:37.140 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:53:37.140 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:53:37.141 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:53:37.141 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 5591, 2022-07-21T01:53:37.141 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:53:37.142 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:53:37.142 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:53:37.143 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 5591 2022-07-21T01:53:37.143 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:53:37.144 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:53:37.144 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:53:37.145 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:53:37.145 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1714699085 2022-07-21T01:53:37.146 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:53:37.146 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:53:37.147 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:53:37.147 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:53:37.148 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:53:37.148 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1714699085 2022-07-21T01:53:37.149 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:53:37.149 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.5591 172.21.15.191:0/1714699085", 2022-07-21T01:53:37.150 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/1714699085", 2022-07-21T01:53:37.150 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:53:37.151 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 26, 2022-07-21T01:53:37.151 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 63, 2022-07-21T01:53:37.152 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:53:37.152 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:53:37.153 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:53:37.153 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:53:37.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:37.157 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:37.158 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:37.166 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:53:37.168 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:53:37.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:37.169 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:37.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:37.229 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:37.230 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:37.238 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:53:37.240 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:53:37.241 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-07-21T01:53:37.242 DEBUG:teuthology.orchestra.run.smithi191:> import sys 2022-07-21T01:53:37.243 DEBUG:teuthology.orchestra.run.smithi191:> import time 2022-07-21T01:53:37.244 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:53:37.245 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:53:37.245 DEBUG:teuthology.orchestra.run.smithi191:> n = 1000 2022-07-21T01:53:37.246 DEBUG:teuthology.orchestra.run.smithi191:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2022-07-21T01:53:37.246 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:53:37.247 DEBUG:teuthology.orchestra.run.smithi191:> if not os.path.exists(os.path.dirname(abs_path)): 2022-07-21T01:53:37.247 DEBUG:teuthology.orchestra.run.smithi191:> os.makedirs(os.path.dirname(abs_path)) 2022-07-21T01:53:37.247 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:53:37.248 DEBUG:teuthology.orchestra.run.smithi191:> for i in range(0, n): 2022-07-21T01:53:37.249 DEBUG:teuthology.orchestra.run.smithi191:> fname = "{0}_{1}".format(abs_path, i) 2022-07-21T01:53:37.250 DEBUG:teuthology.orchestra.run.smithi191:> with open(fname, '"'"'w'"'"') as f: 2022-07-21T01:53:37.250 DEBUG:teuthology.orchestra.run.smithi191:> f.write('"'"'content'"'"') 2022-07-21T01:53:37.251 DEBUG:teuthology.orchestra.run.smithi191:> if True: 2022-07-21T01:53:37.252 DEBUG:teuthology.orchestra.run.smithi191:> f.flush() 2022-07-21T01:53:37.252 DEBUG:teuthology.orchestra.run.smithi191:> os.fsync(f.fileno()) 2022-07-21T01:53:37.253 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:53:45.937 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:53:46.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.051+0000 7f0515236700 1 Processor -- start 2022-07-21T01:53:46.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.051+0000 7f0515236700 1 -- start start 2022-07-21T01:53:46.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.051+0000 7f0515236700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f05100b0570 0x7f05100b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.051+0000 7f0515236700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f05100aea10 0x7f0510141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.051+0000 7f0515236700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05101422c0 0x7f05100ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.051+0000 7f0515236700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0510143b10 con 0x7f05100b0570 2022-07-21T01:53:46.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.051+0000 7f0515236700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0510142750 con 0x7f05100aea10 2022-07-21T01:53:46.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.051+0000 7f0515236700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05101433d0 con 0x7f05101422c0 2022-07-21T01:53:46.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.051+0000 7f050ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f05100b0570 0x7f05100b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f05100b0570 0x7f05100b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36104/0 (socket says 172.21.15.140:36104) 2022-07-21T01:53:46.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050ffff700 1 -- 172.21.15.140:0/1918621698 learned_addr learned my addr 172.21.15.140:0/1918621698 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:46.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050f7fe700 1 --2- 172.21.15.140:0/1918621698 >> v2:172.21.15.152:3300/0 conn(0x7f05100aea10 0x7f0510141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f0514a35700 1 --2- 172.21.15.140:0/1918621698 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05101422c0 0x7f05100ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050ffff700 1 -- 172.21.15.140:0/1918621698 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05101422c0 msgr2=0x7f05100ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:46.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050ffff700 1 --2- 172.21.15.140:0/1918621698 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05101422c0 0x7f05100ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050ffff700 1 -- 172.21.15.140:0/1918621698 >> v2:172.21.15.152:3300/0 conn(0x7f05100aea10 msgr2=0x7f0510141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:46.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050ffff700 1 --2- 172.21.15.140:0/1918621698 >> v2:172.21.15.152:3300/0 conn(0x7f05100aea10 0x7f0510141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050ffff700 1 -- 172.21.15.140:0/1918621698 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0510143670 con 0x7f05100b0570 2022-07-21T01:53:46.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050f7fe700 1 --2- 172.21.15.140:0/1918621698 >> v2:172.21.15.152:3300/0 conn(0x7f05100aea10 0x7f0510141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:46.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050ffff700 1 --2- 172.21.15.140:0/1918621698 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05100b0570 0x7f05100b0950 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f04f8002f70 tx=0x7f04f8006eb0).ready entity=mon.0 client_cookie=317defa8c00b77eb server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:46.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050effd700 1 -- 172.21.15.140:0/1918621698 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04f800d570 con 0x7f05100b0570 2022-07-21T01:53:46.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050effd700 1 -- 172.21.15.140:0/1918621698 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f8000e90 con 0x7f05100b0570 2022-07-21T01:53:46.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.055+0000 7f050effd700 1 -- 172.21.15.140:0/1918621698 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04f800d900 con 0x7f05100b0570 2022-07-21T01:53:46.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.059+0000 7f0515236700 1 -- 172.21.15.140:0/1918621698 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05100b0570 msgr2=0x7f05100b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.059+0000 7f0515236700 1 --2- 172.21.15.140:0/1918621698 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05100b0570 0x7f05100b0950 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f04f8002f70 tx=0x7f04f8006eb0).stop 2022-07-21T01:53:46.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.059+0000 7f0515236700 1 -- 172.21.15.140:0/1918621698 shutdown_connections 2022-07-21T01:53:46.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.059+0000 7f0515236700 1 --2- 172.21.15.140:0/1918621698 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05101422c0 0x7f05100ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.059+0000 7f0515236700 1 --2- 172.21.15.140:0/1918621698 >> v2:172.21.15.152:3300/0 conn(0x7f05100aea10 0x7f0510141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.059+0000 7f0515236700 1 --2- 172.21.15.140:0/1918621698 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05100b0570 0x7f05100b0950 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.059+0000 7f0515236700 1 -- 172.21.15.140:0/1918621698 >> 172.21.15.140:0/1918621698 conn(0x7f05100b9360 msgr2=0x7f05100b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:46.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.059+0000 7f0515236700 1 -- 172.21.15.140:0/1918621698 shutdown_connections 2022-07-21T01:53:46.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.059+0000 7f0515236700 1 -- 172.21.15.140:0/1918621698 wait complete. 2022-07-21T01:53:46.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.059+0000 7f0515236700 1 Processor -- start 2022-07-21T01:53:46.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.059+0000 7f0515236700 1 -- start start 2022-07-21T01:53:46.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f0515236700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05100aea10 0x7f0510112890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f0515236700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05100b0570 0x7f05100b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f0515236700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05101422c0 0x7f05100b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f0515236700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0510152af0 con 0x7f05100b0570 2022-07-21T01:53:46.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f0515236700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05100b1760 con 0x7f05100aea10 2022-07-21T01:53:46.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f0515236700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0510152f70 con 0x7f05101422c0 2022-07-21T01:53:46.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05100b0570 0x7f05100b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05100b0570 0x7f05100b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36112/0 (socket says 172.21.15.140:36112) 2022-07-21T01:53:46.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050f7fe700 1 -- 172.21.15.140:0/480529347 learned_addr learned my addr 172.21.15.140:0/480529347 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:46.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050ffff700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05100aea10 0x7f0510112890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f0514a35700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05101422c0 0x7f05100b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050f7fe700 1 -- 172.21.15.140:0/480529347 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05101422c0 msgr2=0x7f05100b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050f7fe700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05101422c0 0x7f05100b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050f7fe700 1 -- 172.21.15.140:0/480529347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05100aea10 msgr2=0x7f0510112890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050f7fe700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05100aea10 0x7f0510112890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050f7fe700 1 -- 172.21.15.140:0/480529347 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0508009470 con 0x7f05100b0570 2022-07-21T01:53:46.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050f7fe700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05100b0570 0x7f05100b1270 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f050800c000 tx=0x7f05080091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:46.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050d7fa700 1 -- 172.21.15.140:0/480529347 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f050800c920 con 0x7f05100b0570 2022-07-21T01:53:46.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050d7fa700 1 -- 172.21.15.140:0/480529347 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050800ca80 con 0x7f05100b0570 2022-07-21T01:53:46.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f0515236700 1 -- 172.21.15.140:0/480529347 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f800b1e0 con 0x7f05100b0570 2022-07-21T01:53:46.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f050d7fa700 1 -- 172.21.15.140:0/480529347 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f050800cde0 con 0x7f05100b0570 2022-07-21T01:53:46.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.063+0000 7f0515236700 1 -- 172.21.15.140:0/480529347 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05100ae180 con 0x7f05100b0570 2022-07-21T01:53:46.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.067+0000 7f0515236700 1 -- 172.21.15.140:0/480529347 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05100b3910 con 0x7f05100b0570 2022-07-21T01:53:46.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.067+0000 7f050d7fa700 1 -- 172.21.15.140:0/480529347 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0508022070 con 0x7f05100b0570 2022-07-21T01:53:46.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.067+0000 7f050d7fa700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f04f40539e0 0x7f04f4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.067+0000 7f050d7fa700 1 -- 172.21.15.140:0/480529347 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f050806c7c0 con 0x7f05100b0570 2022-07-21T01:53:46.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.067+0000 7f050ffff700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f04f40539e0 0x7f04f4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.067+0000 7f050ffff700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f04f40539e0 0x7f04f4055ea0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f04f801c000 tx=0x7f04f8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:46.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.071+0000 7f050d7fa700 1 -- 172.21.15.140:0/480529347 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0508043640 con 0x7f05100b0570 2022-07-21T01:53:46.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.235+0000 7f0515236700 1 -- 172.21.15.140:0/480529347 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05100b1010 con 0x7f05100b0570 2022-07-21T01:53:46.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.235+0000 7f050d7fa700 1 -- 172.21.15.140:0/480529347 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f0508046c60 con 0x7f05100b0570 2022-07-21T01:53:46.234 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:46.234 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":26,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/689662045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":689662045},{"type":"v1","addr":"172.21.15.140:6835","nonce":689662045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:53:30.476186+0000","modified":"2022-07-21T01:53:31.479177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5899":{"gid":5899,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/74479549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":74479549},{"type":"v1","addr":"172.21.15.152:6833","nonce":74479549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-07-21T01:53:46.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 -- 172.21.15.140:0/480529347 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f04f40539e0 msgr2=0x7f04f4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f04f40539e0 0x7f04f4055ea0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f04f801c000 tx=0x7f04f8015000).stop 2022-07-21T01:53:46.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 -- 172.21.15.140:0/480529347 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05100b0570 msgr2=0x7f05100b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05100b0570 0x7f05100b1270 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f050800c000 tx=0x7f05080091d0).stop 2022-07-21T01:53:46.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 -- 172.21.15.140:0/480529347 shutdown_connections 2022-07-21T01:53:46.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f04f40539e0 0x7f04f4055ea0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05101422c0 0x7f05100b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05100b0570 0x7f05100b1270 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 --2- 172.21.15.140:0/480529347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05100aea10 0x7f0510112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 -- 172.21.15.140:0/480529347 >> 172.21.15.140:0/480529347 conn(0x7f05100b9360 msgr2=0x7f051013f250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:46.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 -- 172.21.15.140:0/480529347 shutdown_connections 2022-07-21T01:53:46.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.239+0000 7f0515236700 1 -- 172.21.15.140:0/480529347 wait complete. 2022-07-21T01:53:46.239 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 26 2022-07-21T01:53:46.251 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_caps 20 2022-07-21T01:53:46.361 INFO:teuthology.orchestra.run.smithi152.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2022-07-21T01:53:46.362 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2022-07-21T01:53:46.363 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:53:46.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.479+0000 7fe6726fb700 1 Processor -- start 2022-07-21T01:53:46.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.479+0000 7fe6726fb700 1 -- start start 2022-07-21T01:53:46.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.479+0000 7fe6726fb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe66c0aebb0 0x7fe66c0b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.479+0000 7fe6726fb700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe66c0acae0 0x7fe66c13f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.479+0000 7fe6726fb700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66c0aef90 con 0x7fe66c13fd60 2022-07-21T01:53:46.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.479+0000 7fe6726fb700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66c0b5d60 con 0x7fe66c0acae0 2022-07-21T01:53:46.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.479+0000 7fe6726fb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0b0270 con 0x7fe66c0aebb0 2022-07-21T01:53:46.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.479+0000 7fe670ef8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe66c0acae0 0x7fe66c13f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.479+0000 7fe670ef8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe66c0acae0 0x7fe66c13f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36118/0 (socket says 172.21.15.140:36118) 2022-07-21T01:53:46.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.479+0000 7fe670ef8700 1 -- 172.21.15.140:0/693462203 learned_addr learned my addr 172.21.15.140:0/693462203 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:46.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.479+0000 7fe6716f9700 1 --2- 172.21.15.140:0/693462203 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe66c0aebb0 0x7fe66c0b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6716f9700 1 -- 172.21.15.140:0/693462203 >> v2:172.21.15.140:3300/0 conn(0x7fe66c0acae0 msgr2=0x7fe66c13f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:46.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6716f9700 1 --2- 172.21.15.140:0/693462203 >> v2:172.21.15.140:3300/0 conn(0x7fe66c0acae0 0x7fe66c13f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6716f9700 1 -- 172.21.15.140:0/693462203 >> v1:172.21.15.152:6789/0 conn(0x7fe66c13fd60 legacy=0x7fe66c0b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe65bfff700 1 -- 172.21.15.140:0/693462203 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64874269 0 0) 0x7fe66c0aef90 con 0x7fe66c13fd60 2022-07-21T01:53:46.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6716f9700 1 -- 172.21.15.140:0/693462203 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c0af200 con 0x7fe66c0aebb0 2022-07-21T01:53:46.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe670ef8700 1 --2- 172.21.15.140:0/693462203 >> v2:172.21.15.140:3300/0 conn(0x7fe66c0acae0 0x7fe66c13f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:46.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6716f9700 1 --2- 172.21.15.140:0/693462203 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe66c0aebb0 0x7fe66c0b0690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fe66c0ae710 tx=0x7fe668006df0).ready entity=mon.2 client_cookie=217f10e3345fb046 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:46.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe65bfff700 1 -- 172.21.15.140:0/693462203 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe66800d570 con 0x7fe66c0aebb0 2022-07-21T01:53:46.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe65bfff700 1 -- 172.21.15.140:0/693462203 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe668000e90 con 0x7fe66c0aebb0 2022-07-21T01:53:46.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe65bfff700 1 -- 172.21.15.140:0/693462203 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe66800d8c0 con 0x7fe66c0aebb0 2022-07-21T01:53:46.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 -- 172.21.15.140:0/693462203 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe66c0aebb0 msgr2=0x7fe66c0b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 --2- 172.21.15.140:0/693462203 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe66c0aebb0 0x7fe66c0b0690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fe66c0ae710 tx=0x7fe668006df0).stop 2022-07-21T01:53:46.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 -- 172.21.15.140:0/693462203 shutdown_connections 2022-07-21T01:53:46.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 --2- 172.21.15.140:0/693462203 >> v2:172.21.15.140:3300/0 conn(0x7fe66c0acae0 0x7fe66c13f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 --2- 172.21.15.140:0/693462203 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe66c0aebb0 0x7fe66c0b0690 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 -- 172.21.15.140:0/693462203 >> 172.21.15.140:0/693462203 conn(0x7fe66c0634e0 msgr2=0x7fe66c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:46.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 -- 172.21.15.140:0/693462203 shutdown_connections 2022-07-21T01:53:46.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 -- 172.21.15.140:0/693462203 wait complete. 2022-07-21T01:53:46.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 Processor -- start 2022-07-21T01:53:46.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 -- start start 2022-07-21T01:53:46.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe66c0acae0 0x7fe66c140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe66c0aebb0 0x7fe66c1519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe66c13fd60 0x7fe66c14da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c13cfb0 con 0x7fe66c0acae0 2022-07-21T01:53:46.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c13acb0 con 0x7fe66c0aebb0 2022-07-21T01:53:46.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6726fb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c151ee0 con 0x7fe66c13fd60 2022-07-21T01:53:46.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6716f9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe66c0acae0 0x7fe66c140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe670ef8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe66c0aebb0 0x7fe66c1519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe6716f9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe66c0acae0 0x7fe66c140b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36126/0 (socket says 172.21.15.140:36126) 2022-07-21T01:53:46.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe670ef8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe66c0aebb0 0x7fe66c1519f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:41864/0 (socket says 172.21.15.140:41864) 2022-07-21T01:53:46.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe670ef8700 1 -- 172.21.15.140:0/1657512310 learned_addr learned my addr 172.21.15.140:0/1657512310 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:46.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.483+0000 7fe671efa700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe66c13fd60 0x7fe66c14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe6716f9700 1 -- 172.21.15.140:0/1657512310 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe66c13fd60 msgr2=0x7fe66c14da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe6716f9700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe66c13fd60 0x7fe66c14da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe6716f9700 1 -- 172.21.15.140:0/1657512310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe66c0aebb0 msgr2=0x7fe66c1519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe6716f9700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe66c0aebb0 0x7fe66c1519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe6716f9700 1 -- 172.21.15.140:0/1657512310 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c00b1e0 con 0x7fe66c0acae0 2022-07-21T01:53:46.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe670ef8700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe66c0aebb0 0x7fe66c1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:46.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe6716f9700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe66c0acae0 0x7fe66c140b20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fe66c152070 tx=0x7fe668014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:46.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe65a7fc700 1 -- 172.21.15.140:0/1657512310 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe668016070 con 0x7fe66c0acae0 2022-07-21T01:53:46.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe65a7fc700 1 -- 172.21.15.140:0/1657512310 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe668013850 con 0x7fe66c0acae0 2022-07-21T01:53:46.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe6726fb700 1 -- 172.21.15.140:0/1657512310 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66800b180 con 0x7fe66c0acae0 2022-07-21T01:53:46.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe65a7fc700 1 -- 172.21.15.140:0/1657512310 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe668013bb0 con 0x7fe66c0acae0 2022-07-21T01:53:46.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe6726fb700 1 -- 172.21.15.140:0/1657512310 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c151550 con 0x7fe66c0acae0 2022-07-21T01:53:46.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe6726fb700 1 -- 172.21.15.140:0/1657512310 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66c151710 con 0x7fe66c0acae0 2022-07-21T01:53:46.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe65a7fc700 1 -- 172.21.15.140:0/1657512310 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe66800f030 con 0x7fe66c0acae0 2022-07-21T01:53:46.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe65a7fc700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe64c053950 0x7fe64c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.487+0000 7fe65a7fc700 1 -- 172.21.15.140:0/1657512310 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe66806cb60 con 0x7fe66c0acae0 2022-07-21T01:53:46.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.491+0000 7fe670ef8700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe64c053950 0x7fe64c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.491+0000 7fe670ef8700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe64c053950 0x7fe64c055e10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fe65c00b000 tx=0x7fe65c006e50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:46.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.495+0000 7fe65a7fc700 1 -- 172.21.15.140:0/1657512310 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe6680439e0 con 0x7fe66c0acae0 2022-07-21T01:53:46.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.655+0000 7fe6726fb700 1 -- 172.21.15.140:0/1657512310 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe66c0af4a0 con 0x7fe66c0acae0 2022-07-21T01:53:46.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.655+0000 7fe65a7fc700 1 -- 172.21.15.140:0/1657512310 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2137 (secure 0 0 0) 0x7fe668023070 con 0x7fe66c0acae0 2022-07-21T01:53:46.657 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:46.658 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":26,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/689662045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":689662045},{"type":"v1","addr":"172.21.15.140:6835","nonce":689662045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:53:30.476186+0000","modified":"2022-07-21T01:53:31.479177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5899":{"gid":5899,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/74479549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":74479549},{"type":"v1","addr":"172.21.15.152:6833","nonce":74479549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-07-21T01:53:46.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 -- 172.21.15.140:0/1657512310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe64c053950 msgr2=0x7fe64c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe64c053950 0x7fe64c055e10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fe65c00b000 tx=0x7fe65c006e50).stop 2022-07-21T01:53:46.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 -- 172.21.15.140:0/1657512310 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe66c0acae0 msgr2=0x7fe66c140b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe66c0acae0 0x7fe66c140b20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fe66c152070 tx=0x7fe668014000).stop 2022-07-21T01:53:46.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 -- 172.21.15.140:0/1657512310 shutdown_connections 2022-07-21T01:53:46.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe64c053950 0x7fe64c055e10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe66c13fd60 0x7fe66c14da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe66c0aebb0 0x7fe66c1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 --2- 172.21.15.140:0/1657512310 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe66c0acae0 0x7fe66c140b20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 -- 172.21.15.140:0/1657512310 >> 172.21.15.140:0/1657512310 conn(0x7fe66c0634e0 msgr2=0x7fe66c0b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:46.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 -- 172.21.15.140:0/1657512310 shutdown_connections 2022-07-21T01:53:46.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.659+0000 7fe6726fb700 1 -- 172.21.15.140:0/1657512310 wait complete. 2022-07-21T01:53:46.666 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 26 2022-07-21T01:53:46.675 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_recall_max_decay_threshold 400 2022-07-21T01:53:46.790 INFO:teuthology.orchestra.run.smithi152.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2022-07-21T01:53:46.791 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2022-07-21T01:53:46.792 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:53:46.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd33aa700 1 Processor -- start 2022-07-21T01:53:46.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd33aa700 1 -- start start 2022-07-21T01:53:46.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd33aa700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0b1e00 0x7f0bcc0ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd33aa700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0bcc0ae5c0 0x7f0bcc143200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd33aa700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bcc0aff40 con 0x7f0bcc0afa30 2022-07-21T01:53:46.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd33aa700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bcc0b0bc0 con 0x7f0bcc0ae5c0 2022-07-21T01:53:46.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd33aa700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0b21e0 con 0x7f0bcc0b1e00 2022-07-21T01:53:46.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd0945700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0bcc0afa30 0x7f0bcc0adeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50278/0 (socket says 172.21.15.140:50278) 2022-07-21T01:53:46.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd0945700 1 -- 172.21.15.140:0/4207057737 learned_addr learned my addr 172.21.15.140:0/4207057737 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:46.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd1947700 1 --2- 172.21.15.140:0/4207057737 >> v2:172.21.15.152:3300/0 conn(0x7f0bcc0ae5c0 0x7f0bcc143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd1146700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0b1e00 0x7f0bcc0ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223557670 0 0) 0x7f0bcc0aff40 con 0x7f0bcc0afa30 2022-07-21T01:53:46.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bb0003260 con 0x7f0bcc0afa30 2022-07-21T01:53:46.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2728371245 0 0) 0x7f0bb0003260 con 0x7f0bcc0afa30 2022-07-21T01:53:46.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0b1e00 msgr2=0x7f0bcc0ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:46.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0b1e00 0x7f0bcc0ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 >> v2:172.21.15.152:3300/0 conn(0x7f0bcc0ae5c0 msgr2=0x7f0bcc143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:46.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 --2- 172.21.15.140:0/4207057737 >> v2:172.21.15.152:3300/0 conn(0x7f0bcc0ae5c0 0x7f0bcc143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc0af510 con 0x7f0bcc0afa30 2022-07-21T01:53:46.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f0bc80019e0 con 0x7f0bcc0afa30 2022-07-21T01:53:46.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 >> v1:172.21.15.140:6789/0 conn(0x7f0bcc0afa30 legacy=0x7f0bcc0adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bb0003750 0x7f0bb0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0b1e00 0x7f0bb0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00049f0 0x7f0bb0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0b0bc0 con 0x7f0bb00049f0 2022-07-21T01:53:46.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0b21e0 con 0x7f0bb0003750 2022-07-21T01:53:46.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb000b750 con 0x7f0bcc0b1e00 2022-07-21T01:53:46.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd0945700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0b1e00 0x7f0bb0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd1947700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00049f0 0x7f0bb0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd1146700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bb0003750 0x7f0bb0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd0945700 1 -- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bb0003750 msgr2=0x7f0bb0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:46.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd0945700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bb0003750 0x7f0bb0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd0945700 1 -- 172.21.15.140:0/4207057737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00049f0 msgr2=0x7f0bb0004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:46.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd0945700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00049f0 0x7f0bb0004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd0945700 1 -- 172.21.15.140:0/4207057737 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb0003260 con 0x7f0bcc0b1e00 2022-07-21T01:53:46.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd1947700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00049f0 0x7f0bb0004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:46.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd0945700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0b1e00 0x7f0bb0004500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f0bc80016f0 tx=0x7f0bcc0af6a0).ready entity=mon.2 client_cookie=bcad93935cb3f1dd server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:46.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bc800cc20 con 0x7f0bcc0b1e00 2022-07-21T01:53:46.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc800cd80 con 0x7f0bcc0b1e00 2022-07-21T01:53:46.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bc3fff700 1 -- 172.21.15.140:0/4207057737 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bc8005b80 con 0x7f0bcc0b1e00 2022-07-21T01:53:46.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd33aa700 1 -- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0b1e00 msgr2=0x7f0bb0004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd33aa700 1 --2- 172.21.15.140:0/4207057737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0b1e00 0x7f0bb0004500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f0bc80016f0 tx=0x7f0bcc0af6a0).stop 2022-07-21T01:53:46.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.903+0000 7f0bd1947700 1 -- 172.21.15.140:0/4207057737 reap_dead start 2022-07-21T01:53:46.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 -- 172.21.15.140:0/4207057737 shutdown_connections 2022-07-21T01:53:46.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 -- 172.21.15.140:0/4207057737 >> 172.21.15.140:0/4207057737 conn(0x7f0bcc0611c0 msgr2=0x7f0bcc062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:46.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 -- 172.21.15.140:0/4207057737 shutdown_connections 2022-07-21T01:53:46.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 -- 172.21.15.140:0/4207057737 wait complete. 2022-07-21T01:53:46.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 Processor -- start 2022-07-21T01:53:46.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 -- start start 2022-07-21T01:53:46.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bcc1478d0 0x7f0bcc142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0a3860 0x7f0bcc142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bcc0a4ed0 0x7f0bcc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc147cb0 con 0x7f0bcc1478d0 2022-07-21T01:53:46.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc142d50 con 0x7f0bcc0a4ed0 2022-07-21T01:53:46.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd33aa700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc067010 con 0x7f0bcc0a3860 2022-07-21T01:53:46.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd1146700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bcc1478d0 0x7f0bcc142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd0945700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0a3860 0x7f0bcc142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd1146700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bcc1478d0 0x7f0bcc142370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36144/0 (socket says 172.21.15.140:36144) 2022-07-21T01:53:46.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd1146700 1 -- 172.21.15.140:0/1223539788 learned_addr learned my addr 172.21.15.140:0/1223539788 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:46.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd1947700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bcc0a4ed0 0x7f0bcc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd0945700 1 -- 172.21.15.140:0/1223539788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bcc0a4ed0 msgr2=0x7f0bcc0a5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd0945700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bcc0a4ed0 0x7f0bcc0a5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd0945700 1 -- 172.21.15.140:0/1223539788 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bcc1478d0 msgr2=0x7f0bcc142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:46.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd0945700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bcc1478d0 0x7f0bcc142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:46.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd0945700 1 -- 172.21.15.140:0/1223539788 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc0a2a80 con 0x7f0bcc0a3860 2022-07-21T01:53:46.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd1146700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bcc1478d0 0x7f0bcc142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:46.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.907+0000 7f0bd0945700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0a3860 0x7f0bcc142860 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f0bc80016f0 tx=0x7f0bc8015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:46.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.911+0000 7f0bc27fc700 1 -- 172.21.15.140:0/1223539788 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bc80072c0 con 0x7f0bcc0a3860 2022-07-21T01:53:46.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.911+0000 7f0bc27fc700 1 -- 172.21.15.140:0/1223539788 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc8007420 con 0x7f0bcc0a3860 2022-07-21T01:53:46.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.911+0000 7f0bc27fc700 1 -- 172.21.15.140:0/1223539788 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0bc8007780 con 0x7f0bcc0a3860 2022-07-21T01:53:46.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.911+0000 7f0bd33aa700 1 -- 172.21.15.140:0/1223539788 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc147ff0 con 0x7f0bcc0a3860 2022-07-21T01:53:46.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.911+0000 7f0bd33aa700 1 -- 172.21.15.140:0/1223539788 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc066d10 con 0x7f0bcc0a3860 2022-07-21T01:53:46.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.911+0000 7f0bd33aa700 1 -- 172.21.15.140:0/1223539788 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc067210 con 0x7f0bcc0a3860 2022-07-21T01:53:46.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.911+0000 7f0bc27fc700 1 -- 172.21.15.140:0/1223539788 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0bc801d070 con 0x7f0bcc0a3860 2022-07-21T01:53:46.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.911+0000 7f0bc27fc700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bb4053a70 0x7f0bb4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:46.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.911+0000 7f0bc27fc700 1 -- 172.21.15.140:0/1223539788 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0bc8019030 con 0x7f0bcc0a3860 2022-07-21T01:53:46.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.911+0000 7f0bd1146700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bb4053a70 0x7f0bb4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:46.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.911+0000 7f0bd1146700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bb4053a70 0x7f0bb4055f30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f0bbc00b000 tx=0x7f0bbc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:46.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:46.919+0000 7f0bc27fc700 1 -- 172.21.15.140:0/1223539788 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0bc8043d20 con 0x7f0bcc0a3860 2022-07-21T01:53:47.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 -- 172.21.15.140:0/1223539788 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bcc0a2490 con 0x7f0bcc0a3860 2022-07-21T01:53:47.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bc27fc700 1 -- 172.21.15.140:0/1223539788 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f0bc8047340 con 0x7f0bcc0a3860 2022-07-21T01:53:47.075 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:47.076 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":26,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/689662045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":689662045},{"type":"v1","addr":"172.21.15.140:6835","nonce":689662045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:53:30.476186+0000","modified":"2022-07-21T01:53:31.479177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5899":{"gid":5899,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/74479549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":74479549},{"type":"v1","addr":"172.21.15.152:6833","nonce":74479549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-07-21T01:53:47.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 -- 172.21.15.140:0/1223539788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bb4053a70 msgr2=0x7f0bb4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:47.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bb4053a70 0x7f0bb4055f30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f0bbc00b000 tx=0x7f0bbc008040).stop 2022-07-21T01:53:47.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 -- 172.21.15.140:0/1223539788 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0a3860 msgr2=0x7f0bcc142860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:47.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0a3860 0x7f0bcc142860 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f0bc80016f0 tx=0x7f0bc8015000).stop 2022-07-21T01:53:47.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 -- 172.21.15.140:0/1223539788 shutdown_connections 2022-07-21T01:53:47.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0bb4053a70 0x7f0bb4055f30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bcc0a4ed0 0x7f0bcc0a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bcc0a3860 0x7f0bcc142860 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 --2- 172.21.15.140:0/1223539788 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bcc1478d0 0x7f0bcc142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 -- 172.21.15.140:0/1223539788 >> 172.21.15.140:0/1223539788 conn(0x7f0bcc0b1e00 msgr2=0x7f0bcc0b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:47.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 -- 172.21.15.140:0/1223539788 shutdown_connections 2022-07-21T01:53:47.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.079+0000 7f0bd33aa700 1 -- 172.21.15.140:0/1223539788 wait complete. 2022-07-21T01:53:47.089 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 26 2022-07-21T01:53:47.091 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b cache drop 10 2022-07-21T01:53:47.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.203+0000 7feb0db97700 1 Processor -- start 2022-07-21T01:53:47.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.203+0000 7feb0db97700 1 -- start start 2022-07-21T01:53:47.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7feb080b62a0 0x7feb080b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7feb080b01e0 0x7feb0813fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb080b6680 con 0x7feb080b3ed0 2022-07-21T01:53:47.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb081465c0 con 0x7feb080b62a0 2022-07-21T01:53:47.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb08142df0 con 0x7feb080b01e0 2022-07-21T01:53:47.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb077fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7feb080b62a0 0x7feb080b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb077fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7feb080b62a0 0x7feb080b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36152/0 (socket says 172.21.15.140:36152) 2022-07-21T01:53:47.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb077fe700 1 -- 172.21.15.140:0/2098818581 learned_addr learned my addr 172.21.15.140:0/2098818581 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:47.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb06ffd700 1 --1- 172.21.15.140:0/2098818581 >> v1:172.21.15.140:6789/0 conn(0x7feb080b3ed0 0x7feb08143530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50300/0 (socket says 172.21.15.140:50300) 2022-07-21T01:53:47.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb07fff700 1 --2- 172.21.15.140:0/2098818581 >> v2:172.21.15.152:3300/0 conn(0x7feb080b01e0 0x7feb0813fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb077fe700 1 -- 172.21.15.140:0/2098818581 >> v2:172.21.15.152:3300/0 conn(0x7feb080b01e0 msgr2=0x7feb0813fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:47.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb077fe700 1 --2- 172.21.15.140:0/2098818581 >> v2:172.21.15.152:3300/0 conn(0x7feb080b01e0 0x7feb0813fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb077fe700 1 -- 172.21.15.140:0/2098818581 >> v1:172.21.15.140:6789/0 conn(0x7feb080b3ed0 legacy=0x7feb08143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:47.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb077fe700 1 -- 172.21.15.140:0/2098818581 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb08143090 con 0x7feb080b62a0 2022-07-21T01:53:47.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb077fe700 1 --2- 172.21.15.140:0/2098818581 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feb080b62a0 0x7feb080b43e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7feb000032f0 tx=0x7feb00009290).ready entity=mon.0 client_cookie=5695edb3bffb3172 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:47.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb067fc700 1 -- 172.21.15.140:0/2098818581 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb0000b870 con 0x7feb080b62a0 2022-07-21T01:53:47.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb067fc700 1 -- 172.21.15.140:0/2098818581 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0000b9d0 con 0x7feb080b62a0 2022-07-21T01:53:47.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb067fc700 1 -- 172.21.15.140:0/2098818581 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb0000bb40 con 0x7feb080b62a0 2022-07-21T01:53:47.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 -- 172.21.15.140:0/2098818581 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feb080b62a0 msgr2=0x7feb080b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:47.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 --2- 172.21.15.140:0/2098818581 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feb080b62a0 0x7feb080b43e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7feb000032f0 tx=0x7feb00009290).stop 2022-07-21T01:53:47.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 -- 172.21.15.140:0/2098818581 shutdown_connections 2022-07-21T01:53:47.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 --2- 172.21.15.140:0/2098818581 >> v2:172.21.15.152:3300/0 conn(0x7feb080b01e0 0x7feb0813fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 --2- 172.21.15.140:0/2098818581 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feb080b62a0 0x7feb080b43e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 -- 172.21.15.140:0/2098818581 >> 172.21.15.140:0/2098818581 conn(0x7feb08066f00 msgr2=0x7feb080672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:47.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 -- 172.21.15.140:0/2098818581 shutdown_connections 2022-07-21T01:53:47.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 -- 172.21.15.140:0/2098818581 wait complete. 2022-07-21T01:53:47.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 Processor -- start 2022-07-21T01:53:47.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 -- start start 2022-07-21T01:53:47.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feb080b01e0 0x7feb080aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.207+0000 7feb0db97700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7feb080b3ed0 0x7feb080af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb0db97700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb080b62a0 0x7feb0813b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb0db97700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb080ae750 con 0x7feb080b01e0 2022-07-21T01:53:47.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb0db97700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb08142880 con 0x7feb080b62a0 2022-07-21T01:53:47.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb0db97700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb081425e0 con 0x7feb080b3ed0 2022-07-21T01:53:47.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb077fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feb080b01e0 0x7feb080aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb077fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feb080b01e0 0x7feb080aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36158/0 (socket says 172.21.15.140:36158) 2022-07-21T01:53:47.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb077fe700 1 -- 172.21.15.140:0/3322056135 learned_addr learned my addr 172.21.15.140:0/3322056135 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:47.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb07fff700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb080b62a0 0x7feb0813b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb06ffd700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7feb080b3ed0 0x7feb080af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb077fe700 1 -- 172.21.15.140:0/3322056135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7feb080b3ed0 msgr2=0x7feb080af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:47.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb077fe700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7feb080b3ed0 0x7feb080af460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb077fe700 1 -- 172.21.15.140:0/3322056135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb080b62a0 msgr2=0x7feb0813b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:47.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb077fe700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb080b62a0 0x7feb0813b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb077fe700 1 -- 172.21.15.140:0/3322056135 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb00009530 con 0x7feb080b01e0 2022-07-21T01:53:47.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb077fe700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feb080b01e0 0x7feb080aebb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7feb000032f0 tx=0x7feb00016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:47.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb04ff9700 1 -- 172.21.15.140:0/3322056135 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb0000b870 con 0x7feb080b01e0 2022-07-21T01:53:47.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb04ff9700 1 -- 172.21.15.140:0/3322056135 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0000bdb0 con 0x7feb080b01e0 2022-07-21T01:53:47.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb0db97700 1 -- 172.21.15.140:0/3322056135 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb081401b0 con 0x7feb080b01e0 2022-07-21T01:53:47.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb04ff9700 1 -- 172.21.15.140:0/3322056135 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb0001f5e0 con 0x7feb080b01e0 2022-07-21T01:53:47.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb0db97700 1 -- 172.21.15.140:0/3322056135 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb081407d0 con 0x7feb080b01e0 2022-07-21T01:53:47.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb04ff9700 1 -- 172.21.15.140:0/3322056135 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7feb00026030 con 0x7feb080b01e0 2022-07-21T01:53:47.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb04ff9700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feae8053980 0x7feae8055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb04ff9700 1 -- 172.21.15.140:0/3322056135 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feb0006ca50 con 0x7feb080b01e0 2022-07-21T01:53:47.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.211+0000 7feb06ffd700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feae8053980 0x7feae8055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb06ffd700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feae8053980 0x7feae8055e40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7feafc00d000 tx=0x7feafc007b40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:47.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb17dee740 1 Processor -- start 2022-07-21T01:53:47.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb17dee740 1 -- start start 2022-07-21T01:53:47.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb17dee740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x1797d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb17dee740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1793790 0x175cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb17dee740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x175d140 0x175b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb17dee740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x16ce210 con 0x1797970 2022-07-21T01:53:47.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb17dee740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x16ae450 con 0x1793790 2022-07-21T01:53:47.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb17dee740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x1793bd0 con 0x175d140 2022-07-21T01:53:47.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb077fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x1797d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb077fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x1797d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36166/0 (socket says 172.21.15.140:36166) 2022-07-21T01:53:47.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb06ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1793790 0x175cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb077fe700 1 -- 172.21.15.140:0/1156726531 learned_addr learned my addr 172.21.15.140:0/1156726531 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:47.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb07fff700 1 --2- 172.21.15.140:0/1156726531 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x175d140 0x175b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb077fe700 1 -- 172.21.15.140:0/1156726531 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x175d140 msgr2=0x175b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:47.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb077fe700 1 --2- 172.21.15.140:0/1156726531 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x175d140 0x175b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb077fe700 1 -- 172.21.15.140:0/1156726531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1793790 msgr2=0x175cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:47.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb077fe700 1 --2- 172.21.15.140:0/1156726531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1793790 0x175cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb077fe700 1 -- 172.21.15.140:0/1156726531 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1705ad0 con 0x1797970 2022-07-21T01:53:47.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb077fe700 1 --2- 172.21.15.140:0/1156726531 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x1797d50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7feb0006b0f0 tx=0x7feb00012000).ready entity=mon.0 client_cookie=f274dceae2d7e7c4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:47.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb0db97700 1 -- 172.21.15.140:0/1156726531 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb00047060 con 0x1797970 2022-07-21T01:53:47.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb0db97700 1 -- 172.21.15.140:0/1156726531 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb00046e80 con 0x1797970 2022-07-21T01:53:47.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.215+0000 7feb0db97700 1 -- 172.21.15.140:0/1156726531 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb000474c0 con 0x1797970 2022-07-21T01:53:47.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 -- 172.21.15.140:0/1156726531 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 msgr2=0x1797d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:47.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 --2- 172.21.15.140:0/1156726531 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x1797d50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7feb0006b0f0 tx=0x7feb00012000).stop 2022-07-21T01:53:47.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 -- 172.21.15.140:0/1156726531 shutdown_connections 2022-07-21T01:53:47.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 --2- 172.21.15.140:0/1156726531 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x175d140 0x175b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 --2- 172.21.15.140:0/1156726531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1793790 0x175cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 --2- 172.21.15.140:0/1156726531 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x1797d50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 -- 172.21.15.140:0/1156726531 >> 172.21.15.140:0/1156726531 conn(0x16e58f0 msgr2=0x1797260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:47.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 -- 172.21.15.140:0/1156726531 shutdown_connections 2022-07-21T01:53:47.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 -- 172.21.15.140:0/1156726531 wait complete. 2022-07-21T01:53:47.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 -- start start 2022-07-21T01:53:47.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 Processor -- start 2022-07-21T01:53:47.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x175d140 0x17a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1793790 0x1791770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1797970 0x1791c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x17a2190 con 0x1793790 2022-07-21T01:53:47.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x1763880 con 0x1797970 2022-07-21T01:53:47.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x17a23b0 con 0x175d140 2022-07-21T01:53:47.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb17dee740 10 client.0 ll_register_callbacks cb 0x17150b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-07-21T01:53:47.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb06ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1793790 0x1791770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb06ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1793790 0x1791770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36174/0 (socket says 172.21.15.140:36174) 2022-07-21T01:53:47.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb06ffd700 1 -- 172.21.15.140:0/22204224 learned_addr learned my addr 172.21.15.140:0/22204224 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:47.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb07fff700 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1797970 0x1791c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.219+0000 7feb077fe700 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x175d140 0x17a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb07fff700 1 -- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x175d140 msgr2=0x17a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:47.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb07fff700 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x175d140 0x17a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb07fff700 1 -- 172.21.15.140:0/22204224 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1793790 msgr2=0x1791770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:47.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb07fff700 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1793790 0x1791770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb07fff700 1 -- 172.21.15.140:0/22204224 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb081407d0 con 0x1797970 2022-07-21T01:53:47.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb06ffd700 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1793790 0x1791770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:47.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb07fff700 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1797970 0x1791c60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x17930f0 tx=0x7feaf8008040).ready entity=mon.1 client_cookie=b30c563a91b3a142 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:47.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feae67fc700 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2022-07-21T01:53:47.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feae67fc700 1 -- 172.21.15.140:0/22204224 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaf8015820 con 0x1797970 2022-07-21T01:53:47.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feae67fc700 1 -- 172.21.15.140:0/22204224 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feaf8013e70 con 0x1797970 2022-07-21T01:53:47.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb17dee740 1 -- 172.21.15.140:0/22204224 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x16c41c0 con 0x1797970 2022-07-21T01:53:47.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feae67fc700 1 -- 172.21.15.140:0/22204224 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaf8015980 con 0x1797970 2022-07-21T01:53:47.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb17dee740 10 client.6106 fetch_fsmap learned FSMap version 26 2022-07-21T01:53:47.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb17dee740 1 -- 172.21.15.140:0/22204224 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x16bd7b0 con 0x1797970 2022-07-21T01:53:47.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feae67fc700 1 -- 172.21.15.140:0/22204224 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x16c41c0 con 0x1797970 2022-07-21T01:53:47.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feae67fc700 1 -- 172.21.15.140:0/22204224 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 26) v1 ==== 1247+0+0 (secure 0 0 0) 0x7feaf8015ae0 con 0x1797970 2022-07-21T01:53:47.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb17dee740 10 client.6106 fetch_fsmap finished waiting for FSMap version 26 2022-07-21T01:53:47.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb17dee740 10 client.6106 resolve_mds: resolved ID 'b' to GID 5899 2022-07-21T01:53:47.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb17dee740 20 client.6106 populate_metadata read hostname 'smithi140' 2022-07-21T01:53:47.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb17dee740 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:6832/74479549,v1:172.21.15.152:6833/74479549] conn(0x17a5430 0x17a5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb17dee740 4 client.6106 mds_command: new command op to 5899 tid=0[{"prefix": "get_command_descriptions"}] 2022-07-21T01:53:47.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb17dee740 1 -- 172.21.15.140:0/22204224 --> [v2:172.21.15.152:6832/74479549,v1:172.21.15.152:6833/74479549] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x16bc240 con 0x17a5430 2022-07-21T01:53:47.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb077fe700 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:6832/74479549,v1:172.21.15.152:6833/74479549] conn(0x17a5430 0x17a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feae67fc700 10 client.6106 ms_handle_connect on v2:172.21.15.152:6832/74479549 2022-07-21T01:53:47.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.223+0000 7feb077fe700 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:6832/74479549,v1:172.21.15.152:6833/74479549] conn(0x17a5430 0x17a5810 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=602af51a47d8d267 server_cookie=fd603c3e7ca93b9a in_seq=0 out_seq=0 2022-07-21T01:53:47.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feae67fc700 1 -- 172.21.15.140:0/22204224 <== mds.0 v2:172.21.15.152:6832/74479549 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x16bc240 con 0x17a5430 2022-07-21T01:53:47.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feae67fc700 10 client.6106 handle_command_reply: tid=0 2022-07-21T01:53:47.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 1 client.6106 shutdown 2022-07-21T01:53:47.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 1 -- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1797970 msgr2=0x1791c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:47.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1797970 0x1791c60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x17930f0 tx=0x7feaf8008040).stop 2022-07-21T01:53:47.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 1 -- 172.21.15.140:0/22204224 shutdown_connections 2022-07-21T01:53:47.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:6832/74479549,v1:172.21.15.152:6833/74479549] conn(0x17a5430 0x17a5810 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1797970 0x1791c60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1793790 0x1791770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 1 --2- 172.21.15.140:0/22204224 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x175d140 0x17a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 1 -- 172.21.15.140:0/22204224 >> 172.21.15.140:0/22204224 conn(0x16e58f0 msgr2=0x1797260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:47.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feae67fc700 0 client.6106 ms_handle_reset on v2:172.21.15.152:6832/74479549 2022-07-21T01:53:47.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 1 -- 172.21.15.140:0/22204224 shutdown_connections 2022-07-21T01:53:47.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 1 -- 172.21.15.140:0/22204224 wait complete. 2022-07-21T01:53:47.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.227+0000 7feb17dee740 20 client.6106 trim_cache size 0 max 16384 2022-07-21T01:53:47.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb17dee740 1 Processor -- start 2022-07-21T01:53:47.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb17dee740 1 -- start start 2022-07-21T01:53:47.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb17dee740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x175d140 0x1792200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb17dee740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1793790 0x1791620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb17dee740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x17a1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb17dee740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x16b39b0 con 0x1797970 2022-07-21T01:53:47.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb17dee740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x16dff10 con 0x175d140 2022-07-21T01:53:47.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb17dee740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x1760690 con 0x1793790 2022-07-21T01:53:47.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb07fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x17a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb07fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x17a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36182/0 (socket says 172.21.15.140:36182) 2022-07-21T01:53:47.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb07fff700 1 -- 172.21.15.140:0/2258259626 learned_addr learned my addr 172.21.15.140:0/2258259626 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:47.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb06ffd700 1 --2- 172.21.15.140:0/2258259626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1793790 0x1791620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb077fe700 1 --2- 172.21.15.140:0/2258259626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x175d140 0x1792200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb07fff700 1 -- 172.21.15.140:0/2258259626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1793790 msgr2=0x1791620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:47.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb07fff700 1 --2- 172.21.15.140:0/2258259626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1793790 0x1791620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb07fff700 1 -- 172.21.15.140:0/2258259626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x175d140 msgr2=0x1792200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:47.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb07fff700 1 --2- 172.21.15.140:0/2258259626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x175d140 0x1792200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb07fff700 1 -- 172.21.15.140:0/2258259626 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x16c63a0 con 0x1797970 2022-07-21T01:53:47.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb07fff700 1 --2- 172.21.15.140:0/2258259626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x17a1030 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x17595f0 tx=0x7feaf8025000).ready entity=mon.0 client_cookie=73baf07f1d1c40aa server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:47.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feaecff9700 1 -- 172.21.15.140:0/2258259626 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaf8014da0 con 0x1797970 2022-07-21T01:53:47.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feaecff9700 1 -- 172.21.15.140:0/2258259626 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf8000e90 con 0x1797970 2022-07-21T01:53:47.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feaecff9700 1 -- 172.21.15.140:0/2258259626 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feaf80145a0 con 0x1797970 2022-07-21T01:53:47.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb17dee740 1 -- 172.21.15.140:0/2258259626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 msgr2=0x17a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.239+0000 7feb17dee740 1 --2- 172.21.15.140:0/2258259626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x17a1030 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x17595f0 tx=0x7feaf8025000).stop 2022-07-21T01:53:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 -- 172.21.15.140:0/2258259626 shutdown_connections 2022-07-21T01:53:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 --2- 172.21.15.140:0/2258259626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x1797970 0x17a1030 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 --2- 172.21.15.140:0/2258259626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1793790 0x1791620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 --2- 172.21.15.140:0/2258259626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x175d140 0x1792200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 -- 172.21.15.140:0/2258259626 >> 172.21.15.140:0/2258259626 conn(0x16e58f0 msgr2=0x17669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:47.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 -- 172.21.15.140:0/2258259626 shutdown_connections 2022-07-21T01:53:47.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 -- 172.21.15.140:0/2258259626 wait complete. 2022-07-21T01:53:47.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 -- start start 2022-07-21T01:53:47.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 Processor -- start 2022-07-21T01:53:47.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x175d140 0x17f8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1793790 0x17f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1797970 0x17f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x17f9810 con 0x175d140 2022-07-21T01:53:47.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x17a1e60 con 0x1797970 2022-07-21T01:53:47.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x175a4d0 con 0x1793790 2022-07-21T01:53:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb17dee740 10 client.0 ll_register_callbacks cb 0x1766220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2022-07-21T01:53:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb077fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x175d140 0x17f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb077fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x175d140 0x17f8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36186/0 (socket says 172.21.15.140:36186) 2022-07-21T01:53:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb077fe700 1 -- 172.21.15.140:0/3705033624 learned_addr learned my addr 172.21.15.140:0/3705033624 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:47.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb07fff700 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1797970 0x17f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb06ffd700 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1793790 0x17f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb077fe700 1 -- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1793790 msgr2=0x17f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:47.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb077fe700 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1793790 0x17f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb077fe700 1 -- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1797970 msgr2=0x17f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:47.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb077fe700 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1797970 0x17f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:47.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb077fe700 1 -- 172.21.15.140:0/3705033624 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x16bd7b0 con 0x175d140 2022-07-21T01:53:47.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feb077fe700 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x175d140 0x17f8940 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7feb00020000 tx=0x7feb0003b280).ready entity=mon.0 client_cookie=4fe0cb6f1f198ac server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:47.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feae77fe700 10 client.0 ms_handle_connect on v2:172.21.15.140:3300/0 2022-07-21T01:53:47.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.243+0000 7feae77fe700 1 -- 172.21.15.140:0/3705033624 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feb00037070 con 0x175d140 2022-07-21T01:53:47.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feae77fe700 1 -- 172.21.15.140:0/3705033624 <== mon.0 v2:172.21.15.140:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feb00043310 con 0x175d140 2022-07-21T01:53:47.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feb17dee740 1 -- 172.21.15.140:0/3705033624 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb00046e80 con 0x175d140 2022-07-21T01:53:47.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feae77fe700 1 -- 172.21.15.140:0/3705033624 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb000434d0 con 0x175d140 2022-07-21T01:53:47.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feb17dee740 10 client.6177 fetch_fsmap learned FSMap version 26 2022-07-21T01:53:47.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feb17dee740 1 -- 172.21.15.140:0/3705033624 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x16c2d90 con 0x175d140 2022-07-21T01:53:47.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feae77fe700 1 -- 172.21.15.140:0/3705033624 <== mon.0 v2:172.21.15.140:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0004c020 con 0x175d140 2022-07-21T01:53:47.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feae77fe700 1 -- 172.21.15.140:0/3705033624 <== mon.0 v2:172.21.15.140:3300/0 5 ==== fsmap(e 26) v1 ==== 1247+0+0 (secure 0 0 0) 0x7feb0006c230 con 0x175d140 2022-07-21T01:53:47.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feb17dee740 10 client.6177 fetch_fsmap finished waiting for FSMap version 26 2022-07-21T01:53:47.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feb17dee740 10 client.6177 resolve_mds: resolved ID 'b' to GID 5899 2022-07-21T01:53:47.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feb17dee740 20 client.6177 populate_metadata read hostname 'smithi140' 2022-07-21T01:53:47.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feb17dee740 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:6832/74479549,v1:172.21.15.152:6833/74479549] conn(0x17f47d0 0x17f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:47.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feb17dee740 4 client.6177 mds_command: new command op to 5899 tid=0[{"prefix": "cache drop", "timeout": 10}] 2022-07-21T01:53:47.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feb17dee740 1 -- 172.21.15.140:0/3705033624 --> [v2:172.21.15.152:6832/74479549,v1:172.21.15.152:6833/74479549] -- command(tid 0: {"prefix": "cache drop", "timeout": 10}) v1 -- 0x15b15e0 con 0x17f47d0 2022-07-21T01:53:47.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feb06ffd700 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:6832/74479549,v1:172.21.15.152:6833/74479549] conn(0x17f47d0 0x17f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:47.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feae77fe700 10 client.6177 ms_handle_connect on v2:172.21.15.152:6832/74479549 2022-07-21T01:53:47.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:47.247+0000 7feb06ffd700 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:6832/74479549,v1:172.21.15.152:6833/74479549] conn(0x17f47d0 0x17f4bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=97ef52e3629ac7a7 server_cookie=432088930d4bfef2 in_seq=0 out_seq=0 2022-07-21T01:53:57.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feae77fe700 1 -- 172.21.15.140:0/3705033624 <== mds.0 v2:172.21.15.152:6832/74479549 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+400 (crc 0 0 0) 0x15b15e0 con 0x17f47d0 2022-07-21T01:53:57.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feae77fe700 10 client.6177 handle_command_reply: tid=0 2022-07-21T01:53:57.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 1 client.6177 shutdown 2022-07-21T01:53:57.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 1 -- 172.21.15.140:0/3705033624 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x175d140 msgr2=0x17f8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:57.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x175d140 0x17f8940 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7feb00020000 tx=0x7feb0003b280).stop 2022-07-21T01:53:57.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 1 -- 172.21.15.140:0/3705033624 shutdown_connections 2022-07-21T01:53:57.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:6832/74479549,v1:172.21.15.152:6833/74479549] conn(0x17f47d0 0x17f4bb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:57.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x1797970 0x17f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:57.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x1793790 0x17f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:57.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 1 --2- 172.21.15.140:0/3705033624 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x175d140 0x17f8940 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:57.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feae77fe700 0 client.6177 ms_handle_reset on v2:172.21.15.152:6832/74479549 2022-07-21T01:53:57.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 1 -- 172.21.15.140:0/3705033624 >> 172.21.15.140:0/3705033624 conn(0x16e58f0 msgr2=0x17918d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:57.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 1 -- 172.21.15.140:0/3705033624 shutdown_connections 2022-07-21T01:53:57.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 1 -- 172.21.15.140:0/3705033624 wait complete. 2022-07-21T01:53:57.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.483+0000 7feb17dee740 20 client.6177 trim_cache size 0 max 16384 2022-07-21T01:53:57.491 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:53:57.492 INFO:teuthology.orchestra.run.smithi140.stdout: "client_recall": { 2022-07-21T01:53:57.494 INFO:teuthology.orchestra.run.smithi140.stdout: "return_code": -110, 2022-07-21T01:53:57.494 INFO:teuthology.orchestra.run.smithi140.stdout: "message": "(110) Connection timed out", 2022-07-21T01:53:57.494 INFO:teuthology.orchestra.run.smithi140.stdout: "recalled": 240 2022-07-21T01:53:57.495 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:57.495 INFO:teuthology.orchestra.run.smithi140.stdout: "flush_journal": { 2022-07-21T01:53:57.495 INFO:teuthology.orchestra.run.smithi140.stdout: "return_code": 0, 2022-07-21T01:53:57.495 INFO:teuthology.orchestra.run.smithi140.stdout: "message": "" 2022-07-21T01:53:57.496 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:57.497 INFO:teuthology.orchestra.run.smithi140.stdout: "trim_cache": { 2022-07-21T01:53:57.497 INFO:teuthology.orchestra.run.smithi140.stdout: "trimmed": 220 2022-07-21T01:53:57.497 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:57.497 INFO:teuthology.orchestra.run.smithi140.stdout: "cache": { 2022-07-21T01:53:57.498 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": { 2022-07-21T01:53:57.498 INFO:teuthology.orchestra.run.smithi140.stdout: "items": 15434, 2022-07-21T01:53:57.498 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes": 2428352 2022-07-21T01:53:57.499 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:57.499 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:57.499 INFO:teuthology.orchestra.run.smithi140.stdout: "duration": 10.236411378 2022-07-21T01:53:57.500 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:53:57.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 -- 172.21.15.140:0/3322056135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feae8053980 msgr2=0x7feae8055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:57.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feae8053980 0x7feae8055e40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7feafc00d000 tx=0x7feafc007b40).stop 2022-07-21T01:53:57.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 -- 172.21.15.140:0/3322056135 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feb080b01e0 msgr2=0x7feb080aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:57.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feb080b01e0 0x7feb080aebb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7feb000032f0 tx=0x7feb00016000).stop 2022-07-21T01:53:57.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 -- 172.21.15.140:0/3322056135 shutdown_connections 2022-07-21T01:53:57.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7feae8053980 0x7feae8055e40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:57.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb080b62a0 0x7feb0813b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:57.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7feb080b3ed0 0x7feb080af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:57.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 --2- 172.21.15.140:0/3322056135 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feb080b01e0 0x7feb080aebb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:57.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 -- 172.21.15.140:0/3322056135 >> 172.21.15.140:0/3322056135 conn(0x7feb08066f00 msgr2=0x7feb08145110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:57.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 -- 172.21.15.140:0/3322056135 shutdown_connections 2022-07-21T01:53:57.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:57.487+0000 7feb0db97700 1 -- 172.21.15.140:0/3322056135 wait complete. 2022-07-21T01:53:57.505 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:53:57.506 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:57.527 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:57.551 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:57.573 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:57.595 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:53:57.617 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:53:57.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:57.618 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:57.619 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:57.628 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:53:57.629 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:53:57.629 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:53:57.630 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:53:57.631 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:57.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22680]: fuse finished with error 0 and tester_r 0 2022-07-21T01:53:57.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:57.687 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:57.688 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:57.695 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:53:57.696 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:53:57.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:53:57.814 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:53:57.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:57.867 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:57.868 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:53:57.876 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:53:57.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:53:57.878 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:53:57.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:53:57.880 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:53:57.886 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:53:57.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:53:57.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:53:57.888 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:53:57.889 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:53:57.896 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:53:57.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:53:57.898 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:53:57.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:53:57.900 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:53:57.945 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:53:57.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:53:57.947 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:53:58.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.063+0000 7f2457138700 1 Processor -- start 2022-07-21T01:53:58.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.063+0000 7f2457138700 1 -- start start 2022-07-21T01:53:58.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2457138700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f24500b0570 0x7f24500b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2457138700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f24500aea10 0x7f2450141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2457138700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24501433d0 con 0x7f24501422c0 2022-07-21T01:53:58.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2457138700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2450143b10 con 0x7f24500aea10 2022-07-21T01:53:58.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2457138700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2450143670 con 0x7f24500b0570 2022-07-21T01:53:58.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2455935700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f24500aea10 0x7f2450141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2456937700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f24501422c0 0x7f24500ad350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50344/0 (socket says 172.21.15.140:50344) 2022-07-21T01:53:58.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2456937700 1 -- 172.21.15.140:0/1408172724 learned_addr learned my addr 172.21.15.140:0/1408172724 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:53:58.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2456136700 1 --2- 172.21.15.140:0/1408172724 >> v2:172.21.15.152:3300/0 conn(0x7f24500b0570 0x7f24500b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2455935700 1 -- 172.21.15.140:0/1408172724 >> v2:172.21.15.152:3300/0 conn(0x7f24500b0570 msgr2=0x7f24500b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:58.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2455935700 1 --2- 172.21.15.140:0/1408172724 >> v2:172.21.15.152:3300/0 conn(0x7f24500b0570 0x7f24500b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2455935700 1 -- 172.21.15.140:0/1408172724 >> v1:172.21.15.140:6789/0 conn(0x7f24501422c0 legacy=0x7f24500ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2455935700 1 -- 172.21.15.140:0/1408172724 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2450142710 con 0x7f24500aea10 2022-07-21T01:53:58.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2455134700 1 -- 172.21.15.140:0/1408172724 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760260550 0 0) 0x7f24501433d0 con 0x7f24501422c0 2022-07-21T01:53:58.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2456136700 1 --2- 172.21.15.140:0/1408172724 >> v2:172.21.15.152:3300/0 conn(0x7f24500b0570 0x7f24500b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:53:58.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2455935700 1 --2- 172.21.15.140:0/1408172724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24500aea10 0x7f2450141dd0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f244c002f70 tx=0x7f244c006df0).ready entity=mon.0 client_cookie=4e06a197f29cd492 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:58.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2455134700 1 -- 172.21.15.140:0/1408172724 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f244c00d570 con 0x7f24500aea10 2022-07-21T01:53:58.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2455134700 1 -- 172.21.15.140:0/1408172724 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c000e90 con 0x7f24500aea10 2022-07-21T01:53:58.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.067+0000 7f2455134700 1 -- 172.21.15.140:0/1408172724 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f244c00d8c0 con 0x7f24500aea10 2022-07-21T01:53:58.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 -- 172.21.15.140:0/1408172724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24500aea10 msgr2=0x7f2450141dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 --2- 172.21.15.140:0/1408172724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24500aea10 0x7f2450141dd0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f244c002f70 tx=0x7f244c006df0).stop 2022-07-21T01:53:58.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 -- 172.21.15.140:0/1408172724 shutdown_connections 2022-07-21T01:53:58.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 --2- 172.21.15.140:0/1408172724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24500aea10 0x7f2450141dd0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 --2- 172.21.15.140:0/1408172724 >> v2:172.21.15.152:3300/0 conn(0x7f24500b0570 0x7f24500b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 -- 172.21.15.140:0/1408172724 >> 172.21.15.140:0/1408172724 conn(0x7f24500b9360 msgr2=0x7f24500b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:58.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 -- 172.21.15.140:0/1408172724 shutdown_connections 2022-07-21T01:53:58.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 -- 172.21.15.140:0/1408172724 wait complete. 2022-07-21T01:53:58.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 Processor -- start 2022-07-21T01:53:58.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 -- start start 2022-07-21T01:53:58.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24500aea10 0x7f24500b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24500b0570 0x7f245013ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24501422c0 0x7f245013f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24500b1e10 con 0x7f24500aea10 2022-07-21T01:53:58.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24500b2040 con 0x7f24501422c0 2022-07-21T01:53:58.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24500b1210 con 0x7f24500b0570 2022-07-21T01:53:58.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2456136700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24500aea10 0x7f24500b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2456136700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24500aea10 0x7f24500b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36200/0 (socket says 172.21.15.140:36200) 2022-07-21T01:53:58.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2456136700 1 -- 172.21.15.140:0/3662883130 learned_addr learned my addr 172.21.15.140:0/3662883130 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:58.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2456937700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24501422c0 0x7f245013f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2455935700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24500b0570 0x7f245013ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2456136700 1 -- 172.21.15.140:0/3662883130 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24500b0570 msgr2=0x7f245013ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2456136700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24500b0570 0x7f245013ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2456136700 1 -- 172.21.15.140:0/3662883130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24501422c0 msgr2=0x7f245013f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2456136700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24501422c0 0x7f245013f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2456136700 1 -- 172.21.15.140:0/3662883130 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244000b180 con 0x7f24500aea10 2022-07-21T01:53:58.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2455935700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24500b0570 0x7f245013ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:53:58.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2456136700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24500aea10 0x7f24500b2a20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f244000c000 tx=0x7f2440006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:58.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f243f7fe700 1 -- 172.21.15.140:0/3662883130 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f244000c890 con 0x7f24500aea10 2022-07-21T01:53:58.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f243f7fe700 1 -- 172.21.15.140:0/3662883130 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244000c9f0 con 0x7f24500aea10 2022-07-21T01:53:58.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 -- 172.21.15.140:0/3662883130 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244c00b180 con 0x7f24500aea10 2022-07-21T01:53:58.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f243f7fe700 1 -- 172.21.15.140:0/3662883130 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f244000cd50 con 0x7f24500aea10 2022-07-21T01:53:58.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.071+0000 7f2457138700 1 -- 172.21.15.140:0/3662883130 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24500b16b0 con 0x7f24500aea10 2022-07-21T01:53:58.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.075+0000 7f2457138700 1 -- 172.21.15.140:0/3662883130 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24500b13d0 con 0x7f24500aea10 2022-07-21T01:53:58.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.075+0000 7f243f7fe700 1 -- 172.21.15.140:0/3662883130 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f244001c460 con 0x7f24500aea10 2022-07-21T01:53:58.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.075+0000 7f243f7fe700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2430053a70 0x7f2430055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.075+0000 7f243f7fe700 1 -- 172.21.15.140:0/3662883130 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f244006c4c0 con 0x7f24500aea10 2022-07-21T01:53:58.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.075+0000 7f2455935700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2430053a70 0x7f2430055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.075+0000 7f2455935700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2430053a70 0x7f2430055f30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f244c018000 tx=0x7f244c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:58.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.083+0000 7f243f7fe700 1 -- 172.21.15.140:0/3662883130 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2440043380 con 0x7f24500aea10 2022-07-21T01:53:58.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.203+0000 7f2457138700 1 -- 172.21.15.140:0/3662883130 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f24500b1010 con 0x7f24500aea10 2022-07-21T01:53:58.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f243f7fe700 1 -- 172.21.15.140:0/3662883130 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v63) v1 ==== 81+0+33744 (secure 0 0 0) 0x7f24400469a0 con 0x7f24500aea10 2022-07-21T01:53:58.204 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:58.204 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:53:58.204 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 63, 2022-07-21T01:53:58.204 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:53:58.205 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:53:58.205 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:53:31.479929+0000", 2022-07-21T01:53:58.205 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:53:58.205 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:53:58.206 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:53:58.206 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:53:58.206 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:53:58.207 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:53:58.209 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:53:58.210 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:53:58.210 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:53:58.210 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:58.210 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:53:58.211 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:53:58.211 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:53:58.211 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:53:58.211 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:53:58.212 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 12, 2022-07-21T01:53:58.212 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:53:58.212 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:53:58.212 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:53:58.212 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:53:58.213 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:53:58.213 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.213 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:53:58.213 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:53:58.214 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:53:58.214 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:53:58.214 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:53:58.215 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:53:58.216 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:53:58.216 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:53:58.216 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:53:58.216 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:53:58.217 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:53:58.217 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:53:58.217 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:53:58.217 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:53:58.218 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:53:58.218 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:53:58.218 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:53:58.219 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:53:58.219 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:53:58.219 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:53:58.219 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:53:58.220 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:53:58.220 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:53:58.220 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.220 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:53:58.221 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:53:58.221 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:53:58.222 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:53:58.222 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:53:58.223 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:53:58.223 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:53:58.223 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:53:58.224 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:53:58.224 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:53:58.225 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:53:58.225 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:53:58.226 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:53:58.226 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:53:58.227 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:53:58.227 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:53:58.228 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:53:58.228 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:53:58.229 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:53:58.229 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:53:58.230 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:53:58.230 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:53:58.231 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:53:58.231 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:53:58.231 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:53:58.232 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:53:58.232 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:53:58.232 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.232 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:53:58.233 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:53:58.233 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:53:58.234 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:53:58.234 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:53:58.235 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:53:58.235 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:53:58.236 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:53:58.236 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:53:58.237 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:53:58.237 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:53:58.238 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:53:58.238 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:53:58.239 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.239 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:53:58.239 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:53:58.240 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.240 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.240 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.241 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:53:58.241 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:53:58.242 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:53:58.242 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:53:58.242 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:53:58.243 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:53:58.243 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:53:58.243 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:53:58.244 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:53:58.244 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:53:58.244 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:53:58.245 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:53:58.245 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:53:58.245 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:53:58.245 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:53:58.246 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:53:58.246 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:53:58.246 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:53:58.246 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:53:58.247 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:53:58.247 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:53:58.247 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:53:58.247 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:53:58.247 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.248 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:53:58.248 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:53:58.248 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:53:58.248 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:53:58.249 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:53:58.249 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:53:58.249 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:53:58.249 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:53:58.250 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:53:58.250 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:53:58.250 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:53:58.250 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:53:58.250 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:53:58.251 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:53:58.251 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:53:58.251 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:53:58.251 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:53:58.252 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:53:58.252 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:53:58.252 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:53:58.252 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:53:58.252 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:53:58.253 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:53:58.253 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:53:58.253 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:53:58.253 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:53:58.254 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:53:58.254 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.254 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:53:58.254 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:53:58.255 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:53:58.255 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:53:58.255 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:53:58.255 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:53:58.255 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:53:58.256 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:53:58.256 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:53:58.256 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:53:58.256 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:53:58.257 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:53:58.257 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:53:58.257 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:53:58.257 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.257 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.258 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.258 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 11, 2022-07-21T01:53:58.258 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:53:58.258 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:53:28.457127+0000", 2022-07-21T01:53:58.259 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:53:58.259 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:53:58.259 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:53:58.259 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:53:58.260 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:53:58.260 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:53:58.261 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:53:58.261 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:53:58.261 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:53:58.261 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:53:58.262 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:53:58.262 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:53:58.262 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:53:58.262 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:53:58.263 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:53:58.263 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:53:58.263 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:53:58.263 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:53:58.264 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:53:58.264 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:53:58.264 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.264 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "62", 2022-07-21T01:53:58.265 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:53:58.265 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:53:58.265 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:53:58.265 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:53:58.266 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:53:58.266 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:53:58.266 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:53:58.266 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:53:58.267 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:53:58.267 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:53:58.267 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:53:58.267 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:53:58.268 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:53:58.268 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:53:58.268 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:53:58.268 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:53:58.268 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:53:58.269 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:53:58.269 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:53:58.269 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:53:58.269 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:53:58.269 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:53:58.270 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:53:58.270 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:53:58.270 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:53:58.270 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:53:58.271 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.271 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:53:58.271 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:53:58.271 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:53:58.271 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:53:58.272 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:53:58.272 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:53:58.272 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:53:58.272 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:53:58.273 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:53:58.273 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:53:58.273 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:53:58.273 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:53:58.273 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:53:58.274 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:53:58.274 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:53:58.274 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.274 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:53:58.275 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:53:58.275 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:53:58.275 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.275 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.275 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.276 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.276 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 12, 2022-07-21T01:53:58.276 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:53:58.277 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:53:29.483080+0000", 2022-07-21T01:53:58.277 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:53:58.277 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:53:58.278 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:53:58.278 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:53:58.278 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:53:58.278 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:53:58.278 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:53:58.279 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:53:58.279 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:53:58.279 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:53:58.279 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:53:58.280 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:53:58.280 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:53:58.280 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:53:58.281 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:53:58.281 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:53:58.281 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:53:58.281 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:53:58.282 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:53:58.282 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:53:58.282 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.282 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "63", 2022-07-21T01:53:58.283 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:53:58.283 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:53:58.283 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:53:58.283 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:53:58.284 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:53:58.284 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:53:58.284 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:53:58.284 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:53:58.284 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:53:58.285 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:53:58.285 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:53:58.285 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:53:58.285 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:53:58.285 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:53:58.286 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:53:58.286 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:53:58.286 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:53:58.286 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:53:58.287 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:53:58.287 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:53:58.287 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:53:58.287 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:53:58.288 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:53:58.288 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:53:58.288 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:53:58.288 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:53:58.289 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.289 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:53:58.289 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:53:58.289 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:53:58.290 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:53:58.290 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:53:58.290 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:53:58.291 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:53:58.291 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:53:58.291 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:53:58.291 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:53:58.292 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:53:58.292 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:53:58.292 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:53:58.292 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:53:58.292 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:53:58.293 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.293 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.293 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.293 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:58.294 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:53:58.294 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.294 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:53:58.294 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:53:58.295 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:58.296 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:58.298 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:58.298 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:58.299 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:58.299 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:58.300 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:58.300 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 61, 2022-07-21T01:53:58.301 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:58.301 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:58.302 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:58.302 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.303 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.303 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.304 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:53:58.305 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:58.305 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.305 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.306 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.306 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:53:58.307 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:58.307 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.308 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.308 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.309 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:58.309 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.309 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.310 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.310 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:53:58.310 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:58.310 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.311 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.311 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.311 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:53:58.311 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:58.312 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.312 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.312 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.312 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:58.312 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.313 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.313 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.313 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:53:58.314 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:58.314 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.315 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.315 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.315 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:53:58.316 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:58.316 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.317 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.317 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.318 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:58.318 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.319 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.319 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.320 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:53:58.320 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:58.320 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.321 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.321 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.322 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:53:58.322 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:53:58.323 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.323 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.323 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.324 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:53:58.324 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:53:58.325 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:53:58.325 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:53:58.326 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:58.326 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:58.327 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:58.327 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.328 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.329 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.329 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:53:58.329 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:53:58.330 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:58.330 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:58.331 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:58.331 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:58.332 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:58.332 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:58.333 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:58.333 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 61, 2022-07-21T01:53:58.333 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:58.334 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:58.334 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:58.335 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.335 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.336 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.336 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:53:58.336 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:58.337 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.337 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.338 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.338 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:53:58.339 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:58.339 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.340 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.340 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.340 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:58.341 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.341 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.342 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.343 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:53:58.343 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:58.343 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.344 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.344 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.345 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:53:58.345 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:58.346 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.346 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.346 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.347 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:58.347 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.348 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.348 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.349 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:53:58.349 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:58.350 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.350 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.350 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.351 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:53:58.351 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:58.352 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.352 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.353 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.353 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:58.354 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.354 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.354 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.355 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:53:58.355 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:58.356 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.356 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.357 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.357 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:53:58.358 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:53:58.358 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.358 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.359 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.359 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:53:58.360 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:53:58.360 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:53:58.360 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:53:58.361 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:58.361 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:58.361 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:58.361 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.362 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.362 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.362 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:53:58.362 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:53:58.363 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:58.363 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:58.363 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:58.363 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:58.363 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:58.364 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:58.364 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:58.364 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 60, 2022-07-21T01:53:58.376 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:58.376 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:58.376 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:58.377 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.377 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.377 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.377 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:53:58.378 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:58.378 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.378 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.378 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.379 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:53:58.379 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:58.379 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.379 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.380 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.380 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:58.380 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.380 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.380 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.381 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:53:58.381 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:58.381 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.381 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.382 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.382 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:53:58.382 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:58.382 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.382 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.383 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.383 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:58.383 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.383 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.384 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.384 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:53:58.384 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:58.384 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.384 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.385 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.385 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:53:58.385 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:58.385 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.386 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.386 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.386 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:58.386 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.387 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.387 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.387 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:53:58.387 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:58.387 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.388 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.388 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.388 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:53:58.388 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:53:58.388 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.389 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.389 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.389 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:53:58.389 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:53:58.390 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:53:58.390 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:53:58.390 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:58.390 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:58.390 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:58.391 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.391 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.391 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.391 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:53:58.392 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:53:58.392 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:58.392 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:58.392 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:58.392 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:58.393 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:58.393 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:58.393 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:58.393 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 49, 2022-07-21T01:53:58.394 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:58.394 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:58.394 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:58.394 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.394 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.395 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.395 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:53:58.395 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:58.395 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.396 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.396 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.396 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:53:58.397 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:58.397 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.397 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.397 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.397 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:58.398 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.398 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.398 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.398 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:53:58.399 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:58.399 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.399 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.399 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.399 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:53:58.400 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:58.400 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.400 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.400 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.401 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:58.401 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.401 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.401 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.401 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:53:58.402 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:58.402 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.402 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.403 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.403 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:53:58.403 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:58.403 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.403 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.404 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.404 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:58.404 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.404 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.405 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.405 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:53:58.405 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:58.406 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.406 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.406 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.406 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:53:58.407 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:53:58.407 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.407 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.407 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.407 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:53:58.408 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:53:58.408 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:53:58.408 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:53:58.408 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:58.409 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:58.409 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:58.409 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.409 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.409 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.410 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:53:58.410 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:53:58.410 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:58.410 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:58.411 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:58.411 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:58.411 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:58.411 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:58.411 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:58.412 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 61, 2022-07-21T01:53:58.412 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:58.412 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:58.412 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:58.413 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.413 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.413 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.413 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:53:58.413 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:58.414 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.414 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.414 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.414 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:53:58.415 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:58.415 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.415 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.415 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.415 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:58.416 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.416 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.416 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.416 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:53:58.417 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:58.417 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.417 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.417 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.417 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:53:58.418 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:58.418 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.418 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.418 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.419 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:58.419 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.419 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.419 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.419 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:53:58.420 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:58.420 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.420 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.420 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.421 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:53:58.421 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:58.421 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.421 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.421 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.422 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:58.422 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.422 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.422 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.423 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:53:58.423 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:58.423 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.423 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.423 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.424 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:53:58.424 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:53:58.424 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.424 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.424 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.425 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:53:58.425 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:53:58.425 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:53:58.425 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:53:58.426 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:58.426 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:58.426 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:58.426 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.426 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.427 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.427 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:53:58.427 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:53:58.427 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:58.428 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:58.428 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:58.428 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:58.429 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:58.429 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:58.429 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:58.430 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 60, 2022-07-21T01:53:58.430 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:58.430 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:58.430 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:58.430 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.431 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.431 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.431 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:53:58.431 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:58.432 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.432 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.432 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.432 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:53:58.432 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:58.433 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.433 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.433 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.433 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:58.434 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.434 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.434 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.434 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:53:58.434 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:58.435 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.435 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.435 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.435 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:53:58.436 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:58.436 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.436 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.436 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.436 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:58.437 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.437 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.437 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.437 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:53:58.438 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:58.438 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.438 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.438 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.438 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:53:58.439 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:58.439 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.439 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.439 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.440 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:58.440 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.440 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.440 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.441 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:53:58.441 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:58.441 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.441 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.441 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.442 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:53:58.442 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:53:58.442 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.443 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.443 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.443 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:53:58.443 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:53:58.444 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:53:58.444 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:53:58.444 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:58.444 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:58.444 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:58.445 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.445 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.445 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.445 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:53:58.446 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:53:58.446 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:58.446 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:58.446 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:58.446 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:58.447 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:58.447 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:58.447 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:58.447 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 61, 2022-07-21T01:53:58.448 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:58.448 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:58.448 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:58.448 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.448 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.449 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.449 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:53:58.449 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:58.449 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.449 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.450 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.450 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:53:58.450 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:58.450 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.451 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.451 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.451 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:58.451 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.451 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.452 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.452 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:53:58.452 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:58.452 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.453 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.453 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.453 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:53:58.453 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:58.453 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.454 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.454 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.454 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:58.454 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.454 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.455 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.455 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:53:58.455 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:58.455 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.456 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.456 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.456 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:53:58.456 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:58.456 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.457 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.457 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.457 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:58.457 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.458 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.458 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.458 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:53:58.458 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:58.459 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.459 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.459 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.459 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:53:58.459 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:53:58.460 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.460 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.460 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.460 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:53:58.461 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:53:58.461 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:53:58.461 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:53:58.461 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:58.462 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:58.462 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:58.462 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.462 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.463 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.463 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:53:58.463 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:53:58.463 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:53:58.464 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:53:58.464 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:53:58.464 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:53:58.464 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:53:58.464 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:53:58.465 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:53:58.465 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 60, 2022-07-21T01:53:58.465 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:53:58.465 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:53:58.465 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:53:58.466 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.466 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.466 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.467 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:53:58.467 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:58.467 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.467 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.468 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.468 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:53:58.468 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:58.468 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.469 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.469 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.469 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:53:58.469 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.470 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.470 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.470 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:53:58.470 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:58.471 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.471 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.471 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.471 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:53:58.472 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:58.472 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.472 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.472 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.473 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:53:58.473 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.473 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.473 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.474 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:53:58.474 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:58.474 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.474 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.475 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.475 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:53:58.475 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:58.475 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.475 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.476 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.476 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:53:58.477 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:53:58.477 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.477 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:53:58.477 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:53:58.478 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:58.478 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.478 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.478 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:53:58.479 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:53:58.479 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:53:58.479 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.479 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.480 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.480 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:53:58.480 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:53:58.480 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:53:58.480 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:53:58.481 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:53:58.481 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:53:58.481 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:53:58.481 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:53:58.482 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.482 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:58.482 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:53:58.482 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.483 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:53:58.483 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:58.483 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:58.483 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:58.484 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:58.484 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:58.484 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:53:58.484 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:58.484 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.485 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.485 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:53:58.485 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:58.485 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:58.486 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:58.486 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:58.486 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:58.486 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:53:58.486 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:58.487 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.487 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.487 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:53:58.487 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:58.488 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:58.488 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:58.488 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:58.488 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:58.489 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:53:58.489 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:58.489 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.489 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.489 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:53:58.490 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:58.490 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:58.490 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:58.490 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:58.490 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:58.491 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:53:58.491 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:58.491 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.492 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.492 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:53:58.492 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:58.492 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:58.493 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:58.493 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:58.493 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:58.493 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:53:58.493 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:58.494 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.494 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.494 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:53:58.494 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:58.495 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:58.495 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:58.495 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:58.495 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:58.495 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:53:58.496 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:58.496 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.496 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.496 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:53:58.497 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:58.497 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:58.497 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:58.497 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:58.497 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:58.498 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:53:58.498 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:58.498 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.498 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:53:58.499 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:53:58.499 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:53:58.499 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:53:58.499 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:53:58.499 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:53:58.500 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:53:58.500 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:53:58.500 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:53:58.500 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.501 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:53:58.501 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:53:58.501 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:53:58.501 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:53:58.502 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:53:58.502 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": {}, 2022-07-21T01:53:58.502 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:53:58.502 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:53:58.503 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:53:58.503 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:53:58.503 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:53:58.503 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:53:58.504 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:53:58.504 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:53:58.504 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:53:58.504 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:53:58.505 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:53:58.505 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:53:58.505 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:53:58.505 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:53:58.506 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:53:58.506 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:53:58.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 -- 172.21.15.140:0/3662883130 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2430053a70 msgr2=0x7f2430055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2430053a70 0x7f2430055f30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f244c018000 tx=0x7f244c014000).stop 2022-07-21T01:53:58.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 -- 172.21.15.140:0/3662883130 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24500aea10 msgr2=0x7f24500b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24500aea10 0x7f24500b2a20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f244000c000 tx=0x7f2440006df0).stop 2022-07-21T01:53:58.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 -- 172.21.15.140:0/3662883130 shutdown_connections 2022-07-21T01:53:58.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2430053a70 0x7f2430055f30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24501422c0 0x7f245013f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24500b0570 0x7f245013ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 --2- 172.21.15.140:0/3662883130 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24500aea10 0x7f24500b2a20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 -- 172.21.15.140:0/3662883130 >> 172.21.15.140:0/3662883130 conn(0x7f24500b9360 msgr2=0x7f24500b7260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:58.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 -- 172.21.15.140:0/3662883130 shutdown_connections 2022-07-21T01:53:58.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.207+0000 7f2457138700 1 -- 172.21.15.140:0/3662883130 wait complete. 2022-07-21T01:53:58.511 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:53:58.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.623+0000 7f07e8a32700 1 Processor -- start 2022-07-21T01:53:58.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.623+0000 7f07e8a32700 1 -- start start 2022-07-21T01:53:58.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.623+0000 7f07e8a32700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f07e40adff0 0x7f07e40b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.623+0000 7f07e8a32700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f07e40af8e0 0x7f07e405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.623+0000 7f07e8a32700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e4147010 con 0x7f07e40b5650 2022-07-21T01:53:58.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.623+0000 7f07e8a32700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e41472b0 con 0x7f07e40adff0 2022-07-21T01:53:58.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.623+0000 7f07e8a32700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e4146d40 con 0x7f07e40af8e0 2022-07-21T01:53:58.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.623+0000 7f07e2ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f07e40adff0 0x7f07e40b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.623+0000 7f07e2ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f07e40adff0 0x7f07e40b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36210/0 (socket says 172.21.15.140:36210) 2022-07-21T01:53:58.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.623+0000 7f07e2ffd700 1 -- 172.21.15.140:0/1446131152 learned_addr learned my addr 172.21.15.140:0/1446131152 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:58.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e3fff700 1 --2- 172.21.15.140:0/1446131152 >> v2:172.21.15.152:3300/0 conn(0x7f07e40af8e0 0x7f07e405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e2ffd700 1 -- 172.21.15.140:0/1446131152 >> v2:172.21.15.152:3300/0 conn(0x7f07e40af8e0 msgr2=0x7f07e405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:58.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e2ffd700 1 --2- 172.21.15.140:0/1446131152 >> v2:172.21.15.152:3300/0 conn(0x7f07e40af8e0 0x7f07e405bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e2ffd700 1 -- 172.21.15.140:0/1446131152 >> v1:172.21.15.152:6789/0 conn(0x7f07e40b5650 legacy=0x7f07e4147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:58.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e2ffd700 1 -- 172.21.15.140:0/1446131152 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e40acae0 con 0x7f07e40adff0 2022-07-21T01:53:58.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e2ffd700 1 --2- 172.21.15.140:0/1446131152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07e40adff0 0x7f07e40b5a30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f07e4060b70 tx=0x7f07d4006e50).ready entity=mon.0 client_cookie=e2ddcf61154114e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:58.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e27fc700 1 -- 172.21.15.140:0/1446131152 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d400d5c0 con 0x7f07e40adff0 2022-07-21T01:53:58.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e27fc700 1 -- 172.21.15.140:0/1446131152 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d4000e90 con 0x7f07e40adff0 2022-07-21T01:53:58.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e27fc700 1 -- 172.21.15.140:0/1446131152 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07d400d950 con 0x7f07e40adff0 2022-07-21T01:53:58.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e8a32700 1 -- 172.21.15.140:0/1446131152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07e40adff0 msgr2=0x7f07e40b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e8a32700 1 --2- 172.21.15.140:0/1446131152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07e40adff0 0x7f07e40b5a30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f07e4060b70 tx=0x7f07d4006e50).stop 2022-07-21T01:53:58.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e8a32700 1 -- 172.21.15.140:0/1446131152 shutdown_connections 2022-07-21T01:53:58.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e8a32700 1 --2- 172.21.15.140:0/1446131152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07e40adff0 0x7f07e40b5a30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e8a32700 1 --2- 172.21.15.140:0/1446131152 >> v2:172.21.15.152:3300/0 conn(0x7f07e40af8e0 0x7f07e405bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e8a32700 1 -- 172.21.15.140:0/1446131152 >> 172.21.15.140:0/1446131152 conn(0x7f07e40634e0 msgr2=0x7f07e40b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:58.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e8a32700 1 -- 172.21.15.140:0/1446131152 shutdown_connections 2022-07-21T01:53:58.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.627+0000 7f07e8a32700 1 -- 172.21.15.140:0/1446131152 wait complete. 2022-07-21T01:53:58.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e8a32700 1 Processor -- start 2022-07-21T01:53:58.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e8a32700 1 -- start start 2022-07-21T01:53:58.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e8a32700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e40adff0 0x7f07e4066a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e8a32700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07e40af8e0 0x7f07e41599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e8a32700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07e40b5650 0x7f07e4159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e8a32700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e4067270 con 0x7f07e40b5650 2022-07-21T01:53:58.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e8a32700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e4067510 con 0x7f07e40adff0 2022-07-21T01:53:58.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e8a32700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e4066f30 con 0x7f07e40af8e0 2022-07-21T01:53:58.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07e40b5650 0x7f07e4159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07e40b5650 0x7f07e4159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36218/0 (socket says 172.21.15.140:36218) 2022-07-21T01:53:58.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e3fff700 1 -- 172.21.15.140:0/2717013411 learned_addr learned my addr 172.21.15.140:0/2717013411 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:58.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e37fe700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e40adff0 0x7f07e4066a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e2ffd700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07e40af8e0 0x7f07e41599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e3fff700 1 -- 172.21.15.140:0/2717013411 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07e40af8e0 msgr2=0x7f07e41599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e3fff700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07e40af8e0 0x7f07e41599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e3fff700 1 -- 172.21.15.140:0/2717013411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e40adff0 msgr2=0x7f07e4066a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e3fff700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e40adff0 0x7f07e4066a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e3fff700 1 -- 172.21.15.140:0/2717013411 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d400b1e0 con 0x7f07e40b5650 2022-07-21T01:53:58.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e3fff700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07e40b5650 0x7f07e4159ee0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f07dc00a000 tx=0x7f07dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:58.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e0ff9700 1 -- 172.21.15.140:0/2717013411 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07dc013620 con 0x7f07e40b5650 2022-07-21T01:53:58.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e0ff9700 1 -- 172.21.15.140:0/2717013411 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc013780 con 0x7f07e40b5650 2022-07-21T01:53:58.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e8a32700 1 -- 172.21.15.140:0/2717013411 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e4067920 con 0x7f07e40b5650 2022-07-21T01:53:58.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e0ff9700 1 -- 172.21.15.140:0/2717013411 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07dc013ae0 con 0x7f07e40b5650 2022-07-21T01:53:58.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.631+0000 7f07e8a32700 1 -- 172.21.15.140:0/2717013411 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e4063fb0 con 0x7f07e40b5650 2022-07-21T01:53:58.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.635+0000 7f07e8a32700 1 -- 172.21.15.140:0/2717013411 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e40574e0 con 0x7f07e40b5650 2022-07-21T01:53:58.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.635+0000 7f07e0ff9700 1 -- 172.21.15.140:0/2717013411 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f07dc013c40 con 0x7f07e40b5650 2022-07-21T01:53:58.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.635+0000 7f07e0ff9700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07c8053a10 0x7f07c8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.635+0000 7f07e0ff9700 1 -- 172.21.15.140:0/2717013411 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f07dc06bd00 con 0x7f07e40b5650 2022-07-21T01:53:58.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.635+0000 7f07e37fe700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07c8053a10 0x7f07c8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.635+0000 7f07e37fe700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07c8053a10 0x7f07c8055ed0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f07cc00c000 tx=0x7f07cc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:58.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.639+0000 7f07e0ff9700 1 -- 172.21.15.140:0/2717013411 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07dc042ab0 con 0x7f07e40b5650 2022-07-21T01:53:58.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 -- 172.21.15.140:0/2717013411 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07e4067ea0 con 0x7f07e40b5650 2022-07-21T01:53:58.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e0ff9700 1 -- 172.21.15.140:0/2717013411 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+2137 (secure 0 0 0) 0x7f07dc0460d0 con 0x7f07e40b5650 2022-07-21T01:53:58.799 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:53:58.800 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":26,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/689662045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":689662045},{"type":"v1","addr":"172.21.15.140:6835","nonce":689662045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:53:30.476186+0000","modified":"2022-07-21T01:53:31.479177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5899":{"gid":5899,"name":"b","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/74479549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":74479549},{"type":"v1","addr":"172.21.15.152:6833","nonce":74479549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-07-21T01:53:58.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 -- 172.21.15.140:0/2717013411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07c8053a10 msgr2=0x7f07c8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07c8053a10 0x7f07c8055ed0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f07cc00c000 tx=0x7f07cc008040).stop 2022-07-21T01:53:58.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 -- 172.21.15.140:0/2717013411 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07e40b5650 msgr2=0x7f07e4159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07e40b5650 0x7f07e4159ee0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f07dc00a000 tx=0x7f07dc008040).stop 2022-07-21T01:53:58.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 -- 172.21.15.140:0/2717013411 shutdown_connections 2022-07-21T01:53:58.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07c8053a10 0x7f07c8055ed0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07e40b5650 0x7f07e4159ee0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07e40af8e0 0x7f07e41599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 --2- 172.21.15.140:0/2717013411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07e40adff0 0x7f07e4066a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 -- 172.21.15.140:0/2717013411 >> 172.21.15.140:0/2717013411 conn(0x7f07e40634e0 msgr2=0x7f07e40b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:58.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 -- 172.21.15.140:0/2717013411 shutdown_connections 2022-07-21T01:53:58.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.803+0000 7f07e8a32700 1 -- 172.21.15.140:0/2717013411 wait complete. 2022-07-21T01:53:58.808 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 26 2022-07-21T01:53:58.812 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:53:58.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.919+0000 7f93bbfff700 1 Processor -- start 2022-07-21T01:53:58.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.919+0000 7f93bbfff700 1 -- start start 2022-07-21T01:53:58.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.919+0000 7f93bbfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f93bc0b1b00 0x7f93bc146280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.919+0000 7f93bbfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f93bc1429d0 0x7f93bc146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.919+0000 7f93bbfff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93bc05d060 con 0x7f93bc0b1230 2022-07-21T01:53:58.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.919+0000 7f93bbfff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93bc0b1610 con 0x7f93bc1429d0 2022-07-21T01:53:58.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.919+0000 7f93bbfff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93bc05ca50 con 0x7f93bc0b1b00 2022-07-21T01:53:58.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.919+0000 7f93bb7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f93bc1429d0 0x7f93bc146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.919+0000 7f93bb7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f93bc1429d0 0x7f93bc146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36226/0 (socket says 172.21.15.140:36226) 2022-07-21T01:53:58.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.919+0000 7f93bb7fe700 1 -- 172.21.15.140:0/2246959548 learned_addr learned my addr 172.21.15.140:0/2246959548 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:58.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93ba7fc700 1 --2- 172.21.15.140:0/2246959548 >> v2:172.21.15.152:3300/0 conn(0x7f93bc0b1b00 0x7f93bc146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bb7fe700 1 -- 172.21.15.140:0/2246959548 >> v2:172.21.15.152:3300/0 conn(0x7f93bc0b1b00 msgr2=0x7f93bc146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:53:58.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bb7fe700 1 --2- 172.21.15.140:0/2246959548 >> v2:172.21.15.152:3300/0 conn(0x7f93bc0b1b00 0x7f93bc146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bb7fe700 1 -- 172.21.15.140:0/2246959548 >> v1:172.21.15.152:6789/0 conn(0x7f93bc0b1230 legacy=0x7f93bc13c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bb7fe700 1 -- 172.21.15.140:0/2246959548 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc067450 con 0x7f93bc1429d0 2022-07-21T01:53:58.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bb7fe700 1 --2- 172.21.15.140:0/2246959548 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93bc1429d0 0x7f93bc146770 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f93b40032f0 tx=0x7f93b4009290).ready entity=mon.0 client_cookie=ee30e84be105d06e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:58.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93b9ffb700 1 -- 172.21.15.140:0/2246959548 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93b400b870 con 0x7f93bc1429d0 2022-07-21T01:53:58.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93b9ffb700 1 -- 172.21.15.140:0/2246959548 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b400b9d0 con 0x7f93bc1429d0 2022-07-21T01:53:58.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93b9ffb700 1 -- 172.21.15.140:0/2246959548 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93b400bb40 con 0x7f93bc1429d0 2022-07-21T01:53:58.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bbfff700 1 -- 172.21.15.140:0/2246959548 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93bc1429d0 msgr2=0x7f93bc146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bbfff700 1 --2- 172.21.15.140:0/2246959548 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93bc1429d0 0x7f93bc146770 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f93b40032f0 tx=0x7f93b4009290).stop 2022-07-21T01:53:58.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bbfff700 1 -- 172.21.15.140:0/2246959548 shutdown_connections 2022-07-21T01:53:58.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bbfff700 1 --2- 172.21.15.140:0/2246959548 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93bc1429d0 0x7f93bc146770 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bbfff700 1 --2- 172.21.15.140:0/2246959548 >> v2:172.21.15.152:3300/0 conn(0x7f93bc0b1b00 0x7f93bc146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bbfff700 1 -- 172.21.15.140:0/2246959548 >> 172.21.15.140:0/2246959548 conn(0x7f93bc064c00 msgr2=0x7f93bc064fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:53:58.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bbfff700 1 -- 172.21.15.140:0/2246959548 shutdown_connections 2022-07-21T01:53:58.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bbfff700 1 -- 172.21.15.140:0/2246959548 wait complete. 2022-07-21T01:53:58.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bbfff700 1 Processor -- start 2022-07-21T01:53:58.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.923+0000 7f93bbfff700 1 -- start start 2022-07-21T01:53:58.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93bbfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93bc0b1230 0x7f93bc140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93bbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93bc0b1b00 0x7f93bc113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93bbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93bc1429d0 0x7f93bc113660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93bbfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1521c0 con 0x7f93bc0b1b00 2022-07-21T01:53:58.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93bbfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc152460 con 0x7f93bc1429d0 2022-07-21T01:53:58.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93bbfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc152700 con 0x7f93bc0b1230 2022-07-21T01:53:58.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93ba7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93bc0b1b00 0x7f93bc113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93ba7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93bc0b1b00 0x7f93bc113170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36230/0 (socket says 172.21.15.140:36230) 2022-07-21T01:53:58.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93ba7fc700 1 -- 172.21.15.140:0/1931279219 learned_addr learned my addr 172.21.15.140:0/1931279219 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:53:58.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93bb7fe700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93bc1429d0 0x7f93bc113660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93baffd700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93bc0b1230 0x7f93bc140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93ba7fc700 1 -- 172.21.15.140:0/1931279219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93bc0b1230 msgr2=0x7f93bc140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93ba7fc700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93bc0b1230 0x7f93bc140c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93ba7fc700 1 -- 172.21.15.140:0/1931279219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93bc1429d0 msgr2=0x7f93bc113660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:53:58.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93ba7fc700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93bc1429d0 0x7f93bc113660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:53:58.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93ba7fc700 1 -- 172.21.15.140:0/1931279219 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b4009530 con 0x7f93bc0b1b00 2022-07-21T01:53:58.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93ba7fc700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93bc0b1b00 0x7f93bc113170 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f93b000c000 tx=0x7f93b0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:58.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93abfff700 1 -- 172.21.15.140:0/1931279219 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93b0010780 con 0x7f93bc0b1b00 2022-07-21T01:53:58.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93abfff700 1 -- 172.21.15.140:0/1931279219 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b00108e0 con 0x7f93bc0b1b00 2022-07-21T01:53:58.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93bbfff700 1 -- 172.21.15.140:0/1931279219 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc141100 con 0x7f93bc0b1b00 2022-07-21T01:53:58.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93abfff700 1 -- 172.21.15.140:0/1931279219 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93b0010c40 con 0x7f93bc0b1b00 2022-07-21T01:53:58.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93bbfff700 1 -- 172.21.15.140:0/1931279219 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93bc145d10 con 0x7f93bc0b1b00 2022-07-21T01:53:58.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.927+0000 7f93bbfff700 1 -- 172.21.15.140:0/1931279219 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93bc1528c0 con 0x7f93bc0b1b00 2022-07-21T01:53:58.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.931+0000 7f93abfff700 1 -- 172.21.15.140:0/1931279219 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f93b001c460 con 0x7f93bc0b1b00 2022-07-21T01:53:58.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.931+0000 7f93abfff700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f93980539e0 0x7f9398055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:53:58.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.931+0000 7f93abfff700 1 -- 172.21.15.140:0/1931279219 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f93b006c280 con 0x7f93bc0b1b00 2022-07-21T01:53:58.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.931+0000 7f93baffd700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f93980539e0 0x7f9398055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:53:58.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.931+0000 7f93baffd700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f93980539e0 0x7f9398055ea0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f93ac00d000 tx=0x7f93ac007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:53:58.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:58.935+0000 7f93abfff700 1 -- 172.21.15.140:0/1931279219 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f93b00430a0 con 0x7f93bc0b1b00 2022-07-21T01:53:59.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:53:59.095+0000 7f93bbfff700 1 -- 172.21.15.140:0/1931279219 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f93bc0676f0 con 0x7f93bc0b1b00 2022-07-21T01:54:00.063 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:54:00.067+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:54:00.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.079+0000 7f93abfff700 1 -- 172.21.15.140:0/1931279219 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7f93b00466c0 con 0x7f93bc0b1b00 2022-07-21T01:54:00.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 -- 172.21.15.140:0/1931279219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f93980539e0 msgr2=0x7f9398055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:00.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f93980539e0 0x7f9398055ea0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f93ac00d000 tx=0x7f93ac007b90).stop 2022-07-21T01:54:00.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 -- 172.21.15.140:0/1931279219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93bc0b1b00 msgr2=0x7f93bc113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:00.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93bc0b1b00 0x7f93bc113170 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f93b000c000 tx=0x7f93b0006df0).stop 2022-07-21T01:54:00.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 -- 172.21.15.140:0/1931279219 shutdown_connections 2022-07-21T01:54:00.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f93980539e0 0x7f9398055ea0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93bc1429d0 0x7f93bc113660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93bc0b1b00 0x7f93bc113170 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 --2- 172.21.15.140:0/1931279219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93bc0b1230 0x7f93bc140c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 -- 172.21.15.140:0/1931279219 >> 172.21.15.140:0/1931279219 conn(0x7f93bc064c00 msgr2=0x7f93bc060f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:00.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 -- 172.21.15.140:0/1931279219 shutdown_connections 2022-07-21T01:54:00.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.083+0000 7f93bbfff700 1 -- 172.21.15.140:0/1931279219 wait complete. 2022-07-21T01:54:00.083 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:54:00.096 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:54:00.110 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:54:00.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 Processor -- start 2022-07-21T01:54:00.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 -- start start 2022-07-21T01:54:00.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1d00b5650 0x7fb1d00b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb1d00adff0 0x7fb1d0147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d0147010 con 0x7fb1d00af8e0 2022-07-21T01:54:00.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d01472b0 con 0x7fb1d00adff0 2022-07-21T01:54:00.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d0146d40 con 0x7fb1d00b5650 2022-07-21T01:54:00.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1cf7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb1d00adff0 0x7fb1d0147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1cf7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb1d00adff0 0x7fb1d0147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36236/0 (socket says 172.21.15.140:36236) 2022-07-21T01:54:00.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1cf7fe700 1 -- 172.21.15.140:0/2553062537 learned_addr learned my addr 172.21.15.140:0/2553062537 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:00.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1cffff700 1 --2- 172.21.15.140:0/2553062537 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1d00b5650 0x7fb1d00b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1cf7fe700 1 -- 172.21.15.140:0/2553062537 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1d00b5650 msgr2=0x7fb1d00b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:00.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1cf7fe700 1 --2- 172.21.15.140:0/2553062537 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1d00b5650 0x7fb1d00b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1cf7fe700 1 -- 172.21.15.140:0/2553062537 >> v1:172.21.15.152:6789/0 conn(0x7fb1d00af8e0 legacy=0x7fb1d005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:00.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1cf7fe700 1 -- 172.21.15.140:0/2553062537 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0147c40 con 0x7fb1d00adff0 2022-07-21T01:54:00.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1cffff700 1 --2- 172.21.15.140:0/2553062537 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1d00b5650 0x7fb1d00b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:54:00.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1cf7fe700 1 --2- 172.21.15.140:0/2553062537 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1d00adff0 0x7fb1d0147750 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fb1d0060b70 tx=0x7fb1b8009250).ready entity=mon.0 client_cookie=b49c7f56ca691a9a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:00.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1ceffd700 1 -- 172.21.15.140:0/2553062537 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b800b860 con 0x7fb1d00adff0 2022-07-21T01:54:00.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1ceffd700 1 -- 172.21.15.140:0/2553062537 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b800b9c0 con 0x7fb1d00adff0 2022-07-21T01:54:00.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1ceffd700 1 -- 172.21.15.140:0/2553062537 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b800bb70 con 0x7fb1d00adff0 2022-07-21T01:54:00.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2553062537 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1d00adff0 msgr2=0x7fb1d0147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:00.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 --2- 172.21.15.140:0/2553062537 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1d00adff0 0x7fb1d0147750 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fb1d0060b70 tx=0x7fb1b8009250).stop 2022-07-21T01:54:00.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2553062537 shutdown_connections 2022-07-21T01:54:00.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 --2- 172.21.15.140:0/2553062537 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1d00adff0 0x7fb1d0147750 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 --2- 172.21.15.140:0/2553062537 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1d00b5650 0x7fb1d00b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.207+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2553062537 >> 172.21.15.140:0/2553062537 conn(0x7fb1d00634e0 msgr2=0x7fb1d00b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:00.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2553062537 shutdown_connections 2022-07-21T01:54:00.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2553062537 wait complete. 2022-07-21T01:54:00.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 Processor -- start 2022-07-21T01:54:00.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 -- start start 2022-07-21T01:54:00.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1d00adff0 0x7fb1d00643a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1d00af8e0 0x7fb1d0144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1d00b5650 0x7fb1d0142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d0110f50 con 0x7fb1d00adff0 2022-07-21T01:54:00.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d0111180 con 0x7fb1d00b5650 2022-07-21T01:54:00.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d0063f00 con 0x7fb1d00af8e0 2022-07-21T01:54:00.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1d00adff0 0x7fb1d00643a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cf7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1d00af8e0 0x7fb1d0144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1d00adff0 0x7fb1d00643a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36244/0 (socket says 172.21.15.140:36244) 2022-07-21T01:54:00.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cffff700 1 -- 172.21.15.140:0/2739071218 learned_addr learned my addr 172.21.15.140:0/2739071218 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:00.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d4da7700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1d00b5650 0x7fb1d0142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cffff700 1 -- 172.21.15.140:0/2739071218 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1d00af8e0 msgr2=0x7fb1d0144cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:00.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cffff700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1d00af8e0 0x7fb1d0144cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cffff700 1 -- 172.21.15.140:0/2739071218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1d00b5650 msgr2=0x7fb1d0142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:00.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cffff700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1d00b5650 0x7fb1d0142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cffff700 1 -- 172.21.15.140:0/2739071218 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b80094d0 con 0x7fb1d00adff0 2022-07-21T01:54:00.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cffff700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1d00adff0 0x7fb1d00643a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fb1c0007000 tx=0x7fb1c0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:00.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cd7fa700 1 -- 172.21.15.140:0/2739071218 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1c000ce80 con 0x7fb1d00adff0 2022-07-21T01:54:00.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cd7fa700 1 -- 172.21.15.140:0/2739071218 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c0010580 con 0x7fb1d00adff0 2022-07-21T01:54:00.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2739071218 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d0067890 con 0x7fb1d00adff0 2022-07-21T01:54:00.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1cd7fa700 1 -- 172.21.15.140:0/2739071218 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1c00108e0 con 0x7fb1d00adff0 2022-07-21T01:54:00.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.211+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2739071218 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d0063b30 con 0x7fb1d00adff0 2022-07-21T01:54:00.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.215+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2739071218 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d005c4a0 con 0x7fb1d00adff0 2022-07-21T01:54:00.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.215+0000 7fb1cd7fa700 1 -- 172.21.15.140:0/2739071218 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb1c0010a40 con 0x7fb1d00adff0 2022-07-21T01:54:00.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.215+0000 7fb1cd7fa700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb1b40539e0 0x7fb1b4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.215+0000 7fb1cd7fa700 1 -- 172.21.15.140:0/2739071218 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb1c006bbd0 con 0x7fb1d00adff0 2022-07-21T01:54:00.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.215+0000 7fb1cf7fe700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb1b40539e0 0x7fb1b4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.215+0000 7fb1cf7fe700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb1b40539e0 0x7fb1b4055ea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fb1b801b000 tx=0x7fb1b8016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:00.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.223+0000 7fb1cd7fa700 1 -- 172.21.15.140:0/2739071218 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb1c0042aa0 con 0x7fb1d00adff0 2022-07-21T01:54:00.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2739071218 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1d00574e0 con 0x7fb1d00adff0 2022-07-21T01:54:00.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1cd7fa700 1 -- 172.21.15.140:0/2739071218 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+1779 (secure 0 0 0) 0x7fb1c0020250 con 0x7fb1d00adff0 2022-07-21T01:54:00.376 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:00.376 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/689662045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":689662045},{"type":"v1","addr":"172.21.15.140:6835","nonce":689662045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:53:30.476186+0000","modified":"2022-07-21T01:54:00.070365+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2022-07-21T01:54:00.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2739071218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb1b40539e0 msgr2=0x7fb1b4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:00.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb1b40539e0 0x7fb1b4055ea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fb1b801b000 tx=0x7fb1b8016000).stop 2022-07-21T01:54:00.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2739071218 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1d00adff0 msgr2=0x7fb1d00643a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:00.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1d00adff0 0x7fb1d00643a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fb1c0007000 tx=0x7fb1c0008040).stop 2022-07-21T01:54:00.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2739071218 shutdown_connections 2022-07-21T01:54:00.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb1b40539e0 0x7fb1b4055ea0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1d00b5650 0x7fb1d0142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1d00af8e0 0x7fb1d0144cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 --2- 172.21.15.140:0/2739071218 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1d00adff0 0x7fb1d00643a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2739071218 >> 172.21.15.140:0/2739071218 conn(0x7fb1d00634e0 msgr2=0x7fb1d00ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:00.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2739071218 shutdown_connections 2022-07-21T01:54:00.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.379+0000 7fb1d55a8700 1 -- 172.21.15.140:0/2739071218 wait complete. 2022-07-21T01:54:00.381 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 27 2022-07-21T01:54:00.388 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:54:00.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd07d39d700 1 Processor -- start 2022-07-21T01:54:00.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd07d39d700 1 -- start start 2022-07-21T01:54:00.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd07d39d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0780b6800 0x7fd0780aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd07d39d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd0780ad0e0 0x7fd0781418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd07d39d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0780ae280 con 0x7fd0780b3cb0 2022-07-21T01:54:00.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd07d39d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0780b6be0 con 0x7fd0780ad0e0 2022-07-21T01:54:00.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd07d39d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780ae4b0 con 0x7fd0780b6800 2022-07-21T01:54:00.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd07cb9c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd0780ad0e0 0x7fd0781418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd0777fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd0780b3cb0 0x7fd078145f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50396/0 (socket says 172.21.15.140:50396) 2022-07-21T01:54:00.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd0777fe700 1 -- 172.21.15.140:0/1732054231 learned_addr learned my addr 172.21.15.140:0/1732054231 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:00.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd077fff700 1 --2- 172.21.15.140:0/1732054231 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0780b6800 0x7fd0780aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd076ffd700 1 -- 172.21.15.140:0/1732054231 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212528966 0 0) 0x7fd0780ae280 con 0x7fd0780b3cb0 2022-07-21T01:54:00.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd076ffd700 1 -- 172.21.15.140:0/1732054231 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd058003260 con 0x7fd0780b3cb0 2022-07-21T01:54:00.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd077fff700 1 -- 172.21.15.140:0/1732054231 >> v2:172.21.15.140:3300/0 conn(0x7fd0780ad0e0 msgr2=0x7fd0781418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:00.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd077fff700 1 --2- 172.21.15.140:0/1732054231 >> v2:172.21.15.140:3300/0 conn(0x7fd0780ad0e0 0x7fd0781418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd077fff700 1 -- 172.21.15.140:0/1732054231 >> v1:172.21.15.140:6789/0 conn(0x7fd0780b3cb0 legacy=0x7fd078145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:00.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd077fff700 1 -- 172.21.15.140:0/1732054231 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780ae750 con 0x7fd0780b6800 2022-07-21T01:54:00.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd07cb9c700 1 --2- 172.21.15.140:0/1732054231 >> v2:172.21.15.140:3300/0 conn(0x7fd0780ad0e0 0x7fd0781418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:00.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd077fff700 1 --2- 172.21.15.140:0/1732054231 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0780b6800 0x7fd0780aebf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd068002f70 tx=0x7fd068006df0).ready entity=mon.2 client_cookie=43d2c0e1f3a0b37f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:00.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd076ffd700 1 -- 172.21.15.140:0/1732054231 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd06800d4a0 con 0x7fd0780b6800 2022-07-21T01:54:00.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd076ffd700 1 -- 172.21.15.140:0/1732054231 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd068000e90 con 0x7fd0780b6800 2022-07-21T01:54:00.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.499+0000 7fd076ffd700 1 -- 172.21.15.140:0/1732054231 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd06800d7f0 con 0x7fd0780b6800 2022-07-21T01:54:00.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.503+0000 7fd07d39d700 1 -- 172.21.15.140:0/1732054231 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0780b6800 msgr2=0x7fd0780aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:00.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.503+0000 7fd07d39d700 1 --2- 172.21.15.140:0/1732054231 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0780b6800 0x7fd0780aebf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fd068002f70 tx=0x7fd068006df0).stop 2022-07-21T01:54:00.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.503+0000 7fd07d39d700 1 -- 172.21.15.140:0/1732054231 shutdown_connections 2022-07-21T01:54:00.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.503+0000 7fd07d39d700 1 --2- 172.21.15.140:0/1732054231 >> v2:172.21.15.140:3300/0 conn(0x7fd0780ad0e0 0x7fd0781418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.503+0000 7fd07d39d700 1 --2- 172.21.15.140:0/1732054231 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0780b6800 0x7fd0780aebf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.503+0000 7fd07d39d700 1 -- 172.21.15.140:0/1732054231 >> 172.21.15.140:0/1732054231 conn(0x7fd0780b9dd0 msgr2=0x7fd0780b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:00.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.503+0000 7fd07d39d700 1 -- 172.21.15.140:0/1732054231 shutdown_connections 2022-07-21T01:54:00.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.503+0000 7fd07d39d700 1 -- 172.21.15.140:0/1732054231 wait complete. 2022-07-21T01:54:00.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.503+0000 7fd07d39d700 1 Processor -- start 2022-07-21T01:54:00.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.503+0000 7fd07d39d700 1 -- start start 2022-07-21T01:54:00.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd07d39d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0780ad0e0 0x7fd078113440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd07d39d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd0780b3cb0 0x7fd078115f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd07d39d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0780b6800 0x7fd078141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd07d39d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd078113dc0 con 0x7fd0780b3cb0 2022-07-21T01:54:00.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd07d39d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd078114060 con 0x7fd0780ad0e0 2022-07-21T01:54:00.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd07d39d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd078116430 con 0x7fd0780b6800 2022-07-21T01:54:00.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd0777fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd0780b3cb0 0x7fd078115f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd0777fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd0780b3cb0 0x7fd078115f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36258/0 (socket says 172.21.15.140:36258) 2022-07-21T01:54:00.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd0777fe700 1 -- 172.21.15.140:0/1589983096 learned_addr learned my addr 172.21.15.140:0/1589983096 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:00.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd077fff700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0780ad0e0 0x7fd078113440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd07cb9c700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0780b6800 0x7fd078141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd077fff700 1 -- 172.21.15.140:0/1589983096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0780b6800 msgr2=0x7fd078141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:00.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd077fff700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0780b6800 0x7fd078141b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd077fff700 1 -- 172.21.15.140:0/1589983096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd0780b3cb0 msgr2=0x7fd078115f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:00.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd077fff700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd0780b3cb0 0x7fd078115f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:00.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd077fff700 1 -- 172.21.15.140:0/1589983096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd070009470 con 0x7fd0780ad0e0 2022-07-21T01:54:00.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd0777fe700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd0780b3cb0 0x7fd078115f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:00.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd077fff700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0780ad0e0 0x7fd078113440 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fd068002f70 tx=0x7fd068014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:00.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd0757fa700 1 -- 172.21.15.140:0/1589983096 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd068016070 con 0x7fd0780ad0e0 2022-07-21T01:54:00.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd0757fa700 1 -- 172.21.15.140:0/1589983096 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0680137d0 con 0x7fd0780ad0e0 2022-07-21T01:54:00.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd07d39d700 1 -- 172.21.15.140:0/1589983096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06800b180 con 0x7fd0780ad0e0 2022-07-21T01:54:00.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd0757fa700 1 -- 172.21.15.140:0/1589983096 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd068013b30 con 0x7fd0780ad0e0 2022-07-21T01:54:00.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.507+0000 7fd07d39d700 1 -- 172.21.15.140:0/1589983096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0780b2ee0 con 0x7fd0780ad0e0 2022-07-21T01:54:00.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.511+0000 7fd0757fa700 1 -- 172.21.15.140:0/1589983096 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd06800f030 con 0x7fd0780ad0e0 2022-07-21T01:54:00.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.511+0000 7fd07d39d700 1 -- 172.21.15.140:0/1589983096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd078112840 con 0x7fd0780ad0e0 2022-07-21T01:54:00.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.511+0000 7fd0757fa700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd05c053b00 0x7fd05c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:00.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.511+0000 7fd0757fa700 1 -- 172.21.15.140:0/1589983096 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd06806ccf0 con 0x7fd0780ad0e0 2022-07-21T01:54:00.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.511+0000 7fd0777fe700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd05c053b00 0x7fd05c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:00.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.511+0000 7fd0777fe700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd05c053b00 0x7fd05c055fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fd06000d000 tx=0x7fd060009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:00.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.519+0000 7fd0757fa700 1 -- 172.21.15.140:0/1589983096 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd078112840 con 0x7fd0780ad0e0 2022-07-21T01:54:00.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:00.683+0000 7fd07d39d700 1 -- 172.21.15.140:0/1589983096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd078112840 con 0x7fd0780ad0e0 2022-07-21T01:54:01.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd0757fa700 1 -- 172.21.15.140:0/1589983096 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd078112840 con 0x7fd0780ad0e0 2022-07-21T01:54:01.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 -- 172.21.15.140:0/1589983096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd05c053b00 msgr2=0x7fd05c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:01.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd05c053b00 0x7fd05c055fc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fd06000d000 tx=0x7fd060009040).stop 2022-07-21T01:54:01.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 -- 172.21.15.140:0/1589983096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0780ad0e0 msgr2=0x7fd078113440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:01.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0780ad0e0 0x7fd078113440 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7fd068002f70 tx=0x7fd068014000).stop 2022-07-21T01:54:01.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 -- 172.21.15.140:0/1589983096 shutdown_connections 2022-07-21T01:54:01.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd05c053b00 0x7fd05c055fc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:01.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0780b6800 0x7fd078141b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:01.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd0780b3cb0 0x7fd078115f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:01.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 --2- 172.21.15.140:0/1589983096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0780ad0e0 0x7fd078113440 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:01.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 -- 172.21.15.140:0/1589983096 >> 172.21.15.140:0/1589983096 conn(0x7fd0780b9dd0 msgr2=0x7fd078145e50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:01.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 -- 172.21.15.140:0/1589983096 shutdown_connections 2022-07-21T01:54:01.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.087+0000 7fd07d39d700 1 -- 172.21.15.140:0/1589983096 wait complete. 2022-07-21T01:54:01.099 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:54:01.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.215+0000 7fb7389dd700 1 Processor -- start 2022-07-21T01:54:01.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.215+0000 7fb7389dd700 1 -- start start 2022-07-21T01:54:01.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7340b5990 0x7fb7340af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:01.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb7341400f0 0x7fb7340ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:01.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7340b5d70 con 0x7fb7340b1400 2022-07-21T01:54:01.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7340acb80 con 0x7fb7341400f0 2022-07-21T01:54:01.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340ace20 con 0x7fb7340b5990 2022-07-21T01:54:01.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb732d9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7340b5990 0x7fb7340af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:01.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb72a59c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb7341400f0 0x7fb7340ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:01.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb732d9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7340b5990 0x7fb7340af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:32994/0 (socket says 172.21.15.140:32994) 2022-07-21T01:54:01.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb732d9d700 1 -- 172.21.15.140:0/2062442036 learned_addr learned my addr 172.21.15.140:0/2062442036 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:01.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb73359e700 1 --1- 172.21.15.140:0/2062442036 >> v1:172.21.15.152:6789/0 conn(0x7fb7340b1400 0x7fb73413c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:51526/0 (socket says 172.21.15.140:51526) 2022-07-21T01:54:01.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb72a59c700 1 -- 172.21.15.140:0/2062442036 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7340b5990 msgr2=0x7fb7340af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:01.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb72a59c700 1 --2- 172.21.15.140:0/2062442036 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7340b5990 0x7fb7340af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:01.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb72a59c700 1 -- 172.21.15.140:0/2062442036 >> v1:172.21.15.152:6789/0 conn(0x7fb7340b1400 legacy=0x7fb73413c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:01.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb72a59c700 1 -- 172.21.15.140:0/2062442036 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7341405e0 con 0x7fb7341400f0 2022-07-21T01:54:01.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb72a59c700 1 --2- 172.21.15.140:0/2062442036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7341400f0 0x7fb7340ad2c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fb71c002f70 tx=0x7fb71c006df0).ready entity=mon.1 client_cookie=3a2ae6f64497a5b8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:01.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb73259c700 1 -- 172.21.15.140:0/2062442036 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb71c014070 con 0x7fb7341400f0 2022-07-21T01:54:01.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb73259c700 1 -- 172.21.15.140:0/2062442036 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c000e90 con 0x7fb7341400f0 2022-07-21T01:54:01.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb73259c700 1 -- 172.21.15.140:0/2062442036 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb71c00fb60 con 0x7fb7341400f0 2022-07-21T01:54:01.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 -- 172.21.15.140:0/2062442036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7341400f0 msgr2=0x7fb7340ad2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:01.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 --2- 172.21.15.140:0/2062442036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7341400f0 0x7fb7340ad2c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fb71c002f70 tx=0x7fb71c006df0).stop 2022-07-21T01:54:01.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 -- 172.21.15.140:0/2062442036 shutdown_connections 2022-07-21T01:54:01.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 --2- 172.21.15.140:0/2062442036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7341400f0 0x7fb7340ad2c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:01.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 --2- 172.21.15.140:0/2062442036 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7340b5990 0x7fb7340af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:01.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 -- 172.21.15.140:0/2062442036 >> 172.21.15.140:0/2062442036 conn(0x7fb734063480 msgr2=0x7fb7340b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:01.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 -- 172.21.15.140:0/2062442036 shutdown_connections 2022-07-21T01:54:01.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.219+0000 7fb7389dd700 1 -- 172.21.15.140:0/2062442036 wait complete. 2022-07-21T01:54:01.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb7389dd700 1 Processor -- start 2022-07-21T01:54:01.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb7389dd700 1 -- start start 2022-07-21T01:54:01.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb7389dd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7340b1400 0x7fb73413dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:01.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb7389dd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7340b5990 0x7fb7341436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:01.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb7389dd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7341400f0 0x7fb73413e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:01.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb7389dd700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb734141260 con 0x7fb7341400f0 2022-07-21T01:54:01.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb7389dd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73413d960 con 0x7fb7340b1400 2022-07-21T01:54:01.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb7389dd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73413dc00 con 0x7fb7340b5990 2022-07-21T01:54:01.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb73359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7341400f0 0x7fb73413e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:01.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb732d9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7340b1400 0x7fb73413dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:01.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb73359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7341400f0 0x7fb73413e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36274/0 (socket says 172.21.15.140:36274) 2022-07-21T01:54:01.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb73359e700 1 -- 172.21.15.140:0/1259989617 learned_addr learned my addr 172.21.15.140:0/1259989617 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:01.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb72a59c700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7340b5990 0x7fb7341436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:01.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb73359e700 1 -- 172.21.15.140:0/1259989617 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7340b5990 msgr2=0x7fb7341436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:01.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb73359e700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7340b5990 0x7fb7341436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:01.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb73359e700 1 -- 172.21.15.140:0/1259989617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7340b1400 msgr2=0x7fb73413dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:01.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb73359e700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7340b1400 0x7fb73413dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:01.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb73359e700 1 -- 172.21.15.140:0/1259989617 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c00b180 con 0x7fb7341400f0 2022-07-21T01:54:01.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.223+0000 7fb73359e700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7341400f0 0x7fb73413e650 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fb7340ad1a0 tx=0x7fb72c007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:01.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.227+0000 7fb730d99700 1 -- 172.21.15.140:0/1259989617 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb72c004ab0 con 0x7fb7341400f0 2022-07-21T01:54:01.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.227+0000 7fb730d99700 1 -- 172.21.15.140:0/1259989617 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c004c10 con 0x7fb7341400f0 2022-07-21T01:54:01.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.227+0000 7fb7389dd700 1 -- 172.21.15.140:0/1259989617 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7340b04d0 con 0x7fb7341400f0 2022-07-21T01:54:01.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.227+0000 7fb730d99700 1 -- 172.21.15.140:0/1259989617 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb72c007170 con 0x7fb7341400f0 2022-07-21T01:54:01.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.227+0000 7fb7389dd700 1 -- 172.21.15.140:0/1259989617 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7340b0a20 con 0x7fb7341400f0 2022-07-21T01:54:01.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.227+0000 7fb7389dd700 1 -- 172.21.15.140:0/1259989617 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb734141da0 con 0x7fb7341400f0 2022-07-21T01:54:01.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.227+0000 7fb730d99700 1 -- 172.21.15.140:0/1259989617 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb72c01b070 con 0x7fb7341400f0 2022-07-21T01:54:01.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.227+0000 7fb730d99700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb70c053b00 0x7fb70c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:01.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.227+0000 7fb730d99700 1 -- 172.21.15.140:0/1259989617 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb72c06b740 con 0x7fb7341400f0 2022-07-21T01:54:01.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.227+0000 7fb732d9d700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb70c053b00 0x7fb70c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:01.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.231+0000 7fb732d9d700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb70c053b00 0x7fb70c055fc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fb724006000 tx=0x7fb724006e50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:01.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.231+0000 7fb730d99700 1 -- 172.21.15.140:0/1259989617 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb72c042570 con 0x7fb7341400f0 2022-07-21T01:54:01.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:01.351+0000 7fb7389dd700 1 -- 172.21.15.140:0/1259989617 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb73413ddc0 con 0x7fb7341400f0 2022-07-21T01:54:01.925 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:54:01.929 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:54:01.933 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:54:01.938 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:54:01.940 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:54:02.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.095+0000 7fb730d99700 1 -- 172.21.15.140:0/1259989617 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) v1 ==== 185+0+0 (secure 0 0 0) 0x7fb72c045b90 con 0x7fb7341400f0 2022-07-21T01:54:02.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 -- 172.21.15.140:0/1259989617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb70c053b00 msgr2=0x7fb70c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:02.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb70c053b00 0x7fb70c055fc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fb724006000 tx=0x7fb724006e50).stop 2022-07-21T01:54:02.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 -- 172.21.15.140:0/1259989617 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7341400f0 msgr2=0x7fb73413e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:02.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7341400f0 0x7fb73413e650 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fb7340ad1a0 tx=0x7fb72c007ef0).stop 2022-07-21T01:54:02.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 -- 172.21.15.140:0/1259989617 shutdown_connections 2022-07-21T01:54:02.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb70c053b00 0x7fb70c055fc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:02.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb7341400f0 0x7fb73413e650 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:02.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7340b5990 0x7fb7341436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:02.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 --2- 172.21.15.140:0/1259989617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7340b1400 0x7fb73413dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:02.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 -- 172.21.15.140:0/1259989617 >> 172.21.15.140:0/1259989617 conn(0x7fb734063480 msgr2=0x7fb734139340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:02.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 -- 172.21.15.140:0/1259989617 shutdown_connections 2022-07-21T01:54:02.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.099+0000 7fb7389dd700 1 -- 172.21.15.140:0/1259989617 wait complete. 2022-07-21T01:54:02.099 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:54:02.109 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:54:02.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.227+0000 7ff005a79700 1 Processor -- start 2022-07-21T01:54:02.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.227+0000 7ff005a79700 1 -- start start 2022-07-21T01:54:02.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff0000b6200 0x7ff0000b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:02.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff0000b49f0 0x7ff0000b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:02.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0000b06c0 0x7ff0000b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:02.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0000b4640 con 0x7ff0000b6200 2022-07-21T01:54:02.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0000b17f0 con 0x7ff0000b49f0 2022-07-21T01:54:02.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000b5190 con 0x7ff0000b06c0 2022-07-21T01:54:02.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff004a77700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff0000b6200 0x7ff0000b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:02.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7feffffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff0000b49f0 0x7ff0000b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:02.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff004a77700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff0000b6200 0x7ff0000b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36278/0 (socket says 172.21.15.140:36278) 2022-07-21T01:54:02.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005278700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0000b06c0 0x7ff0000b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:02.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff004a77700 1 -- 172.21.15.140:0/1818137110 learned_addr learned my addr 172.21.15.140:0/1818137110 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:02.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff004a77700 1 -- 172.21.15.140:0/1818137110 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0000b06c0 msgr2=0x7ff0000b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:02.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff004a77700 1 --2- 172.21.15.140:0/1818137110 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0000b06c0 0x7ff0000b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:02.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff004a77700 1 -- 172.21.15.140:0/1818137110 >> v2:172.21.15.152:3300/0 conn(0x7ff0000b49f0 msgr2=0x7ff0000b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:02.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff004a77700 1 --2- 172.21.15.140:0/1818137110 >> v2:172.21.15.152:3300/0 conn(0x7ff0000b49f0 0x7ff0000b01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:02.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff004a77700 1 -- 172.21.15.140:0/1818137110 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0000b65e0 con 0x7ff0000b6200 2022-07-21T01:54:02.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7feffffff700 1 --2- 172.21.15.140:0/1818137110 >> v2:172.21.15.152:3300/0 conn(0x7ff0000b49f0 0x7ff0000b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:02.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff004a77700 1 --2- 172.21.15.140:0/1818137110 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0000b6200 0x7ff0000b4080 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7feff0002f70 tx=0x7feff0006eb0).ready entity=mon.0 client_cookie=f60f2f7065ae168d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:02.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7fefff7fe700 1 -- 172.21.15.140:0/1818137110 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff000d570 con 0x7ff0000b6200 2022-07-21T01:54:02.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7fefff7fe700 1 -- 172.21.15.140:0/1818137110 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff0000e90 con 0x7ff0000b6200 2022-07-21T01:54:02.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7fefff7fe700 1 -- 172.21.15.140:0/1818137110 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff000d900 con 0x7ff0000b6200 2022-07-21T01:54:02.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 -- 172.21.15.140:0/1818137110 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0000b6200 msgr2=0x7ff0000b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:02.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 --2- 172.21.15.140:0/1818137110 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0000b6200 0x7ff0000b4080 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7feff0002f70 tx=0x7feff0006eb0).stop 2022-07-21T01:54:02.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 -- 172.21.15.140:0/1818137110 shutdown_connections 2022-07-21T01:54:02.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 --2- 172.21.15.140:0/1818137110 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0000b06c0 0x7ff0000b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:02.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 --2- 172.21.15.140:0/1818137110 >> v2:172.21.15.152:3300/0 conn(0x7ff0000b49f0 0x7ff0000b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:02.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 --2- 172.21.15.140:0/1818137110 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0000b6200 0x7ff0000b4080 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:02.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 -- 172.21.15.140:0/1818137110 >> 172.21.15.140:0/1818137110 conn(0x7ff000063d10 msgr2=0x7ff0000b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:02.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 -- 172.21.15.140:0/1818137110 shutdown_connections 2022-07-21T01:54:02.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.231+0000 7ff005a79700 1 -- 172.21.15.140:0/1818137110 wait complete. 2022-07-21T01:54:02.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7ff005a79700 1 Processor -- start 2022-07-21T01:54:02.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7ff005a79700 1 -- start start 2022-07-21T01:54:02.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7ff005a79700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0000b06c0 0x7ff000061070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:02.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7ff005a79700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0000b49f0 0x7ff0000ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:02.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7ff005a79700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0000b6200 0x7ff0000ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:02.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7ff005a79700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff000141900 con 0x7ff0000b49f0 2022-07-21T01:54:02.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7ff005a79700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00013f970 con 0x7ff0000b6200 2022-07-21T01:54:02.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7ff005a79700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff000140940 con 0x7ff0000b06c0 2022-07-21T01:54:02.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7feffffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0000b49f0 0x7ff0000ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:02.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7feffffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0000b49f0 0x7ff0000ad4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36286/0 (socket says 172.21.15.140:36286) 2022-07-21T01:54:02.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7feffffff700 1 -- 172.21.15.140:0/87383790 learned_addr learned my addr 172.21.15.140:0/87383790 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:02.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7ff004a77700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0000b06c0 0x7ff000061070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:02.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7ff005278700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0000b6200 0x7ff0000ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:02.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7feffffff700 1 -- 172.21.15.140:0/87383790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0000b06c0 msgr2=0x7ff000061070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:02.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7feffffff700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0000b06c0 0x7ff000061070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:02.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7feffffff700 1 -- 172.21.15.140:0/87383790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0000b6200 msgr2=0x7ff0000ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:02.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7feffffff700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0000b6200 0x7ff0000ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:02.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.235+0000 7feffffff700 1 -- 172.21.15.140:0/87383790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff400b180 con 0x7ff0000b49f0 2022-07-21T01:54:02.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7feffffff700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0000b49f0 0x7ff0000ad4d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7feff400c000 tx=0x7feff4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:02.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7feffdffb700 1 -- 172.21.15.140:0/87383790 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff400c7e0 con 0x7ff0000b49f0 2022-07-21T01:54:02.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7feffdffb700 1 -- 172.21.15.140:0/87383790 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff400c940 con 0x7ff0000b49f0 2022-07-21T01:54:02.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7ff005a79700 1 -- 172.21.15.140:0/87383790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff000b1e0 con 0x7ff0000b49f0 2022-07-21T01:54:02.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7feffdffb700 1 -- 172.21.15.140:0/87383790 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feff400cca0 con 0x7ff0000b49f0 2022-07-21T01:54:02.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7ff005a79700 1 -- 172.21.15.140:0/87383790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff000066530 con 0x7ff0000b49f0 2022-07-21T01:54:02.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7ff005a79700 1 -- 172.21.15.140:0/87383790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff000141a70 con 0x7ff0000b49f0 2022-07-21T01:54:02.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7feffdffb700 1 -- 172.21.15.140:0/87383790 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7feff401c460 con 0x7ff0000b49f0 2022-07-21T01:54:02.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7feffdffb700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fefe0053a10 0x7fefe0055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:02.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7feffdffb700 1 -- 172.21.15.140:0/87383790 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5721+0+0 (secure 0 0 0) 0x7feff406c2c0 con 0x7ff0000b49f0 2022-07-21T01:54:02.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7ff004a77700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fefe0053a10 0x7fefe0055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:02.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.239+0000 7ff004a77700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fefe0053a10 0x7fefe0055ed0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7feff001c000 tx=0x7feff0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:02.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.243+0000 7feffdffb700 1 -- 172.21.15.140:0/87383790 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff000141a70 con 0x7ff0000b49f0 2022-07-21T01:54:02.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:02.371+0000 7ff005a79700 1 -- 172.21.15.140:0/87383790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff000140b00 con 0x7ff0000b49f0 2022-07-21T01:54:03.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.099+0000 7feffdffb700 1 -- 172.21.15.140:0/87383790 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff000140b00 con 0x7ff0000b49f0 2022-07-21T01:54:03.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 -- 172.21.15.140:0/87383790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fefe0053a10 msgr2=0x7fefe0055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fefe0053a10 0x7fefe0055ed0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7feff001c000 tx=0x7feff0015000).stop 2022-07-21T01:54:03.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 -- 172.21.15.140:0/87383790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0000b49f0 msgr2=0x7ff0000ad4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0000b49f0 0x7ff0000ad4d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7feff400c000 tx=0x7feff4006df0).stop 2022-07-21T01:54:03.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 -- 172.21.15.140:0/87383790 shutdown_connections 2022-07-21T01:54:03.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fefe0053a10 0x7fefe0055ed0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0000b6200 0x7ff0000ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0000b49f0 0x7ff0000ad4d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 --2- 172.21.15.140:0/87383790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0000b06c0 0x7ff000061070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 -- 172.21.15.140:0/87383790 >> 172.21.15.140:0/87383790 conn(0x7ff000063d10 msgr2=0x7ff0000b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:03.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 -- 172.21.15.140:0/87383790 shutdown_connections 2022-07-21T01:54:03.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.103+0000 7ff005a79700 1 -- 172.21.15.140:0/87383790 wait complete. 2022-07-21T01:54:03.107 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:54:03.115 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-07-21T01:54:03.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.227+0000 7f7d11ca4700 1 Processor -- start 2022-07-21T01:54:03.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.227+0000 7f7d11ca4700 1 -- start start 2022-07-21T01:54:03.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d11ca4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7d0c142660 0x7f7d0c0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:03.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d11ca4700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0c143550 con 0x7f7d0c0b1460 2022-07-21T01:54:03.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d11ca4700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0c142b70 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d11ca4700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d0c142e10 con 0x7f7d0c142660 2022-07-21T01:54:03.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d0b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f7d0c0b1460 0x7f7d0c0af090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50438/0 (socket says 172.21.15.140:50438) 2022-07-21T01:54:03.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d0b7fe700 1 -- 172.21.15.140:0/1966188290 learned_addr learned my addr 172.21.15.140:0/1966188290 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:03.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d02ffd700 1 --2- 172.21.15.140:0/1966188290 >> v2:172.21.15.140:3300/0 conn(0x7f7d0c142660 0x7f7d0c0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:03.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d0affd700 1 -- 172.21.15.140:0/1966188290 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1659470324 0 0) 0x7f7d0c143550 con 0x7f7d0c0b1460 2022-07-21T01:54:03.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d0affd700 1 -- 172.21.15.140:0/1966188290 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cec003260 con 0x7f7d0c0b1460 2022-07-21T01:54:03.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d0affd700 1 -- 172.21.15.140:0/1966188290 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584076034 0 0) 0x7f7d0c142b70 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d0affd700 1 -- 172.21.15.140:0/1966188290 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d0c143550 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d0affd700 1 -- 172.21.15.140:0/1966188290 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1216210334 0 0) 0x7f7cec003260 con 0x7f7d0c0b1460 2022-07-21T01:54:03.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d02ffd700 1 -- 172.21.15.140:0/1966188290 >> v1:172.21.15.152:6789/0 conn(0x7f7d0c0b1e60 legacy=0x7f7d0c13f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d02ffd700 1 -- 172.21.15.140:0/1966188290 >> v1:172.21.15.140:6789/0 conn(0x7f7d0c0b1460 legacy=0x7f7d0c0af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d02ffd700 1 -- 172.21.15.140:0/1966188290 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c1430b0 con 0x7f7d0c142660 2022-07-21T01:54:03.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d02ffd700 1 --2- 172.21.15.140:0/1966188290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d0c142660 0x7f7d0c0af7a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f7cfc001110 tx=0x7f7cf4006e80).ready entity=mon.0 client_cookie=25b756f1387d1ea6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:03.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d0affd700 1 -- 172.21.15.140:0/1966188290 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cf4014070 con 0x7f7d0c142660 2022-07-21T01:54:03.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d0affd700 1 -- 172.21.15.140:0/1966188290 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf4000e90 con 0x7f7d0c142660 2022-07-21T01:54:03.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d0affd700 1 -- 172.21.15.140:0/1966188290 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cfc001660 con 0x7f7d0c142660 2022-07-21T01:54:03.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d11ca4700 1 -- 172.21.15.140:0/1966188290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d0c142660 msgr2=0x7f7d0c0af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d11ca4700 1 --2- 172.21.15.140:0/1966188290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d0c142660 0x7f7d0c0af7a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f7cfc001110 tx=0x7f7cf4006e80).stop 2022-07-21T01:54:03.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d11ca4700 1 -- 172.21.15.140:0/1966188290 shutdown_connections 2022-07-21T01:54:03.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d11ca4700 1 --2- 172.21.15.140:0/1966188290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d0c142660 0x7f7d0c0af7a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d11ca4700 1 -- 172.21.15.140:0/1966188290 >> 172.21.15.140:0/1966188290 conn(0x7f7d0c0b23b0 msgr2=0x7f7d0c0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:03.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d11ca4700 1 -- 172.21.15.140:0/1966188290 shutdown_connections 2022-07-21T01:54:03.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.231+0000 7f7d11ca4700 1 -- 172.21.15.140:0/1966188290 wait complete. 2022-07-21T01:54:03.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d11ca4700 1 Processor -- start 2022-07-21T01:54:03.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d11ca4700 1 -- start start 2022-07-21T01:54:03.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d11ca4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d0c0b1460 0x7f7d0c066890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:03.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d11ca4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d0c0b1e60 0x7f7d0c0662b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:03.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d11ca4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d0c142660 0x7f7d0c13f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:03.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d11ca4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c13acc0 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d11ca4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c066ee0 con 0x7f7d0c0b1460 2022-07-21T01:54:03.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d11ca4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c14de40 con 0x7f7d0c142660 2022-07-21T01:54:03.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d02ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d0c0b1e60 0x7f7d0c0662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:03.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d02ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d0c0b1e60 0x7f7d0c0662b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36300/0 (socket says 172.21.15.140:36300) 2022-07-21T01:54:03.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d02ffd700 1 -- 172.21.15.140:0/2302777160 learned_addr learned my addr 172.21.15.140:0/2302777160 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:03.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d0b7fe700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d0c0b1460 0x7f7d0c066890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:03.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d0bfff700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d0c142660 0x7f7d0c13f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:03.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d02ffd700 1 -- 172.21.15.140:0/2302777160 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d0c142660 msgr2=0x7f7d0c13f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d02ffd700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d0c142660 0x7f7d0c13f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d02ffd700 1 -- 172.21.15.140:0/2302777160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d0c0b1460 msgr2=0x7f7d0c066890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d02ffd700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d0c0b1460 0x7f7d0c066890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d02ffd700 1 -- 172.21.15.140:0/2302777160 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf400c310 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d02ffd700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d0c0b1e60 0x7f7d0c0662b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f7cfc001110 tx=0x7f7d0c142d00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:03.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d097fa700 1 -- 172.21.15.140:0/2302777160 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cf4010b90 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d097fa700 1 -- 172.21.15.140:0/2302777160 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf4010cf0 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d11ca4700 1 -- 172.21.15.140:0/2302777160 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0c13f590 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d097fa700 1 -- 172.21.15.140:0/2302777160 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7cf401b620 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.235+0000 7f7d11ca4700 1 -- 172.21.15.140:0/2302777160 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0c063150 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.239+0000 7f7d11ca4700 1 -- 172.21.15.140:0/2302777160 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0c0574e0 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.239+0000 7f7d097fa700 1 -- 172.21.15.140:0/2302777160 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7cf4021030 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.239+0000 7f7d097fa700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cf0053aa0 0x7f7cf0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:03.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.239+0000 7f7d097fa700 1 -- 172.21.15.140:0/2302777160 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f7cf406cb30 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.239+0000 7f7d0b7fe700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cf0053aa0 0x7f7cf0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:03.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.239+0000 7f7d0b7fe700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cf0053aa0 0x7f7cf0055f60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f7cfc00d000 tx=0x7f7cfc000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:03.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.247+0000 7f7d097fa700 1 -- 172.21.15.140:0/2302777160 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7cf4043c20 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.371+0000 7f7d11ca4700 1 -- 172.21.15.140:0/2302777160 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7d0c0636e0 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d097fa700 1 -- 172.21.15.140:0/2302777160 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7cf4047240 con 0x7f7d0c0b1e60 2022-07-21T01:54:03.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 -- 172.21.15.140:0/2302777160 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cf0053aa0 msgr2=0x7f7cf0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cf0053aa0 0x7f7cf0055f60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f7cfc00d000 tx=0x7f7cfc000bc0).stop 2022-07-21T01:54:03.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 -- 172.21.15.140:0/2302777160 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d0c0b1e60 msgr2=0x7f7d0c0662b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d0c0b1e60 0x7f7d0c0662b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f7cfc001110 tx=0x7f7d0c142d00).stop 2022-07-21T01:54:03.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 -- 172.21.15.140:0/2302777160 shutdown_connections 2022-07-21T01:54:03.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cf0053aa0 0x7f7cf0055f60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d0c142660 0x7f7d0c13f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d0c0b1e60 0x7f7d0c0662b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 --2- 172.21.15.140:0/2302777160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d0c0b1460 0x7f7d0c066890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 -- 172.21.15.140:0/2302777160 >> 172.21.15.140:0/2302777160 conn(0x7f7d0c0b23b0 msgr2=0x7f7d0c060140 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:03.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 -- 172.21.15.140:0/2302777160 shutdown_connections 2022-07-21T01:54:03.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.379+0000 7f7d11ca4700 1 -- 172.21.15.140:0/2302777160 wait complete. 2022-07-21T01:54:03.388 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2022-07-21T01:54:03.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f1610a40700 1 Processor -- start 2022-07-21T01:54:03.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f1610a40700 1 -- start start 2022-07-21T01:54:03.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f1610a40700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0ace90 0x7f160c0aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:03.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f1610a40700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160c0b5f10 con 0x7f160c0b13e0 2022-07-21T01:54:03.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f1610a40700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160c0afc40 con 0x7f160c0b5b30 2022-07-21T01:54:03.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f1610a40700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c0afee0 con 0x7f160c0ace90 2022-07-21T01:54:03.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f160ad9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f160c0b13e0 0x7f160c1463d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50456/0 (socket says 172.21.15.140:50456) 2022-07-21T01:54:03.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f160ad9d700 1 -- 172.21.15.140:0/1198102276 learned_addr learned my addr 172.21.15.140:0/1198102276 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:03.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f1609d9b700 1 --2- 172.21.15.140:0/1198102276 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0ace90 0x7f160c0aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:03.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f160959a700 1 -- 172.21.15.140:0/1198102276 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747307214 0 0) 0x7f160c0b5f10 con 0x7f160c0b13e0 2022-07-21T01:54:03.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f160959a700 1 -- 172.21.15.140:0/1198102276 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15f0003260 con 0x7f160c0b13e0 2022-07-21T01:54:03.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f160959a700 1 -- 172.21.15.140:0/1198102276 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632051787 0 0) 0x7f160c0afc40 con 0x7f160c0b5b30 2022-07-21T01:54:03.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f160959a700 1 -- 172.21.15.140:0/1198102276 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f160c0b5f10 con 0x7f160c0b5b30 2022-07-21T01:54:03.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f160959a700 1 -- 172.21.15.140:0/1198102276 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3874903571 0 0) 0x7f15f0003260 con 0x7f160c0b13e0 2022-07-21T01:54:03.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f1609d9b700 1 -- 172.21.15.140:0/1198102276 >> v1:172.21.15.152:6789/0 conn(0x7f160c0b5b30 legacy=0x7f160c0b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f1609d9b700 1 -- 172.21.15.140:0/1198102276 >> v1:172.21.15.140:6789/0 conn(0x7f160c0b13e0 legacy=0x7f160c1463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f1609d9b700 1 -- 172.21.15.140:0/1198102276 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c0ad490 con 0x7f160c0ace90 2022-07-21T01:54:03.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f1609d9b700 1 --2- 172.21.15.140:0/1198102276 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0ace90 0x7f160c0aefb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f1604001110 tx=0x7f15fc0091d0).ready entity=mon.2 client_cookie=d9062ce3e4c51502 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:03.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f160959a700 1 -- 172.21.15.140:0/1198102276 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1604001660 con 0x7f160c0ace90 2022-07-21T01:54:03.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.487+0000 7f160959a700 1 -- 172.21.15.140:0/1198102276 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15fc00f510 con 0x7f160c0ace90 2022-07-21T01:54:03.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f160959a700 1 -- 172.21.15.140:0/1198102276 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15fc00f680 con 0x7f160c0ace90 2022-07-21T01:54:03.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 -- 172.21.15.140:0/1198102276 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0ace90 msgr2=0x7f160c0aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 --2- 172.21.15.140:0/1198102276 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0ace90 0x7f160c0aefb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f1604001110 tx=0x7f15fc0091d0).stop 2022-07-21T01:54:03.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 -- 172.21.15.140:0/1198102276 shutdown_connections 2022-07-21T01:54:03.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 --2- 172.21.15.140:0/1198102276 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0ace90 0x7f160c0aefb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 -- 172.21.15.140:0/1198102276 >> 172.21.15.140:0/1198102276 conn(0x7f160c0635b0 msgr2=0x7f160c0b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:03.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 -- 172.21.15.140:0/1198102276 shutdown_connections 2022-07-21T01:54:03.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 -- 172.21.15.140:0/1198102276 wait complete. 2022-07-21T01:54:03.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 Processor -- start 2022-07-21T01:54:03.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 -- start start 2022-07-21T01:54:03.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f160c0ace90 0x7f160c144710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:03.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0b13e0 0x7f160c064070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:03.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f160c0b5b30 0x7f160c066220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:03.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c144c00 con 0x7f160c0ace90 2022-07-21T01:54:03.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c156e10 con 0x7f160c0b5b30 2022-07-21T01:54:03.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1610a40700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c157060 con 0x7f160c0b13e0 2022-07-21T01:54:03.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f160a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f160c0ace90 0x7f160c144710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:03.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f160a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f160c0ace90 0x7f160c144710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36312/0 (socket says 172.21.15.140:36312) 2022-07-21T01:54:03.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f160a59c700 1 -- 172.21.15.140:0/3749400567 learned_addr learned my addr 172.21.15.140:0/3749400567 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:03.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1609d9b700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0b13e0 0x7f160c064070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:03.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f160ad9d700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f160c0b5b30 0x7f160c066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:03.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f160a59c700 1 -- 172.21.15.140:0/3749400567 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0b13e0 msgr2=0x7f160c064070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f160a59c700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0b13e0 0x7f160c064070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f160a59c700 1 -- 172.21.15.140:0/3749400567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f160c0b5b30 msgr2=0x7f160c066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:03.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f160a59c700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f160c0b5b30 0x7f160c066220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:03.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f160a59c700 1 -- 172.21.15.140:0/3749400567 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc009470 con 0x7f160c0ace90 2022-07-21T01:54:03.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f160ad9d700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f160c0b5b30 0x7f160c066220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:03.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.491+0000 7f1609d9b700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0b13e0 0x7f160c064070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:03.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f160a59c700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f160c0ace90 0x7f160c144710 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f160c143df0 tx=0x7f1600007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:03.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f15fb7fe700 1 -- 172.21.15.140:0/3749400567 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f160000ce00 con 0x7f160c0ace90 2022-07-21T01:54:03.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f15fb7fe700 1 -- 172.21.15.140:0/3749400567 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160000f470 con 0x7f160c0ace90 2022-07-21T01:54:03.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f1610a40700 1 -- 172.21.15.140:0/3749400567 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160c157300 con 0x7f160c0ace90 2022-07-21T01:54:03.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f15fb7fe700 1 -- 172.21.15.140:0/3749400567 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f160000f7d0 con 0x7f160c0ace90 2022-07-21T01:54:03.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f1610a40700 1 -- 172.21.15.140:0/3749400567 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160c145000 con 0x7f160c0ace90 2022-07-21T01:54:03.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f1610a40700 1 -- 172.21.15.140:0/3749400567 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f160c0574e0 con 0x7f160c0ace90 2022-07-21T01:54:03.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f15fb7fe700 1 -- 172.21.15.140:0/3749400567 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1600019030 con 0x7f160c0ace90 2022-07-21T01:54:03.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f15fb7fe700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15e8053a70 0x7f15e8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:03.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f15fb7fe700 1 -- 172.21.15.140:0/3749400567 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f160006b090 con 0x7f160c0ace90 2022-07-21T01:54:03.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f1609d9b700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15e8053a70 0x7f15e8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:03.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.495+0000 7f1609d9b700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15e8053a70 0x7f15e8055f30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f15fc019000 tx=0x7f15fc016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:03.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.503+0000 7f15fb7fe700 1 -- 172.21.15.140:0/3749400567 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1600042290 con 0x7f160c0ace90 2022-07-21T01:54:03.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:03.667+0000 7f1610a40700 1 -- 172.21.15.140:0/3749400567 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f160c067ea0 con 0x7f160c0ace90 2022-07-21T01:54:04.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.111+0000 7f15fb7fe700 1 -- 172.21.15.140:0/3749400567 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v242) v1 ==== 147+0+0 (secure 0 0 0) 0x7f16000458b0 con 0x7f160c0ace90 2022-07-21T01:54:04.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 -- 172.21.15.140:0/3749400567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15e8053a70 msgr2=0x7f15e8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:04.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15e8053a70 0x7f15e8055f30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f15fc019000 tx=0x7f15fc016000).stop 2022-07-21T01:54:04.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 -- 172.21.15.140:0/3749400567 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f160c0ace90 msgr2=0x7f160c144710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:04.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f160c0ace90 0x7f160c144710 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f160c143df0 tx=0x7f1600007ae0).stop 2022-07-21T01:54:04.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 -- 172.21.15.140:0/3749400567 shutdown_connections 2022-07-21T01:54:04.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f15e8053a70 0x7f15e8055f30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:04.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f160c0b5b30 0x7f160c066220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:04.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f160c0b13e0 0x7f160c064070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:04.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 --2- 172.21.15.140:0/3749400567 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f160c0ace90 0x7f160c144710 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:04.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 -- 172.21.15.140:0/3749400567 >> 172.21.15.140:0/3749400567 conn(0x7f160c0635b0 msgr2=0x7f160c148660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:04.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 -- 172.21.15.140:0/3749400567 shutdown_connections 2022-07-21T01:54:04.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.115+0000 7f1610a40700 1 -- 172.21.15.140:0/3749400567 wait complete. 2022-07-21T01:54:04.128 INFO:tasks.cephfs_test_runner:test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2022-07-21T01:54:04.129 INFO:tasks.cephfs_test_runner:Starting test: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) 2022-07-21T01:54:04.130 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2022-07-21T01:54:04.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.239+0000 7fd75ffff700 1 Processor -- start 2022-07-21T01:54:04.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.239+0000 7fd75ffff700 1 -- start start 2022-07-21T01:54:04.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd7600adff0 0x7fd7600b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:04.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd7600af8e0 0x7fd76005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:04.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd760147010 con 0x7fd7600b5650 2022-07-21T01:54:04.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7601472b0 con 0x7fd7600af8e0 2022-07-21T01:54:04.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd760146d40 con 0x7fd7600adff0 2022-07-21T01:54:04.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd7600af8e0 0x7fd76005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:04.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd7600af8e0 0x7fd76005bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36324/0 (socket says 172.21.15.140:36324) 2022-07-21T01:54:04.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75f7fe700 1 -- 172.21.15.140:0/3163578716 learned_addr learned my addr 172.21.15.140:0/3163578716 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:04.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75e7fc700 1 --2- 172.21.15.140:0/3163578716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd7600adff0 0x7fd7600b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:04.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75f7fe700 1 -- 172.21.15.140:0/3163578716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd7600adff0 msgr2=0x7fd7600b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:04.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75f7fe700 1 --2- 172.21.15.140:0/3163578716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd7600adff0 0x7fd7600b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:04.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75f7fe700 1 -- 172.21.15.140:0/3163578716 >> v1:172.21.15.152:6789/0 conn(0x7fd7600b5650 legacy=0x7fd760147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:04.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75f7fe700 1 -- 172.21.15.140:0/3163578716 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7600acae0 con 0x7fd7600af8e0 2022-07-21T01:54:04.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75f7fe700 1 --2- 172.21.15.140:0/3163578716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd7600af8e0 0x7fd76005bd90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7fd758002f70 tx=0x7fd758006eb0).ready entity=mon.0 client_cookie=9cdab610c265c05a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:04.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75dffb700 1 -- 172.21.15.140:0/3163578716 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd75800d5c0 con 0x7fd7600af8e0 2022-07-21T01:54:04.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75dffb700 1 -- 172.21.15.140:0/3163578716 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd758000e90 con 0x7fd7600af8e0 2022-07-21T01:54:04.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75dffb700 1 -- 172.21.15.140:0/3163578716 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd75800d950 con 0x7fd7600af8e0 2022-07-21T01:54:04.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 -- 172.21.15.140:0/3163578716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd7600af8e0 msgr2=0x7fd76005bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:04.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 --2- 172.21.15.140:0/3163578716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd7600af8e0 0x7fd76005bd90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7fd758002f70 tx=0x7fd758006eb0).stop 2022-07-21T01:54:04.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 -- 172.21.15.140:0/3163578716 shutdown_connections 2022-07-21T01:54:04.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 --2- 172.21.15.140:0/3163578716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd7600af8e0 0x7fd76005bd90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:04.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 --2- 172.21.15.140:0/3163578716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd7600adff0 0x7fd7600b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:04.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 -- 172.21.15.140:0/3163578716 >> 172.21.15.140:0/3163578716 conn(0x7fd7600634e0 msgr2=0x7fd7600b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:04.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 -- 172.21.15.140:0/3163578716 shutdown_connections 2022-07-21T01:54:04.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.243+0000 7fd75ffff700 1 -- 172.21.15.140:0/3163578716 wait complete. 2022-07-21T01:54:04.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75ffff700 1 Processor -- start 2022-07-21T01:54:04.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75ffff700 1 -- start start 2022-07-21T01:54:04.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd7600adff0 0x7fd760111730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:04.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7600af8e0 0x7fd760148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:04.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd7600b5650 0x7fd760156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:04.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd760063ae0 con 0x7fd7600b5650 2022-07-21T01:54:04.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd760110d50 con 0x7fd7600af8e0 2022-07-21T01:54:04.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd760110ff0 con 0x7fd7600adff0 2022-07-21T01:54:04.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd7600b5650 0x7fd760156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:04.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75e7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7600af8e0 0x7fd760148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:04.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75e7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7600af8e0 0x7fd760148a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42068/0 (socket says 172.21.15.140:42068) 2022-07-21T01:54:04.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd7600b5650 0x7fd760156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36330/0 (socket says 172.21.15.140:36330) 2022-07-21T01:54:04.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75e7fc700 1 -- 172.21.15.140:0/560398219 learned_addr learned my addr 172.21.15.140:0/560398219 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:04.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75effd700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd7600adff0 0x7fd760111730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:04.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75e7fc700 1 -- 172.21.15.140:0/560398219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd7600adff0 msgr2=0x7fd760111730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:04.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75e7fc700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd7600adff0 0x7fd760111730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:04.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75e7fc700 1 -- 172.21.15.140:0/560398219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd7600b5650 msgr2=0x7fd760156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:04.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75e7fc700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd7600b5650 0x7fd760156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:04.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75e7fc700 1 -- 172.21.15.140:0/560398219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75800b1e0 con 0x7fd7600af8e0 2022-07-21T01:54:04.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75f7fe700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd7600b5650 0x7fd760156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:04.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75e7fc700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7600af8e0 0x7fd760148a10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fd74800a000 tx=0x7fd748008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:04.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd757fff700 1 -- 172.21.15.140:0/560398219 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd748005ad0 con 0x7fd7600af8e0 2022-07-21T01:54:04.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75ffff700 1 -- 172.21.15.140:0/560398219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd760111290 con 0x7fd7600af8e0 2022-07-21T01:54:04.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd757fff700 1 -- 172.21.15.140:0/560398219 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd748005c30 con 0x7fd7600af8e0 2022-07-21T01:54:04.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd757fff700 1 -- 172.21.15.140:0/560398219 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd748006030 con 0x7fd7600af8e0 2022-07-21T01:54:04.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.247+0000 7fd75ffff700 1 -- 172.21.15.140:0/560398219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd760148f00 con 0x7fd7600af8e0 2022-07-21T01:54:04.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.251+0000 7fd75ffff700 1 -- 172.21.15.140:0/560398219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd760110b70 con 0x7fd7600af8e0 2022-07-21T01:54:04.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.251+0000 7fd757fff700 1 -- 172.21.15.140:0/560398219 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd748006190 con 0x7fd7600af8e0 2022-07-21T01:54:04.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.251+0000 7fd757fff700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd744053a70 0x7fd744055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:04.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.251+0000 7fd757fff700 1 -- 172.21.15.140:0/560398219 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fd74806bc50 con 0x7fd7600af8e0 2022-07-21T01:54:04.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.251+0000 7fd75effd700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd744053a70 0x7fd744055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:04.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.251+0000 7fd75effd700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd744053a70 0x7fd744055f30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fd75000d000 tx=0x7fd750007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:04.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.259+0000 7fd757fff700 1 -- 172.21.15.140:0/560398219 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd748042c60 con 0x7fd7600af8e0 2022-07-21T01:54:04.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:04.427+0000 7fd75ffff700 1 -- 172.21.15.140:0/560398219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7fd760111420 con 0x7fd7600af8e0 2022-07-21T01:54:05.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.115+0000 7fd757fff700 1 -- 172.21.15.140:0/560398219 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v243) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd748046280 con 0x7fd7600af8e0 2022-07-21T01:54:05.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.119+0000 7fd75ffff700 1 -- 172.21.15.140:0/560398219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd744053a70 msgr2=0x7fd744055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.123+0000 7fd75ffff700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd744053a70 0x7fd744055f30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fd75000d000 tx=0x7fd750007b90).stop 2022-07-21T01:54:05.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.123+0000 7fd75ffff700 1 -- 172.21.15.140:0/560398219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7600af8e0 msgr2=0x7fd760148a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.123+0000 7fd75ffff700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7600af8e0 0x7fd760148a10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fd74800a000 tx=0x7fd748008040).stop 2022-07-21T01:54:05.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.123+0000 7fd75ffff700 1 -- 172.21.15.140:0/560398219 shutdown_connections 2022-07-21T01:54:05.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.123+0000 7fd75ffff700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd744053a70 0x7fd744055f30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.123+0000 7fd75ffff700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd7600b5650 0x7fd760156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.123+0000 7fd75ffff700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7600af8e0 0x7fd760148a10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.123+0000 7fd75ffff700 1 --2- 172.21.15.140:0/560398219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd7600adff0 0x7fd760111730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.123+0000 7fd75ffff700 1 -- 172.21.15.140:0/560398219 >> 172.21.15.140:0/560398219 conn(0x7fd7600634e0 msgr2=0x7fd7600b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:05.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.123+0000 7fd75ffff700 1 -- 172.21.15.140:0/560398219 shutdown_connections 2022-07-21T01:54:05.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.123+0000 7fd75ffff700 1 -- 172.21.15.140:0/560398219 wait complete. 2022-07-21T01:54:05.134 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:54:05.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb3254700 1 Processor -- start 2022-07-21T01:54:05.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb3254700 1 -- start start 2022-07-21T01:54:05.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb3254700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1fac0b4440 0x7f1fac143590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb3254700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1fac145f90 0x7f1fac13fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb3254700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fac0b6c60 con 0x7f1fac0b6880 2022-07-21T01:54:05.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb3254700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac1429c0 con 0x7f1fac145f90 2022-07-21T01:54:05.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb3254700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fac142bb0 con 0x7f1fac0b4440 2022-07-21T01:54:05.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb2252700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1fac0b6880 0x7f1fac0b3d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50480/0 (socket says 172.21.15.140:50480) 2022-07-21T01:54:05.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb2252700 1 -- 172.21.15.140:0/3584562125 learned_addr learned my addr 172.21.15.140:0/3584562125 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:05.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb2a53700 1 --2- 172.21.15.140:0/3584562125 >> v2:172.21.15.140:3300/0 conn(0x7f1fac145f90 0x7f1fac13fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb1a51700 1 --2- 172.21.15.140:0/3584562125 >> v2:172.21.15.152:3300/0 conn(0x7f1fac0b4440 0x7f1fac143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb1250700 1 -- 172.21.15.140:0/3584562125 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3191728294 0 0) 0x7f1fac0b6c60 con 0x7f1fac0b6880 2022-07-21T01:54:05.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb1250700 1 -- 172.21.15.140:0/3584562125 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f98003260 con 0x7f1fac0b6880 2022-07-21T01:54:05.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb2a53700 1 -- 172.21.15.140:0/3584562125 >> v2:172.21.15.152:3300/0 conn(0x7f1fac0b4440 msgr2=0x7f1fac143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:05.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb2a53700 1 --2- 172.21.15.140:0/3584562125 >> v2:172.21.15.152:3300/0 conn(0x7f1fac0b4440 0x7f1fac143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb2a53700 1 -- 172.21.15.140:0/3584562125 >> v1:172.21.15.140:6789/0 conn(0x7f1fac0b6880 legacy=0x7f1fac0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.255+0000 7f1fb2a53700 1 -- 172.21.15.140:0/3584562125 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac142e50 con 0x7f1fac145f90 2022-07-21T01:54:05.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb2a53700 1 --2- 172.21.15.140:0/3584562125 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fac145f90 0x7f1fac13fa00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f1f9c002f70 tx=0x7f1f9c006df0).ready entity=mon.0 client_cookie=f273e48ce3780297 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:05.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb1250700 1 -- 172.21.15.140:0/3584562125 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f9c00d4f0 con 0x7f1fac145f90 2022-07-21T01:54:05.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb1250700 1 -- 172.21.15.140:0/3584562125 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c000e90 con 0x7f1fac145f90 2022-07-21T01:54:05.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb1250700 1 -- 172.21.15.140:0/3584562125 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f9c00d840 con 0x7f1fac145f90 2022-07-21T01:54:05.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb3254700 1 -- 172.21.15.140:0/3584562125 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fac145f90 msgr2=0x7f1fac13fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb3254700 1 --2- 172.21.15.140:0/3584562125 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fac145f90 0x7f1fac13fa00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f1f9c002f70 tx=0x7f1f9c006df0).stop 2022-07-21T01:54:05.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb3254700 1 -- 172.21.15.140:0/3584562125 shutdown_connections 2022-07-21T01:54:05.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb3254700 1 --2- 172.21.15.140:0/3584562125 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fac145f90 0x7f1fac13fa00 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb3254700 1 --2- 172.21.15.140:0/3584562125 >> v2:172.21.15.152:3300/0 conn(0x7f1fac0b4440 0x7f1fac143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb3254700 1 -- 172.21.15.140:0/3584562125 >> 172.21.15.140:0/3584562125 conn(0x7f1fac0b9e60 msgr2=0x7f1fac0b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:05.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb3254700 1 -- 172.21.15.140:0/3584562125 shutdown_connections 2022-07-21T01:54:05.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb3254700 1 -- 172.21.15.140:0/3584562125 wait complete. 2022-07-21T01:54:05.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb3254700 1 Processor -- start 2022-07-21T01:54:05.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.259+0000 7f1fb3254700 1 -- start start 2022-07-21T01:54:05.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb3254700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fac0b4440 0x7f1fac1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb3254700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fac0b6880 0x7f1fac139bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb3254700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fac145f90 0x7f1fac0b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb3254700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac0ae290 con 0x7f1fac145f90 2022-07-21T01:54:05.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb3254700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac151e60 con 0x7f1fac0b4440 2022-07-21T01:54:05.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb3254700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac152100 con 0x7f1fac0b6880 2022-07-21T01:54:05.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb2a53700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fac145f90 0x7f1fac0b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb2a53700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fac145f90 0x7f1fac0b71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36342/0 (socket says 172.21.15.140:36342) 2022-07-21T01:54:05.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb2a53700 1 -- 172.21.15.140:0/2313010458 learned_addr learned my addr 172.21.15.140:0/2313010458 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:05.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb2252700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fac0b4440 0x7f1fac1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb1a51700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fac0b6880 0x7f1fac139bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb2a53700 1 -- 172.21.15.140:0/2313010458 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fac0b6880 msgr2=0x7f1fac139bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb2a53700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fac0b6880 0x7f1fac139bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb2a53700 1 -- 172.21.15.140:0/2313010458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fac0b4440 msgr2=0x7f1fac1525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb2a53700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fac0b4440 0x7f1fac1525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb2a53700 1 -- 172.21.15.140:0/2313010458 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c00b180 con 0x7f1fac145f90 2022-07-21T01:54:05.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb2a53700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fac145f90 0x7f1fac0b71c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f1f9c002f70 tx=0x7f1f9c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:05.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fab7fe700 1 -- 172.21.15.140:0/2313010458 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f9c016070 con 0x7f1fac145f90 2022-07-21T01:54:05.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fab7fe700 1 -- 172.21.15.140:0/2313010458 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c013850 con 0x7f1fac145f90 2022-07-21T01:54:05.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb3254700 1 -- 172.21.15.140:0/2313010458 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac0b8ce0 con 0x7f1fac145f90 2022-07-21T01:54:05.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fab7fe700 1 -- 172.21.15.140:0/2313010458 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1f9c013bb0 con 0x7f1fac145f90 2022-07-21T01:54:05.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.263+0000 7f1fb3254700 1 -- 172.21.15.140:0/2313010458 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fac152c30 con 0x7f1fac145f90 2022-07-21T01:54:05.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.267+0000 7f1fab7fe700 1 -- 172.21.15.140:0/2313010458 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1f9c025030 con 0x7f1fac145f90 2022-07-21T01:54:05.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.267+0000 7f1fb3254700 1 -- 172.21.15.140:0/2313010458 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fac1430f0 con 0x7f1fac145f90 2022-07-21T01:54:05.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.267+0000 7f1fab7fe700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f8c053b00 0x7f1f8c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.267+0000 7f1fb2252700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f8c053b00 0x7f1f8c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.267+0000 7f1fab7fe700 1 -- 172.21.15.140:0/2313010458 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f1f9c06cc90 con 0x7f1fac145f90 2022-07-21T01:54:05.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.267+0000 7f1fb2252700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f8c053b00 0x7f1f8c055fc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f1fa400d000 tx=0x7f1fa4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:05.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.271+0000 7f1fab7fe700 1 -- 172.21.15.140:0/2313010458 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1f9c043d80 con 0x7f1fac145f90 2022-07-21T01:54:05.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.399+0000 7f1fb3254700 1 -- 172.21.15.140:0/2313010458 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1fac0574e0 con 0x7f1fac145f90 2022-07-21T01:54:05.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.407+0000 7f1fab7fe700 1 -- 172.21.15.140:0/2313010458 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f9c023070 con 0x7f1fac145f90 2022-07-21T01:54:05.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 -- 172.21.15.140:0/2313010458 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f8c053b00 msgr2=0x7f1f8c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f8c053b00 0x7f1f8c055fc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f1fa400d000 tx=0x7f1fa4009040).stop 2022-07-21T01:54:05.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 -- 172.21.15.140:0/2313010458 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fac145f90 msgr2=0x7f1fac0b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fac145f90 0x7f1fac0b71c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f1f9c002f70 tx=0x7f1f9c014000).stop 2022-07-21T01:54:05.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 -- 172.21.15.140:0/2313010458 shutdown_connections 2022-07-21T01:54:05.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1f8c053b00 0x7f1f8c055fc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1fac145f90 0x7f1fac0b71c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1fac0b6880 0x7f1fac139bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 --2- 172.21.15.140:0/2313010458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1fac0b4440 0x7f1fac1525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 -- 172.21.15.140:0/2313010458 >> 172.21.15.140:0/2313010458 conn(0x7f1fac0b9e60 msgr2=0x7f1fac1470d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:05.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 -- 172.21.15.140:0/2313010458 shutdown_connections 2022-07-21T01:54:05.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.411+0000 7f1fb3254700 1 -- 172.21.15.140:0/2313010458 wait complete. 2022-07-21T01:54:05.419 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:54:05.420 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:54:05.439 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:54:05.460 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:54:05.482 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:54:05.503 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:54:05.524 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:54:05.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:05.525 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:54:05.525 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:54:05.533 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:54:05.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:54:05.535 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:54:05.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:05.537 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:54:05.538 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:54:05.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:54:05.548 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:54:05.549 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:54:05.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:05.551 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:54:05.552 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:54:05.574 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:54:05.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:54:05.577 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:54:05.577 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:54:05.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9edfe28700 1 Processor -- start 2022-07-21T01:54:05.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9edfe28700 1 -- start start 2022-07-21T01:54:05.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9edfe28700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9ed80b1e00 0x7f9ed814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9edfe28700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ed80acef0 0x7f9ed8146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9edfe28700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed805d480 con 0x7f9ed805c3f0 2022-07-21T01:54:05.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9edfe28700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed80b21e0 con 0x7f9ed80b1e00 2022-07-21T01:54:05.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9edfe28700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed805cf40 con 0x7f9ed80acef0 2022-07-21T01:54:05.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9edd3c3700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f9ed805c3f0 0x7f9ed80ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50494/0 (socket says 172.21.15.140:50494) 2022-07-21T01:54:05.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9edd3c3700 1 -- 172.21.15.140:0/3932769322 learned_addr learned my addr 172.21.15.140:0/3932769322 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:05.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9eddbc4700 1 --2- 172.21.15.140:0/3932769322 >> v2:172.21.15.140:3300/0 conn(0x7f9ed80b1e00 0x7f9ed814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9ede3c5700 1 --2- 172.21.15.140:0/3932769322 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ed80acef0 0x7f9ed8146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9edcbc2700 1 -- 172.21.15.140:0/3932769322 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536317524 0 0) 0x7f9ed805d480 con 0x7f9ed805c3f0 2022-07-21T01:54:05.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9edcbc2700 1 -- 172.21.15.140:0/3932769322 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec4003260 con 0x7f9ed805c3f0 2022-07-21T01:54:05.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9eddbc4700 1 -- 172.21.15.140:0/3932769322 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ed80acef0 msgr2=0x7f9ed8146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:05.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9eddbc4700 1 --2- 172.21.15.140:0/3932769322 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ed80acef0 0x7f9ed8146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9eddbc4700 1 -- 172.21.15.140:0/3932769322 >> v1:172.21.15.140:6789/0 conn(0x7f9ed805c3f0 legacy=0x7f9ed80ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9eddbc4700 1 -- 172.21.15.140:0/3932769322 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed805d1e0 con 0x7f9ed80b1e00 2022-07-21T01:54:05.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.691+0000 7f9eddbc4700 1 --2- 172.21.15.140:0/3932769322 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ed80b1e00 0x7f9ed814a6d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f9ecc002f70 tx=0x7f9ecc006df0).ready entity=mon.0 client_cookie=678059db9916fe5c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:05.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.695+0000 7f9edcbc2700 1 -- 172.21.15.140:0/3932769322 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ecc00d4a0 con 0x7f9ed80b1e00 2022-07-21T01:54:05.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.695+0000 7f9edcbc2700 1 -- 172.21.15.140:0/3932769322 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc000e90 con 0x7f9ed80b1e00 2022-07-21T01:54:05.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.695+0000 7f9edcbc2700 1 -- 172.21.15.140:0/3932769322 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ecc00d7f0 con 0x7f9ed80b1e00 2022-07-21T01:54:05.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.695+0000 7f9edfe28700 1 -- 172.21.15.140:0/3932769322 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ed80b1e00 msgr2=0x7f9ed814a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.695+0000 7f9edfe28700 1 --2- 172.21.15.140:0/3932769322 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ed80b1e00 0x7f9ed814a6d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f9ecc002f70 tx=0x7f9ecc006df0).stop 2022-07-21T01:54:05.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.695+0000 7f9edfe28700 1 -- 172.21.15.140:0/3932769322 shutdown_connections 2022-07-21T01:54:05.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.695+0000 7f9edfe28700 1 --2- 172.21.15.140:0/3932769322 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ed80acef0 0x7f9ed8146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.695+0000 7f9edfe28700 1 --2- 172.21.15.140:0/3932769322 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ed80b1e00 0x7f9ed814a6d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.695+0000 7f9edfe28700 1 -- 172.21.15.140:0/3932769322 >> 172.21.15.140:0/3932769322 conn(0x7f9ed80611c0 msgr2=0x7f9ed8062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:05.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.695+0000 7f9edfe28700 1 -- 172.21.15.140:0/3932769322 shutdown_connections 2022-07-21T01:54:05.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.695+0000 7f9edfe28700 1 -- 172.21.15.140:0/3932769322 wait complete. 2022-07-21T01:54:05.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edfe28700 1 Processor -- start 2022-07-21T01:54:05.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edfe28700 1 -- start start 2022-07-21T01:54:05.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edfe28700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ed80acef0 0x7f9ed809e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edfe28700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ed80b1e00 0x7f9ed8149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edfe28700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ed814a6d0 0x7f9ed80a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edfe28700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed809e260 con 0x7f9ed814a6d0 2022-07-21T01:54:05.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edfe28700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80a1de0 con 0x7f9ed80acef0 2022-07-21T01:54:05.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edfe28700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed809e4c0 con 0x7f9ed80b1e00 2022-07-21T01:54:05.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ede3c5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ed814a6d0 0x7f9ed80a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ede3c5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ed814a6d0 0x7f9ed80a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36358/0 (socket says 172.21.15.140:36358) 2022-07-21T01:54:05.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ede3c5700 1 -- 172.21.15.140:0/862722376 learned_addr learned my addr 172.21.15.140:0/862722376 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:05.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edd3c3700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ed80b1e00 0x7f9ed8149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9eddbc4700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ed80acef0 0x7f9ed809e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ede3c5700 1 -- 172.21.15.140:0/862722376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ed80b1e00 msgr2=0x7f9ed8149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ede3c5700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ed80b1e00 0x7f9ed8149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ede3c5700 1 -- 172.21.15.140:0/862722376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ed80acef0 msgr2=0x7f9ed809e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:05.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ede3c5700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ed80acef0 0x7f9ed809e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:05.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ede3c5700 1 -- 172.21.15.140:0/862722376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc00b180 con 0x7f9ed814a6d0 2022-07-21T01:54:05.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edd3c3700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ed80b1e00 0x7f9ed8149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:05.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ede3c5700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ed814a6d0 0x7f9ed80a0120 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f9ed000c000 tx=0x7f9ed0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:05.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ed6ffd700 1 -- 172.21.15.140:0/862722376 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ed0010780 con 0x7f9ed814a6d0 2022-07-21T01:54:05.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ed6ffd700 1 -- 172.21.15.140:0/862722376 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed00108e0 con 0x7f9ed814a6d0 2022-07-21T01:54:05.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edfe28700 1 -- 172.21.15.140:0/862722376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed80a1b00 con 0x7f9ed814a6d0 2022-07-21T01:54:05.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9ed6ffd700 1 -- 172.21.15.140:0/862722376 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ed0010c40 con 0x7f9ed814a6d0 2022-07-21T01:54:05.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edfe28700 1 -- 172.21.15.140:0/862722376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed8149390 con 0x7f9ed814a6d0 2022-07-21T01:54:05.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.699+0000 7f9edfe28700 1 -- 172.21.15.140:0/862722376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed80b04b0 con 0x7f9ed814a6d0 2022-07-21T01:54:05.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.703+0000 7f9ed6ffd700 1 -- 172.21.15.140:0/862722376 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9ed001c460 con 0x7f9ed814a6d0 2022-07-21T01:54:05.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.703+0000 7f9ed6ffd700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb8053b00 0x7f9eb8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:05.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.703+0000 7f9ed6ffd700 1 -- 172.21.15.140:0/862722376 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f9ed006bff0 con 0x7f9ed814a6d0 2022-07-21T01:54:05.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.703+0000 7f9eddbc4700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb8053b00 0x7f9eb8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:05.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.703+0000 7f9eddbc4700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb8053b00 0x7f9eb8055fc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f9ecc018000 tx=0x7f9ecc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:05.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.711+0000 7f9ed6ffd700 1 -- 172.21.15.140:0/862722376 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ed0043150 con 0x7f9ed814a6d0 2022-07-21T01:54:05.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.839+0000 7f9edfe28700 1 -- 172.21.15.140:0/862722376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9ed80574e0 con 0x7f9ed814a6d0 2022-07-21T01:54:05.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.839+0000 7f9ed6ffd700 1 -- 172.21.15.140:0/862722376 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v66) v1 ==== 81+0+28922 (secure 0 0 0) 0x7f9ed0046770 con 0x7f9ed814a6d0 2022-07-21T01:54:05.837 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:05.837 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:54:05.837 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 66, 2022-07-21T01:54:05.838 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:54:05.838 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:54:05.838 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:54:03.093681+0000", 2022-07-21T01:54:05.839 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:54:05.839 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:54:05.839 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:54:05.839 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:54:05.840 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:54:05.840 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:54:05.840 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:54:05.840 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:54:05.841 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:54:05.841 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:54:05.841 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:54:05.841 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:54:05.842 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:54:05.842 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:54:05.842 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:54:05.842 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 12, 2022-07-21T01:54:05.843 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:54:05.843 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:54:05.843 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:54:05.843 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:54:05.844 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:54:05.844 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.844 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:54:05.844 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:54:05.845 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:54:05.845 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:54:05.845 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:54:05.845 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:54:05.846 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:54:05.846 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:54:05.846 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:54:05.846 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:54:05.847 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:54:05.847 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:54:05.847 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:54:05.847 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:54:05.848 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:54:05.848 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:54:05.848 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:54:05.848 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:54:05.849 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:54:05.849 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:54:05.849 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:54:05.849 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:54:05.850 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:54:05.850 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.850 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:54:05.850 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:54:05.851 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:54:05.851 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:54:05.851 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:54:05.852 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:54:05.852 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:54:05.852 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:54:05.852 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:54:05.852 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:54:05.853 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:54:05.853 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:54:05.853 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:54:05.854 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:54:05.854 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:54:05.854 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:54:05.854 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:54:05.855 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:54:05.855 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:54:05.855 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:54:05.856 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:54:05.856 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:54:05.856 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:54:05.857 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:54:05.857 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:54:05.857 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:54:05.857 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:54:05.858 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.858 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:54:05.858 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:54:05.858 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:54:05.859 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:54:05.859 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:54:05.859 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:54:05.859 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:54:05.860 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:54:05.860 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:54:05.860 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:54:05.860 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:54:05.861 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:54:05.861 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:54:05.861 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.861 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:54:05.862 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:54:05.862 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.862 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.862 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.863 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:54:05.863 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:54:05.863 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:54:05.863 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:54:05.864 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:54:05.864 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:54:05.864 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:54:05.864 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:54:05.865 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:54:05.865 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:54:05.865 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:54:05.865 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:54:05.866 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:54:05.866 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:54:05.866 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:54:05.866 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:54:05.867 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:54:05.867 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:54:05.867 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:54:05.867 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:54:05.868 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:54:05.868 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:54:05.868 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:54:05.868 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.869 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:54:05.869 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:54:05.869 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:54:05.870 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:54:05.870 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:54:05.870 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:54:05.870 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:54:05.871 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:54:05.871 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:54:05.871 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:54:05.871 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:54:05.872 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:54:05.872 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:54:05.872 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:54:05.872 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:54:05.873 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:54:05.873 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:54:05.873 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:54:05.873 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:54:05.874 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:54:05.874 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:54:05.874 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:54:05.875 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:54:05.875 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:54:05.875 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:54:05.875 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:54:05.875 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:54:05.876 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.876 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:54:05.876 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:54:05.876 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:54:05.877 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:54:05.877 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:54:05.877 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:54:05.877 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:54:05.878 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:54:05.878 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:54:05.878 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:54:05.878 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:54:05.879 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:54:05.879 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:54:05.879 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:54:05.879 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.880 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.880 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:54:05.880 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:54:05.880 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.881 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:54:05.881 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:54:05.881 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:54:05.882 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:54:05.882 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:54:05.882 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:54:05.882 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:54:05.883 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:54:05.883 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:54:05.883 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 61, 2022-07-21T01:54:05.883 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:54:05.884 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:54:05.884 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:54:05.884 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.884 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.885 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.885 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:54:05.885 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:54:05.885 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.886 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.886 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.886 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:54:05.886 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:54:05.887 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.887 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.887 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.888 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:54:05.888 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.888 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.888 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.889 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:54:05.889 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:54:05.889 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.889 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.890 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.890 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:54:05.890 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:54:05.890 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.891 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.891 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.891 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:54:05.891 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.892 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.892 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.892 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:54:05.892 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:54:05.893 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.893 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.893 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.894 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:54:05.894 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:54:05.894 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.894 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.895 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.895 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:54:05.895 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.896 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.896 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.896 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:54:05.897 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:54:05.897 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.897 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.898 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.898 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:54:05.898 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:54:05.898 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.899 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.899 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.899 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:54:05.900 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:54:05.900 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:54:05.900 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:54:05.900 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:54:05.901 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:54:05.901 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:54:05.901 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.901 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.902 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.902 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:54:05.902 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:54:05.902 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:54:05.903 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:54:05.903 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:54:05.903 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:54:05.903 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:54:05.904 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:54:05.904 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:54:05.904 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 61, 2022-07-21T01:54:05.904 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:54:05.905 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:54:05.905 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:54:05.905 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.905 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.906 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.906 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:54:05.906 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:54:05.907 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.907 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.907 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.908 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:54:05.908 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:54:05.908 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.908 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.909 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.909 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:54:05.909 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.909 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.910 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.910 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:54:05.910 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:54:05.910 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.911 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.911 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.911 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:54:05.911 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:54:05.912 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.912 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.912 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.912 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:54:05.913 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.913 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.913 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.913 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:54:05.914 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:54:05.914 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.914 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.914 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.915 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:54:05.915 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:54:05.915 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.916 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.916 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.916 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:54:05.917 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.917 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.917 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.918 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:54:05.918 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:54:05.918 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.918 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.919 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.919 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:54:05.919 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:54:05.919 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.920 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.920 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.920 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:54:05.920 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:54:05.920 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:54:05.921 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:54:05.921 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:54:05.921 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:54:05.921 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:54:05.922 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.922 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.922 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.922 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:54:05.923 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:54:05.923 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:54:05.923 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:54:05.923 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:54:05.924 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:54:05.924 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:54:05.924 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:54:05.924 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:54:05.925 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 60, 2022-07-21T01:54:05.925 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:54:05.925 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:54:05.925 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:54:05.926 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.926 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.926 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.927 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:54:05.927 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:54:05.927 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.927 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.928 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.928 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:54:05.928 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:54:05.929 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.929 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.929 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.929 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:54:05.930 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.930 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.930 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.930 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:54:05.931 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:54:05.931 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.931 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.931 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.932 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:54:05.932 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:54:05.932 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.932 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.933 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.933 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:54:05.933 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.933 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.934 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.934 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:54:05.934 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:54:05.934 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.935 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.935 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.935 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:54:05.935 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:54:05.936 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.936 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.936 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.936 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:54:05.937 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.937 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.937 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.937 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:54:05.938 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:54:05.938 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.938 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.938 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.939 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:54:05.939 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:54:05.939 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.940 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.940 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.940 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:54:05.940 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:54:05.941 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:54:05.941 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:54:05.941 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:54:05.941 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:54:05.941 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:54:05.942 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.942 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.942 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.942 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:54:05.943 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:54:05.943 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:54:05.943 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:54:05.944 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:54:05.944 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:54:05.944 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:54:05.944 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:54:05.944 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:54:05.945 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 49, 2022-07-21T01:54:05.945 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:54:05.945 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:54:05.945 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:54:05.946 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.946 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.946 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.946 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:54:05.947 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:54:05.947 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.947 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.947 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.948 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:54:05.948 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:54:05.948 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.948 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.949 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.949 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:54:05.949 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.949 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.950 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.950 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:54:05.950 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:54:05.950 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.950 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.951 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.951 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:54:05.951 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:54:05.951 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.952 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.952 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.952 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:54:05.952 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.953 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.953 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.953 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:54:05.953 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:54:05.954 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.954 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.954 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.954 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:54:05.955 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:54:05.955 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.955 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.955 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.955 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:54:05.956 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.956 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.956 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.956 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:54:05.957 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:54:05.957 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.957 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.957 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.958 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:54:05.958 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:54:05.958 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.958 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.959 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.959 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:54:05.959 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:54:05.959 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:54:05.960 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:54:05.960 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:54:05.960 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:54:05.961 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:54:05.961 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.961 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.961 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.961 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:54:05.962 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:54:05.962 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:54:05.962 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:54:05.962 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:54:05.963 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:54:05.963 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:54:05.963 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:54:05.963 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:54:05.964 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 61, 2022-07-21T01:54:05.964 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:54:05.964 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:54:05.965 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:54:05.965 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.965 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.965 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.966 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:54:05.966 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:54:05.966 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.966 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.967 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.967 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:54:05.967 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:54:05.968 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.968 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.968 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.968 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:54:05.969 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.969 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.969 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.970 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:54:05.970 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:54:05.970 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.971 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.971 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.971 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:54:05.971 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:54:05.972 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.972 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.972 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.972 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:54:05.973 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.973 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.973 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.974 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:54:05.974 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:54:05.974 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.974 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.975 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.975 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:54:05.975 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:54:05.975 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.976 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.976 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.976 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:54:05.976 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.977 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.977 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.977 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:54:05.977 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:54:05.978 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.978 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.978 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.978 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:54:05.979 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:54:05.979 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.979 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.980 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.980 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:54:05.980 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:54:05.980 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:54:05.981 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:54:05.981 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:54:05.981 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:54:05.981 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:54:05.981 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.982 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.982 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.982 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:54:05.983 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:54:05.983 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:54:05.983 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:54:05.983 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:54:05.984 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:54:05.984 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:54:05.984 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:54:05.984 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:54:05.985 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 60, 2022-07-21T01:54:05.985 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:54:05.985 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:54:05.985 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:54:05.986 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.986 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.986 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.987 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:54:05.987 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:54:05.987 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.987 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.988 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.988 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:54:05.988 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:54:05.988 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.989 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.989 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.989 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:54:05.990 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.990 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.990 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.990 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:54:05.991 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:54:05.991 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.991 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.992 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.992 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:54:05.992 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:54:05.993 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.993 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.993 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.993 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:54:05.994 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.994 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.994 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.995 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:54:05.995 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:54:05.995 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.995 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.996 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.996 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:54:05.996 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:54:05.996 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:05.997 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:05.997 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.997 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:54:05.997 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:05.998 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.998 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:05.998 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:54:05.998 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:54:05.999 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:05.999 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:05.999 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:05.999 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:54:06.000 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:54:06.000 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.000 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.000 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.001 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:54:06.001 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:54:06.001 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:54:06.001 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:54:06.002 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:54:06.002 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:54:06.002 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:54:06.003 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.003 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.003 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.004 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:54:06.004 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:54:06.004 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:54:06.004 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:54:06.005 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:54:06.005 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:54:06.005 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:54:06.005 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:54:06.006 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:54:06.006 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 61, 2022-07-21T01:54:06.006 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:54:06.006 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:54:06.007 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:54:06.007 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:06.007 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.007 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:06.007 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:54:06.008 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:54:06.008 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.008 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.008 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:06.009 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:54:06.009 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:54:06.009 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.009 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.010 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.010 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:54:06.010 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:06.010 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.011 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:06.011 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:54:06.011 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:54:06.011 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.012 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.012 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:06.012 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:54:06.012 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:54:06.013 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.013 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.013 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.014 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:54:06.014 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:06.014 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.014 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:06.015 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:54:06.015 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:54:06.015 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.015 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.016 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:06.016 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:54:06.016 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:54:06.016 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.016 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.017 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.017 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:54:06.017 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:06.017 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.018 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:06.018 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:54:06.018 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:54:06.018 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.019 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.019 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:06.019 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:54:06.019 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:54:06.020 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.020 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.020 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.020 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:54:06.021 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:54:06.021 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:54:06.021 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:54:06.021 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:54:06.021 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:54:06.022 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:54:06.022 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.022 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.022 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.023 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:54:06.023 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:54:06.023 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:54:06.023 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:54:06.024 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:54:06.024 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:54:06.024 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:54:06.025 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:54:06.025 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:54:06.025 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 60, 2022-07-21T01:54:06.025 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:54:06.026 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:54:06.026 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:54:06.026 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:06.026 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.027 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:06.027 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:54:06.027 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:54:06.027 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.028 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.028 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:06.028 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:54:06.028 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:54:06.029 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.029 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.029 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.029 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:54:06.030 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:06.030 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.030 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:06.030 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:54:06.031 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:54:06.031 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.031 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.031 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:06.032 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:54:06.032 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:54:06.032 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.032 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.033 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.033 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:54:06.033 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:06.033 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.034 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:06.034 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:54:06.034 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:54:06.034 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.035 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.035 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:06.035 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:54:06.035 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:54:06.036 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.036 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.036 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.036 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:54:06.037 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:54:06.037 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.037 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:54:06.037 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:54:06.038 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:54:06.038 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.038 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.038 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:54:06.039 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:54:06.039 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:54:06.039 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.039 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.040 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.040 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:54:06.040 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:54:06.040 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:54:06.041 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:54:06.041 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:54:06.041 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:54:06.041 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:54:06.041 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:06.042 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.042 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:54:06.042 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:54:06.043 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.043 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:54:06.043 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:54:06.043 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:54:06.043 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:54:06.044 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:54:06.044 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:54:06.044 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:54:06.044 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:54:06.045 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.045 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.045 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:54:06.045 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:54:06.046 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:54:06.046 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:54:06.046 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:54:06.046 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:54:06.047 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:54:06.047 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:54:06.047 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.048 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.048 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:54:06.048 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:54:06.048 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:54:06.049 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:54:06.049 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:54:06.049 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:54:06.050 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:54:06.050 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:54:06.050 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.050 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.051 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:54:06.051 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:54:06.051 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:54:06.052 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:54:06.052 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:54:06.052 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:54:06.052 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:54:06.053 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:54:06.053 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.053 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.053 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:54:06.054 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:54:06.054 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:54:06.054 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:54:06.054 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:54:06.055 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:54:06.055 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:54:06.055 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:54:06.056 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.056 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.056 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:54:06.056 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:54:06.057 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:54:06.057 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:54:06.057 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:54:06.057 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:54:06.058 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:54:06.058 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:54:06.058 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.058 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.059 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:54:06.059 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:54:06.059 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:54:06.059 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:54:06.060 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:54:06.060 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:54:06.060 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:54:06.061 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:54:06.061 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.061 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:06.061 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:54:06.062 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:54:06.062 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:54:06.062 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:54:06.062 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:54:06.063 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:54:06.063 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:54:06.063 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:54:06.063 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.064 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:54:06.064 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:54:06.064 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:54:06.065 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:54:06.065 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:54:06.065 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:54:06.065 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6833/74479549": "2022-07-22T01:53:59.098633+0000", 2022-07-21T01:54:06.066 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6832/74479549": "2022-07-22T01:53:59.098633+0000" 2022-07-21T01:54:06.066 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.066 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:54:06.067 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:54:06.067 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:54:06.067 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:54:06.067 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:54:06.068 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:54:06.068 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:54:06.068 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:54:06.068 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:06.069 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:06.069 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:54:06.069 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:54:06.069 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:54:06.070 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:54:06.070 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:54:06.070 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:54:06.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 -- 172.21.15.140:0/862722376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb8053b00 msgr2=0x7f9eb8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb8053b00 0x7f9eb8055fc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f9ecc018000 tx=0x7f9ecc014000).stop 2022-07-21T01:54:06.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 -- 172.21.15.140:0/862722376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ed814a6d0 msgr2=0x7f9ed80a0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ed814a6d0 0x7f9ed80a0120 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f9ed000c000 tx=0x7f9ed0006df0).stop 2022-07-21T01:54:06.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 -- 172.21.15.140:0/862722376 shutdown_connections 2022-07-21T01:54:06.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9eb8053b00 0x7f9eb8055fc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ed814a6d0 0x7f9ed80a0120 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ed80b1e00 0x7f9ed8149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 --2- 172.21.15.140:0/862722376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ed80acef0 0x7f9ed809e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 -- 172.21.15.140:0/862722376 >> 172.21.15.140:0/862722376 conn(0x7f9ed805c3f0 msgr2=0x7f9ed80ae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:06.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 -- 172.21.15.140:0/862722376 shutdown_connections 2022-07-21T01:54:06.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:05.843+0000 7f9edfe28700 1 -- 172.21.15.140:0/862722376 wait complete. 2022-07-21T01:54:06.076 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:54:06.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114f59e700 1 Processor -- start 2022-07-21T01:54:06.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114f59e700 1 -- start start 2022-07-21T01:54:06.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114f59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1150063d60 0x7f11500b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114f59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f11500b1440 0x7f11500acae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11500af150 0x7f11500acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114f59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1150064140 con 0x7f11500b1440 2022-07-21T01:54:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114f59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1150143800 con 0x7f1150063d60 2022-07-21T01:54:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1150143aa0 con 0x7f11500af150 2022-07-21T01:54:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114dd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f11500b1440 0x7f11500acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114dd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f11500b1440 0x7f11500acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36362/0 (socket says 172.21.15.140:36362) 2022-07-21T01:54:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114dd9b700 1 -- 172.21.15.140:0/4094809571 learned_addr learned my addr 172.21.15.140:0/4094809571 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:06.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114e59c700 1 --2- 172.21.15.140:0/4094809571 >> v2:172.21.15.152:3300/0 conn(0x7f1150063d60 0x7f11500b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114ed9d700 1 --2- 172.21.15.140:0/4094809571 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11500af150 0x7f11500acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114dd9b700 1 -- 172.21.15.140:0/4094809571 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11500af150 msgr2=0x7f11500acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:06.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.191+0000 7f114dd9b700 1 --2- 172.21.15.140:0/4094809571 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11500af150 0x7f11500acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114dd9b700 1 -- 172.21.15.140:0/4094809571 >> v2:172.21.15.152:3300/0 conn(0x7f1150063d60 msgr2=0x7f11500b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:06.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114dd9b700 1 --2- 172.21.15.140:0/4094809571 >> v2:172.21.15.152:3300/0 conn(0x7f1150063d60 0x7f11500b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114dd9b700 1 -- 172.21.15.140:0/4094809571 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11500b1050 con 0x7f11500b1440 2022-07-21T01:54:06.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114ed9d700 1 --2- 172.21.15.140:0/4094809571 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11500af150 0x7f11500acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:06.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114dd9b700 1 --2- 172.21.15.140:0/4094809571 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11500b1440 0x7f11500acae0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f1138002f70 tx=0x7f1138006e50).ready entity=mon.0 client_cookie=24bd532c804a7959 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:06.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114d59a700 1 -- 172.21.15.140:0/4094809571 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1138006020 con 0x7f11500b1440 2022-07-21T01:54:06.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114d59a700 1 -- 172.21.15.140:0/4094809571 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1138006180 con 0x7f11500b1440 2022-07-21T01:54:06.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114d59a700 1 -- 172.21.15.140:0/4094809571 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1138006520 con 0x7f11500b1440 2022-07-21T01:54:06.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114f59e700 1 -- 172.21.15.140:0/4094809571 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11500b1440 msgr2=0x7f11500acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114f59e700 1 --2- 172.21.15.140:0/4094809571 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11500b1440 0x7f11500acae0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f1138002f70 tx=0x7f1138006e50).stop 2022-07-21T01:54:06.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114f59e700 1 -- 172.21.15.140:0/4094809571 shutdown_connections 2022-07-21T01:54:06.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114f59e700 1 --2- 172.21.15.140:0/4094809571 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11500af150 0x7f11500acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114f59e700 1 --2- 172.21.15.140:0/4094809571 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11500b1440 0x7f11500acae0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114f59e700 1 --2- 172.21.15.140:0/4094809571 >> v2:172.21.15.152:3300/0 conn(0x7f1150063d60 0x7f11500b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114f59e700 1 -- 172.21.15.140:0/4094809571 >> 172.21.15.140:0/4094809571 conn(0x7f11500634e0 msgr2=0x7f11500b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:06.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114f59e700 1 -- 172.21.15.140:0/4094809571 shutdown_connections 2022-07-21T01:54:06.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.195+0000 7f114f59e700 1 -- 172.21.15.140:0/4094809571 wait complete. 2022-07-21T01:54:06.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114f59e700 1 Processor -- start 2022-07-21T01:54:06.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114f59e700 1 -- start start 2022-07-21T01:54:06.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114f59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1150063d60 0x7f11500b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11500af150 0x7f115013a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114f59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11500b1440 0x7f11500b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114f59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1150064620 con 0x7f11500b1440 2022-07-21T01:54:06.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114f59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11500648c0 con 0x7f1150063d60 2022-07-21T01:54:06.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1150064b60 con 0x7f11500af150 2022-07-21T01:54:06.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11500b1440 0x7f11500b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114dd9b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11500af150 0x7f115013a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11500b1440 0x7f11500b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36372/0 (socket says 172.21.15.140:36372) 2022-07-21T01:54:06.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114e59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1150063d60 0x7f11500b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114ed9d700 1 -- 172.21.15.140:0/2529560109 learned_addr learned my addr 172.21.15.140:0/2529560109 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:06.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114ed9d700 1 -- 172.21.15.140:0/2529560109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11500af150 msgr2=0x7f115013a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114ed9d700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11500af150 0x7f115013a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114ed9d700 1 -- 172.21.15.140:0/2529560109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1150063d60 msgr2=0x7f11500b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114ed9d700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1150063d60 0x7f11500b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114ed9d700 1 -- 172.21.15.140:0/2529560109 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113800b1e0 con 0x7f11500b1440 2022-07-21T01:54:06.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f114ed9d700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11500b1440 0x7f11500b37c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f1148006cc0 tx=0x7f1148008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:06.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.199+0000 7f11477fe700 1 -- 172.21.15.140:0/2529560109 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1148013620 con 0x7f11500b1440 2022-07-21T01:54:06.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.203+0000 7f11477fe700 1 -- 172.21.15.140:0/2529560109 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148013780 con 0x7f11500b1440 2022-07-21T01:54:06.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.203+0000 7f114f59e700 1 -- 172.21.15.140:0/2529560109 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1150064310 con 0x7f11500b1440 2022-07-21T01:54:06.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.203+0000 7f11477fe700 1 -- 172.21.15.140:0/2529560109 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1148013ae0 con 0x7f11500b1440 2022-07-21T01:54:06.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.203+0000 7f114f59e700 1 -- 172.21.15.140:0/2529560109 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1150134660 con 0x7f11500b1440 2022-07-21T01:54:06.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.203+0000 7f11477fe700 1 -- 172.21.15.140:0/2529560109 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1148013c40 con 0x7f11500b1440 2022-07-21T01:54:06.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.203+0000 7f114f59e700 1 -- 172.21.15.140:0/2529560109 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1150064d20 con 0x7f11500b1440 2022-07-21T01:54:06.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.203+0000 7f11477fe700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1134053b00 0x7f1134055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.203+0000 7f11477fe700 1 -- 172.21.15.140:0/2529560109 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f114806bf20 con 0x7f11500b1440 2022-07-21T01:54:06.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.203+0000 7f114e59c700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1134053b00 0x7f1134055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.203+0000 7f114e59c700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1134053b00 0x7f1134055fc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f114000b000 tx=0x7f1140006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:06.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.207+0000 7f11477fe700 1 -- 172.21.15.140:0/2529560109 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1148042fb0 con 0x7f11500b1440 2022-07-21T01:54:06.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 -- 172.21.15.140:0/2529560109 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11500574e0 con 0x7f11500b1440 2022-07-21T01:54:06.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f11477fe700 1 -- 172.21.15.140:0/2529560109 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+1209 (secure 0 0 0) 0x7f11480465d0 con 0x7f11500b1440 2022-07-21T01:54:06.367 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:06.368 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/689662045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":689662045},{"type":"v1","addr":"172.21.15.140:6835","nonce":689662045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24},{"gid":6139,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/130775537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":130775537},{"type":"v1","addr":"172.21.15.152:6833","nonce":130775537}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[]} 2022-07-21T01:54:06.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 -- 172.21.15.140:0/2529560109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1134053b00 msgr2=0x7f1134055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1134053b00 0x7f1134055fc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f114000b000 tx=0x7f1140006df0).stop 2022-07-21T01:54:06.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 -- 172.21.15.140:0/2529560109 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11500b1440 msgr2=0x7f11500b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11500b1440 0x7f11500b37c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f1148006cc0 tx=0x7f1148008040).stop 2022-07-21T01:54:06.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 -- 172.21.15.140:0/2529560109 shutdown_connections 2022-07-21T01:54:06.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1134053b00 0x7f1134055fc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11500b1440 0x7f11500b37c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11500af150 0x7f115013a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 --2- 172.21.15.140:0/2529560109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1150063d60 0x7f11500b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 -- 172.21.15.140:0/2529560109 >> 172.21.15.140:0/2529560109 conn(0x7f11500634e0 msgr2=0x7f11500af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:06.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 -- 172.21.15.140:0/2529560109 shutdown_connections 2022-07-21T01:54:06.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.371+0000 7f114f59e700 1 -- 172.21.15.140:0/2529560109 wait complete. 2022-07-21T01:54:06.373 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 28 2022-07-21T01:54:06.380 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:54:06.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01cb209700 1 Processor -- start 2022-07-21T01:54:06.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01cb209700 1 -- start start 2022-07-21T01:54:06.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01cb209700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c40b5650 0x7f01c40b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01cb209700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f01c40adff0 0x7f01c4147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01cb209700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c4147010 con 0x7f01c40af8e0 2022-07-21T01:54:06.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01cb209700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c41472b0 con 0x7f01c40adff0 2022-07-21T01:54:06.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01cb209700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4146d40 con 0x7f01c40b5650 2022-07-21T01:54:06.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9a06700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f01c40adff0 0x7f01c4147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9a06700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f01c40adff0 0x7f01c4147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36376/0 (socket says 172.21.15.140:36376) 2022-07-21T01:54:06.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9a06700 1 -- 172.21.15.140:0/531911371 learned_addr learned my addr 172.21.15.140:0/531911371 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:06.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01ca207700 1 --2- 172.21.15.140:0/531911371 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c40b5650 0x7f01c40b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9205700 1 -- 172.21.15.140:0/531911371 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622971042 0 0) 0x7f01c4147010 con 0x7f01c40af8e0 2022-07-21T01:54:06.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9205700 1 -- 172.21.15.140:0/531911371 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b0003260 con 0x7f01c40af8e0 2022-07-21T01:54:06.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9a06700 1 -- 172.21.15.140:0/531911371 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c40b5650 msgr2=0x7f01c40b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:06.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9a06700 1 --2- 172.21.15.140:0/531911371 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c40b5650 0x7f01c40b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9a06700 1 -- 172.21.15.140:0/531911371 >> v1:172.21.15.152:6789/0 conn(0x7f01c40af8e0 legacy=0x7f01c405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9a06700 1 -- 172.21.15.140:0/531911371 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c4147c40 con 0x7f01c40adff0 2022-07-21T01:54:06.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01ca207700 1 --2- 172.21.15.140:0/531911371 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c40b5650 0x7f01c40b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:06.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9a06700 1 --2- 172.21.15.140:0/531911371 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01c40adff0 0x7f01c4147750 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f01c00032f0 tx=0x7f01c0009250).ready entity=mon.0 client_cookie=b1004346cc838a46 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:06.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9205700 1 -- 172.21.15.140:0/531911371 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01c000b6c0 con 0x7f01c40adff0 2022-07-21T01:54:06.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9205700 1 -- 172.21.15.140:0/531911371 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c000b820 con 0x7f01c40adff0 2022-07-21T01:54:06.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.483+0000 7f01c9205700 1 -- 172.21.15.140:0/531911371 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01c000b990 con 0x7f01c40adff0 2022-07-21T01:54:06.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 -- 172.21.15.140:0/531911371 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01c40adff0 msgr2=0x7f01c4147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 --2- 172.21.15.140:0/531911371 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01c40adff0 0x7f01c4147750 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f01c00032f0 tx=0x7f01c0009250).stop 2022-07-21T01:54:06.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 -- 172.21.15.140:0/531911371 shutdown_connections 2022-07-21T01:54:06.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 --2- 172.21.15.140:0/531911371 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01c40adff0 0x7f01c4147750 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 --2- 172.21.15.140:0/531911371 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c40b5650 0x7f01c40b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 -- 172.21.15.140:0/531911371 >> 172.21.15.140:0/531911371 conn(0x7f01c40634e0 msgr2=0x7f01c40b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:06.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 -- 172.21.15.140:0/531911371 shutdown_connections 2022-07-21T01:54:06.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 -- 172.21.15.140:0/531911371 wait complete. 2022-07-21T01:54:06.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 Processor -- start 2022-07-21T01:54:06.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 -- start start 2022-07-21T01:54:06.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c40af8e0 0x7f01c4065930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01c40b5650 0x7f01c4065e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c411e040 0x7f01c411e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c411d140 con 0x7f01c40b5650 2022-07-21T01:54:06.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c4157120 con 0x7f01c40af8e0 2022-07-21T01:54:06.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01cb209700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40672b0 con 0x7f01c411e040 2022-07-21T01:54:06.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01caa08700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c411e040 0x7f01c411e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01ca207700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c40af8e0 0x7f01c4065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01caa08700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c411e040 0x7f01c411e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:33114/0 (socket says 172.21.15.140:33114) 2022-07-21T01:54:06.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01caa08700 1 -- 172.21.15.140:0/2904356132 learned_addr learned my addr 172.21.15.140:0/2904356132 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:06.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01c9a06700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01c40b5650 0x7f01c4065e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01caa08700 1 -- 172.21.15.140:0/2904356132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c40af8e0 msgr2=0x7f01c4065930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01caa08700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c40af8e0 0x7f01c4065930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01caa08700 1 -- 172.21.15.140:0/2904356132 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01c40b5650 msgr2=0x7f01c4065e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01caa08700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01c40b5650 0x7f01c4065e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01caa08700 1 -- 172.21.15.140:0/2904356132 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b400b180 con 0x7f01c411e040 2022-07-21T01:54:06.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.487+0000 7f01caa08700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c411e040 0x7f01c411e4c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f01bc001110 tx=0x7f01bc007a10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:06.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.491+0000 7f01bb7fe700 1 -- 172.21.15.140:0/2904356132 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01bc012400 con 0x7f01c411e040 2022-07-21T01:54:06.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.491+0000 7f01bb7fe700 1 -- 172.21.15.140:0/2904356132 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc012560 con 0x7f01c411e040 2022-07-21T01:54:06.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.491+0000 7f01cb209700 1 -- 172.21.15.140:0/2904356132 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c00094d0 con 0x7f01c411e040 2022-07-21T01:54:06.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.491+0000 7f01bb7fe700 1 -- 172.21.15.140:0/2904356132 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01bc0128c0 con 0x7f01c411e040 2022-07-21T01:54:06.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.491+0000 7f01cb209700 1 -- 172.21.15.140:0/2904356132 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c411e9b0 con 0x7f01c411e040 2022-07-21T01:54:06.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.491+0000 7f01cb209700 1 -- 172.21.15.140:0/2904356132 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c405c4a0 con 0x7f01c411e040 2022-07-21T01:54:06.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.491+0000 7f01bb7fe700 1 -- 172.21.15.140:0/2904356132 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f01bc010070 con 0x7f01c411e040 2022-07-21T01:54:06.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.491+0000 7f01bb7fe700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f01a40539e0 0x7f01a4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.491+0000 7f01bb7fe700 1 -- 172.21.15.140:0/2904356132 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f01bc06ad50 con 0x7f01c411e040 2022-07-21T01:54:06.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.491+0000 7f01ca207700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f01a40539e0 0x7f01a4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.495+0000 7f01ca207700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f01a40539e0 0x7f01a4055ea0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f01b400b000 tx=0x7f01b4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:06.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.503+0000 7f01bb7fe700 1 -- 172.21.15.140:0/2904356132 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f01bc041eb0 con 0x7f01c411e040 2022-07-21T01:54:06.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.663+0000 7f01cb209700 1 -- 172.21.15.140:0/2904356132 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01c40574e0 con 0x7f01c411e040 2022-07-21T01:54:06.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.663+0000 7f01bb7fe700 1 -- 172.21.15.140:0/2904356132 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+1209 (secure 0 0 0) 0x7f01bc0454d0 con 0x7f01c411e040 2022-07-21T01:54:06.661 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:06.661 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/689662045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":689662045},{"type":"v1","addr":"172.21.15.140:6835","nonce":689662045}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24},{"gid":6139,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/130775537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":130775537},{"type":"v1","addr":"172.21.15.152:6833","nonce":130775537}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":28}],"filesystems":[]} 2022-07-21T01:54:06.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 -- 172.21.15.140:0/2904356132 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f01a40539e0 msgr2=0x7f01a4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f01a40539e0 0x7f01a4055ea0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f01b400b000 tx=0x7f01b4006df0).stop 2022-07-21T01:54:06.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 -- 172.21.15.140:0/2904356132 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c411e040 msgr2=0x7f01c411e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c411e040 0x7f01c411e4c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f01bc001110 tx=0x7f01bc007a10).stop 2022-07-21T01:54:06.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 -- 172.21.15.140:0/2904356132 shutdown_connections 2022-07-21T01:54:06.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f01a40539e0 0x7f01a4055ea0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01c411e040 0x7f01c411e4c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f01c40b5650 0x7f01c4065e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 --2- 172.21.15.140:0/2904356132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01c40af8e0 0x7f01c4065930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 -- 172.21.15.140:0/2904356132 >> 172.21.15.140:0/2904356132 conn(0x7f01c40634e0 msgr2=0x7f01c40afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:06.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 -- 172.21.15.140:0/2904356132 shutdown_connections 2022-07-21T01:54:06.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.667+0000 7f01cb209700 1 -- 172.21.15.140:0/2904356132 wait complete. 2022-07-21T01:54:06.669 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 28 2022-07-21T01:54:06.674 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:54:06.675 INFO:tasks.ceph.mds.a:Stopping old one... 2022-07-21T01:54:06.675 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:54:06.676 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:54:06.676 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:54:06.677 INFO:tasks.ceph.mds.b:Stopping old one... 2022-07-21T01:54:06.677 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:54:06.678 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:54:06.690 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:54:06.691 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:54:06.693 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:54:06.694 DEBUG:teuthology.parallel:result is None 2022-07-21T01:54:06.695 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:54:06.696 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:54:06.699 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:54:06.699 DEBUG:teuthology.parallel:result is None 2022-07-21T01:54:06.700 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:54:06.765 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:54:06.773 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:54:06.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.791+0000 7f1d8958f700 1 Processor -- start 2022-07-21T01:54:06.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.791+0000 7f1d8958f700 1 -- start start 2022-07-21T01:54:06.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098280 0x7f1d84098660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1d8409abb0 0x7f1d8413ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d840b0a90 con 0x7f1d84098b50 2022-07-21T01:54:06.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8409b020 con 0x7f1d8409abb0 2022-07-21T01:54:06.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d840b0550 con 0x7f1d84098280 2022-07-21T01:54:06.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d827fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1d84098b50 0x7f1d8409a4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50556/0 (socket says 172.21.15.140:50556) 2022-07-21T01:54:06.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d827fc700 1 -- 172.21.15.140:0/4151252356 learned_addr learned my addr 172.21.15.140:0/4151252356 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:06.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d837fe700 1 --2- 172.21.15.140:0/4151252356 >> v2:172.21.15.140:3300/0 conn(0x7f1d8409abb0 0x7f1d8413ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d82ffd700 1 --2- 172.21.15.140:0/4151252356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098280 0x7f1d84098660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d81ffb700 1 -- 172.21.15.140:0/4151252356 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043751342 0 0) 0x7f1d840b0a90 con 0x7f1d84098b50 2022-07-21T01:54:06.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d81ffb700 1 -- 172.21.15.140:0/4151252356 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d64003260 con 0x7f1d84098b50 2022-07-21T01:54:06.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d82ffd700 1 -- 172.21.15.140:0/4151252356 >> v2:172.21.15.140:3300/0 conn(0x7f1d8409abb0 msgr2=0x7f1d8413ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:06.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d82ffd700 1 --2- 172.21.15.140:0/4151252356 >> v2:172.21.15.140:3300/0 conn(0x7f1d8409abb0 0x7f1d8413ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d82ffd700 1 -- 172.21.15.140:0/4151252356 >> v1:172.21.15.140:6789/0 conn(0x7f1d84098b50 legacy=0x7f1d8409a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d82ffd700 1 -- 172.21.15.140:0/4151252356 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d840b07f0 con 0x7f1d84098280 2022-07-21T01:54:06.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d82ffd700 1 --2- 172.21.15.140:0/4151252356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098280 0x7f1d84098660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f1d7c0032f0 tx=0x7f1d7c0091d0).ready entity=mon.2 client_cookie=93357293e7173d53 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:06.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d81ffb700 1 -- 172.21.15.140:0/4151252356 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d7c00b600 con 0x7f1d84098280 2022-07-21T01:54:06.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d81ffb700 1 -- 172.21.15.140:0/4151252356 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c00b760 con 0x7f1d84098280 2022-07-21T01:54:06.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d81ffb700 1 -- 172.21.15.140:0/4151252356 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d7c00b8d0 con 0x7f1d84098280 2022-07-21T01:54:06.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 -- 172.21.15.140:0/4151252356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098280 msgr2=0x7f1d84098660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 --2- 172.21.15.140:0/4151252356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098280 0x7f1d84098660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f1d7c0032f0 tx=0x7f1d7c0091d0).stop 2022-07-21T01:54:06.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 -- 172.21.15.140:0/4151252356 shutdown_connections 2022-07-21T01:54:06.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 --2- 172.21.15.140:0/4151252356 >> v2:172.21.15.140:3300/0 conn(0x7f1d8409abb0 0x7f1d8413ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 --2- 172.21.15.140:0/4151252356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098280 0x7f1d84098660 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 -- 172.21.15.140:0/4151252356 >> 172.21.15.140:0/4151252356 conn(0x7f1d840b50a0 msgr2=0x7f1d840b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:06.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 -- 172.21.15.140:0/4151252356 shutdown_connections 2022-07-21T01:54:06.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 -- 172.21.15.140:0/4151252356 wait complete. 2022-07-21T01:54:06.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 Processor -- start 2022-07-21T01:54:06.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.795+0000 7f1d8958f700 1 -- start start 2022-07-21T01:54:06.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d8958f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84098280 0x7f1d84134430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d8958f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098b50 0x7f1d84134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d8958f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8409abb0 0x7f1d84135420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d8958f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d84064de0 con 0x7f1d8409abb0 2022-07-21T01:54:06.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d8958f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d84135f00 con 0x7f1d84098280 2022-07-21T01:54:06.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d8958f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d84065010 con 0x7f1d84098b50 2022-07-21T01:54:06.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8409abb0 0x7f1d84135420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d827fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098b50 0x7f1d84134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d82ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84098280 0x7f1d84134430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d82ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84098280 0x7f1d84134430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42154/0 (socket says 172.21.15.140:42154) 2022-07-21T01:54:06.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8409abb0 0x7f1d84135420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36418/0 (socket says 172.21.15.140:36418) 2022-07-21T01:54:06.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d82ffd700 1 -- 172.21.15.140:0/1361914685 learned_addr learned my addr 172.21.15.140:0/1361914685 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:06.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d827fc700 1 -- 172.21.15.140:0/1361914685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84098280 msgr2=0x7f1d84134430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d827fc700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84098280 0x7f1d84134430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d827fc700 1 -- 172.21.15.140:0/1361914685 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8409abb0 msgr2=0x7f1d84135420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:06.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d827fc700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8409abb0 0x7f1d84135420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:06.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d827fc700 1 -- 172.21.15.140:0/1361914685 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c009470 con 0x7f1d84098b50 2022-07-21T01:54:06.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d82ffd700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84098280 0x7f1d84134430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:06.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d837fe700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8409abb0 0x7f1d84135420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:06.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d827fc700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098b50 0x7f1d84134f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f1d6c001110 tx=0x7f1d6c007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:06.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d7bfff700 1 -- 172.21.15.140:0/1361914685 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d6c0070d0 con 0x7f1d84098b50 2022-07-21T01:54:06.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d7bfff700 1 -- 172.21.15.140:0/1361914685 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c007230 con 0x7f1d84098b50 2022-07-21T01:54:06.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d8958f700 1 -- 172.21.15.140:0/1361914685 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d840652b0 con 0x7f1d84098b50 2022-07-21T01:54:06.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d7bfff700 1 -- 172.21.15.140:0/1361914685 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d6c005610 con 0x7f1d84098b50 2022-07-21T01:54:06.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.799+0000 7f1d8958f700 1 -- 172.21.15.140:0/1361914685 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d84064420 con 0x7f1d84098b50 2022-07-21T01:54:06.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.803+0000 7f1d8958f700 1 -- 172.21.15.140:0/1361914685 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d840646d0 con 0x7f1d84098b50 2022-07-21T01:54:06.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.803+0000 7f1d7bfff700 1 -- 172.21.15.140:0/1361914685 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d6c01a060 con 0x7f1d84098b50 2022-07-21T01:54:06.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.803+0000 7f1d7bfff700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d680539e0 0x7f1d68055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:06.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.803+0000 7f1d82ffd700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d680539e0 0x7f1d68055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:06.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.803+0000 7f1d7bfff700 1 -- 172.21.15.140:0/1361914685 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f1d6c06b310 con 0x7f1d84098b50 2022-07-21T01:54:06.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.803+0000 7f1d82ffd700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d680539e0 0x7f1d68055ea0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f1d7c01b000 tx=0x7f1d7c016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:06.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.811+0000 7f1d7bfff700 1 -- 172.21.15.140:0/1361914685 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d6c042510 con 0x7f1d84098b50 2022-07-21T01:54:06.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:06.947+0000 7f1d8958f700 1 -- 172.21.15.140:0/1361914685 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f1d840af3d0 con 0x7f1d84098b50 2022-07-21T01:54:07.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.415+0000 7f1d7bfff700 1 -- 172.21.15.140:0/1361914685 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v67) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1d6c018070 con 0x7f1d84098b50 2022-07-21T01:54:07.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 -- 172.21.15.140:0/1361914685 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d680539e0 msgr2=0x7f1d68055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:07.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d680539e0 0x7f1d68055ea0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f1d7c01b000 tx=0x7f1d7c016000).stop 2022-07-21T01:54:07.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 -- 172.21.15.140:0/1361914685 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098b50 msgr2=0x7f1d84134f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:07.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098b50 0x7f1d84134f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f1d6c001110 tx=0x7f1d6c007a20).stop 2022-07-21T01:54:07.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 -- 172.21.15.140:0/1361914685 shutdown_connections 2022-07-21T01:54:07.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d680539e0 0x7f1d68055ea0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8409abb0 0x7f1d84135420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d84098b50 0x7f1d84134f30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 --2- 172.21.15.140:0/1361914685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84098280 0x7f1d84134430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 -- 172.21.15.140:0/1361914685 >> 172.21.15.140:0/1361914685 conn(0x7f1d840b50a0 msgr2=0x7f1d84069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:07.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 -- 172.21.15.140:0/1361914685 shutdown_connections 2022-07-21T01:54:07.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.419+0000 7f1d8958f700 1 -- 172.21.15.140:0/1361914685 wait complete. 2022-07-21T01:54:07.423 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:54:07.429 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:54:07.430 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:54:07.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 Processor -- start 2022-07-21T01:54:07.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 -- start start 2022-07-21T01:54:07.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f94b00b1e00 0x7f94b014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 0x7f94b00ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b005d480 con 0x7f94b005bd90 2022-07-21T01:54:07.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b00b21e0 con 0x7f94b00b1e00 2022-07-21T01:54:07.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b005cf40 con 0x7f94b005c3f0 2022-07-21T01:54:07.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b5519700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f94b00b1e00 0x7f94b014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b5d1a700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f94b005bd90 0x7f94b0146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50572/0 (socket says 172.21.15.140:50572) 2022-07-21T01:54:07.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b5d1a700 1 -- 172.21.15.140:0/3312442095 learned_addr learned my addr 172.21.15.140:0/3312442095 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:07.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b4d18700 1 --2- 172.21.15.140:0/3312442095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 0x7f94b00ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b5519700 1 -- 172.21.15.140:0/3312442095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 msgr2=0x7f94b00ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:07.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b5519700 1 --2- 172.21.15.140:0/3312442095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 0x7f94b00ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94affff700 1 -- 172.21.15.140:0/3312442095 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674036689 0 0) 0x7f94b005d480 con 0x7f94b005bd90 2022-07-21T01:54:07.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b5519700 1 -- 172.21.15.140:0/3312442095 >> v1:172.21.15.140:6789/0 conn(0x7f94b005bd90 legacy=0x7f94b0146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:07.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b5519700 1 -- 172.21.15.140:0/3312442095 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b005d1e0 con 0x7f94b00b1e00 2022-07-21T01:54:07.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b5519700 1 --2- 172.21.15.140:0/3312442095 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94b00b1e00 0x7f94b014a6d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f94a00032f0 tx=0x7f94a00091d0).ready entity=mon.0 client_cookie=ca56e63374ee456d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:07.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94affff700 1 -- 172.21.15.140:0/3312442095 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94a0009ea0 con 0x7f94b00b1e00 2022-07-21T01:54:07.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94affff700 1 -- 172.21.15.140:0/3312442095 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a000b8a0 con 0x7f94b00b1e00 2022-07-21T01:54:07.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94affff700 1 -- 172.21.15.140:0/3312442095 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94a000ba10 con 0x7f94b00b1e00 2022-07-21T01:54:07.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 -- 172.21.15.140:0/3312442095 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94b00b1e00 msgr2=0x7f94b014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:07.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 --2- 172.21.15.140:0/3312442095 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94b00b1e00 0x7f94b014a6d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f94a00032f0 tx=0x7f94a00091d0).stop 2022-07-21T01:54:07.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 -- 172.21.15.140:0/3312442095 shutdown_connections 2022-07-21T01:54:07.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 --2- 172.21.15.140:0/3312442095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 0x7f94b00ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 --2- 172.21.15.140:0/3312442095 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94b00b1e00 0x7f94b014a6d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 -- 172.21.15.140:0/3312442095 >> 172.21.15.140:0/3312442095 conn(0x7f94b00611c0 msgr2=0x7f94b0062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:07.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 -- 172.21.15.140:0/3312442095 shutdown_connections 2022-07-21T01:54:07.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.539+0000 7f94b777d700 1 -- 172.21.15.140:0/3312442095 wait complete. 2022-07-21T01:54:07.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b777d700 1 Processor -- start 2022-07-21T01:54:07.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b777d700 1 -- start start 2022-07-21T01:54:07.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b777d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 0x7f94b014bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b777d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b00b1e00 0x7f94b0143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b777d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94b009ea40 0x7f94b01472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b777d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b01481c0 con 0x7f94b009ea40 2022-07-21T01:54:07.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b777d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b00a12b0 con 0x7f94b00b1e00 2022-07-21T01:54:07.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b777d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b00a1550 con 0x7f94b005c3f0 2022-07-21T01:54:07.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5d1a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94b009ea40 0x7f94b01472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b4d18700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b00b1e00 0x7f94b0143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5519700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 0x7f94b014bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5d1a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94b009ea40 0x7f94b01472d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36432/0 (socket says 172.21.15.140:36432) 2022-07-21T01:54:07.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b4d18700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b00b1e00 0x7f94b0143070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42170/0 (socket says 172.21.15.140:42170) 2022-07-21T01:54:07.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5519700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 0x7f94b014bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:33158/0 (socket says 172.21.15.140:33158) 2022-07-21T01:54:07.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5d1a700 1 -- 172.21.15.140:0/859043466 learned_addr learned my addr 172.21.15.140:0/859043466 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:07.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5519700 1 -- 172.21.15.140:0/859043466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b00b1e00 msgr2=0x7f94b0143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:07.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5519700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b00b1e00 0x7f94b0143070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5519700 1 -- 172.21.15.140:0/859043466 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94b009ea40 msgr2=0x7f94b01472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:07.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5519700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94b009ea40 0x7f94b01472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5519700 1 -- 172.21.15.140:0/859043466 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0009470 con 0x7f94b005c3f0 2022-07-21T01:54:07.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5d1a700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94b009ea40 0x7f94b01472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:07.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b5519700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 0x7f94b014bb50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f94b014cb90 tx=0x7f94a0016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:07.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94ae7fc700 1 -- 172.21.15.140:0/859043466 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94a000ba10 con 0x7f94b005c3f0 2022-07-21T01:54:07.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94ae7fc700 1 -- 172.21.15.140:0/859043466 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a001f3f0 con 0x7f94b005c3f0 2022-07-21T01:54:07.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94ae7fc700 1 -- 172.21.15.140:0/859043466 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94a0006b60 con 0x7f94b005c3f0 2022-07-21T01:54:07.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b777d700 1 -- 172.21.15.140:0/859043466 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b0147e40 con 0x7f94b005c3f0 2022-07-21T01:54:07.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.543+0000 7f94b777d700 1 -- 172.21.15.140:0/859043466 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b00a1b90 con 0x7f94b005c3f0 2022-07-21T01:54:07.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.547+0000 7f94b777d700 1 -- 172.21.15.140:0/859043466 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b00a1750 con 0x7f94b005c3f0 2022-07-21T01:54:07.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.547+0000 7f94ae7fc700 1 -- 172.21.15.140:0/859043466 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f94a000f070 con 0x7f94b005c3f0 2022-07-21T01:54:07.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.547+0000 7f94ae7fc700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9490053b00 0x7f9490055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.547+0000 7f94ae7fc700 1 -- 172.21.15.140:0/859043466 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f94a006c380 con 0x7f94b005c3f0 2022-07-21T01:54:07.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.547+0000 7f94b4d18700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9490053b00 0x7f9490055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.547+0000 7f94b4d18700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9490053b00 0x7f9490055fc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f94a400c000 tx=0x7f94a4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:07.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.555+0000 7f94ae7fc700 1 -- 172.21.15.140:0/859043466 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f94b00a1750 con 0x7f94b005c3f0 2022-07-21T01:54:07.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.723+0000 7f94b777d700 1 -- 172.21.15.140:0/859043466 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f94b00b04b0 con 0x7f94b005c3f0 2022-07-21T01:54:07.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.723+0000 7f94ae7fc700 1 -- 172.21.15.140:0/859043466 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f94a0015090 con 0x7f94b005c3f0 2022-07-21T01:54:07.720 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:07.721 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:54:07.721 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:54:07.722 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.722 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:54:07.723 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:54:07.723 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.724 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:54:07.725 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:54:07.725 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:54:07.726 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.726 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.727 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.727 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.728 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:54:07.728 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:54:07.729 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.729 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:54:07.730 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:54:07.730 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:54:07.731 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.731 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.732 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.733 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.733 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:54:07.734 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:54:07.734 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.735 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:54:07.735 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:54:07.736 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.736 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.737 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.737 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.738 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:54:07.739 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:54:07.739 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.740 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:54:07.740 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:54:07.741 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.741 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.742 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.742 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.743 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:54:07.743 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:54:07.744 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.745 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:54:07.745 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:54:07.746 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.746 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.747 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.747 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.748 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:54:07.748 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:54:07.748 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.749 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:54:07.749 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:54:07.750 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.750 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.751 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.751 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.752 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:54:07.753 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:54:07.753 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.754 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:54:07.754 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:54:07.755 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.755 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.756 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.756 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.757 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:54:07.757 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:54:07.758 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.758 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:54:07.759 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:54:07.759 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.760 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.760 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.761 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.761 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:54:07.762 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:54:07.763 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.763 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:54:07.764 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:54:07.764 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.765 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.765 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.766 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.766 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:54:07.767 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:54:07.767 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.768 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:54:07.768 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:54:07.769 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.769 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.769 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.770 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.770 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:54:07.771 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:54:07.771 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.772 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:54:07.772 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:54:07.773 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:54:07.773 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.774 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.774 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.774 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:54:07.775 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:54:07.775 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.775 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:54:07.775 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:54:07.776 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:54:07.776 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.776 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.777 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.777 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:54:07.777 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:54:07.778 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.778 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:54:07.778 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:54:07.779 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:54:07.779 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.779 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.779 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.780 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:54:07.780 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:54:07.780 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.781 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:54:07.781 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:54:07.781 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:54:07.782 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.782 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.782 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.782 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.783 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:54:07.783 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:54:07.783 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.783 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:54:07.784 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:54:07.784 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.785 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.785 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:07.786 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:07.786 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:54:07.786 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:54:07.787 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:54:07.787 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:54:07.787 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:54:07.787 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:54:07.788 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.788 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:07.788 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:54:07.789 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:54:07.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 -- 172.21.15.140:0/859043466 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9490053b00 msgr2=0x7f9490055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:07.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9490053b00 0x7f9490055fc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f94a400c000 tx=0x7f94a4008040).stop 2022-07-21T01:54:07.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 -- 172.21.15.140:0/859043466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 msgr2=0x7f94b014bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:07.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 0x7f94b014bb50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f94b014cb90 tx=0x7f94a0016000).stop 2022-07-21T01:54:07.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 -- 172.21.15.140:0/859043466 shutdown_connections 2022-07-21T01:54:07.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9490053b00 0x7f9490055fc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94b009ea40 0x7f94b01472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b00b1e00 0x7f94b0143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 --2- 172.21.15.140:0/859043466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94b005c3f0 0x7f94b014bb50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 -- 172.21.15.140:0/859043466 >> 172.21.15.140:0/859043466 conn(0x7f94b005bd90 msgr2=0x7f94b01485d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:07.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 -- 172.21.15.140:0/859043466 shutdown_connections 2022-07-21T01:54:07.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.727+0000 7f94b777d700 1 -- 172.21.15.140:0/859043466 wait complete. 2022-07-21T01:54:07.797 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:54:07.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 Processor -- start 2022-07-21T01:54:07.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 -- start start 2022-07-21T01:54:07.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6ac0b1e00 0x7fb6ac14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb6ac05c3f0 0x7fb6ac0ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ac05d480 con 0x7fb6ac05bd90 2022-07-21T01:54:07.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ac0b21e0 con 0x7fb6ac05c3f0 2022-07-21T01:54:07.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac05cf40 con 0x7fb6ac0b1e00 2022-07-21T01:54:07.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6abfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb6ac05c3f0 0x7fb6ac0ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6abfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb6ac05c3f0 0x7fb6ac0ac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36436/0 (socket says 172.21.15.140:36436) 2022-07-21T01:54:07.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6abfff700 1 -- 172.21.15.140:0/3443755813 learned_addr learned my addr 172.21.15.140:0/3443755813 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:07.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b0983700 1 --2- 172.21.15.140:0/3443755813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6ac0b1e00 0x7fb6ac14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6abfff700 1 -- 172.21.15.140:0/3443755813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6ac0b1e00 msgr2=0x7fb6ac14a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:07.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6abfff700 1 --2- 172.21.15.140:0/3443755813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6ac0b1e00 0x7fb6ac14a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6abfff700 1 -- 172.21.15.140:0/3443755813 >> v1:172.21.15.140:6789/0 conn(0x7fb6ac05bd90 legacy=0x7fb6ac146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:07.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6abfff700 1 -- 172.21.15.140:0/3443755813 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac05d1e0 con 0x7fb6ac05c3f0 2022-07-21T01:54:07.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6abfff700 1 --2- 172.21.15.140:0/3443755813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6ac05c3f0 0x7fb6ac0ac7e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fb6a0002f70 tx=0x7fb6a0006eb0).ready entity=mon.0 client_cookie=f2fed0be227d5c4f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:07.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6ab7fe700 1 -- 172.21.15.140:0/3443755813 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6a000d5c0 con 0x7fb6ac05c3f0 2022-07-21T01:54:07.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6ab7fe700 1 -- 172.21.15.140:0/3443755813 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0000e90 con 0x7fb6ac05c3f0 2022-07-21T01:54:07.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6ab7fe700 1 -- 172.21.15.140:0/3443755813 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6a000d950 con 0x7fb6ac05c3f0 2022-07-21T01:54:07.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 -- 172.21.15.140:0/3443755813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6ac05c3f0 msgr2=0x7fb6ac0ac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:07.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 --2- 172.21.15.140:0/3443755813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6ac05c3f0 0x7fb6ac0ac7e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fb6a0002f70 tx=0x7fb6a0006eb0).stop 2022-07-21T01:54:07.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 -- 172.21.15.140:0/3443755813 shutdown_connections 2022-07-21T01:54:07.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 --2- 172.21.15.140:0/3443755813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6ac05c3f0 0x7fb6ac0ac7e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 --2- 172.21.15.140:0/3443755813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6ac0b1e00 0x7fb6ac14a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 -- 172.21.15.140:0/3443755813 >> 172.21.15.140:0/3443755813 conn(0x7fb6ac0611c0 msgr2=0x7fb6ac062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:07.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 -- 172.21.15.140:0/3443755813 shutdown_connections 2022-07-21T01:54:07.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.911+0000 7fb6b2be7700 1 -- 172.21.15.140:0/3443755813 wait complete. 2022-07-21T01:54:07.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b2be7700 1 Processor -- start 2022-07-21T01:54:07.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b2be7700 1 -- start start 2022-07-21T01:54:07.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b2be7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6ac05c3f0 0x7fb6ac146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b2be7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6ac0b1e00 0x7fb6ac0b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b2be7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6ac0a0200 0x7fb6ac0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b2be7700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0a1360 con 0x7fb6ac0a0200 2022-07-21T01:54:07.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b2be7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0a16f0 con 0x7fb6ac05c3f0 2022-07-21T01:54:07.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b2be7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0a1990 con 0x7fb6ac0b1e00 2022-07-21T01:54:07.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b1184700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6ac0a0200 0x7fb6ac0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b1184700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6ac0a0200 0x7fb6ac0b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36446/0 (socket says 172.21.15.140:36446) 2022-07-21T01:54:07.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b1184700 1 -- 172.21.15.140:0/711300396 learned_addr learned my addr 172.21.15.140:0/711300396 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:07.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b0983700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6ac05c3f0 0x7fb6ac146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6abfff700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6ac0b1e00 0x7fb6ac0b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b1184700 1 -- 172.21.15.140:0/711300396 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6ac0b1e00 msgr2=0x7fb6ac0b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:07.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b1184700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6ac0b1e00 0x7fb6ac0b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b1184700 1 -- 172.21.15.140:0/711300396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6ac05c3f0 msgr2=0x7fb6ac146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:07.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b1184700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6ac05c3f0 0x7fb6ac146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:07.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b1184700 1 -- 172.21.15.140:0/711300396 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a000b1e0 con 0x7fb6ac0a0200 2022-07-21T01:54:07.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6abfff700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6ac0b1e00 0x7fb6ac0b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:54:07.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b1184700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6ac0a0200 0x7fb6ac0b4ce0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fb6ac0ac6c0 tx=0x7fb6a4007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:07.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6a9ffb700 1 -- 172.21.15.140:0/711300396 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6a400e4a0 con 0x7fb6ac0a0200 2022-07-21T01:54:07.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6a9ffb700 1 -- 172.21.15.140:0/711300396 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a400e600 con 0x7fb6ac0a0200 2022-07-21T01:54:07.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b2be7700 1 -- 172.21.15.140:0/711300396 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac09ea90 con 0x7fb6ac0a0200 2022-07-21T01:54:07.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6a9ffb700 1 -- 172.21.15.140:0/711300396 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6a400e960 con 0x7fb6ac0a0200 2022-07-21T01:54:07.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.915+0000 7fb6b2be7700 1 -- 172.21.15.140:0/711300396 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac0b5370 con 0x7fb6ac0a0200 2022-07-21T01:54:07.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.919+0000 7fb6b2be7700 1 -- 172.21.15.140:0/711300396 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ac0b04b0 con 0x7fb6ac0a0200 2022-07-21T01:54:07.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.919+0000 7fb6a9ffb700 1 -- 172.21.15.140:0/711300396 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb6a4019030 con 0x7fb6ac0a0200 2022-07-21T01:54:07.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.919+0000 7fb6a9ffb700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb68c053aa0 0x7fb68c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:07.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.919+0000 7fb6a9ffb700 1 -- 172.21.15.140:0/711300396 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb6a406b390 con 0x7fb6ac0a0200 2022-07-21T01:54:07.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.919+0000 7fb6b0983700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb68c053aa0 0x7fb68c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:07.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.919+0000 7fb6b0983700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb68c053aa0 0x7fb68c055f60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fb69c00c000 tx=0x7fb69c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:07.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:07.927+0000 7fb6a9ffb700 1 -- 172.21.15.140:0/711300396 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb6a4042470 con 0x7fb6ac0a0200 2022-07-21T01:54:08.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.047+0000 7fb6b2be7700 1 -- 172.21.15.140:0/711300396 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6ac0574e0 con 0x7fb6ac0a0200 2022-07-21T01:54:08.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.047+0000 7fb6a9ffb700 1 -- 172.21.15.140:0/711300396 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+13278 (secure 0 0 0) 0x7fb6a4045a90 con 0x7fb6ac0a0200 2022-07-21T01:54:08.044 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:08.045 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":67,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:54:07.409920+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:54:08.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 -- 172.21.15.140:0/711300396 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb68c053aa0 msgr2=0x7fb68c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb68c053aa0 0x7fb68c055f60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7fb69c00c000 tx=0x7fb69c008040).stop 2022-07-21T01:54:08.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 -- 172.21.15.140:0/711300396 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6ac0a0200 msgr2=0x7fb6ac0b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6ac0a0200 0x7fb6ac0b4ce0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fb6ac0ac6c0 tx=0x7fb6a4007f40).stop 2022-07-21T01:54:08.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 -- 172.21.15.140:0/711300396 shutdown_connections 2022-07-21T01:54:08.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb68c053aa0 0x7fb68c055f60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6ac0a0200 0x7fb6ac0b4ce0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6ac0b1e00 0x7fb6ac0b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 --2- 172.21.15.140:0/711300396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6ac05c3f0 0x7fb6ac146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 -- 172.21.15.140:0/711300396 >> 172.21.15.140:0/711300396 conn(0x7fb6ac05bd90 msgr2=0x7fb6ac0ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:08.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 -- 172.21.15.140:0/711300396 shutdown_connections 2022-07-21T01:54:08.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.051+0000 7fb6b2be7700 1 -- 172.21.15.140:0/711300396 wait complete. 2022-07-21T01:54:08.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:54:08.059 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:54:08.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.171+0000 7fe5c99f5700 1 Processor -- start 2022-07-21T01:54:08.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.171+0000 7fe5c99f5700 1 -- start start 2022-07-21T01:54:08.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe5c409df40 0x7fe5c4145680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe5c40b4d60 0x7fe5c4145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c40607e0 con 0x7fe5c4060ee0 2022-07-21T01:54:08.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c4060d20 con 0x7fe5c40b4d60 2022-07-21T01:54:08.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c4060a80 con 0x7fe5c409df40 2022-07-21T01:54:08.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c37fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe5c4060ee0 0x7fe5c4146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50598/0 (socket says 172.21.15.140:50598) 2022-07-21T01:54:08.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c37fe700 1 -- 172.21.15.140:0/3534600209 learned_addr learned my addr 172.21.15.140:0/3534600209 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:08.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5ba7fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe5c40b4d60 0x7fe5c4145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c2ffd700 1 --2- 172.21.15.140:0/3534600209 >> v2:172.21.15.152:3300/0 conn(0x7fe5c409df40 0x7fe5c4145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c27fc700 1 -- 172.21.15.140:0/3534600209 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184876304 0 0) 0x7fe5c40607e0 con 0x7fe5c4060ee0 2022-07-21T01:54:08.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c27fc700 1 -- 172.21.15.140:0/3534600209 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5a4003260 con 0x7fe5c4060ee0 2022-07-21T01:54:08.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5ba7fc700 1 -- 172.21.15.140:0/3534600209 >> v2:172.21.15.152:3300/0 conn(0x7fe5c409df40 msgr2=0x7fe5c4145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:08.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5ba7fc700 1 --2- 172.21.15.140:0/3534600209 >> v2:172.21.15.152:3300/0 conn(0x7fe5c409df40 0x7fe5c4145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5ba7fc700 1 -- 172.21.15.140:0/3534600209 >> v1:172.21.15.140:6789/0 conn(0x7fe5c4060ee0 legacy=0x7fe5c4146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5ba7fc700 1 -- 172.21.15.140:0/3534600209 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c409e320 con 0x7fe5c40b4d60 2022-07-21T01:54:08.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5ba7fc700 1 --2- 172.21.15.140:0/3534600209 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5c40b4d60 0x7fe5c4145b70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fe5ac002f70 tx=0x7fe5ac006df0).ready entity=mon.0 client_cookie=10dec2c35e0ea9f8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c27fc700 1 -- 172.21.15.140:0/3534600209 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5ac00d4f0 con 0x7fe5c40b4d60 2022-07-21T01:54:08.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c27fc700 1 -- 172.21.15.140:0/3534600209 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ac000e90 con 0x7fe5c40b4d60 2022-07-21T01:54:08.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c27fc700 1 -- 172.21.15.140:0/3534600209 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5ac00d840 con 0x7fe5c40b4d60 2022-07-21T01:54:08.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3534600209 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5c40b4d60 msgr2=0x7fe5c4145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 --2- 172.21.15.140:0/3534600209 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5c40b4d60 0x7fe5c4145b70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fe5ac002f70 tx=0x7fe5ac006df0).stop 2022-07-21T01:54:08.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3534600209 shutdown_connections 2022-07-21T01:54:08.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 --2- 172.21.15.140:0/3534600209 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5c40b4d60 0x7fe5c4145b70 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 --2- 172.21.15.140:0/3534600209 >> v2:172.21.15.152:3300/0 conn(0x7fe5c409df40 0x7fe5c4145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3534600209 >> 172.21.15.140:0/3534600209 conn(0x7fe5c40b5f30 msgr2=0x7fe5c40b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:08.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3534600209 shutdown_connections 2022-07-21T01:54:08.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.175+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3534600209 wait complete. 2022-07-21T01:54:08.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c99f5700 1 Processor -- start 2022-07-21T01:54:08.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c99f5700 1 -- start start 2022-07-21T01:54:08.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c99f5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5c4060ee0 0x7fe5c40b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c99f5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c409df40 0x7fe5c4155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c99f5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5c40b4d60 0x7fe5c41525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c99f5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c4092ee0 con 0x7fe5c40b4d60 2022-07-21T01:54:08.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c99f5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40929c0 con 0x7fe5c409df40 2022-07-21T01:54:08.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c99f5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c4155a40 con 0x7fe5c4060ee0 2022-07-21T01:54:08.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5c40b4d60 0x7fe5c41525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5c40b4d60 0x7fe5c41525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36460/0 (socket says 172.21.15.140:36460) 2022-07-21T01:54:08.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c37fe700 1 -- 172.21.15.140:0/3141423976 learned_addr learned my addr 172.21.15.140:0/3141423976 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:08.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c2ffd700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5c4060ee0 0x7fe5c40b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5ba7fc700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c409df40 0x7fe5c4155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c37fe700 1 -- 172.21.15.140:0/3141423976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5c4060ee0 msgr2=0x7fe5c40b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c37fe700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5c4060ee0 0x7fe5c40b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c37fe700 1 -- 172.21.15.140:0/3141423976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c409df40 msgr2=0x7fe5c4155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c37fe700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c409df40 0x7fe5c4155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c37fe700 1 -- 172.21.15.140:0/3141423976 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac00b180 con 0x7fe5c40b4d60 2022-07-21T01:54:08.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c37fe700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5c40b4d60 0x7fe5c41525c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fe5bc001490 tx=0x7fe5bc007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c0ff9700 1 -- 172.21.15.140:0/3141423976 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5bc005ba0 con 0x7fe5c40b4d60 2022-07-21T01:54:08.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c0ff9700 1 -- 172.21.15.140:0/3141423976 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc005d00 con 0x7fe5c40b4d60 2022-07-21T01:54:08.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3141423976 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c4092c40 con 0x7fe5c40b4d60 2022-07-21T01:54:08.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c0ff9700 1 -- 172.21.15.140:0/3141423976 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5bc0072c0 con 0x7fe5c40b4d60 2022-07-21T01:54:08.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.179+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3141423976 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c40b19a0 con 0x7fe5c40b4d60 2022-07-21T01:54:08.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.183+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3141423976 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c4151f50 con 0x7fe5c40b4d60 2022-07-21T01:54:08.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.183+0000 7fe5c0ff9700 1 -- 172.21.15.140:0/3141423976 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe5bc010070 con 0x7fe5c40b4d60 2022-07-21T01:54:08.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.183+0000 7fe5c0ff9700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5a8053aa0 0x7fe5a8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.183+0000 7fe5c0ff9700 1 -- 172.21.15.140:0/3141423976 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe5bc06b450 con 0x7fe5c40b4d60 2022-07-21T01:54:08.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.183+0000 7fe5c2ffd700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5a8053aa0 0x7fe5a8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.183+0000 7fe5c2ffd700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5a8053aa0 0x7fe5a8055f60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fe5b4006000 tx=0x7fe5b4006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.187+0000 7fe5c0ff9700 1 -- 172.21.15.140:0/3141423976 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe5bc0426a0 con 0x7fe5c40b4d60 2022-07-21T01:54:08.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.311+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3141423976 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe5c40574e0 con 0x7fe5c40b4d60 2022-07-21T01:54:08.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c0ff9700 1 -- 172.21.15.140:0/3141423976 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v67) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe5bc045cc0 con 0x7fe5c40b4d60 2022-07-21T01:54:08.310 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:54:08.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3141423976 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5a8053aa0 msgr2=0x7fe5a8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5a8053aa0 0x7fe5a8055f60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fe5b4006000 tx=0x7fe5b4006d90).stop 2022-07-21T01:54:08.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3141423976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5c40b4d60 msgr2=0x7fe5c41525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5c40b4d60 0x7fe5c41525c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fe5bc001490 tx=0x7fe5bc007e00).stop 2022-07-21T01:54:08.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3141423976 shutdown_connections 2022-07-21T01:54:08.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5a8053aa0 0x7fe5a8055f60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5c40b4d60 0x7fe5c41525c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5c409df40 0x7fe5c4155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 --2- 172.21.15.140:0/3141423976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5c4060ee0 0x7fe5c40b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3141423976 >> 172.21.15.140:0/3141423976 conn(0x7fe5c40b5f30 msgr2=0x7fe5c40b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:08.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3141423976 shutdown_connections 2022-07-21T01:54:08.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.315+0000 7fe5c99f5700 1 -- 172.21.15.140:0/3141423976 wait complete. 2022-07-21T01:54:08.322 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:54:08.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 Processor -- start 2022-07-21T01:54:08.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 -- start start 2022-07-21T01:54:08.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4bbc0b2210 0x7f4bbc140860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4bbc0967e0 0x7f4bbc0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4bbc0ac680 0x7f4bbc0ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bbc13fe80 con 0x7f4bbc0ac680 2022-07-21T01:54:08.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bbc140120 con 0x7f4bbc0b2210 2022-07-21T01:54:08.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc1403c0 con 0x7f4bbc0967e0 2022-07-21T01:54:08.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc0e3b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4bbc0ac680 0x7f4bbc0ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc0e3b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4bbc0ac680 0x7f4bbc0ab0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36468/0 (socket says 172.21.15.140:36468) 2022-07-21T01:54:08.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc0e3b700 1 -- 172.21.15.140:0/3913345790 learned_addr learned my addr 172.21.15.140:0/3913345790 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:08.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bbb7fe700 1 --2- 172.21.15.140:0/3913345790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4bbc0967e0 0x7f4bbc0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bbbfff700 1 --2- 172.21.15.140:0/3913345790 >> v2:172.21.15.152:3300/0 conn(0x7f4bbc0b2210 0x7f4bbc140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc0e3b700 1 -- 172.21.15.140:0/3913345790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4bbc0967e0 msgr2=0x7f4bbc0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:08.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc0e3b700 1 --2- 172.21.15.140:0/3913345790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4bbc0967e0 0x7f4bbc0b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc0e3b700 1 -- 172.21.15.140:0/3913345790 >> v2:172.21.15.152:3300/0 conn(0x7f4bbc0b2210 msgr2=0x7f4bbc140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:08.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc0e3b700 1 --2- 172.21.15.140:0/3913345790 >> v2:172.21.15.152:3300/0 conn(0x7f4bbc0b2210 0x7f4bbc140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc0e3b700 1 -- 172.21.15.140:0/3913345790 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc0ab5a0 con 0x7f4bbc0ac680 2022-07-21T01:54:08.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bbbfff700 1 --2- 172.21.15.140:0/3913345790 >> v2:172.21.15.152:3300/0 conn(0x7f4bbc0b2210 0x7f4bbc140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:08.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc0e3b700 1 --2- 172.21.15.140:0/3913345790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4bbc0ac680 0x7f4bbc0ab0b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f4bac002f70 tx=0x7f4bac006eb0).ready entity=mon.0 client_cookie=d1b2e09a26a7221a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bbaffd700 1 -- 172.21.15.140:0/3913345790 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bac00d570 con 0x7f4bbc0ac680 2022-07-21T01:54:08.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bbaffd700 1 -- 172.21.15.140:0/3913345790 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac000e90 con 0x7f4bbc0ac680 2022-07-21T01:54:08.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bbaffd700 1 -- 172.21.15.140:0/3913345790 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bac00d900 con 0x7f4bbc0ac680 2022-07-21T01:54:08.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 -- 172.21.15.140:0/3913345790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4bbc0ac680 msgr2=0x7f4bbc0ab0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 --2- 172.21.15.140:0/3913345790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4bbc0ac680 0x7f4bbc0ab0b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f4bac002f70 tx=0x7f4bac006eb0).stop 2022-07-21T01:54:08.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 -- 172.21.15.140:0/3913345790 shutdown_connections 2022-07-21T01:54:08.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 --2- 172.21.15.140:0/3913345790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4bbc0ac680 0x7f4bbc0ab0b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 --2- 172.21.15.140:0/3913345790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4bbc0967e0 0x7f4bbc0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 --2- 172.21.15.140:0/3913345790 >> v2:172.21.15.152:3300/0 conn(0x7f4bbc0b2210 0x7f4bbc140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 -- 172.21.15.140:0/3913345790 >> 172.21.15.140:0/3913345790 conn(0x7f4bbc0b36c0 msgr2=0x7f4bbc0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:08.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 -- 172.21.15.140:0/3913345790 shutdown_connections 2022-07-21T01:54:08.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.427+0000 7f4bc289e700 1 -- 172.21.15.140:0/3913345790 wait complete. 2022-07-21T01:54:08.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bc289e700 1 Processor -- start 2022-07-21T01:54:08.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bc289e700 1 -- start start 2022-07-21T01:54:08.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bc289e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4bbc0967e0 0x7f4bbc0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bc289e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bbc0ac680 0x7f4bbc13e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bc289e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4bbc0b2210 0x7f4bbc0af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bc289e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc0adb40 con 0x7f4bbc0967e0 2022-07-21T01:54:08.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bc289e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc13df20 con 0x7f4bbc0ac680 2022-07-21T01:54:08.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bc289e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc0af9b0 con 0x7f4bbc0b2210 2022-07-21T01:54:08.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bbbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4bbc0967e0 0x7f4bbc0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bbbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4bbc0967e0 0x7f4bbc0affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36470/0 (socket says 172.21.15.140:36470) 2022-07-21T01:54:08.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bbbfff700 1 -- 172.21.15.140:0/481658222 learned_addr learned my addr 172.21.15.140:0/481658222 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:08.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bbb7fe700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bbc0ac680 0x7f4bbc13e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bc0e3b700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4bbc0b2210 0x7f4bbc0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bbbfff700 1 -- 172.21.15.140:0/481658222 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4bbc0b2210 msgr2=0x7f4bbc0af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bbbfff700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4bbc0b2210 0x7f4bbc0af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bbbfff700 1 -- 172.21.15.140:0/481658222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bbc0ac680 msgr2=0x7f4bbc13e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bbbfff700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bbc0ac680 0x7f4bbc13e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bbbfff700 1 -- 172.21.15.140:0/481658222 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac00b1e0 con 0x7f4bbc0967e0 2022-07-21T01:54:08.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bbbfff700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4bbc0967e0 0x7f4bbc0affc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f4bbc09c0b0 tx=0x7f4bb4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bb97fa700 1 -- 172.21.15.140:0/481658222 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bb4011780 con 0x7f4bbc0967e0 2022-07-21T01:54:08.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bb97fa700 1 -- 172.21.15.140:0/481658222 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb40118e0 con 0x7f4bbc0967e0 2022-07-21T01:54:08.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bc289e700 1 -- 172.21.15.140:0/481658222 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bbc0afcf0 con 0x7f4bbc0967e0 2022-07-21T01:54:08.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.431+0000 7f4bb97fa700 1 -- 172.21.15.140:0/481658222 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bb4011c40 con 0x7f4bbc0967e0 2022-07-21T01:54:08.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.435+0000 7f4bc289e700 1 -- 172.21.15.140:0/481658222 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bbc099a10 con 0x7f4bbc0967e0 2022-07-21T01:54:08.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.435+0000 7f4bb97fa700 1 -- 172.21.15.140:0/481658222 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4bb4012420 con 0x7f4bbc0967e0 2022-07-21T01:54:08.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.435+0000 7f4bc289e700 1 -- 172.21.15.140:0/481658222 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bbc0574e0 con 0x7f4bbc0967e0 2022-07-21T01:54:08.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.435+0000 7f4bb97fa700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ba0053980 0x7f4ba0055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.435+0000 7f4bb97fa700 1 -- 172.21.15.140:0/481658222 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f4bb406bce0 con 0x7f4bbc0967e0 2022-07-21T01:54:08.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.439+0000 7f4bbb7fe700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ba0053980 0x7f4ba0055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.439+0000 7f4bbb7fe700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ba0053980 0x7f4ba0055e40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f4ba400c000 tx=0x7f4ba4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.439+0000 7f4bb97fa700 1 -- 172.21.15.140:0/481658222 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4bb4042ed0 con 0x7f4bbc0967e0 2022-07-21T01:54:08.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.563+0000 7f4bc289e700 1 -- 172.21.15.140:0/481658222 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4bbc05db10 con 0x7f4bbc0967e0 2022-07-21T01:54:08.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bb97fa700 1 -- 172.21.15.140:0/481658222 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v67) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4bb40464f0 con 0x7f4bbc0967e0 2022-07-21T01:54:08.562 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:54:08.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 -- 172.21.15.140:0/481658222 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ba0053980 msgr2=0x7f4ba0055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ba0053980 0x7f4ba0055e40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f4ba400c000 tx=0x7f4ba4008040).stop 2022-07-21T01:54:08.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 -- 172.21.15.140:0/481658222 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4bbc0967e0 msgr2=0x7f4bbc0affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4bbc0967e0 0x7f4bbc0affc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f4bbc09c0b0 tx=0x7f4bb4008040).stop 2022-07-21T01:54:08.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 -- 172.21.15.140:0/481658222 shutdown_connections 2022-07-21T01:54:08.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ba0053980 0x7f4ba0055e40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4bbc0b2210 0x7f4bbc0af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bbc0ac680 0x7f4bbc13e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 --2- 172.21.15.140:0/481658222 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4bbc0967e0 0x7f4bbc0affc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 -- 172.21.15.140:0/481658222 >> 172.21.15.140:0/481658222 conn(0x7f4bbc0b36c0 msgr2=0x7f4bbc13ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:08.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 -- 172.21.15.140:0/481658222 shutdown_connections 2022-07-21T01:54:08.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.567+0000 7f4bc289e700 1 -- 172.21.15.140:0/481658222 wait complete. 2022-07-21T01:54:08.574 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:54:08.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05abf29700 1 Processor -- start 2022-07-21T01:54:08.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05abf29700 1 -- start start 2022-07-21T01:54:08.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05abf29700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05a40b1e00 0x7f05a414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05abf29700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f05a405c3f0 0x7f05a40ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05abf29700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f05a405bd90 0x7f05a40accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05abf29700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a405d480 con 0x7f05a405c3f0 2022-07-21T01:54:08.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05abf29700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a40ad1c0 con 0x7f05a405bd90 2022-07-21T01:54:08.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05abf29700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a40b21e0 con 0x7f05a40b1e00 2022-07-21T01:54:08.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a94c4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f05a405c3f0 0x7f05a40ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a9cc5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05a40b1e00 0x7f05a414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a94c4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f05a405c3f0 0x7f05a40ac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36480/0 (socket says 172.21.15.140:36480) 2022-07-21T01:54:08.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a94c4700 1 -- 172.21.15.140:0/2684344462 learned_addr learned my addr 172.21.15.140:0/2684344462 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:08.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05aa4c6700 1 --2- 172.21.15.140:0/2684344462 >> v2:172.21.15.152:3300/0 conn(0x7f05a405bd90 0x7f05a40accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a94c4700 1 -- 172.21.15.140:0/2684344462 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05a40b1e00 msgr2=0x7f05a414a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:08.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a94c4700 1 --2- 172.21.15.140:0/2684344462 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05a40b1e00 0x7f05a414a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a94c4700 1 -- 172.21.15.140:0/2684344462 >> v2:172.21.15.152:3300/0 conn(0x7f05a405bd90 msgr2=0x7f05a40accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:08.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a94c4700 1 --2- 172.21.15.140:0/2684344462 >> v2:172.21.15.152:3300/0 conn(0x7f05a405bd90 0x7f05a40accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a94c4700 1 -- 172.21.15.140:0/2684344462 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a405cf40 con 0x7f05a405c3f0 2022-07-21T01:54:08.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a94c4700 1 --2- 172.21.15.140:0/2684344462 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05a405c3f0 0x7f05a40ac7e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f0594002f70 tx=0x7f0594006eb0).ready entity=mon.0 client_cookie=f43bc8acb0fc55ab server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a8cc3700 1 -- 172.21.15.140:0/2684344462 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f059400d570 con 0x7f05a405c3f0 2022-07-21T01:54:08.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a8cc3700 1 -- 172.21.15.140:0/2684344462 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0594000e90 con 0x7f05a405c3f0 2022-07-21T01:54:08.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.671+0000 7f05a8cc3700 1 -- 172.21.15.140:0/2684344462 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f059400d900 con 0x7f05a405c3f0 2022-07-21T01:54:08.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 -- 172.21.15.140:0/2684344462 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05a405c3f0 msgr2=0x7f05a40ac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 --2- 172.21.15.140:0/2684344462 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05a405c3f0 0x7f05a40ac7e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f0594002f70 tx=0x7f0594006eb0).stop 2022-07-21T01:54:08.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 -- 172.21.15.140:0/2684344462 shutdown_connections 2022-07-21T01:54:08.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 --2- 172.21.15.140:0/2684344462 >> v2:172.21.15.152:3300/0 conn(0x7f05a405bd90 0x7f05a40accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 --2- 172.21.15.140:0/2684344462 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05a405c3f0 0x7f05a40ac7e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 --2- 172.21.15.140:0/2684344462 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05a40b1e00 0x7f05a414a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 -- 172.21.15.140:0/2684344462 >> 172.21.15.140:0/2684344462 conn(0x7f05a40611c0 msgr2=0x7f05a4062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:08.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 -- 172.21.15.140:0/2684344462 shutdown_connections 2022-07-21T01:54:08.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 -- 172.21.15.140:0/2684344462 wait complete. 2022-07-21T01:54:08.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 Processor -- start 2022-07-21T01:54:08.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 -- start start 2022-07-21T01:54:08.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05a405c3f0 0x7f05a409e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05a40b1e00 0x7f05a409fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a40b5bb0 0x7f05a40a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a40b59a0 con 0x7f05a40b1e00 2022-07-21T01:54:08.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a40b4fc0 con 0x7f05a40b5bb0 2022-07-21T01:54:08.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05abf29700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a40b5260 con 0x7f05a405c3f0 2022-07-21T01:54:08.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05a94c4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05a40b1e00 0x7f05a409fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05a94c4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05a40b1e00 0x7f05a409fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36484/0 (socket says 172.21.15.140:36484) 2022-07-21T01:54:08.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05a94c4700 1 -- 172.21.15.140:0/3053627369 learned_addr learned my addr 172.21.15.140:0/3053627369 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:08.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05aa4c6700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a40b5bb0 0x7f05a40a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05a9cc5700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05a405c3f0 0x7f05a409e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05a94c4700 1 -- 172.21.15.140:0/3053627369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05a405c3f0 msgr2=0x7f05a409e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05a94c4700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05a405c3f0 0x7f05a409e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05a94c4700 1 -- 172.21.15.140:0/3053627369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a40b5bb0 msgr2=0x7f05a40a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05a94c4700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a40b5bb0 0x7f05a40a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05a94c4700 1 -- 172.21.15.140:0/3053627369 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059400b1e0 con 0x7f05a40b1e00 2022-07-21T01:54:08.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.675+0000 7f05aa4c6700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a40b5bb0 0x7f05a40a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:08.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f05a94c4700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05a40b1e00 0x7f05a409fd60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f0594002f70 tx=0x7f0594015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f0592ffd700 1 -- 172.21.15.140:0/3053627369 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0594011690 con 0x7f05a40b1e00 2022-07-21T01:54:08.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f0592ffd700 1 -- 172.21.15.140:0/3053627369 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05940117f0 con 0x7f05a40b1e00 2022-07-21T01:54:08.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f05abf29700 1 -- 172.21.15.140:0/3053627369 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a40b5500 con 0x7f05a40b1e00 2022-07-21T01:54:08.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f0592ffd700 1 -- 172.21.15.140:0/3053627369 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0594011b50 con 0x7f05a40b1e00 2022-07-21T01:54:08.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f05abf29700 1 -- 172.21.15.140:0/3053627369 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a40b48c0 con 0x7f05a40b1e00 2022-07-21T01:54:08.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f05abf29700 1 -- 172.21.15.140:0/3053627369 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a40b5690 con 0x7f05a40b1e00 2022-07-21T01:54:08.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f0592ffd700 1 -- 172.21.15.140:0/3053627369 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0594025030 con 0x7f05a40b1e00 2022-07-21T01:54:08.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f0592ffd700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0584053b00 0x7f0584055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f0592ffd700 1 -- 172.21.15.140:0/3053627369 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f059406cf80 con 0x7f05a40b1e00 2022-07-21T01:54:08.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f05a9cc5700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0584053b00 0x7f0584055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.679+0000 7f05a9cc5700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0584053b00 0x7f0584055fc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f059c00b000 tx=0x7f059c0091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.687+0000 7f0592ffd700 1 -- 172.21.15.140:0/3053627369 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f05940441d0 con 0x7f05a40b1e00 2022-07-21T01:54:08.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 -- 172.21.15.140:0/3053627369 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f05a405d1e0 con 0x7f05a40b1e00 2022-07-21T01:54:08.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f0592ffd700 1 -- 172.21.15.140:0/3053627369 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0594023020 con 0x7f05a40b1e00 2022-07-21T01:54:08.815 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:08.815 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:54:08.816 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:08.816 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:54:08.817 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:54:08.817 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:54:08.818 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:54:08.819 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:54:08.819 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:54:08.820 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:54:08.820 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:54:08.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 -- 172.21.15.140:0/3053627369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0584053b00 msgr2=0x7f0584055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0584053b00 0x7f0584055fc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f059c00b000 tx=0x7f059c0091d0).stop 2022-07-21T01:54:08.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 -- 172.21.15.140:0/3053627369 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05a40b1e00 msgr2=0x7f05a409fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05a40b1e00 0x7f05a409fd60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f0594002f70 tx=0x7f0594015000).stop 2022-07-21T01:54:08.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 -- 172.21.15.140:0/3053627369 shutdown_connections 2022-07-21T01:54:08.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0584053b00 0x7f0584055fc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a40b5bb0 0x7f05a40a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05a40b1e00 0x7f05a409fd60 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 --2- 172.21.15.140:0/3053627369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05a405c3f0 0x7f05a409e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 -- 172.21.15.140:0/3053627369 >> 172.21.15.140:0/3053627369 conn(0x7f05a405bd90 msgr2=0x7f05a40afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:08.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 -- 172.21.15.140:0/3053627369 shutdown_connections 2022-07-21T01:54:08.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.819+0000 7f05abf29700 1 -- 172.21.15.140:0/3053627369 wait complete. 2022-07-21T01:54:08.831 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:54:08.832 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:54:08.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.939+0000 7f584cc38700 1 Processor -- start 2022-07-21T01:54:08.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.939+0000 7f584cc38700 1 -- start start 2022-07-21T01:54:08.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584cc38700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58480b5b50 0x7f58480accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584cc38700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f584805c120 0x7f5848144320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584cc38700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480ac5e0 con 0x7f58480b0480 2022-07-21T01:54:08.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584cc38700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58480b5f30 con 0x7f584805c120 2022-07-21T01:54:08.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584cc38700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584805cf70 con 0x7f58480b5b50 2022-07-21T01:54:08.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5845d9b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f58480b0480 0x7f584814a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50638/0 (socket says 172.21.15.140:50638) 2022-07-21T01:54:08.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5845d9b700 1 -- 172.21.15.140:0/3146519412 learned_addr learned my addr 172.21.15.140:0/3146519412 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:08.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5846d9d700 1 --2- 172.21.15.140:0/3146519412 >> v2:172.21.15.152:3300/0 conn(0x7f584805c120 0x7f5848144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584659c700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58480b5b50 0x7f58480accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267773098 0 0) 0x7f58480ac5e0 con 0x7f58480b0480 2022-07-21T01:54:08.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5828003260 con 0x7f58480b0480 2022-07-21T01:54:08.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1583707291 0 0) 0x7f5828003260 con 0x7f58480b0480 2022-07-21T01:54:08.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58480b5b50 msgr2=0x7f58480accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:08.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58480b5b50 0x7f58480accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 >> v2:172.21.15.152:3300/0 conn(0x7f584805c120 msgr2=0x7f5848144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:08.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 --2- 172.21.15.140:0/3146519412 >> v2:172.21.15.152:3300/0 conn(0x7f584805c120 0x7f5848144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480ac810 con 0x7f58480b0480 2022-07-21T01:54:08.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f5838001a00 con 0x7f58480b0480 2022-07-21T01:54:08.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 >> v1:172.21.15.140:6789/0 conn(0x7f58480b0480 legacy=0x7f584814a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5828003750 0x7f5828003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58480b5b50 0x7f5828004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58280049f0 0x7f5828004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480b5f30 con 0x7f58280049f0 2022-07-21T01:54:08.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584805cf70 con 0x7f5828003750 2022-07-21T01:54:08.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582800b750 con 0x7f58480b5b50 2022-07-21T01:54:08.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5846d9d700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58280049f0 0x7f5828004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584659c700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5828003750 0x7f5828003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5845d9b700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58480b5b50 0x7f5828004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5846d9d700 1 -- 172.21.15.140:0/3146519412 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58480b5b50 msgr2=0x7f5828004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:08.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5846d9d700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58480b5b50 0x7f5828004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5846d9d700 1 -- 172.21.15.140:0/3146519412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5828003750 msgr2=0x7f5828003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:08.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5846d9d700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5828003750 0x7f5828003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5846d9d700 1 -- 172.21.15.140:0/3146519412 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5828003260 con 0x7f58280049f0 2022-07-21T01:54:08.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5846d9d700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58280049f0 0x7f5828004e70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f584000c000 tx=0x7f5840006df0).ready entity=mon.0 client_cookie=718f4ab526a835c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f584000e5f0 con 0x7f58280049f0 2022-07-21T01:54:08.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584000e750 con 0x7f58280049f0 2022-07-21T01:54:08.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584559a700 1 -- 172.21.15.140:0/3146519412 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f584000eab0 con 0x7f58280049f0 2022-07-21T01:54:08.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584cc38700 1 -- 172.21.15.140:0/3146519412 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58280049f0 msgr2=0x7f5828004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584cc38700 1 --2- 172.21.15.140:0/3146519412 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58280049f0 0x7f5828004e70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f584000c000 tx=0x7f5840006df0).stop 2022-07-21T01:54:08.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f5846d9d700 1 -- 172.21.15.140:0/3146519412 reap_dead start 2022-07-21T01:54:08.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584cc38700 1 -- 172.21.15.140:0/3146519412 shutdown_connections 2022-07-21T01:54:08.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584cc38700 1 -- 172.21.15.140:0/3146519412 >> 172.21.15.140:0/3146519412 conn(0x7f58480635e0 msgr2=0x7f58480b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:08.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584cc38700 1 -- 172.21.15.140:0/3146519412 shutdown_connections 2022-07-21T01:54:08.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.943+0000 7f584cc38700 1 -- 172.21.15.140:0/3146519412 wait complete. 2022-07-21T01:54:08.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f584cc38700 1 Processor -- start 2022-07-21T01:54:08.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f584cc38700 1 -- start start 2022-07-21T01:54:08.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f584cc38700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58481496f0 0x7f5848149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f584cc38700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f584809e560 0x7f5848156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f584cc38700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f584815a260 0x7f584815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f584cc38700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584809ece0 con 0x7f584815a260 2022-07-21T01:54:08.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f584cc38700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5848159dc0 con 0x7f58481496f0 2022-07-21T01:54:08.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f584cc38700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584809e9a0 con 0x7f584809e560 2022-07-21T01:54:08.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f5846d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f584815a260 0x7f584815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f5846d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f584815a260 0x7f584815a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36506/0 (socket says 172.21.15.140:36506) 2022-07-21T01:54:08.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f584659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58481496f0 0x7f5848149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f5846d9d700 1 -- 172.21.15.140:0/1162074866 learned_addr learned my addr 172.21.15.140:0/1162074866 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:08.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.947+0000 7f5845d9b700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f584809e560 0x7f5848156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f5846d9d700 1 -- 172.21.15.140:0/1162074866 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f584809e560 msgr2=0x7f5848156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f5846d9d700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f584809e560 0x7f5848156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f5846d9d700 1 -- 172.21.15.140:0/1162074866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58481496f0 msgr2=0x7f5848149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:08.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f5846d9d700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58481496f0 0x7f5848149ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:08.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f5846d9d700 1 -- 172.21.15.140:0/1162074866 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5848156b30 con 0x7f584815a260 2022-07-21T01:54:08.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f5846d9d700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f584815a260 0x7f584815a710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f5848061810 tx=0x7f5840017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f583f7fe700 1 -- 172.21.15.140:0/1162074866 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f584001d810 con 0x7f584815a260 2022-07-21T01:54:08.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f583f7fe700 1 -- 172.21.15.140:0/1162074866 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584001d970 con 0x7f584815a260 2022-07-21T01:54:08.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f583f7fe700 1 -- 172.21.15.140:0/1162074866 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f584001dd10 con 0x7f584815a260 2022-07-21T01:54:08.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f584cc38700 1 -- 172.21.15.140:0/1162074866 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5848063a10 con 0x7f584815a260 2022-07-21T01:54:08.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f584cc38700 1 -- 172.21.15.140:0/1162074866 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5848063f40 con 0x7f584815a260 2022-07-21T01:54:08.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f584cc38700 1 -- 172.21.15.140:0/1162074866 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5848159f80 con 0x7f584815a260 2022-07-21T01:54:08.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f583f7fe700 1 -- 172.21.15.140:0/1162074866 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f58400123f0 con 0x7f584815a260 2022-07-21T01:54:08.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f583f7fe700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f582c053a10 0x7f582c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:08.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f583f7fe700 1 -- 172.21.15.140:0/1162074866 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f5840000e40 con 0x7f584815a260 2022-07-21T01:54:08.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.951+0000 7f584659c700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f582c053a10 0x7f582c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:08.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.955+0000 7f584659c700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f582c053a10 0x7f582c055ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f583000b000 tx=0x7f5830006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:08.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:08.959+0000 7f583f7fe700 1 -- 172.21.15.140:0/1162074866 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f58400446e0 con 0x7f584815a260 2022-07-21T01:54:09.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.083+0000 7f584cc38700 1 -- 172.21.15.140:0/1162074866 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f58480574e0 con 0x7f584815a260 2022-07-21T01:54:09.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.431+0000 7f583f7fe700 1 -- 172.21.15.140:0/1162074866 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v68) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5840047d00 con 0x7f584815a260 2022-07-21T01:54:09.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 -- 172.21.15.140:0/1162074866 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f582c053a10 msgr2=0x7f582c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:09.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f582c053a10 0x7f582c055ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f583000b000 tx=0x7f5830006df0).stop 2022-07-21T01:54:09.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 -- 172.21.15.140:0/1162074866 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f584815a260 msgr2=0x7f584815a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:09.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f584815a260 0x7f584815a710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f5848061810 tx=0x7f5840017000).stop 2022-07-21T01:54:09.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 -- 172.21.15.140:0/1162074866 shutdown_connections 2022-07-21T01:54:09.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f582c053a10 0x7f582c055ed0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:09.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f584815a260 0x7f584815a710 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:09.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f584809e560 0x7f5848156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:09.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 --2- 172.21.15.140:0/1162074866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58481496f0 0x7f5848149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:09.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 -- 172.21.15.140:0/1162074866 >> 172.21.15.140:0/1162074866 conn(0x7f58480635e0 msgr2=0x7f5848145010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:09.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 -- 172.21.15.140:0/1162074866 shutdown_connections 2022-07-21T01:54:09.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.435+0000 7f584cc38700 1 -- 172.21.15.140:0/1162074866 wait complete. 2022-07-21T01:54:09.435 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:54:09.446 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:54:09.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.559+0000 7fd95cd41700 1 Processor -- start 2022-07-21T01:54:09.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.559+0000 7fd95cd41700 1 -- start start 2022-07-21T01:54:09.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd95cd41700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd9580adfa0 0x7fd95805d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:09.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd95cd41700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd958143ea0 0x7fd958144350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:09.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd95cd41700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9580b4230 con 0x7fd95805bf30 2022-07-21T01:54:09.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd95cd41700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9580677b0 con 0x7fd9580adfa0 2022-07-21T01:54:09.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd95cd41700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9580b2950 con 0x7fd958143ea0 2022-07-21T01:54:09.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9577fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd95805bf30 0x7fd95813a550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50658/0 (socket says 172.21.15.140:50658) 2022-07-21T01:54:09.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9577fe700 1 -- 172.21.15.140:0/364320452 learned_addr learned my addr 172.21.15.140:0/364320452 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:09.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd956ffd700 1 --2- 172.21.15.140:0/364320452 >> v2:172.21.15.140:3300/0 conn(0x7fd9580adfa0 0x7fd95805d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:09.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd957fff700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd958143ea0 0x7fd958144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:09.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073502079 0 0) 0x7fd9580b4230 con 0x7fd95805bf30 2022-07-21T01:54:09.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd944003260 con 0x7fd95805bf30 2022-07-21T01:54:09.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694515189 0 0) 0x7fd944003260 con 0x7fd95805bf30 2022-07-21T01:54:09.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd958143ea0 msgr2=0x7fd958144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:09.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd958143ea0 0x7fd958144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:09.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 >> v2:172.21.15.140:3300/0 conn(0x7fd9580adfa0 msgr2=0x7fd95805d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:09.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 --2- 172.21.15.140:0/364320452 >> v2:172.21.15.140:3300/0 conn(0x7fd9580adfa0 0x7fd95805d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:09.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9580b3850 con 0x7fd95805bf30 2022-07-21T01:54:09.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fd9500019e0 con 0x7fd95805bf30 2022-07-21T01:54:09.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 >> v1:172.21.15.140:6789/0 conn(0x7fd95805bf30 legacy=0x7fd95813a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:09.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd944003750 0x7fd944003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:09.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944004500 0x7fd944004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:09.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd958143ea0 0x7fd944004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:09.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9580677b0 con 0x7fd944003750 2022-07-21T01:54:09.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9580b2950 con 0x7fd944004500 2022-07-21T01:54:09.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94400b750 con 0x7fd958143ea0 2022-07-21T01:54:09.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9577fe700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd944003750 0x7fd944003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:09.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd956ffd700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944004500 0x7fd944004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:09.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd957fff700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd958143ea0 0x7fd944004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:09.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9577fe700 1 -- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd958143ea0 msgr2=0x7fd944004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:09.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9577fe700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd958143ea0 0x7fd944004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:09.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9577fe700 1 -- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944004500 msgr2=0x7fd944004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:09.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9577fe700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944004500 0x7fd944004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:09.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9577fe700 1 -- 172.21.15.140:0/364320452 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd944003260 con 0x7fd944003750 2022-07-21T01:54:09.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd956ffd700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944004500 0x7fd944004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:54:09.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd957fff700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd958143ea0 0x7fd944004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:54:09.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9577fe700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd944003750 0x7fd944003b70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fd9500016f0 tx=0x7fd9500021d0).ready entity=mon.0 client_cookie=8da402175e7cfe20 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:09.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd95000cc80 con 0x7fd944003750 2022-07-21T01:54:09.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95000cde0 con 0x7fd944003750 2022-07-21T01:54:09.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd9567fc700 1 -- 172.21.15.140:0/364320452 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd950005b80 con 0x7fd944003750 2022-07-21T01:54:09.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd95cd41700 1 -- 172.21.15.140:0/364320452 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd944003750 msgr2=0x7fd944003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:09.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd95cd41700 1 --2- 172.21.15.140:0/364320452 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd944003750 0x7fd944003b70 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fd9500016f0 tx=0x7fd9500021d0).stop 2022-07-21T01:54:09.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd957fff700 1 -- 172.21.15.140:0/364320452 reap_dead start 2022-07-21T01:54:09.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd95cd41700 1 -- 172.21.15.140:0/364320452 shutdown_connections 2022-07-21T01:54:09.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd95cd41700 1 -- 172.21.15.140:0/364320452 >> 172.21.15.140:0/364320452 conn(0x7fd958064b90 msgr2=0x7fd958064f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:09.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.563+0000 7fd95cd41700 1 -- 172.21.15.140:0/364320452 shutdown_connections 2022-07-21T01:54:09.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd95cd41700 1 -- 172.21.15.140:0/364320452 wait complete. 2022-07-21T01:54:09.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd95cd41700 1 Processor -- start 2022-07-21T01:54:09.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd95cd41700 1 -- start start 2022-07-21T01:54:09.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd95cd41700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944003750 0x7fd958151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:09.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd95cd41700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd958061520 0x7fd958060700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:09.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd95cd41700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd95813db80 0x7fd95813e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:09.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd95cd41700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd958152d30 con 0x7fd958061520 2022-07-21T01:54:09.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd95cd41700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95813e4f0 con 0x7fd944003750 2022-07-21T01:54:09.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd95cd41700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd958142190 con 0x7fd95813db80 2022-07-21T01:54:09.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd956ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd958061520 0x7fd958060700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:09.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd956ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd958061520 0x7fd958060700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36526/0 (socket says 172.21.15.140:36526) 2022-07-21T01:54:09.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd956ffd700 1 -- 172.21.15.140:0/2681958341 learned_addr learned my addr 172.21.15.140:0/2681958341 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:09.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd9577fe700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944003750 0x7fd958151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:09.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.567+0000 7fd957fff700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd95813db80 0x7fd95813e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:09.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd956ffd700 1 -- 172.21.15.140:0/2681958341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd95813db80 msgr2=0x7fd95813e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:09.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd956ffd700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd95813db80 0x7fd95813e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:09.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd956ffd700 1 -- 172.21.15.140:0/2681958341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944003750 msgr2=0x7fd958151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:09.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd956ffd700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944003750 0x7fd958151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:09.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd956ffd700 1 -- 172.21.15.140:0/2681958341 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd958142430 con 0x7fd958061520 2022-07-21T01:54:09.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd956ffd700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd958061520 0x7fd958060700 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fd958155590 tx=0x7fd94c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:09.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd954ff9700 1 -- 172.21.15.140:0/2681958341 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd94c017070 con 0x7fd958061520 2022-07-21T01:54:09.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd954ff9700 1 -- 172.21.15.140:0/2681958341 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c000e90 con 0x7fd958061520 2022-07-21T01:54:09.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd95cd41700 1 -- 172.21.15.140:0/2681958341 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd958141e10 con 0x7fd958061520 2022-07-21T01:54:09.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd954ff9700 1 -- 172.21.15.140:0/2681958341 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd94c012950 con 0x7fd958061520 2022-07-21T01:54:09.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd95cd41700 1 -- 172.21.15.140:0/2681958341 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9580b1470 con 0x7fd958061520 2022-07-21T01:54:09.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd95cd41700 1 -- 172.21.15.140:0/2681958341 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9580b3af0 con 0x7fd958061520 2022-07-21T01:54:09.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd954ff9700 1 -- 172.21.15.140:0/2681958341 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd94c012ab0 con 0x7fd958061520 2022-07-21T01:54:09.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.571+0000 7fd954ff9700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd938053b00 0x7fd938055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:09.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.575+0000 7fd9577fe700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd938053b00 0x7fd938055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:09.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.575+0000 7fd954ff9700 1 -- 172.21.15.140:0/2681958341 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fd94c06bd80 con 0x7fd958061520 2022-07-21T01:54:09.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.575+0000 7fd9577fe700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd938053b00 0x7fd938055fc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fd95001a000 tx=0x7fd950015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:09.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.579+0000 7fd954ff9700 1 -- 172.21.15.140:0/2681958341 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd94c042d60 con 0x7fd958061520 2022-07-21T01:54:09.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:09.699+0000 7fd95cd41700 1 -- 172.21.15.140:0/2681958341 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fd958141fd0 con 0x7fd958061520 2022-07-21T01:54:10.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.431+0000 7fd954ff9700 1 -- 172.21.15.140:0/2681958341 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v69) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd94c046380 con 0x7fd958061520 2022-07-21T01:54:10.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 -- 172.21.15.140:0/2681958341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd938053b00 msgr2=0x7fd938055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd938053b00 0x7fd938055fc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fd95001a000 tx=0x7fd950015000).stop 2022-07-21T01:54:10.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 -- 172.21.15.140:0/2681958341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd958061520 msgr2=0x7fd958060700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd958061520 0x7fd958060700 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fd958155590 tx=0x7fd94c008040).stop 2022-07-21T01:54:10.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 -- 172.21.15.140:0/2681958341 shutdown_connections 2022-07-21T01:54:10.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd938053b00 0x7fd938055fc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd95813db80 0x7fd95813e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd958061520 0x7fd958060700 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 --2- 172.21.15.140:0/2681958341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd944003750 0x7fd958151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 -- 172.21.15.140:0/2681958341 >> 172.21.15.140:0/2681958341 conn(0x7fd958064b90 msgr2=0x7fd9581409f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:10.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 -- 172.21.15.140:0/2681958341 shutdown_connections 2022-07-21T01:54:10.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.435+0000 7fd95cd41700 1 -- 172.21.15.140:0/2681958341 wait complete. 2022-07-21T01:54:10.435 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:54:10.447 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:54:10.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d7576700 1 Processor -- start 2022-07-21T01:54:10.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d7576700 1 -- start start 2022-07-21T01:54:10.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d7576700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa4d00b3560 0x7fa4d0099370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:10.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d7576700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4d005cd40 0x7fa4d0060670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:10.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d7576700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d00b3940 con 0x7fa4d0060b60 2022-07-21T01:54:10.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d7576700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d009b990 con 0x7fa4d00b3560 2022-07-21T01:54:10.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d7576700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d009bc30 con 0x7fa4d005cd40 2022-07-21T01:54:10.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d6d75700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa4d0060b60 0x7fa4d00b1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50682/0 (socket says 172.21.15.140:50682) 2022-07-21T01:54:10.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d6d75700 1 -- 172.21.15.140:0/139091674 learned_addr learned my addr 172.21.15.140:0/139091674 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:10.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d5d73700 1 --2- 172.21.15.140:0/139091674 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4d005cd40 0x7fa4d0060670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:10.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d6574700 1 --2- 172.21.15.140:0/139091674 >> v2:172.21.15.152:3300/0 conn(0x7fa4d00b3560 0x7fa4d0099370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:10.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d5572700 1 -- 172.21.15.140:0/139091674 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541905253 0 0) 0x7fa4d00b3940 con 0x7fa4d0060b60 2022-07-21T01:54:10.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d5572700 1 -- 172.21.15.140:0/139091674 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b8003260 con 0x7fa4d0060b60 2022-07-21T01:54:10.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.639+0000 7fa4d6574700 1 -- 172.21.15.140:0/139091674 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4d005cd40 msgr2=0x7fa4d0060670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:10.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d6574700 1 --2- 172.21.15.140:0/139091674 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4d005cd40 0x7fa4d0060670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d6574700 1 -- 172.21.15.140:0/139091674 >> v1:172.21.15.140:6789/0 conn(0x7fa4d0060b60 legacy=0x7fa4d00b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d6574700 1 -- 172.21.15.140:0/139091674 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d009bed0 con 0x7fa4d00b3560 2022-07-21T01:54:10.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d6574700 1 --2- 172.21.15.140:0/139091674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4d00b3560 0x7fa4d0099370 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fa4cc0032f0 tx=0x7fa4cc0091d0).ready entity=mon.1 client_cookie=6f0c1b76cb8f8f8d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:10.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d5572700 1 -- 172.21.15.140:0/139091674 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4cc00b600 con 0x7fa4d00b3560 2022-07-21T01:54:10.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d5572700 1 -- 172.21.15.140:0/139091674 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4cc00b760 con 0x7fa4d00b3560 2022-07-21T01:54:10.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d5572700 1 -- 172.21.15.140:0/139091674 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4cc00b8d0 con 0x7fa4d00b3560 2022-07-21T01:54:10.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d7576700 1 -- 172.21.15.140:0/139091674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4d00b3560 msgr2=0x7fa4d0099370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d7576700 1 --2- 172.21.15.140:0/139091674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4d00b3560 0x7fa4d0099370 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fa4cc0032f0 tx=0x7fa4cc0091d0).stop 2022-07-21T01:54:10.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d7576700 1 -- 172.21.15.140:0/139091674 shutdown_connections 2022-07-21T01:54:10.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d7576700 1 --2- 172.21.15.140:0/139091674 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4d005cd40 0x7fa4d0060670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d7576700 1 --2- 172.21.15.140:0/139091674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4d00b3560 0x7fa4d0099370 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d7576700 1 -- 172.21.15.140:0/139091674 >> 172.21.15.140:0/139091674 conn(0x7fa4d00b6b20 msgr2=0x7fa4d00b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:10.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d7576700 1 -- 172.21.15.140:0/139091674 shutdown_connections 2022-07-21T01:54:10.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d7576700 1 -- 172.21.15.140:0/139091674 wait complete. 2022-07-21T01:54:10.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.643+0000 7fa4d7576700 1 Processor -- start 2022-07-21T01:54:10.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d7576700 1 -- start start 2022-07-21T01:54:10.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d7576700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4d005cd40 0x7fa4d009a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:10.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d7576700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4d0060b60 0x7fa4d01444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:10.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d7576700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4d00b3560 0x7fa4d01449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:10.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d7576700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d013e170 con 0x7fa4d0060b60 2022-07-21T01:54:10.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d7576700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d0144260 con 0x7fa4d005cd40 2022-07-21T01:54:10.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d7576700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d0146d30 con 0x7fa4d00b3560 2022-07-21T01:54:10.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d5d73700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4d0060b60 0x7fa4d01444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:10.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d5d73700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4d0060b60 0x7fa4d01444c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36538/0 (socket says 172.21.15.140:36538) 2022-07-21T01:54:10.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d5d73700 1 -- 172.21.15.140:0/2988891802 learned_addr learned my addr 172.21.15.140:0/2988891802 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:10.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d6574700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4d005cd40 0x7fa4d009a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:10.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d6d75700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4d00b3560 0x7fa4d01449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:10.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d5d73700 1 -- 172.21.15.140:0/2988891802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4d00b3560 msgr2=0x7fa4d01449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d5d73700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4d00b3560 0x7fa4d01449b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d5d73700 1 -- 172.21.15.140:0/2988891802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4d005cd40 msgr2=0x7fa4d009a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d5d73700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4d005cd40 0x7fa4d009a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d5d73700 1 -- 172.21.15.140:0/2988891802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc009470 con 0x7fa4d0060b60 2022-07-21T01:54:10.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d5d73700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4d0060b60 0x7fa4d01444c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fa4c400c000 tx=0x7fa4c4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:10.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4c37fe700 1 -- 172.21.15.140:0/2988891802 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c4010590 con 0x7fa4d0060b60 2022-07-21T01:54:10.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4c37fe700 1 -- 172.21.15.140:0/2988891802 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c40106f0 con 0x7fa4d0060b60 2022-07-21T01:54:10.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d7576700 1 -- 172.21.15.140:0/2988891802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d0146f90 con 0x7fa4d0060b60 2022-07-21T01:54:10.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4c37fe700 1 -- 172.21.15.140:0/2988891802 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c4010a50 con 0x7fa4d0060b60 2022-07-21T01:54:10.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.647+0000 7fa4d7576700 1 -- 172.21.15.140:0/2988891802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d00ad000 con 0x7fa4d0060b60 2022-07-21T01:54:10.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.651+0000 7fa4d7576700 1 -- 172.21.15.140:0/2988891802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d0060ff0 con 0x7fa4d0060b60 2022-07-21T01:54:10.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.651+0000 7fa4c37fe700 1 -- 172.21.15.140:0/2988891802 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa4c4010bb0 con 0x7fa4d0060b60 2022-07-21T01:54:10.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.651+0000 7fa4c37fe700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4b00539e0 0x7fa4b0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:10.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.651+0000 7fa4c37fe700 1 -- 172.21.15.140:0/2988891802 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fa4c406c920 con 0x7fa4d0060b60 2022-07-21T01:54:10.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.651+0000 7fa4d6574700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4b00539e0 0x7fa4b0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:10.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.651+0000 7fa4d6574700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4b00539e0 0x7fa4b0055ea0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fa4cc01b000 tx=0x7fa4cc016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:10.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.655+0000 7fa4c37fe700 1 -- 172.21.15.140:0/2988891802 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa4c40438b0 con 0x7fa4d0060b60 2022-07-21T01:54:10.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.819+0000 7fa4d7576700 1 -- 172.21.15.140:0/2988891802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fa4d0098360 con 0x7fa4d0060b60 2022-07-21T01:54:10.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4c37fe700 1 -- 172.21.15.140:0/2988891802 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v29) v1 ==== 189+0+0 (secure 0 0 0) 0x7fa4c4046ed0 con 0x7fa4d0060b60 2022-07-21T01:54:10.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 -- 172.21.15.140:0/2988891802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4b00539e0 msgr2=0x7fa4b0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4b00539e0 0x7fa4b0055ea0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fa4cc01b000 tx=0x7fa4cc016000).stop 2022-07-21T01:54:10.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 -- 172.21.15.140:0/2988891802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4d0060b60 msgr2=0x7fa4d01444c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4d0060b60 0x7fa4d01444c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fa4c400c000 tx=0x7fa4c4006df0).stop 2022-07-21T01:54:10.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 -- 172.21.15.140:0/2988891802 shutdown_connections 2022-07-21T01:54:10.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4b00539e0 0x7fa4b0055ea0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa4d00b3560 0x7fa4d01449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa4d0060b60 0x7fa4d01444c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 --2- 172.21.15.140:0/2988891802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4d005cd40 0x7fa4d009a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 -- 172.21.15.140:0/2988891802 >> 172.21.15.140:0/2988891802 conn(0x7fa4d00b6b20 msgr2=0x7fa4d00b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:10.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 -- 172.21.15.140:0/2988891802 shutdown_connections 2022-07-21T01:54:10.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.843+0000 7fa4d7576700 1 -- 172.21.15.140:0/2988891802 wait complete. 2022-07-21T01:54:10.844 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 13 and data pool 14 2022-07-21T01:54:10.854 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:54:10.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 Processor -- start 2022-07-21T01:54:10.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 -- start start 2022-07-21T01:54:10.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7efc840af8e0 0x7efc840b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:10.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc84147010 con 0x7efc840b5650 2022-07-21T01:54:10.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc841472b0 con 0x7efc840adff0 2022-07-21T01:54:10.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc84146d40 con 0x7efc840af8e0 2022-07-21T01:54:10.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc83fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7efc840b5650 0x7efc84147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50704/0 (socket says 172.21.15.140:50704) 2022-07-21T01:54:10.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc83fff700 1 -- 172.21.15.140:0/1961839059 learned_addr learned my addr 172.21.15.140:0/1961839059 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:10.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc88c74700 1 --2- 172.21.15.140:0/1961839059 >> v2:172.21.15.140:3300/0 conn(0x7efc840af8e0 0x7efc840b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:10.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc82ffd700 1 -- 172.21.15.140:0/1961839059 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986176472 0 0) 0x7efc84147010 con 0x7efc840b5650 2022-07-21T01:54:10.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc82ffd700 1 -- 172.21.15.140:0/1961839059 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc64003260 con 0x7efc840b5650 2022-07-21T01:54:10.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc82ffd700 1 -- 172.21.15.140:0/1961839059 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304068177 0 0) 0x7efc841472b0 con 0x7efc840adff0 2022-07-21T01:54:10.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc82ffd700 1 -- 172.21.15.140:0/1961839059 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc84147010 con 0x7efc840adff0 2022-07-21T01:54:10.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc88c74700 1 -- 172.21.15.140:0/1961839059 >> v1:172.21.15.152:6789/0 conn(0x7efc840adff0 legacy=0x7efc8405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc88c74700 1 -- 172.21.15.140:0/1961839059 >> v1:172.21.15.140:6789/0 conn(0x7efc840b5650 legacy=0x7efc84147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc88c74700 1 -- 172.21.15.140:0/1961839059 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8405c4a0 con 0x7efc840af8e0 2022-07-21T01:54:10.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc88c74700 1 --2- 172.21.15.140:0/1961839059 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efc840af8e0 0x7efc840b5a30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7efc74002f70 tx=0x7efc74006df0).ready entity=mon.0 client_cookie=ac875a0ca2372b46 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:10.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc82ffd700 1 -- 172.21.15.140:0/1961839059 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc7400d540 con 0x7efc840af8e0 2022-07-21T01:54:10.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc82ffd700 1 -- 172.21.15.140:0/1961839059 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc74000e90 con 0x7efc840af8e0 2022-07-21T01:54:10.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc82ffd700 1 -- 172.21.15.140:0/1961839059 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc7400d890 con 0x7efc840af8e0 2022-07-21T01:54:10.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 -- 172.21.15.140:0/1961839059 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efc840af8e0 msgr2=0x7efc840b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 --2- 172.21.15.140:0/1961839059 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efc840af8e0 0x7efc840b5a30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7efc74002f70 tx=0x7efc74006df0).stop 2022-07-21T01:54:10.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 -- 172.21.15.140:0/1961839059 shutdown_connections 2022-07-21T01:54:10.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 --2- 172.21.15.140:0/1961839059 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efc840af8e0 0x7efc840b5a30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 -- 172.21.15.140:0/1961839059 >> 172.21.15.140:0/1961839059 conn(0x7efc840634e0 msgr2=0x7efc840b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:10.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 -- 172.21.15.140:0/1961839059 shutdown_connections 2022-07-21T01:54:10.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.951+0000 7efc89475700 1 -- 172.21.15.140:0/1961839059 wait complete. 2022-07-21T01:54:10.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc89475700 1 Processor -- start 2022-07-21T01:54:10.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc89475700 1 -- start start 2022-07-21T01:54:10.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc89475700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc840adff0 0x7efc84149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:10.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc89475700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efc840af8e0 0x7efc84146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:10.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc89475700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc840b5650 0x7efc84159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:10.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc89475700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc841573e0 con 0x7efc840af8e0 2022-07-21T01:54:10.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc89475700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc84066b00 con 0x7efc840b5650 2022-07-21T01:54:10.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc89475700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc84110f70 con 0x7efc840adff0 2022-07-21T01:54:10.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efc840af8e0 0x7efc84146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:10.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efc840af8e0 0x7efc84146000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36566/0 (socket says 172.21.15.140:36566) 2022-07-21T01:54:10.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc837fe700 1 -- 172.21.15.140:0/2107236690 learned_addr learned my addr 172.21.15.140:0/2107236690 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:10.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc88c74700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc840b5650 0x7efc84159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:10.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc83fff700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc840adff0 0x7efc84149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:10.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc837fe700 1 -- 172.21.15.140:0/2107236690 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc840adff0 msgr2=0x7efc84149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc837fe700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc840adff0 0x7efc84149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc837fe700 1 -- 172.21.15.140:0/2107236690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc840b5650 msgr2=0x7efc84159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:10.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc837fe700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc840b5650 0x7efc84159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:10.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc837fe700 1 -- 172.21.15.140:0/2107236690 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7400b180 con 0x7efc840af8e0 2022-07-21T01:54:10.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc837fe700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efc840af8e0 0x7efc84146000 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7efc7c001160 tx=0x7efc7c007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:10.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc817fa700 1 -- 172.21.15.140:0/2107236690 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc7c012400 con 0x7efc840af8e0 2022-07-21T01:54:10.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc89475700 1 -- 172.21.15.140:0/2107236690 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc84066860 con 0x7efc840af8e0 2022-07-21T01:54:10.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc817fa700 1 -- 172.21.15.140:0/2107236690 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c012560 con 0x7efc840af8e0 2022-07-21T01:54:10.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc817fa700 1 -- 172.21.15.140:0/2107236690 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc7c0128c0 con 0x7efc840af8e0 2022-07-21T01:54:10.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.955+0000 7efc89475700 1 -- 172.21.15.140:0/2107236690 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc84063990 con 0x7efc840af8e0 2022-07-21T01:54:10.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.959+0000 7efc89475700 1 -- 172.21.15.140:0/2107236690 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc840574e0 con 0x7efc840af8e0 2022-07-21T01:54:10.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.959+0000 7efc817fa700 1 -- 172.21.15.140:0/2107236690 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efc7c010070 con 0x7efc840af8e0 2022-07-21T01:54:10.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.959+0000 7efc817fa700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efc68053a10 0x7efc68055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:10.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.959+0000 7efc817fa700 1 -- 172.21.15.140:0/2107236690 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efc7c06afd0 con 0x7efc840af8e0 2022-07-21T01:54:10.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.959+0000 7efc83fff700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efc68053a10 0x7efc68055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:10.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.963+0000 7efc83fff700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efc68053a10 0x7efc68055ed0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7efc6c00d000 tx=0x7efc6c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:10.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:10.967+0000 7efc817fa700 1 -- 172.21.15.140:0/2107236690 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efc7c041e50 con 0x7efc840af8e0 2022-07-21T01:54:11.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.091+0000 7efc89475700 1 -- 172.21.15.140:0/2107236690 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc84067ea0 con 0x7efc840af8e0 2022-07-21T01:54:11.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc817fa700 1 -- 172.21.15.140:0/2107236690 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16150 (secure 0 0 0) 0x7efc7c045470 con 0x7efc840af8e0 2022-07-21T01:54:11.096 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:11.096 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":72,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:54:10.846151+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:54:09.143212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-07-21T01:54:09.749431+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:54:11.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 -- 172.21.15.140:0/2107236690 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efc68053a10 msgr2=0x7efc68055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efc68053a10 0x7efc68055ed0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7efc6c00d000 tx=0x7efc6c009040).stop 2022-07-21T01:54:11.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 -- 172.21.15.140:0/2107236690 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efc840af8e0 msgr2=0x7efc84146000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efc840af8e0 0x7efc84146000 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7efc7c001160 tx=0x7efc7c007a50).stop 2022-07-21T01:54:11.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 -- 172.21.15.140:0/2107236690 shutdown_connections 2022-07-21T01:54:11.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efc68053a10 0x7efc68055ed0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc840b5650 0x7efc84159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efc840af8e0 0x7efc84146000 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 --2- 172.21.15.140:0/2107236690 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc840adff0 0x7efc84149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 -- 172.21.15.140:0/2107236690 >> 172.21.15.140:0/2107236690 conn(0x7efc840634e0 msgr2=0x7efc84069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:11.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 -- 172.21.15.140:0/2107236690 shutdown_connections 2022-07-21T01:54:11.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.095+0000 7efc89475700 1 -- 172.21.15.140:0/2107236690 wait complete. 2022-07-21T01:54:11.104 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:54:11.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375c7e9700 1 Processor -- start 2022-07-21T01:54:11.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375c7e9700 1 -- start start 2022-07-21T01:54:11.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375c7e9700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f37540acef0 0x7f375414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375c7e9700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375405d480 con 0x7f37540b1e00 2022-07-21T01:54:11.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375c7e9700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37540b21e0 con 0x7f375405c3f0 2022-07-21T01:54:11.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375c7e9700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375405cf40 con 0x7f37540acef0 2022-07-21T01:54:11.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375a585700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f37540b1e00 0x7f37540ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50718/0 (socket says 172.21.15.140:50718) 2022-07-21T01:54:11.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375a585700 1 -- 172.21.15.140:0/1480499062 learned_addr learned my addr 172.21.15.140:0/1480499062 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:11.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915824684 0 0) 0x7f375405d480 con 0x7f37540b1e00 2022-07-21T01:54:11.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3740003260 con 0x7f37540b1e00 2022-07-21T01:54:11.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375ad86700 1 --2- 172.21.15.140:0/1480499062 >> v2:172.21.15.152:3300/0 conn(0x7f37540acef0 0x7f375414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1419507868 0 0) 0x7f3740003260 con 0x7f37540b1e00 2022-07-21T01:54:11.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 >> v2:172.21.15.152:3300/0 conn(0x7f37540acef0 msgr2=0x7f375414a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:11.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 --2- 172.21.15.140:0/1480499062 >> v2:172.21.15.152:3300/0 conn(0x7f37540acef0 0x7f375414a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 >> v1:172.21.15.152:6789/0 conn(0x7f375405c3f0 legacy=0x7f3754146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375405d1e0 con 0x7f37540b1e00 2022-07-21T01:54:11.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f37500019e0 con 0x7f37540b1e00 2022-07-21T01:54:11.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 >> v1:172.21.15.140:6789/0 conn(0x7f37540b1e00 legacy=0x7f37540ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3740003750 0x7f3740003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3740004500 0x7f3740004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3740004ec0 0x7f374000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375405cf40 con 0x7f3740004ec0 2022-07-21T01:54:11.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374000be50 con 0x7f3740003750 2022-07-21T01:54:11.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374000c010 con 0x7f3740004500 2022-07-21T01:54:11.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375ad86700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3740004ec0 0x7f374000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375a585700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3740003750 0x7f3740003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759d84700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3740004500 0x7f3740004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375a585700 1 -- 172.21.15.140:0/1480499062 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3740004500 msgr2=0x7f3740004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:11.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375a585700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3740004500 0x7f3740004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375a585700 1 -- 172.21.15.140:0/1480499062 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3740004ec0 msgr2=0x7f374000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:11.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375a585700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3740004ec0 0x7f374000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375a585700 1 -- 172.21.15.140:0/1480499062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3740003260 con 0x7f3740003750 2022-07-21T01:54:11.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375ad86700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3740004ec0 0x7f374000b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:11.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375ad86700 1 -- 172.21.15.140:0/1480499062 reap_dead start 2022-07-21T01:54:11.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f375a585700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3740003750 0x7f3740003b70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f37500016f0 tx=0x7f37540ac4d0).ready entity=mon.1 client_cookie=bc9b00add62a0af3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:11.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.215+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f375000cc40 con 0x7f3740003750 2022-07-21T01:54:11.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.219+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375000cda0 con 0x7f3740003750 2022-07-21T01:54:11.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.219+0000 7f3759583700 1 -- 172.21.15.140:0/1480499062 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3750005b80 con 0x7f3740003750 2022-07-21T01:54:11.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.219+0000 7f375c7e9700 1 -- 172.21.15.140:0/1480499062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3740003750 msgr2=0x7f3740003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.219+0000 7f375c7e9700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3740003750 0x7f3740003b70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f37500016f0 tx=0x7f37540ac4d0).stop 2022-07-21T01:54:11.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.219+0000 7f375c7e9700 1 -- 172.21.15.140:0/1480499062 shutdown_connections 2022-07-21T01:54:11.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.219+0000 7f375c7e9700 1 --2- 172.21.15.140:0/1480499062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3740003750 0x7f3740003b70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.219+0000 7f375c7e9700 1 -- 172.21.15.140:0/1480499062 >> 172.21.15.140:0/1480499062 conn(0x7f37540611c0 msgr2=0x7f3754062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:11.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.219+0000 7f375c7e9700 1 -- 172.21.15.140:0/1480499062 shutdown_connections 2022-07-21T01:54:11.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.219+0000 7f375c7e9700 1 -- 172.21.15.140:0/1480499062 wait complete. 2022-07-21T01:54:11.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.219+0000 7f375c7e9700 1 Processor -- start 2022-07-21T01:54:11.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.219+0000 7f375c7e9700 1 -- start start 2022-07-21T01:54:11.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375c7e9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37540adab0 0x7f37540a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375c7e9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3754143630 0x7f37540a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375c7e9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37540b5b40 0x7f37540a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375c7e9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375409e1d0 con 0x7f37540adab0 2022-07-21T01:54:11.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375c7e9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37540a02c0 con 0x7f3754143630 2022-07-21T01:54:11.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375c7e9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37540ade90 con 0x7f37540b5b40 2022-07-21T01:54:11.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375a585700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37540adab0 0x7f37540a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375a585700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37540adab0 0x7f37540a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36586/0 (socket says 172.21.15.140:36586) 2022-07-21T01:54:11.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375a585700 1 -- 172.21.15.140:0/2520029033 learned_addr learned my addr 172.21.15.140:0/2520029033 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:11.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375ad86700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37540b5b40 0x7f37540a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f3759d84700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3754143630 0x7f37540a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375a585700 1 -- 172.21.15.140:0/2520029033 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37540b5b40 msgr2=0x7f37540a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375a585700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37540b5b40 0x7f37540a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375a585700 1 -- 172.21.15.140:0/2520029033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3754143630 msgr2=0x7f37540a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375a585700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3754143630 0x7f37540a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375a585700 1 -- 172.21.15.140:0/2520029033 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37540b4860 con 0x7f37540adab0 2022-07-21T01:54:11.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f3759d84700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3754143630 0x7f37540a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:11.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375ad86700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37540b5b40 0x7f37540a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:11.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375a585700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37540adab0 0x7f37540a12d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f37540a0160 tx=0x7f3750013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:11.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f374b7fe700 1 -- 172.21.15.140:0/2520029033 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37500048a0 con 0x7f37540adab0 2022-07-21T01:54:11.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f374b7fe700 1 -- 172.21.15.140:0/2520029033 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3750004a00 con 0x7f37540adab0 2022-07-21T01:54:11.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f374b7fe700 1 -- 172.21.15.140:0/2520029033 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3750004d60 con 0x7f37540adab0 2022-07-21T01:54:11.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375c7e9700 1 -- 172.21.15.140:0/2520029033 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37540b4b00 con 0x7f37540adab0 2022-07-21T01:54:11.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.223+0000 7f375c7e9700 1 -- 172.21.15.140:0/2520029033 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37540a1090 con 0x7f37540adab0 2022-07-21T01:54:11.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.227+0000 7f375c7e9700 1 -- 172.21.15.140:0/2520029033 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37540b04b0 con 0x7f37540adab0 2022-07-21T01:54:11.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.227+0000 7f374b7fe700 1 -- 172.21.15.140:0/2520029033 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f375001c070 con 0x7f37540adab0 2022-07-21T01:54:11.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.227+0000 7f374b7fe700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3734053b30 0x7f3734055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.227+0000 7f374b7fe700 1 -- 172.21.15.140:0/2520029033 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f375006c840 con 0x7f37540adab0 2022-07-21T01:54:11.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.227+0000 7f3759d84700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3734053b30 0x7f3734055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.227+0000 7f3759d84700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3734053b30 0x7f3734055ff0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f374400d000 tx=0x7f3744009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:11.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.231+0000 7f374b7fe700 1 -- 172.21.15.140:0/2520029033 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3750043700 con 0x7f37540adab0 2022-07-21T01:54:11.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.355+0000 7f375c7e9700 1 -- 172.21.15.140:0/2520029033 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37540574e0 con 0x7f37540adab0 2022-07-21T01:54:11.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.355+0000 7f374b7fe700 1 -- 172.21.15.140:0/2520029033 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16150 (secure 0 0 0) 0x7f3750046d20 con 0x7f37540adab0 2022-07-21T01:54:11.353 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:11.353 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":72,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:54:10.846151+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:54:09.143212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-07-21T01:54:09.749431+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:54:11.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 -- 172.21.15.140:0/2520029033 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3734053b30 msgr2=0x7f3734055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3734053b30 0x7f3734055ff0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f374400d000 tx=0x7f3744009040).stop 2022-07-21T01:54:11.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 -- 172.21.15.140:0/2520029033 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37540adab0 msgr2=0x7f37540a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37540adab0 0x7f37540a12d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f37540a0160 tx=0x7f3750013000).stop 2022-07-21T01:54:11.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 -- 172.21.15.140:0/2520029033 shutdown_connections 2022-07-21T01:54:11.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3734053b30 0x7f3734055ff0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37540b5b40 0x7f37540a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3754143630 0x7f37540a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 --2- 172.21.15.140:0/2520029033 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37540adab0 0x7f37540a12d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 -- 172.21.15.140:0/2520029033 >> 172.21.15.140:0/2520029033 conn(0x7f3740003750 msgr2=0x7f37540afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:11.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 -- 172.21.15.140:0/2520029033 shutdown_connections 2022-07-21T01:54:11.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.359+0000 7f375c7e9700 1 -- 172.21.15.140:0/2520029033 wait complete. 2022-07-21T01:54:11.366 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:54:11.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c9aca0700 1 Processor -- start 2022-07-21T01:54:11.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c9aca0700 1 -- start start 2022-07-21T01:54:11.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c9aca0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c940b5b50 0x7f8c94144060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c9aca0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8c9405c450 0x7f8c94144550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c9aca0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c94144a40 0x7f8c940ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c9aca0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c94067c30 con 0x7f8c94144a40 2022-07-21T01:54:11.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c9aca0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c940b5f30 con 0x7f8c9405c450 2022-07-21T01:54:11.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c9aca0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c94067990 con 0x7f8c940b5b50 2022-07-21T01:54:11.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c9923d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c94144a40 0x7f8c940ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c98a3c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c940b5b50 0x7f8c94144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c98a3c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c940b5b50 0x7f8c94144060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:33322/0 (socket says 172.21.15.140:33322) 2022-07-21T01:54:11.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c98a3c700 1 -- 172.21.15.140:0/4273481350 learned_addr learned my addr 172.21.15.140:0/4273481350 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:11.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c93fff700 1 --2- 172.21.15.140:0/4273481350 >> v2:172.21.15.152:3300/0 conn(0x7f8c9405c450 0x7f8c94144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c98a3c700 1 -- 172.21.15.140:0/4273481350 >> v2:172.21.15.152:3300/0 conn(0x7f8c9405c450 msgr2=0x7f8c94144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:11.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c98a3c700 1 --2- 172.21.15.140:0/4273481350 >> v2:172.21.15.152:3300/0 conn(0x7f8c9405c450 0x7f8c94144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c98a3c700 1 -- 172.21.15.140:0/4273481350 >> v2:172.21.15.140:3300/0 conn(0x7f8c94144a40 msgr2=0x7f8c940ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:11.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c98a3c700 1 --2- 172.21.15.140:0/4273481350 >> v2:172.21.15.140:3300/0 conn(0x7f8c94144a40 0x7f8c940ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c98a3c700 1 -- 172.21.15.140:0/4273481350 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9405cc60 con 0x7f8c940b5b50 2022-07-21T01:54:11.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c98a3c700 1 --2- 172.21.15.140:0/4273481350 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c940b5b50 0x7f8c94144060 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f8c8c0032f0 tx=0x7f8c8c009290).ready entity=mon.2 client_cookie=7bbfe30976a7446d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:11.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.463+0000 7f8c937fe700 1 -- 172.21.15.140:0/4273481350 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c8c00b680 con 0x7f8c940b5b50 2022-07-21T01:54:11.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c937fe700 1 -- 172.21.15.140:0/4273481350 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c00b7e0 con 0x7f8c940b5b50 2022-07-21T01:54:11.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c937fe700 1 -- 172.21.15.140:0/4273481350 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c8c00b950 con 0x7f8c940b5b50 2022-07-21T01:54:11.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c9aca0700 1 -- 172.21.15.140:0/4273481350 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c940b5b50 msgr2=0x7f8c94144060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c9aca0700 1 --2- 172.21.15.140:0/4273481350 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c940b5b50 0x7f8c94144060 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f8c8c0032f0 tx=0x7f8c8c009290).stop 2022-07-21T01:54:11.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c9aca0700 1 -- 172.21.15.140:0/4273481350 shutdown_connections 2022-07-21T01:54:11.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c9aca0700 1 --2- 172.21.15.140:0/4273481350 >> v2:172.21.15.140:3300/0 conn(0x7f8c94144a40 0x7f8c940ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c9aca0700 1 --2- 172.21.15.140:0/4273481350 >> v2:172.21.15.152:3300/0 conn(0x7f8c9405c450 0x7f8c94144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c9aca0700 1 --2- 172.21.15.140:0/4273481350 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c940b5b50 0x7f8c94144060 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c9aca0700 1 -- 172.21.15.140:0/4273481350 >> 172.21.15.140:0/4273481350 conn(0x7f8c940635e0 msgr2=0x7f8c940b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:11.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c9aca0700 1 -- 172.21.15.140:0/4273481350 shutdown_connections 2022-07-21T01:54:11.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c9aca0700 1 -- 172.21.15.140:0/4273481350 wait complete. 2022-07-21T01:54:11.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c9aca0700 1 Processor -- start 2022-07-21T01:54:11.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.467+0000 7f8c9aca0700 1 -- start start 2022-07-21T01:54:11.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c9aca0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c9405c450 0x7f8c940b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c9aca0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c940b5b50 0x7f8c94136ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c9aca0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c94144a40 0x7f8c941373e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c9aca0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9413b3a0 con 0x7f8c940b5b50 2022-07-21T01:54:11.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c9aca0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c94153fc0 con 0x7f8c94144a40 2022-07-21T01:54:11.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c9aca0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9413d4f0 con 0x7f8c9405c450 2022-07-21T01:54:11.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c93fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c940b5b50 0x7f8c94136ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c93fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c940b5b50 0x7f8c94136ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36600/0 (socket says 172.21.15.140:36600) 2022-07-21T01:54:11.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c93fff700 1 -- 172.21.15.140:0/520886697 learned_addr learned my addr 172.21.15.140:0/520886697 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:11.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c98a3c700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c9405c450 0x7f8c940b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c9923d700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c94144a40 0x7f8c941373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c93fff700 1 -- 172.21.15.140:0/520886697 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c9405c450 msgr2=0x7f8c940b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c93fff700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c9405c450 0x7f8c940b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c93fff700 1 -- 172.21.15.140:0/520886697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c94144a40 msgr2=0x7f8c941373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c93fff700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c94144a40 0x7f8c941373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c93fff700 1 -- 172.21.15.140:0/520886697 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c009530 con 0x7f8c940b5b50 2022-07-21T01:54:11.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c9923d700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c94144a40 0x7f8c941373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:11.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c93fff700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c940b5b50 0x7f8c94136ef0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f8c9413b660 tx=0x7f8c88006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:11.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c91ffb700 1 -- 172.21.15.140:0/520886697 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c88010780 con 0x7f8c940b5b50 2022-07-21T01:54:11.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c91ffb700 1 -- 172.21.15.140:0/520886697 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c880108e0 con 0x7f8c940b5b50 2022-07-21T01:54:11.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c9aca0700 1 -- 172.21.15.140:0/520886697 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9413ca20 con 0x7f8c940b5b50 2022-07-21T01:54:11.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c91ffb700 1 -- 172.21.15.140:0/520886697 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c88010c40 con 0x7f8c940b5b50 2022-07-21T01:54:11.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.471+0000 7f8c9aca0700 1 -- 172.21.15.140:0/520886697 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c940b4d30 con 0x7f8c940b5b50 2022-07-21T01:54:11.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.475+0000 7f8c9aca0700 1 -- 172.21.15.140:0/520886697 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9405d100 con 0x7f8c940b5b50 2022-07-21T01:54:11.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.475+0000 7f8c91ffb700 1 -- 172.21.15.140:0/520886697 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8c8801c460 con 0x7f8c940b5b50 2022-07-21T01:54:11.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.475+0000 7f8c91ffb700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c74053950 0x7f8c74055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.475+0000 7f8c98a3c700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c74053950 0x7f8c74055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.475+0000 7f8c91ffb700 1 -- 172.21.15.140:0/520886697 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8c8806c490 con 0x7f8c940b5b50 2022-07-21T01:54:11.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.475+0000 7f8c98a3c700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c74053950 0x7f8c74055e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f8c8c01a000 tx=0x7f8c8c017000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:11.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.483+0000 7f8c91ffb700 1 -- 172.21.15.140:0/520886697 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c88043240 con 0x7f8c940b5b50 2022-07-21T01:54:11.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.643+0000 7f8c9aca0700 1 -- 172.21.15.140:0/520886697 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8c9413d630 con 0x7f8c940b5b50 2022-07-21T01:54:11.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.847+0000 7f8c91ffb700 1 -- 172.21.15.140:0/520886697 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8c88046860 con 0x7f8c940b5b50 2022-07-21T01:54:11.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.847+0000 7f8c9aca0700 1 -- 172.21.15.140:0/520886697 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c74053950 msgr2=0x7f8c74055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.851+0000 7f8c9aca0700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c74053950 0x7f8c74055e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f8c8c01a000 tx=0x7f8c8c017000).stop 2022-07-21T01:54:11.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.851+0000 7f8c9aca0700 1 -- 172.21.15.140:0/520886697 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c940b5b50 msgr2=0x7f8c94136ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.851+0000 7f8c9aca0700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c940b5b50 0x7f8c94136ef0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f8c9413b660 tx=0x7f8c88006df0).stop 2022-07-21T01:54:11.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.851+0000 7f8c9aca0700 1 -- 172.21.15.140:0/520886697 shutdown_connections 2022-07-21T01:54:11.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.851+0000 7f8c9aca0700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c74053950 0x7f8c74055e10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.851+0000 7f8c9aca0700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c94144a40 0x7f8c941373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.851+0000 7f8c9aca0700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c940b5b50 0x7f8c94136ef0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.851+0000 7f8c9aca0700 1 --2- 172.21.15.140:0/520886697 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c9405c450 0x7f8c940b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.851+0000 7f8c9aca0700 1 -- 172.21.15.140:0/520886697 >> 172.21.15.140:0/520886697 conn(0x7f8c940635e0 msgr2=0x7f8c940aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:11.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.851+0000 7f8c9aca0700 1 -- 172.21.15.140:0/520886697 shutdown_connections 2022-07-21T01:54:11.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.851+0000 7f8c9aca0700 1 -- 172.21.15.140:0/520886697 wait complete. 2022-07-21T01:54:11.860 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:54:11.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.979+0000 7f07c51b2700 1 Processor -- start 2022-07-21T01:54:11.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.979+0000 7f07c51b2700 1 -- start start 2022-07-21T01:54:11.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.979+0000 7f07c51b2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f07c00b1620 0x7f07c0140b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.979+0000 7f07c51b2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07c00ac390 0x7f07c00aef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:11.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.979+0000 7f07c51b2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c00b0040 con 0x7f07c0141010 2022-07-21T01:54:11.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.979+0000 7f07c51b2700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c00b1a00 con 0x7f07c00b1620 2022-07-21T01:54:11.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.979+0000 7f07c51b2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c00ae830 con 0x7f07c00ac390 2022-07-21T01:54:11.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.979+0000 7f07b7fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f07c0141010 0x7f07c00abc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50754/0 (socket says 172.21.15.140:50754) 2022-07-21T01:54:11.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.979+0000 7f07b7fff700 1 -- 172.21.15.140:0/4102306319 learned_addr learned my addr 172.21.15.140:0/4102306319 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:11.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.979+0000 7f07bed9d700 1 --2- 172.21.15.140:0/4102306319 >> v2:172.21.15.152:3300/0 conn(0x7f07c00b1620 0x7f07c0140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07bf59e700 1 --2- 172.21.15.140:0/4102306319 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07c00ac390 0x7f07c00aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:11.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07be59c700 1 -- 172.21.15.140:0/4102306319 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635850331 0 0) 0x7f07c00b0040 con 0x7f07c0141010 2022-07-21T01:54:11.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07be59c700 1 -- 172.21.15.140:0/4102306319 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07a0003260 con 0x7f07c0141010 2022-07-21T01:54:11.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07bed9d700 1 -- 172.21.15.140:0/4102306319 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07c00ac390 msgr2=0x7f07c00aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:11.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07bed9d700 1 --2- 172.21.15.140:0/4102306319 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07c00ac390 0x7f07c00aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07bed9d700 1 -- 172.21.15.140:0/4102306319 >> v1:172.21.15.140:6789/0 conn(0x7f07c0141010 legacy=0x7f07c00abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07bed9d700 1 -- 172.21.15.140:0/4102306319 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c00aead0 con 0x7f07c00b1620 2022-07-21T01:54:11.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07bed9d700 1 --2- 172.21.15.140:0/4102306319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07c00b1620 0x7f07c0140b20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f07b0002f70 tx=0x7f07b0006df0).ready entity=mon.1 client_cookie=b9da2526da147e1a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:11.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07be59c700 1 -- 172.21.15.140:0/4102306319 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07b000d4a0 con 0x7f07c00b1620 2022-07-21T01:54:11.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07be59c700 1 -- 172.21.15.140:0/4102306319 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b0000e90 con 0x7f07c00b1620 2022-07-21T01:54:11.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07be59c700 1 -- 172.21.15.140:0/4102306319 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07b000d7f0 con 0x7f07c00b1620 2022-07-21T01:54:11.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07c51b2700 1 -- 172.21.15.140:0/4102306319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07c00b1620 msgr2=0x7f07c0140b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:11.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07c51b2700 1 --2- 172.21.15.140:0/4102306319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07c00b1620 0x7f07c0140b20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f07b0002f70 tx=0x7f07b0006df0).stop 2022-07-21T01:54:11.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07c51b2700 1 -- 172.21.15.140:0/4102306319 shutdown_connections 2022-07-21T01:54:11.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07c51b2700 1 --2- 172.21.15.140:0/4102306319 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07c00ac390 0x7f07c00aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:11.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07c51b2700 1 --2- 172.21.15.140:0/4102306319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07c00b1620 0x7f07c0140b20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07c51b2700 1 -- 172.21.15.140:0/4102306319 >> 172.21.15.140:0/4102306319 conn(0x7f07c00b23d0 msgr2=0x7f07c00b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:12.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07c51b2700 1 -- 172.21.15.140:0/4102306319 shutdown_connections 2022-07-21T01:54:12.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.983+0000 7f07c51b2700 1 -- 172.21.15.140:0/4102306319 wait complete. 2022-07-21T01:54:12.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07c51b2700 1 Processor -- start 2022-07-21T01:54:12.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07c51b2700 1 -- start start 2022-07-21T01:54:12.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07c51b2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07c00ac390 0x7f07c0090130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07c51b2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07c00b1620 0x7f07c01442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07c51b2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07c0141010 0x7f07c0065730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07c51b2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c0065300 con 0x7f07c0141010 2022-07-21T01:54:12.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07c51b2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c0150b40 con 0x7f07c00b1620 2022-07-21T01:54:12.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07c51b2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c0150de0 con 0x7f07c00ac390 2022-07-21T01:54:12.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bf59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07c0141010 0x7f07c0065730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bed9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07c00ac390 0x7f07c0090130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bf59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07c0141010 0x7f07c0065730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36616/0 (socket says 172.21.15.140:36616) 2022-07-21T01:54:12.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bf59e700 1 -- 172.21.15.140:0/1382764284 learned_addr learned my addr 172.21.15.140:0/1382764284 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:12.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bed9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07c00ac390 0x7f07c0090130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:33344/0 (socket says 172.21.15.140:33344) 2022-07-21T01:54:12.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07b7fff700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07c00b1620 0x7f07c01442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bf59e700 1 -- 172.21.15.140:0/1382764284 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07c00ac390 msgr2=0x7f07c0090130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bf59e700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07c00ac390 0x7f07c0090130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bf59e700 1 -- 172.21.15.140:0/1382764284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07c00b1620 msgr2=0x7f07c01442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bf59e700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07c00b1620 0x7f07c01442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bf59e700 1 -- 172.21.15.140:0/1382764284 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b000b180 con 0x7f07c0141010 2022-07-21T01:54:12.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bf59e700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07c0141010 0x7f07c0065730 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f07b8006e00 tx=0x7f07b80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:12.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bcd99700 1 -- 172.21.15.140:0/1382764284 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07b800c640 con 0x7f07c0141010 2022-07-21T01:54:12.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bcd99700 1 -- 172.21.15.140:0/1382764284 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b800c7a0 con 0x7f07c0141010 2022-07-21T01:54:12.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07c51b2700 1 -- 172.21.15.140:0/1382764284 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c008f610 con 0x7f07c0141010 2022-07-21T01:54:12.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07bcd99700 1 -- 172.21.15.140:0/1382764284 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07b800cb00 con 0x7f07c0141010 2022-07-21T01:54:12.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.987+0000 7f07c51b2700 1 -- 172.21.15.140:0/1382764284 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c008f8f0 con 0x7f07c0141010 2022-07-21T01:54:12.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.991+0000 7f07bcd99700 1 -- 172.21.15.140:0/1382764284 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f07b800cc60 con 0x7f07c0141010 2022-07-21T01:54:12.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.991+0000 7f07c51b2700 1 -- 172.21.15.140:0/1382764284 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c0065470 con 0x7f07c0141010 2022-07-21T01:54:12.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.991+0000 7f07bcd99700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07a4053a70 0x7f07a4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.991+0000 7f07bcd99700 1 -- 172.21.15.140:0/1382764284 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f07b806c220 con 0x7f07c0141010 2022-07-21T01:54:12.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.991+0000 7f07bed9d700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07a4053a70 0x7f07a4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.991+0000 7f07bed9d700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07a4053a70 0x7f07a4055f30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f07b001a000 tx=0x7f07b0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:12.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:11.999+0000 7f07bcd99700 1 -- 172.21.15.140:0/1382764284 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f07c0065470 con 0x7f07c0141010 2022-07-21T01:54:12.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.159+0000 7f07c51b2700 1 -- 172.21.15.140:0/1382764284 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07c0150980 con 0x7f07c0141010 2022-07-21T01:54:12.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.159+0000 7f07bcd99700 1 -- 172.21.15.140:0/1382764284 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f07b8043030 con 0x7f07c0141010 2022-07-21T01:54:12.159 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:12.159 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":30,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/4173658089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4173658089},{"type":"v1","addr":"172.21.15.152:6833","nonce":4173658089}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:54:10.830421+0000","modified":"2022-07-21T01:54:11.841578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6208":{"gid":6208,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/167323020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":167323020},{"type":"v1","addr":"172.21.15.140:6835","nonce":167323020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-07-21T01:54:12.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 -- 172.21.15.140:0/1382764284 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07a4053a70 msgr2=0x7f07a4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07a4053a70 0x7f07a4055f30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f07b001a000 tx=0x7f07b0014000).stop 2022-07-21T01:54:12.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 -- 172.21.15.140:0/1382764284 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07c0141010 msgr2=0x7f07c0065730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07c0141010 0x7f07c0065730 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f07b8006e00 tx=0x7f07b80091d0).stop 2022-07-21T01:54:12.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 -- 172.21.15.140:0/1382764284 shutdown_connections 2022-07-21T01:54:12.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f07a4053a70 0x7f07a4055f30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07c0141010 0x7f07c0065730 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07c00b1620 0x7f07c01442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 --2- 172.21.15.140:0/1382764284 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f07c00ac390 0x7f07c0090130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 -- 172.21.15.140:0/1382764284 >> 172.21.15.140:0/1382764284 conn(0x7f07c00b23d0 msgr2=0x7f07c013f310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:12.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 -- 172.21.15.140:0/1382764284 shutdown_connections 2022-07-21T01:54:12.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.163+0000 7f07c51b2700 1 -- 172.21.15.140:0/1382764284 wait complete. 2022-07-21T01:54:12.166 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 30 2022-07-21T01:54:12.170 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:54:12.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.275+0000 7fc38f59e700 1 Processor -- start 2022-07-21T01:54:12.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.275+0000 7fc38f59e700 1 -- start start 2022-07-21T01:54:12.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.275+0000 7fc38f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3900adff0 0x7fc3900b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.275+0000 7fc38f59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc390147010 con 0x7fc3900af8e0 2022-07-21T01:54:12.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.275+0000 7fc38f59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3901472b0 con 0x7fc3900b5650 2022-07-21T01:54:12.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.275+0000 7fc38f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc390146d40 con 0x7fc3900adff0 2022-07-21T01:54:12.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.275+0000 7fc38ed9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc3900af8e0 0x7fc39005bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50770/0 (socket says 172.21.15.140:50770) 2022-07-21T01:54:12.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.275+0000 7fc38ed9d700 1 -- 172.21.15.140:0/3451556763 learned_addr learned my addr 172.21.15.140:0/3451556763 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:12.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38dd9b700 1 --2- 172.21.15.140:0/3451556763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3900adff0 0x7fc3900b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38d59a700 1 -- 172.21.15.140:0/3451556763 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305671683 0 0) 0x7fc390147010 con 0x7fc3900af8e0 2022-07-21T01:54:12.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38d59a700 1 -- 172.21.15.140:0/3451556763 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc374003260 con 0x7fc3900af8e0 2022-07-21T01:54:12.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38dd9b700 1 -- 172.21.15.140:0/3451556763 >> v1:172.21.15.152:6789/0 conn(0x7fc3900b5650 legacy=0x7fc390147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38dd9b700 1 -- 172.21.15.140:0/3451556763 >> v1:172.21.15.140:6789/0 conn(0x7fc3900af8e0 legacy=0x7fc39005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38dd9b700 1 -- 172.21.15.140:0/3451556763 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39005c4a0 con 0x7fc3900adff0 2022-07-21T01:54:12.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38dd9b700 1 --2- 172.21.15.140:0/3451556763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3900adff0 0x7fc3900b5a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fc378002f70 tx=0x7fc378006d90).ready entity=mon.2 client_cookie=61d2a93c13804bee server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:12.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38d59a700 1 -- 172.21.15.140:0/3451556763 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc37800d4a0 con 0x7fc3900adff0 2022-07-21T01:54:12.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38d59a700 1 -- 172.21.15.140:0/3451556763 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378000e90 con 0x7fc3900adff0 2022-07-21T01:54:12.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38d59a700 1 -- 172.21.15.140:0/3451556763 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc37800d7f0 con 0x7fc3900adff0 2022-07-21T01:54:12.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38f59e700 1 -- 172.21.15.140:0/3451556763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3900adff0 msgr2=0x7fc3900b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38f59e700 1 --2- 172.21.15.140:0/3451556763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3900adff0 0x7fc3900b5a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fc378002f70 tx=0x7fc378006d90).stop 2022-07-21T01:54:12.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38f59e700 1 -- 172.21.15.140:0/3451556763 shutdown_connections 2022-07-21T01:54:12.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38f59e700 1 --2- 172.21.15.140:0/3451556763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3900adff0 0x7fc3900b5a30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38f59e700 1 -- 172.21.15.140:0/3451556763 >> 172.21.15.140:0/3451556763 conn(0x7fc3900634e0 msgr2=0x7fc3900b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:12.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38f59e700 1 -- 172.21.15.140:0/3451556763 shutdown_connections 2022-07-21T01:54:12.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.279+0000 7fc38f59e700 1 -- 172.21.15.140:0/3451556763 wait complete. 2022-07-21T01:54:12.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38f59e700 1 Processor -- start 2022-07-21T01:54:12.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38f59e700 1 -- start start 2022-07-21T01:54:12.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38f59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3900adff0 0x7fc390110c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38f59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3900af8e0 0x7fc39014a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3900b5650 0x7fc390159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38f59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900649c0 con 0x7fc3900af8e0 2022-07-21T01:54:12.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38f59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39010f820 con 0x7fc3900adff0 2022-07-21T01:54:12.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39010fac0 con 0x7fc3900b5650 2022-07-21T01:54:12.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38dd9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3900af8e0 0x7fc39014a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38dd9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3900af8e0 0x7fc39014a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36628/0 (socket says 172.21.15.140:36628) 2022-07-21T01:54:12.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38dd9b700 1 -- 172.21.15.140:0/864547784 learned_addr learned my addr 172.21.15.140:0/864547784 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:12.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38ed9d700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3900b5650 0x7fc390159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38e59c700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3900adff0 0x7fc390110c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.283+0000 7fc38dd9b700 1 -- 172.21.15.140:0/864547784 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3900b5650 msgr2=0x7fc390159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc38dd9b700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3900b5650 0x7fc390159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc38dd9b700 1 -- 172.21.15.140:0/864547784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3900adff0 msgr2=0x7fc390110c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc38dd9b700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3900adff0 0x7fc390110c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc38dd9b700 1 -- 172.21.15.140:0/864547784 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37800b120 con 0x7fc3900af8e0 2022-07-21T01:54:12.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc38dd9b700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3900af8e0 0x7fc39014a2f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fc378002f70 tx=0x7fc378014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:12.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc3877fe700 1 -- 172.21.15.140:0/864547784 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc378011b60 con 0x7fc3900af8e0 2022-07-21T01:54:12.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc3877fe700 1 -- 172.21.15.140:0/864547784 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378011cc0 con 0x7fc3900af8e0 2022-07-21T01:54:12.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc38f59e700 1 -- 172.21.15.140:0/864547784 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc390111c30 con 0x7fc3900af8e0 2022-07-21T01:54:12.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc3877fe700 1 -- 172.21.15.140:0/864547784 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc378006220 con 0x7fc3900af8e0 2022-07-21T01:54:12.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc38f59e700 1 -- 172.21.15.140:0/864547784 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc390064e20 con 0x7fc3900af8e0 2022-07-21T01:54:12.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc38f59e700 1 -- 172.21.15.140:0/864547784 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3900574e0 con 0x7fc3900af8e0 2022-07-21T01:54:12.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc3877fe700 1 -- 172.21.15.140:0/864547784 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc378023030 con 0x7fc3900af8e0 2022-07-21T01:54:12.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc3877fe700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc368053a70 0x7fc368055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc3877fe700 1 -- 172.21.15.140:0/864547784 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc37806c6f0 con 0x7fc3900af8e0 2022-07-21T01:54:12.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc38e59c700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc368053a70 0x7fc368055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.287+0000 7fc38e59c700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc368053a70 0x7fc368055f30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fc38000d000 tx=0x7fc380007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:12.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.291+0000 7fc3877fe700 1 -- 172.21.15.140:0/864547784 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc378043610 con 0x7fc3900af8e0 2022-07-21T01:54:12.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.419+0000 7fc38f59e700 1 -- 172.21.15.140:0/864547784 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc390067ea0 con 0x7fc3900af8e0 2022-07-21T01:54:12.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.419+0000 7fc3877fe700 1 -- 172.21.15.140:0/864547784 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16137 (secure 0 0 0) 0x7fc378021090 con 0x7fc3900af8e0 2022-07-21T01:54:12.417 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:12.418 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":73,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:54:11.862650+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:54:09.143212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-07-21T01:54:09.749431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:54:12.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 -- 172.21.15.140:0/864547784 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc368053a70 msgr2=0x7fc368055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc368053a70 0x7fc368055f30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fc38000d000 tx=0x7fc380007b70).stop 2022-07-21T01:54:12.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 -- 172.21.15.140:0/864547784 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3900af8e0 msgr2=0x7fc39014a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3900af8e0 0x7fc39014a2f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7fc378002f70 tx=0x7fc378014000).stop 2022-07-21T01:54:12.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 -- 172.21.15.140:0/864547784 shutdown_connections 2022-07-21T01:54:12.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc368053a70 0x7fc368055f30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3900b5650 0x7fc390159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc3900af8e0 0x7fc39014a2f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 --2- 172.21.15.140:0/864547784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3900adff0 0x7fc390110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 -- 172.21.15.140:0/864547784 >> 172.21.15.140:0/864547784 conn(0x7fc3900634e0 msgr2=0x7fc3901460c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:12.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 -- 172.21.15.140:0/864547784 shutdown_connections 2022-07-21T01:54:12.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.423+0000 7fc38f59e700 1 -- 172.21.15.140:0/864547784 wait complete. 2022-07-21T01:54:12.430 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:54:12.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.543+0000 7f6a697d1700 1 Processor -- start 2022-07-21T01:54:12.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.543+0000 7f6a697d1700 1 -- start start 2022-07-21T01:54:12.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.543+0000 7f6a697d1700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6a64146290 0x7f6a640ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.543+0000 7f6a697d1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a640948d0 con 0x7f6a640ab250 2022-07-21T01:54:12.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.543+0000 7f6a697d1700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a64094b70 con 0x7f6a6405c7c0 2022-07-21T01:54:12.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.543+0000 7f6a697d1700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a64093db0 con 0x7f6a64146290 2022-07-21T01:54:12.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.543+0000 7f6a68fd0700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f6a640ab250 0x7f6a641420a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50784/0 (socket says 172.21.15.140:50784) 2022-07-21T01:54:12.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.543+0000 7f6a68fd0700 1 -- 172.21.15.140:0/2510374071 learned_addr learned my addr 172.21.15.140:0/2510374071 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:12.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.543+0000 7f6a5bfff700 1 --2- 172.21.15.140:0/2510374071 >> v2:172.21.15.152:3300/0 conn(0x7f6a64146290 0x7f6a640ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a637fe700 1 -- 172.21.15.140:0/2510374071 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062845091 0 0) 0x7f6a640948d0 con 0x7f6a640ab250 2022-07-21T01:54:12.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a637fe700 1 -- 172.21.15.140:0/2510374071 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a44003260 con 0x7f6a640ab250 2022-07-21T01:54:12.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a637fe700 1 -- 172.21.15.140:0/2510374071 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886346420 0 0) 0x7f6a64094b70 con 0x7f6a6405c7c0 2022-07-21T01:54:12.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a637fe700 1 -- 172.21.15.140:0/2510374071 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a640948d0 con 0x7f6a6405c7c0 2022-07-21T01:54:12.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a5bfff700 1 -- 172.21.15.140:0/2510374071 >> v1:172.21.15.152:6789/0 conn(0x7f6a6405c7c0 legacy=0x7f6a640952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a5bfff700 1 -- 172.21.15.140:0/2510374071 >> v1:172.21.15.140:6789/0 conn(0x7f6a640ab250 legacy=0x7f6a641420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a637fe700 1 -- 172.21.15.140:0/2510374071 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1833729503 0 0) 0x7f6a44003260 con 0x7f6a640ab250 2022-07-21T01:54:12.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a5bfff700 1 -- 172.21.15.140:0/2510374071 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64094e10 con 0x7f6a64146290 2022-07-21T01:54:12.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a5bfff700 1 --2- 172.21.15.140:0/2510374071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a64146290 0x7f6a640ab760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f6a5c001490 tx=0x7f6a4c006df0).ready entity=mon.1 client_cookie=712b6ee7f0508364 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:12.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a637fe700 1 -- 172.21.15.140:0/2510374071 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a5c0019e0 con 0x7f6a64146290 2022-07-21T01:54:12.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a637fe700 1 -- 172.21.15.140:0/2510374071 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c000e90 con 0x7f6a64146290 2022-07-21T01:54:12.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a637fe700 1 -- 172.21.15.140:0/2510374071 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c012460 con 0x7f6a64146290 2022-07-21T01:54:12.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a697d1700 1 -- 172.21.15.140:0/2510374071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a64146290 msgr2=0x7f6a640ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a697d1700 1 --2- 172.21.15.140:0/2510374071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a64146290 0x7f6a640ab760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f6a5c001490 tx=0x7f6a4c006df0).stop 2022-07-21T01:54:12.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a697d1700 1 -- 172.21.15.140:0/2510374071 shutdown_connections 2022-07-21T01:54:12.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a697d1700 1 --2- 172.21.15.140:0/2510374071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a64146290 0x7f6a640ab760 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a697d1700 1 -- 172.21.15.140:0/2510374071 >> 172.21.15.140:0/2510374071 conn(0x7f6a640b6210 msgr2=0x7f6a640b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:12.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a697d1700 1 -- 172.21.15.140:0/2510374071 shutdown_connections 2022-07-21T01:54:12.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a697d1700 1 -- 172.21.15.140:0/2510374071 wait complete. 2022-07-21T01:54:12.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a697d1700 1 Processor -- start 2022-07-21T01:54:12.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.547+0000 7f6a697d1700 1 -- start start 2022-07-21T01:54:12.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a697d1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6a6405c7c0 0x7f6a64097340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a697d1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a640ab250 0x7f6a641422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a697d1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a64146290 0x7f6a64155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a697d1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a64062150 con 0x7f6a640ab250 2022-07-21T01:54:12.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a697d1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a64155740 con 0x7f6a64146290 2022-07-21T01:54:12.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a697d1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a640b10a0 con 0x7f6a6405c7c0 2022-07-21T01:54:12.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a5bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a640ab250 0x7f6a641422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a5bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a640ab250 0x7f6a641422f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36642/0 (socket says 172.21.15.140:36642) 2022-07-21T01:54:12.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a5bfff700 1 -- 172.21.15.140:0/460378168 learned_addr learned my addr 172.21.15.140:0/460378168 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:12.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a68fd0700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a64146290 0x7f6a64155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a63fff700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6a6405c7c0 0x7f6a64097340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a5bfff700 1 -- 172.21.15.140:0/460378168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6a6405c7c0 msgr2=0x7f6a64097340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a5bfff700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6a6405c7c0 0x7f6a64097340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a5bfff700 1 -- 172.21.15.140:0/460378168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a64146290 msgr2=0x7f6a64155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a5bfff700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a64146290 0x7f6a64155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a5bfff700 1 -- 172.21.15.140:0/460378168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c00b180 con 0x7f6a640ab250 2022-07-21T01:54:12.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a5bfff700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a640ab250 0x7f6a641422f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f6a5c001490 tx=0x7f6a4c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:12.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a61ffb700 1 -- 172.21.15.140:0/460378168 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c016070 con 0x7f6a640ab250 2022-07-21T01:54:12.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a61ffb700 1 -- 172.21.15.140:0/460378168 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c00bec0 con 0x7f6a640ab250 2022-07-21T01:54:12.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a697d1700 1 -- 172.21.15.140:0/460378168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a640655b0 con 0x7f6a640ab250 2022-07-21T01:54:12.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a61ffb700 1 -- 172.21.15.140:0/460378168 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a4c012460 con 0x7f6a640ab250 2022-07-21T01:54:12.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a697d1700 1 -- 172.21.15.140:0/460378168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a64156270 con 0x7f6a640ab250 2022-07-21T01:54:12.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a697d1700 1 -- 172.21.15.140:0/460378168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a640b0a00 con 0x7f6a640ab250 2022-07-21T01:54:12.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a61ffb700 1 -- 172.21.15.140:0/460378168 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6a4c025030 con 0x7f6a640ab250 2022-07-21T01:54:12.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a61ffb700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a480539e0 0x7f6a48055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a61ffb700 1 -- 172.21.15.140:0/460378168 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6a4c06ccf0 con 0x7f6a640ab250 2022-07-21T01:54:12.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.551+0000 7f6a63fff700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a480539e0 0x7f6a48055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.555+0000 7f6a63fff700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a480539e0 0x7f6a48055ea0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f6a5400b000 tx=0x7f6a54009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:12.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.559+0000 7f6a61ffb700 1 -- 172.21.15.140:0/460378168 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6a4c043c10 con 0x7f6a640ab250 2022-07-21T01:54:12.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.723+0000 7f6a697d1700 1 -- 172.21.15.140:0/460378168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6a64062290 con 0x7f6a640ab250 2022-07-21T01:54:12.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.731+0000 7f6a61ffb700 1 -- 172.21.15.140:0/460378168 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6a4c023020 con 0x7f6a640ab250 2022-07-21T01:54:12.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 -- 172.21.15.140:0/460378168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a480539e0 msgr2=0x7f6a48055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a480539e0 0x7f6a48055ea0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f6a5400b000 tx=0x7f6a54009040).stop 2022-07-21T01:54:12.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 -- 172.21.15.140:0/460378168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a640ab250 msgr2=0x7f6a641422f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a640ab250 0x7f6a641422f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f6a5c001490 tx=0x7f6a4c014000).stop 2022-07-21T01:54:12.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 -- 172.21.15.140:0/460378168 shutdown_connections 2022-07-21T01:54:12.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a480539e0 0x7f6a48055ea0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a64146290 0x7f6a64155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a640ab250 0x7f6a641422f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 --2- 172.21.15.140:0/460378168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6a6405c7c0 0x7f6a64097340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 -- 172.21.15.140:0/460378168 >> 172.21.15.140:0/460378168 conn(0x7f6a640b6210 msgr2=0x7f6a640b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:12.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 -- 172.21.15.140:0/460378168 shutdown_connections 2022-07-21T01:54:12.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.735+0000 7f6a697d1700 1 -- 172.21.15.140:0/460378168 wait complete. 2022-07-21T01:54:12.735 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:54:12.746 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:54:12.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.855+0000 7fabe89bd700 1 Processor -- start 2022-07-21T01:54:12.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.855+0000 7fabe89bd700 1 -- start start 2022-07-21T01:54:12.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.855+0000 7fabe89bd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe40acc10 0x7fabe40acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fabe4096680 0x7fabe40b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe40abde0 con 0x7fabe4095b10 2022-07-21T01:54:12.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe4096c80 con 0x7fabe4096680 2022-07-21T01:54:12.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe4098910 con 0x7fabe40acc10 2022-07-21T01:54:12.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe359e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fabe4096680 0x7fabe40b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe359e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fabe4096680 0x7fabe40b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36652/0 (socket says 172.21.15.140:36652) 2022-07-21T01:54:12.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe359e700 1 -- 172.21.15.140:0/3530406071 learned_addr learned my addr 172.21.15.140:0/3530406071 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:12.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe2d9d700 1 --2- 172.21.15.140:0/3530406071 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe40acc10 0x7fabe40acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe359e700 1 -- 172.21.15.140:0/3530406071 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe40acc10 msgr2=0x7fabe40acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:12.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe359e700 1 --2- 172.21.15.140:0/3530406071 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe40acc10 0x7fabe40acff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe359e700 1 -- 172.21.15.140:0/3530406071 >> v1:172.21.15.140:6789/0 conn(0x7fabe4095b10 legacy=0x7fabe4095f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe359e700 1 -- 172.21.15.140:0/3530406071 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe4098b20 con 0x7fabe4096680 2022-07-21T01:54:12.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe1d9b700 1 -- 172.21.15.140:0/3530406071 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720735096 0 0) 0x7fabe40abde0 con 0x7fabe4095b10 2022-07-21T01:54:12.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe2d9d700 1 --2- 172.21.15.140:0/3530406071 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe40acc10 0x7fabe40acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:12.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe359e700 1 --2- 172.21.15.140:0/3530406071 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fabe4096680 0x7fabe40b0c20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fabd4002f70 tx=0x7fabd4006f40).ready entity=mon.0 client_cookie=c37a904739456286 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:12.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe1d9b700 1 -- 172.21.15.140:0/3530406071 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabd400dc70 con 0x7fabe4096680 2022-07-21T01:54:12.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe1d9b700 1 -- 172.21.15.140:0/3530406071 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd4000e90 con 0x7fabe4096680 2022-07-21T01:54:12.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe1d9b700 1 -- 172.21.15.140:0/3530406071 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabd400be90 con 0x7fabe4096680 2022-07-21T01:54:12.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- 172.21.15.140:0/3530406071 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fabe4096680 msgr2=0x7fabe40b0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 --2- 172.21.15.140:0/3530406071 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fabe4096680 0x7fabe40b0c20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fabd4002f70 tx=0x7fabd4006f40).stop 2022-07-21T01:54:12.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- 172.21.15.140:0/3530406071 shutdown_connections 2022-07-21T01:54:12.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 --2- 172.21.15.140:0/3530406071 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fabe4096680 0x7fabe40b0c20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 --2- 172.21.15.140:0/3530406071 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe40acc10 0x7fabe40acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- 172.21.15.140:0/3530406071 >> 172.21.15.140:0/3530406071 conn(0x7fabe40b6210 msgr2=0x7fabe40b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:12.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- 172.21.15.140:0/3530406071 shutdown_connections 2022-07-21T01:54:12.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- 172.21.15.140:0/3530406071 wait complete. 2022-07-21T01:54:12.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 Processor -- start 2022-07-21T01:54:12.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- start start 2022-07-21T01:54:12.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabe4095b10 0x7fabe405fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe4096680 0x7fabe40ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fabe40acc10 0x7fabe41512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe4151ea0 con 0x7fabe40acc10 2022-07-21T01:54:12.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe4151c00 con 0x7fabe4095b10 2022-07-21T01:54:12.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe89bd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe40af330 con 0x7fabe4096680 2022-07-21T01:54:12.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fabe40acc10 0x7fabe41512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fabe40acc10 0x7fabe41512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36656/0 (socket says 172.21.15.140:36656) 2022-07-21T01:54:12.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe359e700 1 -- 172.21.15.140:0/3329357748 learned_addr learned my addr 172.21.15.140:0/3329357748 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:12.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe2d9d700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabe4095b10 0x7fabe405fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.859+0000 7fabe259c700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe4096680 0x7fabe40ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabe359e700 1 -- 172.21.15.140:0/3329357748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe4096680 msgr2=0x7fabe40ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabe359e700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe4096680 0x7fabe40ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabe359e700 1 -- 172.21.15.140:0/3329357748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabe4095b10 msgr2=0x7fabe405fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:12.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabe359e700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabe4095b10 0x7fabe405fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:12.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabe359e700 1 -- 172.21.15.140:0/3329357748 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd400b460 con 0x7fabe40acc10 2022-07-21T01:54:12.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabe259c700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe4096680 0x7fabe40ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:12.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabe359e700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fabe40acc10 0x7fabe41512d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fabd4002f70 tx=0x7fabe40abf70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:12.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabd3fff700 1 -- 172.21.15.140:0/3329357748 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabd4010cf0 con 0x7fabe40acc10 2022-07-21T01:54:12.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabd3fff700 1 -- 172.21.15.140:0/3329357748 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd4010e50 con 0x7fabe40acc10 2022-07-21T01:54:12.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabe89bd700 1 -- 172.21.15.140:0/3329357748 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe405ff50 con 0x7fabe40acc10 2022-07-21T01:54:12.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabd3fff700 1 -- 172.21.15.140:0/3329357748 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabd4024640 con 0x7fabe40acc10 2022-07-21T01:54:12.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabe89bd700 1 -- 172.21.15.140:0/3329357748 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe40ae070 con 0x7fabe40acc10 2022-07-21T01:54:12.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.863+0000 7fabe89bd700 1 -- 172.21.15.140:0/3329357748 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe40574e0 con 0x7fabe40acc10 2022-07-21T01:54:12.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.867+0000 7fabd3fff700 1 -- 172.21.15.140:0/3329357748 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fabd4021070 con 0x7fabe40acc10 2022-07-21T01:54:12.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.867+0000 7fabd3fff700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fabc00539e0 0x7fabc0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:12.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.867+0000 7fabd3fff700 1 -- 172.21.15.140:0/3329357748 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fabd406f2a0 con 0x7fabe40acc10 2022-07-21T01:54:12.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.867+0000 7fabe2d9d700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fabc00539e0 0x7fabc0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:12.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.867+0000 7fabe2d9d700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fabc00539e0 0x7fabc0055ea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fabdc00c000 tx=0x7fabdc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:12.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:12.875+0000 7fabd3fff700 1 -- 172.21.15.140:0/3329357748 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fabd40460b0 con 0x7fabe40acc10 2022-07-21T01:54:13.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.035+0000 7fabe89bd700 1 -- 172.21.15.140:0/3329357748 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fabe40447e0 con 0x7fabe40acc10 2022-07-21T01:54:13.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.043+0000 7fabd3fff700 1 -- 172.21.15.140:0/3329357748 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v28) v1 ==== 184+0+0 (secure 0 0 0) 0x7fabd40496d0 con 0x7fabe40acc10 2022-07-21T01:54:13.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 -- 172.21.15.140:0/3329357748 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fabc00539e0 msgr2=0x7fabc0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fabc00539e0 0x7fabc0055ea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fabdc00c000 tx=0x7fabdc008040).stop 2022-07-21T01:54:13.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 -- 172.21.15.140:0/3329357748 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fabe40acc10 msgr2=0x7fabe41512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fabe40acc10 0x7fabe41512d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7fabd4002f70 tx=0x7fabe40abf70).stop 2022-07-21T01:54:13.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 -- 172.21.15.140:0/3329357748 shutdown_connections 2022-07-21T01:54:13.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fabc00539e0 0x7fabc0055ea0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fabe40acc10 0x7fabe41512d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fabe4096680 0x7fabe40ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 --2- 172.21.15.140:0/3329357748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fabe4095b10 0x7fabe405fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 -- 172.21.15.140:0/3329357748 >> 172.21.15.140:0/3329357748 conn(0x7fabe40b6210 msgr2=0x7fabe40b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:13.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 -- 172.21.15.140:0/3329357748 shutdown_connections 2022-07-21T01:54:13.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.047+0000 7fabe89bd700 1 -- 172.21.15.140:0/3329357748 wait complete. 2022-07-21T01:54:13.049 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:54:13.059 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:54:13.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.187+0000 7fc40ffff700 1 Processor -- start 2022-07-21T01:54:13.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.187+0000 7fc40ffff700 1 -- start start 2022-07-21T01:54:13.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.187+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4100b6800 0x7fc4100aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.187+0000 7fc40ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc4100b3cb0 0x7fc410145f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.187+0000 7fc40ffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4100ae280 con 0x7fc4100ad0e0 2022-07-21T01:54:13.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.187+0000 7fc40ffff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100b6be0 con 0x7fc4100b3cb0 2022-07-21T01:54:13.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.187+0000 7fc40ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100ae4b0 con 0x7fc4100b6800 2022-07-21T01:54:13.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.187+0000 7fc40f7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc4100ad0e0 0x7fc41005bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50812/0 (socket says 172.21.15.140:50812) 2022-07-21T01:54:13.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.187+0000 7fc40f7fe700 1 -- 172.21.15.140:0/3702212199 learned_addr learned my addr 172.21.15.140:0/3702212199 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:13.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.187+0000 7fc40effd700 1 --2- 172.21.15.140:0/3702212199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4100b6800 0x7fc4100aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.187+0000 7fc40e7fc700 1 --2- 172.21.15.140:0/3702212199 >> v2:172.21.15.152:3300/0 conn(0x7fc4100b3cb0 0x7fc410145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40dffb700 1 -- 172.21.15.140:0/3702212199 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547710540 0 0) 0x7fc4100ae280 con 0x7fc4100ad0e0 2022-07-21T01:54:13.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40dffb700 1 -- 172.21.15.140:0/3702212199 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3f0003260 con 0x7fc4100ad0e0 2022-07-21T01:54:13.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40effd700 1 -- 172.21.15.140:0/3702212199 >> v2:172.21.15.152:3300/0 conn(0x7fc4100b3cb0 msgr2=0x7fc410145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:13.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40effd700 1 --2- 172.21.15.140:0/3702212199 >> v2:172.21.15.152:3300/0 conn(0x7fc4100b3cb0 0x7fc410145f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40effd700 1 -- 172.21.15.140:0/3702212199 >> v1:172.21.15.140:6789/0 conn(0x7fc4100ad0e0 legacy=0x7fc41005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40effd700 1 -- 172.21.15.140:0/3702212199 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4100ae750 con 0x7fc4100b6800 2022-07-21T01:54:13.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40effd700 1 --2- 172.21.15.140:0/3702212199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4100b6800 0x7fc4100aebf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc4080032f0 tx=0x7fc4080091d0).ready entity=mon.2 client_cookie=71349e09a616479c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:13.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40dffb700 1 -- 172.21.15.140:0/3702212199 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40800b600 con 0x7fc4100b6800 2022-07-21T01:54:13.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40dffb700 1 -- 172.21.15.140:0/3702212199 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40800b760 con 0x7fc4100b6800 2022-07-21T01:54:13.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40dffb700 1 -- 172.21.15.140:0/3702212199 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40800b8d0 con 0x7fc4100b6800 2022-07-21T01:54:13.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40ffff700 1 -- 172.21.15.140:0/3702212199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4100b6800 msgr2=0x7fc4100aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40ffff700 1 --2- 172.21.15.140:0/3702212199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4100b6800 0x7fc4100aebf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc4080032f0 tx=0x7fc4080091d0).stop 2022-07-21T01:54:13.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40ffff700 1 -- 172.21.15.140:0/3702212199 shutdown_connections 2022-07-21T01:54:13.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40ffff700 1 --2- 172.21.15.140:0/3702212199 >> v2:172.21.15.152:3300/0 conn(0x7fc4100b3cb0 0x7fc410145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40ffff700 1 --2- 172.21.15.140:0/3702212199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4100b6800 0x7fc4100aebf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40ffff700 1 -- 172.21.15.140:0/3702212199 >> 172.21.15.140:0/3702212199 conn(0x7fc4100b9dd0 msgr2=0x7fc4100b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:13.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40ffff700 1 -- 172.21.15.140:0/3702212199 shutdown_connections 2022-07-21T01:54:13.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.191+0000 7fc40ffff700 1 -- 172.21.15.140:0/3702212199 wait complete. 2022-07-21T01:54:13.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40ffff700 1 Processor -- start 2022-07-21T01:54:13.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40ffff700 1 -- start start 2022-07-21T01:54:13.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4100ad0e0 0x7fc410115150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4100b3cb0 0x7fc4100b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4100b6800 0x7fc410151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100b30c0 con 0x7fc4100b3cb0 2022-07-21T01:54:13.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc410151fb0 con 0x7fc4100ad0e0 2022-07-21T01:54:13.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4101521a0 con 0x7fc4100b6800 2022-07-21T01:54:13.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40e7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4100b3cb0 0x7fc4100b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40effd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4100ad0e0 0x7fc410115150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40e7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4100b3cb0 0x7fc4100b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36670/0 (socket says 172.21.15.140:36670) 2022-07-21T01:54:13.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40e7fc700 1 -- 172.21.15.140:0/318082975 learned_addr learned my addr 172.21.15.140:0/318082975 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:13.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40f7fe700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4100b6800 0x7fc410151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40e7fc700 1 -- 172.21.15.140:0/318082975 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4100b6800 msgr2=0x7fc410151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40e7fc700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4100b6800 0x7fc410151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40e7fc700 1 -- 172.21.15.140:0/318082975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4100ad0e0 msgr2=0x7fc410115150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40e7fc700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4100ad0e0 0x7fc410115150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.195+0000 7fc40e7fc700 1 -- 172.21.15.140:0/318082975 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc408009470 con 0x7fc4100b3cb0 2022-07-21T01:54:13.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.199+0000 7fc40e7fc700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4100b3cb0 0x7fc4100b2900 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fc3f800c000 tx=0x7fc3f8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:13.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.199+0000 7fc407fff700 1 -- 172.21.15.140:0/318082975 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3f8010780 con 0x7fc4100b3cb0 2022-07-21T01:54:13.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.199+0000 7fc407fff700 1 -- 172.21.15.140:0/318082975 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f80108e0 con 0x7fc4100b3cb0 2022-07-21T01:54:13.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.199+0000 7fc40ffff700 1 -- 172.21.15.140:0/318082975 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc410152440 con 0x7fc4100b3cb0 2022-07-21T01:54:13.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.199+0000 7fc407fff700 1 -- 172.21.15.140:0/318082975 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3f8010c40 con 0x7fc4100b3cb0 2022-07-21T01:54:13.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.199+0000 7fc40ffff700 1 -- 172.21.15.140:0/318082975 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc410141670 con 0x7fc4100b3cb0 2022-07-21T01:54:13.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.199+0000 7fc40ffff700 1 -- 172.21.15.140:0/318082975 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4100b5530 con 0x7fc4100b3cb0 2022-07-21T01:54:13.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.199+0000 7fc407fff700 1 -- 172.21.15.140:0/318082975 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc3f801c460 con 0x7fc4100b3cb0 2022-07-21T01:54:13.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.199+0000 7fc407fff700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3f4053a70 0x7fc3f4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.199+0000 7fc407fff700 1 -- 172.21.15.140:0/318082975 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc3f806c2b0 con 0x7fc4100b3cb0 2022-07-21T01:54:13.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.203+0000 7fc40effd700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3f4053a70 0x7fc3f4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.203+0000 7fc40effd700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3f4053a70 0x7fc3f4055f30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fc40801b000 tx=0x7fc408016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:13.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.207+0000 7fc407fff700 1 -- 172.21.15.140:0/318082975 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc3f8043170 con 0x7fc4100b3cb0 2022-07-21T01:54:13.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.371+0000 7fc40ffff700 1 -- 172.21.15.140:0/318082975 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc4100574e0 con 0x7fc4100b3cb0 2022-07-21T01:54:13.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.379+0000 7fc407fff700 1 -- 172.21.15.140:0/318082975 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v29) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc3f8046790 con 0x7fc4100b3cb0 2022-07-21T01:54:13.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 -- 172.21.15.140:0/318082975 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3f4053a70 msgr2=0x7fc3f4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3f4053a70 0x7fc3f4055f30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fc40801b000 tx=0x7fc408016000).stop 2022-07-21T01:54:13.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 -- 172.21.15.140:0/318082975 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4100b3cb0 msgr2=0x7fc4100b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4100b3cb0 0x7fc4100b2900 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fc3f800c000 tx=0x7fc3f8006df0).stop 2022-07-21T01:54:13.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 -- 172.21.15.140:0/318082975 shutdown_connections 2022-07-21T01:54:13.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc3f4053a70 0x7fc3f4055f30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc4100b6800 0x7fc410151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc4100b3cb0 0x7fc4100b2900 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 --2- 172.21.15.140:0/318082975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc4100ad0e0 0x7fc410115150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 -- 172.21.15.140:0/318082975 >> 172.21.15.140:0/318082975 conn(0x7fc4100b9dd0 msgr2=0x7fc410148170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:13.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 -- 172.21.15.140:0/318082975 shutdown_connections 2022-07-21T01:54:13.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.383+0000 7fc40ffff700 1 -- 172.21.15.140:0/318082975 wait complete. 2022-07-21T01:54:13.382 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:54:13.391 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:54:13.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.515+0000 7f769ae41700 1 Processor -- start 2022-07-21T01:54:13.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.515+0000 7f769ae41700 1 -- start start 2022-07-21T01:54:13.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.515+0000 7f769ae41700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76941424e0 0x7f76940b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.515+0000 7f769ae41700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f76940aea10 0x7f76940ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.515+0000 7f769ae41700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76941433d0 con 0x7f76940b0570 2022-07-21T01:54:13.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.515+0000 7f769ae41700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7694143b10 con 0x7f76940aea10 2022-07-21T01:54:13.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.515+0000 7f769ae41700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7694143670 con 0x7f76941424e0 2022-07-21T01:54:13.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769963e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76941424e0 0x7f76940b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769a640700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f76940aea10 0x7f76940ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f7699e3f700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f76940b0570 0x7f7694141dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:51936/0 (socket says 172.21.15.140:51936) 2022-07-21T01:54:13.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f7699e3f700 1 -- 172.21.15.140:0/2264082488 learned_addr learned my addr 172.21.15.140:0/2264082488 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:13.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769a640700 1 -- 172.21.15.140:0/2264082488 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76941424e0 msgr2=0x7f76940b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:13.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769a640700 1 --2- 172.21.15.140:0/2264082488 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76941424e0 0x7f76940b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769a640700 1 -- 172.21.15.140:0/2264082488 >> v1:172.21.15.152:6789/0 conn(0x7f76940b0570 legacy=0x7f7694141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769a640700 1 -- 172.21.15.140:0/2264082488 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76940b1010 con 0x7f76940aea10 2022-07-21T01:54:13.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769a640700 1 --2- 172.21.15.140:0/2264082488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76940aea10 0x7f76940ad350 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f7684002f70 tx=0x7f7684006df0).ready entity=mon.1 client_cookie=472e72a0581651dc server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:13.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f7698e3d700 1 -- 172.21.15.140:0/2264082488 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f768400d570 con 0x7f76940aea10 2022-07-21T01:54:13.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f7698e3d700 1 -- 172.21.15.140:0/2264082488 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7684000e90 con 0x7f76940aea10 2022-07-21T01:54:13.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f7698e3d700 1 -- 172.21.15.140:0/2264082488 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f768400d8c0 con 0x7f76940aea10 2022-07-21T01:54:13.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769ae41700 1 -- 172.21.15.140:0/2264082488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76940aea10 msgr2=0x7f76940ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769ae41700 1 --2- 172.21.15.140:0/2264082488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76940aea10 0x7f76940ad350 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f7684002f70 tx=0x7f7684006df0).stop 2022-07-21T01:54:13.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769ae41700 1 -- 172.21.15.140:0/2264082488 shutdown_connections 2022-07-21T01:54:13.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769ae41700 1 --2- 172.21.15.140:0/2264082488 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76941424e0 0x7f76940b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769ae41700 1 --2- 172.21.15.140:0/2264082488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76940aea10 0x7f76940ad350 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769ae41700 1 -- 172.21.15.140:0/2264082488 >> 172.21.15.140:0/2264082488 conn(0x7f76940b9360 msgr2=0x7f76940b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:13.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769ae41700 1 -- 172.21.15.140:0/2264082488 shutdown_connections 2022-07-21T01:54:13.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769ae41700 1 -- 172.21.15.140:0/2264082488 wait complete. 2022-07-21T01:54:13.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769ae41700 1 Processor -- start 2022-07-21T01:54:13.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.519+0000 7f769ae41700 1 -- start start 2022-07-21T01:54:13.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769ae41700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76940aea10 0x7f7694114d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769ae41700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76940b0570 0x7f76940b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769ae41700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76941424e0 0x7f76940b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769ae41700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769413e930 con 0x7f76940b0570 2022-07-21T01:54:13.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769ae41700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7694114340 con 0x7f76941424e0 2022-07-21T01:54:13.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769ae41700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76941145e0 con 0x7f76940aea10 2022-07-21T01:54:13.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769963e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76940b0570 0x7f76940b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769a640700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76941424e0 0x7f76940b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f7699e3f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76940aea10 0x7f7694114d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769963e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76940b0570 0x7f76940b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36684/0 (socket says 172.21.15.140:36684) 2022-07-21T01:54:13.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769963e700 1 -- 172.21.15.140:0/2669103014 learned_addr learned my addr 172.21.15.140:0/2669103014 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:13.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769963e700 1 -- 172.21.15.140:0/2669103014 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76940aea10 msgr2=0x7f7694114d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769963e700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76940aea10 0x7f7694114d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769963e700 1 -- 172.21.15.140:0/2669103014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76941424e0 msgr2=0x7f76940b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769963e700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76941424e0 0x7f76940b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769963e700 1 -- 172.21.15.140:0/2669103014 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768400b180 con 0x7f76940b0570 2022-07-21T01:54:13.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f7699e3f700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76940aea10 0x7f7694114d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:13.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769a640700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76941424e0 0x7f76940b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:13.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769963e700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76940b0570 0x7f76940b4ca0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f768800c000 tx=0x7f76880091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:13.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f7692ffd700 1 -- 172.21.15.140:0/2669103014 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7688012600 con 0x7f76940b0570 2022-07-21T01:54:13.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769ae41700 1 -- 172.21.15.140:0/2669103014 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7694114880 con 0x7f76940b0570 2022-07-21T01:54:13.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f7692ffd700 1 -- 172.21.15.140:0/2669103014 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7688012760 con 0x7f76940b0570 2022-07-21T01:54:13.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f7692ffd700 1 -- 172.21.15.140:0/2669103014 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7688012ac0 con 0x7f76940b0570 2022-07-21T01:54:13.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769ae41700 1 -- 172.21.15.140:0/2669103014 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76940b3ba0 con 0x7f76940b0570 2022-07-21T01:54:13.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.523+0000 7f769ae41700 1 -- 172.21.15.140:0/2669103014 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7694114a10 con 0x7f76940b0570 2022-07-21T01:54:13.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.527+0000 7f7692ffd700 1 -- 172.21.15.140:0/2669103014 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7688022070 con 0x7f76940b0570 2022-07-21T01:54:13.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.527+0000 7f7692ffd700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f76740539e0 0x7f7674055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:13.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.527+0000 7f7692ffd700 1 -- 172.21.15.140:0/2669103014 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f768806c660 con 0x7f76940b0570 2022-07-21T01:54:13.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.527+0000 7f7699e3f700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f76740539e0 0x7f7674055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:13.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.527+0000 7f7699e3f700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f76740539e0 0x7f7674055ea0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f768c00d000 tx=0x7f768c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:13.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.531+0000 7f7692ffd700 1 -- 172.21.15.140:0/2669103014 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f76880433f0 con 0x7f76940b0570 2022-07-21T01:54:13.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.691+0000 7f769ae41700 1 -- 172.21.15.140:0/2669103014 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76940574e0 con 0x7f76940b0570 2022-07-21T01:54:13.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.691+0000 7f7692ffd700 1 -- 172.21.15.140:0/2669103014 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f7688046a10 con 0x7f76940b0570 2022-07-21T01:54:13.690 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:13.690 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":30,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/4173658089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4173658089},{"type":"v1","addr":"172.21.15.152:6833","nonce":4173658089}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:54:10.830421+0000","modified":"2022-07-21T01:54:11.841578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6208":{"gid":6208,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/167323020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":167323020},{"type":"v1","addr":"172.21.15.140:6835","nonce":167323020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-07-21T01:54:13.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 -- 172.21.15.140:0/2669103014 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f76740539e0 msgr2=0x7f7674055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f76740539e0 0x7f7674055ea0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f768c00d000 tx=0x7f768c009040).stop 2022-07-21T01:54:13.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 -- 172.21.15.140:0/2669103014 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76940b0570 msgr2=0x7f76940b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:13.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76940b0570 0x7f76940b4ca0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f768800c000 tx=0x7f76880091d0).stop 2022-07-21T01:54:13.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 -- 172.21.15.140:0/2669103014 shutdown_connections 2022-07-21T01:54:13.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f76740539e0 0x7f7674055ea0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76941424e0 0x7f76940b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f76940b0570 0x7f76940b4ca0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 --2- 172.21.15.140:0/2669103014 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f76940aea10 0x7f7694114d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:13.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 -- 172.21.15.140:0/2669103014 >> 172.21.15.140:0/2669103014 conn(0x7f76940b9360 msgr2=0x7f7694145d90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:13.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 -- 172.21.15.140:0/2669103014 shutdown_connections 2022-07-21T01:54:13.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:13.695+0000 7f769ae41700 1 -- 172.21.15.140:0/2669103014 wait complete. 2022-07-21T01:54:13.695 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 30 2022-07-21T01:54:13.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:54:10.830421+0000', 'modified': '2022-07-21T01:54:11.841578+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6208}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6208': {'gid': 6208, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/167323020', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 167323020}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 167323020}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:54:13.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:54:13.906 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:54:14.023 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":6208,"want_state":"up:active","state":"up:active","rank_uptime":3.1751871170000001,"mdsmap_epoch":30,"osdmap_epoch":73,"osdmap_epoch_barrier":72,"uptime":7.2469605289999999} 2022-07-21T01:54:14.024 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 6208, "mdsmap_epoch": 30, "osdmap_epoch": 73, "osdmap_epoch_barrier": 72, "rank_uptime": 3.175187117, "state": "up:active", "uptime": 7.246960529, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:54:14.025 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:54:14.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.135+0000 7fd3f359e700 1 Processor -- start 2022-07-21T01:54:14.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.135+0000 7fd3f359e700 1 -- start start 2022-07-21T01:54:14.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.135+0000 7fd3f359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3f40adff0 0x7fd3f40b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.135+0000 7fd3f359e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd3f40af8e0 0x7fd3f405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.135+0000 7fd3f359e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f4147010 con 0x7fd3f40b5650 2022-07-21T01:54:14.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.135+0000 7fd3f359e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f41472b0 con 0x7fd3f40af8e0 2022-07-21T01:54:14.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.135+0000 7fd3f359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f4146d40 con 0x7fd3f40adff0 2022-07-21T01:54:14.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f2d9d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd3f40af8e0 0x7fd3f405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f2d9d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd3f40af8e0 0x7fd3f405bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42434/0 (socket says 172.21.15.140:42434) 2022-07-21T01:54:14.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f2d9d700 1 -- 172.21.15.140:0/1398235462 learned_addr learned my addr 172.21.15.140:0/1398235462 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:14.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f259c700 1 --1- 172.21.15.140:0/1398235462 >> v1:172.21.15.152:6789/0 conn(0x7fd3f40b5650 0x7fd3f4147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:51950/0 (socket says 172.21.15.140:51950) 2022-07-21T01:54:14.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f1d9b700 1 --2- 172.21.15.140:0/1398235462 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3f40adff0 0x7fd3f40b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f2d9d700 1 -- 172.21.15.140:0/1398235462 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3f40adff0 msgr2=0x7fd3f40b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:14.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f2d9d700 1 --2- 172.21.15.140:0/1398235462 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3f40adff0 0x7fd3f40b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f2d9d700 1 -- 172.21.15.140:0/1398235462 >> v1:172.21.15.152:6789/0 conn(0x7fd3f40b5650 legacy=0x7fd3f4147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f2d9d700 1 -- 172.21.15.140:0/1398235462 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f40acae0 con 0x7fd3f40af8e0 2022-07-21T01:54:14.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f2d9d700 1 --2- 172.21.15.140:0/1398235462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f40af8e0 0x7fd3f405bd90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fd3ec0032f0 tx=0x7fd3ec009290).ready entity=mon.1 client_cookie=4197d8ce29d75f6d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f159a700 1 -- 172.21.15.140:0/1398235462 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3ec00b8a0 con 0x7fd3f40af8e0 2022-07-21T01:54:14.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f159a700 1 -- 172.21.15.140:0/1398235462 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ec00ba00 con 0x7fd3f40af8e0 2022-07-21T01:54:14.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f159a700 1 -- 172.21.15.140:0/1398235462 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3ec00bb70 con 0x7fd3f40af8e0 2022-07-21T01:54:14.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f359e700 1 -- 172.21.15.140:0/1398235462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f40af8e0 msgr2=0x7fd3f405bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f359e700 1 --2- 172.21.15.140:0/1398235462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f40af8e0 0x7fd3f405bd90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fd3ec0032f0 tx=0x7fd3ec009290).stop 2022-07-21T01:54:14.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f359e700 1 -- 172.21.15.140:0/1398235462 shutdown_connections 2022-07-21T01:54:14.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f359e700 1 --2- 172.21.15.140:0/1398235462 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3f40adff0 0x7fd3f40b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f359e700 1 --2- 172.21.15.140:0/1398235462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f40af8e0 0x7fd3f405bd90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f359e700 1 -- 172.21.15.140:0/1398235462 >> 172.21.15.140:0/1398235462 conn(0x7fd3f40634e0 msgr2=0x7fd3f40b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:14.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f359e700 1 -- 172.21.15.140:0/1398235462 shutdown_connections 2022-07-21T01:54:14.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f359e700 1 -- 172.21.15.140:0/1398235462 wait complete. 2022-07-21T01:54:14.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f359e700 1 Processor -- start 2022-07-21T01:54:14.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.139+0000 7fd3f359e700 1 -- start start 2022-07-21T01:54:14.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3f40adff0 0x7fd3f4111830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3f40af8e0 0x7fd3f4156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f359e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f40b5650 0x7fd3f4149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f359e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f410fc80 con 0x7fd3f40af8e0 2022-07-21T01:54:14.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f359e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f410ff20 con 0x7fd3f40b5650 2022-07-21T01:54:14.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f410e1d0 con 0x7fd3f40adff0 2022-07-21T01:54:14.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f1d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3f40af8e0 0x7fd3f4156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f1d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3f40af8e0 0x7fd3f4156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36696/0 (socket says 172.21.15.140:36696) 2022-07-21T01:54:14.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f1d9b700 1 -- 172.21.15.140:0/2604850244 learned_addr learned my addr 172.21.15.140:0/2604850244 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:14.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f259c700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3f40adff0 0x7fd3f4111830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f2d9d700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f40b5650 0x7fd3f4149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f1d9b700 1 -- 172.21.15.140:0/2604850244 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3f40adff0 msgr2=0x7fd3f4111830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f1d9b700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3f40adff0 0x7fd3f4111830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f1d9b700 1 -- 172.21.15.140:0/2604850244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f40b5650 msgr2=0x7fd3f4149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f1d9b700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f40b5650 0x7fd3f4149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f1d9b700 1 -- 172.21.15.140:0/2604850244 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec009530 con 0x7fd3f40af8e0 2022-07-21T01:54:14.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f2d9d700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f40b5650 0x7fd3f4149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:14.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f1d9b700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3f40af8e0 0x7fd3f4156420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7fd3dc00a000 tx=0x7fd3dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3eb7fe700 1 -- 172.21.15.140:0/2604850244 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3dc011850 con 0x7fd3f40af8e0 2022-07-21T01:54:14.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3eb7fe700 1 -- 172.21.15.140:0/2604850244 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc0119b0 con 0x7fd3f40af8e0 2022-07-21T01:54:14.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f359e700 1 -- 172.21.15.140:0/2604850244 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f410f5b0 con 0x7fd3f40af8e0 2022-07-21T01:54:14.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3eb7fe700 1 -- 172.21.15.140:0/2604850244 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3dc011d10 con 0x7fd3f40af8e0 2022-07-21T01:54:14.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f359e700 1 -- 172.21.15.140:0/2604850244 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f4066f30 con 0x7fd3f40af8e0 2022-07-21T01:54:14.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.143+0000 7fd3f359e700 1 -- 172.21.15.140:0/2604850244 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f410e390 con 0x7fd3f40af8e0 2022-07-21T01:54:14.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.147+0000 7fd3eb7fe700 1 -- 172.21.15.140:0/2604850244 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd3dc01c460 con 0x7fd3f40af8e0 2022-07-21T01:54:14.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.147+0000 7fd3eb7fe700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd3cc053a70 0x7fd3cc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.147+0000 7fd3eb7fe700 1 -- 172.21.15.140:0/2604850244 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd3dc06c490 con 0x7fd3f40af8e0 2022-07-21T01:54:14.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.147+0000 7fd3f259c700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd3cc053a70 0x7fd3cc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.147+0000 7fd3f259c700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd3cc053a70 0x7fd3cc055f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fd3e400d000 tx=0x7fd3e4000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.151+0000 7fd3eb7fe700 1 -- 172.21.15.140:0/2604850244 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd3dc043350 con 0x7fd3f40af8e0 2022-07-21T01:54:14.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.279+0000 7fd3f359e700 1 -- 172.21.15.140:0/2604850244 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3f40574e0 con 0x7fd3f40af8e0 2022-07-21T01:54:14.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.279+0000 7fd3eb7fe700 1 -- 172.21.15.140:0/2604850244 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16137 (secure 0 0 0) 0x7fd3dc046970 con 0x7fd3f40af8e0 2022-07-21T01:54:14.277 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:14.277 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":73,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:54:11.862650+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:54:09.143212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-07-21T01:54:09.749431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:54:14.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.279+0000 7fd3f359e700 1 -- 172.21.15.140:0/2604850244 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd3cc053a70 msgr2=0x7fd3cc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.283+0000 7fd3f359e700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd3cc053a70 0x7fd3cc055f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fd3e400d000 tx=0x7fd3e4000bc0).stop 2022-07-21T01:54:14.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.283+0000 7fd3f359e700 1 -- 172.21.15.140:0/2604850244 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3f40af8e0 msgr2=0x7fd3f4156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.283+0000 7fd3f359e700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3f40af8e0 0x7fd3f4156420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7fd3dc00a000 tx=0x7fd3dc008040).stop 2022-07-21T01:54:14.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.283+0000 7fd3f359e700 1 -- 172.21.15.140:0/2604850244 shutdown_connections 2022-07-21T01:54:14.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.283+0000 7fd3f359e700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd3cc053a70 0x7fd3cc055f30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.283+0000 7fd3f359e700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f40b5650 0x7fd3f4149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.283+0000 7fd3f359e700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3f40af8e0 0x7fd3f4156420 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.283+0000 7fd3f359e700 1 --2- 172.21.15.140:0/2604850244 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3f40adff0 0x7fd3f4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.283+0000 7fd3f359e700 1 -- 172.21.15.140:0/2604850244 >> 172.21.15.140:0/2604850244 conn(0x7fd3f40634e0 msgr2=0x7fd3f40b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:14.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.283+0000 7fd3f359e700 1 -- 172.21.15.140:0/2604850244 shutdown_connections 2022-07-21T01:54:14.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.283+0000 7fd3f359e700 1 -- 172.21.15.140:0/2604850244 wait complete. 2022-07-21T01:54:14.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:54:09.143212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:54:09.749431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:54:14.291 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:54:14.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.415+0000 7f72d1a8b700 1 Processor -- start 2022-07-21T01:54:14.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.415+0000 7f72d1a8b700 1 -- start start 2022-07-21T01:54:14.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.415+0000 7f72d1a8b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72cc0b3cf0 0x7f72cc142a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.415+0000 7f72d1a8b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f72cc142f40 0x7f72cc1419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.415+0000 7f72d1a8b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc0ba180 con 0x7f72cc0b9da0 2022-07-21T01:54:14.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.415+0000 7f72d1a8b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72cc0ad5b0 con 0x7f72cc142f40 2022-07-21T01:54:14.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.415+0000 7f72d1a8b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc1461b0 con 0x7f72cc0b3cf0 2022-07-21T01:54:14.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.415+0000 7f72d128a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f72cc142f40 0x7f72cc1419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.415+0000 7f72cbfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72cc0b3cf0 0x7f72cc142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.415+0000 7f72d0a89700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f72cc0b9da0 0x7f72cc1468f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:51964/0 (socket says 172.21.15.140:51964) 2022-07-21T01:54:14.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.415+0000 7f72d0a89700 1 -- 172.21.15.140:0/2849053667 learned_addr learned my addr 172.21.15.140:0/2849053667 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:14.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72cbfff700 1 -- 172.21.15.140:0/2849053667 >> v2:172.21.15.152:3300/0 conn(0x7f72cc142f40 msgr2=0x7f72cc1419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:14.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72cbfff700 1 --2- 172.21.15.140:0/2849053667 >> v2:172.21.15.152:3300/0 conn(0x7f72cc142f40 0x7f72cc1419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72cbfff700 1 -- 172.21.15.140:0/2849053667 >> v1:172.21.15.152:6789/0 conn(0x7f72cc0b9da0 legacy=0x7f72cc1468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72cbfff700 1 -- 172.21.15.140:0/2849053667 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc146450 con 0x7f72cc0b3cf0 2022-07-21T01:54:14.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72cbfff700 1 --2- 172.21.15.140:0/2849053667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72cc0b3cf0 0x7f72cc142a50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f72bc002f70 tx=0x7f72bc006df0).ready entity=mon.2 client_cookie=7a79df7844970835 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72cb7fe700 1 -- 172.21.15.140:0/2849053667 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72bc00d570 con 0x7f72cc0b3cf0 2022-07-21T01:54:14.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72cb7fe700 1 -- 172.21.15.140:0/2849053667 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72bc000e90 con 0x7f72cc0b3cf0 2022-07-21T01:54:14.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72cb7fe700 1 -- 172.21.15.140:0/2849053667 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72bc00d8c0 con 0x7f72cc0b3cf0 2022-07-21T01:54:14.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72d1a8b700 1 -- 172.21.15.140:0/2849053667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72cc0b3cf0 msgr2=0x7f72cc142a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72d1a8b700 1 --2- 172.21.15.140:0/2849053667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72cc0b3cf0 0x7f72cc142a50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f72bc002f70 tx=0x7f72bc006df0).stop 2022-07-21T01:54:14.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72d1a8b700 1 -- 172.21.15.140:0/2849053667 shutdown_connections 2022-07-21T01:54:14.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72d1a8b700 1 --2- 172.21.15.140:0/2849053667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72cc0b3cf0 0x7f72cc142a50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72d1a8b700 1 --2- 172.21.15.140:0/2849053667 >> v2:172.21.15.152:3300/0 conn(0x7f72cc142f40 0x7f72cc1419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72d1a8b700 1 -- 172.21.15.140:0/2849053667 >> 172.21.15.140:0/2849053667 conn(0x7f72cc0678c0 msgr2=0x7f72cc0b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:14.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72d1a8b700 1 -- 172.21.15.140:0/2849053667 shutdown_connections 2022-07-21T01:54:14.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72d1a8b700 1 -- 172.21.15.140:0/2849053667 wait complete. 2022-07-21T01:54:14.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72d1a8b700 1 Processor -- start 2022-07-21T01:54:14.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.419+0000 7f72d1a8b700 1 -- start start 2022-07-21T01:54:14.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d1a8b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72cc0b3cf0 0x7f72cc0b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d1a8b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72cc0b9da0 0x7f72cc155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d1a8b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72cc142f40 0x7f72cc0b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d1a8b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc148c20 con 0x7f72cc142f40 2022-07-21T01:54:14.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d1a8b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0b2280 con 0x7f72cc0b9da0 2022-07-21T01:54:14.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d1a8b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc148e50 con 0x7f72cc0b3cf0 2022-07-21T01:54:14.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d128a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72cc142f40 0x7f72cc0b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72cbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72cc0b9da0 0x7f72cc155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d0a89700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72cc0b3cf0 0x7f72cc0b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72cbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72cc0b9da0 0x7f72cc155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42450/0 (socket says 172.21.15.140:42450) 2022-07-21T01:54:14.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72cbfff700 1 -- 172.21.15.140:0/3405426949 learned_addr learned my addr 172.21.15.140:0/3405426949 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:14.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d128a700 1 -- 172.21.15.140:0/3405426949 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72cc0b3cf0 msgr2=0x7f72cc0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d128a700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72cc0b3cf0 0x7f72cc0b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d128a700 1 -- 172.21.15.140:0/3405426949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72cc0b9da0 msgr2=0x7f72cc155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d128a700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72cc0b9da0 0x7f72cc155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d128a700 1 -- 172.21.15.140:0/3405426949 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc00b180 con 0x7f72cc142f40 2022-07-21T01:54:14.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72cbfff700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72cc0b9da0 0x7f72cc155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:14.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d128a700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72cc142f40 0x7f72cc0b26e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f72c400c000 tx=0x7f72c40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72c9ffb700 1 -- 172.21.15.140:0/3405426949 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72c4012600 con 0x7f72cc142f40 2022-07-21T01:54:14.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72c9ffb700 1 -- 172.21.15.140:0/3405426949 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c4012760 con 0x7f72cc142f40 2022-07-21T01:54:14.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72c9ffb700 1 -- 172.21.15.140:0/3405426949 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f72c4012ac0 con 0x7f72cc142f40 2022-07-21T01:54:14.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d1a8b700 1 -- 172.21.15.140:0/3405426949 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc0b0c80 con 0x7f72cc142f40 2022-07-21T01:54:14.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.423+0000 7f72d1a8b700 1 -- 172.21.15.140:0/3405426949 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc0af060 con 0x7f72cc142f40 2022-07-21T01:54:14.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.427+0000 7f72d1a8b700 1 -- 172.21.15.140:0/3405426949 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc0b23f0 con 0x7f72cc142f40 2022-07-21T01:54:14.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.427+0000 7f72c9ffb700 1 -- 172.21.15.140:0/3405426949 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f72c4022070 con 0x7f72cc142f40 2022-07-21T01:54:14.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.427+0000 7f72c9ffb700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ac053b00 0x7f72ac055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.427+0000 7f72c9ffb700 1 -- 172.21.15.140:0/3405426949 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f72c406c4c0 con 0x7f72cc142f40 2022-07-21T01:54:14.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.427+0000 7f72d0a89700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ac053b00 0x7f72ac055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.427+0000 7f72d0a89700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ac053b00 0x7f72ac055fc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f72c000d000 tx=0x7f72c0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.431+0000 7f72c9ffb700 1 -- 172.21.15.140:0/3405426949 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f72c40432e0 con 0x7f72cc142f40 2022-07-21T01:54:14.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 -- 172.21.15.140:0/3405426949 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f72cc0667f0 con 0x7f72cc142f40 2022-07-21T01:54:14.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72c9ffb700 1 -- 172.21.15.140:0/3405426949 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v73) v1 ==== 110+0+10 (secure 0 0 0) 0x7f72c4016020 con 0x7f72cc142f40 2022-07-21T01:54:14.552 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:54:14.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 -- 172.21.15.140:0/3405426949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ac053b00 msgr2=0x7f72ac055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ac053b00 0x7f72ac055fc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f72c000d000 tx=0x7f72c0009040).stop 2022-07-21T01:54:14.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 -- 172.21.15.140:0/3405426949 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72cc142f40 msgr2=0x7f72cc0b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72cc142f40 0x7f72cc0b26e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f72c400c000 tx=0x7f72c40091d0).stop 2022-07-21T01:54:14.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 -- 172.21.15.140:0/3405426949 shutdown_connections 2022-07-21T01:54:14.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f72ac053b00 0x7f72ac055fc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f72cc142f40 0x7f72cc0b26e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72cc0b9da0 0x7f72cc155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 --2- 172.21.15.140:0/3405426949 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72cc0b3cf0 0x7f72cc0b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 -- 172.21.15.140:0/3405426949 >> 172.21.15.140:0/3405426949 conn(0x7f72cc0678c0 msgr2=0x7f72cc0b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:14.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 -- 172.21.15.140:0/3405426949 shutdown_connections 2022-07-21T01:54:14.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.555+0000 7f72d1a8b700 1 -- 172.21.15.140:0/3405426949 wait complete. 2022-07-21T01:54:14.575 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:54:14.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78f562700 1 Processor -- start 2022-07-21T01:54:14.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78f562700 1 -- start start 2022-07-21T01:54:14.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78f562700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc7880b2550 0x7fc7880ab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78f562700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc7880ac650 0x7fc788095b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78f562700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc788143ec0 con 0x7fc788143940 2022-07-21T01:54:14.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78f562700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc788140c10 con 0x7fc7880b2550 2022-07-21T01:54:14.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78f562700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7880b2930 con 0x7fc7880ac650 2022-07-21T01:54:14.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78daff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc788143940 0x7fc788096000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50868/0 (socket says 172.21.15.140:50868) 2022-07-21T01:54:14.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78daff700 1 -- 172.21.15.140:0/1443413649 learned_addr learned my addr 172.21.15.140:0/1443413649 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:14.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78d2fe700 1 --2- 172.21.15.140:0/1443413649 >> v2:172.21.15.140:3300/0 conn(0x7fc7880b2550 0x7fc7880ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78cafd700 1 --2- 172.21.15.140:0/1443413649 >> v2:172.21.15.152:3300/0 conn(0x7fc7880ac650 0x7fc788095b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc77ffff700 1 -- 172.21.15.140:0/1443413649 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903923340 0 0) 0x7fc788143ec0 con 0x7fc788143940 2022-07-21T01:54:14.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc77ffff700 1 -- 172.21.15.140:0/1443413649 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc76c003260 con 0x7fc788143940 2022-07-21T01:54:14.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78d2fe700 1 -- 172.21.15.140:0/1443413649 >> v2:172.21.15.152:3300/0 conn(0x7fc7880ac650 msgr2=0x7fc788095b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:14.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78d2fe700 1 --2- 172.21.15.140:0/1443413649 >> v2:172.21.15.152:3300/0 conn(0x7fc7880ac650 0x7fc788095b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78d2fe700 1 -- 172.21.15.140:0/1443413649 >> v1:172.21.15.140:6789/0 conn(0x7fc788143940 legacy=0x7fc788096000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78d2fe700 1 -- 172.21.15.140:0/1443413649 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78805fcf0 con 0x7fc7880b2550 2022-07-21T01:54:14.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78d2fe700 1 --2- 172.21.15.140:0/1443413649 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7880b2550 0x7fc7880ab900 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fc778002f70 tx=0x7fc778006df0).ready entity=mon.0 client_cookie=9c6fbb39651cf721 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc77ffff700 1 -- 172.21.15.140:0/1443413649 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc77800d4f0 con 0x7fc7880b2550 2022-07-21T01:54:14.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc77ffff700 1 -- 172.21.15.140:0/1443413649 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc778000e90 con 0x7fc7880b2550 2022-07-21T01:54:14.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc77ffff700 1 -- 172.21.15.140:0/1443413649 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc77800d840 con 0x7fc7880b2550 2022-07-21T01:54:14.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78f562700 1 -- 172.21.15.140:0/1443413649 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7880b2550 msgr2=0x7fc7880ab900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78f562700 1 --2- 172.21.15.140:0/1443413649 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7880b2550 0x7fc7880ab900 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7fc778002f70 tx=0x7fc778006df0).stop 2022-07-21T01:54:14.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.687+0000 7fc78f562700 1 -- 172.21.15.140:0/1443413649 shutdown_connections 2022-07-21T01:54:14.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 --2- 172.21.15.140:0/1443413649 >> v2:172.21.15.152:3300/0 conn(0x7fc7880ac650 0x7fc788095b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 --2- 172.21.15.140:0/1443413649 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7880b2550 0x7fc7880ab900 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 -- 172.21.15.140:0/1443413649 >> 172.21.15.140:0/1443413649 conn(0x7fc7880b36c0 msgr2=0x7fc7880b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:14.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 -- 172.21.15.140:0/1443413649 shutdown_connections 2022-07-21T01:54:14.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 -- 172.21.15.140:0/1443413649 wait complete. 2022-07-21T01:54:14.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 Processor -- start 2022-07-21T01:54:14.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 -- start start 2022-07-21T01:54:14.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7880ac650 0x7fc78809b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7880b2550 0x7fc788152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc788143940 0x7fc788152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78813f150 con 0x7fc7880ac650 2022-07-21T01:54:14.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc788099b90 con 0x7fc7880b2550 2022-07-21T01:54:14.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78f562700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78809b410 con 0x7fc788143940 2022-07-21T01:54:14.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78d2fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7880ac650 0x7fc78809b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78d2fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7880ac650 0x7fc78809b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36724/0 (socket says 172.21.15.140:36724) 2022-07-21T01:54:14.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78d2fe700 1 -- 172.21.15.140:0/3725624486 learned_addr learned my addr 172.21.15.140:0/3725624486 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:14.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78cafd700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7880b2550 0x7fc788152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78daff700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc788143940 0x7fc788152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78d2fe700 1 -- 172.21.15.140:0/3725624486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc788143940 msgr2=0x7fc788152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78d2fe700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc788143940 0x7fc788152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78d2fe700 1 -- 172.21.15.140:0/3725624486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7880b2550 msgr2=0x7fc788152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78d2fe700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7880b2550 0x7fc788152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.691+0000 7fc78d2fe700 1 -- 172.21.15.140:0/3725624486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77800b180 con 0x7fc7880ac650 2022-07-21T01:54:14.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc78d2fe700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7880ac650 0x7fc78809b8b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fc778002f70 tx=0x7fc778014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc77e7fc700 1 -- 172.21.15.140:0/3725624486 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc778012860 con 0x7fc7880ac650 2022-07-21T01:54:14.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc77e7fc700 1 -- 172.21.15.140:0/3725624486 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7780129c0 con 0x7fc7880ac650 2022-07-21T01:54:14.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc78f562700 1 -- 172.21.15.140:0/3725624486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7880b0160 con 0x7fc7880ac650 2022-07-21T01:54:14.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc77e7fc700 1 -- 172.21.15.140:0/3725624486 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc778012d20 con 0x7fc7880ac650 2022-07-21T01:54:14.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc78f562700 1 -- 172.21.15.140:0/3725624486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc788099910 con 0x7fc7880ac650 2022-07-21T01:54:14.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc78f562700 1 -- 172.21.15.140:0/3725624486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc788099d50 con 0x7fc7880ac650 2022-07-21T01:54:14.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc77e7fc700 1 -- 172.21.15.140:0/3725624486 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc77800f030 con 0x7fc7880ac650 2022-07-21T01:54:14.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc77e7fc700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc770053aa0 0x7fc770055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc77e7fc700 1 -- 172.21.15.140:0/3725624486 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc77806f120 con 0x7fc7880ac650 2022-07-21T01:54:14.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc78cafd700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc770053aa0 0x7fc770055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.695+0000 7fc78cafd700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc770053aa0 0x7fc770055f60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fc78400b000 tx=0x7fc784008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.703+0000 7fc77e7fc700 1 -- 172.21.15.140:0/3725624486 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc7780460f0 con 0x7fc7880ac650 2022-07-21T01:54:14.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 -- 172.21.15.140:0/3725624486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7880ab1c0 con 0x7fc7880ac650 2022-07-21T01:54:14.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc77e7fc700 1 -- 172.21.15.140:0/3725624486 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v73) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc778049710 con 0x7fc7880ac650 2022-07-21T01:54:14.819 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:54:14.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 -- 172.21.15.140:0/3725624486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc770053aa0 msgr2=0x7fc770055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc770053aa0 0x7fc770055f60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fc78400b000 tx=0x7fc784008040).stop 2022-07-21T01:54:14.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 -- 172.21.15.140:0/3725624486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7880ac650 msgr2=0x7fc78809b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7880ac650 0x7fc78809b8b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fc778002f70 tx=0x7fc778014000).stop 2022-07-21T01:54:14.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 -- 172.21.15.140:0/3725624486 shutdown_connections 2022-07-21T01:54:14.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc770053aa0 0x7fc770055f60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc788143940 0x7fc788152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7880b2550 0x7fc788152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 --2- 172.21.15.140:0/3725624486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7880ac650 0x7fc78809b8b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 -- 172.21.15.140:0/3725624486 >> 172.21.15.140:0/3725624486 conn(0x7fc7880b36c0 msgr2=0x7fc78813fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:14.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 -- 172.21.15.140:0/3725624486 shutdown_connections 2022-07-21T01:54:14.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.823+0000 7fc78f562700 1 -- 172.21.15.140:0/3725624486 wait complete. 2022-07-21T01:54:14.832 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:54:14.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9b59e700 1 Processor -- start 2022-07-21T01:54:14.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9b59e700 1 -- start start 2022-07-21T01:54:14.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9b59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1a9c142ac0 0x7f1a9c142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9b59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1a9c0accf0 0x7f1a9c0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a9c0b08a0 0x7f1a9c145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9b59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9c143430 con 0x7f1a9c142ac0 2022-07-21T01:54:14.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9b59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a9c0b0d10 con 0x7f1a9c0accf0 2022-07-21T01:54:14.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c0aef20 con 0x7f1a9c0b08a0 2022-07-21T01:54:14.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9a59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1a9c142ac0 0x7f1a9c142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9a59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1a9c142ac0 0x7f1a9c142ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36732/0 (socket says 172.21.15.140:36732) 2022-07-21T01:54:14.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9a59c700 1 -- 172.21.15.140:0/3307638518 learned_addr learned my addr 172.21.15.140:0/3307638518 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:14.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a99d9b700 1 --2- 172.21.15.140:0/3307638518 >> v2:172.21.15.152:3300/0 conn(0x7f1a9c0accf0 0x7f1a9c0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.947+0000 7f1a9ad9d700 1 --2- 172.21.15.140:0/3307638518 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a9c0b08a0 0x7f1a9c145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9a59c700 1 -- 172.21.15.140:0/3307638518 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a9c0b08a0 msgr2=0x7f1a9c145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:14.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9a59c700 1 --2- 172.21.15.140:0/3307638518 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a9c0b08a0 0x7f1a9c145ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9a59c700 1 -- 172.21.15.140:0/3307638518 >> v2:172.21.15.152:3300/0 conn(0x7f1a9c0accf0 msgr2=0x7f1a9c0b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:14.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9a59c700 1 --2- 172.21.15.140:0/3307638518 >> v2:172.21.15.152:3300/0 conn(0x7f1a9c0accf0 0x7f1a9c0b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9a59c700 1 -- 172.21.15.140:0/3307638518 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c143660 con 0x7f1a9c142ac0 2022-07-21T01:54:14.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9a59c700 1 --2- 172.21.15.140:0/3307638518 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a9c142ac0 0x7f1a9c142ea0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f1a90002f70 tx=0x7f1a90006eb0).ready entity=mon.0 client_cookie=bf353a9f826561ed server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9959a700 1 -- 172.21.15.140:0/3307638518 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a9000d570 con 0x7f1a9c142ac0 2022-07-21T01:54:14.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9959a700 1 -- 172.21.15.140:0/3307638518 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a90000e90 con 0x7f1a9c142ac0 2022-07-21T01:54:14.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9959a700 1 -- 172.21.15.140:0/3307638518 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a9000d900 con 0x7f1a9c142ac0 2022-07-21T01:54:14.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 -- 172.21.15.140:0/3307638518 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a9c142ac0 msgr2=0x7f1a9c142ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 --2- 172.21.15.140:0/3307638518 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a9c142ac0 0x7f1a9c142ea0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f1a90002f70 tx=0x7f1a90006eb0).stop 2022-07-21T01:54:14.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 -- 172.21.15.140:0/3307638518 shutdown_connections 2022-07-21T01:54:14.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 --2- 172.21.15.140:0/3307638518 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a9c0b08a0 0x7f1a9c145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 --2- 172.21.15.140:0/3307638518 >> v2:172.21.15.152:3300/0 conn(0x7f1a9c0accf0 0x7f1a9c0b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 --2- 172.21.15.140:0/3307638518 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a9c142ac0 0x7f1a9c142ea0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 -- 172.21.15.140:0/3307638518 >> 172.21.15.140:0/3307638518 conn(0x7f1a9c0b93c0 msgr2=0x7f1a9c0b97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:14.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 -- 172.21.15.140:0/3307638518 shutdown_connections 2022-07-21T01:54:14.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 -- 172.21.15.140:0/3307638518 wait complete. 2022-07-21T01:54:14.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 Processor -- start 2022-07-21T01:54:14.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 -- start start 2022-07-21T01:54:14.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a9c0accf0 0x7f1a9c13b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a9c0b08a0 0x7f1a9c152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a9c142ac0 0x7f1a9c152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c151cd0 con 0x7f1a9c142ac0 2022-07-21T01:54:14.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c144370 con 0x7f1a9c0b08a0 2022-07-21T01:54:14.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9c155ed0 con 0x7f1a9c0accf0 2022-07-21T01:54:14.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a9c142ac0 0x7f1a9c152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a9c142ac0 0x7f1a9c152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36742/0 (socket says 172.21.15.140:36742) 2022-07-21T01:54:14.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.951+0000 7f1a9ad9d700 1 -- 172.21.15.140:0/2213687026 learned_addr learned my addr 172.21.15.140:0/2213687026 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:14.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a9a59c700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a9c0accf0 0x7f1a9c13b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a99d9b700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a9c0b08a0 0x7f1a9c152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a9ad9d700 1 -- 172.21.15.140:0/2213687026 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a9c0accf0 msgr2=0x7f1a9c13b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a9ad9d700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a9c0accf0 0x7f1a9c13b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a9ad9d700 1 -- 172.21.15.140:0/2213687026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a9c0b08a0 msgr2=0x7f1a9c152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:14.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a9ad9d700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a9c0b08a0 0x7f1a9c152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:14.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a9ad9d700 1 -- 172.21.15.140:0/2213687026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9000b1e0 con 0x7f1a9c142ac0 2022-07-21T01:54:14.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a9ad9d700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a9c142ac0 0x7f1a9c152850 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f1a94006e00 tx=0x7f1a940091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a8b7fe700 1 -- 172.21.15.140:0/2213687026 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a9400eeb0 con 0x7f1a9c142ac0 2022-07-21T01:54:14.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a8b7fe700 1 -- 172.21.15.140:0/2213687026 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9400c850 con 0x7f1a9c142ac0 2022-07-21T01:54:14.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a9b59e700 1 -- 172.21.15.140:0/2213687026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9c151ec0 con 0x7f1a9c142ac0 2022-07-21T01:54:14.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a8b7fe700 1 -- 172.21.15.140:0/2213687026 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a9400cbb0 con 0x7f1a9c142ac0 2022-07-21T01:54:14.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a9b59e700 1 -- 172.21.15.140:0/2213687026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a9c152d40 con 0x7f1a9c142ac0 2022-07-21T01:54:14.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a9b59e700 1 -- 172.21.15.140:0/2213687026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a9c1560d0 con 0x7f1a9c142ac0 2022-07-21T01:54:14.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.955+0000 7f1a8b7fe700 1 -- 172.21.15.140:0/2213687026 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1a94022070 con 0x7f1a9c142ac0 2022-07-21T01:54:14.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.959+0000 7f1a8b7fe700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a78053950 0x7f1a78055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:14.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.959+0000 7f1a9a59c700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a78053950 0x7f1a78055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:14.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.959+0000 7f1a8b7fe700 1 -- 172.21.15.140:0/2213687026 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1a9406c640 con 0x7f1a9c142ac0 2022-07-21T01:54:14.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.959+0000 7f1a9a59c700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a78053950 0x7f1a78055e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f1a9001c000 tx=0x7f1a90015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:14.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:14.963+0000 7f1a8b7fe700 1 -- 172.21.15.140:0/2213687026 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1a94043450 con 0x7f1a9c142ac0 2022-07-21T01:54:15.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.087+0000 7f1a9b59e700 1 -- 172.21.15.140:0/2213687026 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a9c0b0f40 con 0x7f1a9c142ac0 2022-07-21T01:54:15.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.087+0000 7f1a8b7fe700 1 -- 172.21.15.140:0/2213687026 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1a94046a70 con 0x7f1a9c142ac0 2022-07-21T01:54:15.086 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:54:15.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 -- 172.21.15.140:0/2213687026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a78053950 msgr2=0x7f1a78055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a78053950 0x7f1a78055e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f1a9001c000 tx=0x7f1a90015000).stop 2022-07-21T01:54:15.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 -- 172.21.15.140:0/2213687026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a9c142ac0 msgr2=0x7f1a9c152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a9c142ac0 0x7f1a9c152850 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f1a94006e00 tx=0x7f1a940091d0).stop 2022-07-21T01:54:15.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 -- 172.21.15.140:0/2213687026 shutdown_connections 2022-07-21T01:54:15.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1a78053950 0x7f1a78055e10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1a9c142ac0 0x7f1a9c152850 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a9c0b08a0 0x7f1a9c152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 --2- 172.21.15.140:0/2213687026 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1a9c0accf0 0x7f1a9c13b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 -- 172.21.15.140:0/2213687026 >> 172.21.15.140:0/2213687026 conn(0x7f1a9c0b93c0 msgr2=0x7f1a9c0ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:15.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 -- 172.21.15.140:0/2213687026 shutdown_connections 2022-07-21T01:54:15.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.091+0000 7f1a9b59e700 1 -- 172.21.15.140:0/2213687026 wait complete. 2022-07-21T01:54:15.098 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:54:15.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83ecaa9700 1 Processor -- start 2022-07-21T01:54:15.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83ecaa9700 1 -- start start 2022-07-21T01:54:15.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83ecaa9700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f83e80b2ec0 0x7f83e80b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83ecaa9700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f83e805d160 0x7f83e8142210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83ecaa9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83e8142700 0x7f83e81461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83ecaa9700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e8142b70 con 0x7f83e805d160 2022-07-21T01:54:15.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83ecaa9700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e80b4ea0 con 0x7f83e80b2ec0 2022-07-21T01:54:15.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83ecaa9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80b3f10 con 0x7f83e8142700 2022-07-21T01:54:15.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83e6ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f83e805d160 0x7f83e8142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83e6ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f83e805d160 0x7f83e8142210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36748/0 (socket says 172.21.15.140:36748) 2022-07-21T01:54:15.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83e6ffd700 1 -- 172.21.15.140:0/862769662 learned_addr learned my addr 172.21.15.140:0/862769662 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:15.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83e7fff700 1 --2- 172.21.15.140:0/862769662 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83e8142700 0x7f83e81461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.211+0000 7f83e77fe700 1 --2- 172.21.15.140:0/862769662 >> v2:172.21.15.152:3300/0 conn(0x7f83e80b2ec0 0x7f83e80b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e6ffd700 1 -- 172.21.15.140:0/862769662 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83e8142700 msgr2=0x7f83e81461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:15.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e6ffd700 1 --2- 172.21.15.140:0/862769662 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83e8142700 0x7f83e81461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e6ffd700 1 -- 172.21.15.140:0/862769662 >> v2:172.21.15.152:3300/0 conn(0x7f83e80b2ec0 msgr2=0x7f83e80b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:15.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e6ffd700 1 --2- 172.21.15.140:0/862769662 >> v2:172.21.15.152:3300/0 conn(0x7f83e80b2ec0 0x7f83e80b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e6ffd700 1 -- 172.21.15.140:0/862769662 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80b3790 con 0x7f83e805d160 2022-07-21T01:54:15.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e6ffd700 1 --2- 172.21.15.140:0/862769662 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83e805d160 0x7f83e8142210 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f83e80b4fe0 tx=0x7f83d8006eb0).ready entity=mon.0 client_cookie=8faea7d2685f9946 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:15.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e67fc700 1 -- 172.21.15.140:0/862769662 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83d800e610 con 0x7f83e805d160 2022-07-21T01:54:15.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e67fc700 1 -- 172.21.15.140:0/862769662 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d8000e90 con 0x7f83e805d160 2022-07-21T01:54:15.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e67fc700 1 -- 172.21.15.140:0/862769662 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83d800e9a0 con 0x7f83e805d160 2022-07-21T01:54:15.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 -- 172.21.15.140:0/862769662 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83e805d160 msgr2=0x7f83e8142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 --2- 172.21.15.140:0/862769662 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83e805d160 0x7f83e8142210 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f83e80b4fe0 tx=0x7f83d8006eb0).stop 2022-07-21T01:54:15.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 -- 172.21.15.140:0/862769662 shutdown_connections 2022-07-21T01:54:15.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 --2- 172.21.15.140:0/862769662 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83e8142700 0x7f83e81461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 --2- 172.21.15.140:0/862769662 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83e805d160 0x7f83e8142210 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 --2- 172.21.15.140:0/862769662 >> v2:172.21.15.152:3300/0 conn(0x7f83e80b2ec0 0x7f83e80b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 -- 172.21.15.140:0/862769662 >> 172.21.15.140:0/862769662 conn(0x7f83e8066e20 msgr2=0x7f83e8067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:15.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 -- 172.21.15.140:0/862769662 shutdown_connections 2022-07-21T01:54:15.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 -- 172.21.15.140:0/862769662 wait complete. 2022-07-21T01:54:15.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 Processor -- start 2022-07-21T01:54:15.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 -- start start 2022-07-21T01:54:15.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e805d160 0x7f83e8112660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83e80b2ec0 0x7f83e8152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83e8142700 0x7f83e8152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e813c660 con 0x7f83e80b2ec0 2022-07-21T01:54:15.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e81531d0 con 0x7f83e805d160 2022-07-21T01:54:15.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83ecaa9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8152f30 con 0x7f83e8142700 2022-07-21T01:54:15.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83e80b2ec0 0x7f83e8152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e6ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83e80b2ec0 0x7f83e8152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36754/0 (socket says 172.21.15.140:36754) 2022-07-21T01:54:15.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e6ffd700 1 -- 172.21.15.140:0/4132615972 learned_addr learned my addr 172.21.15.140:0/4132615972 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:15.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e77fe700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e805d160 0x7f83e8112660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.215+0000 7f83e7fff700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83e8142700 0x7f83e8152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83e6ffd700 1 -- 172.21.15.140:0/4132615972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83e8142700 msgr2=0x7f83e8152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83e6ffd700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83e8142700 0x7f83e8152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83e6ffd700 1 -- 172.21.15.140:0/4132615972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e805d160 msgr2=0x7f83e8112660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83e6ffd700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e805d160 0x7f83e8112660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83e6ffd700 1 -- 172.21.15.140:0/4132615972 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d800b1e0 con 0x7f83e80b2ec0 2022-07-21T01:54:15.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83e77fe700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e805d160 0x7f83e8112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:15.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83e6ffd700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83e80b2ec0 0x7f83e8152230 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f83e80b4fe0 tx=0x7f83d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:15.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83e4ff9700 1 -- 172.21.15.140:0/4132615972 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83d8021070 con 0x7f83e80b2ec0 2022-07-21T01:54:15.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83ecaa9700 1 -- 172.21.15.140:0/4132615972 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e8146860 con 0x7f83e80b2ec0 2022-07-21T01:54:15.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83e4ff9700 1 -- 172.21.15.140:0/4132615972 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d80116e0 con 0x7f83e80b2ec0 2022-07-21T01:54:15.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83e4ff9700 1 -- 172.21.15.140:0/4132615972 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83d8011a40 con 0x7f83e80b2ec0 2022-07-21T01:54:15.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83ecaa9700 1 -- 172.21.15.140:0/4132615972 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e81533b0 con 0x7f83e80b2ec0 2022-07-21T01:54:15.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83ecaa9700 1 -- 172.21.15.140:0/4132615972 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e81469f0 con 0x7f83e80b2ec0 2022-07-21T01:54:15.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.219+0000 7f83e4ff9700 1 -- 172.21.15.140:0/4132615972 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f83d800f070 con 0x7f83e80b2ec0 2022-07-21T01:54:15.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.223+0000 7f83e4ff9700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83c8053aa0 0x7f83c8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.223+0000 7f83e4ff9700 1 -- 172.21.15.140:0/4132615972 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f83d806d0c0 con 0x7f83e80b2ec0 2022-07-21T01:54:15.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.223+0000 7f83e77fe700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83c8053aa0 0x7f83c8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.223+0000 7f83e77fe700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83c8053aa0 0x7f83c8055f60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f83dc00b000 tx=0x7f83dc006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:15.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.227+0000 7f83e4ff9700 1 -- 172.21.15.140:0/4132615972 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f83d8043f40 con 0x7f83e80b2ec0 2022-07-21T01:54:15.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.351+0000 7f83ecaa9700 1 -- 172.21.15.140:0/4132615972 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f83e80b2620 con 0x7f83e80b2ec0 2022-07-21T01:54:15.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83e4ff9700 1 -- 172.21.15.140:0/4132615972 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+10 (secure 0 0 0) 0x7f83d8047560 con 0x7f83e80b2ec0 2022-07-21T01:54:15.351 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:54:15.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 -- 172.21.15.140:0/4132615972 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83c8053aa0 msgr2=0x7f83c8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83c8053aa0 0x7f83c8055f60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f83dc00b000 tx=0x7f83dc006df0).stop 2022-07-21T01:54:15.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 -- 172.21.15.140:0/4132615972 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83e80b2ec0 msgr2=0x7f83e8152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83e80b2ec0 0x7f83e8152230 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f83e80b4fe0 tx=0x7f83d8015000).stop 2022-07-21T01:54:15.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 -- 172.21.15.140:0/4132615972 shutdown_connections 2022-07-21T01:54:15.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83c8053aa0 0x7f83c8055f60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83e8142700 0x7f83e8152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83e80b2ec0 0x7f83e8152230 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 --2- 172.21.15.140:0/4132615972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83e805d160 0x7f83e8112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 -- 172.21.15.140:0/4132615972 >> 172.21.15.140:0/4132615972 conn(0x7f83e8066e20 msgr2=0x7f83e805bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:15.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 -- 172.21.15.140:0/4132615972 shutdown_connections 2022-07-21T01:54:15.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.355+0000 7f83ecaa9700 1 -- 172.21.15.140:0/4132615972 wait complete. 2022-07-21T01:54:15.362 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:54:15.363 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:54:15.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8292c80700 1 Processor -- start 2022-07-21T01:54:15.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8292c80700 1 -- start start 2022-07-21T01:54:15.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8292c80700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f828c0b5650 0x7f828c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8292c80700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f828c0af8e0 0x7f828c05bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8292c80700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f828c147010 con 0x7f828c0adff0 2022-07-21T01:54:15.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8292c80700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f828c1472b0 con 0x7f828c0b5650 2022-07-21T01:54:15.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8292c80700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c146d40 con 0x7f828c0af8e0 2022-07-21T01:54:15.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8291c7e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f828c0b5650 0x7f828c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f829247f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f828c0af8e0 0x7f828c05bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8291c7e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f828c0b5650 0x7f828c0b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42500/0 (socket says 172.21.15.140:42500) 2022-07-21T01:54:15.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8291c7e700 1 -- 172.21.15.140:0/27016200 learned_addr learned my addr 172.21.15.140:0/27016200 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:15.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f829147d700 1 --1- 172.21.15.140:0/27016200 >> v1:172.21.15.152:6789/0 conn(0x7f828c0adff0 0x7f828c147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:52022/0 (socket says 172.21.15.140:52022) 2022-07-21T01:54:15.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8291c7e700 1 -- 172.21.15.140:0/27016200 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f828c0af8e0 msgr2=0x7f828c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:15.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8291c7e700 1 --2- 172.21.15.140:0/27016200 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f828c0af8e0 0x7f828c05bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8291c7e700 1 -- 172.21.15.140:0/27016200 >> v1:172.21.15.152:6789/0 conn(0x7f828c0adff0 legacy=0x7f828c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8291c7e700 1 -- 172.21.15.140:0/27016200 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c0acae0 con 0x7f828c0b5650 2022-07-21T01:54:15.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.467+0000 7f8291c7e700 1 --2- 172.21.15.140:0/27016200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c0b5650 0x7f828c0b5a30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f827c002f70 tx=0x7f827c006eb0).ready entity=mon.1 client_cookie=398d65dafdff40a3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:15.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8290c7c700 1 -- 172.21.15.140:0/27016200 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f827c00d570 con 0x7f828c0b5650 2022-07-21T01:54:15.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8290c7c700 1 -- 172.21.15.140:0/27016200 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c000e90 con 0x7f828c0b5650 2022-07-21T01:54:15.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8290c7c700 1 -- 172.21.15.140:0/27016200 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f827c00d900 con 0x7f828c0b5650 2022-07-21T01:54:15.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8292c80700 1 -- 172.21.15.140:0/27016200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c0b5650 msgr2=0x7f828c0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8292c80700 1 --2- 172.21.15.140:0/27016200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c0b5650 0x7f828c0b5a30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f827c002f70 tx=0x7f827c006eb0).stop 2022-07-21T01:54:15.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8292c80700 1 -- 172.21.15.140:0/27016200 shutdown_connections 2022-07-21T01:54:15.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8292c80700 1 --2- 172.21.15.140:0/27016200 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f828c0af8e0 0x7f828c05bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8292c80700 1 --2- 172.21.15.140:0/27016200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c0b5650 0x7f828c0b5a30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8292c80700 1 -- 172.21.15.140:0/27016200 >> 172.21.15.140:0/27016200 conn(0x7f828c0634e0 msgr2=0x7f828c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:15.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8292c80700 1 -- 172.21.15.140:0/27016200 shutdown_connections 2022-07-21T01:54:15.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8292c80700 1 -- 172.21.15.140:0/27016200 wait complete. 2022-07-21T01:54:15.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.471+0000 7f8292c80700 1 Processor -- start 2022-07-21T01:54:15.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8292c80700 1 -- start start 2022-07-21T01:54:15.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8292c80700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f828c0adff0 0x7f828c066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8292c80700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f828c0af8e0 0x7f828c1435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8292c80700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c0b5650 0x7f828c143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8292c80700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c111dd0 con 0x7f828c0adff0 2022-07-21T01:54:15.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8292c80700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c0671a0 con 0x7f828c0b5650 2022-07-21T01:54:15.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8292c80700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c143fb0 con 0x7f828c0af8e0 2022-07-21T01:54:15.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8291c7e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f828c0adff0 0x7f828c066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8291c7e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f828c0adff0 0x7f828c066cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36766/0 (socket says 172.21.15.140:36766) 2022-07-21T01:54:15.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8291c7e700 1 -- 172.21.15.140:0/3757838984 learned_addr learned my addr 172.21.15.140:0/3757838984 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:15.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f829247f700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c0b5650 0x7f828c143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f829147d700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f828c0af8e0 0x7f828c1435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8291c7e700 1 -- 172.21.15.140:0/3757838984 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f828c0af8e0 msgr2=0x7f828c1435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8291c7e700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f828c0af8e0 0x7f828c1435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8291c7e700 1 -- 172.21.15.140:0/3757838984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c0b5650 msgr2=0x7f828c143ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8291c7e700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c0b5650 0x7f828c143ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8291c7e700 1 -- 172.21.15.140:0/3757838984 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c00b1e0 con 0x7f828c0adff0 2022-07-21T01:54:15.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8291c7e700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f828c0adff0 0x7f828c066cb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f828c111f10 tx=0x7f827c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:15.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f828affd700 1 -- 172.21.15.140:0/3757838984 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f827c006b40 con 0x7f828c0adff0 2022-07-21T01:54:15.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8292c80700 1 -- 172.21.15.140:0/3757838984 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c0673e0 con 0x7f828c0adff0 2022-07-21T01:54:15.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f8292c80700 1 -- 172.21.15.140:0/3757838984 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828c111ac0 con 0x7f828c0adff0 2022-07-21T01:54:15.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f828affd700 1 -- 172.21.15.140:0/3757838984 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c00d8f0 con 0x7f828c0adff0 2022-07-21T01:54:15.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.475+0000 7f828affd700 1 -- 172.21.15.140:0/3757838984 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f827c011de0 con 0x7f828c0adff0 2022-07-21T01:54:15.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.479+0000 7f8292c80700 1 -- 172.21.15.140:0/3757838984 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8268003410 con 0x7f828c0adff0 2022-07-21T01:54:15.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.479+0000 7f828affd700 1 -- 172.21.15.140:0/3757838984 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f827c01a040 con 0x7f828c0adff0 2022-07-21T01:54:15.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.479+0000 7f828affd700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f826c053a70 0x7f826c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:15.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.479+0000 7f829147d700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f826c053a70 0x7f826c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:15.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.479+0000 7f828affd700 1 -- 172.21.15.140:0/3757838984 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f827c06ce70 con 0x7f828c0adff0 2022-07-21T01:54:15.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.479+0000 7f829147d700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f826c053a70 0x7f826c055f30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f828000b000 tx=0x7f8280007140).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:15.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.487+0000 7f828affd700 1 -- 172.21.15.140:0/3757838984 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f827c043c00 con 0x7f828c0adff0 2022-07-21T01:54:15.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.639+0000 7f8292c80700 1 -- 172.21.15.140:0/3757838984 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8268003c40 con 0x7f828c0adff0 2022-07-21T01:54:15.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.639+0000 7f828affd700 1 -- 172.21.15.140:0/3757838984 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f827c023090 con 0x7f828c0adff0 2022-07-21T01:54:15.638 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:15.638 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":30,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/4173658089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4173658089},{"type":"v1","addr":"172.21.15.152:6833","nonce":4173658089}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:54:10.830421+0000","modified":"2022-07-21T01:54:11.841578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6208":{"gid":6208,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/167323020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":167323020},{"type":"v1","addr":"172.21.15.140:6835","nonce":167323020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-07-21T01:54:15.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 -- 172.21.15.140:0/3757838984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f826c053a70 msgr2=0x7f826c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f826c053a70 0x7f826c055f30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f828000b000 tx=0x7f8280007140).stop 2022-07-21T01:54:15.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 -- 172.21.15.140:0/3757838984 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f828c0adff0 msgr2=0x7f828c066cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:15.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f828c0adff0 0x7f828c066cb0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f828c111f10 tx=0x7f827c015000).stop 2022-07-21T01:54:15.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 -- 172.21.15.140:0/3757838984 shutdown_connections 2022-07-21T01:54:15.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f826c053a70 0x7f826c055f30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f828c0b5650 0x7f828c143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f828c0af8e0 0x7f828c1435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 --2- 172.21.15.140:0/3757838984 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f828c0adff0 0x7f828c066cb0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:15.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 -- 172.21.15.140:0/3757838984 >> 172.21.15.140:0/3757838984 conn(0x7f828c0634e0 msgr2=0x7f828c1465f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:15.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 -- 172.21.15.140:0/3757838984 shutdown_connections 2022-07-21T01:54:15.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:15.643+0000 7f8292c80700 1 -- 172.21.15.140:0/3757838984 wait complete. 2022-07-21T01:54:15.646 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 30 2022-07-21T01:54:15.857 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:54:10.830421+0000', 'modified': '2022-07-21T01:54:11.841578+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6208}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6208': {'gid': 6208, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/167323020', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 167323020}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 167323020}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:54:15.858 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:54:15.859 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:54:15.975 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":6208,"want_state":"up:active","state":"up:active","rank_uptime":5.1269027579999999,"mdsmap_epoch":30,"osdmap_epoch":73,"osdmap_epoch_barrier":72,"uptime":9.1986783699999997} 2022-07-21T01:54:15.975 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 6208, "mdsmap_epoch": 30, "osdmap_epoch": 73, "osdmap_epoch_barrier": 72, "rank_uptime": 5.126902758, "state": "up:active", "uptime": 9.19867837, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:54:15.976 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:54:15.977 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:54:15.978 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:54:15.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:15.979 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:54:15.990 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:54:16.006 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:54:16.007 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:54:16.065 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:54:16.066 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:54:16.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:16.069 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:54:16.075 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:54:16.076 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:54:16.078 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:54:16.096 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:54:16.097 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:54:16.152 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:54:16.160+0000 7f5799b68080 -1 init, newargv = 0x555820e553d0 newargc=9 2022-07-21T01:54:16.152 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22775]: starting ceph client 2022-07-21T01:54:16.152 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:54:16.153 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:54:16.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:16.155 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:54:16.160 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22775]: starting fuse 2022-07-21T01:54:17.261 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:54:17.280 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:54:17.281 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:54:17.337 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:54:17.338 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:54:17.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:17.341 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:54:17.348 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:54:17.349 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:54:17.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:17.351 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:54:17.352 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:54:17.352 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:54:17.353 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:54:17.354 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:54:17.355 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:54:17.356 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:54:17.357 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:54:17.358 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:54:17.359 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:54:17.360 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:54:17.360 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:54:17.361 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:54:17.362 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:54:17.363 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:54:17.365 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:54:17.366 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:54:17.367 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:54:17.367 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:54:17.368 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:54:17.369 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:54:17.370 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:54:17.372 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:54:17.432 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.22775.asok 2022-07-21T01:54:17.437 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22775.asok 2022-07-21T01:54:17.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:17.439 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22775.asok status 2022-07-21T01:54:17.532 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:54:17.533 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:54:17.533 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:54:17.534 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:54:17.534 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:54:17.534 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:54:17.534 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:54:17.535 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "22775", 2022-07-21T01:54:17.535 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:54:17.535 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:54:17.535 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:54:17.536 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:54:17.536 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 6367, 2022-07-21T01:54:17.536 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:54:17.536 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:54:17.536 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:54:17.537 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 6367 2022-07-21T01:54:17.537 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:54:17.537 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:54:17.538 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:54:17.538 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:54:17.538 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1019294232 2022-07-21T01:54:17.538 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:54:17.539 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:54:17.539 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:54:17.539 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:54:17.539 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:54:17.540 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1019294232 2022-07-21T01:54:17.540 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:54:17.540 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.6367 172.21.15.191:0/1019294232", 2022-07-21T01:54:17.540 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/1019294232", 2022-07-21T01:54:17.541 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:54:17.541 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 30, 2022-07-21T01:54:17.541 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 73, 2022-07-21T01:54:17.541 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:54:17.542 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:54:17.542 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:54:17.542 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:54:17.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:17.544 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:54:17.545 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:54:17.552 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:54:17.553 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:54:17.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:17.555 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:54:17.617 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:54:17.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fade5703700 1 Processor -- start 2022-07-21T01:54:17.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fade5703700 1 -- start start 2022-07-21T01:54:17.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fade5703700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade005c3f0 0x7fade014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:17.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fade5703700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fade00acef0 0x7fade0146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:17.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fade5703700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade005d480 con 0x7fade00b1e00 2022-07-21T01:54:17.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fade5703700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade00b21e0 con 0x7fade00acef0 2022-07-21T01:54:17.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fade5703700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade005cf40 con 0x7fade005c3f0 2022-07-21T01:54:17.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fadde7fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade005c3f0 0x7fade014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:17.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7faddf7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fade00acef0 0x7fade0146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:17.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fadde7fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade005c3f0 0x7fade014a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:33504/0 (socket says 172.21.15.140:33504) 2022-07-21T01:54:17.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7faddeffd700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fade00b1e00 0x7fade00ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:52034/0 (socket says 172.21.15.140:52034) 2022-07-21T01:54:17.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7faddf7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fade00acef0 0x7fade0146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42518/0 (socket says 172.21.15.140:42518) 2022-07-21T01:54:17.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7faddeffd700 1 -- 172.21.15.140:0/2284580034 learned_addr learned my addr 172.21.15.140:0/2284580034 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:17.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7faddf7fe700 1 -- 172.21.15.140:0/2284580034 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade005c3f0 msgr2=0x7fade014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:17.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7faddf7fe700 1 --2- 172.21.15.140:0/2284580034 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade005c3f0 0x7fade014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:17.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7faddf7fe700 1 -- 172.21.15.140:0/2284580034 >> v1:172.21.15.152:6789/0 conn(0x7fade00b1e00 legacy=0x7fade00ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:17.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7faddf7fe700 1 -- 172.21.15.140:0/2284580034 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade005d1e0 con 0x7fade00acef0 2022-07-21T01:54:17.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fadde7fc700 1 --2- 172.21.15.140:0/2284580034 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade005c3f0 0x7fade014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:17.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7faddf7fe700 1 --2- 172.21.15.140:0/2284580034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fade00acef0 0x7fade0146410 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fadd80032f0 tx=0x7fadd80091d0).ready entity=mon.1 client_cookie=89e02dfd9257d98 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:17.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fadddffb700 1 -- 172.21.15.140:0/2284580034 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadd800b6c0 con 0x7fade00acef0 2022-07-21T01:54:17.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fadddffb700 1 -- 172.21.15.140:0/2284580034 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd800b820 con 0x7fade00acef0 2022-07-21T01:54:17.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.735+0000 7fadddffb700 1 -- 172.21.15.140:0/2284580034 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadd800b990 con 0x7fade00acef0 2022-07-21T01:54:17.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.739+0000 7fade5703700 1 -- 172.21.15.140:0/2284580034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fade00acef0 msgr2=0x7fade0146410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:17.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.739+0000 7fade5703700 1 --2- 172.21.15.140:0/2284580034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fade00acef0 0x7fade0146410 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fadd80032f0 tx=0x7fadd80091d0).stop 2022-07-21T01:54:17.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.739+0000 7fade5703700 1 -- 172.21.15.140:0/2284580034 shutdown_connections 2022-07-21T01:54:17.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.739+0000 7fade5703700 1 --2- 172.21.15.140:0/2284580034 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade005c3f0 0x7fade014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:17.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.739+0000 7fade5703700 1 --2- 172.21.15.140:0/2284580034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fade00acef0 0x7fade0146410 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:17.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.739+0000 7fade5703700 1 -- 172.21.15.140:0/2284580034 >> 172.21.15.140:0/2284580034 conn(0x7fade00611c0 msgr2=0x7fade0062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:17.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.739+0000 7fade5703700 1 -- 172.21.15.140:0/2284580034 shutdown_connections 2022-07-21T01:54:17.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.739+0000 7fade5703700 1 -- 172.21.15.140:0/2284580034 wait complete. 2022-07-21T01:54:17.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.739+0000 7fade5703700 1 Processor -- start 2022-07-21T01:54:17.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.739+0000 7fade5703700 1 -- start start 2022-07-21T01:54:17.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fade5703700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fade00acef0 0x7fade0159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:17.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fade5703700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fade00b1e00 0x7fade01570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:17.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fade5703700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade0064d30 0x7fade0142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:17.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fade5703700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0063cb0 con 0x7fade00acef0 2022-07-21T01:54:17.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fade5703700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade015a820 con 0x7fade00b1e00 2022-07-21T01:54:17.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fade5703700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade015aac0 con 0x7fade0064d30 2022-07-21T01:54:17.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7faddeffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fade00acef0 0x7fade0159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:17.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7faddeffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fade00acef0 0x7fade0159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36780/0 (socket says 172.21.15.140:36780) 2022-07-21T01:54:17.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7faddeffd700 1 -- 172.21.15.140:0/3491953126 learned_addr learned my addr 172.21.15.140:0/3491953126 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:17.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fadde7fc700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fade00b1e00 0x7fade01570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:17.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7faddf7fe700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade0064d30 0x7fade0142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:17.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7faddeffd700 1 -- 172.21.15.140:0/3491953126 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade0064d30 msgr2=0x7fade0142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:17.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7faddeffd700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade0064d30 0x7fade0142320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:17.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7faddeffd700 1 -- 172.21.15.140:0/3491953126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fade00b1e00 msgr2=0x7fade01570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:17.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7faddeffd700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fade00b1e00 0x7fade01570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:17.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7faddeffd700 1 -- 172.21.15.140:0/3491953126 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd400b180 con 0x7fade00acef0 2022-07-21T01:54:17.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7faddeffd700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fade00acef0 0x7fade0159f30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fade00ac6c0 tx=0x7fadd0007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:17.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fadcffff700 1 -- 172.21.15.140:0/3491953126 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadd00070d0 con 0x7fade00acef0 2022-07-21T01:54:17.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fadcffff700 1 -- 172.21.15.140:0/3491953126 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd0007230 con 0x7fade00acef0 2022-07-21T01:54:17.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fade5703700 1 -- 172.21.15.140:0/3491953126 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd8009470 con 0x7fade00acef0 2022-07-21T01:54:17.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fadcffff700 1 -- 172.21.15.140:0/3491953126 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fadd0005610 con 0x7fade00acef0 2022-07-21T01:54:17.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fade5703700 1 -- 172.21.15.140:0/3491953126 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade015a420 con 0x7fade00acef0 2022-07-21T01:54:17.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.743+0000 7fade5703700 1 -- 172.21.15.140:0/3491953126 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade0064300 con 0x7fade00acef0 2022-07-21T01:54:17.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.747+0000 7fadcffff700 1 -- 172.21.15.140:0/3491953126 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fadd0010070 con 0x7fade00acef0 2022-07-21T01:54:17.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.747+0000 7fadcffff700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fadbc0539e0 0x7fadbc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:17.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.747+0000 7fadcffff700 1 -- 172.21.15.140:0/3491953126 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fadd006b3f0 con 0x7fade00acef0 2022-07-21T01:54:17.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.747+0000 7fadde7fc700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fadbc0539e0 0x7fadbc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:17.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.747+0000 7fadde7fc700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fadbc0539e0 0x7fadbc055ea0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fadd400b000 tx=0x7fadd4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:17.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.751+0000 7fadcffff700 1 -- 172.21.15.140:0/3491953126 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fade0064300 con 0x7fade00acef0 2022-07-21T01:54:17.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.875+0000 7fade5703700 1 -- 172.21.15.140:0/3491953126 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fade0064300 con 0x7fade00acef0 2022-07-21T01:54:17.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fadcffff700 1 -- 172.21.15.140:0/3491953126 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16137 (secure 0 0 0) 0x7fade0064300 con 0x7fade00acef0 2022-07-21T01:54:17.874 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:17.875 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":73,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:54:11.862650+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:54:09.143212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-07-21T01:54:09.749431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:54:17.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 -- 172.21.15.140:0/3491953126 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fadbc0539e0 msgr2=0x7fadbc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:17.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fadbc0539e0 0x7fadbc055ea0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fadd400b000 tx=0x7fadd4006df0).stop 2022-07-21T01:54:17.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 -- 172.21.15.140:0/3491953126 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fade00acef0 msgr2=0x7fade0159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:17.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fade00acef0 0x7fade0159f30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fade00ac6c0 tx=0x7fadd0007ca0).stop 2022-07-21T01:54:17.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 -- 172.21.15.140:0/3491953126 shutdown_connections 2022-07-21T01:54:17.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fadbc0539e0 0x7fadbc055ea0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:17.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fade0064d30 0x7fade0142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:17.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fade00b1e00 0x7fade01570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:17.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 --2- 172.21.15.140:0/3491953126 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fade00acef0 0x7fade0159f30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:17.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 -- 172.21.15.140:0/3491953126 >> 172.21.15.140:0/3491953126 conn(0x7fade005c3f0 msgr2=0x7fade0143c40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:17.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 -- 172.21.15.140:0/3491953126 shutdown_connections 2022-07-21T01:54:17.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:17.879+0000 7fade5703700 1 -- 172.21.15.140:0/3491953126 wait complete. 2022-07-21T01:54:17.897 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:54:09.143212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:54:09.749431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:54:17.898 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:54:18.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda3a6d1700 1 Processor -- start 2022-07-21T01:54:18.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda3a6d1700 1 -- start start 2022-07-21T01:54:18.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda3a6d1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 0x7fda340b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda3a6d1700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fda340ab8a0 0x7fda3413fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda3a6d1700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3405f840 con 0x7fda340b1a00 2022-07-21T01:54:18.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda3a6d1700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda340acab0 con 0x7fda340ab8a0 2022-07-21T01:54:18.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda3a6d1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda34098a80 con 0x7fda34095880 2022-07-21T01:54:18.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda38ece700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 0x7fda340b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda39ed0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fda340ab8a0 0x7fda3413fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda39ed0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fda340ab8a0 0x7fda3413fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42532/0 (socket says 172.21.15.140:42532) 2022-07-21T01:54:18.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda39ed0700 1 -- 172.21.15.140:0/2628392576 learned_addr learned my addr 172.21.15.140:0/2628392576 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:18.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda38ece700 1 --2- 172.21.15.140:0/2628392576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 0x7fda340b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:33518/0 (socket says 172.21.15.140:33518) 2022-07-21T01:54:18.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda396cf700 1 --1- 172.21.15.140:0/2628392576 >> v1:172.21.15.152:6789/0 conn(0x7fda340b1a00 0x7fda340ab190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:52048/0 (socket says 172.21.15.140:52048) 2022-07-21T01:54:18.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda38ece700 1 -- 172.21.15.140:0/2628392576 >> v2:172.21.15.152:3300/0 conn(0x7fda340ab8a0 msgr2=0x7fda3413fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:18.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda38ece700 1 --2- 172.21.15.140:0/2628392576 >> v2:172.21.15.152:3300/0 conn(0x7fda340ab8a0 0x7fda3413fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda38ece700 1 -- 172.21.15.140:0/2628392576 >> v1:172.21.15.152:6789/0 conn(0x7fda340b1a00 legacy=0x7fda340ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda38ece700 1 -- 172.21.15.140:0/2628392576 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda34143870 con 0x7fda34095880 2022-07-21T01:54:18.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.015+0000 7fda38ece700 1 --2- 172.21.15.140:0/2628392576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 0x7fda340b1de0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fda30002f70 tx=0x7fda30006df0).ready entity=mon.2 client_cookie=b6ab80369031bbd1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda2bfff700 1 -- 172.21.15.140:0/2628392576 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda3000d570 con 0x7fda34095880 2022-07-21T01:54:18.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda2bfff700 1 -- 172.21.15.140:0/2628392576 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda30000e90 con 0x7fda34095880 2022-07-21T01:54:18.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda2bfff700 1 -- 172.21.15.140:0/2628392576 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda3000d8c0 con 0x7fda34095880 2022-07-21T01:54:18.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda3a6d1700 1 -- 172.21.15.140:0/2628392576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 msgr2=0x7fda340b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda3a6d1700 1 --2- 172.21.15.140:0/2628392576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 0x7fda340b1de0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fda30002f70 tx=0x7fda30006df0).stop 2022-07-21T01:54:18.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda3a6d1700 1 -- 172.21.15.140:0/2628392576 shutdown_connections 2022-07-21T01:54:18.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda3a6d1700 1 --2- 172.21.15.140:0/2628392576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 0x7fda340b1de0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda3a6d1700 1 --2- 172.21.15.140:0/2628392576 >> v2:172.21.15.152:3300/0 conn(0x7fda340ab8a0 0x7fda3413fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda3a6d1700 1 -- 172.21.15.140:0/2628392576 >> 172.21.15.140:0/2628392576 conn(0x7fda340b2af0 msgr2=0x7fda340b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:18.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda3a6d1700 1 -- 172.21.15.140:0/2628392576 shutdown_connections 2022-07-21T01:54:18.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda3a6d1700 1 -- 172.21.15.140:0/2628392576 wait complete. 2022-07-21T01:54:18.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda3a6d1700 1 Processor -- start 2022-07-21T01:54:18.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.019+0000 7fda3a6d1700 1 -- start start 2022-07-21T01:54:18.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda3a6d1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 0x7fda34139a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda3a6d1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda340ab8a0 0x7fda34099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda3a6d1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda340b1a00 0x7fda34095c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda3a6d1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340979d0 con 0x7fda340ab8a0 2022-07-21T01:54:18.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda3a6d1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda34097c00 con 0x7fda340b1a00 2022-07-21T01:54:18.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda3a6d1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda34099640 con 0x7fda34095880 2022-07-21T01:54:18.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda38ece700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda340ab8a0 0x7fda34099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda38ece700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda340ab8a0 0x7fda34099ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36796/0 (socket says 172.21.15.140:36796) 2022-07-21T01:54:18.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda38ece700 1 -- 172.21.15.140:0/4100477275 learned_addr learned my addr 172.21.15.140:0/4100477275 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:18.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda396cf700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 0x7fda34139a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda39ed0700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda340b1a00 0x7fda34095c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda38ece700 1 -- 172.21.15.140:0/4100477275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 msgr2=0x7fda34139a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda38ece700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 0x7fda34139a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda38ece700 1 -- 172.21.15.140:0/4100477275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda340b1a00 msgr2=0x7fda34095c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda38ece700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda340b1a00 0x7fda34095c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda38ece700 1 -- 172.21.15.140:0/4100477275 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3000b180 con 0x7fda340ab8a0 2022-07-21T01:54:18.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda38ece700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda340ab8a0 0x7fda34099ae0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fda30002f70 tx=0x7fda30014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda2a7fc700 1 -- 172.21.15.140:0/4100477275 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda30016070 con 0x7fda340ab8a0 2022-07-21T01:54:18.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda2a7fc700 1 -- 172.21.15.140:0/4100477275 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3000d8f0 con 0x7fda340ab8a0 2022-07-21T01:54:18.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda3a6d1700 1 -- 172.21.15.140:0/4100477275 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda34139f20 con 0x7fda340ab8a0 2022-07-21T01:54:18.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda2a7fc700 1 -- 172.21.15.140:0/4100477275 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda30013de0 con 0x7fda340ab8a0 2022-07-21T01:54:18.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda3a6d1700 1 -- 172.21.15.140:0/4100477275 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3409bf70 con 0x7fda340ab8a0 2022-07-21T01:54:18.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.023+0000 7fda3a6d1700 1 -- 172.21.15.140:0/4100477275 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda34099800 con 0x7fda340ab8a0 2022-07-21T01:54:18.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.027+0000 7fda2a7fc700 1 -- 172.21.15.140:0/4100477275 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fda3000f030 con 0x7fda340ab8a0 2022-07-21T01:54:18.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.027+0000 7fda2a7fc700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda1c053aa0 0x7fda1c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.027+0000 7fda2a7fc700 1 -- 172.21.15.140:0/4100477275 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fda3006cc20 con 0x7fda340ab8a0 2022-07-21T01:54:18.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.027+0000 7fda396cf700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda1c053aa0 0x7fda1c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.027+0000 7fda396cf700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda1c053aa0 0x7fda1c055f60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fda2400d000 tx=0x7fda24007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.031+0000 7fda2a7fc700 1 -- 172.21.15.140:0/4100477275 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fda30043aa0 con 0x7fda340ab8a0 2022-07-21T01:54:18.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.159+0000 7fda3a6d1700 1 -- 172.21.15.140:0/4100477275 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fda340ac3e0 con 0x7fda340ab8a0 2022-07-21T01:54:18.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.159+0000 7fda2a7fc700 1 -- 172.21.15.140:0/4100477275 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v73) v1 ==== 110+0+10 (secure 0 0 0) 0x7fda30023020 con 0x7fda340ab8a0 2022-07-21T01:54:18.154 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:54:18.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 -- 172.21.15.140:0/4100477275 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda1c053aa0 msgr2=0x7fda1c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda1c053aa0 0x7fda1c055f60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fda2400d000 tx=0x7fda24007b90).stop 2022-07-21T01:54:18.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 -- 172.21.15.140:0/4100477275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda340ab8a0 msgr2=0x7fda34099ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda340ab8a0 0x7fda34099ae0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fda30002f70 tx=0x7fda30014000).stop 2022-07-21T01:54:18.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 -- 172.21.15.140:0/4100477275 shutdown_connections 2022-07-21T01:54:18.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda1c053aa0 0x7fda1c055f60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda340b1a00 0x7fda34095c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda340ab8a0 0x7fda34099ae0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 --2- 172.21.15.140:0/4100477275 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda34095880 0x7fda34139a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 -- 172.21.15.140:0/4100477275 >> 172.21.15.140:0/4100477275 conn(0x7fda340b2af0 msgr2=0x7fda340b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:18.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 -- 172.21.15.140:0/4100477275 shutdown_connections 2022-07-21T01:54:18.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.163+0000 7fda3a6d1700 1 -- 172.21.15.140:0/4100477275 wait complete. 2022-07-21T01:54:18.174 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:54:18.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8fc50700 1 Processor -- start 2022-07-21T01:54:18.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8fc50700 1 -- start start 2022-07-21T01:54:18.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8fc50700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbc880ae8d0 0x7fbc880af960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8fc50700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc8813f720 0x7fbc8813fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8fc50700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc880b2a10 con 0x7fbc880b2630 2022-07-21T01:54:18.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8fc50700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc880afff0 con 0x7fbc880ae8d0 2022-07-21T01:54:18.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8fc50700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880b0290 con 0x7fbc8813f720 2022-07-21T01:54:18.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8d9ec700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fbc880b2630 0x7fbc880ae1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50948/0 (socket says 172.21.15.140:50948) 2022-07-21T01:54:18.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8d9ec700 1 -- 172.21.15.140:0/4285548539 learned_addr learned my addr 172.21.15.140:0/4285548539 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:18.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8e1ed700 1 --2- 172.21.15.140:0/4285548539 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc8813f720 0x7fbc8813fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8d1eb700 1 --2- 172.21.15.140:0/4285548539 >> v2:172.21.15.152:3300/0 conn(0x7fbc880ae8d0 0x7fbc880af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8c9ea700 1 -- 172.21.15.140:0/4285548539 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865040779 0 0) 0x7fbc880b2a10 con 0x7fbc880b2630 2022-07-21T01:54:18.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8c9ea700 1 -- 172.21.15.140:0/4285548539 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc70003260 con 0x7fbc880b2630 2022-07-21T01:54:18.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8d1eb700 1 -- 172.21.15.140:0/4285548539 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc8813f720 msgr2=0x7fbc8813fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:18.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8d1eb700 1 --2- 172.21.15.140:0/4285548539 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc8813f720 0x7fbc8813fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8d1eb700 1 -- 172.21.15.140:0/4285548539 >> v1:172.21.15.140:6789/0 conn(0x7fbc880b2630 legacy=0x7fbc880ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8d1eb700 1 -- 172.21.15.140:0/4285548539 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880af220 con 0x7fbc880ae8d0 2022-07-21T01:54:18.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.287+0000 7fbc8e1ed700 1 --2- 172.21.15.140:0/4285548539 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc8813f720 0x7fbc8813fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:18.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8d1eb700 1 --2- 172.21.15.140:0/4285548539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc880ae8d0 0x7fbc880af960 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fbc84002f70 tx=0x7fbc84006df0).ready entity=mon.1 client_cookie=200c4211095a7016 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8c9ea700 1 -- 172.21.15.140:0/4285548539 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc8400d4a0 con 0x7fbc880ae8d0 2022-07-21T01:54:18.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8c9ea700 1 -- 172.21.15.140:0/4285548539 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc84000e90 con 0x7fbc880ae8d0 2022-07-21T01:54:18.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8c9ea700 1 -- 172.21.15.140:0/4285548539 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc8400d7f0 con 0x7fbc880ae8d0 2022-07-21T01:54:18.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8fc50700 1 -- 172.21.15.140:0/4285548539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc880ae8d0 msgr2=0x7fbc880af960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8fc50700 1 --2- 172.21.15.140:0/4285548539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc880ae8d0 0x7fbc880af960 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fbc84002f70 tx=0x7fbc84006df0).stop 2022-07-21T01:54:18.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8fc50700 1 -- 172.21.15.140:0/4285548539 shutdown_connections 2022-07-21T01:54:18.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8fc50700 1 --2- 172.21.15.140:0/4285548539 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc8813f720 0x7fbc8813fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8fc50700 1 --2- 172.21.15.140:0/4285548539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc880ae8d0 0x7fbc880af960 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8fc50700 1 -- 172.21.15.140:0/4285548539 >> 172.21.15.140:0/4285548539 conn(0x7fbc88062fc0 msgr2=0x7fbc880633a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:18.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8fc50700 1 -- 172.21.15.140:0/4285548539 shutdown_connections 2022-07-21T01:54:18.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8fc50700 1 -- 172.21.15.140:0/4285548539 wait complete. 2022-07-21T01:54:18.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8fc50700 1 Processor -- start 2022-07-21T01:54:18.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.291+0000 7fbc8fc50700 1 -- start start 2022-07-21T01:54:18.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8fc50700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc880ae8d0 0x7fbc8813d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8fc50700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc880b2630 0x7fbc88065780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8fc50700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc8813f720 0x7fbc88065c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8fc50700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8813e9c0 con 0x7fbc880ae8d0 2022-07-21T01:54:18.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8fc50700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88064920 con 0x7fbc880b2630 2022-07-21T01:54:18.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8fc50700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8813e040 con 0x7fbc8813f720 2022-07-21T01:54:18.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8d9ec700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc880ae8d0 0x7fbc8813d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8d9ec700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc880ae8d0 0x7fbc8813d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36810/0 (socket says 172.21.15.140:36810) 2022-07-21T01:54:18.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8e1ed700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc8813f720 0x7fbc88065c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8d9ec700 1 -- 172.21.15.140:0/557205634 learned_addr learned my addr 172.21.15.140:0/557205634 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:18.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8d1eb700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc880b2630 0x7fbc88065780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8d9ec700 1 -- 172.21.15.140:0/557205634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc8813f720 msgr2=0x7fbc88065c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8d9ec700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc8813f720 0x7fbc88065c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8d9ec700 1 -- 172.21.15.140:0/557205634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc880b2630 msgr2=0x7fbc88065780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8d9ec700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc880b2630 0x7fbc88065780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8d9ec700 1 -- 172.21.15.140:0/557205634 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc80009470 con 0x7fbc880ae8d0 2022-07-21T01:54:18.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8d9ec700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc880ae8d0 0x7fbc8813d140 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fbc78001110 tx=0x7fbc78007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc76ffd700 1 -- 172.21.15.140:0/557205634 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc78007110 con 0x7fbc880ae8d0 2022-07-21T01:54:18.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc76ffd700 1 -- 172.21.15.140:0/557205634 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc78007270 con 0x7fbc880ae8d0 2022-07-21T01:54:18.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8fc50700 1 -- 172.21.15.140:0/557205634 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8400b180 con 0x7fbc880ae8d0 2022-07-21T01:54:18.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc76ffd700 1 -- 172.21.15.140:0/557205634 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc78005030 con 0x7fbc880ae8d0 2022-07-21T01:54:18.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8fc50700 1 -- 172.21.15.140:0/557205634 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8813e580 con 0x7fbc880ae8d0 2022-07-21T01:54:18.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc76ffd700 1 -- 172.21.15.140:0/557205634 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbc7801b070 con 0x7fbc880ae8d0 2022-07-21T01:54:18.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.295+0000 7fbc8fc50700 1 -- 172.21.15.140:0/557205634 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc880af4c0 con 0x7fbc880ae8d0 2022-07-21T01:54:18.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.299+0000 7fbc76ffd700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc68053a70 0x7fbc68055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.299+0000 7fbc76ffd700 1 -- 172.21.15.140:0/557205634 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbc7806b520 con 0x7fbc880ae8d0 2022-07-21T01:54:18.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.299+0000 7fbc8d1eb700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc68053a70 0x7fbc68055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.299+0000 7fbc8d1eb700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc68053a70 0x7fbc68055f30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fbc8401a000 tx=0x7fbc84014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.299+0000 7fbc76ffd700 1 -- 172.21.15.140:0/557205634 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbc780423c0 con 0x7fbc880ae8d0 2022-07-21T01:54:18.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.427+0000 7fbc8fc50700 1 -- 172.21.15.140:0/557205634 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbc880574e0 con 0x7fbc880ae8d0 2022-07-21T01:54:18.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.427+0000 7fbc76ffd700 1 -- 172.21.15.140:0/557205634 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v73) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbc780459e0 con 0x7fbc880ae8d0 2022-07-21T01:54:18.424 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:54:18.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 -- 172.21.15.140:0/557205634 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc68053a70 msgr2=0x7fbc68055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc68053a70 0x7fbc68055f30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7fbc8401a000 tx=0x7fbc84014000).stop 2022-07-21T01:54:18.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 -- 172.21.15.140:0/557205634 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc880ae8d0 msgr2=0x7fbc8813d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc880ae8d0 0x7fbc8813d140 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fbc78001110 tx=0x7fbc78007a80).stop 2022-07-21T01:54:18.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 -- 172.21.15.140:0/557205634 shutdown_connections 2022-07-21T01:54:18.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc68053a70 0x7fbc68055f30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc8813f720 0x7fbc88065c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc880b2630 0x7fbc88065780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 --2- 172.21.15.140:0/557205634 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc880ae8d0 0x7fbc8813d140 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 -- 172.21.15.140:0/557205634 >> 172.21.15.140:0/557205634 conn(0x7fbc88062fc0 msgr2=0x7fbc88138fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:18.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 -- 172.21.15.140:0/557205634 shutdown_connections 2022-07-21T01:54:18.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.431+0000 7fbc8fc50700 1 -- 172.21.15.140:0/557205634 wait complete. 2022-07-21T01:54:18.442 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:54:18.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.551+0000 7fecb53cb700 1 Processor -- start 2022-07-21T01:54:18.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.551+0000 7fecb53cb700 1 -- start start 2022-07-21T01:54:18.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecb00b59f0 0x7fecb005c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fecb00addd0 0x7fecb0142d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb005d2f0 con 0x7fecb00af940 2022-07-21T01:54:18.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb005ccd0 con 0x7fecb00addd0 2022-07-21T01:54:18.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb0146350 con 0x7fecb00b59f0 2022-07-21T01:54:18.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7feca7fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fecb00af940 0x7fecb005c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50962/0 (socket says 172.21.15.140:50962) 2022-07-21T01:54:18.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7feca7fff700 1 -- 172.21.15.140:0/1235744254 learned_addr learned my addr 172.21.15.140:0/1235744254 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:18.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb4bca700 1 --2- 172.21.15.140:0/1235744254 >> v2:172.21.15.140:3300/0 conn(0x7fecb00addd0 0x7fecb0142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecaf7fe700 1 -- 172.21.15.140:0/1235744254 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225804229 0 0) 0x7fecb005d2f0 con 0x7fecb00af940 2022-07-21T01:54:18.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecaf7fe700 1 -- 172.21.15.140:0/1235744254 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec90003260 con 0x7fecb00af940 2022-07-21T01:54:18.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecaffff700 1 --2- 172.21.15.140:0/1235744254 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecb00b59f0 0x7fecb005c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb4bca700 1 -- 172.21.15.140:0/1235744254 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecb00b59f0 msgr2=0x7fecb005c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:18.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb4bca700 1 --2- 172.21.15.140:0/1235744254 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecb00b59f0 0x7fecb005c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecaf7fe700 1 -- 172.21.15.140:0/1235744254 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1365507162 0 0) 0x7fec90003260 con 0x7fecb00af940 2022-07-21T01:54:18.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb4bca700 1 -- 172.21.15.140:0/1235744254 >> v1:172.21.15.140:6789/0 conn(0x7fecb00af940 legacy=0x7fecb005c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb4bca700 1 -- 172.21.15.140:0/1235744254 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00b5dd0 con 0x7fecb00addd0 2022-07-21T01:54:18.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb4bca700 1 --2- 172.21.15.140:0/1235744254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecb00addd0 0x7fecb0142d00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7fec98001800 tx=0x7feca80091d0).ready entity=mon.0 client_cookie=36415ff5a301186a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecaf7fe700 1 -- 172.21.15.140:0/1235744254 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec98001660 con 0x7fecb00addd0 2022-07-21T01:54:18.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecaf7fe700 1 -- 172.21.15.140:0/1235744254 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca800b580 con 0x7fecb00addd0 2022-07-21T01:54:18.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecaf7fe700 1 -- 172.21.15.140:0/1235744254 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feca800b6f0 con 0x7fecb00addd0 2022-07-21T01:54:18.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 -- 172.21.15.140:0/1235744254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecb00addd0 msgr2=0x7fecb0142d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 --2- 172.21.15.140:0/1235744254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecb00addd0 0x7fecb0142d00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7fec98001800 tx=0x7feca80091d0).stop 2022-07-21T01:54:18.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 -- 172.21.15.140:0/1235744254 shutdown_connections 2022-07-21T01:54:18.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 --2- 172.21.15.140:0/1235744254 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecb00addd0 0x7fecb0142d00 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 --2- 172.21.15.140:0/1235744254 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecb00b59f0 0x7fecb005c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 -- 172.21.15.140:0/1235744254 >> 172.21.15.140:0/1235744254 conn(0x7fecb0063480 msgr2=0x7fecb00b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:18.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 -- 172.21.15.140:0/1235744254 shutdown_connections 2022-07-21T01:54:18.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 -- 172.21.15.140:0/1235744254 wait complete. 2022-07-21T01:54:18.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.555+0000 7fecb53cb700 1 Processor -- start 2022-07-21T01:54:18.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb53cb700 1 -- start start 2022-07-21T01:54:18.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb53cb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecb00addd0 0x7fecb0145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb53cb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecb00af940 0x7fecb0155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb53cb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecb00b59f0 0x7fecb0155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb53cb700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb0112df0 con 0x7fecb00b59f0 2022-07-21T01:54:18.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb53cb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00661a0 con 0x7fecb00addd0 2022-07-21T01:54:18.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb53cb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb01126b0 con 0x7fecb00af940 2022-07-21T01:54:18.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb4bca700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecb00b59f0 0x7fecb0155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7feca7fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecb00af940 0x7fecb0155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecaffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecb00addd0 0x7fecb0145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecaffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecb00addd0 0x7fecb0145760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42562/0 (socket says 172.21.15.140:42562) 2022-07-21T01:54:18.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecaffff700 1 -- 172.21.15.140:0/3470025341 learned_addr learned my addr 172.21.15.140:0/3470025341 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:18.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb4bca700 1 -- 172.21.15.140:0/3470025341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecb00af940 msgr2=0x7fecb0155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb4bca700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecb00af940 0x7fecb0155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb4bca700 1 -- 172.21.15.140:0/3470025341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecb00addd0 msgr2=0x7fecb0145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb4bca700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecb00addd0 0x7fecb0145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb4bca700 1 -- 172.21.15.140:0/3470025341 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca8009470 con 0x7fecb00b59f0 2022-07-21T01:54:18.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecaffff700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecb00addd0 0x7fecb0145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:18.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb4bca700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecb00b59f0 0x7fecb0155820 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7fec98001800 tx=0x7feca8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecadffb700 1 -- 172.21.15.140:0/3470025341 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feca800b6f0 con 0x7fecb00b59f0 2022-07-21T01:54:18.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecadffb700 1 -- 172.21.15.140:0/3470025341 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca80032d0 con 0x7fecb00b59f0 2022-07-21T01:54:18.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb53cb700 1 -- 172.21.15.140:0/3470025341 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb0112950 con 0x7fecb00b59f0 2022-07-21T01:54:18.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecadffb700 1 -- 172.21.15.140:0/3470025341 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feca8021630 con 0x7fecb00b59f0 2022-07-21T01:54:18.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.559+0000 7fecb53cb700 1 -- 172.21.15.140:0/3470025341 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb0064ed0 con 0x7fecb00b59f0 2022-07-21T01:54:18.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.563+0000 7fecb53cb700 1 -- 172.21.15.140:0/3470025341 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb0065fc0 con 0x7fecb00b59f0 2022-07-21T01:54:18.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.563+0000 7fecadffb700 1 -- 172.21.15.140:0/3470025341 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7feca800f070 con 0x7fecb00b59f0 2022-07-21T01:54:18.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.563+0000 7fecadffb700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec94053a70 0x7fec94055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.563+0000 7fecadffb700 1 -- 172.21.15.140:0/3470025341 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feca806cfb0 con 0x7fecb00b59f0 2022-07-21T01:54:18.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.563+0000 7fecaffff700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec94053a70 0x7fec94055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.563+0000 7fecaffff700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec94053a70 0x7fec94055f30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7feca0008000 tx=0x7feca0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.567+0000 7fecadffb700 1 -- 172.21.15.140:0/3470025341 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7feca8043dc0 con 0x7fecb00b59f0 2022-07-21T01:54:18.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.695+0000 7fecb53cb700 1 -- 172.21.15.140:0/3470025341 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fecb0112ae0 con 0x7fecb00b59f0 2022-07-21T01:54:18.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.695+0000 7fecadffb700 1 -- 172.21.15.140:0/3470025341 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+10 (secure 0 0 0) 0x7feca80150a0 con 0x7fecb00b59f0 2022-07-21T01:54:18.691 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:54:18.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 -- 172.21.15.140:0/3470025341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec94053a70 msgr2=0x7fec94055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec94053a70 0x7fec94055f30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7feca0008000 tx=0x7feca0008040).stop 2022-07-21T01:54:18.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 -- 172.21.15.140:0/3470025341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecb00b59f0 msgr2=0x7fecb0155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecb00b59f0 0x7fecb0155820 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7fec98001800 tx=0x7feca8016000).stop 2022-07-21T01:54:18.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 -- 172.21.15.140:0/3470025341 shutdown_connections 2022-07-21T01:54:18.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec94053a70 0x7fec94055f30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecb00b59f0 0x7fecb0155820 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fecb00af940 0x7fecb0155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 --2- 172.21.15.140:0/3470025341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecb00addd0 0x7fecb0145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 -- 172.21.15.140:0/3470025341 >> 172.21.15.140:0/3470025341 conn(0x7fecb0063480 msgr2=0x7fecb013eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:18.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 -- 172.21.15.140:0/3470025341 shutdown_connections 2022-07-21T01:54:18.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.699+0000 7fecb53cb700 1 -- 172.21.15.140:0/3470025341 wait complete. 2022-07-21T01:54:18.708 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:54:18.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.823+0000 7fd75da08700 1 Processor -- start 2022-07-21T01:54:18.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.823+0000 7fd75da08700 1 -- start start 2022-07-21T01:54:18.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.823+0000 7fd75da08700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd75805cda0 0x7fd75805d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.823+0000 7fd75da08700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd758142860 0x7fd758062cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.823+0000 7fd75da08700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7580b1df0 con 0x7fd758062180 2022-07-21T01:54:18.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.823+0000 7fd75da08700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7580b2090 con 0x7fd758142860 2022-07-21T01:54:18.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.823+0000 7fd75da08700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd758146140 con 0x7fd75805cda0 2022-07-21T01:54:18.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd7577fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd758142860 0x7fd758062cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd7567fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd758062180 0x7fd758142150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50978/0 (socket says 172.21.15.140:50978) 2022-07-21T01:54:18.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd7567fc700 1 -- 172.21.15.140:0/1317222658 learned_addr learned my addr 172.21.15.140:0/1317222658 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:18.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd756ffd700 1 --2- 172.21.15.140:0/1317222658 >> v2:172.21.15.152:3300/0 conn(0x7fd75805cda0 0x7fd75805d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd7577fe700 1 -- 172.21.15.140:0/1317222658 >> v2:172.21.15.152:3300/0 conn(0x7fd75805cda0 msgr2=0x7fd75805d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:18.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd7577fe700 1 --2- 172.21.15.140:0/1317222658 >> v2:172.21.15.152:3300/0 conn(0x7fd75805cda0 0x7fd75805d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd7577fe700 1 -- 172.21.15.140:0/1317222658 >> v1:172.21.15.140:6789/0 conn(0x7fd758062180 legacy=0x7fd758142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd7577fe700 1 -- 172.21.15.140:0/1317222658 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7581462b0 con 0x7fd758142860 2022-07-21T01:54:18.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd7577fe700 1 --2- 172.21.15.140:0/1317222658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd758142860 0x7fd758062cb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7fd7500032f0 tx=0x7fd7500091d0).ready entity=mon.0 client_cookie=ab3285ff572f3727 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd755ffb700 1 -- 172.21.15.140:0/1317222658 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd75000b680 con 0x7fd758142860 2022-07-21T01:54:18.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd755ffb700 1 -- 172.21.15.140:0/1317222658 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75000b7e0 con 0x7fd758142860 2022-07-21T01:54:18.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd755ffb700 1 -- 172.21.15.140:0/1317222658 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd75000b950 con 0x7fd758142860 2022-07-21T01:54:18.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd75da08700 1 -- 172.21.15.140:0/1317222658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd758142860 msgr2=0x7fd758062cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd75da08700 1 --2- 172.21.15.140:0/1317222658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd758142860 0x7fd758062cb0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7fd7500032f0 tx=0x7fd7500091d0).stop 2022-07-21T01:54:18.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd75da08700 1 -- 172.21.15.140:0/1317222658 shutdown_connections 2022-07-21T01:54:18.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd75da08700 1 --2- 172.21.15.140:0/1317222658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd758142860 0x7fd758062cb0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd75da08700 1 --2- 172.21.15.140:0/1317222658 >> v2:172.21.15.152:3300/0 conn(0x7fd75805cda0 0x7fd75805d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd75da08700 1 -- 172.21.15.140:0/1317222658 >> 172.21.15.140:0/1317222658 conn(0x7fd7580b7da0 msgr2=0x7fd7580b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:18.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd75da08700 1 -- 172.21.15.140:0/1317222658 shutdown_connections 2022-07-21T01:54:18.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd75da08700 1 -- 172.21.15.140:0/1317222658 wait complete. 2022-07-21T01:54:18.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd75da08700 1 Processor -- start 2022-07-21T01:54:18.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.827+0000 7fd75da08700 1 -- start start 2022-07-21T01:54:18.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd75da08700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd75805cda0 0x7fd7581441e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd75da08700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd758062180 0x7fd75809c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd75da08700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd758142860 0x7fd75809cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd75da08700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758153140 con 0x7fd758062180 2022-07-21T01:54:18.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd75da08700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758152ea0 con 0x7fd75805cda0 2022-07-21T01:54:18.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd75da08700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7581482a0 con 0x7fd758142860 2022-07-21T01:54:18.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd7567fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd758062180 0x7fd75809c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd756ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd75805cda0 0x7fd7581441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd756ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd75805cda0 0x7fd7581441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42576/0 (socket says 172.21.15.140:42576) 2022-07-21T01:54:18.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd7567fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd758062180 0x7fd75809c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36838/0 (socket says 172.21.15.140:36838) 2022-07-21T01:54:18.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd7577fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd758142860 0x7fd75809cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd7567fc700 1 -- 172.21.15.140:0/836396387 learned_addr learned my addr 172.21.15.140:0/836396387 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:18.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd7567fc700 1 -- 172.21.15.140:0/836396387 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd758142860 msgr2=0x7fd75809cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd7567fc700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd758142860 0x7fd75809cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd7567fc700 1 -- 172.21.15.140:0/836396387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd75805cda0 msgr2=0x7fd7581441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd7567fc700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd75805cda0 0x7fd7581441e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd7567fc700 1 -- 172.21.15.140:0/836396387 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd750009470 con 0x7fd758062180 2022-07-21T01:54:18.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd7567fc700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd758062180 0x7fd75809c6a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fd7580b5090 tx=0x7fd748007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd747fff700 1 -- 172.21.15.140:0/836396387 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7480070d0 con 0x7fd758062180 2022-07-21T01:54:18.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd747fff700 1 -- 172.21.15.140:0/836396387 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd748007230 con 0x7fd758062180 2022-07-21T01:54:18.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd747fff700 1 -- 172.21.15.140:0/836396387 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd748005610 con 0x7fd758062180 2022-07-21T01:54:18.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd75da08700 1 -- 172.21.15.140:0/836396387 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75809d080 con 0x7fd758062180 2022-07-21T01:54:18.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.831+0000 7fd75da08700 1 -- 172.21.15.140:0/836396387 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75805f8b0 con 0x7fd758062180 2022-07-21T01:54:18.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.835+0000 7fd747fff700 1 -- 172.21.15.140:0/836396387 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd748010070 con 0x7fd758062180 2022-07-21T01:54:18.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.835+0000 7fd75da08700 1 -- 172.21.15.140:0/836396387 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd758148460 con 0x7fd758062180 2022-07-21T01:54:18.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.835+0000 7fd747fff700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd734053980 0x7fd734055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:18.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.835+0000 7fd747fff700 1 -- 172.21.15.140:0/836396387 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd74806b3b0 con 0x7fd758062180 2022-07-21T01:54:18.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.835+0000 7fd756ffd700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd734053980 0x7fd734055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:18.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.835+0000 7fd756ffd700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd734053980 0x7fd734055e40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fd74c00b000 tx=0x7fd74c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:18.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.843+0000 7fd747fff700 1 -- 172.21.15.140:0/836396387 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7480422d0 con 0x7fd758062180 2022-07-21T01:54:18.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.967+0000 7fd75da08700 1 -- 172.21.15.140:0/836396387 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd758066700 con 0x7fd758062180 2022-07-21T01:54:18.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.967+0000 7fd747fff700 1 -- 172.21.15.140:0/836396387 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd748018070 con 0x7fd758062180 2022-07-21T01:54:18.964 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:54:18.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 -- 172.21.15.140:0/836396387 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd734053980 msgr2=0x7fd734055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd734053980 0x7fd734055e40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fd74c00b000 tx=0x7fd74c006df0).stop 2022-07-21T01:54:18.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 -- 172.21.15.140:0/836396387 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd758062180 msgr2=0x7fd75809c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:18.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd758062180 0x7fd75809c6a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7fd7580b5090 tx=0x7fd748007c00).stop 2022-07-21T01:54:18.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 -- 172.21.15.140:0/836396387 shutdown_connections 2022-07-21T01:54:18.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd734053980 0x7fd734055e40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd758142860 0x7fd75809cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd758062180 0x7fd75809c6a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 --2- 172.21.15.140:0/836396387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd75805cda0 0x7fd7581441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:18.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 -- 172.21.15.140:0/836396387 >> 172.21.15.140:0/836396387 conn(0x7fd7580b7da0 msgr2=0x7fd758067550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:18.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 -- 172.21.15.140:0/836396387 shutdown_connections 2022-07-21T01:54:18.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:18.971+0000 7fd75da08700 1 -- 172.21.15.140:0/836396387 wait complete. 2022-07-21T01:54:18.977 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:54:18.978 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:54:19.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862e109700 1 Processor -- start 2022-07-21T01:54:19.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862e109700 1 -- start start 2022-07-21T01:54:19.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862e109700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f862805bd90 0x7f86280b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:19.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862e109700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86280aea30 con 0x7f86280b1200 2022-07-21T01:54:19.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862e109700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86280aec60 con 0x7f86280b5650 2022-07-21T01:54:19.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862e109700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8628142970 con 0x7f862805bd90 2022-07-21T01:54:19.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862c906700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f86280b1200 0x7f8628142d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:50990/0 (socket says 172.21.15.140:50990) 2022-07-21T01:54:19.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862c906700 1 -- 172.21.15.140:0/2549540667 learned_addr learned my addr 172.21.15.140:0/2549540667 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:19.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862d908700 1 --2- 172.21.15.140:0/2549540667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f862805bd90 0x7f86280b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:19.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f8613fff700 1 -- 172.21.15.140:0/2549540667 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667040366 0 0) 0x7f86280aea30 con 0x7f86280b1200 2022-07-21T01:54:19.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f8613fff700 1 -- 172.21.15.140:0/2549540667 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f860c003260 con 0x7f86280b1200 2022-07-21T01:54:19.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862d908700 1 -- 172.21.15.140:0/2549540667 >> v1:172.21.15.152:6789/0 conn(0x7f86280b5650 legacy=0x7f862805c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:19.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862d908700 1 -- 172.21.15.140:0/2549540667 >> v1:172.21.15.140:6789/0 conn(0x7f86280b1200 legacy=0x7f8628142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:19.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f8613fff700 1 -- 172.21.15.140:0/2549540667 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246707575 0 0) 0x7f86280aec60 con 0x7f86280b5650 2022-07-21T01:54:19.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.087+0000 7f862d908700 1 -- 172.21.15.140:0/2549540667 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86280af180 con 0x7f862805bd90 2022-07-21T01:54:19.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862d908700 1 --2- 172.21.15.140:0/2549540667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f862805bd90 0x7f86280b5a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f86280adec0 tx=0x7f86200091d0).ready entity=mon.2 client_cookie=4231b79c13553197 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:19.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f8613fff700 1 -- 172.21.15.140:0/2549540667 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f862000b6b0 con 0x7f862805bd90 2022-07-21T01:54:19.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f8613fff700 1 -- 172.21.15.140:0/2549540667 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862000b810 con 0x7f862805bd90 2022-07-21T01:54:19.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f8613fff700 1 -- 172.21.15.140:0/2549540667 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f862000b980 con 0x7f862805bd90 2022-07-21T01:54:19.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 -- 172.21.15.140:0/2549540667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f862805bd90 msgr2=0x7f86280b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:19.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 --2- 172.21.15.140:0/2549540667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f862805bd90 0x7f86280b5a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f86280adec0 tx=0x7f86200091d0).stop 2022-07-21T01:54:19.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 -- 172.21.15.140:0/2549540667 shutdown_connections 2022-07-21T01:54:19.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 --2- 172.21.15.140:0/2549540667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f862805bd90 0x7f86280b5a30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:19.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 -- 172.21.15.140:0/2549540667 >> 172.21.15.140:0/2549540667 conn(0x7f86280634e0 msgr2=0x7f86280b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:19.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 -- 172.21.15.140:0/2549540667 shutdown_connections 2022-07-21T01:54:19.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 -- 172.21.15.140:0/2549540667 wait complete. 2022-07-21T01:54:19.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 Processor -- start 2022-07-21T01:54:19.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 -- start start 2022-07-21T01:54:19.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f862805bd90 0x7f862813c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:19.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86280b1200 0x7f8628140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:19.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86280b5650 0x7f86281554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:19.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86281455c0 con 0x7f862805bd90 2022-07-21T01:54:19.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8628141440 con 0x7f86280b1200 2022-07-21T01:54:19.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862e109700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8628155c80 con 0x7f86280b5650 2022-07-21T01:54:19.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.091+0000 7f862d107700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f862805bd90 0x7f862813c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:19.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862c906700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86280b1200 0x7f8628140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:19.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862d107700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f862805bd90 0x7f862813c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36850/0 (socket says 172.21.15.140:36850) 2022-07-21T01:54:19.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862d107700 1 -- 172.21.15.140:0/3969919153 learned_addr learned my addr 172.21.15.140:0/3969919153 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:19.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862d908700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86280b5650 0x7f86281554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:19.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862d107700 1 -- 172.21.15.140:0/3969919153 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86280b5650 msgr2=0x7f86281554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:19.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862d107700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86280b5650 0x7f86281554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:19.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862d107700 1 -- 172.21.15.140:0/3969919153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86280b1200 msgr2=0x7f8628140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:19.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862d107700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86280b1200 0x7f8628140f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:19.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862d107700 1 -- 172.21.15.140:0/3969919153 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8620009470 con 0x7f862805bd90 2022-07-21T01:54:19.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862d107700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f862805bd90 0x7f862813c130 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f8628141580 tx=0x7f861c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:19.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f861affd700 1 -- 172.21.15.140:0/3969919153 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f861c012400 con 0x7f862805bd90 2022-07-21T01:54:19.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f861affd700 1 -- 172.21.15.140:0/3969919153 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c012560 con 0x7f862805bd90 2022-07-21T01:54:19.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862e109700 1 -- 172.21.15.140:0/3969919153 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8628155f20 con 0x7f862805bd90 2022-07-21T01:54:19.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f861affd700 1 -- 172.21.15.140:0/3969919153 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f861c0128c0 con 0x7f862805bd90 2022-07-21T01:54:19.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862e109700 1 -- 172.21.15.140:0/3969919153 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862813b3d0 con 0x7f862805bd90 2022-07-21T01:54:19.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f862e109700 1 -- 172.21.15.140:0/3969919153 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86280574e0 con 0x7f862805bd90 2022-07-21T01:54:19.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f861affd700 1 -- 172.21.15.140:0/3969919153 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f861c019030 con 0x7f862805bd90 2022-07-21T01:54:19.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.095+0000 7f861affd700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8604053980 0x7f8604055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:19.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.099+0000 7f861affd700 1 -- 172.21.15.140:0/3969919153 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f861c06b260 con 0x7f862805bd90 2022-07-21T01:54:19.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.099+0000 7f862c906700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8604053980 0x7f8604055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:19.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.099+0000 7f862c906700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8604053980 0x7f8604055e40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f862400d000 tx=0x7f86240027f0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:19.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.103+0000 7f861affd700 1 -- 172.21.15.140:0/3969919153 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f861c0420e0 con 0x7f862805bd90 2022-07-21T01:54:19.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.267+0000 7f862e109700 1 -- 172.21.15.140:0/3969919153 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8628067ea0 con 0x7f862805bd90 2022-07-21T01:54:19.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.267+0000 7f861affd700 1 -- 172.21.15.140:0/3969919153 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f861c045700 con 0x7f862805bd90 2022-07-21T01:54:19.265 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:19.265 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":30,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/4173658089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4173658089},{"type":"v1","addr":"172.21.15.152:6833","nonce":4173658089}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:54:10.830421+0000","modified":"2022-07-21T01:54:11.841578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6208":{"gid":6208,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/167323020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":167323020},{"type":"v1","addr":"172.21.15.140:6835","nonce":167323020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-07-21T01:54:19.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 -- 172.21.15.140:0/3969919153 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8604053980 msgr2=0x7f8604055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:19.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8604053980 0x7f8604055e40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f862400d000 tx=0x7f86240027f0).stop 2022-07-21T01:54:19.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 -- 172.21.15.140:0/3969919153 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f862805bd90 msgr2=0x7f862813c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:19.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f862805bd90 0x7f862813c130 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f8628141580 tx=0x7f861c007c50).stop 2022-07-21T01:54:19.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 -- 172.21.15.140:0/3969919153 shutdown_connections 2022-07-21T01:54:19.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8604053980 0x7f8604055e40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:19.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86280b5650 0x7f86281554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:19.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86280b1200 0x7f8628140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:19.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 --2- 172.21.15.140:0/3969919153 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f862805bd90 0x7f862813c130 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:19.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 -- 172.21.15.140:0/3969919153 >> 172.21.15.140:0/3969919153 conn(0x7f86280634e0 msgr2=0x7f86280b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:19.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 -- 172.21.15.140:0/3969919153 shutdown_connections 2022-07-21T01:54:19.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:19.271+0000 7f862e109700 1 -- 172.21.15.140:0/3969919153 wait complete. 2022-07-21T01:54:19.273 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 30 2022-07-21T01:54:19.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:54:10.830421+0000', 'modified': '2022-07-21T01:54:11.841578+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6208}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6208': {'gid': 6208, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/167323020', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 167323020}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 167323020}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:54:19.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:54:19.487 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:54:19.605 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":6208,"want_state":"up:active","state":"up:active","rank_uptime":8.7576214280000002,"mdsmap_epoch":30,"osdmap_epoch":73,"osdmap_epoch_barrier":72,"uptime":12.829398137} 2022-07-21T01:54:19.605 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 6208, "mdsmap_epoch": 30, "osdmap_epoch": 73, "osdmap_epoch_barrier": 72, "rank_uptime": 8.757621428, "state": "up:active", "uptime": 12.829398137, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:54:19.606 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:54:19.606 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-07-21T01:54:19.607 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-07-21T01:54:19.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:19.608 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:54:19.617 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:54:19.676 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:54:19.677 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:54:19.690 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:54:19.691 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:54:19.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:19.693 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:54:19.697 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:54:19.698 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:54:19.701 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:54:19.716 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:54:19.718 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:54:19.729 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:54:19.731 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:54:19.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:19.733 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:54:19.775 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2022-07-21T01:54:19.777+0000 7fade650a080 -1 init, newargv = 0x559e0651c2f0 newargc=9ceph-fuse[22371]: starting ceph client 2022-07-21T01:54:19.776 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr: 2022-07-21T01:54:19.783 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22371]: starting fuse 2022-07-21T01:54:20.883 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:54:20.903 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:54:20.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:54:20.962 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:54:20.963 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:54:20.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:20.964 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:54:20.970 INFO:teuthology.orchestra.run.smithi192.stdout:52 2022-07-21T01:54:20.970 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:54:20.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:20.971 DEBUG:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2022-07-21T01:54:20.971 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2022-07-21T01:54:20.972 DEBUG:teuthology.orchestra.run.smithi192:> import re 2022-07-21T01:54:20.972 DEBUG:teuthology.orchestra.run.smithi192:> import os 2022-07-21T01:54:20.973 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2022-07-21T01:54:20.973 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:54:20.974 DEBUG:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2022-07-21T01:54:20.974 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-07-21T01:54:20.975 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2022-07-21T01:54:20.976 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:54:20.976 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2022-07-21T01:54:20.977 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2022-07-21T01:54:20.977 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2022-07-21T01:54:20.978 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2022-07-21T01:54:20.978 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:54:20.979 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2022-07-21T01:54:20.979 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:54:20.980 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:54:20.980 DEBUG:teuthology.orchestra.run.smithi192:> return f 2022-07-21T01:54:20.981 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:54:20.981 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:54:20.982 DEBUG:teuthology.orchestra.run.smithi192:> print(find_socket("client.1")) 2022-07-21T01:54:20.982 DEBUG:teuthology.orchestra.run.smithi192:> ' 2022-07-21T01:54:21.057 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.22371.asok 2022-07-21T01:54:21.063 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22371.asok 2022-07-21T01:54:21.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:21.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22371.asok status 2022-07-21T01:54:21.153 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2022-07-21T01:54:21.154 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2022-07-21T01:54:21.154 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:54:21.154 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:54:21.154 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2022-07-21T01:54:21.155 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2022-07-21T01:54:21.155 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-07-21T01:54:21.155 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "22371", 2022-07-21T01:54:21.155 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2022-07-21T01:54:21.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:54:21.156 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2022-07-21T01:54:21.156 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2022-07-21T01:54:21.156 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 6498, 2022-07-21T01:54:21.157 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2022-07-21T01:54:21.157 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2022-07-21T01:54:21.157 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2022-07-21T01:54:21.157 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 6498 2022-07-21T01:54:21.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:54:21.158 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:54:21.158 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:54:21.158 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:54:21.159 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3500337191 2022-07-21T01:54:21.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2022-07-21T01:54:21.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:54:21.159 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:54:21.160 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:54:21.160 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:54:21.160 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3500337191 2022-07-21T01:54:21.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:54:21.161 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.6498 172.21.15.192:0/3500337191", 2022-07-21T01:54:21.161 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/3500337191", 2022-07-21T01:54:21.161 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2022-07-21T01:54:21.161 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 30, 2022-07-21T01:54:21.162 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 73, 2022-07-21T01:54:21.162 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:54:21.162 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false, 2022-07-21T01:54:21.162 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2022-07-21T01:54:21.163 INFO:teuthology.orchestra.run.smithi192.stdout:} 2022-07-21T01:54:21.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:21.165 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:54:21.165 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:54:21.173 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:54:21.174 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:54:21.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:21.175 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:54:21.233 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:54:21.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed0a8cd700 1 Processor -- start 2022-07-21T01:54:21.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed0a8cd700 1 -- start start 2022-07-21T01:54:21.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed0a8cd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed040aea10 0x7fed040b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed0a8cd700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed041433d0 con 0x7fed041424e0 2022-07-21T01:54:21.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed0a8cd700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed04143670 con 0x7fed040b0570 2022-07-21T01:54:21.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed0a8cd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed04143b10 con 0x7fed040aea10 2022-07-21T01:54:21.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed090ca700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fed041424e0 0x7fed040ad350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51004/0 (socket says 172.21.15.140:51004) 2022-07-21T01:54:21.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed090ca700 1 -- 172.21.15.140:0/3065740064 learned_addr learned my addr 172.21.15.140:0/3065740064 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:54:21.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed0a0cc700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed040aea10 0x7fed040b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 97790884 0 0) 0x7fed041433d0 con 0x7fed041424e0 2022-07-21T01:54:21.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecec003260 con 0x7fed041424e0 2022-07-21T01:54:21.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066942473 0 0) 0x7fed04143670 con 0x7fed040b0570 2022-07-21T01:54:21.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed041433d0 con 0x7fed040b0570 2022-07-21T01:54:21.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2178866590 0 0) 0x7fecec003260 con 0x7fed041424e0 2022-07-21T01:54:21.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed040aea10 msgr2=0x7fed040b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:21.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed040aea10 0x7fed040b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 >> v1:172.21.15.152:6789/0 conn(0x7fed040b0570 legacy=0x7fed04141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed040b1010 con 0x7fed041424e0 2022-07-21T01:54:21.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fecf4001660 con 0x7fed041424e0 2022-07-21T01:54:21.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed0a0cc700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed040aea10 0x7fed040b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:21.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 >> v1:172.21.15.140:6789/0 conn(0x7fed041424e0 legacy=0x7fed040ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecec003910 0x7fecec003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed040aea10 0x7fecec004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecec0049f0 0x7fecec004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed04143b10 con 0x7fecec003910 2022-07-21T01:54:21.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec00b750 con 0x7fecec0049f0 2022-07-21T01:54:21.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec00b910 con 0x7fed040aea10 2022-07-21T01:54:21.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed098cb700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecec003910 0x7fecec003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed090ca700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed040aea10 0x7fecec004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed0a0cc700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecec0049f0 0x7fecec004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed090ca700 1 -- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecec0049f0 msgr2=0x7fecec004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:21.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed090ca700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fecec0049f0 0x7fecec004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed090ca700 1 -- 172.21.15.140:0/3065740064 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecec003910 msgr2=0x7fecec003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:21.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed090ca700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecec003910 0x7fecec003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed090ca700 1 -- 172.21.15.140:0/3065740064 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc009470 con 0x7fed040aea10 2022-07-21T01:54:21.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed098cb700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fecec003910 0x7fecec003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:54:21.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.351+0000 7fed090ca700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed040aea10 0x7fecec004500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fecec004470 tx=0x7fecf4001dd0).ready entity=mon.2 client_cookie=23ea3fa6c04941c4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:21.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecf40070d0 con 0x7fed040aea10 2022-07-21T01:54:21.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf4007230 con 0x7fed040aea10 2022-07-21T01:54:21.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed088c9700 1 -- 172.21.15.140:0/3065740064 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fecf400c8b0 con 0x7fed040aea10 2022-07-21T01:54:21.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 -- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed040aea10 msgr2=0x7fecec004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 --2- 172.21.15.140:0/3065740064 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed040aea10 0x7fecec004500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fecec004470 tx=0x7fecf4001dd0).stop 2022-07-21T01:54:21.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a0cc700 1 -- 172.21.15.140:0/3065740064 reap_dead start 2022-07-21T01:54:21.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 -- 172.21.15.140:0/3065740064 shutdown_connections 2022-07-21T01:54:21.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 -- 172.21.15.140:0/3065740064 >> 172.21.15.140:0/3065740064 conn(0x7fed040b9360 msgr2=0x7fed040b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:21.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 -- 172.21.15.140:0/3065740064 shutdown_connections 2022-07-21T01:54:21.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 -- 172.21.15.140:0/3065740064 wait complete. 2022-07-21T01:54:21.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 Processor -- start 2022-07-21T01:54:21.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 -- start start 2022-07-21T01:54:21.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fed040aea10 0x7fed0413eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04151bf0 0x7fed0413d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed04146660 0x7fed0413dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed040b2230 con 0x7fed040aea10 2022-07-21T01:54:21.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0405d180 con 0x7fed04151bf0 2022-07-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a8cd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed04155160 con 0x7fed04146660 2022-07-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed098cb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fed040aea10 0x7fed0413eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed098cb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fed040aea10 0x7fed0413eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36870/0 (socket says 172.21.15.140:36870) 2022-07-21T01:54:21.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed098cb700 1 -- 172.21.15.140:0/625583155 learned_addr learned my addr 172.21.15.140:0/625583155 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:21.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed0a0cc700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed04146660 0x7fed0413dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.355+0000 7fed090ca700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04151bf0 0x7fed0413d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fed098cb700 1 -- 172.21.15.140:0/625583155 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed04146660 msgr2=0x7fed0413dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fed098cb700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed04146660 0x7fed0413dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fed098cb700 1 -- 172.21.15.140:0/625583155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04151bf0 msgr2=0x7fed0413d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fed098cb700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04151bf0 0x7fed0413d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fed098cb700 1 -- 172.21.15.140:0/625583155 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed041454b0 con 0x7fed040aea10 2022-07-21T01:54:21.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fed098cb700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fed040aea10 0x7fed0413eda0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fed04152050 tx=0x7fed00007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:21.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fecf2ffd700 1 -- 172.21.15.140:0/625583155 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed00011780 con 0x7fed040aea10 2022-07-21T01:54:21.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fecf2ffd700 1 -- 172.21.15.140:0/625583155 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed000118e0 con 0x7fed040aea10 2022-07-21T01:54:21.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fed0a8cd700 1 -- 172.21.15.140:0/625583155 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed04145750 con 0x7fed040aea10 2022-07-21T01:54:21.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fecf2ffd700 1 -- 172.21.15.140:0/625583155 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed00011c40 con 0x7fed040aea10 2022-07-21T01:54:21.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fed0a8cd700 1 -- 172.21.15.140:0/625583155 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0413e1e0 con 0x7fed040aea10 2022-07-21T01:54:21.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fed0a8cd700 1 -- 172.21.15.140:0/625583155 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed040574e0 con 0x7fed040aea10 2022-07-21T01:54:21.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fecf2ffd700 1 -- 172.21.15.140:0/625583155 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fed0001b030 con 0x7fed040aea10 2022-07-21T01:54:21.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fecf2ffd700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fece4053b30 0x7fece4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fed090ca700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fece4053b30 0x7fece4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.359+0000 7fecf2ffd700 1 -- 172.21.15.140:0/625583155 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fed0006be70 con 0x7fed040aea10 2022-07-21T01:54:21.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.363+0000 7fed090ca700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fece4053b30 0x7fece4055ff0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fecf4018000 tx=0x7fecf4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:21.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.367+0000 7fecf2ffd700 1 -- 172.21.15.140:0/625583155 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fed00042cf0 con 0x7fed040aea10 2022-07-21T01:54:21.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 -- 172.21.15.140:0/625583155 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed0405d470 con 0x7fed040aea10 2022-07-21T01:54:21.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fecf2ffd700 1 -- 172.21.15.140:0/625583155 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2143 (secure 0 0 0) 0x7fed00046310 con 0x7fed040aea10 2022-07-21T01:54:21.523 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:21.523 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":30,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/4173658089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4173658089},{"type":"v1","addr":"172.21.15.152:6833","nonce":4173658089}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:54:10.830421+0000","modified":"2022-07-21T01:54:11.841578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6208":{"gid":6208,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/167323020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":167323020},{"type":"v1","addr":"172.21.15.140:6835","nonce":167323020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-07-21T01:54:21.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 -- 172.21.15.140:0/625583155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fece4053b30 msgr2=0x7fece4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fece4053b30 0x7fece4055ff0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fecf4018000 tx=0x7fecf4015000).stop 2022-07-21T01:54:21.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 -- 172.21.15.140:0/625583155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fed040aea10 msgr2=0x7fed0413eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fed040aea10 0x7fed0413eda0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fed04152050 tx=0x7fed00007a00).stop 2022-07-21T01:54:21.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 -- 172.21.15.140:0/625583155 shutdown_connections 2022-07-21T01:54:21.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fece4053b30 0x7fece4055ff0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fed04146660 0x7fed0413dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed04151bf0 0x7fed0413d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 --2- 172.21.15.140:0/625583155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fed040aea10 0x7fed0413eda0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 -- 172.21.15.140:0/625583155 >> 172.21.15.140:0/625583155 conn(0x7fed040b9360 msgr2=0x7fed041410b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:21.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 -- 172.21.15.140:0/625583155 shutdown_connections 2022-07-21T01:54:21.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.527+0000 7fed0a8cd700 1 -- 172.21.15.140:0/625583155 wait complete. 2022-07-21T01:54:21.532 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 30 2022-07-21T01:54:21.537 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:54:21.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 Processor -- start 2022-07-21T01:54:21.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 -- start start 2022-07-21T01:54:21.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f57c40aea20 0x7f57c4140a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f57c40675f0 0x7f57c4140f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c40aee00 con 0x7f57c4141470 2022-07-21T01:54:21.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c4144470 con 0x7f57c40675f0 2022-07-21T01:54:21.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c4144710 con 0x7f57c40aea20 2022-07-21T01:54:21.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57ba7fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f57c40675f0 0x7f57c4140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57ba7fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f57c40675f0 0x7f57c4140f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36878/0 (socket says 172.21.15.140:36878) 2022-07-21T01:54:21.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57ba7fc700 1 -- 172.21.15.140:0/4147835417 learned_addr learned my addr 172.21.15.140:0/4147835417 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:21.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c2ffd700 1 --2- 172.21.15.140:0/4147835417 >> v2:172.21.15.152:3300/0 conn(0x7f57c40aea20 0x7f57c4140a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57ba7fc700 1 -- 172.21.15.140:0/4147835417 >> v2:172.21.15.152:3300/0 conn(0x7f57c40aea20 msgr2=0x7f57c4140a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:21.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57ba7fc700 1 --2- 172.21.15.140:0/4147835417 >> v2:172.21.15.152:3300/0 conn(0x7f57c40aea20 0x7f57c4140a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57ba7fc700 1 -- 172.21.15.140:0/4147835417 >> v1:172.21.15.152:6789/0 conn(0x7f57c4141470 legacy=0x7f57c413a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:54:21.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57ba7fc700 1 -- 172.21.15.140:0/4147835417 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c41449b0 con 0x7f57c40675f0 2022-07-21T01:54:21.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57ba7fc700 1 --2- 172.21.15.140:0/4147835417 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57c40675f0 0x7f57c4140f80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f57c4063ee0 tx=0x7f57ac006eb0).ready entity=mon.0 client_cookie=5b6046242dba5861 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:21.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c27fc700 1 -- 172.21.15.140:0/4147835417 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57ac00d5c0 con 0x7f57c40675f0 2022-07-21T01:54:21.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c27fc700 1 -- 172.21.15.140:0/4147835417 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac000e90 con 0x7f57c40675f0 2022-07-21T01:54:21.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c27fc700 1 -- 172.21.15.140:0/4147835417 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57ac00d950 con 0x7f57c40675f0 2022-07-21T01:54:21.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 -- 172.21.15.140:0/4147835417 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57c40675f0 msgr2=0x7f57c4140f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 --2- 172.21.15.140:0/4147835417 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57c40675f0 0x7f57c4140f80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f57c4063ee0 tx=0x7f57ac006eb0).stop 2022-07-21T01:54:21.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 -- 172.21.15.140:0/4147835417 shutdown_connections 2022-07-21T01:54:21.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 --2- 172.21.15.140:0/4147835417 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57c40675f0 0x7f57c4140f80 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 --2- 172.21.15.140:0/4147835417 >> v2:172.21.15.152:3300/0 conn(0x7f57c40aea20 0x7f57c4140a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 -- 172.21.15.140:0/4147835417 >> 172.21.15.140:0/4147835417 conn(0x7f57c40611c0 msgr2=0x7f57c4062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:21.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 -- 172.21.15.140:0/4147835417 shutdown_connections 2022-07-21T01:54:21.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.639+0000 7f57c93f5700 1 -- 172.21.15.140:0/4147835417 wait complete. 2022-07-21T01:54:21.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c93f5700 1 Processor -- start 2022-07-21T01:54:21.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c93f5700 1 -- start start 2022-07-21T01:54:21.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c93f5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57c40aea20 0x7f57c413d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c93f5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57c4141470 0x7f57c4140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c93f5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c413bbb0 0x7f57c4140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c93f5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c413e6c0 con 0x7f57c40aea20 2022-07-21T01:54:21.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c93f5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c413fc20 con 0x7f57c413bbb0 2022-07-21T01:54:21.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c93f5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c413fec0 con 0x7f57c4141470 2022-07-21T01:54:21.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c2ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57c40aea20 0x7f57c413d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c2ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57c40aea20 0x7f57c413d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36884/0 (socket says 172.21.15.140:36884) 2022-07-21T01:54:21.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c2ffd700 1 -- 172.21.15.140:0/730520616 learned_addr learned my addr 172.21.15.140:0/730520616 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:54:21.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57ba7fc700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57c4141470 0x7f57c4140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c37fe700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c413bbb0 0x7f57c4140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c2ffd700 1 -- 172.21.15.140:0/730520616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57c4141470 msgr2=0x7f57c4140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c2ffd700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57c4141470 0x7f57c4140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c2ffd700 1 -- 172.21.15.140:0/730520616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c413bbb0 msgr2=0x7f57c4140670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c2ffd700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c413bbb0 0x7f57c4140670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c2ffd700 1 -- 172.21.15.140:0/730520616 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac00b1e0 con 0x7f57c40aea20 2022-07-21T01:54:21.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57ba7fc700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57c4141470 0x7f57c4140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:54:21.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c37fe700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c413bbb0 0x7f57c4140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:54:21.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.643+0000 7f57c2ffd700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57c40aea20 0x7f57c413d590 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f57b400a000 tx=0x7f57b4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:21.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.647+0000 7f57c0ff9700 1 -- 172.21.15.140:0/730520616 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57b4011780 con 0x7f57c40aea20 2022-07-21T01:54:21.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.647+0000 7f57c0ff9700 1 -- 172.21.15.140:0/730520616 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b40118e0 con 0x7f57c40aea20 2022-07-21T01:54:21.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.647+0000 7f57c93f5700 1 -- 172.21.15.140:0/730520616 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c413b770 con 0x7f57c40aea20 2022-07-21T01:54:21.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.647+0000 7f57c0ff9700 1 -- 172.21.15.140:0/730520616 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57b4011c40 con 0x7f57c40aea20 2022-07-21T01:54:21.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.647+0000 7f57c93f5700 1 -- 172.21.15.140:0/730520616 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c4140d00 con 0x7f57c40aea20 2022-07-21T01:54:21.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.647+0000 7f57c93f5700 1 -- 172.21.15.140:0/730520616 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c413b910 con 0x7f57c40aea20 2022-07-21T01:54:21.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.647+0000 7f57c0ff9700 1 -- 172.21.15.140:0/730520616 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f57b4012420 con 0x7f57c40aea20 2022-07-21T01:54:21.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.647+0000 7f57c0ff9700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57a80539e0 0x7f57a8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:54:21.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.647+0000 7f57c0ff9700 1 -- 172.21.15.140:0/730520616 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f57b406bdd0 con 0x7f57c40aea20 2022-07-21T01:54:21.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.647+0000 7f57ba7fc700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57a80539e0 0x7f57a8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:54:21.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.647+0000 7f57ba7fc700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57a80539e0 0x7f57a8055ea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f57ac01a000 tx=0x7f57ac015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:54:21.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.651+0000 7f57c0ff9700 1 -- 172.21.15.140:0/730520616 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57c413b910 con 0x7f57c40aea20 2022-07-21T01:54:21.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.811+0000 7f57c93f5700 1 -- 172.21.15.140:0/730520616 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57c4141250 con 0x7f57c40aea20 2022-07-21T01:54:21.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c0ff9700 1 -- 172.21.15.140:0/730520616 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f57b4042b00 con 0x7f57c40aea20 2022-07-21T01:54:21.810 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:54:21.810 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":30,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/4173658089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4173658089},{"type":"v1","addr":"172.21.15.152:6833","nonce":4173658089}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:54:10.830421+0000","modified":"2022-07-21T01:54:11.841578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6208":{"gid":6208,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/167323020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":167323020},{"type":"v1","addr":"172.21.15.140:6835","nonce":167323020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-07-21T01:54:21.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 -- 172.21.15.140:0/730520616 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57a80539e0 msgr2=0x7f57a8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57a80539e0 0x7f57a8055ea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f57ac01a000 tx=0x7f57ac015000).stop 2022-07-21T01:54:21.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 -- 172.21.15.140:0/730520616 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57c40aea20 msgr2=0x7f57c413d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:54:21.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57c40aea20 0x7f57c413d590 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f57b400a000 tx=0x7f57b4008040).stop 2022-07-21T01:54:21.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 -- 172.21.15.140:0/730520616 shutdown_connections 2022-07-21T01:54:21.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57a80539e0 0x7f57a8055ea0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57c413bbb0 0x7f57c4140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57c4141470 0x7f57c4140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 --2- 172.21.15.140:0/730520616 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57c40aea20 0x7f57c413d590 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:54:21.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 -- 172.21.15.140:0/730520616 >> 172.21.15.140:0/730520616 conn(0x7f57c40675f0 msgr2=0x7f57c40b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:54:21.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 -- 172.21.15.140:0/730520616 shutdown_connections 2022-07-21T01:54:21.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:54:21.815+0000 7f57c93f5700 1 -- 172.21.15.140:0/730520616 wait complete. 2022-07-21T01:54:21.816 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 30 2022-07-21T01:54:21.823 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_cap_revoke_eviction_timeout 30.0 2022-07-21T01:54:21.924 INFO:teuthology.orchestra.run.smithi140.stdout:{"success":"mds_cap_revoke_eviction_timeout = '30.000000' "} 2022-07-21T01:54:21.925 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_cap_revoke_eviction_timeout = '30.000000' " } 2022-07-21T01:54:21.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:21.927 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:54:21.928 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:54:21.937 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:54:21.939 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:54:21.941 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-07-21T01:54:21.942 DEBUG:teuthology.orchestra.run.smithi191:> import time 2022-07-21T01:54:21.943 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:54:21.943 DEBUG:teuthology.orchestra.run.smithi191:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2022-07-21T01:54:21.944 DEBUG:teuthology.orchestra.run.smithi191:> f.write('"'"'content'"'"') 2022-07-21T01:54:21.944 DEBUG:teuthology.orchestra.run.smithi191:> f.flush() 2022-07-21T01:54:21.945 DEBUG:teuthology.orchestra.run.smithi191:> f.write('"'"'content2'"'"') 2022-07-21T01:54:21.945 DEBUG:teuthology.orchestra.run.smithi191:> while True: 2022-07-21T01:54:21.946 DEBUG:teuthology.orchestra.run.smithi191:> time.sleep(1) 2022-07-21T01:54:21.946 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:54:21.949 DEBUG:teuthology.orchestra.run.smithi191:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2022-07-21T01:54:21.966 INFO:teuthology.orchestra.run.smithi191.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2022-07-21T01:54:21.967 DEBUG:teuthology.orchestra.run:got remote process result: 2 2022-07-21T01:54:22.969 DEBUG:teuthology.orchestra.run.smithi191:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2022-07-21T01:54:22.987 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.0/background_file 2022-07-21T01:54:22.989 DEBUG:tasks.cephfs.mount:File background_file became visible from 0 after 1s 2022-07-21T01:54:22.990 DEBUG:teuthology.orchestra.run.smithi192:> sudo ls /home/ubuntu/cephtest/mnt.1/background_file 2022-07-21T01:54:23.025 INFO:teuthology.orchestra.run.smithi192.stdout:/home/ubuntu/cephtest/mnt.1/background_file 2022-07-21T01:54:23.026 DEBUG:tasks.cephfs.mount:File background_file became visible from 1 after 0s 2022-07-21T01:54:23.027 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:54:23.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:54:23.043 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:54:23.043 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:54:23.052 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:54:23.053 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:54:23.054 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-07-21T01:54:23.054 DEBUG:teuthology.orchestra.run.smithi192:> import os 2022-07-21T01:54:23.055 DEBUG:teuthology.orchestra.run.smithi192:> import time 2022-07-21T01:54:23.055 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:54:23.056 DEBUG:teuthology.orchestra.run.smithi192:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2022-07-21T01:54:23.056 DEBUG:teuthology.orchestra.run.smithi192:> try: 2022-07-21T01:54:23.056 DEBUG:teuthology.orchestra.run.smithi192:> while True: 2022-07-21T01:54:23.057 DEBUG:teuthology.orchestra.run.smithi192:> os.write(fd, b'"'"'content'"'"') 2022-07-21T01:54:23.057 DEBUG:teuthology.orchestra.run.smithi192:> time.sleep(1) 2022-07-21T01:54:23.057 DEBUG:teuthology.orchestra.run.smithi192:> if not False: 2022-07-21T01:54:23.058 DEBUG:teuthology.orchestra.run.smithi192:> break 2022-07-21T01:54:23.058 DEBUG:teuthology.orchestra.run.smithi192:> except IOError as e: 2022-07-21T01:54:23.059 DEBUG:teuthology.orchestra.run.smithi192:> pass 2022-07-21T01:54:23.059 DEBUG:teuthology.orchestra.run.smithi192:> os.close(fd) 2022-07-21T01:54:23.060 DEBUG:teuthology.orchestra.run.smithi192:> ' 2022-07-21T01:54:32.015 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:54:32.018 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:54:32.021 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:54:32.024 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:54:32.027 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:55:02.095 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:55:02.099 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:55:02.102 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:55:02.105 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:55:02.108 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:55:02.825 INFO:tasks.cephfs.test_misc:cap_waiter waited 39.723318099975586s 2022-07-21T01:55:02.826 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls --format=json 2022-07-21T01:55:02.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.938+0000 7f100bdc5700 1 Processor -- start 2022-07-21T01:55:02.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.938+0000 7f100bdc5700 1 -- start start 2022-07-21T01:55:02.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.938+0000 7f100bdc5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f10040afb80 0x7f1004061080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:02.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.938+0000 7f100bdc5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100414a750 con 0x7f1004097a80 2022-07-21T01:55:02.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.938+0000 7f100bdc5700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10040b2840 con 0x7f1004060ca0 2022-07-21T01:55:02.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.938+0000 7f100bdc5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1004148430 con 0x7f10040afb80 2022-07-21T01:55:02.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.938+0000 7f1009360700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1004097a80 0x7f1004146850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51042/0 (socket says 172.21.15.140:51042) 2022-07-21T01:55:02.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.938+0000 7f1009360700 1 -- 172.21.15.140:0/1329155083 learned_addr learned my addr 172.21.15.140:0/1329155083 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:02.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.938+0000 7f100a362700 1 --2- 172.21.15.140:0/1329155083 >> v2:172.21.15.152:3300/0 conn(0x7f10040afb80 0x7f1004061080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:02.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f1008b5f700 1 -- 172.21.15.140:0/1329155083 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974555035 0 0) 0x7f10040b2840 con 0x7f1004060ca0 2022-07-21T01:55:02.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f1008b5f700 1 -- 172.21.15.140:0/1329155083 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fec003260 con 0x7f1004060ca0 2022-07-21T01:55:02.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f1008b5f700 1 -- 172.21.15.140:0/1329155083 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147678737 0 0) 0x7f100414a750 con 0x7f1004097a80 2022-07-21T01:55:02.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f1008b5f700 1 -- 172.21.15.140:0/1329155083 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10040b2840 con 0x7f1004097a80 2022-07-21T01:55:02.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100a362700 1 -- 172.21.15.140:0/1329155083 >> v1:172.21.15.152:6789/0 conn(0x7f1004060ca0 legacy=0x7f100414ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:02.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100a362700 1 -- 172.21.15.140:0/1329155083 >> v1:172.21.15.140:6789/0 conn(0x7f1004097a80 legacy=0x7f1004146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:02.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100a362700 1 -- 172.21.15.140:0/1329155083 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10040ac110 con 0x7f10040afb80 2022-07-21T01:55:02.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100a362700 1 --2- 172.21.15.140:0/1329155083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10040afb80 0x7f1004061080 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f1000002f70 tx=0x7f1000006df0).ready entity=mon.1 client_cookie=3c444ee3601df961 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:02.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f1008b5f700 1 -- 172.21.15.140:0/1329155083 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f100000d540 con 0x7f10040afb80 2022-07-21T01:55:02.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f1008b5f700 1 -- 172.21.15.140:0/1329155083 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1000000e90 con 0x7f10040afb80 2022-07-21T01:55:02.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f1008b5f700 1 -- 172.21.15.140:0/1329155083 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f100000d890 con 0x7f10040afb80 2022-07-21T01:55:02.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100bdc5700 1 -- 172.21.15.140:0/1329155083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10040afb80 msgr2=0x7f1004061080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:02.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100bdc5700 1 --2- 172.21.15.140:0/1329155083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10040afb80 0x7f1004061080 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f1000002f70 tx=0x7f1000006df0).stop 2022-07-21T01:55:02.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100bdc5700 1 -- 172.21.15.140:0/1329155083 shutdown_connections 2022-07-21T01:55:02.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100bdc5700 1 --2- 172.21.15.140:0/1329155083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10040afb80 0x7f1004061080 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:02.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100bdc5700 1 -- 172.21.15.140:0/1329155083 >> 172.21.15.140:0/1329155083 conn(0x7f10040b5cb0 msgr2=0x7f10040b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:02.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100bdc5700 1 -- 172.21.15.140:0/1329155083 shutdown_connections 2022-07-21T01:55:02.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100bdc5700 1 -- 172.21.15.140:0/1329155083 wait complete. 2022-07-21T01:55:02.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100bdc5700 1 Processor -- start 2022-07-21T01:55:02.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.942+0000 7f100bdc5700 1 -- start start 2022-07-21T01:55:02.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f100bdc5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1004060ca0 0x7f10041598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:02.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f100bdc5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1004097a80 0x7f1004149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:02.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f100bdc5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10040afb80 0x7f10040acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:02.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f100bdc5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100409a7c0 con 0x7f1004060ca0 2022-07-21T01:55:02.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f100bdc5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100409aa20 con 0x7f1004097a80 2022-07-21T01:55:02.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f100bdc5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10040aca00 con 0x7f10040afb80 2022-07-21T01:55:02.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f1009b61700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1004060ca0 0x7f10041598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:02.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f1009b61700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1004060ca0 0x7f10041598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36900/0 (socket says 172.21.15.140:36900) 2022-07-21T01:55:02.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f1009b61700 1 -- 172.21.15.140:0/1963899444 learned_addr learned my addr 172.21.15.140:0/1963899444 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:02.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f100a362700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10040afb80 0x7f10040acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:02.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f1009360700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1004097a80 0x7f1004149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:02.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f1009b61700 1 -- 172.21.15.140:0/1963899444 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10040afb80 msgr2=0x7f10040acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:02.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f1009b61700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10040afb80 0x7f10040acea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:02.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f1009b61700 1 -- 172.21.15.140:0/1963899444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1004097a80 msgr2=0x7f1004149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:02.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f1009b61700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1004097a80 0x7f1004149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:02.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f1009b61700 1 -- 172.21.15.140:0/1963899444 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100000b180 con 0x7f1004060ca0 2022-07-21T01:55:02.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f1009b61700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1004060ca0 0x7f10041598d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f0ffc001490 tx=0x7f0ffc007e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:02.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f0ff2ffd700 1 -- 172.21.15.140:0/1963899444 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ffc00e4a0 con 0x7f1004060ca0 2022-07-21T01:55:02.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f0ff2ffd700 1 -- 172.21.15.140:0/1963899444 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc00e600 con 0x7f1004060ca0 2022-07-21T01:55:02.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f100bdc5700 1 -- 172.21.15.140:0/1963899444 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100414a140 con 0x7f1004060ca0 2022-07-21T01:55:02.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f0ff2ffd700 1 -- 172.21.15.140:0/1963899444 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ffc00e960 con 0x7f1004060ca0 2022-07-21T01:55:02.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f100bdc5700 1 -- 172.21.15.140:0/1963899444 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1004063bb0 con 0x7f1004060ca0 2022-07-21T01:55:02.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.946+0000 7f100bdc5700 1 -- 172.21.15.140:0/1963899444 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10040acbc0 con 0x7f1004060ca0 2022-07-21T01:55:02.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.950+0000 7f0ff2ffd700 1 -- 172.21.15.140:0/1963899444 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0ffc019030 con 0x7f1004060ca0 2022-07-21T01:55:02.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.950+0000 7f0ff2ffd700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fe4053a70 0x7f0fe4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:02.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.950+0000 7f0ff2ffd700 1 -- 172.21.15.140:0/1963899444 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f0ffc06b670 con 0x7f1004060ca0 2022-07-21T01:55:02.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.950+0000 7f1009360700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fe4053a70 0x7f0fe4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:02.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.950+0000 7f1009360700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fe4053a70 0x7f0fe4055f30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f0ff400d000 tx=0x7f0ff4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:02.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:02.954+0000 7f0ff2ffd700 1 -- 172.21.15.140:0/1963899444 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0ffc0424c0 con 0x7f1004060ca0 2022-07-21T01:55:03.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 -- 172.21.15.140:0/1963899444 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist ls", "format": "json"} v 0) v1 -- 0x7f10040574e0 con 0x7f1004060ca0 2022-07-21T01:55:03.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f0ff2ffd700 1 -- 172.21.15.140:0/1963899444 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls", "format": "json"}]=0 listed 1 entries v74) v1 ==== 98+0+81 (secure 0 0 0) 0x7f0ffc045ae0 con 0x7f1004060ca0 2022-07-21T01:55:03.090 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:03.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 -- 172.21.15.140:0/1963899444 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fe4053a70 msgr2=0x7f0fe4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fe4053a70 0x7f0fe4055f30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f0ff400d000 tx=0x7f0ff4009040).stop 2022-07-21T01:55:03.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 -- 172.21.15.140:0/1963899444 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1004060ca0 msgr2=0x7f10041598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1004060ca0 0x7f10041598d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f0ffc001490 tx=0x7f0ffc007e80).stop 2022-07-21T01:55:03.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 -- 172.21.15.140:0/1963899444 shutdown_connections 2022-07-21T01:55:03.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fe4053a70 0x7f0fe4055f30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f10040afb80 0x7f10040acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1004097a80 0x7f1004149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 --2- 172.21.15.140:0/1963899444 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1004060ca0 0x7f10041598d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 -- 172.21.15.140:0/1963899444 >> 172.21.15.140:0/1963899444 conn(0x7f10040b5cb0 msgr2=0x7f10040b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:03.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 -- 172.21.15.140:0/1963899444 shutdown_connections 2022-07-21T01:55:03.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.090+0000 7f100bdc5700 1 -- 172.21.15.140:0/1963899444 wait complete. 2022-07-21T01:55:03.096 INFO:teuthology.orchestra.run.smithi140.stderr:listed 1 entries 2022-07-21T01:55:03.103 INFO:teuthology.orchestra.run.smithi140.stdout:[{"addr":"172.21.15.191:0/1019294232","until":"2022-07-21T02:54:56.780716+0000"}] 2022-07-21T01:55:03.119 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2022-07-21T01:55:03.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:03.120 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:03.121 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:03.128 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2022-07-21T01:55:03.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:03.130 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-07-21T01:55:03.131 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:55:03.131 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:55:03.131 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:03.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:03.191 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:03.191 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:03.202 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:55:03.202 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:55:03.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:55:03.204 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:03.211 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:03.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88f10e2700 1 Processor -- start 2022-07-21T01:55:03.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88f10e2700 1 -- start start 2022-07-21T01:55:03.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88f10e2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f88ec0adff0 0x7f88ec0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88f10e2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec147010 con 0x7f88ec0af8e0 2022-07-21T01:55:03.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88f10e2700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec1472b0 con 0x7f88ec0b5650 2022-07-21T01:55:03.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88f10e2700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ec146d40 con 0x7f88ec0adff0 2022-07-21T01:55:03.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88eb7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f88ec0adff0 0x7f88ec0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88eb7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f88ec0adff0 0x7f88ec0b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36908/0 (socket says 172.21.15.140:36908) 2022-07-21T01:55:03.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88eb7fe700 1 -- 172.21.15.140:0/3304484038 learned_addr learned my addr 172.21.15.140:0/3304484038 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:03.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88eb7fe700 1 -- 172.21.15.140:0/3304484038 >> v1:172.21.15.152:6789/0 conn(0x7f88ec0b5650 legacy=0x7f88ec147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88eb7fe700 1 -- 172.21.15.140:0/3304484038 >> v1:172.21.15.140:6789/0 conn(0x7f88ec0af8e0 legacy=0x7f88ec05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88eaffd700 1 -- 172.21.15.140:0/3304484038 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337218750 0 0) 0x7f88ec147010 con 0x7f88ec0af8e0 2022-07-21T01:55:03.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88eb7fe700 1 -- 172.21.15.140:0/3304484038 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec05c4a0 con 0x7f88ec0adff0 2022-07-21T01:55:03.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88eb7fe700 1 --2- 172.21.15.140:0/3304484038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88ec0adff0 0x7f88ec0b5a30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f88d4002f70 tx=0x7f88d4006e50).ready entity=mon.0 client_cookie=e482cacf7b9a8b03 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:03.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88eaffd700 1 -- 172.21.15.140:0/3304484038 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88d400d660 con 0x7f88ec0adff0 2022-07-21T01:55:03.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88eaffd700 1 -- 172.21.15.140:0/3304484038 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d4000e90 con 0x7f88ec0adff0 2022-07-21T01:55:03.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.322+0000 7f88eaffd700 1 -- 172.21.15.140:0/3304484038 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88d400d9b0 con 0x7f88ec0adff0 2022-07-21T01:55:03.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 -- 172.21.15.140:0/3304484038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88ec0adff0 msgr2=0x7f88ec0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 --2- 172.21.15.140:0/3304484038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88ec0adff0 0x7f88ec0b5a30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f88d4002f70 tx=0x7f88d4006e50).stop 2022-07-21T01:55:03.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 -- 172.21.15.140:0/3304484038 shutdown_connections 2022-07-21T01:55:03.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 --2- 172.21.15.140:0/3304484038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88ec0adff0 0x7f88ec0b5a30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 -- 172.21.15.140:0/3304484038 >> 172.21.15.140:0/3304484038 conn(0x7f88ec0634e0 msgr2=0x7f88ec0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:03.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 -- 172.21.15.140:0/3304484038 shutdown_connections 2022-07-21T01:55:03.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 -- 172.21.15.140:0/3304484038 wait complete. 2022-07-21T01:55:03.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 Processor -- start 2022-07-21T01:55:03.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 -- start start 2022-07-21T01:55:03.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88ec0adff0 0x7f88ec10e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ec0af8e0 0x7f88ec14a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88ec0b5650 0x7f88ec14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec14abc0 con 0x7f88ec0b5650 2022-07-21T01:55:03.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec156620 con 0x7f88ec0af8e0 2022-07-21T01:55:03.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f10e2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec1568c0 con 0x7f88ec0adff0 2022-07-21T01:55:03.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f08e1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88ec0b5650 0x7f88ec14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f08e1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88ec0b5650 0x7f88ec14a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36918/0 (socket says 172.21.15.140:36918) 2022-07-21T01:55:03.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88f08e1700 1 -- 172.21.15.140:0/2430198298 learned_addr learned my addr 172.21.15.140:0/2430198298 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:03.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88eb7fe700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ec0af8e0 0x7f88ec14a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.326+0000 7f88ebfff700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88ec0adff0 0x7f88ec10e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88f08e1700 1 -- 172.21.15.140:0/2430198298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88ec0adff0 msgr2=0x7f88ec10e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88f08e1700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88ec0adff0 0x7f88ec10e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88f08e1700 1 -- 172.21.15.140:0/2430198298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ec0af8e0 msgr2=0x7f88ec14a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88f08e1700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ec0af8e0 0x7f88ec14a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88f08e1700 1 -- 172.21.15.140:0/2430198298 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d400b1e0 con 0x7f88ec0b5650 2022-07-21T01:55:03.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88ebfff700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88ec0adff0 0x7f88ec10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:03.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88f08e1700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88ec0b5650 0x7f88ec14a6d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f88e4001250 tx=0x7f88e4007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:03.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88e97fa700 1 -- 172.21.15.140:0/2430198298 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e400e4a0 con 0x7f88ec0b5650 2022-07-21T01:55:03.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88e97fa700 1 -- 172.21.15.140:0/2430198298 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e400e600 con 0x7f88ec0b5650 2022-07-21T01:55:03.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88e97fa700 1 -- 172.21.15.140:0/2430198298 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e400e960 con 0x7f88ec0b5650 2022-07-21T01:55:03.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88f10e2700 1 -- 172.21.15.140:0/2430198298 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ec156b60 con 0x7f88ec0b5650 2022-07-21T01:55:03.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88f10e2700 1 -- 172.21.15.140:0/2430198298 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ec156310 con 0x7f88ec0b5650 2022-07-21T01:55:03.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88f10e2700 1 -- 172.21.15.140:0/2430198298 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ec156cf0 con 0x7f88ec0b5650 2022-07-21T01:55:03.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88e97fa700 1 -- 172.21.15.140:0/2430198298 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f88e4010070 con 0x7f88ec0b5650 2022-07-21T01:55:03.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88e97fa700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88c40539e0 0x7f88c4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88e97fa700 1 -- 172.21.15.140:0/2430198298 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f88e406b310 con 0x7f88ec0b5650 2022-07-21T01:55:03.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.330+0000 7f88ebfff700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88c40539e0 0x7f88c4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.334+0000 7f88ebfff700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88c40539e0 0x7f88c4055ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f88dc00b000 tx=0x7f88dc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:03.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.334+0000 7f88e97fa700 1 -- 172.21.15.140:0/2430198298 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f88e4042200 con 0x7f88ec0b5650 2022-07-21T01:55:03.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.462+0000 7f88f10e2700 1 -- 172.21.15.140:0/2430198298 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88ec0574e0 con 0x7f88ec0b5650 2022-07-21T01:55:03.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.462+0000 7f88e97fa700 1 -- 172.21.15.140:0/2430198298 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16199 (secure 0 0 0) 0x7f88e4045820 con 0x7f88ec0b5650 2022-07-21T01:55:03.465 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:03.466 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":74,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:54:57.078135+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:54:09.143212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2022-07-21T01:54:09.749431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.191:0/1019294232":"2022-07-21T02:54:56.780716+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:03.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 -- 172.21.15.140:0/2430198298 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88c40539e0 msgr2=0x7f88c4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88c40539e0 0x7f88c4055ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f88dc00b000 tx=0x7f88dc009040).stop 2022-07-21T01:55:03.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 -- 172.21.15.140:0/2430198298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88ec0b5650 msgr2=0x7f88ec14a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88ec0b5650 0x7f88ec14a6d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f88e4001250 tx=0x7f88e4007dc0).stop 2022-07-21T01:55:03.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 -- 172.21.15.140:0/2430198298 shutdown_connections 2022-07-21T01:55:03.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88c40539e0 0x7f88c4055ea0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88ec0b5650 0x7f88ec14a6d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ec0af8e0 0x7f88ec14a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 --2- 172.21.15.140:0/2430198298 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88ec0adff0 0x7f88ec10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 -- 172.21.15.140:0/2430198298 >> 172.21.15.140:0/2430198298 conn(0x7f88ec0634e0 msgr2=0x7f88ec1460f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:03.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 -- 172.21.15.140:0/2430198298 shutdown_connections 2022-07-21T01:55:03.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.466+0000 7f88f10e2700 1 -- 172.21.15.140:0/2430198298 wait complete. 2022-07-21T01:55:03.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:54:09.143212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:54:09.749431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:55:03.483 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:55:03.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.590+0000 7fe33ef37700 1 Processor -- start 2022-07-21T01:55:03.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.590+0000 7fe33ef37700 1 -- start start 2022-07-21T01:55:03.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.590+0000 7fe33ef37700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe33805cf40 0x7fe3380ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.590+0000 7fe33ef37700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3380b3870 0x7fe338140f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.590+0000 7fe33ef37700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe338140aa0 con 0x7fe338062ec0 2022-07-21T01:55:03.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.590+0000 7fe33ef37700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe338144000 con 0x7fe33805cf40 2022-07-21T01:55:03.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.590+0000 7fe33ef37700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380b1ce0 con 0x7fe3380b3870 2022-07-21T01:55:03.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.590+0000 7fe33ccd3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe33805cf40 0x7fe3380ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.590+0000 7fe33ccd3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe33805cf40 0x7fe3380ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36922/0 (socket says 172.21.15.140:36922) 2022-07-21T01:55:03.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.590+0000 7fe33ccd3700 1 -- 172.21.15.140:0/3014086443 learned_addr learned my addr 172.21.15.140:0/3014086443 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:03.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.590+0000 7fe337fff700 1 --2- 172.21.15.140:0/3014086443 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3380b3870 0x7fe338140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ccd3700 1 -- 172.21.15.140:0/3014086443 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3380b3870 msgr2=0x7fe338140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:03.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ccd3700 1 --2- 172.21.15.140:0/3014086443 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3380b3870 0x7fe338140f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ccd3700 1 -- 172.21.15.140:0/3014086443 >> v1:172.21.15.152:6789/0 conn(0x7fe338062ec0 legacy=0x7fe3381442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ccd3700 1 -- 172.21.15.140:0/3014086443 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3380b1e20 con 0x7fe33805cf40 2022-07-21T01:55:03.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ccd3700 1 --2- 172.21.15.140:0/3014086443 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe33805cf40 0x7fe3380ad260 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7fe3280032f0 tx=0x7fe328009290).ready entity=mon.0 client_cookie=598fbf88f291db11 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:03.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe3377fe700 1 -- 172.21.15.140:0/3014086443 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe32800b8a0 con 0x7fe33805cf40 2022-07-21T01:55:03.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe3377fe700 1 -- 172.21.15.140:0/3014086443 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32800ba00 con 0x7fe33805cf40 2022-07-21T01:55:03.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe3377fe700 1 -- 172.21.15.140:0/3014086443 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe32800bb70 con 0x7fe33805cf40 2022-07-21T01:55:03.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ef37700 1 -- 172.21.15.140:0/3014086443 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe33805cf40 msgr2=0x7fe3380ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ef37700 1 --2- 172.21.15.140:0/3014086443 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe33805cf40 0x7fe3380ad260 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7fe3280032f0 tx=0x7fe328009290).stop 2022-07-21T01:55:03.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ef37700 1 -- 172.21.15.140:0/3014086443 shutdown_connections 2022-07-21T01:55:03.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ef37700 1 --2- 172.21.15.140:0/3014086443 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3380b3870 0x7fe338140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ef37700 1 --2- 172.21.15.140:0/3014086443 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe33805cf40 0x7fe3380ad260 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ef37700 1 -- 172.21.15.140:0/3014086443 >> 172.21.15.140:0/3014086443 conn(0x7fe3380b7d30 msgr2=0x7fe3380b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:03.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ef37700 1 -- 172.21.15.140:0/3014086443 shutdown_connections 2022-07-21T01:55:03.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ef37700 1 -- 172.21.15.140:0/3014086443 wait complete. 2022-07-21T01:55:03.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ef37700 1 Processor -- start 2022-07-21T01:55:03.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.594+0000 7fe33ef37700 1 -- start start 2022-07-21T01:55:03.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe33ef37700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe33805cf40 0x7fe33813ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe33ef37700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe338062ec0 0x7fe33814ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe33ef37700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3380b3870 0x7fe338153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe33ef37700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33809cd90 con 0x7fe338062ec0 2022-07-21T01:55:03.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe33ef37700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3381535f0 con 0x7fe33805cf40 2022-07-21T01:55:03.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe33ef37700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33809cfc0 con 0x7fe3380b3870 2022-07-21T01:55:03.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe337fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe338062ec0 0x7fe33814ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe337fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe338062ec0 0x7fe33814ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36930/0 (socket says 172.21.15.140:36930) 2022-07-21T01:55:03.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe337fff700 1 -- 172.21.15.140:0/3382368594 learned_addr learned my addr 172.21.15.140:0/3382368594 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:03.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe33ccd3700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe33805cf40 0x7fe33813ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe33d4d4700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3380b3870 0x7fe338153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe337fff700 1 -- 172.21.15.140:0/3382368594 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3380b3870 msgr2=0x7fe338153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe337fff700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3380b3870 0x7fe338153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe337fff700 1 -- 172.21.15.140:0/3382368594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe33805cf40 msgr2=0x7fe33813ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe337fff700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe33805cf40 0x7fe33813ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe337fff700 1 -- 172.21.15.140:0/3382368594 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe328009530 con 0x7fe338062ec0 2022-07-21T01:55:03.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe33d4d4700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3380b3870 0x7fe338153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:03.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe337fff700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe338062ec0 0x7fe33814ffe0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7fe32000c000 tx=0x7fe320006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:03.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe335ffb700 1 -- 172.21.15.140:0/3382368594 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe320010780 con 0x7fe338062ec0 2022-07-21T01:55:03.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe335ffb700 1 -- 172.21.15.140:0/3382368594 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3200108e0 con 0x7fe338062ec0 2022-07-21T01:55:03.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe33ef37700 1 -- 172.21.15.140:0/3382368594 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe338153850 con 0x7fe338062ec0 2022-07-21T01:55:03.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe335ffb700 1 -- 172.21.15.140:0/3382368594 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe320010c40 con 0x7fe338062ec0 2022-07-21T01:55:03.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.598+0000 7fe33ef37700 1 -- 172.21.15.140:0/3382368594 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe338150550 con 0x7fe338062ec0 2022-07-21T01:55:03.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.602+0000 7fe33ef37700 1 -- 172.21.15.140:0/3382368594 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3380574e0 con 0x7fe338062ec0 2022-07-21T01:55:03.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.602+0000 7fe335ffb700 1 -- 172.21.15.140:0/3382368594 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe32001c460 con 0x7fe338062ec0 2022-07-21T01:55:03.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.602+0000 7fe335ffb700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe31c053950 0x7fe31c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.602+0000 7fe335ffb700 1 -- 172.21.15.140:0/3382368594 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6116+0+0 (secure 0 0 0) 0x7fe32006c3c0 con 0x7fe338062ec0 2022-07-21T01:55:03.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.602+0000 7fe33ccd3700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe31c053950 0x7fe31c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.602+0000 7fe33ccd3700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe31c053950 0x7fe31c055e10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fe32801d000 tx=0x7fe328016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:03.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.610+0000 7fe335ffb700 1 -- 172.21.15.140:0/3382368594 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe320043250 con 0x7fe338062ec0 2022-07-21T01:55:03.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.738+0000 7fe33ef37700 1 -- 172.21.15.140:0/3382368594 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe3380b4b40 con 0x7fe338062ec0 2022-07-21T01:55:03.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.738+0000 7fe335ffb700 1 -- 172.21.15.140:0/3382368594 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v74) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe320046870 con 0x7fe338062ec0 2022-07-21T01:55:03.738 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:55:03.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.738+0000 7fe33ef37700 1 -- 172.21.15.140:0/3382368594 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe31c053950 msgr2=0x7fe31c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.738+0000 7fe33ef37700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe31c053950 0x7fe31c055e10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fe32801d000 tx=0x7fe328016000).stop 2022-07-21T01:55:03.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.738+0000 7fe33ef37700 1 -- 172.21.15.140:0/3382368594 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe338062ec0 msgr2=0x7fe33814ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.738+0000 7fe33ef37700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe338062ec0 0x7fe33814ffe0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7fe32000c000 tx=0x7fe320006df0).stop 2022-07-21T01:55:03.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.742+0000 7fe33ef37700 1 -- 172.21.15.140:0/3382368594 shutdown_connections 2022-07-21T01:55:03.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.742+0000 7fe33ef37700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe31c053950 0x7fe31c055e10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.742+0000 7fe33ef37700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3380b3870 0x7fe338153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.742+0000 7fe33ef37700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe338062ec0 0x7fe33814ffe0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.742+0000 7fe33ef37700 1 --2- 172.21.15.140:0/3382368594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe33805cf40 0x7fe33813ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.742+0000 7fe33ef37700 1 -- 172.21.15.140:0/3382368594 >> 172.21.15.140:0/3382368594 conn(0x7fe3380b7d30 msgr2=0x7fe3380b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:03.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.742+0000 7fe33ef37700 1 -- 172.21.15.140:0/3382368594 shutdown_connections 2022-07-21T01:55:03.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.742+0000 7fe33ef37700 1 -- 172.21.15.140:0/3382368594 wait complete. 2022-07-21T01:55:03.756 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:55:03.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.862+0000 7ff3cb392700 1 Processor -- start 2022-07-21T01:55:03.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.862+0000 7ff3cb392700 1 -- start start 2022-07-21T01:55:03.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff3c405d160 0x7ff3c40b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3c41461c0 0x7ff3c41465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c40b3f10 con 0x7ff3c40b2ec0 2022-07-21T01:55:03.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c40b4ea0 con 0x7ff3c41461c0 2022-07-21T01:55:03.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c40b3790 con 0x7ff3c405d160 2022-07-21T01:55:03.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cab91700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3c41461c0 0x7ff3c41465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cab91700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3c41461c0 0x7ff3c41465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36940/0 (socket says 172.21.15.140:36940) 2022-07-21T01:55:03.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cab91700 1 -- 172.21.15.140:0/2734172844 learned_addr learned my addr 172.21.15.140:0/2734172844 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:03.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3c9b8f700 1 --2- 172.21.15.140:0/2734172844 >> v2:172.21.15.152:3300/0 conn(0x7ff3c405d160 0x7ff3c40b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cab91700 1 -- 172.21.15.140:0/2734172844 >> v2:172.21.15.152:3300/0 conn(0x7ff3c405d160 msgr2=0x7ff3c40b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:03.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cab91700 1 --2- 172.21.15.140:0/2734172844 >> v2:172.21.15.152:3300/0 conn(0x7ff3c405d160 0x7ff3c40b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cab91700 1 -- 172.21.15.140:0/2734172844 >> v1:172.21.15.152:6789/0 conn(0x7ff3c40b2ec0 legacy=0x7ff3c4142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cab91700 1 -- 172.21.15.140:0/2734172844 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c40b2620 con 0x7ff3c41461c0 2022-07-21T01:55:03.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cab91700 1 --2- 172.21.15.140:0/2734172844 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c41461c0 0x7ff3c41465f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7ff3bc0032f0 tx=0x7ff3bc009290).ready entity=mon.0 client_cookie=36bd5fa1baae4647 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:03.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3c938e700 1 -- 172.21.15.140:0/2734172844 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3bc00b870 con 0x7ff3c41461c0 2022-07-21T01:55:03.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3c938e700 1 -- 172.21.15.140:0/2734172844 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc00b9d0 con 0x7ff3c41461c0 2022-07-21T01:55:03.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3c938e700 1 -- 172.21.15.140:0/2734172844 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3bc00bb40 con 0x7ff3c41461c0 2022-07-21T01:55:03.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 -- 172.21.15.140:0/2734172844 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c41461c0 msgr2=0x7ff3c41465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 --2- 172.21.15.140:0/2734172844 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c41461c0 0x7ff3c41465f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7ff3bc0032f0 tx=0x7ff3bc009290).stop 2022-07-21T01:55:03.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 -- 172.21.15.140:0/2734172844 shutdown_connections 2022-07-21T01:55:03.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 --2- 172.21.15.140:0/2734172844 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c41461c0 0x7ff3c41465f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 --2- 172.21.15.140:0/2734172844 >> v2:172.21.15.152:3300/0 conn(0x7ff3c405d160 0x7ff3c40b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 -- 172.21.15.140:0/2734172844 >> 172.21.15.140:0/2734172844 conn(0x7ff3c4066e20 msgr2=0x7ff3c4067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:03.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 -- 172.21.15.140:0/2734172844 shutdown_connections 2022-07-21T01:55:03.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.866+0000 7ff3cb392700 1 -- 172.21.15.140:0/2734172844 wait complete. 2022-07-21T01:55:03.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3cb392700 1 Processor -- start 2022-07-21T01:55:03.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3cb392700 1 -- start start 2022-07-21T01:55:03.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3cb392700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c405d160 0x7ff3c4145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3cb392700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c40b2ec0 0x7ff3c413e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3cb392700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3c41461c0 0x7ff3c41440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3cb392700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c41523a0 con 0x7ff3c40b2ec0 2022-07-21T01:55:03.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3cb392700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40b8870 con 0x7ff3c405d160 2022-07-21T01:55:03.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3cb392700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40b8b90 con 0x7ff3c41461c0 2022-07-21T01:55:03.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3c9b8f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c40b2ec0 0x7ff3c413e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3c9b8f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c40b2ec0 0x7ff3c413e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36944/0 (socket says 172.21.15.140:36944) 2022-07-21T01:55:03.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3ca390700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c405d160 0x7ff3c4145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3c9b8f700 1 -- 172.21.15.140:0/2676865419 learned_addr learned my addr 172.21.15.140:0/2676865419 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:03.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.870+0000 7ff3cab91700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3c41461c0 0x7ff3c41440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3cab91700 1 -- 172.21.15.140:0/2676865419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c405d160 msgr2=0x7ff3c4145560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3cab91700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c405d160 0x7ff3c4145560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3cab91700 1 -- 172.21.15.140:0/2676865419 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c40b2ec0 msgr2=0x7ff3c413e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:03.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3cab91700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c40b2ec0 0x7ff3c413e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:03.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3cab91700 1 -- 172.21.15.140:0/2676865419 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc009530 con 0x7ff3c41461c0 2022-07-21T01:55:03.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3c9b8f700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c40b2ec0 0x7ff3c413e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:03.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3ca390700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c405d160 0x7ff3c4145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:03.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3cab91700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3c41461c0 0x7ff3c41440c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7ff3bc0032f0 tx=0x7ff3bc016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:03.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3b37fe700 1 -- 172.21.15.140:0/2676865419 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3bc00b870 con 0x7ff3c41461c0 2022-07-21T01:55:03.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3b37fe700 1 -- 172.21.15.140:0/2676865419 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc021440 con 0x7ff3c41461c0 2022-07-21T01:55:03.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3cb392700 1 -- 172.21.15.140:0/2676865419 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c40b8e30 con 0x7ff3c41461c0 2022-07-21T01:55:03.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3b37fe700 1 -- 172.21.15.140:0/2676865419 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3bc0217a0 con 0x7ff3c41461c0 2022-07-21T01:55:03.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3cb392700 1 -- 172.21.15.140:0/2676865419 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c40b84f0 con 0x7ff3c41461c0 2022-07-21T01:55:03.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3cb392700 1 -- 172.21.15.140:0/2676865419 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c40b8ff0 con 0x7ff3c41461c0 2022-07-21T01:55:03.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3b37fe700 1 -- 172.21.15.140:0/2676865419 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff3bc00f070 con 0x7ff3c41461c0 2022-07-21T01:55:03.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.874+0000 7ff3b37fe700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3a40539e0 0x7ff3a4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:03.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.878+0000 7ff3b37fe700 1 -- 172.21.15.140:0/2676865419 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6116+0+0 (secure 0 0 0) 0x7ff3bc06d380 con 0x7ff3c41461c0 2022-07-21T01:55:03.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.878+0000 7ff3ca390700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3a40539e0 0x7ff3a4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:03.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.878+0000 7ff3ca390700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3a40539e0 0x7ff3a4055ea0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7ff3c000d000 tx=0x7ff3c0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:03.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:03.886+0000 7ff3b37fe700 1 -- 172.21.15.140:0/2676865419 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff3bc044160 con 0x7ff3c41461c0 2022-07-21T01:55:04.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.010+0000 7ff3cb392700 1 -- 172.21.15.140:0/2676865419 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff3c40b28c0 con 0x7ff3c41461c0 2022-07-21T01:55:04.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3b37fe700 1 -- 172.21.15.140:0/2676865419 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff3bc015020 con 0x7ff3c41461c0 2022-07-21T01:55:04.014 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:04.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 -- 172.21.15.140:0/2676865419 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3a40539e0 msgr2=0x7ff3a4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3a40539e0 0x7ff3a4055ea0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7ff3c000d000 tx=0x7ff3c0009040).stop 2022-07-21T01:55:04.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 -- 172.21.15.140:0/2676865419 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3c41461c0 msgr2=0x7ff3c41440c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3c41461c0 0x7ff3c41440c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7ff3bc0032f0 tx=0x7ff3bc016000).stop 2022-07-21T01:55:04.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 -- 172.21.15.140:0/2676865419 shutdown_connections 2022-07-21T01:55:04.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3a40539e0 0x7ff3a4055ea0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3c41461c0 0x7ff3c41440c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c40b2ec0 0x7ff3c413e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 --2- 172.21.15.140:0/2676865419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c405d160 0x7ff3c4145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 -- 172.21.15.140:0/2676865419 >> 172.21.15.140:0/2676865419 conn(0x7ff3c4066e20 msgr2=0x7ff3c40b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:04.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 -- 172.21.15.140:0/2676865419 shutdown_connections 2022-07-21T01:55:04.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.014+0000 7ff3cb392700 1 -- 172.21.15.140:0/2676865419 wait complete. 2022-07-21T01:55:04.029 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:55:04.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.134+0000 7faec4ad6700 1 Processor -- start 2022-07-21T01:55:04.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.134+0000 7faec4ad6700 1 -- start start 2022-07-21T01:55:04.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7faec00b9da0 0x7faec01468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faec00b3cf0 0x7faec0142a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faec0142f40 0x7faec005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec00ad5b0 con 0x7faec00b3cf0 2022-07-21T01:55:04.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec00ba180 con 0x7faec00b9da0 2022-07-21T01:55:04.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec01461b0 con 0x7faec0142f40 2022-07-21T01:55:04.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebeffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faec00b3cf0 0x7faec0142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebeffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faec00b3cf0 0x7faec0142a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36952/0 (socket says 172.21.15.140:36952) 2022-07-21T01:55:04.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebeffd700 1 -- 172.21.15.140:0/917701074 learned_addr learned my addr 172.21.15.140:0/917701074 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:04.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebffff700 1 --2- 172.21.15.140:0/917701074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faec0142f40 0x7faec005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebf7fe700 1 --2- 172.21.15.140:0/917701074 >> v2:172.21.15.152:3300/0 conn(0x7faec00b9da0 0x7faec01468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebeffd700 1 -- 172.21.15.140:0/917701074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faec0142f40 msgr2=0x7faec005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:04.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebeffd700 1 --2- 172.21.15.140:0/917701074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faec0142f40 0x7faec005bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebeffd700 1 -- 172.21.15.140:0/917701074 >> v2:172.21.15.152:3300/0 conn(0x7faec00b9da0 msgr2=0x7faec01468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:04.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebeffd700 1 --2- 172.21.15.140:0/917701074 >> v2:172.21.15.152:3300/0 conn(0x7faec00b9da0 0x7faec01468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebeffd700 1 -- 172.21.15.140:0/917701074 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec0146450 con 0x7faec00b3cf0 2022-07-21T01:55:04.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebffff700 1 --2- 172.21.15.140:0/917701074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faec0142f40 0x7faec005bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:04.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebeffd700 1 --2- 172.21.15.140:0/917701074 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faec00b3cf0 0x7faec0142a50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7faea8002f70 tx=0x7faea8006eb0).ready entity=mon.0 client_cookie=596dcf938af7dd77 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:04.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebe7fc700 1 -- 172.21.15.140:0/917701074 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faea800d570 con 0x7faec00b3cf0 2022-07-21T01:55:04.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebe7fc700 1 -- 172.21.15.140:0/917701074 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea8000e90 con 0x7faec00b3cf0 2022-07-21T01:55:04.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faebe7fc700 1 -- 172.21.15.140:0/917701074 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faea800d900 con 0x7faec00b3cf0 2022-07-21T01:55:04.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 -- 172.21.15.140:0/917701074 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faec00b3cf0 msgr2=0x7faec0142a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 --2- 172.21.15.140:0/917701074 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faec00b3cf0 0x7faec0142a50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7faea8002f70 tx=0x7faea8006eb0).stop 2022-07-21T01:55:04.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 -- 172.21.15.140:0/917701074 shutdown_connections 2022-07-21T01:55:04.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 --2- 172.21.15.140:0/917701074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faec0142f40 0x7faec005bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 --2- 172.21.15.140:0/917701074 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faec00b3cf0 0x7faec0142a50 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 --2- 172.21.15.140:0/917701074 >> v2:172.21.15.152:3300/0 conn(0x7faec00b9da0 0x7faec01468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 -- 172.21.15.140:0/917701074 >> 172.21.15.140:0/917701074 conn(0x7faec00678c0 msgr2=0x7faec00b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:04.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 -- 172.21.15.140:0/917701074 shutdown_connections 2022-07-21T01:55:04.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 -- 172.21.15.140:0/917701074 wait complete. 2022-07-21T01:55:04.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 Processor -- start 2022-07-21T01:55:04.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.138+0000 7faec4ad6700 1 -- start start 2022-07-21T01:55:04.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faec4ad6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faec00b3cf0 0x7faec00b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faec4ad6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faec00b9da0 0x7faec00b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faec4ad6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faec0142f40 0x7faec0154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faec4ad6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00b0d70 con 0x7faec00b3cf0 2022-07-21T01:55:04.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faec4ad6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00b0880 con 0x7faec0142f40 2022-07-21T01:55:04.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faec4ad6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec00b0ad0 con 0x7faec00b9da0 2022-07-21T01:55:04.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebf7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faec00b3cf0 0x7faec00b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebf7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faec00b3cf0 0x7faec00b8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36956/0 (socket says 172.21.15.140:36956) 2022-07-21T01:55:04.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebf7fe700 1 -- 172.21.15.140:0/4076031632 learned_addr learned my addr 172.21.15.140:0/4076031632 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:04.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebeffd700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faec00b9da0 0x7faec00b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebffff700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faec0142f40 0x7faec0154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebf7fe700 1 -- 172.21.15.140:0/4076031632 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faec00b9da0 msgr2=0x7faec00b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebf7fe700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faec00b9da0 0x7faec00b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebf7fe700 1 -- 172.21.15.140:0/4076031632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faec0142f40 msgr2=0x7faec0154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebf7fe700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faec0142f40 0x7faec0154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebf7fe700 1 -- 172.21.15.140:0/4076031632 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea800b1e0 con 0x7faec00b3cf0 2022-07-21T01:55:04.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebf7fe700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faec00b3cf0 0x7faec00b8ad0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7faeb000c000 tx=0x7faeb0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:04.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebcff9700 1 -- 172.21.15.140:0/4076031632 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faeb0010780 con 0x7faec00b3cf0 2022-07-21T01:55:04.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebcff9700 1 -- 172.21.15.140:0/4076031632 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb00108e0 con 0x7faec00b3cf0 2022-07-21T01:55:04.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faec4ad6700 1 -- 172.21.15.140:0/4076031632 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec0113570 con 0x7faec00b3cf0 2022-07-21T01:55:04.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebcff9700 1 -- 172.21.15.140:0/4076031632 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faeb0010c40 con 0x7faec00b3cf0 2022-07-21T01:55:04.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faec4ad6700 1 -- 172.21.15.140:0/4076031632 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec00b2370 con 0x7faec00b3cf0 2022-07-21T01:55:04.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faec4ad6700 1 -- 172.21.15.140:0/4076031632 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec00574e0 con 0x7faec00b3cf0 2022-07-21T01:55:04.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebcff9700 1 -- 172.21.15.140:0/4076031632 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faeb001c460 con 0x7faec00b3cf0 2022-07-21T01:55:04.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebcff9700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faea4053aa0 0x7faea4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.142+0000 7faebcff9700 1 -- 172.21.15.140:0/4076031632 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6116+0+0 (secure 0 0 0) 0x7faeb006cf90 con 0x7faec00b3cf0 2022-07-21T01:55:04.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.146+0000 7faebeffd700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faea4053aa0 0x7faea4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.146+0000 7faebeffd700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faea4053aa0 0x7faea4055f60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7faea801a000 tx=0x7faea8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:04.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.150+0000 7faebcff9700 1 -- 172.21.15.140:0/4076031632 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faeb0043e20 con 0x7faec00b3cf0 2022-07-21T01:55:04.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 -- 172.21.15.140:0/4076031632 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faec00699b0 con 0x7faec00b3cf0 2022-07-21T01:55:04.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faebcff9700 1 -- 172.21.15.140:0/4076031632 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+10 (secure 0 0 0) 0x7faeb0047440 con 0x7faec00b3cf0 2022-07-21T01:55:04.274 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:04.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 -- 172.21.15.140:0/4076031632 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faea4053aa0 msgr2=0x7faea4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faea4053aa0 0x7faea4055f60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7faea801a000 tx=0x7faea8015000).stop 2022-07-21T01:55:04.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 -- 172.21.15.140:0/4076031632 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faec00b3cf0 msgr2=0x7faec00b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faec00b3cf0 0x7faec00b8ad0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7faeb000c000 tx=0x7faeb0006df0).stop 2022-07-21T01:55:04.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 -- 172.21.15.140:0/4076031632 shutdown_connections 2022-07-21T01:55:04.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faea4053aa0 0x7faea4055f60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faec0142f40 0x7faec0154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faec00b9da0 0x7faec00b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 --2- 172.21.15.140:0/4076031632 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faec00b3cf0 0x7faec00b8ad0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 -- 172.21.15.140:0/4076031632 >> 172.21.15.140:0/4076031632 conn(0x7faec00678c0 msgr2=0x7faec005bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:04.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 -- 172.21.15.140:0/4076031632 shutdown_connections 2022-07-21T01:55:04.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.274+0000 7faec4ad6700 1 -- 172.21.15.140:0/4076031632 wait complete. 2022-07-21T01:55:04.293 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:55:04.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.402+0000 7f78abfff700 1 Processor -- start 2022-07-21T01:55:04.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.402+0000 7f78abfff700 1 -- start start 2022-07-21T01:55:04.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac0b59d0 0x7f78ac05d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f78ac128d70 0x7f78ac13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac0b5db0 con 0x7f78ac0af180 2022-07-21T01:55:04.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ac0ad800 con 0x7f78ac128d70 2022-07-21T01:55:04.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0adaa0 con 0x7f78ac0b59d0 2022-07-21T01:55:04.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78ab7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f78ac128d70 0x7f78ac13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78aaffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac0b59d0 0x7f78ac05d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78ab7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f78ac128d70 0x7f78ac13fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42708/0 (socket says 172.21.15.140:42708) 2022-07-21T01:55:04.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78aaffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac0b59d0 0x7f78ac05d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:33694/0 (socket says 172.21.15.140:33694) 2022-07-21T01:55:04.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78ab7fe700 1 -- 172.21.15.140:0/559595056 learned_addr learned my addr 172.21.15.140:0/559595056 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:04.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78aa7fc700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f78ac0af180 0x7f78ac128660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:52224/0 (socket says 172.21.15.140:52224) 2022-07-21T01:55:04.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78aaffd700 1 -- 172.21.15.140:0/559595056 >> v2:172.21.15.152:3300/0 conn(0x7f78ac128d70 msgr2=0x7f78ac13fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:04.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78aaffd700 1 --2- 172.21.15.140:0/559595056 >> v2:172.21.15.152:3300/0 conn(0x7f78ac128d70 0x7f78ac13fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78aaffd700 1 -- 172.21.15.140:0/559595056 >> v1:172.21.15.152:6789/0 conn(0x7f78ac0af180 legacy=0x7f78ac128660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78aaffd700 1 -- 172.21.15.140:0/559595056 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac0ad4f0 con 0x7f78ac0b59d0 2022-07-21T01:55:04.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78ab7fe700 1 --2- 172.21.15.140:0/559595056 >> v2:172.21.15.152:3300/0 conn(0x7f78ac128d70 0x7f78ac13fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:04.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78aaffd700 1 --2- 172.21.15.140:0/559595056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac0b59d0 0x7f78ac05d1c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f789c002f70 tx=0x7f789c006df0).ready entity=mon.2 client_cookie=26ae4549dbd54eb8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:04.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78a9ffb700 1 -- 172.21.15.140:0/559595056 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f789c00d570 con 0x7f78ac0b59d0 2022-07-21T01:55:04.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78a9ffb700 1 -- 172.21.15.140:0/559595056 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c000e90 con 0x7f78ac0b59d0 2022-07-21T01:55:04.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78a9ffb700 1 -- 172.21.15.140:0/559595056 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f789c00d8c0 con 0x7f78ac0b59d0 2022-07-21T01:55:04.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 -- 172.21.15.140:0/559595056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac0b59d0 msgr2=0x7f78ac05d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 --2- 172.21.15.140:0/559595056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac0b59d0 0x7f78ac05d1c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f789c002f70 tx=0x7f789c006df0).stop 2022-07-21T01:55:04.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 -- 172.21.15.140:0/559595056 shutdown_connections 2022-07-21T01:55:04.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 --2- 172.21.15.140:0/559595056 >> v2:172.21.15.152:3300/0 conn(0x7f78ac128d70 0x7f78ac13fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 --2- 172.21.15.140:0/559595056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac0b59d0 0x7f78ac05d1c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 -- 172.21.15.140:0/559595056 >> 172.21.15.140:0/559595056 conn(0x7f78ac0634c0 msgr2=0x7f78ac0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:04.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.406+0000 7f78abfff700 1 -- 172.21.15.140:0/559595056 shutdown_connections 2022-07-21T01:55:04.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78abfff700 1 -- 172.21.15.140:0/559595056 wait complete. 2022-07-21T01:55:04.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78abfff700 1 Processor -- start 2022-07-21T01:55:04.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78abfff700 1 -- start start 2022-07-21T01:55:04.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78abfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78ac0af180 0x7f78ac1263a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78abfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0b59d0 0x7f78ac0b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78abfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac128d70 0x7f78ac127550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78abfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac140bc0 con 0x7f78ac0af180 2022-07-21T01:55:04.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78abfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac140df0 con 0x7f78ac0b59d0 2022-07-21T01:55:04.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78abfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac141090 con 0x7f78ac128d70 2022-07-21T01:55:04.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78aaffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78ac0af180 0x7f78ac1263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78aaffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78ac0af180 0x7f78ac1263a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36970/0 (socket says 172.21.15.140:36970) 2022-07-21T01:55:04.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78aaffd700 1 -- 172.21.15.140:0/2167139891 learned_addr learned my addr 172.21.15.140:0/2167139891 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:04.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78aa7fc700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0b59d0 0x7f78ac0b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.410+0000 7f78ab7fe700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac128d70 0x7f78ac127550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f78aaffd700 1 -- 172.21.15.140:0/2167139891 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac128d70 msgr2=0x7f78ac127550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f78aaffd700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac128d70 0x7f78ac127550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f78aaffd700 1 -- 172.21.15.140:0/2167139891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0b59d0 msgr2=0x7f78ac0b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f78aaffd700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0b59d0 0x7f78ac0b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f78aaffd700 1 -- 172.21.15.140:0/2167139891 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a4009530 con 0x7f78ac0af180 2022-07-21T01:55:04.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f78aaffd700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78ac0af180 0x7f78ac1263a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f789c002f70 tx=0x7f789c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:04.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f789bfff700 1 -- 172.21.15.140:0/2167139891 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f789c00dea0 con 0x7f78ac0af180 2022-07-21T01:55:04.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f789bfff700 1 -- 172.21.15.140:0/2167139891 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c013de0 con 0x7f78ac0af180 2022-07-21T01:55:04.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f78abfff700 1 -- 172.21.15.140:0/2167139891 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c00b180 con 0x7f78ac0af180 2022-07-21T01:55:04.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f789bfff700 1 -- 172.21.15.140:0/2167139891 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f789c01e5f0 con 0x7f78ac0af180 2022-07-21T01:55:04.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f78abfff700 1 -- 172.21.15.140:0/2167139891 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac142670 con 0x7f78ac0af180 2022-07-21T01:55:04.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f78abfff700 1 -- 172.21.15.140:0/2167139891 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ac125320 con 0x7f78ac0af180 2022-07-21T01:55:04.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.414+0000 7f789bfff700 1 -- 172.21.15.140:0/2167139891 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f789c025030 con 0x7f78ac0af180 2022-07-21T01:55:04.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.418+0000 7f789bfff700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7888053a10 0x7f7888055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.418+0000 7f789bfff700 1 -- 172.21.15.140:0/2167139891 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f789c06cbc0 con 0x7f78ac0af180 2022-07-21T01:55:04.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.418+0000 7f78aa7fc700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7888053a10 0x7f7888055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.418+0000 7f78aa7fc700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7888053a10 0x7f7888055ed0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f78a000d000 tx=0x7f78a0007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:04.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.422+0000 7f789bfff700 1 -- 172.21.15.140:0/2167139891 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f789c043ab0 con 0x7f78ac0af180 2022-07-21T01:55:04.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.546+0000 7f78abfff700 1 -- 172.21.15.140:0/2167139891 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78ac0574e0 con 0x7f78ac0af180 2022-07-21T01:55:04.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f789bfff700 1 -- 172.21.15.140:0/2167139891 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+10 (secure 0 0 0) 0x7f789c023070 con 0x7f78ac0af180 2022-07-21T01:55:04.549 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:04.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 -- 172.21.15.140:0/2167139891 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7888053a10 msgr2=0x7f7888055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7888053a10 0x7f7888055ed0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f78a000d000 tx=0x7f78a0007b80).stop 2022-07-21T01:55:04.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 -- 172.21.15.140:0/2167139891 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78ac0af180 msgr2=0x7f78ac1263a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78ac0af180 0x7f78ac1263a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f789c002f70 tx=0x7f789c014000).stop 2022-07-21T01:55:04.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 -- 172.21.15.140:0/2167139891 shutdown_connections 2022-07-21T01:55:04.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7888053a10 0x7f7888055ed0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78ac128d70 0x7f78ac127550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78ac0b59d0 0x7f78ac0b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 --2- 172.21.15.140:0/2167139891 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78ac0af180 0x7f78ac1263a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 -- 172.21.15.140:0/2167139891 >> 172.21.15.140:0/2167139891 conn(0x7f78ac0634c0 msgr2=0x7f78ac145880 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:04.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 -- 172.21.15.140:0/2167139891 shutdown_connections 2022-07-21T01:55:04.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.550+0000 7f78abfff700 1 -- 172.21.15.140:0/2167139891 wait complete. 2022-07-21T01:55:04.562 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:55:04.562 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:04.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a030d6700 1 Processor -- start 2022-07-21T01:55:04.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a030d6700 1 -- start start 2022-07-21T01:55:04.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a030d6700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f49fc0adff0 0x7f49fc0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a030d6700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49fc147010 con 0x7f49fc0af8e0 2022-07-21T01:55:04.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a030d6700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49fc1472b0 con 0x7f49fc0b5650 2022-07-21T01:55:04.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a030d6700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49fc146d40 con 0x7f49fc0adff0 2022-07-21T01:55:04.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a028d5700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f49fc0af8e0 0x7f49fc05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51126/0 (socket says 172.21.15.140:51126) 2022-07-21T01:55:04.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a028d5700 1 -- 172.21.15.140:0/582050572 learned_addr learned my addr 172.21.15.140:0/582050572 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:04.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a018d3700 1 --2- 172.21.15.140:0/582050572 >> v2:172.21.15.152:3300/0 conn(0x7f49fc0adff0 0x7f49fc0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a010d2700 1 -- 172.21.15.140:0/582050572 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536800223 0 0) 0x7f49fc147010 con 0x7f49fc0af8e0 2022-07-21T01:55:04.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a010d2700 1 -- 172.21.15.140:0/582050572 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49dc003260 con 0x7f49fc0af8e0 2022-07-21T01:55:04.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a010d2700 1 -- 172.21.15.140:0/582050572 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280321392 0 0) 0x7f49fc1472b0 con 0x7f49fc0b5650 2022-07-21T01:55:04.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.662+0000 7f4a010d2700 1 -- 172.21.15.140:0/582050572 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49fc147010 con 0x7f49fc0b5650 2022-07-21T01:55:04.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a018d3700 1 -- 172.21.15.140:0/582050572 >> v1:172.21.15.152:6789/0 conn(0x7f49fc0b5650 legacy=0x7f49fc147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a018d3700 1 -- 172.21.15.140:0/582050572 >> v1:172.21.15.140:6789/0 conn(0x7f49fc0af8e0 legacy=0x7f49fc05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a018d3700 1 -- 172.21.15.140:0/582050572 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc05c4a0 con 0x7f49fc0adff0 2022-07-21T01:55:04.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a018d3700 1 --2- 172.21.15.140:0/582050572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49fc0adff0 0x7f49fc0b5a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f49f40032f0 tx=0x7f49f40091a0).ready entity=mon.1 client_cookie=c53a5705dd6852e2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:04.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a010d2700 1 -- 172.21.15.140:0/582050572 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49f400b600 con 0x7f49fc0adff0 2022-07-21T01:55:04.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a010d2700 1 -- 172.21.15.140:0/582050572 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f400b760 con 0x7f49fc0adff0 2022-07-21T01:55:04.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a010d2700 1 -- 172.21.15.140:0/582050572 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49f400b8d0 con 0x7f49fc0adff0 2022-07-21T01:55:04.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a030d6700 1 -- 172.21.15.140:0/582050572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49fc0adff0 msgr2=0x7f49fc0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a030d6700 1 --2- 172.21.15.140:0/582050572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49fc0adff0 0x7f49fc0b5a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f49f40032f0 tx=0x7f49f40091a0).stop 2022-07-21T01:55:04.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a030d6700 1 -- 172.21.15.140:0/582050572 shutdown_connections 2022-07-21T01:55:04.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a030d6700 1 --2- 172.21.15.140:0/582050572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49fc0adff0 0x7f49fc0b5a30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a030d6700 1 -- 172.21.15.140:0/582050572 >> 172.21.15.140:0/582050572 conn(0x7f49fc0634e0 msgr2=0x7f49fc0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:04.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a030d6700 1 -- 172.21.15.140:0/582050572 shutdown_connections 2022-07-21T01:55:04.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.666+0000 7f4a030d6700 1 -- 172.21.15.140:0/582050572 wait complete. 2022-07-21T01:55:04.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a030d6700 1 Processor -- start 2022-07-21T01:55:04.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a030d6700 1 -- start start 2022-07-21T01:55:04.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a030d6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49fc0adff0 0x7f49fc10fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a030d6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49fc0af8e0 0x7f49fc110c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a030d6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49fc0b5650 0x7f49fc064440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a030d6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc10f6c0 con 0x7f49fc0b5650 2022-07-21T01:55:04.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a030d6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc063fe0 con 0x7f49fc0adff0 2022-07-21T01:55:04.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a030d6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc110100 con 0x7f49fc0af8e0 2022-07-21T01:55:04.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a028d5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49fc0b5650 0x7f49fc064440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a028d5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49fc0b5650 0x7f49fc064440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:36986/0 (socket says 172.21.15.140:36986) 2022-07-21T01:55:04.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a028d5700 1 -- 172.21.15.140:0/3555768117 learned_addr learned my addr 172.21.15.140:0/3555768117 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:04.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a018d3700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49fc0af8e0 0x7f49fc110c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a020d4700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49fc0adff0 0x7f49fc10fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a028d5700 1 -- 172.21.15.140:0/3555768117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49fc0af8e0 msgr2=0x7f49fc110c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a028d5700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49fc0af8e0 0x7f49fc110c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a028d5700 1 -- 172.21.15.140:0/3555768117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49fc0adff0 msgr2=0x7f49fc10fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a028d5700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49fc0adff0 0x7f49fc10fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a028d5700 1 -- 172.21.15.140:0/3555768117 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f4009420 con 0x7f49fc0b5650 2022-07-21T01:55:04.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a020d4700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49fc0adff0 0x7f49fc10fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:04.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a028d5700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49fc0b5650 0x7f49fc064440 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f49e4001110 tx=0x7f49e4007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:04.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f49fb7fe700 1 -- 172.21.15.140:0/3555768117 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49e40070d0 con 0x7f49fc0b5650 2022-07-21T01:55:04.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a030d6700 1 -- 172.21.15.140:0/3555768117 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49fc1103a0 con 0x7f49fc0b5650 2022-07-21T01:55:04.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f49fb7fe700 1 -- 172.21.15.140:0/3555768117 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e4007230 con 0x7f49fc0b5650 2022-07-21T01:55:04.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f49fb7fe700 1 -- 172.21.15.140:0/3555768117 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f49e4005610 con 0x7f49fc0b5650 2022-07-21T01:55:04.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.670+0000 7f4a030d6700 1 -- 172.21.15.140:0/3555768117 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc110600 con 0x7f49fc0b5650 2022-07-21T01:55:04.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.674+0000 7f4a030d6700 1 -- 172.21.15.140:0/3555768117 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49fc10ff50 con 0x7f49fc0b5650 2022-07-21T01:55:04.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.674+0000 7f49fb7fe700 1 -- 172.21.15.140:0/3555768117 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f49e4010070 con 0x7f49fc0b5650 2022-07-21T01:55:04.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.674+0000 7f49fb7fe700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f49e00539e0 0x7f49e0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:04.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.674+0000 7f4a020d4700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f49e00539e0 0x7f49e0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:04.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.674+0000 7f49fb7fe700 1 -- 172.21.15.140:0/3555768117 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f49e406b770 con 0x7f49fc0b5650 2022-07-21T01:55:04.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.674+0000 7f4a020d4700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f49e00539e0 0x7f49e0055ea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f49ec00b000 tx=0x7f49ec009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:04.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.682+0000 7f49fb7fe700 1 -- 172.21.15.140:0/3555768117 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f49e40424d0 con 0x7f49fc0b5650 2022-07-21T01:55:04.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.842+0000 7f4a030d6700 1 -- 172.21.15.140:0/3555768117 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49fc0574e0 con 0x7f49fc0b5650 2022-07-21T01:55:04.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.842+0000 7f49fb7fe700 1 -- 172.21.15.140:0/3555768117 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f49e4045af0 con 0x7f49fc0b5650 2022-07-21T01:55:04.844 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:04.844 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":30,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/4173658089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4173658089},{"type":"v1","addr":"172.21.15.152:6833","nonce":4173658089}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:54:10.830421+0000","modified":"2022-07-21T01:54:11.841578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6208":{"gid":6208,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/167323020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":167323020},{"type":"v1","addr":"172.21.15.140:6835","nonce":167323020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-07-21T01:55:04.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 -- 172.21.15.140:0/3555768117 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f49e00539e0 msgr2=0x7f49e0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f49e00539e0 0x7f49e0055ea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f49ec00b000 tx=0x7f49ec009040).stop 2022-07-21T01:55:04.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 -- 172.21.15.140:0/3555768117 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49fc0b5650 msgr2=0x7f49fc064440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:04.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49fc0b5650 0x7f49fc064440 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f49e4001110 tx=0x7f49e4007a80).stop 2022-07-21T01:55:04.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 -- 172.21.15.140:0/3555768117 shutdown_connections 2022-07-21T01:55:04.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f49e00539e0 0x7f49e0055ea0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f49fc0b5650 0x7f49fc064440 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49fc0af8e0 0x7f49fc110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 --2- 172.21.15.140:0/3555768117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49fc0adff0 0x7f49fc10fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:04.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 -- 172.21.15.140:0/3555768117 >> 172.21.15.140:0/3555768117 conn(0x7f49fc0634e0 msgr2=0x7f49fc0b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:04.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 -- 172.21.15.140:0/3555768117 shutdown_connections 2022-07-21T01:55:04.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:04.846+0000 7f4a030d6700 1 -- 172.21.15.140:0/3555768117 wait complete. 2022-07-21T01:55:04.853 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 30 2022-07-21T01:55:05.063 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:54:10.830421+0000', 'modified': '2022-07-21T01:54:11.841578+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6208}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6208': {'gid': 6208, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/167323020', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 167323020}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 167323020}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:55:05.064 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:55:05.065 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:55:05.179 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":6208,"want_state":"up:active","state":"up:active","rank_uptime":54.328294053999997,"mdsmap_epoch":30,"osdmap_epoch":74,"osdmap_epoch_barrier":74,"uptime":58.400073034000002} 2022-07-21T01:55:05.179 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 6208, "mdsmap_epoch": 30, "osdmap_epoch": 74, "osdmap_epoch_barrier": 74, "rank_uptime": 54.328294054, "state": "up:active", "uptime": 58.400073034, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:55:05.180 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:55:05.180 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:55:05.181 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:55:05.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:05.182 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:05.190 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:55:05.205 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:05.206 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:05.217 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:05.218 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:05.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:05.220 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:55:05.226 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:55:05.226 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:05.270 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:55:05.287 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:05.287 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:05.296 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:05.297 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:05.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:05.299 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:55:05.343 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22878]: starting ceph client 2022-07-21T01:55:05.344 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:55:05.344+0000 7f3f498cf080 -1 init, newargv = 0x55a4514e1210 newargc=9 2022-07-21T01:55:05.352 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22878]: starting fuse 2022-07-21T01:55:06.447 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:55:06.467 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:06.468 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:06.480 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:06.481 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:06.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:06.484 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:55:06.531 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:55:06.531 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:55:06.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:06.532 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:55:06.533 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:55:06.533 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:55:06.533 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:55:06.534 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:55:06.534 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:06.535 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:55:06.535 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:55:06.536 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:55:06.536 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:06.536 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:55:06.537 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:55:06.537 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:55:06.538 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:55:06.538 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:06.539 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:55:06.539 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:55:06.539 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:55:06.540 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:55:06.540 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:55:06.541 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:06.541 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:55:06.541 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:55:06.587 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.22878.asok 2022-07-21T01:55:06.593 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22878.asok 2022-07-21T01:55:06.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:06.595 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22878.asok status 2022-07-21T01:55:06.682 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:55:06.683 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:55:06.683 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:55:06.684 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:55:06.684 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:55:06.685 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:55:06.685 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:55:06.685 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "22878", 2022-07-21T01:55:06.686 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:55:06.686 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:06.687 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:55:06.687 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:55:06.688 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 5912, 2022-07-21T01:55:06.688 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:55:06.688 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:55:06.689 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:55:06.689 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 5912 2022-07-21T01:55:06.689 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:06.689 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:55:06.690 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:55:06.690 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:55:06.690 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3908948240 2022-07-21T01:55:06.690 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:55:06.691 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:06.691 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:55:06.691 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:55:06.691 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:55:06.692 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 3908948240 2022-07-21T01:55:06.692 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:06.692 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.5912 172.21.15.191:0/3908948240", 2022-07-21T01:55:06.693 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/3908948240", 2022-07-21T01:55:06.693 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:55:06.693 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 30, 2022-07-21T01:55:06.694 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 74, 2022-07-21T01:55:06.694 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:55:06.694 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:55:06.694 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:55:06.695 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:55:06.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:06.697 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:06.697 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:06.704 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:55:06.705 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:55:06.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:06.706 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:06.726 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:55:06.726 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:06.746 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:06.768 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:06.787 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:06.807 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:06.828 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:55:06.829 INFO:tasks.cephfs.mount:Terminating background process 2022-07-21T01:55:06.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:06.829 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:06.830 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:06.838 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:55:06.840 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:55:06.840 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:55:06.840 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:55:06.841 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:06.857 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22878]: fuse finished with error 0 and tester_r 0 2022-07-21T01:55:06.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:06.859 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:06.860 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:06.907 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:55:06.909 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:55:06.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:55:06.940 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:06.948 INFO:tasks.cephfs.mount:Terminating background process 2022-07-21T01:55:06.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:06.949 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:06.950 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:06.959 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:55:06.960 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:55:06.961 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2022-07-21T01:55:06.961 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:55:06.962 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:06.978 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22371]: fuse finished with error 0 and tester_r 0 2022-07-21T01:55:06.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:06.981 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:06.982 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:06.990 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2022-07-21T01:55:06.991 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:55:07.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:55:07.109 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:07.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:07.117 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:07.118 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:55:07.125 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:55:07.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:07.127 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:55:07.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:55:07.128 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:55:07.135 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:55:07.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:07.137 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:55:07.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3067282700 1 Processor -- start 2022-07-21T01:55:07.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3067282700 1 -- start start 2022-07-21T01:55:07.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3067282700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600adff0 0x7f30600b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3067282700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f30600af8e0 0x7f306005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3067282700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3060147010 con 0x7f30600b5650 2022-07-21T01:55:07.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3067282700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30601472b0 con 0x7f30600af8e0 2022-07-21T01:55:07.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3067282700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3060146d40 con 0x7f30600adff0 2022-07-21T01:55:07.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3066280700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f30600b5650 0x7f3060147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51136/0 (socket says 172.21.15.140:51136) 2022-07-21T01:55:07.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3066280700 1 -- 172.21.15.140:0/1108600808 learned_addr learned my addr 172.21.15.140:0/1108600808 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:07.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3066a81700 1 --2- 172.21.15.140:0/1108600808 >> v2:172.21.15.152:3300/0 conn(0x7f30600af8e0 0x7f306005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3065a7f700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600adff0 0x7f30600b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529700917 0 0) 0x7f3060147010 con 0x7f30600b5650 2022-07-21T01:55:07.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3048003260 con 0x7f30600b5650 2022-07-21T01:55:07.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1476093595 0 0) 0x7f3048003260 con 0x7f30600b5650 2022-07-21T01:55:07.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600adff0 msgr2=0x7f30600b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:07.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600adff0 0x7f30600b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 >> v2:172.21.15.152:3300/0 conn(0x7f30600af8e0 msgr2=0x7f306005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:07.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 --2- 172.21.15.140:0/1108600808 >> v2:172.21.15.152:3300/0 conn(0x7f30600af8e0 0x7f306005bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30600acae0 con 0x7f30600b5650 2022-07-21T01:55:07.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f3050001660 con 0x7f30600b5650 2022-07-21T01:55:07.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 >> v1:172.21.15.140:6789/0 conn(0x7f30600b5650 legacy=0x7f3060147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600adff0 0x7f3048003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3048003c80 0x7f3048004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30480049f0 0x7f3048004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30601472b0 con 0x7f3048003c80 2022-07-21T01:55:07.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3060146d40 con 0x7f30480049f0 2022-07-21T01:55:07.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304800b750 con 0x7f30600adff0 2022-07-21T01:55:07.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3065a7f700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3048003c80 0x7f3048004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3066280700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600adff0 0x7f3048003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3066a81700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30480049f0 0x7f3048004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3065a7f700 1 -- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600adff0 msgr2=0x7f3048003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:07.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3065a7f700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30600adff0 0x7f3048003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3065a7f700 1 -- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30480049f0 msgr2=0x7f3048004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:07.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3065a7f700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30480049f0 0x7f3048004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3065a7f700 1 -- 172.21.15.140:0/1108600808 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3048003260 con 0x7f3048003c80 2022-07-21T01:55:07.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3066a81700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30480049f0 0x7f3048004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:55:07.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f3065a7f700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3048003c80 0x7f3048004500 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f30480054d0 tx=0x7f305c006d90).ready entity=mon.0 client_cookie=7e0d17d122fae09a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:07.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f305c013450 con 0x7f3048003c80 2022-07-21T01:55:07.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c0135b0 con 0x7f3048003c80 2022-07-21T01:55:07.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.246+0000 7f306527e700 1 -- 172.21.15.140:0/1108600808 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f305c013910 con 0x7f3048003c80 2022-07-21T01:55:07.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 -- 172.21.15.140:0/1108600808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3048003c80 msgr2=0x7f3048004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 --2- 172.21.15.140:0/1108600808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3048003c80 0x7f3048004500 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f30480054d0 tx=0x7f305c006d90).stop 2022-07-21T01:55:07.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3066a81700 1 -- 172.21.15.140:0/1108600808 reap_dead start 2022-07-21T01:55:07.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 -- 172.21.15.140:0/1108600808 shutdown_connections 2022-07-21T01:55:07.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 -- 172.21.15.140:0/1108600808 >> 172.21.15.140:0/1108600808 conn(0x7f30600634e0 msgr2=0x7f30600b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:07.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 -- 172.21.15.140:0/1108600808 shutdown_connections 2022-07-21T01:55:07.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 -- 172.21.15.140:0/1108600808 wait complete. 2022-07-21T01:55:07.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 Processor -- start 2022-07-21T01:55:07.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 -- start start 2022-07-21T01:55:07.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3048003c80 0x7f3060063da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3060111810 0x7f3060064940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3060159980 0x7f3060159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3060064290 con 0x7f3060159980 2022-07-21T01:55:07.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3060065120 con 0x7f3060111810 2022-07-21T01:55:07.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30600674a0 con 0x7f3048003c80 2022-07-21T01:55:07.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3066a81700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3060159980 0x7f3060159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3066a81700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3060159980 0x7f3060159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37006/0 (socket says 172.21.15.140:37006) 2022-07-21T01:55:07.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3066a81700 1 -- 172.21.15.140:0/2439626597 learned_addr learned my addr 172.21.15.140:0/2439626597 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:07.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3065a7f700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3060111810 0x7f3060064940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3066280700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3048003c80 0x7f3060063da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3066a81700 1 -- 172.21.15.140:0/2439626597 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3048003c80 msgr2=0x7f3060063da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3066a81700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3048003c80 0x7f3060063da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3066a81700 1 -- 172.21.15.140:0/2439626597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3060111810 msgr2=0x7f3060064940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3066a81700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3060111810 0x7f3060064940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3066a81700 1 -- 172.21.15.140:0/2439626597 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3060064e30 con 0x7f3060159980 2022-07-21T01:55:07.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3065a7f700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3060111810 0x7f3060064940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:07.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3066a81700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3060159980 0x7f3060159e30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f305800b000 tx=0x7f3058008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:07.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f304f7fe700 1 -- 172.21.15.140:0/2439626597 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3058019070 con 0x7f3060159980 2022-07-21T01:55:07.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f304f7fe700 1 -- 172.21.15.140:0/2439626597 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3058014810 con 0x7f3060159980 2022-07-21T01:55:07.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 -- 172.21.15.140:0/2439626597 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3060111430 con 0x7f3060159980 2022-07-21T01:55:07.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f304f7fe700 1 -- 172.21.15.140:0/2439626597 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3058014b70 con 0x7f3060159980 2022-07-21T01:55:07.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.250+0000 7f3067282700 1 -- 172.21.15.140:0/2439626597 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30600678c0 con 0x7f3060159980 2022-07-21T01:55:07.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.254+0000 7f3067282700 1 -- 172.21.15.140:0/2439626597 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30600b5a30 con 0x7f3060159980 2022-07-21T01:55:07.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.254+0000 7f304f7fe700 1 -- 172.21.15.140:0/2439626597 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3058012040 con 0x7f3060159980 2022-07-21T01:55:07.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.254+0000 7f304f7fe700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3040053a10 0x7f3040055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.254+0000 7f304f7fe700 1 -- 172.21.15.140:0/2439626597 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f305806c0b0 con 0x7f3060159980 2022-07-21T01:55:07.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.254+0000 7f3066280700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3040053a10 0x7f3040055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.254+0000 7f3066280700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3040053a10 0x7f3040055ed0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f305000d000 tx=0x7f3050009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:07.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.258+0000 7f304f7fe700 1 -- 172.21.15.140:0/2439626597 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3058042f00 con 0x7f3060159980 2022-07-21T01:55:07.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.386+0000 7f3067282700 1 -- 172.21.15.140:0/2439626597 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f30600574e0 con 0x7f3060159980 2022-07-21T01:55:07.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.386+0000 7f304f7fe700 1 -- 172.21.15.140:0/2439626597 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v74) v1 ==== 81+0+33821 (secure 0 0 0) 0x7f3058046520 con 0x7f3060159980 2022-07-21T01:55:07.389 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:07.389 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:55:07.389 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 74, 2022-07-21T01:55:07.390 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:55:07.390 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:55:07.390 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:54:57.078135+0000", 2022-07-21T01:55:07.390 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:55:07.391 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:55:07.391 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:55:07.391 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:55:07.391 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:55:07.392 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:55:07.392 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:55:07.392 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:55:07.392 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:55:07.393 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:07.393 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:55:07.393 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:55:07.393 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:55:07.393 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:55:07.394 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:55:07.394 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 14, 2022-07-21T01:55:07.394 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:55:07.395 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:55:07.395 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:55:07.395 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:55:07.396 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:55:07.396 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.397 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:55:07.397 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:55:07.398 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:55:07.398 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:07.399 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:07.399 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:07.399 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:07.400 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:07.400 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:07.400 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:07.400 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:07.400 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:55:07.401 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:55:07.401 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:55:07.401 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:55:07.401 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:55:07.402 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:07.402 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:07.402 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:07.402 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:07.403 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:07.403 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:07.403 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:07.403 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.403 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:55:07.404 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:07.404 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:07.404 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:07.404 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:07.404 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:07.405 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:07.405 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:07.405 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:07.405 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:07.406 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:07.406 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:07.406 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:07.406 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:07.407 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:07.407 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:07.407 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:07.407 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:07.407 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:07.408 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:07.408 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:07.408 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:07.408 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:07.408 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:07.409 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:07.409 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:07.409 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:07.409 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.410 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:07.410 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:07.410 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:07.410 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:07.410 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:07.411 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:07.411 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:07.411 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:07.411 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:07.411 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:07.412 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:07.412 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:55:07.412 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:55:07.412 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.413 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:07.413 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:55:07.413 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.413 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.414 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.414 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:55:07.414 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:55:07.415 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:55:07.415 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:07.415 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:07.416 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:07.416 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:07.416 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:07.416 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:07.417 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:07.417 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:07.417 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:55:07.417 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:55:07.417 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:55:07.418 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:55:07.418 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:55:07.418 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:07.418 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:07.419 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:07.419 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:07.419 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:07.419 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:07.419 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:07.420 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.420 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:55:07.420 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:07.420 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:07.420 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:07.421 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:07.421 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:07.421 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:07.421 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:07.422 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:07.422 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:07.422 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:07.422 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:07.423 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:07.423 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:07.423 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:07.423 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:07.424 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:07.424 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:07.424 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:07.424 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:07.425 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:07.425 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:07.425 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:07.426 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:07.426 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:07.426 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:07.426 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:07.427 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.427 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:07.427 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:07.427 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:07.428 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:07.428 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:07.428 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:07.428 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:07.428 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:07.429 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:07.429 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:07.429 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:07.429 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:55:07.430 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:07.430 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:55:07.430 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.430 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.430 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.431 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 13, 2022-07-21T01:55:07.431 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:55:07.432 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:54:09.143212+0000", 2022-07-21T01:55:07.432 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:07.432 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:07.432 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:07.433 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:07.433 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:07.433 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:07.433 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:07.433 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:07.434 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:55:07.434 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:55:07.434 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:55:07.434 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:55:07.435 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:55:07.435 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:07.435 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:07.435 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:07.436 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:07.436 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:07.436 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:07.437 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:07.437 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.437 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "71", 2022-07-21T01:55:07.437 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:07.438 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:07.438 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:07.438 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:07.439 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:07.439 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:07.439 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:07.439 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:07.440 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:07.440 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:07.440 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:07.440 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:07.441 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:07.441 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:07.441 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:07.441 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:07.441 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:07.442 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:07.442 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:07.442 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:07.442 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:07.443 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:07.443 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:07.443 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:07.443 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:07.444 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:07.444 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.444 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:07.444 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:07.444 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:07.445 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:07.445 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:07.445 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:07.445 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:07.446 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:07.446 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:07.446 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:07.446 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:07.446 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:55:07.447 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:55:07.447 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:55:07.447 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:55:07.447 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.448 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:07.448 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:55:07.448 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:55:07.448 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.449 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.449 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.449 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.449 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 14, 2022-07-21T01:55:07.449 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:55:07.450 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:54:09.749431+0000", 2022-07-21T01:55:07.450 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:07.450 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:07.450 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:07.451 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:07.451 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:07.451 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:07.451 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:07.451 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:07.452 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:55:07.452 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:55:07.452 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:55:07.452 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:55:07.453 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:55:07.453 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:07.453 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:07.453 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:07.454 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:07.454 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:07.454 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:07.454 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:07.454 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.455 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "73", 2022-07-21T01:55:07.455 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:07.455 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:07.455 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:07.456 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:07.456 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:07.456 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:07.456 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:07.457 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:07.457 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:07.457 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:07.457 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:07.457 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:07.458 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:07.458 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:07.458 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:07.458 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:07.459 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:07.459 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:07.459 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:07.459 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:07.459 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:07.460 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:07.460 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:07.460 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:07.460 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:07.461 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:07.461 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.461 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:07.461 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:07.461 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:07.462 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:07.462 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:07.462 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:07.462 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:07.463 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:07.463 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:07.463 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:07.463 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:07.463 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:55:07.464 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:07.464 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:55:07.464 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:55:07.464 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.465 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.465 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.465 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:07.465 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:55:07.466 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.466 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:55:07.466 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:55:07.466 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:07.466 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:07.467 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:07.467 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:07.467 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:07.467 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:07.468 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:07.468 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 69, 2022-07-21T01:55:07.468 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:07.468 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:07.468 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:07.469 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.469 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.469 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.469 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:55:07.470 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:07.470 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.470 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.470 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.471 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:55:07.471 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:07.471 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.471 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.471 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.472 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:07.472 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.472 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.472 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.473 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:55:07.473 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:07.473 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.473 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.473 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.474 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:55:07.474 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:07.474 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.474 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.475 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.475 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:07.475 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.475 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.475 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.476 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:55:07.476 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:07.476 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.476 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.477 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.477 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:55:07.477 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:07.477 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.477 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.478 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.478 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:07.478 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.478 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.479 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.479 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:55:07.479 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:07.479 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.480 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.480 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.480 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:55:07.480 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:07.480 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.481 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.481 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.481 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:55:07.481 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:55:07.482 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:55:07.482 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:55:07.482 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:07.482 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:07.482 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:07.483 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.483 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.483 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.483 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:55:07.484 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:55:07.484 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:07.484 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:07.484 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:07.484 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:07.485 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:07.485 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:07.485 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:07.485 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 61, 2022-07-21T01:55:07.486 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:07.486 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:07.486 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:07.486 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.487 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.487 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.487 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:55:07.487 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:07.487 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.488 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.488 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.488 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:55:07.488 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:07.489 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.489 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.489 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.489 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:07.490 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.490 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.490 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.490 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:55:07.490 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:07.491 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.491 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.491 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.491 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:55:07.492 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:07.492 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.492 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.492 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.492 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:07.493 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.493 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.493 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.493 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:55:07.494 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:07.494 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.494 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.494 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.495 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:55:07.495 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:07.495 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.495 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.496 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.496 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:07.496 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.496 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.496 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.497 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:55:07.497 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:07.497 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.497 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.498 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.498 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:55:07.498 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:07.498 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.498 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.499 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.499 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:55:07.499 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:55:07.499 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:55:07.500 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:55:07.500 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:07.500 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:07.500 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:07.500 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.501 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.501 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.501 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:55:07.501 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:55:07.502 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:07.502 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:07.502 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:07.502 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:07.503 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:07.503 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:07.503 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:07.503 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 69, 2022-07-21T01:55:07.503 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:07.504 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:07.504 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:07.504 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.504 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.505 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.505 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:55:07.505 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:07.505 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.505 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.506 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.506 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:55:07.506 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:07.506 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.507 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.507 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.507 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:07.507 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.507 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.508 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.508 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:55:07.508 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:07.508 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.509 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.509 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.509 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:55:07.509 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:07.509 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.510 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.510 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.510 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:07.510 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.511 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.511 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.511 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:55:07.511 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:07.512 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.512 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.512 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.512 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:55:07.512 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:07.513 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.513 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.513 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.513 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:07.514 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.514 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.514 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.514 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:55:07.515 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:07.515 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.515 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.515 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.515 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:55:07.516 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:07.516 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.516 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.516 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.517 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:55:07.517 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:55:07.517 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:55:07.517 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:55:07.517 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:07.518 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:07.518 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:07.518 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.518 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.519 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.519 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:55:07.519 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:55:07.519 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:07.519 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:07.520 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:07.520 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:07.520 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:07.520 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:07.521 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:07.521 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 69, 2022-07-21T01:55:07.521 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:07.521 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:07.521 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:07.522 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.522 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.522 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.522 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:55:07.523 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:07.523 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.523 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.523 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.524 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:55:07.524 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:07.524 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.524 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.524 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.525 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:07.525 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.525 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.525 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.526 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:55:07.526 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:07.526 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.526 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.526 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.527 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:55:07.527 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:07.527 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.527 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.528 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.528 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:07.528 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.528 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.529 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.529 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:55:07.529 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:07.529 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.529 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.530 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.530 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:55:07.530 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:07.530 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.531 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.531 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.531 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:07.531 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.532 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.532 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.532 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:55:07.532 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:07.533 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.533 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.533 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.533 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:55:07.533 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:07.534 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.534 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.534 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.534 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:55:07.535 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:55:07.535 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:55:07.535 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:55:07.535 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:07.535 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:07.536 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:07.536 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.536 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.536 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.537 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:55:07.537 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:55:07.537 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:07.537 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:07.538 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:07.538 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:07.538 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:07.538 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:07.539 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:07.539 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 68, 2022-07-21T01:55:07.539 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:07.539 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:07.540 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:07.540 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.540 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.540 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.540 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:55:07.541 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:07.541 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.541 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.541 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.542 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:55:07.542 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:07.542 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.542 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.542 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.543 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:07.543 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.543 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.543 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.544 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:55:07.544 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:07.544 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.544 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.544 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.545 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:55:07.545 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:07.545 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.545 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.546 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.546 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:07.546 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.546 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.546 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.547 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:55:07.547 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:07.547 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.547 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.548 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.548 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:55:07.548 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:07.548 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.549 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.549 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.549 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:07.549 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.549 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.550 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.550 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:55:07.550 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:07.550 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.551 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.551 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.551 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:55:07.551 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:07.552 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.552 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.552 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.552 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:55:07.552 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:55:07.553 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:55:07.553 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:55:07.553 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:07.553 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:07.554 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:07.554 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.554 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.554 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.554 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:55:07.555 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:55:07.555 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:07.555 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:07.555 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:07.556 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:07.556 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:07.556 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:07.556 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:07.557 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 69, 2022-07-21T01:55:07.557 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:07.557 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:07.557 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:07.557 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.558 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.558 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.558 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:55:07.558 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:07.559 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.559 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.559 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.559 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:55:07.560 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:07.560 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.560 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.560 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.561 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:07.561 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.561 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.561 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.561 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:55:07.562 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:07.562 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.562 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.562 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.563 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:55:07.563 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:07.563 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.563 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.564 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.564 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:07.564 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.564 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.564 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.565 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:55:07.565 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:07.565 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.565 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.566 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.566 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:55:07.566 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:07.566 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.567 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.567 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.567 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:07.567 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.567 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.568 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.568 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:55:07.568 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:07.568 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.569 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.569 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.569 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:55:07.569 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:07.569 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.570 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.570 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.570 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:55:07.570 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:55:07.571 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:55:07.571 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:55:07.571 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:07.571 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:07.572 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:07.572 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.572 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.572 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.572 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:55:07.573 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:55:07.573 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:07.573 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:07.573 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:07.574 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:07.574 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:07.574 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:07.574 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:07.574 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 69, 2022-07-21T01:55:07.575 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:07.575 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:07.575 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:07.575 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.576 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.576 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.576 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:55:07.576 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:07.576 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.577 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.577 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.577 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:55:07.577 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:07.578 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.578 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.578 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.578 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:07.578 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.579 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.579 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.579 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:55:07.579 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:07.580 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.580 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.580 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.580 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:55:07.581 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:07.581 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.581 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.581 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.582 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:07.582 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.582 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.582 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.582 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:55:07.583 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:07.583 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.583 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.583 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.584 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:55:07.584 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:07.584 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.584 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.585 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.585 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:07.585 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.585 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.585 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.586 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:55:07.586 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:07.586 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.586 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.587 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.587 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:55:07.587 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:07.587 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.588 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.588 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.588 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:55:07.588 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:55:07.589 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:55:07.589 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:55:07.589 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:07.589 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:07.589 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:07.590 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.590 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.590 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.590 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:55:07.591 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:55:07.591 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:07.591 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:07.592 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:07.592 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:07.592 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:07.592 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:07.593 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:07.593 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 68, 2022-07-21T01:55:07.593 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:07.593 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:07.593 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:07.594 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.594 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.594 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.594 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:55:07.595 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:07.595 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.595 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.595 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.596 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:55:07.596 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:07.596 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.596 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.596 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.597 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:07.597 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.597 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.597 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.598 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:55:07.598 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:07.598 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.598 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.599 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.599 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:55:07.599 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:07.599 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.600 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.600 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.600 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:07.600 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.600 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.601 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.601 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:55:07.601 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:07.601 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.601 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.602 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.602 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:55:07.602 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:07.602 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.603 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.603 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.603 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:07.603 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:07.604 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.604 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:07.604 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:55:07.604 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:07.604 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.605 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.605 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:07.605 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:55:07.605 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:07.606 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.606 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.606 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.606 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:55:07.606 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:55:07.607 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:55:07.607 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:55:07.607 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:07.607 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:07.608 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:07.608 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:07.608 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.608 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:07.609 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:55:07.609 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.609 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:55:07.609 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:07.609 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:07.610 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:07.610 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:07.610 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:07.610 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:55:07.611 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:07.611 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.611 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.611 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:55:07.612 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:07.612 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:07.612 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:07.612 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:07.612 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:07.613 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:55:07.613 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:07.613 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.613 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.614 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:55:07.614 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:07.614 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:07.614 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:07.615 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:07.615 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:07.615 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:55:07.615 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:07.615 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.616 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.616 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:55:07.616 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:07.616 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:07.617 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:07.617 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:07.617 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:07.617 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:55:07.617 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:07.618 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.618 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.618 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:55:07.618 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:07.619 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:07.619 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:07.619 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:07.619 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:07.620 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:55:07.620 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:07.620 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.620 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.620 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:55:07.621 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:07.621 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:07.621 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:07.621 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:07.621 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:07.622 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:55:07.622 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:07.622 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.622 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.623 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:55:07.623 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:07.623 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:07.623 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:07.624 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:07.624 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:07.624 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:55:07.624 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:07.625 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.625 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:07.625 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:55:07.625 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:07.625 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:07.626 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:07.626 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:07.626 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:07.626 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:55:07.627 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:07.627 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.627 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:07.627 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:55:07.627 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:55:07.628 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:55:07.628 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:55:07.628 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:55:07.628 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.191:0/1019294232": "2022-07-21T02:54:56.780716+0000" 2022-07-21T01:55:07.629 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.629 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:55:07.629 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:55:07.629 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:55:07.630 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:55:07.630 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:55:07.630 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:55:07.630 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:55:07.630 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:55:07.631 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:07.631 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:07.631 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:55:07.631 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:55:07.632 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:55:07.632 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:55:07.632 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:55:07.632 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:55:07.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 -- 172.21.15.140:0/2439626597 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3040053a10 msgr2=0x7f3040055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3040053a10 0x7f3040055ed0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f305000d000 tx=0x7f3050009040).stop 2022-07-21T01:55:07.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 -- 172.21.15.140:0/2439626597 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3060159980 msgr2=0x7f3060159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3060159980 0x7f3060159e30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f305800b000 tx=0x7f3058008040).stop 2022-07-21T01:55:07.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 -- 172.21.15.140:0/2439626597 shutdown_connections 2022-07-21T01:55:07.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3040053a10 0x7f3040055ed0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3060159980 0x7f3060159e30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3060111810 0x7f3060064940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 --2- 172.21.15.140:0/2439626597 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3048003c80 0x7f3060063da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 -- 172.21.15.140:0/2439626597 >> 172.21.15.140:0/2439626597 conn(0x7f30600634e0 msgr2=0x7f306014a250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:07.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 -- 172.21.15.140:0/2439626597 shutdown_connections 2022-07-21T01:55:07.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.390+0000 7f3067282700 1 -- 172.21.15.140:0/2439626597 wait complete. 2022-07-21T01:55:07.638 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:07.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 Processor -- start 2022-07-21T01:55:07.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 -- start start 2022-07-21T01:55:07.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9e0c0adff0 0x7f9e0c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0c147010 con 0x7f9e0c0b5650 2022-07-21T01:55:07.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0c1472b0 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0c146d40 con 0x7f9e0c0adff0 2022-07-21T01:55:07.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e0b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f9e0c0b5650 0x7f9e0c147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51156/0 (socket says 172.21.15.140:51156) 2022-07-21T01:55:07.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e0b7fe700 1 -- 172.21.15.140:0/3946347734 learned_addr learned my addr 172.21.15.140:0/3946347734 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:07.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e03fff700 1 --2- 172.21.15.140:0/3946347734 >> v2:172.21.15.152:3300/0 conn(0x7f9e0c0adff0 0x7f9e0c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e0affd700 1 -- 172.21.15.140:0/3946347734 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103129303 0 0) 0x7f9e0c147010 con 0x7f9e0c0b5650 2022-07-21T01:55:07.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e0affd700 1 -- 172.21.15.140:0/3946347734 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dec003260 con 0x7f9e0c0b5650 2022-07-21T01:55:07.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e03fff700 1 -- 172.21.15.140:0/3946347734 >> v1:172.21.15.152:6789/0 conn(0x7f9e0c0af8e0 legacy=0x7f9e0c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e03fff700 1 -- 172.21.15.140:0/3946347734 >> v1:172.21.15.140:6789/0 conn(0x7f9e0c0b5650 legacy=0x7f9e0c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e03fff700 1 -- 172.21.15.140:0/3946347734 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c05c4a0 con 0x7f9e0c0adff0 2022-07-21T01:55:07.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e03fff700 1 --2- 172.21.15.140:0/3946347734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e0c0adff0 0x7f9e0c0b5a30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f9df4002f70 tx=0x7f9df4006d90).ready entity=mon.1 client_cookie=47ba8a09308fd1c6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:07.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e0affd700 1 -- 172.21.15.140:0/3946347734 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9df400d4a0 con 0x7f9e0c0adff0 2022-07-21T01:55:07.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e0affd700 1 -- 172.21.15.140:0/3946347734 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4000e90 con 0x7f9e0c0adff0 2022-07-21T01:55:07.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e0affd700 1 -- 172.21.15.140:0/3946347734 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9df400d7f0 con 0x7f9e0c0adff0 2022-07-21T01:55:07.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 -- 172.21.15.140:0/3946347734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e0c0adff0 msgr2=0x7f9e0c0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 --2- 172.21.15.140:0/3946347734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e0c0adff0 0x7f9e0c0b5a30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f9df4002f70 tx=0x7f9df4006d90).stop 2022-07-21T01:55:07.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 -- 172.21.15.140:0/3946347734 shutdown_connections 2022-07-21T01:55:07.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 --2- 172.21.15.140:0/3946347734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e0c0adff0 0x7f9e0c0b5a30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 -- 172.21.15.140:0/3946347734 >> 172.21.15.140:0/3946347734 conn(0x7f9e0c0634e0 msgr2=0x7f9e0c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:07.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 -- 172.21.15.140:0/3946347734 shutdown_connections 2022-07-21T01:55:07.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.750+0000 7f9e10d2e700 1 -- 172.21.15.140:0/3946347734 wait complete. 2022-07-21T01:55:07.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e10d2e700 1 Processor -- start 2022-07-21T01:55:07.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e10d2e700 1 -- start start 2022-07-21T01:55:07.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e10d2e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e0c0adff0 0x7f9e0c146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e10d2e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e0c0af8e0 0x7f9e0c063f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e10d2e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e0c0b5650 0x7f9e0c064450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e10d2e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c110d70 con 0x7f9e0c0b5650 2022-07-21T01:55:07.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e10d2e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c065d60 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e10d2e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c156dd0 con 0x7f9e0c0adff0 2022-07-21T01:55:07.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e03fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e0c0af8e0 0x7f9e0c063f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e03fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e0c0af8e0 0x7f9e0c063f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42758/0 (socket says 172.21.15.140:42758) 2022-07-21T01:55:07.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e0b7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e0c0adff0 0x7f9e0c146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e03fff700 1 -- 172.21.15.140:0/2094753090 learned_addr learned my addr 172.21.15.140:0/2094753090 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:07.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e0bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e0c0b5650 0x7f9e0c064450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e03fff700 1 -- 172.21.15.140:0/2094753090 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e0c0adff0 msgr2=0x7f9e0c146000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e03fff700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e0c0adff0 0x7f9e0c146000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e03fff700 1 -- 172.21.15.140:0/2094753090 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e0c0b5650 msgr2=0x7f9e0c064450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e03fff700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e0c0b5650 0x7f9e0c064450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e03fff700 1 -- 172.21.15.140:0/2094753090 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df400b120 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e0bfff700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e0c0b5650 0x7f9e0c064450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:07.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e03fff700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e0c0af8e0 0x7f9e0c063f60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f9df4002f70 tx=0x7f9df4014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:07.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e097fa700 1 -- 172.21.15.140:0/2094753090 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9df4020070 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e097fa700 1 -- 172.21.15.140:0/2094753090 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4011880 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e10d2e700 1 -- 172.21.15.140:0/2094753090 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0c157070 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e097fa700 1 -- 172.21.15.140:0/2094753090 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9df4011be0 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.754+0000 7f9e10d2e700 1 -- 172.21.15.140:0/2094753090 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0c064940 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.758+0000 7f9e10d2e700 1 -- 172.21.15.140:0/2094753090 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0c157230 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.758+0000 7f9e097fa700 1 -- 172.21.15.140:0/2094753090 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9df400f070 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.758+0000 7f9e097fa700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9df0053aa0 0x7f9df0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:07.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.758+0000 7f9e097fa700 1 -- 172.21.15.140:0/2094753090 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f9df406c710 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.758+0000 7f9e0b7fe700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9df0053aa0 0x7f9df0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:07.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.758+0000 7f9e0b7fe700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9df0053aa0 0x7f9df0055f60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f9dfc00d000 tx=0x7f9dfc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:07.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.766+0000 7f9e097fa700 1 -- 172.21.15.140:0/2094753090 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9df4043600 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.926+0000 7f9e10d2e700 1 -- 172.21.15.140:0/2094753090 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e0c0574e0 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.926+0000 7f9e097fa700 1 -- 172.21.15.140:0/2094753090 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f9df4006920 con 0x7f9e0c0af8e0 2022-07-21T01:55:07.928 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:07.928 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":30,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/4173658089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4173658089},{"type":"v1","addr":"172.21.15.152:6833","nonce":4173658089}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:54:10.830421+0000","modified":"2022-07-21T01:54:11.841578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6208":{"gid":6208,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/167323020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":167323020},{"type":"v1","addr":"172.21.15.140:6835","nonce":167323020}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-07-21T01:55:07.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 -- 172.21.15.140:0/2094753090 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9df0053aa0 msgr2=0x7f9df0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9df0053aa0 0x7f9df0055f60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f9dfc00d000 tx=0x7f9dfc009040).stop 2022-07-21T01:55:07.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 -- 172.21.15.140:0/2094753090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e0c0af8e0 msgr2=0x7f9e0c063f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:07.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e0c0af8e0 0x7f9e0c063f60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f9df4002f70 tx=0x7f9df4014000).stop 2022-07-21T01:55:07.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 -- 172.21.15.140:0/2094753090 shutdown_connections 2022-07-21T01:55:07.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9df0053aa0 0x7f9df0055f60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e0c0b5650 0x7f9e0c064450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e0c0af8e0 0x7f9e0c063f60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 --2- 172.21.15.140:0/2094753090 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e0c0adff0 0x7f9e0c146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:07.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 -- 172.21.15.140:0/2094753090 >> 172.21.15.140:0/2094753090 conn(0x7f9e0c0634e0 msgr2=0x7f9e0c0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:07.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 -- 172.21.15.140:0/2094753090 shutdown_connections 2022-07-21T01:55:07.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:07.930+0000 7f9e10d2e700 1 -- 172.21.15.140:0/2094753090 wait complete. 2022-07-21T01:55:07.934 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 30 2022-07-21T01:55:07.946 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:55:08.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.046+0000 7f147c59b700 1 Processor -- start 2022-07-21T01:55:08.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.046+0000 7f147c59b700 1 -- start start 2022-07-21T01:55:08.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147c59b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1474098a20 0x7f14740b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:08.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147c59b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1474062160 con 0x7f14740ac200 2022-07-21T01:55:08.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147c59b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1474062670 con 0x7f1474061d80 2022-07-21T01:55:08.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147c59b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1474095880 con 0x7f1474098a20 2022-07-21T01:55:08.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147ab38700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f14740ac200 0x7f14740ac690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51174/0 (socket says 172.21.15.140:51174) 2022-07-21T01:55:08.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147ab38700 1 -- 172.21.15.140:0/728842584 learned_addr learned my addr 172.21.15.140:0/728842584 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:08.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f1479b36700 1 --2- 172.21.15.140:0/728842584 >> v2:172.21.15.152:3300/0 conn(0x7f1474098a20 0x7f14740b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:08.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f1479335700 1 -- 172.21.15.140:0/728842584 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546599198 0 0) 0x7f1474062160 con 0x7f14740ac200 2022-07-21T01:55:08.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f1479335700 1 -- 172.21.15.140:0/728842584 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f145c003260 con 0x7f14740ac200 2022-07-21T01:55:08.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f1479b36700 1 -- 172.21.15.140:0/728842584 >> v1:172.21.15.152:6789/0 conn(0x7f1474061d80 legacy=0x7f14740b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:08.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f1479b36700 1 -- 172.21.15.140:0/728842584 >> v1:172.21.15.140:6789/0 conn(0x7f14740ac200 legacy=0x7f14740ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:08.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f1479b36700 1 -- 172.21.15.140:0/728842584 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1474095a70 con 0x7f1474098a20 2022-07-21T01:55:08.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f1479b36700 1 --2- 172.21.15.140:0/728842584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1474098a20 0x7f14740b1260 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f1464002f70 tx=0x7f1464006df0).ready entity=mon.1 client_cookie=5db2f8edab1f259 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:08.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f1479335700 1 -- 172.21.15.140:0/728842584 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f146400d610 con 0x7f1474098a20 2022-07-21T01:55:08.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f1479335700 1 -- 172.21.15.140:0/728842584 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1464000e90 con 0x7f1474098a20 2022-07-21T01:55:08.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f1479335700 1 -- 172.21.15.140:0/728842584 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f146400d960 con 0x7f1474098a20 2022-07-21T01:55:08.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147c59b700 1 -- 172.21.15.140:0/728842584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1474098a20 msgr2=0x7f14740b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:08.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147c59b700 1 --2- 172.21.15.140:0/728842584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1474098a20 0x7f14740b1260 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f1464002f70 tx=0x7f1464006df0).stop 2022-07-21T01:55:08.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147c59b700 1 -- 172.21.15.140:0/728842584 shutdown_connections 2022-07-21T01:55:08.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147c59b700 1 --2- 172.21.15.140:0/728842584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1474098a20 0x7f14740b1260 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:08.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147c59b700 1 -- 172.21.15.140:0/728842584 >> 172.21.15.140:0/728842584 conn(0x7f14740b2c50 msgr2=0x7f14740b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:08.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147c59b700 1 -- 172.21.15.140:0/728842584 shutdown_connections 2022-07-21T01:55:08.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.050+0000 7f147c59b700 1 -- 172.21.15.140:0/728842584 wait complete. 2022-07-21T01:55:08.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147c59b700 1 Processor -- start 2022-07-21T01:55:08.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147c59b700 1 -- start start 2022-07-21T01:55:08.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147c59b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1474061d80 0x7f1474065de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:08.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147c59b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1474098a20 0x7f147413f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:08.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147c59b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14740ac200 0x7f147413f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:08.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147c59b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1474146ed0 con 0x7f1474098a20 2022-07-21T01:55:08.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147c59b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1474066990 con 0x7f14740ac200 2022-07-21T01:55:08.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147c59b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1474146420 con 0x7f1474061d80 2022-07-21T01:55:08.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f1479b36700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1474098a20 0x7f147413f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:08.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f1479b36700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1474098a20 0x7f147413f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37032/0 (socket says 172.21.15.140:37032) 2022-07-21T01:55:08.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f1479b36700 1 -- 172.21.15.140:0/2006406985 learned_addr learned my addr 172.21.15.140:0/2006406985 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:08.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147ab38700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14740ac200 0x7f147413f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:08.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147a337700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1474061d80 0x7f1474065de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:08.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f1479b36700 1 -- 172.21.15.140:0/2006406985 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1474061d80 msgr2=0x7f1474065de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:08.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f1479b36700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1474061d80 0x7f1474065de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:08.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f1479b36700 1 -- 172.21.15.140:0/2006406985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14740ac200 msgr2=0x7f147413f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:08.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f1479b36700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14740ac200 0x7f147413f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:08.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f1479b36700 1 -- 172.21.15.140:0/2006406985 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146400b180 con 0x7f1474098a20 2022-07-21T01:55:08.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f1479b36700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1474098a20 0x7f147413f450 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f1464002f70 tx=0x7f1464014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:08.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f14637fe700 1 -- 172.21.15.140:0/2006406985 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f146400d290 con 0x7f1474098a20 2022-07-21T01:55:08.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147c59b700 1 -- 172.21.15.140:0/2006406985 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14740675c0 con 0x7f1474098a20 2022-07-21T01:55:08.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f14637fe700 1 -- 172.21.15.140:0/2006406985 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146401e400 con 0x7f1474098a20 2022-07-21T01:55:08.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f14637fe700 1 -- 172.21.15.140:0/2006406985 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f146401e760 con 0x7f1474098a20 2022-07-21T01:55:08.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.054+0000 7f147c59b700 1 -- 172.21.15.140:0/2006406985 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14740aeb90 con 0x7f1474098a20 2022-07-21T01:55:08.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.058+0000 7f147c59b700 1 -- 172.21.15.140:0/2006406985 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14740623d0 con 0x7f1474098a20 2022-07-21T01:55:08.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.058+0000 7f14637fe700 1 -- 172.21.15.140:0/2006406985 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f146401c070 con 0x7f1474098a20 2022-07-21T01:55:08.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.058+0000 7f14637fe700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1454053a70 0x7f1454055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:08.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.058+0000 7f14637fe700 1 -- 172.21.15.140:0/2006406985 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f146406cbf0 con 0x7f1474098a20 2022-07-21T01:55:08.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.058+0000 7f147a337700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1454053a70 0x7f1454055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:08.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.058+0000 7f147a337700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1454053a70 0x7f1454055f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f147000b000 tx=0x7f1470009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:08.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.066+0000 7f14637fe700 1 -- 172.21.15.140:0/2006406985 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1464043950 con 0x7f1474098a20 2022-07-21T01:55:08.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:08.226+0000 7f147c59b700 1 -- 172.21.15.140:0/2006406985 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f14740574e0 con 0x7f1474098a20 2022-07-21T01:55:09.097 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:55:09.098+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:55:09.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.110+0000 7f14637fe700 1 -- 172.21.15.140:0/2006406985 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1464023070 con 0x7f1474098a20 2022-07-21T01:55:09.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 -- 172.21.15.140:0/2006406985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1454053a70 msgr2=0x7f1454055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1454053a70 0x7f1454055f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f147000b000 tx=0x7f1470009040).stop 2022-07-21T01:55:09.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 -- 172.21.15.140:0/2006406985 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1474098a20 msgr2=0x7f147413f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1474098a20 0x7f147413f450 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f1464002f70 tx=0x7f1464014000).stop 2022-07-21T01:55:09.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 -- 172.21.15.140:0/2006406985 shutdown_connections 2022-07-21T01:55:09.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1454053a70 0x7f1454055f30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14740ac200 0x7f147413f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1474098a20 0x7f147413f450 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 --2- 172.21.15.140:0/2006406985 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1474061d80 0x7f1474065de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 -- 172.21.15.140:0/2006406985 >> 172.21.15.140:0/2006406985 conn(0x7f14740b2c50 msgr2=0x7f14740b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:09.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 -- 172.21.15.140:0/2006406985 shutdown_connections 2022-07-21T01:55:09.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.114+0000 7f147c59b700 1 -- 172.21.15.140:0/2006406985 wait complete. 2022-07-21T01:55:09.131 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:55:09.132 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:09.140 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:55:09.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd263c700 1 Processor -- start 2022-07-21T01:55:09.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd263c700 1 -- start start 2022-07-21T01:55:09.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd263c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5ccc098ae0 0x7f5ccc061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:09.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd263c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ccc0b22d0 con 0x7f5ccc062040 2022-07-21T01:55:09.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd263c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ccc0abe50 con 0x7f5ccc0b1ef0 2022-07-21T01:55:09.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd263c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ccc0ac0f0 con 0x7f5ccc098ae0 2022-07-21T01:55:09.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd0e39700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f5ccc062040 0x7f5ccc146020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51198/0 (socket says 172.21.15.140:51198) 2022-07-21T01:55:09.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd0e39700 1 -- 172.21.15.140:0/385649295 learned_addr learned my addr 172.21.15.140:0/385649295 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:09.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd1e3b700 1 --2- 172.21.15.140:0/385649295 >> v2:172.21.15.140:3300/0 conn(0x7f5ccc098ae0 0x7f5ccc061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:09.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cbffff700 1 -- 172.21.15.140:0/385649295 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805926309 0 0) 0x7f5ccc0b22d0 con 0x7f5ccc062040 2022-07-21T01:55:09.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cbffff700 1 -- 172.21.15.140:0/385649295 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb0003260 con 0x7f5ccc062040 2022-07-21T01:55:09.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cbffff700 1 -- 172.21.15.140:0/385649295 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727357003 0 0) 0x7f5ccc0abe50 con 0x7f5ccc0b1ef0 2022-07-21T01:55:09.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd1e3b700 1 -- 172.21.15.140:0/385649295 >> v1:172.21.15.152:6789/0 conn(0x7f5ccc0b1ef0 legacy=0x7f5ccc061050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd1e3b700 1 -- 172.21.15.140:0/385649295 >> v1:172.21.15.140:6789/0 conn(0x7f5ccc062040 legacy=0x7f5ccc146020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd1e3b700 1 -- 172.21.15.140:0/385649295 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ccc0ac390 con 0x7f5ccc098ae0 2022-07-21T01:55:09.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd1e3b700 1 --2- 172.21.15.140:0/385649295 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc098ae0 0x7f5ccc061760 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f5cc8002f70 tx=0x7f5cc8006df0).ready entity=mon.0 client_cookie=3c9de74b53013fcd server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:09.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cbffff700 1 -- 172.21.15.140:0/385649295 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cc800d610 con 0x7f5ccc098ae0 2022-07-21T01:55:09.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cbffff700 1 -- 172.21.15.140:0/385649295 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc8000e90 con 0x7f5ccc098ae0 2022-07-21T01:55:09.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cbffff700 1 -- 172.21.15.140:0/385649295 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cc800d960 con 0x7f5ccc098ae0 2022-07-21T01:55:09.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd263c700 1 -- 172.21.15.140:0/385649295 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc098ae0 msgr2=0x7f5ccc061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd263c700 1 --2- 172.21.15.140:0/385649295 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc098ae0 0x7f5ccc061760 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f5cc8002f70 tx=0x7f5cc8006df0).stop 2022-07-21T01:55:09.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd263c700 1 -- 172.21.15.140:0/385649295 shutdown_connections 2022-07-21T01:55:09.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd263c700 1 --2- 172.21.15.140:0/385649295 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc098ae0 0x7f5ccc061760 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.230+0000 7f5cd263c700 1 -- 172.21.15.140:0/385649295 >> 172.21.15.140:0/385649295 conn(0x7f5ccc0b2ca0 msgr2=0x7f5ccc0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:09.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 -- 172.21.15.140:0/385649295 shutdown_connections 2022-07-21T01:55:09.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 -- 172.21.15.140:0/385649295 wait complete. 2022-07-21T01:55:09.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 Processor -- start 2022-07-21T01:55:09.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 -- start start 2022-07-21T01:55:09.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc062040 0x7f5ccc0ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:09.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc098ae0 0x7f5ccc144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:09.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc0b1ef0 0x7f5ccc148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:09.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc14b470 con 0x7f5ccc098ae0 2022-07-21T01:55:09.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc0af360 con 0x7f5ccc0b1ef0 2022-07-21T01:55:09.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc0af600 con 0x7f5ccc062040 2022-07-21T01:55:09.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd0e39700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc098ae0 0x7f5ccc144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:09.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd0e39700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc098ae0 0x7f5ccc144050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37058/0 (socket says 172.21.15.140:37058) 2022-07-21T01:55:09.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd0e39700 1 -- 172.21.15.140:0/2251334589 learned_addr learned my addr 172.21.15.140:0/2251334589 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:09.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd1e3b700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc0b1ef0 0x7f5ccc148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:09.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd163a700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc062040 0x7f5ccc0ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:09.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd0e39700 1 -- 172.21.15.140:0/2251334589 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc062040 msgr2=0x7f5ccc0ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd0e39700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc062040 0x7f5ccc0ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd0e39700 1 -- 172.21.15.140:0/2251334589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc0b1ef0 msgr2=0x7f5ccc148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd0e39700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc0b1ef0 0x7f5ccc148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd0e39700 1 -- 172.21.15.140:0/2251334589 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc800b180 con 0x7f5ccc098ae0 2022-07-21T01:55:09.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd1e3b700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc0b1ef0 0x7f5ccc148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:09.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd0e39700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc098ae0 0x7f5ccc144050 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f5cc4001490 tx=0x7f5cc4007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:09.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cbe7fc700 1 -- 172.21.15.140:0/2251334589 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cc4005ba0 con 0x7f5ccc098ae0 2022-07-21T01:55:09.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cbe7fc700 1 -- 172.21.15.140:0/2251334589 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc4005d00 con 0x7f5ccc098ae0 2022-07-21T01:55:09.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 -- 172.21.15.140:0/2251334589 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ccc099230 con 0x7f5ccc098ae0 2022-07-21T01:55:09.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cbe7fc700 1 -- 172.21.15.140:0/2251334589 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5cc40072c0 con 0x7f5ccc098ae0 2022-07-21T01:55:09.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.234+0000 7f5cd263c700 1 -- 172.21.15.140:0/2251334589 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ccc0b0100 con 0x7f5ccc098ae0 2022-07-21T01:55:09.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.238+0000 7f5cbe7fc700 1 -- 172.21.15.140:0/2251334589 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5cc4010070 con 0x7f5ccc098ae0 2022-07-21T01:55:09.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.238+0000 7f5cd263c700 1 -- 172.21.15.140:0/2251334589 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ccc0af7c0 con 0x7f5ccc098ae0 2022-07-21T01:55:09.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.238+0000 7f5cbe7fc700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cb4053950 0x7f5cb4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:09.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.238+0000 7f5cbe7fc700 1 -- 172.21.15.140:0/2251334589 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f5cc406bbc0 con 0x7f5ccc098ae0 2022-07-21T01:55:09.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.238+0000 7f5cd163a700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cb4053950 0x7f5cb4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:09.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.238+0000 7f5cd163a700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cb4053950 0x7f5cb4055e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f5cc000b000 tx=0x7f5cc0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:09.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.242+0000 7f5cbe7fc700 1 -- 172.21.15.140:0/2251334589 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5cc4042950 con 0x7f5ccc098ae0 2022-07-21T01:55:09.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.402+0000 7f5cd263c700 1 -- 172.21.15.140:0/2251334589 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ccc0574e0 con 0x7f5ccc098ae0 2022-07-21T01:55:09.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.402+0000 7f5cbe7fc700 1 -- 172.21.15.140:0/2251334589 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+1782 (secure 0 0 0) 0x7f5cc4045f70 con 0x7f5ccc098ae0 2022-07-21T01:55:09.404 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:09.404 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":31,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/4173658089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4173658089},{"type":"v1","addr":"172.21.15.152:6833","nonce":4173658089}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:54:10.830421+0000","modified":"2022-07-21T01:55:09.100847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2022-07-21T01:55:09.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 -- 172.21.15.140:0/2251334589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cb4053950 msgr2=0x7f5cb4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cb4053950 0x7f5cb4055e10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f5cc000b000 tx=0x7f5cc0009040).stop 2022-07-21T01:55:09.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 -- 172.21.15.140:0/2251334589 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc098ae0 msgr2=0x7f5ccc144050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc098ae0 0x7f5ccc144050 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f5cc4001490 tx=0x7f5cc4007e20).stop 2022-07-21T01:55:09.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 -- 172.21.15.140:0/2251334589 shutdown_connections 2022-07-21T01:55:09.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5cb4053950 0x7f5cb4055e10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ccc0b1ef0 0x7f5ccc148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5ccc098ae0 0x7f5ccc144050 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 --2- 172.21.15.140:0/2251334589 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5ccc062040 0x7f5ccc0ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 -- 172.21.15.140:0/2251334589 >> 172.21.15.140:0/2251334589 conn(0x7f5ccc0b2ca0 msgr2=0x7f5ccc145b50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:09.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 -- 172.21.15.140:0/2251334589 shutdown_connections 2022-07-21T01:55:09.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.406+0000 7f5cd263c700 1 -- 172.21.15.140:0/2251334589 wait complete. 2022-07-21T01:55:09.409 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 31 2022-07-21T01:55:09.416 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:55:09.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.522+0000 7f57e0699700 1 Processor -- start 2022-07-21T01:55:09.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.522+0000 7f57e0699700 1 -- start start 2022-07-21T01:55:09.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f57d80b5320 0x7f57d805c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:09.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57d805c990 0x7f57d8146430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:09.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d80b0c20 con 0x7f57d80b9eb0 2022-07-21T01:55:09.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d80ba290 con 0x7f57d80b5320 2022-07-21T01:55:09.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8142e00 con 0x7f57d805c990 2022-07-21T01:55:09.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57de435700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f57d80b9eb0 0x7f57d805bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51210/0 (socket says 172.21.15.140:51210) 2022-07-21T01:55:09.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57de435700 1 -- 172.21.15.140:0/1384629968 learned_addr learned my addr 172.21.15.140:0/1384629968 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:09.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57ddc34700 1 --2- 172.21.15.140:0/1384629968 >> v2:172.21.15.140:3300/0 conn(0x7f57d80b5320 0x7f57d805c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:09.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57dec36700 1 --2- 172.21.15.140:0/1384629968 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57d805c990 0x7f57d8146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:09.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57dd433700 1 -- 172.21.15.140:0/1384629968 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656668015 0 0) 0x7f57d80b0c20 con 0x7f57d80b9eb0 2022-07-21T01:55:09.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57dd433700 1 -- 172.21.15.140:0/1384629968 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57c0003260 con 0x7f57d80b9eb0 2022-07-21T01:55:09.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57ddc34700 1 -- 172.21.15.140:0/1384629968 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57d805c990 msgr2=0x7f57d8146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:09.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57ddc34700 1 --2- 172.21.15.140:0/1384629968 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57d805c990 0x7f57d8146430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57ddc34700 1 -- 172.21.15.140:0/1384629968 >> v1:172.21.15.140:6789/0 conn(0x7f57d80b9eb0 legacy=0x7f57d805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57ddc34700 1 -- 172.21.15.140:0/1384629968 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d80afed0 con 0x7f57d80b5320 2022-07-21T01:55:09.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57ddc34700 1 --2- 172.21.15.140:0/1384629968 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57d80b5320 0x7f57d805c4a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f57c8002f70 tx=0x7f57c8006df0).ready entity=mon.0 client_cookie=95efc5bc0980c1af server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:09.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57dd433700 1 -- 172.21.15.140:0/1384629968 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57c800d4a0 con 0x7f57d80b5320 2022-07-21T01:55:09.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57dd433700 1 -- 172.21.15.140:0/1384629968 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c8000e90 con 0x7f57d80b5320 2022-07-21T01:55:09.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57dd433700 1 -- 172.21.15.140:0/1384629968 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57c800d7f0 con 0x7f57d80b5320 2022-07-21T01:55:09.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 -- 172.21.15.140:0/1384629968 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57d80b5320 msgr2=0x7f57d805c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 --2- 172.21.15.140:0/1384629968 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57d80b5320 0x7f57d805c4a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f57c8002f70 tx=0x7f57c8006df0).stop 2022-07-21T01:55:09.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 -- 172.21.15.140:0/1384629968 shutdown_connections 2022-07-21T01:55:09.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 --2- 172.21.15.140:0/1384629968 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57d805c990 0x7f57d8146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 --2- 172.21.15.140:0/1384629968 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57d80b5320 0x7f57d805c4a0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 -- 172.21.15.140:0/1384629968 >> 172.21.15.140:0/1384629968 conn(0x7f57d8067980 msgr2=0x7f57d80b9460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:09.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 -- 172.21.15.140:0/1384629968 shutdown_connections 2022-07-21T01:55:09.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.526+0000 7f57e0699700 1 -- 172.21.15.140:0/1384629968 wait complete. 2022-07-21T01:55:09.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57e0699700 1 Processor -- start 2022-07-21T01:55:09.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57e0699700 1 -- start start 2022-07-21T01:55:09.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57e0699700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57d805c990 0x7f57d80af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:09.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57e0699700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57d80b5320 0x7f57d8155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:09.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57e0699700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57d80b9eb0 0x7f57d8151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:09.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57e0699700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8155440 con 0x7f57d805c990 2022-07-21T01:55:09.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57e0699700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d81556e0 con 0x7f57d80b9eb0 2022-07-21T01:55:09.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57e0699700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8114f30 con 0x7f57d80b5320 2022-07-21T01:55:09.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57de435700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57d805c990 0x7f57d80af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:09.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57de435700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57d805c990 0x7f57d80af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37070/0 (socket says 172.21.15.140:37070) 2022-07-21T01:55:09.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57de435700 1 -- 172.21.15.140:0/2868894765 learned_addr learned my addr 172.21.15.140:0/2868894765 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:09.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57ddc34700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57d80b5320 0x7f57d8155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:09.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57dec36700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57d80b9eb0 0x7f57d8151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:09.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57de435700 1 -- 172.21.15.140:0/2868894765 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57d80b5320 msgr2=0x7f57d8155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57de435700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57d80b5320 0x7f57d8155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57de435700 1 -- 172.21.15.140:0/2868894765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57d80b9eb0 msgr2=0x7f57d8151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:09.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57de435700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57d80b9eb0 0x7f57d8151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:09.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57de435700 1 -- 172.21.15.140:0/2868894765 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c800b180 con 0x7f57d805c990 2022-07-21T01:55:09.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57dec36700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57d80b9eb0 0x7f57d8151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:55:09.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57de435700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57d805c990 0x7f57d80af220 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f57d4001110 tx=0x7f57d4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:09.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57c77fe700 1 -- 172.21.15.140:0/2868894765 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d4007e10 con 0x7f57d805c990 2022-07-21T01:55:09.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57c77fe700 1 -- 172.21.15.140:0/2868894765 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d40072e0 con 0x7f57d805c990 2022-07-21T01:55:09.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57e0699700 1 -- 172.21.15.140:0/2868894765 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d8156070 con 0x7f57d805c990 2022-07-21T01:55:09.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57c77fe700 1 -- 172.21.15.140:0/2868894765 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57d4005800 con 0x7f57d805c990 2022-07-21T01:55:09.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.530+0000 7f57e0699700 1 -- 172.21.15.140:0/2868894765 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d8152400 con 0x7f57d805c990 2022-07-21T01:55:09.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.534+0000 7f57c77fe700 1 -- 172.21.15.140:0/2868894765 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f57d401a060 con 0x7f57d805c990 2022-07-21T01:55:09.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.534+0000 7f57e0699700 1 -- 172.21.15.140:0/2868894765 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d8114810 con 0x7f57d805c990 2022-07-21T01:55:09.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.534+0000 7f57c77fe700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57b80539e0 0x7f57b8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:09.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.534+0000 7f57ddc34700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57b80539e0 0x7f57b8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:09.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.534+0000 7f57c77fe700 1 -- 172.21.15.140:0/2868894765 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f57d406b750 con 0x7f57d805c990 2022-07-21T01:55:09.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.534+0000 7f57ddc34700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57b80539e0 0x7f57b8055ea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f57c8018000 tx=0x7f57c8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:09.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.538+0000 7f57c77fe700 1 -- 172.21.15.140:0/2868894765 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57d8114810 con 0x7f57d805c990 2022-07-21T01:55:09.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:09.694+0000 7f57e0699700 1 -- 172.21.15.140:0/2868894765 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f57d8114810 con 0x7f57d805c990 2022-07-21T01:55:10.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57c77fe700 1 -- 172.21.15.140:0/2868894765 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f57d8114810 con 0x7f57d805c990 2022-07-21T01:55:10.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 -- 172.21.15.140:0/2868894765 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57b80539e0 msgr2=0x7f57b8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:10.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57b80539e0 0x7f57b8055ea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f57c8018000 tx=0x7f57c8014000).stop 2022-07-21T01:55:10.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 -- 172.21.15.140:0/2868894765 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57d805c990 msgr2=0x7f57d80af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:10.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57d805c990 0x7f57d80af220 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f57d4001110 tx=0x7f57d4007ad0).stop 2022-07-21T01:55:10.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 -- 172.21.15.140:0/2868894765 shutdown_connections 2022-07-21T01:55:10.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57b80539e0 0x7f57b8055ea0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:10.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57d80b9eb0 0x7f57d8151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:10.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57d80b5320 0x7f57d8155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:10.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 --2- 172.21.15.140:0/2868894765 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57d805c990 0x7f57d80af220 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:10.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 -- 172.21.15.140:0/2868894765 >> 172.21.15.140:0/2868894765 conn(0x7f57d8067980 msgr2=0x7f57d80b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:10.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 -- 172.21.15.140:0/2868894765 shutdown_connections 2022-07-21T01:55:10.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.113+0000 7f57e0699700 1 -- 172.21.15.140:0/2868894765 wait complete. 2022-07-21T01:55:10.133 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:55:10.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.245+0000 7f5904e25700 1 Processor -- start 2022-07-21T01:55:10.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.245+0000 7f5904e25700 1 -- start start 2022-07-21T01:55:10.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.245+0000 7f5904e25700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f590005cf40 0x7f5900062f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:10.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.245+0000 7f5904e25700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5900142b30 0x7f5900142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:10.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.245+0000 7f5904e25700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59000adfa0 con 0x7f59000adb50 2022-07-21T01:55:10.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.245+0000 7f5904e25700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5900147110 con 0x7f590005cf40 2022-07-21T01:55:10.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.245+0000 7f5904e25700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59001462c0 con 0x7f5900142b30 2022-07-21T01:55:10.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fe59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f590005cf40 0x7f5900062f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:10.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fe59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f590005cf40 0x7f5900062f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37078/0 (socket says 172.21.15.140:37078) 2022-07-21T01:55:10.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fe59c700 1 -- 172.21.15.140:0/3861615416 learned_addr learned my addr 172.21.15.140:0/3861615416 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:10.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fed9d700 1 --2- 172.21.15.140:0/3861615416 >> v2:172.21.15.152:3300/0 conn(0x7f5900142b30 0x7f5900142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:10.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fe59c700 1 -- 172.21.15.140:0/3861615416 >> v2:172.21.15.152:3300/0 conn(0x7f5900142b30 msgr2=0x7f5900142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:10.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fe59c700 1 --2- 172.21.15.140:0/3861615416 >> v2:172.21.15.152:3300/0 conn(0x7f5900142b30 0x7f5900142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:10.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fe59c700 1 -- 172.21.15.140:0/3861615416 >> v1:172.21.15.152:6789/0 conn(0x7f59000adb50 legacy=0x7f5900146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:10.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fe59c700 1 -- 172.21.15.140:0/3861615416 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900146560 con 0x7f590005cf40 2022-07-21T01:55:10.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fe59c700 1 --2- 172.21.15.140:0/3861615416 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f590005cf40 0x7f5900062f30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f58f0002f70 tx=0x7f58f0006eb0).ready entity=mon.0 client_cookie=b42c025e99f4a30 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:10.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fdd9b700 1 -- 172.21.15.140:0/3861615416 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f000d5c0 con 0x7f590005cf40 2022-07-21T01:55:10.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fdd9b700 1 -- 172.21.15.140:0/3861615416 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0000e90 con 0x7f590005cf40 2022-07-21T01:55:10.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f58fdd9b700 1 -- 172.21.15.140:0/3861615416 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f000d950 con 0x7f590005cf40 2022-07-21T01:55:10.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 -- 172.21.15.140:0/3861615416 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f590005cf40 msgr2=0x7f5900062f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:10.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 --2- 172.21.15.140:0/3861615416 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f590005cf40 0x7f5900062f30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f58f0002f70 tx=0x7f58f0006eb0).stop 2022-07-21T01:55:10.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 -- 172.21.15.140:0/3861615416 shutdown_connections 2022-07-21T01:55:10.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 --2- 172.21.15.140:0/3861615416 >> v2:172.21.15.152:3300/0 conn(0x7f5900142b30 0x7f5900142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:10.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 --2- 172.21.15.140:0/3861615416 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f590005cf40 0x7f5900062f30 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:10.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 -- 172.21.15.140:0/3861615416 >> 172.21.15.140:0/3861615416 conn(0x7f59000b7d00 msgr2=0x7f59000b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:10.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 -- 172.21.15.140:0/3861615416 shutdown_connections 2022-07-21T01:55:10.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 -- 172.21.15.140:0/3861615416 wait complete. 2022-07-21T01:55:10.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 Processor -- start 2022-07-21T01:55:10.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 -- start start 2022-07-21T01:55:10.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590005cf40 0x7f59000b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:10.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59000adb50 0x7f59000ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:10.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5900142b30 0x7f590009ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:10.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590009bcb0 con 0x7f5900142b30 2022-07-21T01:55:10.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590009a430 con 0x7f590005cf40 2022-07-21T01:55:10.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.249+0000 7f5904e25700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590009a6d0 con 0x7f59000adb50 2022-07-21T01:55:10.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58fed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5900142b30 0x7f590009ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:10.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58fed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5900142b30 0x7f590009ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37088/0 (socket says 172.21.15.140:37088) 2022-07-21T01:55:10.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58fed9d700 1 -- 172.21.15.140:0/298620601 learned_addr learned my addr 172.21.15.140:0/298620601 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:10.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58f5d9b700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59000adb50 0x7f59000ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:10.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58fe59c700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590005cf40 0x7f59000b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:10.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58fed9d700 1 -- 172.21.15.140:0/298620601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59000adb50 msgr2=0x7f59000ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:10.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58fed9d700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59000adb50 0x7f59000ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:10.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58fed9d700 1 -- 172.21.15.140:0/298620601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590005cf40 msgr2=0x7f59000b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:10.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58fed9d700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590005cf40 0x7f59000b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:10.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58fed9d700 1 -- 172.21.15.140:0/298620601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f000b1e0 con 0x7f5900142b30 2022-07-21T01:55:10.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58fe59c700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590005cf40 0x7f59000b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:10.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58fed9d700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5900142b30 0x7f590009ab70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f58f8006e00 tx=0x7f58f80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:10.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58f7fff700 1 -- 172.21.15.140:0/298620601 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f800c670 con 0x7f5900142b30 2022-07-21T01:55:10.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58f7fff700 1 -- 172.21.15.140:0/298620601 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f800c7d0 con 0x7f5900142b30 2022-07-21T01:55:10.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f5904e25700 1 -- 172.21.15.140:0/298620601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590009d170 con 0x7f5900142b30 2022-07-21T01:55:10.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58f7fff700 1 -- 172.21.15.140:0/298620601 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58f800cb30 con 0x7f5900142b30 2022-07-21T01:55:10.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f5904e25700 1 -- 172.21.15.140:0/298620601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59000b0a10 con 0x7f5900142b30 2022-07-21T01:55:10.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f5904e25700 1 -- 172.21.15.140:0/298620601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59000b0490 con 0x7f5900142b30 2022-07-21T01:55:10.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.253+0000 7f58f7fff700 1 -- 172.21.15.140:0/298620601 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f58f800cc90 con 0x7f5900142b30 2022-07-21T01:55:10.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.257+0000 7f58f7fff700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f58d8053aa0 0x7f58d8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:10.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.257+0000 7f58f7fff700 1 -- 172.21.15.140:0/298620601 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6202+0+0 (secure 0 0 0) 0x7f58f806c160 con 0x7f5900142b30 2022-07-21T01:55:10.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.257+0000 7f58fe59c700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f58d8053aa0 0x7f58d8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:10.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.257+0000 7f58fe59c700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f58d8053aa0 0x7f58d8055f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f58f001a000 tx=0x7f58f0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:10.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.261+0000 7f58f7fff700 1 -- 172.21.15.140:0/298620601 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f59000b0490 con 0x7f5900142b30 2022-07-21T01:55:10.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:10.385+0000 7f5904e25700 1 -- 172.21.15.140:0/298620601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f59000b0490 con 0x7f5900142b30 2022-07-21T01:55:11.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.117+0000 7f58f7fff700 1 -- 172.21.15.140:0/298620601 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) v1 ==== 185+0+0 (secure 0 0 0) 0x7f59000b0490 con 0x7f5900142b30 2022-07-21T01:55:11.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 -- 172.21.15.140:0/298620601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f58d8053aa0 msgr2=0x7f58d8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:11.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f58d8053aa0 0x7f58d8055f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f58f001a000 tx=0x7f58f0015000).stop 2022-07-21T01:55:11.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 -- 172.21.15.140:0/298620601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5900142b30 msgr2=0x7f590009ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:11.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5900142b30 0x7f590009ab70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f58f8006e00 tx=0x7f58f80091d0).stop 2022-07-21T01:55:11.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 -- 172.21.15.140:0/298620601 shutdown_connections 2022-07-21T01:55:11.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f58d8053aa0 0x7f58d8055f60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:11.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5900142b30 0x7f590009ab70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:11.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f59000adb50 0x7f59000ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:11.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 --2- 172.21.15.140:0/298620601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590005cf40 0x7f59000b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:11.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 -- 172.21.15.140:0/298620601 >> 172.21.15.140:0/298620601 conn(0x7f59000b7d00 msgr2=0x7f59000b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:11.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 -- 172.21.15.140:0/298620601 shutdown_connections 2022-07-21T01:55:11.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.121+0000 7f5904e25700 1 -- 172.21.15.140:0/298620601 wait complete. 2022-07-21T01:55:11.129 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:55:11.135 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:55:11.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.245+0000 7f0fc587e700 1 Processor -- start 2022-07-21T01:55:11.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.245+0000 7f0fc587e700 1 -- start start 2022-07-21T01:55:11.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fc00b3cf0 0x7f0fc0146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:11.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0fc0146530 0x7f0fc0141a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:11.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc0066b10 con 0x7f0fc00b6840 2022-07-21T01:55:11.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc00b5c90 con 0x7f0fc0146530 2022-07-21T01:55:11.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00b6c20 con 0x7f0fc00b3cf0 2022-07-21T01:55:11.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc507d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0fc0146530 0x7f0fc0141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:11.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc487c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0fc00b6840 0x7f0fc00b5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51240/0 (socket says 172.21.15.140:51240) 2022-07-21T01:55:11.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc487c700 1 -- 172.21.15.140:0/3786741078 learned_addr learned my addr 172.21.15.140:0/3786741078 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:11.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fbffff700 1 --2- 172.21.15.140:0/3786741078 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fc00b3cf0 0x7f0fc0146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:11.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fbf7fe700 1 -- 172.21.15.140:0/3786741078 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270489905 0 0) 0x7f0fc0066b10 con 0x7f0fc00b6840 2022-07-21T01:55:11.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fbf7fe700 1 -- 172.21.15.140:0/3786741078 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fac003260 con 0x7f0fc00b6840 2022-07-21T01:55:11.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc507d700 1 -- 172.21.15.140:0/3786741078 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fc00b3cf0 msgr2=0x7f0fc0146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:11.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc507d700 1 --2- 172.21.15.140:0/3786741078 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fc00b3cf0 0x7f0fc0146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:11.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc507d700 1 -- 172.21.15.140:0/3786741078 >> v1:172.21.15.140:6789/0 conn(0x7f0fc00b6840 legacy=0x7f0fc00b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:11.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc507d700 1 -- 172.21.15.140:0/3786741078 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc00aead0 con 0x7f0fc0146530 2022-07-21T01:55:11.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc507d700 1 --2- 172.21.15.140:0/3786741078 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fc0146530 0x7f0fc0141a90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f0fb80032f0 tx=0x7f0fb80091d0).ready entity=mon.0 client_cookie=b992d6a5c84bc987 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:11.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fbf7fe700 1 -- 172.21.15.140:0/3786741078 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fb800b600 con 0x7f0fc0146530 2022-07-21T01:55:11.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fbf7fe700 1 -- 172.21.15.140:0/3786741078 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb800b760 con 0x7f0fc0146530 2022-07-21T01:55:11.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fbf7fe700 1 -- 172.21.15.140:0/3786741078 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fb800b8d0 con 0x7f0fc0146530 2022-07-21T01:55:11.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- 172.21.15.140:0/3786741078 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fc0146530 msgr2=0x7f0fc0141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:11.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 --2- 172.21.15.140:0/3786741078 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fc0146530 0x7f0fc0141a90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f0fb80032f0 tx=0x7f0fb80091d0).stop 2022-07-21T01:55:11.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- 172.21.15.140:0/3786741078 shutdown_connections 2022-07-21T01:55:11.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 --2- 172.21.15.140:0/3786741078 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fc0146530 0x7f0fc0141a90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:11.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 --2- 172.21.15.140:0/3786741078 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fc00b3cf0 0x7f0fc0146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:11.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- 172.21.15.140:0/3786741078 >> 172.21.15.140:0/3786741078 conn(0x7f0fc00b9e10 msgr2=0x7f0fc00b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:11.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- 172.21.15.140:0/3786741078 shutdown_connections 2022-07-21T01:55:11.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- 172.21.15.140:0/3786741078 wait complete. 2022-07-21T01:55:11.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 Processor -- start 2022-07-21T01:55:11.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- start start 2022-07-21T01:55:11.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fc00b3cf0 0x7f0fc00aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:11.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fc00b6840 0x7f0fc00b8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:11.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fc0146530 0x7f0fc01148e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:11.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00b15c0 con 0x7f0fc00b6840 2022-07-21T01:55:11.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00b17f0 con 0x7f0fc00b3cf0 2022-07-21T01:55:11.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.249+0000 7f0fc587e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc00b1a90 con 0x7f0fc0146530 2022-07-21T01:55:11.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fbffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fc00b6840 0x7f0fc00b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:11.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fc507d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fc0146530 0x7f0fc01148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:11.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fbffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fc00b6840 0x7f0fc00b8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37100/0 (socket says 172.21.15.140:37100) 2022-07-21T01:55:11.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fbffff700 1 -- 172.21.15.140:0/524186983 learned_addr learned my addr 172.21.15.140:0/524186983 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:11.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fc487c700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fc00b3cf0 0x7f0fc00aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:11.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fc507d700 1 -- 172.21.15.140:0/524186983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fc00b3cf0 msgr2=0x7f0fc00aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:11.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fc507d700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fc00b3cf0 0x7f0fc00aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:11.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fc507d700 1 -- 172.21.15.140:0/524186983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fc00b6840 msgr2=0x7f0fc00b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:11.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fc507d700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fc00b6840 0x7f0fc00b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:11.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fc507d700 1 -- 172.21.15.140:0/524186983 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb8009470 con 0x7f0fc0146530 2022-07-21T01:55:11.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fbffff700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fc00b6840 0x7f0fc00b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:11.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fc507d700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fc0146530 0x7f0fc01148e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f0fc0113c60 tx=0x7f0fb8016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:11.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fbdffb700 1 -- 172.21.15.140:0/524186983 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fb800b600 con 0x7f0fc0146530 2022-07-21T01:55:11.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fbdffb700 1 -- 172.21.15.140:0/524186983 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb8011b20 con 0x7f0fc0146530 2022-07-21T01:55:11.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fc587e700 1 -- 172.21.15.140:0/524186983 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc0112f00 con 0x7f0fc0146530 2022-07-21T01:55:11.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fbdffb700 1 -- 172.21.15.140:0/524186983 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fb8011e80 con 0x7f0fc0146530 2022-07-21T01:55:11.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fc587e700 1 -- 172.21.15.140:0/524186983 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc00b29f0 con 0x7f0fc0146530 2022-07-21T01:55:11.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.253+0000 7f0fc587e700 1 -- 172.21.15.140:0/524186983 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc00b1c50 con 0x7f0fc0146530 2022-07-21T01:55:11.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.257+0000 7f0fbdffb700 1 -- 172.21.15.140:0/524186983 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0fb800f070 con 0x7f0fc0146530 2022-07-21T01:55:11.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.257+0000 7f0fbdffb700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fa0053a70 0x7f0fa0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:11.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.257+0000 7f0fbdffb700 1 -- 172.21.15.140:0/524186983 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5764+0+0 (secure 0 0 0) 0x7f0fb806c5a0 con 0x7f0fc0146530 2022-07-21T01:55:11.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.257+0000 7f0fc487c700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fa0053a70 0x7f0fa0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:11.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.257+0000 7f0fc487c700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fa0053a70 0x7f0fa0055f30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f0fb000d000 tx=0x7f0fb0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:11.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.265+0000 7f0fbdffb700 1 -- 172.21.15.140:0/524186983 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0fb8043460 con 0x7f0fc0146530 2022-07-21T01:55:11.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:11.393+0000 7f0fc587e700 1 -- 172.21.15.140:0/524186983 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0fc00aed70 con 0x7f0fc0146530 2022-07-21T01:55:12.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.129+0000 7f0fbdffb700 1 -- 172.21.15.140:0/524186983 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0fb80150d0 con 0x7f0fc0146530 2022-07-21T01:55:12.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 -- 172.21.15.140:0/524186983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fa0053a70 msgr2=0x7f0fa0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fa0053a70 0x7f0fa0055f30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f0fb000d000 tx=0x7f0fb0009040).stop 2022-07-21T01:55:12.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 -- 172.21.15.140:0/524186983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fc0146530 msgr2=0x7f0fc01148e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fc0146530 0x7f0fc01148e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f0fc0113c60 tx=0x7f0fb8016000).stop 2022-07-21T01:55:12.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 -- 172.21.15.140:0/524186983 shutdown_connections 2022-07-21T01:55:12.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fa0053a70 0x7f0fa0055f30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fc0146530 0x7f0fc01148e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fc00b6840 0x7f0fc00b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 --2- 172.21.15.140:0/524186983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fc00b3cf0 0x7f0fc00aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 -- 172.21.15.140:0/524186983 >> 172.21.15.140:0/524186983 conn(0x7f0fc00b9e10 msgr2=0x7f0fc00b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:12.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 -- 172.21.15.140:0/524186983 shutdown_connections 2022-07-21T01:55:12.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.133+0000 7f0fc587e700 1 -- 172.21.15.140:0/524186983 wait complete. 2022-07-21T01:55:12.138 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:55:12.148 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-07-21T01:55:12.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.257+0000 7f8969eb5700 1 Processor -- start 2022-07-21T01:55:12.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.257+0000 7f8969eb5700 1 -- start start 2022-07-21T01:55:12.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.257+0000 7f8969eb5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8964066670 0x7f896405cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.257+0000 7f8969eb5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f896405bfa0 0x7f896413f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.257+0000 7f8969eb5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89640b2220 con 0x7f89640b7ca0 2022-07-21T01:55:12.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.257+0000 7f8969eb5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89640b16f0 con 0x7f8964066670 2022-07-21T01:55:12.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.257+0000 7f8969eb5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896405c4e0 con 0x7f896405bfa0 2022-07-21T01:55:12.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.257+0000 7f89637fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f89640b7ca0 0x7f8964143f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51252/0 (socket says 172.21.15.140:51252) 2022-07-21T01:55:12.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.257+0000 7f89637fe700 1 -- 172.21.15.140:0/3240798987 learned_addr learned my addr 172.21.15.140:0/3240798987 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:12.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.257+0000 7f8963fff700 1 --2- 172.21.15.140:0/3240798987 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f896405bfa0 0x7f896413f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8962ffd700 1 --2- 172.21.15.140:0/3240798987 >> v2:172.21.15.152:3300/0 conn(0x7f8964066670 0x7f896405cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f89627fc700 1 -- 172.21.15.140:0/3240798987 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700822113 0 0) 0x7f89640b2220 con 0x7f89640b7ca0 2022-07-21T01:55:12.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f89627fc700 1 -- 172.21.15.140:0/3240798987 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8950003260 con 0x7f89640b7ca0 2022-07-21T01:55:12.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8963fff700 1 -- 172.21.15.140:0/3240798987 >> v2:172.21.15.152:3300/0 conn(0x7f8964066670 msgr2=0x7f896405cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:12.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8963fff700 1 --2- 172.21.15.140:0/3240798987 >> v2:172.21.15.152:3300/0 conn(0x7f8964066670 0x7f896405cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8963fff700 1 -- 172.21.15.140:0/3240798987 >> v1:172.21.15.140:6789/0 conn(0x7f89640b7ca0 legacy=0x7f8964143f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8963fff700 1 -- 172.21.15.140:0/3240798987 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89640b8080 con 0x7f896405bfa0 2022-07-21T01:55:12.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8962ffd700 1 --2- 172.21.15.140:0/3240798987 >> v2:172.21.15.152:3300/0 conn(0x7f8964066670 0x7f896405cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:12.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8963fff700 1 --2- 172.21.15.140:0/3240798987 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f896405bfa0 0x7f896413f990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f8954002f70 tx=0x7f8954006df0).ready entity=mon.2 client_cookie=53875666665687d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:12.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f89627fc700 1 -- 172.21.15.140:0/3240798987 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f895400d4a0 con 0x7f896405bfa0 2022-07-21T01:55:12.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f89627fc700 1 -- 172.21.15.140:0/3240798987 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8954000e90 con 0x7f896405bfa0 2022-07-21T01:55:12.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f89627fc700 1 -- 172.21.15.140:0/3240798987 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f895400d7f0 con 0x7f896405bfa0 2022-07-21T01:55:12.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 -- 172.21.15.140:0/3240798987 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f896405bfa0 msgr2=0x7f896413f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 --2- 172.21.15.140:0/3240798987 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f896405bfa0 0x7f896413f990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f8954002f70 tx=0x7f8954006df0).stop 2022-07-21T01:55:12.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 -- 172.21.15.140:0/3240798987 shutdown_connections 2022-07-21T01:55:12.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 --2- 172.21.15.140:0/3240798987 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f896405bfa0 0x7f896413f990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 --2- 172.21.15.140:0/3240798987 >> v2:172.21.15.152:3300/0 conn(0x7f8964066670 0x7f896405cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 -- 172.21.15.140:0/3240798987 >> 172.21.15.140:0/3240798987 conn(0x7f8964065760 msgr2=0x7f89640b7250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:12.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 -- 172.21.15.140:0/3240798987 shutdown_connections 2022-07-21T01:55:12.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 -- 172.21.15.140:0/3240798987 wait complete. 2022-07-21T01:55:12.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 Processor -- start 2022-07-21T01:55:12.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 -- start start 2022-07-21T01:55:12.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f896405bfa0 0x7f8964111450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8964066670 0x7f8964111e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89640b7ca0 0x7f896414fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641144b0 con 0x7f896405bfa0 2022-07-21T01:55:12.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8964110550 con 0x7f8964066670 2022-07-21T01:55:12.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8969eb5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89640b0150 con 0x7f89640b7ca0 2022-07-21T01:55:12.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f89637fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f896405bfa0 0x7f8964111450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8962ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8964066670 0x7f8964111e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f89637fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f896405bfa0 0x7f8964111450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37112/0 (socket says 172.21.15.140:37112) 2022-07-21T01:55:12.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f89637fe700 1 -- 172.21.15.140:0/3691633936 learned_addr learned my addr 172.21.15.140:0/3691633936 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:12.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8963fff700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89640b7ca0 0x7f896414fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8962ffd700 1 -- 172.21.15.140:0/3691633936 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89640b7ca0 msgr2=0x7f896414fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.261+0000 7f8962ffd700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89640b7ca0 0x7f896414fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8962ffd700 1 -- 172.21.15.140:0/3691633936 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f896405bfa0 msgr2=0x7f8964111450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8962ffd700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f896405bfa0 0x7f8964111450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8962ffd700 1 -- 172.21.15.140:0/3691633936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8958009470 con 0x7f8964066670 2022-07-21T01:55:12.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8963fff700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89640b7ca0 0x7f896414fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:12.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8962ffd700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8964066670 0x7f8964111e90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f895800c000 tx=0x7f89580091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:12.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8960ff9700 1 -- 172.21.15.140:0/3691633936 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f895800c850 con 0x7f8964066670 2022-07-21T01:55:12.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8960ff9700 1 -- 172.21.15.140:0/3691633936 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895800c9b0 con 0x7f8964066670 2022-07-21T01:55:12.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8969eb5700 1 -- 172.21.15.140:0/3691633936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f895400b180 con 0x7f8964066670 2022-07-21T01:55:12.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8960ff9700 1 -- 172.21.15.140:0/3691633936 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f895800cd10 con 0x7f8964066670 2022-07-21T01:55:12.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8969eb5700 1 -- 172.21.15.140:0/3691633936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89641141e0 con 0x7f8964066670 2022-07-21T01:55:12.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8969eb5700 1 -- 172.21.15.140:0/3691633936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893c003410 con 0x7f8964066670 2022-07-21T01:55:12.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8960ff9700 1 -- 172.21.15.140:0/3691633936 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8958022070 con 0x7f8964066670 2022-07-21T01:55:12.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8960ff9700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8944053b00 0x7f8944055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f8960ff9700 1 -- 172.21.15.140:0/3691633936 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f895806c330 con 0x7f8964066670 2022-07-21T01:55:12.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f89637fe700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8944053b00 0x7f8944055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.265+0000 7f89637fe700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8944053b00 0x7f8944055fc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f895c00d000 tx=0x7f895c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:12.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.273+0000 7f8960ff9700 1 -- 172.21.15.140:0/3691633936 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8958043460 con 0x7f8964066670 2022-07-21T01:55:12.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.397+0000 7f8969eb5700 1 -- 172.21.15.140:0/3691633936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f893c003c40 con 0x7f8964066670 2022-07-21T01:55:12.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.405+0000 7f8960ff9700 1 -- 172.21.15.140:0/3691633936 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8958046a80 con 0x7f8964066670 2022-07-21T01:55:12.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 -- 172.21.15.140:0/3691633936 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8944053b00 msgr2=0x7f8944055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8944053b00 0x7f8944055fc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f895c00d000 tx=0x7f895c009040).stop 2022-07-21T01:55:12.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 -- 172.21.15.140:0/3691633936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8964066670 msgr2=0x7f8964111e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8964066670 0x7f8964111e90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f895800c000 tx=0x7f89580091d0).stop 2022-07-21T01:55:12.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 -- 172.21.15.140:0/3691633936 shutdown_connections 2022-07-21T01:55:12.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8944053b00 0x7f8944055fc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89640b7ca0 0x7f896414fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8964066670 0x7f8964111e90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 --2- 172.21.15.140:0/3691633936 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f896405bfa0 0x7f8964111450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 -- 172.21.15.140:0/3691633936 >> 172.21.15.140:0/3691633936 conn(0x7f8964065760 msgr2=0x7f8964140e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:12.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 -- 172.21.15.140:0/3691633936 shutdown_connections 2022-07-21T01:55:12.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.409+0000 7f8969eb5700 1 -- 172.21.15.140:0/3691633936 wait complete. 2022-07-21T01:55:12.424 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2022-07-21T01:55:12.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f19808b7700 1 Processor -- start 2022-07-21T01:55:12.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f19808b7700 1 -- start start 2022-07-21T01:55:12.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f19808b7700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f197c14ada0 0x7f197c146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f19808b7700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c0afbc0 con 0x7f197c0b59d0 2022-07-21T01:55:12.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f19808b7700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c0b00e0 con 0x7f197c0b14f0 2022-07-21T01:55:12.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f19808b7700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197c148080 con 0x7f197c14ada0 2022-07-21T01:55:12.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f197ad9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f197c0b59d0 0x7f197c0b07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51266/0 (socket says 172.21.15.140:51266) 2022-07-21T01:55:12.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f197ad9d700 1 -- 172.21.15.140:0/3726346892 learned_addr learned my addr 172.21.15.140:0/3726346892 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:12.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f197a59c700 1 --2- 172.21.15.140:0/3726346892 >> v2:172.21.15.152:3300/0 conn(0x7f197c14ada0 0x7f197c146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347251871 0 0) 0x7f197c0afbc0 con 0x7f197c0b59d0 2022-07-21T01:55:12.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1960003260 con 0x7f197c0b59d0 2022-07-21T01:55:12.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3700550374 0 0) 0x7f1960003260 con 0x7f197c0b59d0 2022-07-21T01:55:12.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 >> v2:172.21.15.152:3300/0 conn(0x7f197c14ada0 msgr2=0x7f197c146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:12.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 --2- 172.21.15.140:0/3726346892 >> v2:172.21.15.152:3300/0 conn(0x7f197c14ada0 0x7f197c146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 >> v1:172.21.15.152:6789/0 conn(0x7f197c0b14f0 legacy=0x7f197c146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0b5db0 con 0x7f197c0b59d0 2022-07-21T01:55:12.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f1974001a00 con 0x7f197c0b59d0 2022-07-21T01:55:12.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 >> v1:172.21.15.140:6789/0 conn(0x7f197c0b59d0 legacy=0x7f197c0b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 --2- 172.21.15.140:0/3726346892 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1960003750 0x7f1960003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 --2- 172.21.15.140:0/3726346892 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 0x7f1960004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 --2- 172.21.15.140:0/3726346892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1960004e70 0x7f196000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c148080 con 0x7f1960004500 2022-07-21T01:55:12.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196000bca0 con 0x7f1960004e70 2022-07-21T01:55:12.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196000be60 con 0x7f1960003750 2022-07-21T01:55:12.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f197a59c700 1 --2- 172.21.15.140:0/3726346892 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 0x7f1960004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f197ad9d700 1 --2- 172.21.15.140:0/3726346892 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1960003750 0x7f1960003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.569+0000 7f197b59e700 1 --2- 172.21.15.140:0/3726346892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1960004e70 0x7f196000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197a59c700 1 -- 172.21.15.140:0/3726346892 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1960003750 msgr2=0x7f1960003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:12.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197a59c700 1 --2- 172.21.15.140:0/3726346892 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1960003750 0x7f1960003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197a59c700 1 -- 172.21.15.140:0/3726346892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1960004e70 msgr2=0x7f196000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:12.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197a59c700 1 --2- 172.21.15.140:0/3726346892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1960004e70 0x7f196000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197a59c700 1 -- 172.21.15.140:0/3726346892 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960003260 con 0x7f1960004500 2022-07-21T01:55:12.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197b59e700 1 -- 172.21.15.140:0/3726346892 reap_dead start 2022-07-21T01:55:12.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197a59c700 1 --2- 172.21.15.140:0/3726346892 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 0x7f1960004980 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f197000c000 tx=0x7f1970006df0).ready entity=mon.0 client_cookie=92211610abdd33b4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:12.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f197000e5f0 con 0x7f1960004500 2022-07-21T01:55:12.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197000e750 con 0x7f1960004500 2022-07-21T01:55:12.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f1979d9b700 1 -- 172.21.15.140:0/3726346892 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f197000eab0 con 0x7f1960004500 2022-07-21T01:55:12.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 -- 172.21.15.140:0/3726346892 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 msgr2=0x7f1960004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 --2- 172.21.15.140:0/3726346892 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 0x7f1960004980 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f197000c000 tx=0x7f1970006df0).stop 2022-07-21T01:55:12.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 -- 172.21.15.140:0/3726346892 shutdown_connections 2022-07-21T01:55:12.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 --2- 172.21.15.140:0/3726346892 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 0x7f1960004980 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 -- 172.21.15.140:0/3726346892 >> 172.21.15.140:0/3726346892 conn(0x7f197c0634c0 msgr2=0x7f197c0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:12.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 -- 172.21.15.140:0/3726346892 shutdown_connections 2022-07-21T01:55:12.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 -- 172.21.15.140:0/3726346892 wait complete. 2022-07-21T01:55:12.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 Processor -- start 2022-07-21T01:55:12.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 -- start start 2022-07-21T01:55:12.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 0x7f197c14c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f197c1499a0 0x7f197c064280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f197c065d90 0x7f197c142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c0b3b00 con 0x7f1960004500 2022-07-21T01:55:12.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c14a6c0 con 0x7f197c065d90 2022-07-21T01:55:12.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f19808b7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c14a960 con 0x7f197c1499a0 2022-07-21T01:55:12.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 0x7f197c14c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 0x7f197c14c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37132/0 (socket says 172.21.15.140:37132) 2022-07-21T01:55:12.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197ad9d700 1 -- 172.21.15.140:0/1397761528 learned_addr learned my addr 172.21.15.140:0/1397761528 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:12.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197b59e700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f197c065d90 0x7f197c142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197a59c700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f197c1499a0 0x7f197c064280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197ad9d700 1 -- 172.21.15.140:0/1397761528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f197c1499a0 msgr2=0x7f197c064280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197ad9d700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f197c1499a0 0x7f197c064280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197ad9d700 1 -- 172.21.15.140:0/1397761528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f197c065d90 msgr2=0x7f197c142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:12.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.573+0000 7f197ad9d700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f197c065d90 0x7f197c142360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:12.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f197ad9d700 1 -- 172.21.15.140:0/1397761528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c063de0 con 0x7f1960004500 2022-07-21T01:55:12.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f197ad9d700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 0x7f197c14c710 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f197400d000 tx=0x7f19740021d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:12.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f196bfff700 1 -- 172.21.15.140:0/1397761528 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f197400fa60 con 0x7f1960004500 2022-07-21T01:55:12.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f196bfff700 1 -- 172.21.15.140:0/1397761528 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197400fbc0 con 0x7f1960004500 2022-07-21T01:55:12.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f19808b7700 1 -- 172.21.15.140:0/1397761528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c067770 con 0x7f1960004500 2022-07-21T01:55:12.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f196bfff700 1 -- 172.21.15.140:0/1397761528 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1974012350 con 0x7f1960004500 2022-07-21T01:55:12.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f19808b7700 1 -- 172.21.15.140:0/1397761528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c066210 con 0x7f1960004500 2022-07-21T01:55:12.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f19808b7700 1 -- 172.21.15.140:0/1397761528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c0b0340 con 0x7f1960004500 2022-07-21T01:55:12.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f196bfff700 1 -- 172.21.15.140:0/1397761528 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f197401b030 con 0x7f1960004500 2022-07-21T01:55:12.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f196bfff700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1958053b30 0x7f1958055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:12.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f196bfff700 1 -- 172.21.15.140:0/1397761528 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f197406bb40 con 0x7f1960004500 2022-07-21T01:55:12.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.577+0000 7f197a59c700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1958053b30 0x7f1958055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:12.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.581+0000 7f197a59c700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1958053b30 0x7f1958055ff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f197001b000 tx=0x7f1970017000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:12.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.585+0000 7f196bfff700 1 -- 172.21.15.140:0/1397761528 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1974042d10 con 0x7f1960004500 2022-07-21T01:55:12.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:12.753+0000 7f19808b7700 1 -- 172.21.15.140:0/1397761528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7f197c0574e0 con 0x7f1960004500 2022-07-21T01:55:13.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.129+0000 7f196bfff700 1 -- 172.21.15.140:0/1397761528 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v290) v1 ==== 139+0+0 (secure 0 0 0) 0x7f1974019030 con 0x7f1960004500 2022-07-21T01:55:13.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 -- 172.21.15.140:0/1397761528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1958053b30 msgr2=0x7f1958055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:13.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1958053b30 0x7f1958055ff0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f197001b000 tx=0x7f1970017000).stop 2022-07-21T01:55:13.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 -- 172.21.15.140:0/1397761528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 msgr2=0x7f197c14c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:13.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 0x7f197c14c710 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f197400d000 tx=0x7f19740021d0).stop 2022-07-21T01:55:13.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 -- 172.21.15.140:0/1397761528 shutdown_connections 2022-07-21T01:55:13.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1958053b30 0x7f1958055ff0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:13.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f197c065d90 0x7f197c142360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:13.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f197c1499a0 0x7f197c064280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:13.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 --2- 172.21.15.140:0/1397761528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1960004500 0x7f197c14c710 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:13.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 -- 172.21.15.140:0/1397761528 >> 172.21.15.140:0/1397761528 conn(0x7f197c0634c0 msgr2=0x7f197c1467a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:13.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 -- 172.21.15.140:0/1397761528 shutdown_connections 2022-07-21T01:55:13.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.133+0000 7f19808b7700 1 -- 172.21.15.140:0/1397761528 wait complete. 2022-07-21T01:55:13.150 INFO:tasks.cephfs_test_runner:test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) ... ok 2022-07-21T01:55:13.150 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode (tasks.cephfs.test_misc.TestMisc) 2022-07-21T01:55:13.151 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2022-07-21T01:55:13.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef62c2700 1 Processor -- start 2022-07-21T01:55:13.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef62c2700 1 -- start start 2022-07-21T01:55:13.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef62c2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ef00b4440 0x7f4ef0143590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:13.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef62c2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef00b6c60 con 0x7f4ef0145f90 2022-07-21T01:55:13.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef62c2700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef0142bb0 con 0x7f4ef00b6880 2022-07-21T01:55:13.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef62c2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef01429c0 con 0x7f4ef00b4440 2022-07-21T01:55:13.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef5ac1700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4ef0145f90 0x7f4ef013fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51288/0 (socket says 172.21.15.140:51288) 2022-07-21T01:55:13.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef5ac1700 1 -- 172.21.15.140:0/3207115362 learned_addr learned my addr 172.21.15.140:0/3207115362 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:13.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef4abf700 1 --2- 172.21.15.140:0/3207115362 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ef00b4440 0x7f4ef0143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:13.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4eeffff700 1 -- 172.21.15.140:0/3207115362 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26722518 0 0) 0x7f4ef00b6c60 con 0x7f4ef0145f90 2022-07-21T01:55:13.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4eeffff700 1 -- 172.21.15.140:0/3207115362 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed0003260 con 0x7f4ef0145f90 2022-07-21T01:55:13.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef4abf700 1 -- 172.21.15.140:0/3207115362 >> v1:172.21.15.152:6789/0 conn(0x7f4ef00b6880 legacy=0x7f4ef00b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:13.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef4abf700 1 -- 172.21.15.140:0/3207115362 >> v1:172.21.15.140:6789/0 conn(0x7f4ef0145f90 legacy=0x7f4ef013fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:13.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef4abf700 1 -- 172.21.15.140:0/3207115362 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef0142e50 con 0x7f4ef00b4440 2022-07-21T01:55:13.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4ef4abf700 1 --2- 172.21.15.140:0/3207115362 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ef00b4440 0x7f4ef0143590 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f4ed8002f70 tx=0x7f4ed8006df0).ready entity=mon.2 client_cookie=cceae8ae70fbf0a0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:13.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4eeffff700 1 -- 172.21.15.140:0/3207115362 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ed800d570 con 0x7f4ef00b4440 2022-07-21T01:55:13.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4eeffff700 1 -- 172.21.15.140:0/3207115362 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed8000e90 con 0x7f4ef00b4440 2022-07-21T01:55:13.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.257+0000 7f4eeffff700 1 -- 172.21.15.140:0/3207115362 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ed800d8c0 con 0x7f4ef00b4440 2022-07-21T01:55:13.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3207115362 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ef00b4440 msgr2=0x7f4ef0143590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:13.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 --2- 172.21.15.140:0/3207115362 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ef00b4440 0x7f4ef0143590 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f4ed8002f70 tx=0x7f4ed8006df0).stop 2022-07-21T01:55:13.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3207115362 shutdown_connections 2022-07-21T01:55:13.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 --2- 172.21.15.140:0/3207115362 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ef00b4440 0x7f4ef0143590 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:13.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3207115362 >> 172.21.15.140:0/3207115362 conn(0x7f4ef00b9e60 msgr2=0x7f4ef00b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:13.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3207115362 shutdown_connections 2022-07-21T01:55:13.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3207115362 wait complete. 2022-07-21T01:55:13.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 Processor -- start 2022-07-21T01:55:13.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 -- start start 2022-07-21T01:55:13.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ef00b4440 0x7f4ef00b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:13.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef00b6880 0x7f4ef0144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:13.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ef0145f90 0x7f4ef0152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:13.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef0151ea0 con 0x7f4ef0145f90 2022-07-21T01:55:13.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00aee00 con 0x7f4ef00b6880 2022-07-21T01:55:13.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef62c2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef00b2f10 con 0x7f4ef00b4440 2022-07-21T01:55:13.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef5ac1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ef0145f90 0x7f4ef0152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:13.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef5ac1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ef0145f90 0x7f4ef0152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37150/0 (socket says 172.21.15.140:37150) 2022-07-21T01:55:13.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef5ac1700 1 -- 172.21.15.140:0/3185492370 learned_addr learned my addr 172.21.15.140:0/3185492370 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:13.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef4abf700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef00b6880 0x7f4ef0144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:13.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.261+0000 7f4ef52c0700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ef00b4440 0x7f4ef00b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:13.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4ef5ac1700 1 -- 172.21.15.140:0/3185492370 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ef00b4440 msgr2=0x7f4ef00b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:13.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4ef5ac1700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ef00b4440 0x7f4ef00b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:13.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4ef5ac1700 1 -- 172.21.15.140:0/3185492370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef00b6880 msgr2=0x7f4ef0144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:13.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4ef5ac1700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef00b6880 0x7f4ef0144a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:13.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4ef5ac1700 1 -- 172.21.15.140:0/3185492370 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed800b180 con 0x7f4ef0145f90 2022-07-21T01:55:13.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4ef5ac1700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ef0145f90 0x7f4ef0152530 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f4ee0001110 tx=0x7f4ee0007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:13.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4eee7fc700 1 -- 172.21.15.140:0/3185492370 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ee00070d0 con 0x7f4ef0145f90 2022-07-21T01:55:13.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4eee7fc700 1 -- 172.21.15.140:0/3185492370 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee0007230 con 0x7f4ef0145f90 2022-07-21T01:55:13.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3185492370 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef00b3130 con 0x7f4ef0145f90 2022-07-21T01:55:13.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4eee7fc700 1 -- 172.21.15.140:0/3185492370 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ee0005610 con 0x7f4ef0145f90 2022-07-21T01:55:13.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3185492370 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef0144f70 con 0x7f4ef0145f90 2022-07-21T01:55:13.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3185492370 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef01430f0 con 0x7f4ef0145f90 2022-07-21T01:55:13.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4eee7fc700 1 -- 172.21.15.140:0/3185492370 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4ee001a060 con 0x7f4ef0145f90 2022-07-21T01:55:13.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4eee7fc700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ed4053b00 0x7f4ed4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:13.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4eee7fc700 1 -- 172.21.15.140:0/3185492370 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f4ee006b3d0 con 0x7f4ef0145f90 2022-07-21T01:55:13.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4ef52c0700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ed4053b00 0x7f4ed4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:13.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.265+0000 7f4ef52c0700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ed4053b00 0x7f4ed4055fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f4ee800b000 tx=0x7f4ee8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:13.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.269+0000 7f4eee7fc700 1 -- 172.21.15.140:0/3185492370 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4ee00425a0 con 0x7f4ef0145f90 2022-07-21T01:55:13.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:13.437+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3185492370 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f4ef00574e0 con 0x7f4ef0145f90 2022-07-21T01:55:14.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.133+0000 7f4eee7fc700 1 -- 172.21.15.140:0/3185492370 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v291) v1 ==== 129+0+0 (secure 0 0 0) 0x7f4ee0018070 con 0x7f4ef0145f90 2022-07-21T01:55:14.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3185492370 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ed4053b00 msgr2=0x7f4ed4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ed4053b00 0x7f4ed4055fc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f4ee800b000 tx=0x7f4ee8009040).stop 2022-07-21T01:55:14.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3185492370 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ef0145f90 msgr2=0x7f4ef0152530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ef0145f90 0x7f4ef0152530 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f4ee0001110 tx=0x7f4ee0007a80).stop 2022-07-21T01:55:14.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3185492370 shutdown_connections 2022-07-21T01:55:14.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4ed4053b00 0x7f4ed4055fc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ef0145f90 0x7f4ef0152530 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ef00b6880 0x7f4ef0144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 --2- 172.21.15.140:0/3185492370 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ef00b4440 0x7f4ef00b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3185492370 >> 172.21.15.140:0/3185492370 conn(0x7f4ef00b9e60 msgr2=0x7f4ef013ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:14.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3185492370 shutdown_connections 2022-07-21T01:55:14.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.137+0000 7f4ef62c2700 1 -- 172.21.15.140:0/3185492370 wait complete. 2022-07-21T01:55:14.153 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:55:14.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.261+0000 7ff4fa127700 1 Processor -- start 2022-07-21T01:55:14.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.261+0000 7ff4fa127700 1 -- start start 2022-07-21T01:55:14.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4fa127700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f405d0e0 0x7ff4f4146920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4fa127700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff4f40b5960 0x7ff4f4142b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4fa127700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f405d4c0 con 0x7ff4f4143060 2022-07-21T01:55:14.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4fa127700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f41461e0 con 0x7ff4f40b5960 2022-07-21T01:55:14.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4fa127700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f4146480 con 0x7ff4f405d0e0 2022-07-21T01:55:14.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f3fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff4f4143060 0x7ff4f409b6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51306/0 (socket says 172.21.15.140:51306) 2022-07-21T01:55:14.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f3fff700 1 -- 172.21.15.140:0/1802036776 learned_addr learned my addr 172.21.15.140:0/1802036776 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:14.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f37fe700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f405d0e0 0x7ff4f4146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f2ffd700 1 --2- 172.21.15.140:0/1802036776 >> v2:172.21.15.152:3300/0 conn(0x7ff4f40b5960 0x7ff4f4142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714826251 0 0) 0x7ff4f405d4c0 con 0x7ff4f4143060 2022-07-21T01:55:14.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4d4003260 con 0x7ff4f4143060 2022-07-21T01:55:14.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3594824154 0 0) 0x7ff4d4003260 con 0x7ff4f4143060 2022-07-21T01:55:14.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f405d0e0 msgr2=0x7ff4f4146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:14.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f405d0e0 0x7ff4f4146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 >> v2:172.21.15.152:3300/0 conn(0x7ff4f40b5960 msgr2=0x7ff4f4142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:14.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 --2- 172.21.15.140:0/1802036776 >> v2:172.21.15.152:3300/0 conn(0x7ff4f40b5960 0x7ff4f4142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f40b01d0 con 0x7ff4f4143060 2022-07-21T01:55:14.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7ff4ec0019e0 con 0x7ff4f4143060 2022-07-21T01:55:14.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 >> v1:172.21.15.140:6789/0 conn(0x7ff4f4143060 legacy=0x7ff4f409b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4d4003750 0x7ff4d4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f405d0e0 0x7ff4d4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d40049f0 0x7ff4d4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f41461e0 con 0x7ff4d4003750 2022-07-21T01:55:14.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f4146480 con 0x7ff4d40049f0 2022-07-21T01:55:14.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d400b750 con 0x7ff4f405d0e0 2022-07-21T01:55:14.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f37fe700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4d4003750 0x7ff4d4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f3fff700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d40049f0 0x7ff4d4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f2ffd700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f405d0e0 0x7ff4d4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f37fe700 1 -- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f405d0e0 msgr2=0x7ff4d4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:14.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f37fe700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f405d0e0 0x7ff4d4004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f37fe700 1 -- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d40049f0 msgr2=0x7ff4d4004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:14.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f37fe700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d40049f0 0x7ff4d4004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f37fe700 1 -- 172.21.15.140:0/1802036776 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d4003260 con 0x7ff4d4003750 2022-07-21T01:55:14.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f2ffd700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f405d0e0 0x7ff4d4004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:14.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f37fe700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4d4003750 0x7ff4d4003b70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7ff4dc00c000 tx=0x7ff4dc006df0).ready entity=mon.0 client_cookie=5ad10fe33b92ae5 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:14.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4dc00e830 con 0x7ff4d4003750 2022-07-21T01:55:14.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc00e990 con 0x7ff4d4003750 2022-07-21T01:55:14.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.265+0000 7ff4f27fc700 1 -- 172.21.15.140:0/1802036776 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4dc00ecf0 con 0x7ff4d4003750 2022-07-21T01:55:14.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 -- 172.21.15.140:0/1802036776 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4d4003750 msgr2=0x7ff4d4003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 --2- 172.21.15.140:0/1802036776 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4d4003750 0x7ff4d4003b70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7ff4dc00c000 tx=0x7ff4dc006df0).stop 2022-07-21T01:55:14.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f3fff700 1 -- 172.21.15.140:0/1802036776 reap_dead start 2022-07-21T01:55:14.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 -- 172.21.15.140:0/1802036776 shutdown_connections 2022-07-21T01:55:14.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 -- 172.21.15.140:0/1802036776 >> 172.21.15.140:0/1802036776 conn(0x7ff4f40b6a20 msgr2=0x7ff4f4066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:14.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 -- 172.21.15.140:0/1802036776 shutdown_connections 2022-07-21T01:55:14.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 -- 172.21.15.140:0/1802036776 wait complete. 2022-07-21T01:55:14.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 Processor -- start 2022-07-21T01:55:14.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 -- start start 2022-07-21T01:55:14.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d4003750 0x7ff4f40b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4f40b3350 0x7ff4f4146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f4146d30 0x7ff4f4152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f41471e0 con 0x7ff4f40b3350 2022-07-21T01:55:14.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f40b2dc0 con 0x7ff4d4003750 2022-07-21T01:55:14.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f4149a90 con 0x7ff4f4146d30 2022-07-21T01:55:14.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f2ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4f40b3350 0x7ff4f4146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f2ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4f40b3350 0x7ff4f4146840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37170/0 (socket says 172.21.15.140:37170) 2022-07-21T01:55:14.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f2ffd700 1 -- 172.21.15.140:0/4206388560 learned_addr learned my addr 172.21.15.140:0/4206388560 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:14.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f37fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d4003750 0x7ff4f40b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f3fff700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f4146d30 0x7ff4f4152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f2ffd700 1 -- 172.21.15.140:0/4206388560 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f4146d30 msgr2=0x7ff4f4152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f2ffd700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f4146d30 0x7ff4f4152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f2ffd700 1 -- 172.21.15.140:0/4206388560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d4003750 msgr2=0x7ff4f40b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f2ffd700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d4003750 0x7ff4f40b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f2ffd700 1 -- 172.21.15.140:0/4206388560 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f4149cf0 con 0x7ff4f40b3350 2022-07-21T01:55:14.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f3fff700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f4146d30 0x7ff4f4152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:14.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f2ffd700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4f40b3350 0x7ff4f4146840 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7ff4e400b000 tx=0x7ff4e4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:14.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f0ff9700 1 -- 172.21.15.140:0/4206388560 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4e4014680 con 0x7ff4f40b3350 2022-07-21T01:55:14.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f0ff9700 1 -- 172.21.15.140:0/4206388560 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e40147e0 con 0x7ff4f40b3350 2022-07-21T01:55:14.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4fa127700 1 -- 172.21.15.140:0/4206388560 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f4149f90 con 0x7ff4f40b3350 2022-07-21T01:55:14.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.269+0000 7ff4f0ff9700 1 -- 172.21.15.140:0/4206388560 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4e4014b40 con 0x7ff4f40b3350 2022-07-21T01:55:14.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.273+0000 7ff4fa127700 1 -- 172.21.15.140:0/4206388560 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f4152720 con 0x7ff4f40b3350 2022-07-21T01:55:14.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.273+0000 7ff4fa127700 1 -- 172.21.15.140:0/4206388560 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f40afdc0 con 0x7ff4f40b3350 2022-07-21T01:55:14.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.273+0000 7ff4f0ff9700 1 -- 172.21.15.140:0/4206388560 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff4e4024070 con 0x7ff4f40b3350 2022-07-21T01:55:14.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.273+0000 7ff4f0ff9700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4d8053aa0 0x7ff4d8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.273+0000 7ff4f37fe700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4d8053aa0 0x7ff4d8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.273+0000 7ff4f0ff9700 1 -- 172.21.15.140:0/4206388560 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff4e406c3f0 con 0x7ff4f40b3350 2022-07-21T01:55:14.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.277+0000 7ff4f37fe700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4d8053aa0 0x7ff4d8055f60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7ff4dc01a000 tx=0x7ff4dc019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:14.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.281+0000 7ff4f0ff9700 1 -- 172.21.15.140:0/4206388560 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff4e40435c0 con 0x7ff4f40b3350 2022-07-21T01:55:14.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.405+0000 7ff4fa127700 1 -- 172.21.15.140:0/4206388560 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff4f40574e0 con 0x7ff4f40b3350 2022-07-21T01:55:14.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4f0ff9700 1 -- 172.21.15.140:0/4206388560 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff4e4046be0 con 0x7ff4f40b3350 2022-07-21T01:55:14.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 -- 172.21.15.140:0/4206388560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4d8053aa0 msgr2=0x7ff4d8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4d8053aa0 0x7ff4d8055f60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7ff4dc01a000 tx=0x7ff4dc019000).stop 2022-07-21T01:55:14.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 -- 172.21.15.140:0/4206388560 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4f40b3350 msgr2=0x7ff4f4146840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4f40b3350 0x7ff4f4146840 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7ff4e400b000 tx=0x7ff4e4008040).stop 2022-07-21T01:55:14.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 -- 172.21.15.140:0/4206388560 shutdown_connections 2022-07-21T01:55:14.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4d8053aa0 0x7ff4d8055f60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4f4146d30 0x7ff4f4152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4f40b3350 0x7ff4f4146840 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 --2- 172.21.15.140:0/4206388560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4d4003750 0x7ff4f40b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 -- 172.21.15.140:0/4206388560 >> 172.21.15.140:0/4206388560 conn(0x7ff4f40b6a20 msgr2=0x7ff4f40b8980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:14.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 -- 172.21.15.140:0/4206388560 shutdown_connections 2022-07-21T01:55:14.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.413+0000 7ff4fa127700 1 -- 172.21.15.140:0/4206388560 wait complete. 2022-07-21T01:55:14.430 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:55:14.430 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:14.448 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:14.468 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:14.489 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:14.511 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:14.533 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:55:14.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:14.534 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:14.535 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:14.542 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:55:14.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:14.544 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:55:14.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:14.545 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:14.545 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:14.561 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:55:14.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:14.563 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:55:14.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:14.564 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:14.564 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:55:14.571 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:55:14.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:14.573 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:55:14.574 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:55:14.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.681+0000 7f8a18301700 1 Processor -- start 2022-07-21T01:55:14.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.681+0000 7f8a18301700 1 -- start start 2022-07-21T01:55:14.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.681+0000 7f8a18301700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8a100b9ef0 0x7f8a100b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.681+0000 7f8a18301700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a100ba2d0 con 0x7f8a100b2960 2022-07-21T01:55:14.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.681+0000 7f8a18301700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a100b4560 con 0x7f8a1005d160 2022-07-21T01:55:14.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.681+0000 7f8a18301700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a100b4800 con 0x7f8a100b9ef0 2022-07-21T01:55:14.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.681+0000 7f8a1689e700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f8a100b2960 0x7f8a10142210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51326/0 (socket says 172.21.15.140:51326) 2022-07-21T01:55:14.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.681+0000 7f8a1689e700 1 -- 172.21.15.140:0/2860139330 learned_addr learned my addr 172.21.15.140:0/2860139330 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:14.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.681+0000 7f8a1609d700 1 --2- 172.21.15.140:0/2860139330 >> v2:172.21.15.152:3300/0 conn(0x7f8a100b9ef0 0x7f8a100b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.681+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778481249 0 0) 0x7f8a100ba2d0 con 0x7f8a100b2960 2022-07-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.681+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89f8003260 con 0x7f8a100b2960 2022-07-21T01:55:14.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582700639 0 0) 0x7f8a100b4560 con 0x7f8a1005d160 2022-07-21T01:55:14.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a100ba2d0 con 0x7f8a1005d160 2022-07-21T01:55:14.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3424176774 0 0) 0x7f89f8003260 con 0x7f8a100b2960 2022-07-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 >> v2:172.21.15.152:3300/0 conn(0x7f8a100b9ef0 msgr2=0x7f8a100b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:14.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 --2- 172.21.15.140:0/2860139330 >> v2:172.21.15.152:3300/0 conn(0x7f8a100b9ef0 0x7f8a100b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 >> v1:172.21.15.152:6789/0 conn(0x7f8a1005d160 legacy=0x7f8a100b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a100b1db0 con 0x7f8a100b2960 2022-07-21T01:55:14.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f8a080019e0 con 0x7f8a100b2960 2022-07-21T01:55:14.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 >> v1:172.21.15.140:6789/0 conn(0x7f8a100b2960 legacy=0x7f8a10142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 --2- 172.21.15.140:0/2860139330 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89f80038e0 0x7f89f8003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 --2- 172.21.15.140:0/2860139330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89f8004500 0x7f89f8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 --2- 172.21.15.140:0/2860139330 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 0x7f89f800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a100b4800 con 0x7f89f8004ec0 2022-07-21T01:55:14.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f800be30 con 0x7f89f8004500 2022-07-21T01:55:14.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f800bff0 con 0x7f89f80038e0 2022-07-21T01:55:14.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1689e700 1 --2- 172.21.15.140:0/2860139330 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 0x7f89f800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1609d700 1 --2- 172.21.15.140:0/2860139330 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89f80038e0 0x7f89f8003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1589c700 1 --2- 172.21.15.140:0/2860139330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89f8004500 0x7f89f8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1689e700 1 -- 172.21.15.140:0/2860139330 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89f80038e0 msgr2=0x7f89f8003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1689e700 1 --2- 172.21.15.140:0/2860139330 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89f80038e0 0x7f89f8003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1689e700 1 -- 172.21.15.140:0/2860139330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89f8004500 msgr2=0x7f89f8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:14.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1689e700 1 --2- 172.21.15.140:0/2860139330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89f8004500 0x7f89f8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1689e700 1 -- 172.21.15.140:0/2860139330 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f8003260 con 0x7f89f8004ec0 2022-07-21T01:55:14.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1689e700 1 -- 172.21.15.140:0/2860139330 reap_dead start 2022-07-21T01:55:14.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1689e700 1 --2- 172.21.15.140:0/2860139330 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 0x7f89f800b750 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f8a080017b0 tx=0x7f8a100b1910).ready entity=mon.0 client_cookie=f8d4d224d514514 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a0800cbd0 con 0x7f89f8004ec0 2022-07-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0800cd30 con 0x7f89f8004ec0 2022-07-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a1509b700 1 -- 172.21.15.140:0/2860139330 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a0800cea0 con 0x7f89f8004ec0 2022-07-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a18301700 1 -- 172.21.15.140:0/2860139330 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 msgr2=0x7f89f800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a18301700 1 --2- 172.21.15.140:0/2860139330 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 0x7f89f800b750 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f8a080017b0 tx=0x7f8a100b1910).stop 2022-07-21T01:55:14.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a18301700 1 -- 172.21.15.140:0/2860139330 shutdown_connections 2022-07-21T01:55:14.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a18301700 1 --2- 172.21.15.140:0/2860139330 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 0x7f89f800b750 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a18301700 1 -- 172.21.15.140:0/2860139330 >> 172.21.15.140:0/2860139330 conn(0x7f8a100679d0 msgr2=0x7f8a100b94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:14.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a18301700 1 -- 172.21.15.140:0/2860139330 shutdown_connections 2022-07-21T01:55:14.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.685+0000 7f8a18301700 1 -- 172.21.15.140:0/2860139330 wait complete. 2022-07-21T01:55:14.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a18301700 1 Processor -- start 2022-07-21T01:55:14.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a18301700 1 -- start start 2022-07-21T01:55:14.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a18301700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 0x7f8a10140b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a18301700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a10144430 0x7f8a10146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a18301700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a100af3d0 0x7f8a10146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a18301700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a100ae7e0 con 0x7f89f8004ec0 2022-07-21T01:55:14.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a18301700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a100aeeb0 con 0x7f8a10144430 2022-07-21T01:55:14.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a18301700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a10145230 con 0x7f8a100af3d0 2022-07-21T01:55:14.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1609d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 0x7f8a10140b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1609d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 0x7f8a10140b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37188/0 (socket says 172.21.15.140:37188) 2022-07-21T01:55:14.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1609d700 1 -- 172.21.15.140:0/1932891187 learned_addr learned my addr 172.21.15.140:0/1932891187 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:14.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1589c700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a10144430 0x7f8a10146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1689e700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a100af3d0 0x7f8a10146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1609d700 1 -- 172.21.15.140:0/1932891187 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a100af3d0 msgr2=0x7f8a10146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1609d700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a100af3d0 0x7f8a10146f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1609d700 1 -- 172.21.15.140:0/1932891187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a10144430 msgr2=0x7f8a10146a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:14.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1609d700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a10144430 0x7f8a10146a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:14.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1609d700 1 -- 172.21.15.140:0/1932891187 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a10145490 con 0x7f89f8004ec0 2022-07-21T01:55:14.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1689e700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a100af3d0 0x7f8a10146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:14.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a1609d700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 0x7f8a10140b20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f8a0c00b000 tx=0x7f8a0c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:14.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f89ff7fe700 1 -- 172.21.15.140:0/1932891187 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a0c00b460 con 0x7f89f8004ec0 2022-07-21T01:55:14.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f89ff7fe700 1 -- 172.21.15.140:0/1932891187 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0c00f730 con 0x7f89f8004ec0 2022-07-21T01:55:14.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f89ff7fe700 1 -- 172.21.15.140:0/1932891187 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a0c00f8a0 con 0x7f89f8004ec0 2022-07-21T01:55:14.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.689+0000 7f8a18301700 1 -- 172.21.15.140:0/1932891187 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a100aea10 con 0x7f89f8004ec0 2022-07-21T01:55:14.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.693+0000 7f8a18301700 1 -- 172.21.15.140:0/1932891187 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a10147620 con 0x7f89f8004ec0 2022-07-21T01:55:14.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.693+0000 7f8a18301700 1 -- 172.21.15.140:0/1932891187 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a100b3780 con 0x7f89f8004ec0 2022-07-21T01:55:14.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.693+0000 7f89ff7fe700 1 -- 172.21.15.140:0/1932891187 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8a0c024070 con 0x7f89f8004ec0 2022-07-21T01:55:14.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.693+0000 7f89ff7fe700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89f0053b30 0x7f89f0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:14.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.693+0000 7f89ff7fe700 1 -- 172.21.15.140:0/1932891187 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f8a0c06c6a0 con 0x7f89f8004ec0 2022-07-21T01:55:14.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.693+0000 7f8a1589c700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89f0053b30 0x7f89f0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:14.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.693+0000 7f8a1589c700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89f0053b30 0x7f89f0055ff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f8a00003000 tx=0x7f8a00007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:14.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.697+0000 7f89ff7fe700 1 -- 172.21.15.140:0/1932891187 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8a0c043760 con 0x7f89f8004ec0 2022-07-21T01:55:14.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.825+0000 7f8a18301700 1 -- 172.21.15.140:0/1932891187 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8a100574e0 con 0x7f89f8004ec0 2022-07-21T01:55:14.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.825+0000 7f89ff7fe700 1 -- 172.21.15.140:0/1932891187 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v77) v1 ==== 81+0+28997 (secure 0 0 0) 0x7f8a0c046d80 con 0x7f89f8004ec0 2022-07-21T01:55:14.828 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:14.829 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:55:14.829 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 77, 2022-07-21T01:55:14.829 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:55:14.830 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:55:14.830 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:55:12.117273+0000", 2022-07-21T01:55:14.830 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:55:14.830 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:55:14.831 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:55:14.831 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:55:14.831 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:55:14.831 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:55:14.832 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:55:14.832 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:55:14.832 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:55:14.832 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:14.833 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:55:14.833 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:55:14.833 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:55:14.833 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:55:14.834 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:55:14.834 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 14, 2022-07-21T01:55:14.834 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:55:14.834 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:55:14.835 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:55:14.835 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:55:14.835 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:55:14.836 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.836 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:55:14.836 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:55:14.836 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:55:14.837 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:14.837 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:14.837 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:14.837 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:14.838 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:14.838 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:14.838 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:14.838 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:14.839 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:55:14.839 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:55:14.839 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:55:14.839 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:55:14.840 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:55:14.840 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:14.840 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:14.840 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:14.841 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:14.841 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:14.841 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:14.841 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:14.841 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.842 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:55:14.842 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:14.842 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:14.842 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:14.843 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:14.843 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:14.843 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:14.843 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:14.844 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:14.844 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:14.844 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:14.844 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:14.845 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:14.845 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:14.845 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:14.845 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:14.846 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:14.846 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:14.846 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:14.847 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:14.847 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:14.847 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:14.847 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:14.848 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:14.848 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:14.848 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:14.848 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:14.849 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.849 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:14.849 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:14.849 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:14.850 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:14.850 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:14.850 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:14.851 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:14.851 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:14.851 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:14.851 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:14.852 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:14.852 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:55:14.852 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:55:14.852 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.853 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:14.853 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:55:14.853 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.853 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.854 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.854 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:55:14.854 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:55:14.854 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:55:14.854 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:14.855 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:14.855 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:14.855 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:14.855 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:14.856 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:14.856 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:14.856 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:14.856 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:55:14.857 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:55:14.857 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:55:14.857 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:55:14.857 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:55:14.858 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:14.858 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:14.858 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:14.858 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:14.859 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:14.859 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:14.859 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:14.859 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.859 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:55:14.860 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:14.860 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:14.860 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:14.860 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:14.861 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:14.861 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:14.861 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:14.862 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:14.862 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:14.862 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:14.862 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:14.863 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:14.863 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:14.863 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:14.863 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:14.864 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:14.864 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:14.864 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:14.864 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:14.865 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:14.865 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:14.865 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:14.865 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:14.866 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:14.866 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:14.866 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:14.866 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.867 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:14.867 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:14.867 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:14.868 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:14.868 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:14.868 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:14.868 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:14.869 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:14.869 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:14.869 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:14.869 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:14.870 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:55:14.870 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:14.870 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:55:14.870 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.871 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.871 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:14.871 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:55:14.871 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.872 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:55:14.872 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:55:14.872 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:14.872 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:14.873 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:14.873 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:14.873 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:14.874 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:14.874 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:14.874 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 69, 2022-07-21T01:55:14.874 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:14.875 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:14.875 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:14.875 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.875 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.876 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.876 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:55:14.876 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:14.876 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.877 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.877 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.877 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:55:14.877 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:14.878 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.878 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.878 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.879 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:14.879 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.879 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.880 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.880 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:55:14.880 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:14.880 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.881 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.881 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.881 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:55:14.882 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:14.882 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.882 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.883 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.883 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:14.883 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.883 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.883 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.884 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:55:14.884 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:14.884 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.884 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.885 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.885 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:55:14.885 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:14.885 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.886 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.886 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.886 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:14.886 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.887 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.887 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.887 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:55:14.887 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:14.888 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.888 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.888 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.888 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:55:14.888 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:14.889 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.889 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.889 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.889 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:55:14.890 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:55:14.890 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:55:14.890 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:55:14.890 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:14.891 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:14.891 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:14.891 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.891 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.892 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.892 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:55:14.892 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:55:14.892 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:14.893 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:14.893 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:14.893 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:14.893 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:14.894 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:14.894 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:14.894 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 61, 2022-07-21T01:55:14.894 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:14.895 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:14.895 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:14.895 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.895 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.896 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.896 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:55:14.896 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:14.896 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.896 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.897 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.897 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:55:14.897 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:14.897 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.898 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.898 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.898 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:14.898 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.899 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.899 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.899 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:55:14.899 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:14.900 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.900 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.900 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.900 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:55:14.901 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:14.901 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.901 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.901 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.902 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:14.902 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.902 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.903 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.903 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:55:14.903 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:14.903 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.903 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.904 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.904 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:55:14.904 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:14.905 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.905 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.905 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.905 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:14.906 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.906 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.906 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.906 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:55:14.907 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:14.907 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.907 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.907 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.908 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:55:14.908 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:14.908 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.908 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.909 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.909 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:55:14.909 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:55:14.909 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:55:14.910 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:55:14.910 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:14.910 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:14.910 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:14.910 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.911 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.911 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.911 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:55:14.911 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:55:14.912 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:14.912 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:14.912 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:14.913 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:14.913 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:14.913 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:14.913 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:14.914 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 69, 2022-07-21T01:55:14.914 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:14.914 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:14.914 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:14.914 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.915 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.915 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.915 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:55:14.915 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:14.916 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.916 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.916 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.916 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:55:14.917 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:14.917 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.917 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.917 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.918 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:14.918 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.918 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.918 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.919 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:55:14.919 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:14.919 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.919 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.919 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.920 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:55:14.920 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:14.920 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.921 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.921 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.921 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:14.921 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.922 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.922 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.922 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:55:14.922 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:14.923 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.923 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.923 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.923 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:55:14.924 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:14.924 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.924 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.924 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.925 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:14.925 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.925 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.925 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.926 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:55:14.926 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:14.926 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.926 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.926 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.927 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:55:14.927 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:14.927 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.927 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.928 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.928 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:55:14.928 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:55:14.928 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:55:14.929 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:55:14.929 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:14.929 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:14.929 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:14.930 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.930 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.930 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.930 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:55:14.931 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:55:14.931 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:14.931 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:14.931 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:14.932 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:14.932 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:14.932 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:14.932 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:14.932 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 69, 2022-07-21T01:55:14.933 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:14.933 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:14.933 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:14.933 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.934 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.934 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.934 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:55:14.935 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:14.935 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.935 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.935 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.936 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:55:14.936 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:14.936 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.936 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.937 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.937 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:14.937 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.937 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.938 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.938 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:55:14.938 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:14.938 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.939 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.939 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.939 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:55:14.939 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:14.940 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.940 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.940 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.940 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:14.941 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.941 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.941 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.941 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:55:14.941 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:14.942 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.942 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.942 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.942 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:55:14.943 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:14.943 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.943 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.943 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.944 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:14.944 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.944 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.944 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.945 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:55:14.945 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:14.945 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.945 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.946 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.946 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:55:14.946 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:14.946 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.947 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.947 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.947 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:55:14.948 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:55:14.948 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:55:14.948 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:55:14.949 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:14.949 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:14.949 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:14.950 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.950 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.950 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.951 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:55:14.951 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:55:14.952 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:14.952 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:14.952 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:14.952 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:14.953 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:14.953 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:14.953 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:14.953 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 68, 2022-07-21T01:55:14.954 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:14.954 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:14.954 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:14.954 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.955 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.955 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.955 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:55:14.955 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:14.955 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.956 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.956 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.956 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:55:14.956 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:14.957 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.957 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.957 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.957 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:14.958 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.958 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.958 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.958 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:55:14.958 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:14.959 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.959 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.959 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.959 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:55:14.960 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:14.960 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.960 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.960 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.961 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:14.961 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.961 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.961 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.962 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:55:14.962 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:14.962 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.962 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.963 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.963 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:55:14.963 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:14.963 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.964 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.964 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.964 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:14.964 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.965 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.965 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.965 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:55:14.966 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:14.966 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.967 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.967 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.967 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:55:14.968 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:14.968 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.968 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.969 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.969 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:55:14.969 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:55:14.969 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:55:14.970 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:55:14.970 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:14.970 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:14.970 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:14.971 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.971 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.971 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.971 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:55:14.972 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:55:14.972 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:14.972 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:14.972 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:14.973 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:14.973 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:14.973 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:14.973 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:14.974 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 69, 2022-07-21T01:55:14.974 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:14.974 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:14.975 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:14.975 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.975 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.975 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.976 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:55:14.976 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:14.976 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.976 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.976 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.977 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:55:14.977 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:14.977 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.977 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.978 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.978 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:14.978 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.978 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.979 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.979 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:55:14.979 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:14.979 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.980 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.980 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.980 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:55:14.980 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:14.981 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.981 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.981 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.981 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:14.982 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.982 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.982 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.983 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:55:14.983 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:14.983 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.983 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.984 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.984 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:55:14.984 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:14.984 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.985 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.985 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.985 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:14.985 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.985 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.986 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.986 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:55:14.986 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:14.986 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.987 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.987 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.987 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:55:14.987 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:14.988 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.988 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.988 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.988 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:55:14.989 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:55:14.989 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:55:14.989 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:55:14.989 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:14.989 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:14.990 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:14.990 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.990 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.990 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.991 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:55:14.991 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:55:14.991 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:14.991 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:14.992 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:14.992 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:14.992 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:14.992 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:14.993 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:14.993 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 69, 2022-07-21T01:55:14.993 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:14.993 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:14.993 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:14.994 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.994 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.994 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.994 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:55:14.995 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:14.995 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.995 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.995 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.996 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:55:14.996 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:14.996 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:14.996 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:14.997 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.997 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:14.997 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:14.997 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.997 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:14.998 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:55:14.998 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:14.998 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:14.999 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:14.999 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:14.999 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:55:14.999 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:14.999 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:15.000 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:15.000 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.000 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:15.000 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:15.001 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.001 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:15.001 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:55:15.001 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:15.002 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.002 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.002 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:15.002 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:55:15.003 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:15.003 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:15.003 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:15.003 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.004 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:15.004 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:15.004 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.004 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:15.005 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:55:15.005 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:15.005 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.005 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.006 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:15.006 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:55:15.006 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:15.006 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:15.007 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:15.007 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.007 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:55:15.007 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:55:15.008 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:55:15.008 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:55:15.008 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:15.009 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:15.009 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:15.009 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:15.010 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.010 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.010 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:55:15.010 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:55:15.011 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:15.011 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:15.011 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:15.011 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:15.012 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:15.012 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:15.012 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:15.012 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 68, 2022-07-21T01:55:15.012 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:15.013 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:15.013 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:15.013 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:15.014 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.014 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:15.014 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:55:15.014 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:15.015 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.015 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.015 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:15.015 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:55:15.016 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:15.016 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:15.016 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:15.016 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.016 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:15.017 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:15.017 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.017 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:15.018 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:55:15.018 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:15.018 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.018 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.018 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:15.019 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:55:15.019 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:15.019 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:15.019 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:15.020 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.020 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:15.020 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:15.020 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.021 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:15.021 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:55:15.021 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:15.021 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.022 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.022 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:15.022 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:55:15.022 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:15.023 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:15.023 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:15.023 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.024 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:15.024 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:15.024 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.024 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:15.025 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:55:15.025 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:15.025 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.025 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.025 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:15.026 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:55:15.026 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:15.026 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:15.026 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:15.027 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.027 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:55:15.027 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:55:15.028 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:55:15.028 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:55:15.029 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:15.029 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:15.029 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:15.030 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:15.030 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:15.031 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:15.031 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:55:15.032 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.032 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:55:15.032 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:15.032 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:15.033 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:15.033 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:15.033 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:15.033 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:55:15.034 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:15.034 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.034 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.034 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:55:15.035 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:15.035 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:15.035 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:15.035 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:15.036 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:15.036 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:55:15.036 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:15.036 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.037 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.037 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:55:15.037 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:15.038 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:15.038 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:15.038 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:15.038 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:15.039 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:55:15.039 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:15.039 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.039 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.040 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:55:15.040 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:15.040 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:15.040 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:15.040 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:15.041 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:15.041 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:55:15.041 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:15.042 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.042 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.042 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:55:15.042 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:15.043 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:15.043 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:15.043 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:15.043 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:15.044 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:55:15.044 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:15.044 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.044 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.045 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:55:15.045 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:15.045 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:15.045 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:15.045 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:15.046 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:15.046 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:55:15.046 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:15.047 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.047 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.047 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:55:15.048 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:15.048 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:15.048 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:15.048 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:15.049 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:15.049 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:55:15.049 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:15.049 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.049 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:15.050 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:55:15.050 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:15.050 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:15.050 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:15.051 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:15.051 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:15.051 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:55:15.051 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:15.052 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:15.052 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:15.052 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:55:15.052 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:55:15.053 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:55:15.053 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:55:15.053 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:55:15.053 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6835/167323020": "2022-07-22T01:55:08.230961+0000", 2022-07-21T01:55:15.054 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6834/167323020": "2022-07-22T01:55:08.230961+0000", 2022-07-21T01:55:15.054 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.191:0/1019294232": "2022-07-21T02:54:56.780716+0000" 2022-07-21T01:55:15.054 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.054 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:55:15.055 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:55:15.055 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:55:15.056 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:55:15.056 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:55:15.057 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:55:15.057 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:55:15.058 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:55:15.058 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:15.058 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:15.059 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:55:15.059 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:55:15.060 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:55:15.060 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:55:15.060 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:55:15.061 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:55:15.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 -- 172.21.15.140:0/1932891187 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89f0053b30 msgr2=0x7f89f0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89f0053b30 0x7f89f0055ff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f8a00003000 tx=0x7f8a00007a00).stop 2022-07-21T01:55:15.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 -- 172.21.15.140:0/1932891187 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 msgr2=0x7f8a10140b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 0x7f8a10140b20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f8a0c00b000 tx=0x7f8a0c006df0).stop 2022-07-21T01:55:15.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 -- 172.21.15.140:0/1932891187 shutdown_connections 2022-07-21T01:55:15.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89f0053b30 0x7f89f0055ff0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8a100af3d0 0x7f8a10146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a10144430 0x7f8a10146a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 --2- 172.21.15.140:0/1932891187 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89f8004ec0 0x7f8a10140b20 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 -- 172.21.15.140:0/1932891187 >> 172.21.15.140:0/1932891187 conn(0x7f8a100679d0 msgr2=0x7f8a1005bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:15.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 -- 172.21.15.140:0/1932891187 shutdown_connections 2022-07-21T01:55:15.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:14.829+0000 7f8a18301700 1 -- 172.21.15.140:0/1932891187 wait complete. 2022-07-21T01:55:15.069 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:15.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9c54b4700 1 Processor -- start 2022-07-21T01:55:15.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9c54b4700 1 -- start start 2022-07-21T01:55:15.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9c54b4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc9c01424e0 0x7fc9c00b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9c54b4700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c01433d0 con 0x7fc9c00b0570 2022-07-21T01:55:15.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9c54b4700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c0143670 con 0x7fc9c00aea10 2022-07-21T01:55:15.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9c54b4700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c0143b10 con 0x7fc9c01424e0 2022-07-21T01:55:15.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9bffff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc9c00b0570 0x7fc9c0141dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51342/0 (socket says 172.21.15.140:51342) 2022-07-21T01:55:15.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9bffff700 1 -- 172.21.15.140:0/1940498221 learned_addr learned my addr 172.21.15.140:0/1940498221 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:15.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9bf7fe700 1 --2- 172.21.15.140:0/1940498221 >> v2:172.21.15.152:3300/0 conn(0x7fc9c01424e0 0x7fc9c00b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9beffd700 1 -- 172.21.15.140:0/1940498221 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790596701 0 0) 0x7fc9c01433d0 con 0x7fc9c00b0570 2022-07-21T01:55:15.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9beffd700 1 -- 172.21.15.140:0/1940498221 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9a0003260 con 0x7fc9c00b0570 2022-07-21T01:55:15.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9bf7fe700 1 -- 172.21.15.140:0/1940498221 >> v1:172.21.15.152:6789/0 conn(0x7fc9c00aea10 legacy=0x7fc9c00ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9bf7fe700 1 -- 172.21.15.140:0/1940498221 >> v1:172.21.15.140:6789/0 conn(0x7fc9c00b0570 legacy=0x7fc9c0141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9bf7fe700 1 -- 172.21.15.140:0/1940498221 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c00b1010 con 0x7fc9c01424e0 2022-07-21T01:55:15.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9bf7fe700 1 --2- 172.21.15.140:0/1940498221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9c01424e0 0x7fc9c00b0950 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fc9a80032f0 tx=0x7fc9a80091d0).ready entity=mon.1 client_cookie=9d30c793663c97eb server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:15.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.181+0000 7fc9beffd700 1 -- 172.21.15.140:0/1940498221 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9a800b6c0 con 0x7fc9c01424e0 2022-07-21T01:55:15.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.185+0000 7fc9beffd700 1 -- 172.21.15.140:0/1940498221 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a800b820 con 0x7fc9c01424e0 2022-07-21T01:55:15.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.185+0000 7fc9beffd700 1 -- 172.21.15.140:0/1940498221 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9a800b990 con 0x7fc9c01424e0 2022-07-21T01:55:15.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.185+0000 7fc9c54b4700 1 -- 172.21.15.140:0/1940498221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9c01424e0 msgr2=0x7fc9c00b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.185+0000 7fc9c54b4700 1 --2- 172.21.15.140:0/1940498221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9c01424e0 0x7fc9c00b0950 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fc9a80032f0 tx=0x7fc9a80091d0).stop 2022-07-21T01:55:15.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.185+0000 7fc9c54b4700 1 -- 172.21.15.140:0/1940498221 shutdown_connections 2022-07-21T01:55:15.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.185+0000 7fc9c54b4700 1 --2- 172.21.15.140:0/1940498221 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9c01424e0 0x7fc9c00b0950 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.185+0000 7fc9c54b4700 1 -- 172.21.15.140:0/1940498221 >> 172.21.15.140:0/1940498221 conn(0x7fc9c00b9360 msgr2=0x7fc9c00b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:15.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.185+0000 7fc9c54b4700 1 -- 172.21.15.140:0/1940498221 shutdown_connections 2022-07-21T01:55:15.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.185+0000 7fc9c54b4700 1 -- 172.21.15.140:0/1940498221 wait complete. 2022-07-21T01:55:15.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9c54b4700 1 Processor -- start 2022-07-21T01:55:15.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9c54b4700 1 -- start start 2022-07-21T01:55:15.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9c54b4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9c00aea10 0x7fc9c0149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9c54b4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc9c00b0570 0x7fc9c00b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9c54b4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9c01424e0 0x7fc9c0151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9c54b4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c0146ad0 con 0x7fc9c00b0570 2022-07-21T01:55:15.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9c54b4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00b2f10 con 0x7fc9c00aea10 2022-07-21T01:55:15.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9c54b4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00b4700 con 0x7fc9c01424e0 2022-07-21T01:55:15.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bf7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc9c00b0570 0x7fc9c00b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bf7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc9c00b0570 0x7fc9c00b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37204/0 (socket says 172.21.15.140:37204) 2022-07-21T01:55:15.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bf7fe700 1 -- 172.21.15.140:0/2021942984 learned_addr learned my addr 172.21.15.140:0/2021942984 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:15.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bffff700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9c00aea10 0x7fc9c0149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9c4cb3700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9c01424e0 0x7fc9c0151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bf7fe700 1 -- 172.21.15.140:0/2021942984 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9c01424e0 msgr2=0x7fc9c0151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bf7fe700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9c01424e0 0x7fc9c0151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bf7fe700 1 -- 172.21.15.140:0/2021942984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9c00aea10 msgr2=0x7fc9c0149a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bf7fe700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9c00aea10 0x7fc9c0149a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bf7fe700 1 -- 172.21.15.140:0/2021942984 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a8009470 con 0x7fc9c00b0570 2022-07-21T01:55:15.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bffff700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9c00aea10 0x7fc9c0149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:15.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bf7fe700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc9c00b0570 0x7fc9c00b4ba0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fc9a80032f0 tx=0x7fc9a8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:15.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bd7fa700 1 -- 172.21.15.140:0/2021942984 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9a800b6c0 con 0x7fc9c00b0570 2022-07-21T01:55:15.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bd7fa700 1 -- 172.21.15.140:0/2021942984 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a80107a0 con 0x7fc9c00b0570 2022-07-21T01:55:15.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9c54b4700 1 -- 172.21.15.140:0/2021942984 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c0149f60 con 0x7fc9c00b0570 2022-07-21T01:55:15.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9bd7fa700 1 -- 172.21.15.140:0/2021942984 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9a8010b00 con 0x7fc9c00b0570 2022-07-21T01:55:15.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.189+0000 7fc9c54b4700 1 -- 172.21.15.140:0/2021942984 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c00b5230 con 0x7fc9c00b0570 2022-07-21T01:55:15.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.193+0000 7fc9c54b4700 1 -- 172.21.15.140:0/2021942984 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c00b3110 con 0x7fc9c00b0570 2022-07-21T01:55:15.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.193+0000 7fc9bd7fa700 1 -- 172.21.15.140:0/2021942984 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc9a800f070 con 0x7fc9c00b0570 2022-07-21T01:55:15.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.193+0000 7fc9bd7fa700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc9a4053a70 0x7fc9a4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.193+0000 7fc9bd7fa700 1 -- 172.21.15.140:0/2021942984 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fc9a806d260 con 0x7fc9c00b0570 2022-07-21T01:55:15.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.193+0000 7fc9bffff700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc9a4053a70 0x7fc9a4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.193+0000 7fc9bffff700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc9a4053a70 0x7fc9a4055f30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fc9b000b000 tx=0x7fc9b0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:15.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.201+0000 7fc9bd7fa700 1 -- 172.21.15.140:0/2021942984 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc9a8044430 con 0x7fc9c00b0570 2022-07-21T01:55:15.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.361+0000 7fc9c54b4700 1 -- 172.21.15.140:0/2021942984 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9c00ace30 con 0x7fc9c00b0570 2022-07-21T01:55:15.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.361+0000 7fc9bd7fa700 1 -- 172.21.15.140:0/2021942984 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+1215 (secure 0 0 0) 0x7fc9a8015070 con 0x7fc9c00b0570 2022-07-21T01:55:15.365 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:15.366 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":33,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/4173658089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4173658089},{"type":"v1","addr":"172.21.15.152:6833","nonce":4173658089}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":6588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/1984033450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1984033450},{"type":"v1","addr":"172.21.15.140:6835","nonce":1984033450}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[]} 2022-07-21T01:55:15.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 -- 172.21.15.140:0/2021942984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc9a4053a70 msgr2=0x7fc9a4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc9a4053a70 0x7fc9a4055f30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fc9b000b000 tx=0x7fc9b0009040).stop 2022-07-21T01:55:15.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 -- 172.21.15.140:0/2021942984 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc9c00b0570 msgr2=0x7fc9c00b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc9c00b0570 0x7fc9c00b4ba0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fc9a80032f0 tx=0x7fc9a8016000).stop 2022-07-21T01:55:15.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 -- 172.21.15.140:0/2021942984 shutdown_connections 2022-07-21T01:55:15.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc9a4053a70 0x7fc9a4055f30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9c01424e0 0x7fc9c0151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc9c00b0570 0x7fc9c00b4ba0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 --2- 172.21.15.140:0/2021942984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9c00aea10 0x7fc9c0149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 -- 172.21.15.140:0/2021942984 >> 172.21.15.140:0/2021942984 conn(0x7fc9c00b9360 msgr2=0x7fc9c01410c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:15.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 -- 172.21.15.140:0/2021942984 shutdown_connections 2022-07-21T01:55:15.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.365+0000 7fc9c54b4700 1 -- 172.21.15.140:0/2021942984 wait complete. 2022-07-21T01:55:15.370 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 33 2022-07-21T01:55:15.378 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:15.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a8bd6700 1 Processor -- start 2022-07-21T01:55:15.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a8bd6700 1 -- start start 2022-07-21T01:55:15.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a8bd6700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd2a41424e0 0x7fd2a40b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a8bd6700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2a40aea10 0x7fd2a40ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a8bd6700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a41433d0 con 0x7fd2a40b0570 2022-07-21T01:55:15.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a8bd6700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a4143b10 con 0x7fd2a40aea10 2022-07-21T01:55:15.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a8bd6700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a4143670 con 0x7fd2a41424e0 2022-07-21T01:55:15.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a3fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2a40aea10 0x7fd2a40ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a2ffd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd2a41424e0 0x7fd2a40b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a3fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2a40aea10 0x7fd2a40ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37214/0 (socket says 172.21.15.140:37214) 2022-07-21T01:55:15.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a2ffd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd2a41424e0 0x7fd2a40b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42950/0 (socket says 172.21.15.140:42950) 2022-07-21T01:55:15.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a3fff700 1 -- 172.21.15.140:0/2763888166 learned_addr learned my addr 172.21.15.140:0/2763888166 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:15.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a2ffd700 1 -- 172.21.15.140:0/2763888166 >> v2:172.21.15.140:3300/0 conn(0x7fd2a40aea10 msgr2=0x7fd2a40ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:15.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a2ffd700 1 --2- 172.21.15.140:0/2763888166 >> v2:172.21.15.140:3300/0 conn(0x7fd2a40aea10 0x7fd2a40ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a2ffd700 1 -- 172.21.15.140:0/2763888166 >> v1:172.21.15.152:6789/0 conn(0x7fd2a40b0570 legacy=0x7fd2a4141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a2ffd700 1 -- 172.21.15.140:0/2763888166 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a40b1010 con 0x7fd2a41424e0 2022-07-21T01:55:15.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a2ffd700 1 --2- 172.21.15.140:0/2763888166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a41424e0 0x7fd2a40b0950 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fd28c002f70 tx=0x7fd28c006df0).ready entity=mon.1 client_cookie=da44aa44bafdf753 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:15.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a27fc700 1 -- 172.21.15.140:0/2763888166 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd28c00d570 con 0x7fd2a41424e0 2022-07-21T01:55:15.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a27fc700 1 -- 172.21.15.140:0/2763888166 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c000e90 con 0x7fd2a41424e0 2022-07-21T01:55:15.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.477+0000 7fd2a27fc700 1 -- 172.21.15.140:0/2763888166 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd28c00d8c0 con 0x7fd2a41424e0 2022-07-21T01:55:15.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.481+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/2763888166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a41424e0 msgr2=0x7fd2a40b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.481+0000 7fd2a8bd6700 1 --2- 172.21.15.140:0/2763888166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a41424e0 0x7fd2a40b0950 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7fd28c002f70 tx=0x7fd28c006df0).stop 2022-07-21T01:55:15.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.481+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/2763888166 shutdown_connections 2022-07-21T01:55:15.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.481+0000 7fd2a8bd6700 1 --2- 172.21.15.140:0/2763888166 >> v2:172.21.15.140:3300/0 conn(0x7fd2a40aea10 0x7fd2a40ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.481+0000 7fd2a8bd6700 1 --2- 172.21.15.140:0/2763888166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a41424e0 0x7fd2a40b0950 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.481+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/2763888166 >> 172.21.15.140:0/2763888166 conn(0x7fd2a40b9360 msgr2=0x7fd2a40b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:15.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.481+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/2763888166 shutdown_connections 2022-07-21T01:55:15.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.481+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/2763888166 wait complete. 2022-07-21T01:55:15.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.481+0000 7fd2a8bd6700 1 Processor -- start 2022-07-21T01:55:15.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.481+0000 7fd2a8bd6700 1 -- start start 2022-07-21T01:55:15.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a8bd6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2a40aea10 0x7fd2a4155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a8bd6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2a40b0570 0x7fd2a4155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a8bd6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a41424e0 0x7fd2a40b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a8bd6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a40b3ef0 con 0x7fd2a40b0570 2022-07-21T01:55:15.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a8bd6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a4112d60 con 0x7fd2a41424e0 2022-07-21T01:55:15.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a8bd6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a40b4db0 con 0x7fd2a40aea10 2022-07-21T01:55:15.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a2ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2a40b0570 0x7fd2a4155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a2ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2a40b0570 0x7fd2a4155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37218/0 (socket says 172.21.15.140:37218) 2022-07-21T01:55:15.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a2ffd700 1 -- 172.21.15.140:0/197105344 learned_addr learned my addr 172.21.15.140:0/197105344 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:15.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a3fff700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a41424e0 0x7fd2a40b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a37fe700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2a40aea10 0x7fd2a4155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a2ffd700 1 -- 172.21.15.140:0/197105344 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2a40aea10 msgr2=0x7fd2a4155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a2ffd700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2a40aea10 0x7fd2a4155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a2ffd700 1 -- 172.21.15.140:0/197105344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a41424e0 msgr2=0x7fd2a40b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a2ffd700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a41424e0 0x7fd2a40b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a2ffd700 1 -- 172.21.15.140:0/197105344 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c00b180 con 0x7fd2a40b0570 2022-07-21T01:55:15.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a2ffd700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2a40b0570 0x7fd2a4155d10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7fd2a4113ea0 tx=0x7fd28c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:15.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a0ff9700 1 -- 172.21.15.140:0/197105344 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd28c016070 con 0x7fd2a40b0570 2022-07-21T01:55:15.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a0ff9700 1 -- 172.21.15.140:0/197105344 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c00d8f0 con 0x7fd2a40b0570 2022-07-21T01:55:15.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/197105344 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a40b5050 con 0x7fd2a40b0570 2022-07-21T01:55:15.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a0ff9700 1 -- 172.21.15.140:0/197105344 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd28c013d60 con 0x7fd2a40b0570 2022-07-21T01:55:15.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/197105344 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a40b3890 con 0x7fd2a40b0570 2022-07-21T01:55:15.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a0ff9700 1 -- 172.21.15.140:0/197105344 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd28c025030 con 0x7fd2a40b0570 2022-07-21T01:55:15.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a0ff9700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd288053950 0x7fd288055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a0ff9700 1 -- 172.21.15.140:0/197105344 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fd28c06caf0 con 0x7fd2a40b0570 2022-07-21T01:55:15.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/197105344 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a40b4c00 con 0x7fd2a40b0570 2022-07-21T01:55:15.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.485+0000 7fd2a37fe700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd288053950 0x7fd288055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.489+0000 7fd2a37fe700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd288053950 0x7fd288055e10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fd29400d000 tx=0x7fd294009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:15.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.493+0000 7fd2a0ff9700 1 -- 172.21.15.140:0/197105344 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd28c023070 con 0x7fd2a40b0570 2022-07-21T01:55:15.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.657+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/197105344 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2a40ace30 con 0x7fd2a40b0570 2022-07-21T01:55:15.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.657+0000 7fd2a0ff9700 1 -- 172.21.15.140:0/197105344 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+1215 (secure 0 0 0) 0x7fd28c043cc0 con 0x7fd2a40b0570 2022-07-21T01:55:15.660 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:15.660 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":33,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/4173658089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":4173658089},{"type":"v1","addr":"172.21.15.152:6833","nonce":4173658089}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":6588,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/1984033450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1984033450},{"type":"v1","addr":"172.21.15.140:6835","nonce":1984033450}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":33}],"filesystems":[]} 2022-07-21T01:55:15.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.657+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/197105344 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd288053950 msgr2=0x7fd288055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.657+0000 7fd2a8bd6700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd288053950 0x7fd288055e10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fd29400d000 tx=0x7fd294009040).stop 2022-07-21T01:55:15.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.657+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/197105344 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2a40b0570 msgr2=0x7fd2a4155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.657+0000 7fd2a8bd6700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2a40b0570 0x7fd2a4155d10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7fd2a4113ea0 tx=0x7fd28c014000).stop 2022-07-21T01:55:15.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.661+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/197105344 shutdown_connections 2022-07-21T01:55:15.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.661+0000 7fd2a8bd6700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd288053950 0x7fd288055e10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.661+0000 7fd2a8bd6700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a41424e0 0x7fd2a40b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.661+0000 7fd2a8bd6700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2a40b0570 0x7fd2a4155d10 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.661+0000 7fd2a8bd6700 1 --2- 172.21.15.140:0/197105344 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2a40aea10 0x7fd2a4155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.661+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/197105344 >> 172.21.15.140:0/197105344 conn(0x7fd2a40b9360 msgr2=0x7fd2a413f250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:15.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.661+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/197105344 shutdown_connections 2022-07-21T01:55:15.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.661+0000 7fd2a8bd6700 1 -- 172.21.15.140:0/197105344 wait complete. 2022-07-21T01:55:15.665 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 33 2022-07-21T01:55:15.673 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:55:15.673 INFO:tasks.ceph.mds.a:Stopping old one... 2022-07-21T01:55:15.674 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:55:15.674 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:55:15.674 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:55:15.674 INFO:tasks.ceph.mds.b:Stopping old one... 2022-07-21T01:55:15.675 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:55:15.675 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:55:15.683 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:55:15.684 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:55:15.690 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:55:15.690 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:55:15.731 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:55:15.732 DEBUG:teuthology.parallel:result is None 2022-07-21T01:55:15.735 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:55:15.736 DEBUG:teuthology.parallel:result is None 2022-07-21T01:55:15.736 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:55:15.801 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:55:15.812 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:55:15.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.837+0000 7f4c3f09f700 1 Processor -- start 2022-07-21T01:55:15.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.837+0000 7f4c3f09f700 1 -- start start 2022-07-21T01:55:15.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3f09f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4c380b6200 0x7f4c380ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3f09f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c380677d0 con 0x7f4c380b4af0 2022-07-21T01:55:15.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3f09f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c380b65e0 con 0x7f4c38140060 2022-07-21T01:55:15.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3f09f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c380aca50 con 0x7f4c380b6200 2022-07-21T01:55:15.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3d89c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4c380b4af0 0x7f4c3813f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51384/0 (socket says 172.21.15.140:51384) 2022-07-21T01:55:15.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3d89c700 1 -- 172.21.15.140:0/3635684228 learned_addr learned my addr 172.21.15.140:0/3635684228 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:15.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3e09d700 1 --2- 172.21.15.140:0/3635684228 >> v2:172.21.15.152:3300/0 conn(0x7f4c380b6200 0x7f4c380ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3d09b700 1 -- 172.21.15.140:0/3635684228 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090209282 0 0) 0x7f4c380677d0 con 0x7f4c380b4af0 2022-07-21T01:55:15.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3d09b700 1 -- 172.21.15.140:0/3635684228 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c20003260 con 0x7f4c380b4af0 2022-07-21T01:55:15.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3e09d700 1 -- 172.21.15.140:0/3635684228 >> v1:172.21.15.152:6789/0 conn(0x7f4c38140060 legacy=0x7f4c380b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3e09d700 1 -- 172.21.15.140:0/3635684228 >> v1:172.21.15.140:6789/0 conn(0x7f4c380b4af0 legacy=0x7f4c3813f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3e09d700 1 -- 172.21.15.140:0/3635684228 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c38067530 con 0x7f4c380b6200 2022-07-21T01:55:15.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.841+0000 7f4c3e09d700 1 --2- 172.21.15.140:0/3635684228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c380b6200 0x7f4c380ac560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f4c34002f70 tx=0x7f4c34006df0).ready entity=mon.1 client_cookie=f78c980f8077b30a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:15.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.845+0000 7f4c3d09b700 1 -- 172.21.15.140:0/3635684228 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c3400d610 con 0x7f4c380b6200 2022-07-21T01:55:15.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.845+0000 7f4c3d09b700 1 -- 172.21.15.140:0/3635684228 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c34000e90 con 0x7f4c380b6200 2022-07-21T01:55:15.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.845+0000 7f4c3d09b700 1 -- 172.21.15.140:0/3635684228 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c3400d960 con 0x7f4c380b6200 2022-07-21T01:55:15.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.845+0000 7f4c3f09f700 1 -- 172.21.15.140:0/3635684228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c380b6200 msgr2=0x7f4c380ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.845+0000 7f4c3f09f700 1 --2- 172.21.15.140:0/3635684228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c380b6200 0x7f4c380ac560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f4c34002f70 tx=0x7f4c34006df0).stop 2022-07-21T01:55:15.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.845+0000 7f4c3f09f700 1 -- 172.21.15.140:0/3635684228 shutdown_connections 2022-07-21T01:55:15.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.845+0000 7f4c3f09f700 1 --2- 172.21.15.140:0/3635684228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c380b6200 0x7f4c380ac560 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.845+0000 7f4c3f09f700 1 -- 172.21.15.140:0/3635684228 >> 172.21.15.140:0/3635684228 conn(0x7f4c38063d10 msgr2=0x7f4c380b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:15.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.845+0000 7f4c3f09f700 1 -- 172.21.15.140:0/3635684228 shutdown_connections 2022-07-21T01:55:15.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.845+0000 7f4c3f09f700 1 -- 172.21.15.140:0/3635684228 wait complete. 2022-07-21T01:55:15.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3f09f700 1 Processor -- start 2022-07-21T01:55:15.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3f09f700 1 -- start start 2022-07-21T01:55:15.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3f09f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c380b4af0 0x7f4c38150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3f09f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c380b6200 0x7f4c380667e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3f09f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c38140060 0x7f4c3813b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3f09f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c381466e0 con 0x7f4c380b6200 2022-07-21T01:55:15.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3f09f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c381411d0 con 0x7f4c380b4af0 2022-07-21T01:55:15.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3f09f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c38141470 con 0x7f4c38140060 2022-07-21T01:55:15.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3d89c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c380b6200 0x7f4c380667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3d89c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c380b6200 0x7f4c380667e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37244/0 (socket says 172.21.15.140:37244) 2022-07-21T01:55:15.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3e09d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c380b4af0 0x7f4c38150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3d89c700 1 -- 172.21.15.140:0/2642531637 learned_addr learned my addr 172.21.15.140:0/2642531637 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:15.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3e89e700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c38140060 0x7f4c3813b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3d89c700 1 -- 172.21.15.140:0/2642531637 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c38140060 msgr2=0x7f4c3813b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3d89c700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c38140060 0x7f4c3813b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3d89c700 1 -- 172.21.15.140:0/2642531637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c380b4af0 msgr2=0x7f4c38150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:15.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3d89c700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c380b4af0 0x7f4c38150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:15.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3d89c700 1 -- 172.21.15.140:0/2642531637 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3400b180 con 0x7f4c380b6200 2022-07-21T01:55:15.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3d89c700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c380b6200 0x7f4c380667e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f4c28001110 tx=0x7f4c28007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:15.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c277fe700 1 -- 172.21.15.140:0/2642531637 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c280070d0 con 0x7f4c380b6200 2022-07-21T01:55:15.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c277fe700 1 -- 172.21.15.140:0/2642531637 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c28007230 con 0x7f4c380b6200 2022-07-21T01:55:15.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3f09f700 1 -- 172.21.15.140:0/2642531637 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c38141710 con 0x7f4c380b6200 2022-07-21T01:55:15.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c277fe700 1 -- 172.21.15.140:0/2642531637 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c28005610 con 0x7f4c380b6200 2022-07-21T01:55:15.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.849+0000 7f4c3f09f700 1 -- 172.21.15.140:0/2642531637 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c38140da0 con 0x7f4c380b6200 2022-07-21T01:55:15.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.853+0000 7f4c3f09f700 1 -- 172.21.15.140:0/2642531637 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c381418a0 con 0x7f4c380b6200 2022-07-21T01:55:15.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.853+0000 7f4c277fe700 1 -- 172.21.15.140:0/2642531637 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4c28010070 con 0x7f4c380b6200 2022-07-21T01:55:15.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.853+0000 7f4c277fe700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c18053aa0 0x7f4c18055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:15.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.853+0000 7f4c277fe700 1 -- 172.21.15.140:0/2642531637 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f4c2806b1b0 con 0x7f4c380b6200 2022-07-21T01:55:15.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.853+0000 7f4c3e09d700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c18053aa0 0x7f4c18055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:15.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.853+0000 7f4c3e09d700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c18053aa0 0x7f4c18055f60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f4c3401a000 tx=0x7f4c34014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:15.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.857+0000 7f4c277fe700 1 -- 172.21.15.140:0/2642531637 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4c28042300 con 0x7f4c380b6200 2022-07-21T01:55:15.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:15.997+0000 7f4c3f09f700 1 -- 172.21.15.140:0/2642531637 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f4c380b5220 con 0x7f4c380b6200 2022-07-21T01:55:16.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.861+0000 7f4c277fe700 1 -- 172.21.15.140:0/2642531637 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4c28045920 con 0x7f4c380b6200 2022-07-21T01:55:16.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 -- 172.21.15.140:0/2642531637 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c18053aa0 msgr2=0x7f4c18055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:16.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c18053aa0 0x7f4c18055f60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f4c3401a000 tx=0x7f4c34014000).stop 2022-07-21T01:55:16.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 -- 172.21.15.140:0/2642531637 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c380b6200 msgr2=0x7f4c380667e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:16.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c380b6200 0x7f4c380667e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f4c28001110 tx=0x7f4c28007ad0).stop 2022-07-21T01:55:16.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 -- 172.21.15.140:0/2642531637 shutdown_connections 2022-07-21T01:55:16.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c18053aa0 0x7f4c18055f60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:16.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c38140060 0x7f4c3813b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:16.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c380b6200 0x7f4c380667e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:16.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 --2- 172.21.15.140:0/2642531637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c380b4af0 0x7f4c38150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:16.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 -- 172.21.15.140:0/2642531637 >> 172.21.15.140:0/2642531637 conn(0x7f4c38063d10 msgr2=0x7f4c38069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:16.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 -- 172.21.15.140:0/2642531637 shutdown_connections 2022-07-21T01:55:16.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.865+0000 7f4c3f09f700 1 -- 172.21.15.140:0/2642531637 wait complete. 2022-07-21T01:55:16.872 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:55:16.884 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:55:16.884 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:55:16.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.989+0000 7fec0ed72700 1 Processor -- start 2022-07-21T01:55:16.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.989+0000 7fec0ed72700 1 -- start start 2022-07-21T01:55:16.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0ed72700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec080b18d0 0x7fec08143710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:16.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0ed72700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fec0813ec50 0x7fec0813f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:16.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0ed72700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec080afac0 con 0x7fec080b61d0 2022-07-21T01:55:16.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0ed72700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec080add80 con 0x7fec0813ec50 2022-07-21T01:55:16.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0ed72700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec080b65b0 con 0x7fec080b18d0 2022-07-21T01:55:17.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0e571700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fec0813ec50 0x7fec0813f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0d56f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec080b18d0 0x7fec08143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0e571700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fec0813ec50 0x7fec0813f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:42996/0 (socket says 172.21.15.140:42996) 2022-07-21T01:55:17.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0dd70700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fec080b61d0 0x7fec08143000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:52512/0 (socket says 172.21.15.140:52512) 2022-07-21T01:55:17.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0e571700 1 -- 172.21.15.140:0/1082237077 learned_addr learned my addr 172.21.15.140:0/1082237077 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:17.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0e571700 1 -- 172.21.15.140:0/1082237077 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec080b18d0 msgr2=0x7fec08143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:17.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0e571700 1 --2- 172.21.15.140:0/1082237077 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec080b18d0 0x7fec08143710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0e571700 1 -- 172.21.15.140:0/1082237077 >> v1:172.21.15.152:6789/0 conn(0x7fec080b61d0 legacy=0x7fec08143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0e571700 1 -- 172.21.15.140:0/1082237077 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0805c3e0 con 0x7fec0813ec50 2022-07-21T01:55:17.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0d56f700 1 --2- 172.21.15.140:0/1082237077 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec080b18d0 0x7fec08143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:17.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0e571700 1 --2- 172.21.15.140:0/1082237077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec0813ec50 0x7fec0813f080 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fec000032f0 tx=0x7fec00009290).ready entity=mon.1 client_cookie=734fa57916b47e80 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0cd6e700 1 -- 172.21.15.140:0/1082237077 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0000b8a0 con 0x7fec0813ec50 2022-07-21T01:55:17.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0cd6e700 1 -- 172.21.15.140:0/1082237077 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0000ba00 con 0x7fec0813ec50 2022-07-21T01:55:17.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0cd6e700 1 -- 172.21.15.140:0/1082237077 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0000bb70 con 0x7fec0813ec50 2022-07-21T01:55:17.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0ed72700 1 -- 172.21.15.140:0/1082237077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec0813ec50 msgr2=0x7fec0813f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0ed72700 1 --2- 172.21.15.140:0/1082237077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec0813ec50 0x7fec0813f080 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fec000032f0 tx=0x7fec00009290).stop 2022-07-21T01:55:17.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.993+0000 7fec0ed72700 1 -- 172.21.15.140:0/1082237077 shutdown_connections 2022-07-21T01:55:17.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 --2- 172.21.15.140:0/1082237077 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec080b18d0 0x7fec08143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 --2- 172.21.15.140:0/1082237077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec0813ec50 0x7fec0813f080 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 -- 172.21.15.140:0/1082237077 >> 172.21.15.140:0/1082237077 conn(0x7fec08063cf0 msgr2=0x7fec080b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:17.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 -- 172.21.15.140:0/1082237077 shutdown_connections 2022-07-21T01:55:17.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 -- 172.21.15.140:0/1082237077 wait complete. 2022-07-21T01:55:17.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 Processor -- start 2022-07-21T01:55:17.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 -- start start 2022-07-21T01:55:17.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec080b18d0 0x7fec0813fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec080b61d0 0x7fec081390d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0813ec50 0x7fec081395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec08065f70 con 0x7fec0813ec50 2022-07-21T01:55:17.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec08139ab0 con 0x7fec080b61d0 2022-07-21T01:55:17.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0ed72700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec08065170 con 0x7fec080b18d0 2022-07-21T01:55:17.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0e571700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0813ec50 0x7fec081395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0dd70700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec080b18d0 0x7fec0813fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0e571700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0813ec50 0x7fec081395c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37262/0 (socket says 172.21.15.140:37262) 2022-07-21T01:55:17.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0d56f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec080b61d0 0x7fec081390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0e571700 1 -- 172.21.15.140:0/3751229807 learned_addr learned my addr 172.21.15.140:0/3751229807 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:17.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0e571700 1 -- 172.21.15.140:0/3751229807 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec080b18d0 msgr2=0x7fec0813fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0e571700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec080b18d0 0x7fec0813fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0e571700 1 -- 172.21.15.140:0/3751229807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec080b61d0 msgr2=0x7fec081390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0e571700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec080b61d0 0x7fec081390d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0e571700 1 -- 172.21.15.140:0/3751229807 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf800b180 con 0x7fec0813ec50 2022-07-21T01:55:17.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:16.997+0000 7fec0d56f700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec080b61d0 0x7fec081390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:17.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.001+0000 7fec0e571700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0813ec50 0x7fec081395c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fec000032f0 tx=0x7fec00016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.001+0000 7febf6ffd700 1 -- 172.21.15.140:0/3751229807 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0000b8a0 con 0x7fec0813ec50 2022-07-21T01:55:17.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.001+0000 7febf6ffd700 1 -- 172.21.15.140:0/3751229807 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec00021630 con 0x7fec0813ec50 2022-07-21T01:55:17.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.001+0000 7fec0ed72700 1 -- 172.21.15.140:0/3751229807 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec00009530 con 0x7fec0813ec50 2022-07-21T01:55:17.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.001+0000 7febf6ffd700 1 -- 172.21.15.140:0/3751229807 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec00021990 con 0x7fec0813ec50 2022-07-21T01:55:17.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.001+0000 7fec0ed72700 1 -- 172.21.15.140:0/3751229807 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec08066f70 con 0x7fec0813ec50 2022-07-21T01:55:17.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.001+0000 7fec0ed72700 1 -- 172.21.15.140:0/3751229807 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec080679b0 con 0x7fec0813ec50 2022-07-21T01:55:17.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.001+0000 7febf6ffd700 1 -- 172.21.15.140:0/3751229807 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fec00027020 con 0x7fec0813ec50 2022-07-21T01:55:17.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.001+0000 7febf6ffd700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 0x7febe8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.001+0000 7febf6ffd700 1 -- 172.21.15.140:0/3751229807 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fec0006cdb0 con 0x7fec0813ec50 2022-07-21T01:55:17.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.001+0000 7fec0dd70700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 0x7febe8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.005+0000 7fec0dd70700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 0x7febe8055fc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fec0400d000 tx=0x7fec04007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.009+0000 7febf6ffd700 1 -- 172.21.15.140:0/3751229807 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fec000440b0 con 0x7fec0813ec50 2022-07-21T01:55:17.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 -- 172.21.15.140:0/3751229807 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fec080574e0 con 0x7fec0813ec50 2022-07-21T01:55:17.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7febf6ffd700 1 -- 172.21.15.140:0/3751229807 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v29) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fec00015070 con 0x7fec0813ec50 2022-07-21T01:55:17.176 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:17.176 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:55:17.177 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:55:17.177 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.177 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:55:17.177 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:55:17.178 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.178 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:55:17.178 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:17.179 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:17.179 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.179 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.179 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.180 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.180 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:55:17.180 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:55:17.180 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.181 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:55:17.181 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:17.181 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:17.181 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.181 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.182 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.182 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.182 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:55:17.182 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:55:17.183 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.183 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:17.183 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:17.183 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.184 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.184 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.184 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.184 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:55:17.185 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:55:17.185 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.185 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:17.185 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:17.185 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.186 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.186 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.186 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.186 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:55:17.187 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:55:17.187 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.187 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:17.187 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:17.187 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.188 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.188 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.188 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.188 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:55:17.189 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:55:17.189 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.189 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:17.189 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:17.189 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.190 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.190 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.190 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.190 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:55:17.191 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:55:17.191 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.191 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:17.191 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:17.192 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.192 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.192 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.192 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.192 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:55:17.193 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:55:17.193 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.193 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:17.193 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:17.194 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.194 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.194 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.194 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.194 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:55:17.195 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:55:17.195 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.195 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:17.195 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:17.196 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.196 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.196 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.196 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.196 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:55:17.197 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:55:17.197 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.197 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:17.197 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:17.197 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.198 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.198 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.198 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.198 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:55:17.199 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:55:17.199 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.199 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:55:17.199 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:55:17.199 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:55:17.200 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.200 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.200 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.200 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:55:17.201 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:55:17.201 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.201 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:55:17.201 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:55:17.201 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:55:17.202 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.202 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.202 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.202 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:55:17.202 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:55:17.203 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.203 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:55:17.203 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:55:17.203 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:55:17.204 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.204 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.204 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.204 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:55:17.204 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:55:17.205 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.205 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:55:17.205 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:17.205 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:17.206 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.206 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.206 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.206 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.206 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:55:17.207 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:55:17.207 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.207 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:55:17.207 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:55:17.207 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.208 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.208 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:17.208 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:17.208 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:55:17.209 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:55:17.209 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:17.209 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:55:17.209 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:55:17.209 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:17.210 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.210 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:17.210 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:17.223 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:55:17.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 -- 172.21.15.140:0/3751229807 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 msgr2=0x7febe8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 0x7febe8055fc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7fec0400d000 tx=0x7fec04007b80).stop 2022-07-21T01:55:17.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 -- 172.21.15.140:0/3751229807 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0813ec50 msgr2=0x7fec081395c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0813ec50 0x7fec081395c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fec000032f0 tx=0x7fec00016000).stop 2022-07-21T01:55:17.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 -- 172.21.15.140:0/3751229807 shutdown_connections 2022-07-21T01:55:17.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7febe8053b00 0x7febe8055fc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec0813ec50 0x7fec081395c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec080b61d0 0x7fec081390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 --2- 172.21.15.140:0/3751229807 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec080b18d0 0x7fec0813fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 -- 172.21.15.140:0/3751229807 >> 172.21.15.140:0/3751229807 conn(0x7fec08063cf0 msgr2=0x7fec080b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:17.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 -- 172.21.15.140:0/3751229807 shutdown_connections 2022-07-21T01:55:17.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.173+0000 7fec0ed72700 1 -- 172.21.15.140:0/3751229807 wait complete. 2022-07-21T01:55:17.230 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:17.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.333+0000 7f2b157f8700 1 Processor -- start 2022-07-21T01:55:17.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.333+0000 7f2b157f8700 1 -- start start 2022-07-21T01:55:17.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b1005c3f0 0x7f2b1014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1005d480 con 0x7f2b100b1e00 2022-07-21T01:55:17.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b100b21e0 con 0x7f2b100acef0 2022-07-21T01:55:17.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1005cf40 con 0x7f2b1005c3f0 2022-07-21T01:55:17.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0effd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2b100b1e00 0x7f2b100ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51414/0 (socket says 172.21.15.140:51414) 2022-07-21T01:55:17.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0effd700 1 -- 172.21.15.140:0/1236394669 learned_addr learned my addr 172.21.15.140:0/1236394669 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:17.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0e7fc700 1 --2- 172.21.15.140:0/1236394669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b1005c3f0 0x7f2b1014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0dffb700 1 -- 172.21.15.140:0/1236394669 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023142310 0 0) 0x7f2b1005d480 con 0x7f2b100b1e00 2022-07-21T01:55:17.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0dffb700 1 -- 172.21.15.140:0/1236394669 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af0003260 con 0x7f2b100b1e00 2022-07-21T01:55:17.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0dffb700 1 -- 172.21.15.140:0/1236394669 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879559977 0 0) 0x7f2b100b21e0 con 0x7f2b100acef0 2022-07-21T01:55:17.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0dffb700 1 -- 172.21.15.140:0/1236394669 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b1005d480 con 0x7f2b100acef0 2022-07-21T01:55:17.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0e7fc700 1 -- 172.21.15.140:0/1236394669 >> v1:172.21.15.152:6789/0 conn(0x7f2b100acef0 legacy=0x7f2b10146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0e7fc700 1 -- 172.21.15.140:0/1236394669 >> v1:172.21.15.140:6789/0 conn(0x7f2b100b1e00 legacy=0x7f2b100ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0e7fc700 1 -- 172.21.15.140:0/1236394669 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1005d1e0 con 0x7f2b1005c3f0 2022-07-21T01:55:17.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0e7fc700 1 --2- 172.21.15.140:0/1236394669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b1005c3f0 0x7f2b1014a6d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f2b000032f0 tx=0x7f2b000091d0).ready entity=mon.2 client_cookie=6619a244a173a646 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0dffb700 1 -- 172.21.15.140:0/1236394669 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b0000b640 con 0x7f2b1005c3f0 2022-07-21T01:55:17.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0dffb700 1 -- 172.21.15.140:0/1236394669 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0000b7a0 con 0x7f2b1005c3f0 2022-07-21T01:55:17.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b0dffb700 1 -- 172.21.15.140:0/1236394669 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b0000b910 con 0x7f2b1005c3f0 2022-07-21T01:55:17.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 -- 172.21.15.140:0/1236394669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b1005c3f0 msgr2=0x7f2b1014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 --2- 172.21.15.140:0/1236394669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b1005c3f0 0x7f2b1014a6d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f2b000032f0 tx=0x7f2b000091d0).stop 2022-07-21T01:55:17.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 -- 172.21.15.140:0/1236394669 shutdown_connections 2022-07-21T01:55:17.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 --2- 172.21.15.140:0/1236394669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b1005c3f0 0x7f2b1014a6d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 -- 172.21.15.140:0/1236394669 >> 172.21.15.140:0/1236394669 conn(0x7f2b100611c0 msgr2=0x7f2b10062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:17.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 -- 172.21.15.140:0/1236394669 shutdown_connections 2022-07-21T01:55:17.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 -- 172.21.15.140:0/1236394669 wait complete. 2022-07-21T01:55:17.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.337+0000 7f2b157f8700 1 Processor -- start 2022-07-21T01:55:17.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b157f8700 1 -- start start 2022-07-21T01:55:17.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b157f8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b100acef0 0x7f2b10063dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b157f8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b100b1e00 0x7f2b100b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b157f8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b1009e5a0 0x7f2b100b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b157f8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b101565a0 con 0x7f2b100acef0 2022-07-21T01:55:17.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b157f8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b10157580 con 0x7f2b100b1e00 2022-07-21T01:55:17.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b157f8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b100a1d10 con 0x7f2b1009e5a0 2022-07-21T01:55:17.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b0effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b100acef0 0x7f2b10063dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b0effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b100acef0 0x7f2b10063dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37272/0 (socket says 172.21.15.140:37272) 2022-07-21T01:55:17.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b0effd700 1 -- 172.21.15.140:0/3534107386 learned_addr learned my addr 172.21.15.140:0/3534107386 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:17.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b0e7fc700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b100b1e00 0x7f2b100b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b0f7fe700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b1009e5a0 0x7f2b100b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b0effd700 1 -- 172.21.15.140:0/3534107386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b1009e5a0 msgr2=0x7f2b100b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b0effd700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b1009e5a0 0x7f2b100b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b0effd700 1 -- 172.21.15.140:0/3534107386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b100b1e00 msgr2=0x7f2b100b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b0effd700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b100b1e00 0x7f2b100b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b0effd700 1 -- 172.21.15.140:0/3534107386 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b00009470 con 0x7f2b100acef0 2022-07-21T01:55:17.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b0effd700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b100acef0 0x7f2b10063dc0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f2af80010d0 tx=0x7f2af8007b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b07fff700 1 -- 172.21.15.140:0/3534107386 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2af800ce50 con 0x7f2b100acef0 2022-07-21T01:55:17.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b157f8700 1 -- 172.21.15.140:0/3534107386 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b100b5840 con 0x7f2b100acef0 2022-07-21T01:55:17.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b07fff700 1 -- 172.21.15.140:0/3534107386 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af80070d0 con 0x7f2b100acef0 2022-07-21T01:55:17.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b07fff700 1 -- 172.21.15.140:0/3534107386 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2af8007430 con 0x7f2b100acef0 2022-07-21T01:55:17.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.341+0000 7f2b157f8700 1 -- 172.21.15.140:0/3534107386 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b100642b0 con 0x7f2b100acef0 2022-07-21T01:55:17.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.345+0000 7f2b07fff700 1 -- 172.21.15.140:0/3534107386 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2af8010070 con 0x7f2b100acef0 2022-07-21T01:55:17.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.345+0000 7f2b157f8700 1 -- 172.21.15.140:0/3534107386 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b100a1ed0 con 0x7f2b100acef0 2022-07-21T01:55:17.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.345+0000 7f2b07fff700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2af40539e0 0x7f2af4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.345+0000 7f2b07fff700 1 -- 172.21.15.140:0/3534107386 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2af806b220 con 0x7f2b100acef0 2022-07-21T01:55:17.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.345+0000 7f2b0e7fc700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2af40539e0 0x7f2af4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.345+0000 7f2b0e7fc700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2af40539e0 0x7f2af4055ea0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f2b00019000 tx=0x7f2b00016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.349+0000 7f2b07fff700 1 -- 172.21.15.140:0/3534107386 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2af8042360 con 0x7f2b100acef0 2022-07-21T01:55:17.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.473+0000 7f2b157f8700 1 -- 172.21.15.140:0/3534107386 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b100b04b0 con 0x7f2b100acef0 2022-07-21T01:55:17.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.473+0000 7f2b07fff700 1 -- 172.21.15.140:0/3534107386 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+13278 (secure 0 0 0) 0x7f2af8045980 con 0x7f2b100acef0 2022-07-21T01:55:17.477 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:17.477 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":80,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:16.851973+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:17.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 -- 172.21.15.140:0/3534107386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2af40539e0 msgr2=0x7f2af4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2af40539e0 0x7f2af4055ea0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f2b00019000 tx=0x7f2b00016000).stop 2022-07-21T01:55:17.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 -- 172.21.15.140:0/3534107386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b100acef0 msgr2=0x7f2b10063dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b100acef0 0x7f2b10063dc0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f2af80010d0 tx=0x7f2af8007b00).stop 2022-07-21T01:55:17.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 -- 172.21.15.140:0/3534107386 shutdown_connections 2022-07-21T01:55:17.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2af40539e0 0x7f2af4055ea0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b1009e5a0 0x7f2b100b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b100b1e00 0x7f2b100b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 --2- 172.21.15.140:0/3534107386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b100acef0 0x7f2b10063dc0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 -- 172.21.15.140:0/3534107386 >> 172.21.15.140:0/3534107386 conn(0x7f2b1005c3f0 msgr2=0x7f2b100ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:17.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 -- 172.21.15.140:0/3534107386 shutdown_connections 2022-07-21T01:55:17.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.477+0000 7f2b157f8700 1 -- 172.21.15.140:0/3534107386 wait complete. 2022-07-21T01:55:17.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:55:17.490 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:55:17.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d86022700 1 Processor -- start 2022-07-21T01:55:17.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d86022700 1 -- start start 2022-07-21T01:55:17.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d86022700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4d800acbe0 0x7f4d800ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d86022700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80141ee0 0x7f4d80142370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d86022700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d800ab850 con 0x7f4d8005f840 2022-07-21T01:55:17.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d86022700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d801460e0 con 0x7f4d800acbe0 2022-07-21T01:55:17.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d86022700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80146380 con 0x7f4d80141ee0 2022-07-21T01:55:17.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d8481f700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4d8005f840 0x7f4d80146ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51426/0 (socket says 172.21.15.140:51426) 2022-07-21T01:55:17.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d8481f700 1 -- 172.21.15.140:0/981921266 learned_addr learned my addr 172.21.15.140:0/981921266 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:17.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d85821700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80141ee0 0x7f4d80142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277491820 0 0) 0x7f4d800ab850 con 0x7f4d8005f840 2022-07-21T01:55:17.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d85020700 1 --2- 172.21.15.140:0/981921266 >> v2:172.21.15.152:3300/0 conn(0x7f4d800acbe0 0x7f4d800ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d60003260 con 0x7f4d8005f840 2022-07-21T01:55:17.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4203193823 0 0) 0x7f4d60003260 con 0x7f4d8005f840 2022-07-21T01:55:17.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80141ee0 msgr2=0x7f4d80142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:17.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80141ee0 0x7f4d80142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 >> v2:172.21.15.152:3300/0 conn(0x7f4d800acbe0 msgr2=0x7f4d800ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:17.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 --2- 172.21.15.140:0/981921266 >> v2:172.21.15.152:3300/0 conn(0x7f4d800acbe0 0x7f4d800ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d80146620 con 0x7f4d8005f840 2022-07-21T01:55:17.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f4d70001660 con 0x7f4d8005f840 2022-07-21T01:55:17.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d85020700 1 --2- 172.21.15.140:0/981921266 >> v2:172.21.15.152:3300/0 conn(0x7f4d800acbe0 0x7f4d800ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:17.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 >> v1:172.21.15.140:6789/0 conn(0x7f4d8005f840 legacy=0x7f4d80146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80141ee0 0x7f4d60003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d60003c80 0x7f4d60004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d600049f0 0x7f4d60004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d801460e0 con 0x7f4d60003c80 2022-07-21T01:55:17.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80146380 con 0x7f4d600049f0 2022-07-21T01:55:17.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6000b780 con 0x7f4d80141ee0 2022-07-21T01:55:17.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d8481f700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d60003c80 0x7f4d60004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d85020700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80141ee0 0x7f4d60003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d85821700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d600049f0 0x7f4d60004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d85020700 1 -- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d600049f0 msgr2=0x7f4d60004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:17.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d85020700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d600049f0 0x7f4d60004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d85020700 1 -- 172.21.15.140:0/981921266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d60003c80 msgr2=0x7f4d60004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:17.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d85020700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d60003c80 0x7f4d60004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d85020700 1 -- 172.21.15.140:0/981921266 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d60003260 con 0x7f4d80141ee0 2022-07-21T01:55:17.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.593+0000 7f4d8481f700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d60003c80 0x7f4d60004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:17.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d85020700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80141ee0 0x7f4d60003750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f4d6800a000 tx=0x7f4d68008040).ready entity=mon.2 client_cookie=9934e10ed8762408 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d6800c070 con 0x7f4d80141ee0 2022-07-21T01:55:17.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6800e5c0 con 0x7f4d80141ee0 2022-07-21T01:55:17.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d7ffff700 1 -- 172.21.15.140:0/981921266 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d6800e920 con 0x7f4d80141ee0 2022-07-21T01:55:17.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d86022700 1 -- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80141ee0 msgr2=0x7f4d60003750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d86022700 1 --2- 172.21.15.140:0/981921266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80141ee0 0x7f4d60003750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f4d6800a000 tx=0x7f4d68008040).stop 2022-07-21T01:55:17.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d85821700 1 -- 172.21.15.140:0/981921266 reap_dead start 2022-07-21T01:55:17.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d86022700 1 -- 172.21.15.140:0/981921266 shutdown_connections 2022-07-21T01:55:17.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d86022700 1 -- 172.21.15.140:0/981921266 >> 172.21.15.140:0/981921266 conn(0x7f4d800b2af0 msgr2=0x7f4d800b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:17.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d86022700 1 -- 172.21.15.140:0/981921266 shutdown_connections 2022-07-21T01:55:17.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d86022700 1 -- 172.21.15.140:0/981921266 wait complete. 2022-07-21T01:55:17.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d86022700 1 Processor -- start 2022-07-21T01:55:17.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.597+0000 7f4d86022700 1 -- start start 2022-07-21T01:55:17.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d86022700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d80141ee0 0x7f4d800af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d86022700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d800ad9f0 0x7f4d80152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d86022700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80152600 0x7f4d8005ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d86022700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80152b00 con 0x7f4d800ad9f0 2022-07-21T01:55:17.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d86022700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80152da0 con 0x7f4d80141ee0 2022-07-21T01:55:17.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d86022700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800af1c0 con 0x7f4d80152600 2022-07-21T01:55:17.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d8481f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d800ad9f0 0x7f4d80152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d8481f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d800ad9f0 0x7f4d80152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37294/0 (socket says 172.21.15.140:37294) 2022-07-21T01:55:17.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d8481f700 1 -- 172.21.15.140:0/3524807038 learned_addr learned my addr 172.21.15.140:0/3524807038 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:17.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d85821700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80152600 0x7f4d8005ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d85020700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d80141ee0 0x7f4d800af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d85821700 1 -- 172.21.15.140:0/3524807038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d80141ee0 msgr2=0x7f4d800af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d85821700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d80141ee0 0x7f4d800af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d85821700 1 -- 172.21.15.140:0/3524807038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d800ad9f0 msgr2=0x7f4d80152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d85821700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d800ad9f0 0x7f4d80152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d85821700 1 -- 172.21.15.140:0/3524807038 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800afb50 con 0x7f4d80152600 2022-07-21T01:55:17.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d8481f700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d800ad9f0 0x7f4d80152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:17.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d85821700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80152600 0x7f4d8005ff00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4d7800b000 tx=0x7f4d78006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d7e7fc700 1 -- 172.21.15.140:0/3524807038 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d78019070 con 0x7f4d80152600 2022-07-21T01:55:17.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d7e7fc700 1 -- 172.21.15.140:0/3524807038 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d78000e90 con 0x7f4d80152600 2022-07-21T01:55:17.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d86022700 1 -- 172.21.15.140:0/3524807038 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d800ad5f0 con 0x7f4d80152600 2022-07-21T01:55:17.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d7e7fc700 1 -- 172.21.15.140:0/3524807038 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d78013870 con 0x7f4d80152600 2022-07-21T01:55:17.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d86022700 1 -- 172.21.15.140:0/3524807038 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d80060610 con 0x7f4d80152600 2022-07-21T01:55:17.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.601+0000 7f4d86022700 1 -- 172.21.15.140:0/3524807038 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d800ad7e0 con 0x7f4d80152600 2022-07-21T01:55:17.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.605+0000 7f4d7e7fc700 1 -- 172.21.15.140:0/3524807038 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4d780139d0 con 0x7f4d80152600 2022-07-21T01:55:17.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.605+0000 7f4d7e7fc700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d64053b30 0x7f4d64055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.605+0000 7f4d7e7fc700 1 -- 172.21.15.140:0/3524807038 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f4d7806b6d0 con 0x7f4d80152600 2022-07-21T01:55:17.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.605+0000 7f4d85020700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d64053b30 0x7f4d64055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.605+0000 7f4d85020700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d64053b30 0x7f4d64055ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f4d6801c000 tx=0x7f4d68008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.613+0000 7f4d7e7fc700 1 -- 172.21.15.140:0/3524807038 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4d780428a0 con 0x7f4d80152600 2022-07-21T01:55:17.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.737+0000 7f4d86022700 1 -- 172.21.15.140:0/3524807038 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4d800574e0 con 0x7f4d80152600 2022-07-21T01:55:17.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d7e7fc700 1 -- 172.21.15.140:0/3524807038 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v80) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4d78045ec0 con 0x7f4d80152600 2022-07-21T01:55:17.742 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:55:17.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 -- 172.21.15.140:0/3524807038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d64053b30 msgr2=0x7f4d64055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d64053b30 0x7f4d64055ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f4d6801c000 tx=0x7f4d68008040).stop 2022-07-21T01:55:17.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 -- 172.21.15.140:0/3524807038 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80152600 msgr2=0x7f4d8005ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80152600 0x7f4d8005ff00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f4d7800b000 tx=0x7f4d78006df0).stop 2022-07-21T01:55:17.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 -- 172.21.15.140:0/3524807038 shutdown_connections 2022-07-21T01:55:17.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d64053b30 0x7f4d64055ff0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d80152600 0x7f4d8005ff00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d800ad9f0 0x7f4d80152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 --2- 172.21.15.140:0/3524807038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d80141ee0 0x7f4d800af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 -- 172.21.15.140:0/3524807038 >> 172.21.15.140:0/3524807038 conn(0x7f4d800b2af0 msgr2=0x7f4d8013f420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:17.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 -- 172.21.15.140:0/3524807038 shutdown_connections 2022-07-21T01:55:17.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.741+0000 7f4d86022700 1 -- 172.21.15.140:0/3524807038 wait complete. 2022-07-21T01:55:17.756 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:55:17.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f8c5e700 1 Processor -- start 2022-07-21T01:55:17.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f8c5e700 1 -- start start 2022-07-21T01:55:17.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f8c5e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa7f40b59d0 0x7fa7f40b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f8c5e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f40afbc0 con 0x7fa7f414ada0 2022-07-21T01:55:17.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f8c5e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f40b00e0 con 0x7fa7f40b14f0 2022-07-21T01:55:17.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f8c5e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f4148080 con 0x7fa7f40b59d0 2022-07-21T01:55:17.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f37fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa7f40b59d0 0x7fa7f40b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f2ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa7f414ada0 0x7fa7f405bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51448/0 (socket says 172.21.15.140:51448) 2022-07-21T01:55:17.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f2ffd700 1 -- 172.21.15.140:0/1253148230 learned_addr learned my addr 172.21.15.140:0/1253148230 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:17.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f37fe700 1 -- 172.21.15.140:0/1253148230 >> v1:172.21.15.152:6789/0 conn(0x7fa7f40b14f0 legacy=0x7fa7f405c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f37fe700 1 -- 172.21.15.140:0/1253148230 >> v1:172.21.15.140:6789/0 conn(0x7fa7f414ada0 legacy=0x7fa7f405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f37fe700 1 -- 172.21.15.140:0/1253148230 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40b5db0 con 0x7fa7f40b59d0 2022-07-21T01:55:17.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f37fe700 1 --2- 172.21.15.140:0/1253148230 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7f40b59d0 0x7fa7f40b07e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fa7e8002f70 tx=0x7fa7e8006df0).ready entity=mon.0 client_cookie=72038a6244d8252b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f27fc700 1 -- 172.21.15.140:0/1253148230 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7e800d610 con 0x7fa7f40b59d0 2022-07-21T01:55:17.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f27fc700 1 -- 172.21.15.140:0/1253148230 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e8000e90 con 0x7fa7f40b59d0 2022-07-21T01:55:17.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.861+0000 7fa7f27fc700 1 -- 172.21.15.140:0/1253148230 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7e800d9a0 con 0x7fa7f40b59d0 2022-07-21T01:55:17.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.865+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/1253148230 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7f40b59d0 msgr2=0x7fa7f40b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.865+0000 7fa7f8c5e700 1 --2- 172.21.15.140:0/1253148230 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7f40b59d0 0x7fa7f40b07e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fa7e8002f70 tx=0x7fa7e8006df0).stop 2022-07-21T01:55:17.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.865+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/1253148230 shutdown_connections 2022-07-21T01:55:17.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.865+0000 7fa7f8c5e700 1 --2- 172.21.15.140:0/1253148230 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7f40b59d0 0x7fa7f40b07e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.865+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/1253148230 >> 172.21.15.140:0/1253148230 conn(0x7fa7f40634c0 msgr2=0x7fa7f40b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:17.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.865+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/1253148230 shutdown_connections 2022-07-21T01:55:17.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.865+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/1253148230 wait complete. 2022-07-21T01:55:17.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.865+0000 7fa7f8c5e700 1 Processor -- start 2022-07-21T01:55:17.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.865+0000 7fa7f8c5e700 1 -- start start 2022-07-21T01:55:17.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f8c5e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7f40b14f0 0x7fa7f40b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f8c5e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7f40b59d0 0x7fa7f4159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f8c5e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7f414ada0 0x7fa7f4159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f8c5e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f40ae0f0 con 0x7fa7f414ada0 2022-07-21T01:55:17.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f8c5e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f410fe00 con 0x7fa7f40b14f0 2022-07-21T01:55:17.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f8c5e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f41100a0 con 0x7fa7f40b59d0 2022-07-21T01:55:17.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7f414ada0 0x7fa7f4159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7f414ada0 0x7fa7f4159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37308/0 (socket says 172.21.15.140:37308) 2022-07-21T01:55:17.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f3fff700 1 -- 172.21.15.140:0/3240147152 learned_addr learned my addr 172.21.15.140:0/3240147152 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:17.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f37fe700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7f40b14f0 0x7fa7f40b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f2ffd700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7f40b59d0 0x7fa7f4159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f3fff700 1 -- 172.21.15.140:0/3240147152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7f40b59d0 msgr2=0x7fa7f4159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f3fff700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7f40b59d0 0x7fa7f4159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f3fff700 1 -- 172.21.15.140:0/3240147152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7f40b14f0 msgr2=0x7fa7f40b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:17.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f3fff700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7f40b14f0 0x7fa7f40b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:17.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f3fff700 1 -- 172.21.15.140:0/3240147152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e800b180 con 0x7fa7f414ada0 2022-07-21T01:55:17.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f3fff700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7f414ada0 0x7fa7f4159c80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fa7f40639a0 tx=0x7fa7ec007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f0ff9700 1 -- 172.21.15.140:0/3240147152 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7ec00f470 con 0x7fa7f414ada0 2022-07-21T01:55:17.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f0ff9700 1 -- 172.21.15.140:0/3240147152 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec00f5d0 con 0x7fa7f414ada0 2022-07-21T01:55:17.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/3240147152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f4112110 con 0x7fa7f414ada0 2022-07-21T01:55:17.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f0ff9700 1 -- 172.21.15.140:0/3240147152 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7ec00f930 con 0x7fa7f414ada0 2022-07-21T01:55:17.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/3240147152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f4110540 con 0x7fa7f414ada0 2022-07-21T01:55:17.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/3240147152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f4110260 con 0x7fa7f414ada0 2022-07-21T01:55:17.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f0ff9700 1 -- 172.21.15.140:0/3240147152 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa7ec019030 con 0x7fa7f414ada0 2022-07-21T01:55:17.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.869+0000 7fa7f0ff9700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7d4053aa0 0x7fa7d4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:17.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.873+0000 7fa7f0ff9700 1 -- 172.21.15.140:0/3240147152 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa7ec06b220 con 0x7fa7f414ada0 2022-07-21T01:55:17.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.873+0000 7fa7f37fe700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7d4053aa0 0x7fa7d4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:17.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.873+0000 7fa7f37fe700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7d4053aa0 0x7fa7d4055f60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fa7e801a000 tx=0x7fa7e8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:17.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:17.877+0000 7fa7f0ff9700 1 -- 172.21.15.140:0/3240147152 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa7ec042470 con 0x7fa7f414ada0 2022-07-21T01:55:18.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.001+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/3240147152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa7f40b0340 con 0x7fa7f414ada0 2022-07-21T01:55:18.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.005+0000 7fa7f0ff9700 1 -- 172.21.15.140:0/3240147152 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa7ec017020 con 0x7fa7f414ada0 2022-07-21T01:55:18.007 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:18.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.005+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/3240147152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7d4053aa0 msgr2=0x7fa7d4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.005+0000 7fa7f8c5e700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7d4053aa0 0x7fa7d4055f60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fa7e801a000 tx=0x7fa7e8014000).stop 2022-07-21T01:55:18.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.005+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/3240147152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7f414ada0 msgr2=0x7fa7f4159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.005+0000 7fa7f8c5e700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7f414ada0 0x7fa7f4159c80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fa7f40639a0 tx=0x7fa7ec007f00).stop 2022-07-21T01:55:18.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.009+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/3240147152 shutdown_connections 2022-07-21T01:55:18.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.009+0000 7fa7f8c5e700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7d4053aa0 0x7fa7d4055f60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.009+0000 7fa7f8c5e700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7f414ada0 0x7fa7f4159c80 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.009+0000 7fa7f8c5e700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7f40b59d0 0x7fa7f4159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.009+0000 7fa7f8c5e700 1 --2- 172.21.15.140:0/3240147152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7f40b14f0 0x7fa7f40b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.009+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/3240147152 >> 172.21.15.140:0/3240147152 conn(0x7fa7f40634c0 msgr2=0x7fa7f40b4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:18.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.009+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/3240147152 shutdown_connections 2022-07-21T01:55:18.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.009+0000 7fa7f8c5e700 1 -- 172.21.15.140:0/3240147152 wait complete. 2022-07-21T01:55:18.028 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:55:18.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00e087b700 1 Processor -- start 2022-07-21T01:55:18.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00e087b700 1 -- start start 2022-07-21T01:55:18.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00e087b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f00dc0ac3a0 0x7f00dc143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00e087b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 0x7f00dc13e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00e087b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00dc060980 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00e087b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00dc0b22d0 con 0x7f00dc0ac3a0 2022-07-21T01:55:18.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00e087b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc098810 con 0x7f00dc05fa50 2022-07-21T01:55:18.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00dad9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f00dc0b1ef0 0x7f00dc142f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51460/0 (socket says 172.21.15.140:51460) 2022-07-21T01:55:18.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00dad9d700 1 -- 172.21.15.140:0/1015624523 learned_addr learned my addr 172.21.15.140:0/1015624523 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:18.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00da59c700 1 --2- 172.21.15.140:0/1015624523 >> v2:172.21.15.140:3300/0 conn(0x7f00dc0ac3a0 0x7f00dc143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00db59e700 1 --2- 172.21.15.140:0/1015624523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 0x7f00dc13e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00d9d9b700 1 -- 172.21.15.140:0/1015624523 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856734994 0 0) 0x7f00dc060980 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00d9d9b700 1 -- 172.21.15.140:0/1015624523 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00bc003260 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00db59e700 1 -- 172.21.15.140:0/1015624523 >> v2:172.21.15.140:3300/0 conn(0x7f00dc0ac3a0 msgr2=0x7f00dc143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:18.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00db59e700 1 --2- 172.21.15.140:0/1015624523 >> v2:172.21.15.140:3300/0 conn(0x7f00dc0ac3a0 0x7f00dc143690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00db59e700 1 -- 172.21.15.140:0/1015624523 >> v1:172.21.15.140:6789/0 conn(0x7f00dc0b1ef0 legacy=0x7f00dc142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00db59e700 1 -- 172.21.15.140:0/1015624523 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc060bb0 con 0x7f00dc05fa50 2022-07-21T01:55:18.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.129+0000 7f00da59c700 1 --2- 172.21.15.140:0/1015624523 >> v2:172.21.15.140:3300/0 conn(0x7f00dc0ac3a0 0x7f00dc143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:18.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00db59e700 1 --2- 172.21.15.140:0/1015624523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 0x7f00dc13e9a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f00cc002f70 tx=0x7f00cc006df0).ready entity=mon.2 client_cookie=1a21b956cf97749b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:18.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00d9d9b700 1 -- 172.21.15.140:0/1015624523 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00cc00d4a0 con 0x7f00dc05fa50 2022-07-21T01:55:18.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00d9d9b700 1 -- 172.21.15.140:0/1015624523 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc000e90 con 0x7f00dc05fa50 2022-07-21T01:55:18.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00d9d9b700 1 -- 172.21.15.140:0/1015624523 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00cc00d7f0 con 0x7f00dc05fa50 2022-07-21T01:55:18.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00e087b700 1 -- 172.21.15.140:0/1015624523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 msgr2=0x7f00dc13e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00e087b700 1 --2- 172.21.15.140:0/1015624523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 0x7f00dc13e9a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f00cc002f70 tx=0x7f00cc006df0).stop 2022-07-21T01:55:18.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00e087b700 1 -- 172.21.15.140:0/1015624523 shutdown_connections 2022-07-21T01:55:18.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00e087b700 1 --2- 172.21.15.140:0/1015624523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 0x7f00dc13e9a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00e087b700 1 --2- 172.21.15.140:0/1015624523 >> v2:172.21.15.140:3300/0 conn(0x7f00dc0ac3a0 0x7f00dc143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00e087b700 1 -- 172.21.15.140:0/1015624523 >> 172.21.15.140:0/1015624523 conn(0x7f00dc0b2ca0 msgr2=0x7f00dc0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:18.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00e087b700 1 -- 172.21.15.140:0/1015624523 shutdown_connections 2022-07-21T01:55:18.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.133+0000 7f00e087b700 1 -- 172.21.15.140:0/1015624523 wait complete. 2022-07-21T01:55:18.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00e087b700 1 Processor -- start 2022-07-21T01:55:18.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00e087b700 1 -- start start 2022-07-21T01:55:18.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00e087b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 0x7f00dc139bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00e087b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00dc0ac3a0 0x7f00dc1528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00e087b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00dc0b1ef0 0x7f00dc14ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00e087b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc0adf50 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00e087b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc1521a0 con 0x7f00dc0ac3a0 2022-07-21T01:55:18.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00e087b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc152440 con 0x7f00dc05fa50 2022-07-21T01:55:18.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00db59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00dc0b1ef0 0x7f00dc14ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00da59c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00dc0ac3a0 0x7f00dc1528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00dad9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 0x7f00dc139bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00db59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00dc0b1ef0 0x7f00dc14ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37324/0 (socket says 172.21.15.140:37324) 2022-07-21T01:55:18.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00db59e700 1 -- 172.21.15.140:0/2523264542 learned_addr learned my addr 172.21.15.140:0/2523264542 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:18.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00db59e700 1 -- 172.21.15.140:0/2523264542 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 msgr2=0x7f00dc139bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00db59e700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 0x7f00dc139bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00db59e700 1 -- 172.21.15.140:0/2523264542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00dc0ac3a0 msgr2=0x7f00dc1528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00db59e700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00dc0ac3a0 0x7f00dc1528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00db59e700 1 -- 172.21.15.140:0/2523264542 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c400b180 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00dad9d700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 0x7f00dc139bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:18.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.137+0000 7f00db59e700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00dc0b1ef0 0x7f00dc14ead0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f00cc002f70 tx=0x7f00cc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:18.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.141+0000 7f00d3fff700 1 -- 172.21.15.140:0/2523264542 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00cc016070 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.141+0000 7f00d3fff700 1 -- 172.21.15.140:0/2523264542 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc012740 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.141+0000 7f00e087b700 1 -- 172.21.15.140:0/2523264542 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00cc00b180 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.141+0000 7f00d3fff700 1 -- 172.21.15.140:0/2523264542 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00cc012aa0 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.141+0000 7f00e087b700 1 -- 172.21.15.140:0/2523264542 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00dc152e50 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.141+0000 7f00d3fff700 1 -- 172.21.15.140:0/2523264542 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f00cc00f030 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.141+0000 7f00e087b700 1 -- 172.21.15.140:0/2523264542 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00dc152600 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.141+0000 7f00d3fff700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f00c0053aa0 0x7f00c0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.141+0000 7f00d3fff700 1 -- 172.21.15.140:0/2523264542 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f00cc06c260 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.141+0000 7f00dad9d700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f00c0053aa0 0x7f00c0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.141+0000 7f00dad9d700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f00c0053aa0 0x7f00c0055f60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f00d400d000 tx=0x7f00d4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:18.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.149+0000 7f00d3fff700 1 -- 172.21.15.140:0/2523264542 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f00cc043410 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 -- 172.21.15.140:0/2523264542 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f00dc062270 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00d3fff700 1 -- 172.21.15.140:0/2523264542 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+143 (secure 0 0 0) 0x7f00cc022020 con 0x7f00dc0b1ef0 2022-07-21T01:55:18.280 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:18.280 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:55:18.280 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:18.280 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:55:18.281 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:55:18.281 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:18.281 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:18.282 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:55:18.282 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:55:18.282 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:18.283 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:55:18.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 -- 172.21.15.140:0/2523264542 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f00c0053aa0 msgr2=0x7f00c0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f00c0053aa0 0x7f00c0055f60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f00d400d000 tx=0x7f00d4009040).stop 2022-07-21T01:55:18.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 -- 172.21.15.140:0/2523264542 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00dc0b1ef0 msgr2=0x7f00dc14ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00dc0b1ef0 0x7f00dc14ead0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f00cc002f70 tx=0x7f00cc014000).stop 2022-07-21T01:55:18.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 -- 172.21.15.140:0/2523264542 shutdown_connections 2022-07-21T01:55:18.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f00c0053aa0 0x7f00c0055f60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00dc0b1ef0 0x7f00dc14ead0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00dc0ac3a0 0x7f00dc1528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 --2- 172.21.15.140:0/2523264542 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00dc05fa50 0x7f00dc139bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 -- 172.21.15.140:0/2523264542 >> 172.21.15.140:0/2523264542 conn(0x7f00dc0b2ca0 msgr2=0x7f00dc1401d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:18.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 -- 172.21.15.140:0/2523264542 shutdown_connections 2022-07-21T01:55:18.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.277+0000 7f00e087b700 1 -- 172.21.15.140:0/2523264542 wait complete. 2022-07-21T01:55:18.292 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:55:18.293 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:55:18.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.393+0000 7fd6db136700 1 Processor -- start 2022-07-21T01:55:18.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.393+0000 7fd6db136700 1 -- start start 2022-07-21T01:55:18.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.393+0000 7fd6db136700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd6d40acfa0 0x7fd6d4142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.393+0000 7fd6db136700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd6d40b1030 0x7fd6d4142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.393+0000 7fd6db136700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d40ad380 con 0x7fd6d40b02e0 2022-07-21T01:55:18.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.393+0000 7fd6db136700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d413f030 con 0x7fd6d40b1030 2022-07-21T01:55:18.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.393+0000 7fd6db136700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d40af470 con 0x7fd6d40acfa0 2022-07-21T01:55:18.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6da935700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd6d40b02e0 0x7fd6d40ae4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51476/0 (socket says 172.21.15.140:51476) 2022-07-21T01:55:18.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6da935700 1 -- 172.21.15.140:0/362238951 learned_addr learned my addr 172.21.15.140:0/362238951 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:18.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9933700 1 --2- 172.21.15.140:0/362238951 >> v2:172.21.15.140:3300/0 conn(0x7fd6d40b1030 0x7fd6d4142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6da134700 1 --2- 172.21.15.140:0/362238951 >> v2:172.21.15.152:3300/0 conn(0x7fd6d40acfa0 0x7fd6d4142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9132700 1 -- 172.21.15.140:0/362238951 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494475841 0 0) 0x7fd6d40ad380 con 0x7fd6d40b02e0 2022-07-21T01:55:18.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9132700 1 -- 172.21.15.140:0/362238951 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6c0003260 con 0x7fd6d40b02e0 2022-07-21T01:55:18.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9933700 1 -- 172.21.15.140:0/362238951 >> v2:172.21.15.152:3300/0 conn(0x7fd6d40acfa0 msgr2=0x7fd6d4142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:18.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9933700 1 --2- 172.21.15.140:0/362238951 >> v2:172.21.15.152:3300/0 conn(0x7fd6d40acfa0 0x7fd6d4142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9132700 1 -- 172.21.15.140:0/362238951 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2324174023 0 0) 0x7fd6c0003260 con 0x7fd6d40b02e0 2022-07-21T01:55:18.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9933700 1 -- 172.21.15.140:0/362238951 >> v1:172.21.15.140:6789/0 conn(0x7fd6d40b02e0 legacy=0x7fd6d40ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9933700 1 -- 172.21.15.140:0/362238951 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d40b1630 con 0x7fd6d40b1030 2022-07-21T01:55:18.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9933700 1 --2- 172.21.15.140:0/362238951 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6d40b1030 0x7fd6d4142fb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fd6cc001b80 tx=0x7fd6c8006df0).ready entity=mon.0 client_cookie=d34b40ac95dc6096 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:18.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9132700 1 -- 172.21.15.140:0/362238951 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6cc0019e0 con 0x7fd6d40b1030 2022-07-21T01:55:18.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9132700 1 -- 172.21.15.140:0/362238951 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c8000e90 con 0x7fd6d40b1030 2022-07-21T01:55:18.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6d9132700 1 -- 172.21.15.140:0/362238951 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c8012460 con 0x7fd6d40b1030 2022-07-21T01:55:18.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6db136700 1 -- 172.21.15.140:0/362238951 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6d40b1030 msgr2=0x7fd6d4142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6db136700 1 --2- 172.21.15.140:0/362238951 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6d40b1030 0x7fd6d4142fb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fd6cc001b80 tx=0x7fd6c8006df0).stop 2022-07-21T01:55:18.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6db136700 1 -- 172.21.15.140:0/362238951 shutdown_connections 2022-07-21T01:55:18.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6db136700 1 --2- 172.21.15.140:0/362238951 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6d40b1030 0x7fd6d4142fb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6db136700 1 --2- 172.21.15.140:0/362238951 >> v2:172.21.15.152:3300/0 conn(0x7fd6d40acfa0 0x7fd6d4142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6db136700 1 -- 172.21.15.140:0/362238951 >> 172.21.15.140:0/362238951 conn(0x7fd6d40632d0 msgr2=0x7fd6d40636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:18.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6db136700 1 -- 172.21.15.140:0/362238951 shutdown_connections 2022-07-21T01:55:18.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.397+0000 7fd6db136700 1 -- 172.21.15.140:0/362238951 wait complete. 2022-07-21T01:55:18.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6db136700 1 Processor -- start 2022-07-21T01:55:18.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6db136700 1 -- start start 2022-07-21T01:55:18.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6db136700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6d40acfa0 0x7fd6d4064600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6db136700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d40b02e0 0x7fd6d4133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6db136700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6d40b1030 0x7fd6d4135c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6db136700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d4136c10 con 0x7fd6d40acfa0 2022-07-21T01:55:18.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6db136700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d4136950 con 0x7fd6d40b02e0 2022-07-21T01:55:18.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6db136700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d4136eb0 con 0x7fd6d40b1030 2022-07-21T01:55:18.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6da134700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6d40acfa0 0x7fd6d4064600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6da134700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6d40acfa0 0x7fd6d4064600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37334/0 (socket says 172.21.15.140:37334) 2022-07-21T01:55:18.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6da134700 1 -- 172.21.15.140:0/3133343282 learned_addr learned my addr 172.21.15.140:0/3133343282 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:18.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6d9933700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d40b02e0 0x7fd6d4133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6da935700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6d40b1030 0x7fd6d4135c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6da134700 1 -- 172.21.15.140:0/3133343282 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6d40b1030 msgr2=0x7fd6d4135c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6da134700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6d40b1030 0x7fd6d4135c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6da134700 1 -- 172.21.15.140:0/3133343282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d40b02e0 msgr2=0x7fd6d4133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:18.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6da134700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d40b02e0 0x7fd6d4133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:18.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6da134700 1 -- 172.21.15.140:0/3133343282 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c800b180 con 0x7fd6d40acfa0 2022-07-21T01:55:18.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6da134700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6d40acfa0 0x7fd6d4064600 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7fd6c400c000 tx=0x7fd6c4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:18.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6d37fe700 1 -- 172.21.15.140:0/3133343282 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c4011750 con 0x7fd6d40acfa0 2022-07-21T01:55:18.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6d37fe700 1 -- 172.21.15.140:0/3133343282 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c40118b0 con 0x7fd6d40acfa0 2022-07-21T01:55:18.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6db136700 1 -- 172.21.15.140:0/3133343282 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d4134130 con 0x7fd6d40acfa0 2022-07-21T01:55:18.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6d37fe700 1 -- 172.21.15.140:0/3133343282 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c4011c10 con 0x7fd6d40acfa0 2022-07-21T01:55:18.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.401+0000 7fd6db136700 1 -- 172.21.15.140:0/3133343282 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d4066f90 con 0x7fd6d40acfa0 2022-07-21T01:55:18.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.405+0000 7fd6db136700 1 -- 172.21.15.140:0/3133343282 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d412de00 con 0x7fd6d40acfa0 2022-07-21T01:55:18.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.405+0000 7fd6d37fe700 1 -- 172.21.15.140:0/3133343282 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd6c4011d70 con 0x7fd6d40acfa0 2022-07-21T01:55:18.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.405+0000 7fd6d37fe700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6b4053aa0 0x7fd6b4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:18.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.405+0000 7fd6d37fe700 1 -- 172.21.15.140:0/3133343282 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd6c406bfa0 con 0x7fd6d40acfa0 2022-07-21T01:55:18.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.405+0000 7fd6d9933700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6b4053aa0 0x7fd6b4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:18.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.405+0000 7fd6d9933700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6b4053aa0 0x7fd6b4055f60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fd6c801a000 tx=0x7fd6c8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:18.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.413+0000 7fd6d37fe700 1 -- 172.21.15.140:0/3133343282 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd6c40431f0 con 0x7fd6d40acfa0 2022-07-21T01:55:18.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:18.537+0000 7fd6db136700 1 -- 172.21.15.140:0/3133343282 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fd6d41339e0 con 0x7fd6d40acfa0 2022-07-21T01:55:19.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.181+0000 7fd6d37fe700 1 -- 172.21.15.140:0/3133343282 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v81) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd6c4046810 con 0x7fd6d40acfa0 2022-07-21T01:55:19.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 -- 172.21.15.140:0/3133343282 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6b4053aa0 msgr2=0x7fd6b4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:19.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6b4053aa0 0x7fd6b4055f60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fd6c801a000 tx=0x7fd6c8014000).stop 2022-07-21T01:55:19.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 -- 172.21.15.140:0/3133343282 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6d40acfa0 msgr2=0x7fd6d4064600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:19.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6d40acfa0 0x7fd6d4064600 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7fd6c400c000 tx=0x7fd6c4006df0).stop 2022-07-21T01:55:19.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 -- 172.21.15.140:0/3133343282 shutdown_connections 2022-07-21T01:55:19.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6b4053aa0 0x7fd6b4055f60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:19.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6d40b1030 0x7fd6d4135c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:19.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6d40b02e0 0x7fd6d4133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:19.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 --2- 172.21.15.140:0/3133343282 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6d40acfa0 0x7fd6d4064600 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:19.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 -- 172.21.15.140:0/3133343282 >> 172.21.15.140:0/3133343282 conn(0x7fd6d40632d0 msgr2=0x7fd6d4137b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:19.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 -- 172.21.15.140:0/3133343282 shutdown_connections 2022-07-21T01:55:19.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.189+0000 7fd6db136700 1 -- 172.21.15.140:0/3133343282 wait complete. 2022-07-21T01:55:19.195 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:55:19.212 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:55:19.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.309+0000 7ff49e120700 1 Processor -- start 2022-07-21T01:55:19.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.309+0000 7ff49e120700 1 -- start start 2022-07-21T01:55:19.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49e120700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff4980b32a0 0x7ff4981468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:19.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49e120700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4980b3f10 con 0x7ff4980b2ec0 2022-07-21T01:55:19.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49e120700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4980b2620 con 0x7ff49805d160 2022-07-21T01:55:19.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49e120700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4980b4ea0 con 0x7ff4980b32a0 2022-07-21T01:55:19.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49d91f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff4980b32a0 0x7ff4981468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:19.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49d11e700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff4980b2ec0 0x7ff498142210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51486/0 (socket says 172.21.15.140:51486) 2022-07-21T01:55:19.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49d11e700 1 -- 172.21.15.140:0/1501910811 learned_addr learned my addr 172.21.15.140:0/1501910811 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:19.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49d91f700 1 -- 172.21.15.140:0/1501910811 >> v1:172.21.15.152:6789/0 conn(0x7ff49805d160 legacy=0x7ff4981461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:19.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49d91f700 1 -- 172.21.15.140:0/1501910811 >> v1:172.21.15.140:6789/0 conn(0x7ff4980b2ec0 legacy=0x7ff498142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:19.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff48ffff700 1 -- 172.21.15.140:0/1501910811 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774408540 0 0) 0x7ff4980b2620 con 0x7ff49805d160 2022-07-21T01:55:19.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49d91f700 1 -- 172.21.15.140:0/1501910811 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980b28c0 con 0x7ff4980b32a0 2022-07-21T01:55:19.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49d91f700 1 --2- 172.21.15.140:0/1501910811 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4980b32a0 0x7ff4981468d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7ff494001110 tx=0x7ff4900091d0).ready entity=mon.0 client_cookie=324e576fa1640ba1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:19.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff48ffff700 1 -- 172.21.15.140:0/1501910811 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff490009ea0 con 0x7ff4980b32a0 2022-07-21T01:55:19.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff48ffff700 1 -- 172.21.15.140:0/1501910811 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49000b760 con 0x7ff4980b32a0 2022-07-21T01:55:19.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff48ffff700 1 -- 172.21.15.140:0/1501910811 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff49000b8d0 con 0x7ff4980b32a0 2022-07-21T01:55:19.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49e120700 1 -- 172.21.15.140:0/1501910811 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4980b32a0 msgr2=0x7ff4981468d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:19.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49e120700 1 --2- 172.21.15.140:0/1501910811 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4980b32a0 0x7ff4981468d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7ff494001110 tx=0x7ff4900091d0).stop 2022-07-21T01:55:19.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49e120700 1 -- 172.21.15.140:0/1501910811 shutdown_connections 2022-07-21T01:55:19.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49e120700 1 --2- 172.21.15.140:0/1501910811 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4980b32a0 0x7ff4981468d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:19.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49e120700 1 -- 172.21.15.140:0/1501910811 >> 172.21.15.140:0/1501910811 conn(0x7ff498066e20 msgr2=0x7ff498067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:19.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49e120700 1 -- 172.21.15.140:0/1501910811 shutdown_connections 2022-07-21T01:55:19.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.313+0000 7ff49e120700 1 -- 172.21.15.140:0/1501910811 wait complete. 2022-07-21T01:55:19.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49e120700 1 Processor -- start 2022-07-21T01:55:19.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49e120700 1 -- start start 2022-07-21T01:55:19.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49e120700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49805d160 0x7ff4981440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:19.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49e120700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4980b2ec0 0x7ff498152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:19.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49e120700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4980b32a0 0x7ff4980add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:19.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49e120700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff498146a80 con 0x7ff4980b32a0 2022-07-21T01:55:19.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49e120700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980ad460 con 0x7ff49805d160 2022-07-21T01:55:19.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49e120700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4981445b0 con 0x7ff4980b2ec0 2022-07-21T01:55:19.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49d91f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4980b32a0 0x7ff4980add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:19.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49d91f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4980b32a0 0x7ff4980add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37350/0 (socket says 172.21.15.140:37350) 2022-07-21T01:55:19.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49d11e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49805d160 0x7ff4981440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:19.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49d91f700 1 -- 172.21.15.140:0/3597860054 learned_addr learned my addr 172.21.15.140:0/3597860054 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:19.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49c91d700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4980b2ec0 0x7ff498152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:19.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49d91f700 1 -- 172.21.15.140:0/3597860054 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4980b2ec0 msgr2=0x7ff498152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:19.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49d91f700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4980b2ec0 0x7ff498152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:19.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49d91f700 1 -- 172.21.15.140:0/3597860054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49805d160 msgr2=0x7ff4981440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:19.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49d91f700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49805d160 0x7ff4981440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:19.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49d91f700 1 -- 172.21.15.140:0/3597860054 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff490009470 con 0x7ff4980b32a0 2022-07-21T01:55:19.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49d91f700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4980b32a0 0x7ff4980add60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7ff494001110 tx=0x7ff490016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:19.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff48e7fc700 1 -- 172.21.15.140:0/3597860054 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff49000b8d0 con 0x7ff4980b32a0 2022-07-21T01:55:19.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff48e7fc700 1 -- 172.21.15.140:0/3597860054 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4900032d0 con 0x7ff4980b32a0 2022-07-21T01:55:19.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49e120700 1 -- 172.21.15.140:0/3597860054 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4980acae0 con 0x7ff4980b32a0 2022-07-21T01:55:19.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff48e7fc700 1 -- 172.21.15.140:0/3597860054 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff490009ea0 con 0x7ff4980b32a0 2022-07-21T01:55:19.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.317+0000 7ff49e120700 1 -- 172.21.15.140:0/3597860054 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4980afaf0 con 0x7ff4980b32a0 2022-07-21T01:55:19.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.321+0000 7ff49e120700 1 -- 172.21.15.140:0/3597860054 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4980ad620 con 0x7ff4980b32a0 2022-07-21T01:55:19.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.321+0000 7ff48e7fc700 1 -- 172.21.15.140:0/3597860054 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff490026030 con 0x7ff4980b32a0 2022-07-21T01:55:19.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.321+0000 7ff48e7fc700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff480053950 0x7ff480055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:19.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.321+0000 7ff48e7fc700 1 -- 172.21.15.140:0/3597860054 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5607+0+0 (secure 0 0 0) 0x7ff49006c860 con 0x7ff4980b32a0 2022-07-21T01:55:19.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.321+0000 7ff49d11e700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff480053950 0x7ff480055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:19.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.321+0000 7ff49d11e700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff480053950 0x7ff480055e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7ff49400d000 tx=0x7ff494009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:19.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.329+0000 7ff48e7fc700 1 -- 172.21.15.140:0/3597860054 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff490047320 con 0x7ff4980b32a0 2022-07-21T01:55:19.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:19.461+0000 7ff49e120700 1 -- 172.21.15.140:0/3597860054 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7ff4980b4a00 con 0x7ff4980b32a0 2022-07-21T01:55:20.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.185+0000 7ff48e7fc700 1 -- 172.21.15.140:0/3597860054 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v82) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff4900240a0 con 0x7ff4980b32a0 2022-07-21T01:55:20.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 -- 172.21.15.140:0/3597860054 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff480053950 msgr2=0x7ff480055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff480053950 0x7ff480055e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7ff49400d000 tx=0x7ff494009040).stop 2022-07-21T01:55:20.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 -- 172.21.15.140:0/3597860054 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4980b32a0 msgr2=0x7ff4980add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4980b32a0 0x7ff4980add60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7ff494001110 tx=0x7ff490016000).stop 2022-07-21T01:55:20.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 -- 172.21.15.140:0/3597860054 shutdown_connections 2022-07-21T01:55:20.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff480053950 0x7ff480055e10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4980b32a0 0x7ff4980add60 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4980b2ec0 0x7ff498152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 --2- 172.21.15.140:0/3597860054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff49805d160 0x7ff4981440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 -- 172.21.15.140:0/3597860054 >> 172.21.15.140:0/3597860054 conn(0x7ff498066e20 msgr2=0x7ff4980b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:20.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 -- 172.21.15.140:0/3597860054 shutdown_connections 2022-07-21T01:55:20.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.193+0000 7ff49e120700 1 -- 172.21.15.140:0/3597860054 wait complete. 2022-07-21T01:55:20.198 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:55:20.210 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:55:20.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f9e66f700 1 Processor -- start 2022-07-21T01:55:20.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f9e66f700 1 -- start start 2022-07-21T01:55:20.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f9e66f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8f980b6690 0x7f8f980b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f9e66f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f980ae670 0x7f8f980aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f9e66f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f980acae0 con 0x7f8f980b59a0 2022-07-21T01:55:20.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f9e66f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f98147100 con 0x7f8f980b6690 2022-07-21T01:55:20.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f9e66f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980acd10 con 0x7f8f980ae670 2022-07-21T01:55:20.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f977fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f8f980b59a0 0x7f8f9813c970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51502/0 (socket says 172.21.15.140:51502) 2022-07-21T01:55:20.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f977fe700 1 -- 172.21.15.140:0/2301359215 learned_addr learned my addr 172.21.15.140:0/2301359215 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:20.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f97fff700 1 --2- 172.21.15.140:0/2301359215 >> v2:172.21.15.140:3300/0 conn(0x7f8f980b6690 0x7f8f980b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f9cc0c700 1 --2- 172.21.15.140:0/2301359215 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f980ae670 0x7f8f980aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581797348 0 0) 0x7f8f980acae0 con 0x7f8f980b59a0 2022-07-21T01:55:20.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.317+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f84003260 con 0x7f8f980b59a0 2022-07-21T01:55:20.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 250840995 0 0) 0x7f8f84003260 con 0x7f8f980b59a0 2022-07-21T01:55:20.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f980ae670 msgr2=0x7f8f980aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:20.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 --2- 172.21.15.140:0/2301359215 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f980ae670 0x7f8f980aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 >> v2:172.21.15.140:3300/0 conn(0x7f8f980b6690 msgr2=0x7f8f980b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:20.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 --2- 172.21.15.140:0/2301359215 >> v2:172.21.15.140:3300/0 conn(0x7f8f980b6690 0x7f8f980b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980b5df0 con 0x7f8f980b59a0 2022-07-21T01:55:20.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f8f88001660 con 0x7f8f980b59a0 2022-07-21T01:55:20.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 >> v1:172.21.15.140:6789/0 conn(0x7f8f980b59a0 legacy=0x7f8f9813c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 --2- 172.21.15.140:0/2301359215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f84003750 0x7f8f84003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 --2- 172.21.15.140:0/2301359215 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f84004500 0x7f8f84004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 --2- 172.21.15.140:0/2301359215 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f980ae670 0x7f8f84004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98147100 con 0x7f8f84004500 2022-07-21T01:55:20.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980acd10 con 0x7f8f84003750 2022-07-21T01:55:20.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8400b750 con 0x7f8f980ae670 2022-07-21T01:55:20.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f977fe700 1 --2- 172.21.15.140:0/2301359215 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f84004500 0x7f8f84004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f97fff700 1 --2- 172.21.15.140:0/2301359215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f84003750 0x7f8f84003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f977fe700 1 -- 172.21.15.140:0/2301359215 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f980ae670 msgr2=0x7f8f84004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:20.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f977fe700 1 --2- 172.21.15.140:0/2301359215 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f980ae670 0x7f8f84004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-07-21T01:55:20.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f977fe700 1 -- 172.21.15.140:0/2301359215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f84003750 msgr2=0x7f8f84003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:20.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f977fe700 1 --2- 172.21.15.140:0/2301359215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f84003750 0x7f8f84003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f977fe700 1 -- 172.21.15.140:0/2301359215 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f84003260 con 0x7f8f84004500 2022-07-21T01:55:20.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f977fe700 1 --2- 172.21.15.140:0/2301359215 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f84004500 0x7f8f84004980 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f8f88001370 tx=0x7f8f88001e50).ready entity=mon.0 client_cookie=f73bc62a8db2d785 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:20.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f880070d0 con 0x7f8f84004500 2022-07-21T01:55:20.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88007230 con 0x7f8f84004500 2022-07-21T01:55:20.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f96ffd700 1 -- 172.21.15.140:0/2301359215 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f8800c9c0 con 0x7f8f84004500 2022-07-21T01:55:20.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 -- 172.21.15.140:0/2301359215 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f84004500 msgr2=0x7f8f84004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 --2- 172.21.15.140:0/2301359215 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f84004500 0x7f8f84004980 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f8f88001370 tx=0x7f8f88001e50).stop 2022-07-21T01:55:20.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9cc0c700 1 -- 172.21.15.140:0/2301359215 reap_dead start 2022-07-21T01:55:20.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 -- 172.21.15.140:0/2301359215 shutdown_connections 2022-07-21T01:55:20.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 -- 172.21.15.140:0/2301359215 >> 172.21.15.140:0/2301359215 conn(0x7f8f980b9f70 msgr2=0x7f8f980b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:20.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 -- 172.21.15.140:0/2301359215 shutdown_connections 2022-07-21T01:55:20.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 -- 172.21.15.140:0/2301359215 wait complete. 2022-07-21T01:55:20.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 Processor -- start 2022-07-21T01:55:20.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 -- start start 2022-07-21T01:55:20.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f98140f90 0x7f8f98113dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f98112e90 0x7f8f98156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f98156650 0x7f8f980b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98155a90 con 0x7f8f98112e90 2022-07-21T01:55:20.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98141370 con 0x7f8f98140f90 2022-07-21T01:55:20.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f9e66f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98116f30 con 0x7f8f98156650 2022-07-21T01:55:20.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f977fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f98112e90 0x7f8f98156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f977fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f98112e90 0x7f8f98156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37368/0 (socket says 172.21.15.140:37368) 2022-07-21T01:55:20.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.321+0000 7f8f977fe700 1 -- 172.21.15.140:0/1771513388 learned_addr learned my addr 172.21.15.140:0/1771513388 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:20.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f97fff700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f98140f90 0x7f8f98113dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f9cc0c700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f98156650 0x7f8f980b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f977fe700 1 -- 172.21.15.140:0/1771513388 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f98156650 msgr2=0x7f8f980b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f977fe700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f98156650 0x7f8f980b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f977fe700 1 -- 172.21.15.140:0/1771513388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f98140f90 msgr2=0x7f8f98113dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f977fe700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f98140f90 0x7f8f98113dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f977fe700 1 -- 172.21.15.140:0/1771513388 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f981171d0 con 0x7f8f98112e90 2022-07-21T01:55:20.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f97fff700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f98140f90 0x7f8f98113dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:20.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f977fe700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f98112e90 0x7f8f98156160 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f8f88001370 tx=0x7f8f88015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:20.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f957fa700 1 -- 172.21.15.140:0/1771513388 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f880047d0 con 0x7f8f98112e90 2022-07-21T01:55:20.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f957fa700 1 -- 172.21.15.140:0/1771513388 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88004930 con 0x7f8f98112e90 2022-07-21T01:55:20.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f9e66f700 1 -- 172.21.15.140:0/1771513388 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f98155cc0 con 0x7f8f98112e90 2022-07-21T01:55:20.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f957fa700 1 -- 172.21.15.140:0/1771513388 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f88004c90 con 0x7f8f98112e90 2022-07-21T01:55:20.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f9e66f700 1 -- 172.21.15.140:0/1771513388 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f98112a90 con 0x7f8f98112e90 2022-07-21T01:55:20.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f9e66f700 1 -- 172.21.15.140:0/1771513388 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f98155e50 con 0x7f8f98112e90 2022-07-21T01:55:20.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f957fa700 1 -- 172.21.15.140:0/1771513388 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8f88010070 con 0x7f8f98112e90 2022-07-21T01:55:20.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f957fa700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f78053b00 0x7f8f78055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f957fa700 1 -- 172.21.15.140:0/1771513388 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f8f8806caf0 con 0x7f8f98112e90 2022-07-21T01:55:20.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.325+0000 7f8f97fff700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f78053b00 0x7f8f78055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.329+0000 7f8f97fff700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f78053b00 0x7f8f78055fc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f8f8c00b000 tx=0x7f8f8c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:20.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.333+0000 7f8f957fa700 1 -- 172.21.15.140:0/1771513388 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8f880439e0 con 0x7f8f98112e90 2022-07-21T01:55:20.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.501+0000 7f8f9e66f700 1 -- 172.21.15.140:0/1771513388 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f8f980acfb0 con 0x7f8f98112e90 2022-07-21T01:55:20.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.509+0000 7f8f957fa700 1 -- 172.21.15.140:0/1771513388 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v35) v1 ==== 189+0+0 (secure 0 0 0) 0x7f8f88024030 con 0x7f8f98112e90 2022-07-21T01:55:20.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 -- 172.21.15.140:0/1771513388 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f78053b00 msgr2=0x7f8f78055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f78053b00 0x7f8f78055fc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f8f8c00b000 tx=0x7f8f8c008040).stop 2022-07-21T01:55:20.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 -- 172.21.15.140:0/1771513388 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f98112e90 msgr2=0x7f8f98156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f98112e90 0x7f8f98156160 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f8f88001370 tx=0x7f8f88015000).stop 2022-07-21T01:55:20.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 -- 172.21.15.140:0/1771513388 shutdown_connections 2022-07-21T01:55:20.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8f78053b00 0x7f8f78055fc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8f98156650 0x7f8f980b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8f98112e90 0x7f8f98156160 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 --2- 172.21.15.140:0/1771513388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f98140f90 0x7f8f98113dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 -- 172.21.15.140:0/1771513388 >> 172.21.15.140:0/1771513388 conn(0x7f8f980b9f70 msgr2=0x7f8f9805cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:20.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 -- 172.21.15.140:0/1771513388 shutdown_connections 2022-07-21T01:55:20.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.513+0000 7f8f9e66f700 1 -- 172.21.15.140:0/1771513388 wait complete. 2022-07-21T01:55:20.517 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 15 and data pool 16 2022-07-21T01:55:20.531 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:20.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f142179e700 1 Processor -- start 2022-07-21T01:55:20.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f142179e700 1 -- start start 2022-07-21T01:55:20.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f142179e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f141c0afa10 0x7f141c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f142179e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c0b42a0 con 0x7f141c0b3de0 2022-07-21T01:55:20.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f142179e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c060c90 con 0x7f141c0608b0 2022-07-21T01:55:20.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f142179e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0af410 con 0x7f141c0afa10 2022-07-21T01:55:20.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f141b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f141c0b3de0 0x7f141c1452d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51534/0 (socket says 172.21.15.140:51534) 2022-07-21T01:55:20.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f141b7fe700 1 -- 172.21.15.140:0/1291777489 learned_addr learned my addr 172.21.15.140:0/1291777489 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:20.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f141a7fc700 1 --2- 172.21.15.140:0/1291777489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f141c0afa10 0x7f141c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f1419ffb700 1 -- 172.21.15.140:0/1291777489 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768081726 0 0) 0x7f141c0b42a0 con 0x7f141c0b3de0 2022-07-21T01:55:20.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f1419ffb700 1 -- 172.21.15.140:0/1291777489 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1400003260 con 0x7f141c0b3de0 2022-07-21T01:55:20.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f1419ffb700 1 -- 172.21.15.140:0/1291777489 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552823292 0 0) 0x7f141c060c90 con 0x7f141c0608b0 2022-07-21T01:55:20.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f1419ffb700 1 -- 172.21.15.140:0/1291777489 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f141c0b42a0 con 0x7f141c0608b0 2022-07-21T01:55:20.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f141a7fc700 1 -- 172.21.15.140:0/1291777489 >> v1:172.21.15.152:6789/0 conn(0x7f141c0608b0 legacy=0x7f141c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f141a7fc700 1 -- 172.21.15.140:0/1291777489 >> v1:172.21.15.140:6789/0 conn(0x7f141c0b3de0 legacy=0x7f141c1452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f141a7fc700 1 -- 172.21.15.140:0/1291777489 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0af6b0 con 0x7f141c0afa10 2022-07-21T01:55:20.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f141a7fc700 1 --2- 172.21.15.140:0/1291777489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f141c0afa10 0x7f141c0b46d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f1410002f70 tx=0x7f1410006df0).ready entity=mon.2 client_cookie=73a583de2054770a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:20.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f1419ffb700 1 -- 172.21.15.140:0/1291777489 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f141000d5c0 con 0x7f141c0afa10 2022-07-21T01:55:20.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f1419ffb700 1 -- 172.21.15.140:0/1291777489 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1410000e90 con 0x7f141c0afa10 2022-07-21T01:55:20.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.617+0000 7f1419ffb700 1 -- 172.21.15.140:0/1291777489 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f141000d910 con 0x7f141c0afa10 2022-07-21T01:55:20.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.621+0000 7f142179e700 1 -- 172.21.15.140:0/1291777489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f141c0afa10 msgr2=0x7f141c0b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.621+0000 7f142179e700 1 --2- 172.21.15.140:0/1291777489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f141c0afa10 0x7f141c0b46d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f1410002f70 tx=0x7f1410006df0).stop 2022-07-21T01:55:20.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.621+0000 7f142179e700 1 -- 172.21.15.140:0/1291777489 shutdown_connections 2022-07-21T01:55:20.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.621+0000 7f142179e700 1 --2- 172.21.15.140:0/1291777489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f141c0afa10 0x7f141c0b46d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.621+0000 7f142179e700 1 -- 172.21.15.140:0/1291777489 >> 172.21.15.140:0/1291777489 conn(0x7f141c0b5ee0 msgr2=0x7f141c0b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:20.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.621+0000 7f142179e700 1 -- 172.21.15.140:0/1291777489 shutdown_connections 2022-07-21T01:55:20.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.621+0000 7f142179e700 1 -- 172.21.15.140:0/1291777489 wait complete. 2022-07-21T01:55:20.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.621+0000 7f142179e700 1 Processor -- start 2022-07-21T01:55:20.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.621+0000 7f142179e700 1 -- start start 2022-07-21T01:55:20.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f142179e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f141c0608b0 0x7f141c0b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f142179e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f141c0afa10 0x7f141c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f142179e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f141c0b3de0 0x7f141c151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f142179e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0b3520 con 0x7f141c0afa10 2022-07-21T01:55:20.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f142179e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0b37c0 con 0x7f141c0608b0 2022-07-21T01:55:20.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f142179e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c091550 con 0x7f141c0b3de0 2022-07-21T01:55:20.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f141a7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f141c0afa10 0x7f141c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f141a7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f141c0afa10 0x7f141c0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37392/0 (socket says 172.21.15.140:37392) 2022-07-21T01:55:20.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f141a7fc700 1 -- 172.21.15.140:0/2081245360 learned_addr learned my addr 172.21.15.140:0/2081245360 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:20.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f141b7fe700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f141c0b3de0 0x7f141c151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f141affd700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f141c0608b0 0x7f141c0b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f141a7fc700 1 -- 172.21.15.140:0/2081245360 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f141c0b3de0 msgr2=0x7f141c151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f141a7fc700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f141c0b3de0 0x7f141c151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f141a7fc700 1 -- 172.21.15.140:0/2081245360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f141c0608b0 msgr2=0x7f141c0b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f141a7fc700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f141c0608b0 0x7f141c0b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f141a7fc700 1 -- 172.21.15.140:0/2081245360 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141000b180 con 0x7f141c0afa10 2022-07-21T01:55:20.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f141a7fc700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f141c0afa10 0x7f141c0b2b90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f1410002f70 tx=0x7f1410014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:20.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f140bfff700 1 -- 172.21.15.140:0/2081245360 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1410016070 con 0x7f141c0afa10 2022-07-21T01:55:20.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f140bfff700 1 -- 172.21.15.140:0/2081245360 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14100137d0 con 0x7f141c0afa10 2022-07-21T01:55:20.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f140bfff700 1 -- 172.21.15.140:0/2081245360 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1410013b30 con 0x7f141c0afa10 2022-07-21T01:55:20.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f142179e700 1 -- 172.21.15.140:0/2081245360 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c0b1430 con 0x7f141c0afa10 2022-07-21T01:55:20.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.625+0000 7f142179e700 1 -- 172.21.15.140:0/2081245360 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c0b1d70 con 0x7f141c0afa10 2022-07-21T01:55:20.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.629+0000 7f142179e700 1 -- 172.21.15.140:0/2081245360 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141c091710 con 0x7f141c0afa10 2022-07-21T01:55:20.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.629+0000 7f140bfff700 1 -- 172.21.15.140:0/2081245360 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f141000f030 con 0x7f141c0afa10 2022-07-21T01:55:20.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.629+0000 7f140bfff700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f13f8053a10 0x7f13f8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.629+0000 7f140bfff700 1 -- 172.21.15.140:0/2081245360 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f141006c9b0 con 0x7f141c0afa10 2022-07-21T01:55:20.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.629+0000 7f141affd700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f13f8053a10 0x7f13f8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.629+0000 7f141affd700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f13f8053a10 0x7f13f8055ed0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f140c00d000 tx=0x7f140c007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:20.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.633+0000 7f140bfff700 1 -- 172.21.15.140:0/2081245360 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1410043870 con 0x7f141c0afa10 2022-07-21T01:55:20.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.757+0000 7f142179e700 1 -- 172.21.15.140:0/2081245360 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f141c0574e0 con 0x7f141c0afa10 2022-07-21T01:55:20.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f140bfff700 1 -- 172.21.15.140:0/2081245360 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16150 (secure 0 0 0) 0x7f141002ff30 con 0x7f141c0afa10 2022-07-21T01:55:20.763 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:20.763 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":83,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:20.506631+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:55:18.595416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-07-21T01:55:19.516058+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:20.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 -- 172.21.15.140:0/2081245360 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f13f8053a10 msgr2=0x7f13f8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f13f8053a10 0x7f13f8055ed0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f140c00d000 tx=0x7f140c007a00).stop 2022-07-21T01:55:20.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 -- 172.21.15.140:0/2081245360 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f141c0afa10 msgr2=0x7f141c0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f141c0afa10 0x7f141c0b2b90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f1410002f70 tx=0x7f1410014000).stop 2022-07-21T01:55:20.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 -- 172.21.15.140:0/2081245360 shutdown_connections 2022-07-21T01:55:20.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f13f8053a10 0x7f13f8055ed0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f141c0b3de0 0x7f141c151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f141c0afa10 0x7f141c0b2b90 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 --2- 172.21.15.140:0/2081245360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f141c0608b0 0x7f141c0b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 -- 172.21.15.140:0/2081245360 >> 172.21.15.140:0/2081245360 conn(0x7f141c0b5ee0 msgr2=0x7f141c05c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:20.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 -- 172.21.15.140:0/2081245360 shutdown_connections 2022-07-21T01:55:20.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.761+0000 7f142179e700 1 -- 172.21.15.140:0/2081245360 wait complete. 2022-07-21T01:55:20.776 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:20.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.868+0000 7f68d5ffb700 1 Processor -- start 2022-07-21T01:55:20.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.868+0000 7f68d5ffb700 1 -- start start 2022-07-21T01:55:20.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68d00b1e00 0x7f68d014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f68d005c3f0 0x7f68d00ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d005d480 con 0x7f68d005bd90 2022-07-21T01:55:20.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d00b21e0 con 0x7f68d005c3f0 2022-07-21T01:55:20.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d005cf40 con 0x7f68d00b1e00 2022-07-21T01:55:20.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68ceffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f68d005c3f0 0x7f68d00ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68ceffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f68d005c3f0 0x7f68d00ac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37398/0 (socket says 172.21.15.140:37398) 2022-07-21T01:55:20.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68ceffd700 1 -- 172.21.15.140:0/4054809923 learned_addr learned my addr 172.21.15.140:0/4054809923 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:20.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68cf7fe700 1 --2- 172.21.15.140:0/4054809923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68d00b1e00 0x7f68d014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68ceffd700 1 -- 172.21.15.140:0/4054809923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68d00b1e00 msgr2=0x7f68d014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:20.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68ceffd700 1 --2- 172.21.15.140:0/4054809923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68d00b1e00 0x7f68d014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68ceffd700 1 -- 172.21.15.140:0/4054809923 >> v1:172.21.15.152:6789/0 conn(0x7f68d005bd90 legacy=0x7f68d0146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68ceffd700 1 -- 172.21.15.140:0/4054809923 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d005d1e0 con 0x7f68d005c3f0 2022-07-21T01:55:20.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68ceffd700 1 --2- 172.21.15.140:0/4054809923 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68d005c3f0 0x7f68d00ac7e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f68c00032f0 tx=0x7f68c0009290).ready entity=mon.0 client_cookie=edc95062c0576a1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:20.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68ce7fc700 1 -- 172.21.15.140:0/4054809923 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68c000b8a0 con 0x7f68d005c3f0 2022-07-21T01:55:20.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68ce7fc700 1 -- 172.21.15.140:0/4054809923 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c000ba00 con 0x7f68d005c3f0 2022-07-21T01:55:20.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68ce7fc700 1 -- 172.21.15.140:0/4054809923 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68c000bb70 con 0x7f68d005c3f0 2022-07-21T01:55:20.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 -- 172.21.15.140:0/4054809923 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68d005c3f0 msgr2=0x7f68d00ac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 --2- 172.21.15.140:0/4054809923 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68d005c3f0 0x7f68d00ac7e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f68c00032f0 tx=0x7f68c0009290).stop 2022-07-21T01:55:20.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 -- 172.21.15.140:0/4054809923 shutdown_connections 2022-07-21T01:55:20.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 --2- 172.21.15.140:0/4054809923 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68d005c3f0 0x7f68d00ac7e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 --2- 172.21.15.140:0/4054809923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68d00b1e00 0x7f68d014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 -- 172.21.15.140:0/4054809923 >> 172.21.15.140:0/4054809923 conn(0x7f68d00611c0 msgr2=0x7f68d0062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:20.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 -- 172.21.15.140:0/4054809923 shutdown_connections 2022-07-21T01:55:20.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.872+0000 7f68d5ffb700 1 -- 172.21.15.140:0/4054809923 wait complete. 2022-07-21T01:55:20.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.876+0000 7f68d5ffb700 1 Processor -- start 2022-07-21T01:55:20.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.876+0000 7f68d5ffb700 1 -- start start 2022-07-21T01:55:20.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.876+0000 7f68d5ffb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68d005c3f0 0x7f68d0149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.876+0000 7f68d5ffb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68d00b1e00 0x7f68d0141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.876+0000 7f68d5ffb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68d00b3720 0x7f68d0141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.876+0000 7f68d5ffb700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d00b2ec0 con 0x7f68d005c3f0 2022-07-21T01:55:20.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.876+0000 7f68d5ffb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d009ea90 con 0x7f68d00b3720 2022-07-21T01:55:20.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.876+0000 7f68d5ffb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d009ed30 con 0x7f68d00b1e00 2022-07-21T01:55:20.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.876+0000 7f68cf7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68d005c3f0 0x7f68d0149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.876+0000 7f68cf7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68d005c3f0 0x7f68d0149c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37404/0 (socket says 172.21.15.140:37404) 2022-07-21T01:55:20.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.876+0000 7f68cf7fe700 1 -- 172.21.15.140:0/2235996290 learned_addr learned my addr 172.21.15.140:0/2235996290 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:20.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68ceffd700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68d00b1e00 0x7f68d0141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68cffff700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68d00b3720 0x7f68d0141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68cf7fe700 1 -- 172.21.15.140:0/2235996290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68d00b1e00 msgr2=0x7f68d0141ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68cf7fe700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68d00b1e00 0x7f68d0141ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68cf7fe700 1 -- 172.21.15.140:0/2235996290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68d00b3720 msgr2=0x7f68d0141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:20.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68cf7fe700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68d00b3720 0x7f68d0141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:20.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68cf7fe700 1 -- 172.21.15.140:0/2235996290 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0009530 con 0x7f68d005c3f0 2022-07-21T01:55:20.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68cf7fe700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68d005c3f0 0x7f68d0149c20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f68c40030b0 tx=0x7f68c4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:20.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68ccff9700 1 -- 172.21.15.140:0/2235996290 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68c4011780 con 0x7f68d005c3f0 2022-07-21T01:55:20.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68ccff9700 1 -- 172.21.15.140:0/2235996290 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c40118e0 con 0x7f68d005c3f0 2022-07-21T01:55:20.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68d5ffb700 1 -- 172.21.15.140:0/2235996290 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d00b30f0 con 0x7f68d005c3f0 2022-07-21T01:55:20.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68ccff9700 1 -- 172.21.15.140:0/2235996290 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68c4011c40 con 0x7f68d005c3f0 2022-07-21T01:55:20.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68d5ffb700 1 -- 172.21.15.140:0/2235996290 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d01424a0 con 0x7f68d005c3f0 2022-07-21T01:55:20.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68d5ffb700 1 -- 172.21.15.140:0/2235996290 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d00b3280 con 0x7f68d005c3f0 2022-07-21T01:55:20.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.880+0000 7f68ccff9700 1 -- 172.21.15.140:0/2235996290 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f68c401c460 con 0x7f68d005c3f0 2022-07-21T01:55:20.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.884+0000 7f68ccff9700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68b00539e0 0x7f68b0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:20.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.884+0000 7f68ccff9700 1 -- 172.21.15.140:0/2235996290 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f68c406c2e0 con 0x7f68d005c3f0 2022-07-21T01:55:20.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.884+0000 7f68ceffd700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68b00539e0 0x7f68b0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:20.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.884+0000 7f68ceffd700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68b00539e0 0x7f68b0055ea0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f68c0019000 tx=0x7f68c0016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:20.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:20.888+0000 7f68ccff9700 1 -- 172.21.15.140:0/2235996290 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f68c40431a0 con 0x7f68d005c3f0 2022-07-21T01:55:21.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.012+0000 7f68d5ffb700 1 -- 172.21.15.140:0/2235996290 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68d00b04b0 con 0x7f68d005c3f0 2022-07-21T01:55:21.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.012+0000 7f68ccff9700 1 -- 172.21.15.140:0/2235996290 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16150 (secure 0 0 0) 0x7f68d00b04b0 con 0x7f68d005c3f0 2022-07-21T01:55:21.017 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:21.018 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":83,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:20.506631+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:55:18.595416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-07-21T01:55:19.516058+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:21.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 -- 172.21.15.140:0/2235996290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68b00539e0 msgr2=0x7f68b0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68b00539e0 0x7f68b0055ea0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f68c0019000 tx=0x7f68c0016000).stop 2022-07-21T01:55:21.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 -- 172.21.15.140:0/2235996290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68d005c3f0 msgr2=0x7f68d0149c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68d005c3f0 0x7f68d0149c20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f68c40030b0 tx=0x7f68c4008040).stop 2022-07-21T01:55:21.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 -- 172.21.15.140:0/2235996290 shutdown_connections 2022-07-21T01:55:21.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68b00539e0 0x7f68b0055ea0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68d00b3720 0x7f68d0141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68d00b1e00 0x7f68d0141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 --2- 172.21.15.140:0/2235996290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68d005c3f0 0x7f68d0149c20 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 -- 172.21.15.140:0/2235996290 >> 172.21.15.140:0/2235996290 conn(0x7f68d005bd90 msgr2=0x7f68d01454e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:21.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 -- 172.21.15.140:0/2235996290 shutdown_connections 2022-07-21T01:55:21.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.016+0000 7f68d5ffb700 1 -- 172.21.15.140:0/2235996290 wait complete. 2022-07-21T01:55:21.031 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:55:21.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0359d700 1 Processor -- start 2022-07-21T01:55:21.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0359d700 1 -- start start 2022-07-21T01:55:21.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0359d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1dfc0b3ee0 0x7f1dfc0b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0359d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1dfc0b1280 0x7f1dfc143270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0359d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc0b42c0 con 0x7f1dfc143760 2022-07-21T01:55:21.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0359d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc143cd0 con 0x7f1dfc0b3ee0 2022-07-21T01:55:21.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0359d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc066c90 con 0x7f1dfc0b1280 2022-07-21T01:55:21.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e02d9c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1dfc143760 0x7f1dfc13fbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51560/0 (socket says 172.21.15.140:51560) 2022-07-21T01:55:21.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e02d9c700 1 -- 172.21.15.140:0/1016918222 learned_addr learned my addr 172.21.15.140:0/1016918222 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:21.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0259b700 1 --2- 172.21.15.140:0/1016918222 >> v2:172.21.15.152:3300/0 conn(0x7f1dfc0b3ee0 0x7f1dfc0b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e01d9a700 1 --2- 172.21.15.140:0/1016918222 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1dfc0b1280 0x7f1dfc143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e01599700 1 -- 172.21.15.140:0/1016918222 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139691370 0 0) 0x7f1dfc0b42c0 con 0x7f1dfc143760 2022-07-21T01:55:21.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e01599700 1 -- 172.21.15.140:0/1016918222 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1de4003260 con 0x7f1dfc143760 2022-07-21T01:55:21.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0259b700 1 -- 172.21.15.140:0/1016918222 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1dfc0b1280 msgr2=0x7f1dfc143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:21.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0259b700 1 --2- 172.21.15.140:0/1016918222 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1dfc0b1280 0x7f1dfc143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0259b700 1 -- 172.21.15.140:0/1016918222 >> v1:172.21.15.140:6789/0 conn(0x7f1dfc143760 legacy=0x7f1dfc13fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.136+0000 7f1e0259b700 1 -- 172.21.15.140:0/1016918222 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc142b30 con 0x7f1dfc0b3ee0 2022-07-21T01:55:21.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0259b700 1 --2- 172.21.15.140:0/1016918222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc0b3ee0 0x7f1dfc0b1b10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f1df8002f70 tx=0x7f1df8006df0).ready entity=mon.1 client_cookie=f6673dfb8e936c15 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:21.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e01599700 1 -- 172.21.15.140:0/1016918222 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1df800d4a0 con 0x7f1dfc0b3ee0 2022-07-21T01:55:21.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e01599700 1 -- 172.21.15.140:0/1016918222 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df8000e90 con 0x7f1dfc0b3ee0 2022-07-21T01:55:21.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e01599700 1 -- 172.21.15.140:0/1016918222 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1df800d7f0 con 0x7f1dfc0b3ee0 2022-07-21T01:55:21.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 -- 172.21.15.140:0/1016918222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc0b3ee0 msgr2=0x7f1dfc0b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 --2- 172.21.15.140:0/1016918222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc0b3ee0 0x7f1dfc0b1b10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f1df8002f70 tx=0x7f1df8006df0).stop 2022-07-21T01:55:21.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 -- 172.21.15.140:0/1016918222 shutdown_connections 2022-07-21T01:55:21.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 --2- 172.21.15.140:0/1016918222 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1dfc0b1280 0x7f1dfc143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 --2- 172.21.15.140:0/1016918222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc0b3ee0 0x7f1dfc0b1b10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 -- 172.21.15.140:0/1016918222 >> 172.21.15.140:0/1016918222 conn(0x7f1dfc064c50 msgr2=0x7f1dfc065030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:21.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 -- 172.21.15.140:0/1016918222 shutdown_connections 2022-07-21T01:55:21.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 -- 172.21.15.140:0/1016918222 wait complete. 2022-07-21T01:55:21.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 Processor -- start 2022-07-21T01:55:21.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 -- start start 2022-07-21T01:55:21.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1dfc0b1280 0x7f1dfc1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc0b3ee0 0x7f1dfc13fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1dfc143760 0x7f1dfc140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc156440 con 0x7f1dfc143760 2022-07-21T01:55:21.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc155830 con 0x7f1dfc0b3ee0 2022-07-21T01:55:21.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e0359d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc142530 con 0x7f1dfc0b1280 2022-07-21T01:55:21.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e02d9c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1dfc143760 0x7f1dfc140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e02d9c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1dfc143760 0x7f1dfc140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37422/0 (socket says 172.21.15.140:37422) 2022-07-21T01:55:21.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e02d9c700 1 -- 172.21.15.140:0/433439124 learned_addr learned my addr 172.21.15.140:0/433439124 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:21.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.140+0000 7f1e01d9a700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc0b3ee0 0x7f1dfc13fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1e0259b700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1dfc0b1280 0x7f1dfc1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1e02d9c700 1 -- 172.21.15.140:0/433439124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1dfc0b1280 msgr2=0x7f1dfc1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1e02d9c700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1dfc0b1280 0x7f1dfc1466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1e02d9c700 1 -- 172.21.15.140:0/433439124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc0b3ee0 msgr2=0x7f1dfc13fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1e02d9c700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc0b3ee0 0x7f1dfc13fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1e02d9c700 1 -- 172.21.15.140:0/433439124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df800b180 con 0x7f1dfc143760 2022-07-21T01:55:21.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1e01d9a700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc0b3ee0 0x7f1dfc13fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:21.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1e02d9c700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1dfc143760 0x7f1dfc140150 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f1df4001490 tx=0x7f1df4007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:21.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1deb7fe700 1 -- 172.21.15.140:0/433439124 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1df4005c70 con 0x7f1dfc143760 2022-07-21T01:55:21.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1deb7fe700 1 -- 172.21.15.140:0/433439124 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df4005dd0 con 0x7f1dfc143760 2022-07-21T01:55:21.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1deb7fe700 1 -- 172.21.15.140:0/433439124 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1df40072c0 con 0x7f1dfc143760 2022-07-21T01:55:21.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1e0359d700 1 -- 172.21.15.140:0/433439124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc1427d0 con 0x7f1dfc143760 2022-07-21T01:55:21.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1e0359d700 1 -- 172.21.15.140:0/433439124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc151f70 con 0x7f1dfc143760 2022-07-21T01:55:21.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1e0359d700 1 -- 172.21.15.140:0/433439124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc0b3340 con 0x7f1dfc143760 2022-07-21T01:55:21.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.144+0000 7f1deb7fe700 1 -- 172.21.15.140:0/433439124 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1df4010070 con 0x7f1dfc143760 2022-07-21T01:55:21.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.148+0000 7f1deb7fe700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ddc053b00 0x7f1ddc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.148+0000 7f1deb7fe700 1 -- 172.21.15.140:0/433439124 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1df406b7f0 con 0x7f1dfc143760 2022-07-21T01:55:21.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.148+0000 7f1e0259b700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ddc053b00 0x7f1ddc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.148+0000 7f1e0259b700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ddc053b00 0x7f1ddc055fc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f1df801a000 tx=0x7f1df8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:21.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.152+0000 7f1deb7fe700 1 -- 172.21.15.140:0/433439124 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1df4042710 con 0x7f1dfc143760 2022-07-21T01:55:21.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.320+0000 7f1e0359d700 1 -- 172.21.15.140:0/433439124 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1dfc0574e0 con 0x7f1dfc143760 2022-07-21T01:55:21.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.512+0000 7f1deb7fe700 1 -- 172.21.15.140:0/433439124 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1df4045d30 con 0x7f1dfc143760 2022-07-21T01:55:21.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 -- 172.21.15.140:0/433439124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ddc053b00 msgr2=0x7f1ddc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ddc053b00 0x7f1ddc055fc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f1df801a000 tx=0x7f1df8014000).stop 2022-07-21T01:55:21.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 -- 172.21.15.140:0/433439124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1dfc143760 msgr2=0x7f1dfc140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1dfc143760 0x7f1dfc140150 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f1df4001490 tx=0x7f1df4007da0).stop 2022-07-21T01:55:21.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 -- 172.21.15.140:0/433439124 shutdown_connections 2022-07-21T01:55:21.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ddc053b00 0x7f1ddc055fc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1dfc143760 0x7f1dfc140150 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1dfc0b3ee0 0x7f1dfc13fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 --2- 172.21.15.140:0/433439124 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1dfc0b1280 0x7f1dfc1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 -- 172.21.15.140:0/433439124 >> 172.21.15.140:0/433439124 conn(0x7f1dfc064c50 msgr2=0x7f1dfc12bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:21.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 -- 172.21.15.140:0/433439124 shutdown_connections 2022-07-21T01:55:21.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.520+0000 7f1e0359d700 1 -- 172.21.15.140:0/433439124 wait complete. 2022-07-21T01:55:21.534 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:21.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.640+0000 7f48ef96d700 1 Processor -- start 2022-07-21T01:55:21.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.640+0000 7f48ef96d700 1 -- start start 2022-07-21T01:55:21.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.640+0000 7f48ef96d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f48e80b1620 0x7f48e805bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.640+0000 7f48ef96d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f48e805fd20 0x7f48e8142080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.640+0000 7f48ef96d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e8060160 con 0x7f48e8142570 2022-07-21T01:55:21.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.640+0000 7f48ef96d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e80b1a00 con 0x7f48e80b1620 2022-07-21T01:55:21.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.640+0000 7f48ef96d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e80ac750 con 0x7f48e805fd20 2022-07-21T01:55:21.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.640+0000 7f48edf0a700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f48e8142570 0x7f48e81462b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51576/0 (socket says 172.21.15.140:51576) 2022-07-21T01:55:21.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.640+0000 7f48edf0a700 1 -- 172.21.15.140:0/3220278652 learned_addr learned my addr 172.21.15.140:0/3220278652 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:21.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.640+0000 7f48ed709700 1 --2- 172.21.15.140:0/3220278652 >> v2:172.21.15.152:3300/0 conn(0x7f48e80b1620 0x7f48e805bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.640+0000 7f48ecf08700 1 --2- 172.21.15.140:0/3220278652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f48e805fd20 0x7f48e8142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48d7fff700 1 -- 172.21.15.140:0/3220278652 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687859707 0 0) 0x7f48e8060160 con 0x7f48e8142570 2022-07-21T01:55:21.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48d7fff700 1 -- 172.21.15.140:0/3220278652 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48d0003260 con 0x7f48e8142570 2022-07-21T01:55:21.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ecf08700 1 -- 172.21.15.140:0/3220278652 >> v2:172.21.15.152:3300/0 conn(0x7f48e80b1620 msgr2=0x7f48e805bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:21.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ecf08700 1 --2- 172.21.15.140:0/3220278652 >> v2:172.21.15.152:3300/0 conn(0x7f48e80b1620 0x7f48e805bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ecf08700 1 -- 172.21.15.140:0/3220278652 >> v1:172.21.15.140:6789/0 conn(0x7f48e8142570 legacy=0x7f48e81462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48d7fff700 1 -- 172.21.15.140:0/3220278652 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3071930115 0 0) 0x7f48d0003260 con 0x7f48e8142570 2022-07-21T01:55:21.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ecf08700 1 -- 172.21.15.140:0/3220278652 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e80abd70 con 0x7f48e805fd20 2022-07-21T01:55:21.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ed709700 1 --2- 172.21.15.140:0/3220278652 >> v2:172.21.15.152:3300/0 conn(0x7f48e80b1620 0x7f48e805bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:21.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ecf08700 1 --2- 172.21.15.140:0/3220278652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f48e805fd20 0x7f48e8142080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f48e0001b80 tx=0x7f48d8006df0).ready entity=mon.2 client_cookie=53e71b2283b1dc22 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:21.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48d7fff700 1 -- 172.21.15.140:0/3220278652 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48e00019e0 con 0x7f48e805fd20 2022-07-21T01:55:21.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48d7fff700 1 -- 172.21.15.140:0/3220278652 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d8000e90 con 0x7f48e805fd20 2022-07-21T01:55:21.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48d7fff700 1 -- 172.21.15.140:0/3220278652 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48d8012460 con 0x7f48e805fd20 2022-07-21T01:55:21.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ef96d700 1 -- 172.21.15.140:0/3220278652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f48e805fd20 msgr2=0x7f48e8142080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ef96d700 1 --2- 172.21.15.140:0/3220278652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f48e805fd20 0x7f48e8142080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f48e0001b80 tx=0x7f48d8006df0).stop 2022-07-21T01:55:21.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ef96d700 1 -- 172.21.15.140:0/3220278652 shutdown_connections 2022-07-21T01:55:21.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ef96d700 1 --2- 172.21.15.140:0/3220278652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f48e805fd20 0x7f48e8142080 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ef96d700 1 --2- 172.21.15.140:0/3220278652 >> v2:172.21.15.152:3300/0 conn(0x7f48e80b1620 0x7f48e805bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ef96d700 1 -- 172.21.15.140:0/3220278652 >> 172.21.15.140:0/3220278652 conn(0x7f48e80b23d0 msgr2=0x7f48e80b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:21.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ef96d700 1 -- 172.21.15.140:0/3220278652 shutdown_connections 2022-07-21T01:55:21.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.644+0000 7f48ef96d700 1 -- 172.21.15.140:0/3220278652 wait complete. 2022-07-21T01:55:21.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ef96d700 1 Processor -- start 2022-07-21T01:55:21.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ef96d700 1 -- start start 2022-07-21T01:55:21.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ef96d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f48e805fd20 0x7f48e80ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ef96d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48e80b1620 0x7f48e80ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ef96d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f48e8142570 0x7f48e813d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ef96d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e80afc10 con 0x7f48e805fd20 2022-07-21T01:55:21.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ef96d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e80af970 con 0x7f48e80b1620 2022-07-21T01:55:21.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ef96d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e80af6b0 con 0x7f48e8142570 2022-07-21T01:55:21.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ed709700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f48e805fd20 0x7f48e80ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ed709700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f48e805fd20 0x7f48e80ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37434/0 (socket says 172.21.15.140:37434) 2022-07-21T01:55:21.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ed709700 1 -- 172.21.15.140:0/1769038562 learned_addr learned my addr 172.21.15.140:0/1769038562 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:21.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48edf0a700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f48e8142570 0x7f48e813d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ecf08700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48e80b1620 0x7f48e80ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ed709700 1 -- 172.21.15.140:0/1769038562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f48e8142570 msgr2=0x7f48e813d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ed709700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f48e8142570 0x7f48e813d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ed709700 1 -- 172.21.15.140:0/1769038562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48e80b1620 msgr2=0x7f48e80ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ed709700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48e80b1620 0x7f48e80ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ed709700 1 -- 172.21.15.140:0/1769038562 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e400b180 con 0x7f48e805fd20 2022-07-21T01:55:21.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ed709700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f48e805fd20 0x7f48e80ad130 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f48e400c000 tx=0x7f48e4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:21.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48d67fc700 1 -- 172.21.15.140:0/1769038562 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48e400c7e0 con 0x7f48e805fd20 2022-07-21T01:55:21.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48d67fc700 1 -- 172.21.15.140:0/1769038562 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e400c940 con 0x7f48e805fd20 2022-07-21T01:55:21.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ef96d700 1 -- 172.21.15.140:0/1769038562 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d800b180 con 0x7f48e805fd20 2022-07-21T01:55:21.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48d67fc700 1 -- 172.21.15.140:0/1769038562 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48e400cca0 con 0x7f48e805fd20 2022-07-21T01:55:21.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.648+0000 7f48ef96d700 1 -- 172.21.15.140:0/1769038562 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e80b0000 con 0x7f48e805fd20 2022-07-21T01:55:21.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.652+0000 7f48ef96d700 1 -- 172.21.15.140:0/1769038562 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e80ac010 con 0x7f48e805fd20 2022-07-21T01:55:21.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.652+0000 7f48d67fc700 1 -- 172.21.15.140:0/1769038562 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f48e401c460 con 0x7f48e805fd20 2022-07-21T01:55:21.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.652+0000 7f48d67fc700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f48c8053a70 0x7f48c8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.652+0000 7f48d67fc700 1 -- 172.21.15.140:0/1769038562 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f48e406c4b0 con 0x7f48e805fd20 2022-07-21T01:55:21.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.652+0000 7f48ecf08700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f48c8053a70 0x7f48c8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.652+0000 7f48ecf08700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f48c8053a70 0x7f48c8055f30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f48d801a000 tx=0x7f48d8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:21.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.660+0000 7f48d67fc700 1 -- 172.21.15.140:0/1769038562 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f48e40432d0 con 0x7f48e805fd20 2022-07-21T01:55:21.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 -- 172.21.15.140:0/1769038562 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48e80574e0 con 0x7f48e805fd20 2022-07-21T01:55:21.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48d67fc700 1 -- 172.21.15.140:0/1769038562 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2146 (secure 0 0 0) 0x7f48e40468f0 con 0x7f48e805fd20 2022-07-21T01:55:21.819 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:21.820 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":36,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/2489766347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2489766347},{"type":"v1","addr":"172.21.15.152:6833","nonce":2489766347}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:20.506956+0000","modified":"2022-07-21T01:55:21.509498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6586":{"gid":6586,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2461699636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2461699636},{"type":"v1","addr":"172.21.15.140:6835","nonce":2461699636}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-07-21T01:55:21.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 -- 172.21.15.140:0/1769038562 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f48c8053a70 msgr2=0x7f48c8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f48c8053a70 0x7f48c8055f30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f48d801a000 tx=0x7f48d8014000).stop 2022-07-21T01:55:21.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 -- 172.21.15.140:0/1769038562 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f48e805fd20 msgr2=0x7f48e80ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f48e805fd20 0x7f48e80ad130 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f48e400c000 tx=0x7f48e4006df0).stop 2022-07-21T01:55:21.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 -- 172.21.15.140:0/1769038562 shutdown_connections 2022-07-21T01:55:21.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f48c8053a70 0x7f48c8055f30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f48e8142570 0x7f48e813d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f48e80b1620 0x7f48e80ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 --2- 172.21.15.140:0/1769038562 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f48e805fd20 0x7f48e80ad130 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 -- 172.21.15.140:0/1769038562 >> 172.21.15.140:0/1769038562 conn(0x7f48e80b23d0 msgr2=0x7f48e81437d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:21.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 -- 172.21.15.140:0/1769038562 shutdown_connections 2022-07-21T01:55:21.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.820+0000 7f48ef96d700 1 -- 172.21.15.140:0/1769038562 wait complete. 2022-07-21T01:55:21.824 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 36 2022-07-21T01:55:21.832 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:21.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.928+0000 7efce339b700 1 Processor -- start 2022-07-21T01:55:21.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.928+0000 7efce339b700 1 -- start start 2022-07-21T01:55:21.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce339b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc05c3f0 0x7efcdc14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce339b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcdc05d480 con 0x7efcdc0acef0 2022-07-21T01:55:21.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce339b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcdc0b21e0 con 0x7efcdc0b1e00 2022-07-21T01:55:21.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce339b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc05cf40 con 0x7efcdc05c3f0 2022-07-21T01:55:21.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce1938700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7efcdc0acef0 0x7efcdc146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51588/0 (socket says 172.21.15.140:51588) 2022-07-21T01:55:21.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce1938700 1 -- 172.21.15.140:0/3186044305 learned_addr learned my addr 172.21.15.140:0/3186044305 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:21.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce0936700 1 --2- 172.21.15.140:0/3186044305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc05c3f0 0x7efcdc14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efcd3fff700 1 -- 172.21.15.140:0/3186044305 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350212384 0 0) 0x7efcdc05d480 con 0x7efcdc0acef0 2022-07-21T01:55:21.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efcd3fff700 1 -- 172.21.15.140:0/3186044305 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcc0003260 con 0x7efcdc0acef0 2022-07-21T01:55:21.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce0936700 1 -- 172.21.15.140:0/3186044305 >> v1:172.21.15.152:6789/0 conn(0x7efcdc0b1e00 legacy=0x7efcdc0ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce0936700 1 -- 172.21.15.140:0/3186044305 >> v1:172.21.15.140:6789/0 conn(0x7efcdc0acef0 legacy=0x7efcdc146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce0936700 1 -- 172.21.15.140:0/3186044305 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc05d1e0 con 0x7efcdc05c3f0 2022-07-21T01:55:21.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce0936700 1 --2- 172.21.15.140:0/3186044305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc05c3f0 0x7efcdc14a6d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7efccc002f70 tx=0x7efccc006df0).ready entity=mon.2 client_cookie=400c1f19b76c522a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:21.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efcd3fff700 1 -- 172.21.15.140:0/3186044305 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efccc00d570 con 0x7efcdc05c3f0 2022-07-21T01:55:21.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efcd3fff700 1 -- 172.21.15.140:0/3186044305 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc000e90 con 0x7efcdc05c3f0 2022-07-21T01:55:21.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efcd3fff700 1 -- 172.21.15.140:0/3186044305 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efccc00d8c0 con 0x7efcdc05c3f0 2022-07-21T01:55:21.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce339b700 1 -- 172.21.15.140:0/3186044305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc05c3f0 msgr2=0x7efcdc14a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce339b700 1 --2- 172.21.15.140:0/3186044305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc05c3f0 0x7efcdc14a6d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7efccc002f70 tx=0x7efccc006df0).stop 2022-07-21T01:55:21.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce339b700 1 -- 172.21.15.140:0/3186044305 shutdown_connections 2022-07-21T01:55:21.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce339b700 1 --2- 172.21.15.140:0/3186044305 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc05c3f0 0x7efcdc14a6d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce339b700 1 -- 172.21.15.140:0/3186044305 >> 172.21.15.140:0/3186044305 conn(0x7efcdc0611c0 msgr2=0x7efcdc062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:21.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce339b700 1 -- 172.21.15.140:0/3186044305 shutdown_connections 2022-07-21T01:55:21.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.932+0000 7efce339b700 1 -- 172.21.15.140:0/3186044305 wait complete. 2022-07-21T01:55:21.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce339b700 1 Processor -- start 2022-07-21T01:55:21.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce339b700 1 -- start start 2022-07-21T01:55:21.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce339b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc0acef0 0x7efcdc14c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce339b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efcdc0b1e00 0x7efcdc063e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce339b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcdc0b55d0 0x7efcdc144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce339b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0661c0 con 0x7efcdc0b1e00 2022-07-21T01:55:21.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce339b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc067ae0 con 0x7efcdc0b55d0 2022-07-21T01:55:21.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce339b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc067840 con 0x7efcdc0acef0 2022-07-21T01:55:21.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce0936700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efcdc0b1e00 0x7efcdc063e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce0936700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efcdc0b1e00 0x7efcdc063e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37448/0 (socket says 172.21.15.140:37448) 2022-07-21T01:55:21.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce0936700 1 -- 172.21.15.140:0/1159451863 learned_addr learned my addr 172.21.15.140:0/1159451863 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:21.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce1137700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc0acef0 0x7efcdc14c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce1938700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcdc0b55d0 0x7efcdc144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce0936700 1 -- 172.21.15.140:0/1159451863 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc0acef0 msgr2=0x7efcdc14c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce0936700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc0acef0 0x7efcdc14c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce0936700 1 -- 172.21.15.140:0/1159451863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcdc0b55d0 msgr2=0x7efcdc144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:21.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce0936700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcdc0b55d0 0x7efcdc144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:21.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce0936700 1 -- 172.21.15.140:0/1159451863 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc00b180 con 0x7efcdc0b1e00 2022-07-21T01:55:21.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce1137700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc0acef0 0x7efcdc14c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:21.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce1938700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcdc0b55d0 0x7efcdc144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:21.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce0936700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efcdc0b1e00 0x7efcdc063e10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7efccc002f70 tx=0x7efccc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:21.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efcd27fc700 1 -- 172.21.15.140:0/1159451863 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efccc016070 con 0x7efcdc0b1e00 2022-07-21T01:55:21.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efcd27fc700 1 -- 172.21.15.140:0/1159451863 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc013930 con 0x7efcdc0b1e00 2022-07-21T01:55:21.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce339b700 1 -- 172.21.15.140:0/1159451863 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc064300 con 0x7efcdc0b1e00 2022-07-21T01:55:21.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efcd27fc700 1 -- 172.21.15.140:0/1159451863 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efccc013c90 con 0x7efcdc0b1e00 2022-07-21T01:55:21.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.936+0000 7efce339b700 1 -- 172.21.15.140:0/1159451863 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcdc1449b0 con 0x7efcdc0b1e00 2022-07-21T01:55:21.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.940+0000 7efce339b700 1 -- 172.21.15.140:0/1159451863 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcdc066330 con 0x7efcdc0b1e00 2022-07-21T01:55:21.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.940+0000 7efcd27fc700 1 -- 172.21.15.140:0/1159451863 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efccc024030 con 0x7efcdc0b1e00 2022-07-21T01:55:21.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.940+0000 7efcd27fc700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efcc4053a10 0x7efcc4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:21.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.940+0000 7efcd27fc700 1 -- 172.21.15.140:0/1159451863 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efccc06cad0 con 0x7efcdc0b1e00 2022-07-21T01:55:21.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.940+0000 7efce1137700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efcc4053a10 0x7efcc4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:21.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.940+0000 7efce1137700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efcc4053a10 0x7efcc4055ed0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7efcd400b000 tx=0x7efcd4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:21.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:21.944+0000 7efcd27fc700 1 -- 172.21.15.140:0/1159451863 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efccc0439f0 con 0x7efcdc0b1e00 2022-07-21T01:55:22.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.068+0000 7efce339b700 1 -- 172.21.15.140:0/1159451863 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcdc0574e0 con 0x7efcdc0b1e00 2022-07-21T01:55:22.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.068+0000 7efcd27fc700 1 -- 172.21.15.140:0/1159451863 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16137 (secure 0 0 0) 0x7efccc022070 con 0x7efcdc0b1e00 2022-07-21T01:55:22.068 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:22.069 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":84,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:21.510210+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:55:18.595416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-07-21T01:55:19.516058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:22.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 -- 172.21.15.140:0/1159451863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efcc4053a10 msgr2=0x7efcc4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efcc4053a10 0x7efcc4055ed0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7efcd400b000 tx=0x7efcd4009040).stop 2022-07-21T01:55:22.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 -- 172.21.15.140:0/1159451863 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efcdc0b1e00 msgr2=0x7efcdc063e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efcdc0b1e00 0x7efcdc063e10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7efccc002f70 tx=0x7efccc014000).stop 2022-07-21T01:55:22.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 -- 172.21.15.140:0/1159451863 shutdown_connections 2022-07-21T01:55:22.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efcc4053a10 0x7efcc4055ed0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcdc0b55d0 0x7efcdc144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efcdc0b1e00 0x7efcdc063e10 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 --2- 172.21.15.140:0/1159451863 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efcdc0acef0 0x7efcdc14c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 -- 172.21.15.140:0/1159451863 >> 172.21.15.140:0/1159451863 conn(0x7efcdc05c3f0 msgr2=0x7efcdc0ad6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:22.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 -- 172.21.15.140:0/1159451863 shutdown_connections 2022-07-21T01:55:22.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.072+0000 7efce339b700 1 -- 172.21.15.140:0/1159451863 wait complete. 2022-07-21T01:55:22.083 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:55:22.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa327485700 1 Processor -- start 2022-07-21T01:55:22.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa327485700 1 -- start start 2022-07-21T01:55:22.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa327485700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3200b1600 0x7fa3200aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa327485700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa32005fd90 0x7fa3200613d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa327485700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3200b19e0 con 0x7fa3200618c0 2022-07-21T01:55:22.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa327485700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa320060fe0 con 0x7fa32005fd90 2022-07-21T01:55:22.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa327485700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3200ac740 con 0x7fa3200b1600 2022-07-21T01:55:22.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa325221700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3200b1600 0x7fa3200aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa325a22700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa3200618c0 0x7fa320146280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51604/0 (socket says 172.21.15.140:51604) 2022-07-21T01:55:22.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa325221700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3200b1600 0x7fa3200aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:34184/0 (socket says 172.21.15.140:34184) 2022-07-21T01:55:22.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa325221700 1 -- 172.21.15.140:0/611619666 learned_addr learned my addr 172.21.15.140:0/611619666 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:22.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa324a20700 1 --2- 172.21.15.140:0/611619666 >> v2:172.21.15.152:3300/0 conn(0x7fa32005fd90 0x7fa3200613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa325221700 1 -- 172.21.15.140:0/611619666 >> v2:172.21.15.152:3300/0 conn(0x7fa32005fd90 msgr2=0x7fa3200613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:22.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa325221700 1 --2- 172.21.15.140:0/611619666 >> v2:172.21.15.152:3300/0 conn(0x7fa32005fd90 0x7fa3200613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa325221700 1 -- 172.21.15.140:0/611619666 >> v1:172.21.15.140:6789/0 conn(0x7fa3200618c0 legacy=0x7fa320146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa325221700 1 -- 172.21.15.140:0/611619666 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32009b900 con 0x7fa3200b1600 2022-07-21T01:55:22.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa324a20700 1 --2- 172.21.15.140:0/611619666 >> v2:172.21.15.152:3300/0 conn(0x7fa32005fd90 0x7fa3200613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:22.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.200+0000 7fa325221700 1 --2- 172.21.15.140:0/611619666 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3200b1600 0x7fa3200aa9d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fa31c002f70 tx=0x7fa31c006e50).ready entity=mon.2 client_cookie=9013d577cbfa7c05 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:22.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.208+0000 7fa317fff700 1 -- 172.21.15.140:0/611619666 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa31c00d5c0 con 0x7fa3200b1600 2022-07-21T01:55:22.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.208+0000 7fa317fff700 1 -- 172.21.15.140:0/611619666 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31c000e90 con 0x7fa3200b1600 2022-07-21T01:55:22.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.208+0000 7fa317fff700 1 -- 172.21.15.140:0/611619666 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa31c00d950 con 0x7fa3200b1600 2022-07-21T01:55:22.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.208+0000 7fa327485700 1 -- 172.21.15.140:0/611619666 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3200b1600 msgr2=0x7fa3200aa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.208+0000 7fa327485700 1 --2- 172.21.15.140:0/611619666 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3200b1600 0x7fa3200aa9d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fa31c002f70 tx=0x7fa31c006e50).stop 2022-07-21T01:55:22.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.208+0000 7fa327485700 1 -- 172.21.15.140:0/611619666 shutdown_connections 2022-07-21T01:55:22.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.208+0000 7fa327485700 1 --2- 172.21.15.140:0/611619666 >> v2:172.21.15.152:3300/0 conn(0x7fa32005fd90 0x7fa3200613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.208+0000 7fa327485700 1 --2- 172.21.15.140:0/611619666 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3200b1600 0x7fa3200aa9d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.208+0000 7fa327485700 1 -- 172.21.15.140:0/611619666 >> 172.21.15.140:0/611619666 conn(0x7fa3200b23b0 msgr2=0x7fa3200b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:22.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa327485700 1 -- 172.21.15.140:0/611619666 shutdown_connections 2022-07-21T01:55:22.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa327485700 1 -- 172.21.15.140:0/611619666 wait complete. 2022-07-21T01:55:22.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa327485700 1 Processor -- start 2022-07-21T01:55:22.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa327485700 1 -- start start 2022-07-21T01:55:22.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa327485700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa32005fd90 0x7fa32013fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa327485700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3200618c0 0x7fa320148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa327485700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3200acca0 0x7fa32008fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa327485700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3200adce0 con 0x7fa3200acca0 2022-07-21T01:55:22.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa327485700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa320141560 con 0x7fa3200618c0 2022-07-21T01:55:22.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa327485700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa320141800 con 0x7fa32005fd90 2022-07-21T01:55:22.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa325a22700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3200acca0 0x7fa32008fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa325a22700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3200acca0 0x7fa32008fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37464/0 (socket says 172.21.15.140:37464) 2022-07-21T01:55:22.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa325a22700 1 -- 172.21.15.140:0/2743491894 learned_addr learned my addr 172.21.15.140:0/2743491894 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:22.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa324a20700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3200618c0 0x7fa320148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa325221700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa32005fd90 0x7fa32013fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa325a22700 1 -- 172.21.15.140:0/2743491894 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa32005fd90 msgr2=0x7fa32013fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.212+0000 7fa325a22700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa32005fd90 0x7fa32013fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa325a22700 1 -- 172.21.15.140:0/2743491894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3200618c0 msgr2=0x7fa320148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa325a22700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3200618c0 0x7fa320148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa325a22700 1 -- 172.21.15.140:0/2743491894 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31000b180 con 0x7fa3200acca0 2022-07-21T01:55:22.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa325a22700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3200acca0 0x7fa32008fdb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fa3200624b0 tx=0x7fa318007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:22.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa3167fc700 1 -- 172.21.15.140:0/2743491894 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa31800ce60 con 0x7fa3200acca0 2022-07-21T01:55:22.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa3167fc700 1 -- 172.21.15.140:0/2743491894 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa318004ab0 con 0x7fa3200acca0 2022-07-21T01:55:22.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa3167fc700 1 -- 172.21.15.140:0/2743491894 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa318004e10 con 0x7fa3200acca0 2022-07-21T01:55:22.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa327485700 1 -- 172.21.15.140:0/2743491894 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c00b1e0 con 0x7fa3200acca0 2022-07-21T01:55:22.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa327485700 1 -- 172.21.15.140:0/2743491894 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa320149080 con 0x7fa3200acca0 2022-07-21T01:55:22.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa327485700 1 -- 172.21.15.140:0/2743491894 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3200aeba0 con 0x7fa3200acca0 2022-07-21T01:55:22.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa3167fc700 1 -- 172.21.15.140:0/2743491894 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa318007170 con 0x7fa3200acca0 2022-07-21T01:55:22.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa3167fc700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa308053a70 0x7fa308055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa3167fc700 1 -- 172.21.15.140:0/2743491894 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa31806b940 con 0x7fa3200acca0 2022-07-21T01:55:22.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa325221700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa308053a70 0x7fa308055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.216+0000 7fa325221700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa308053a70 0x7fa308055f30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fa31c01a000 tx=0x7fa31c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:22.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.224+0000 7fa3167fc700 1 -- 172.21.15.140:0/2743491894 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa3200aeba0 con 0x7fa3200acca0 2022-07-21T01:55:22.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.392+0000 7fa327485700 1 -- 172.21.15.140:0/2743491894 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa3200aeba0 con 0x7fa3200acca0 2022-07-21T01:55:22.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa3167fc700 1 -- 172.21.15.140:0/2743491894 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v30) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa3200aeba0 con 0x7fa3200acca0 2022-07-21T01:55:22.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 -- 172.21.15.140:0/2743491894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa308053a70 msgr2=0x7fa308055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa308053a70 0x7fa308055f30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fa31c01a000 tx=0x7fa31c014000).stop 2022-07-21T01:55:22.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 -- 172.21.15.140:0/2743491894 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3200acca0 msgr2=0x7fa32008fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3200acca0 0x7fa32008fdb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fa3200624b0 tx=0x7fa318007f10).stop 2022-07-21T01:55:22.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 -- 172.21.15.140:0/2743491894 shutdown_connections 2022-07-21T01:55:22.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa308053a70 0x7fa308055f30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa3200acca0 0x7fa32008fdb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3200618c0 0x7fa320148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 --2- 172.21.15.140:0/2743491894 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa32005fd90 0x7fa32013fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 -- 172.21.15.140:0/2743491894 >> 172.21.15.140:0/2743491894 conn(0x7fa3200b23b0 msgr2=0x7fa3200b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:22.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 -- 172.21.15.140:0/2743491894 shutdown_connections 2022-07-21T01:55:22.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.400+0000 7fa327485700 1 -- 172.21.15.140:0/2743491894 wait complete. 2022-07-21T01:55:22.409 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:55:22.413 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:55:22.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.524+0000 7fe94f80f700 1 Processor -- start 2022-07-21T01:55:22.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.524+0000 7fe94f80f700 1 -- start start 2022-07-21T01:55:22.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.524+0000 7fe94f80f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9480b4040 0x7fe948143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.524+0000 7fe94f80f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe94813f8d0 0x7fe94813fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.524+0000 7fe94f80f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe94805d0d0 con 0x7fe948065fb0 2022-07-21T01:55:22.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.524+0000 7fe94f80f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9480b4420 con 0x7fe94813f8d0 2022-07-21T01:55:22.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.524+0000 7fe94f80f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94805c0a0 con 0x7fe9480b4040 2022-07-21T01:55:22.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.524+0000 7fe94ddac700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe94813f8d0 0x7fe94813fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.524+0000 7fe94cdaa700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe948065fb0 0x7fe948066440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51614/0 (socket says 172.21.15.140:51614) 2022-07-21T01:55:22.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.524+0000 7fe94cdaa700 1 -- 172.21.15.140:0/1475055906 learned_addr learned my addr 172.21.15.140:0/1475055906 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:22.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.524+0000 7fe94d5ab700 1 --2- 172.21.15.140:0/1475055906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9480b4040 0x7fe948143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe937fff700 1 -- 172.21.15.140:0/1475055906 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248213701 0 0) 0x7fe94805d0d0 con 0x7fe948065fb0 2022-07-21T01:55:22.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe937fff700 1 -- 172.21.15.140:0/1475055906 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe930003260 con 0x7fe948065fb0 2022-07-21T01:55:22.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94ddac700 1 -- 172.21.15.140:0/1475055906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9480b4040 msgr2=0x7fe948143fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:22.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94ddac700 1 --2- 172.21.15.140:0/1475055906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9480b4040 0x7fe948143fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94ddac700 1 -- 172.21.15.140:0/1475055906 >> v1:172.21.15.140:6789/0 conn(0x7fe948065fb0 legacy=0x7fe948066440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94ddac700 1 -- 172.21.15.140:0/1475055906 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94805ce30 con 0x7fe94813f8d0 2022-07-21T01:55:22.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94ddac700 1 --2- 172.21.15.140:0/1475055906 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe94813f8d0 0x7fe94813fd40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fe9400032f0 tx=0x7fe9400091d0).ready entity=mon.0 client_cookie=935c1bdc3ae445f0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:22.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe937fff700 1 -- 172.21.15.140:0/1475055906 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94000b600 con 0x7fe94813f8d0 2022-07-21T01:55:22.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe937fff700 1 -- 172.21.15.140:0/1475055906 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94000b760 con 0x7fe94813f8d0 2022-07-21T01:55:22.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe937fff700 1 -- 172.21.15.140:0/1475055906 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94000b8d0 con 0x7fe94813f8d0 2022-07-21T01:55:22.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94f80f700 1 -- 172.21.15.140:0/1475055906 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe94813f8d0 msgr2=0x7fe94813fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94f80f700 1 --2- 172.21.15.140:0/1475055906 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe94813f8d0 0x7fe94813fd40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fe9400032f0 tx=0x7fe9400091d0).stop 2022-07-21T01:55:22.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94f80f700 1 -- 172.21.15.140:0/1475055906 shutdown_connections 2022-07-21T01:55:22.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94f80f700 1 --2- 172.21.15.140:0/1475055906 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe94813f8d0 0x7fe94813fd40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94f80f700 1 --2- 172.21.15.140:0/1475055906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9480b4040 0x7fe948143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94f80f700 1 -- 172.21.15.140:0/1475055906 >> 172.21.15.140:0/1475055906 conn(0x7fe948064d00 msgr2=0x7fe9480650e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:22.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94f80f700 1 -- 172.21.15.140:0/1475055906 shutdown_connections 2022-07-21T01:55:22.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94f80f700 1 -- 172.21.15.140:0/1475055906 wait complete. 2022-07-21T01:55:22.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94f80f700 1 Processor -- start 2022-07-21T01:55:22.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.528+0000 7fe94f80f700 1 -- start start 2022-07-21T01:55:22.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94f80f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe948065fb0 0x7fe94813d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94f80f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9480b4040 0x7fe948153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94f80f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe94813f8d0 0x7fe948150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94f80f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe948143760 con 0x7fe94813f8d0 2022-07-21T01:55:22.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94f80f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe948143a00 con 0x7fe948065fb0 2022-07-21T01:55:22.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94f80f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe948143ca0 con 0x7fe9480b4040 2022-07-21T01:55:22.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94ddac700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe94813f8d0 0x7fe948150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94ddac700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe94813f8d0 0x7fe948150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37478/0 (socket says 172.21.15.140:37478) 2022-07-21T01:55:22.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94ddac700 1 -- 172.21.15.140:0/3156420122 learned_addr learned my addr 172.21.15.140:0/3156420122 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:22.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94d5ab700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe948065fb0 0x7fe94813d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94cdaa700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9480b4040 0x7fe948153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94ddac700 1 -- 172.21.15.140:0/3156420122 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9480b4040 msgr2=0x7fe948153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94ddac700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9480b4040 0x7fe948153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94ddac700 1 -- 172.21.15.140:0/3156420122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe948065fb0 msgr2=0x7fe94813d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94ddac700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe948065fb0 0x7fe94813d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94ddac700 1 -- 172.21.15.140:0/3156420122 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe940009470 con 0x7fe94813f8d0 2022-07-21T01:55:22.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94ddac700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe94813f8d0 0x7fe948150010 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7fe9400032f0 tx=0x7fe940016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:22.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe9367fc700 1 -- 172.21.15.140:0/3156420122 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94000b8d0 con 0x7fe94813f8d0 2022-07-21T01:55:22.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe9367fc700 1 -- 172.21.15.140:0/3156420122 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94001f3f0 con 0x7fe94813f8d0 2022-07-21T01:55:22.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94f80f700 1 -- 172.21.15.140:0/3156420122 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe948154190 con 0x7fe94813f8d0 2022-07-21T01:55:22.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe9367fc700 1 -- 172.21.15.140:0/3156420122 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94000b600 con 0x7fe94813f8d0 2022-07-21T01:55:22.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94f80f700 1 -- 172.21.15.140:0/3156420122 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94813dfe0 con 0x7fe94813f8d0 2022-07-21T01:55:22.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe9367fc700 1 -- 172.21.15.140:0/3156420122 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe940026030 con 0x7fe94813f8d0 2022-07-21T01:55:22.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.532+0000 7fe94f80f700 1 -- 172.21.15.140:0/3156420122 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94814fd50 con 0x7fe94813f8d0 2022-07-21T01:55:22.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.536+0000 7fe9367fc700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe928053a70 0x7fe928055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.536+0000 7fe9367fc700 1 -- 172.21.15.140:0/3156420122 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe94006cb00 con 0x7fe94813f8d0 2022-07-21T01:55:22.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.536+0000 7fe94d5ab700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe928053a70 0x7fe928055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.536+0000 7fe94d5ab700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe928053a70 0x7fe928055f30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fe94400b000 tx=0x7fe944006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:22.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.540+0000 7fe9367fc700 1 -- 172.21.15.140:0/3156420122 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe94814fd50 con 0x7fe94813f8d0 2022-07-21T01:55:22.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.704+0000 7fe94f80f700 1 -- 172.21.15.140:0/3156420122 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe94814fd50 con 0x7fe94813f8d0 2022-07-21T01:55:22.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe9367fc700 1 -- 172.21.15.140:0/3156420122 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v31) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe94814fd50 con 0x7fe94813f8d0 2022-07-21T01:55:22.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 -- 172.21.15.140:0/3156420122 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe928053a70 msgr2=0x7fe928055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe928053a70 0x7fe928055f30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fe94400b000 tx=0x7fe944006df0).stop 2022-07-21T01:55:22.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 -- 172.21.15.140:0/3156420122 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe94813f8d0 msgr2=0x7fe948150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe94813f8d0 0x7fe948150010 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7fe9400032f0 tx=0x7fe940016000).stop 2022-07-21T01:55:22.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 -- 172.21.15.140:0/3156420122 shutdown_connections 2022-07-21T01:55:22.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe928053a70 0x7fe928055f30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe94813f8d0 0x7fe948150010 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9480b4040 0x7fe948153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 --2- 172.21.15.140:0/3156420122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe948065fb0 0x7fe94813d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 -- 172.21.15.140:0/3156420122 >> 172.21.15.140:0/3156420122 conn(0x7fe948064d00 msgr2=0x7fe948140980 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:22.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 -- 172.21.15.140:0/3156420122 shutdown_connections 2022-07-21T01:55:22.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.712+0000 7fe94f80f700 1 -- 172.21.15.140:0/3156420122 wait complete. 2022-07-21T01:55:22.717 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:55:22.726 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:55:22.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f55188d9700 1 Processor -- start 2022-07-21T01:55:22.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f55188d9700 1 -- start start 2022-07-21T01:55:22.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f55188d9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 0x7f55140ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f55188d9700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5514143a10 0x7f5514143e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f55188d9700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55140b0b80 con 0x7f55140b5970 2022-07-21T01:55:22.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f55188d9700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f551405d4d0 con 0x7f5514143a10 2022-07-21T01:55:22.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f55188d9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55140b5d50 con 0x7f55140af8c0 2022-07-21T01:55:22.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f5513fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5514143a10 0x7f5514143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f5513fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5514143a10 0x7f5514143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37486/0 (socket says 172.21.15.140:37486) 2022-07-21T01:55:22.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f5513fff700 1 -- 172.21.15.140:0/453175268 learned_addr learned my addr 172.21.15.140:0/453175268 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:22.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f5512ffd700 1 --2- 172.21.15.140:0/453175268 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 0x7f55140ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f5513fff700 1 -- 172.21.15.140:0/453175268 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 msgr2=0x7f55140ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:22.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f5513fff700 1 --2- 172.21.15.140:0/453175268 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 0x7f55140ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f5513fff700 1 -- 172.21.15.140:0/453175268 >> v1:172.21.15.152:6789/0 conn(0x7f55140b5970 legacy=0x7f55140ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f5513fff700 1 -- 172.21.15.140:0/453175268 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140b0db0 con 0x7f5514143a10 2022-07-21T01:55:22.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f5512ffd700 1 --2- 172.21.15.140:0/453175268 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 0x7f55140ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:22.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f5513fff700 1 --2- 172.21.15.140:0/453175268 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5514143a10 0x7f5514143e80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f550c0032f0 tx=0x7f550c009290).ready entity=mon.0 client_cookie=59027f8b81fea2b9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:22.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f55127fc700 1 -- 172.21.15.140:0/453175268 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f550c00b8a0 con 0x7f5514143a10 2022-07-21T01:55:22.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f55127fc700 1 -- 172.21.15.140:0/453175268 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550c00ba00 con 0x7f5514143a10 2022-07-21T01:55:22.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.844+0000 7f55127fc700 1 -- 172.21.15.140:0/453175268 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f550c00bb70 con 0x7f5514143a10 2022-07-21T01:55:22.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.848+0000 7f55188d9700 1 -- 172.21.15.140:0/453175268 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5514143a10 msgr2=0x7f5514143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.848+0000 7f55188d9700 1 --2- 172.21.15.140:0/453175268 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5514143a10 0x7f5514143e80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f550c0032f0 tx=0x7f550c009290).stop 2022-07-21T01:55:22.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.848+0000 7f55188d9700 1 -- 172.21.15.140:0/453175268 shutdown_connections 2022-07-21T01:55:22.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.848+0000 7f55188d9700 1 --2- 172.21.15.140:0/453175268 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5514143a10 0x7f5514143e80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.848+0000 7f55188d9700 1 --2- 172.21.15.140:0/453175268 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 0x7f55140ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.848+0000 7f55188d9700 1 -- 172.21.15.140:0/453175268 >> 172.21.15.140:0/453175268 conn(0x7f5514063490 msgr2=0x7f55140b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:22.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.848+0000 7f55188d9700 1 -- 172.21.15.140:0/453175268 shutdown_connections 2022-07-21T01:55:22.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.848+0000 7f55188d9700 1 -- 172.21.15.140:0/453175268 wait complete. 2022-07-21T01:55:22.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.848+0000 7f55188d9700 1 Processor -- start 2022-07-21T01:55:22.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.848+0000 7f55188d9700 1 -- start start 2022-07-21T01:55:22.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f55188d9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 0x7f5514141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f55188d9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55140b5970 0x7f5514140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f55188d9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5514143a10 0x7f5514156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f55188d9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5514156220 con 0x7f5514143a10 2022-07-21T01:55:22.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f55188d9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5514156460 con 0x7f55140b5970 2022-07-21T01:55:22.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f55188d9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5514156700 con 0x7f55140af8c0 2022-07-21T01:55:22.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5513fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5514143a10 0x7f5514156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5512ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55140b5970 0x7f5514140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f55137fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 0x7f5514141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5512ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55140b5970 0x7f5514140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:43230/0 (socket says 172.21.15.140:43230) 2022-07-21T01:55:22.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f55137fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 0x7f5514141630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:34218/0 (socket says 172.21.15.140:34218) 2022-07-21T01:55:22.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5512ffd700 1 -- 172.21.15.140:0/1786507356 learned_addr learned my addr 172.21.15.140:0/1786507356 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:22.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5513fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5514143a10 0x7f5514156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37492/0 (socket says 172.21.15.140:37492) 2022-07-21T01:55:22.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5512ffd700 1 -- 172.21.15.140:0/1786507356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 msgr2=0x7f5514141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5512ffd700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 0x7f5514141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5512ffd700 1 -- 172.21.15.140:0/1786507356 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5514143a10 msgr2=0x7f5514156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:22.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5512ffd700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5514143a10 0x7f5514156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:22.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5512ffd700 1 -- 172.21.15.140:0/1786507356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc00b180 con 0x7f55140b5970 2022-07-21T01:55:22.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5513fff700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5514143a10 0x7f5514156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:22.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5512ffd700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55140b5970 0x7f5514140530 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f54fc00c000 tx=0x7f54fc006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:22.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5510ff9700 1 -- 172.21.15.140:0/1786507356 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54fc010780 con 0x7f55140b5970 2022-07-21T01:55:22.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5510ff9700 1 -- 172.21.15.140:0/1786507356 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc0108e0 con 0x7f55140b5970 2022-07-21T01:55:22.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f55188d9700 1 -- 172.21.15.140:0/1786507356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550c009530 con 0x7f55140b5970 2022-07-21T01:55:22.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f5510ff9700 1 -- 172.21.15.140:0/1786507356 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54fc010c40 con 0x7f55140b5970 2022-07-21T01:55:22.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.852+0000 7f55188d9700 1 -- 172.21.15.140:0/1786507356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5514141b20 con 0x7f55140b5970 2022-07-21T01:55:22.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.856+0000 7f55188d9700 1 -- 172.21.15.140:0/1786507356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55140acae0 con 0x7f55140b5970 2022-07-21T01:55:22.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.856+0000 7f5510ff9700 1 -- 172.21.15.140:0/1786507356 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f54fc01c460 con 0x7f55140b5970 2022-07-21T01:55:22.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.856+0000 7f5510ff9700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54f8053b00 0x7f54f8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:22.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.856+0000 7f5510ff9700 1 -- 172.21.15.140:0/1786507356 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f54fc06c4f0 con 0x7f55140b5970 2022-07-21T01:55:22.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.856+0000 7f55137fe700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54f8053b00 0x7f54f8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:22.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.856+0000 7f55137fe700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54f8053b00 0x7f54f8055fc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f550400d000 tx=0x7f5504007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:22.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:22.864+0000 7f5510ff9700 1 -- 172.21.15.140:0/1786507356 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f54fc0432a0 con 0x7f55140b5970 2022-07-21T01:55:23.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.028+0000 7f55188d9700 1 -- 172.21.15.140:0/1786507356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f55140574e0 con 0x7f55140b5970 2022-07-21T01:55:23.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.040+0000 7f5510ff9700 1 -- 172.21.15.140:0/1786507356 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7f54fc0468c0 con 0x7f55140b5970 2022-07-21T01:55:23.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.040+0000 7f55188d9700 1 -- 172.21.15.140:0/1786507356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54f8053b00 msgr2=0x7f54f8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.040+0000 7f55188d9700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54f8053b00 0x7f54f8055fc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f550400d000 tx=0x7f5504007b90).stop 2022-07-21T01:55:23.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.040+0000 7f55188d9700 1 -- 172.21.15.140:0/1786507356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55140b5970 msgr2=0x7f5514140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.040+0000 7f55188d9700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55140b5970 0x7f5514140530 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f54fc00c000 tx=0x7f54fc006df0).stop 2022-07-21T01:55:23.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.040+0000 7f55188d9700 1 -- 172.21.15.140:0/1786507356 shutdown_connections 2022-07-21T01:55:23.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.040+0000 7f55188d9700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54f8053b00 0x7f54f8055fc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.040+0000 7f55188d9700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5514143a10 0x7f5514156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.040+0000 7f55188d9700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55140b5970 0x7f5514140530 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.040+0000 7f55188d9700 1 --2- 172.21.15.140:0/1786507356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55140af8c0 0x7f5514141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.040+0000 7f55188d9700 1 -- 172.21.15.140:0/1786507356 >> 172.21.15.140:0/1786507356 conn(0x7f5514063490 msgr2=0x7f55140b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:23.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.044+0000 7f55188d9700 1 -- 172.21.15.140:0/1786507356 shutdown_connections 2022-07-21T01:55:23.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.044+0000 7f55188d9700 1 -- 172.21.15.140:0/1786507356 wait complete. 2022-07-21T01:55:23.045 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:55:23.057 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:23.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.160+0000 7f1172cf3700 1 Processor -- start 2022-07-21T01:55:23.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.160+0000 7f1172cf3700 1 -- start start 2022-07-21T01:55:23.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.160+0000 7f1172cf3700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f116c0b5650 0x7f116c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.160+0000 7f1172cf3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f116c0adff0 0x7f116c147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116c147010 con 0x7f116c0af8e0 2022-07-21T01:55:23.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c1472b0 con 0x7f116c0adff0 2022-07-21T01:55:23.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c146d40 con 0x7f116c0b5650 2022-07-21T01:55:23.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f11714f0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f116c0adff0 0x7f116c147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f11714f0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f116c0adff0 0x7f116c147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37498/0 (socket says 172.21.15.140:37498) 2022-07-21T01:55:23.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f11714f0700 1 -- 172.21.15.140:0/1699512978 learned_addr learned my addr 172.21.15.140:0/1699512978 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:23.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f11724f2700 1 --1- 172.21.15.140:0/1699512978 >> v1:172.21.15.152:6789/0 conn(0x7f116c0af8e0 0x7f116c05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:52760/0 (socket says 172.21.15.140:52760) 2022-07-21T01:55:23.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1171cf1700 1 --2- 172.21.15.140:0/1699512978 >> v2:172.21.15.152:3300/0 conn(0x7f116c0b5650 0x7f116c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1170cef700 1 -- 172.21.15.140:0/1699512978 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104014388 0 0) 0x7f116c147010 con 0x7f116c0af8e0 2022-07-21T01:55:23.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1170cef700 1 -- 172.21.15.140:0/1699512978 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1154003260 con 0x7f116c0af8e0 2022-07-21T01:55:23.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f11714f0700 1 -- 172.21.15.140:0/1699512978 >> v2:172.21.15.152:3300/0 conn(0x7f116c0b5650 msgr2=0x7f116c0b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:23.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f11714f0700 1 --2- 172.21.15.140:0/1699512978 >> v2:172.21.15.152:3300/0 conn(0x7f116c0b5650 0x7f116c0b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f11714f0700 1 -- 172.21.15.140:0/1699512978 >> v1:172.21.15.152:6789/0 conn(0x7f116c0af8e0 legacy=0x7f116c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f11714f0700 1 -- 172.21.15.140:0/1699512978 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c147c40 con 0x7f116c0adff0 2022-07-21T01:55:23.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1171cf1700 1 --2- 172.21.15.140:0/1699512978 >> v2:172.21.15.152:3300/0 conn(0x7f116c0b5650 0x7f116c0b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:23.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f11714f0700 1 --2- 172.21.15.140:0/1699512978 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f116c0adff0 0x7f116c147750 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f115c002f70 tx=0x7f115c006e50).ready entity=mon.0 client_cookie=f26637f060310e7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:23.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1170cef700 1 -- 172.21.15.140:0/1699512978 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f115c00d5c0 con 0x7f116c0adff0 2022-07-21T01:55:23.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1170cef700 1 -- 172.21.15.140:0/1699512978 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c000e90 con 0x7f116c0adff0 2022-07-21T01:55:23.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1170cef700 1 -- 172.21.15.140:0/1699512978 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f115c00d950 con 0x7f116c0adff0 2022-07-21T01:55:23.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 -- 172.21.15.140:0/1699512978 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f116c0adff0 msgr2=0x7f116c147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 --2- 172.21.15.140:0/1699512978 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f116c0adff0 0x7f116c147750 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f115c002f70 tx=0x7f115c006e50).stop 2022-07-21T01:55:23.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 -- 172.21.15.140:0/1699512978 shutdown_connections 2022-07-21T01:55:23.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 --2- 172.21.15.140:0/1699512978 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f116c0adff0 0x7f116c147750 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 --2- 172.21.15.140:0/1699512978 >> v2:172.21.15.152:3300/0 conn(0x7f116c0b5650 0x7f116c0b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 -- 172.21.15.140:0/1699512978 >> 172.21.15.140:0/1699512978 conn(0x7f116c0634e0 msgr2=0x7f116c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:23.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 -- 172.21.15.140:0/1699512978 shutdown_connections 2022-07-21T01:55:23.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 -- 172.21.15.140:0/1699512978 wait complete. 2022-07-21T01:55:23.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 Processor -- start 2022-07-21T01:55:23.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.164+0000 7f1172cf3700 1 -- start start 2022-07-21T01:55:23.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1172cf3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f116c0adff0 0x7f116c0b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1172cf3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f116c0af8e0 0x7f116c066fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1172cf3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f116c0b5650 0x7f116c065c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1172cf3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c145310 con 0x7f116c0af8e0 2022-07-21T01:55:23.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1172cf3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c1112b0 con 0x7f116c0b5650 2022-07-21T01:55:23.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1172cf3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c063990 con 0x7f116c0adff0 2022-07-21T01:55:23.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1171cf1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f116c0adff0 0x7f116c0b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f11714f0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f116c0af8e0 0x7f116c066fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f11724f2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f116c0b5650 0x7f116c065c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1171cf1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f116c0adff0 0x7f116c0b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:34232/0 (socket says 172.21.15.140:34232) 2022-07-21T01:55:23.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1171cf1700 1 -- 172.21.15.140:0/1948876534 learned_addr learned my addr 172.21.15.140:0/1948876534 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:23.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f11714f0700 1 -- 172.21.15.140:0/1948876534 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f116c0adff0 msgr2=0x7f116c0b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f11714f0700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f116c0adff0 0x7f116c0b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f11714f0700 1 -- 172.21.15.140:0/1948876534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f116c0b5650 msgr2=0x7f116c065c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f11714f0700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f116c0b5650 0x7f116c065c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f11714f0700 1 -- 172.21.15.140:0/1948876534 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1164009470 con 0x7f116c0af8e0 2022-07-21T01:55:23.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f11724f2700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f116c0b5650 0x7f116c065c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:23.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1171cf1700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f116c0adff0 0x7f116c0b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:23.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f11714f0700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f116c0af8e0 0x7f116c066fa0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7f115c002f70 tx=0x7f115c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:23.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f115affd700 1 -- 172.21.15.140:0/1948876534 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f115c016070 con 0x7f116c0af8e0 2022-07-21T01:55:23.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f115affd700 1 -- 172.21.15.140:0/1948876534 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c00de20 con 0x7f116c0af8e0 2022-07-21T01:55:23.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1172cf3700 1 -- 172.21.15.140:0/1948876534 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c00b1e0 con 0x7f116c0af8e0 2022-07-21T01:55:23.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f115affd700 1 -- 172.21.15.140:0/1948876534 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f115c013da0 con 0x7f116c0af8e0 2022-07-21T01:55:23.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1172cf3700 1 -- 172.21.15.140:0/1948876534 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c111010 con 0x7f116c0af8e0 2022-07-21T01:55:23.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.168+0000 7f1172cf3700 1 -- 172.21.15.140:0/1948876534 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116c110b90 con 0x7f116c0af8e0 2022-07-21T01:55:23.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.172+0000 7f115affd700 1 -- 172.21.15.140:0/1948876534 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f115c00f030 con 0x7f116c0af8e0 2022-07-21T01:55:23.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.172+0000 7f115affd700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f114c0539e0 0x7f114c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.172+0000 7f115affd700 1 -- 172.21.15.140:0/1948876534 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f115c06cd00 con 0x7f116c0af8e0 2022-07-21T01:55:23.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.172+0000 7f1171cf1700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f114c0539e0 0x7f114c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.172+0000 7f1171cf1700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f114c0539e0 0x7f114c055ea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f116400b000 tx=0x7f11640091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:23.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.176+0000 7f115affd700 1 -- 172.21.15.140:0/1948876534 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f115c070030 con 0x7f116c0af8e0 2022-07-21T01:55:23.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.340+0000 7f1172cf3700 1 -- 172.21.15.140:0/1948876534 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f116c05c4a0 con 0x7f116c0af8e0 2022-07-21T01:55:23.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.340+0000 7f115affd700 1 -- 172.21.15.140:0/1948876534 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2146 (secure 0 0 0) 0x7f115c043cd0 con 0x7f116c0af8e0 2022-07-21T01:55:23.340 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:23.341 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":36,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/2489766347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2489766347},{"type":"v1","addr":"172.21.15.152:6833","nonce":2489766347}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:20.506956+0000","modified":"2022-07-21T01:55:21.509498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6586":{"gid":6586,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2461699636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2461699636},{"type":"v1","addr":"172.21.15.140:6835","nonce":2461699636}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-07-21T01:55:23.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.340+0000 7f1172cf3700 1 -- 172.21.15.140:0/1948876534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f114c0539e0 msgr2=0x7f114c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.340+0000 7f1172cf3700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f114c0539e0 0x7f114c055ea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f116400b000 tx=0x7f11640091d0).stop 2022-07-21T01:55:23.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.344+0000 7f1172cf3700 1 -- 172.21.15.140:0/1948876534 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f116c0af8e0 msgr2=0x7f116c066fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.344+0000 7f1172cf3700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f116c0af8e0 0x7f116c066fa0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7f115c002f70 tx=0x7f115c014000).stop 2022-07-21T01:55:23.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.344+0000 7f1172cf3700 1 -- 172.21.15.140:0/1948876534 shutdown_connections 2022-07-21T01:55:23.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.344+0000 7f1172cf3700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f114c0539e0 0x7f114c055ea0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.344+0000 7f1172cf3700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f116c0b5650 0x7f116c065c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.344+0000 7f1172cf3700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f116c0af8e0 0x7f116c066fa0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.344+0000 7f1172cf3700 1 --2- 172.21.15.140:0/1948876534 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f116c0adff0 0x7f116c0b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.344+0000 7f1172cf3700 1 -- 172.21.15.140:0/1948876534 >> 172.21.15.140:0/1948876534 conn(0x7f116c0634e0 msgr2=0x7f116c146110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:23.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.344+0000 7f1172cf3700 1 -- 172.21.15.140:0/1948876534 shutdown_connections 2022-07-21T01:55:23.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.344+0000 7f1172cf3700 1 -- 172.21.15.140:0/1948876534 wait complete. 2022-07-21T01:55:23.345 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 36 2022-07-21T01:55:23.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:55:20.506956+0000', 'modified': '2022-07-21T01:55:21.509498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6586}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6586': {'gid': 6586, 'name': 'a', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/2461699636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 2461699636}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 2461699636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:55:23.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:55:23.558 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:55:23.674 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":6586,"want_state":"up:active","state":"up:active","rank_uptime":3.1555014739999998,"mdsmap_epoch":36,"osdmap_epoch":84,"osdmap_epoch_barrier":83,"uptime":7.8606374719999996} 2022-07-21T01:55:23.675 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 6586, "mdsmap_epoch": 36, "osdmap_epoch": 84, "osdmap_epoch_barrier": 83, "rank_uptime": 3.155501474, "state": "up:active", "uptime": 7.860637472, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:55:23.676 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:23.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 Processor -- start 2022-07-21T01:55:23.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 -- start start 2022-07-21T01:55:23.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f43340b5650 0x7f43340b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43340adff0 0x7f4334147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f43340af8e0 0x7f433405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4334147010 con 0x7f43340af8e0 2022-07-21T01:55:23.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43341472b0 con 0x7f43340b5650 2022-07-21T01:55:23.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4334146d40 con 0x7f43340adff0 2022-07-21T01:55:23.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f4339f32700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f43340af8e0 0x7f433405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f4339f32700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f43340af8e0 0x7f433405bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37514/0 (socket says 172.21.15.140:37514) 2022-07-21T01:55:23.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f4339f32700 1 -- 172.21.15.140:0/2579167050 learned_addr learned my addr 172.21.15.140:0/2579167050 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:23.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f4338f30700 1 --2- 172.21.15.140:0/2579167050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43340adff0 0x7f4334147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f4339731700 1 --2- 172.21.15.140:0/2579167050 >> v2:172.21.15.152:3300/0 conn(0x7f43340b5650 0x7f43340b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f4339f32700 1 -- 172.21.15.140:0/2579167050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43340adff0 msgr2=0x7f4334147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:23.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f4339f32700 1 --2- 172.21.15.140:0/2579167050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43340adff0 0x7f4334147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f4339f32700 1 -- 172.21.15.140:0/2579167050 >> v2:172.21.15.152:3300/0 conn(0x7f43340b5650 msgr2=0x7f43340b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:23.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f4339f32700 1 --2- 172.21.15.140:0/2579167050 >> v2:172.21.15.152:3300/0 conn(0x7f43340b5650 0x7f43340b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f4339f32700 1 -- 172.21.15.140:0/2579167050 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4334147c40 con 0x7f43340af8e0 2022-07-21T01:55:23.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f4339f32700 1 --2- 172.21.15.140:0/2579167050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43340af8e0 0x7f433405bd90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f432c0032f0 tx=0x7f432c009290).ready entity=mon.0 client_cookie=7705c97dddc97ac1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:23.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f432bfff700 1 -- 172.21.15.140:0/2579167050 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432c00b870 con 0x7f43340af8e0 2022-07-21T01:55:23.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f432bfff700 1 -- 172.21.15.140:0/2579167050 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c00b9d0 con 0x7f43340af8e0 2022-07-21T01:55:23.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f432bfff700 1 -- 172.21.15.140:0/2579167050 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432c00bb40 con 0x7f43340af8e0 2022-07-21T01:55:23.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 -- 172.21.15.140:0/2579167050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43340af8e0 msgr2=0x7f433405bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 --2- 172.21.15.140:0/2579167050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43340af8e0 0x7f433405bd90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f432c0032f0 tx=0x7f432c009290).stop 2022-07-21T01:55:23.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 -- 172.21.15.140:0/2579167050 shutdown_connections 2022-07-21T01:55:23.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 --2- 172.21.15.140:0/2579167050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43340af8e0 0x7f433405bd90 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 --2- 172.21.15.140:0/2579167050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43340adff0 0x7f4334147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 --2- 172.21.15.140:0/2579167050 >> v2:172.21.15.152:3300/0 conn(0x7f43340b5650 0x7f43340b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 -- 172.21.15.140:0/2579167050 >> 172.21.15.140:0/2579167050 conn(0x7f43340634e0 msgr2=0x7f43340b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:23.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 -- 172.21.15.140:0/2579167050 shutdown_connections 2022-07-21T01:55:23.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.784+0000 7f433a733700 1 -- 172.21.15.140:0/2579167050 wait complete. 2022-07-21T01:55:23.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f433a733700 1 Processor -- start 2022-07-21T01:55:23.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f433a733700 1 -- start start 2022-07-21T01:55:23.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f433a733700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43340adff0 0x7f4334149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f433a733700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43340af8e0 0x7f4334066f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f433a733700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43340b5650 0x7f4334067480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f433a733700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43341562a0 con 0x7f43340b5650 2022-07-21T01:55:23.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f433a733700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4334149b40 con 0x7f43340adff0 2022-07-21T01:55:23.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f433a733700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43341564d0 con 0x7f43340af8e0 2022-07-21T01:55:23.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4339f32700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43340b5650 0x7f4334067480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4339f32700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43340b5650 0x7f4334067480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37520/0 (socket says 172.21.15.140:37520) 2022-07-21T01:55:23.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4339f32700 1 -- 172.21.15.140:0/545482652 learned_addr learned my addr 172.21.15.140:0/545482652 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:23.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4338f30700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43340af8e0 0x7f4334066f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4339731700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43340adff0 0x7f4334149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4339f32700 1 -- 172.21.15.140:0/545482652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43340af8e0 msgr2=0x7f4334066f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4339f32700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43340af8e0 0x7f4334066f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4339f32700 1 -- 172.21.15.140:0/545482652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43340adff0 msgr2=0x7f4334149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4339f32700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43340adff0 0x7f4334149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4339f32700 1 -- 172.21.15.140:0/545482652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c009530 con 0x7f43340b5650 2022-07-21T01:55:23.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4339731700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43340adff0 0x7f4334149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:55:23.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f4339f32700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43340b5650 0x7f4334067480 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f432c0032f0 tx=0x7f432c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:23.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f432a7fc700 1 -- 172.21.15.140:0/545482652 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432c00b870 con 0x7f43340b5650 2022-07-21T01:55:23.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f432a7fc700 1 -- 172.21.15.140:0/545482652 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c0105b0 con 0x7f43340b5650 2022-07-21T01:55:23.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f433a733700 1 -- 172.21.15.140:0/545482652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4334067970 con 0x7f43340b5650 2022-07-21T01:55:23.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f432a7fc700 1 -- 172.21.15.140:0/545482652 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432c010910 con 0x7f43340b5650 2022-07-21T01:55:23.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f433a733700 1 -- 172.21.15.140:0/545482652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433410fa90 con 0x7f43340b5650 2022-07-21T01:55:23.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.788+0000 7f433a733700 1 -- 172.21.15.140:0/545482652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43341566d0 con 0x7f43340b5650 2022-07-21T01:55:23.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.792+0000 7f432a7fc700 1 -- 172.21.15.140:0/545482652 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f432c027020 con 0x7f43340b5650 2022-07-21T01:55:23.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.792+0000 7f432a7fc700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f431c0539e0 0x7f431c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:23.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.792+0000 7f432a7fc700 1 -- 172.21.15.140:0/545482652 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f432c06d390 con 0x7f43340b5650 2022-07-21T01:55:23.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.792+0000 7f4339731700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f431c0539e0 0x7f431c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:23.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.792+0000 7f4339731700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f431c0539e0 0x7f431c055ea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f4324008000 tx=0x7f4324008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:23.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.796+0000 7f432a7fc700 1 -- 172.21.15.140:0/545482652 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f432c0442b0 con 0x7f43340b5650 2022-07-21T01:55:23.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.924+0000 7f433a733700 1 -- 172.21.15.140:0/545482652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43340574e0 con 0x7f43340b5650 2022-07-21T01:55:23.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.924+0000 7f432a7fc700 1 -- 172.21.15.140:0/545482652 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f432c015020 con 0x7f43340b5650 2022-07-21T01:55:23.926 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:23.927 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":84,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:21.510210+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:55:18.595416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-07-21T01:55:19.516058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:23.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 -- 172.21.15.140:0/545482652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f431c0539e0 msgr2=0x7f431c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f431c0539e0 0x7f431c055ea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f4324008000 tx=0x7f4324008040).stop 2022-07-21T01:55:23.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 -- 172.21.15.140:0/545482652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43340b5650 msgr2=0x7f4334067480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:23.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43340b5650 0x7f4334067480 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f432c0032f0 tx=0x7f432c016000).stop 2022-07-21T01:55:23.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 -- 172.21.15.140:0/545482652 shutdown_connections 2022-07-21T01:55:23.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f431c0539e0 0x7f431c055ea0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43340b5650 0x7f4334067480 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43340af8e0 0x7f4334066f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 --2- 172.21.15.140:0/545482652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43340adff0 0x7f4334149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:23.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 -- 172.21.15.140:0/545482652 >> 172.21.15.140:0/545482652 conn(0x7f43340634e0 msgr2=0x7f43340afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:23.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 -- 172.21.15.140:0/545482652 shutdown_connections 2022-07-21T01:55:23.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:23.928+0000 7f433a733700 1 -- 172.21.15.140:0/545482652 wait complete. 2022-07-21T01:55:23.943 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:55:18.595416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:55:19.516058+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:55:23.944 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:55:24.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f772ae81700 1 Processor -- start 2022-07-21T01:55:24.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f772ae81700 1 -- start start 2022-07-21T01:55:24.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f772ae81700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f77240b59f0 0x7f772405c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f772ae81700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 0x7f772405c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f772ae81700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772405d2f0 con 0x7f77240addd0 2022-07-21T01:55:24.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f772ae81700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7724146350 con 0x7f77240b59f0 2022-07-21T01:55:24.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f772ae81700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77240b5dd0 con 0x7f77240af940 2022-07-21T01:55:24.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f772967e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 0x7f772405c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f772967e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 0x7f772405c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:34256/0 (socket says 172.21.15.140:34256) 2022-07-21T01:55:24.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f7729e7f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f77240b59f0 0x7f772405c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f772967e700 1 -- 172.21.15.140:0/413821405 learned_addr learned my addr 172.21.15.140:0/413821405 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:24.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.052+0000 7f7728e7d700 1 -- 172.21.15.140:0/413821405 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371188243 0 0) 0x7f772405d2f0 con 0x7f77240addd0 2022-07-21T01:55:24.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f7728e7d700 1 -- 172.21.15.140:0/413821405 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7710003260 con 0x7f77240addd0 2022-07-21T01:55:24.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772967e700 1 -- 172.21.15.140:0/413821405 >> v2:172.21.15.152:3300/0 conn(0x7f77240b59f0 msgr2=0x7f772405c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:24.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772967e700 1 --2- 172.21.15.140:0/413821405 >> v2:172.21.15.152:3300/0 conn(0x7f77240b59f0 0x7f772405c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772967e700 1 -- 172.21.15.140:0/413821405 >> v1:172.21.15.152:6789/0 conn(0x7f77240addd0 legacy=0x7f7724141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772967e700 1 -- 172.21.15.140:0/413821405 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77241460b0 con 0x7f77240af940 2022-07-21T01:55:24.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772967e700 1 --2- 172.21.15.140:0/413821405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 0x7f772405c5c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f77200032f0 tx=0x7f7720009290).ready entity=mon.2 client_cookie=fe325f664164de02 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f7728e7d700 1 -- 172.21.15.140:0/413821405 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f772000b6c0 con 0x7f77240af940 2022-07-21T01:55:24.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f7728e7d700 1 -- 172.21.15.140:0/413821405 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772000b820 con 0x7f77240af940 2022-07-21T01:55:24.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f7728e7d700 1 -- 172.21.15.140:0/413821405 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f772000b990 con 0x7f77240af940 2022-07-21T01:55:24.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772ae81700 1 -- 172.21.15.140:0/413821405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 msgr2=0x7f772405c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772ae81700 1 --2- 172.21.15.140:0/413821405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 0x7f772405c5c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f77200032f0 tx=0x7f7720009290).stop 2022-07-21T01:55:24.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772ae81700 1 -- 172.21.15.140:0/413821405 shutdown_connections 2022-07-21T01:55:24.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772ae81700 1 --2- 172.21.15.140:0/413821405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 0x7f772405c5c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772ae81700 1 --2- 172.21.15.140:0/413821405 >> v2:172.21.15.152:3300/0 conn(0x7f77240b59f0 0x7f772405c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772ae81700 1 -- 172.21.15.140:0/413821405 >> 172.21.15.140:0/413821405 conn(0x7f7724063480 msgr2=0x7f77240b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:24.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772ae81700 1 -- 172.21.15.140:0/413821405 shutdown_connections 2022-07-21T01:55:24.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772ae81700 1 -- 172.21.15.140:0/413821405 wait complete. 2022-07-21T01:55:24.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772ae81700 1 Processor -- start 2022-07-21T01:55:24.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.056+0000 7f772ae81700 1 -- start start 2022-07-21T01:55:24.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f772ae81700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f77240addd0 0x7f772405c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f772ae81700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 0x7f772413e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f772ae81700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77240b59f0 0x7f7724113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f772ae81700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7724155280 con 0x7f77240addd0 2022-07-21T01:55:24.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f772ae81700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7724064f10 con 0x7f77240b59f0 2022-07-21T01:55:24.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f772ae81700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7724067090 con 0x7f77240af940 2022-07-21T01:55:24.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f7729e7f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f77240addd0 0x7f772405c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f7729e7f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f77240addd0 0x7f772405c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37530/0 (socket says 172.21.15.140:37530) 2022-07-21T01:55:24.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f7729e7f700 1 -- 172.21.15.140:0/1074113585 learned_addr learned my addr 172.21.15.140:0/1074113585 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:24.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f772967e700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 0x7f772413e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f772a680700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77240b59f0 0x7f7724113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f7729e7f700 1 -- 172.21.15.140:0/1074113585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 msgr2=0x7f772413e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f7729e7f700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 0x7f772413e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f7729e7f700 1 -- 172.21.15.140:0/1074113585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77240b59f0 msgr2=0x7f7724113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f7729e7f700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77240b59f0 0x7f7724113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f7729e7f700 1 -- 172.21.15.140:0/1074113585 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7720009530 con 0x7f77240addd0 2022-07-21T01:55:24.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f7729e7f700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f77240addd0 0x7f772405c0d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f771400c000 tx=0x7f7714006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f771affd700 1 -- 172.21.15.140:0/1074113585 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7714011750 con 0x7f77240addd0 2022-07-21T01:55:24.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f771affd700 1 -- 172.21.15.140:0/1074113585 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77140118b0 con 0x7f77240addd0 2022-07-21T01:55:24.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f772ae81700 1 -- 172.21.15.140:0/1074113585 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7724067330 con 0x7f77240addd0 2022-07-21T01:55:24.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f771affd700 1 -- 172.21.15.140:0/1074113585 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7714011c10 con 0x7f77240addd0 2022-07-21T01:55:24.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f772ae81700 1 -- 172.21.15.140:0/1074113585 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772405c5c0 con 0x7f77240addd0 2022-07-21T01:55:24.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.060+0000 7f772ae81700 1 -- 172.21.15.140:0/1074113585 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77240677d0 con 0x7f77240addd0 2022-07-21T01:55:24.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.064+0000 7f771affd700 1 -- 172.21.15.140:0/1074113585 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7714011d70 con 0x7f77240addd0 2022-07-21T01:55:24.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.064+0000 7f771affd700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f77040539e0 0x7f7704055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.064+0000 7f771affd700 1 -- 172.21.15.140:0/1074113585 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f771406c310 con 0x7f77240addd0 2022-07-21T01:55:24.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.064+0000 7f772967e700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f77040539e0 0x7f7704055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.064+0000 7f772967e700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f77040539e0 0x7f7704055ea0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f772001b000 tx=0x7f7720016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.072+0000 7f771affd700 1 -- 172.21.15.140:0/1074113585 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f77240677d0 con 0x7f77240addd0 2022-07-21T01:55:24.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 -- 172.21.15.140:0/1074113585 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f77240677d0 con 0x7f77240addd0 2022-07-21T01:55:24.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f771affd700 1 -- 172.21.15.140:0/1074113585 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v84) v1 ==== 110+0+10 (secure 0 0 0) 0x7f77240677d0 con 0x7f77240addd0 2022-07-21T01:55:24.195 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:55:24.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 -- 172.21.15.140:0/1074113585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f77040539e0 msgr2=0x7f7704055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f77040539e0 0x7f7704055ea0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f772001b000 tx=0x7f7720016000).stop 2022-07-21T01:55:24.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 -- 172.21.15.140:0/1074113585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f77240addd0 msgr2=0x7f772405c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f77240addd0 0x7f772405c0d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f771400c000 tx=0x7f7714006df0).stop 2022-07-21T01:55:24.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 -- 172.21.15.140:0/1074113585 shutdown_connections 2022-07-21T01:55:24.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f77040539e0 0x7f7704055ea0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77240b59f0 0x7f7724113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f77240af940 0x7f772413e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 --2- 172.21.15.140:0/1074113585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f77240addd0 0x7f772405c0d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 -- 172.21.15.140:0/1074113585 >> 172.21.15.140:0/1074113585 conn(0x7f7724063480 msgr2=0x7f77241458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:24.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 -- 172.21.15.140:0/1074113585 shutdown_connections 2022-07-21T01:55:24.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.196+0000 7f772ae81700 1 -- 172.21.15.140:0/1074113585 wait complete. 2022-07-21T01:55:24.207 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:55:24.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.308+0000 7fc63bfff700 1 Processor -- start 2022-07-21T01:55:24.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.308+0000 7fc63bfff700 1 -- start start 2022-07-21T01:55:24.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b4ee0 0x7fc63c13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63bfff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c0b5d00 con 0x7fc63c142c40 2022-07-21T01:55:24.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63bfff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c13f090 con 0x7fc63c0b5920 2022-07-21T01:55:24.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c13f330 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc63c142c40 0x7fc63c13e2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51688/0 (socket says 172.21.15.140:51688) 2022-07-21T01:55:24.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63b7fe700 1 -- 172.21.15.140:0/466476199 learned_addr learned my addr 172.21.15.140:0/466476199 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:24.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc6327fc700 1 --2- 172.21.15.140:0/466476199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b4ee0 0x7fc63c13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63a7fc700 1 -- 172.21.15.140:0/466476199 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099404810 0 0) 0x7fc63c0b5d00 con 0x7fc63c142c40 2022-07-21T01:55:24.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63a7fc700 1 -- 172.21.15.140:0/466476199 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc61c004370 con 0x7fc63c142c40 2022-07-21T01:55:24.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63a7fc700 1 -- 172.21.15.140:0/466476199 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233817093 0 0) 0x7fc63c13f090 con 0x7fc63c0b5920 2022-07-21T01:55:24.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63a7fc700 1 -- 172.21.15.140:0/466476199 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc63c0b5d00 con 0x7fc63c0b5920 2022-07-21T01:55:24.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc6327fc700 1 -- 172.21.15.140:0/466476199 >> v1:172.21.15.152:6789/0 conn(0x7fc63c0b5920 legacy=0x7fc63c142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc6327fc700 1 -- 172.21.15.140:0/466476199 >> v1:172.21.15.140:6789/0 conn(0x7fc63c142c40 legacy=0x7fc63c13e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc6327fc700 1 -- 172.21.15.140:0/466476199 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c13f5d0 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc6327fc700 1 --2- 172.21.15.140:0/466476199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b4ee0 0x7fc63c13fa70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fc624002f70 tx=0x7fc624006df0).ready entity=mon.2 client_cookie=3ec10cad6d4b3652 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63a7fc700 1 -- 172.21.15.140:0/466476199 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc62400d5c0 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63a7fc700 1 -- 172.21.15.140:0/466476199 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc624000e90 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.312+0000 7fc63a7fc700 1 -- 172.21.15.140:0/466476199 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc62400d910 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.316+0000 7fc63bfff700 1 -- 172.21.15.140:0/466476199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b4ee0 msgr2=0x7fc63c13fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.316+0000 7fc63bfff700 1 --2- 172.21.15.140:0/466476199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b4ee0 0x7fc63c13fa70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fc624002f70 tx=0x7fc624006df0).stop 2022-07-21T01:55:24.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.316+0000 7fc63bfff700 1 -- 172.21.15.140:0/466476199 shutdown_connections 2022-07-21T01:55:24.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.316+0000 7fc63bfff700 1 --2- 172.21.15.140:0/466476199 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b4ee0 0x7fc63c13fa70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.316+0000 7fc63bfff700 1 -- 172.21.15.140:0/466476199 >> 172.21.15.140:0/466476199 conn(0x7fc63c0632b0 msgr2=0x7fc63c063690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:24.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.316+0000 7fc63bfff700 1 -- 172.21.15.140:0/466476199 shutdown_connections 2022-07-21T01:55:24.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.316+0000 7fc63bfff700 1 -- 172.21.15.140:0/466476199 wait complete. 2022-07-21T01:55:24.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.316+0000 7fc63bfff700 1 Processor -- start 2022-07-21T01:55:24.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.316+0000 7fc63bfff700 1 -- start start 2022-07-21T01:55:24.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc63c0b4ee0 0x7fc63c0b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b5920 0x7fc63c145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc63c142c40 0x7fc63c1457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63bfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c14d860 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63bfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0ad770 con 0x7fc63c142c40 2022-07-21T01:55:24.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0b2cf0 con 0x7fc63c0b5920 2022-07-21T01:55:24.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc63c0b4ee0 0x7fc63c0b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc63c0b4ee0 0x7fc63c0b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37544/0 (socket says 172.21.15.140:37544) 2022-07-21T01:55:24.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63affd700 1 -- 172.21.15.140:0/1854744758 learned_addr learned my addr 172.21.15.140:0/1854744758 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:24.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc6327fc700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b5920 0x7fc63c145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63b7fe700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc63c142c40 0x7fc63c1457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63affd700 1 -- 172.21.15.140:0/1854744758 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b5920 msgr2=0x7fc63c145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63affd700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b5920 0x7fc63c145300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63affd700 1 -- 172.21.15.140:0/1854744758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc63c142c40 msgr2=0x7fc63c1457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63affd700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc63c142c40 0x7fc63c1457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63affd700 1 -- 172.21.15.140:0/1854744758 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62400b180 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63b7fe700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc63c142c40 0x7fc63c1457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:55:24.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc6327fc700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b5920 0x7fc63c145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:55:24.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63affd700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc63c0b4ee0 0x7fc63c0b11d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7fc62c001110 tx=0x7fc62c007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc638ff9700 1 -- 172.21.15.140:0/1854744758 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc62c00f470 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc638ff9700 1 -- 172.21.15.140:0/1854744758 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc62c00f5d0 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc638ff9700 1 -- 172.21.15.140:0/1854744758 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc62c00f930 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63bfff700 1 -- 172.21.15.140:0/1854744758 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63c0b2f90 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63bfff700 1 -- 172.21.15.140:0/1854744758 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63c0b28d0 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc638ff9700 1 -- 172.21.15.140:0/1854744758 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc62c019030 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc63bfff700 1 -- 172.21.15.140:0/1854744758 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63c09ba80 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc638ff9700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc620053aa0 0x7fc620055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc638ff9700 1 -- 172.21.15.140:0/1854744758 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc62c06b410 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc6327fc700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc620053aa0 0x7fc620055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.320+0000 7fc6327fc700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc620053aa0 0x7fc620055f60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fc624015000 tx=0x7fc624014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.324+0000 7fc638ff9700 1 -- 172.21.15.140:0/1854744758 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc62c042330 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 -- 172.21.15.140:0/1854744758 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc63c0574e0 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc638ff9700 1 -- 172.21.15.140:0/1854744758 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v84) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc62c045950 con 0x7fc63c0b4ee0 2022-07-21T01:55:24.452 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:24.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 -- 172.21.15.140:0/1854744758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc620053aa0 msgr2=0x7fc620055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc620053aa0 0x7fc620055f60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fc624015000 tx=0x7fc624014000).stop 2022-07-21T01:55:24.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 -- 172.21.15.140:0/1854744758 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc63c0b4ee0 msgr2=0x7fc63c0b11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc63c0b4ee0 0x7fc63c0b11d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7fc62c001110 tx=0x7fc62c007ae0).stop 2022-07-21T01:55:24.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 -- 172.21.15.140:0/1854744758 shutdown_connections 2022-07-21T01:55:24.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc620053aa0 0x7fc620055f60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc63c142c40 0x7fc63c1457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc63c0b5920 0x7fc63c145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 --2- 172.21.15.140:0/1854744758 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc63c0b4ee0 0x7fc63c0b11d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 -- 172.21.15.140:0/1854744758 >> 172.21.15.140:0/1854744758 conn(0x7fc63c0632b0 msgr2=0x7fc63c137f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:24.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 -- 172.21.15.140:0/1854744758 shutdown_connections 2022-07-21T01:55:24.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.452+0000 7fc63bfff700 1 -- 172.21.15.140:0/1854744758 wait complete. 2022-07-21T01:55:24.465 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:55:24.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.576+0000 7f4d90686700 1 Processor -- start 2022-07-21T01:55:24.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.576+0000 7f4d90686700 1 -- start start 2022-07-21T01:55:24.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.576+0000 7f4d90686700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4d880b5b50 0x7f4d880accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.576+0000 7f4d90686700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d8805c120 0x7f4d88144320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.576+0000 7f4d90686700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d880ac5e0 con 0x7f4d880b0480 2022-07-21T01:55:24.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.576+0000 7f4d90686700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d880b5f30 con 0x7f4d880b5b50 2022-07-21T01:55:24.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.576+0000 7f4d90686700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8805cf70 con 0x7f4d8805c120 2022-07-21T01:55:24.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.576+0000 7f4d8e422700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4d880b5b50 0x7f4d880accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.576+0000 7f4d8e422700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4d880b5b50 0x7f4d880accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37552/0 (socket says 172.21.15.140:37552) 2022-07-21T01:55:24.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.576+0000 7f4d8e422700 1 -- 172.21.15.140:0/1020734498 learned_addr learned my addr 172.21.15.140:0/1020734498 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:24.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.576+0000 7f4d8ec23700 1 --2- 172.21.15.140:0/1020734498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d8805c120 0x7f4d88144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d8e422700 1 -- 172.21.15.140:0/1020734498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d8805c120 msgr2=0x7f4d88144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:24.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d8e422700 1 --2- 172.21.15.140:0/1020734498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d8805c120 0x7f4d88144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d8e422700 1 -- 172.21.15.140:0/1020734498 >> v1:172.21.15.152:6789/0 conn(0x7f4d880b0480 legacy=0x7f4d8814a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d8e422700 1 -- 172.21.15.140:0/1020734498 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d880ac810 con 0x7f4d880b5b50 2022-07-21T01:55:24.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d8ec23700 1 --2- 172.21.15.140:0/1020734498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d8805c120 0x7f4d88144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:24.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d8e422700 1 --2- 172.21.15.140:0/1020734498 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d880b5b50 0x7f4d880accb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f4d84002f70 tx=0x7f4d84006eb0).ready entity=mon.0 client_cookie=8397f796cda1e787 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d8d420700 1 -- 172.21.15.140:0/1020734498 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d8400d5c0 con 0x7f4d880b5b50 2022-07-21T01:55:24.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d8d420700 1 -- 172.21.15.140:0/1020734498 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d84000e90 con 0x7f4d880b5b50 2022-07-21T01:55:24.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d8d420700 1 -- 172.21.15.140:0/1020734498 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d8400d950 con 0x7f4d880b5b50 2022-07-21T01:55:24.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d90686700 1 -- 172.21.15.140:0/1020734498 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d880b5b50 msgr2=0x7f4d880accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d90686700 1 --2- 172.21.15.140:0/1020734498 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d880b5b50 0x7f4d880accb0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f4d84002f70 tx=0x7f4d84006eb0).stop 2022-07-21T01:55:24.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d90686700 1 -- 172.21.15.140:0/1020734498 shutdown_connections 2022-07-21T01:55:24.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d90686700 1 --2- 172.21.15.140:0/1020734498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d8805c120 0x7f4d88144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d90686700 1 --2- 172.21.15.140:0/1020734498 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d880b5b50 0x7f4d880accb0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d90686700 1 -- 172.21.15.140:0/1020734498 >> 172.21.15.140:0/1020734498 conn(0x7f4d880635e0 msgr2=0x7f4d880b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:24.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d90686700 1 -- 172.21.15.140:0/1020734498 shutdown_connections 2022-07-21T01:55:24.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d90686700 1 -- 172.21.15.140:0/1020734498 wait complete. 2022-07-21T01:55:24.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d90686700 1 Processor -- start 2022-07-21T01:55:24.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.580+0000 7f4d90686700 1 -- start start 2022-07-21T01:55:24.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d90686700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d8805c120 0x7f4d88159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d90686700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d880b0480 0x7f4d88148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d90686700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d880b5b50 0x7f4d881569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d90686700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8809e520 con 0x7f4d8805c120 2022-07-21T01:55:24.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d90686700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8809e7c0 con 0x7f4d880b0480 2022-07-21T01:55:24.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d90686700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d88148c00 con 0x7f4d880b5b50 2022-07-21T01:55:24.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8e422700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d8805c120 0x7f4d88159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8e422700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d8805c120 0x7f4d88159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37558/0 (socket says 172.21.15.140:37558) 2022-07-21T01:55:24.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8e422700 1 -- 172.21.15.140:0/3361290578 learned_addr learned my addr 172.21.15.140:0/3361290578 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:24.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8ec23700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d880b5b50 0x7f4d881569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8dc21700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d880b0480 0x7f4d88148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8e422700 1 -- 172.21.15.140:0/3361290578 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d880b5b50 msgr2=0x7f4d881569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8e422700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d880b5b50 0x7f4d881569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8e422700 1 -- 172.21.15.140:0/3361290578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d880b0480 msgr2=0x7f4d88148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8e422700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d880b0480 0x7f4d88148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8e422700 1 -- 172.21.15.140:0/3361290578 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8400b1e0 con 0x7f4d8805c120 2022-07-21T01:55:24.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8ec23700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d880b5b50 0x7f4d881569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:24.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8dc21700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d880b0480 0x7f4d88148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:24.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d8e422700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d8805c120 0x7f4d88159ab0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f4d84002f70 tx=0x7f4d84015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d7b7fe700 1 -- 172.21.15.140:0/3361290578 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d84011950 con 0x7f4d8805c120 2022-07-21T01:55:24.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d7b7fe700 1 -- 172.21.15.140:0/3361290578 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d84011ab0 con 0x7f4d8805c120 2022-07-21T01:55:24.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d7b7fe700 1 -- 172.21.15.140:0/3361290578 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d84011e10 con 0x7f4d8805c120 2022-07-21T01:55:24.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d90686700 1 -- 172.21.15.140:0/3361290578 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8809e210 con 0x7f4d8805c120 2022-07-21T01:55:24.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d90686700 1 -- 172.21.15.140:0/3361290578 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d880a1fb0 con 0x7f4d8805c120 2022-07-21T01:55:24.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d90686700 1 -- 172.21.15.140:0/3361290578 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8809f460 con 0x7f4d8805c120 2022-07-21T01:55:24.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.584+0000 7f4d7b7fe700 1 -- 172.21.15.140:0/3361290578 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4d84025030 con 0x7f4d8805c120 2022-07-21T01:55:24.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.588+0000 7f4d7b7fe700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d68053950 0x7f4d68055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.588+0000 7f4d7b7fe700 1 -- 172.21.15.140:0/3361290578 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4d8406d310 con 0x7f4d8805c120 2022-07-21T01:55:24.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.588+0000 7f4d8dc21700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d68053950 0x7f4d68055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.588+0000 7f4d8dc21700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d68053950 0x7f4d68055e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f4d7c00b000 tx=0x7f4d7c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.596+0000 7f4d7b7fe700 1 -- 172.21.15.140:0/3361290578 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4d8809f460 con 0x7f4d8805c120 2022-07-21T01:55:24.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 -- 172.21.15.140:0/3361290578 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d8809f460 con 0x7f4d8805c120 2022-07-21T01:55:24.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d7b7fe700 1 -- 172.21.15.140:0/3361290578 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4d84023020 con 0x7f4d8805c120 2022-07-21T01:55:24.719 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:24.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 -- 172.21.15.140:0/3361290578 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d68053950 msgr2=0x7f4d68055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d68053950 0x7f4d68055e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f4d7c00b000 tx=0x7f4d7c009040).stop 2022-07-21T01:55:24.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 -- 172.21.15.140:0/3361290578 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d8805c120 msgr2=0x7f4d88159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d8805c120 0x7f4d88159ab0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f4d84002f70 tx=0x7f4d84015000).stop 2022-07-21T01:55:24.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 -- 172.21.15.140:0/3361290578 shutdown_connections 2022-07-21T01:55:24.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d68053950 0x7f4d68055e10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d880b5b50 0x7f4d881569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d880b0480 0x7f4d88148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 --2- 172.21.15.140:0/3361290578 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d8805c120 0x7f4d88159ab0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 -- 172.21.15.140:0/3361290578 >> 172.21.15.140:0/3361290578 conn(0x7f4d880635e0 msgr2=0x7f4d8814a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:24.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 -- 172.21.15.140:0/3361290578 shutdown_connections 2022-07-21T01:55:24.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.720+0000 7f4d90686700 1 -- 172.21.15.140:0/3361290578 wait complete. 2022-07-21T01:55:24.732 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:55:24.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.844+0000 7f6c02ad5700 1 Processor -- start 2022-07-21T01:55:24.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.844+0000 7f6c02ad5700 1 -- start start 2022-07-21T01:55:24.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6bfc0b4b10 0x7f6bfc0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6bfc0b0350 0x7f6bfc13e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bfc0aabe0 con 0x7f6bfc0b0890 2022-07-21T01:55:24.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc0b4ef0 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0aae10 con 0x7f6bfc0b0350 2022-07-21T01:55:24.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c01ad3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6bfc0b4b10 0x7f6bfc0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c01ad3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6bfc0b4b10 0x7f6bfc0af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37566/0 (socket says 172.21.15.140:37566) 2022-07-21T01:55:24.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c01ad3700 1 -- 172.21.15.140:0/3144992516 learned_addr learned my addr 172.21.15.140:0/3144992516 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:24.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c022d4700 1 --2- 172.21.15.140:0/3144992516 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6bfc0b0350 0x7f6bfc13e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c01ad3700 1 -- 172.21.15.140:0/3144992516 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6bfc0b0350 msgr2=0x7f6bfc13e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:24.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c01ad3700 1 --2- 172.21.15.140:0/3144992516 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6bfc0b0350 0x7f6bfc13e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c01ad3700 1 -- 172.21.15.140:0/3144992516 >> v1:172.21.15.140:6789/0 conn(0x7f6bfc0b0890 legacy=0x7f6bfc0ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c01ad3700 1 -- 172.21.15.140:0/3144992516 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0ab0b0 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c00ad1700 1 -- 172.21.15.140:0/3144992516 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799355310 0 0) 0x7f6bfc0aabe0 con 0x7f6bfc0b0890 2022-07-21T01:55:24.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c022d4700 1 --2- 172.21.15.140:0/3144992516 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6bfc0b0350 0x7f6bfc13e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:55:24.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c01ad3700 1 --2- 172.21.15.140:0/3144992516 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6bfc0b4b10 0x7f6bfc0af8b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f6bf8002f70 tx=0x7f6bf8006f40).ready entity=mon.0 client_cookie=c54e53f01d9cf83 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c00ad1700 1 -- 172.21.15.140:0/3144992516 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf800dc70 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c00ad1700 1 -- 172.21.15.140:0/3144992516 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf8000e90 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c00ad1700 1 -- 172.21.15.140:0/3144992516 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf800be90 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 -- 172.21.15.140:0/3144992516 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6bfc0b4b10 msgr2=0x7f6bfc0af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 --2- 172.21.15.140:0/3144992516 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6bfc0b4b10 0x7f6bfc0af8b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f6bf8002f70 tx=0x7f6bf8006f40).stop 2022-07-21T01:55:24.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 -- 172.21.15.140:0/3144992516 shutdown_connections 2022-07-21T01:55:24.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 --2- 172.21.15.140:0/3144992516 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6bfc0b0350 0x7f6bfc13e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 --2- 172.21.15.140:0/3144992516 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6bfc0b4b10 0x7f6bfc0af8b0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 -- 172.21.15.140:0/3144992516 >> 172.21.15.140:0/3144992516 conn(0x7f6bfc0632b0 msgr2=0x7f6bfc063690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:24.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 -- 172.21.15.140:0/3144992516 shutdown_connections 2022-07-21T01:55:24.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.848+0000 7f6c02ad5700 1 -- 172.21.15.140:0/3144992516 wait complete. 2022-07-21T01:55:24.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c02ad5700 1 Processor -- start 2022-07-21T01:55:24.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c02ad5700 1 -- start start 2022-07-21T01:55:24.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c02ad5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bfc0b0350 0x7f6bfc0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c02ad5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6bfc0b0890 0x7f6bfc13a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c02ad5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6bfc0b4b10 0x7f6bfc0add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c02ad5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc13a7a0 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c02ad5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc142c30 con 0x7f6bfc0b0350 2022-07-21T01:55:24.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c02ad5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc142990 con 0x7f6bfc0b0890 2022-07-21T01:55:24.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c022d4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6bfc0b4b10 0x7f6bfc0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c022d4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6bfc0b4b10 0x7f6bfc0add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37576/0 (socket says 172.21.15.140:37576) 2022-07-21T01:55:24.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c022d4700 1 -- 172.21.15.140:0/40948991 learned_addr learned my addr 172.21.15.140:0/40948991 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:24.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c01ad3700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bfc0b0350 0x7f6bfc0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c012d2700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6bfc0b0890 0x7f6bfc13a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c022d4700 1 -- 172.21.15.140:0/40948991 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6bfc0b0890 msgr2=0x7f6bfc13a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c022d4700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6bfc0b0890 0x7f6bfc13a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c022d4700 1 -- 172.21.15.140:0/40948991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bfc0b0350 msgr2=0x7f6bfc0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c022d4700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bfc0b0350 0x7f6bfc0b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c022d4700 1 -- 172.21.15.140:0/40948991 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf800b460 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c022d4700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6bfc0b4b10 0x7f6bfc0add00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f6bf4007000 tx=0x7f6bf4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6beaffd700 1 -- 172.21.15.140:0/40948991 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf4015070 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6beaffd700 1 -- 172.21.15.140:0/40948991 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf4010580 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c02ad5700 1 -- 172.21.15.140:0/40948991 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc13dce0 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6beaffd700 1 -- 172.21.15.140:0/40948991 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6bf40108e0 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.852+0000 7f6c02ad5700 1 -- 172.21.15.140:0/40948991 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc13ae30 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.856+0000 7f6c02ad5700 1 -- 172.21.15.140:0/40948991 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc13dea0 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.856+0000 7f6beaffd700 1 -- 172.21.15.140:0/40948991 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6bf4010a40 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.856+0000 7f6beaffd700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6bdc053a10 0x7f6bdc055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:24.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.856+0000 7f6beaffd700 1 -- 172.21.15.140:0/40948991 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6bf406c0f0 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.856+0000 7f6c01ad3700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6bdc053a10 0x7f6bdc055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:24.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.856+0000 7f6c01ad3700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6bdc053a10 0x7f6bdc055ed0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f6bf801b000 tx=0x7f6bfc142e00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:24.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.864+0000 7f6beaffd700 1 -- 172.21.15.140:0/40948991 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6bf4046e60 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.988+0000 7f6c02ad5700 1 -- 172.21.15.140:0/40948991 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6bfc0574e0 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.992+0000 7f6beaffd700 1 -- 172.21.15.140:0/40948991 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6bf4042f70 con 0x7f6bfc0b4b10 2022-07-21T01:55:24.991 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:24.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 -- 172.21.15.140:0/40948991 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6bdc053a10 msgr2=0x7f6bdc055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6bdc053a10 0x7f6bdc055ed0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f6bf801b000 tx=0x7f6bfc142e00).stop 2022-07-21T01:55:24.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 -- 172.21.15.140:0/40948991 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6bfc0b4b10 msgr2=0x7f6bfc0add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:24.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6bfc0b4b10 0x7f6bfc0add00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f6bf4007000 tx=0x7f6bf4008040).stop 2022-07-21T01:55:24.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 -- 172.21.15.140:0/40948991 shutdown_connections 2022-07-21T01:55:24.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6bdc053a10 0x7f6bdc055ed0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6bfc0b4b10 0x7f6bfc0add00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6bfc0b0890 0x7f6bfc13a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 --2- 172.21.15.140:0/40948991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bfc0b0350 0x7f6bfc0b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:24.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 -- 172.21.15.140:0/40948991 >> 172.21.15.140:0/40948991 conn(0x7f6bfc0632b0 msgr2=0x7f6bfc13e700 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:24.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 -- 172.21.15.140:0/40948991 shutdown_connections 2022-07-21T01:55:24.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:24.996+0000 7f6c02ad5700 1 -- 172.21.15.140:0/40948991 wait complete. 2022-07-21T01:55:25.010 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:55:25.011 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:25.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.112+0000 7f1591b1c700 1 Processor -- start 2022-07-21T01:55:25.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.112+0000 7f1591b1c700 1 -- start start 2022-07-21T01:55:25.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1591b1c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f158c0b5650 0x7f158c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:25.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1591b1c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c147010 con 0x7f158c0adff0 2022-07-21T01:55:25.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1591b1c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158c1472b0 con 0x7f158c0af8e0 2022-07-21T01:55:25.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1591b1c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158c146d40 con 0x7f158c0b5650 2022-07-21T01:55:25.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f158c0adff0 0x7f158c147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51726/0 (socket says 172.21.15.140:51726) 2022-07-21T01:55:25.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158bfff700 1 -- 172.21.15.140:0/666404473 learned_addr learned my addr 172.21.15.140:0/666404473 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:25.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1590b1a700 1 --2- 172.21.15.140:0/666404473 >> v2:172.21.15.152:3300/0 conn(0x7f158c0b5650 0x7f158c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:25.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927923944 0 0) 0x7f158c147010 con 0x7f158c0adff0 2022-07-21T01:55:25.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1570003260 con 0x7f158c0adff0 2022-07-21T01:55:25.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3748346957 0 0) 0x7f1570003260 con 0x7f158c0adff0 2022-07-21T01:55:25.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 >> v2:172.21.15.152:3300/0 conn(0x7f158c0b5650 msgr2=0x7f158c0b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:25.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 --2- 172.21.15.140:0/666404473 >> v2:172.21.15.152:3300/0 conn(0x7f158c0b5650 0x7f158c0b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:25.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 >> v1:172.21.15.152:6789/0 conn(0x7f158c0af8e0 legacy=0x7f158c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:25.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c05c4a0 con 0x7f158c0adff0 2022-07-21T01:55:25.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f15740017e0 con 0x7f158c0adff0 2022-07-21T01:55:25.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 >> v1:172.21.15.140:6789/0 conn(0x7f158c0adff0 legacy=0x7f158c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:25.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 --2- 172.21.15.140:0/666404473 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1570003750 0x7f1570003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:25.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 --2- 172.21.15.140:0/666404473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1570004500 0x7f1570004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:25.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 --2- 172.21.15.140:0/666404473 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1570004e70 0x7f157000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:25.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c146d40 con 0x7f1570003750 2022-07-21T01:55:25.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157000bdd0 con 0x7f1570004500 2022-07-21T01:55:25.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157000bf60 con 0x7f1570004e70 2022-07-21T01:55:25.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158bfff700 1 --2- 172.21.15.140:0/666404473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1570004500 0x7f1570004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:25.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1590b1a700 1 --2- 172.21.15.140:0/666404473 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1570003750 0x7f1570003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:25.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f159131b700 1 --2- 172.21.15.140:0/666404473 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1570004e70 0x7f157000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:25.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158bfff700 1 -- 172.21.15.140:0/666404473 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1570004e70 msgr2=0x7f157000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:25.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158bfff700 1 --2- 172.21.15.140:0/666404473 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1570004e70 0x7f157000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:25.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158bfff700 1 -- 172.21.15.140:0/666404473 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1570003750 msgr2=0x7f1570003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:25.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158bfff700 1 --2- 172.21.15.140:0/666404473 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1570003750 0x7f1570003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:25.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158bfff700 1 -- 172.21.15.140:0/666404473 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570003260 con 0x7f1570004500 2022-07-21T01:55:25.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f159131b700 1 -- 172.21.15.140:0/666404473 reap_dead start 2022-07-21T01:55:25.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158bfff700 1 --2- 172.21.15.140:0/666404473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1570004500 0x7f1570004980 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f15740014f0 tx=0x7f1574007590).ready entity=mon.1 client_cookie=541c8671690c4e2f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:25.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15740070d0 con 0x7f1570004500 2022-07-21T01:55:25.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1574007230 con 0x7f1570004500 2022-07-21T01:55:25.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f158b7fe700 1 -- 172.21.15.140:0/666404473 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15740073a0 con 0x7f1570004500 2022-07-21T01:55:25.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1591b1c700 1 -- 172.21.15.140:0/666404473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1570004500 msgr2=0x7f1570004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:25.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1591b1c700 1 --2- 172.21.15.140:0/666404473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1570004500 0x7f1570004980 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f15740014f0 tx=0x7f1574007590).stop 2022-07-21T01:55:25.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1591b1c700 1 -- 172.21.15.140:0/666404473 shutdown_connections 2022-07-21T01:55:25.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1591b1c700 1 --2- 172.21.15.140:0/666404473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1570004500 0x7f1570004980 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:25.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1591b1c700 1 -- 172.21.15.140:0/666404473 >> 172.21.15.140:0/666404473 conn(0x7f158c0634e0 msgr2=0x7f158c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:25.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1591b1c700 1 -- 172.21.15.140:0/666404473 shutdown_connections 2022-07-21T01:55:25.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.116+0000 7f1591b1c700 1 -- 172.21.15.140:0/666404473 wait complete. 2022-07-21T01:55:25.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f1591b1c700 1 Processor -- start 2022-07-21T01:55:25.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f1591b1c700 1 -- start start 2022-07-21T01:55:25.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f1591b1c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1570004500 0x7f158c11dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:25.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f1591b1c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f158c11e140 0x7f158c11ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:25.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f1591b1c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f158c11b240 0x7f158c11b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:25.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f1591b1c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c11e5a0 con 0x7f158c11e140 2022-07-21T01:55:25.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f1591b1c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158c11bbe0 con 0x7f158c11b240 2022-07-21T01:55:25.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f1591b1c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158c15a590 con 0x7f1570004500 2022-07-21T01:55:25.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f158bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f158c11e140 0x7f158c11ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:25.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f158bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f158c11e140 0x7f158c11ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37594/0 (socket says 172.21.15.140:37594) 2022-07-21T01:55:25.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f158bfff700 1 -- 172.21.15.140:0/3093834173 learned_addr learned my addr 172.21.15.140:0/3093834173 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:25.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f1590b1a700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1570004500 0x7f158c11dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:25.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f159131b700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f158c11b240 0x7f158c11b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:25.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f158bfff700 1 -- 172.21.15.140:0/3093834173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1570004500 msgr2=0x7f158c11dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:25.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f158bfff700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1570004500 0x7f158c11dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:25.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f158bfff700 1 -- 172.21.15.140:0/3093834173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f158c11b240 msgr2=0x7f158c11b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:25.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f158bfff700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f158c11b240 0x7f158c11b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:25.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.120+0000 7f158bfff700 1 -- 172.21.15.140:0/3093834173 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c066e80 con 0x7f158c11e140 2022-07-21T01:55:25.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f158bfff700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f158c11e140 0x7f158c11ad50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f15740014f0 tx=0x7f1574015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:25.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f1589ffb700 1 -- 172.21.15.140:0/3093834173 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15740070d0 con 0x7f158c11e140 2022-07-21T01:55:25.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f1589ffb700 1 -- 172.21.15.140:0/3093834173 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15740077a0 con 0x7f158c11e140 2022-07-21T01:55:25.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f1591b1c700 1 -- 172.21.15.140:0/3093834173 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158c067320 con 0x7f158c11e140 2022-07-21T01:55:25.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f1589ffb700 1 -- 172.21.15.140:0/3093834173 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f15740056d0 con 0x7f158c11e140 2022-07-21T01:55:25.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f1591b1c700 1 -- 172.21.15.140:0/3093834173 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158c067710 con 0x7f158c11e140 2022-07-21T01:55:25.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f1591b1c700 1 -- 172.21.15.140:0/3093834173 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158c15a3f0 con 0x7f158c11e140 2022-07-21T01:55:25.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f1589ffb700 1 -- 172.21.15.140:0/3093834173 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1574026070 con 0x7f158c11e140 2022-07-21T01:55:25.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f1589ffb700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1564053a70 0x7f1564055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:25.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f1589ffb700 1 -- 172.21.15.140:0/3093834173 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f157406c270 con 0x7f158c11e140 2022-07-21T01:55:25.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f1590b1a700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1564053a70 0x7f1564055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:25.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.124+0000 7f1590b1a700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1564053a70 0x7f1564055f30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f157c00b000 tx=0x7f157c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:25.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.132+0000 7f1589ffb700 1 -- 172.21.15.140:0/3093834173 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f158c15a3f0 con 0x7f158c11e140 2022-07-21T01:55:25.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.292+0000 7f1591b1c700 1 -- 172.21.15.140:0/3093834173 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f158c15a3f0 con 0x7f158c11e140 2022-07-21T01:55:25.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.292+0000 7f1589ffb700 1 -- 172.21.15.140:0/3093834173 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2146 (secure 0 0 0) 0x7f158c15a3f0 con 0x7f158c11e140 2022-07-21T01:55:25.295 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:25.295 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":36,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/2489766347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2489766347},{"type":"v1","addr":"172.21.15.152:6833","nonce":2489766347}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:20.506956+0000","modified":"2022-07-21T01:55:21.509498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6586":{"gid":6586,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2461699636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2461699636},{"type":"v1","addr":"172.21.15.140:6835","nonce":2461699636}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-07-21T01:55:25.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 -- 172.21.15.140:0/3093834173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1564053a70 msgr2=0x7f1564055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:25.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1564053a70 0x7f1564055f30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f157c00b000 tx=0x7f157c006df0).stop 2022-07-21T01:55:25.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 -- 172.21.15.140:0/3093834173 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f158c11e140 msgr2=0x7f158c11ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:25.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f158c11e140 0x7f158c11ad50 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f15740014f0 tx=0x7f1574015000).stop 2022-07-21T01:55:25.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 -- 172.21.15.140:0/3093834173 shutdown_connections 2022-07-21T01:55:25.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1564053a70 0x7f1564055f30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:25.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f158c11b240 0x7f158c11b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:25.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f158c11e140 0x7f158c11ad50 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:25.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 --2- 172.21.15.140:0/3093834173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1570004500 0x7f158c11dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:25.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 -- 172.21.15.140:0/3093834173 >> 172.21.15.140:0/3093834173 conn(0x7f158c0634e0 msgr2=0x7f158c0ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:25.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 -- 172.21.15.140:0/3093834173 shutdown_connections 2022-07-21T01:55:25.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:25.296+0000 7f1591b1c700 1 -- 172.21.15.140:0/3093834173 wait complete. 2022-07-21T01:55:25.305 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 36 2022-07-21T01:55:25.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:55:20.506956+0000', 'modified': '2022-07-21T01:55:21.509498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6586}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6586': {'gid': 6586, 'name': 'a', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/2461699636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 2461699636}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 2461699636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:55:25.510 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:55:25.511 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:55:25.625 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":6586,"want_state":"up:active","state":"up:active","rank_uptime":5.1073828910000003,"mdsmap_epoch":36,"osdmap_epoch":84,"osdmap_epoch_barrier":83,"uptime":9.8125170619999995} 2022-07-21T01:55:25.626 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 6586, "mdsmap_epoch": 36, "osdmap_epoch": 84, "osdmap_epoch_barrier": 83, "rank_uptime": 5.107382891, "state": "up:active", "uptime": 9.812517062, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:55:25.627 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:55:25.627 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:55:25.628 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:55:25.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:25.628 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:25.637 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:55:25.695 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:25.696 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:25.706 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:25.707 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:25.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:25.708 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:55:25.714 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:55:25.715 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:25.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:55:25.733 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:25.734 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:25.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:55:25.790+0000 7fc24feec080 -1 init, newargv = 0x55d6550d5210 newargc=9 2022-07-21T01:55:25.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22958]: starting ceph client 2022-07-21T01:55:25.793 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:25.795 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:25.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:25.797 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:55:25.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22958]: starting fuse 2022-07-21T01:55:25.806 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:55:25.806 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:55:25.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:25.807 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:55:25.807 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:55:25.808 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:55:25.808 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:55:25.809 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:55:25.809 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:25.810 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:55:25.810 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:55:25.811 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:55:25.811 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:25.812 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:55:25.812 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:55:25.812 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:55:25.813 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:55:25.813 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:25.814 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:55:25.814 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:55:25.815 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:55:25.815 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:55:25.816 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:55:25.816 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:25.817 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:55:25.817 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:55:25.892 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.22958.asok 2022-07-21T01:55:25.897 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.22958.asok 2022-07-21T01:55:25.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:25.898 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.22958.asok status 2022-07-21T01:55:25.985 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:55:25.986 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:55:25.987 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:55:25.987 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:55:25.988 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:55:25.988 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:55:25.988 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:55:25.988 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "22958", 2022-07-21T01:55:25.989 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:55:25.989 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:25.989 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:55:25.989 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:55:25.990 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 6113, 2022-07-21T01:55:25.990 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:55:25.990 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:55:25.990 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:55:25.991 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 6113 2022-07-21T01:55:25.991 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:25.991 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:55:25.991 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:55:25.991 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:55:25.992 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 400764524 2022-07-21T01:55:25.992 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:55:25.992 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:25.993 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:55:25.993 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:55:25.993 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:55:25.994 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 400764524 2022-07-21T01:55:25.994 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:25.994 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.6113 172.21.15.191:0/400764524", 2022-07-21T01:55:25.994 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/400764524", 2022-07-21T01:55:25.995 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:55:25.995 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 36, 2022-07-21T01:55:25.995 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 84, 2022-07-21T01:55:25.995 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:55:25.996 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:55:25.996 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:55:25.997 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:55:25.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:25.999 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:25.999 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:26.006 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:55:26.008 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:55:26.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:26.009 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:26.069 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:26.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 Processor -- start 2022-07-21T01:55:26.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 -- start start 2022-07-21T01:55:26.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbb840b1e00 0x7fbb8414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb840acef0 0x7fbb84146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8405d480 con 0x7fbb8405c3f0 2022-07-21T01:55:26.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb840b21e0 con 0x7fbb840b1e00 2022-07-21T01:55:26.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8405cf40 con 0x7fbb840acef0 2022-07-21T01:55:26.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8944e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbb840b1e00 0x7fbb8414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb88c4d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fbb8405c3f0 0x7fbb840ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51748/0 (socket says 172.21.15.140:51748) 2022-07-21T01:55:26.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb88c4d700 1 -- 172.21.15.140:0/3106284006 learned_addr learned my addr 172.21.15.140:0/3106284006 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:26.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb89c4f700 1 --2- 172.21.15.140:0/3106284006 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb840acef0 0x7fbb84146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8944e700 1 -- 172.21.15.140:0/3106284006 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb840acef0 msgr2=0x7fbb84146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:26.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8944e700 1 --2- 172.21.15.140:0/3106284006 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb840acef0 0x7fbb84146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8944e700 1 -- 172.21.15.140:0/3106284006 >> v1:172.21.15.140:6789/0 conn(0x7fbb8405c3f0 legacy=0x7fbb840ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8944e700 1 -- 172.21.15.140:0/3106284006 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8405d1e0 con 0x7fbb840b1e00 2022-07-21T01:55:26.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8944e700 1 --2- 172.21.15.140:0/3106284006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbb840b1e00 0x7fbb8414a6d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7fbb80002f70 tx=0x7fbb80006df0).ready entity=mon.0 client_cookie=a06ef68f28412417 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb7bfff700 1 -- 172.21.15.140:0/3106284006 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8000d570 con 0x7fbb840b1e00 2022-07-21T01:55:26.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb7bfff700 1 -- 172.21.15.140:0/3106284006 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb80000e90 con 0x7fbb840b1e00 2022-07-21T01:55:26.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb7bfff700 1 -- 172.21.15.140:0/3106284006 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8000d8c0 con 0x7fbb840b1e00 2022-07-21T01:55:26.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3106284006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbb840b1e00 msgr2=0x7fbb8414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 --2- 172.21.15.140:0/3106284006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbb840b1e00 0x7fbb8414a6d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7fbb80002f70 tx=0x7fbb80006df0).stop 2022-07-21T01:55:26.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3106284006 shutdown_connections 2022-07-21T01:55:26.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 --2- 172.21.15.140:0/3106284006 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb840acef0 0x7fbb84146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 --2- 172.21.15.140:0/3106284006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbb840b1e00 0x7fbb8414a6d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3106284006 >> 172.21.15.140:0/3106284006 conn(0x7fbb840611c0 msgr2=0x7fbb84062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:26.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3106284006 shutdown_connections 2022-07-21T01:55:26.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.180+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3106284006 wait complete. 2022-07-21T01:55:26.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8b6b2700 1 Processor -- start 2022-07-21T01:55:26.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8b6b2700 1 -- start start 2022-07-21T01:55:26.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8b6b2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbb840acef0 0x7fbb840674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8b6b2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb840b1e00 0x7fbb84063a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8b6b2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb84063f70 0x7fbb8409e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8b6b2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb84156bb0 con 0x7fbb840acef0 2022-07-21T01:55:26.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8b6b2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb84156e50 con 0x7fbb84063f70 2022-07-21T01:55:26.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8b6b2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb841570f0 con 0x7fbb840b1e00 2022-07-21T01:55:26.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8944e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbb840acef0 0x7fbb840674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8944e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbb840acef0 0x7fbb840674e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37606/0 (socket says 172.21.15.140:37606) 2022-07-21T01:55:26.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8944e700 1 -- 172.21.15.140:0/3541794517 learned_addr learned my addr 172.21.15.140:0/3541794517 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:26.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb88c4d700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb840b1e00 0x7fbb84063a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb89c4f700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb84063f70 0x7fbb8409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8944e700 1 -- 172.21.15.140:0/3541794517 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb840b1e00 msgr2=0x7fbb84063a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8944e700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb840b1e00 0x7fbb84063a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8944e700 1 -- 172.21.15.140:0/3541794517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb84063f70 msgr2=0x7fbb8409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8944e700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb84063f70 0x7fbb8409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8944e700 1 -- 172.21.15.140:0/3541794517 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8000b180 con 0x7fbb840acef0 2022-07-21T01:55:26.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8944e700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbb840acef0 0x7fbb840674e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7fbb80002f70 tx=0x7fbb80015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb7a7fc700 1 -- 172.21.15.140:0/3541794517 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8000d910 con 0x7fbb840acef0 2022-07-21T01:55:26.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb7a7fc700 1 -- 172.21.15.140:0/3541794517 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb80011b80 con 0x7fbb840acef0 2022-07-21T01:55:26.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3541794517 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8414a120 con 0x7fbb840acef0 2022-07-21T01:55:26.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb7a7fc700 1 -- 172.21.15.140:0/3541794517 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8001e460 con 0x7fbb840acef0 2022-07-21T01:55:26.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.184+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3541794517 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb841566a0 con 0x7fbb840acef0 2022-07-21T01:55:26.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.188+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3541794517 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb841572b0 con 0x7fbb840acef0 2022-07-21T01:55:26.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.188+0000 7fbb7a7fc700 1 -- 172.21.15.140:0/3541794517 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbb8001a040 con 0x7fbb840acef0 2022-07-21T01:55:26.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.188+0000 7fbb7a7fc700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb6c053b30 0x7fbb6c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.188+0000 7fbb7a7fc700 1 -- 172.21.15.140:0/3541794517 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb8006cd10 con 0x7fbb840acef0 2022-07-21T01:55:26.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.188+0000 7fbb88c4d700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb6c053b30 0x7fbb6c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.188+0000 7fbb88c4d700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb6c053b30 0x7fbb6c055ff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fbb7400d000 tx=0x7fbb74009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.196+0000 7fbb7a7fc700 1 -- 172.21.15.140:0/3541794517 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbb80072020 con 0x7fbb840acef0 2022-07-21T01:55:26.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.320+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3541794517 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb840b04b0 con 0x7fbb840acef0 2022-07-21T01:55:26.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb7a7fc700 1 -- 172.21.15.140:0/3541794517 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16137 (secure 0 0 0) 0x7fbb80023090 con 0x7fbb840acef0 2022-07-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:26.323 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":84,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:21.510210+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:55:18.595416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2022-07-21T01:55:19.516058+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:26.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3541794517 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb6c053b30 msgr2=0x7fbb6c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb6c053b30 0x7fbb6c055ff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fbb7400d000 tx=0x7fbb74009040).stop 2022-07-21T01:55:26.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3541794517 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbb840acef0 msgr2=0x7fbb840674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbb840acef0 0x7fbb840674e0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7fbb80002f70 tx=0x7fbb80015000).stop 2022-07-21T01:55:26.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3541794517 shutdown_connections 2022-07-21T01:55:26.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb6c053b30 0x7fbb6c055ff0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb84063f70 0x7fbb8409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb840b1e00 0x7fbb84063a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 --2- 172.21.15.140:0/3541794517 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbb840acef0 0x7fbb840674e0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3541794517 >> 172.21.15.140:0/3541794517 conn(0x7fbb8405c3f0 msgr2=0x7fbb840afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:26.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3541794517 shutdown_connections 2022-07-21T01:55:26.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.324+0000 7fbb8b6b2700 1 -- 172.21.15.140:0/3541794517 wait complete. 2022-07-21T01:55:26.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:55:18.595416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:55:19.516058+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:55:26.337 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:55:26.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.444+0000 7fe9612b3700 1 Processor -- start 2022-07-21T01:55:26.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe9612b3700 1 -- start start 2022-07-21T01:55:26.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe9612b3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b59f0 0x7fe95c1407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe9612b3700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c0b5dd0 con 0x7fe95c140cd0 2022-07-21T01:55:26.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe9612b3700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c0652a0 con 0x7fe95c0aef30 2022-07-21T01:55:26.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe9612b3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0b1750 con 0x7fe95c0b59f0 2022-07-21T01:55:26.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe95c140cd0 0x7fe95c1445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51762/0 (socket says 172.21.15.140:51762) 2022-07-21T01:55:26.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95b7fe700 1 -- 172.21.15.140:0/1447920216 learned_addr learned my addr 172.21.15.140:0/1447920216 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:26.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95bfff700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b59f0 0x7fe95c1407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159360990 0 0) 0x7fe95c0b5dd0 con 0x7fe95c140cd0 2022-07-21T01:55:26.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe93c003260 con 0x7fe95c140cd0 2022-07-21T01:55:26.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 756605528 0 0) 0x7fe93c003260 con 0x7fe95c140cd0 2022-07-21T01:55:26.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b59f0 msgr2=0x7fe95c1407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:26.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b59f0 0x7fe95c1407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 >> v1:172.21.15.152:6789/0 conn(0x7fe95c0aef30 legacy=0x7fe95c129340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c143e60 con 0x7fe95c140cd0 2022-07-21T01:55:26.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fe944001660 con 0x7fe95c140cd0 2022-07-21T01:55:26.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 >> v1:172.21.15.140:6789/0 conn(0x7fe95c140cd0 legacy=0x7fe95c1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 0x7fe93c003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b59f0 0x7fe93c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe93c0049f0 0x7fe93c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0b1750 con 0x7fe93c003780 2022-07-21T01:55:26.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c00b750 con 0x7fe93c0049f0 2022-07-21T01:55:26.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c00b8c0 con 0x7fe95c0b59f0 2022-07-21T01:55:26.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95bfff700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 0x7fe93c003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95b7fe700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b59f0 0x7fe93c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe960ab2700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe93c0049f0 0x7fe93c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95bfff700 1 -- 172.21.15.140:0/1447920216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b59f0 msgr2=0x7fe93c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:26.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95bfff700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b59f0 0x7fe93c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95bfff700 1 -- 172.21.15.140:0/1447920216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe93c0049f0 msgr2=0x7fe93c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:26.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95bfff700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe93c0049f0 0x7fe93c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.448+0000 7fe95bfff700 1 -- 172.21.15.140:0/1447920216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c003260 con 0x7fe93c003780 2022-07-21T01:55:26.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe95bfff700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 0x7fe93c003b60 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fe94c00c000 tx=0x7fe94c006df0).ready entity=mon.0 client_cookie=ee853b081fbbef1c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94c00e7d0 con 0x7fe93c003780 2022-07-21T01:55:26.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c00e930 con 0x7fe93c003780 2022-07-21T01:55:26.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe95affd700 1 -- 172.21.15.140:0/1447920216 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94c00ec90 con 0x7fe93c003780 2022-07-21T01:55:26.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe9612b3700 1 -- 172.21.15.140:0/1447920216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 msgr2=0x7fe93c003b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe9612b3700 1 --2- 172.21.15.140:0/1447920216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 0x7fe93c003b60 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fe94c00c000 tx=0x7fe94c006df0).stop 2022-07-21T01:55:26.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe960ab2700 1 -- 172.21.15.140:0/1447920216 reap_dead start 2022-07-21T01:55:26.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe9612b3700 1 -- 172.21.15.140:0/1447920216 shutdown_connections 2022-07-21T01:55:26.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe9612b3700 1 -- 172.21.15.140:0/1447920216 >> 172.21.15.140:0/1447920216 conn(0x7fe95c063480 msgr2=0x7fe95c0b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:26.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe9612b3700 1 -- 172.21.15.140:0/1447920216 shutdown_connections 2022-07-21T01:55:26.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe9612b3700 1 -- 172.21.15.140:0/1447920216 wait complete. 2022-07-21T01:55:26.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe9612b3700 1 Processor -- start 2022-07-21T01:55:26.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.452+0000 7fe9612b3700 1 -- start start 2022-07-21T01:55:26.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe9612b3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 0x7fe95c143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe9612b3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b4f50 0x7fe95c151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe9612b3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe95c151b70 0x7fe95c0b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe9612b3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c12d8d0 con 0x7fe93c003780 2022-07-21T01:55:26.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe9612b3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c151220 con 0x7fe95c151b70 2022-07-21T01:55:26.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe9612b3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c151ff0 con 0x7fe95c0b4f50 2022-07-21T01:55:26.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe95bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 0x7fe95c143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe95bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 0x7fe95c143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37628/0 (socket says 172.21.15.140:37628) 2022-07-21T01:55:26.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe95bfff700 1 -- 172.21.15.140:0/3178367665 learned_addr learned my addr 172.21.15.140:0/3178367665 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:26.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe95b7fe700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b4f50 0x7fe95c151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe960ab2700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe95c151b70 0x7fe95c0b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe95bfff700 1 -- 172.21.15.140:0/3178367665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b4f50 msgr2=0x7fe95c151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe95bfff700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b4f50 0x7fe95c151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe95bfff700 1 -- 172.21.15.140:0/3178367665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe95c151b70 msgr2=0x7fe95c0b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe95bfff700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe95c151b70 0x7fe95c0b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe95bfff700 1 -- 172.21.15.140:0/3178367665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c12d630 con 0x7fe93c003780 2022-07-21T01:55:26.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe95b7fe700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b4f50 0x7fe95c151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:26.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe960ab2700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe95c151b70 0x7fe95c0b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:26.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe95bfff700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 0x7fe95c143480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fe94c00c000 tx=0x7fe94c019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe9597fa700 1 -- 172.21.15.140:0/3178367665 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94c01b820 con 0x7fe93c003780 2022-07-21T01:55:26.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe9597fa700 1 -- 172.21.15.140:0/3178367665 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c000e90 con 0x7fe93c003780 2022-07-21T01:55:26.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe9612b3700 1 -- 172.21.15.140:0/3178367665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c0b4000 con 0x7fe93c003780 2022-07-21T01:55:26.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe9597fa700 1 -- 172.21.15.140:0/3178367665 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe94c01bb70 con 0x7fe93c003780 2022-07-21T01:55:26.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.456+0000 7fe9612b3700 1 -- 172.21.15.140:0/3178367665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c0b4cf0 con 0x7fe93c003780 2022-07-21T01:55:26.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.460+0000 7fe9612b3700 1 -- 172.21.15.140:0/3178367665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c144100 con 0x7fe93c003780 2022-07-21T01:55:26.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.460+0000 7fe9597fa700 1 -- 172.21.15.140:0/3178367665 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe94c02d070 con 0x7fe93c003780 2022-07-21T01:55:26.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.460+0000 7fe9597fa700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe940053a70 0x7fe940055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.460+0000 7fe9597fa700 1 -- 172.21.15.140:0/3178367665 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe94c06d730 con 0x7fe93c003780 2022-07-21T01:55:26.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.460+0000 7fe95b7fe700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe940053a70 0x7fe940055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.460+0000 7fe95b7fe700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe940053a70 0x7fe940055f30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fe944005000 tx=0x7fe944009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.468+0000 7fe9597fa700 1 -- 172.21.15.140:0/3178367665 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe94c044570 con 0x7fe93c003780 2022-07-21T01:55:26.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.588+0000 7fe9612b3700 1 -- 172.21.15.140:0/3178367665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe95c0574e0 con 0x7fe93c003780 2022-07-21T01:55:26.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.588+0000 7fe9597fa700 1 -- 172.21.15.140:0/3178367665 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v84) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe94c06be60 con 0x7fe93c003780 2022-07-21T01:55:26.590 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:55:26.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 -- 172.21.15.140:0/3178367665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe940053a70 msgr2=0x7fe940055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe940053a70 0x7fe940055f30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7fe944005000 tx=0x7fe944009040).stop 2022-07-21T01:55:26.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 -- 172.21.15.140:0/3178367665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 msgr2=0x7fe95c143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 0x7fe95c143480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fe94c00c000 tx=0x7fe94c019000).stop 2022-07-21T01:55:26.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 -- 172.21.15.140:0/3178367665 shutdown_connections 2022-07-21T01:55:26.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe940053a70 0x7fe940055f30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe95c151b70 0x7fe95c0b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe95c0b4f50 0x7fe95c151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 --2- 172.21.15.140:0/3178367665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe93c003780 0x7fe95c143480 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 -- 172.21.15.140:0/3178367665 >> 172.21.15.140:0/3178367665 conn(0x7fe95c063480 msgr2=0x7fe95c066190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:26.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 -- 172.21.15.140:0/3178367665 shutdown_connections 2022-07-21T01:55:26.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.592+0000 7fe9612b3700 1 -- 172.21.15.140:0/3178367665 wait complete. 2022-07-21T01:55:26.607 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:55:26.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.716+0000 7f0e6b59e700 1 Processor -- start 2022-07-21T01:55:26.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.716+0000 7f0e6b59e700 1 -- start start 2022-07-21T01:55:26.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.716+0000 7f0e6b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c14ada0 0x7f0e6c146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.716+0000 7f0e6b59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6c0afbc0 con 0x7f0e6c0b14f0 2022-07-21T01:55:26.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.716+0000 7f0e6b59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6c0b00e0 con 0x7f0e6c0b59d0 2022-07-21T01:55:26.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.716+0000 7f0e6b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c148080 con 0x7f0e6c14ada0 2022-07-21T01:55:26.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.716+0000 7f0e6ad9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0e6c0b14f0 0x7f0e6c146530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51784/0 (socket says 172.21.15.140:51784) 2022-07-21T01:55:26.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.716+0000 7f0e6ad9d700 1 -- 172.21.15.140:0/2342873593 learned_addr learned my addr 172.21.15.140:0/2342873593 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:26.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.716+0000 7f0e69d9b700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c14ada0 0x7f0e6c146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.716+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388331710 0 0) 0x7f0e6c0afbc0 con 0x7f0e6c0b14f0 2022-07-21T01:55:26.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.716+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e50003260 con 0x7f0e6c0b14f0 2022-07-21T01:55:26.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3665680869 0 0) 0x7f0e50003260 con 0x7f0e6c0b14f0 2022-07-21T01:55:26.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c14ada0 msgr2=0x7f0e6c146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:26.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c14ada0 0x7f0e6c146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 >> v1:172.21.15.152:6789/0 conn(0x7f0e6c0b59d0 legacy=0x7f0e6c0b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c0b5db0 con 0x7f0e6c0b14f0 2022-07-21T01:55:26.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f0e64001a00 con 0x7f0e6c0b14f0 2022-07-21T01:55:26.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 >> v1:172.21.15.140:6789/0 conn(0x7f0e6c0b14f0 legacy=0x7f0e6c146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 0x7f0e50003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e50004500 0x7f0e50004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c14ada0 0x7f0e50004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c148080 con 0x7f0e50003780 2022-07-21T01:55:26.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5000b7d0 con 0x7f0e50004500 2022-07-21T01:55:26.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5000b990 con 0x7f0e6c14ada0 2022-07-21T01:55:26.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6a59c700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 0x7f0e50003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e69d9b700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e50004500 0x7f0e50004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6ad9d700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c14ada0 0x7f0e50004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6a59c700 1 -- 172.21.15.140:0/2342873593 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c14ada0 msgr2=0x7f0e50004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:26.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6a59c700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c14ada0 0x7f0e50004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6a59c700 1 -- 172.21.15.140:0/2342873593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e50004500 msgr2=0x7f0e50004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:26.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6a59c700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e50004500 0x7f0e50004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6a59c700 1 -- 172.21.15.140:0/2342873593 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e50003260 con 0x7f0e50003780 2022-07-21T01:55:26.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6a59c700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 0x7f0e50003bc0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f0e6c0b2d00 tx=0x7f0e60007a00).ready entity=mon.0 client_cookie=25ca411cebc95372 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e60007d10 con 0x7f0e50003780 2022-07-21T01:55:26.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e60007030 con 0x7f0e50003780 2022-07-21T01:55:26.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6959a700 1 -- 172.21.15.140:0/2342873593 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e60007390 con 0x7f0e50003780 2022-07-21T01:55:26.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6b59e700 1 -- 172.21.15.140:0/2342873593 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 msgr2=0x7f0e50003bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6b59e700 1 --2- 172.21.15.140:0/2342873593 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 0x7f0e50003bc0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f0e6c0b2d00 tx=0x7f0e60007a00).stop 2022-07-21T01:55:26.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6ad9d700 1 -- 172.21.15.140:0/2342873593 reap_dead start 2022-07-21T01:55:26.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6b59e700 1 -- 172.21.15.140:0/2342873593 shutdown_connections 2022-07-21T01:55:26.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6b59e700 1 -- 172.21.15.140:0/2342873593 >> 172.21.15.140:0/2342873593 conn(0x7f0e6c0634c0 msgr2=0x7f0e6c0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:26.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6b59e700 1 -- 172.21.15.140:0/2342873593 shutdown_connections 2022-07-21T01:55:26.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.720+0000 7f0e6b59e700 1 -- 172.21.15.140:0/2342873593 wait complete. 2022-07-21T01:55:26.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6b59e700 1 Processor -- start 2022-07-21T01:55:26.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6b59e700 1 -- start start 2022-07-21T01:55:26.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 0x7f0e6c146240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c159360 0x7f0e6c1119e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e6c065b00 0x7f0e6c111ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c10e170 con 0x7f0e50003780 2022-07-21T01:55:26.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 0x7f0e6c146240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c067330 con 0x7f0e6c065b00 2022-07-21T01:55:26.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c10e3a0 con 0x7f0e6c159360 2022-07-21T01:55:26.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 0x7f0e6c146240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37646/0 (socket says 172.21.15.140:37646) 2022-07-21T01:55:26.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6a59c700 1 -- 172.21.15.140:0/1218413969 learned_addr learned my addr 172.21.15.140:0/1218413969 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:26.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e69d9b700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c159360 0x7f0e6c1119e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6ad9d700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e6c065b00 0x7f0e6c111ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6a59c700 1 -- 172.21.15.140:0/1218413969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c159360 msgr2=0x7f0e6c1119e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6a59c700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c159360 0x7f0e6c1119e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6a59c700 1 -- 172.21.15.140:0/1218413969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e6c065b00 msgr2=0x7f0e6c111ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6a59c700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e6c065b00 0x7f0e6c111ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6a59c700 1 -- 172.21.15.140:0/1218413969 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c10e640 con 0x7f0e50003780 2022-07-21T01:55:26.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e69d9b700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c159360 0x7f0e6c1119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:26.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e6a59c700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 0x7f0e6c146240 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f0e6c0b2d00 tx=0x7f0e60015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.724+0000 7f0e5b7fe700 1 -- 172.21.15.140:0/1218413969 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e600127b0 con 0x7f0e50003780 2022-07-21T01:55:26.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.728+0000 7f0e5b7fe700 1 -- 172.21.15.140:0/1218413969 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e60012910 con 0x7f0e50003780 2022-07-21T01:55:26.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.728+0000 7f0e6b59e700 1 -- 172.21.15.140:0/1218413969 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c10e8e0 con 0x7f0e50003780 2022-07-21T01:55:26.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.728+0000 7f0e5b7fe700 1 -- 172.21.15.140:0/1218413969 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e60012c90 con 0x7f0e50003780 2022-07-21T01:55:26.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.728+0000 7f0e6b59e700 1 -- 172.21.15.140:0/1218413969 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c065700 con 0x7f0e50003780 2022-07-21T01:55:26.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.728+0000 7f0e6b59e700 1 -- 172.21.15.140:0/1218413969 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c0b0340 con 0x7f0e50003780 2022-07-21T01:55:26.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.728+0000 7f0e5b7fe700 1 -- 172.21.15.140:0/1218413969 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0e6001d070 con 0x7f0e50003780 2022-07-21T01:55:26.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.728+0000 7f0e5b7fe700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e48053aa0 0x7f0e48055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.728+0000 7f0e5b7fe700 1 -- 172.21.15.140:0/1218413969 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0e60001e40 con 0x7f0e50003780 2022-07-21T01:55:26.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.728+0000 7f0e69d9b700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e48053aa0 0x7f0e48055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.728+0000 7f0e69d9b700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e48053aa0 0x7f0e48055f60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f0e5c00b000 tx=0x7f0e5c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.732+0000 7f0e5b7fe700 1 -- 172.21.15.140:0/1218413969 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e60043d90 con 0x7f0e50003780 2022-07-21T01:55:26.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.856+0000 7f0e6b59e700 1 -- 172.21.15.140:0/1218413969 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0e6c0574e0 con 0x7f0e50003780 2022-07-21T01:55:26.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.856+0000 7f0e5b7fe700 1 -- 172.21.15.140:0/1218413969 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v84) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0e60025070 con 0x7f0e50003780 2022-07-21T01:55:26.857 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:26.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 -- 172.21.15.140:0/1218413969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e48053aa0 msgr2=0x7f0e48055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e48053aa0 0x7f0e48055f60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f0e5c00b000 tx=0x7f0e5c008040).stop 2022-07-21T01:55:26.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 -- 172.21.15.140:0/1218413969 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 msgr2=0x7f0e6c146240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 0x7f0e6c146240 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f0e6c0b2d00 tx=0x7f0e60015000).stop 2022-07-21T01:55:26.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 -- 172.21.15.140:0/1218413969 shutdown_connections 2022-07-21T01:55:26.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e48053aa0 0x7f0e48055f60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e6c065b00 0x7f0e6c111ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e6c159360 0x7f0e6c1119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 --2- 172.21.15.140:0/1218413969 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e50003780 0x7f0e6c146240 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 -- 172.21.15.140:0/1218413969 >> 172.21.15.140:0/1218413969 conn(0x7f0e6c0634c0 msgr2=0x7f0e6c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:26.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 -- 172.21.15.140:0/1218413969 shutdown_connections 2022-07-21T01:55:26.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.860+0000 7f0e6b59e700 1 -- 172.21.15.140:0/1218413969 wait complete. 2022-07-21T01:55:26.870 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:55:26.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.976+0000 7f4ea994f700 1 Processor -- start 2022-07-21T01:55:26.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.976+0000 7f4ea994f700 1 -- start start 2022-07-21T01:55:26.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4ea40b5b50 0x7f4ea4143150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ea40b0cc0 0x7f4ea4143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea40b5f30 con 0x7f4ea4143b30 2022-07-21T01:55:26.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea4140950 con 0x7f4ea40b5b50 2022-07-21T01:55:26.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40af1a0 con 0x7f4ea40b0cc0 2022-07-21T01:55:26.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea37fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4ea4143b30 0x7f4ea40afaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51800/0 (socket says 172.21.15.140:51800) 2022-07-21T01:55:26.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea37fe700 1 -- 172.21.15.140:0/4212132237 learned_addr learned my addr 172.21.15.140:0/4212132237 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:26.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea27fc700 1 --2- 172.21.15.140:0/4212132237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ea40b0cc0 0x7f4ea4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea2ffd700 1 --2- 172.21.15.140:0/4212132237 >> v2:172.21.15.152:3300/0 conn(0x7f4ea40b5b50 0x7f4ea4143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea1ffb700 1 -- 172.21.15.140:0/4212132237 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523820560 0 0) 0x7f4ea40b5f30 con 0x7f4ea4143b30 2022-07-21T01:55:26.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea1ffb700 1 -- 172.21.15.140:0/4212132237 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e88003260 con 0x7f4ea4143b30 2022-07-21T01:55:26.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea27fc700 1 -- 172.21.15.140:0/4212132237 >> v2:172.21.15.152:3300/0 conn(0x7f4ea40b5b50 msgr2=0x7f4ea4143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:26.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea27fc700 1 --2- 172.21.15.140:0/4212132237 >> v2:172.21.15.152:3300/0 conn(0x7f4ea40b5b50 0x7f4ea4143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea27fc700 1 -- 172.21.15.140:0/4212132237 >> v1:172.21.15.140:6789/0 conn(0x7f4ea4143b30 legacy=0x7f4ea40afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea27fc700 1 -- 172.21.15.140:0/4212132237 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40af440 con 0x7f4ea40b0cc0 2022-07-21T01:55:26.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea2ffd700 1 --2- 172.21.15.140:0/4212132237 >> v2:172.21.15.152:3300/0 conn(0x7f4ea40b5b50 0x7f4ea4143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:26.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea27fc700 1 --2- 172.21.15.140:0/4212132237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ea40b0cc0 0x7f4ea4143640 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f4e8c002f70 tx=0x7f4e8c006df0).ready entity=mon.2 client_cookie=84049ec48696c543 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea1ffb700 1 -- 172.21.15.140:0/4212132237 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e8c00d4a0 con 0x7f4ea40b0cc0 2022-07-21T01:55:26.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea1ffb700 1 -- 172.21.15.140:0/4212132237 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c000e90 con 0x7f4ea40b0cc0 2022-07-21T01:55:26.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea1ffb700 1 -- 172.21.15.140:0/4212132237 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e8c00d7f0 con 0x7f4ea40b0cc0 2022-07-21T01:55:26.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 -- 172.21.15.140:0/4212132237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ea40b0cc0 msgr2=0x7f4ea4143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 --2- 172.21.15.140:0/4212132237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ea40b0cc0 0x7f4ea4143640 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f4e8c002f70 tx=0x7f4e8c006df0).stop 2022-07-21T01:55:26.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 -- 172.21.15.140:0/4212132237 shutdown_connections 2022-07-21T01:55:26.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 --2- 172.21.15.140:0/4212132237 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ea40b0cc0 0x7f4ea4143640 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 --2- 172.21.15.140:0/4212132237 >> v2:172.21.15.152:3300/0 conn(0x7f4ea40b5b50 0x7f4ea4143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 -- 172.21.15.140:0/4212132237 >> 172.21.15.140:0/4212132237 conn(0x7f4ea40635e0 msgr2=0x7f4ea40b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:26.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 -- 172.21.15.140:0/4212132237 shutdown_connections 2022-07-21T01:55:26.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 -- 172.21.15.140:0/4212132237 wait complete. 2022-07-21T01:55:26.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 Processor -- start 2022-07-21T01:55:26.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.980+0000 7f4ea994f700 1 -- start start 2022-07-21T01:55:26.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea994f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ea40b0cc0 0x7f4ea40b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea994f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ea40b5b50 0x7f4ea414a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea994f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ea4143b30 0x7f4ea414a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea994f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4140770 con 0x7f4ea4143b30 2022-07-21T01:55:26.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea994f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4149620 con 0x7f4ea40b5b50 2022-07-21T01:55:26.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea994f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4149850 con 0x7f4ea40b0cc0 2022-07-21T01:55:26.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ea4143b30 0x7f4ea414a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea2ffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ea40b0cc0 0x7f4ea40b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ea4143b30 0x7f4ea414a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37664/0 (socket says 172.21.15.140:37664) 2022-07-21T01:55:26.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea37fe700 1 -- 172.21.15.140:0/3864729894 learned_addr learned my addr 172.21.15.140:0/3864729894 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:26.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea27fc700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ea40b5b50 0x7f4ea414a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea37fe700 1 -- 172.21.15.140:0/3864729894 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ea40b0cc0 msgr2=0x7f4ea40b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea37fe700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ea40b0cc0 0x7f4ea40b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea37fe700 1 -- 172.21.15.140:0/3864729894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ea40b5b50 msgr2=0x7f4ea414a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:26.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea37fe700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ea40b5b50 0x7f4ea414a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:26.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea37fe700 1 -- 172.21.15.140:0/3864729894 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e94009470 con 0x7f4ea4143b30 2022-07-21T01:55:26.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea37fe700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ea4143b30 0x7f4ea414a710 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f4e9c001110 tx=0x7f4e9c007a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4e9bfff700 1 -- 172.21.15.140:0/3864729894 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e9c00cd50 con 0x7f4ea4143b30 2022-07-21T01:55:26.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4e9bfff700 1 -- 172.21.15.140:0/3864729894 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c0070d0 con 0x7f4ea4143b30 2022-07-21T01:55:26.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4e9bfff700 1 -- 172.21.15.140:0/3864729894 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e9c007430 con 0x7f4ea4143b30 2022-07-21T01:55:26.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea994f700 1 -- 172.21.15.140:0/3864729894 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8c00b180 con 0x7f4ea4143b30 2022-07-21T01:55:26.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea994f700 1 -- 172.21.15.140:0/3864729894 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea4066da0 con 0x7f4ea4143b30 2022-07-21T01:55:26.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4ea994f700 1 -- 172.21.15.140:0/3864729894 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea413e100 con 0x7f4ea4143b30 2022-07-21T01:55:26.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.984+0000 7f4e9bfff700 1 -- 172.21.15.140:0/3864729894 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4e9c010070 con 0x7f4ea4143b30 2022-07-21T01:55:26.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.988+0000 7f4e9bfff700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e7c053b00 0x7f4e7c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:26.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.988+0000 7f4e9bfff700 1 -- 172.21.15.140:0/3864729894 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4e9c06b2e0 con 0x7f4ea4143b30 2022-07-21T01:55:26.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.988+0000 7f4ea2ffd700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e7c053b00 0x7f4e7c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:26.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.988+0000 7f4ea2ffd700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e7c053b00 0x7f4e7c055fc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f4e9400b000 tx=0x7f4e940091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:26.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:26.992+0000 7f4e9bfff700 1 -- 172.21.15.140:0/3864729894 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4e9c042200 con 0x7f4ea4143b30 2022-07-21T01:55:27.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.116+0000 7f4ea994f700 1 -- 172.21.15.140:0/3864729894 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ea40af6e0 con 0x7f4ea4143b30 2022-07-21T01:55:27.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4e9bfff700 1 -- 172.21.15.140:0/3864729894 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4e9c018020 con 0x7f4ea4143b30 2022-07-21T01:55:27.119 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:27.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 -- 172.21.15.140:0/3864729894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e7c053b00 msgr2=0x7f4e7c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e7c053b00 0x7f4e7c055fc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f4e9400b000 tx=0x7f4e940091d0).stop 2022-07-21T01:55:27.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 -- 172.21.15.140:0/3864729894 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ea4143b30 msgr2=0x7f4ea414a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ea4143b30 0x7f4ea414a710 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f4e9c001110 tx=0x7f4e9c007a20).stop 2022-07-21T01:55:27.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 -- 172.21.15.140:0/3864729894 shutdown_connections 2022-07-21T01:55:27.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e7c053b00 0x7f4e7c055fc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4ea4143b30 0x7f4ea414a710 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ea40b5b50 0x7f4ea414a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 --2- 172.21.15.140:0/3864729894 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4ea40b0cc0 0x7f4ea40b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 -- 172.21.15.140:0/3864729894 >> 172.21.15.140:0/3864729894 conn(0x7f4ea40635e0 msgr2=0x7f4ea4140e50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:27.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 -- 172.21.15.140:0/3864729894 shutdown_connections 2022-07-21T01:55:27.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.120+0000 7f4ea994f700 1 -- 172.21.15.140:0/3864729894 wait complete. 2022-07-21T01:55:27.132 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:55:27.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1471ae700 1 Processor -- start 2022-07-21T01:55:27.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1471ae700 1 -- start start 2022-07-21T01:55:27.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1471ae700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1400aa9d0 0x7fa1400b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1471ae700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa140143f60 0x7fa1400abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1471ae700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1401443f0 con 0x7fa1400b58f0 2022-07-21T01:55:27.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1471ae700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400aae10 con 0x7fa140143f60 2022-07-21T01:55:27.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1471ae700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400b4150 con 0x7fa1400aa9d0 2022-07-21T01:55:27.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1469ad700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa140143f60 0x7fa1400abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1469ad700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa140143f60 0x7fa1400abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37672/0 (socket says 172.21.15.140:37672) 2022-07-21T01:55:27.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1469ad700 1 -- 172.21.15.140:0/2868932888 learned_addr learned my addr 172.21.15.140:0/2868932888 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:27.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1459ab700 1 --2- 172.21.15.140:0/2868932888 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1400aa9d0 0x7fa1400b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1469ad700 1 -- 172.21.15.140:0/2868932888 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1400aa9d0 msgr2=0x7fa1400b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:27.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1469ad700 1 --2- 172.21.15.140:0/2868932888 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1400aa9d0 0x7fa1400b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1469ad700 1 -- 172.21.15.140:0/2868932888 >> v1:172.21.15.152:6789/0 conn(0x7fa1400b58f0 legacy=0x7fa1400b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1469ad700 1 -- 172.21.15.140:0/2868932888 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400b43f0 con 0x7fa140143f60 2022-07-21T01:55:27.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1459ab700 1 --2- 172.21.15.140:0/2868932888 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1400aa9d0 0x7fa1400b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:27.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1469ad700 1 --2- 172.21.15.140:0/2868932888 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa140143f60 0x7fa1400abfb0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fa1380032f0 tx=0x7fa138009290).ready entity=mon.0 client_cookie=16b8593566f082ec server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:27.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1451aa700 1 -- 172.21.15.140:0/2868932888 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa13800b870 con 0x7fa140143f60 2022-07-21T01:55:27.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1451aa700 1 -- 172.21.15.140:0/2868932888 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13800b9d0 con 0x7fa140143f60 2022-07-21T01:55:27.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.236+0000 7fa1451aa700 1 -- 172.21.15.140:0/2868932888 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa13800bb80 con 0x7fa140143f60 2022-07-21T01:55:27.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 -- 172.21.15.140:0/2868932888 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa140143f60 msgr2=0x7fa1400abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 --2- 172.21.15.140:0/2868932888 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa140143f60 0x7fa1400abfb0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fa1380032f0 tx=0x7fa138009290).stop 2022-07-21T01:55:27.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 -- 172.21.15.140:0/2868932888 shutdown_connections 2022-07-21T01:55:27.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 --2- 172.21.15.140:0/2868932888 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa140143f60 0x7fa1400abfb0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 --2- 172.21.15.140:0/2868932888 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1400aa9d0 0x7fa1400b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 -- 172.21.15.140:0/2868932888 >> 172.21.15.140:0/2868932888 conn(0x7fa1400632b0 msgr2=0x7fa140063690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:27.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 -- 172.21.15.140:0/2868932888 shutdown_connections 2022-07-21T01:55:27.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 -- 172.21.15.140:0/2868932888 wait complete. 2022-07-21T01:55:27.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 Processor -- start 2022-07-21T01:55:27.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 -- start start 2022-07-21T01:55:27.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1400aa9d0 0x7fa140065310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1400b58f0 0x7fa1401458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa140143f60 0x7fa140066680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14009b620 con 0x7fa140143f60 2022-07-21T01:55:27.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa140145f70 con 0x7fa1400aa9d0 2022-07-21T01:55:27.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1471ae700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa140066350 con 0x7fa1400b58f0 2022-07-21T01:55:27.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1459ab700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1400b58f0 0x7fa1401458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1469ad700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa140143f60 0x7fa140066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1461ac700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1400aa9d0 0x7fa140065310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1461ac700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1400aa9d0 0x7fa140065310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:43414/0 (socket says 172.21.15.140:43414) 2022-07-21T01:55:27.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.240+0000 7fa1461ac700 1 -- 172.21.15.140:0/1464490501 learned_addr learned my addr 172.21.15.140:0/1464490501 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:27.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1469ad700 1 -- 172.21.15.140:0/1464490501 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1400b58f0 msgr2=0x7fa1401458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1469ad700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1400b58f0 0x7fa1401458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1469ad700 1 -- 172.21.15.140:0/1464490501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1400aa9d0 msgr2=0x7fa140065310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1469ad700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1400aa9d0 0x7fa140065310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1469ad700 1 -- 172.21.15.140:0/1464490501 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa138009530 con 0x7fa140143f60 2022-07-21T01:55:27.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1461ac700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1400aa9d0 0x7fa140065310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:27.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1469ad700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa140143f60 0x7fa140066680 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7fa14009b760 tx=0x7fa138016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:27.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa12f7fe700 1 -- 172.21.15.140:0/1464490501 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa138021440 con 0x7fa140143f60 2022-07-21T01:55:27.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa12f7fe700 1 -- 172.21.15.140:0/1464490501 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1380215a0 con 0x7fa140143f60 2022-07-21T01:55:27.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1471ae700 1 -- 172.21.15.140:0/1464490501 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1400accd0 con 0x7fa140143f60 2022-07-21T01:55:27.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa12f7fe700 1 -- 172.21.15.140:0/1464490501 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa138021900 con 0x7fa140143f60 2022-07-21T01:55:27.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1471ae700 1 -- 172.21.15.140:0/1464490501 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa140064c70 con 0x7fa140143f60 2022-07-21T01:55:27.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1471ae700 1 -- 172.21.15.140:0/1464490501 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa140145da0 con 0x7fa140143f60 2022-07-21T01:55:27.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa12f7fe700 1 -- 172.21.15.140:0/1464490501 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa13800f070 con 0x7fa140143f60 2022-07-21T01:55:27.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa12f7fe700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1200539e0 0x7fa120055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa12f7fe700 1 -- 172.21.15.140:0/1464490501 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa13806d0b0 con 0x7fa140143f60 2022-07-21T01:55:27.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1461ac700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1200539e0 0x7fa120055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.244+0000 7fa1461ac700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1200539e0 0x7fa120055ea0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fa13c00d000 tx=0x7fa13c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:27.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.248+0000 7fa12f7fe700 1 -- 172.21.15.140:0/1464490501 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa138043fd0 con 0x7fa140143f60 2022-07-21T01:55:27.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.372+0000 7fa1471ae700 1 -- 172.21.15.140:0/1464490501 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa1400b4690 con 0x7fa140143f60 2022-07-21T01:55:27.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.372+0000 7fa12f7fe700 1 -- 172.21.15.140:0/1464490501 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa138015020 con 0x7fa140143f60 2022-07-21T01:55:27.374 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:27.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 -- 172.21.15.140:0/1464490501 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1200539e0 msgr2=0x7fa120055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1200539e0 0x7fa120055ea0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fa13c00d000 tx=0x7fa13c009040).stop 2022-07-21T01:55:27.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 -- 172.21.15.140:0/1464490501 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa140143f60 msgr2=0x7fa140066680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa140143f60 0x7fa140066680 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7fa14009b760 tx=0x7fa138016000).stop 2022-07-21T01:55:27.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 -- 172.21.15.140:0/1464490501 shutdown_connections 2022-07-21T01:55:27.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1200539e0 0x7fa120055ea0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa140143f60 0x7fa140066680 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa1400b58f0 0x7fa1401458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 --2- 172.21.15.140:0/1464490501 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1400aa9d0 0x7fa140065310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 -- 172.21.15.140:0/1464490501 >> 172.21.15.140:0/1464490501 conn(0x7fa1400632b0 msgr2=0x7fa1400b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:27.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 -- 172.21.15.140:0/1464490501 shutdown_connections 2022-07-21T01:55:27.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.376+0000 7fa1471ae700 1 -- 172.21.15.140:0/1464490501 wait complete. 2022-07-21T01:55:27.387 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:55:27.388 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:27.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9863e71700 1 Processor -- start 2022-07-21T01:55:27.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9863e71700 1 -- start start 2022-07-21T01:55:27.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9863e71700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c05c3f0 0x7f985c14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9863e71700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f985c0acef0 0x7f985c146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9863e71700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c05d480 con 0x7f985c0b1e00 2022-07-21T01:55:27.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9863e71700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985c0b21e0 con 0x7f985c0acef0 2022-07-21T01:55:27.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9863e71700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c05cf40 con 0x7f985c05c3f0 2022-07-21T01:55:27.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9861c0d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f985c0b1e00 0x7f985c0ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51828/0 (socket says 172.21.15.140:51828) 2022-07-21T01:55:27.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9861c0d700 1 -- 172.21.15.140:0/2619836572 learned_addr learned my addr 172.21.15.140:0/2619836572 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:27.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f986140c700 1 --2- 172.21.15.140:0/2619836572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c05c3f0 0x7f985c14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f986240e700 1 --2- 172.21.15.140:0/2619836572 >> v2:172.21.15.152:3300/0 conn(0x7f985c0acef0 0x7f985c146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116808320 0 0) 0x7f985c05d480 con 0x7f985c0b1e00 2022-07-21T01:55:27.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9840003260 con 0x7f985c0b1e00 2022-07-21T01:55:27.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 850831916 0 0) 0x7f9840003260 con 0x7f985c0b1e00 2022-07-21T01:55:27.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c05c3f0 msgr2=0x7f985c14a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:27.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 --2- 172.21.15.140:0/2619836572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c05c3f0 0x7f985c14a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 >> v2:172.21.15.152:3300/0 conn(0x7f985c0acef0 msgr2=0x7f985c146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:27.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 --2- 172.21.15.140:0/2619836572 >> v2:172.21.15.152:3300/0 conn(0x7f985c0acef0 0x7f985c146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c05d1e0 con 0x7f985c0b1e00 2022-07-21T01:55:27.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f9858001660 con 0x7f985c0b1e00 2022-07-21T01:55:27.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 >> v1:172.21.15.140:6789/0 conn(0x7f985c0b1e00 legacy=0x7f985c0ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 --2- 172.21.15.140:0/2619836572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c05c3f0 0x7f9840003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 --2- 172.21.15.140:0/2619836572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9840003c80 0x7f9840004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 --2- 172.21.15.140:0/2619836572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98400049f0 0x7f9840004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0b21e0 con 0x7f9840003c80 2022-07-21T01:55:27.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c05cf40 con 0x7f98400049f0 2022-07-21T01:55:27.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984000b750 con 0x7f985c05c3f0 2022-07-21T01:55:27.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f986140c700 1 --2- 172.21.15.140:0/2619836572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9840003c80 0x7f9840004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f986140c700 1 -- 172.21.15.140:0/2619836572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c05c3f0 msgr2=0x7f9840003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:27.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9861c0d700 1 --2- 172.21.15.140:0/2619836572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c05c3f0 0x7f9840003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f986140c700 1 --2- 172.21.15.140:0/2619836572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c05c3f0 0x7f9840003750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f986140c700 1 -- 172.21.15.140:0/2619836572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98400049f0 msgr2=0x7f9840004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:27.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f986140c700 1 --2- 172.21.15.140:0/2619836572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f98400049f0 0x7f9840004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2022-07-21T01:55:27.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f986140c700 1 -- 172.21.15.140:0/2619836572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9840003260 con 0x7f9840003c80 2022-07-21T01:55:27.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f986140c700 1 --2- 172.21.15.140:0/2619836572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9840003c80 0x7f9840004500 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f9840004470 tx=0x7f98540091d0).ready entity=mon.0 client_cookie=b7ae8508e688d934 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:27.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f985400e7e0 con 0x7f9840003c80 2022-07-21T01:55:27.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985400e940 con 0x7f9840003c80 2022-07-21T01:55:27.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.492+0000 7f9860c0b700 1 -- 172.21.15.140:0/2619836572 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f985400eca0 con 0x7f9840003c80 2022-07-21T01:55:27.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 -- 172.21.15.140:0/2619836572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9840003c80 msgr2=0x7f9840004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 --2- 172.21.15.140:0/2619836572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9840003c80 0x7f9840004500 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f9840004470 tx=0x7f98540091d0).stop 2022-07-21T01:55:27.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f986240e700 1 -- 172.21.15.140:0/2619836572 reap_dead start 2022-07-21T01:55:27.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 -- 172.21.15.140:0/2619836572 shutdown_connections 2022-07-21T01:55:27.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 -- 172.21.15.140:0/2619836572 >> 172.21.15.140:0/2619836572 conn(0x7f985c0611c0 msgr2=0x7f985c062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:27.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 -- 172.21.15.140:0/2619836572 shutdown_connections 2022-07-21T01:55:27.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 -- 172.21.15.140:0/2619836572 wait complete. 2022-07-21T01:55:27.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 Processor -- start 2022-07-21T01:55:27.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 -- start start 2022-07-21T01:55:27.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c145040 0x7f985c145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f985c1571a0 0x7f985c159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f985c15a1e0 0x7f985c15a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c15ab50 con 0x7f985c15a1e0 2022-07-21T01:55:27.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c063d60 con 0x7f985c1571a0 2022-07-21T01:55:27.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9863e71700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c064000 con 0x7f985c145040 2022-07-21T01:55:27.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f986240e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f985c15a1e0 0x7f985c15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f986240e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f985c15a1e0 0x7f985c15a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37698/0 (socket says 172.21.15.140:37698) 2022-07-21T01:55:27.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f986240e700 1 -- 172.21.15.140:0/3282081181 learned_addr learned my addr 172.21.15.140:0/3282081181 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:27.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f986140c700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f985c1571a0 0x7f985c159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f9861c0d700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c145040 0x7f985c145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f986240e700 1 -- 172.21.15.140:0/3282081181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c145040 msgr2=0x7f985c145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f986240e700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c145040 0x7f985c145420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.496+0000 7f986240e700 1 -- 172.21.15.140:0/3282081181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f985c1571a0 msgr2=0x7f985c159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f986240e700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f985c1571a0 0x7f985c159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f986240e700 1 -- 172.21.15.140:0/3282081181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c146bd0 con 0x7f985c15a1e0 2022-07-21T01:55:27.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f9861c0d700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c145040 0x7f985c145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:27.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f986240e700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f985c15a1e0 0x7f985c15a660 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f9850000e90 tx=0x7f9850006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:27.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f984effd700 1 -- 172.21.15.140:0/3282081181 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98500106b0 con 0x7f985c15a1e0 2022-07-21T01:55:27.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f984effd700 1 -- 172.21.15.140:0/3282081181 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9850010810 con 0x7f985c15a1e0 2022-07-21T01:55:27.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f9863e71700 1 -- 172.21.15.140:0/3282081181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985c157600 con 0x7f985c15a1e0 2022-07-21T01:55:27.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f984effd700 1 -- 172.21.15.140:0/3282081181 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9850010b70 con 0x7f985c15a1e0 2022-07-21T01:55:27.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f9863e71700 1 -- 172.21.15.140:0/3282081181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985c159900 con 0x7f985c15a1e0 2022-07-21T01:55:27.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f9863e71700 1 -- 172.21.15.140:0/3282081181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985c063b80 con 0x7f985c15a1e0 2022-07-21T01:55:27.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f984effd700 1 -- 172.21.15.140:0/3282081181 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f985001a030 con 0x7f985c15a1e0 2022-07-21T01:55:27.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f984effd700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9844053a70 0x7f9844055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:27.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f984effd700 1 -- 172.21.15.140:0/3282081181 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f985006b9f0 con 0x7f985c15a1e0 2022-07-21T01:55:27.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f9861c0d700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9844053a70 0x7f9844055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:27.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.500+0000 7f9861c0d700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9844053a70 0x7f9844055f30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f985800c000 tx=0x7f9858009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:27.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.504+0000 7f984effd700 1 -- 172.21.15.140:0/3282081181 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f985006de60 con 0x7f985c15a1e0 2022-07-21T01:55:27.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.664+0000 7f9863e71700 1 -- 172.21.15.140:0/3282081181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f985c0641c0 con 0x7f985c15a1e0 2022-07-21T01:55:27.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.664+0000 7f984effd700 1 -- 172.21.15.140:0/3282081181 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2146 (secure 0 0 0) 0x7f9850046340 con 0x7f985c15a1e0 2022-07-21T01:55:27.665 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:27.665 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":36,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/2489766347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2489766347},{"type":"v1","addr":"172.21.15.152:6833","nonce":2489766347}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:20.506956+0000","modified":"2022-07-21T01:55:21.509498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6586":{"gid":6586,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2461699636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2461699636},{"type":"v1","addr":"172.21.15.140:6835","nonce":2461699636}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-07-21T01:55:27.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.664+0000 7f9863e71700 1 -- 172.21.15.140:0/3282081181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9844053a70 msgr2=0x7f9844055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.664+0000 7f9863e71700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9844053a70 0x7f9844055f30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f985800c000 tx=0x7f9858009040).stop 2022-07-21T01:55:27.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.664+0000 7f9863e71700 1 -- 172.21.15.140:0/3282081181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f985c15a1e0 msgr2=0x7f985c15a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:27.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.664+0000 7f9863e71700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f985c15a1e0 0x7f985c15a660 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f9850000e90 tx=0x7f9850006df0).stop 2022-07-21T01:55:27.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.668+0000 7f9863e71700 1 -- 172.21.15.140:0/3282081181 shutdown_connections 2022-07-21T01:55:27.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.668+0000 7f9863e71700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9844053a70 0x7f9844055f30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.668+0000 7f9863e71700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f985c15a1e0 0x7f985c15a660 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.668+0000 7f9863e71700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f985c1571a0 0x7f985c159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.668+0000 7f9863e71700 1 --2- 172.21.15.140:0/3282081181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f985c145040 0x7f985c145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:27.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.668+0000 7f9863e71700 1 -- 172.21.15.140:0/3282081181 >> 172.21.15.140:0/3282081181 conn(0x7f9840003c80 msgr2=0x7f985c144400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:27.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.668+0000 7f9863e71700 1 -- 172.21.15.140:0/3282081181 shutdown_connections 2022-07-21T01:55:27.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:27.668+0000 7f9863e71700 1 -- 172.21.15.140:0/3282081181 wait complete. 2022-07-21T01:55:27.673 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 36 2022-07-21T01:55:27.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:55:20.506956+0000', 'modified': '2022-07-21T01:55:21.509498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6586}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6586': {'gid': 6586, 'name': 'a', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/2461699636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 2461699636}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 2461699636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:55:27.880 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:55:27.881 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:55:27.996 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":6586,"want_state":"up:active","state":"up:active","rank_uptime":7.4769574109999999,"mdsmap_epoch":36,"osdmap_epoch":84,"osdmap_epoch_barrier":83,"uptime":12.182092272} 2022-07-21T01:55:27.996 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 6586, "mdsmap_epoch": 36, "osdmap_epoch": 84, "osdmap_epoch_barrier": 83, "rank_uptime": 7.476957411, "state": "up:active", "uptime": 12.182092272, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:55:27.997 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:55:27.997 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-07-21T01:55:27.998 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-07-21T01:55:27.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:27.999 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:28.008 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:55:28.023 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:28.024 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:28.083 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:28.084 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:28.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:28.086 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:55:28.091 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:55:28.091 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:28.094 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:55:28.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:28.111 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:28.166 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:28.167 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:28.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:28.169 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:55:28.170 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2022-07-21T01:55:28.164+0000 7fdca5eb3080 -1 init, newargv = 0x564fbeda72b0 newargc=9 2022-07-21T01:55:28.170 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22479]: starting ceph client 2022-07-21T01:55:28.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22479]: starting fuse 2022-07-21T01:55:29.276 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:55:29.295 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:29.296 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:29.354 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:29.356 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:29.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:29.357 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:55:29.364 INFO:teuthology.orchestra.run.smithi192.stdout:52 2022-07-21T01:55:29.365 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:55:29.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:29.366 DEBUG:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2022-07-21T01:55:29.366 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2022-07-21T01:55:29.367 DEBUG:teuthology.orchestra.run.smithi192:> import re 2022-07-21T01:55:29.367 DEBUG:teuthology.orchestra.run.smithi192:> import os 2022-07-21T01:55:29.368 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2022-07-21T01:55:29.368 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:55:29.369 DEBUG:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2022-07-21T01:55:29.369 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-07-21T01:55:29.370 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2022-07-21T01:55:29.370 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:55:29.371 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2022-07-21T01:55:29.371 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2022-07-21T01:55:29.372 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2022-07-21T01:55:29.372 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2022-07-21T01:55:29.372 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:55:29.373 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2022-07-21T01:55:29.373 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:55:29.374 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:55:29.374 DEBUG:teuthology.orchestra.run.smithi192:> return f 2022-07-21T01:55:29.375 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:55:29.375 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:55:29.375 DEBUG:teuthology.orchestra.run.smithi192:> print(find_socket("client.1")) 2022-07-21T01:55:29.376 DEBUG:teuthology.orchestra.run.smithi192:> ' 2022-07-21T01:55:29.421 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.22479.asok 2022-07-21T01:55:29.426 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22479.asok 2022-07-21T01:55:29.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:29.428 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22479.asok status 2022-07-21T01:55:29.519 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2022-07-21T01:55:29.519 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2022-07-21T01:55:29.519 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:55:29.520 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:55:29.520 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2022-07-21T01:55:29.520 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2022-07-21T01:55:29.521 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-07-21T01:55:29.521 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "22479", 2022-07-21T01:55:29.521 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2022-07-21T01:55:29.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:55:29.522 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2022-07-21T01:55:29.522 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2022-07-21T01:55:29.522 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 6155, 2022-07-21T01:55:29.522 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2022-07-21T01:55:29.523 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2022-07-21T01:55:29.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2022-07-21T01:55:29.523 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 6155 2022-07-21T01:55:29.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:55:29.524 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:55:29.525 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:55:29.525 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:55:29.525 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2178788615 2022-07-21T01:55:29.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2022-07-21T01:55:29.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:55:29.526 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:55:29.526 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:55:29.526 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:55:29.527 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2178788615 2022-07-21T01:55:29.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:55:29.527 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.6155 172.21.15.192:0/2178788615", 2022-07-21T01:55:29.527 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/2178788615", 2022-07-21T01:55:29.528 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2022-07-21T01:55:29.528 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 36, 2022-07-21T01:55:29.528 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 84, 2022-07-21T01:55:29.529 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:55:29.529 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false, 2022-07-21T01:55:29.529 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2022-07-21T01:55:29.530 INFO:teuthology.orchestra.run.smithi192.stdout:} 2022-07-21T01:55:29.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:29.532 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:29.532 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:29.540 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:55:29.541 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:55:29.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:29.542 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:29.558 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:29.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d599ef700 1 Processor -- start 2022-07-21T01:55:29.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d599ef700 1 -- start start 2022-07-21T01:55:29.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d599ef700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d540adff0 0x7f4d540b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:29.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d599ef700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4d540af8e0 0x7f4d5405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:29.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d599ef700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d54147010 con 0x7f4d540b5650 2022-07-21T01:55:29.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d599ef700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d541472b0 con 0x7f4d540af8e0 2022-07-21T01:55:29.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d599ef700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d54146d40 con 0x7f4d540adff0 2022-07-21T01:55:29.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d589ed700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4d540b5650 0x7f4d54147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51850/0 (socket says 172.21.15.140:51850) 2022-07-21T01:55:29.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d589ed700 1 -- 172.21.15.140:0/1322781005 learned_addr learned my addr 172.21.15.140:0/1322781005 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:29.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d53fff700 1 --2- 172.21.15.140:0/1322781005 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d540adff0 0x7f4d540b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:29.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d591ee700 1 --2- 172.21.15.140:0/1322781005 >> v2:172.21.15.152:3300/0 conn(0x7f4d540af8e0 0x7f4d5405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:29.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d537fe700 1 -- 172.21.15.140:0/1322781005 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1191770808 0 0) 0x7f4d54147010 con 0x7f4d540b5650 2022-07-21T01:55:29.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.672+0000 7f4d537fe700 1 -- 172.21.15.140:0/1322781005 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d40003260 con 0x7f4d540b5650 2022-07-21T01:55:29.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d53fff700 1 -- 172.21.15.140:0/1322781005 >> v2:172.21.15.152:3300/0 conn(0x7f4d540af8e0 msgr2=0x7f4d5405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:29.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d53fff700 1 --2- 172.21.15.140:0/1322781005 >> v2:172.21.15.152:3300/0 conn(0x7f4d540af8e0 0x7f4d5405bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:29.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d53fff700 1 -- 172.21.15.140:0/1322781005 >> v1:172.21.15.140:6789/0 conn(0x7f4d540b5650 legacy=0x7f4d54147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:29.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d53fff700 1 -- 172.21.15.140:0/1322781005 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d540acae0 con 0x7f4d540adff0 2022-07-21T01:55:29.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d591ee700 1 --2- 172.21.15.140:0/1322781005 >> v2:172.21.15.152:3300/0 conn(0x7f4d540af8e0 0x7f4d5405bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:29.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d53fff700 1 --2- 172.21.15.140:0/1322781005 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d540adff0 0x7f4d540b5a30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f4d440032f0 tx=0x7f4d440091a0).ready entity=mon.2 client_cookie=2791ecbc9c538fe0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:29.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d537fe700 1 -- 172.21.15.140:0/1322781005 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d4400b600 con 0x7f4d540adff0 2022-07-21T01:55:29.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d537fe700 1 -- 172.21.15.140:0/1322781005 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4400b760 con 0x7f4d540adff0 2022-07-21T01:55:29.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d537fe700 1 -- 172.21.15.140:0/1322781005 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d4400b8d0 con 0x7f4d540adff0 2022-07-21T01:55:29.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 -- 172.21.15.140:0/1322781005 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d540adff0 msgr2=0x7f4d540b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:29.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 --2- 172.21.15.140:0/1322781005 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d540adff0 0x7f4d540b5a30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f4d440032f0 tx=0x7f4d440091a0).stop 2022-07-21T01:55:29.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 -- 172.21.15.140:0/1322781005 shutdown_connections 2022-07-21T01:55:29.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 --2- 172.21.15.140:0/1322781005 >> v2:172.21.15.152:3300/0 conn(0x7f4d540af8e0 0x7f4d5405bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:29.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 --2- 172.21.15.140:0/1322781005 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d540adff0 0x7f4d540b5a30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:29.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 -- 172.21.15.140:0/1322781005 >> 172.21.15.140:0/1322781005 conn(0x7f4d540634e0 msgr2=0x7f4d540b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:29.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 -- 172.21.15.140:0/1322781005 shutdown_connections 2022-07-21T01:55:29.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 -- 172.21.15.140:0/1322781005 wait complete. 2022-07-21T01:55:29.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 Processor -- start 2022-07-21T01:55:29.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 -- start start 2022-07-21T01:55:29.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d540adff0 0x7f4d54063990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:29.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540af8e0 0x7f4d54067330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:29.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d540b5650 0x7f4d540647f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:29.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d54111460 con 0x7f4d540b5650 2022-07-21T01:55:29.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d54064fe0 con 0x7f4d540af8e0 2022-07-21T01:55:29.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d599ef700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d54064160 con 0x7f4d540adff0 2022-07-21T01:55:29.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d591ee700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d540b5650 0x7f4d540647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:29.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d53fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540af8e0 0x7f4d54067330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:29.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d589ed700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d540adff0 0x7f4d54063990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:29.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d53fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540af8e0 0x7f4d54067330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:43450/0 (socket says 172.21.15.140:43450) 2022-07-21T01:55:29.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d53fff700 1 -- 172.21.15.140:0/1451799790 learned_addr learned my addr 172.21.15.140:0/1451799790 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:29.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.676+0000 7f4d591ee700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d540b5650 0x7f4d540647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37712/0 (socket says 172.21.15.140:37712) 2022-07-21T01:55:29.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.680+0000 7f4d591ee700 1 -- 172.21.15.140:0/1451799790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d540adff0 msgr2=0x7f4d54063990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:29.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.680+0000 7f4d591ee700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d540adff0 0x7f4d54063990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:29.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.680+0000 7f4d591ee700 1 -- 172.21.15.140:0/1451799790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540af8e0 msgr2=0x7f4d54067330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:29.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.680+0000 7f4d591ee700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540af8e0 0x7f4d54067330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:29.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.680+0000 7f4d591ee700 1 -- 172.21.15.140:0/1451799790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c00b180 con 0x7f4d540b5650 2022-07-21T01:55:29.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.680+0000 7f4d591ee700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d540b5650 0x7f4d540647f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f4d4c00c000 tx=0x7f4d4c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:29.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.680+0000 7f4d51ffb700 1 -- 172.21.15.140:0/1451799790 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d4c010780 con 0x7f4d540b5650 2022-07-21T01:55:29.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.680+0000 7f4d51ffb700 1 -- 172.21.15.140:0/1451799790 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c0108e0 con 0x7f4d540b5650 2022-07-21T01:55:29.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.680+0000 7f4d599ef700 1 -- 172.21.15.140:0/1451799790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d44009420 con 0x7f4d540b5650 2022-07-21T01:55:29.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.680+0000 7f4d51ffb700 1 -- 172.21.15.140:0/1451799790 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d4c010c40 con 0x7f4d540b5650 2022-07-21T01:55:29.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.680+0000 7f4d599ef700 1 -- 172.21.15.140:0/1451799790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d54064ce0 con 0x7f4d540b5650 2022-07-21T01:55:29.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.684+0000 7f4d599ef700 1 -- 172.21.15.140:0/1451799790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d54111060 con 0x7f4d540b5650 2022-07-21T01:55:29.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.684+0000 7f4d51ffb700 1 -- 172.21.15.140:0/1451799790 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4d4c01c460 con 0x7f4d540b5650 2022-07-21T01:55:29.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.684+0000 7f4d51ffb700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d34053a70 0x7f4d34055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:29.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.684+0000 7f4d51ffb700 1 -- 172.21.15.140:0/1451799790 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4d4c06c3e0 con 0x7f4d540b5650 2022-07-21T01:55:29.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.684+0000 7f4d589ed700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d34053a70 0x7f4d34055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:29.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.684+0000 7f4d589ed700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d34053a70 0x7f4d34055f30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f4d4800d000 tx=0x7f4d48009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:29.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.692+0000 7f4d51ffb700 1 -- 172.21.15.140:0/1451799790 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4d54111060 con 0x7f4d540b5650 2022-07-21T01:55:29.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 -- 172.21.15.140:0/1451799790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d54111060 con 0x7f4d540b5650 2022-07-21T01:55:29.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d51ffb700 1 -- 172.21.15.140:0/1451799790 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2146 (secure 0 0 0) 0x7f4d54111060 con 0x7f4d540b5650 2022-07-21T01:55:29.856 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:29.856 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":36,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/2489766347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2489766347},{"type":"v1","addr":"172.21.15.152:6833","nonce":2489766347}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:20.506956+0000","modified":"2022-07-21T01:55:21.509498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6586":{"gid":6586,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2461699636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2461699636},{"type":"v1","addr":"172.21.15.140:6835","nonce":2461699636}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-07-21T01:55:29.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 -- 172.21.15.140:0/1451799790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d34053a70 msgr2=0x7f4d34055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:29.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d34053a70 0x7f4d34055f30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f4d4800d000 tx=0x7f4d48009040).stop 2022-07-21T01:55:29.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 -- 172.21.15.140:0/1451799790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d540b5650 msgr2=0x7f4d540647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:29.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d540b5650 0x7f4d540647f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f4d4c00c000 tx=0x7f4d4c006df0).stop 2022-07-21T01:55:29.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 -- 172.21.15.140:0/1451799790 shutdown_connections 2022-07-21T01:55:29.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4d34053a70 0x7f4d34055f30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:29.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4d540b5650 0x7f4d540647f0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:29.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d540af8e0 0x7f4d54067330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:29.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 --2- 172.21.15.140:0/1451799790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4d540adff0 0x7f4d54063990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:29.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 -- 172.21.15.140:0/1451799790 >> 172.21.15.140:0/1451799790 conn(0x7f4d540634e0 msgr2=0x7f4d540b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:29.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 -- 172.21.15.140:0/1451799790 shutdown_connections 2022-07-21T01:55:29.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:29.856+0000 7f4d599ef700 1 -- 172.21.15.140:0/1451799790 wait complete. 2022-07-21T01:55:29.861 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 36 2022-07-21T01:55:29.869 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1 2022-07-21T01:55:29.976 INFO:teuthology.orchestra.run.smithi140.stdout:{"path":"/","ino":1,"rdev":0,"ctime":"2022-07-21T01:55:26.068375+0000","btime":"2022-07-21T01:55:20.512957+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":16,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":0,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2022-07-21T01:55:20.512957+0000","atime":"0.000000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2022-07-21T01:55:26.068375+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2022-07-21T01:55:20.512957+0000"},"version":3,"file_data_version":0,"xattr_version":1,"backtrace_version":3,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":3,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"authpin":1},"nref":9,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"xlockdone","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{"gather_set":[],"state":"sync","is_leased":false,"num_rdlocks":1,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent"],"client_caps":[{"client_id":6113,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2},{"client_id":6155,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[]} 2022-07-21T01:55:29.978 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "2022-07-21T01:55:20.512957+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "0.000000", "auth_pins": 2, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "xlockdone", "xlock_by": {} }, "backtrace_version": 3, "btime": "2022-07-21T01:55:20.512957+0000", "change_attr": 1, "client_caps": [ { "client_id": 6113, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" }, { "client_id": 6155, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" } ], "client_ranges": [], "ctime": "2022-07-21T01:55:26.068375+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 16, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2022-07-21T01:55:20.512957+0000", "nestlock": {}, "nlink": 1, "nref": 9, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/", "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 3, "request": 1 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2022-07-21T01:55:26.068375+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": { "gather_set": [], "is_leased": false, "num_rdlocks": 1, "num_wrlocks": 0, "num_xlocks": 0, "state": "sync", "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 0, "truncate_size": 18446744073709551615, "uid": 0, "version": 3, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } 2022-07-21T01:55:29.979 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:55:29.980 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:30.001 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:30.022 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:30.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:30.065 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:30.086 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:55:30.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:30.087 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:30.087 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:30.095 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:55:30.096 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:55:30.097 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:55:30.097 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:55:30.097 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:30.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[22958]: fuse finished with error 0 and tester_r 0 2022-07-21T01:55:30.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:30.157 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:30.158 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:30.165 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:55:30.165 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:55:30.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:55:30.189 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:30.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:30.198 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:30.199 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:30.208 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:55:30.209 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:55:30.209 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2022-07-21T01:55:30.210 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:55:30.210 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:30.224 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22479]: fuse finished with error 0 and tester_r 0 2022-07-21T01:55:30.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:30.226 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:30.227 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:30.278 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2022-07-21T01:55:30.279 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:55:30.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:55:30.367 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:30.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:30.375 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:30.375 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:55:30.382 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:55:30.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:30.384 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:55:30.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:55:30.385 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:55:30.392 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:55:30.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:30.394 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:55:30.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf8281700 1 Processor -- start 2022-07-21T01:55:30.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf8281700 1 -- start start 2022-07-21T01:55:30.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf8281700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf00ab8b0 0x7fedf00b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:30.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf8281700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fedf00618e0 0x7fedf0061d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:30.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf8281700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf005f970 con 0x7fedf00b21f0 2022-07-21T01:55:30.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf8281700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf0060b90 con 0x7fedf00618e0 2022-07-21T01:55:30.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf8281700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf014b640 con 0x7fedf00ab8b0 2022-07-21T01:55:30.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf601d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fedf00b21f0 0x7fedf0146790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51862/0 (socket says 172.21.15.140:51862) 2022-07-21T01:55:30.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf601d700 1 -- 172.21.15.140:0/1494557158 learned_addr learned my addr 172.21.15.140:0/1494557158 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:30.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf581c700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf00ab8b0 0x7fedf00b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:30.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf681e700 1 --2- 172.21.15.140:0/1494557158 >> v2:172.21.15.152:3300/0 conn(0x7fedf00618e0 0x7fedf0061d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:30.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554662003 0 0) 0x7fedf005f970 con 0x7fedf00b21f0 2022-07-21T01:55:30.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feddc003260 con 0x7fedf00b21f0 2022-07-21T01:55:30.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2658879550 0 0) 0x7feddc003260 con 0x7fedf00b21f0 2022-07-21T01:55:30.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf00ab8b0 msgr2=0x7fedf00b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:30.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf00ab8b0 0x7fedf00b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:30.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 >> v2:172.21.15.152:3300/0 conn(0x7fedf00618e0 msgr2=0x7fedf0061d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:30.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 --2- 172.21.15.140:0/1494557158 >> v2:172.21.15.152:3300/0 conn(0x7fedf00618e0 0x7fedf0061d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:30.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf005fc10 con 0x7fedf00b21f0 2022-07-21T01:55:30.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fede4001660 con 0x7fedf00b21f0 2022-07-21T01:55:30.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 >> v1:172.21.15.140:6789/0 conn(0x7fedf00b21f0 legacy=0x7fedf0146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:30.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feddc003780 0x7feddc003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:30.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feddc004500 0x7feddc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:30.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf00ab8b0 0x7feddc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:30.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0060b90 con 0x7feddc004500 2022-07-21T01:55:30.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf014b640 con 0x7feddc003780 2022-07-21T01:55:30.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc00b750 con 0x7fedf00ab8b0 2022-07-21T01:55:30.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf581c700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feddc004500 0x7feddc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:30.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf601d700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feddc003780 0x7feddc003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:30.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf681e700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf00ab8b0 0x7feddc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:30.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf581c700 1 -- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf00ab8b0 msgr2=0x7feddc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:30.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf581c700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf00ab8b0 0x7feddc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:30.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf581c700 1 -- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feddc003780 msgr2=0x7feddc003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:30.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf581c700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feddc003780 0x7feddc003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:30.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf581c700 1 -- 172.21.15.140:0/1494557158 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc003260 con 0x7feddc004500 2022-07-21T01:55:30.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf601d700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feddc003780 0x7feddc003ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:30.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf581c700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feddc004500 0x7feddc004980 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fede000c000 tx=0x7fede00091d0).ready entity=mon.0 client_cookie=5b59a6adc8369c99 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:30.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fede000e7e0 con 0x7feddc004500 2022-07-21T01:55:30.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede000e940 con 0x7feddc004500 2022-07-21T01:55:30.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.504+0000 7fedf501b700 1 -- 172.21.15.140:0/1494557158 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fede000eca0 con 0x7feddc004500 2022-07-21T01:55:30.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 -- 172.21.15.140:0/1494557158 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feddc004500 msgr2=0x7feddc004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:30.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 --2- 172.21.15.140:0/1494557158 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7feddc004500 0x7feddc004980 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fede000c000 tx=0x7fede00091d0).stop 2022-07-21T01:55:30.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf681e700 1 -- 172.21.15.140:0/1494557158 reap_dead start 2022-07-21T01:55:30.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 -- 172.21.15.140:0/1494557158 shutdown_connections 2022-07-21T01:55:30.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 -- 172.21.15.140:0/1494557158 >> 172.21.15.140:0/1494557158 conn(0x7fedf00b3690 msgr2=0x7fedf00b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:30.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 -- 172.21.15.140:0/1494557158 shutdown_connections 2022-07-21T01:55:30.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 -- 172.21.15.140:0/1494557158 wait complete. 2022-07-21T01:55:30.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 Processor -- start 2022-07-21T01:55:30.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 -- start start 2022-07-21T01:55:30.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feddc004500 0x7fedf009bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:30.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf009a740 0x7fedf00b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:30.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fedf00977d0 0x7fedf0097c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:30.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf009ad30 con 0x7fedf00977d0 2022-07-21T01:55:30.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf009b5c0 con 0x7feddc004500 2022-07-21T01:55:30.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf8281700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf009b860 con 0x7fedf009a740 2022-07-21T01:55:30.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf681e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fedf00977d0 0x7fedf0097c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:30.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf681e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fedf00977d0 0x7fedf0097c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37732/0 (socket says 172.21.15.140:37732) 2022-07-21T01:55:30.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf681e700 1 -- 172.21.15.140:0/1244275208 learned_addr learned my addr 172.21.15.140:0/1244275208 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:30.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf581c700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf009a740 0x7fedf00b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:30.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.508+0000 7fedf601d700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feddc004500 0x7fedf009bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:30.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedf681e700 1 -- 172.21.15.140:0/1244275208 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf009a740 msgr2=0x7fedf00b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:30.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedf681e700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf009a740 0x7fedf00b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:30.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedf681e700 1 -- 172.21.15.140:0/1244275208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feddc004500 msgr2=0x7fedf009bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:30.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedf681e700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feddc004500 0x7fedf009bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:30.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedf681e700 1 -- 172.21.15.140:0/1244275208 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0098150 con 0x7fedf00977d0 2022-07-21T01:55:30.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedf581c700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf009a740 0x7fedf00b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:30.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedf681e700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fedf00977d0 0x7fedf0097c60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fede800b000 tx=0x7fede8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:30.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedef7fe700 1 -- 172.21.15.140:0/1244275208 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fede8010ea0 con 0x7fedf00977d0 2022-07-21T01:55:30.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedef7fe700 1 -- 172.21.15.140:0/1244275208 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede8012540 con 0x7fedf00977d0 2022-07-21T01:55:30.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedf8281700 1 -- 172.21.15.140:0/1244275208 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf00b4f80 con 0x7fedf00977d0 2022-07-21T01:55:30.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedef7fe700 1 -- 172.21.15.140:0/1244275208 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fede80128a0 con 0x7fedf00977d0 2022-07-21T01:55:30.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedf8281700 1 -- 172.21.15.140:0/1244275208 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf0099160 con 0x7fedf00977d0 2022-07-21T01:55:30.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedf8281700 1 -- 172.21.15.140:0/1244275208 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf005feb0 con 0x7fedf00977d0 2022-07-21T01:55:30.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedef7fe700 1 -- 172.21.15.140:0/1244275208 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fede801c030 con 0x7fedf00977d0 2022-07-21T01:55:30.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedef7fe700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fedd0053b30 0x7fedd0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:30.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedef7fe700 1 -- 172.21.15.140:0/1244275208 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fede806baf0 con 0x7fedf00977d0 2022-07-21T01:55:30.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.512+0000 7fedf601d700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fedd0053b30 0x7fedd0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:30.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.516+0000 7fedf601d700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fedd0053b30 0x7fedd0055ff0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fede400d000 tx=0x7fede4007040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:30.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.520+0000 7fedef7fe700 1 -- 172.21.15.140:0/1244275208 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fede80428a0 con 0x7fedf00977d0 2022-07-21T01:55:30.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 -- 172.21.15.140:0/1244275208 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fedf00574e0 con 0x7fedf00977d0 2022-07-21T01:55:30.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedef7fe700 1 -- 172.21.15.140:0/1244275208 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v84) v1 ==== 81+0+33744 (secure 0 0 0) 0x7fede8045ec0 con 0x7fedf00977d0 2022-07-21T01:55:30.645 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:30.645 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:55:30.645 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 84, 2022-07-21T01:55:30.646 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:55:30.647 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:55:30.647 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:55:21.510210+0000", 2022-07-21T01:55:30.648 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:55:30.648 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:55:30.648 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:55:30.648 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:55:30.649 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:55:30.649 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:55:30.649 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:55:30.649 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:55:30.650 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:55:30.650 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:30.650 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:55:30.651 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:55:30.651 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:55:30.652 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:55:30.652 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:55:30.652 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 16, 2022-07-21T01:55:30.653 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:55:30.653 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:55:30.653 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:55:30.654 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:55:30.654 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:55:30.654 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.655 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:55:30.655 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:55:30.655 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:55:30.655 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:30.656 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:30.656 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:30.656 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:30.656 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:30.657 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:30.657 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:30.657 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:30.657 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:55:30.658 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:55:30.658 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:55:30.658 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:55:30.659 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:55:30.659 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:30.659 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:30.659 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:30.659 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:30.660 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:30.660 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:30.660 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:30.660 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.661 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:55:30.661 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:30.661 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:30.662 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:30.662 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:30.663 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:30.663 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:30.663 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:30.664 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:30.664 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:30.664 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:30.665 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:30.665 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:30.665 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:30.666 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:30.666 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:30.666 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:30.667 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:30.667 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:30.667 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:30.667 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:30.668 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:30.668 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:30.668 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:30.668 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:30.669 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:30.669 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:30.669 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.669 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:30.670 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:30.670 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:30.670 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:30.670 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:30.671 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:30.671 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:30.671 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:30.671 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:30.672 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:30.672 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:30.673 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:55:30.673 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:55:30.673 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.674 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:30.674 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:55:30.674 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.675 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.675 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.676 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:55:30.676 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:55:30.676 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:55:30.676 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:30.677 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:30.677 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:30.677 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:30.677 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:30.678 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:30.678 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:30.679 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:30.679 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:55:30.679 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:55:30.680 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:55:30.680 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:55:30.680 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:55:30.680 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:30.681 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:30.681 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:30.681 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:30.681 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:30.682 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:30.682 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:30.682 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.683 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:55:30.683 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:30.683 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:30.684 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:30.684 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:30.684 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:30.685 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:30.685 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:30.685 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:30.686 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:30.686 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:30.686 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:30.687 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:30.687 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:30.687 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:30.687 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:30.688 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:30.688 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:30.688 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:30.688 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:30.689 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:30.689 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:30.689 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:30.689 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:30.690 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:30.690 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:30.690 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:30.690 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.691 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:30.691 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:30.691 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:30.692 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:30.692 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:30.692 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:30.692 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:30.693 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:30.693 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:30.693 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:30.693 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:30.694 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:55:30.694 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:30.695 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:55:30.695 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.696 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.696 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.696 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 15, 2022-07-21T01:55:30.697 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:55:30.697 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:55:18.595416+0000", 2022-07-21T01:55:30.697 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:30.698 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:30.698 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:30.698 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:30.699 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:30.699 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:30.699 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:30.699 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:30.700 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:55:30.700 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:55:30.700 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:55:30.700 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:55:30.700 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:55:30.701 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:30.701 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:30.701 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:30.701 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:30.701 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:30.702 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:30.702 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:30.702 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.703 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "83", 2022-07-21T01:55:30.703 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:30.703 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:30.703 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:30.704 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:30.704 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:30.704 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:30.704 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:30.704 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:30.705 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:30.705 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:30.705 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:30.706 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:30.706 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:30.706 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:30.706 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:30.707 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:30.707 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:30.707 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:30.708 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:30.708 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:30.708 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:30.708 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:30.709 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:30.709 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:30.709 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:30.709 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:30.709 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.710 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:30.710 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:30.710 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:30.710 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:30.711 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:30.711 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:30.711 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:30.711 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:30.712 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:30.712 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:30.712 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:30.712 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:55:30.712 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:55:30.713 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:55:30.713 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:55:30.713 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.713 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:30.714 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:55:30.714 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:55:30.714 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.714 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.715 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.715 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.715 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 16, 2022-07-21T01:55:30.715 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:55:30.716 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:55:19.516058+0000", 2022-07-21T01:55:30.716 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:30.716 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:30.716 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:30.717 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:30.717 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:30.717 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:30.717 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:30.718 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:30.718 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:55:30.718 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:55:30.719 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:55:30.719 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:55:30.719 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:55:30.720 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:30.720 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:30.720 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:30.720 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:30.721 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:30.721 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:30.721 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:30.721 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.721 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "84", 2022-07-21T01:55:30.722 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:30.722 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:30.722 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:30.722 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:30.723 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:30.723 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:30.723 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:30.723 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:30.724 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:30.724 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:30.724 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:30.724 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:30.724 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:30.725 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:30.725 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:30.725 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:30.725 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:30.726 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:30.726 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:30.726 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:30.726 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:30.727 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:30.727 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:30.727 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:30.727 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:30.728 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:30.728 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.728 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:30.728 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:30.728 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:30.729 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:30.729 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:30.729 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:30.729 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:30.729 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:30.730 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:30.730 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:30.730 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:30.730 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:55:30.731 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:30.731 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:55:30.731 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:55:30.731 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.731 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.732 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.732 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:30.732 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:55:30.732 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.733 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:55:30.733 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:55:30.733 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:30.733 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:30.734 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:30.734 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:30.734 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:30.734 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:30.735 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:30.735 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:30.735 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:30.735 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:30.736 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:30.736 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.736 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.736 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.736 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:55:30.737 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:30.737 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.737 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.737 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.738 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:55:30.738 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:30.738 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.738 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.738 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.739 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:30.739 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.739 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.739 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.740 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:55:30.740 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:30.740 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.740 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.741 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.741 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:55:30.741 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:30.741 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.742 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.742 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.742 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:30.742 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.743 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.743 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.743 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:55:30.743 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:30.743 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.744 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.744 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.744 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:55:30.744 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:30.745 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.745 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.745 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.745 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:30.745 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.746 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.746 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.746 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:55:30.746 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:30.747 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.747 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.747 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.747 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:55:30.747 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:30.748 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.748 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.748 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.748 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:55:30.749 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:55:30.749 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:55:30.749 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:55:30.749 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:30.749 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:30.750 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:30.750 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.750 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.750 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.751 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:55:30.751 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:55:30.751 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:30.751 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:30.752 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:30.752 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:30.752 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:30.752 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:30.752 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:30.753 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 81, 2022-07-21T01:55:30.753 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:30.753 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:30.753 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:30.754 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.754 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.754 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.754 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:55:30.755 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:30.755 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.755 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.755 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.755 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:55:30.756 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:30.756 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.756 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.756 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.757 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:30.757 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.757 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.757 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.758 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:55:30.758 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:30.758 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.759 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.759 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.759 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:55:30.759 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:30.760 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.760 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.760 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.760 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:30.760 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.761 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.761 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.761 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:55:30.761 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:30.762 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.762 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.762 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.762 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:55:30.762 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:30.763 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.763 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.763 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.763 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:30.764 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.764 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.764 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.764 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:55:30.764 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:30.765 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.765 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.765 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.765 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:55:30.765 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:30.766 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.766 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.766 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.767 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:55:30.767 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:55:30.767 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:55:30.767 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:55:30.768 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:30.768 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:30.768 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:30.769 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.769 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.769 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.769 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:55:30.770 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:55:30.770 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:30.770 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:30.770 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:30.771 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:30.771 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:30.771 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:30.771 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:30.771 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:30.772 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:30.772 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:30.772 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:30.772 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.773 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.773 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.773 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:55:30.773 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:30.773 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.774 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.774 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.774 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:55:30.774 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:30.775 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.775 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.775 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.775 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:30.775 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.776 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.776 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.776 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:55:30.776 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:30.777 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.777 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.777 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.777 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:55:30.778 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:30.778 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.778 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.779 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.779 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:30.779 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.779 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.780 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.780 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:55:30.780 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:30.780 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.781 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.781 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.781 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:55:30.781 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:30.781 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.782 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.782 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.782 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:30.782 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.783 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.783 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.783 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:55:30.783 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:30.783 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.784 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.784 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.784 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:55:30.784 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:30.785 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.785 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.785 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.785 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:55:30.785 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:55:30.786 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:55:30.786 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:55:30.786 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:30.786 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:30.787 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:30.787 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.787 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.787 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.788 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:55:30.788 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:55:30.788 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:30.788 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:30.789 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:30.789 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:30.789 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:30.789 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:30.790 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:30.790 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:30.790 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:30.790 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:30.791 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:30.791 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.791 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.791 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.791 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:55:30.792 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:30.792 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.792 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.792 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.792 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:55:30.793 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:30.793 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.793 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.793 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.794 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:30.794 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.794 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.794 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.794 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:55:30.795 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:30.795 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.795 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.795 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.796 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:55:30.796 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:30.796 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.797 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.797 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.797 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:30.797 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.797 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.798 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.798 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:55:30.798 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:30.799 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.799 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.799 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.799 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:55:30.799 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:30.800 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.800 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.800 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.800 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:30.801 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.801 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.801 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.801 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:55:30.801 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:30.802 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.802 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.802 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.802 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:55:30.803 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:30.803 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.803 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.803 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.803 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:55:30.804 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:55:30.804 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:55:30.804 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:55:30.804 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:30.805 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:30.805 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:30.805 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.805 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.805 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.806 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:55:30.806 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:55:30.806 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:30.806 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:30.806 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:30.807 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:30.807 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:30.807 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:30.807 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:30.808 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:30.808 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:30.808 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:30.808 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:30.808 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.809 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.809 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.809 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:55:30.809 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:30.810 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.810 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.810 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.810 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:55:30.811 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:30.811 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.811 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.812 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.812 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:30.812 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.812 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.812 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.813 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:55:30.813 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:30.813 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.813 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.814 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.814 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:55:30.814 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:30.814 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.815 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.815 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.815 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:30.815 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.816 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.816 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.816 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:55:30.817 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:30.817 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.817 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.817 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.818 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:55:30.818 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:30.818 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.818 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.819 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.819 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:30.819 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.819 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.820 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.820 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:55:30.820 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:30.820 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.821 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.821 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.821 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:55:30.821 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:30.821 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.822 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.822 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.822 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:55:30.822 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:55:30.823 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:55:30.823 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:55:30.823 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:30.823 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:30.824 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:30.824 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.824 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.824 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.825 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:55:30.825 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:55:30.825 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:30.825 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:30.825 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:30.826 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:30.826 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:30.826 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:30.826 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:30.827 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:30.827 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:30.827 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:30.827 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:30.828 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.828 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.828 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.829 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:55:30.829 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:30.829 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.829 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.829 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.830 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:55:30.830 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:30.830 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.830 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.831 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.831 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:30.831 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.831 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.832 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.832 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:55:30.832 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:30.832 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.832 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.833 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.833 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:55:30.833 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:30.833 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.834 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.834 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.834 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:30.834 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.834 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.835 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.835 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:55:30.835 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:30.835 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.836 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.836 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.836 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:55:30.836 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:30.837 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.837 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.837 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.837 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:30.837 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.838 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.838 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.838 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:55:30.838 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:30.839 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.839 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.839 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.839 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:55:30.839 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:30.840 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.840 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.840 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.840 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:55:30.840 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:55:30.841 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:55:30.841 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:55:30.841 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:30.841 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:30.842 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:30.842 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.842 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.842 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.842 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:55:30.843 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:55:30.843 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:30.843 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:30.843 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:30.844 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:30.844 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:30.844 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:30.844 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:30.844 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:30.845 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:30.845 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:30.845 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:30.845 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.845 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.846 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.846 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:55:30.846 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:30.846 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.847 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.847 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.847 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:55:30.847 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:30.847 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.848 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.848 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.848 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:30.848 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.849 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.849 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.849 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:55:30.849 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:30.849 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.850 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.850 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.850 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:55:30.850 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:30.851 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.851 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.851 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.852 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:30.852 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.852 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.852 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.852 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:55:30.853 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:30.853 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.853 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.853 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.854 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:55:30.854 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:30.854 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.854 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.854 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.855 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:30.855 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.855 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.855 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.856 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:55:30.856 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:30.856 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.856 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.856 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.857 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:55:30.857 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:30.857 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.857 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.857 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.858 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:55:30.858 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:55:30.858 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:55:30.858 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:55:30.859 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:30.859 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:30.859 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:30.859 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.860 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.860 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.860 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:55:30.860 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:55:30.861 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:30.861 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:30.861 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:30.862 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:30.862 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:30.862 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:30.862 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:30.863 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 68, 2022-07-21T01:55:30.863 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:30.863 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:30.863 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:30.864 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.864 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.864 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.864 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:55:30.864 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:30.865 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.865 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.865 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.865 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:55:30.866 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:30.866 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.866 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.866 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.866 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:30.867 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.867 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.867 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.867 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:55:30.868 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:30.868 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.868 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.868 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.868 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:55:30.869 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:30.869 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.869 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.869 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.870 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:30.870 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.870 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.870 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.870 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:55:30.871 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:30.871 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.871 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.872 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.872 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:55:30.872 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:30.872 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.873 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.873 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.873 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:30.873 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:30.873 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.874 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:30.874 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:55:30.874 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:30.874 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.875 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.875 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:30.875 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:55:30.875 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:30.875 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.876 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.876 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.876 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:55:30.876 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:55:30.877 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:55:30.877 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:55:30.877 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:30.877 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:30.877 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:30.878 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:30.878 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.878 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:30.878 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:55:30.879 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.879 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:55:30.879 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:30.879 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:30.879 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:30.880 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:30.880 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:30.880 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:55:30.880 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:30.881 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.881 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.881 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:55:30.881 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:30.882 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:30.882 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:30.882 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:30.882 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:30.883 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:55:30.883 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:30.883 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.883 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.884 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:55:30.884 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:30.884 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:30.885 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:30.885 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:30.885 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:30.885 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:55:30.885 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:30.886 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.886 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.886 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:55:30.886 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:30.887 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:30.887 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:30.887 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:30.887 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:30.887 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:55:30.888 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:30.888 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.888 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.888 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:55:30.889 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:30.889 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:30.889 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:30.889 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:30.890 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:30.890 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:55:30.890 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:30.890 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.891 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.891 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:55:30.891 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:30.891 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:30.892 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:30.892 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:30.892 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:30.893 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:55:30.893 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:30.894 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.894 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.894 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:55:30.895 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:30.895 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:30.895 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:30.896 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:30.896 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:30.897 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:55:30.897 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:30.897 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.897 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:30.897 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:55:30.898 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:30.898 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:30.898 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:30.898 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:30.899 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:30.899 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:55:30.899 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:30.899 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.899 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:30.900 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:55:30.900 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:55:30.900 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:55:30.900 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:55:30.901 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": {}, 2022-07-21T01:55:30.901 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:55:30.901 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:55:30.901 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:55:30.901 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:55:30.902 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:55:30.902 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:55:30.902 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:55:30.902 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:55:30.903 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:30.903 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:30.903 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:55:30.903 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:55:30.904 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:55:30.904 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:55:30.904 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:55:30.904 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:55:30.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 -- 172.21.15.140:0/1244275208 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fedd0053b30 msgr2=0x7fedd0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:30.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fedd0053b30 0x7fedd0055ff0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fede400d000 tx=0x7fede4007040).stop 2022-07-21T01:55:30.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 -- 172.21.15.140:0/1244275208 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fedf00977d0 msgr2=0x7fedf0097c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:30.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fedf00977d0 0x7fedf0097c60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fede800b000 tx=0x7fede8008040).stop 2022-07-21T01:55:30.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 -- 172.21.15.140:0/1244275208 shutdown_connections 2022-07-21T01:55:30.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fedd0053b30 0x7fedd0055ff0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:30.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fedf00977d0 0x7fedf0097c60 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:30.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fedf009a740 0x7fedf00b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:30.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 --2- 172.21.15.140:0/1244275208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feddc004500 0x7fedf009bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:30.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 -- 172.21.15.140:0/1244275208 >> 172.21.15.140:0/1244275208 conn(0x7fedf00b3690 msgr2=0x7fedf0145990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:30.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 -- 172.21.15.140:0/1244275208 shutdown_connections 2022-07-21T01:55:30.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:30.644+0000 7fedf8281700 1 -- 172.21.15.140:0/1244275208 wait complete. 2022-07-21T01:55:30.911 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:31.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9f59e700 1 Processor -- start 2022-07-21T01:55:31.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9f59e700 1 -- start start 2022-07-21T01:55:31.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9f59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0ba00adff0 0x7f0ba00b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:31.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9f59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba0147010 con 0x7f0ba00af8e0 2022-07-21T01:55:31.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9f59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba01472b0 con 0x7f0ba00b5650 2022-07-21T01:55:31.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9f59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba0146d40 con 0x7f0ba00adff0 2022-07-21T01:55:31.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9ed9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0ba00af8e0 0x7f0ba005bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51886/0 (socket says 172.21.15.140:51886) 2022-07-21T01:55:31.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9ed9d700 1 -- 172.21.15.140:0/1592457800 learned_addr learned my addr 172.21.15.140:0/1592457800 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:31.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9dd9b700 1 --2- 172.21.15.140:0/1592457800 >> v2:172.21.15.152:3300/0 conn(0x7f0ba00adff0 0x7f0ba00b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:31.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9d59a700 1 -- 172.21.15.140:0/1592457800 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507926873 0 0) 0x7f0ba0147010 con 0x7f0ba00af8e0 2022-07-21T01:55:31.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9d59a700 1 -- 172.21.15.140:0/1592457800 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b84003260 con 0x7f0ba00af8e0 2022-07-21T01:55:31.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.020+0000 7f0b9d59a700 1 -- 172.21.15.140:0/1592457800 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542493210 0 0) 0x7f0ba01472b0 con 0x7f0ba00b5650 2022-07-21T01:55:31.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9d59a700 1 -- 172.21.15.140:0/1592457800 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ba0147010 con 0x7f0ba00b5650 2022-07-21T01:55:31.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9d59a700 1 -- 172.21.15.140:0/1592457800 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 804921148 0 0) 0x7f0b84003260 con 0x7f0ba00af8e0 2022-07-21T01:55:31.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9dd9b700 1 -- 172.21.15.140:0/1592457800 >> v1:172.21.15.152:6789/0 conn(0x7f0ba00b5650 legacy=0x7f0ba0147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:31.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9dd9b700 1 -- 172.21.15.140:0/1592457800 >> v1:172.21.15.140:6789/0 conn(0x7f0ba00af8e0 legacy=0x7f0ba005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:31.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9dd9b700 1 -- 172.21.15.140:0/1592457800 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba005c4a0 con 0x7f0ba00adff0 2022-07-21T01:55:31.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9dd9b700 1 --2- 172.21.15.140:0/1592457800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba00adff0 0x7f0ba00b5a30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f0b90001110 tx=0x7f0b94006d90).ready entity=mon.1 client_cookie=7a80b485459644fe server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:31.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9d59a700 1 -- 172.21.15.140:0/1592457800 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b90001660 con 0x7f0ba00adff0 2022-07-21T01:55:31.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9d59a700 1 -- 172.21.15.140:0/1592457800 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b94000e90 con 0x7f0ba00adff0 2022-07-21T01:55:31.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9d59a700 1 -- 172.21.15.140:0/1592457800 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b9400beb0 con 0x7f0ba00adff0 2022-07-21T01:55:31.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1592457800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba00adff0 msgr2=0x7f0ba00b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:31.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9f59e700 1 --2- 172.21.15.140:0/1592457800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba00adff0 0x7f0ba00b5a30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f0b90001110 tx=0x7f0b94006d90).stop 2022-07-21T01:55:31.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1592457800 shutdown_connections 2022-07-21T01:55:31.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9f59e700 1 --2- 172.21.15.140:0/1592457800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba00adff0 0x7f0ba00b5a30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1592457800 >> 172.21.15.140:0/1592457800 conn(0x7f0ba00634e0 msgr2=0x7f0ba00b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:31.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1592457800 shutdown_connections 2022-07-21T01:55:31.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1592457800 wait complete. 2022-07-21T01:55:31.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9f59e700 1 Processor -- start 2022-07-21T01:55:31.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.024+0000 7f0b9f59e700 1 -- start start 2022-07-21T01:55:31.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ba00adff0 0x7f0ba01455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:31.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9f59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba00af8e0 0x7f0ba010f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:31.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9f59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ba00b5650 0x7f0ba010f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:31.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9f59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba0144420 con 0x7f0ba00b5650 2022-07-21T01:55:31.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9f59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba0066470 con 0x7f0ba00af8e0 2022-07-21T01:55:31.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba0144600 con 0x7f0ba00adff0 2022-07-21T01:55:31.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ba00b5650 0x7f0ba010f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:31.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9e59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ba00adff0 0x7f0ba01455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:31.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9dd9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba00af8e0 0x7f0ba010f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:31.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ba00b5650 0x7f0ba010f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37746/0 (socket says 172.21.15.140:37746) 2022-07-21T01:55:31.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9ed9d700 1 -- 172.21.15.140:0/1684009053 learned_addr learned my addr 172.21.15.140:0/1684009053 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:31.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9ed9d700 1 -- 172.21.15.140:0/1684009053 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ba00adff0 msgr2=0x7f0ba01455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:31.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9ed9d700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ba00adff0 0x7f0ba01455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9ed9d700 1 -- 172.21.15.140:0/1684009053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba00af8e0 msgr2=0x7f0ba010f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:31.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9ed9d700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba00af8e0 0x7f0ba010f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9ed9d700 1 -- 172.21.15.140:0/1684009053 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9400b120 con 0x7f0ba00b5650 2022-07-21T01:55:31.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9ed9d700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ba00b5650 0x7f0ba010f5b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f0ba0110a40 tx=0x7f0b90001720).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:31.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b8f7fe700 1 -- 172.21.15.140:0/1684009053 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b90007170 con 0x7f0ba00b5650 2022-07-21T01:55:31.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b8f7fe700 1 -- 172.21.15.140:0/1684009053 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b900072d0 con 0x7f0ba00b5650 2022-07-21T01:55:31.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1684009053 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba0111530 con 0x7f0ba00b5650 2022-07-21T01:55:31.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b8f7fe700 1 -- 172.21.15.140:0/1684009053 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b90004710 con 0x7f0ba00b5650 2022-07-21T01:55:31.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.028+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1684009053 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba01112d0 con 0x7f0ba00b5650 2022-07-21T01:55:31.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.032+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1684009053 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba00574e0 con 0x7f0ba00b5650 2022-07-21T01:55:31.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.032+0000 7f0b8f7fe700 1 -- 172.21.15.140:0/1684009053 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0b9001b070 con 0x7f0ba00b5650 2022-07-21T01:55:31.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.032+0000 7f0b8f7fe700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b7c053950 0x7f0b7c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:31.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.032+0000 7f0b8f7fe700 1 -- 172.21.15.140:0/1684009053 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0b9006b430 con 0x7f0ba00b5650 2022-07-21T01:55:31.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.032+0000 7f0b9e59c700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b7c053950 0x7f0b7c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:31.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.032+0000 7f0b9e59c700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b7c053950 0x7f0b7c055e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f0b9800d000 tx=0x7f0b98007da0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:31.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.040+0000 7f0b8f7fe700 1 -- 172.21.15.140:0/1684009053 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b90042350 con 0x7f0ba00b5650 2022-07-21T01:55:31.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1684009053 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ba00adc10 con 0x7f0ba00b5650 2022-07-21T01:55:31.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b8f7fe700 1 -- 172.21.15.140:0/1684009053 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+2146 (secure 0 0 0) 0x7f0b90045970 con 0x7f0ba00b5650 2022-07-21T01:55:31.201 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:31.201 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":36,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/2489766347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2489766347},{"type":"v1","addr":"172.21.15.152:6833","nonce":2489766347}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:20.506956+0000","modified":"2022-07-21T01:55:21.509498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6586},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6586":{"gid":6586,"name":"a","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2461699636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2461699636},{"type":"v1","addr":"172.21.15.140:6835","nonce":2461699636}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-07-21T01:55:31.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1684009053 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b7c053950 msgr2=0x7f0b7c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:31.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b7c053950 0x7f0b7c055e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f0b9800d000 tx=0x7f0b98007da0).stop 2022-07-21T01:55:31.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1684009053 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ba00b5650 msgr2=0x7f0ba010f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:31.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ba00b5650 0x7f0ba010f5b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f0ba0110a40 tx=0x7f0b90001720).stop 2022-07-21T01:55:31.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1684009053 shutdown_connections 2022-07-21T01:55:31.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b7c053950 0x7f0b7c055e10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ba00b5650 0x7f0ba010f5b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ba00af8e0 0x7f0ba010f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 --2- 172.21.15.140:0/1684009053 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ba00adff0 0x7f0ba01455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1684009053 >> 172.21.15.140:0/1684009053 conn(0x7f0ba00634e0 msgr2=0x7f0ba00ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:31.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1684009053 shutdown_connections 2022-07-21T01:55:31.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.200+0000 7f0b9f59e700 1 -- 172.21.15.140:0/1684009053 wait complete. 2022-07-21T01:55:31.208 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 36 2022-07-21T01:55:31.213 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:55:31.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.316+0000 7f42422c6700 1 Processor -- start 2022-07-21T01:55:31.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.316+0000 7f42422c6700 1 -- start start 2022-07-21T01:55:31.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f42422c6700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f423c0ab250 0x7f423c0ac080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:31.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f42422c6700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f423c0b2770 0x7f423c14a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:31.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f42422c6700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423c05cd40 con 0x7f423c05c960 2022-07-21T01:55:31.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f42422c6700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c062030 con 0x7f423c0ab250 2022-07-21T01:55:31.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f42422c6700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c061600 con 0x7f423c0b2770 2022-07-21T01:55:31.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f423c0ab250 0x7f423c0ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:31.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f423c05c960 0x7f423c0ab970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51896/0 (socket says 172.21.15.140:51896) 2022-07-21T01:55:31.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423bfff700 1 -- 172.21.15.140:0/1682869539 learned_addr learned my addr 172.21.15.140:0/1682869539 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:31.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f4240863700 1 --2- 172.21.15.140:0/1682869539 >> v2:172.21.15.152:3300/0 conn(0x7f423c0b2770 0x7f423c14a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:31.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423affd700 1 -- 172.21.15.140:0/1682869539 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879293333 0 0) 0x7f423c05cd40 con 0x7f423c05c960 2022-07-21T01:55:31.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423affd700 1 -- 172.21.15.140:0/1682869539 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4228003260 con 0x7f423c05c960 2022-07-21T01:55:31.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423b7fe700 1 -- 172.21.15.140:0/1682869539 >> v2:172.21.15.152:3300/0 conn(0x7f423c0b2770 msgr2=0x7f423c14a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:31.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423b7fe700 1 --2- 172.21.15.140:0/1682869539 >> v2:172.21.15.152:3300/0 conn(0x7f423c0b2770 0x7f423c14a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423b7fe700 1 -- 172.21.15.140:0/1682869539 >> v1:172.21.15.140:6789/0 conn(0x7f423c05c960 legacy=0x7f423c0ab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:31.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423b7fe700 1 -- 172.21.15.140:0/1682869539 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c061b90 con 0x7f423c0ab250 2022-07-21T01:55:31.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423b7fe700 1 --2- 172.21.15.140:0/1682869539 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f423c0ab250 0x7f423c0ac080 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f422c0032f0 tx=0x7f422c0091d0).ready entity=mon.0 client_cookie=f99df1147a08ff9b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:31.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423affd700 1 -- 172.21.15.140:0/1682869539 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f422c00b600 con 0x7f423c0ab250 2022-07-21T01:55:31.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423affd700 1 -- 172.21.15.140:0/1682869539 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422c00b760 con 0x7f423c0ab250 2022-07-21T01:55:31.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f423affd700 1 -- 172.21.15.140:0/1682869539 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f422c00b8d0 con 0x7f423c0ab250 2022-07-21T01:55:31.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.320+0000 7f42422c6700 1 -- 172.21.15.140:0/1682869539 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f423c0ab250 msgr2=0x7f423c0ac080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:31.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.324+0000 7f42422c6700 1 --2- 172.21.15.140:0/1682869539 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f423c0ab250 0x7f423c0ac080 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f422c0032f0 tx=0x7f422c0091d0).stop 2022-07-21T01:55:31.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.324+0000 7f42422c6700 1 -- 172.21.15.140:0/1682869539 shutdown_connections 2022-07-21T01:55:31.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.324+0000 7f42422c6700 1 --2- 172.21.15.140:0/1682869539 >> v2:172.21.15.152:3300/0 conn(0x7f423c0b2770 0x7f423c14a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.324+0000 7f42422c6700 1 --2- 172.21.15.140:0/1682869539 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f423c0ab250 0x7f423c0ac080 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.324+0000 7f42422c6700 1 -- 172.21.15.140:0/1682869539 >> 172.21.15.140:0/1682869539 conn(0x7f423c0b2c50 msgr2=0x7f423c0b3030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:31.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.324+0000 7f42422c6700 1 -- 172.21.15.140:0/1682869539 shutdown_connections 2022-07-21T01:55:31.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.324+0000 7f42422c6700 1 -- 172.21.15.140:0/1682869539 wait complete. 2022-07-21T01:55:31.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.324+0000 7f42422c6700 1 Processor -- start 2022-07-21T01:55:31.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.324+0000 7f42422c6700 1 -- start start 2022-07-21T01:55:31.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42422c6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f423c05c960 0x7f423c1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:31.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42422c6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f423c0ab250 0x7f423c148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:31.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42422c6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f423c0b2770 0x7f423c0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:31.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42422c6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c097770 con 0x7f423c05c960 2022-07-21T01:55:31.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42422c6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c095880 con 0x7f423c0b2770 2022-07-21T01:55:31.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42422c6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c095b20 con 0x7f423c0ab250 2022-07-21T01:55:31.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f423bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f423c05c960 0x7f423c1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:31.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f423bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f423c05c960 0x7f423c1597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37756/0 (socket says 172.21.15.140:37756) 2022-07-21T01:55:31.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f423bfff700 1 -- 172.21.15.140:0/2915598094 learned_addr learned my addr 172.21.15.140:0/2915598094 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:31.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f423b7fe700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f423c0ab250 0x7f423c148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:31.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f4240863700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f423c0b2770 0x7f423c0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:31.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f423bfff700 1 -- 172.21.15.140:0/2915598094 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f423c0ab250 msgr2=0x7f423c148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:31.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f423bfff700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f423c0ab250 0x7f423c148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f423bfff700 1 -- 172.21.15.140:0/2915598094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f423c0b2770 msgr2=0x7f423c0b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:31.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f423bfff700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f423c0b2770 0x7f423c0b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:31.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f423bfff700 1 -- 172.21.15.140:0/2915598094 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c009470 con 0x7f423c05c960 2022-07-21T01:55:31.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f423bfff700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f423c05c960 0x7f423c1597d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f4230001110 tx=0x7f4230007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:31.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42397fa700 1 -- 172.21.15.140:0/2915598094 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4230007e10 con 0x7f423c05c960 2022-07-21T01:55:31.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42397fa700 1 -- 172.21.15.140:0/2915598094 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42300072e0 con 0x7f423c05c960 2022-07-21T01:55:31.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42397fa700 1 -- 172.21.15.140:0/2915598094 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4230005800 con 0x7f423c05c960 2022-07-21T01:55:31.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42422c6700 1 -- 172.21.15.140:0/2915598094 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c0b04a0 con 0x7f423c05c960 2022-07-21T01:55:31.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42422c6700 1 -- 172.21.15.140:0/2915598094 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c148d00 con 0x7f423c05c960 2022-07-21T01:55:31.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.328+0000 7f42422c6700 1 -- 172.21.15.140:0/2915598094 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c061330 con 0x7f423c05c960 2022-07-21T01:55:31.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.332+0000 7f42397fa700 1 -- 172.21.15.140:0/2915598094 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4230010070 con 0x7f423c05c960 2022-07-21T01:55:31.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.332+0000 7f42397fa700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f421c053b00 0x7f421c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:31.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.332+0000 7f42397fa700 1 -- 172.21.15.140:0/2915598094 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f423006b620 con 0x7f423c05c960 2022-07-21T01:55:31.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.332+0000 7f423b7fe700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f421c053b00 0x7f421c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:31.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.332+0000 7f423b7fe700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f421c053b00 0x7f421c055fc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f422c019000 tx=0x7f422c016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:31.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.340+0000 7f42397fa700 1 -- 172.21.15.140:0/2915598094 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4230042540 con 0x7f423c05c960 2022-07-21T01:55:31.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:31.500+0000 7f42422c6700 1 -- 172.21.15.140:0/2915598094 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f423c0574e0 con 0x7f423c05c960 2022-07-21T01:55:32.156 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:55:32.159 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:55:32.162 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:55:32.164 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:55:32.167 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:55:32.250 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:55:32.248+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:55:32.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.264+0000 7f42397fa700 1 -- 172.21.15.140:0/2915598094 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4230045b60 con 0x7f423c05c960 2022-07-21T01:55:32.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 -- 172.21.15.140:0/2915598094 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f421c053b00 msgr2=0x7f421c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f421c053b00 0x7f421c055fc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f422c019000 tx=0x7f422c016000).stop 2022-07-21T01:55:32.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 -- 172.21.15.140:0/2915598094 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f423c05c960 msgr2=0x7f423c1597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f423c05c960 0x7f423c1597d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f4230001110 tx=0x7f4230007ad0).stop 2022-07-21T01:55:32.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 -- 172.21.15.140:0/2915598094 shutdown_connections 2022-07-21T01:55:32.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f421c053b00 0x7f421c055fc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f423c0b2770 0x7f423c0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f423c0ab250 0x7f423c148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 --2- 172.21.15.140:0/2915598094 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f423c05c960 0x7f423c1597d0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 -- 172.21.15.140:0/2915598094 >> 172.21.15.140:0/2915598094 conn(0x7f423c0b2c50 msgr2=0x7f423c0989e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:32.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 -- 172.21.15.140:0/2915598094 shutdown_connections 2022-07-21T01:55:32.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.268+0000 7f42422c6700 1 -- 172.21.15.140:0/2915598094 wait complete. 2022-07-21T01:55:32.271 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:55:32.285 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:32.292 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:55:32.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 Processor -- start 2022-07-21T01:55:32.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 -- start start 2022-07-21T01:55:32.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fda540b1e00 0x7fda5414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fda5405c3f0 0x7fda540ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda5405bd90 0x7fda540accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5405d480 con 0x7fda540b1e00 2022-07-21T01:55:32.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda540ad1c0 con 0x7fda5405c3f0 2022-07-21T01:55:32.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda540b21e0 con 0x7fda5405bd90 2022-07-21T01:55:32.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda52ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fda540b1e00 0x7fda5414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda52ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fda540b1e00 0x7fda5414a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37776/0 (socket says 172.21.15.140:37776) 2022-07-21T01:55:32.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda52ffd700 1 -- 172.21.15.140:0/619042033 learned_addr learned my addr 172.21.15.140:0/619042033 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:32.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda527fc700 1 --2- 172.21.15.140:0/619042033 >> v2:172.21.15.152:3300/0 conn(0x7fda5405c3f0 0x7fda540ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda537fe700 1 --2- 172.21.15.140:0/619042033 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda5405bd90 0x7fda540accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda52ffd700 1 -- 172.21.15.140:0/619042033 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda5405bd90 msgr2=0x7fda540accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:32.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda52ffd700 1 --2- 172.21.15.140:0/619042033 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda5405bd90 0x7fda540accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda52ffd700 1 -- 172.21.15.140:0/619042033 >> v2:172.21.15.152:3300/0 conn(0x7fda5405c3f0 msgr2=0x7fda540ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:32.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda52ffd700 1 --2- 172.21.15.140:0/619042033 >> v2:172.21.15.152:3300/0 conn(0x7fda5405c3f0 0x7fda540ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda52ffd700 1 -- 172.21.15.140:0/619042033 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5405cf40 con 0x7fda540b1e00 2022-07-21T01:55:32.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda52ffd700 1 --2- 172.21.15.140:0/619042033 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda540b1e00 0x7fda5414a6d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7fda48002f70 tx=0x7fda48006eb0).ready entity=mon.0 client_cookie=43f66eac3fdf4e07 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:32.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda51ffb700 1 -- 172.21.15.140:0/619042033 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4800d570 con 0x7fda540b1e00 2022-07-21T01:55:32.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda51ffb700 1 -- 172.21.15.140:0/619042033 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda48000e90 con 0x7fda540b1e00 2022-07-21T01:55:32.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda51ffb700 1 -- 172.21.15.140:0/619042033 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda4800d900 con 0x7fda540b1e00 2022-07-21T01:55:32.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 -- 172.21.15.140:0/619042033 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda540b1e00 msgr2=0x7fda5414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 --2- 172.21.15.140:0/619042033 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda540b1e00 0x7fda5414a6d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7fda48002f70 tx=0x7fda48006eb0).stop 2022-07-21T01:55:32.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 -- 172.21.15.140:0/619042033 shutdown_connections 2022-07-21T01:55:32.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 --2- 172.21.15.140:0/619042033 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda5405bd90 0x7fda540accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 --2- 172.21.15.140:0/619042033 >> v2:172.21.15.152:3300/0 conn(0x7fda5405c3f0 0x7fda540ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 --2- 172.21.15.140:0/619042033 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda540b1e00 0x7fda5414a6d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 -- 172.21.15.140:0/619042033 >> 172.21.15.140:0/619042033 conn(0x7fda540611c0 msgr2=0x7fda54062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:32.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 -- 172.21.15.140:0/619042033 shutdown_connections 2022-07-21T01:55:32.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.388+0000 7fda593a0700 1 -- 172.21.15.140:0/619042033 wait complete. 2022-07-21T01:55:32.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda593a0700 1 Processor -- start 2022-07-21T01:55:32.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda593a0700 1 -- start start 2022-07-21T01:55:32.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda593a0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda5405c3f0 0x7fda540ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda593a0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda540b1e00 0x7fda5413d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda593a0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda54148000 0x7fda5413d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda593a0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda54063b90 con 0x7fda5405c3f0 2022-07-21T01:55:32.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda593a0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda541438f0 con 0x7fda540b1e00 2022-07-21T01:55:32.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda593a0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda54143650 con 0x7fda54148000 2022-07-21T01:55:32.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda52ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda5405c3f0 0x7fda540ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda52ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda5405c3f0 0x7fda540ab390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37782/0 (socket says 172.21.15.140:37782) 2022-07-21T01:55:32.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda52ffd700 1 -- 172.21.15.140:0/2665414120 learned_addr learned my addr 172.21.15.140:0/2665414120 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:32.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda537fe700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda54148000 0x7fda5413d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda527fc700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda540b1e00 0x7fda5413d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda52ffd700 1 -- 172.21.15.140:0/2665414120 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda54148000 msgr2=0x7fda5413d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda52ffd700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda54148000 0x7fda5413d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda52ffd700 1 -- 172.21.15.140:0/2665414120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda540b1e00 msgr2=0x7fda5413d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda52ffd700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda540b1e00 0x7fda5413d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda52ffd700 1 -- 172.21.15.140:0/2665414120 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4800b1e0 con 0x7fda5405c3f0 2022-07-21T01:55:32.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda52ffd700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda5405c3f0 0x7fda540ab390 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fda48002f70 tx=0x7fda48015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:32.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.392+0000 7fda43fff700 1 -- 172.21.15.140:0/2665414120 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda48011970 con 0x7fda5405c3f0 2022-07-21T01:55:32.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.396+0000 7fda43fff700 1 -- 172.21.15.140:0/2665414120 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda48011ad0 con 0x7fda5405c3f0 2022-07-21T01:55:32.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.396+0000 7fda43fff700 1 -- 172.21.15.140:0/2665414120 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fda48011e30 con 0x7fda5405c3f0 2022-07-21T01:55:32.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.396+0000 7fda593a0700 1 -- 172.21.15.140:0/2665414120 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5413dd80 con 0x7fda5405c3f0 2022-07-21T01:55:32.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.396+0000 7fda593a0700 1 -- 172.21.15.140:0/2665414120 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda54063ff0 con 0x7fda5405c3f0 2022-07-21T01:55:32.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.396+0000 7fda43fff700 1 -- 172.21.15.140:0/2665414120 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fda48025030 con 0x7fda5405c3f0 2022-07-21T01:55:32.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.396+0000 7fda593a0700 1 -- 172.21.15.140:0/2665414120 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda54063d00 con 0x7fda5405c3f0 2022-07-21T01:55:32.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.396+0000 7fda43fff700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda30053b00 0x7fda30055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.396+0000 7fda43fff700 1 -- 172.21.15.140:0/2665414120 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fda4806ce60 con 0x7fda5405c3f0 2022-07-21T01:55:32.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.396+0000 7fda527fc700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda30053b00 0x7fda30055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.396+0000 7fda527fc700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda30053b00 0x7fda30055fc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fda4400b000 tx=0x7fda440091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:32.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.400+0000 7fda43fff700 1 -- 172.21.15.140:0/2665414120 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fda48043d30 con 0x7fda5405c3f0 2022-07-21T01:55:32.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.560+0000 7fda593a0700 1 -- 172.21.15.140:0/2665414120 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda5405d1e0 con 0x7fda5405c3f0 2022-07-21T01:55:32.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda43fff700 1 -- 172.21.15.140:0/2665414120 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+1782 (secure 0 0 0) 0x7fda48023020 con 0x7fda5405c3f0 2022-07-21T01:55:32.564 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:32.565 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":37,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/2489766347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2489766347},{"type":"v1","addr":"172.21.15.152:6833","nonce":2489766347}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:20.506956+0000","modified":"2022-07-21T01:55:32.252438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2022-07-21T01:55:32.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 -- 172.21.15.140:0/2665414120 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda30053b00 msgr2=0x7fda30055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda30053b00 0x7fda30055fc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fda4400b000 tx=0x7fda440091d0).stop 2022-07-21T01:55:32.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 -- 172.21.15.140:0/2665414120 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda5405c3f0 msgr2=0x7fda540ab390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda5405c3f0 0x7fda540ab390 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7fda48002f70 tx=0x7fda48015000).stop 2022-07-21T01:55:32.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 -- 172.21.15.140:0/2665414120 shutdown_connections 2022-07-21T01:55:32.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fda30053b00 0x7fda30055fc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fda54148000 0x7fda5413d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda540b1e00 0x7fda5413d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 --2- 172.21.15.140:0/2665414120 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fda5405c3f0 0x7fda540ab390 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 -- 172.21.15.140:0/2665414120 >> 172.21.15.140:0/2665414120 conn(0x7fda5405bd90 msgr2=0x7fda541458a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:32.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 -- 172.21.15.140:0/2665414120 shutdown_connections 2022-07-21T01:55:32.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.564+0000 7fda593a0700 1 -- 172.21.15.140:0/2665414120 wait complete. 2022-07-21T01:55:32.572 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 37 2022-07-21T01:55:32.576 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:55:32.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.680+0000 7f55c5039700 1 Processor -- start 2022-07-21T01:55:32.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.680+0000 7f55c5039700 1 -- start start 2022-07-21T01:55:32.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c5039700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55c00ad0e0 0x7f55c00aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c5039700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c00ae280 con 0x7f55c00b3cb0 2022-07-21T01:55:32.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c5039700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c00b6be0 con 0x7f55c00b6800 2022-07-21T01:55:32.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c5039700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00ae4b0 con 0x7f55c00ad0e0 2022-07-21T01:55:32.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55bf7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f55c00b3cb0 0x7f55c01418f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51938/0 (socket says 172.21.15.140:51938) 2022-07-21T01:55:32.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55bf7fe700 1 -- 172.21.15.140:0/1415839169 learned_addr learned my addr 172.21.15.140:0/1415839169 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:32.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c4838700 1 --2- 172.21.15.140:0/1415839169 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55c00ad0e0 0x7f55c00aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55beffd700 1 -- 172.21.15.140:0/1415839169 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978999343 0 0) 0x7f55c00ae280 con 0x7f55c00b3cb0 2022-07-21T01:55:32.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55beffd700 1 -- 172.21.15.140:0/1415839169 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55ac003260 con 0x7f55c00b3cb0 2022-07-21T01:55:32.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55beffd700 1 -- 172.21.15.140:0/1415839169 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537583797 0 0) 0x7f55c00b6be0 con 0x7f55c00b6800 2022-07-21T01:55:32.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55beffd700 1 -- 172.21.15.140:0/1415839169 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55c00ae280 con 0x7f55c00b6800 2022-07-21T01:55:32.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55beffd700 1 -- 172.21.15.140:0/1415839169 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2358834553 0 0) 0x7f55ac003260 con 0x7f55c00b3cb0 2022-07-21T01:55:32.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c4838700 1 -- 172.21.15.140:0/1415839169 >> v1:172.21.15.152:6789/0 conn(0x7f55c00b6800 legacy=0x7f55c0145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c4838700 1 -- 172.21.15.140:0/1415839169 >> v1:172.21.15.140:6789/0 conn(0x7f55c00b3cb0 legacy=0x7f55c01418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c4838700 1 -- 172.21.15.140:0/1415839169 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c00ae750 con 0x7f55c00ad0e0 2022-07-21T01:55:32.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c4838700 1 --2- 172.21.15.140:0/1415839169 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55c00ad0e0 0x7f55c00aebf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f55b4001110 tx=0x7f55b80091d0).ready entity=mon.2 client_cookie=757bd62bd4662d79 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:32.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55beffd700 1 -- 172.21.15.140:0/1415839169 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55b4001660 con 0x7f55c00ad0e0 2022-07-21T01:55:32.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55beffd700 1 -- 172.21.15.140:0/1415839169 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b800f510 con 0x7f55c00ad0e0 2022-07-21T01:55:32.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55beffd700 1 -- 172.21.15.140:0/1415839169 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55b800f680 con 0x7f55c00ad0e0 2022-07-21T01:55:32.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c5039700 1 -- 172.21.15.140:0/1415839169 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55c00ad0e0 msgr2=0x7f55c00aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c5039700 1 --2- 172.21.15.140:0/1415839169 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55c00ad0e0 0x7f55c00aebf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f55b4001110 tx=0x7f55b80091d0).stop 2022-07-21T01:55:32.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c5039700 1 -- 172.21.15.140:0/1415839169 shutdown_connections 2022-07-21T01:55:32.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c5039700 1 --2- 172.21.15.140:0/1415839169 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55c00ad0e0 0x7f55c00aebf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c5039700 1 -- 172.21.15.140:0/1415839169 >> 172.21.15.140:0/1415839169 conn(0x7f55c00b9dd0 msgr2=0x7f55c00b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:32.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c5039700 1 -- 172.21.15.140:0/1415839169 shutdown_connections 2022-07-21T01:55:32.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.684+0000 7f55c5039700 1 -- 172.21.15.140:0/1415839169 wait complete. 2022-07-21T01:55:32.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55c5039700 1 Processor -- start 2022-07-21T01:55:32.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55c5039700 1 -- start start 2022-07-21T01:55:32.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55c5039700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f55c00ad0e0 0x7f55c00af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55c5039700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55c00b3cb0 0x7f55c013ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55c5039700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55c00b6800 0x7f55c013f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55c5039700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00b13b0 con 0x7f55c00ad0e0 2022-07-21T01:55:32.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55c5039700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00b26c0 con 0x7f55c00b6800 2022-07-21T01:55:32.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55c5039700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00b28f0 con 0x7f55c00b3cb0 2022-07-21T01:55:32.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f55c00ad0e0 0x7f55c00af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f55c00ad0e0 0x7f55c00af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37796/0 (socket says 172.21.15.140:37796) 2022-07-21T01:55:32.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bf7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55c00b3cb0 0x7f55c013ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bffff700 1 -- 172.21.15.140:0/2007813824 learned_addr learned my addr 172.21.15.140:0/2007813824 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:32.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55c4838700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55c00b6800 0x7f55c013f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bffff700 1 -- 172.21.15.140:0/2007813824 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55c00b3cb0 msgr2=0x7f55c013ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bffff700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55c00b3cb0 0x7f55c013ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bffff700 1 -- 172.21.15.140:0/2007813824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55c00b6800 msgr2=0x7f55c013f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:32.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bffff700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55c00b6800 0x7f55c013f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:32.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bffff700 1 -- 172.21.15.140:0/2007813824 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8009470 con 0x7f55c00ad0e0 2022-07-21T01:55:32.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bffff700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f55c00ad0e0 0x7f55c00af520 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f55b0001110 tx=0x7f55b0007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:32.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bd7fa700 1 -- 172.21.15.140:0/2007813824 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55b0012400 con 0x7f55c00ad0e0 2022-07-21T01:55:32.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bd7fa700 1 -- 172.21.15.140:0/2007813824 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b0012560 con 0x7f55c00ad0e0 2022-07-21T01:55:32.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55c5039700 1 -- 172.21.15.140:0/2007813824 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c00b2b90 con 0x7f55c00ad0e0 2022-07-21T01:55:32.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55bd7fa700 1 -- 172.21.15.140:0/2007813824 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f55b00128c0 con 0x7f55c00ad0e0 2022-07-21T01:55:32.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.688+0000 7f55c5039700 1 -- 172.21.15.140:0/2007813824 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c013f640 con 0x7f55c00ad0e0 2022-07-21T01:55:32.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.692+0000 7f55c5039700 1 -- 172.21.15.140:0/2007813824 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c00b5530 con 0x7f55c00ad0e0 2022-07-21T01:55:32.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.692+0000 7f55bd7fa700 1 -- 172.21.15.140:0/2007813824 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f55b0019030 con 0x7f55c00ad0e0 2022-07-21T01:55:32.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.692+0000 7f55bd7fa700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f55a0053a70 0x7f55a0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:32.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.692+0000 7f55bd7fa700 1 -- 172.21.15.140:0/2007813824 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f55b006b2a0 con 0x7f55c00ad0e0 2022-07-21T01:55:32.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.692+0000 7f55bf7fe700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f55a0053a70 0x7f55a0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:32.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.692+0000 7f55bf7fe700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f55a0053a70 0x7f55a0055f30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f55b400d000 tx=0x7f55b4000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:32.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.700+0000 7f55bd7fa700 1 -- 172.21.15.140:0/2007813824 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f55b00420d0 con 0x7f55c00ad0e0 2022-07-21T01:55:32.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:32.864+0000 7f55c5039700 1 -- 172.21.15.140:0/2007813824 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f55c00574e0 con 0x7f55c00ad0e0 2022-07-21T01:55:33.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.512+0000 7f55bd7fa700 1 -- 172.21.15.140:0/2007813824 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7f55b00456f0 con 0x7f55c00ad0e0 2022-07-21T01:55:33.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 -- 172.21.15.140:0/2007813824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f55a0053a70 msgr2=0x7f55a0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:33.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f55a0053a70 0x7f55a0055f30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f55b400d000 tx=0x7f55b4000bc0).stop 2022-07-21T01:55:33.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 -- 172.21.15.140:0/2007813824 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f55c00ad0e0 msgr2=0x7f55c00af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:33.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f55c00ad0e0 0x7f55c00af520 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f55b0001110 tx=0x7f55b0007ae0).stop 2022-07-21T01:55:33.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 -- 172.21.15.140:0/2007813824 shutdown_connections 2022-07-21T01:55:33.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f55a0053a70 0x7f55a0055f30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:33.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55c00b6800 0x7f55c013f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:33.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f55c00b3cb0 0x7f55c013ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:33.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 --2- 172.21.15.140:0/2007813824 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f55c00ad0e0 0x7f55c00af520 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:33.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 -- 172.21.15.140:0/2007813824 >> 172.21.15.140:0/2007813824 conn(0x7f55c00b9dd0 msgr2=0x7f55c00b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:33.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 -- 172.21.15.140:0/2007813824 shutdown_connections 2022-07-21T01:55:33.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.516+0000 7f55c5039700 1 -- 172.21.15.140:0/2007813824 wait complete. 2022-07-21T01:55:33.534 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:55:33.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.648+0000 7f8c089d0700 1 Processor -- start 2022-07-21T01:55:33.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.648+0000 7f8c089d0700 1 -- start start 2022-07-21T01:55:33.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c040b3cf0 0x7f8c04146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:33.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c04146530 0x7f8c04141a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:33.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c04066b10 con 0x7f8c040b6840 2022-07-21T01:55:33.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c040b5c90 con 0x7f8c04146530 2022-07-21T01:55:33.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040b6c20 con 0x7f8c040b3cf0 2022-07-21T01:55:33.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c03fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c04146530 0x7f8c04141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:33.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c03fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c04146530 0x7f8c04141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37808/0 (socket says 172.21.15.140:37808) 2022-07-21T01:55:33.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c03fff700 1 -- 172.21.15.140:0/3260301465 learned_addr learned my addr 172.21.15.140:0/3260301465 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:33.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c02ffd700 1 --2- 172.21.15.140:0/3260301465 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c040b3cf0 0x7f8c04146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:33.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c03fff700 1 -- 172.21.15.140:0/3260301465 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c040b3cf0 msgr2=0x7f8c04146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:33.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c03fff700 1 --2- 172.21.15.140:0/3260301465 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c040b3cf0 0x7f8c04146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:33.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c03fff700 1 -- 172.21.15.140:0/3260301465 >> v1:172.21.15.152:6789/0 conn(0x7f8c040b6840 legacy=0x7f8c040b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:33.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c03fff700 1 -- 172.21.15.140:0/3260301465 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c040aead0 con 0x7f8c04146530 2022-07-21T01:55:33.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c03fff700 1 --2- 172.21.15.140:0/3260301465 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c04146530 0x7f8c04141a90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f8bf4002f70 tx=0x7f8bf4006eb0).ready entity=mon.0 client_cookie=f44fdd7dfd714d29 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:33.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c027fc700 1 -- 172.21.15.140:0/3260301465 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bf400d5c0 con 0x7f8c04146530 2022-07-21T01:55:33.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c027fc700 1 -- 172.21.15.140:0/3260301465 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4000e90 con 0x7f8c04146530 2022-07-21T01:55:33.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c027fc700 1 -- 172.21.15.140:0/3260301465 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bf400d950 con 0x7f8c04146530 2022-07-21T01:55:33.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 -- 172.21.15.140:0/3260301465 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c04146530 msgr2=0x7f8c04141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:33.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 --2- 172.21.15.140:0/3260301465 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c04146530 0x7f8c04141a90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7f8bf4002f70 tx=0x7f8bf4006eb0).stop 2022-07-21T01:55:33.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 -- 172.21.15.140:0/3260301465 shutdown_connections 2022-07-21T01:55:33.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 --2- 172.21.15.140:0/3260301465 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c04146530 0x7f8c04141a90 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:33.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 --2- 172.21.15.140:0/3260301465 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c040b3cf0 0x7f8c04146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:33.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 -- 172.21.15.140:0/3260301465 >> 172.21.15.140:0/3260301465 conn(0x7f8c040b9e10 msgr2=0x7f8c040b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:33.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 -- 172.21.15.140:0/3260301465 shutdown_connections 2022-07-21T01:55:33.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.652+0000 7f8c089d0700 1 -- 172.21.15.140:0/3260301465 wait complete. 2022-07-21T01:55:33.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c089d0700 1 Processor -- start 2022-07-21T01:55:33.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c089d0700 1 -- start start 2022-07-21T01:55:33.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c089d0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c040b3cf0 0x7f8c04152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:33.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c089d0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c040b6840 0x7f8c040b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:33.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c089d0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c04146530 0x7f8c040b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:33.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c089d0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c041520f0 con 0x7f8c040b6840 2022-07-21T01:55:33.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c089d0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04152390 con 0x7f8c040b3cf0 2022-07-21T01:55:33.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c089d0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04152630 con 0x7f8c04146530 2022-07-21T01:55:33.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c02ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c040b6840 0x7f8c040b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:33.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c02ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c040b6840 0x7f8c040b8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37812/0 (socket says 172.21.15.140:37812) 2022-07-21T01:55:33.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c02ffd700 1 -- 172.21.15.140:0/3238038490 learned_addr learned my addr 172.21.15.140:0/3238038490 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:33.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c03fff700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c04146530 0x7f8c040b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:33.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c037fe700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c040b3cf0 0x7f8c04152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:33.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c02ffd700 1 -- 172.21.15.140:0/3238038490 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c04146530 msgr2=0x7f8c040b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:33.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c02ffd700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c04146530 0x7f8c040b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:33.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c02ffd700 1 -- 172.21.15.140:0/3238038490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c040b3cf0 msgr2=0x7f8c04152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:33.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c02ffd700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c040b3cf0 0x7f8c04152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:33.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c02ffd700 1 -- 172.21.15.140:0/3238038490 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf400b1e0 con 0x7f8c040b6840 2022-07-21T01:55:33.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c037fe700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c040b3cf0 0x7f8c04152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:33.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c02ffd700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c040b6840 0x7f8c040b8b30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f8bec00a000 tx=0x7f8bec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:33.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c00ff9700 1 -- 172.21.15.140:0/3238038490 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bec0106e0 con 0x7f8c040b6840 2022-07-21T01:55:33.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c00ff9700 1 -- 172.21.15.140:0/3238038490 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec010840 con 0x7f8c040b6840 2022-07-21T01:55:33.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c089d0700 1 -- 172.21.15.140:0/3238038490 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c040afbc0 con 0x7f8c040b6840 2022-07-21T01:55:33.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c00ff9700 1 -- 172.21.15.140:0/3238038490 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8bec010ba0 con 0x7f8c040b6840 2022-07-21T01:55:33.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.656+0000 7f8c089d0700 1 -- 172.21.15.140:0/3238038490 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c04155250 con 0x7f8c040b6840 2022-07-21T01:55:33.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.660+0000 7f8c089d0700 1 -- 172.21.15.140:0/3238038490 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c041527f0 con 0x7f8c040b6840 2022-07-21T01:55:33.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.660+0000 7f8c00ff9700 1 -- 172.21.15.140:0/3238038490 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8bec01a030 con 0x7f8c040b6840 2022-07-21T01:55:33.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.660+0000 7f8c00ff9700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8be8053a70 0x7f8be8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:33.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.660+0000 7f8c00ff9700 1 -- 172.21.15.140:0/3238038490 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f8bec06bb40 con 0x7f8c040b6840 2022-07-21T01:55:33.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.660+0000 7f8c037fe700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8be8053a70 0x7f8be8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:33.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.660+0000 7f8c037fe700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8be8053a70 0x7f8be8055f30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f8bfc00d000 tx=0x7f8bfc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:33.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.664+0000 7f8c00ff9700 1 -- 172.21.15.140:0/3238038490 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8bec0429b0 con 0x7f8c040b6840 2022-07-21T01:55:33.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:33.792+0000 7f8c089d0700 1 -- 172.21.15.140:0/3238038490 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c04151ef0 con 0x7f8c040b6840 2022-07-21T01:55:34.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.524+0000 7f8c00ff9700 1 -- 172.21.15.140:0/3238038490 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8bec045fd0 con 0x7f8c040b6840 2022-07-21T01:55:34.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 -- 172.21.15.140:0/3238038490 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8be8053a70 msgr2=0x7f8be8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:34.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8be8053a70 0x7f8be8055f30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f8bfc00d000 tx=0x7f8bfc009040).stop 2022-07-21T01:55:34.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 -- 172.21.15.140:0/3238038490 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c040b6840 msgr2=0x7f8c040b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:34.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c040b6840 0x7f8c040b8b30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f8bec00a000 tx=0x7f8bec008040).stop 2022-07-21T01:55:34.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 -- 172.21.15.140:0/3238038490 shutdown_connections 2022-07-21T01:55:34.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8be8053a70 0x7f8be8055f30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:34.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c04146530 0x7f8c040b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:34.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c040b6840 0x7f8c040b8b30 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:34.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 --2- 172.21.15.140:0/3238038490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c040b3cf0 0x7f8c04152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:34.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 -- 172.21.15.140:0/3238038490 >> 172.21.15.140:0/3238038490 conn(0x7f8c040b9e10 msgr2=0x7f8c04142f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:34.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 -- 172.21.15.140:0/3238038490 shutdown_connections 2022-07-21T01:55:34.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.528+0000 7f8c089d0700 1 -- 172.21.15.140:0/3238038490 wait complete. 2022-07-21T01:55:34.532 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:55:34.541 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:55:34.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.656+0000 7f4214892700 1 Processor -- start 2022-07-21T01:55:34.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.656+0000 7f4214892700 1 -- start start 2022-07-21T01:55:34.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100b3cf0 0x7f4210146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:34.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4210146530 0x7f4210141a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:34.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4210066b10 con 0x7f42100b6840 2022-07-21T01:55:34.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42100b5c90 con 0x7f4210146530 2022-07-21T01:55:34.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42100b6c20 con 0x7f42100b3cf0 2022-07-21T01:55:34.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4210146530 0x7f4210141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:34.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420f7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f42100b6840 0x7f42100b5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:51964/0 (socket says 172.21.15.140:51964) 2022-07-21T01:55:34.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420f7fe700 1 -- 172.21.15.140:0/1397421870 learned_addr learned my addr 172.21.15.140:0/1397421870 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:34.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420effd700 1 --2- 172.21.15.140:0/1397421870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100b3cf0 0x7f4210146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:34.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420ffff700 1 -- 172.21.15.140:0/1397421870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100b3cf0 msgr2=0x7f4210146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:34.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420ffff700 1 --2- 172.21.15.140:0/1397421870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100b3cf0 0x7f4210146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:34.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420ffff700 1 -- 172.21.15.140:0/1397421870 >> v1:172.21.15.140:6789/0 conn(0x7f42100b6840 legacy=0x7f42100b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:34.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420ffff700 1 -- 172.21.15.140:0/1397421870 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42100aead0 con 0x7f4210146530 2022-07-21T01:55:34.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420ffff700 1 --2- 172.21.15.140:0/1397421870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4210146530 0x7f4210141a90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f42080032f0 tx=0x7f42080091d0).ready entity=mon.0 client_cookie=fb749fdd5d886298 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:34.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420e7fc700 1 -- 172.21.15.140:0/1397421870 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420800b6c0 con 0x7f4210146530 2022-07-21T01:55:34.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420e7fc700 1 -- 172.21.15.140:0/1397421870 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420800b820 con 0x7f4210146530 2022-07-21T01:55:34.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f420e7fc700 1 -- 172.21.15.140:0/1397421870 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420800b990 con 0x7f4210146530 2022-07-21T01:55:34.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 -- 172.21.15.140:0/1397421870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4210146530 msgr2=0x7f4210141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:34.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 --2- 172.21.15.140:0/1397421870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4210146530 0x7f4210141a90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7f42080032f0 tx=0x7f42080091d0).stop 2022-07-21T01:55:34.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 -- 172.21.15.140:0/1397421870 shutdown_connections 2022-07-21T01:55:34.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 --2- 172.21.15.140:0/1397421870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4210146530 0x7f4210141a90 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:34.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 --2- 172.21.15.140:0/1397421870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100b3cf0 0x7f4210146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:34.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 -- 172.21.15.140:0/1397421870 >> 172.21.15.140:0/1397421870 conn(0x7f42100b9e10 msgr2=0x7f42100b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:34.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 -- 172.21.15.140:0/1397421870 shutdown_connections 2022-07-21T01:55:34.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.660+0000 7f4214892700 1 -- 172.21.15.140:0/1397421870 wait complete. 2022-07-21T01:55:34.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.664+0000 7f4214892700 1 Processor -- start 2022-07-21T01:55:34.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.664+0000 7f4214892700 1 -- start start 2022-07-21T01:55:34.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f4214892700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100b3cf0 0x7f42100b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:34.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f4214892700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42100b6840 0x7f4210155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:34.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f4214892700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4210146530 0x7f42101151f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:34.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f4214892700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4210114dc0 con 0x7f4210146530 2022-07-21T01:55:34.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f4214892700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42100b2110 con 0x7f42100b6840 2022-07-21T01:55:34.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f4214892700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42100b23b0 con 0x7f42100b3cf0 2022-07-21T01:55:34.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4210146530 0x7f42101151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:34.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420f7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100b3cf0 0x7f42100b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:34.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4210146530 0x7f42101151f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37826/0 (socket says 172.21.15.140:37826) 2022-07-21T01:55:34.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420ffff700 1 -- 172.21.15.140:0/1600436520 learned_addr learned my addr 172.21.15.140:0/1600436520 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:34.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420effd700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42100b6840 0x7f4210155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:34.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420ffff700 1 -- 172.21.15.140:0/1600436520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100b3cf0 msgr2=0x7f42100b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:34.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420ffff700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100b3cf0 0x7f42100b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:34.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420ffff700 1 -- 172.21.15.140:0/1600436520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42100b6840 msgr2=0x7f4210155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:34.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420ffff700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42100b6840 0x7f4210155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:34.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420ffff700 1 -- 172.21.15.140:0/1600436520 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4208009470 con 0x7f4210146530 2022-07-21T01:55:34.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420ffff700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4210146530 0x7f42101151f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f42080032f0 tx=0x7f4208016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:34.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420cff9700 1 -- 172.21.15.140:0/1600436520 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4208011e20 con 0x7f4210146530 2022-07-21T01:55:34.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420cff9700 1 -- 172.21.15.140:0/1600436520 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4208021440 con 0x7f4210146530 2022-07-21T01:55:34.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f420cff9700 1 -- 172.21.15.140:0/1600436520 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f42080217a0 con 0x7f4210146530 2022-07-21T01:55:34.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f4214892700 1 -- 172.21.15.140:0/1600436520 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42100b1d90 con 0x7f4210146530 2022-07-21T01:55:34.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f4214892700 1 -- 172.21.15.140:0/1600436520 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4210112b30 con 0x7f4210146530 2022-07-21T01:55:34.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.668+0000 7f4214892700 1 -- 172.21.15.140:0/1600436520 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4210114f30 con 0x7f4210146530 2022-07-21T01:55:34.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.672+0000 7f420cff9700 1 -- 172.21.15.140:0/1600436520 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f420800f070 con 0x7f4210146530 2022-07-21T01:55:34.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.672+0000 7f420cff9700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41f4053980 0x7f41f4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:34.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.672+0000 7f420cff9700 1 -- 172.21.15.140:0/1600436520 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f420806c880 con 0x7f4210146530 2022-07-21T01:55:34.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.672+0000 7f420f7fe700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41f4053980 0x7f41f4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:34.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.672+0000 7f420f7fe700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41f4053980 0x7f41f4055e40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f420000d000 tx=0x7f4200009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:34.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.680+0000 7f420cff9700 1 -- 172.21.15.140:0/1600436520 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4210114f30 con 0x7f4210146530 2022-07-21T01:55:34.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:34.804+0000 7f4214892700 1 -- 172.21.15.140:0/1600436520 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4210112b30 con 0x7f4210146530 2022-07-21T01:55:35.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.528+0000 7f420cff9700 1 -- 172.21.15.140:0/1600436520 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4210112b30 con 0x7f4210146530 2022-07-21T01:55:35.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 -- 172.21.15.140:0/1600436520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41f4053980 msgr2=0x7f41f4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41f4053980 0x7f41f4055e40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f420000d000 tx=0x7f4200009040).stop 2022-07-21T01:55:35.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 -- 172.21.15.140:0/1600436520 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4210146530 msgr2=0x7f42101151f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4210146530 0x7f42101151f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f42080032f0 tx=0x7f4208016000).stop 2022-07-21T01:55:35.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 -- 172.21.15.140:0/1600436520 shutdown_connections 2022-07-21T01:55:35.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f41f4053980 0x7f41f4055e40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4210146530 0x7f42101151f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42100b6840 0x7f4210155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 --2- 172.21.15.140:0/1600436520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100b3cf0 0x7f42100b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 -- 172.21.15.140:0/1600436520 >> 172.21.15.140:0/1600436520 conn(0x7f42100b9e10 msgr2=0x7f421005bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:35.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 -- 172.21.15.140:0/1600436520 shutdown_connections 2022-07-21T01:55:35.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.532+0000 7f4214892700 1 -- 172.21.15.140:0/1600436520 wait complete. 2022-07-21T01:55:35.535 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:55:35.544 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-07-21T01:55:35.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 Processor -- start 2022-07-21T01:55:35.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 -- start start 2022-07-21T01:55:35.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3c580af3e0 0x7f3c58146780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 0x7f3c5814a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c58146c70 con 0x7f3c580b5990 2022-07-21T01:55:35.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c580ad050 con 0x7f3c580af3e0 2022-07-21T01:55:35.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580b17e0 con 0x7f3c5805d210 2022-07-21T01:55:35.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c57fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3c580af3e0 0x7f3c58146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5d2da700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 0x7f3c5814a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5d2da700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 0x7f3c5814a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:34564/0 (socket says 172.21.15.140:34564) 2022-07-21T01:55:35.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c57fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3c580af3e0 0x7f3c58146780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:43570/0 (socket says 172.21.15.140:43570) 2022-07-21T01:55:35.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5cad9700 1 --1- 172.21.15.140:0/379636730 >> v1:172.21.15.152:6789/0 conn(0x7f3c580b5990 0x7f3c58146070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:53090/0 (socket says 172.21.15.140:53090) 2022-07-21T01:55:35.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5d2da700 1 -- 172.21.15.140:0/379636730 learned_addr learned my addr 172.21.15.140:0/379636730 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:35.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c57fff700 1 -- 172.21.15.140:0/379636730 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 msgr2=0x7f3c5814a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:35.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c57fff700 1 --2- 172.21.15.140:0/379636730 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 0x7f3c5814a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c57fff700 1 -- 172.21.15.140:0/379636730 >> v1:172.21.15.152:6789/0 conn(0x7f3c580b5990 legacy=0x7f3c58146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c57fff700 1 -- 172.21.15.140:0/379636730 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c580b5d70 con 0x7f3c580af3e0 2022-07-21T01:55:35.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5d2da700 1 --2- 172.21.15.140:0/379636730 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 0x7f3c5814a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:35.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c57fff700 1 --2- 172.21.15.140:0/379636730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c580af3e0 0x7f3c58146780 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f3c500032f0 tx=0x7f3c500091d0).ready entity=mon.1 client_cookie=f3da8e2453ed9052 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:35.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c577fe700 1 -- 172.21.15.140:0/379636730 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c5000b6c0 con 0x7f3c580af3e0 2022-07-21T01:55:35.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c577fe700 1 -- 172.21.15.140:0/379636730 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5000b820 con 0x7f3c580af3e0 2022-07-21T01:55:35.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c577fe700 1 -- 172.21.15.140:0/379636730 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c5000b990 con 0x7f3c580af3e0 2022-07-21T01:55:35.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 -- 172.21.15.140:0/379636730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c580af3e0 msgr2=0x7f3c58146780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 --2- 172.21.15.140:0/379636730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c580af3e0 0x7f3c58146780 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f3c500032f0 tx=0x7f3c500091d0).stop 2022-07-21T01:55:35.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 -- 172.21.15.140:0/379636730 shutdown_connections 2022-07-21T01:55:35.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 --2- 172.21.15.140:0/379636730 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 0x7f3c5814a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 --2- 172.21.15.140:0/379636730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c580af3e0 0x7f3c58146780 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 -- 172.21.15.140:0/379636730 >> 172.21.15.140:0/379636730 conn(0x7f3c58063480 msgr2=0x7f3c580b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:35.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.647+0000 7f3c5dadb700 1 -- 172.21.15.140:0/379636730 shutdown_connections 2022-07-21T01:55:35.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5dadb700 1 -- 172.21.15.140:0/379636730 wait complete. 2022-07-21T01:55:35.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5dadb700 1 Processor -- start 2022-07-21T01:55:35.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5dadb700 1 -- start start 2022-07-21T01:55:35.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5dadb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 0x7f3c58064200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5dadb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c580af3e0 0x7f3c58064a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5dadb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3c580b5990 0x7f3c58064f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5dadb700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58145b00 con 0x7f3c580b5990 2022-07-21T01:55:35.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5dadb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58145600 con 0x7f3c580af3e0 2022-07-21T01:55:35.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5dadb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58145860 con 0x7f3c5805d210 2022-07-21T01:55:35.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5d2da700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3c580b5990 0x7f3c58064f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5d2da700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3c580b5990 0x7f3c58064f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37840/0 (socket says 172.21.15.140:37840) 2022-07-21T01:55:35.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5d2da700 1 -- 172.21.15.140:0/2432380900 learned_addr learned my addr 172.21.15.140:0/2432380900 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:35.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c57fff700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c580af3e0 0x7f3c58064a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5cad9700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 0x7f3c58064200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5d2da700 1 -- 172.21.15.140:0/2432380900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 msgr2=0x7f3c58064200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5d2da700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 0x7f3c58064200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5d2da700 1 -- 172.21.15.140:0/2432380900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c580af3e0 msgr2=0x7f3c58064a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5d2da700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c580af3e0 0x7f3c58064a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5d2da700 1 -- 172.21.15.140:0/2432380900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4800b1e0 con 0x7f3c580b5990 2022-07-21T01:55:35.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5d2da700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3c580b5990 0x7f3c58064f70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f3c4800c000 tx=0x7f3c48006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:35.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c55ffb700 1 -- 172.21.15.140:0/2432380900 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c48012600 con 0x7f3c580b5990 2022-07-21T01:55:35.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c55ffb700 1 -- 172.21.15.140:0/2432380900 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c48012760 con 0x7f3c580b5990 2022-07-21T01:55:35.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c5dadb700 1 -- 172.21.15.140:0/2432380900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c50009470 con 0x7f3c580b5990 2022-07-21T01:55:35.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.651+0000 7f3c55ffb700 1 -- 172.21.15.140:0/2432380900 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c48012ac0 con 0x7f3c580b5990 2022-07-21T01:55:35.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.655+0000 7f3c5dadb700 1 -- 172.21.15.140:0/2432380900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c580678c0 con 0x7f3c580b5990 2022-07-21T01:55:35.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.655+0000 7f3c55ffb700 1 -- 172.21.15.140:0/2432380900 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3c48022070 con 0x7f3c580b5990 2022-07-21T01:55:35.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.655+0000 7f3c5dadb700 1 -- 172.21.15.140:0/2432380900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c580afa60 con 0x7f3c580b5990 2022-07-21T01:55:35.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.655+0000 7f3c55ffb700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3c30053a70 0x7f3c30055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.655+0000 7f3c55ffb700 1 -- 172.21.15.140:0/2432380900 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f3c4806c160 con 0x7f3c580b5990 2022-07-21T01:55:35.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.655+0000 7f3c5cad9700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3c30053a70 0x7f3c30055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.655+0000 7f3c5cad9700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3c30053a70 0x7f3c30055f30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f3c4000d000 tx=0x7f3c40007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:35.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.659+0000 7f3c55ffb700 1 -- 172.21.15.140:0/2432380900 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3c480432a0 con 0x7f3c580b5990 2022-07-21T01:55:35.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.783+0000 7f3c5dadb700 1 -- 172.21.15.140:0/2432380900 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3c580574e0 con 0x7f3c580b5990 2022-07-21T01:55:35.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.791+0000 7f3c55ffb700 1 -- 172.21.15.140:0/2432380900 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3c480160a0 con 0x7f3c580b5990 2022-07-21T01:55:35.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 -- 172.21.15.140:0/2432380900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3c30053a70 msgr2=0x7f3c30055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3c30053a70 0x7f3c30055f30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f3c4000d000 tx=0x7f3c40007b90).stop 2022-07-21T01:55:35.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 -- 172.21.15.140:0/2432380900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3c580b5990 msgr2=0x7f3c58064f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3c580b5990 0x7f3c58064f70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f3c4800c000 tx=0x7f3c48006eb0).stop 2022-07-21T01:55:35.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 -- 172.21.15.140:0/2432380900 shutdown_connections 2022-07-21T01:55:35.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3c30053a70 0x7f3c30055f30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3c580b5990 0x7f3c58064f70 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c580af3e0 0x7f3c58064a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 --2- 172.21.15.140:0/2432380900 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3c5805d210 0x7f3c58064200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 -- 172.21.15.140:0/2432380900 >> 172.21.15.140:0/2432380900 conn(0x7f3c58063480 msgr2=0x7f3c580b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:35.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 -- 172.21.15.140:0/2432380900 shutdown_connections 2022-07-21T01:55:35.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.795+0000 7f3c5dadb700 1 -- 172.21.15.140:0/2432380900 wait complete. 2022-07-21T01:55:35.809 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2022-07-21T01:55:35.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.911+0000 7f9502c23700 1 Processor -- start 2022-07-21T01:55:35.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.911+0000 7f9502c23700 1 -- start start 2022-07-21T01:55:35.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.911+0000 7f9502c23700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 0x7f94fc0b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.911+0000 7f9502c23700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f94fc0b14f0 0x7f94fc147050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.911+0000 7f9502c23700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94fc0afbc0 con 0x7f94fc14ada0 2022-07-21T01:55:35.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.911+0000 7f9502c23700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc148080 con 0x7f94fc0b14f0 2022-07-21T01:55:35.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.911+0000 7f9502c23700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0b00e0 con 0x7f94fc0b59d0 2022-07-21T01:55:35.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502422700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f94fc0b14f0 0x7f94fc147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9501c21700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 0x7f94fc0b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9501420700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f94fc14ada0 0x7f94fc05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:53104/0 (socket says 172.21.15.140:53104) 2022-07-21T01:55:35.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9501420700 1 -- 172.21.15.140:0/1830875318 learned_addr learned my addr 172.21.15.140:0/1830875318 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:35.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9501c21700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 0x7f94fc0b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:34576/0 (socket says 172.21.15.140:34576) 2022-07-21T01:55:35.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9500c1f700 1 -- 172.21.15.140:0/1830875318 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713366286 0 0) 0x7f94fc0afbc0 con 0x7f94fc14ada0 2022-07-21T01:55:35.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9500c1f700 1 -- 172.21.15.140:0/1830875318 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94e8003260 con 0x7f94fc14ada0 2022-07-21T01:55:35.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502422700 1 -- 172.21.15.140:0/1830875318 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 msgr2=0x7f94fc0b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:35.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502422700 1 --2- 172.21.15.140:0/1830875318 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 0x7f94fc0b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502422700 1 -- 172.21.15.140:0/1830875318 >> v1:172.21.15.152:6789/0 conn(0x7f94fc14ada0 legacy=0x7f94fc05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502422700 1 -- 172.21.15.140:0/1830875318 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc0b5db0 con 0x7f94fc0b14f0 2022-07-21T01:55:35.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9501c21700 1 --2- 172.21.15.140:0/1830875318 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 0x7f94fc0b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:35.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502422700 1 --2- 172.21.15.140:0/1830875318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94fc0b14f0 0x7f94fc147050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f94f4002f70 tx=0x7f94f4006df0).ready entity=mon.1 client_cookie=b19ffb64b5e5dbf0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:35.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9500c1f700 1 -- 172.21.15.140:0/1830875318 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94f400d4a0 con 0x7f94fc0b14f0 2022-07-21T01:55:35.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9500c1f700 1 -- 172.21.15.140:0/1830875318 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f4000e90 con 0x7f94fc0b14f0 2022-07-21T01:55:35.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9500c1f700 1 -- 172.21.15.140:0/1830875318 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94f400d7f0 con 0x7f94fc0b14f0 2022-07-21T01:55:35.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502c23700 1 -- 172.21.15.140:0/1830875318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94fc0b14f0 msgr2=0x7f94fc147050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502c23700 1 --2- 172.21.15.140:0/1830875318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94fc0b14f0 0x7f94fc147050 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f94f4002f70 tx=0x7f94f4006df0).stop 2022-07-21T01:55:35.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502c23700 1 -- 172.21.15.140:0/1830875318 shutdown_connections 2022-07-21T01:55:35.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502c23700 1 --2- 172.21.15.140:0/1830875318 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94fc0b14f0 0x7f94fc147050 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502c23700 1 --2- 172.21.15.140:0/1830875318 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 0x7f94fc0b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502c23700 1 -- 172.21.15.140:0/1830875318 >> 172.21.15.140:0/1830875318 conn(0x7f94fc0634c0 msgr2=0x7f94fc0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:35.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502c23700 1 -- 172.21.15.140:0/1830875318 shutdown_connections 2022-07-21T01:55:35.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.915+0000 7f9502c23700 1 -- 172.21.15.140:0/1830875318 wait complete. 2022-07-21T01:55:35.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9502c23700 1 Processor -- start 2022-07-21T01:55:35.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9502c23700 1 -- start start 2022-07-21T01:55:35.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9502c23700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94fc0b14f0 0x7f94fc1442f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9502c23700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 0x7f94fc14a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9502c23700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94fc14ada0 0x7f94fc0674c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9502c23700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0679b0 con 0x7f94fc0b14f0 2022-07-21T01:55:35.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9502c23700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc066980 con 0x7f94fc14ada0 2022-07-21T01:55:35.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9502c23700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc067020 con 0x7f94fc0b59d0 2022-07-21T01:55:35.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9501c21700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94fc0b14f0 0x7f94fc1442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9501c21700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94fc0b14f0 0x7f94fc1442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37850/0 (socket says 172.21.15.140:37850) 2022-07-21T01:55:35.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9502422700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94fc14ada0 0x7f94fc0674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9501c21700 1 -- 172.21.15.140:0/48730667 learned_addr learned my addr 172.21.15.140:0/48730667 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:35.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.919+0000 7f9501420700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 0x7f94fc14a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9501420700 1 -- 172.21.15.140:0/48730667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94fc14ada0 msgr2=0x7f94fc0674c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9501420700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94fc14ada0 0x7f94fc0674c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9501420700 1 -- 172.21.15.140:0/48730667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94fc0b14f0 msgr2=0x7f94fc1442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:35.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9501420700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94fc0b14f0 0x7f94fc1442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:35.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9501420700 1 -- 172.21.15.140:0/48730667 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec00b180 con 0x7f94fc0b59d0 2022-07-21T01:55:35.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9502422700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94fc14ada0 0x7f94fc0674c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:35.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9501c21700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94fc0b14f0 0x7f94fc1442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:35.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9501420700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 0x7f94fc14a0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f94f0001490 tx=0x7f94f0007d60).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:35.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f94faffd700 1 -- 172.21.15.140:0/48730667 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94f000ce90 con 0x7f94fc0b59d0 2022-07-21T01:55:35.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f94faffd700 1 -- 172.21.15.140:0/48730667 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f0005ba0 con 0x7f94fc0b59d0 2022-07-21T01:55:35.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f94faffd700 1 -- 172.21.15.140:0/48730667 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94f00048a0 con 0x7f94fc0b59d0 2022-07-21T01:55:35.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9502c23700 1 -- 172.21.15.140:0/48730667 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f400b180 con 0x7f94fc0b59d0 2022-07-21T01:55:35.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9502c23700 1 -- 172.21.15.140:0/48730667 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc10e170 con 0x7f94fc0b59d0 2022-07-21T01:55:35.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9502c23700 1 -- 172.21.15.140:0/48730667 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94fc0b0340 con 0x7f94fc0b59d0 2022-07-21T01:55:35.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f94faffd700 1 -- 172.21.15.140:0/48730667 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f94f001b070 con 0x7f94fc0b59d0 2022-07-21T01:55:35.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f94faffd700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f94dc053950 0x7f94dc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:35.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f94faffd700 1 -- 172.21.15.140:0/48730667 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f94f006b330 con 0x7f94fc0b59d0 2022-07-21T01:55:35.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.923+0000 7f9501c21700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f94dc053950 0x7f94dc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:35.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.927+0000 7f9501c21700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f94dc053950 0x7f94dc055e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f94ec00b000 tx=0x7f94ec006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:35.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:35.935+0000 7f94faffd700 1 -- 172.21.15.140:0/48730667 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f94f0042530 con 0x7f94fc0b59d0 2022-07-21T01:55:36.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.103+0000 7f9502c23700 1 -- 172.21.15.140:0/48730667 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f94fc0574e0 con 0x7f94fc0b59d0 2022-07-21T01:55:36.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.539+0000 7f94faffd700 1 -- 172.21.15.140:0/48730667 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v313) v1 ==== 126+0+0 (secure 0 0 0) 0x7f94f0045b50 con 0x7f94fc0b59d0 2022-07-21T01:55:36.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 -- 172.21.15.140:0/48730667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f94dc053950 msgr2=0x7f94dc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:36.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f94dc053950 0x7f94dc055e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f94ec00b000 tx=0x7f94ec006df0).stop 2022-07-21T01:55:36.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 -- 172.21.15.140:0/48730667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 msgr2=0x7f94fc14a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:36.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 0x7f94fc14a0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f94f0001490 tx=0x7f94f0007d60).stop 2022-07-21T01:55:36.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 -- 172.21.15.140:0/48730667 shutdown_connections 2022-07-21T01:55:36.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f94dc053950 0x7f94dc055e10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:36.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94fc14ada0 0x7f94fc0674c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:36.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f94fc0b59d0 0x7f94fc14a0c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:36.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 --2- 172.21.15.140:0/48730667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f94fc0b14f0 0x7f94fc1442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:36.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 -- 172.21.15.140:0/48730667 >> 172.21.15.140:0/48730667 conn(0x7f94fc0634c0 msgr2=0x7f94fc145020 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:36.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 -- 172.21.15.140:0/48730667 shutdown_connections 2022-07-21T01:55:36.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.543+0000 7f9502c23700 1 -- 172.21.15.140:0/48730667 wait complete. 2022-07-21T01:55:36.560 INFO:tasks.cephfs_test_runner:test_dump_inode (tasks.cephfs.test_misc.TestMisc) ... ok 2022-07-21T01:55:36.560 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) 2022-07-21T01:55:36.561 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2022-07-21T01:55:36.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc671355700 1 Processor -- start 2022-07-21T01:55:36.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc671355700 1 -- start start 2022-07-21T01:55:36.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc671355700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc66c0b59d0 0x7fc66c0b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:36.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc671355700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc66c14ada0 0x7fc66c0b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:36.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc671355700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66c0afbc0 con 0x7fc66c0b14f0 2022-07-21T01:55:36.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc671355700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c148080 con 0x7fc66c14ada0 2022-07-21T01:55:36.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc671355700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0b00e0 con 0x7fc66c0b59d0 2022-07-21T01:55:36.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc66b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc66c14ada0 0x7fc66c0b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:36.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc670b54700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc66c0b14f0 0x7fc66c05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52008/0 (socket says 172.21.15.140:52008) 2022-07-21T01:55:36.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc670b54700 1 -- 172.21.15.140:0/652204439 learned_addr learned my addr 172.21.15.140:0/652204439 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:36.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc66bfff700 1 --2- 172.21.15.140:0/652204439 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc66c0b59d0 0x7fc66c0b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:36.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.667+0000 7fc66affd700 1 -- 172.21.15.140:0/652204439 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216946018 0 0) 0x7fc66c0afbc0 con 0x7fc66c0b14f0 2022-07-21T01:55:36.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc66affd700 1 -- 172.21.15.140:0/652204439 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc658003260 con 0x7fc66c0b14f0 2022-07-21T01:55:36.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc66b7fe700 1 -- 172.21.15.140:0/652204439 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc66c0b59d0 msgr2=0x7fc66c0b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:36.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc66affd700 1 -- 172.21.15.140:0/652204439 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1385218150 0 0) 0x7fc658003260 con 0x7fc66c0b14f0 2022-07-21T01:55:36.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc66b7fe700 1 --2- 172.21.15.140:0/652204439 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc66c0b59d0 0x7fc66c0b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:36.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc66b7fe700 1 -- 172.21.15.140:0/652204439 >> v1:172.21.15.140:6789/0 conn(0x7fc66c0b14f0 legacy=0x7fc66c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:36.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc66b7fe700 1 -- 172.21.15.140:0/652204439 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c0b5db0 con 0x7fc66c14ada0 2022-07-21T01:55:36.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc66b7fe700 1 --2- 172.21.15.140:0/652204439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc66c14ada0 0x7fc66c0b0cd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fc65c001820 tx=0x7fc660006df0).ready entity=mon.0 client_cookie=682ff3df27a94e93 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:36.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc66affd700 1 -- 172.21.15.140:0/652204439 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc65c001680 con 0x7fc66c14ada0 2022-07-21T01:55:36.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc66affd700 1 -- 172.21.15.140:0/652204439 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc660000e90 con 0x7fc66c14ada0 2022-07-21T01:55:36.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc66affd700 1 -- 172.21.15.140:0/652204439 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc660012460 con 0x7fc66c14ada0 2022-07-21T01:55:36.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc671355700 1 -- 172.21.15.140:0/652204439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc66c14ada0 msgr2=0x7fc66c0b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:36.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc671355700 1 --2- 172.21.15.140:0/652204439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc66c14ada0 0x7fc66c0b0cd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7fc65c001820 tx=0x7fc660006df0).stop 2022-07-21T01:55:36.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc671355700 1 -- 172.21.15.140:0/652204439 shutdown_connections 2022-07-21T01:55:36.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc671355700 1 --2- 172.21.15.140:0/652204439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc66c14ada0 0x7fc66c0b0cd0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:36.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc671355700 1 --2- 172.21.15.140:0/652204439 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc66c0b59d0 0x7fc66c0b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:36.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc671355700 1 -- 172.21.15.140:0/652204439 >> 172.21.15.140:0/652204439 conn(0x7fc66c0634c0 msgr2=0x7fc66c0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:36.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc671355700 1 -- 172.21.15.140:0/652204439 shutdown_connections 2022-07-21T01:55:36.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.671+0000 7fc671355700 1 -- 172.21.15.140:0/652204439 wait complete. 2022-07-21T01:55:36.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc671355700 1 Processor -- start 2022-07-21T01:55:36.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc671355700 1 -- start start 2022-07-21T01:55:36.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc671355700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc66c0b14f0 0x7fc66c149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:36.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc671355700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc66c0b59d0 0x7fc66c10ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:36.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc671355700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc66c14ada0 0x7fc66c155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:36.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc671355700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c10e530 con 0x7fc66c14ada0 2022-07-21T01:55:36.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc671355700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c10e7d0 con 0x7fc66c0b59d0 2022-07-21T01:55:36.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc671355700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c10ea70 con 0x7fc66c0b14f0 2022-07-21T01:55:36.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc670b54700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc66c14ada0 0x7fc66c155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:36.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc670b54700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc66c14ada0 0x7fc66c155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37868/0 (socket says 172.21.15.140:37868) 2022-07-21T01:55:36.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc670b54700 1 -- 172.21.15.140:0/3198890109 learned_addr learned my addr 172.21.15.140:0/3198890109 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:36.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc66bfff700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc66c0b14f0 0x7fc66c149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:36.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc66b7fe700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc66c0b59d0 0x7fc66c10ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:36.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc670b54700 1 -- 172.21.15.140:0/3198890109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc66c0b14f0 msgr2=0x7fc66c149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:36.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc670b54700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc66c0b14f0 0x7fc66c149860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:36.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc670b54700 1 -- 172.21.15.140:0/3198890109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc66c0b59d0 msgr2=0x7fc66c10ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:36.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc670b54700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc66c0b59d0 0x7fc66c10ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:36.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc670b54700 1 -- 172.21.15.140:0/3198890109 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66000b180 con 0x7fc66c14ada0 2022-07-21T01:55:36.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.675+0000 7fc670b54700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc66c14ada0 0x7fc66c155dd0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fc65c0018e0 tx=0x7fc65c001e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:36.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.679+0000 7fc6697fa700 1 -- 172.21.15.140:0/3198890109 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc65c007170 con 0x7fc66c14ada0 2022-07-21T01:55:36.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.679+0000 7fc6697fa700 1 -- 172.21.15.140:0/3198890109 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c0072d0 con 0x7fc66c14ada0 2022-07-21T01:55:36.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.679+0000 7fc671355700 1 -- 172.21.15.140:0/3198890109 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c066880 con 0x7fc66c14ada0 2022-07-21T01:55:36.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.679+0000 7fc6697fa700 1 -- 172.21.15.140:0/3198890109 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc65c004710 con 0x7fc66c14ada0 2022-07-21T01:55:36.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.679+0000 7fc671355700 1 -- 172.21.15.140:0/3198890109 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c0659c0 con 0x7fc66c14ada0 2022-07-21T01:55:36.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.679+0000 7fc671355700 1 -- 172.21.15.140:0/3198890109 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c10ec30 con 0x7fc66c14ada0 2022-07-21T01:55:36.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.679+0000 7fc6697fa700 1 -- 172.21.15.140:0/3198890109 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc65c004870 con 0x7fc66c14ada0 2022-07-21T01:55:36.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.679+0000 7fc6697fa700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc64c053b00 0x7fc64c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:36.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.679+0000 7fc6697fa700 1 -- 172.21.15.140:0/3198890109 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fc65c06b610 con 0x7fc66c14ada0 2022-07-21T01:55:36.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.679+0000 7fc66bfff700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc64c053b00 0x7fc64c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:36.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.683+0000 7fc66bfff700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc64c053b00 0x7fc64c055fc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fc66400b000 tx=0x7fc6640091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:36.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.687+0000 7fc6697fa700 1 -- 172.21.15.140:0/3198890109 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc65c042700 con 0x7fc66c14ada0 2022-07-21T01:55:36.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:36.859+0000 7fc671355700 1 -- 172.21.15.140:0/3198890109 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7fc66c0b0340 con 0x7fc66c14ada0 2022-07-21T01:55:37.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.543+0000 7fc6697fa700 1 -- 172.21.15.140:0/3198890109 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v314) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc65c045d20 con 0x7fc66c14ada0 2022-07-21T01:55:37.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 -- 172.21.15.140:0/3198890109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc64c053b00 msgr2=0x7fc64c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:37.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc64c053b00 0x7fc64c055fc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fc66400b000 tx=0x7fc6640091d0).stop 2022-07-21T01:55:37.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 -- 172.21.15.140:0/3198890109 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc66c14ada0 msgr2=0x7fc66c155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:37.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc66c14ada0 0x7fc66c155dd0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7fc65c0018e0 tx=0x7fc65c001e40).stop 2022-07-21T01:55:37.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 -- 172.21.15.140:0/3198890109 shutdown_connections 2022-07-21T01:55:37.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc64c053b00 0x7fc64c055fc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc66c14ada0 0x7fc66c155dd0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc66c0b59d0 0x7fc66c10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 --2- 172.21.15.140:0/3198890109 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc66c0b14f0 0x7fc66c149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 -- 172.21.15.140:0/3198890109 >> 172.21.15.140:0/3198890109 conn(0x7fc66c0634c0 msgr2=0x7fc66c1452a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:37.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 -- 172.21.15.140:0/3198890109 shutdown_connections 2022-07-21T01:55:37.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.547+0000 7fc671355700 1 -- 172.21.15.140:0/3198890109 wait complete. 2022-07-21T01:55:37.561 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:55:37.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8983a8e700 1 Processor -- start 2022-07-21T01:55:37.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8983a8e700 1 -- start start 2022-07-21T01:55:37.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8983a8e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f897c0967e0 0x7f897c140860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:37.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8983a8e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f897c0ac680 0x7f897c0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:37.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8983a8e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c13fe80 con 0x7f897c0b2210 2022-07-21T01:55:37.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8983a8e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c140120 con 0x7f897c0967e0 2022-07-21T01:55:37.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8983a8e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c1403c0 con 0x7f897c0ac680 2022-07-21T01:55:37.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8981029700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f897c0967e0 0x7f897c140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:37.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8981029700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f897c0967e0 0x7f897c140860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37876/0 (socket says 172.21.15.140:37876) 2022-07-21T01:55:37.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8981029700 1 -- 172.21.15.140:0/2745974991 learned_addr learned my addr 172.21.15.140:0/2745974991 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:37.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f898202b700 1 --2- 172.21.15.140:0/2745974991 >> v2:172.21.15.152:3300/0 conn(0x7f897c0ac680 0x7f897c0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:37.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8981029700 1 -- 172.21.15.140:0/2745974991 >> v2:172.21.15.152:3300/0 conn(0x7f897c0ac680 msgr2=0x7f897c0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:37.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8981029700 1 --2- 172.21.15.140:0/2745974991 >> v2:172.21.15.152:3300/0 conn(0x7f897c0ac680 0x7f897c0b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8981029700 1 -- 172.21.15.140:0/2745974991 >> v1:172.21.15.140:6789/0 conn(0x7f897c0b2210 legacy=0x7f897c0ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:37.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.679+0000 7f8981029700 1 -- 172.21.15.140:0/2745974991 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0ab7c0 con 0x7f897c0967e0 2022-07-21T01:55:37.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8981029700 1 --2- 172.21.15.140:0/2745974991 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f897c0967e0 0x7f897c140860 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f896c002f70 tx=0x7f896c006f40).ready entity=mon.0 client_cookie=51e5e44f504a0816 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:37.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8980828700 1 -- 172.21.15.140:0/2745974991 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896c00dc70 con 0x7f897c0967e0 2022-07-21T01:55:37.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8980828700 1 -- 172.21.15.140:0/2745974991 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c000e90 con 0x7f897c0967e0 2022-07-21T01:55:37.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8980828700 1 -- 172.21.15.140:0/2745974991 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896c00be90 con 0x7f897c0967e0 2022-07-21T01:55:37.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8983a8e700 1 -- 172.21.15.140:0/2745974991 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f897c0967e0 msgr2=0x7f897c140860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:37.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8983a8e700 1 --2- 172.21.15.140:0/2745974991 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f897c0967e0 0x7f897c140860 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f896c002f70 tx=0x7f896c006f40).stop 2022-07-21T01:55:37.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8983a8e700 1 -- 172.21.15.140:0/2745974991 shutdown_connections 2022-07-21T01:55:37.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8983a8e700 1 --2- 172.21.15.140:0/2745974991 >> v2:172.21.15.152:3300/0 conn(0x7f897c0ac680 0x7f897c0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8983a8e700 1 --2- 172.21.15.140:0/2745974991 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f897c0967e0 0x7f897c140860 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8983a8e700 1 -- 172.21.15.140:0/2745974991 >> 172.21.15.140:0/2745974991 conn(0x7f897c0b36c0 msgr2=0x7f897c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:37.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8983a8e700 1 -- 172.21.15.140:0/2745974991 shutdown_connections 2022-07-21T01:55:37.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8983a8e700 1 -- 172.21.15.140:0/2745974991 wait complete. 2022-07-21T01:55:37.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8983a8e700 1 Processor -- start 2022-07-21T01:55:37.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.683+0000 7f8983a8e700 1 -- start start 2022-07-21T01:55:37.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f8983a8e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c0967e0 0x7f897c13c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:37.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f8983a8e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f897c0ac680 0x7f897c0ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:37.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f8983a8e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f897c0b2210 0x7f897c0adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:37.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f8983a8e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c142790 con 0x7f897c0b2210 2022-07-21T01:55:37.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f8983a8e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c142a30 con 0x7f897c0967e0 2022-07-21T01:55:37.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f8983a8e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c143520 con 0x7f897c0ac680 2022-07-21T01:55:37.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f898202b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f897c0b2210 0x7f897c0adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:37.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f898202b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f897c0b2210 0x7f897c0adc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37884/0 (socket says 172.21.15.140:37884) 2022-07-21T01:55:37.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f898202b700 1 -- 172.21.15.140:0/2749695052 learned_addr learned my addr 172.21.15.140:0/2749695052 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:37.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f8981029700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f897c0ac680 0x7f897c0ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:37.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f898182a700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c0967e0 0x7f897c13c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:37.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f898202b700 1 -- 172.21.15.140:0/2749695052 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f897c0ac680 msgr2=0x7f897c0ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:37.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f898202b700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f897c0ac680 0x7f897c0ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f898202b700 1 -- 172.21.15.140:0/2749695052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c0967e0 msgr2=0x7f897c13c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:37.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f898202b700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c0967e0 0x7f897c13c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f898202b700 1 -- 172.21.15.140:0/2749695052 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c00b460 con 0x7f897c0b2210 2022-07-21T01:55:37.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f898202b700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f897c0b2210 0x7f897c0adc50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f897400a000 tx=0x7f8974008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:37.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f896affd700 1 -- 172.21.15.140:0/2749695052 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8974013620 con 0x7f897c0b2210 2022-07-21T01:55:37.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f896affd700 1 -- 172.21.15.140:0/2749695052 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8974013780 con 0x7f897c0b2210 2022-07-21T01:55:37.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f8983a8e700 1 -- 172.21.15.140:0/2749695052 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c1514e0 con 0x7f897c0b2210 2022-07-21T01:55:37.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f896affd700 1 -- 172.21.15.140:0/2749695052 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8974013ae0 con 0x7f897c0b2210 2022-07-21T01:55:37.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f8983a8e700 1 -- 172.21.15.140:0/2749695052 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c0b03c0 con 0x7f897c0b2210 2022-07-21T01:55:37.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f8983a8e700 1 -- 172.21.15.140:0/2749695052 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c13eef0 con 0x7f897c0b2210 2022-07-21T01:55:37.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f896affd700 1 -- 172.21.15.140:0/2749695052 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8974013c40 con 0x7f897c0b2210 2022-07-21T01:55:37.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f896affd700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f895c053950 0x7f895c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:37.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f896affd700 1 -- 172.21.15.140:0/2749695052 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f897406bad0 con 0x7f897c0b2210 2022-07-21T01:55:37.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.687+0000 7f898182a700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f895c053950 0x7f895c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:37.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.691+0000 7f898182a700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f895c053950 0x7f895c055e10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f897800b000 tx=0x7f8978009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:37.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.691+0000 7f896affd700 1 -- 172.21.15.140:0/2749695052 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8974042c70 con 0x7f897c0b2210 2022-07-21T01:55:37.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.819+0000 7f8983a8e700 1 -- 172.21.15.140:0/2749695052 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f897c0aba10 con 0x7f897c0b2210 2022-07-21T01:55:37.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.827+0000 7f896affd700 1 -- 172.21.15.140:0/2749695052 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8974046290 con 0x7f897c0b2210 2022-07-21T01:55:37.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 -- 172.21.15.140:0/2749695052 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f895c053950 msgr2=0x7f895c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:37.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f895c053950 0x7f895c055e10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f897800b000 tx=0x7f8978009040).stop 2022-07-21T01:55:37.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 -- 172.21.15.140:0/2749695052 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f897c0b2210 msgr2=0x7f897c0adc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:37.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f897c0b2210 0x7f897c0adc50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f897400a000 tx=0x7f8974008040).stop 2022-07-21T01:55:37.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 -- 172.21.15.140:0/2749695052 shutdown_connections 2022-07-21T01:55:37.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f895c053950 0x7f895c055e10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f897c0b2210 0x7f897c0adc50 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f897c0ac680 0x7f897c0ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 --2- 172.21.15.140:0/2749695052 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f897c0967e0 0x7f897c13c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:37.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 -- 172.21.15.140:0/2749695052 >> 172.21.15.140:0/2749695052 conn(0x7f897c0b36c0 msgr2=0x7f897c13b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:37.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 -- 172.21.15.140:0/2749695052 shutdown_connections 2022-07-21T01:55:37.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:37.831+0000 7f8983a8e700 1 -- 172.21.15.140:0/2749695052 wait complete. 2022-07-21T01:55:37.845 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:55:37.846 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:37.860 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:37.881 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:37.902 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:37.923 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:37.944 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:55:37.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:37.945 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:37.945 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:37.952 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:55:37.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:37.954 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:55:37.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:37.955 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:37.955 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:37.963 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:55:37.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:37.965 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:55:37.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:37.966 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:37.966 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:55:37.974 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:55:37.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:37.975 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:55:37.976 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:55:38.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4ac6cc700 1 Processor -- start 2022-07-21T01:55:38.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4ac6cc700 1 -- start start 2022-07-21T01:55:38.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4ac6cc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe4a40b1620 0x7fe4a4142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4ac6cc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe4a413ee40 0x7fe4a41499b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4ac6cc700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a41426b0 con 0x7fe4a4142fd0 2022-07-21T01:55:38.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4ac6cc700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a4060b50 con 0x7fe4a40b1620 2022-07-21T01:55:38.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4ac6cc700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a40b1a00 con 0x7fe4a413ee40 2022-07-21T01:55:38.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4aa468700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe4a40b1620 0x7fe4a4142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4aa468700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe4a40b1620 0x7fe4a4142ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37888/0 (socket says 172.21.15.140:37888) 2022-07-21T01:55:38.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4a9c67700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe4a4142fd0 0x7fe4a413e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52034/0 (socket says 172.21.15.140:52034) 2022-07-21T01:55:38.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4aa468700 1 -- 172.21.15.140:0/264554757 learned_addr learned my addr 172.21.15.140:0/264554757 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:38.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.087+0000 7fe4aac69700 1 --2- 172.21.15.140:0/264554757 >> v2:172.21.15.152:3300/0 conn(0x7fe4a413ee40 0x7fe4a41499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4aa468700 1 -- 172.21.15.140:0/264554757 >> v2:172.21.15.152:3300/0 conn(0x7fe4a413ee40 msgr2=0x7fe4a41499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:38.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4aa468700 1 --2- 172.21.15.140:0/264554757 >> v2:172.21.15.152:3300/0 conn(0x7fe4a413ee40 0x7fe4a41499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4aa468700 1 -- 172.21.15.140:0/264554757 >> v1:172.21.15.140:6789/0 conn(0x7fe4a4142fd0 legacy=0x7fe4a413e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4a9466700 1 -- 172.21.15.140:0/264554757 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729017191 0 0) 0x7fe4a41426b0 con 0x7fe4a4142fd0 2022-07-21T01:55:38.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4aa468700 1 -- 172.21.15.140:0/264554757 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a405f840 con 0x7fe4a40b1620 2022-07-21T01:55:38.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4aa468700 1 --2- 172.21.15.140:0/264554757 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4a40b1620 0x7fe4a4142ae0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7fe4a0002f70 tx=0x7fe4a0006f40).ready entity=mon.0 client_cookie=1d29fd966ae7b092 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:38.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4a9466700 1 -- 172.21.15.140:0/264554757 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4a000c880 con 0x7fe4a40b1620 2022-07-21T01:55:38.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4a9466700 1 -- 172.21.15.140:0/264554757 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a0000e90 con 0x7fe4a40b1620 2022-07-21T01:55:38.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4a9466700 1 -- 172.21.15.140:0/264554757 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4a000cbd0 con 0x7fe4a40b1620 2022-07-21T01:55:38.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/264554757 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4a40b1620 msgr2=0x7fe4a4142ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4ac6cc700 1 --2- 172.21.15.140:0/264554757 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4a40b1620 0x7fe4a4142ae0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7fe4a0002f70 tx=0x7fe4a0006f40).stop 2022-07-21T01:55:38.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/264554757 shutdown_connections 2022-07-21T01:55:38.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4ac6cc700 1 --2- 172.21.15.140:0/264554757 >> v2:172.21.15.152:3300/0 conn(0x7fe4a413ee40 0x7fe4a41499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4ac6cc700 1 --2- 172.21.15.140:0/264554757 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4a40b1620 0x7fe4a4142ae0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/264554757 >> 172.21.15.140:0/264554757 conn(0x7fe4a40b23d0 msgr2=0x7fe4a40b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:38.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/264554757 shutdown_connections 2022-07-21T01:55:38.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.091+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/264554757 wait complete. 2022-07-21T01:55:38.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4ac6cc700 1 Processor -- start 2022-07-21T01:55:38.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4ac6cc700 1 -- start start 2022-07-21T01:55:38.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4ac6cc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4a40b1620 0x7fe4a40ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4ac6cc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4a413ee40 0x7fe4a4144f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4ac6cc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4a4142fd0 0x7fe4a40ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4ac6cc700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40ade90 con 0x7fe4a413ee40 2022-07-21T01:55:38.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4ac6cc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40ada20 con 0x7fe4a40b1620 2022-07-21T01:55:38.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4ac6cc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a4144840 con 0x7fe4a4142fd0 2022-07-21T01:55:38.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4a9c67700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4a413ee40 0x7fe4a4144f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4aa468700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4a40b1620 0x7fe4a40ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4a9c67700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4a413ee40 0x7fe4a4144f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37896/0 (socket says 172.21.15.140:37896) 2022-07-21T01:55:38.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4a9c67700 1 -- 172.21.15.140:0/3389469077 learned_addr learned my addr 172.21.15.140:0/3389469077 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:38.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4aac69700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4a4142fd0 0x7fe4a40ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4a9c67700 1 -- 172.21.15.140:0/3389469077 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4a4142fd0 msgr2=0x7fe4a40ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4a9c67700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4a4142fd0 0x7fe4a40ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4a9c67700 1 -- 172.21.15.140:0/3389469077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4a40b1620 msgr2=0x7fe4a40ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4a9c67700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4a40b1620 0x7fe4a40ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4a9c67700 1 -- 172.21.15.140:0/3389469077 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a000a360 con 0x7fe4a413ee40 2022-07-21T01:55:38.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.095+0000 7fe4a9c67700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4a413ee40 0x7fe4a4144f80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fe494000ed0 tx=0x7fe494007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:38.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.099+0000 7fe4937fe700 1 -- 172.21.15.140:0/3389469077 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe49400ce50 con 0x7fe4a413ee40 2022-07-21T01:55:38.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.099+0000 7fe4937fe700 1 -- 172.21.15.140:0/3389469077 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4940070d0 con 0x7fe4a413ee40 2022-07-21T01:55:38.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.099+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/3389469077 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a4144ae0 con 0x7fe4a413ee40 2022-07-21T01:55:38.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.099+0000 7fe4937fe700 1 -- 172.21.15.140:0/3389469077 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe494007430 con 0x7fe4a413ee40 2022-07-21T01:55:38.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.099+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/3389469077 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a40af860 con 0x7fe4a413ee40 2022-07-21T01:55:38.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.099+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/3389469077 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a4144c70 con 0x7fe4a413ee40 2022-07-21T01:55:38.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.099+0000 7fe4937fe700 1 -- 172.21.15.140:0/3389469077 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe49401a060 con 0x7fe4a413ee40 2022-07-21T01:55:38.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.099+0000 7fe4937fe700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4840539e0 0x7fe484055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.099+0000 7fe4937fe700 1 -- 172.21.15.140:0/3389469077 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fe49406b280 con 0x7fe4a413ee40 2022-07-21T01:55:38.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.099+0000 7fe4aa468700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4840539e0 0x7fe484055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.099+0000 7fe4aa468700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4840539e0 0x7fe484055ea0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7fe4a001b000 tx=0x7fe4a0006f40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:38.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.107+0000 7fe4937fe700 1 -- 172.21.15.140:0/3389469077 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe494042480 con 0x7fe4a413ee40 2022-07-21T01:55:38.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.235+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/3389469077 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe4a405fae0 con 0x7fe4a413ee40 2022-07-21T01:55:38.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.235+0000 7fe4937fe700 1 -- 172.21.15.140:0/3389469077 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v87) v1 ==== 81+0+28926 (secure 0 0 0) 0x7fe494018070 con 0x7fe4a413ee40 2022-07-21T01:55:38.239 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:38.239 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:55:38.240 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 87, 2022-07-21T01:55:38.240 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:55:38.240 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:55:38.240 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:55:35.521048+0000", 2022-07-21T01:55:38.241 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:55:38.241 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:55:38.241 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:55:38.241 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:55:38.242 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:55:38.242 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:55:38.242 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:55:38.242 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:55:38.242 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:55:38.243 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:38.243 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:55:38.243 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:55:38.243 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:55:38.244 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:55:38.244 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:55:38.244 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 16, 2022-07-21T01:55:38.244 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:55:38.245 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:55:38.245 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:55:38.245 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:55:38.245 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:55:38.246 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.246 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:55:38.246 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:55:38.246 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:55:38.246 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:38.247 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:38.247 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:38.247 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:38.247 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:38.248 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:38.248 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:38.248 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:38.248 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:55:38.249 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:55:38.249 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:55:38.249 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:55:38.250 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:55:38.250 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:38.250 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:38.251 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:38.251 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:38.251 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:38.251 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:38.252 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:38.252 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.252 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:55:38.252 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:38.253 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:38.253 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:38.253 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:38.253 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:38.253 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:38.254 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:38.254 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:38.254 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:38.254 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:38.254 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:38.255 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:38.255 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:38.255 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:38.255 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:38.256 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:38.256 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:38.256 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:38.256 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:38.256 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:38.257 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:38.257 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:38.257 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:38.257 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:38.258 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:38.258 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:38.258 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.258 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:38.259 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:38.259 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:38.259 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:38.259 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:38.259 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:38.260 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:38.260 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:38.260 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:38.260 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:38.261 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:38.261 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:55:38.261 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:55:38.261 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.261 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:38.262 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:55:38.262 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.262 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.262 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.263 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:55:38.263 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:55:38.263 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:55:38.263 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:38.263 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:38.264 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:38.264 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:38.264 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:38.265 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:38.266 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:38.266 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:38.266 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:55:38.266 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:55:38.267 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:55:38.267 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:55:38.267 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:55:38.267 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:38.268 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:38.268 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:38.268 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:38.268 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:38.268 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:38.269 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:38.269 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.269 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:55:38.269 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:38.269 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:38.270 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:38.270 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:38.270 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:38.270 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:38.271 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:38.271 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:38.271 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:38.271 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:38.271 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:38.272 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:38.272 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:38.272 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:38.272 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:38.273 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:38.273 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:38.273 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:38.273 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:38.273 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:38.274 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:38.274 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:38.274 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:38.275 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:38.275 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:38.275 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:38.275 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.275 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:38.276 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:38.276 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:38.276 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:38.276 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:38.276 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:38.277 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:38.277 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:38.277 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:38.277 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:38.278 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:38.279 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:55:38.279 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:38.279 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:55:38.280 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.280 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.280 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:38.280 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:55:38.281 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.281 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:55:38.281 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:55:38.281 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:38.281 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:38.282 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:38.282 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:38.282 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:38.282 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:38.283 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:38.283 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:38.283 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:38.283 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:38.283 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:38.284 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.284 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.284 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.284 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:55:38.285 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:38.285 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.285 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.285 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.285 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:55:38.286 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:38.286 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.286 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.286 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.286 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:38.287 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.287 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.287 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.288 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:55:38.288 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:38.288 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.288 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.288 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.289 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:55:38.289 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:38.289 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.289 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.290 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.290 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:38.290 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.290 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.290 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.291 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:55:38.291 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:38.291 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.291 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.291 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.292 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:55:38.292 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:38.292 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.292 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.293 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.293 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:38.293 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.293 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.293 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.294 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:55:38.294 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:38.294 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.294 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.294 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.295 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:55:38.295 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:38.295 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.295 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.296 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.296 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:55:38.296 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:55:38.296 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:55:38.296 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:55:38.297 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:38.297 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:38.297 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:38.297 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.297 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.298 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.298 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:55:38.298 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:55:38.298 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:38.299 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:38.299 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:38.299 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:38.299 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:38.299 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:38.300 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:38.300 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 81, 2022-07-21T01:55:38.300 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:38.300 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:38.301 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:38.301 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.301 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.301 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.302 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:55:38.302 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:38.302 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.302 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.303 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.303 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:55:38.303 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:38.303 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.304 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.304 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.304 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:38.304 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.305 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.305 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.305 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:55:38.305 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:38.306 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.306 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.306 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.306 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:55:38.307 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:38.307 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.307 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.307 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.307 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:38.308 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.308 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.308 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.308 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:55:38.308 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:38.309 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.309 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.309 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.309 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:55:38.310 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:38.310 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.310 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.310 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.310 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:38.311 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.311 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.311 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.311 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:55:38.312 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:38.312 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.312 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.312 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.312 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:55:38.313 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:38.313 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.313 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.313 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.313 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:55:38.314 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:55:38.314 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:55:38.314 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:55:38.314 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:38.315 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:38.315 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:38.315 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.315 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.315 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.316 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:55:38.316 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:55:38.317 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:38.317 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:38.318 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:38.318 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:38.318 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:38.318 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:38.318 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:38.319 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:38.319 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:38.319 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:38.319 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:38.320 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.320 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.320 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.320 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:55:38.320 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:38.321 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.321 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.321 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.321 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:55:38.321 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:38.322 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.322 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.322 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.322 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:38.323 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.323 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.323 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.323 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:55:38.323 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:38.324 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.324 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.324 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.325 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:55:38.325 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:38.325 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.325 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.325 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.326 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:38.326 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.326 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.326 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.327 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:55:38.327 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:38.327 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.327 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.327 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.328 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:55:38.328 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:38.328 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.328 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.328 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.329 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:38.329 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.329 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.330 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.330 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:55:38.331 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:38.332 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.332 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.333 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.333 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:55:38.334 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:38.334 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.335 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.335 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.336 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:55:38.336 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:55:38.337 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:55:38.337 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:55:38.338 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:38.338 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:38.339 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:38.339 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.340 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.340 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.341 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:55:38.341 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:55:38.342 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:38.342 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:38.342 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:38.343 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:38.343 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:38.344 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:38.344 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:38.345 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:38.345 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:38.346 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:38.346 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:38.347 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.347 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.348 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.348 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:55:38.349 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:38.349 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.350 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.350 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.351 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:55:38.351 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:38.351 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.352 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.352 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.352 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:38.353 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.353 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.354 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.355 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:55:38.355 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:38.356 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.356 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.357 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.358 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:55:38.358 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:38.359 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.359 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.360 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.360 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:38.360 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.361 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.361 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.362 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:55:38.362 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:38.363 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.363 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.364 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.364 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:55:38.365 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:38.365 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.366 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.366 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.367 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:38.367 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.368 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.368 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.368 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:55:38.368 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:38.369 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.369 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.370 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.370 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:55:38.371 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:38.371 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.372 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.372 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.373 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:55:38.373 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:55:38.374 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:55:38.374 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:55:38.375 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:38.375 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:38.375 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:38.375 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.376 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.376 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.377 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:55:38.377 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:55:38.378 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:38.378 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:38.379 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:38.379 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:38.380 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:38.380 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:38.381 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:38.381 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:38.381 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:38.381 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:38.382 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:38.382 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.382 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.383 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.383 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:55:38.383 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:38.383 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.383 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.384 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.384 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:55:38.384 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:38.384 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.385 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.385 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.385 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:38.385 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.386 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.386 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.386 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:55:38.386 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:38.387 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.387 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.387 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.387 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:55:38.388 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:38.388 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.388 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.388 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.389 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:38.389 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.389 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.389 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.390 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:55:38.390 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:38.390 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.390 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.390 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.391 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:55:38.391 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:38.391 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.391 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.392 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.392 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:38.392 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.392 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.392 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.393 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:55:38.393 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:38.393 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.393 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.394 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.394 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:55:38.394 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:38.394 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.395 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.395 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.395 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:55:38.395 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:55:38.396 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:55:38.396 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:55:38.396 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:38.396 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:38.397 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:38.397 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.397 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.397 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.398 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:55:38.398 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:55:38.398 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:38.398 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:38.399 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:38.399 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:38.399 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:38.399 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:38.400 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:38.400 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:38.400 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:38.400 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:38.401 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:38.401 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.401 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.401 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.401 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:55:38.402 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:38.402 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.402 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.402 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.403 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:55:38.403 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:38.403 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.403 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.404 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.404 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:38.404 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.404 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.405 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.405 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:55:38.405 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:38.405 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.405 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.406 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.406 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:55:38.407 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:38.407 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.407 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.407 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.407 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:38.408 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.408 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.408 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.408 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:55:38.408 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:38.409 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.409 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.409 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.409 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:55:38.410 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:38.410 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.410 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.410 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.411 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:38.411 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.411 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.411 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.412 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:55:38.412 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:38.412 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.412 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.412 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.413 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:55:38.413 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:38.414 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.414 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.415 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.415 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:55:38.415 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:55:38.416 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:55:38.416 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:55:38.417 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:38.417 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:38.418 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:38.418 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.419 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.419 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.420 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:55:38.420 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:55:38.420 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:38.421 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:38.421 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:38.422 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:38.422 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:38.423 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:38.423 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:38.423 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:38.424 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:38.424 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:38.425 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:38.425 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.425 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.426 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.426 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:55:38.426 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:38.427 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.427 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.427 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.427 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:55:38.427 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:38.428 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.428 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.428 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.428 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:38.428 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.429 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.429 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.429 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:55:38.429 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:38.430 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.430 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.430 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.430 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:55:38.431 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:38.431 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.431 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.431 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.432 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:38.432 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.432 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.432 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.433 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:55:38.433 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:38.433 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.434 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.434 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.434 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:55:38.434 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:38.435 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.435 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.435 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.435 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:38.436 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.436 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.436 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.436 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:55:38.437 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:38.437 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.437 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.437 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.437 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:55:38.438 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:38.438 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.438 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.438 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.439 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:55:38.439 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:55:38.439 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:55:38.439 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:55:38.439 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:38.440 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:38.440 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:38.440 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.440 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.441 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.441 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:55:38.441 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:55:38.441 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:38.441 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:38.442 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:38.442 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:38.442 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:38.442 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:38.443 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:38.443 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 68, 2022-07-21T01:55:38.443 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:38.443 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:38.443 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:38.444 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.444 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.444 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.444 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:55:38.445 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:38.445 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.445 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.445 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.445 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:55:38.446 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:38.446 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.446 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.446 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.447 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:38.447 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.447 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.447 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.447 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:55:38.448 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:38.448 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.448 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.448 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.449 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:55:38.449 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:38.449 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.449 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.449 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.450 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:38.450 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.450 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.451 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.451 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:55:38.451 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:38.451 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.452 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.452 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.452 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:55:38.452 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:38.452 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.453 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.453 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.453 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:38.453 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:38.454 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.454 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:38.454 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:55:38.455 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:38.455 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.455 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.455 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:38.456 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:55:38.456 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:38.456 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.456 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.457 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.457 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:55:38.457 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:55:38.457 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:55:38.458 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:55:38.458 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:38.458 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:38.458 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:38.459 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:38.459 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.459 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:38.459 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:55:38.459 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.460 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:55:38.460 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:38.460 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:38.460 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:38.461 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:38.461 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:38.461 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:55:38.461 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:38.462 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.462 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.462 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:55:38.462 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:38.463 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:38.463 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:38.463 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:38.463 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:38.463 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:55:38.464 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:38.464 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.464 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.464 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:55:38.465 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:38.465 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:38.465 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:38.465 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:38.466 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:38.466 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:55:38.466 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:38.466 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.467 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.467 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:55:38.467 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:38.467 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:38.468 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:38.468 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:38.468 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:38.468 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:55:38.468 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:38.469 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.469 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.469 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:55:38.470 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:38.470 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:38.470 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:38.470 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:38.470 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:38.471 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:55:38.471 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:38.471 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.471 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.472 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:55:38.472 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:38.472 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:38.472 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:38.472 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:38.473 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:38.473 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:55:38.473 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:38.473 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.474 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.474 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:55:38.474 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:38.474 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:38.475 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:38.475 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:38.475 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:38.475 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:55:38.476 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:38.476 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.476 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:38.476 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:55:38.477 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:38.477 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:38.477 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:38.477 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:38.478 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:38.478 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:55:38.478 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:38.478 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.479 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:38.479 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:55:38.479 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:55:38.479 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:55:38.479 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:55:38.480 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:55:38.480 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6835/2461699636": "2022-07-22T01:55:31.502923+0000", 2022-07-21T01:55:38.480 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6834/2461699636": "2022-07-22T01:55:31.502923+0000" 2022-07-21T01:55:38.480 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.481 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:55:38.481 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:55:38.481 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:55:38.481 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:55:38.482 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:55:38.482 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:55:38.482 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:55:38.482 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:55:38.482 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:38.483 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:38.483 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:55:38.483 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:55:38.483 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:55:38.483 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:55:38.484 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:55:38.484 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:55:38.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/3389469077 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4840539e0 msgr2=0x7fe484055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4840539e0 0x7fe484055ea0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7fe4a001b000 tx=0x7fe4a0006f40).stop 2022-07-21T01:55:38.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/3389469077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4a413ee40 msgr2=0x7fe4a4144f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4a413ee40 0x7fe4a4144f80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fe494000ed0 tx=0x7fe494007c80).stop 2022-07-21T01:55:38.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/3389469077 shutdown_connections 2022-07-21T01:55:38.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4840539e0 0x7fe484055ea0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4a4142fd0 0x7fe4a40ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4a413ee40 0x7fe4a4144f80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 --2- 172.21.15.140:0/3389469077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4a40b1620 0x7fe4a40ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/3389469077 >> 172.21.15.140:0/3389469077 conn(0x7fe4a40b23d0 msgr2=0x7fe4a4140510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:38.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/3389469077 shutdown_connections 2022-07-21T01:55:38.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.239+0000 7fe4ac6cc700 1 -- 172.21.15.140:0/3389469077 wait complete. 2022-07-21T01:55:38.490 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:38.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 Processor -- start 2022-07-21T01:55:38.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 -- start start 2022-07-21T01:55:38.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbbdc0b5650 0x7fbbdc0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbbdc0af8e0 0x7fbbdc05bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbdc147010 con 0x7fbbdc0adff0 2022-07-21T01:55:38.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbdc1472b0 con 0x7fbbdc0b5650 2022-07-21T01:55:38.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbdc146d40 con 0x7fbbdc0af8e0 2022-07-21T01:55:38.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe1554700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbbdc0b5650 0x7fbbdc0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe1554700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbbdc0b5650 0x7fbbdc0b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37904/0 (socket says 172.21.15.140:37904) 2022-07-21T01:55:38.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe1554700 1 -- 172.21.15.140:0/1859077259 learned_addr learned my addr 172.21.15.140:0/1859077259 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:38.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe1d55700 1 --2- 172.21.15.140:0/1859077259 >> v2:172.21.15.152:3300/0 conn(0x7fbbdc0af8e0 0x7fbbdc05bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe1554700 1 -- 172.21.15.140:0/1859077259 >> v2:172.21.15.152:3300/0 conn(0x7fbbdc0af8e0 msgr2=0x7fbbdc05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:38.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe1554700 1 --2- 172.21.15.140:0/1859077259 >> v2:172.21.15.152:3300/0 conn(0x7fbbdc0af8e0 0x7fbbdc05bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe1554700 1 -- 172.21.15.140:0/1859077259 >> v1:172.21.15.152:6789/0 conn(0x7fbbdc0adff0 legacy=0x7fbbdc147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe1554700 1 -- 172.21.15.140:0/1859077259 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc0acae0 con 0x7fbbdc0b5650 2022-07-21T01:55:38.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe1554700 1 --2- 172.21.15.140:0/1859077259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbbdc0b5650 0x7fbbdc0b5a30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fbbd8002f70 tx=0x7fbbd8006eb0).ready entity=mon.0 client_cookie=a20f89a14beddef6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:38.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbcbfff700 1 -- 172.21.15.140:0/1859077259 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbd800d5c0 con 0x7fbbdc0b5650 2022-07-21T01:55:38.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbcbfff700 1 -- 172.21.15.140:0/1859077259 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd8000e90 con 0x7fbbdc0b5650 2022-07-21T01:55:38.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbcbfff700 1 -- 172.21.15.140:0/1859077259 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbd800d950 con 0x7fbbdc0b5650 2022-07-21T01:55:38.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 -- 172.21.15.140:0/1859077259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbbdc0b5650 msgr2=0x7fbbdc0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 --2- 172.21.15.140:0/1859077259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbbdc0b5650 0x7fbbdc0b5a30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fbbd8002f70 tx=0x7fbbd8006eb0).stop 2022-07-21T01:55:38.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 -- 172.21.15.140:0/1859077259 shutdown_connections 2022-07-21T01:55:38.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 --2- 172.21.15.140:0/1859077259 >> v2:172.21.15.152:3300/0 conn(0x7fbbdc0af8e0 0x7fbbdc05bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 --2- 172.21.15.140:0/1859077259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbbdc0b5650 0x7fbbdc0b5a30 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 -- 172.21.15.140:0/1859077259 >> 172.21.15.140:0/1859077259 conn(0x7fbbdc0634e0 msgr2=0x7fbbdc0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:38.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 -- 172.21.15.140:0/1859077259 shutdown_connections 2022-07-21T01:55:38.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.603+0000 7fbbe2556700 1 -- 172.21.15.140:0/1859077259 wait complete. 2022-07-21T01:55:38.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe2556700 1 Processor -- start 2022-07-21T01:55:38.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe2556700 1 -- start start 2022-07-21T01:55:38.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe2556700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbbdc0adff0 0x7fbbdc0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe2556700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbbdc0af8e0 0x7fbbdc10f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe2556700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbbdc0b5650 0x7fbbdc10f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe2556700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc067160 con 0x7fbbdc0b5650 2022-07-21T01:55:38.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe2556700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc111140 con 0x7fbbdc0adff0 2022-07-21T01:55:38.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe2556700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc1113e0 con 0x7fbbdc0af8e0 2022-07-21T01:55:38.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe1d55700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbbdc0b5650 0x7fbbdc10f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe1d55700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbbdc0b5650 0x7fbbdc10f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37912/0 (socket says 172.21.15.140:37912) 2022-07-21T01:55:38.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe1d55700 1 -- 172.21.15.140:0/2158947494 learned_addr learned my addr 172.21.15.140:0/2158947494 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:38.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe1554700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbbdc0adff0 0x7fbbdc0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe0d53700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbbdc0af8e0 0x7fbbdc10f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe1d55700 1 -- 172.21.15.140:0/2158947494 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbbdc0af8e0 msgr2=0x7fbbdc10f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe1d55700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbbdc0af8e0 0x7fbbdc10f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe1d55700 1 -- 172.21.15.140:0/2158947494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbbdc0adff0 msgr2=0x7fbbdc0b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe1d55700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbbdc0adff0 0x7fbbdc0b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe1d55700 1 -- 172.21.15.140:0/2158947494 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd800b1e0 con 0x7fbbdc0b5650 2022-07-21T01:55:38.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe1d55700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbbdc0b5650 0x7fbbdc10f5b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7fbbd4006e00 tx=0x7fbbd40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:38.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbca7fc700 1 -- 172.21.15.140:0/2158947494 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbd400eeb0 con 0x7fbbdc0b5650 2022-07-21T01:55:38.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbca7fc700 1 -- 172.21.15.140:0/2158947494 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd400c850 con 0x7fbbdc0b5650 2022-07-21T01:55:38.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe2556700 1 -- 172.21.15.140:0/2158947494 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbdc1104e0 con 0x7fbbdc0b5650 2022-07-21T01:55:38.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbca7fc700 1 -- 172.21.15.140:0/2158947494 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbd400cbb0 con 0x7fbbdc0b5650 2022-07-21T01:55:38.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.607+0000 7fbbe2556700 1 -- 172.21.15.140:0/2158947494 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbdc066f20 con 0x7fbbdc0b5650 2022-07-21T01:55:38.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.611+0000 7fbbe2556700 1 -- 172.21.15.140:0/2158947494 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbdc0574e0 con 0x7fbbdc0b5650 2022-07-21T01:55:38.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.611+0000 7fbbca7fc700 1 -- 172.21.15.140:0/2158947494 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbbd4022070 con 0x7fbbdc0b5650 2022-07-21T01:55:38.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.611+0000 7fbbca7fc700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbbbc053b00 0x7fbbbc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.611+0000 7fbbca7fc700 1 -- 172.21.15.140:0/2158947494 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fbbd406c170 con 0x7fbbdc0b5650 2022-07-21T01:55:38.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.611+0000 7fbbe1554700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbbbc053b00 0x7fbbbc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.611+0000 7fbbe1554700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbbbc053b00 0x7fbbbc055fc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fbbd801c000 tx=0x7fbbd8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:38.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.619+0000 7fbbca7fc700 1 -- 172.21.15.140:0/2158947494 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbbd4043310 con 0x7fbbdc0b5650 2022-07-21T01:55:38.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.783+0000 7fbbe2556700 1 -- 172.21.15.140:0/2158947494 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbdc067ea0 con 0x7fbbdc0b5650 2022-07-21T01:55:38.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.783+0000 7fbbca7fc700 1 -- 172.21.15.140:0/2158947494 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+1215 (secure 0 0 0) 0x7fbbd4046930 con 0x7fbbdc0b5650 2022-07-21T01:55:38.787 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:38.787 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/2489766347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2489766347},{"type":"v1","addr":"172.21.15.152:6833","nonce":2489766347}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6921,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/4103099115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":4103099115},{"type":"v1","addr":"172.21.15.140:6835","nonce":4103099115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":39}],"filesystems":[]} 2022-07-21T01:55:38.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 -- 172.21.15.140:0/2158947494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbbbc053b00 msgr2=0x7fbbbc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbbbc053b00 0x7fbbbc055fc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fbbd801c000 tx=0x7fbbd8015000).stop 2022-07-21T01:55:38.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 -- 172.21.15.140:0/2158947494 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbbdc0b5650 msgr2=0x7fbbdc10f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbbdc0b5650 0x7fbbdc10f5b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7fbbd4006e00 tx=0x7fbbd40091d0).stop 2022-07-21T01:55:38.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 -- 172.21.15.140:0/2158947494 shutdown_connections 2022-07-21T01:55:38.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbbbc053b00 0x7fbbbc055fc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbbdc0b5650 0x7fbbdc10f5b0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbbdc0af8e0 0x7fbbdc10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 --2- 172.21.15.140:0/2158947494 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbbdc0adff0 0x7fbbdc0b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 -- 172.21.15.140:0/2158947494 >> 172.21.15.140:0/2158947494 conn(0x7fbbdc0634e0 msgr2=0x7fbbdc0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:38.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 -- 172.21.15.140:0/2158947494 shutdown_connections 2022-07-21T01:55:38.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.787+0000 7fbbe2556700 1 -- 172.21.15.140:0/2158947494 wait complete. 2022-07-21T01:55:38.792 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 39 2022-07-21T01:55:38.799 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:38.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.895+0000 7f839f2f8700 1 Processor -- start 2022-07-21T01:55:38.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.895+0000 7f839f2f8700 1 -- start start 2022-07-21T01:55:38.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f83981424e0 0x7f83980b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f83980aea10 0x7f83980ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83981433d0 con 0x7f83980b0570 2022-07-21T01:55:38.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8398143b10 con 0x7f83981424e0 2022-07-21T01:55:38.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8398143670 con 0x7f83980aea10 2022-07-21T01:55:38.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839daf5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f83981424e0 0x7f83980b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839daf5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f83981424e0 0x7f83980b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37918/0 (socket says 172.21.15.140:37918) 2022-07-21T01:55:38.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839daf5700 1 -- 172.21.15.140:0/241341695 learned_addr learned my addr 172.21.15.140:0/241341695 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:38.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839eaf7700 1 --2- 172.21.15.140:0/241341695 >> v2:172.21.15.152:3300/0 conn(0x7f83980aea10 0x7f83980ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839daf5700 1 -- 172.21.15.140:0/241341695 >> v2:172.21.15.152:3300/0 conn(0x7f83980aea10 msgr2=0x7f83980ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:38.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839daf5700 1 --2- 172.21.15.140:0/241341695 >> v2:172.21.15.152:3300/0 conn(0x7f83980aea10 0x7f83980ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839daf5700 1 -- 172.21.15.140:0/241341695 >> v1:172.21.15.152:6789/0 conn(0x7f83980b0570 legacy=0x7f8398141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839daf5700 1 -- 172.21.15.140:0/241341695 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83980b1010 con 0x7f83981424e0 2022-07-21T01:55:38.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839daf5700 1 --2- 172.21.15.140:0/241341695 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83981424e0 0x7f83980b0950 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f8394002f70 tx=0x7f8394006eb0).ready entity=mon.0 client_cookie=214f93d659a7e353 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:38.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839d2f4700 1 -- 172.21.15.140:0/241341695 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839400d5c0 con 0x7f83981424e0 2022-07-21T01:55:38.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839d2f4700 1 -- 172.21.15.140:0/241341695 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8394000e90 con 0x7f83981424e0 2022-07-21T01:55:38.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839d2f4700 1 -- 172.21.15.140:0/241341695 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839400d950 con 0x7f83981424e0 2022-07-21T01:55:38.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 -- 172.21.15.140:0/241341695 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83981424e0 msgr2=0x7f83980b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 --2- 172.21.15.140:0/241341695 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83981424e0 0x7f83980b0950 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f8394002f70 tx=0x7f8394006eb0).stop 2022-07-21T01:55:38.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 -- 172.21.15.140:0/241341695 shutdown_connections 2022-07-21T01:55:38.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 --2- 172.21.15.140:0/241341695 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83981424e0 0x7f83980b0950 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 --2- 172.21.15.140:0/241341695 >> v2:172.21.15.152:3300/0 conn(0x7f83980aea10 0x7f83980ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 -- 172.21.15.140:0/241341695 >> 172.21.15.140:0/241341695 conn(0x7f83980b9360 msgr2=0x7f83980b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:38.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 -- 172.21.15.140:0/241341695 shutdown_connections 2022-07-21T01:55:38.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.899+0000 7f839f2f8700 1 -- 172.21.15.140:0/241341695 wait complete. 2022-07-21T01:55:38.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839f2f8700 1 Processor -- start 2022-07-21T01:55:38.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839f2f8700 1 -- start start 2022-07-21T01:55:38.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839f2f8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83980aea10 0x7f83981141e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839f2f8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83980b0570 0x7f83980b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839f2f8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83981424e0 0x7f83980b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839f2f8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8398114f10 con 0x7f83981424e0 2022-07-21T01:55:38.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839f2f8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8398113580 con 0x7f83980b0570 2022-07-21T01:55:38.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839f2f8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83981149d0 con 0x7f83980aea10 2022-07-21T01:55:38.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839eaf7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83981424e0 0x7f83980b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839eaf7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83981424e0 0x7f83980b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37926/0 (socket says 172.21.15.140:37926) 2022-07-21T01:55:38.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839eaf7700 1 -- 172.21.15.140:0/291114753 learned_addr learned my addr 172.21.15.140:0/291114753 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:38.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839daf5700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83980b0570 0x7f83980b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839e2f6700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83980aea10 0x7f83981141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839eaf7700 1 -- 172.21.15.140:0/291114753 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83980aea10 msgr2=0x7f83981141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839eaf7700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83980aea10 0x7f83981141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839eaf7700 1 -- 172.21.15.140:0/291114753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83980b0570 msgr2=0x7f83980b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:38.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839eaf7700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83980b0570 0x7f83980b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:38.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839eaf7700 1 -- 172.21.15.140:0/291114753 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839400b1e0 con 0x7f83981424e0 2022-07-21T01:55:38.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839eaf7700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83981424e0 0x7f83980b27d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f8390006e00 tx=0x7f83900091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:38.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f83877fe700 1 -- 172.21.15.140:0/291114753 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839000c640 con 0x7f83981424e0 2022-07-21T01:55:38.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839f2f8700 1 -- 172.21.15.140:0/291114753 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8398114c70 con 0x7f83981424e0 2022-07-21T01:55:38.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f83877fe700 1 -- 172.21.15.140:0/291114753 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839000c7a0 con 0x7f83981424e0 2022-07-21T01:55:38.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f83877fe700 1 -- 172.21.15.140:0/291114753 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f839000cb00 con 0x7f83981424e0 2022-07-21T01:55:38.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.903+0000 7f839f2f8700 1 -- 172.21.15.140:0/291114753 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83980b3450 con 0x7f83981424e0 2022-07-21T01:55:38.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.907+0000 7f839f2f8700 1 -- 172.21.15.140:0/291114753 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83980ace30 con 0x7f83981424e0 2022-07-21T01:55:38.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.907+0000 7f83877fe700 1 -- 172.21.15.140:0/291114753 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f839000cc60 con 0x7f83981424e0 2022-07-21T01:55:38.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.907+0000 7f83877fe700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8378053b00 0x7f8378055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:38.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.907+0000 7f839e2f6700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8378053b00 0x7f8378055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:38.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.907+0000 7f83877fe700 1 -- 172.21.15.140:0/291114753 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f839006beb0 con 0x7f83981424e0 2022-07-21T01:55:38.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.907+0000 7f839e2f6700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8378053b00 0x7f8378055fc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f838800d000 tx=0x7f8388000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:38.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:38.911+0000 7f83877fe700 1 -- 172.21.15.140:0/291114753 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8390043010 con 0x7f83981424e0 2022-07-21T01:55:39.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 -- 172.21.15.140:0/291114753 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83980574e0 con 0x7f83981424e0 2022-07-21T01:55:39.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f83877fe700 1 -- 172.21.15.140:0/291114753 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+1215 (secure 0 0 0) 0x7f8390046630 con 0x7f83981424e0 2022-07-21T01:55:39.076 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:39.076 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/2489766347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":2489766347},{"type":"v1","addr":"172.21.15.152:6833","nonce":2489766347}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":34},{"gid":6921,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/4103099115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":4103099115},{"type":"v1","addr":"172.21.15.140:6835","nonce":4103099115}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":39}],"filesystems":[]} 2022-07-21T01:55:39.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 -- 172.21.15.140:0/291114753 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8378053b00 msgr2=0x7f8378055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:39.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8378053b00 0x7f8378055fc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f838800d000 tx=0x7f8388000bc0).stop 2022-07-21T01:55:39.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 -- 172.21.15.140:0/291114753 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83981424e0 msgr2=0x7f83980b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:39.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83981424e0 0x7f83980b27d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f8390006e00 tx=0x7f83900091d0).stop 2022-07-21T01:55:39.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 -- 172.21.15.140:0/291114753 shutdown_connections 2022-07-21T01:55:39.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8378053b00 0x7f8378055fc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:39.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83981424e0 0x7f83980b27d0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:39.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83980b0570 0x7f83980b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:39.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 --2- 172.21.15.140:0/291114753 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83980aea10 0x7f83981141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:39.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 -- 172.21.15.140:0/291114753 >> 172.21.15.140:0/291114753 conn(0x7f83980b9360 msgr2=0x7f839813f180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:39.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 -- 172.21.15.140:0/291114753 shutdown_connections 2022-07-21T01:55:39.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.075+0000 7f839f2f8700 1 -- 172.21.15.140:0/291114753 wait complete. 2022-07-21T01:55:39.082 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 39 2022-07-21T01:55:39.088 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:55:39.089 INFO:tasks.ceph.mds.a:Stopping old one... 2022-07-21T01:55:39.089 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:55:39.090 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:55:39.090 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:55:39.090 INFO:tasks.ceph.mds.b:Stopping old one... 2022-07-21T01:55:39.091 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:55:39.091 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:55:39.099 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:55:39.099 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:55:39.102 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:55:39.102 DEBUG:teuthology.parallel:result is None 2022-07-21T01:55:39.106 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:55:39.107 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:55:39.109 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:55:39.110 DEBUG:teuthology.parallel:result is None 2022-07-21T01:55:39.110 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:55:39.170 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:55:39.178 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:55:39.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa71dfed700 1 Processor -- start 2022-07-21T01:55:39.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa71dfed700 1 -- start start 2022-07-21T01:55:39.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa71dfed700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa718140060 0x7fa7180ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:39.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa71dfed700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa7180b4af0 0x7fa7180b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:39.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa71dfed700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7180677d0 con 0x7fa7180b6200 2022-07-21T01:55:39.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa71dfed700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7180b65e0 con 0x7fa718140060 2022-07-21T01:55:39.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa71dfed700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7180aca50 con 0x7fa7180b4af0 2022-07-21T01:55:39.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa71cfeb700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa7180b6200 0x7fa71813f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52088/0 (socket says 172.21.15.140:52088) 2022-07-21T01:55:39.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa71cfeb700 1 -- 172.21.15.140:0/3713791217 learned_addr learned my addr 172.21.15.140:0/3713791217 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:39.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa717fff700 1 --2- 172.21.15.140:0/3713791217 >> v2:172.21.15.140:3300/0 conn(0x7fa718140060 0x7fa7180ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:39.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa71d7ec700 1 --2- 172.21.15.140:0/3713791217 >> v2:172.21.15.152:3300/0 conn(0x7fa7180b4af0 0x7fa7180b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:39.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa717fff700 1 -- 172.21.15.140:0/3713791217 >> v2:172.21.15.152:3300/0 conn(0x7fa7180b4af0 msgr2=0x7fa7180b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:39.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa717fff700 1 --2- 172.21.15.140:0/3713791217 >> v2:172.21.15.152:3300/0 conn(0x7fa7180b4af0 0x7fa7180b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:39.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa717fff700 1 -- 172.21.15.140:0/3713791217 >> v1:172.21.15.140:6789/0 conn(0x7fa7180b6200 legacy=0x7fa71813f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:39.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa717fff700 1 -- 172.21.15.140:0/3713791217 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa718067530 con 0x7fa718140060 2022-07-21T01:55:39.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.219+0000 7fa71d7ec700 1 --2- 172.21.15.140:0/3713791217 >> v2:172.21.15.152:3300/0 conn(0x7fa7180b4af0 0x7fa7180b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:55:39.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa717fff700 1 --2- 172.21.15.140:0/3713791217 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa718140060 0x7fa7180ac560 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7fa70c002f70 tx=0x7fa70c006df0).ready entity=mon.0 client_cookie=f652478f4e6aecc8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:39.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa7177fe700 1 -- 172.21.15.140:0/3713791217 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa70c00d5c0 con 0x7fa718140060 2022-07-21T01:55:39.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa7177fe700 1 -- 172.21.15.140:0/3713791217 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c000e90 con 0x7fa718140060 2022-07-21T01:55:39.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa7177fe700 1 -- 172.21.15.140:0/3713791217 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa70c00d910 con 0x7fa718140060 2022-07-21T01:55:39.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 -- 172.21.15.140:0/3713791217 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa718140060 msgr2=0x7fa7180ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:39.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 --2- 172.21.15.140:0/3713791217 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa718140060 0x7fa7180ac560 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7fa70c002f70 tx=0x7fa70c006df0).stop 2022-07-21T01:55:39.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 -- 172.21.15.140:0/3713791217 shutdown_connections 2022-07-21T01:55:39.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 --2- 172.21.15.140:0/3713791217 >> v2:172.21.15.152:3300/0 conn(0x7fa7180b4af0 0x7fa7180b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:39.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 --2- 172.21.15.140:0/3713791217 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa718140060 0x7fa7180ac560 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:39.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 -- 172.21.15.140:0/3713791217 >> 172.21.15.140:0/3713791217 conn(0x7fa718063d10 msgr2=0x7fa7180b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:39.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 -- 172.21.15.140:0/3713791217 shutdown_connections 2022-07-21T01:55:39.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 -- 172.21.15.140:0/3713791217 wait complete. 2022-07-21T01:55:39.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 Processor -- start 2022-07-21T01:55:39.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 -- start start 2022-07-21T01:55:39.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7180b4af0 0x7fa7180665f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:39.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7180b6200 0x7fa718152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:39.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa718140060 0x7fa718152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:39.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718151f40 con 0x7fa718140060 2022-07-21T01:55:39.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa718060830 con 0x7fa7180b4af0 2022-07-21T01:55:39.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71dfed700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa718155690 con 0x7fa7180b6200 2022-07-21T01:55:39.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71d7ec700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa718140060 0x7fa718152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:39.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71d7ec700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa718140060 0x7fa718152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37952/0 (socket says 172.21.15.140:37952) 2022-07-21T01:55:39.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71d7ec700 1 -- 172.21.15.140:0/2140363468 learned_addr learned my addr 172.21.15.140:0/2140363468 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:39.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa717fff700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7180b6200 0x7fa718152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:39.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71cfeb700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7180b4af0 0x7fa7180665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:39.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71d7ec700 1 -- 172.21.15.140:0/2140363468 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7180b6200 msgr2=0x7fa718152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:39.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71d7ec700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7180b6200 0x7fa718152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:39.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71d7ec700 1 -- 172.21.15.140:0/2140363468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7180b4af0 msgr2=0x7fa7180665f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:39.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71d7ec700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7180b4af0 0x7fa7180665f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:39.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa71d7ec700 1 -- 172.21.15.140:0/2140363468 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c00b180 con 0x7fa718140060 2022-07-21T01:55:39.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.223+0000 7fa717fff700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7180b6200 0x7fa718152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:39.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa71d7ec700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa718140060 0x7fa718152860 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7fa710007000 tx=0x7fa710008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:39.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa715ffb700 1 -- 172.21.15.140:0/2140363468 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa710015070 con 0x7fa718140060 2022-07-21T01:55:39.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa715ffb700 1 -- 172.21.15.140:0/2140363468 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa710010580 con 0x7fa718140060 2022-07-21T01:55:39.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa715ffb700 1 -- 172.21.15.140:0/2140363468 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7100108e0 con 0x7fa718140060 2022-07-21T01:55:39.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa71dfed700 1 -- 172.21.15.140:0/2140363468 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa718153070 con 0x7fa718140060 2022-07-21T01:55:39.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa71dfed700 1 -- 172.21.15.140:0/2140363468 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa718152d50 con 0x7fa718140060 2022-07-21T01:55:39.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa71dfed700 1 -- 172.21.15.140:0/2140363468 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa718155890 con 0x7fa718140060 2022-07-21T01:55:39.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa715ffb700 1 -- 172.21.15.140:0/2140363468 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa710010a40 con 0x7fa718140060 2022-07-21T01:55:39.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa715ffb700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6f8053a70 0x7fa6f8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:39.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa715ffb700 1 -- 172.21.15.140:0/2140363468 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fa71006bf20 con 0x7fa718140060 2022-07-21T01:55:39.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa71cfeb700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6f8053a70 0x7fa6f8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:39.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.227+0000 7fa71cfeb700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6f8053a70 0x7fa6f8055f30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fa70800d000 tx=0x7fa708009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:39.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.235+0000 7fa715ffb700 1 -- 172.21.15.140:0/2140363468 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa710043080 con 0x7fa718140060 2022-07-21T01:55:39.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:39.371+0000 7fa71dfed700 1 -- 172.21.15.140:0/2140363468 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fa7180b5220 con 0x7fa718140060 2022-07-21T01:55:40.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.203+0000 7fa715ffb700 1 -- 172.21.15.140:0/2140363468 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v89) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa7100466a0 con 0x7fa718140060 2022-07-21T01:55:40.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 -- 172.21.15.140:0/2140363468 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6f8053a70 msgr2=0x7fa6f8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6f8053a70 0x7fa6f8055f30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7fa70800d000 tx=0x7fa708009040).stop 2022-07-21T01:55:40.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 -- 172.21.15.140:0/2140363468 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa718140060 msgr2=0x7fa718152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa718140060 0x7fa718152860 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7fa710007000 tx=0x7fa710008040).stop 2022-07-21T01:55:40.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 -- 172.21.15.140:0/2140363468 shutdown_connections 2022-07-21T01:55:40.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa6f8053a70 0x7fa6f8055f30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa718140060 0x7fa718152860 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa7180b6200 0x7fa718152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 --2- 172.21.15.140:0/2140363468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7180b4af0 0x7fa7180665f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 -- 172.21.15.140:0/2140363468 >> 172.21.15.140:0/2140363468 conn(0x7fa718063d10 msgr2=0x7fa71813e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:40.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 -- 172.21.15.140:0/2140363468 shutdown_connections 2022-07-21T01:55:40.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.207+0000 7fa71dfed700 1 -- 172.21.15.140:0/2140363468 wait complete. 2022-07-21T01:55:40.212 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:55:40.224 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:55:40.225 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:55:40.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa55efaa700 1 Processor -- start 2022-07-21T01:55:40.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa55efaa700 1 -- start start 2022-07-21T01:55:40.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa55efaa700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5580b1480 0x7fa55813fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa55efaa700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa558060ed0 0x7fa558140360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa55efaa700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa558140850 con 0x7fa55805bd90 2022-07-21T01:55:40.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa55efaa700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa558061800 con 0x7fa558060ed0 2022-07-21T01:55:40.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa55efaa700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55805c460 con 0x7fa5580b1480 2022-07-21T01:55:40.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa557fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa558060ed0 0x7fa558140360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa55d547700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa55805bd90 0x7fa55813c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52106/0 (socket says 172.21.15.140:52106) 2022-07-21T01:55:40.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa55d547700 1 -- 172.21.15.140:0/3701427742 learned_addr learned my addr 172.21.15.140:0/3701427742 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:40.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa557fff700 1 --2- 172.21.15.140:0/3701427742 >> v2:172.21.15.140:3300/0 conn(0x7fa558060ed0 0x7fa558140360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37958/0 (socket says 172.21.15.140:37958) 2022-07-21T01:55:40.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa55cd46700 1 --2- 172.21.15.140:0/3701427742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5580b1480 0x7fa55813fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa557fff700 1 -- 172.21.15.140:0/3701427742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5580b1480 msgr2=0x7fa55813fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:40.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa557fff700 1 --2- 172.21.15.140:0/3701427742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5580b1480 0x7fa55813fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa557fff700 1 -- 172.21.15.140:0/3701427742 >> v1:172.21.15.140:6789/0 conn(0x7fa55805bd90 legacy=0x7fa55813c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa557fff700 1 -- 172.21.15.140:0/3701427742 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558143a20 con 0x7fa558060ed0 2022-07-21T01:55:40.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa557fff700 1 --2- 172.21.15.140:0/3701427742 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa558060ed0 0x7fa558140360 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7fa558140a00 tx=0x7fa540006df0).ready entity=mon.0 client_cookie=87830b3ebaa4921c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:40.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa5577fe700 1 -- 172.21.15.140:0/3701427742 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa54000d570 con 0x7fa558060ed0 2022-07-21T01:55:40.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa5577fe700 1 -- 172.21.15.140:0/3701427742 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa540000e90 con 0x7fa558060ed0 2022-07-21T01:55:40.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.331+0000 7fa5577fe700 1 -- 172.21.15.140:0/3701427742 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa54000d8c0 con 0x7fa558060ed0 2022-07-21T01:55:40.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.335+0000 7fa55efaa700 1 -- 172.21.15.140:0/3701427742 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa558060ed0 msgr2=0x7fa558140360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.335+0000 7fa55efaa700 1 --2- 172.21.15.140:0/3701427742 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa558060ed0 0x7fa558140360 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7fa558140a00 tx=0x7fa540006df0).stop 2022-07-21T01:55:40.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.335+0000 7fa55efaa700 1 -- 172.21.15.140:0/3701427742 shutdown_connections 2022-07-21T01:55:40.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.335+0000 7fa55efaa700 1 --2- 172.21.15.140:0/3701427742 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa558060ed0 0x7fa558140360 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.335+0000 7fa55efaa700 1 --2- 172.21.15.140:0/3701427742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5580b1480 0x7fa55813fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.335+0000 7fa55efaa700 1 -- 172.21.15.140:0/3701427742 >> 172.21.15.140:0/3701427742 conn(0x7fa5580b23d0 msgr2=0x7fa5580b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:40.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.335+0000 7fa55efaa700 1 -- 172.21.15.140:0/3701427742 shutdown_connections 2022-07-21T01:55:40.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.335+0000 7fa55efaa700 1 -- 172.21.15.140:0/3701427742 wait complete. 2022-07-21T01:55:40.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.335+0000 7fa55efaa700 1 Processor -- start 2022-07-21T01:55:40.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.335+0000 7fa55efaa700 1 -- start start 2022-07-21T01:55:40.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55efaa700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa55805bd90 0x7fa558093fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55efaa700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa558060ed0 0x7fa5580aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55efaa700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5580b1480 0x7fa5581536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55efaa700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580913e0 con 0x7fa5580b1480 2022-07-21T01:55:40.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55efaa700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580918b0 con 0x7fa558060ed0 2022-07-21T01:55:40.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55efaa700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa558153ba0 con 0x7fa55805bd90 2022-07-21T01:55:40.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55d547700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5580b1480 0x7fa5581536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55d547700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5580b1480 0x7fa5581536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37966/0 (socket says 172.21.15.140:37966) 2022-07-21T01:55:40.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55d547700 1 -- 172.21.15.140:0/3239007617 learned_addr learned my addr 172.21.15.140:0/3239007617 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:40.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa557fff700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa558060ed0 0x7fa5580aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55cd46700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa55805bd90 0x7fa558093fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55d547700 1 -- 172.21.15.140:0/3239007617 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa55805bd90 msgr2=0x7fa558093fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55d547700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa55805bd90 0x7fa558093fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55d547700 1 -- 172.21.15.140:0/3239007617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa558060ed0 msgr2=0x7fa5580aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55d547700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa558060ed0 0x7fa5580aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55d547700 1 -- 172.21.15.140:0/3239007617 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54000b180 con 0x7fa5580b1480 2022-07-21T01:55:40.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55d547700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5580b1480 0x7fa5581536b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7fa558062e30 tx=0x7fa548007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:40.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa555ffb700 1 -- 172.21.15.140:0/3239007617 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5480070d0 con 0x7fa5580b1480 2022-07-21T01:55:40.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa555ffb700 1 -- 172.21.15.140:0/3239007617 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa548007230 con 0x7fa5580b1480 2022-07-21T01:55:40.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55efaa700 1 -- 172.21.15.140:0/3239007617 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa558091610 con 0x7fa5580b1480 2022-07-21T01:55:40.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa555ffb700 1 -- 172.21.15.140:0/3239007617 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5480055f0 con 0x7fa5580b1480 2022-07-21T01:55:40.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.339+0000 7fa55efaa700 1 -- 172.21.15.140:0/3239007617 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa558092040 con 0x7fa5580b1480 2022-07-21T01:55:40.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.343+0000 7fa55efaa700 1 -- 172.21.15.140:0/3239007617 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5580911b0 con 0x7fa5580b1480 2022-07-21T01:55:40.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.343+0000 7fa555ffb700 1 -- 172.21.15.140:0/3239007617 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa548010070 con 0x7fa5580b1480 2022-07-21T01:55:40.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.343+0000 7fa555ffb700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa53c053b30 0x7fa53c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.343+0000 7fa555ffb700 1 -- 172.21.15.140:0/3239007617 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa54806b000 con 0x7fa5580b1480 2022-07-21T01:55:40.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.343+0000 7fa55cd46700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa53c053b30 0x7fa53c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.343+0000 7fa55cd46700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa53c053b30 0x7fa53c055ff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fa55000b000 tx=0x7fa5500091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:40.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.351+0000 7fa555ffb700 1 -- 172.21.15.140:0/3239007617 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa5580911b0 con 0x7fa5580b1480 2022-07-21T01:55:40.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.515+0000 7fa55efaa700 1 -- 172.21.15.140:0/3239007617 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa5580911b0 con 0x7fa5580b1480 2022-07-21T01:55:40.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.515+0000 7fa555ffb700 1 -- 172.21.15.140:0/3239007617 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v32) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fa5580911b0 con 0x7fa5580b1480 2022-07-21T01:55:40.517 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:40.517 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:55:40.517 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:55:40.518 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.518 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:55:40.518 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:55:40.518 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.519 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:55:40.519 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:40.519 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:40.519 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.519 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.520 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.520 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.520 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:55:40.520 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:55:40.521 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.521 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:55:40.521 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:40.521 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:40.521 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.522 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.522 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.522 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.522 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:55:40.523 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:55:40.523 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.523 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:40.523 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:40.524 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.524 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.524 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.524 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.524 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:55:40.525 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:55:40.525 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.525 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:40.525 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:40.525 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.526 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.526 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.526 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.526 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:55:40.527 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:55:40.527 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.527 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:40.527 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:40.527 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.528 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.528 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.528 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.528 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:55:40.529 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:55:40.529 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.529 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:40.529 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:40.529 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.530 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.530 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.530 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.530 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:55:40.531 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:55:40.531 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.531 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:40.531 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:40.531 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.532 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.532 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.532 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.532 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:55:40.532 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:55:40.533 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.533 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:40.533 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:40.533 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.534 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.534 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.534 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.534 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:55:40.534 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:55:40.535 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.535 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:40.535 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:40.535 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.535 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.536 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.536 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.536 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:55:40.536 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:55:40.537 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.537 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:40.537 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:40.537 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.537 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.538 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.538 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.538 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:55:40.538 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:55:40.539 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.539 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:55:40.539 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:55:40.539 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:55:40.539 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.540 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.540 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.540 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:55:40.540 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:55:40.541 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.541 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:55:40.541 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:55:40.541 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:55:40.541 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.542 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.542 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.542 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:55:40.542 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:55:40.543 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.543 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:55:40.543 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:55:40.543 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:55:40.544 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.544 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.544 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.544 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:55:40.544 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:55:40.545 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.545 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:55:40.545 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:55:40.545 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:55:40.545 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.546 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.546 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.546 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.546 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:55:40.547 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:55:40.547 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.547 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:55:40.547 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:55:40.547 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.548 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.548 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:40.548 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:40.548 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:55:40.548 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:55:40.549 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:55:40.549 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:55:40.549 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:55:40.549 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:55:40.550 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.550 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:40.550 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:40.550 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:55:40.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 -- 172.21.15.140:0/3239007617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa53c053b30 msgr2=0x7fa53c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa53c053b30 0x7fa53c055ff0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fa55000b000 tx=0x7fa5500091d0).stop 2022-07-21T01:55:40.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 -- 172.21.15.140:0/3239007617 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5580b1480 msgr2=0x7fa5581536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5580b1480 0x7fa5581536b0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7fa558062e30 tx=0x7fa548007bb0).stop 2022-07-21T01:55:40.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 -- 172.21.15.140:0/3239007617 shutdown_connections 2022-07-21T01:55:40.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa53c053b30 0x7fa53c055ff0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5580b1480 0x7fa5581536b0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa558060ed0 0x7fa5580aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 --2- 172.21.15.140:0/3239007617 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa55805bd90 0x7fa558093fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 -- 172.21.15.140:0/3239007617 >> 172.21.15.140:0/3239007617 conn(0x7fa5580b23d0 msgr2=0x7fa5580ac2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:40.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 -- 172.21.15.140:0/3239007617 shutdown_connections 2022-07-21T01:55:40.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.519+0000 7fa55efaa700 1 -- 172.21.15.140:0/3239007617 wait complete. 2022-07-21T01:55:40.555 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:40.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7ac9e2700 1 Processor -- start 2022-07-21T01:55:40.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7ac9e2700 1 -- start start 2022-07-21T01:55:40.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7ac9e2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc7a80b5650 0x7fc7a80b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7ac9e2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc7a80adff0 0x7fc7a8147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7ac9e2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7a80af8e0 0x7fc7a805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7ac9e2700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a8147010 con 0x7fc7a80adff0 2022-07-21T01:55:40.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7ac9e2700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a81472b0 con 0x7fc7a80b5650 2022-07-21T01:55:40.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7ac9e2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a8146d40 con 0x7fc7a80af8e0 2022-07-21T01:55:40.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a6ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc7a80adff0 0x7fc7a8147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a6ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc7a80adff0 0x7fc7a8147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37970/0 (socket says 172.21.15.140:37970) 2022-07-21T01:55:40.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a6ffd700 1 -- 172.21.15.140:0/2421842622 learned_addr learned my addr 172.21.15.140:0/2421842622 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:40.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a77fe700 1 --2- 172.21.15.140:0/2421842622 >> v2:172.21.15.152:3300/0 conn(0x7fc7a80b5650 0x7fc7a80b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a7fff700 1 --2- 172.21.15.140:0/2421842622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7a80af8e0 0x7fc7a805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a6ffd700 1 -- 172.21.15.140:0/2421842622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7a80af8e0 msgr2=0x7fc7a805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:40.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a6ffd700 1 --2- 172.21.15.140:0/2421842622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7a80af8e0 0x7fc7a805bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a6ffd700 1 -- 172.21.15.140:0/2421842622 >> v2:172.21.15.152:3300/0 conn(0x7fc7a80b5650 msgr2=0x7fc7a80b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:40.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a6ffd700 1 --2- 172.21.15.140:0/2421842622 >> v2:172.21.15.152:3300/0 conn(0x7fc7a80b5650 0x7fc7a80b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a6ffd700 1 -- 172.21.15.140:0/2421842622 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a8147c40 con 0x7fc7a80adff0 2022-07-21T01:55:40.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a77fe700 1 --2- 172.21.15.140:0/2421842622 >> v2:172.21.15.152:3300/0 conn(0x7fc7a80b5650 0x7fc7a80b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:40.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.659+0000 7fc7a6ffd700 1 --2- 172.21.15.140:0/2421842622 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7a80adff0 0x7fc7a8147750 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7fc7980032f0 tx=0x7fc798009250).ready entity=mon.0 client_cookie=c34862277d29eeb8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:40.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7a67fc700 1 -- 172.21.15.140:0/2421842622 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc79800b870 con 0x7fc7a80adff0 2022-07-21T01:55:40.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7a67fc700 1 -- 172.21.15.140:0/2421842622 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79800b9d0 con 0x7fc7a80adff0 2022-07-21T01:55:40.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7a67fc700 1 -- 172.21.15.140:0/2421842622 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc79800bb40 con 0x7fc7a80adff0 2022-07-21T01:55:40.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/2421842622 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7a80adff0 msgr2=0x7fc7a8147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7ac9e2700 1 --2- 172.21.15.140:0/2421842622 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7a80adff0 0x7fc7a8147750 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7fc7980032f0 tx=0x7fc798009250).stop 2022-07-21T01:55:40.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/2421842622 shutdown_connections 2022-07-21T01:55:40.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7ac9e2700 1 --2- 172.21.15.140:0/2421842622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7a80af8e0 0x7fc7a805bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7ac9e2700 1 --2- 172.21.15.140:0/2421842622 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7a80adff0 0x7fc7a8147750 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7ac9e2700 1 --2- 172.21.15.140:0/2421842622 >> v2:172.21.15.152:3300/0 conn(0x7fc7a80b5650 0x7fc7a80b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/2421842622 >> 172.21.15.140:0/2421842622 conn(0x7fc7a80634e0 msgr2=0x7fc7a80b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:40.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/2421842622 shutdown_connections 2022-07-21T01:55:40.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/2421842622 wait complete. 2022-07-21T01:55:40.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7ac9e2700 1 Processor -- start 2022-07-21T01:55:40.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.663+0000 7fc7ac9e2700 1 -- start start 2022-07-21T01:55:40.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7ac9e2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7a80adff0 0x7fc7a8111120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7ac9e2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7a80af8e0 0x7fc7a8145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7ac9e2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7a80b5650 0x7fc7a8156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7ac9e2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a8157490 con 0x7fc7a80adff0 2022-07-21T01:55:40.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7ac9e2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80645e0 con 0x7fc7a80b5650 2022-07-21T01:55:40.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7ac9e2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a8064840 con 0x7fc7a80af8e0 2022-07-21T01:55:40.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7a80adff0 0x7fc7a8111120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7a80adff0 0x7fc7a8111120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37976/0 (socket says 172.21.15.140:37976) 2022-07-21T01:55:40.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a77fe700 1 -- 172.21.15.140:0/886345565 learned_addr learned my addr 172.21.15.140:0/886345565 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:40.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a6ffd700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7a80af8e0 0x7fc7a8145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a7fff700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7a80b5650 0x7fc7a8156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a77fe700 1 -- 172.21.15.140:0/886345565 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7a80af8e0 msgr2=0x7fc7a8145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a77fe700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7a80af8e0 0x7fc7a8145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a77fe700 1 -- 172.21.15.140:0/886345565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7a80b5650 msgr2=0x7fc7a8156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a77fe700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7a80b5650 0x7fc7a8156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a77fe700 1 -- 172.21.15.140:0/886345565 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980094d0 con 0x7fc7a80adff0 2022-07-21T01:55:40.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a6ffd700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7a80af8e0 0x7fc7a8145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:40.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a77fe700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7a80adff0 0x7fc7a8111120 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7fc790006940 tx=0x7fc790008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:40.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a4ff9700 1 -- 172.21.15.140:0/886345565 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc790011780 con 0x7fc7a80adff0 2022-07-21T01:55:40.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a4ff9700 1 -- 172.21.15.140:0/886345565 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7900118e0 con 0x7fc7a80adff0 2022-07-21T01:55:40.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a4ff9700 1 -- 172.21.15.140:0/886345565 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc790011c40 con 0x7fc7a80adff0 2022-07-21T01:55:40.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/886345565 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a8064ae0 con 0x7fc7a80adff0 2022-07-21T01:55:40.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/886345565 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a814a1b0 con 0x7fc7a80adff0 2022-07-21T01:55:40.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/886345565 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a80574e0 con 0x7fc7a80adff0 2022-07-21T01:55:40.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.667+0000 7fc7a4ff9700 1 -- 172.21.15.140:0/886345565 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc79001c460 con 0x7fc7a80adff0 2022-07-21T01:55:40.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.671+0000 7fc7a4ff9700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc780053a70 0x7fc780055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.671+0000 7fc7a4ff9700 1 -- 172.21.15.140:0/886345565 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc79006c080 con 0x7fc7a80adff0 2022-07-21T01:55:40.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.671+0000 7fc7a6ffd700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc780053a70 0x7fc780055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.671+0000 7fc7a6ffd700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc780053a70 0x7fc780055f30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fc798019000 tx=0x7fc798016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:40.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.675+0000 7fc7a4ff9700 1 -- 172.21.15.140:0/886345565 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc790043270 con 0x7fc7a80adff0 2022-07-21T01:55:40.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.795+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/886345565 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7a81117b0 con 0x7fc7a80adff0 2022-07-21T01:55:40.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.795+0000 7fc7a4ff9700 1 -- 172.21.15.140:0/886345565 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+13278 (secure 0 0 0) 0x7fc790046890 con 0x7fc7a80adff0 2022-07-21T01:55:40.798 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:40.799 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":89,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:40.199496+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:40.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/886345565 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc780053a70 msgr2=0x7fc780055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc780053a70 0x7fc780055f30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fc798019000 tx=0x7fc798016000).stop 2022-07-21T01:55:40.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/886345565 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7a80adff0 msgr2=0x7fc7a8111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7a80adff0 0x7fc7a8111120 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7fc790006940 tx=0x7fc790008040).stop 2022-07-21T01:55:40.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/886345565 shutdown_connections 2022-07-21T01:55:40.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc780053a70 0x7fc780055f30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7a80b5650 0x7fc7a8156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7a80af8e0 0x7fc7a8145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 --2- 172.21.15.140:0/886345565 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc7a80adff0 0x7fc7a8111120 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/886345565 >> 172.21.15.140:0/886345565 conn(0x7fc7a80634e0 msgr2=0x7fc7a80b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:40.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/886345565 shutdown_connections 2022-07-21T01:55:40.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.799+0000 7fc7ac9e2700 1 -- 172.21.15.140:0/886345565 wait complete. 2022-07-21T01:55:40.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:55:40.813 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:55:40.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5acaee700 1 Processor -- start 2022-07-21T01:55:40.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5acaee700 1 -- start start 2022-07-21T01:55:40.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5acaee700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5a80b59f0 0x7ff5a81473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5acaee700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff5a80af940 0x7ff5a8147890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5acaee700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a805cc70 con 0x7ff5a80b1400 2022-07-21T01:55:40.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5acaee700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a80b1070 con 0x7ff5a80af940 2022-07-21T01:55:40.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5acaee700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80b5dd0 con 0x7ff5a80b59f0 2022-07-21T01:55:40.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5a6ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff5a80af940 0x7ff5a8147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5a6ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff5a80af940 0x7ff5a8147890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37984/0 (socket says 172.21.15.140:37984) 2022-07-21T01:55:40.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5a6ffd700 1 -- 172.21.15.140:0/1955875258 learned_addr learned my addr 172.21.15.140:0/1955875258 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:40.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5a77fe700 1 --2- 172.21.15.140:0/1955875258 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5a80b59f0 0x7ff5a81473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5a6ffd700 1 -- 172.21.15.140:0/1955875258 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5a80b59f0 msgr2=0x7ff5a81473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:40.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5a6ffd700 1 --2- 172.21.15.140:0/1955875258 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5a80b59f0 0x7ff5a81473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5a6ffd700 1 -- 172.21.15.140:0/1955875258 >> v1:172.21.15.152:6789/0 conn(0x7ff5a80b1400 legacy=0x7ff5a814a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5a6ffd700 1 -- 172.21.15.140:0/1955875258 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a805c760 con 0x7ff5a80af940 2022-07-21T01:55:40.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.919+0000 7ff5a6ffd700 1 --2- 172.21.15.140:0/1955875258 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff5a80af940 0x7ff5a8147890 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7ff590002f70 tx=0x7ff590006eb0).ready entity=mon.0 client_cookie=fd12f19bd92080d2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:40.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5a67fc700 1 -- 172.21.15.140:0/1955875258 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff59000d5c0 con 0x7ff5a80af940 2022-07-21T01:55:40.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5a67fc700 1 -- 172.21.15.140:0/1955875258 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff590000e90 con 0x7ff5a80af940 2022-07-21T01:55:40.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5a67fc700 1 -- 172.21.15.140:0/1955875258 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff59000d950 con 0x7ff5a80af940 2022-07-21T01:55:40.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5acaee700 1 -- 172.21.15.140:0/1955875258 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff5a80af940 msgr2=0x7ff5a8147890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5acaee700 1 --2- 172.21.15.140:0/1955875258 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff5a80af940 0x7ff5a8147890 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7ff590002f70 tx=0x7ff590006eb0).stop 2022-07-21T01:55:40.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5acaee700 1 -- 172.21.15.140:0/1955875258 shutdown_connections 2022-07-21T01:55:40.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5acaee700 1 --2- 172.21.15.140:0/1955875258 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff5a80af940 0x7ff5a8147890 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5acaee700 1 --2- 172.21.15.140:0/1955875258 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5a80b59f0 0x7ff5a81473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5acaee700 1 -- 172.21.15.140:0/1955875258 >> 172.21.15.140:0/1955875258 conn(0x7ff5a8063480 msgr2=0x7ff5a80b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:40.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5acaee700 1 -- 172.21.15.140:0/1955875258 shutdown_connections 2022-07-21T01:55:40.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5acaee700 1 -- 172.21.15.140:0/1955875258 wait complete. 2022-07-21T01:55:40.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5acaee700 1 Processor -- start 2022-07-21T01:55:40.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.923+0000 7ff5acaee700 1 -- start start 2022-07-21T01:55:40.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5acaee700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff5a80af940 0x7ff5a8067310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5acaee700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a80b1400 0x7ff5a80b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5acaee700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5a80b59f0 0x7ff5a8066670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5acaee700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a810fc50 con 0x7ff5a80af940 2022-07-21T01:55:40.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5acaee700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a81108b0 con 0x7ff5a80b1400 2022-07-21T01:55:40.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5acaee700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a8110b50 con 0x7ff5a80b59f0 2022-07-21T01:55:40.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff5a80af940 0x7ff5a8067310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff5a80af940 0x7ff5a8067310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:37990/0 (socket says 172.21.15.140:37990) 2022-07-21T01:55:40.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a77fe700 1 -- 172.21.15.140:0/4210682447 learned_addr learned my addr 172.21.15.140:0/4210682447 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:40.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a6ffd700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a80b1400 0x7ff5a80b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a7fff700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5a80b59f0 0x7ff5a8066670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a77fe700 1 -- 172.21.15.140:0/4210682447 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5a80b59f0 msgr2=0x7ff5a8066670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a77fe700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5a80b59f0 0x7ff5a8066670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a77fe700 1 -- 172.21.15.140:0/4210682447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a80b1400 msgr2=0x7ff5a80b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:40.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a77fe700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a80b1400 0x7ff5a80b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:40.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a77fe700 1 -- 172.21.15.140:0/4210682447 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59000b1e0 con 0x7ff5a80af940 2022-07-21T01:55:40.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a77fe700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff5a80af940 0x7ff5a8067310 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7ff5980030b0 tx=0x7ff598008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:40.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a4ff9700 1 -- 172.21.15.140:0/4210682447 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff598011780 con 0x7ff5a80af940 2022-07-21T01:55:40.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a4ff9700 1 -- 172.21.15.140:0/4210682447 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5980118e0 con 0x7ff5a80af940 2022-07-21T01:55:40.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5acaee700 1 -- 172.21.15.140:0/4210682447 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a8110ff0 con 0x7ff5a80af940 2022-07-21T01:55:40.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5a4ff9700 1 -- 172.21.15.140:0/4210682447 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff598011c40 con 0x7ff5a80af940 2022-07-21T01:55:40.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5acaee700 1 -- 172.21.15.140:0/4210682447 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a8111510 con 0x7ff5a80af940 2022-07-21T01:55:40.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.927+0000 7ff5acaee700 1 -- 172.21.15.140:0/4210682447 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a8110d50 con 0x7ff5a80af940 2022-07-21T01:55:40.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.931+0000 7ff5a4ff9700 1 -- 172.21.15.140:0/4210682447 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff598012420 con 0x7ff5a80af940 2022-07-21T01:55:40.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.931+0000 7ff5a4ff9700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff580053b00 0x7ff580055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:40.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.931+0000 7ff5a6ffd700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff580053b00 0x7ff580055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:40.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.931+0000 7ff5a4ff9700 1 -- 172.21.15.140:0/4210682447 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff59806b8c0 con 0x7ff5a80af940 2022-07-21T01:55:40.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.931+0000 7ff5a6ffd700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff580053b00 0x7ff580055fc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7ff59001c000 tx=0x7ff590015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:40.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:40.935+0000 7ff5a4ff9700 1 -- 172.21.15.140:0/4210682447 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff5a8110d50 con 0x7ff5a80af940 2022-07-21T01:55:41.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.063+0000 7ff5acaee700 1 -- 172.21.15.140:0/4210682447 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff5a8110d50 con 0x7ff5a80af940 2022-07-21T01:55:41.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.063+0000 7ff5a4ff9700 1 -- 172.21.15.140:0/4210682447 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v89) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff5a8110d50 con 0x7ff5a80af940 2022-07-21T01:55:41.065 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:55:41.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 -- 172.21.15.140:0/4210682447 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff580053b00 msgr2=0x7ff580055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff580053b00 0x7ff580055fc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7ff59001c000 tx=0x7ff590015000).stop 2022-07-21T01:55:41.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 -- 172.21.15.140:0/4210682447 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff5a80af940 msgr2=0x7ff5a8067310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff5a80af940 0x7ff5a8067310 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7ff5980030b0 tx=0x7ff598008040).stop 2022-07-21T01:55:41.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 -- 172.21.15.140:0/4210682447 shutdown_connections 2022-07-21T01:55:41.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff580053b00 0x7ff580055fc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff5a80b59f0 0x7ff5a8066670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5a80b1400 0x7ff5a80b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 --2- 172.21.15.140:0/4210682447 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff5a80af940 0x7ff5a8067310 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 -- 172.21.15.140:0/4210682447 >> 172.21.15.140:0/4210682447 conn(0x7ff5a8063480 msgr2=0x7ff5a80adc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:41.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 -- 172.21.15.140:0/4210682447 shutdown_connections 2022-07-21T01:55:41.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.067+0000 7ff5acaee700 1 -- 172.21.15.140:0/4210682447 wait complete. 2022-07-21T01:55:41.078 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:55:41.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.187+0000 7fc8aeb2a700 1 Processor -- start 2022-07-21T01:55:41.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.187+0000 7fc8aeb2a700 1 -- start start 2022-07-21T01:55:41.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc8a80b2210 0x7fc8a8140860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc8a80967e0 0x7fc8a80b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a813fe80 con 0x7fc8a80ac680 2022-07-21T01:55:41.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a8140120 con 0x7fc8a80967e0 2022-07-21T01:55:41.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a81403c0 con 0x7fc8a80b2210 2022-07-21T01:55:41.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8a7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc8a80967e0 0x7fc8a80b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8a7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc8a80967e0 0x7fc8a80b25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38000/0 (socket says 172.21.15.140:38000) 2022-07-21T01:55:41.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8a7fff700 1 -- 172.21.15.140:0/3405559298 learned_addr learned my addr 172.21.15.140:0/3405559298 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:41.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8ac8c6700 1 --2- 172.21.15.140:0/3405559298 >> v2:172.21.15.152:3300/0 conn(0x7fc8a80b2210 0x7fc8a8140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8a7fff700 1 -- 172.21.15.140:0/3405559298 >> v2:172.21.15.152:3300/0 conn(0x7fc8a80b2210 msgr2=0x7fc8a8140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:41.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8a7fff700 1 --2- 172.21.15.140:0/3405559298 >> v2:172.21.15.152:3300/0 conn(0x7fc8a80b2210 0x7fc8a8140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8a7fff700 1 -- 172.21.15.140:0/3405559298 >> v1:172.21.15.152:6789/0 conn(0x7fc8a80ac680 legacy=0x7fc8a80ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8a7fff700 1 -- 172.21.15.140:0/3405559298 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80ab7c0 con 0x7fc8a80967e0 2022-07-21T01:55:41.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8a7fff700 1 --2- 172.21.15.140:0/3405559298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc8a80967e0 0x7fc8a80b25f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fc89c002f70 tx=0x7fc89c006eb0).ready entity=mon.0 client_cookie=e041c70b7be1e258 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:41.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8a77fe700 1 -- 172.21.15.140:0/3405559298 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc89c00d5c0 con 0x7fc8a80967e0 2022-07-21T01:55:41.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8a77fe700 1 -- 172.21.15.140:0/3405559298 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c000e90 con 0x7fc8a80967e0 2022-07-21T01:55:41.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8a77fe700 1 -- 172.21.15.140:0/3405559298 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc89c00d950 con 0x7fc8a80967e0 2022-07-21T01:55:41.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/3405559298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc8a80967e0 msgr2=0x7fc8a80b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 --2- 172.21.15.140:0/3405559298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc8a80967e0 0x7fc8a80b25f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fc89c002f70 tx=0x7fc89c006eb0).stop 2022-07-21T01:55:41.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/3405559298 shutdown_connections 2022-07-21T01:55:41.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 --2- 172.21.15.140:0/3405559298 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc8a80967e0 0x7fc8a80b25f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 --2- 172.21.15.140:0/3405559298 >> v2:172.21.15.152:3300/0 conn(0x7fc8a80b2210 0x7fc8a8140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/3405559298 >> 172.21.15.140:0/3405559298 conn(0x7fc8a80b36c0 msgr2=0x7fc8a80b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:41.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/3405559298 shutdown_connections 2022-07-21T01:55:41.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.191+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/3405559298 wait complete. 2022-07-21T01:55:41.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8aeb2a700 1 Processor -- start 2022-07-21T01:55:41.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8aeb2a700 1 -- start start 2022-07-21T01:55:41.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8aeb2a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8a80967e0 0x7fc8a81442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8aeb2a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc8a80ac680 0x7fc8a805c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8aeb2a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc8a80b2210 0x7fc8a805cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8aeb2a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80b4690 con 0x7fc8a80b2210 2022-07-21T01:55:41.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8aeb2a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80b53a0 con 0x7fc8a80967e0 2022-07-21T01:55:41.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8aeb2a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80b5640 con 0x7fc8a80ac680 2022-07-21T01:55:41.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8ad0c7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc8a80b2210 0x7fc8a805cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8ad0c7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc8a80b2210 0x7fc8a805cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38008/0 (socket says 172.21.15.140:38008) 2022-07-21T01:55:41.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8ad0c7700 1 -- 172.21.15.140:0/2760593128 learned_addr learned my addr 172.21.15.140:0/2760593128 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:41.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8ac8c6700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8a80967e0 0x7fc8a81442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.195+0000 7fc8a7fff700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc8a80ac680 0x7fc8a805c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8ad0c7700 1 -- 172.21.15.140:0/2760593128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc8a80ac680 msgr2=0x7fc8a805c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8ad0c7700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc8a80ac680 0x7fc8a805c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8ad0c7700 1 -- 172.21.15.140:0/2760593128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8a80967e0 msgr2=0x7fc8a81442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8ad0c7700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8a80967e0 0x7fc8a81442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8ad0c7700 1 -- 172.21.15.140:0/2760593128 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c00b1e0 con 0x7fc8a80b2210 2022-07-21T01:55:41.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8ad0c7700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc8a80b2210 0x7fc8a805cb10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fc8a8062610 tx=0x7fc8a0007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:41.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8a5ffb700 1 -- 172.21.15.140:0/2760593128 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8a000e4a0 con 0x7fc8a80b2210 2022-07-21T01:55:41.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8a5ffb700 1 -- 172.21.15.140:0/2760593128 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a000e600 con 0x7fc8a80b2210 2022-07-21T01:55:41.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/2760593128 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a80b58e0 con 0x7fc8a80b2210 2022-07-21T01:55:41.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8a5ffb700 1 -- 172.21.15.140:0/2760593128 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8a000e960 con 0x7fc8a80b2210 2022-07-21T01:55:41.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/2760593128 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a80b4ea0 con 0x7fc8a80b2210 2022-07-21T01:55:41.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/2760593128 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a80b42c0 con 0x7fc8a80b2210 2022-07-21T01:55:41.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.199+0000 7fc8a5ffb700 1 -- 172.21.15.140:0/2760593128 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc8a0019030 con 0x7fc8a80b2210 2022-07-21T01:55:41.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.203+0000 7fc8a5ffb700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc888053b00 0x7fc888055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.203+0000 7fc8a5ffb700 1 -- 172.21.15.140:0/2760593128 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc8a006b380 con 0x7fc8a80b2210 2022-07-21T01:55:41.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.203+0000 7fc8ac8c6700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc888053b00 0x7fc888055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.203+0000 7fc8ac8c6700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc888053b00 0x7fc888055fc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fc89800b000 tx=0x7fc898006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:41.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.207+0000 7fc8a5ffb700 1 -- 172.21.15.140:0/2760593128 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc8a80b42c0 con 0x7fc8a80b2210 2022-07-21T01:55:41.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.331+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/2760593128 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc8a80b42c0 con 0x7fc8a80b2210 2022-07-21T01:55:41.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8a5ffb700 1 -- 172.21.15.140:0/2760593128 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v89) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc8a80b42c0 con 0x7fc8a80b2210 2022-07-21T01:55:41.335 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:41.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/2760593128 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc888053b00 msgr2=0x7fc888055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc888053b00 0x7fc888055fc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7fc89800b000 tx=0x7fc898006df0).stop 2022-07-21T01:55:41.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/2760593128 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc8a80b2210 msgr2=0x7fc8a805cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc8a80b2210 0x7fc8a805cb10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fc8a8062610 tx=0x7fc8a0007f40).stop 2022-07-21T01:55:41.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/2760593128 shutdown_connections 2022-07-21T01:55:41.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc888053b00 0x7fc888055fc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc8a80b2210 0x7fc8a805cb10 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc8a80ac680 0x7fc8a805c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 --2- 172.21.15.140:0/2760593128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8a80967e0 0x7fc8a81442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/2760593128 >> 172.21.15.140:0/2760593128 conn(0x7fc8a80b36c0 msgr2=0x7fc8a813f250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:41.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/2760593128 shutdown_connections 2022-07-21T01:55:41.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.335+0000 7fc8aeb2a700 1 -- 172.21.15.140:0/2760593128 wait complete. 2022-07-21T01:55:41.348 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:55:41.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.447+0000 7f9e5cd3a700 1 Processor -- start 2022-07-21T01:55:41.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 -- start start 2022-07-21T01:55:41.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9e580b34f0 0x7f9e58140050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e580acaf0 con 0x7f9e58061d80 2022-07-21T01:55:41.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e580b2190 con 0x7f9e580b3110 2022-07-21T01:55:41.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e580621d0 con 0x7f9e580b34f0 2022-07-21T01:55:41.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e55d9b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f9e58061d80 0x7f9e5813f940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52158/0 (socket says 172.21.15.140:52158) 2022-07-21T01:55:41.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e55d9b700 1 -- 172.21.15.140:0/117845275 learned_addr learned my addr 172.21.15.140:0/117845275 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:41.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e56d9d700 1 --2- 172.21.15.140:0/117845275 >> v2:172.21.15.152:3300/0 conn(0x7f9e580b34f0 0x7f9e58140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5559a700 1 -- 172.21.15.140:0/117845275 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330548942 0 0) 0x7f9e580acaf0 con 0x7f9e58061d80 2022-07-21T01:55:41.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5559a700 1 -- 172.21.15.140:0/117845275 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e3c003260 con 0x7f9e58061d80 2022-07-21T01:55:41.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e56d9d700 1 -- 172.21.15.140:0/117845275 >> v1:172.21.15.152:6789/0 conn(0x7f9e580b3110 legacy=0x7f9e581414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e56d9d700 1 -- 172.21.15.140:0/117845275 >> v1:172.21.15.140:6789/0 conn(0x7f9e58061d80 legacy=0x7f9e5813f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e56d9d700 1 -- 172.21.15.140:0/117845275 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580b3960 con 0x7f9e580b34f0 2022-07-21T01:55:41.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e56d9d700 1 --2- 172.21.15.140:0/117845275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e580b34f0 0x7f9e58140050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f9e580ad7d0 tx=0x7f9e50006df0).ready entity=mon.1 client_cookie=cd9f1f835d22c9eb server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:41.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5559a700 1 -- 172.21.15.140:0/117845275 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e5000d610 con 0x7f9e580b34f0 2022-07-21T01:55:41.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5559a700 1 -- 172.21.15.140:0/117845275 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50000e90 con 0x7f9e580b34f0 2022-07-21T01:55:41.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5559a700 1 -- 172.21.15.140:0/117845275 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e5000d960 con 0x7f9e580b34f0 2022-07-21T01:55:41.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/117845275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e580b34f0 msgr2=0x7f9e58140050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 --2- 172.21.15.140:0/117845275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e580b34f0 0x7f9e58140050 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f9e580ad7d0 tx=0x7f9e50006df0).stop 2022-07-21T01:55:41.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/117845275 shutdown_connections 2022-07-21T01:55:41.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 --2- 172.21.15.140:0/117845275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e580b34f0 0x7f9e58140050 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/117845275 >> 172.21.15.140:0/117845275 conn(0x7f9e580b47e0 msgr2=0x7f9e5805bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:41.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/117845275 shutdown_connections 2022-07-21T01:55:41.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.451+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/117845275 wait complete. 2022-07-21T01:55:41.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e5cd3a700 1 Processor -- start 2022-07-21T01:55:41.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e5cd3a700 1 -- start start 2022-07-21T01:55:41.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e5cd3a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e58061d80 0x7f9e58143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e5cd3a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e580b3110 0x7f9e580650b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e5cd3a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e580b34f0 0x7f9e58097710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e5cd3a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58097010 con 0x7f9e58061d80 2022-07-21T01:55:41.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e5cd3a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58097270 con 0x7f9e580b3110 2022-07-21T01:55:41.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e5cd3a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e581503e0 con 0x7f9e580b34f0 2022-07-21T01:55:41.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e5659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e58061d80 0x7f9e58143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e5659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e58061d80 0x7f9e58143230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38020/0 (socket says 172.21.15.140:38020) 2022-07-21T01:55:41.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e55d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e580b3110 0x7f9e580650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e5659c700 1 -- 172.21.15.140:0/349327098 learned_addr learned my addr 172.21.15.140:0/349327098 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:41.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.455+0000 7f9e56d9d700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e580b34f0 0x7f9e58097710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e5659c700 1 -- 172.21.15.140:0/349327098 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e580b34f0 msgr2=0x7f9e58097710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e5659c700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e580b34f0 0x7f9e58097710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e5659c700 1 -- 172.21.15.140:0/349327098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e580b3110 msgr2=0x7f9e580650b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e5659c700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e580b3110 0x7f9e580650b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e5659c700 1 -- 172.21.15.140:0/349327098 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5000b180 con 0x7f9e58061d80 2022-07-21T01:55:41.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e5659c700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e58061d80 0x7f9e58143230 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f9e58068040 tx=0x7f9e48007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:41.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e4f7fe700 1 -- 172.21.15.140:0/349327098 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e48007030 con 0x7f9e58061d80 2022-07-21T01:55:41.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e4f7fe700 1 -- 172.21.15.140:0/349327098 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e48007190 con 0x7f9e58061d80 2022-07-21T01:55:41.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/349327098 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e580655a0 con 0x7f9e58061d80 2022-07-21T01:55:41.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e4f7fe700 1 -- 172.21.15.140:0/349327098 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e48006510 con 0x7f9e58061d80 2022-07-21T01:55:41.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/349327098 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e580989f0 con 0x7f9e58061d80 2022-07-21T01:55:41.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/349327098 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e580b2630 con 0x7f9e58061d80 2022-07-21T01:55:41.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e4f7fe700 1 -- 172.21.15.140:0/349327098 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9e48010030 con 0x7f9e58061d80 2022-07-21T01:55:41.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e4f7fe700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9e30053b00 0x7f9e30055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e4f7fe700 1 -- 172.21.15.140:0/349327098 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9e4806b6f0 con 0x7f9e58061d80 2022-07-21T01:55:41.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e55d9b700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9e30053b00 0x7f9e30055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.459+0000 7f9e55d9b700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9e30053b00 0x7f9e30055fc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f9e4000d000 tx=0x7f9e40009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:41.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.467+0000 7f9e4f7fe700 1 -- 172.21.15.140:0/349327098 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9e48042940 con 0x7f9e58061d80 2022-07-21T01:55:41.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.587+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/349327098 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9e580574e0 con 0x7f9e58061d80 2022-07-21T01:55:41.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.587+0000 7f9e4f7fe700 1 -- 172.21.15.140:0/349327098 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) v1 ==== 84+0+143 (secure 0 0 0) 0x7f9e48045f60 con 0x7f9e58061d80 2022-07-21T01:55:41.591 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:41.591 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:55:41.592 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:41.592 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:55:41.592 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:55:41.592 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:41.593 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:41.593 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:55:41.593 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:55:41.593 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:41.594 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:55:41.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/349327098 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9e30053b00 msgr2=0x7f9e30055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9e30053b00 0x7f9e30055fc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f9e4000d000 tx=0x7f9e40009040).stop 2022-07-21T01:55:41.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/349327098 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e58061d80 msgr2=0x7f9e58143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e58061d80 0x7f9e58143230 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f9e58068040 tx=0x7f9e48007c10).stop 2022-07-21T01:55:41.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/349327098 shutdown_connections 2022-07-21T01:55:41.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9e30053b00 0x7f9e30055fc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e580b34f0 0x7f9e58097710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e580b3110 0x7f9e580650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 --2- 172.21.15.140:0/349327098 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9e58061d80 0x7f9e58143230 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/349327098 >> 172.21.15.140:0/349327098 conn(0x7f9e580b47e0 msgr2=0x7f9e58099460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:41.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/349327098 shutdown_connections 2022-07-21T01:55:41.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.591+0000 7f9e5cd3a700 1 -- 172.21.15.140:0/349327098 wait complete. 2022-07-21T01:55:41.607 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:55:41.608 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:55:41.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e191a1700 1 Processor -- start 2022-07-21T01:55:41.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e191a1700 1 -- start start 2022-07-21T01:55:41.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e191a1700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1e140b9da0 0x7f1e141468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e191a1700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1e141419a0 0x7f1e14141e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e191a1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e140ba180 con 0x7f1e140b3cf0 2022-07-21T01:55:41.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e191a1700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e140ad5b0 con 0x7f1e141419a0 2022-07-21T01:55:41.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e191a1700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e141461b0 con 0x7f1e140b9da0 2022-07-21T01:55:41.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e189a0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1e141419a0 0x7f1e14141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e137fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1e140b3cf0 0x7f1e14142a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52174/0 (socket says 172.21.15.140:52174) 2022-07-21T01:55:41.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e137fe700 1 -- 172.21.15.140:0/1110759042 learned_addr learned my addr 172.21.15.140:0/1110759042 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:41.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e13fff700 1 --2- 172.21.15.140:0/1110759042 >> v2:172.21.15.152:3300/0 conn(0x7f1e140b9da0 0x7f1e141468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e189a0700 1 -- 172.21.15.140:0/1110759042 >> v2:172.21.15.152:3300/0 conn(0x7f1e140b9da0 msgr2=0x7f1e141468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:41.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e189a0700 1 --2- 172.21.15.140:0/1110759042 >> v2:172.21.15.152:3300/0 conn(0x7f1e140b9da0 0x7f1e141468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e189a0700 1 -- 172.21.15.140:0/1110759042 >> v1:172.21.15.140:6789/0 conn(0x7f1e140b3cf0 legacy=0x7f1e14142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e12ffd700 1 -- 172.21.15.140:0/1110759042 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1867638298 0 0) 0x7f1e140ba180 con 0x7f1e140b3cf0 2022-07-21T01:55:41.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e189a0700 1 -- 172.21.15.140:0/1110759042 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e14146450 con 0x7f1e141419a0 2022-07-21T01:55:41.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.715+0000 7f1e13fff700 1 --2- 172.21.15.140:0/1110759042 >> v2:172.21.15.152:3300/0 conn(0x7f1e140b9da0 0x7f1e141468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:41.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e189a0700 1 --2- 172.21.15.140:0/1110759042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1e141419a0 0x7f1e14141e50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f1e0c0032f0 tx=0x7f1e0c0091d0).ready entity=mon.0 client_cookie=b3f0dcf0c4b1f66e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:41.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e12ffd700 1 -- 172.21.15.140:0/1110759042 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e0c009ea0 con 0x7f1e141419a0 2022-07-21T01:55:41.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e12ffd700 1 -- 172.21.15.140:0/1110759042 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c00b870 con 0x7f1e141419a0 2022-07-21T01:55:41.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e12ffd700 1 -- 172.21.15.140:0/1110759042 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e0c00b9e0 con 0x7f1e141419a0 2022-07-21T01:55:41.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 -- 172.21.15.140:0/1110759042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1e141419a0 msgr2=0x7f1e14141e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 --2- 172.21.15.140:0/1110759042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1e141419a0 0x7f1e14141e50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f1e0c0032f0 tx=0x7f1e0c0091d0).stop 2022-07-21T01:55:41.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 -- 172.21.15.140:0/1110759042 shutdown_connections 2022-07-21T01:55:41.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 --2- 172.21.15.140:0/1110759042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1e141419a0 0x7f1e14141e50 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 --2- 172.21.15.140:0/1110759042 >> v2:172.21.15.152:3300/0 conn(0x7f1e140b9da0 0x7f1e141468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 -- 172.21.15.140:0/1110759042 >> 172.21.15.140:0/1110759042 conn(0x7f1e140678c0 msgr2=0x7f1e140b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:41.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 -- 172.21.15.140:0/1110759042 shutdown_connections 2022-07-21T01:55:41.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 -- 172.21.15.140:0/1110759042 wait complete. 2022-07-21T01:55:41.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 Processor -- start 2022-07-21T01:55:41.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 -- start start 2022-07-21T01:55:41.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1e140b3cf0 0x7f1e1413ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e140b9da0 0x7f1e1413d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1e141419a0 0x7f1e1413dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14155ea0 con 0x7f1e140b3cf0 2022-07-21T01:55:41.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e141444f0 con 0x7f1e140b9da0 2022-07-21T01:55:41.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.719+0000 7f1e191a1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14144de0 con 0x7f1e141419a0 2022-07-21T01:55:41.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e13fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1e140b3cf0 0x7f1e1413ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e13fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1e140b3cf0 0x7f1e1413ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38032/0 (socket says 172.21.15.140:38032) 2022-07-21T01:55:41.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e13fff700 1 -- 172.21.15.140:0/3220791472 learned_addr learned my addr 172.21.15.140:0/3220791472 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:41.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e189a0700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1e141419a0 0x7f1e1413dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e137fe700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e140b9da0 0x7f1e1413d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e13fff700 1 -- 172.21.15.140:0/3220791472 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1e141419a0 msgr2=0x7f1e1413dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e13fff700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1e141419a0 0x7f1e1413dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e13fff700 1 -- 172.21.15.140:0/3220791472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e140b9da0 msgr2=0x7f1e1413d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:41.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e13fff700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e140b9da0 0x7f1e1413d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:41.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e13fff700 1 -- 172.21.15.140:0/3220791472 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c009470 con 0x7f1e140b3cf0 2022-07-21T01:55:41.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e137fe700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e140b9da0 0x7f1e1413d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:55:41.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e13fff700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1e140b3cf0 0x7f1e1413ee00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f1e08006940 tx=0x7f1e08008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:41.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e117fa700 1 -- 172.21.15.140:0/3220791472 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e08011780 con 0x7f1e140b3cf0 2022-07-21T01:55:41.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e117fa700 1 -- 172.21.15.140:0/3220791472 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e080118e0 con 0x7f1e140b3cf0 2022-07-21T01:55:41.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e191a1700 1 -- 172.21.15.140:0/3220791472 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e14145080 con 0x7f1e140b3cf0 2022-07-21T01:55:41.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e117fa700 1 -- 172.21.15.140:0/3220791472 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e08011c40 con 0x7f1e140b3cf0 2022-07-21T01:55:41.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e191a1700 1 -- 172.21.15.140:0/3220791472 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e14145520 con 0x7f1e140b3cf0 2022-07-21T01:55:41.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e191a1700 1 -- 172.21.15.140:0/3220791472 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e140574e0 con 0x7f1e140b3cf0 2022-07-21T01:55:41.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e117fa700 1 -- 172.21.15.140:0/3220791472 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1e08012420 con 0x7f1e140b3cf0 2022-07-21T01:55:41.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e117fa700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1df4053aa0 0x7f1df4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:41.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.723+0000 7f1e137fe700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1df4053aa0 0x7f1df4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:41.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.727+0000 7f1e117fa700 1 -- 172.21.15.140:0/3220791472 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f1e0806bbc0 con 0x7f1e140b3cf0 2022-07-21T01:55:41.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.727+0000 7f1e137fe700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1df4053aa0 0x7f1df4055f60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f1e04008000 tx=0x7f1e04009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:41.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.735+0000 7f1e117fa700 1 -- 172.21.15.140:0/3220791472 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1e08042ca0 con 0x7f1e140b3cf0 2022-07-21T01:55:41.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:41.859+0000 7f1e191a1700 1 -- 172.21.15.140:0/3220791472 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f1e140699b0 con 0x7f1e140b3cf0 2022-07-21T01:55:42.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.019+0000 7f1e117fa700 1 -- 172.21.15.140:0/3220791472 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v90) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1e080462c0 con 0x7f1e140b3cf0 2022-07-21T01:55:42.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 -- 172.21.15.140:0/3220791472 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1df4053aa0 msgr2=0x7f1df4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:42.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1df4053aa0 0x7f1df4055f60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f1e04008000 tx=0x7f1e04009040).stop 2022-07-21T01:55:42.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 -- 172.21.15.140:0/3220791472 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1e140b3cf0 msgr2=0x7f1e1413ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:42.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1e140b3cf0 0x7f1e1413ee00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f1e08006940 tx=0x7f1e08008040).stop 2022-07-21T01:55:42.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 -- 172.21.15.140:0/3220791472 shutdown_connections 2022-07-21T01:55:42.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1df4053aa0 0x7f1df4055f60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:42.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1e141419a0 0x7f1e1413dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:42.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e140b9da0 0x7f1e1413d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:42.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 --2- 172.21.15.140:0/3220791472 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1e140b3cf0 0x7f1e1413ee00 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:42.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 -- 172.21.15.140:0/3220791472 >> 172.21.15.140:0/3220791472 conn(0x7f1e140678c0 msgr2=0x7f1e1405bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:42.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 -- 172.21.15.140:0/3220791472 shutdown_connections 2022-07-21T01:55:42.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.023+0000 7f1e191a1700 1 -- 172.21.15.140:0/3220791472 wait complete. 2022-07-21T01:55:42.027 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:55:42.038 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:55:42.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbee157a700 1 Processor -- start 2022-07-21T01:55:42.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbee157a700 1 -- start start 2022-07-21T01:55:42.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbee157a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbedc0b6880 0x7fbedc143590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:42.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbee157a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc145f90 0x7fbedc0b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:42.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbee157a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc0b6c60 con 0x7fbedc0b4220 2022-07-21T01:55:42.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbee157a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbedc1429c0 con 0x7fbedc0b6880 2022-07-21T01:55:42.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbee157a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc142bb0 con 0x7fbedc145f90 2022-07-21T01:55:42.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbee0d79700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fbedc0b4220 0x7fbedc13fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52188/0 (socket says 172.21.15.140:52188) 2022-07-21T01:55:42.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbee0d79700 1 -- 172.21.15.140:0/3897210384 learned_addr learned my addr 172.21.15.140:0/3897210384 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:42.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedbfff700 1 --2- 172.21.15.140:0/3897210384 >> v2:172.21.15.152:3300/0 conn(0x7fbedc0b6880 0x7fbedc143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:42.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedb7fe700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc145f90 0x7fbedc0b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:42.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627997403 0 0) 0x7fbedc0b6c60 con 0x7fbedc0b4220 2022-07-21T01:55:42.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbebc003260 con 0x7fbedc0b4220 2022-07-21T01:55:42.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2990339198 0 0) 0x7fbebc003260 con 0x7fbedc0b4220 2022-07-21T01:55:42.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc145f90 msgr2=0x7fbedc0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:42.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc145f90 0x7fbedc0b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:42.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 >> v2:172.21.15.152:3300/0 conn(0x7fbedc0b6880 msgr2=0x7fbedc143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:42.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 --2- 172.21.15.140:0/3897210384 >> v2:172.21.15.152:3300/0 conn(0x7fbedc0b6880 0x7fbedc143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:42.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc142e50 con 0x7fbedc0b4220 2022-07-21T01:55:42.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fbed4001660 con 0x7fbedc0b4220 2022-07-21T01:55:42.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 >> v1:172.21.15.140:6789/0 conn(0x7fbedc0b4220 legacy=0x7fbedc13fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:42.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbebc003750 0x7fbebc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:42.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.143+0000 7fbedaffd700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc145f90 0x7fbebc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:42.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedaffd700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc0049f0 0x7fbebc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:42.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc142bb0 con 0x7fbebc003750 2022-07-21T01:55:42.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc004330 con 0x7fbebc0049f0 2022-07-21T01:55:42.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc00b7a0 con 0x7fbedc145f90 2022-07-21T01:55:42.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedbfff700 1 --2- 172.21.15.140:0/3897210384 >> v2:172.21.15.152:3300/0 conn(0x7fbedc0b6880 0x7fbedc143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:42.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedbfff700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbebc003750 0x7fbebc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:42.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedb7fe700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc145f90 0x7fbebc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:42.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbee0d79700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc0049f0 0x7fbebc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:42.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedbfff700 1 -- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc145f90 msgr2=0x7fbebc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:42.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedbfff700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc145f90 0x7fbebc004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:42.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedbfff700 1 -- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc0049f0 msgr2=0x7fbebc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:42.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedbfff700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc0049f0 0x7fbebc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:42.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedbfff700 1 -- 172.21.15.140:0/3897210384 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc003260 con 0x7fbebc003750 2022-07-21T01:55:42.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedb7fe700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc145f90 0x7fbebc004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:42.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbee0d79700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc0049f0 0x7fbebc004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:42.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedbfff700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbebc003750 0x7fbebc003b70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7fbebc004470 tx=0x7fbecc0091d0).ready entity=mon.0 client_cookie=4b85830fa6aa250b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:42.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbecc00e920 con 0x7fbebc003750 2022-07-21T01:55:42.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc00ea80 con 0x7fbebc003750 2022-07-21T01:55:42.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbedaffd700 1 -- 172.21.15.140:0/3897210384 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbecc00ede0 con 0x7fbebc003750 2022-07-21T01:55:42.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbee157a700 1 -- 172.21.15.140:0/3897210384 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbebc003750 msgr2=0x7fbebc003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:42.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbee157a700 1 --2- 172.21.15.140:0/3897210384 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbebc003750 0x7fbebc003b70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7fbebc004470 tx=0x7fbecc0091d0).stop 2022-07-21T01:55:42.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbee0d79700 1 -- 172.21.15.140:0/3897210384 reap_dead start 2022-07-21T01:55:42.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbee157a700 1 -- 172.21.15.140:0/3897210384 shutdown_connections 2022-07-21T01:55:42.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbee157a700 1 -- 172.21.15.140:0/3897210384 >> 172.21.15.140:0/3897210384 conn(0x7fbedc0b9e60 msgr2=0x7fbedc0b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:42.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbee157a700 1 -- 172.21.15.140:0/3897210384 shutdown_connections 2022-07-21T01:55:42.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbee157a700 1 -- 172.21.15.140:0/3897210384 wait complete. 2022-07-21T01:55:42.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbee157a700 1 Processor -- start 2022-07-21T01:55:42.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.147+0000 7fbee157a700 1 -- start start 2022-07-21T01:55:42.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbee157a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc003750 0x7fbedc147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:42.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbee157a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbedc0b1e60 0x7fbedc0acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:42.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbee157a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc0acfd0 0x7fbedc155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:42.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbee157a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0b3540 con 0x7fbedc0b1e60 2022-07-21T01:55:42.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbee157a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc140ab0 con 0x7fbebc003750 2022-07-21T01:55:42.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbee157a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc140d50 con 0x7fbedc0acfd0 2022-07-21T01:55:42.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbedb7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbedc0b1e60 0x7fbedc0acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:42.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbedb7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbedc0b1e60 0x7fbedc0acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38054/0 (socket says 172.21.15.140:38054) 2022-07-21T01:55:42.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbedb7fe700 1 -- 172.21.15.140:0/3934543743 learned_addr learned my addr 172.21.15.140:0/3934543743 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:42.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbedbfff700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc003750 0x7fbedc147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:42.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbee0d79700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc0acfd0 0x7fbedc155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:42.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbedb7fe700 1 -- 172.21.15.140:0/3934543743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc0acfd0 msgr2=0x7fbedc155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:42.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbedb7fe700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc0acfd0 0x7fbedc155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:42.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbedb7fe700 1 -- 172.21.15.140:0/3934543743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc003750 msgr2=0x7fbedc147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:42.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbedb7fe700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc003750 0x7fbedc147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:42.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbedb7fe700 1 -- 172.21.15.140:0/3934543743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc0b22a0 con 0x7fbedc0b1e60 2022-07-21T01:55:42.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbedbfff700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc003750 0x7fbedc147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:42.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbedb7fe700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbedc0b1e60 0x7fbedc0acae0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7fbec400b000 tx=0x7fbec4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:42.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbed97fa700 1 -- 172.21.15.140:0/3934543743 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbec4010e60 con 0x7fbedc0b1e60 2022-07-21T01:55:42.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbed97fa700 1 -- 172.21.15.140:0/3934543743 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec400e8d0 con 0x7fbedc0b1e60 2022-07-21T01:55:42.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbee157a700 1 -- 172.21.15.140:0/3934543743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc13f320 con 0x7fbedc0b1e60 2022-07-21T01:55:42.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbed97fa700 1 -- 172.21.15.140:0/3934543743 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbec400ec30 con 0x7fbedc0b1e60 2022-07-21T01:55:42.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbee157a700 1 -- 172.21.15.140:0/3934543743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc141390 con 0x7fbedc0b1e60 2022-07-21T01:55:42.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbee157a700 1 -- 172.21.15.140:0/3934543743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc0b5c20 con 0x7fbedc0b1e60 2022-07-21T01:55:42.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.151+0000 7fbed97fa700 1 -- 172.21.15.140:0/3934543743 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbec4024070 con 0x7fbedc0b1e60 2022-07-21T01:55:42.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.155+0000 7fbed97fa700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbec0053aa0 0x7fbec0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:42.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.155+0000 7fbed97fa700 1 -- 172.21.15.140:0/3934543743 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fbec406c6f0 con 0x7fbedc0b1e60 2022-07-21T01:55:42.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.155+0000 7fbedbfff700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbec0053aa0 0x7fbec0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:42.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.155+0000 7fbedbfff700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbec0053aa0 0x7fbec0055f60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fbecc01c000 tx=0x7fbecc01a000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:42.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.159+0000 7fbed97fa700 1 -- 172.21.15.140:0/3934543743 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbedc0b5c20 con 0x7fbedc0b1e60 2022-07-21T01:55:42.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:42.287+0000 7fbee157a700 1 -- 172.21.15.140:0/3934543743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fbedc0b5c20 con 0x7fbedc0b1e60 2022-07-21T01:55:43.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.031+0000 7fbed97fa700 1 -- 172.21.15.140:0/3934543743 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v91) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbedc0b5c20 con 0x7fbedc0b1e60 2022-07-21T01:55:43.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 -- 172.21.15.140:0/3934543743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbec0053aa0 msgr2=0x7fbec0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbec0053aa0 0x7fbec0055f60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fbecc01c000 tx=0x7fbecc01a000).stop 2022-07-21T01:55:43.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 -- 172.21.15.140:0/3934543743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbedc0b1e60 msgr2=0x7fbedc0acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbedc0b1e60 0x7fbedc0acae0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7fbec400b000 tx=0x7fbec4008040).stop 2022-07-21T01:55:43.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 -- 172.21.15.140:0/3934543743 shutdown_connections 2022-07-21T01:55:43.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbec0053aa0 0x7fbec0055f60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbedc0acfd0 0x7fbedc155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbedc0b1e60 0x7fbedc0acae0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 --2- 172.21.15.140:0/3934543743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbebc003750 0x7fbedc147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 -- 172.21.15.140:0/3934543743 >> 172.21.15.140:0/3934543743 conn(0x7fbedc0b9e60 msgr2=0x7fbedc139050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:43.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 -- 172.21.15.140:0/3934543743 shutdown_connections 2022-07-21T01:55:43.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.035+0000 7fbee157a700 1 -- 172.21.15.140:0/3934543743 wait complete. 2022-07-21T01:55:43.042 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:55:43.050 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:55:43.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.151+0000 7f9756e2d700 1 Processor -- start 2022-07-21T01:55:43.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.151+0000 7f9756e2d700 1 -- start start 2022-07-21T01:55:43.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97500b57f0 0x7f97500af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f975005c3b0 0x7f975005c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9750065040 con 0x7f97500aecf0 2022-07-21T01:55:43.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975005bf10 con 0x7f975005c3b0 2022-07-21T01:55:43.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9750064d70 con 0x7f97500b57f0 2022-07-21T01:55:43.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f975562a700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f97500aecf0 0x7f9750143d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52210/0 (socket says 172.21.15.140:52210) 2022-07-21T01:55:43.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f975562a700 1 -- 172.21.15.140:0/2377439730 learned_addr learned my addr 172.21.15.140:0/2377439730 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:43.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9755e2b700 1 --2- 172.21.15.140:0/2377439730 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97500b57f0 0x7f97500af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f975662c700 1 --2- 172.21.15.140:0/2377439730 >> v2:172.21.15.152:3300/0 conn(0x7f975005c3b0 0x7f975005c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9754e29700 1 -- 172.21.15.140:0/2377439730 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761252340 0 0) 0x7f9750065040 con 0x7f97500aecf0 2022-07-21T01:55:43.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9754e29700 1 -- 172.21.15.140:0/2377439730 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9738003260 con 0x7f97500aecf0 2022-07-21T01:55:43.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9755e2b700 1 -- 172.21.15.140:0/2377439730 >> v2:172.21.15.152:3300/0 conn(0x7f975005c3b0 msgr2=0x7f975005c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:43.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9755e2b700 1 --2- 172.21.15.140:0/2377439730 >> v2:172.21.15.152:3300/0 conn(0x7f975005c3b0 0x7f975005c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9755e2b700 1 -- 172.21.15.140:0/2377439730 >> v1:172.21.15.140:6789/0 conn(0x7f97500aecf0 legacy=0x7f9750143d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9755e2b700 1 -- 172.21.15.140:0/2377439730 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97500b5bd0 con 0x7f97500b57f0 2022-07-21T01:55:43.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f975662c700 1 --2- 172.21.15.140:0/2377439730 >> v2:172.21.15.152:3300/0 conn(0x7f975005c3b0 0x7f975005c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:43.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9755e2b700 1 --2- 172.21.15.140:0/2377439730 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97500b57f0 0x7f97500af270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f974c002f70 tx=0x7f974c006d90).ready entity=mon.2 client_cookie=60f42e307a399591 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:43.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9754e29700 1 -- 172.21.15.140:0/2377439730 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f974c00e590 con 0x7f97500b57f0 2022-07-21T01:55:43.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9754e29700 1 -- 172.21.15.140:0/2377439730 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c000e90 con 0x7f97500b57f0 2022-07-21T01:55:43.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9754e29700 1 -- 172.21.15.140:0/2377439730 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f974c00e8e0 con 0x7f97500b57f0 2022-07-21T01:55:43.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 -- 172.21.15.140:0/2377439730 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97500b57f0 msgr2=0x7f97500af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 --2- 172.21.15.140:0/2377439730 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97500b57f0 0x7f97500af270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f974c002f70 tx=0x7f974c006d90).stop 2022-07-21T01:55:43.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 -- 172.21.15.140:0/2377439730 shutdown_connections 2022-07-21T01:55:43.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 --2- 172.21.15.140:0/2377439730 >> v2:172.21.15.152:3300/0 conn(0x7f975005c3b0 0x7f975005c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 --2- 172.21.15.140:0/2377439730 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97500b57f0 0x7f97500af270 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 -- 172.21.15.140:0/2377439730 >> 172.21.15.140:0/2377439730 conn(0x7f9750063480 msgr2=0x7f97500b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:43.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 -- 172.21.15.140:0/2377439730 shutdown_connections 2022-07-21T01:55:43.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.155+0000 7f9756e2d700 1 -- 172.21.15.140:0/2377439730 wait complete. 2022-07-21T01:55:43.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9756e2d700 1 Processor -- start 2022-07-21T01:55:43.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9756e2d700 1 -- start start 2022-07-21T01:55:43.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9756e2d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f975005c3b0 0x7f97501407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9756e2d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97500aecf0 0x7f9750150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9756e2d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97500b57f0 0x7f975013dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9756e2d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975013e110 con 0x7f975005c3b0 2022-07-21T01:55:43.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9756e2d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9750151450 con 0x7f97500aecf0 2022-07-21T01:55:43.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9756e2d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975013d4e0 con 0x7f97500b57f0 2022-07-21T01:55:43.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9755e2b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f975005c3b0 0x7f97501407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9755e2b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f975005c3b0 0x7f97501407d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38066/0 (socket says 172.21.15.140:38066) 2022-07-21T01:55:43.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9755e2b700 1 -- 172.21.15.140:0/430517549 learned_addr learned my addr 172.21.15.140:0/430517549 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:43.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f975562a700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97500aecf0 0x7f9750150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f975662c700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97500b57f0 0x7f975013dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9755e2b700 1 -- 172.21.15.140:0/430517549 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97500b57f0 msgr2=0x7f975013dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9755e2b700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97500b57f0 0x7f975013dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9755e2b700 1 -- 172.21.15.140:0/430517549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97500aecf0 msgr2=0x7f9750150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9755e2b700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97500aecf0 0x7f9750150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9755e2b700 1 -- 172.21.15.140:0/430517549 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9748009470 con 0x7f975005c3b0 2022-07-21T01:55:43.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9755e2b700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f975005c3b0 0x7f97501407d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f974c002f70 tx=0x7f974c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:43.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9742ffd700 1 -- 172.21.15.140:0/430517549 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f974c011950 con 0x7f975005c3b0 2022-07-21T01:55:43.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9742ffd700 1 -- 172.21.15.140:0/430517549 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c011ab0 con 0x7f975005c3b0 2022-07-21T01:55:43.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9756e2d700 1 -- 172.21.15.140:0/430517549 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c00b120 con 0x7f975005c3b0 2022-07-21T01:55:43.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9742ffd700 1 -- 172.21.15.140:0/430517549 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f974c011e10 con 0x7f975005c3b0 2022-07-21T01:55:43.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.159+0000 7f9756e2d700 1 -- 172.21.15.140:0/430517549 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975013ccc0 con 0x7f975005c3b0 2022-07-21T01:55:43.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.163+0000 7f9756e2d700 1 -- 172.21.15.140:0/430517549 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97501512a0 con 0x7f975005c3b0 2022-07-21T01:55:43.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.163+0000 7f9742ffd700 1 -- 172.21.15.140:0/430517549 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f974c024030 con 0x7f975005c3b0 2022-07-21T01:55:43.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.163+0000 7f9742ffd700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9730053a70 0x7f9730055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.163+0000 7f9742ffd700 1 -- 172.21.15.140:0/430517549 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f974c06cdb0 con 0x7f975005c3b0 2022-07-21T01:55:43.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.163+0000 7f975562a700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9730053a70 0x7f9730055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.163+0000 7f975562a700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9730053a70 0x7f9730055f30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f974400b000 tx=0x7f9744009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:43.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.171+0000 7f9742ffd700 1 -- 172.21.15.140:0/430517549 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f974c043d40 con 0x7f975005c3b0 2022-07-21T01:55:43.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.339+0000 7f9756e2d700 1 -- 172.21.15.140:0/430517549 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f975013a3f0 con 0x7f975005c3b0 2022-07-21T01:55:43.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.347+0000 7f9742ffd700 1 -- 172.21.15.140:0/430517549 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 17 and data pool 18 v41) v1 ==== 189+0+0 (secure 0 0 0) 0x7f974c015020 con 0x7f975005c3b0 2022-07-21T01:55:43.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 -- 172.21.15.140:0/430517549 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9730053a70 msgr2=0x7f9730055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9730053a70 0x7f9730055f30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f974400b000 tx=0x7f9744009040).stop 2022-07-21T01:55:43.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 -- 172.21.15.140:0/430517549 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f975005c3b0 msgr2=0x7f97501407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f975005c3b0 0x7f97501407d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f974c002f70 tx=0x7f974c014000).stop 2022-07-21T01:55:43.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 -- 172.21.15.140:0/430517549 shutdown_connections 2022-07-21T01:55:43.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9730053a70 0x7f9730055f30 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97500b57f0 0x7f975013dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97500aecf0 0x7f9750150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 --2- 172.21.15.140:0/430517549 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f975005c3b0 0x7f97501407d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 -- 172.21.15.140:0/430517549 >> 172.21.15.140:0/430517549 conn(0x7f9750063480 msgr2=0x7f9750065450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:43.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 -- 172.21.15.140:0/430517549 shutdown_connections 2022-07-21T01:55:43.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.351+0000 7f9756e2d700 1 -- 172.21.15.140:0/430517549 wait complete. 2022-07-21T01:55:43.357 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 17 and data pool 18 2022-07-21T01:55:43.367 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:43.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faf00891700 1 Processor -- start 2022-07-21T01:55:43.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faf00891700 1 -- start start 2022-07-21T01:55:43.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faf00891700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc05fa50 0x7faefc14a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faf00891700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc060980 con 0x7faefc0ac3a0 2022-07-21T01:55:43.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faf00891700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc0b22d0 con 0x7faefc0b1ef0 2022-07-21T01:55:43.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faf00891700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc098810 con 0x7faefc05fa50 2022-07-21T01:55:43.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefaffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7faefc0ac3a0 0x7faefc143690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52230/0 (socket says 172.21.15.140:52230) 2022-07-21T01:55:43.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefaffd700 1 -- 172.21.15.140:0/1043970978 learned_addr learned my addr 172.21.15.140:0/1043970978 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:43.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefbfff700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc05fa50 0x7faefc14a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597039792 0 0) 0x7faefc060980 con 0x7faefc0ac3a0 2022-07-21T01:55:43.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faedc003260 con 0x7faefc0ac3a0 2022-07-21T01:55:43.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3920355140 0 0) 0x7faedc003260 con 0x7faefc0ac3a0 2022-07-21T01:55:43.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc05fa50 msgr2=0x7faefc14a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:43.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc05fa50 0x7faefc14a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 >> v1:172.21.15.152:6789/0 conn(0x7faefc0b1ef0 legacy=0x7faefc142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc060bb0 con 0x7faefc0ac3a0 2022-07-21T01:55:43.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7faef40019e0 con 0x7faefc0ac3a0 2022-07-21T01:55:43.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 >> v1:172.21.15.140:6789/0 conn(0x7faefc0ac3a0 legacy=0x7faefc143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faedc003780 0x7faedc003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc05fa50 0x7faedc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faedc0049f0 0x7faedc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc098810 con 0x7faedc0049f0 2022-07-21T01:55:43.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc00da40 con 0x7faedc003780 2022-07-21T01:55:43.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faedc00dc00 con 0x7faefc05fa50 2022-07-21T01:55:43.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefbfff700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faedc0049f0 0x7faedc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefb7fe700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faedc003780 0x7faedc003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefaffd700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc05fa50 0x7faedc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefb7fe700 1 -- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc05fa50 msgr2=0x7faedc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:43.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefb7fe700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc05fa50 0x7faedc004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefb7fe700 1 -- 172.21.15.140:0/1043970978 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faedc0049f0 msgr2=0x7faedc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:43.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefb7fe700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faedc0049f0 0x7faedc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefb7fe700 1 -- 172.21.15.140:0/1043970978 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc003260 con 0x7faedc003780 2022-07-21T01:55:43.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefbfff700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faedc0049f0 0x7faedc004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:43.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefb7fe700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faedc003780 0x7faedc003ba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7faefc0626a0 tx=0x7faee4007a50).ready entity=mon.1 client_cookie=4e6ad406d562360f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:43.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faee4007030 con 0x7faedc003780 2022-07-21T01:55:43.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee4007190 con 0x7faedc003780 2022-07-21T01:55:43.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.459+0000 7faefa7fc700 1 -- 172.21.15.140:0/1043970978 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faee4006560 con 0x7faedc003780 2022-07-21T01:55:43.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 -- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faedc003780 msgr2=0x7faedc003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 --2- 172.21.15.140:0/1043970978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faedc003780 0x7faedc003ba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7faefc0626a0 tx=0x7faee4007a50).stop 2022-07-21T01:55:43.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faefbfff700 1 -- 172.21.15.140:0/1043970978 reap_dead start 2022-07-21T01:55:43.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 -- 172.21.15.140:0/1043970978 shutdown_connections 2022-07-21T01:55:43.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 -- 172.21.15.140:0/1043970978 >> 172.21.15.140:0/1043970978 conn(0x7faefc0b2ca0 msgr2=0x7faefc0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:43.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 -- 172.21.15.140:0/1043970978 shutdown_connections 2022-07-21T01:55:43.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 -- 172.21.15.140:0/1043970978 wait complete. 2022-07-21T01:55:43.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 Processor -- start 2022-07-21T01:55:43.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 -- start start 2022-07-21T01:55:43.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc140e80 0x7faefc141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faefc0995e0 0x7faefc151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faefc14dcb0 0x7faefc14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc14ef30 con 0x7faefc0995e0 2022-07-21T01:55:43.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc14e650 con 0x7faefc14dcb0 2022-07-21T01:55:43.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faf00891700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc152150 con 0x7faefc140e80 2022-07-21T01:55:43.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faefaffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faefc0995e0 0x7faefc151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faefaffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faefc0995e0 0x7faefc151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38098/0 (socket says 172.21.15.140:38098) 2022-07-21T01:55:43.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faefaffd700 1 -- 172.21.15.140:0/920839204 learned_addr learned my addr 172.21.15.140:0/920839204 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:43.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faefb7fe700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc140e80 0x7faefc141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.463+0000 7faefbfff700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faefc14dcb0 0x7faefc14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faefaffd700 1 -- 172.21.15.140:0/920839204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc140e80 msgr2=0x7faefc141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faefaffd700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc140e80 0x7faefc141260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faefaffd700 1 -- 172.21.15.140:0/920839204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faefc14dcb0 msgr2=0x7faefc14e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faefaffd700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faefc14dcb0 0x7faefc14e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faefaffd700 1 -- 172.21.15.140:0/920839204 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc151280 con 0x7faefc0995e0 2022-07-21T01:55:43.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faefaffd700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faefc0995e0 0x7faefc151c60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7faefc141a50 tx=0x7faefc13fa30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:43.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faef8ff9700 1 -- 172.21.15.140:0/920839204 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faef400f880 con 0x7faefc0995e0 2022-07-21T01:55:43.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faef8ff9700 1 -- 172.21.15.140:0/920839204 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef400f9e0 con 0x7faefc0995e0 2022-07-21T01:55:43.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faf00891700 1 -- 172.21.15.140:0/920839204 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc151520 con 0x7faefc0995e0 2022-07-21T01:55:43.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faef8ff9700 1 -- 172.21.15.140:0/920839204 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faef400fd40 con 0x7faefc0995e0 2022-07-21T01:55:43.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faf00891700 1 -- 172.21.15.140:0/920839204 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc0ae0c0 con 0x7faefc0995e0 2022-07-21T01:55:43.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faf00891700 1 -- 172.21.15.140:0/920839204 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faefc062270 con 0x7faefc0995e0 2022-07-21T01:55:43.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.467+0000 7faef8ff9700 1 -- 172.21.15.140:0/920839204 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faef40133f0 con 0x7faefc0995e0 2022-07-21T01:55:43.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.471+0000 7faef8ff9700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faee0053aa0 0x7faee0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.471+0000 7faef8ff9700 1 -- 172.21.15.140:0/920839204 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faef406bb10 con 0x7faefc0995e0 2022-07-21T01:55:43.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.471+0000 7faefb7fe700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faee0053aa0 0x7faee0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.471+0000 7faefb7fe700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faee0053aa0 0x7faee0055f60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7faee4019000 tx=0x7faee4013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:43.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.475+0000 7faef8ff9700 1 -- 172.21.15.140:0/920839204 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faef40429d0 con 0x7faefc0995e0 2022-07-21T01:55:43.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.599+0000 7faf00891700 1 -- 172.21.15.140:0/920839204 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faefc0574e0 con 0x7faefc0995e0 2022-07-21T01:55:43.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.599+0000 7faef8ff9700 1 -- 172.21.15.140:0/920839204 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16150 (secure 0 0 0) 0x7faef4019020 con 0x7faefc0995e0 2022-07-21T01:55:43.602 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:43.602 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":93,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:43.342907+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:55:41.913815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-07-21T01:55:42.343580+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:43.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 -- 172.21.15.140:0/920839204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faee0053aa0 msgr2=0x7faee0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faee0053aa0 0x7faee0055f60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7faee4019000 tx=0x7faee4013000).stop 2022-07-21T01:55:43.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 -- 172.21.15.140:0/920839204 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faefc0995e0 msgr2=0x7faefc151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faefc0995e0 0x7faefc151c60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7faefc141a50 tx=0x7faefc13fa30).stop 2022-07-21T01:55:43.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 -- 172.21.15.140:0/920839204 shutdown_connections 2022-07-21T01:55:43.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faee0053aa0 0x7faee0055f60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faefc14dcb0 0x7faefc14e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faefc0995e0 0x7faefc151c60 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 --2- 172.21.15.140:0/920839204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faefc140e80 0x7faefc141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 -- 172.21.15.140:0/920839204 >> 172.21.15.140:0/920839204 conn(0x7faefc0b2ca0 msgr2=0x7faefc1377c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:43.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 -- 172.21.15.140:0/920839204 shutdown_connections 2022-07-21T01:55:43.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.603+0000 7faf00891700 1 -- 172.21.15.140:0/920839204 wait complete. 2022-07-21T01:55:43.615 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:43.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.715+0000 7ff8d7ae5700 1 Processor -- start 2022-07-21T01:55:43.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.715+0000 7ff8d7ae5700 1 -- start start 2022-07-21T01:55:43.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff8d00b1e00 0x7ff8d014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8d005c3f0 0x7ff8d00ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff8d005bd90 0x7ff8d00accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d005d480 con 0x7ff8d00b1e00 2022-07-21T01:55:43.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d00ad1c0 con 0x7ff8d005bd90 2022-07-21T01:55:43.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d00b21e0 con 0x7ff8d005c3f0 2022-07-21T01:55:43.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d5881700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff8d00b1e00 0x7ff8d014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d5881700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff8d00b1e00 0x7ff8d014a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38104/0 (socket says 172.21.15.140:38104) 2022-07-21T01:55:43.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d5881700 1 -- 172.21.15.140:0/1950751110 learned_addr learned my addr 172.21.15.140:0/1950751110 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:43.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d6082700 1 --2- 172.21.15.140:0/1950751110 >> v2:172.21.15.152:3300/0 conn(0x7ff8d005bd90 0x7ff8d00accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d5080700 1 --2- 172.21.15.140:0/1950751110 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8d005c3f0 0x7ff8d00ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d5881700 1 -- 172.21.15.140:0/1950751110 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8d005c3f0 msgr2=0x7ff8d00ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:43.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d5881700 1 --2- 172.21.15.140:0/1950751110 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8d005c3f0 0x7ff8d00ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d5881700 1 -- 172.21.15.140:0/1950751110 >> v2:172.21.15.152:3300/0 conn(0x7ff8d005bd90 msgr2=0x7ff8d00accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:43.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d5881700 1 --2- 172.21.15.140:0/1950751110 >> v2:172.21.15.152:3300/0 conn(0x7ff8d005bd90 0x7ff8d00accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d5881700 1 -- 172.21.15.140:0/1950751110 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d005cf40 con 0x7ff8d00b1e00 2022-07-21T01:55:43.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d6082700 1 --2- 172.21.15.140:0/1950751110 >> v2:172.21.15.152:3300/0 conn(0x7ff8d005bd90 0x7ff8d00accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:43.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d5881700 1 --2- 172.21.15.140:0/1950751110 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8d00b1e00 0x7ff8d014a6d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7ff8c80032f0 tx=0x7ff8c8009290).ready entity=mon.0 client_cookie=2f7386c7555dc024 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:43.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d487f700 1 -- 172.21.15.140:0/1950751110 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8c800b870 con 0x7ff8d00b1e00 2022-07-21T01:55:43.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d487f700 1 -- 172.21.15.140:0/1950751110 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c800b9d0 con 0x7ff8d00b1e00 2022-07-21T01:55:43.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d487f700 1 -- 172.21.15.140:0/1950751110 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8c800bb40 con 0x7ff8d00b1e00 2022-07-21T01:55:43.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/1950751110 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8d00b1e00 msgr2=0x7ff8d014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 --2- 172.21.15.140:0/1950751110 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8d00b1e00 0x7ff8d014a6d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7ff8c80032f0 tx=0x7ff8c8009290).stop 2022-07-21T01:55:43.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/1950751110 shutdown_connections 2022-07-21T01:55:43.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 --2- 172.21.15.140:0/1950751110 >> v2:172.21.15.152:3300/0 conn(0x7ff8d005bd90 0x7ff8d00accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 --2- 172.21.15.140:0/1950751110 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8d005c3f0 0x7ff8d00ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 --2- 172.21.15.140:0/1950751110 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8d00b1e00 0x7ff8d014a6d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/1950751110 >> 172.21.15.140:0/1950751110 conn(0x7ff8d00611c0 msgr2=0x7ff8d0062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:43.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/1950751110 shutdown_connections 2022-07-21T01:55:43.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.719+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/1950751110 wait complete. 2022-07-21T01:55:43.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d7ae5700 1 Processor -- start 2022-07-21T01:55:43.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d7ae5700 1 -- start start 2022-07-21T01:55:43.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d7ae5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8d005c3f0 0x7ff8d00ab660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d7ae5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8d00b1e00 0x7ff8d015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d7ae5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8d014a630 0x7ff8d015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d7ae5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0149930 con 0x7ff8d014a630 2022-07-21T01:55:43.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d7ae5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0149b60 con 0x7ff8d005c3f0 2022-07-21T01:55:43.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d7ae5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0149e00 con 0x7ff8d00b1e00 2022-07-21T01:55:43.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d6082700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8d014a630 0x7ff8d015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d6082700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8d014a630 0x7ff8d015a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38112/0 (socket says 172.21.15.140:38112) 2022-07-21T01:55:43.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d6082700 1 -- 172.21.15.140:0/3183983940 learned_addr learned my addr 172.21.15.140:0/3183983940 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:43.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d5881700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8d005c3f0 0x7ff8d00ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d5080700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8d00b1e00 0x7ff8d015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d6082700 1 -- 172.21.15.140:0/3183983940 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8d00b1e00 msgr2=0x7ff8d015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d6082700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8d00b1e00 0x7ff8d015a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d6082700 1 -- 172.21.15.140:0/3183983940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8d005c3f0 msgr2=0x7ff8d00ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d6082700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8d005c3f0 0x7ff8d00ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d6082700 1 -- 172.21.15.140:0/3183983940 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c8009530 con 0x7ff8d014a630 2022-07-21T01:55:43.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d6082700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8d014a630 0x7ff8d015a7b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7ff8cc006940 tx=0x7ff8cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:43.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8beffd700 1 -- 172.21.15.140:0/3183983940 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8cc011780 con 0x7ff8d014a630 2022-07-21T01:55:43.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8beffd700 1 -- 172.21.15.140:0/3183983940 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc0118e0 con 0x7ff8d014a630 2022-07-21T01:55:43.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/3183983940 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d014a0a0 con 0x7ff8d014a630 2022-07-21T01:55:43.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8beffd700 1 -- 172.21.15.140:0/3183983940 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff8cc011c40 con 0x7ff8d014a630 2022-07-21T01:55:43.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.723+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/3183983940 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d0149220 con 0x7ff8d014a630 2022-07-21T01:55:43.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.727+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/3183983940 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d014a230 con 0x7ff8d014a630 2022-07-21T01:55:43.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.727+0000 7ff8beffd700 1 -- 172.21.15.140:0/3183983940 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff8cc01c460 con 0x7ff8d014a630 2022-07-21T01:55:43.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.727+0000 7ff8beffd700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff8b0053aa0 0x7ff8b0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.727+0000 7ff8beffd700 1 -- 172.21.15.140:0/3183983940 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff8cc06c2c0 con 0x7ff8d014a630 2022-07-21T01:55:43.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.727+0000 7ff8d5881700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff8b0053aa0 0x7ff8b0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.727+0000 7ff8d5881700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff8b0053aa0 0x7ff8b0055f60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7ff8c801a000 tx=0x7ff8c8016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:43.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.735+0000 7ff8beffd700 1 -- 172.21.15.140:0/3183983940 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff8cc043180 con 0x7ff8d014a630 2022-07-21T01:55:43.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.859+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/3183983940 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8d00574e0 con 0x7ff8d014a630 2022-07-21T01:55:43.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8beffd700 1 -- 172.21.15.140:0/3183983940 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16150 (secure 0 0 0) 0x7ff8cc0467a0 con 0x7ff8d014a630 2022-07-21T01:55:43.864 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:43.865 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":93,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:43.342907+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:55:41.913815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-07-21T01:55:42.343580+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:43.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/3183983940 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff8b0053aa0 msgr2=0x7ff8b0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff8b0053aa0 0x7ff8b0055f60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7ff8c801a000 tx=0x7ff8c8016000).stop 2022-07-21T01:55:43.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/3183983940 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8d014a630 msgr2=0x7ff8d015a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8d014a630 0x7ff8d015a7b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7ff8cc006940 tx=0x7ff8cc008040).stop 2022-07-21T01:55:43.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/3183983940 shutdown_connections 2022-07-21T01:55:43.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff8b0053aa0 0x7ff8b0055f60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8d014a630 0x7ff8d015a7b0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8d00b1e00 0x7ff8d015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 --2- 172.21.15.140:0/3183983940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8d005c3f0 0x7ff8d00ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/3183983940 >> 172.21.15.140:0/3183983940 conn(0x7ff8d005bd90 msgr2=0x7ff8d014d200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:43.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/3183983940 shutdown_connections 2022-07-21T01:55:43.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.863+0000 7ff8d7ae5700 1 -- 172.21.15.140:0/3183983940 wait complete. 2022-07-21T01:55:43.878 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:55:43.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.983+0000 7fbaedabb700 1 Processor -- start 2022-07-21T01:55:43.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.983+0000 7fbaedabb700 1 -- start start 2022-07-21T01:55:43.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbae80b62a0 0x7fbae80b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b3ed0 0x7fbae8143530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:43.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae80b6680 con 0x7fbae8143a20 2022-07-21T01:55:43.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae81465c0 con 0x7fbae80b62a0 2022-07-21T01:55:43.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae8142df0 con 0x7fbae80b3ed0 2022-07-21T01:55:43.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae77fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbae80b62a0 0x7fbae80b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae6ffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b3ed0 0x7fbae8143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:43.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae7fff700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbae8143a20 0x7fbae813fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:53382/0 (socket says 172.21.15.140:53382) 2022-07-21T01:55:43.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae7fff700 1 -- 172.21.15.140:0/2563471048 learned_addr learned my addr 172.21.15.140:0/2563471048 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:43.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae67fc700 1 -- 172.21.15.140:0/2563471048 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052134363 0 0) 0x7fbae80b6680 con 0x7fbae8143a20 2022-07-21T01:55:43.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae67fc700 1 -- 172.21.15.140:0/2563471048 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac8003260 con 0x7fbae8143a20 2022-07-21T01:55:43.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae6ffd700 1 -- 172.21.15.140:0/2563471048 >> v2:172.21.15.152:3300/0 conn(0x7fbae80b62a0 msgr2=0x7fbae80b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:43.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae6ffd700 1 --2- 172.21.15.140:0/2563471048 >> v2:172.21.15.152:3300/0 conn(0x7fbae80b62a0 0x7fbae80b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae6ffd700 1 -- 172.21.15.140:0/2563471048 >> v1:172.21.15.152:6789/0 conn(0x7fbae8143a20 legacy=0x7fbae813fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae6ffd700 1 -- 172.21.15.140:0/2563471048 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae8143090 con 0x7fbae80b3ed0 2022-07-21T01:55:43.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae6ffd700 1 --2- 172.21.15.140:0/2563471048 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b3ed0 0x7fbae8143530 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fbad0002f70 tx=0x7fbad0006df0).ready entity=mon.2 client_cookie=14047525113c21e1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:43.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae67fc700 1 -- 172.21.15.140:0/2563471048 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbad000d4a0 con 0x7fbae80b3ed0 2022-07-21T01:55:43.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae67fc700 1 -- 172.21.15.140:0/2563471048 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad0000e90 con 0x7fbae80b3ed0 2022-07-21T01:55:43.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbae67fc700 1 -- 172.21.15.140:0/2563471048 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbad000d7f0 con 0x7fbae80b3ed0 2022-07-21T01:55:43.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 -- 172.21.15.140:0/2563471048 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b3ed0 msgr2=0x7fbae8143530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:43.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 --2- 172.21.15.140:0/2563471048 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b3ed0 0x7fbae8143530 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fbad0002f70 tx=0x7fbad0006df0).stop 2022-07-21T01:55:43.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 -- 172.21.15.140:0/2563471048 shutdown_connections 2022-07-21T01:55:43.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 --2- 172.21.15.140:0/2563471048 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b3ed0 0x7fbae8143530 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 --2- 172.21.15.140:0/2563471048 >> v2:172.21.15.152:3300/0 conn(0x7fbae80b62a0 0x7fbae80b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:43.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 -- 172.21.15.140:0/2563471048 >> 172.21.15.140:0/2563471048 conn(0x7fbae8066f00 msgr2=0x7fbae80672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:43.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 -- 172.21.15.140:0/2563471048 shutdown_connections 2022-07-21T01:55:43.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.987+0000 7fbaedabb700 1 -- 172.21.15.140:0/2563471048 wait complete. 2022-07-21T01:55:44.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbaedabb700 1 Processor -- start 2022-07-21T01:55:44.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbaedabb700 1 -- start start 2022-07-21T01:55:44.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbaedabb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae80b3ed0 0x7fbae8140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbaedabb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b62a0 0x7fbae81420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbaedabb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbae8143a20 0x7fbae813cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbaedabb700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae8140290 con 0x7fbae8143a20 2022-07-21T01:55:44.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbaedabb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae81142f0 con 0x7fbae80b3ed0 2022-07-21T01:55:44.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbaedabb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae8142820 con 0x7fbae80b62a0 2022-07-21T01:55:44.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbae8143a20 0x7fbae813cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae77fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae80b3ed0 0x7fbae8140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae6ffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b62a0 0x7fbae81420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbae8143a20 0x7fbae813cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38128/0 (socket says 172.21.15.140:38128) 2022-07-21T01:55:44.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae7fff700 1 -- 172.21.15.140:0/1252423156 learned_addr learned my addr 172.21.15.140:0/1252423156 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:44.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae7fff700 1 -- 172.21.15.140:0/1252423156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b62a0 msgr2=0x7fbae81420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae7fff700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b62a0 0x7fbae81420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae7fff700 1 -- 172.21.15.140:0/1252423156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae80b3ed0 msgr2=0x7fbae8140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae7fff700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae80b3ed0 0x7fbae8140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae7fff700 1 -- 172.21.15.140:0/1252423156 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad000b180 con 0x7fbae8143a20 2022-07-21T01:55:44.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae6ffd700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b62a0 0x7fbae81420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:44.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae7fff700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbae8143a20 0x7fbae813cb00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fbae0001490 tx=0x7fbae0007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:44.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae4ff9700 1 -- 172.21.15.140:0/1252423156 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbae0005c70 con 0x7fbae8143a20 2022-07-21T01:55:44.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae4ff9700 1 -- 172.21.15.140:0/1252423156 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae0005dd0 con 0x7fbae8143a20 2022-07-21T01:55:44.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbaedabb700 1 -- 172.21.15.140:0/1252423156 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae8113ff0 con 0x7fbae8143a20 2022-07-21T01:55:44.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbae4ff9700 1 -- 172.21.15.140:0/1252423156 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbae00072c0 con 0x7fbae8143a20 2022-07-21T01:55:44.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.991+0000 7fbaedabb700 1 -- 172.21.15.140:0/1252423156 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae81408e0 con 0x7fbae8143a20 2022-07-21T01:55:44.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.995+0000 7fbaedabb700 1 -- 172.21.15.140:0/1252423156 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae8141b50 con 0x7fbae8143a20 2022-07-21T01:55:44.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.995+0000 7fbae4ff9700 1 -- 172.21.15.140:0/1252423156 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbae0010070 con 0x7fbae8143a20 2022-07-21T01:55:44.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.995+0000 7fbae4ff9700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbacc0539e0 0x7fbacc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.995+0000 7fbae4ff9700 1 -- 172.21.15.140:0/1252423156 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbae006b990 con 0x7fbae8143a20 2022-07-21T01:55:44.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.995+0000 7fbae77fe700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbacc0539e0 0x7fbacc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.995+0000 7fbae77fe700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbacc0539e0 0x7fbacc055ea0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fbad800b000 tx=0x7fbad8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:44.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:43.999+0000 7fbae4ff9700 1 -- 172.21.15.140:0/1252423156 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbae00427a0 con 0x7fbae8143a20 2022-07-21T01:55:44.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.155+0000 7fbaedabb700 1 -- 172.21.15.140:0/1252423156 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbae81462b0 con 0x7fbae8143a20 2022-07-21T01:55:44.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.355+0000 7fbae4ff9700 1 -- 172.21.15.140:0/1252423156 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbae0045dc0 con 0x7fbae8143a20 2022-07-21T01:55:44.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 -- 172.21.15.140:0/1252423156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbacc0539e0 msgr2=0x7fbacc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbacc0539e0 0x7fbacc055ea0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fbad800b000 tx=0x7fbad8006df0).stop 2022-07-21T01:55:44.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 -- 172.21.15.140:0/1252423156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbae8143a20 msgr2=0x7fbae813cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbae8143a20 0x7fbae813cb00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7fbae0001490 tx=0x7fbae0007dc0).stop 2022-07-21T01:55:44.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 -- 172.21.15.140:0/1252423156 shutdown_connections 2022-07-21T01:55:44.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbacc0539e0 0x7fbacc055ea0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbae8143a20 0x7fbae813cb00 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbae80b62a0 0x7fbae81420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 --2- 172.21.15.140:0/1252423156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbae80b3ed0 0x7fbae8140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 -- 172.21.15.140:0/1252423156 >> 172.21.15.140:0/1252423156 conn(0x7fbae8066f00 msgr2=0x7fbae80b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:44.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 -- 172.21.15.140:0/1252423156 shutdown_connections 2022-07-21T01:55:44.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.359+0000 7fbaedabb700 1 -- 172.21.15.140:0/1252423156 wait complete. 2022-07-21T01:55:44.376 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:44.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.475+0000 7f0fdb27b700 1 Processor -- start 2022-07-21T01:55:44.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.475+0000 7f0fdb27b700 1 -- start start 2022-07-21T01:55:44.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.475+0000 7f0fdb27b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0fd40b3ca0 0x7f0fd40b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.475+0000 7f0fdb27b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0fd413cb70 0x7f0fd4143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.475+0000 7f0fdb27b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd4142ba0 con 0x7f0fd413c790 2022-07-21T01:55:44.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd41429b0 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd4146330 con 0x7f0fd413cb70 2022-07-21T01:55:44.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fda279700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0fd413c790 0x7f0fd40b9a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52278/0 (socket says 172.21.15.140:52278) 2022-07-21T01:55:44.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fda279700 1 -- 172.21.15.140:0/1557863206 learned_addr learned my addr 172.21.15.140:0/1557863206 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:44.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9a78700 1 --2- 172.21.15.140:0/1557863206 >> v2:172.21.15.140:3300/0 conn(0x7f0fd40b3ca0 0x7f0fd40b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdaa7a700 1 --2- 172.21.15.140:0/1557863206 >> v2:172.21.15.152:3300/0 conn(0x7f0fd413cb70 0x7f0fd4143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9277700 1 -- 172.21.15.140:0/1557863206 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214153277 0 0) 0x7f0fd4142ba0 con 0x7f0fd413c790 2022-07-21T01:55:44.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9277700 1 -- 172.21.15.140:0/1557863206 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fbc003260 con 0x7f0fd413c790 2022-07-21T01:55:44.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9a78700 1 -- 172.21.15.140:0/1557863206 >> v2:172.21.15.152:3300/0 conn(0x7f0fd413cb70 msgr2=0x7f0fd4143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:44.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9a78700 1 --2- 172.21.15.140:0/1557863206 >> v2:172.21.15.152:3300/0 conn(0x7f0fd413cb70 0x7f0fd4143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9a78700 1 -- 172.21.15.140:0/1557863206 >> v1:172.21.15.140:6789/0 conn(0x7f0fd413c790 legacy=0x7f0fd40b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9277700 1 -- 172.21.15.140:0/1557863206 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1055820959 0 0) 0x7f0fbc003260 con 0x7f0fd413c790 2022-07-21T01:55:44.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9a78700 1 -- 172.21.15.140:0/1557863206 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd40ba120 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9a78700 1 --2- 172.21.15.140:0/1557863206 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fd40b3ca0 0x7f0fd40b4710 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f0fd0001800 tx=0x7f0fc4006df0).ready entity=mon.0 client_cookie=693fddd81afd5ca4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:44.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9277700 1 -- 172.21.15.140:0/1557863206 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd0001660 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9277700 1 -- 172.21.15.140:0/1557863206 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc4000e90 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fd9277700 1 -- 172.21.15.140:0/1557863206 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fc4012460 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 -- 172.21.15.140:0/1557863206 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fd40b3ca0 msgr2=0x7f0fd40b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 --2- 172.21.15.140:0/1557863206 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fd40b3ca0 0x7f0fd40b4710 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f0fd0001800 tx=0x7f0fc4006df0).stop 2022-07-21T01:55:44.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 -- 172.21.15.140:0/1557863206 shutdown_connections 2022-07-21T01:55:44.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 --2- 172.21.15.140:0/1557863206 >> v2:172.21.15.152:3300/0 conn(0x7f0fd413cb70 0x7f0fd4143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 --2- 172.21.15.140:0/1557863206 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fd40b3ca0 0x7f0fd40b4710 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 -- 172.21.15.140:0/1557863206 >> 172.21.15.140:0/1557863206 conn(0x7f0fd40678b0 msgr2=0x7f0fd40b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:44.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 -- 172.21.15.140:0/1557863206 shutdown_connections 2022-07-21T01:55:44.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 -- 172.21.15.140:0/1557863206 wait complete. 2022-07-21T01:55:44.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 Processor -- start 2022-07-21T01:55:44.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.479+0000 7f0fdb27b700 1 -- start start 2022-07-21T01:55:44.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fdb27b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fd40b3ca0 0x7f0fd40b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fdb27b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fd413c790 0x7f0fd40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fdb27b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fd413cb70 0x7f0fd40af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fdb27b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40ae010 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fdb27b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40add70 con 0x7f0fd413cb70 2022-07-21T01:55:44.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fdb27b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd4140030 con 0x7f0fd413c790 2022-07-21T01:55:44.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fda279700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fd40b3ca0 0x7f0fd40b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fda279700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fd40b3ca0 0x7f0fd40b17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38138/0 (socket says 172.21.15.140:38138) 2022-07-21T01:55:44.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fda279700 1 -- 172.21.15.140:0/2039705952 learned_addr learned my addr 172.21.15.140:0/2039705952 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:44.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fdaa7a700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fd413cb70 0x7f0fd40af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fd9a78700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fd413c790 0x7f0fd40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fda279700 1 -- 172.21.15.140:0/2039705952 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fd413c790 msgr2=0x7f0fd40aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fda279700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fd413c790 0x7f0fd40aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fda279700 1 -- 172.21.15.140:0/2039705952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fd413cb70 msgr2=0x7f0fd40af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fda279700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fd413cb70 0x7f0fd40af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fda279700 1 -- 172.21.15.140:0/2039705952 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc400b180 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fda279700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fd40b3ca0 0x7f0fd40b17d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f0fd00018c0 tx=0x7f0fd0001230).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:44.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fc37fe700 1 -- 172.21.15.140:0/2039705952 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd000ce50 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fc37fe700 1 -- 172.21.15.140:0/2039705952 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0007170 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fdb27b700 1 -- 172.21.15.140:0/2039705952 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd40ae7b0 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fc37fe700 1 -- 172.21.15.140:0/2039705952 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fd0004730 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.483+0000 7f0fdb27b700 1 -- 172.21.15.140:0/2039705952 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd4139990 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.487+0000 7f0fdb27b700 1 -- 172.21.15.140:0/2039705952 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd41401f0 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.487+0000 7f0fc37fe700 1 -- 172.21.15.140:0/2039705952 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0fd0004890 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.487+0000 7f0fc37fe700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fb4053a70 0x7f0fb4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.487+0000 7f0fc37fe700 1 -- 172.21.15.140:0/2039705952 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0fd006ba40 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.487+0000 7f0fd9a78700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fb4053a70 0x7f0fb4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.487+0000 7f0fd9a78700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fb4053a70 0x7f0fb4055f30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f0fc401a000 tx=0x7f0fc4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:44.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.491+0000 7f0fc37fe700 1 -- 172.21.15.140:0/2039705952 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0fd00427f0 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.655+0000 7f0fdb27b700 1 -- 172.21.15.140:0/2039705952 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fd40574e0 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.655+0000 7f0fc37fe700 1 -- 172.21.15.140:0/2039705952 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f0fd0045e10 con 0x7f0fd40b3ca0 2022-07-21T01:55:44.658 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:44.659 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/518920349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":518920349},{"type":"v1","addr":"172.21.15.152:6833","nonce":518920349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:43.343194+0000","modified":"2022-07-21T01:55:44.346904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2478727503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2478727503},{"type":"v1","addr":"172.21.15.140:6835","nonce":2478727503}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-07-21T01:55:44.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 -- 172.21.15.140:0/2039705952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fb4053a70 msgr2=0x7f0fb4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fb4053a70 0x7f0fb4055f30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f0fc401a000 tx=0x7f0fc4014000).stop 2022-07-21T01:55:44.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 -- 172.21.15.140:0/2039705952 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fd40b3ca0 msgr2=0x7f0fd40b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fd40b3ca0 0x7f0fd40b17d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f0fd00018c0 tx=0x7f0fd0001230).stop 2022-07-21T01:55:44.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 -- 172.21.15.140:0/2039705952 shutdown_connections 2022-07-21T01:55:44.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fb4053a70 0x7f0fb4055f30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0fd413cb70 0x7f0fd40af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0fd413c790 0x7f0fd40aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 --2- 172.21.15.140:0/2039705952 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0fd40b3ca0 0x7f0fd40b17d0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 -- 172.21.15.140:0/2039705952 >> 172.21.15.140:0/2039705952 conn(0x7f0fd40678b0 msgr2=0x7f0fd4145670 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:44.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 -- 172.21.15.140:0/2039705952 shutdown_connections 2022-07-21T01:55:44.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.659+0000 7f0fdb27b700 1 -- 172.21.15.140:0/2039705952 wait complete. 2022-07-21T01:55:44.664 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 42 2022-07-21T01:55:44.672 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:44.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3ba3f8700 1 Processor -- start 2022-07-21T01:55:44.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3ba3f8700 1 -- start start 2022-07-21T01:55:44.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3ba3f8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd3b40b1200 0x7fd3b40b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3ba3f8700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd3b405bd90 0x7fd3b4142d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3ba3f8700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b40aea30 con 0x7fd3b40b5650 2022-07-21T01:55:44.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3ba3f8700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b4142970 con 0x7fd3b40b1200 2022-07-21T01:55:44.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3ba3f8700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b40aec60 con 0x7fd3b405bd90 2022-07-21T01:55:44.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3b8bf5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd3b40b1200 0x7fd3b40b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3b93f6700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd3b40b5650 0x7fd3b405c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52292/0 (socket says 172.21.15.140:52292) 2022-07-21T01:55:44.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3b93f6700 1 -- 172.21.15.140:0/3433213874 learned_addr learned my addr 172.21.15.140:0/3433213874 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:44.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3b9bf7700 1 --2- 172.21.15.140:0/3433213874 >> v2:172.21.15.152:3300/0 conn(0x7fd3b405bd90 0x7fd3b4142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3b8bf5700 1 -- 172.21.15.140:0/3433213874 >> v2:172.21.15.152:3300/0 conn(0x7fd3b405bd90 msgr2=0x7fd3b4142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:44.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3b8bf5700 1 --2- 172.21.15.140:0/3433213874 >> v2:172.21.15.152:3300/0 conn(0x7fd3b405bd90 0x7fd3b4142d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3b8bf5700 1 -- 172.21.15.140:0/3433213874 >> v1:172.21.15.140:6789/0 conn(0x7fd3b40b5650 legacy=0x7fd3b405c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3b8bf5700 1 -- 172.21.15.140:0/3433213874 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b40af180 con 0x7fd3b40b1200 2022-07-21T01:55:44.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3abfff700 1 -- 172.21.15.140:0/3433213874 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819711946 0 0) 0x7fd3b40aea30 con 0x7fd3b40b5650 2022-07-21T01:55:44.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3b8bf5700 1 --2- 172.21.15.140:0/3433213874 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3b40b1200 0x7fd3b40b5a30 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fd3a4002f70 tx=0x7fd3a4006d90).ready entity=mon.0 client_cookie=6eac1a420f4f9c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:44.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3abfff700 1 -- 172.21.15.140:0/3433213874 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3a400d4a0 con 0x7fd3b40b1200 2022-07-21T01:55:44.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3abfff700 1 -- 172.21.15.140:0/3433213874 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a4000e90 con 0x7fd3b40b1200 2022-07-21T01:55:44.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.771+0000 7fd3abfff700 1 -- 172.21.15.140:0/3433213874 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3a400d7f0 con 0x7fd3b40b1200 2022-07-21T01:55:44.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.775+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/3433213874 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3b40b1200 msgr2=0x7fd3b40b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.775+0000 7fd3ba3f8700 1 --2- 172.21.15.140:0/3433213874 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3b40b1200 0x7fd3b40b5a30 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fd3a4002f70 tx=0x7fd3a4006d90).stop 2022-07-21T01:55:44.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.775+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/3433213874 shutdown_connections 2022-07-21T01:55:44.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.775+0000 7fd3ba3f8700 1 --2- 172.21.15.140:0/3433213874 >> v2:172.21.15.152:3300/0 conn(0x7fd3b405bd90 0x7fd3b4142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.775+0000 7fd3ba3f8700 1 --2- 172.21.15.140:0/3433213874 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3b40b1200 0x7fd3b40b5a30 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.775+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/3433213874 >> 172.21.15.140:0/3433213874 conn(0x7fd3b40634e0 msgr2=0x7fd3b40b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:44.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.775+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/3433213874 shutdown_connections 2022-07-21T01:55:44.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.775+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/3433213874 wait complete. 2022-07-21T01:55:44.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.775+0000 7fd3ba3f8700 1 Processor -- start 2022-07-21T01:55:44.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.775+0000 7fd3ba3f8700 1 -- start start 2022-07-21T01:55:44.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3ba3f8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3b405bd90 0x7fd3b4063be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3ba3f8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3b40b1200 0x7fd3b413d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3ba3f8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3b40b5650 0x7fd3b4066e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3ba3f8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b4156380 con 0x7fd3b405bd90 2022-07-21T01:55:44.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b93f6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3b405bd90 0x7fd3b4063be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3ba3f8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b41559a0 con 0x7fd3b40b1200 2022-07-21T01:55:44.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3ba3f8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b4155c40 con 0x7fd3b40b5650 2022-07-21T01:55:44.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b93f6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3b405bd90 0x7fd3b4063be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38152/0 (socket says 172.21.15.140:38152) 2022-07-21T01:55:44.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b93f6700 1 -- 172.21.15.140:0/1509475663 learned_addr learned my addr 172.21.15.140:0/1509475663 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:44.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b8bf5700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3b40b1200 0x7fd3b413d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b9bf7700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3b40b5650 0x7fd3b4066e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b93f6700 1 -- 172.21.15.140:0/1509475663 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3b40b5650 msgr2=0x7fd3b4066e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b93f6700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3b40b5650 0x7fd3b4066e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b93f6700 1 -- 172.21.15.140:0/1509475663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3b40b1200 msgr2=0x7fd3b413d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b93f6700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3b40b1200 0x7fd3b413d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b93f6700 1 -- 172.21.15.140:0/1509475663 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a400b120 con 0x7fd3b405bd90 2022-07-21T01:55:44.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b8bf5700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3b40b1200 0x7fd3b413d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:44.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3b93f6700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3b405bd90 0x7fd3b4063be0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fd3b0000bc0 tx=0x7fd3b0007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:44.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3aa7fc700 1 -- 172.21.15.140:0/1509475663 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b00070d0 con 0x7fd3b405bd90 2022-07-21T01:55:44.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3aa7fc700 1 -- 172.21.15.140:0/1509475663 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b0007230 con 0x7fd3b405bd90 2022-07-21T01:55:44.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3aa7fc700 1 -- 172.21.15.140:0/1509475663 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b0005610 con 0x7fd3b405bd90 2022-07-21T01:55:44.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/1509475663 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b4155ee0 con 0x7fd3b405bd90 2022-07-21T01:55:44.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.779+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/1509475663 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b4155500 con 0x7fd3b405bd90 2022-07-21T01:55:44.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.783+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/1509475663 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b41557b0 con 0x7fd3b405bd90 2022-07-21T01:55:44.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.783+0000 7fd3aa7fc700 1 -- 172.21.15.140:0/1509475663 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd3b0010070 con 0x7fd3b405bd90 2022-07-21T01:55:44.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.783+0000 7fd3aa7fc700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd39c053b00 0x7fd39c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:44.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.783+0000 7fd3aa7fc700 1 -- 172.21.15.140:0/1509475663 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd3b006b6b0 con 0x7fd3b405bd90 2022-07-21T01:55:44.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.783+0000 7fd3b8bf5700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd39c053b00 0x7fd39c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:44.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.783+0000 7fd3b8bf5700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd39c053b00 0x7fd39c055fc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fd3a4018000 tx=0x7fd3a4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:44.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.787+0000 7fd3aa7fc700 1 -- 172.21.15.140:0/1509475663 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd3b00424c0 con 0x7fd3b405bd90 2022-07-21T01:55:44.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.915+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/1509475663 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3b4156070 con 0x7fd3b405bd90 2022-07-21T01:55:44.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.915+0000 7fd3aa7fc700 1 -- 172.21.15.140:0/1509475663 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16137 (secure 0 0 0) 0x7fd3b0045ae0 con 0x7fd3b405bd90 2022-07-21T01:55:44.917 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:44.918 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":94,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:44.346577+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:55:41.913815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-07-21T01:55:42.343580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:44.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/1509475663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd39c053b00 msgr2=0x7fd39c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd39c053b00 0x7fd39c055fc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fd3a4018000 tx=0x7fd3a4014000).stop 2022-07-21T01:55:44.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/1509475663 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3b405bd90 msgr2=0x7fd3b4063be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:44.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3b405bd90 0x7fd3b4063be0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fd3b0000bc0 tx=0x7fd3b0007c90).stop 2022-07-21T01:55:44.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/1509475663 shutdown_connections 2022-07-21T01:55:44.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd39c053b00 0x7fd39c055fc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3b40b5650 0x7fd3b4066e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3b40b1200 0x7fd3b413d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 --2- 172.21.15.140:0/1509475663 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd3b405bd90 0x7fd3b4063be0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:44.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/1509475663 >> 172.21.15.140:0/1509475663 conn(0x7fd3b40634e0 msgr2=0x7fd3b40adb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:44.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/1509475663 shutdown_connections 2022-07-21T01:55:44.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:44.919+0000 7fd3ba3f8700 1 -- 172.21.15.140:0/1509475663 wait complete. 2022-07-21T01:55:44.931 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:55:45.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.035+0000 7fa4828a2700 1 Processor -- start 2022-07-21T01:55:45.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.035+0000 7fa4828a2700 1 -- start start 2022-07-21T01:55:45.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa47c0b6800 0x7fa47c0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa47c0b3cb0 0x7fa47c145f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c0ae280 con 0x7fa47c0ad0e0 2022-07-21T01:55:45.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c0b6be0 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c0ae4b0 con 0x7fa47c0b6800 2022-07-21T01:55:45.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa48109f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa47c0b3cb0 0x7fa47c145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4818a0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa47c0b6800 0x7fa47c0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4818a0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa47c0b6800 0x7fa47c0aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:43900/0 (socket says 172.21.15.140:43900) 2022-07-21T01:55:45.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4818a0700 1 -- 172.21.15.140:0/1825270423 learned_addr learned my addr 172.21.15.140:0/1825270423 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:45.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa48109f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa47c0b3cb0 0x7fa47c145f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38164/0 (socket says 172.21.15.140:38164) 2022-07-21T01:55:45.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4820a1700 1 --1- 172.21.15.140:0/1825270423 >> v1:172.21.15.152:6789/0 conn(0x7fa47c0ad0e0 0x7fa47c05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:53422/0 (socket says 172.21.15.140:53422) 2022-07-21T01:55:45.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa48109f700 1 -- 172.21.15.140:0/1825270423 >> v2:172.21.15.152:3300/0 conn(0x7fa47c0b6800 msgr2=0x7fa47c0aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:45.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa48109f700 1 --2- 172.21.15.140:0/1825270423 >> v2:172.21.15.152:3300/0 conn(0x7fa47c0b6800 0x7fa47c0aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa48109f700 1 -- 172.21.15.140:0/1825270423 >> v1:172.21.15.152:6789/0 conn(0x7fa47c0ad0e0 legacy=0x7fa47c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa48089e700 1 -- 172.21.15.140:0/1825270423 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441934112 0 0) 0x7fa47c0ae280 con 0x7fa47c0ad0e0 2022-07-21T01:55:45.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa48109f700 1 -- 172.21.15.140:0/1825270423 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0ae750 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa48109f700 1 --2- 172.21.15.140:0/1825270423 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa47c0b3cb0 0x7fa47c145f40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7fa46c002f70 tx=0x7fa46c006df0).ready entity=mon.0 client_cookie=8072f76375d57aea server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:45.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa48089e700 1 -- 172.21.15.140:0/1825270423 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46c00d570 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa48089e700 1 -- 172.21.15.140:0/1825270423 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c000e90 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa48089e700 1 -- 172.21.15.140:0/1825270423 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46c00d8c0 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 -- 172.21.15.140:0/1825270423 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa47c0b3cb0 msgr2=0x7fa47c145f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 --2- 172.21.15.140:0/1825270423 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa47c0b3cb0 0x7fa47c145f40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7fa46c002f70 tx=0x7fa46c006df0).stop 2022-07-21T01:55:45.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 -- 172.21.15.140:0/1825270423 shutdown_connections 2022-07-21T01:55:45.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 --2- 172.21.15.140:0/1825270423 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa47c0b3cb0 0x7fa47c145f40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 --2- 172.21.15.140:0/1825270423 >> v2:172.21.15.152:3300/0 conn(0x7fa47c0b6800 0x7fa47c0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 -- 172.21.15.140:0/1825270423 >> 172.21.15.140:0/1825270423 conn(0x7fa47c0b9dd0 msgr2=0x7fa47c0b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:45.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 -- 172.21.15.140:0/1825270423 shutdown_connections 2022-07-21T01:55:45.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.039+0000 7fa4828a2700 1 -- 172.21.15.140:0/1825270423 wait complete. 2022-07-21T01:55:45.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4828a2700 1 Processor -- start 2022-07-21T01:55:45.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4828a2700 1 -- start start 2022-07-21T01:55:45.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4828a2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa47c0ad0e0 0x7fa47c149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4828a2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa47c0b3cb0 0x7fa47c144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4828a2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa47c0b6800 0x7fa47c13e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4828a2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0af1e0 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4828a2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0af480 con 0x7fa47c0ad0e0 2022-07-21T01:55:45.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4828a2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c151e00 con 0x7fa47c0b6800 2022-07-21T01:55:45.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa48109f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa47c0b3cb0 0x7fa47c144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa48109f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa47c0b3cb0 0x7fa47c144360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38168/0 (socket says 172.21.15.140:38168) 2022-07-21T01:55:45.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa48109f700 1 -- 172.21.15.140:0/1187062266 learned_addr learned my addr 172.21.15.140:0/1187062266 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:45.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4818a0700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa47c0ad0e0 0x7fa47c149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4820a1700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa47c0b6800 0x7fa47c13e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa48109f700 1 -- 172.21.15.140:0/1187062266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa47c0b6800 msgr2=0x7fa47c13e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa48109f700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa47c0b6800 0x7fa47c13e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa48109f700 1 -- 172.21.15.140:0/1187062266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa47c0ad0e0 msgr2=0x7fa47c149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa48109f700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa47c0ad0e0 0x7fa47c149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa48109f700 1 -- 172.21.15.140:0/1187062266 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c00b180 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa48109f700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa47c0b3cb0 0x7fa47c144360 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7fa46c002f70 tx=0x7fa46c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:45.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa46affd700 1 -- 172.21.15.140:0/1187062266 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46c00dea0 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa46affd700 1 -- 172.21.15.140:0/1187062266 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c013e90 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4828a2700 1 -- 172.21.15.140:0/1187062266 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c144850 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa46affd700 1 -- 172.21.15.140:0/1187062266 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa46c01e5f0 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.043+0000 7fa4828a2700 1 -- 172.21.15.140:0/1187062266 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c145dd0 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.047+0000 7fa4828a2700 1 -- 172.21.15.140:0/1187062266 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c0b5530 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.047+0000 7fa46affd700 1 -- 172.21.15.140:0/1187062266 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa46c025030 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.047+0000 7fa46affd700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa45c053b00 0x7fa45c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.047+0000 7fa46affd700 1 -- 172.21.15.140:0/1187062266 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa46c06ccb0 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.047+0000 7fa4818a0700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa45c053b00 0x7fa45c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.047+0000 7fa4818a0700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa45c053b00 0x7fa45c055fc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fa478008000 tx=0x7fa478008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:45.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.055+0000 7fa46affd700 1 -- 172.21.15.140:0/1187062266 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa46c043ac0 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.215+0000 7fa4828a2700 1 -- 172.21.15.140:0/1187062266 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa47c0574e0 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa46affd700 1 -- 172.21.15.140:0/1187062266 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa46c023070 con 0x7fa47c0b3cb0 2022-07-21T01:55:45.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa4828a2700 1 -- 172.21.15.140:0/1187062266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa45c053b00 msgr2=0x7fa45c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa4828a2700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa45c053b00 0x7fa45c055fc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fa478008000 tx=0x7fa478008040).stop 2022-07-21T01:55:45.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa4828a2700 1 -- 172.21.15.140:0/1187062266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa47c0b3cb0 msgr2=0x7fa47c144360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa4828a2700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa47c0b3cb0 0x7fa47c144360 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7fa46c002f70 tx=0x7fa46c014000).stop 2022-07-21T01:55:45.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa4828a2700 1 -- 172.21.15.140:0/1187062266 shutdown_connections 2022-07-21T01:55:45.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa4828a2700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa45c053b00 0x7fa45c055fc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa4828a2700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa47c0b6800 0x7fa47c13e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa4828a2700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa47c0b3cb0 0x7fa47c144360 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa4828a2700 1 --2- 172.21.15.140:0/1187062266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa47c0ad0e0 0x7fa47c149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa4828a2700 1 -- 172.21.15.140:0/1187062266 >> 172.21.15.140:0/1187062266 conn(0x7fa47c0b9dd0 msgr2=0x7fa47c0b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:45.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.223+0000 7fa4828a2700 1 -- 172.21.15.140:0/1187062266 shutdown_connections 2022-07-21T01:55:45.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.227+0000 7fa4828a2700 1 -- 172.21.15.140:0/1187062266 wait complete. 2022-07-21T01:55:45.230 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:55:45.238 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:55:45.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c94c1700 1 Processor -- start 2022-07-21T01:55:45.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c94c1700 1 -- start start 2022-07-21T01:55:45.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c94c1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2c413a770 0x7fc2c4067120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c94c1700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc2c4067610 0x7fc2c405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c94c1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c40b4340 con 0x7fc2c405c280 2022-07-21T01:55:45.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c94c1700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c4066680 con 0x7fc2c4067610 2022-07-21T01:55:45.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c94c1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40b1d80 con 0x7fc2c413a770 2022-07-21T01:55:45.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c37fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc2c4067610 0x7fc2c405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c37fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc2c4067610 0x7fc2c405bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38174/0 (socket says 172.21.15.140:38174) 2022-07-21T01:55:45.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c37fe700 1 -- 172.21.15.140:0/299512508 learned_addr learned my addr 172.21.15.140:0/299512508 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:45.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c3fff700 1 --2- 172.21.15.140:0/299512508 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2c413a770 0x7fc2c4067120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c37fe700 1 -- 172.21.15.140:0/299512508 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2c413a770 msgr2=0x7fc2c4067120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:45.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c2ffd700 1 -- 172.21.15.140:0/299512508 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250031514 0 0) 0x7fc2c40b4340 con 0x7fc2c405c280 2022-07-21T01:55:45.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c37fe700 1 --2- 172.21.15.140:0/299512508 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2c413a770 0x7fc2c4067120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c37fe700 1 -- 172.21.15.140:0/299512508 >> v1:172.21.15.140:6789/0 conn(0x7fc2c405c280 legacy=0x7fc2c4140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c37fe700 1 -- 172.21.15.140:0/299512508 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c40b1ae0 con 0x7fc2c4067610 2022-07-21T01:55:45.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c37fe700 1 --2- 172.21.15.140:0/299512508 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2c4067610 0x7fc2c405bd90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7fc2b4002f70 tx=0x7fc2b4006eb0).ready entity=mon.0 client_cookie=584ccd35d33754c8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:45.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c2ffd700 1 -- 172.21.15.140:0/299512508 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2b400d570 con 0x7fc2c4067610 2022-07-21T01:55:45.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c2ffd700 1 -- 172.21.15.140:0/299512508 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b4000e90 con 0x7fc2c4067610 2022-07-21T01:55:45.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.355+0000 7fc2c2ffd700 1 -- 172.21.15.140:0/299512508 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2b400d8c0 con 0x7fc2c4067610 2022-07-21T01:55:45.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 -- 172.21.15.140:0/299512508 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2c4067610 msgr2=0x7fc2c405bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 --2- 172.21.15.140:0/299512508 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2c4067610 0x7fc2c405bd90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7fc2b4002f70 tx=0x7fc2b4006eb0).stop 2022-07-21T01:55:45.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 -- 172.21.15.140:0/299512508 shutdown_connections 2022-07-21T01:55:45.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 --2- 172.21.15.140:0/299512508 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2c4067610 0x7fc2c405bd90 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 --2- 172.21.15.140:0/299512508 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2c413a770 0x7fc2c4067120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 -- 172.21.15.140:0/299512508 >> 172.21.15.140:0/299512508 conn(0x7fc2c4064c00 msgr2=0x7fc2c4064fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:45.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 -- 172.21.15.140:0/299512508 shutdown_connections 2022-07-21T01:55:45.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 -- 172.21.15.140:0/299512508 wait complete. 2022-07-21T01:55:45.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 Processor -- start 2022-07-21T01:55:45.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 -- start start 2022-07-21T01:55:45.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2c405c280 0x7fc2c413d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2c4067610 0x7fc2c4139a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2c413a770 0x7fc2c4139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c4140020 con 0x7fc2c413a770 2022-07-21T01:55:45.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c413ad20 con 0x7fc2c4067610 2022-07-21T01:55:45.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c413ee50 con 0x7fc2c405c280 2022-07-21T01:55:45.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c8cc0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2c413a770 0x7fc2c4139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c8cc0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2c413a770 0x7fc2c4139ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38182/0 (socket says 172.21.15.140:38182) 2022-07-21T01:55:45.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c8cc0700 1 -- 172.21.15.140:0/3570503162 learned_addr learned my addr 172.21.15.140:0/3570503162 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:45.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c3fff700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2c405c280 0x7fc2c413d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c37fe700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2c4067610 0x7fc2c4139a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c8cc0700 1 -- 172.21.15.140:0/3570503162 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2c405c280 msgr2=0x7fc2c413d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c8cc0700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2c405c280 0x7fc2c413d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c8cc0700 1 -- 172.21.15.140:0/3570503162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2c4067610 msgr2=0x7fc2c4139a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c8cc0700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2c4067610 0x7fc2c4139a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c8cc0700 1 -- 172.21.15.140:0/3570503162 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b400b1e0 con 0x7fc2c413a770 2022-07-21T01:55:45.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c8cc0700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2c413a770 0x7fc2c4139ef0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fc2bc001250 tx=0x7fc2bc007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:45.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c17fa700 1 -- 172.21.15.140:0/3570503162 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2bc00e4a0 con 0x7fc2c413a770 2022-07-21T01:55:45.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c17fa700 1 -- 172.21.15.140:0/3570503162 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc00e600 con 0x7fc2c413a770 2022-07-21T01:55:45.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c94c1700 1 -- 172.21.15.140:0/3570503162 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c413a3e0 con 0x7fc2c413a770 2022-07-21T01:55:45.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.359+0000 7fc2c17fa700 1 -- 172.21.15.140:0/3570503162 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc2bc00e960 con 0x7fc2c413a770 2022-07-21T01:55:45.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.363+0000 7fc2c94c1700 1 -- 172.21.15.140:0/3570503162 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c413f0b0 con 0x7fc2c413a770 2022-07-21T01:55:45.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.363+0000 7fc2c94c1700 1 -- 172.21.15.140:0/3570503162 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c41412d0 con 0x7fc2c413a770 2022-07-21T01:55:45.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.363+0000 7fc2c17fa700 1 -- 172.21.15.140:0/3570503162 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc2bc010070 con 0x7fc2c413a770 2022-07-21T01:55:45.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.363+0000 7fc2c17fa700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc2a8053a70 0x7fc2a8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.363+0000 7fc2c17fa700 1 -- 172.21.15.140:0/3570503162 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc2bc06b3d0 con 0x7fc2c413a770 2022-07-21T01:55:45.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.363+0000 7fc2c3fff700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc2a8053a70 0x7fc2a8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.363+0000 7fc2c3fff700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc2a8053a70 0x7fc2a8055f30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fc2ac00b000 tx=0x7fc2ac006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:45.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.371+0000 7fc2c17fa700 1 -- 172.21.15.140:0/3570503162 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc2bc0421e0 con 0x7fc2c413a770 2022-07-21T01:55:45.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.535+0000 7fc2c94c1700 1 -- 172.21.15.140:0/3570503162 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc2c405d270 con 0x7fc2c413a770 2022-07-21T01:55:45.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.543+0000 7fc2c17fa700 1 -- 172.21.15.140:0/3570503162 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc2bc045800 con 0x7fc2c413a770 2022-07-21T01:55:45.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.543+0000 7fc2c94c1700 1 -- 172.21.15.140:0/3570503162 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc2a8053a70 msgr2=0x7fc2a8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.543+0000 7fc2c94c1700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc2a8053a70 0x7fc2a8055f30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fc2ac00b000 tx=0x7fc2ac006df0).stop 2022-07-21T01:55:45.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.543+0000 7fc2c94c1700 1 -- 172.21.15.140:0/3570503162 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2c413a770 msgr2=0x7fc2c4139ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.543+0000 7fc2c94c1700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2c413a770 0x7fc2c4139ef0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fc2bc001250 tx=0x7fc2bc007dc0).stop 2022-07-21T01:55:45.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.547+0000 7fc2c94c1700 1 -- 172.21.15.140:0/3570503162 shutdown_connections 2022-07-21T01:55:45.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.547+0000 7fc2c94c1700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc2a8053a70 0x7fc2a8055f30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.547+0000 7fc2c94c1700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2c413a770 0x7fc2c4139ef0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.547+0000 7fc2c94c1700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2c4067610 0x7fc2c4139a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.547+0000 7fc2c94c1700 1 --2- 172.21.15.140:0/3570503162 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc2c405c280 0x7fc2c413d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.547+0000 7fc2c94c1700 1 -- 172.21.15.140:0/3570503162 >> 172.21.15.140:0/3570503162 conn(0x7fc2c4064c00 msgr2=0x7fc2c4141e10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:45.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.547+0000 7fc2c94c1700 1 -- 172.21.15.140:0/3570503162 shutdown_connections 2022-07-21T01:55:45.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.547+0000 7fc2c94c1700 1 -- 172.21.15.140:0/3570503162 wait complete. 2022-07-21T01:55:45.551 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:55:45.561 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:55:45.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.663+0000 7f820df92700 1 Processor -- start 2022-07-21T01:55:45.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.663+0000 7f820df92700 1 -- start start 2022-07-21T01:55:45.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f82080b5630 0x7f82080b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f820813c7c0 0x7f820805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82080b0a10 con 0x7f820813cc50 2022-07-21T01:55:45.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82080b0cb0 con 0x7f82080b5630 2022-07-21T01:55:45.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82080b0f50 con 0x7f820813c7c0 2022-07-21T01:55:45.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820cf90700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f82080b5630 0x7f82080b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f8207fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f820813c7c0 0x7f820805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820cf90700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f82080b5630 0x7f82080b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38190/0 (socket says 172.21.15.140:38190) 2022-07-21T01:55:45.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820cf90700 1 -- 172.21.15.140:0/3105900790 learned_addr learned my addr 172.21.15.140:0/3105900790 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:45.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f8207fff700 1 -- 172.21.15.140:0/3105900790 >> v2:172.21.15.140:3300/0 conn(0x7f82080b5630 msgr2=0x7f82080b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:45.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f8207fff700 1 --2- 172.21.15.140:0/3105900790 >> v2:172.21.15.140:3300/0 conn(0x7f82080b5630 0x7f82080b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f8207fff700 1 -- 172.21.15.140:0/3105900790 >> v1:172.21.15.152:6789/0 conn(0x7f820813cc50 legacy=0x7f820805c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f8207fff700 1 -- 172.21.15.140:0/3105900790 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080afa10 con 0x7f820813c7c0 2022-07-21T01:55:45.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820cf90700 1 --2- 172.21.15.140:0/3105900790 >> v2:172.21.15.140:3300/0 conn(0x7f82080b5630 0x7f82080b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:45.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f8207fff700 1 --2- 172.21.15.140:0/3105900790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f820813c7c0 0x7f820805bd90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f81f0002f70 tx=0x7f81f0006df0).ready entity=mon.1 client_cookie=12f072f68455b3d5 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:45.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f82077fe700 1 -- 172.21.15.140:0/3105900790 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81f000d570 con 0x7f820813c7c0 2022-07-21T01:55:45.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f82077fe700 1 -- 172.21.15.140:0/3105900790 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f0000e90 con 0x7f820813c7c0 2022-07-21T01:55:45.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f82077fe700 1 -- 172.21.15.140:0/3105900790 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81f000d8c0 con 0x7f820813c7c0 2022-07-21T01:55:45.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 -- 172.21.15.140:0/3105900790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f820813c7c0 msgr2=0x7f820805bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 --2- 172.21.15.140:0/3105900790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f820813c7c0 0x7f820805bd90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f81f0002f70 tx=0x7f81f0006df0).stop 2022-07-21T01:55:45.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 -- 172.21.15.140:0/3105900790 shutdown_connections 2022-07-21T01:55:45.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 --2- 172.21.15.140:0/3105900790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f820813c7c0 0x7f820805bd90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 --2- 172.21.15.140:0/3105900790 >> v2:172.21.15.140:3300/0 conn(0x7f82080b5630 0x7f82080b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 -- 172.21.15.140:0/3105900790 >> 172.21.15.140:0/3105900790 conn(0x7f8208063490 msgr2=0x7f82080b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:45.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 -- 172.21.15.140:0/3105900790 shutdown_connections 2022-07-21T01:55:45.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.667+0000 7f820df92700 1 -- 172.21.15.140:0/3105900790 wait complete. 2022-07-21T01:55:45.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.671+0000 7f820df92700 1 Processor -- start 2022-07-21T01:55:45.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.671+0000 7f820df92700 1 -- start start 2022-07-21T01:55:45.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820df92700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82080b5630 0x7f82081458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820df92700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f820813c7c0 0x7f820813ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820df92700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f820813cc50 0x7f820813f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820df92700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82081421a0 con 0x7f820813c7c0 2022-07-21T01:55:45.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820df92700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8208144370 con 0x7f820813cc50 2022-07-21T01:55:45.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820df92700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8208144610 con 0x7f82080b5630 2022-07-21T01:55:45.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f8207fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f820813c7c0 0x7f820813ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820cf90700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82080b5630 0x7f82081458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f8207fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f820813c7c0 0x7f820813ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38198/0 (socket says 172.21.15.140:38198) 2022-07-21T01:55:45.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f8207fff700 1 -- 172.21.15.140:0/4009227997 learned_addr learned my addr 172.21.15.140:0/4009227997 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:45.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820d791700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f820813cc50 0x7f820813f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820cf90700 1 -- 172.21.15.140:0/4009227997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f820813cc50 msgr2=0x7f820813f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820cf90700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f820813cc50 0x7f820813f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820cf90700 1 -- 172.21.15.140:0/4009227997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f820813c7c0 msgr2=0x7f820813ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820cf90700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f820813c7c0 0x7f820813ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820cf90700 1 -- 172.21.15.140:0/4009227997 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f800b1e0 con 0x7f82080b5630 2022-07-21T01:55:45.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f8207fff700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f820813c7c0 0x7f820813ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:45.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820cf90700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82080b5630 0x7f82081458f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f81f800c000 tx=0x7f81f8006eb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:45.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f8205ffb700 1 -- 172.21.15.140:0/4009227997 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81f8012600 con 0x7f82080b5630 2022-07-21T01:55:45.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f8205ffb700 1 -- 172.21.15.140:0/4009227997 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f8012760 con 0x7f82080b5630 2022-07-21T01:55:45.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820df92700 1 -- 172.21.15.140:0/4009227997 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f000b180 con 0x7f82080b5630 2022-07-21T01:55:45.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f8205ffb700 1 -- 172.21.15.140:0/4009227997 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81f8012ac0 con 0x7f82080b5630 2022-07-21T01:55:45.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820df92700 1 -- 172.21.15.140:0/4009227997 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8208145190 con 0x7f82080b5630 2022-07-21T01:55:45.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f820df92700 1 -- 172.21.15.140:0/4009227997 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820813ed90 con 0x7f82080b5630 2022-07-21T01:55:45.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.675+0000 7f8205ffb700 1 -- 172.21.15.140:0/4009227997 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f81f8012c20 con 0x7f82080b5630 2022-07-21T01:55:45.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.679+0000 7f8205ffb700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f81ec053a70 0x7f81ec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:45.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.679+0000 7f8205ffb700 1 -- 172.21.15.140:0/4009227997 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f81f806bf90 con 0x7f82080b5630 2022-07-21T01:55:45.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.679+0000 7f8207fff700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f81ec053a70 0x7f81ec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:45.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.679+0000 7f8207fff700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f81ec053a70 0x7f81ec055f30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f81f001a000 tx=0x7f81f0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:45.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.687+0000 7f8205ffb700 1 -- 172.21.15.140:0/4009227997 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f81f8042e10 con 0x7f82080b5630 2022-07-21T01:55:45.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.855+0000 7f820df92700 1 -- 172.21.15.140:0/4009227997 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f82080574e0 con 0x7f82080b5630 2022-07-21T01:55:45.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.867+0000 7f8205ffb700 1 -- 172.21.15.140:0/4009227997 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v35) v1 ==== 184+0+0 (secure 0 0 0) 0x7f81f800f2a0 con 0x7f82080b5630 2022-07-21T01:55:45.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 -- 172.21.15.140:0/4009227997 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f81ec053a70 msgr2=0x7f81ec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f81ec053a70 0x7f81ec055f30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f81f001a000 tx=0x7f81f0014000).stop 2022-07-21T01:55:45.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 -- 172.21.15.140:0/4009227997 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82080b5630 msgr2=0x7f82081458f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:45.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82080b5630 0x7f82081458f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f81f800c000 tx=0x7f81f8006eb0).stop 2022-07-21T01:55:45.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 -- 172.21.15.140:0/4009227997 shutdown_connections 2022-07-21T01:55:45.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f81ec053a70 0x7f81ec055f30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f820813cc50 0x7f820813f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f820813c7c0 0x7f820813ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 --2- 172.21.15.140:0/4009227997 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82080b5630 0x7f82081458f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:45.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 -- 172.21.15.140:0/4009227997 >> 172.21.15.140:0/4009227997 conn(0x7f8208063490 msgr2=0x7f82080acae0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:45.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 -- 172.21.15.140:0/4009227997 shutdown_connections 2022-07-21T01:55:45.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.871+0000 7f820df92700 1 -- 172.21.15.140:0/4009227997 wait complete. 2022-07-21T01:55:45.875 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:55:45.885 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:46.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.987+0000 7fb4e67b6700 1 Processor -- start 2022-07-21T01:55:46.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.987+0000 7fb4e67b6700 1 -- start start 2022-07-21T01:55:46.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb4e00b5650 0x7fb4e00b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb4e00b1200 0x7fb4e005c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4e005bd90 0x7fb4e005c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e0142970 con 0x7fb4e00b1200 2022-07-21T01:55:46.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e00aea30 con 0x7fb4e00b5650 2022-07-21T01:55:46.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e00aec60 con 0x7fb4e005bd90 2022-07-21T01:55:46.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e4fb3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb4e00b1200 0x7fb4e005c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e4fb3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb4e00b1200 0x7fb4e005c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38202/0 (socket says 172.21.15.140:38202) 2022-07-21T01:55:46.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e4fb3700 1 -- 172.21.15.140:0/3801102701 learned_addr learned my addr 172.21.15.140:0/3801102701 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:46.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e5fb5700 1 --2- 172.21.15.140:0/3801102701 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4e005bd90 0x7fb4e005c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e57b4700 1 --2- 172.21.15.140:0/3801102701 >> v2:172.21.15.152:3300/0 conn(0x7fb4e00b5650 0x7fb4e00b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e4fb3700 1 -- 172.21.15.140:0/3801102701 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4e005bd90 msgr2=0x7fb4e005c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:46.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e4fb3700 1 --2- 172.21.15.140:0/3801102701 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4e005bd90 0x7fb4e005c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e4fb3700 1 -- 172.21.15.140:0/3801102701 >> v2:172.21.15.152:3300/0 conn(0x7fb4e00b5650 msgr2=0x7fb4e00b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:46.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e4fb3700 1 --2- 172.21.15.140:0/3801102701 >> v2:172.21.15.152:3300/0 conn(0x7fb4e00b5650 0x7fb4e00b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e4fb3700 1 -- 172.21.15.140:0/3801102701 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e00af180 con 0x7fb4e00b1200 2022-07-21T01:55:46.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e4fb3700 1 --2- 172.21.15.140:0/3801102701 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4e00b1200 0x7fb4e005c4f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fb4d0002f70 tx=0x7fb4d0006e50).ready entity=mon.0 client_cookie=a5c29cea80901281 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:46.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4cffff700 1 -- 172.21.15.140:0/3801102701 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4d000d570 con 0x7fb4e00b1200 2022-07-21T01:55:46.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4cffff700 1 -- 172.21.15.140:0/3801102701 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d0000e90 con 0x7fb4e00b1200 2022-07-21T01:55:46.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4cffff700 1 -- 172.21.15.140:0/3801102701 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4d000d900 con 0x7fb4e00b1200 2022-07-21T01:55:46.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 -- 172.21.15.140:0/3801102701 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4e00b1200 msgr2=0x7fb4e005c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 --2- 172.21.15.140:0/3801102701 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4e00b1200 0x7fb4e005c4f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fb4d0002f70 tx=0x7fb4d0006e50).stop 2022-07-21T01:55:46.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 -- 172.21.15.140:0/3801102701 shutdown_connections 2022-07-21T01:55:46.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 --2- 172.21.15.140:0/3801102701 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4e005bd90 0x7fb4e005c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 --2- 172.21.15.140:0/3801102701 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4e00b1200 0x7fb4e005c4f0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 --2- 172.21.15.140:0/3801102701 >> v2:172.21.15.152:3300/0 conn(0x7fb4e00b5650 0x7fb4e00b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 -- 172.21.15.140:0/3801102701 >> 172.21.15.140:0/3801102701 conn(0x7fb4e00634e0 msgr2=0x7fb4e00b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:46.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 -- 172.21.15.140:0/3801102701 shutdown_connections 2022-07-21T01:55:46.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.991+0000 7fb4e67b6700 1 -- 172.21.15.140:0/3801102701 wait complete. 2022-07-21T01:55:46.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e67b6700 1 Processor -- start 2022-07-21T01:55:46.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e67b6700 1 -- start start 2022-07-21T01:55:46.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e67b6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4e005bd90 0x7fb4e0067510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e67b6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4e00b1200 0x7fb4e00b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e67b6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4e00b5650 0x7fb4e0063d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e67b6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e013ead0 con 0x7fb4e005bd90 2022-07-21T01:55:46.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e67b6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e013ecf0 con 0x7fb4e00b5650 2022-07-21T01:55:46.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e67b6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e013ced0 con 0x7fb4e00b1200 2022-07-21T01:55:46.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e57b4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4e005bd90 0x7fb4e0067510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e57b4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4e005bd90 0x7fb4e0067510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38208/0 (socket says 172.21.15.140:38208) 2022-07-21T01:55:46.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e57b4700 1 -- 172.21.15.140:0/135494974 learned_addr learned my addr 172.21.15.140:0/135494974 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:46.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e4fb3700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4e00b1200 0x7fb4e00b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e5fb5700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4e00b5650 0x7fb4e0063d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e57b4700 1 -- 172.21.15.140:0/135494974 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4e00b1200 msgr2=0x7fb4e00b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e57b4700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4e00b1200 0x7fb4e00b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e57b4700 1 -- 172.21.15.140:0/135494974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4e00b5650 msgr2=0x7fb4e0063d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e57b4700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4e00b5650 0x7fb4e0063d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e57b4700 1 -- 172.21.15.140:0/135494974 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d000b1e0 con 0x7fb4e005bd90 2022-07-21T01:55:46.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e57b4700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4e005bd90 0x7fb4e0067510 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fb4dc0030b0 tx=0x7fb4dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:46.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4ce7fc700 1 -- 172.21.15.140:0/135494974 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4dc011780 con 0x7fb4e005bd90 2022-07-21T01:55:46.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4ce7fc700 1 -- 172.21.15.140:0/135494974 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc0118e0 con 0x7fb4e005bd90 2022-07-21T01:55:46.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e67b6700 1 -- 172.21.15.140:0/135494974 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e00b3970 con 0x7fb4e005bd90 2022-07-21T01:55:46.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4ce7fc700 1 -- 172.21.15.140:0/135494974 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4dc011c40 con 0x7fb4e005bd90 2022-07-21T01:55:46.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.995+0000 7fb4e67b6700 1 -- 172.21.15.140:0/135494974 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e013ded0 con 0x7fb4e005bd90 2022-07-21T01:55:46.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.999+0000 7fb4e67b6700 1 -- 172.21.15.140:0/135494974 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e013d150 con 0x7fb4e005bd90 2022-07-21T01:55:46.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.999+0000 7fb4ce7fc700 1 -- 172.21.15.140:0/135494974 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb4dc012420 con 0x7fb4e005bd90 2022-07-21T01:55:46.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.999+0000 7fb4ce7fc700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb4c0053aa0 0x7fb4c0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.999+0000 7fb4ce7fc700 1 -- 172.21.15.140:0/135494974 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb4dc06bc80 con 0x7fb4e005bd90 2022-07-21T01:55:46.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.999+0000 7fb4e4fb3700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb4c0053aa0 0x7fb4c0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:45.999+0000 7fb4e4fb3700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb4c0053aa0 0x7fb4c0055f60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fb4d001c000 tx=0x7fb4d0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:46.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.003+0000 7fb4ce7fc700 1 -- 172.21.15.140:0/135494974 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb4dc042a30 con 0x7fb4e005bd90 2022-07-21T01:55:46.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 -- 172.21.15.140:0/135494974 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4e00b3b60 con 0x7fb4e005bd90 2022-07-21T01:55:46.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4ce7fc700 1 -- 172.21.15.140:0/135494974 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2143 (secure 0 0 0) 0x7fb4dc046050 con 0x7fb4e005bd90 2022-07-21T01:55:46.169 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:46.169 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/518920349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":518920349},{"type":"v1","addr":"172.21.15.152:6833","nonce":518920349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:43.343194+0000","modified":"2022-07-21T01:55:44.346904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2478727503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2478727503},{"type":"v1","addr":"172.21.15.140:6835","nonce":2478727503}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-07-21T01:55:46.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 -- 172.21.15.140:0/135494974 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb4c0053aa0 msgr2=0x7fb4c0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb4c0053aa0 0x7fb4c0055f60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fb4d001c000 tx=0x7fb4d0015000).stop 2022-07-21T01:55:46.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 -- 172.21.15.140:0/135494974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4e005bd90 msgr2=0x7fb4e0067510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4e005bd90 0x7fb4e0067510 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fb4dc0030b0 tx=0x7fb4dc008040).stop 2022-07-21T01:55:46.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 -- 172.21.15.140:0/135494974 shutdown_connections 2022-07-21T01:55:46.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb4c0053aa0 0x7fb4c0055f60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4e00b5650 0x7fb4e0063d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4e00b1200 0x7fb4e00b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 --2- 172.21.15.140:0/135494974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb4e005bd90 0x7fb4e0067510 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 -- 172.21.15.140:0/135494974 >> 172.21.15.140:0/135494974 conn(0x7fb4e00634e0 msgr2=0x7fb4e0143e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:46.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 -- 172.21.15.140:0/135494974 shutdown_connections 2022-07-21T01:55:46.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.167+0000 7fb4e67b6700 1 -- 172.21.15.140:0/135494974 wait complete. 2022-07-21T01:55:46.174 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 42 2022-07-21T01:55:46.383 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:55:43.343194+0000', 'modified': '2022-07-21T01:55:44.346904+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7002}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7002': {'gid': 7002, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/2478727503', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 2478727503}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 2478727503}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:55:46.384 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:55:46.385 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:55:46.503 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":7002,"want_state":"up:active","state":"up:active","rank_uptime":3.1495028779999998,"mdsmap_epoch":42,"osdmap_epoch":94,"osdmap_epoch_barrier":93,"uptime":7.3216173920000003} 2022-07-21T01:55:46.504 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 7002, "mdsmap_epoch": 42, "osdmap_epoch": 94, "osdmap_epoch_barrier": 93, "rank_uptime": 3.149502878, "state": "up:active", "uptime": 7.321617392, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:55:46.505 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:46.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e44c59700 1 Processor -- start 2022-07-21T01:55:46.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e44c59700 1 -- start start 2022-07-21T01:55:46.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e44c59700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e400b5650 0x7f3e400b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e44c59700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3e400adff0 0x7f3e40147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e44c59700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3e400af8e0 0x7f3e4005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e44c59700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e40147010 con 0x7f3e400af8e0 2022-07-21T01:55:46.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e44c59700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e401472b0 con 0x7f3e400adff0 2022-07-21T01:55:46.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e44c59700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e40146d40 con 0x7f3e400b5650 2022-07-21T01:55:46.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e3ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3e400af8e0 0x7f3e4005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e3ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3e400af8e0 0x7f3e4005bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38218/0 (socket says 172.21.15.140:38218) 2022-07-21T01:55:46.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e3ffff700 1 -- 172.21.15.140:0/3514650829 learned_addr learned my addr 172.21.15.140:0/3514650829 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:46.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e3f7fe700 1 --2- 172.21.15.140:0/3514650829 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e400b5650 0x7f3e400b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.607+0000 7f3e3effd700 1 --2- 172.21.15.140:0/3514650829 >> v2:172.21.15.152:3300/0 conn(0x7f3e400adff0 0x7f3e40147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e3ffff700 1 -- 172.21.15.140:0/3514650829 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e400b5650 msgr2=0x7f3e400b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:46.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e3ffff700 1 --2- 172.21.15.140:0/3514650829 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e400b5650 0x7f3e400b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e3ffff700 1 -- 172.21.15.140:0/3514650829 >> v2:172.21.15.152:3300/0 conn(0x7f3e400adff0 msgr2=0x7f3e40147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:46.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e3ffff700 1 --2- 172.21.15.140:0/3514650829 >> v2:172.21.15.152:3300/0 conn(0x7f3e400adff0 0x7f3e40147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e3ffff700 1 -- 172.21.15.140:0/3514650829 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40147c40 con 0x7f3e400af8e0 2022-07-21T01:55:46.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e3effd700 1 --2- 172.21.15.140:0/3514650829 >> v2:172.21.15.152:3300/0 conn(0x7f3e400adff0 0x7f3e40147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:46.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e3f7fe700 1 --2- 172.21.15.140:0/3514650829 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e400b5650 0x7f3e400b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:46.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e3ffff700 1 --2- 172.21.15.140:0/3514650829 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e400af8e0 0x7f3e4005bd90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f3e380032f0 tx=0x7f3e38009290).ready entity=mon.0 client_cookie=5da2a975cdbdc4c1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:46.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e3e7fc700 1 -- 172.21.15.140:0/3514650829 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e3800b680 con 0x7f3e400af8e0 2022-07-21T01:55:46.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e3e7fc700 1 -- 172.21.15.140:0/3514650829 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3800b7e0 con 0x7f3e400af8e0 2022-07-21T01:55:46.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e3e7fc700 1 -- 172.21.15.140:0/3514650829 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e3800b950 con 0x7f3e400af8e0 2022-07-21T01:55:46.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 -- 172.21.15.140:0/3514650829 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e400af8e0 msgr2=0x7f3e4005bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 --2- 172.21.15.140:0/3514650829 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e400af8e0 0x7f3e4005bd90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f3e380032f0 tx=0x7f3e38009290).stop 2022-07-21T01:55:46.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 -- 172.21.15.140:0/3514650829 shutdown_connections 2022-07-21T01:55:46.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 --2- 172.21.15.140:0/3514650829 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e400af8e0 0x7f3e4005bd90 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 --2- 172.21.15.140:0/3514650829 >> v2:172.21.15.152:3300/0 conn(0x7f3e400adff0 0x7f3e40147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 --2- 172.21.15.140:0/3514650829 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e400b5650 0x7f3e400b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 -- 172.21.15.140:0/3514650829 >> 172.21.15.140:0/3514650829 conn(0x7f3e400634e0 msgr2=0x7f3e400b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:46.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 -- 172.21.15.140:0/3514650829 shutdown_connections 2022-07-21T01:55:46.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 -- 172.21.15.140:0/3514650829 wait complete. 2022-07-21T01:55:46.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 Processor -- start 2022-07-21T01:55:46.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 -- start start 2022-07-21T01:55:46.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e400adff0 0x7f3e40065090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e400af8e0 0x7f3e401499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e400b5650 0x7f3e40144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e40111910 con 0x7f3e400af8e0 2022-07-21T01:55:46.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400646b0 con 0x7f3e400adff0 2022-07-21T01:55:46.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.611+0000 7f3e44c59700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e40064950 con 0x7f3e400b5650 2022-07-21T01:55:46.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e400af8e0 0x7f3e401499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e400af8e0 0x7f3e401499a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38224/0 (socket says 172.21.15.140:38224) 2022-07-21T01:55:46.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3effd700 1 -- 172.21.15.140:0/3255485788 learned_addr learned my addr 172.21.15.140:0/3255485788 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:46.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3ffff700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e400b5650 0x7f3e40144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3f7fe700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e400adff0 0x7f3e40065090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3f7fe700 1 -- 172.21.15.140:0/3255485788 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e400b5650 msgr2=0x7f3e40144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3f7fe700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e400b5650 0x7f3e40144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3f7fe700 1 -- 172.21.15.140:0/3255485788 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e400af8e0 msgr2=0x7f3e401499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3f7fe700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e400af8e0 0x7f3e401499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3f7fe700 1 -- 172.21.15.140:0/3255485788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2800b120 con 0x7f3e400adff0 2022-07-21T01:55:46.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3f7fe700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e400adff0 0x7f3e40065090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f3e3000c000 tx=0x7f3e30006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:46.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3cff9700 1 -- 172.21.15.140:0/3255485788 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e30010780 con 0x7f3e400adff0 2022-07-21T01:55:46.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e44c59700 1 -- 172.21.15.140:0/3255485788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3000b180 con 0x7f3e400adff0 2022-07-21T01:55:46.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3cff9700 1 -- 172.21.15.140:0/3255485788 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e300108e0 con 0x7f3e400adff0 2022-07-21T01:55:46.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3cff9700 1 -- 172.21.15.140:0/3255485788 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e30010c40 con 0x7f3e400adff0 2022-07-21T01:55:46.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e44c59700 1 -- 172.21.15.140:0/3255485788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e38009530 con 0x7f3e400adff0 2022-07-21T01:55:46.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e44c59700 1 -- 172.21.15.140:0/3255485788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e40064df0 con 0x7f3e400adff0 2022-07-21T01:55:46.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.615+0000 7f3e3cff9700 1 -- 172.21.15.140:0/3255485788 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3e3001c460 con 0x7f3e400adff0 2022-07-21T01:55:46.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.619+0000 7f3e3cff9700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e180539e0 0x7f3e18055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.619+0000 7f3e3cff9700 1 -- 172.21.15.140:0/3255485788 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3e3006c370 con 0x7f3e400adff0 2022-07-21T01:55:46.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.619+0000 7f3e3effd700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e180539e0 0x7f3e18055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.619+0000 7f3e3effd700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e180539e0 0x7f3e18055ea0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f3e28008000 tx=0x7f3e28008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:46.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.627+0000 7f3e3cff9700 1 -- 172.21.15.140:0/3255485788 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3e40064df0 con 0x7f3e400adff0 2022-07-21T01:55:46.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.755+0000 7f3e44c59700 1 -- 172.21.15.140:0/3255485788 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e40064df0 con 0x7f3e400adff0 2022-07-21T01:55:46.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.755+0000 7f3e3cff9700 1 -- 172.21.15.140:0/3255485788 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16137 (secure 0 0 0) 0x7f3e40064df0 con 0x7f3e400adff0 2022-07-21T01:55:46.759 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:46.760 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":94,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:44.346577+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:55:41.913815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-07-21T01:55:42.343580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:46.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 -- 172.21.15.140:0/3255485788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e180539e0 msgr2=0x7f3e18055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e180539e0 0x7f3e18055ea0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f3e28008000 tx=0x7f3e28008040).stop 2022-07-21T01:55:46.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 -- 172.21.15.140:0/3255485788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e400adff0 msgr2=0x7f3e40065090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e400adff0 0x7f3e40065090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f3e3000c000 tx=0x7f3e30006df0).stop 2022-07-21T01:55:46.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 -- 172.21.15.140:0/3255485788 shutdown_connections 2022-07-21T01:55:46.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e180539e0 0x7f3e18055ea0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e400b5650 0x7f3e40144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e400af8e0 0x7f3e401499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 --2- 172.21.15.140:0/3255485788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e400adff0 0x7f3e40065090 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 -- 172.21.15.140:0/3255485788 >> 172.21.15.140:0/3255485788 conn(0x7f3e400634e0 msgr2=0x7f3e4010e130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:46.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 -- 172.21.15.140:0/3255485788 shutdown_connections 2022-07-21T01:55:46.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.759+0000 7f3e44c59700 1 -- 172.21.15.140:0/3255485788 wait complete. 2022-07-21T01:55:46.772 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:55:41.913815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:55:42.343580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:55:46.773 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:55:46.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.879+0000 7fd860b8e700 1 Processor -- start 2022-07-21T01:55:46.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.879+0000 7fd860b8e700 1 -- start start 2022-07-21T01:55:46.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd85c0b3330 0x7fd85c05cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd85c0b3710 0x7fd85c09aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c144e70 con 0x7fd85c144a20 2022-07-21T01:55:46.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c144040 con 0x7fd85c0b3330 2022-07-21T01:55:46.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c1442e0 con 0x7fd85c0b3710 2022-07-21T01:55:46.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85a59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd85c0b3330 0x7fd85c05cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85a59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd85c0b3330 0x7fd85c05cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38230/0 (socket says 172.21.15.140:38230) 2022-07-21T01:55:46.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85a59c700 1 -- 172.21.15.140:0/683900515 learned_addr learned my addr 172.21.15.140:0/683900515 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:46.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd859d9b700 1 --2- 172.21.15.140:0/683900515 >> v2:172.21.15.152:3300/0 conn(0x7fd85c0b3710 0x7fd85c09aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85959a700 1 -- 172.21.15.140:0/683900515 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202948384 0 0) 0x7fd85c144e70 con 0x7fd85c144a20 2022-07-21T01:55:46.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85959a700 1 -- 172.21.15.140:0/683900515 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd840003260 con 0x7fd85c144a20 2022-07-21T01:55:46.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85a59c700 1 -- 172.21.15.140:0/683900515 >> v2:172.21.15.152:3300/0 conn(0x7fd85c0b3710 msgr2=0x7fd85c09aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:46.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85a59c700 1 --2- 172.21.15.140:0/683900515 >> v2:172.21.15.152:3300/0 conn(0x7fd85c0b3710 0x7fd85c09aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85a59c700 1 -- 172.21.15.140:0/683900515 >> v1:172.21.15.152:6789/0 conn(0x7fd85c144a20 legacy=0x7fd85c0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85a59c700 1 -- 172.21.15.140:0/683900515 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c144580 con 0x7fd85c0b3330 2022-07-21T01:55:46.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd859d9b700 1 --2- 172.21.15.140:0/683900515 >> v2:172.21.15.152:3300/0 conn(0x7fd85c0b3710 0x7fd85c09aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:46.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85a59c700 1 --2- 172.21.15.140:0/683900515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85c0b3330 0x7fd85c05cbb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7fd84c002f70 tx=0x7fd84c006eb0).ready entity=mon.0 client_cookie=bcb87a83f407c4ed server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:46.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85959a700 1 -- 172.21.15.140:0/683900515 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd84c00d610 con 0x7fd85c0b3330 2022-07-21T01:55:46.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85959a700 1 -- 172.21.15.140:0/683900515 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c000e90 con 0x7fd85c0b3330 2022-07-21T01:55:46.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd85959a700 1 -- 172.21.15.140:0/683900515 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd84c00d9a0 con 0x7fd85c0b3330 2022-07-21T01:55:46.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 -- 172.21.15.140:0/683900515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85c0b3330 msgr2=0x7fd85c05cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 --2- 172.21.15.140:0/683900515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85c0b3330 0x7fd85c05cbb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7fd84c002f70 tx=0x7fd84c006eb0).stop 2022-07-21T01:55:46.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 -- 172.21.15.140:0/683900515 shutdown_connections 2022-07-21T01:55:46.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 --2- 172.21.15.140:0/683900515 >> v2:172.21.15.152:3300/0 conn(0x7fd85c0b3710 0x7fd85c09aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 --2- 172.21.15.140:0/683900515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85c0b3330 0x7fd85c05cbb0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 -- 172.21.15.140:0/683900515 >> 172.21.15.140:0/683900515 conn(0x7fd85c0b6c50 msgr2=0x7fd85c0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:46.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 -- 172.21.15.140:0/683900515 shutdown_connections 2022-07-21T01:55:46.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.883+0000 7fd860b8e700 1 -- 172.21.15.140:0/683900515 wait complete. 2022-07-21T01:55:46.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd860b8e700 1 Processor -- start 2022-07-21T01:55:46.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd860b8e700 1 -- start start 2022-07-21T01:55:46.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd860b8e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85c0b3710 0x7fd85c0ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd860b8e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd85c144a20 0x7fd85c0ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd860b8e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd85c05f840 0x7fd85c0aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd860b8e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0609c0 con 0x7fd85c0b3710 2022-07-21T01:55:46.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd860b8e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0602f0 con 0x7fd85c144a20 2022-07-21T01:55:46.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd860b8e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c05fd80 con 0x7fd85c05f840 2022-07-21T01:55:46.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd85a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85c0b3710 0x7fd85c0ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd859d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd85c144a20 0x7fd85c0ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd85ad9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd85c05f840 0x7fd85c0aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd85ad9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd85c05f840 0x7fd85c0aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:34968/0 (socket says 172.21.15.140:34968) 2022-07-21T01:55:46.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd85ad9d700 1 -- 172.21.15.140:0/2642089583 learned_addr learned my addr 172.21.15.140:0/2642089583 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:46.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd85a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85c0b3710 0x7fd85c0ab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38240/0 (socket says 172.21.15.140:38240) 2022-07-21T01:55:46.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd85ad9d700 1 -- 172.21.15.140:0/2642089583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd85c144a20 msgr2=0x7fd85c0ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd85ad9d700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd85c144a20 0x7fd85c0ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd85ad9d700 1 -- 172.21.15.140:0/2642089583 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85c0b3710 msgr2=0x7fd85c0ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:46.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd85ad9d700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85c0b3710 0x7fd85c0ab9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:46.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd85ad9d700 1 -- 172.21.15.140:0/2642089583 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85000b180 con 0x7fd85c05f840 2022-07-21T01:55:46.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd85ad9d700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd85c05f840 0x7fd85c0aec70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fd85c060b00 tx=0x7fd854007dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:46.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd84b7fe700 1 -- 172.21.15.140:0/2642089583 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd854002310 con 0x7fd85c05f840 2022-07-21T01:55:46.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd84b7fe700 1 -- 172.21.15.140:0/2642089583 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd854002470 con 0x7fd85c05f840 2022-07-21T01:55:46.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd860b8e700 1 -- 172.21.15.140:0/2642089583 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84c00b1e0 con 0x7fd85c05f840 2022-07-21T01:55:46.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd84b7fe700 1 -- 172.21.15.140:0/2642089583 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8540017a0 con 0x7fd85c05f840 2022-07-21T01:55:46.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.887+0000 7fd860b8e700 1 -- 172.21.15.140:0/2642089583 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c097de0 con 0x7fd85c05f840 2022-07-21T01:55:46.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.891+0000 7fd860b8e700 1 -- 172.21.15.140:0/2642089583 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c0618f0 con 0x7fd85c05f840 2022-07-21T01:55:46.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.891+0000 7fd84b7fe700 1 -- 172.21.15.140:0/2642089583 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd854010030 con 0x7fd85c05f840 2022-07-21T01:55:46.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.891+0000 7fd84b7fe700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd838053980 0x7fd838055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:46.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.891+0000 7fd84b7fe700 1 -- 172.21.15.140:0/2642089583 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd85406be70 con 0x7fd85c05f840 2022-07-21T01:55:46.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.891+0000 7fd85a59c700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd838053980 0x7fd838055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:46.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.891+0000 7fd85a59c700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd838053980 0x7fd838055e40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fd84c01c000 tx=0x7fd84c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:46.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:46.899+0000 7fd84b7fe700 1 -- 172.21.15.140:0/2642089583 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd854042c90 con 0x7fd85c05f840 2022-07-21T01:55:47.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.027+0000 7fd860b8e700 1 -- 172.21.15.140:0/2642089583 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd85c0574e0 con 0x7fd85c05f840 2022-07-21T01:55:47.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.027+0000 7fd84b7fe700 1 -- 172.21.15.140:0/2642089583 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v94) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd8540462b0 con 0x7fd85c05f840 2022-07-21T01:55:47.030 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:55:47.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 -- 172.21.15.140:0/2642089583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd838053980 msgr2=0x7fd838055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd838053980 0x7fd838055e40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fd84c01c000 tx=0x7fd84c015000).stop 2022-07-21T01:55:47.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 -- 172.21.15.140:0/2642089583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd85c05f840 msgr2=0x7fd85c0aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd85c05f840 0x7fd85c0aec70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fd85c060b00 tx=0x7fd854007dd0).stop 2022-07-21T01:55:47.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 -- 172.21.15.140:0/2642089583 shutdown_connections 2022-07-21T01:55:47.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd838053980 0x7fd838055e40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd85c05f840 0x7fd85c0aec70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd85c144a20 0x7fd85c0ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 --2- 172.21.15.140:0/2642089583 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd85c0b3710 0x7fd85c0ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 -- 172.21.15.140:0/2642089583 >> 172.21.15.140:0/2642089583 conn(0x7fd85c0b6c50 msgr2=0x7fd85c140660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:47.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 -- 172.21.15.140:0/2642089583 shutdown_connections 2022-07-21T01:55:47.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.031+0000 7fd860b8e700 1 -- 172.21.15.140:0/2642089583 wait complete. 2022-07-21T01:55:47.042 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:55:47.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.147+0000 7f1ff9dea700 1 Processor -- start 2022-07-21T01:55:47.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.147+0000 7f1ff9dea700 1 -- start start 2022-07-21T01:55:47.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1ff40adfa0 0x7f1ff405d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1ff4143ea0 0x7f1ff4144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff40b4230 con 0x7f1ff405bf30 2022-07-21T01:55:47.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff40677b0 con 0x7f1ff4143ea0 2022-07-21T01:55:47.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff40b2950 con 0x7f1ff40adfa0 2022-07-21T01:55:47.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff95e9700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1ff4143ea0 0x7f1ff4144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff95e9700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1ff4143ea0 0x7f1ff4144330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38248/0 (socket says 172.21.15.140:38248) 2022-07-21T01:55:47.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff95e9700 1 -- 172.21.15.140:0/1931776381 learned_addr learned my addr 172.21.15.140:0/1931776381 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:47.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff3fff700 1 --2- 172.21.15.140:0/1931776381 >> v2:172.21.15.152:3300/0 conn(0x7f1ff40adfa0 0x7f1ff405d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff95e9700 1 -- 172.21.15.140:0/1931776381 >> v2:172.21.15.152:3300/0 conn(0x7f1ff40adfa0 msgr2=0x7f1ff405d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:47.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff95e9700 1 --2- 172.21.15.140:0/1931776381 >> v2:172.21.15.152:3300/0 conn(0x7f1ff40adfa0 0x7f1ff405d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff95e9700 1 -- 172.21.15.140:0/1931776381 >> v1:172.21.15.152:6789/0 conn(0x7f1ff405bf30 legacy=0x7f1ff413a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff95e9700 1 -- 172.21.15.140:0/1931776381 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff40b3850 con 0x7f1ff4143ea0 2022-07-21T01:55:47.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff3fff700 1 --2- 172.21.15.140:0/1931776381 >> v2:172.21.15.152:3300/0 conn(0x7f1ff40adfa0 0x7f1ff405d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:47.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff95e9700 1 --2- 172.21.15.140:0/1931776381 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff4143ea0 0x7f1ff4144330 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f1fec0032f0 tx=0x7f1fec009290).ready entity=mon.0 client_cookie=96be177f58fb6f86 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:47.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff37fe700 1 -- 172.21.15.140:0/1931776381 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fec00b870 con 0x7f1ff4143ea0 2022-07-21T01:55:47.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff37fe700 1 -- 172.21.15.140:0/1931776381 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fec00b9d0 con 0x7f1ff4143ea0 2022-07-21T01:55:47.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff37fe700 1 -- 172.21.15.140:0/1931776381 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fec00bb40 con 0x7f1ff4143ea0 2022-07-21T01:55:47.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 -- 172.21.15.140:0/1931776381 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff4143ea0 msgr2=0x7f1ff4144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 --2- 172.21.15.140:0/1931776381 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff4143ea0 0x7f1ff4144330 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f1fec0032f0 tx=0x7f1fec009290).stop 2022-07-21T01:55:47.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 -- 172.21.15.140:0/1931776381 shutdown_connections 2022-07-21T01:55:47.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 --2- 172.21.15.140:0/1931776381 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff4143ea0 0x7f1ff4144330 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 --2- 172.21.15.140:0/1931776381 >> v2:172.21.15.152:3300/0 conn(0x7f1ff40adfa0 0x7f1ff405d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 -- 172.21.15.140:0/1931776381 >> 172.21.15.140:0/1931776381 conn(0x7f1ff4064b90 msgr2=0x7f1ff4064f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:47.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 -- 172.21.15.140:0/1931776381 shutdown_connections 2022-07-21T01:55:47.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.151+0000 7f1ff9dea700 1 -- 172.21.15.140:0/1931776381 wait complete. 2022-07-21T01:55:47.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff9dea700 1 Processor -- start 2022-07-21T01:55:47.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff9dea700 1 -- start start 2022-07-21T01:55:47.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff9dea700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff405bf30 0x7f1ff40b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff9dea700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff40adfa0 0x7f1ff4153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff9dea700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ff4143ea0 0x7f1ff4153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff9dea700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff413f640 con 0x7f1ff40adfa0 2022-07-21T01:55:47.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff9dea700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff413f3a0 con 0x7f1ff4143ea0 2022-07-21T01:55:47.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff9dea700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4143850 con 0x7f1ff405bf30 2022-07-21T01:55:47.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff40adfa0 0x7f1ff4153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff40adfa0 0x7f1ff4153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38250/0 (socket says 172.21.15.140:38250) 2022-07-21T01:55:47.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff3fff700 1 -- 172.21.15.140:0/2306563667 learned_addr learned my addr 172.21.15.140:0/2306563667 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:47.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff8de8700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff405bf30 0x7f1ff40b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff95e9700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ff4143ea0 0x7f1ff4153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff3fff700 1 -- 172.21.15.140:0/2306563667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff405bf30 msgr2=0x7f1ff40b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff3fff700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff405bf30 0x7f1ff40b6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff3fff700 1 -- 172.21.15.140:0/2306563667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ff4143ea0 msgr2=0x7f1ff4153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff3fff700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ff4143ea0 0x7f1ff4153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff3fff700 1 -- 172.21.15.140:0/2306563667 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec009530 con 0x7f1ff40adfa0 2022-07-21T01:55:47.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff3fff700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff40adfa0 0x7f1ff4153710 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f1fe4006940 tx=0x7f1fe4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:47.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff1ffb700 1 -- 172.21.15.140:0/2306563667 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fe4011780 con 0x7f1ff40adfa0 2022-07-21T01:55:47.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff1ffb700 1 -- 172.21.15.140:0/2306563667 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe40118e0 con 0x7f1ff40adfa0 2022-07-21T01:55:47.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff9dea700 1 -- 172.21.15.140:0/2306563667 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff4140360 con 0x7f1ff40adfa0 2022-07-21T01:55:47.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff1ffb700 1 -- 172.21.15.140:0/2306563667 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fe4011c40 con 0x7f1ff40adfa0 2022-07-21T01:55:47.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff9dea700 1 -- 172.21.15.140:0/2306563667 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff413e600 con 0x7f1ff40adfa0 2022-07-21T01:55:47.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff9dea700 1 -- 172.21.15.140:0/2306563667 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff40b3af0 con 0x7f1ff40adfa0 2022-07-21T01:55:47.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.155+0000 7f1ff1ffb700 1 -- 172.21.15.140:0/2306563667 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1fe401c460 con 0x7f1ff40adfa0 2022-07-21T01:55:47.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.159+0000 7f1ff1ffb700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053a10 0x7f1fd8055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.159+0000 7f1ff1ffb700 1 -- 172.21.15.140:0/2306563667 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1fe406c230 con 0x7f1ff40adfa0 2022-07-21T01:55:47.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.159+0000 7f1ff8de8700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053a10 0x7f1fd8055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.159+0000 7f1ff8de8700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053a10 0x7f1fd8055ed0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f1fdc00b000 tx=0x7f1fdc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:47.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.163+0000 7f1ff1ffb700 1 -- 172.21.15.140:0/2306563667 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1fe4043050 con 0x7f1ff40adfa0 2022-07-21T01:55:47.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.295+0000 7f1ff9dea700 1 -- 172.21.15.140:0/2306563667 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1ff40574e0 con 0x7f1ff40adfa0 2022-07-21T01:55:47.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.295+0000 7f1ff1ffb700 1 -- 172.21.15.140:0/2306563667 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v94) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1fe4046670 con 0x7f1ff40adfa0 2022-07-21T01:55:47.298 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:47.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 -- 172.21.15.140:0/2306563667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053a10 msgr2=0x7f1fd8055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053a10 0x7f1fd8055ed0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f1fdc00b000 tx=0x7f1fdc009040).stop 2022-07-21T01:55:47.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 -- 172.21.15.140:0/2306563667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff40adfa0 msgr2=0x7f1ff4153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff40adfa0 0x7f1ff4153710 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f1fe4006940 tx=0x7f1fe4008040).stop 2022-07-21T01:55:47.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 -- 172.21.15.140:0/2306563667 shutdown_connections 2022-07-21T01:55:47.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053a10 0x7f1fd8055ed0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ff4143ea0 0x7f1ff4153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff40adfa0 0x7f1ff4153710 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 --2- 172.21.15.140:0/2306563667 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff405bf30 0x7f1ff40b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 -- 172.21.15.140:0/2306563667 >> 172.21.15.140:0/2306563667 conn(0x7f1ff4064b90 msgr2=0x7f1ff413d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:47.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 -- 172.21.15.140:0/2306563667 shutdown_connections 2022-07-21T01:55:47.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.299+0000 7f1ff9dea700 1 -- 172.21.15.140:0/2306563667 wait complete. 2022-07-21T01:55:47.316 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:55:47.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 Processor -- start 2022-07-21T01:55:47.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 -- start start 2022-07-21T01:55:47.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbf9005cf40 0x7fbf901387d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90061600 0x7fbf90138cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf90142e90 con 0x7fbf900603e0 2022-07-21T01:55:47.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf900b20c0 con 0x7fbf9005cf40 2022-07-21T01:55:47.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9013fe20 con 0x7fbf90061600 2022-07-21T01:55:47.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbf9005cf40 0x7fbf901387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbf9005cf40 0x7fbf901387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38258/0 (socket says 172.21.15.140:38258) 2022-07-21T01:55:47.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8f7fe700 1 -- 172.21.15.140:0/3668310269 learned_addr learned my addr 172.21.15.140:0/3668310269 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:47.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8ffff700 1 --1- 172.21.15.140:0/3668310269 >> v1:172.21.15.140:6789/0 conn(0x7fbf900603e0 0x7fbf90060820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52408/0 (socket says 172.21.15.140:52408) 2022-07-21T01:55:47.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8effd700 1 --2- 172.21.15.140:0/3668310269 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90061600 0x7fbf90138cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8f7fe700 1 -- 172.21.15.140:0/3668310269 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90061600 msgr2=0x7fbf90138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:47.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8f7fe700 1 --2- 172.21.15.140:0/3668310269 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90061600 0x7fbf90138cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8f7fe700 1 -- 172.21.15.140:0/3668310269 >> v1:172.21.15.140:6789/0 conn(0x7fbf900603e0 legacy=0x7fbf90060820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8f7fe700 1 -- 172.21.15.140:0/3668310269 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf900b19b0 con 0x7fbf9005cf40 2022-07-21T01:55:47.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8f7fe700 1 --2- 172.21.15.140:0/3668310269 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf9005cf40 0x7fbf901387d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fbf84002f70 tx=0x7fbf84006eb0).ready entity=mon.0 client_cookie=2c3e80cb45f9cc5c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:47.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8e7fc700 1 -- 172.21.15.140:0/3668310269 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf8400d5c0 con 0x7fbf9005cf40 2022-07-21T01:55:47.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8e7fc700 1 -- 172.21.15.140:0/3668310269 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf84000e90 con 0x7fbf9005cf40 2022-07-21T01:55:47.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf8e7fc700 1 -- 172.21.15.140:0/3668310269 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf8400d950 con 0x7fbf9005cf40 2022-07-21T01:55:47.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 -- 172.21.15.140:0/3668310269 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf9005cf40 msgr2=0x7fbf901387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 --2- 172.21.15.140:0/3668310269 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf9005cf40 0x7fbf901387d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fbf84002f70 tx=0x7fbf84006eb0).stop 2022-07-21T01:55:47.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 -- 172.21.15.140:0/3668310269 shutdown_connections 2022-07-21T01:55:47.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 --2- 172.21.15.140:0/3668310269 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90061600 0x7fbf90138cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 --2- 172.21.15.140:0/3668310269 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf9005cf40 0x7fbf901387d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 -- 172.21.15.140:0/3668310269 >> 172.21.15.140:0/3668310269 conn(0x7fbf900b6270 msgr2=0x7fbf900b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:47.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.431+0000 7fbf95f33700 1 -- 172.21.15.140:0/3668310269 shutdown_connections 2022-07-21T01:55:47.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf95f33700 1 -- 172.21.15.140:0/3668310269 wait complete. 2022-07-21T01:55:47.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf95f33700 1 Processor -- start 2022-07-21T01:55:47.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf95f33700 1 -- start start 2022-07-21T01:55:47.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf95f33700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf9005cf40 0x7fbf9013ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf95f33700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf900603e0 0x7fbf90133db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf95f33700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90061600 0x7fbf901383b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf95f33700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9013b430 con 0x7fbf9005cf40 2022-07-21T01:55:47.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf95f33700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf90137f10 con 0x7fbf900603e0 2022-07-21T01:55:47.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf95f33700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9013b140 con 0x7fbf90061600 2022-07-21T01:55:47.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf9005cf40 0x7fbf9013ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf9005cf40 0x7fbf9013ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38264/0 (socket says 172.21.15.140:38264) 2022-07-21T01:55:47.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8f7fe700 1 -- 172.21.15.140:0/3070594009 learned_addr learned my addr 172.21.15.140:0/3070594009 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:47.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8effd700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf900603e0 0x7fbf90133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8ffff700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90061600 0x7fbf901383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8f7fe700 1 -- 172.21.15.140:0/3070594009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90061600 msgr2=0x7fbf901383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8f7fe700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90061600 0x7fbf901383b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8f7fe700 1 -- 172.21.15.140:0/3070594009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf900603e0 msgr2=0x7fbf90133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8f7fe700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf900603e0 0x7fbf90133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8f7fe700 1 -- 172.21.15.140:0/3070594009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8400b1e0 con 0x7fbf9005cf40 2022-07-21T01:55:47.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8effd700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf900603e0 0x7fbf90133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:55:47.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8f7fe700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf9005cf40 0x7fbf9013ac50 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7fbf84002f70 tx=0x7fbf84015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:47.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8cff9700 1 -- 172.21.15.140:0/3070594009 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf84020a40 con 0x7fbf9005cf40 2022-07-21T01:55:47.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8cff9700 1 -- 172.21.15.140:0/3070594009 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf84020ba0 con 0x7fbf9005cf40 2022-07-21T01:55:47.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf8cff9700 1 -- 172.21.15.140:0/3070594009 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbf84011670 con 0x7fbf9005cf40 2022-07-21T01:55:47.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf95f33700 1 -- 172.21.15.140:0/3070594009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf901342a0 con 0x7fbf9005cf40 2022-07-21T01:55:47.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.435+0000 7fbf95f33700 1 -- 172.21.15.140:0/3070594009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf900ac3f0 con 0x7fbf9005cf40 2022-07-21T01:55:47.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.439+0000 7fbf95f33700 1 -- 172.21.15.140:0/3070594009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf90138be0 con 0x7fbf9005cf40 2022-07-21T01:55:47.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.439+0000 7fbf8cff9700 1 -- 172.21.15.140:0/3070594009 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbf8401a040 con 0x7fbf9005cf40 2022-07-21T01:55:47.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.439+0000 7fbf8cff9700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf70053b00 0x7fbf70055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.439+0000 7fbf8effd700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf70053b00 0x7fbf70055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.439+0000 7fbf8cff9700 1 -- 172.21.15.140:0/3070594009 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbf840716d0 con 0x7fbf9005cf40 2022-07-21T01:55:47.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.439+0000 7fbf8effd700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf70053b00 0x7fbf70055fc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fbf80008000 tx=0x7fbf80008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:47.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.447+0000 7fbf8cff9700 1 -- 172.21.15.140:0/3070594009 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbf840485f0 con 0x7fbf9005cf40 2022-07-21T01:55:47.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.571+0000 7fbf95f33700 1 -- 172.21.15.140:0/3070594009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf901380d0 con 0x7fbf9005cf40 2022-07-21T01:55:47.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf8cff9700 1 -- 172.21.15.140:0/3070594009 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+10 (secure 0 0 0) 0x7fbf8404bc10 con 0x7fbf9005cf40 2022-07-21T01:55:47.578 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:47.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 -- 172.21.15.140:0/3070594009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf70053b00 msgr2=0x7fbf70055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf70053b00 0x7fbf70055fc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fbf80008000 tx=0x7fbf80008040).stop 2022-07-21T01:55:47.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 -- 172.21.15.140:0/3070594009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf9005cf40 msgr2=0x7fbf9013ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf9005cf40 0x7fbf9013ac50 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7fbf84002f70 tx=0x7fbf84015000).stop 2022-07-21T01:55:47.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 -- 172.21.15.140:0/3070594009 shutdown_connections 2022-07-21T01:55:47.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbf70053b00 0x7fbf70055fc0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf90061600 0x7fbf901383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf900603e0 0x7fbf90133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 --2- 172.21.15.140:0/3070594009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf9005cf40 0x7fbf9013ac50 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 -- 172.21.15.140:0/3070594009 >> 172.21.15.140:0/3070594009 conn(0x7fbf900b6270 msgr2=0x7fbf90142060 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:47.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 -- 172.21.15.140:0/3070594009 shutdown_connections 2022-07-21T01:55:47.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.575+0000 7fbf95f33700 1 -- 172.21.15.140:0/3070594009 wait complete. 2022-07-21T01:55:47.589 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:55:47.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.699+0000 7f1240f96700 1 Processor -- start 2022-07-21T01:55:47.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.699+0000 7f1240f96700 1 -- start start 2022-07-21T01:55:47.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.699+0000 7f1240f96700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f123c0b1fe0 0x7f123c0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.699+0000 7f1240f96700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f123c142810 0x7f123c142e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.699+0000 7f1240f96700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123c0ac980 con 0x7f123c0ac5a0 2022-07-21T01:55:47.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.699+0000 7f1240f96700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123c0b33e0 con 0x7f123c0b1fe0 2022-07-21T01:55:47.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.699+0000 7f1240f96700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0b3680 con 0x7f123c142810 2022-07-21T01:55:47.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123a59c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f123c0ac5a0 0x7f123c142100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52418/0 (socket says 172.21.15.140:52418) 2022-07-21T01:55:47.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123a59c700 1 -- 172.21.15.140:0/2212717180 learned_addr learned my addr 172.21.15.140:0/2212717180 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:47.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f1239d9b700 1 --2- 172.21.15.140:0/2212717180 >> v2:172.21.15.152:3300/0 conn(0x7f123c0b1fe0 0x7f123c0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123ad9d700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f123c142810 0x7f123c142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617586225 0 0) 0x7f123c0ac980 con 0x7f123c0ac5a0 2022-07-21T01:55:47.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1220003260 con 0x7f123c0ac5a0 2022-07-21T01:55:47.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3233696218 0 0) 0x7f1220003260 con 0x7f123c0ac5a0 2022-07-21T01:55:47.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f123c142810 msgr2=0x7f123c142e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:47.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f123c142810 0x7f123c142e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 >> v2:172.21.15.152:3300/0 conn(0x7f123c0b1fe0 msgr2=0x7f123c0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:47.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 --2- 172.21.15.140:0/2212717180 >> v2:172.21.15.152:3300/0 conn(0x7f123c0b1fe0 0x7f123c0b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c0b3920 con 0x7f123c0ac5a0 2022-07-21T01:55:47.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f1230001660 con 0x7f123c0ac5a0 2022-07-21T01:55:47.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 >> v1:172.21.15.140:6789/0 conn(0x7f123c0ac5a0 legacy=0x7f123c142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1220003750 0x7f1220003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1220004500 0x7f1220004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f123c142810 0x7f1220004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0b33e0 con 0x7f1220003750 2022-07-21T01:55:47.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c0b3680 con 0x7f1220004500 2022-07-21T01:55:47.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122000b750 con 0x7f123c142810 2022-07-21T01:55:47.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123a59c700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1220003750 0x7f1220003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123ad9d700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f123c142810 0x7f1220004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f1239d9b700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1220004500 0x7f1220004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123a59c700 1 -- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f123c142810 msgr2=0x7f1220004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:47.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123a59c700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f123c142810 0x7f1220004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123a59c700 1 -- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1220004500 msgr2=0x7f1220004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:47.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123a59c700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1220004500 0x7f1220004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123a59c700 1 -- 172.21.15.140:0/2212717180 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1220003260 con 0x7f1220003750 2022-07-21T01:55:47.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f1239d9b700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1220004500 0x7f1220004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:55:47.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123ad9d700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f123c142810 0x7f1220004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:55:47.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123a59c700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1220003750 0x7f1220003b70 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f1230001370 tx=0x7f1230001e50).ready entity=mon.0 client_cookie=f73c02fb2ce35ef0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:47.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12300070d0 con 0x7f1220003750 2022-07-21T01:55:47.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1230007230 con 0x7f1220003750 2022-07-21T01:55:47.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123959a700 1 -- 172.21.15.140:0/2212717180 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f123000c9c0 con 0x7f1220003750 2022-07-21T01:55:47.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f1240f96700 1 -- 172.21.15.140:0/2212717180 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1220003750 msgr2=0x7f1220003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f1240f96700 1 --2- 172.21.15.140:0/2212717180 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1220003750 0x7f1220003b70 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f1230001370 tx=0x7f1230001e50).stop 2022-07-21T01:55:47.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f123ad9d700 1 -- 172.21.15.140:0/2212717180 reap_dead start 2022-07-21T01:55:47.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f1240f96700 1 -- 172.21.15.140:0/2212717180 shutdown_connections 2022-07-21T01:55:47.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f1240f96700 1 -- 172.21.15.140:0/2212717180 >> 172.21.15.140:0/2212717180 conn(0x7f123c0b47c0 msgr2=0x7f123c05bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:47.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f1240f96700 1 -- 172.21.15.140:0/2212717180 shutdown_connections 2022-07-21T01:55:47.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.703+0000 7f1240f96700 1 -- 172.21.15.140:0/2212717180 wait complete. 2022-07-21T01:55:47.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.707+0000 7f1240f96700 1 Processor -- start 2022-07-21T01:55:47.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.707+0000 7f1240f96700 1 -- start start 2022-07-21T01:55:47.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.707+0000 7f1240f96700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1220003750 0x7f123c062b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.707+0000 7f1240f96700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c064b90 0x7f123c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.707+0000 7f1240f96700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f123c151e30 0x7f123c1522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.707+0000 7f1240f96700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c155500 con 0x7f123c151e30 2022-07-21T01:55:47.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.707+0000 7f1240f96700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123c1557a0 con 0x7f123c064b90 2022-07-21T01:55:47.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.707+0000 7f1240f96700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123c140080 con 0x7f1220003750 2022-07-21T01:55:47.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f123ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f123c151e30 0x7f123c1522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f123ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f123c151e30 0x7f123c1522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38288/0 (socket says 172.21.15.140:38288) 2022-07-21T01:55:47.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f123ad9d700 1 -- 172.21.15.140:0/3073367095 learned_addr learned my addr 172.21.15.140:0/3073367095 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:47.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f123a59c700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1220003750 0x7f123c062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f1239d9b700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c064b90 0x7f123c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f123ad9d700 1 -- 172.21.15.140:0/3073367095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1220003750 msgr2=0x7f123c062b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f123ad9d700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1220003750 0x7f123c062b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f123ad9d700 1 -- 172.21.15.140:0/3073367095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c064b90 msgr2=0x7f123c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f123ad9d700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c064b90 0x7f123c155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f123ad9d700 1 -- 172.21.15.140:0/3073367095 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c063070 con 0x7f123c151e30 2022-07-21T01:55:47.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f123ad9d700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f123c151e30 0x7f123c1522b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f123400b000 tx=0x7f1234008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:47.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f122b7fe700 1 -- 172.21.15.140:0/3073367095 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1234014810 con 0x7f123c151e30 2022-07-21T01:55:47.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f122b7fe700 1 -- 172.21.15.140:0/3073367095 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1234014970 con 0x7f123c151e30 2022-07-21T01:55:47.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f1240f96700 1 -- 172.21.15.140:0/3073367095 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123c156130 con 0x7f123c151e30 2022-07-21T01:55:47.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f122b7fe700 1 -- 172.21.15.140:0/3073367095 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1234014cd0 con 0x7f123c151e30 2022-07-21T01:55:47.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f1240f96700 1 -- 172.21.15.140:0/3073367095 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f123c0aee50 con 0x7f123c151e30 2022-07-21T01:55:47.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f1240f96700 1 -- 172.21.15.140:0/3073367095 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f123c060230 con 0x7f123c151e30 2022-07-21T01:55:47.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f122b7fe700 1 -- 172.21.15.140:0/3073367095 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1234012040 con 0x7f123c151e30 2022-07-21T01:55:47.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f122b7fe700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1218053980 0x7f1218055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f122b7fe700 1 -- 172.21.15.140:0/3073367095 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f123406c310 con 0x7f123c151e30 2022-07-21T01:55:47.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.711+0000 7f123a59c700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1218053980 0x7f1218055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.715+0000 7f123a59c700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1218053980 0x7f1218055e40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f1230019000 tx=0x7f1230015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:47.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.719+0000 7f122b7fe700 1 -- 172.21.15.140:0/3073367095 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1234043130 con 0x7f123c151e30 2022-07-21T01:55:47.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 -- 172.21.15.140:0/3073367095 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f123c0574e0 con 0x7f123c151e30 2022-07-21T01:55:47.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f122b7fe700 1 -- 172.21.15.140:0/3073367095 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+10 (secure 0 0 0) 0x7f12340180a0 con 0x7f123c151e30 2022-07-21T01:55:47.852 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:47.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 -- 172.21.15.140:0/3073367095 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1218053980 msgr2=0x7f1218055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1218053980 0x7f1218055e40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f1230019000 tx=0x7f1230015000).stop 2022-07-21T01:55:47.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 -- 172.21.15.140:0/3073367095 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f123c151e30 msgr2=0x7f123c1522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f123c151e30 0x7f123c1522b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f123400b000 tx=0x7f1234008040).stop 2022-07-21T01:55:47.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 -- 172.21.15.140:0/3073367095 shutdown_connections 2022-07-21T01:55:47.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1218053980 0x7f1218055e40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f123c151e30 0x7f123c1522b0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f123c064b90 0x7f123c155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 --2- 172.21.15.140:0/3073367095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1220003750 0x7f123c062b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 -- 172.21.15.140:0/3073367095 >> 172.21.15.140:0/3073367095 conn(0x7f123c0b47c0 msgr2=0x7f123c1408e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:47.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 -- 172.21.15.140:0/3073367095 shutdown_connections 2022-07-21T01:55:47.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.847+0000 7f1240f96700 1 -- 172.21.15.140:0/3073367095 wait complete. 2022-07-21T01:55:47.862 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:55:47.863 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:47.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0729f700 1 Processor -- start 2022-07-21T01:55:47.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0729f700 1 -- start start 2022-07-21T01:55:47.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0729f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b0005bd90 0x7f3b000b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0729f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b000aea30 con 0x7f3b000b1200 2022-07-21T01:55:47.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0729f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b000aec60 con 0x7f3b000b5650 2022-07-21T01:55:47.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0729f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00142970 con 0x7f3b0005bd90 2022-07-21T01:55:47.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b05a9c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f3b000b1200 0x7f3b00142d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52438/0 (socket says 172.21.15.140:52438) 2022-07-21T01:55:47.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b05a9c700 1 -- 172.21.15.140:0/1514872743 learned_addr learned my addr 172.21.15.140:0/1514872743 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:47.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b06a9e700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b0005bd90 0x7f3b000b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565611855 0 0) 0x7f3b000aea30 con 0x7f3b000b1200 2022-07-21T01:55:47.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aec003260 con 0x7f3b000b1200 2022-07-21T01:55:47.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326808438 0 0) 0x7f3b000aec60 con 0x7f3b000b5650 2022-07-21T01:55:47.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b000aea30 con 0x7f3b000b5650 2022-07-21T01:55:47.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 322935212 0 0) 0x7f3aec003260 con 0x7f3b000b1200 2022-07-21T01:55:47.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b0005bd90 msgr2=0x7f3b000b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:47.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b0005bd90 0x7f3b000b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 >> v1:172.21.15.152:6789/0 conn(0x7f3b000b5650 legacy=0x7f3b0005c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b000af180 con 0x7f3b000b1200 2022-07-21T01:55:47.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f3afc001b60 con 0x7f3b000b1200 2022-07-21T01:55:47.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 >> v1:172.21.15.140:6789/0 conn(0x7f3b000b1200 legacy=0x7f3b00142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b0005bd90 0x7f3aec003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3aec003e20 0x7f3aec004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aec0049f0 0x7f3aec004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:47.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec00b800 con 0x7f3aec003e20 2022-07-21T01:55:47.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec00b940 con 0x7f3aec0049f0 2022-07-21T01:55:47.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec00bb00 con 0x7f3b0005bd90 2022-07-21T01:55:47.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.963+0000 7f3b05a9c700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3aec003e20 0x7f3aec004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b06a9e700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aec0049f0 0x7f3aec004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0629d700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b0005bd90 0x7f3aec003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:47.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b05a9c700 1 -- 172.21.15.140:0/1514872743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b0005bd90 msgr2=0x7f3aec003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:47.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b05a9c700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b0005bd90 0x7f3aec003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b05a9c700 1 -- 172.21.15.140:0/1514872743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aec0049f0 msgr2=0x7f3aec004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:47.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b05a9c700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aec0049f0 0x7f3aec004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:47.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b05a9c700 1 -- 172.21.15.140:0/1514872743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec003260 con 0x7f3aec003e20 2022-07-21T01:55:47.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b05a9c700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3aec003e20 0x7f3aec004500 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f3afc001930 tx=0x7f3afc007910).ready entity=mon.0 client_cookie=994d686becd1dc03 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:47.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3afc00cbe0 con 0x7f3aec003e20 2022-07-21T01:55:47.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc00cd40 con 0x7f3aec003e20 2022-07-21T01:55:47.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0529b700 1 -- 172.21.15.140:0/1514872743 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3afc005a50 con 0x7f3aec003e20 2022-07-21T01:55:47.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0729f700 1 -- 172.21.15.140:0/1514872743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3aec003e20 msgr2=0x7f3aec004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:47.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0729f700 1 --2- 172.21.15.140:0/1514872743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3aec003e20 0x7f3aec004500 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f3afc001930 tx=0x7f3afc007910).stop 2022-07-21T01:55:47.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b06a9e700 1 -- 172.21.15.140:0/1514872743 reap_dead start 2022-07-21T01:55:47.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0729f700 1 -- 172.21.15.140:0/1514872743 shutdown_connections 2022-07-21T01:55:47.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0729f700 1 -- 172.21.15.140:0/1514872743 >> 172.21.15.140:0/1514872743 conn(0x7f3b000634e0 msgr2=0x7f3b000b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:47.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0729f700 1 -- 172.21.15.140:0/1514872743 shutdown_connections 2022-07-21T01:55:47.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0729f700 1 -- 172.21.15.140:0/1514872743 wait complete. 2022-07-21T01:55:47.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0729f700 1 Processor -- start 2022-07-21T01:55:47.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.967+0000 7f3b0729f700 1 -- start start 2022-07-21T01:55:47.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b0729f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3aec003e20 0x7f3b00155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:48.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b0729f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b00145150 0x7f3b0013d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:48.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b0729f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b0013a960 0x7f3b001387a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:48.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b0729f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00145e50 con 0x7f3b0013a960 2022-07-21T01:55:48.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b0729f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b001553c0 con 0x7f3b00145150 2022-07-21T01:55:48.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b0729f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00155660 con 0x7f3aec003e20 2022-07-21T01:55:48.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b06a9e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b0013a960 0x7f3b001387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:48.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b0629d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3aec003e20 0x7f3b00155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:48.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b06a9e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b0013a960 0x7f3b001387a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38308/0 (socket says 172.21.15.140:38308) 2022-07-21T01:55:48.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b06a9e700 1 -- 172.21.15.140:0/3275678862 learned_addr learned my addr 172.21.15.140:0/3275678862 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:48.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b05a9c700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b00145150 0x7f3b0013d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:48.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b06a9e700 1 -- 172.21.15.140:0/3275678862 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3aec003e20 msgr2=0x7f3b00155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:48.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b06a9e700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3aec003e20 0x7f3b00155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:48.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b06a9e700 1 -- 172.21.15.140:0/3275678862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b00145150 msgr2=0x7f3b0013d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:48.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b06a9e700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b00145150 0x7f3b0013d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:48.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b06a9e700 1 -- 172.21.15.140:0/3275678862 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00155900 con 0x7f3b0013a960 2022-07-21T01:55:48.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b06a9e700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b0013a960 0x7f3b001387a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f3af800b000 tx=0x7f3af8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:48.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3af77fe700 1 -- 172.21.15.140:0/3275678862 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af8019070 con 0x7f3b0013a960 2022-07-21T01:55:48.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3af77fe700 1 -- 172.21.15.140:0/3275678862 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af8000e90 con 0x7f3b0013a960 2022-07-21T01:55:48.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b0729f700 1 -- 172.21.15.140:0/3275678862 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0013dd40 con 0x7f3b0013a960 2022-07-21T01:55:48.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3af77fe700 1 -- 172.21.15.140:0/3275678862 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af8013870 con 0x7f3b0013a960 2022-07-21T01:55:48.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b0729f700 1 -- 172.21.15.140:0/3275678862 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b00138eb0 con 0x7f3b0013a960 2022-07-21T01:55:48.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3b0729f700 1 -- 172.21.15.140:0/3275678862 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b00155a90 con 0x7f3b0013a960 2022-07-21T01:55:48.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.971+0000 7f3af77fe700 1 -- 172.21.15.140:0/3275678862 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3af80139d0 con 0x7f3b0013a960 2022-07-21T01:55:48.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.975+0000 7f3af77fe700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053b00 0x7f3ae0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:48.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.975+0000 7f3af77fe700 1 -- 172.21.15.140:0/3275678862 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3af806baf0 con 0x7f3b0013a960 2022-07-21T01:55:48.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.975+0000 7f3b0629d700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053b00 0x7f3ae0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:48.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.975+0000 7f3b0629d700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053b00 0x7f3ae0055fc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f3af0003000 tx=0x7f3af0007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:48.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:47.979+0000 7f3af77fe700 1 -- 172.21.15.140:0/3275678862 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3af8042a10 con 0x7f3b0013a960 2022-07-21T01:55:48.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.139+0000 7f3b0729f700 1 -- 172.21.15.140:0/3275678862 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b000574e0 con 0x7f3b0013a960 2022-07-21T01:55:48.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.139+0000 7f3af77fe700 1 -- 172.21.15.140:0/3275678862 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f3af8046030 con 0x7f3b0013a960 2022-07-21T01:55:48.144 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:48.144 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/518920349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":518920349},{"type":"v1","addr":"172.21.15.152:6833","nonce":518920349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:43.343194+0000","modified":"2022-07-21T01:55:44.346904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2478727503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2478727503},{"type":"v1","addr":"172.21.15.140:6835","nonce":2478727503}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-07-21T01:55:48.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 -- 172.21.15.140:0/3275678862 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053b00 msgr2=0x7f3ae0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:48.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053b00 0x7f3ae0055fc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f3af0003000 tx=0x7f3af0007a00).stop 2022-07-21T01:55:48.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 -- 172.21.15.140:0/3275678862 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b0013a960 msgr2=0x7f3b001387a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:48.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b0013a960 0x7f3b001387a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f3af800b000 tx=0x7f3af8006df0).stop 2022-07-21T01:55:48.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 -- 172.21.15.140:0/3275678862 shutdown_connections 2022-07-21T01:55:48.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053b00 0x7f3ae0055fc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:48.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b0013a960 0x7f3b001387a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:48.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b00145150 0x7f3b0013d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:48.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 --2- 172.21.15.140:0/3275678862 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3aec003e20 0x7f3b00155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:48.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 -- 172.21.15.140:0/3275678862 >> 172.21.15.140:0/3275678862 conn(0x7f3b000634e0 msgr2=0x7f3b000b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:48.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 -- 172.21.15.140:0/3275678862 shutdown_connections 2022-07-21T01:55:48.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:48.143+0000 7f3b0729f700 1 -- 172.21.15.140:0/3275678862 wait complete. 2022-07-21T01:55:48.153 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 42 2022-07-21T01:55:48.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:55:43.343194+0000', 'modified': '2022-07-21T01:55:44.346904+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7002}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7002': {'gid': 7002, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/2478727503', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 2478727503}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 2478727503}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:55:48.359 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:55:48.360 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:55:48.477 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":7002,"want_state":"up:active","state":"up:active","rank_uptime":5.1219510550000003,"mdsmap_epoch":42,"osdmap_epoch":94,"osdmap_epoch_barrier":93,"uptime":9.2940656159999993} 2022-07-21T01:55:48.478 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 7002, "mdsmap_epoch": 42, "osdmap_epoch": 94, "osdmap_epoch_barrier": 93, "rank_uptime": 5.121951055, "state": "up:active", "uptime": 9.294065616, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:55:48.479 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:55:48.479 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:55:48.480 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:55:48.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:48.482 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:48.491 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:55:48.506 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:48.507 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:48.568 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:48.569 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:48.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:48.571 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:55:48.576 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:55:48.576 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:48.579 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:55:48.597 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:48.598 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:48.651 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23039]: starting ceph client 2022-07-21T01:55:48.652 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:55:48.647+0000 7f2576e5c080 -1 init, newargv = 0x55b0b5a05680 newargc=9 2022-07-21T01:55:48.661 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23039]: starting fuse 2022-07-21T01:55:48.662 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:48.663 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:48.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:48.664 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:55:48.670 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:55:48.670 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:55:48.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:48.671 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:55:48.672 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:55:48.673 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:55:48.673 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:55:48.673 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:55:48.674 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:48.674 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:55:48.675 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:55:48.675 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:55:48.676 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:48.676 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:55:48.677 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:55:48.677 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:55:48.678 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:55:48.678 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:48.679 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:55:48.679 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:55:48.679 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:55:48.680 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:55:48.680 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:55:48.681 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:48.681 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:55:48.681 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:55:48.724 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.23039.asok 2022-07-21T01:55:48.729 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23039.asok 2022-07-21T01:55:48.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:48.730 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23039.asok status 2022-07-21T01:55:48.870 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:55:48.871 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:55:48.871 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:55:48.871 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:55:48.871 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:55:48.871 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:55:48.872 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:55:48.872 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "23039", 2022-07-21T01:55:48.872 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:55:48.872 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:48.873 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:55:48.873 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:55:48.873 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 6356, 2022-07-21T01:55:48.873 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:55:48.873 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:55:48.874 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:55:48.874 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 6356 2022-07-21T01:55:48.875 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:48.875 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:55:48.876 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:55:48.876 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:55:48.877 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1278448270 2022-07-21T01:55:48.877 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:55:48.877 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:48.878 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:55:48.878 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:55:48.878 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:55:48.878 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 1278448270 2022-07-21T01:55:48.879 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:55:48.879 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.6356 172.21.15.191:0/1278448270", 2022-07-21T01:55:48.879 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/1278448270", 2022-07-21T01:55:48.879 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:55:48.880 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 42, 2022-07-21T01:55:48.880 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 94, 2022-07-21T01:55:48.880 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:55:48.880 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:55:48.880 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:55:48.881 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:55:48.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:48.883 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:48.883 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:48.892 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:55:48.893 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:55:48.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:48.894 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:48.955 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:55:49.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.063+0000 7fe1368c4700 1 Processor -- start 2022-07-21T01:55:49.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.063+0000 7fe1368c4700 1 -- start start 2022-07-21T01:55:49.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.063+0000 7fe1368c4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe13005c3f0 0x7fe13014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.063+0000 7fe1368c4700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13005d480 con 0x7fe1300acef0 2022-07-21T01:55:49.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.063+0000 7fe1368c4700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1300b21e0 con 0x7fe1300b1e00 2022-07-21T01:55:49.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.063+0000 7fe1368c4700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13005cf40 con 0x7fe13005c3f0 2022-07-21T01:55:49.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.063+0000 7fe134e61700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe1300acef0 0x7fe130146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52462/0 (socket says 172.21.15.140:52462) 2022-07-21T01:55:49.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.063+0000 7fe134e61700 1 -- 172.21.15.140:0/1088083080 learned_addr learned my addr 172.21.15.140:0/1088083080 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:49.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.063+0000 7fe12f7fe700 1 --2- 172.21.15.140:0/1088083080 >> v2:172.21.15.152:3300/0 conn(0x7fe13005c3f0 0x7fe13014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.063+0000 7fe12effd700 1 -- 172.21.15.140:0/1088083080 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049599428 0 0) 0x7fe13005d480 con 0x7fe1300acef0 2022-07-21T01:55:49.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe12effd700 1 -- 172.21.15.140:0/1088083080 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe11c003260 con 0x7fe1300acef0 2022-07-21T01:55:49.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe12f7fe700 1 -- 172.21.15.140:0/1088083080 >> v1:172.21.15.152:6789/0 conn(0x7fe1300b1e00 legacy=0x7fe1300ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe12effd700 1 -- 172.21.15.140:0/1088083080 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239486740 0 0) 0x7fe1300b21e0 con 0x7fe1300b1e00 2022-07-21T01:55:49.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe12f7fe700 1 -- 172.21.15.140:0/1088083080 >> v1:172.21.15.140:6789/0 conn(0x7fe1300acef0 legacy=0x7fe130146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe12f7fe700 1 -- 172.21.15.140:0/1088083080 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13005d1e0 con 0x7fe13005c3f0 2022-07-21T01:55:49.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe12f7fe700 1 --2- 172.21.15.140:0/1088083080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13005c3f0 0x7fe13014a6d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fe124002f70 tx=0x7fe124006df0).ready entity=mon.1 client_cookie=c2ae27c864b62ffe server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe12effd700 1 -- 172.21.15.140:0/1088083080 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe12400d610 con 0x7fe13005c3f0 2022-07-21T01:55:49.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe12effd700 1 -- 172.21.15.140:0/1088083080 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe124000e90 con 0x7fe13005c3f0 2022-07-21T01:55:49.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe12effd700 1 -- 172.21.15.140:0/1088083080 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe12400d960 con 0x7fe13005c3f0 2022-07-21T01:55:49.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe1368c4700 1 -- 172.21.15.140:0/1088083080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13005c3f0 msgr2=0x7fe13014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe1368c4700 1 --2- 172.21.15.140:0/1088083080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13005c3f0 0x7fe13014a6d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7fe124002f70 tx=0x7fe124006df0).stop 2022-07-21T01:55:49.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe1368c4700 1 -- 172.21.15.140:0/1088083080 shutdown_connections 2022-07-21T01:55:49.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe1368c4700 1 --2- 172.21.15.140:0/1088083080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe13005c3f0 0x7fe13014a6d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe1368c4700 1 -- 172.21.15.140:0/1088083080 >> 172.21.15.140:0/1088083080 conn(0x7fe1300611c0 msgr2=0x7fe130062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:49.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe1368c4700 1 -- 172.21.15.140:0/1088083080 shutdown_connections 2022-07-21T01:55:49.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe1368c4700 1 -- 172.21.15.140:0/1088083080 wait complete. 2022-07-21T01:55:49.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe1368c4700 1 Processor -- start 2022-07-21T01:55:49.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.067+0000 7fe1368c4700 1 -- start start 2022-07-21T01:55:49.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe1368c4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1300acef0 0x7fe1300b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe1368c4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1300b1e00 0x7fe130063c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe1368c4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1300b2ec0 0x7fe130064100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe1368c4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1301456a0 con 0x7fe1300acef0 2022-07-21T01:55:49.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe1368c4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe130148db0 con 0x7fe1300b2ec0 2022-07-21T01:55:49.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe1368c4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe130149050 con 0x7fe1300b1e00 2022-07-21T01:55:49.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1300acef0 0x7fe1300b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe134e61700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1300b2ec0 0x7fe130064100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12f7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1300b1e00 0x7fe130063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1300acef0 0x7fe1300b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38320/0 (socket says 172.21.15.140:38320) 2022-07-21T01:55:49.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12ffff700 1 -- 172.21.15.140:0/3709033849 learned_addr learned my addr 172.21.15.140:0/3709033849 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:49.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12ffff700 1 -- 172.21.15.140:0/3709033849 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1300b1e00 msgr2=0x7fe130063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12ffff700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1300b1e00 0x7fe130063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12ffff700 1 -- 172.21.15.140:0/3709033849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1300b2ec0 msgr2=0x7fe130064100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12ffff700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1300b2ec0 0x7fe130064100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12ffff700 1 -- 172.21.15.140:0/3709033849 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12400b180 con 0x7fe1300acef0 2022-07-21T01:55:49.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12f7fe700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1300b1e00 0x7fe130063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:49.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12ffff700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1300acef0 0x7fe1300b3a40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7fe128001250 tx=0x7fe1280013f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12d7fa700 1 -- 172.21.15.140:0/3709033849 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe12800f470 con 0x7fe1300acef0 2022-07-21T01:55:49.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12d7fa700 1 -- 172.21.15.140:0/3709033849 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12800f5d0 con 0x7fe1300acef0 2022-07-21T01:55:49.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe1368c4700 1 -- 172.21.15.140:0/3709033849 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1300b4a70 con 0x7fe1300acef0 2022-07-21T01:55:49.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe12d7fa700 1 -- 172.21.15.140:0/3709033849 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe12800f930 con 0x7fe1300acef0 2022-07-21T01:55:49.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe1368c4700 1 -- 172.21.15.140:0/3709033849 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe130064c30 con 0x7fe1300acef0 2022-07-21T01:55:49.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.071+0000 7fe1368c4700 1 -- 172.21.15.140:0/3709033849 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1300b04b0 con 0x7fe1300acef0 2022-07-21T01:55:49.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.075+0000 7fe12d7fa700 1 -- 172.21.15.140:0/3709033849 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe128019030 con 0x7fe1300acef0 2022-07-21T01:55:49.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.075+0000 7fe12d7fa700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe110053950 0x7fe110055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.075+0000 7fe12d7fa700 1 -- 172.21.15.140:0/3709033849 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe12806b630 con 0x7fe1300acef0 2022-07-21T01:55:49.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.075+0000 7fe12f7fe700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe110053950 0x7fe110055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.075+0000 7fe12f7fe700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe110053950 0x7fe110055e10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7fe12401a000 tx=0x7fe124014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.083+0000 7fe12d7fa700 1 -- 172.21.15.140:0/3709033849 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1280424b0 con 0x7fe1300acef0 2022-07-21T01:55:49.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.207+0000 7fe1368c4700 1 -- 172.21.15.140:0/3709033849 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1300574e0 con 0x7fe1300acef0 2022-07-21T01:55:49.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.207+0000 7fe12d7fa700 1 -- 172.21.15.140:0/3709033849 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16137 (secure 0 0 0) 0x7fe128017020 con 0x7fe1300acef0 2022-07-21T01:55:49.211 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:49.212 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":94,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:55:44.346577+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:55:41.913815+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2022-07-21T01:55:42.343580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:55:49.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 -- 172.21.15.140:0/3709033849 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe110053950 msgr2=0x7fe110055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe110053950 0x7fe110055e10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7fe12401a000 tx=0x7fe124014000).stop 2022-07-21T01:55:49.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 -- 172.21.15.140:0/3709033849 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1300acef0 msgr2=0x7fe1300b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1300acef0 0x7fe1300b3a40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7fe128001250 tx=0x7fe1280013f0).stop 2022-07-21T01:55:49.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 -- 172.21.15.140:0/3709033849 shutdown_connections 2022-07-21T01:55:49.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe110053950 0x7fe110055e10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1300b2ec0 0x7fe130064100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1300b1e00 0x7fe130063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 --2- 172.21.15.140:0/3709033849 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1300acef0 0x7fe1300b3a40 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 -- 172.21.15.140:0/3709033849 >> 172.21.15.140:0/3709033849 conn(0x7fe13005c3f0 msgr2=0x7fe130069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:49.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 -- 172.21.15.140:0/3709033849 shutdown_connections 2022-07-21T01:55:49.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.211+0000 7fe1368c4700 1 -- 172.21.15.140:0/3709033849 wait complete. 2022-07-21T01:55:49.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:55:41.913815+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:55:42.343580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:55:49.225 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:55:49.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.331+0000 7fc92e648700 1 Processor -- start 2022-07-21T01:55:49.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.331+0000 7fc92e648700 1 -- start start 2022-07-21T01:55:49.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92e648700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc92805c120 0x7fc928144320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92e648700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9280ac5e0 con 0x7fc9280b0480 2022-07-21T01:55:49.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92e648700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9280b5f30 con 0x7fc9280b5b50 2022-07-21T01:55:49.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92e648700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9280ad3c0 con 0x7fc92805c120 2022-07-21T01:55:49.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc9277fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc9280b0480 0x7fc92814a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52476/0 (socket says 172.21.15.140:52476) 2022-07-21T01:55:49.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc9277fe700 1 -- 172.21.15.140:0/514933117 learned_addr learned my addr 172.21.15.140:0/514933117 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:49.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92cbe5700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc92805c120 0x7fc928144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262750219 0 0) 0x7fc9280ac5e0 con 0x7fc9280b0480 2022-07-21T01:55:49.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc908003260 con 0x7fc9280b0480 2022-07-21T01:55:49.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 35884046 0 0) 0x7fc908003260 con 0x7fc9280b0480 2022-07-21T01:55:49.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc92805c120 msgr2=0x7fc928144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:49.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc92805c120 0x7fc928144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 >> v1:172.21.15.152:6789/0 conn(0x7fc9280b5b50 legacy=0x7fc9280accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92805cf70 con 0x7fc9280b0480 2022-07-21T01:55:49.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fc910001680 con 0x7fc9280b0480 2022-07-21T01:55:49.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 >> v1:172.21.15.140:6789/0 conn(0x7fc9280b0480 legacy=0x7fc92814a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc908003780 0x7fc908003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc92805c120 0x7fc908004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9080049f0 0x7fc908004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9280ad3c0 con 0x7fc908003780 2022-07-21T01:55:49.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90800b750 con 0x7fc9080049f0 2022-07-21T01:55:49.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90800b8e0 con 0x7fc92805c120 2022-07-21T01:55:49.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc927fff700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc908003780 0x7fc908003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc9277fe700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc92805c120 0x7fc908004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92cbe5700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9080049f0 0x7fc908004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc927fff700 1 -- 172.21.15.140:0/514933117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc92805c120 msgr2=0x7fc908004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:49.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc927fff700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc92805c120 0x7fc908004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc927fff700 1 -- 172.21.15.140:0/514933117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9080049f0 msgr2=0x7fc908004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:49.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc927fff700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9080049f0 0x7fc908004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc927fff700 1 -- 172.21.15.140:0/514933117 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc908003260 con 0x7fc908003780 2022-07-21T01:55:49.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc927fff700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc908003780 0x7fc908003b60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fc920001490 tx=0x7fc920007d80).ready entity=mon.0 client_cookie=256be70c886cd8cc server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9200017b0 con 0x7fc908003780 2022-07-21T01:55:49.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc920006890 con 0x7fc908003780 2022-07-21T01:55:49.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc926ffd700 1 -- 172.21.15.140:0/514933117 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc920006bf0 con 0x7fc908003780 2022-07-21T01:55:49.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92e648700 1 -- 172.21.15.140:0/514933117 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc908003780 msgr2=0x7fc908003b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92e648700 1 --2- 172.21.15.140:0/514933117 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc908003780 0x7fc908003b60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7fc920001490 tx=0x7fc920007d80).stop 2022-07-21T01:55:49.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92cbe5700 1 -- 172.21.15.140:0/514933117 reap_dead start 2022-07-21T01:55:49.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92e648700 1 -- 172.21.15.140:0/514933117 shutdown_connections 2022-07-21T01:55:49.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92e648700 1 -- 172.21.15.140:0/514933117 >> 172.21.15.140:0/514933117 conn(0x7fc9280635e0 msgr2=0x7fc9280b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:49.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92e648700 1 -- 172.21.15.140:0/514933117 shutdown_connections 2022-07-21T01:55:49.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.335+0000 7fc92e648700 1 -- 172.21.15.140:0/514933117 wait complete. 2022-07-21T01:55:49.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc92e648700 1 Processor -- start 2022-07-21T01:55:49.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc92e648700 1 -- start start 2022-07-21T01:55:49.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc92e648700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc908003780 0x7fc92809e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc92e648700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc928063f90 0x7fc9281478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc92e648700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc928147db0 0x7fc928149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc92e648700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9280662d0 con 0x7fc928147db0 2022-07-21T01:55:49.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc92e648700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc928157490 con 0x7fc928063f90 2022-07-21T01:55:49.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc92e648700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9281571f0 con 0x7fc908003780 2022-07-21T01:55:49.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc92cbe5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc928147db0 0x7fc928149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc927fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc908003780 0x7fc92809e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc9277fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc928063f90 0x7fc9281478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc9277fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc928063f90 0x7fc9281478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:44080/0 (socket says 172.21.15.140:44080) 2022-07-21T01:55:49.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc927fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc908003780 0x7fc92809e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:35068/0 (socket says 172.21.15.140:35068) 2022-07-21T01:55:49.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc9277fe700 1 -- 172.21.15.140:0/1583524358 learned_addr learned my addr 172.21.15.140:0/1583524358 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:49.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc9277fe700 1 -- 172.21.15.140:0/1583524358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc908003780 msgr2=0x7fc92809e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc9277fe700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc908003780 0x7fc92809e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc9277fe700 1 -- 172.21.15.140:0/1583524358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc928147db0 msgr2=0x7fc928149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc9277fe700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc928147db0 0x7fc928149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc9277fe700 1 -- 172.21.15.140:0/1583524358 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92809e880 con 0x7fc928063f90 2022-07-21T01:55:49.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc92cbe5700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc928147db0 0x7fc928149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:49.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.339+0000 7fc9277fe700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc928063f90 0x7fc9281478c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fc92809e0b0 tx=0x7fc9280b19a0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.343+0000 7fc9257fa700 1 -- 172.21.15.140:0/1583524358 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9100116b0 con 0x7fc928063f90 2022-07-21T01:55:49.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.343+0000 7fc9257fa700 1 -- 172.21.15.140:0/1583524358 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc910011810 con 0x7fc928063f90 2022-07-21T01:55:49.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.343+0000 7fc92e648700 1 -- 172.21.15.140:0/1583524358 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc928063b90 con 0x7fc928063f90 2022-07-21T01:55:49.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.343+0000 7fc9257fa700 1 -- 172.21.15.140:0/1583524358 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc910011b70 con 0x7fc928063f90 2022-07-21T01:55:49.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.343+0000 7fc92e648700 1 -- 172.21.15.140:0/1583524358 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc928067790 con 0x7fc928063f90 2022-07-21T01:55:49.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.343+0000 7fc92e648700 1 -- 172.21.15.140:0/1583524358 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9280ac810 con 0x7fc928063f90 2022-07-21T01:55:49.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.343+0000 7fc9257fa700 1 -- 172.21.15.140:0/1583524358 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc910011cd0 con 0x7fc928063f90 2022-07-21T01:55:49.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.343+0000 7fc9257fa700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc90c053b00 0x7fc90c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.343+0000 7fc9257fa700 1 -- 172.21.15.140:0/1583524358 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc91006ba00 con 0x7fc928063f90 2022-07-21T01:55:49.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.343+0000 7fc927fff700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc90c053b00 0x7fc90c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.343+0000 7fc927fff700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc90c053b00 0x7fc90c055fc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fc92001b000 tx=0x7fc920016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.351+0000 7fc9257fa700 1 -- 172.21.15.140:0/1583524358 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc910042880 con 0x7fc928063f90 2022-07-21T01:55:49.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.479+0000 7fc92e648700 1 -- 172.21.15.140:0/1583524358 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc9280574e0 con 0x7fc928063f90 2022-07-21T01:55:49.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.479+0000 7fc9257fa700 1 -- 172.21.15.140:0/1583524358 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v94) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc910019020 con 0x7fc928063f90 2022-07-21T01:55:49.482 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:55:49.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 -- 172.21.15.140:0/1583524358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc90c053b00 msgr2=0x7fc90c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc90c053b00 0x7fc90c055fc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fc92001b000 tx=0x7fc920016000).stop 2022-07-21T01:55:49.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 -- 172.21.15.140:0/1583524358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc928063f90 msgr2=0x7fc9281478c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc928063f90 0x7fc9281478c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fc92809e0b0 tx=0x7fc9280b19a0).stop 2022-07-21T01:55:49.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 -- 172.21.15.140:0/1583524358 shutdown_connections 2022-07-21T01:55:49.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc90c053b00 0x7fc90c055fc0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc928147db0 0x7fc928149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc928063f90 0x7fc9281478c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 --2- 172.21.15.140:0/1583524358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc908003780 0x7fc92809e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 -- 172.21.15.140:0/1583524358 >> 172.21.15.140:0/1583524358 conn(0x7fc9280635e0 msgr2=0x7fc9281440f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:49.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 -- 172.21.15.140:0/1583524358 shutdown_connections 2022-07-21T01:55:49.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.483+0000 7fc92e648700 1 -- 172.21.15.140:0/1583524358 wait complete. 2022-07-21T01:55:49.512 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:55:49.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 Processor -- start 2022-07-21T01:55:49.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 -- start start 2022-07-21T01:55:49.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0d140988b0 0x7f0d141422c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0d141427b0 0x7f0d14060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140afdb0 con 0x7f0d141462a0 2022-07-21T01:55:49.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d140b0050 con 0x7f0d141427b0 2022-07-21T01:55:49.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1405fcd0 con 0x7f0d140988b0 2022-07-21T01:55:49.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d137fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0d141427b0 0x7f0d14060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d12ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0d141462a0 0x7f0d14146680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52492/0 (socket says 172.21.15.140:52492) 2022-07-21T01:55:49.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d137fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0d141427b0 0x7f0d14060170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38350/0 (socket says 172.21.15.140:38350) 2022-07-21T01:55:49.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d12ffd700 1 -- 172.21.15.140:0/2331407635 learned_addr learned my addr 172.21.15.140:0/2331407635 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:49.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d127fc700 1 --2- 172.21.15.140:0/2331407635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0d140988b0 0x7f0d141422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d137fe700 1 -- 172.21.15.140:0/2331407635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0d140988b0 msgr2=0x7f0d141422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:49.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d137fe700 1 --2- 172.21.15.140:0/2331407635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0d140988b0 0x7f0d141422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d137fe700 1 -- 172.21.15.140:0/2331407635 >> v1:172.21.15.140:6789/0 conn(0x7f0d141462a0 legacy=0x7f0d14146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d11ffb700 1 -- 172.21.15.140:0/2331407635 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019573859 0 0) 0x7f0d140afdb0 con 0x7f0d141462a0 2022-07-21T01:55:49.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d137fe700 1 -- 172.21.15.140:0/2331407635 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d140afae0 con 0x7f0d141427b0 2022-07-21T01:55:49.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d137fe700 1 --2- 172.21.15.140:0/2331407635 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0d141427b0 0x7f0d14060170 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f0d0c0032f0 tx=0x7f0d0c0091d0).ready entity=mon.0 client_cookie=f252b4ad5e60cd5a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d11ffb700 1 -- 172.21.15.140:0/2331407635 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d0c00b6c0 con 0x7f0d141427b0 2022-07-21T01:55:49.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d11ffb700 1 -- 172.21.15.140:0/2331407635 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c00b820 con 0x7f0d141427b0 2022-07-21T01:55:49.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d11ffb700 1 -- 172.21.15.140:0/2331407635 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d0c00b990 con 0x7f0d141427b0 2022-07-21T01:55:49.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 -- 172.21.15.140:0/2331407635 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0d141427b0 msgr2=0x7f0d14060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 --2- 172.21.15.140:0/2331407635 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0d141427b0 0x7f0d14060170 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f0d0c0032f0 tx=0x7f0d0c0091d0).stop 2022-07-21T01:55:49.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 -- 172.21.15.140:0/2331407635 shutdown_connections 2022-07-21T01:55:49.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 --2- 172.21.15.140:0/2331407635 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0d141427b0 0x7f0d14060170 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.623+0000 7f0d19805700 1 --2- 172.21.15.140:0/2331407635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0d140988b0 0x7f0d141422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d19805700 1 -- 172.21.15.140:0/2331407635 >> 172.21.15.140:0/2331407635 conn(0x7f0d140b5f00 msgr2=0x7f0d140b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:49.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d19805700 1 -- 172.21.15.140:0/2331407635 shutdown_connections 2022-07-21T01:55:49.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d19805700 1 -- 172.21.15.140:0/2331407635 wait complete. 2022-07-21T01:55:49.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d19805700 1 Processor -- start 2022-07-21T01:55:49.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d19805700 1 -- start start 2022-07-21T01:55:49.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d19805700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0d140988b0 0x7f0d140b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d19805700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d141427b0 0x7f0d14151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d19805700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0d141462a0 0x7f0d140b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d19805700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141444b0 con 0x7f0d141462a0 2022-07-21T01:55:49.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d19805700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14144750 con 0x7f0d141427b0 2022-07-21T01:55:49.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d19805700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141449f0 con 0x7f0d140988b0 2022-07-21T01:55:49.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d137fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0d141462a0 0x7f0d140b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d137fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0d141462a0 0x7f0d140b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38356/0 (socket says 172.21.15.140:38356) 2022-07-21T01:55:49.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d137fe700 1 -- 172.21.15.140:0/4129392378 learned_addr learned my addr 172.21.15.140:0/4129392378 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:49.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d12ffd700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0d140988b0 0x7f0d140b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d127fc700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d141427b0 0x7f0d14151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d137fe700 1 -- 172.21.15.140:0/4129392378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0d140988b0 msgr2=0x7f0d140b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d137fe700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0d140988b0 0x7f0d140b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d137fe700 1 -- 172.21.15.140:0/4129392378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d141427b0 msgr2=0x7f0d14151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d137fe700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d141427b0 0x7f0d14151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d137fe700 1 -- 172.21.15.140:0/4129392378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c009470 con 0x7f0d141462a0 2022-07-21T01:55:49.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d137fe700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0d141462a0 0x7f0d140b0c90 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f0d0c0032f0 tx=0x7f0d0c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d03fff700 1 -- 172.21.15.140:0/4129392378 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d0c00b990 con 0x7f0d141462a0 2022-07-21T01:55:49.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.627+0000 7f0d03fff700 1 -- 172.21.15.140:0/4129392378 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c021630 con 0x7f0d141462a0 2022-07-21T01:55:49.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.631+0000 7f0d19805700 1 -- 172.21.15.140:0/4129392378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d141441a0 con 0x7f0d141462a0 2022-07-21T01:55:49.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.631+0000 7f0d03fff700 1 -- 172.21.15.140:0/4129392378 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d0c00b6c0 con 0x7f0d141462a0 2022-07-21T01:55:49.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.631+0000 7f0d19805700 1 -- 172.21.15.140:0/4129392378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1409f540 con 0x7f0d141462a0 2022-07-21T01:55:49.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.631+0000 7f0d19805700 1 -- 172.21.15.140:0/4129392378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d14144bb0 con 0x7f0d141462a0 2022-07-21T01:55:49.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.631+0000 7f0d03fff700 1 -- 172.21.15.140:0/4129392378 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0d0c029070 con 0x7f0d141462a0 2022-07-21T01:55:49.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.631+0000 7f0d03fff700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0cf0053a70 0x7f0cf0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.631+0000 7f0d03fff700 1 -- 172.21.15.140:0/4129392378 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0d0c06cf10 con 0x7f0d141462a0 2022-07-21T01:55:49.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.631+0000 7f0d12ffd700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0cf0053a70 0x7f0cf0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.635+0000 7f0d12ffd700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0cf0053a70 0x7f0cf0055f30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f0d0400d000 tx=0x7f0d04007c20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.639+0000 7f0d03fff700 1 -- 172.21.15.140:0/4129392378 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0d0c043eb0 con 0x7f0d141462a0 2022-07-21T01:55:49.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.759+0000 7f0d19805700 1 -- 172.21.15.140:0/4129392378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0d14146d90 con 0x7f0d141462a0 2022-07-21T01:55:49.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.759+0000 7f0d03fff700 1 -- 172.21.15.140:0/4129392378 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v94) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0d0c0150a0 con 0x7f0d141462a0 2022-07-21T01:55:49.763 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:49.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 -- 172.21.15.140:0/4129392378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0cf0053a70 msgr2=0x7f0cf0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0cf0053a70 0x7f0cf0055f30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f0d0400d000 tx=0x7f0d04007c20).stop 2022-07-21T01:55:49.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 -- 172.21.15.140:0/4129392378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0d141462a0 msgr2=0x7f0d140b0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0d141462a0 0x7f0d140b0c90 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f0d0c0032f0 tx=0x7f0d0c016000).stop 2022-07-21T01:55:49.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 -- 172.21.15.140:0/4129392378 shutdown_connections 2022-07-21T01:55:49.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0cf0053a70 0x7f0cf0055f30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0d141462a0 0x7f0d140b0c90 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0d141427b0 0x7f0d14151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 --2- 172.21.15.140:0/4129392378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0d140988b0 0x7f0d140b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 -- 172.21.15.140:0/4129392378 >> 172.21.15.140:0/4129392378 conn(0x7f0d140b5f00 msgr2=0x7f0d1405bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:49.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 -- 172.21.15.140:0/4129392378 shutdown_connections 2022-07-21T01:55:49.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.763+0000 7f0d19805700 1 -- 172.21.15.140:0/4129392378 wait complete. 2022-07-21T01:55:49.781 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:55:49.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.879+0000 7f7b3337b700 1 Processor -- start 2022-07-21T01:55:49.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.879+0000 7f7b3337b700 1 -- start start 2022-07-21T01:55:49.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.879+0000 7f7b3337b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 0x7f7b2c1445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.879+0000 7f7b3337b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b2c144a90 0x7f7b2c129340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b2c0b5dd0 con 0x7f7b2c0b59f0 2022-07-21T01:55:49.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b2c0b1750 con 0x7f7b2c144a90 2022-07-21T01:55:49.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c140ef0 con 0x7f7b2c0aef30 2022-07-21T01:55:49.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b31b78700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 0x7f7b2c1445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b32b7a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b2c144a90 0x7f7b2c129340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b32b7a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b2c144a90 0x7f7b2c129340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:44104/0 (socket says 172.21.15.140:44104) 2022-07-21T01:55:49.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b31b78700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 0x7f7b2c1445a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:35090/0 (socket says 172.21.15.140:35090) 2022-07-21T01:55:49.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b32379700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7b2c0b59f0 0x7f7b2c1407e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:53620/0 (socket says 172.21.15.140:53620) 2022-07-21T01:55:49.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b32b7a700 1 -- 172.21.15.140:0/3597035696 learned_addr learned my addr 172.21.15.140:0/3597035696 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:49.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b32b7a700 1 -- 172.21.15.140:0/3597035696 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 msgr2=0x7f7b2c1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:49.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b32b7a700 1 --2- 172.21.15.140:0/3597035696 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 0x7f7b2c1445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b32b7a700 1 -- 172.21.15.140:0/3597035696 >> v1:172.21.15.152:6789/0 conn(0x7f7b2c0b59f0 legacy=0x7f7b2c1407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b32b7a700 1 -- 172.21.15.140:0/3597035696 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c0652a0 con 0x7f7b2c144a90 2022-07-21T01:55:49.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b32b7a700 1 --2- 172.21.15.140:0/3597035696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b2c144a90 0x7f7b2c129340 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f7b240032f0 tx=0x7f7b24009290).ready entity=mon.1 client_cookie=4f09188e00331ab7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b31377700 1 -- 172.21.15.140:0/3597035696 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b2400b8a0 con 0x7f7b2c144a90 2022-07-21T01:55:49.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b31377700 1 -- 172.21.15.140:0/3597035696 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2400ba00 con 0x7f7b2c144a90 2022-07-21T01:55:49.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b31377700 1 -- 172.21.15.140:0/3597035696 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b2400bb70 con 0x7f7b2c144a90 2022-07-21T01:55:49.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 -- 172.21.15.140:0/3597035696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b2c144a90 msgr2=0x7f7b2c129340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 --2- 172.21.15.140:0/3597035696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b2c144a90 0x7f7b2c129340 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f7b240032f0 tx=0x7f7b24009290).stop 2022-07-21T01:55:49.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 -- 172.21.15.140:0/3597035696 shutdown_connections 2022-07-21T01:55:49.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 --2- 172.21.15.140:0/3597035696 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 0x7f7b2c1445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 --2- 172.21.15.140:0/3597035696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b2c144a90 0x7f7b2c129340 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 -- 172.21.15.140:0/3597035696 >> 172.21.15.140:0/3597035696 conn(0x7f7b2c063480 msgr2=0x7f7b2c0b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:49.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 -- 172.21.15.140:0/3597035696 shutdown_connections 2022-07-21T01:55:49.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 -- 172.21.15.140:0/3597035696 wait complete. 2022-07-21T01:55:49.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.883+0000 7f7b3337b700 1 Processor -- start 2022-07-21T01:55:49.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b3337b700 1 -- start start 2022-07-21T01:55:49.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b3337b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 0x7f7b2c134680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b3337b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b2c0b59f0 0x7f7b2c065870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b3337b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b2c144a90 0x7f7b2c0acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b3337b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0ae410 con 0x7f7b2c0b59f0 2022-07-21T01:55:49.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b3337b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0ae640 con 0x7f7b2c144a90 2022-07-21T01:55:49.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b3337b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0ae8e0 con 0x7f7b2c0aef30 2022-07-21T01:55:49.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b31b78700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b2c0b59f0 0x7f7b2c065870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b31b78700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b2c0b59f0 0x7f7b2c065870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38366/0 (socket says 172.21.15.140:38366) 2022-07-21T01:55:49.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b31b78700 1 -- 172.21.15.140:0/4152010262 learned_addr learned my addr 172.21.15.140:0/4152010262 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:49.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b32b7a700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b2c144a90 0x7f7b2c0acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b32379700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 0x7f7b2c134680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b31b78700 1 -- 172.21.15.140:0/4152010262 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 msgr2=0x7f7b2c134680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b31b78700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 0x7f7b2c134680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b31b78700 1 -- 172.21.15.140:0/4152010262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b2c144a90 msgr2=0x7f7b2c0acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:49.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b31b78700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b2c144a90 0x7f7b2c0acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:49.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b31b78700 1 -- 172.21.15.140:0/4152010262 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b24009530 con 0x7f7b2c0b59f0 2022-07-21T01:55:49.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b32379700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 0x7f7b2c134680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:49.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b31b78700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b2c0b59f0 0x7f7b2c065870 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f7b1400c000 tx=0x7f7b14006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b2b7fe700 1 -- 172.21.15.140:0/4152010262 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b14010780 con 0x7f7b2c0b59f0 2022-07-21T01:55:49.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b2b7fe700 1 -- 172.21.15.140:0/4152010262 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b140108e0 con 0x7f7b2c0b59f0 2022-07-21T01:55:49.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b2b7fe700 1 -- 172.21.15.140:0/4152010262 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b14010c40 con 0x7f7b2c0b59f0 2022-07-21T01:55:49.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b3337b700 1 -- 172.21.15.140:0/4152010262 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c0acae0 con 0x7f7b2c0b59f0 2022-07-21T01:55:49.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b3337b700 1 -- 172.21.15.140:0/4152010262 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c130a80 con 0x7f7b2c0b59f0 2022-07-21T01:55:49.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.887+0000 7f7b3337b700 1 -- 172.21.15.140:0/4152010262 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2c0acc70 con 0x7f7b2c0b59f0 2022-07-21T01:55:49.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.891+0000 7f7b2b7fe700 1 -- 172.21.15.140:0/4152010262 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7b1401c460 con 0x7f7b2c0b59f0 2022-07-21T01:55:49.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.891+0000 7f7b2b7fe700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b10053aa0 0x7f7b10055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:49.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.891+0000 7f7b2b7fe700 1 -- 172.21.15.140:0/4152010262 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7b1406c470 con 0x7f7b2c0b59f0 2022-07-21T01:55:49.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.891+0000 7f7b32379700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b10053aa0 0x7f7b10055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:49.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.891+0000 7f7b32379700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b10053aa0 0x7f7b10055f60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f7b1c00b000 tx=0x7f7b1c0070e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:49.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:49.895+0000 7f7b2b7fe700 1 -- 172.21.15.140:0/4152010262 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7b14043290 con 0x7f7b2c0b59f0 2022-07-21T01:55:50.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.018+0000 7f7b3337b700 1 -- 172.21.15.140:0/4152010262 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7b2c143e60 con 0x7f7b2c0b59f0 2022-07-21T01:55:50.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.018+0000 7f7b2b7fe700 1 -- 172.21.15.140:0/4152010262 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7b140468b0 con 0x7f7b2c0b59f0 2022-07-21T01:55:50.023 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:50.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 -- 172.21.15.140:0/4152010262 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b10053aa0 msgr2=0x7f7b10055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b10053aa0 0x7f7b10055f60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f7b1c00b000 tx=0x7f7b1c0070e0).stop 2022-07-21T01:55:50.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 -- 172.21.15.140:0/4152010262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b2c0b59f0 msgr2=0x7f7b2c065870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b2c0b59f0 0x7f7b2c065870 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f7b1400c000 tx=0x7f7b14006df0).stop 2022-07-21T01:55:50.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 -- 172.21.15.140:0/4152010262 shutdown_connections 2022-07-21T01:55:50.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b10053aa0 0x7f7b10055f60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b2c144a90 0x7f7b2c0acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b2c0b59f0 0x7f7b2c065870 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 --2- 172.21.15.140:0/4152010262 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b2c0aef30 0x7f7b2c134680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 -- 172.21.15.140:0/4152010262 >> 172.21.15.140:0/4152010262 conn(0x7f7b2c063480 msgr2=0x7f7b2c0b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:50.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 -- 172.21.15.140:0/4152010262 shutdown_connections 2022-07-21T01:55:50.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.022+0000 7f7b3337b700 1 -- 172.21.15.140:0/4152010262 wait complete. 2022-07-21T01:55:50.039 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:55:50.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 Processor -- start 2022-07-21T01:55:50.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 -- start start 2022-07-21T01:55:50.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c0aa9d0 0x7fef2c0b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef2c143f60 0x7fef2c0abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2c1443f0 con 0x7fef2c0b58f0 2022-07-21T01:55:50.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2c0aae10 con 0x7fef2c143f60 2022-07-21T01:55:50.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c0b4150 con 0x7fef2c0aa9d0 2022-07-21T01:55:50.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef2bfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef2c143f60 0x7fef2c0abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef23fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c0aa9d0 0x7fef2c0b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef23fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c0aa9d0 0x7fef2c0b4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:35104/0 (socket says 172.21.15.140:35104) 2022-07-21T01:55:50.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef2bfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef2c143f60 0x7fef2c0abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:44118/0 (socket says 172.21.15.140:44118) 2022-07-21T01:55:50.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef23fff700 1 -- 172.21.15.140:0/2428715274 learned_addr learned my addr 172.21.15.140:0/2428715274 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:50.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef23fff700 1 -- 172.21.15.140:0/2428715274 >> v2:172.21.15.152:3300/0 conn(0x7fef2c143f60 msgr2=0x7fef2c0abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:50.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef23fff700 1 --2- 172.21.15.140:0/2428715274 >> v2:172.21.15.152:3300/0 conn(0x7fef2c143f60 0x7fef2c0abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef23fff700 1 -- 172.21.15.140:0/2428715274 >> v1:172.21.15.152:6789/0 conn(0x7fef2c0b58f0 legacy=0x7fef2c0b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef23fff700 1 -- 172.21.15.140:0/2428715274 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c0b43f0 con 0x7fef2c0aa9d0 2022-07-21T01:55:50.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef2bfff700 1 --2- 172.21.15.140:0/2428715274 >> v2:172.21.15.152:3300/0 conn(0x7fef2c143f60 0x7fef2c0abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:50.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef23fff700 1 --2- 172.21.15.140:0/2428715274 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c0aa9d0 0x7fef2c0b4b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fef14002f70 tx=0x7fef14006eb0).ready entity=mon.2 client_cookie=fa354dd71a419995 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:50.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef2affd700 1 -- 172.21.15.140:0/2428715274 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef1400d570 con 0x7fef2c0aa9d0 2022-07-21T01:55:50.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef2affd700 1 -- 172.21.15.140:0/2428715274 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef14000e90 con 0x7fef2c0aa9d0 2022-07-21T01:55:50.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef2affd700 1 -- 172.21.15.140:0/2428715274 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef1400d900 con 0x7fef2c0aa9d0 2022-07-21T01:55:50.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 -- 172.21.15.140:0/2428715274 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c0aa9d0 msgr2=0x7fef2c0b4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 --2- 172.21.15.140:0/2428715274 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c0aa9d0 0x7fef2c0b4b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fef14002f70 tx=0x7fef14006eb0).stop 2022-07-21T01:55:50.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 -- 172.21.15.140:0/2428715274 shutdown_connections 2022-07-21T01:55:50.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 --2- 172.21.15.140:0/2428715274 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c0aa9d0 0x7fef2c0b4b90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 --2- 172.21.15.140:0/2428715274 >> v2:172.21.15.152:3300/0 conn(0x7fef2c143f60 0x7fef2c0abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 -- 172.21.15.140:0/2428715274 >> 172.21.15.140:0/2428715274 conn(0x7fef2c0632b0 msgr2=0x7fef2c063690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:50.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.138+0000 7fef3095c700 1 -- 172.21.15.140:0/2428715274 shutdown_connections 2022-07-21T01:55:50.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef3095c700 1 -- 172.21.15.140:0/2428715274 wait complete. 2022-07-21T01:55:50.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef3095c700 1 Processor -- start 2022-07-21T01:55:50.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef3095c700 1 -- start start 2022-07-21T01:55:50.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef3095c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef2c0aa9d0 0x7fef2c060b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef3095c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef2c0b58f0 0x7fef2c153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef3095c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c143f60 0x7fef2c14fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef3095c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c153450 con 0x7fef2c0b58f0 2022-07-21T01:55:50.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef3095c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c1536f0 con 0x7fef2c0aa9d0 2022-07-21T01:55:50.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef3095c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c1446d0 con 0x7fef2c143f60 2022-07-21T01:55:50.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef23fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef2c0b58f0 0x7fef2c153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef23fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef2c0b58f0 0x7fef2c153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38382/0 (socket says 172.21.15.140:38382) 2022-07-21T01:55:50.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef23fff700 1 -- 172.21.15.140:0/3096885602 learned_addr learned my addr 172.21.15.140:0/3096885602 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:50.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef2bfff700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c143f60 0x7fef2c14fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef2b7fe700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef2c0aa9d0 0x7fef2c060b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef23fff700 1 -- 172.21.15.140:0/3096885602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c143f60 msgr2=0x7fef2c14fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef23fff700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c143f60 0x7fef2c14fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef23fff700 1 -- 172.21.15.140:0/3096885602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef2c0aa9d0 msgr2=0x7fef2c060b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef23fff700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef2c0aa9d0 0x7fef2c060b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef23fff700 1 -- 172.21.15.140:0/3096885602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef24009470 con 0x7fef2c0b58f0 2022-07-21T01:55:50.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef23fff700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef2c0b58f0 0x7fef2c153b90 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7fef14002f70 tx=0x7fef14015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:50.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.142+0000 7fef297fa700 1 -- 172.21.15.140:0/3096885602 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef140118a0 con 0x7fef2c0b58f0 2022-07-21T01:55:50.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.146+0000 7fef297fa700 1 -- 172.21.15.140:0/3096885602 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef14011a00 con 0x7fef2c0b58f0 2022-07-21T01:55:50.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.146+0000 7fef3095c700 1 -- 172.21.15.140:0/3096885602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1400b1e0 con 0x7fef2c0b58f0 2022-07-21T01:55:50.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.146+0000 7fef297fa700 1 -- 172.21.15.140:0/3096885602 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef14011d60 con 0x7fef2c0b58f0 2022-07-21T01:55:50.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.146+0000 7fef3095c700 1 -- 172.21.15.140:0/3096885602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2c150250 con 0x7fef2c0b58f0 2022-07-21T01:55:50.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.146+0000 7fef3095c700 1 -- 172.21.15.140:0/3096885602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2c0b4690 con 0x7fef2c0b58f0 2022-07-21T01:55:50.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.146+0000 7fef297fa700 1 -- 172.21.15.140:0/3096885602 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fef14025030 con 0x7fef2c0b58f0 2022-07-21T01:55:50.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.146+0000 7fef297fa700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef040539e0 0x7fef04055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.146+0000 7fef297fa700 1 -- 172.21.15.140:0/3096885602 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fef1406d1b0 con 0x7fef2c0b58f0 2022-07-21T01:55:50.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.146+0000 7fef2b7fe700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef040539e0 0x7fef04055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.146+0000 7fef2b7fe700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef040539e0 0x7fef04055ea0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fef1c00d000 tx=0x7fef1c000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:50.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.154+0000 7fef297fa700 1 -- 172.21.15.140:0/3096885602 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fef14044000 con 0x7fef2c0b58f0 2022-07-21T01:55:50.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.282+0000 7fef3095c700 1 -- 172.21.15.140:0/3096885602 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fef2c0574e0 con 0x7fef2c0b58f0 2022-07-21T01:55:50.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.282+0000 7fef297fa700 1 -- 172.21.15.140:0/3096885602 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+10 (secure 0 0 0) 0x7fef140230a0 con 0x7fef2c0b58f0 2022-07-21T01:55:50.286 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:55:50.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 -- 172.21.15.140:0/3096885602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef040539e0 msgr2=0x7fef04055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef040539e0 0x7fef04055ea0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fef1c00d000 tx=0x7fef1c000bc0).stop 2022-07-21T01:55:50.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 -- 172.21.15.140:0/3096885602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef2c0b58f0 msgr2=0x7fef2c153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef2c0b58f0 0x7fef2c153b90 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7fef14002f70 tx=0x7fef14015000).stop 2022-07-21T01:55:50.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 -- 172.21.15.140:0/3096885602 shutdown_connections 2022-07-21T01:55:50.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef040539e0 0x7fef04055ea0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef2c143f60 0x7fef2c14fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef2c0b58f0 0x7fef2c153b90 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 --2- 172.21.15.140:0/3096885602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef2c0aa9d0 0x7fef2c060b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 -- 172.21.15.140:0/3096885602 >> 172.21.15.140:0/3096885602 conn(0x7fef2c0632b0 msgr2=0x7fef2c1409f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:50.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 -- 172.21.15.140:0/3096885602 shutdown_connections 2022-07-21T01:55:50.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.286+0000 7fef3095c700 1 -- 172.21.15.140:0/3096885602 wait complete. 2022-07-21T01:55:50.299 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:55:50.300 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:50.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.394+0000 7f6391f63700 1 Processor -- start 2022-07-21T01:55:50.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.394+0000 7f6391f63700 1 -- start start 2022-07-21T01:55:50.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f638c0b1e00 0x7f638c14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f638c0acef0 0x7f638c146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f638c05d480 con 0x7f638c05c3f0 2022-07-21T01:55:50.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c0b21e0 con 0x7f638c0b1e00 2022-07-21T01:55:50.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638c05cf40 con 0x7f638c0acef0 2022-07-21T01:55:50.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638affd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f638c05c3f0 0x7f638c0ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52534/0 (socket says 172.21.15.140:52534) 2022-07-21T01:55:50.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638affd700 1 -- 172.21.15.140:0/4070262042 learned_addr learned my addr 172.21.15.140:0/4070262042 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:50.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638b7fe700 1 --2- 172.21.15.140:0/4070262042 >> v2:172.21.15.140:3300/0 conn(0x7f638c0b1e00 0x7f638c14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638bfff700 1 --2- 172.21.15.140:0/4070262042 >> v2:172.21.15.152:3300/0 conn(0x7f638c0acef0 0x7f638c146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638a7fc700 1 -- 172.21.15.140:0/4070262042 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153272139 0 0) 0x7f638c05d480 con 0x7f638c05c3f0 2022-07-21T01:55:50.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638a7fc700 1 -- 172.21.15.140:0/4070262042 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f636c003260 con 0x7f638c05c3f0 2022-07-21T01:55:50.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638b7fe700 1 -- 172.21.15.140:0/4070262042 >> v2:172.21.15.152:3300/0 conn(0x7f638c0acef0 msgr2=0x7f638c146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:50.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638b7fe700 1 --2- 172.21.15.140:0/4070262042 >> v2:172.21.15.152:3300/0 conn(0x7f638c0acef0 0x7f638c146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638b7fe700 1 -- 172.21.15.140:0/4070262042 >> v1:172.21.15.140:6789/0 conn(0x7f638c05c3f0 legacy=0x7f638c0ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638b7fe700 1 -- 172.21.15.140:0/4070262042 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c05d1e0 con 0x7f638c0b1e00 2022-07-21T01:55:50.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638b7fe700 1 --2- 172.21.15.140:0/4070262042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f638c0b1e00 0x7f638c14a6d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f6374002f70 tx=0x7f6374006df0).ready entity=mon.0 client_cookie=d9f389cb9feca906 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:50.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638a7fc700 1 -- 172.21.15.140:0/4070262042 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637400d4f0 con 0x7f638c0b1e00 2022-07-21T01:55:50.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638a7fc700 1 -- 172.21.15.140:0/4070262042 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6374000e90 con 0x7f638c0b1e00 2022-07-21T01:55:50.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f638a7fc700 1 -- 172.21.15.140:0/4070262042 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f637400d840 con 0x7f638c0b1e00 2022-07-21T01:55:50.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 -- 172.21.15.140:0/4070262042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f638c0b1e00 msgr2=0x7f638c14a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 --2- 172.21.15.140:0/4070262042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f638c0b1e00 0x7f638c14a6d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f6374002f70 tx=0x7f6374006df0).stop 2022-07-21T01:55:50.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 -- 172.21.15.140:0/4070262042 shutdown_connections 2022-07-21T01:55:50.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 --2- 172.21.15.140:0/4070262042 >> v2:172.21.15.152:3300/0 conn(0x7f638c0acef0 0x7f638c146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 --2- 172.21.15.140:0/4070262042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f638c0b1e00 0x7f638c14a6d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 -- 172.21.15.140:0/4070262042 >> 172.21.15.140:0/4070262042 conn(0x7f638c0611c0 msgr2=0x7f638c062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:50.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 -- 172.21.15.140:0/4070262042 shutdown_connections 2022-07-21T01:55:50.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.398+0000 7f6391f63700 1 -- 172.21.15.140:0/4070262042 wait complete. 2022-07-21T01:55:50.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f6391f63700 1 Processor -- start 2022-07-21T01:55:50.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f6391f63700 1 -- start start 2022-07-21T01:55:50.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f6391f63700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f638c0acef0 0x7f638c14a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f6391f63700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f638c149190 0x7f638c146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f6391f63700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f638c156930 0x7f638c15a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f6391f63700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c149c00 con 0x7f638c156930 2022-07-21T01:55:50.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f6391f63700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c149df0 con 0x7f638c149190 2022-07-21T01:55:50.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f6391f63700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638c15a000 con 0x7f638c0acef0 2022-07-21T01:55:50.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f638bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f638c156930 0x7f638c15a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f638bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f638c156930 0x7f638c15a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38398/0 (socket says 172.21.15.140:38398) 2022-07-21T01:55:50.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f638bfff700 1 -- 172.21.15.140:0/1328822704 learned_addr learned my addr 172.21.15.140:0/1328822704 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:50.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f638b7fe700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f638c0acef0 0x7f638c14a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f638affd700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f638c149190 0x7f638c146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f638bfff700 1 -- 172.21.15.140:0/1328822704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f638c0acef0 msgr2=0x7f638c14a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f638bfff700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f638c0acef0 0x7f638c14a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f638bfff700 1 -- 172.21.15.140:0/1328822704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f638c149190 msgr2=0x7f638c146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f638bfff700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f638c149190 0x7f638c146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.402+0000 7f638bfff700 1 -- 172.21.15.140:0/1328822704 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637400b180 con 0x7f638c156930 2022-07-21T01:55:50.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f638bfff700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f638c156930 0x7f638c15a4a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f638400c000 tx=0x7f6384006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:50.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f6388ff9700 1 -- 172.21.15.140:0/1328822704 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6384011750 con 0x7f638c156930 2022-07-21T01:55:50.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f6388ff9700 1 -- 172.21.15.140:0/1328822704 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63840118b0 con 0x7f638c156930 2022-07-21T01:55:50.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f6391f63700 1 -- 172.21.15.140:0/1328822704 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c156db0 con 0x7f638c156930 2022-07-21T01:55:50.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f6388ff9700 1 -- 172.21.15.140:0/1328822704 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6384011c10 con 0x7f638c156930 2022-07-21T01:55:50.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f6391f63700 1 -- 172.21.15.140:0/1328822704 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c15abb0 con 0x7f638c156930 2022-07-21T01:55:50.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f6391f63700 1 -- 172.21.15.140:0/1328822704 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c149fb0 con 0x7f638c156930 2022-07-21T01:55:50.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f6388ff9700 1 -- 172.21.15.140:0/1328822704 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6384011d70 con 0x7f638c156930 2022-07-21T01:55:50.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f6388ff9700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6370053b00 0x7f6370055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:50.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f6388ff9700 1 -- 172.21.15.140:0/1328822704 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f638406c1d0 con 0x7f638c156930 2022-07-21T01:55:50.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f638b7fe700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6370053b00 0x7f6370055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:50.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.406+0000 7f638b7fe700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6370053b00 0x7f6370055fc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f637401a000 tx=0x7f6374014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:50.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.414+0000 7f6388ff9700 1 -- 172.21.15.140:0/1328822704 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f63840430f0 con 0x7f638c156930 2022-07-21T01:55:50.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.574+0000 7f6391f63700 1 -- 172.21.15.140:0/1328822704 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f638c159e10 con 0x7f638c156930 2022-07-21T01:55:50.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.574+0000 7f6388ff9700 1 -- 172.21.15.140:0/1328822704 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f6384046710 con 0x7f638c156930 2022-07-21T01:55:50.577 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:50.577 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/518920349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":518920349},{"type":"v1","addr":"172.21.15.152:6833","nonce":518920349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:43.343194+0000","modified":"2022-07-21T01:55:44.346904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2478727503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2478727503},{"type":"v1","addr":"172.21.15.140:6835","nonce":2478727503}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-07-21T01:55:50.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.574+0000 7f6391f63700 1 -- 172.21.15.140:0/1328822704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6370053b00 msgr2=0x7f6370055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.574+0000 7f6391f63700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6370053b00 0x7f6370055fc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f637401a000 tx=0x7f6374014000).stop 2022-07-21T01:55:50.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.574+0000 7f6391f63700 1 -- 172.21.15.140:0/1328822704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f638c156930 msgr2=0x7f638c15a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:50.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.574+0000 7f6391f63700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f638c156930 0x7f638c15a4a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f638400c000 tx=0x7f6384006df0).stop 2022-07-21T01:55:50.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.578+0000 7f6391f63700 1 -- 172.21.15.140:0/1328822704 shutdown_connections 2022-07-21T01:55:50.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.578+0000 7f6391f63700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6370053b00 0x7f6370055fc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.578+0000 7f6391f63700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f638c156930 0x7f638c15a4a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.578+0000 7f6391f63700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f638c149190 0x7f638c146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.578+0000 7f6391f63700 1 --2- 172.21.15.140:0/1328822704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f638c0acef0 0x7f638c14a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:50.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.578+0000 7f6391f63700 1 -- 172.21.15.140:0/1328822704 >> 172.21.15.140:0/1328822704 conn(0x7f638c05c3f0 msgr2=0x7f638c069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:50.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.578+0000 7f6391f63700 1 -- 172.21.15.140:0/1328822704 shutdown_connections 2022-07-21T01:55:50.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:50.578+0000 7f6391f63700 1 -- 172.21.15.140:0/1328822704 wait complete. 2022-07-21T01:55:50.583 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 42 2022-07-21T01:55:50.793 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:55:43.343194+0000', 'modified': '2022-07-21T01:55:44.346904+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7002}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7002': {'gid': 7002, 'name': 'a', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/2478727503', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 2478727503}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 2478727503}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:55:50.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:55:50.795 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:55:50.910 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":7002,"want_state":"up:active","state":"up:active","rank_uptime":7.5567736429999997,"mdsmap_epoch":42,"osdmap_epoch":94,"osdmap_epoch_barrier":93,"uptime":11.728888264} 2022-07-21T01:55:50.911 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 7002, "mdsmap_epoch": 42, "osdmap_epoch": 94, "osdmap_epoch_barrier": 93, "rank_uptime": 7.556773643, "state": "up:active", "uptime": 11.728888264, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:55:50.912 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:55:50.912 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-07-21T01:55:50.913 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-07-21T01:55:50.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:50.914 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:50.923 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:55:50.981 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:50.981 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:50.996 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:50.997 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:50.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:51.008 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:55:51.015 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:55:51.016 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:51.019 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:55:51.035 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:51.036 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:51.090 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:51.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2022-07-21T01:55:51.087+0000 7f5e87d18080 -1 init, newargv = 0x55e912299410 newargc=9 2022-07-21T01:55:51.091 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22564]: starting ceph client 2022-07-21T01:55:51.092 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:51.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:51.093 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:55:51.099 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22564]: starting fuse 2022-07-21T01:55:52.199 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:55:52.218 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:55:52.219 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:55:52.275 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:55:52.276 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:55:52.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:52.278 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:55:52.283 INFO:teuthology.orchestra.run.smithi192.stdout:52 2022-07-21T01:55:52.283 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:55:52.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:52.284 DEBUG:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2022-07-21T01:55:52.285 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2022-07-21T01:55:52.286 DEBUG:teuthology.orchestra.run.smithi192:> import re 2022-07-21T01:55:52.286 DEBUG:teuthology.orchestra.run.smithi192:> import os 2022-07-21T01:55:52.286 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2022-07-21T01:55:52.287 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:55:52.287 DEBUG:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2022-07-21T01:55:52.288 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-07-21T01:55:52.288 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2022-07-21T01:55:52.289 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:55:52.289 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2022-07-21T01:55:52.290 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2022-07-21T01:55:52.290 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2022-07-21T01:55:52.291 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2022-07-21T01:55:52.291 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:55:52.292 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2022-07-21T01:55:52.292 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:55:52.293 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:55:52.293 DEBUG:teuthology.orchestra.run.smithi192:> return f 2022-07-21T01:55:52.294 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:55:52.294 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:55:52.294 DEBUG:teuthology.orchestra.run.smithi192:> print(find_socket("client.1")) 2022-07-21T01:55:52.295 DEBUG:teuthology.orchestra.run.smithi192:> ' 2022-07-21T01:55:52.339 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.22564.asok 2022-07-21T01:55:52.344 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22564.asok 2022-07-21T01:55:52.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:52.345 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22564.asok status 2022-07-21T01:55:52.435 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2022-07-21T01:55:52.436 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2022-07-21T01:55:52.436 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:55:52.436 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:55:52.436 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2022-07-21T01:55:52.437 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2022-07-21T01:55:52.437 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-07-21T01:55:52.437 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "22564", 2022-07-21T01:55:52.437 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2022-07-21T01:55:52.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:55:52.438 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2022-07-21T01:55:52.438 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2022-07-21T01:55:52.438 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 6392, 2022-07-21T01:55:52.439 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2022-07-21T01:55:52.439 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2022-07-21T01:55:52.439 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2022-07-21T01:55:52.439 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 6392 2022-07-21T01:55:52.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:55:52.440 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:55:52.440 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:55:52.441 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:55:52.441 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2428209736 2022-07-21T01:55:52.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2022-07-21T01:55:52.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:55:52.442 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:55:52.442 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:55:52.442 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:55:52.442 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2428209736 2022-07-21T01:55:52.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:55:52.443 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.6392 172.21.15.192:0/2428209736", 2022-07-21T01:55:52.443 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/2428209736", 2022-07-21T01:55:52.443 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2022-07-21T01:55:52.443 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 42, 2022-07-21T01:55:52.444 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 94, 2022-07-21T01:55:52.444 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:55:52.444 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false, 2022-07-21T01:55:52.444 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2022-07-21T01:55:52.445 INFO:teuthology.orchestra.run.smithi192.stdout:} 2022-07-21T01:55:52.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:52.446 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:52.447 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:52.454 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:55:52.455 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:55:52.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:52.456 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:52.472 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-07-21T01:55:52.472 DEBUG:teuthology.orchestra.run.smithi191:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir -p foo 2022-07-21T01:55:52.499 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-07-21T01:55:52.499 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:55:52.500 DEBUG:teuthology.orchestra.run.smithi191:> import stat 2022-07-21T01:55:52.500 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:55:52.501 DEBUG:teuthology.orchestra.run.smithi191:> print(os.stat("/home/ubuntu/cephtest/mnt.0/foo").st_ino) 2022-07-21T01:55:52.501 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:55:52.582 INFO:teuthology.orchestra.run.smithi191.stdout:1099511627776 2022-07-21T01:55:52.772 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:52.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.878+0000 7f7017fff700 1 Processor -- start 2022-07-21T01:55:52.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.878+0000 7f7017fff700 1 -- start start 2022-07-21T01:55:52.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f70180b5650 0x7f70180b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:52.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f70180af8e0 0x7f701805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:52.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7018147010 con 0x7f70180adff0 2022-07-21T01:55:52.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70181472b0 con 0x7f70180af8e0 2022-07-21T01:55:52.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7018146d40 con 0x7f70180b5650 2022-07-21T01:55:52.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f70167fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f70180adff0 0x7f7018147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52548/0 (socket says 172.21.15.140:52548) 2022-07-21T01:55:52.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f70167fc700 1 -- 172.21.15.140:0/18638055 learned_addr learned my addr 172.21.15.140:0/18638055 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:52.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f70177fe700 1 --2- 172.21.15.140:0/18638055 >> v2:172.21.15.140:3300/0 conn(0x7f70180af8e0 0x7f701805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:52.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7016ffd700 1 --2- 172.21.15.140:0/18638055 >> v2:172.21.15.152:3300/0 conn(0x7f70180b5650 0x7f70180b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:52.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7015ffb700 1 -- 172.21.15.140:0/18638055 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486749270 0 0) 0x7f7018147010 con 0x7f70180adff0 2022-07-21T01:55:52.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7015ffb700 1 -- 172.21.15.140:0/18638055 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff8003260 con 0x7f70180adff0 2022-07-21T01:55:52.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7016ffd700 1 -- 172.21.15.140:0/18638055 >> v2:172.21.15.140:3300/0 conn(0x7f70180af8e0 msgr2=0x7f701805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:52.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7016ffd700 1 --2- 172.21.15.140:0/18638055 >> v2:172.21.15.140:3300/0 conn(0x7f70180af8e0 0x7f701805bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:52.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7016ffd700 1 -- 172.21.15.140:0/18638055 >> v1:172.21.15.140:6789/0 conn(0x7f70180adff0 legacy=0x7f7018147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:52.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7016ffd700 1 -- 172.21.15.140:0/18638055 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70180acae0 con 0x7f70180b5650 2022-07-21T01:55:52.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f70177fe700 1 --2- 172.21.15.140:0/18638055 >> v2:172.21.15.140:3300/0 conn(0x7f70180af8e0 0x7f701805bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:52.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7016ffd700 1 --2- 172.21.15.140:0/18638055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70180b5650 0x7f70180b5a30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f7008002f70 tx=0x7f7008006df0).ready entity=mon.1 client_cookie=6f9b228b2586d7e7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:52.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7015ffb700 1 -- 172.21.15.140:0/18638055 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f700800d4f0 con 0x7f70180b5650 2022-07-21T01:55:52.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7015ffb700 1 -- 172.21.15.140:0/18638055 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7008000e90 con 0x7f70180b5650 2022-07-21T01:55:52.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7015ffb700 1 -- 172.21.15.140:0/18638055 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f700800d840 con 0x7f70180b5650 2022-07-21T01:55:52.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 -- 172.21.15.140:0/18638055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70180b5650 msgr2=0x7f70180b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:52.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 --2- 172.21.15.140:0/18638055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70180b5650 0x7f70180b5a30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f7008002f70 tx=0x7f7008006df0).stop 2022-07-21T01:55:52.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 -- 172.21.15.140:0/18638055 shutdown_connections 2022-07-21T01:55:52.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 --2- 172.21.15.140:0/18638055 >> v2:172.21.15.140:3300/0 conn(0x7f70180af8e0 0x7f701805bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:52.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 --2- 172.21.15.140:0/18638055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70180b5650 0x7f70180b5a30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:52.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 -- 172.21.15.140:0/18638055 >> 172.21.15.140:0/18638055 conn(0x7f70180634e0 msgr2=0x7f70180b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:52.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 -- 172.21.15.140:0/18638055 shutdown_connections 2022-07-21T01:55:52.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 -- 172.21.15.140:0/18638055 wait complete. 2022-07-21T01:55:52.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.882+0000 7f7017fff700 1 Processor -- start 2022-07-21T01:55:52.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f7017fff700 1 -- start start 2022-07-21T01:55:52.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f7017fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70180adff0 0x7f7018110e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:52.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f7017fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70180af8e0 0x7f7018142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:52.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f7017fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70180b5650 0x7f70181433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:52.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f7017fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7018110170 con 0x7f70180af8e0 2022-07-21T01:55:52.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f7017fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70181438c0 con 0x7f70180b5650 2022-07-21T01:55:52.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f7017fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70180676b0 con 0x7f70180adff0 2022-07-21T01:55:52.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f70167fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70180af8e0 0x7f7018142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:52.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f70167fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70180af8e0 0x7f7018142ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38410/0 (socket says 172.21.15.140:38410) 2022-07-21T01:55:52.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f70167fc700 1 -- 172.21.15.140:0/1713173180 learned_addr learned my addr 172.21.15.140:0/1713173180 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:52.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f70177fe700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70180b5650 0x7f70181433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:52.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f7016ffd700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70180adff0 0x7f7018110e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:52.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f70167fc700 1 -- 172.21.15.140:0/1713173180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70180adff0 msgr2=0x7f7018110e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:52.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f70167fc700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70180adff0 0x7f7018110e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:52.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f70167fc700 1 -- 172.21.15.140:0/1713173180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70180b5650 msgr2=0x7f70181433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:52.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f70167fc700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70180b5650 0x7f70181433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:52.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f70167fc700 1 -- 172.21.15.140:0/1713173180 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701000b180 con 0x7f70180af8e0 2022-07-21T01:55:52.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f70167fc700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70180af8e0 0x7f7018142ee0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f7000001610 tx=0x7f7000007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:52.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f700ffff700 1 -- 172.21.15.140:0/1713173180 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7000005a70 con 0x7f70180af8e0 2022-07-21T01:55:52.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f700ffff700 1 -- 172.21.15.140:0/1713173180 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7000005bd0 con 0x7f70180af8e0 2022-07-21T01:55:52.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f7017fff700 1 -- 172.21.15.140:0/1713173180 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700800b180 con 0x7f70180af8e0 2022-07-21T01:55:52.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f700ffff700 1 -- 172.21.15.140:0/1713173180 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70000070d0 con 0x7f70180af8e0 2022-07-21T01:55:52.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.886+0000 7f7017fff700 1 -- 172.21.15.140:0/1713173180 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7018067410 con 0x7f70180af8e0 2022-07-21T01:55:52.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.890+0000 7f7017fff700 1 -- 172.21.15.140:0/1713173180 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7018066f90 con 0x7f70180af8e0 2022-07-21T01:55:52.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.890+0000 7f700ffff700 1 -- 172.21.15.140:0/1713173180 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7000010070 con 0x7f70180af8e0 2022-07-21T01:55:52.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.890+0000 7f700ffff700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ffc053a70 0x7f6ffc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:52.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.890+0000 7f700ffff700 1 -- 172.21.15.140:0/1713173180 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f700006b8b0 con 0x7f70180af8e0 2022-07-21T01:55:52.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.890+0000 7f7016ffd700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ffc053a70 0x7f6ffc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:52.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.890+0000 7f7016ffd700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ffc053a70 0x7f6ffc055f30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f700801a000 tx=0x7f7008014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:52.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:52.898+0000 7f700ffff700 1 -- 172.21.15.140:0/1713173180 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f70000426c0 con 0x7f70180af8e0 2022-07-21T01:55:53.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 -- 172.21.15.140:0/1713173180 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f701805c280 con 0x7f70180af8e0 2022-07-21T01:55:53.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f700ffff700 1 -- 172.21.15.140:0/1713173180 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f7000045ce0 con 0x7f70180af8e0 2022-07-21T01:55:53.058 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:53.059 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/518920349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":518920349},{"type":"v1","addr":"172.21.15.152:6833","nonce":518920349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:43.343194+0000","modified":"2022-07-21T01:55:44.346904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2478727503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2478727503},{"type":"v1","addr":"172.21.15.140:6835","nonce":2478727503}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-07-21T01:55:53.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 -- 172.21.15.140:0/1713173180 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ffc053a70 msgr2=0x7f6ffc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:53.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ffc053a70 0x7f6ffc055f30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f700801a000 tx=0x7f7008014000).stop 2022-07-21T01:55:53.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 -- 172.21.15.140:0/1713173180 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70180af8e0 msgr2=0x7f7018142ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:53.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70180af8e0 0x7f7018142ee0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f7000001610 tx=0x7f7000007c70).stop 2022-07-21T01:55:53.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 -- 172.21.15.140:0/1713173180 shutdown_connections 2022-07-21T01:55:53.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6ffc053a70 0x7f6ffc055f30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:53.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70180b5650 0x7f70181433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:53.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f70180af8e0 0x7f7018142ee0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:53.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 --2- 172.21.15.140:0/1713173180 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70180adff0 0x7f7018110e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:53.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 -- 172.21.15.140:0/1713173180 >> 172.21.15.140:0/1713173180 conn(0x7f70180634e0 msgr2=0x7f70181442a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:53.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 -- 172.21.15.140:0/1713173180 shutdown_connections 2022-07-21T01:55:53.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.054+0000 7f7017fff700 1 -- 172.21.15.140:0/1713173180 wait complete. 2022-07-21T01:55:53.067 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 42 2022-07-21T01:55:53.074 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 0x10000000000 2022-07-21T01:55:53.179 INFO:teuthology.orchestra.run.smithi140.stdout:{"path":"/foo","ino":1099511627776,"rdev":0,"ctime":"2022-07-21T01:55:52.490923+0000","btime":"2022-07-21T01:55:52.490923+0000","mode":16877,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2022-07-21T01:55:52.490923+0000","atime":"2022-07-21T01:55:52.490923+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":6356,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":6356,"want_loner":6356,"mds_caps_wanted":[]} 2022-07-21T01:55:53.181 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2022-07-21T01:55:52.490923+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "backtrace_version": 2, "btime": "2022-07-21T01:55:52.490923+0000", "change_attr": 0, "client_caps": [ { "client_id": 6356, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2022-07-21T01:55:52.490923+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "flocklock": {}, "gid": 0, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 6356, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2022-07-21T01:55:52.490923+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/foo", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": 6356, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "xattrs": [] } 2022-07-21T01:55:53.182 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:55:53.183 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:53.204 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:53.227 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:53.248 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:53.268 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:55:53.290 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:55:53.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:53.291 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:53.292 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:53.300 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:55:53.301 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:55:53.302 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:55:53.302 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:55:53.303 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:53.360 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23039]: fuse finished with error 0 and tester_r 0 2022-07-21T01:55:53.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:53.363 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:53.363 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:53.370 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:55:53.371 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:55:53.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:55:53.451 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:55:53.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:53.459 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:53.460 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:53.468 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:55:53.469 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:55:53.470 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2022-07-21T01:55:53.470 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:55:53.470 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:53.486 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22564]: fuse finished with error 0 and tester_r 0 2022-07-21T01:55:53.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:53.488 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:53.489 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:53.496 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2022-07-21T01:55:53.497 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:55:53.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:55:53.615 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:55:53.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:55:53.623 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:55:53.624 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:55:53.631 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:55:53.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:53.633 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:55:53.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:55:53.634 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:55:53.681 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:55:53.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:55:53.683 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:55:53.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.790+0000 7f749ec22700 1 Processor -- start 2022-07-21T01:55:53.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.790+0000 7f749ec22700 1 -- start start 2022-07-21T01:55:53.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749ec22700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f749805c3f0 0x7f749814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:53.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749ec22700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749805d480 con 0x7f74980b1e00 2022-07-21T01:55:53.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749ec22700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74980b21e0 con 0x7f74980acef0 2022-07-21T01:55:53.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749ec22700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749805cf40 con 0x7f749805c3f0 2022-07-21T01:55:53.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749c9be700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f74980b1e00 0x7f74980ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52564/0 (socket says 172.21.15.140:52564) 2022-07-21T01:55:53.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749c9be700 1 -- 172.21.15.140:0/223286968 learned_addr learned my addr 172.21.15.140:0/223286968 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:53.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f7497fff700 1 --2- 172.21.15.140:0/223286968 >> v2:172.21.15.152:3300/0 conn(0x7f749805c3f0 0x7f749814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:53.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f74977fe700 1 -- 172.21.15.140:0/223286968 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181520174 0 0) 0x7f749805d480 con 0x7f74980b1e00 2022-07-21T01:55:53.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f74977fe700 1 -- 172.21.15.140:0/223286968 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7478003260 con 0x7f74980b1e00 2022-07-21T01:55:53.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f74977fe700 1 -- 172.21.15.140:0/223286968 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55492972 0 0) 0x7f74980b21e0 con 0x7f74980acef0 2022-07-21T01:55:53.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f74977fe700 1 -- 172.21.15.140:0/223286968 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f749805d480 con 0x7f74980acef0 2022-07-21T01:55:53.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f7497fff700 1 -- 172.21.15.140:0/223286968 >> v1:172.21.15.152:6789/0 conn(0x7f74980acef0 legacy=0x7f7498146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:53.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f7497fff700 1 -- 172.21.15.140:0/223286968 >> v1:172.21.15.140:6789/0 conn(0x7f74980b1e00 legacy=0x7f74980ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:53.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f7497fff700 1 -- 172.21.15.140:0/223286968 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749805d1e0 con 0x7f749805c3f0 2022-07-21T01:55:53.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f7497fff700 1 --2- 172.21.15.140:0/223286968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f749805c3f0 0x7f749814a6d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f74880032f0 tx=0x7f74880091d0).ready entity=mon.1 client_cookie=2c42c794f76fa0c5 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:53.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f74977fe700 1 -- 172.21.15.140:0/223286968 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f748800b600 con 0x7f749805c3f0 2022-07-21T01:55:53.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f74977fe700 1 -- 172.21.15.140:0/223286968 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748800b760 con 0x7f749805c3f0 2022-07-21T01:55:53.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f74977fe700 1 -- 172.21.15.140:0/223286968 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f748800b8d0 con 0x7f749805c3f0 2022-07-21T01:55:53.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749ec22700 1 -- 172.21.15.140:0/223286968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f749805c3f0 msgr2=0x7f749814a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:53.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749ec22700 1 --2- 172.21.15.140:0/223286968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f749805c3f0 0x7f749814a6d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f74880032f0 tx=0x7f74880091d0).stop 2022-07-21T01:55:53.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749ec22700 1 -- 172.21.15.140:0/223286968 shutdown_connections 2022-07-21T01:55:53.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749ec22700 1 --2- 172.21.15.140:0/223286968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f749805c3f0 0x7f749814a6d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:53.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749ec22700 1 -- 172.21.15.140:0/223286968 >> 172.21.15.140:0/223286968 conn(0x7f74980611c0 msgr2=0x7f7498062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:53.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749ec22700 1 -- 172.21.15.140:0/223286968 shutdown_connections 2022-07-21T01:55:53.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.794+0000 7f749ec22700 1 -- 172.21.15.140:0/223286968 wait complete. 2022-07-21T01:55:53.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.798+0000 7f749ec22700 1 Processor -- start 2022-07-21T01:55:53.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.798+0000 7f749ec22700 1 -- start start 2022-07-21T01:55:53.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.798+0000 7f749ec22700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74980acef0 0x7f7498063a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:53.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.798+0000 7f749ec22700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74980b1e00 0x7f749814e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:53.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749ec22700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74980b5a20 0x7f7498156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:53.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749ec22700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749815a870 con 0x7f74980acef0 2022-07-21T01:55:53.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749ec22700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749815ad40 con 0x7f74980b5a20 2022-07-21T01:55:53.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749ec22700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7498063f70 con 0x7f74980b1e00 2022-07-21T01:55:53.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749c9be700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74980acef0 0x7f7498063a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:53.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749c9be700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74980acef0 0x7f7498063a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38422/0 (socket says 172.21.15.140:38422) 2022-07-21T01:55:53.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749c9be700 1 -- 172.21.15.140:0/1282357355 learned_addr learned my addr 172.21.15.140:0/1282357355 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:53.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f7497fff700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74980b1e00 0x7f749814e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:53.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749d1bf700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74980b5a20 0x7f7498156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:53.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749c9be700 1 -- 172.21.15.140:0/1282357355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74980b1e00 msgr2=0x7f749814e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:53.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749c9be700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74980b1e00 0x7f749814e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:53.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749c9be700 1 -- 172.21.15.140:0/1282357355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74980b5a20 msgr2=0x7f7498156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:53.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749c9be700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74980b5a20 0x7f7498156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:53.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749c9be700 1 -- 172.21.15.140:0/1282357355 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7488009470 con 0x7f74980acef0 2022-07-21T01:55:53.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749c9be700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74980acef0 0x7f7498063a80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f74980660b0 tx=0x7f7480007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:53.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f7495ffb700 1 -- 172.21.15.140:0/1282357355 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f748000ce00 con 0x7f74980acef0 2022-07-21T01:55:53.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f7495ffb700 1 -- 172.21.15.140:0/1282357355 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74800070d0 con 0x7f74980acef0 2022-07-21T01:55:53.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749ec22700 1 -- 172.21.15.140:0/1282357355 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749815aaa0 con 0x7f74980acef0 2022-07-21T01:55:53.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f7495ffb700 1 -- 172.21.15.140:0/1282357355 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7480007430 con 0x7f74980acef0 2022-07-21T01:55:53.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749ec22700 1 -- 172.21.15.140:0/1282357355 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749814e710 con 0x7f74980acef0 2022-07-21T01:55:53.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f749ec22700 1 -- 172.21.15.140:0/1282357355 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74980b5460 con 0x7f74980acef0 2022-07-21T01:55:53.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.802+0000 7f7495ffb700 1 -- 172.21.15.140:0/1282357355 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7480010070 con 0x7f74980acef0 2022-07-21T01:55:53.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.806+0000 7f7495ffb700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f747c053aa0 0x7f747c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:53.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.806+0000 7f7495ffb700 1 -- 172.21.15.140:0/1282357355 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f748006b480 con 0x7f74980acef0 2022-07-21T01:55:53.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.806+0000 7f7497fff700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f747c053aa0 0x7f747c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:53.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.806+0000 7f7497fff700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f747c053aa0 0x7f747c055f60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f7488019000 tx=0x7f7488016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:53.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.810+0000 7f7495ffb700 1 -- 172.21.15.140:0/1282357355 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f74980b5460 con 0x7f74980acef0 2022-07-21T01:55:53.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.938+0000 7f749ec22700 1 -- 172.21.15.140:0/1282357355 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f74980b5460 con 0x7f74980acef0 2022-07-21T01:55:53.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.942+0000 7f7495ffb700 1 -- 172.21.15.140:0/1282357355 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v94) v1 ==== 81+0+33744 (secure 0 0 0) 0x7f74980b5460 con 0x7f74980acef0 2022-07-21T01:55:53.946 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:53.946 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:55:53.947 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 94, 2022-07-21T01:55:53.947 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:55:53.947 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:55:53.948 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:55:44.346577+0000", 2022-07-21T01:55:53.948 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:55:53.948 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:55:53.948 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:55:53.949 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:55:53.949 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:55:53.949 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:55:53.949 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:55:53.950 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:55:53.950 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:55:53.950 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:53.951 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:55:53.951 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:55:53.951 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:55:53.951 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:55:53.952 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:55:53.952 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 18, 2022-07-21T01:55:53.952 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:55:53.952 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:55:53.953 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:55:53.953 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:55:53.953 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:55:53.953 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:53.954 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:55:53.954 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:55:53.955 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:55:53.955 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:53.956 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:53.956 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:53.957 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:53.957 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:53.957 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:53.957 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:53.958 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:53.958 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:55:53.959 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:55:53.960 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:55:53.960 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:55:53.961 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:55:53.961 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:53.962 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:53.962 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:53.963 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:53.963 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:53.964 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:53.964 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:53.965 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:53.965 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:55:53.966 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:53.966 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:53.967 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:53.968 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:53.968 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:53.969 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:53.969 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:53.970 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:53.970 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:53.971 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:53.971 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:53.972 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:53.972 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:53.972 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:53.973 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:53.973 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:53.973 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:53.974 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:53.975 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:53.975 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:53.976 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:53.976 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:53.977 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:53.977 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:53.978 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:53.979 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:53.979 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:53.980 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:53.980 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:53.981 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:53.981 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:53.982 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:53.982 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:53.983 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:53.984 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:53.984 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:53.985 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:53.985 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:53.986 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:55:53.986 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:55:53.987 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:53.987 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:53.988 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:55:53.988 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:53.989 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:53.989 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:53.990 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:55:53.990 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:55:53.991 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:55:53.992 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:53.992 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:53.993 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:53.993 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:53.994 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:53.994 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:53.995 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:53.995 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:53.996 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:55:53.996 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:55:53.997 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:55:53.997 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:55:53.998 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:55:53.998 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:53.999 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:54.000 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:54.000 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:54.001 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:54.001 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:54.002 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:54.002 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.003 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:55:54.003 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:54.004 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:54.004 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:54.005 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:54.005 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:54.006 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:54.007 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:54.007 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:54.008 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:54.008 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:54.009 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:54.009 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:54.010 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:54.010 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:54.011 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:54.012 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:54.013 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:54.013 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:54.014 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:54.014 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:54.015 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:54.015 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:54.016 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:54.016 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:54.017 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:54.018 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:54.018 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.019 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:54.019 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:54.020 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:54.020 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:54.021 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:54.021 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:54.021 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:54.021 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:54.022 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:54.022 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:54.022 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:54.023 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:55:54.023 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:54.023 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:55:54.024 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.024 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.024 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.024 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 17, 2022-07-21T01:55:54.024 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:55:54.025 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:55:41.913815+0000", 2022-07-21T01:55:54.025 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:54.025 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:54.026 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:54.026 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:54.026 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:54.027 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:54.027 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:54.027 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:54.028 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:55:54.028 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:55:54.028 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:55:54.028 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:55:54.029 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:55:54.029 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:54.029 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:54.029 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:54.030 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:54.030 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:54.030 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:54.030 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:54.031 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.031 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "93", 2022-07-21T01:55:54.031 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:54.031 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:54.032 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:54.032 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:54.032 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:54.032 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:54.033 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:54.033 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:54.033 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:54.033 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:54.034 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:54.034 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:54.034 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:54.034 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:54.035 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:54.035 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:54.035 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:54.035 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:54.036 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:54.036 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:54.036 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:54.037 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:54.037 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:54.037 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:54.037 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:54.038 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:54.038 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.038 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:54.038 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:54.039 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:54.039 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:54.039 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:54.039 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:54.040 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:54.040 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:54.040 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:54.040 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:54.041 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:54.041 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:55:54.041 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:55:54.041 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:55:54.042 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:55:54.042 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.042 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:54.042 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:55:54.043 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:55:54.043 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.043 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.043 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.044 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.044 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 18, 2022-07-21T01:55:54.044 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:55:54.044 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:55:42.343580+0000", 2022-07-21T01:55:54.045 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:55:54.045 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:55:54.045 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:55:54.046 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:55:54.046 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:55:54.046 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:55:54.046 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:55:54.047 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:55:54.047 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:55:54.047 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:55:54.047 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:55:54.048 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:55:54.048 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:55:54.048 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:55:54.048 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:55:54.049 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:55:54.049 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:55:54.049 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:55:54.049 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:55:54.050 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:55:54.050 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.050 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "94", 2022-07-21T01:55:54.050 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:55:54.051 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:55:54.051 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:55:54.051 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:55:54.051 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:55:54.051 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:55:54.052 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:55:54.052 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:55:54.052 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:55:54.052 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:55:54.053 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:55:54.053 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:55:54.053 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:55:54.053 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:55:54.054 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:55:54.054 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:55:54.054 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:55:54.055 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:55:54.055 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:55:54.055 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:55:54.056 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:55:54.056 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:55:54.056 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:55:54.056 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:55:54.057 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:55:54.057 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:55:54.057 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.057 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:55:54.058 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:55:54.058 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:55:54.058 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:55:54.058 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:55:54.058 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:55:54.059 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:55:54.059 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:55:54.059 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:55:54.059 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:55:54.060 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:55:54.060 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:55:54.060 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:55:54.060 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:55:54.061 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:55:54.061 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.061 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.061 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.062 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:54.062 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:55:54.062 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.062 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:55:54.063 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:55:54.063 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:54.063 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:54.063 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:54.063 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:54.064 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:54.064 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:54.064 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:54.064 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 91, 2022-07-21T01:55:54.065 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:54.065 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:54.065 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:54.065 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.066 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.066 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.066 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:55:54.066 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:54.066 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.067 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.067 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.067 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:55:54.067 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:54.068 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.068 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.068 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.068 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:54.069 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.069 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.069 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.069 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:55:54.069 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:54.070 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.070 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.070 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.070 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:55:54.071 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:54.071 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.071 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.071 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.072 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:54.072 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.072 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.072 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.073 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:55:54.073 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:54.073 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.073 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.074 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.074 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:55:54.074 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:54.074 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.075 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.075 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.075 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:54.075 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.075 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.076 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.076 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:55:54.076 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:54.076 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.077 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.077 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.077 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:55:54.077 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:55:54.078 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.078 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.078 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.078 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:55:54.079 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:55:54.079 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:55:54.079 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:55:54.079 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:54.080 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:54.080 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:54.080 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.081 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.081 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.081 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:55:54.081 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:55:54.082 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:54.082 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:54.082 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:54.083 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:54.083 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:54.083 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:54.084 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:54.084 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 91, 2022-07-21T01:55:54.084 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:54.084 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:54.085 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:54.085 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.085 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.085 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.086 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:55:54.086 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:54.086 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.086 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.087 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.087 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:55:54.087 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:54.088 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.088 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.088 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.088 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:54.089 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.089 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.089 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.090 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:55:54.090 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:54.090 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.090 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.091 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.091 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:55:54.091 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:54.091 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.092 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.092 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.092 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:54.092 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.093 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.093 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.093 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:55:54.093 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:54.094 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.094 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.094 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.094 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:55:54.095 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:54.095 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.095 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.095 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.096 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:54.096 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.096 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.096 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.097 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:55:54.097 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:54.097 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.097 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.097 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.098 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:55:54.098 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:55:54.098 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.098 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.099 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.099 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:55:54.099 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:55:54.099 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:55:54.100 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:55:54.100 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:54.100 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:54.100 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:54.101 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.101 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.101 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.101 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:55:54.101 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:55:54.102 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:54.102 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:54.102 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:54.102 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:54.103 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:54.103 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:54.103 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:54.103 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 91, 2022-07-21T01:55:54.104 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:54.104 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:54.104 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:54.104 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.104 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.105 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.105 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:55:54.105 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:54.105 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.106 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.106 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.106 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:55:54.106 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:54.107 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.107 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.107 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.107 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:54.108 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.108 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.108 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.108 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:55:54.109 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:54.109 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.109 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.109 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.109 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:55:54.110 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:54.110 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.110 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.110 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.111 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:54.111 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.111 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.112 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.112 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:55:54.112 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:54.113 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.113 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.113 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.113 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:55:54.114 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:54.114 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.114 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.114 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.114 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:54.115 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.115 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.115 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.115 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:55:54.116 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:54.116 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.116 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.116 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.117 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:55:54.117 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:55:54.117 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.117 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.117 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.118 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:55:54.118 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:55:54.118 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:55:54.118 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:55:54.119 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:54.119 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:54.119 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:54.119 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.120 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.120 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.120 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:55:54.120 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:55:54.121 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:54.121 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:54.121 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:54.121 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:54.122 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:54.122 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:54.122 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:54.122 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 92, 2022-07-21T01:55:54.123 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:54.123 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:54.123 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:54.123 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.124 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.124 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.124 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:55:54.124 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:54.125 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.125 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.125 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.125 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:55:54.126 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:54.126 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.126 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.126 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.126 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:54.127 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.127 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.127 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.127 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:55:54.128 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:54.128 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.128 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.128 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.129 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:55:54.129 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:54.129 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.129 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.129 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.130 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:54.130 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.130 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.130 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.131 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:55:54.131 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:54.131 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.131 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.131 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.132 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:55:54.132 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:54.132 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.133 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.133 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.133 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:54.133 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.133 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.134 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.134 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:55:54.134 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:54.135 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.135 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.135 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.135 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:55:54.135 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:55:54.136 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.136 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.136 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.136 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:55:54.137 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:55:54.137 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:55:54.137 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:55:54.137 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:54.138 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:54.138 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:54.138 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.138 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.138 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.139 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:55:54.139 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:55:54.139 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:54.139 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:54.140 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:54.140 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:54.140 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:54.140 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:54.141 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:54.141 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 90, 2022-07-21T01:55:54.141 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:54.141 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:54.141 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:54.142 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.142 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.142 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.142 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:55:54.143 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:54.143 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.143 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.143 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.144 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:55:54.144 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:54.144 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.144 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.145 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.145 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:54.145 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.145 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.146 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.146 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:55:54.146 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:54.146 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.147 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.147 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.147 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:55:54.147 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:54.147 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.148 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.148 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.148 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:54.148 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.149 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.149 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.149 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:55:54.149 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:54.150 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.150 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.150 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.150 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:55:54.150 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:54.151 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.151 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.151 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.151 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:54.152 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.152 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.152 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.152 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:55:54.153 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:54.153 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.153 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.153 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.153 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:55:54.154 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:55:54.154 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.154 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.154 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.155 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:55:54.155 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:55:54.155 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:55:54.155 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:55:54.156 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:54.156 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:54.156 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:54.156 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.156 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.157 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.157 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:55:54.157 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:55:54.157 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:54.158 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:54.158 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:54.158 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:54.158 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:54.159 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:54.159 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:54.159 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:55:54.159 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:54.160 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:54.160 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:54.160 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.160 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.160 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.161 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:55:54.161 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:54.161 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.161 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.162 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.162 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:55:54.162 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:54.162 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.163 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.163 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.163 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:54.163 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.163 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.164 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.164 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:55:54.164 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:54.164 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.165 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.165 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.165 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:55:54.165 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:54.165 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.166 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.166 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.166 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:54.166 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.167 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.167 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.167 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:55:54.167 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:54.168 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.168 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.168 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.169 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:55:54.169 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:54.169 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.169 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.169 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.170 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:54.170 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.170 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.170 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.171 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:55:54.171 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:54.171 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.171 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.172 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.172 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:55:54.172 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:55:54.172 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.172 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.173 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.173 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:55:54.173 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:55:54.174 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:55:54.174 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:55:54.174 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:54.174 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:54.175 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:54.175 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.175 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.175 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.175 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:55:54.176 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:55:54.176 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:54.176 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:54.176 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:54.177 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:54.177 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:54.177 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:54.177 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:54.178 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 90, 2022-07-21T01:55:54.178 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:54.178 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:54.178 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:54.178 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.179 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.179 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.179 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:55:54.179 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:54.180 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.180 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.180 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.180 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:55:54.181 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:54.181 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.181 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.181 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.181 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:54.182 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.182 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.182 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.182 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:55:54.183 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:54.183 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.183 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.183 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.184 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:55:54.184 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:54.184 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.184 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.184 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.185 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:54.185 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.185 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.185 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.186 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:55:54.186 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:54.186 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.186 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.187 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.187 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:55:54.187 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:54.187 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.187 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.188 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.188 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:54.188 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.188 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.189 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.189 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:55:54.189 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:54.189 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.190 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.190 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.190 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:55:54.190 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:55:54.190 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.191 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.191 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.191 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:55:54.191 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:55:54.192 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:55:54.192 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:55:54.192 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:54.192 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:54.193 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:54.193 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.193 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.193 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.194 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:55:54.194 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:55:54.194 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:55:54.194 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:55:54.194 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:55:54.195 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:55:54.195 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:55:54.195 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:55:54.195 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:55:54.196 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 91, 2022-07-21T01:55:54.196 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:55:54.196 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:55:54.196 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:55:54.197 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.197 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.197 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.197 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:55:54.198 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:54.198 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.198 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.198 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.198 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:55:54.199 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:54.199 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.199 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.199 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.200 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:55:54.200 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.200 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.200 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.201 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:55:54.201 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:54.201 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.201 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.201 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.202 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:55:54.202 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:54.202 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.202 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.203 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.203 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:55:54.203 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.203 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.204 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.204 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:55:54.204 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:54.204 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.205 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.205 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.205 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:55:54.205 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:54.206 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.206 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.206 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.206 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:55:54.207 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:55:54.207 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.207 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:55:54.207 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:55:54.207 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:54.208 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.208 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.208 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:55:54.208 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:55:54.209 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:55:54.209 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.209 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.209 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.210 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:55:54.210 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:55:54.210 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:55:54.210 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:55:54.210 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:55:54.211 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:55:54.211 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:55:54.211 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:55:54.211 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.212 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:54.212 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:55:54.212 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.212 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:55:54.213 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:54.213 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:54.213 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:54.213 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:54.214 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:54.214 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:55:54.214 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:54.214 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.215 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.215 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:55:54.215 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:54.215 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:54.216 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:54.216 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:54.216 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:54.216 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:55:54.216 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:54.217 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.217 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.217 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:55:54.217 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:54.218 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:54.218 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:54.218 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:54.218 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:54.219 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:55:54.219 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:54.219 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.219 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.220 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:55:54.220 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:54.220 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:54.220 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:54.221 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:54.221 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:54.221 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:55:54.221 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:54.221 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.222 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.222 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:55:54.222 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:54.222 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:54.223 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:54.223 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:54.223 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:54.223 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:55:54.224 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:54.224 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.224 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.224 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:55:54.225 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:54.225 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:54.225 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:54.225 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:54.226 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:54.226 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:55:54.226 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:54.226 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.226 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.227 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:55:54.227 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:54.227 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:54.227 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:54.228 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:54.228 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:54.228 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:55:54.228 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:54.229 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.229 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:55:54.229 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:55:54.229 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:55:54.230 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:55:54.230 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:55:54.230 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:55:54.230 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:55:54.231 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:55:54.231 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:55:54.231 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.231 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:55:54.232 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:55:54.232 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:55:54.232 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:55:54.232 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:55:54.232 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": {}, 2022-07-21T01:55:54.233 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:55:54.233 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:55:54.233 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:55:54.233 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:55:54.234 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:55:54.234 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:55:54.234 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:55:54.234 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:55:54.235 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:55:54.235 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:55:54.235 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:55:54.235 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:55:54.235 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:55:54.236 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:55:54.236 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:55:54.236 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:55:54.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.942+0000 7f749ec22700 1 -- 172.21.15.140:0/1282357355 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f747c053aa0 msgr2=0x7f747c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.942+0000 7f749ec22700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f747c053aa0 0x7f747c055f60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f7488019000 tx=0x7f7488016000).stop 2022-07-21T01:55:54.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.942+0000 7f749ec22700 1 -- 172.21.15.140:0/1282357355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74980acef0 msgr2=0x7f7498063a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.942+0000 7f749ec22700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74980acef0 0x7f7498063a80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f74980660b0 tx=0x7f7480007b50).stop 2022-07-21T01:55:54.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.946+0000 7f749ec22700 1 -- 172.21.15.140:0/1282357355 shutdown_connections 2022-07-21T01:55:54.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.946+0000 7f749ec22700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f747c053aa0 0x7f747c055f60 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.946+0000 7f749ec22700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74980b5a20 0x7f7498156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.946+0000 7f749ec22700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f74980b1e00 0x7f749814e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.946+0000 7f749ec22700 1 --2- 172.21.15.140:0/1282357355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f74980acef0 0x7f7498063a80 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.946+0000 7f749ec22700 1 -- 172.21.15.140:0/1282357355 >> 172.21.15.140:0/1282357355 conn(0x7f749805c3f0 msgr2=0x7f7498145030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:54.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.946+0000 7f749ec22700 1 -- 172.21.15.140:0/1282357355 shutdown_connections 2022-07-21T01:55:54.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:53.946+0000 7f749ec22700 1 -- 172.21.15.140:0/1282357355 wait complete. 2022-07-21T01:55:54.242 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:54.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.342+0000 7f7b6a78f700 1 Processor -- start 2022-07-21T01:55:54.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.342+0000 7f7b6a78f700 1 -- start start 2022-07-21T01:55:54.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7b640b5650 0x7f7b640b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b640adff0 0x7f7b64147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b64147010 con 0x7f7b640af8e0 2022-07-21T01:55:54.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b641472b0 con 0x7f7b640b5650 2022-07-21T01:55:54.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b64146d40 con 0x7f7b640adff0 2022-07-21T01:55:54.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6978d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7b640b5650 0x7f7b640b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6978d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7b640b5650 0x7f7b640b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38432/0 (socket says 172.21.15.140:38432) 2022-07-21T01:55:54.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6978d700 1 -- 172.21.15.140:0/1255751050 learned_addr learned my addr 172.21.15.140:0/1255751050 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:54.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b69f8e700 1 --1- 172.21.15.140:0/1255751050 >> v1:172.21.15.152:6789/0 conn(0x7f7b640af8e0 0x7f7b6405bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:53694/0 (socket says 172.21.15.140:53694) 2022-07-21T01:55:54.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b68f8c700 1 --2- 172.21.15.140:0/1255751050 >> v2:172.21.15.152:3300/0 conn(0x7f7b640adff0 0x7f7b64147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6978d700 1 -- 172.21.15.140:0/1255751050 >> v2:172.21.15.152:3300/0 conn(0x7f7b640adff0 msgr2=0x7f7b64147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:54.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6978d700 1 --2- 172.21.15.140:0/1255751050 >> v2:172.21.15.152:3300/0 conn(0x7f7b640adff0 0x7f7b64147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6978d700 1 -- 172.21.15.140:0/1255751050 >> v1:172.21.15.152:6789/0 conn(0x7f7b640af8e0 legacy=0x7f7b6405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6978d700 1 -- 172.21.15.140:0/1255751050 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64147c40 con 0x7f7b640b5650 2022-07-21T01:55:54.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6978d700 1 --2- 172.21.15.140:0/1255751050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b640b5650 0x7f7b640b5a30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f7b54002f70 tx=0x7f7b54006eb0).ready entity=mon.0 client_cookie=db445aecdc4d00e9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:54.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b5bfff700 1 -- 172.21.15.140:0/1255751050 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b5400d5c0 con 0x7f7b640b5650 2022-07-21T01:55:54.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b5bfff700 1 -- 172.21.15.140:0/1255751050 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b54000e90 con 0x7f7b640b5650 2022-07-21T01:55:54.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b5bfff700 1 -- 172.21.15.140:0/1255751050 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b5400d950 con 0x7f7b640b5650 2022-07-21T01:55:54.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 -- 172.21.15.140:0/1255751050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b640b5650 msgr2=0x7f7b640b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 --2- 172.21.15.140:0/1255751050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b640b5650 0x7f7b640b5a30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f7b54002f70 tx=0x7f7b54006eb0).stop 2022-07-21T01:55:54.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 -- 172.21.15.140:0/1255751050 shutdown_connections 2022-07-21T01:55:54.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 --2- 172.21.15.140:0/1255751050 >> v2:172.21.15.152:3300/0 conn(0x7f7b640adff0 0x7f7b64147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 --2- 172.21.15.140:0/1255751050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b640b5650 0x7f7b640b5a30 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 -- 172.21.15.140:0/1255751050 >> 172.21.15.140:0/1255751050 conn(0x7f7b640634e0 msgr2=0x7f7b640b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:54.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 -- 172.21.15.140:0/1255751050 shutdown_connections 2022-07-21T01:55:54.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.346+0000 7f7b6a78f700 1 -- 172.21.15.140:0/1255751050 wait complete. 2022-07-21T01:55:54.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b6a78f700 1 Processor -- start 2022-07-21T01:55:54.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b6a78f700 1 -- start start 2022-07-21T01:55:54.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b6a78f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b640adff0 0x7f7b64063990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b6a78f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b640af8e0 0x7f7b64066ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b6a78f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b640b5650 0x7f7b640673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b6a78f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64110f50 con 0x7f7b640af8e0 2022-07-21T01:55:54.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b6a78f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64065e20 con 0x7f7b640adff0 2022-07-21T01:55:54.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b6a78f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64157540 con 0x7f7b640b5650 2022-07-21T01:55:54.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b68f8c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b640af8e0 0x7f7b64066ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b68f8c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b640af8e0 0x7f7b64066ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38438/0 (socket says 172.21.15.140:38438) 2022-07-21T01:55:54.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b68f8c700 1 -- 172.21.15.140:0/329696239 learned_addr learned my addr 172.21.15.140:0/329696239 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:54.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b6978d700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b640adff0 0x7f7b64063990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b69f8e700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b640b5650 0x7f7b640673c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b68f8c700 1 -- 172.21.15.140:0/329696239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b640b5650 msgr2=0x7f7b640673c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b68f8c700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b640b5650 0x7f7b640673c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b68f8c700 1 -- 172.21.15.140:0/329696239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b640adff0 msgr2=0x7f7b64063990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b68f8c700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b640adff0 0x7f7b64063990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b68f8c700 1 -- 172.21.15.140:0/329696239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5400b1e0 con 0x7f7b640af8e0 2022-07-21T01:55:54.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.350+0000 7f7b68f8c700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b640af8e0 0x7f7b64066ed0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f7b6000c000 tx=0x7f7b600091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:54.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.354+0000 7f7b5a7fc700 1 -- 172.21.15.140:0/329696239 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b6000c670 con 0x7f7b640af8e0 2022-07-21T01:55:54.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.354+0000 7f7b5a7fc700 1 -- 172.21.15.140:0/329696239 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6000c7d0 con 0x7f7b640af8e0 2022-07-21T01:55:54.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.354+0000 7f7b6a78f700 1 -- 172.21.15.140:0/329696239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b641577e0 con 0x7f7b640af8e0 2022-07-21T01:55:54.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.354+0000 7f7b5a7fc700 1 -- 172.21.15.140:0/329696239 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b6000cb30 con 0x7f7b640af8e0 2022-07-21T01:55:54.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.354+0000 7f7b6a78f700 1 -- 172.21.15.140:0/329696239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b64111da0 con 0x7f7b640af8e0 2022-07-21T01:55:54.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.354+0000 7f7b6a78f700 1 -- 172.21.15.140:0/329696239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b64157360 con 0x7f7b640af8e0 2022-07-21T01:55:54.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.354+0000 7f7b5a7fc700 1 -- 172.21.15.140:0/329696239 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7b60022070 con 0x7f7b640af8e0 2022-07-21T01:55:54.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.354+0000 7f7b5a7fc700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b4c053a70 0x7f7b4c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.354+0000 7f7b5a7fc700 1 -- 172.21.15.140:0/329696239 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7b6006c530 con 0x7f7b640af8e0 2022-07-21T01:55:54.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.354+0000 7f7b6978d700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b4c053a70 0x7f7b4c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.354+0000 7f7b6978d700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b4c053a70 0x7f7b4c055f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f7b5401c000 tx=0x7f7b54015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:54.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.362+0000 7f7b5a7fc700 1 -- 172.21.15.140:0/329696239 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7b600433d0 con 0x7f7b640af8e0 2022-07-21T01:55:54.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.526+0000 7f7b6a78f700 1 -- 172.21.15.140:0/329696239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b6405c4a0 con 0x7f7b640af8e0 2022-07-21T01:55:54.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.526+0000 7f7b5a7fc700 1 -- 172.21.15.140:0/329696239 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f7b60016020 con 0x7f7b640af8e0 2022-07-21T01:55:54.531 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:54.531 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":42,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/518920349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":518920349},{"type":"v1","addr":"172.21.15.152:6833","nonce":518920349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:43.343194+0000","modified":"2022-07-21T01:55:44.346904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7002":{"gid":7002,"name":"a","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2478727503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2478727503},{"type":"v1","addr":"172.21.15.140:6835","nonce":2478727503}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-07-21T01:55:54.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 -- 172.21.15.140:0/329696239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b4c053a70 msgr2=0x7f7b4c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b4c053a70 0x7f7b4c055f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f7b5401c000 tx=0x7f7b54015000).stop 2022-07-21T01:55:54.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 -- 172.21.15.140:0/329696239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b640af8e0 msgr2=0x7f7b64066ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b640af8e0 0x7f7b64066ed0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7f7b6000c000 tx=0x7f7b600091a0).stop 2022-07-21T01:55:54.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 -- 172.21.15.140:0/329696239 shutdown_connections 2022-07-21T01:55:54.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b4c053a70 0x7f7b4c055f30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b640b5650 0x7f7b640673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b640af8e0 0x7f7b64066ed0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 --2- 172.21.15.140:0/329696239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b640adff0 0x7f7b64063990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 -- 172.21.15.140:0/329696239 >> 172.21.15.140:0/329696239 conn(0x7f7b640634e0 msgr2=0x7f7b640afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:54.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 -- 172.21.15.140:0/329696239 shutdown_connections 2022-07-21T01:55:54.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.530+0000 7f7b6a78f700 1 -- 172.21.15.140:0/329696239 wait complete. 2022-07-21T01:55:54.536 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 42 2022-07-21T01:55:54.544 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:55:54.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f3198d8a700 1 Processor -- start 2022-07-21T01:55:54.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f3198d8a700 1 -- start start 2022-07-21T01:55:54.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f3198d8a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f31940b5990 0x7f3194146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f3198d8a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3194146c70 con 0x7f319405d210 2022-07-21T01:55:54.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f3198d8a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31940b5d70 con 0x7f31940af3e0 2022-07-21T01:55:54.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f3198d8a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31940ad050 con 0x7f31940b5990 2022-07-21T01:55:54.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f3193fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f319405d210 0x7f319414a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52594/0 (socket says 172.21.15.140:52594) 2022-07-21T01:55:54.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f3193fff700 1 -- 172.21.15.140:0/2143247153 learned_addr learned my addr 172.21.15.140:0/2143247153 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:54.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31937fe700 1 --2- 172.21.15.140:0/2143247153 >> v2:172.21.15.152:3300/0 conn(0x7f31940b5990 0x7f3194146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032905288 0 0) 0x7f3194146c70 con 0x7f319405d210 2022-07-21T01:55:54.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3180003260 con 0x7f319405d210 2022-07-21T01:55:54.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162938706 0 0) 0x7f31940b5d70 con 0x7f31940af3e0 2022-07-21T01:55:54.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3194146c70 con 0x7f31940af3e0 2022-07-21T01:55:54.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2572567020 0 0) 0x7f3180003260 con 0x7f319405d210 2022-07-21T01:55:54.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 >> v2:172.21.15.152:3300/0 conn(0x7f31940b5990 msgr2=0x7f3194146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:54.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 --2- 172.21.15.140:0/2143247153 >> v2:172.21.15.152:3300/0 conn(0x7f31940b5990 0x7f3194146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 >> v1:172.21.15.152:6789/0 conn(0x7f31940af3e0 legacy=0x7f3194146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31940b1010 con 0x7f319405d210 2022-07-21T01:55:54.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f3184001660 con 0x7f319405d210 2022-07-21T01:55:54.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 >> v1:172.21.15.140:6789/0 conn(0x7f319405d210 legacy=0x7f319414a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 --2- 172.21.15.140:0/2143247153 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3180003930 0x7f3180003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 --2- 172.21.15.140:0/2143247153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3180004500 0x7f3180004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 --2- 172.21.15.140:0/2143247153 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 0x7f318000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318000bdd0 con 0x7f3180004e70 2022-07-21T01:55:54.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318000bf10 con 0x7f3180004500 2022-07-21T01:55:54.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318000c0d0 con 0x7f3180003930 2022-07-21T01:55:54.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f3193fff700 1 --2- 172.21.15.140:0/2143247153 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 0x7f318000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f3192ffd700 1 --2- 172.21.15.140:0/2143247153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3180004500 0x7f3180004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.650+0000 7f31937fe700 1 --2- 172.21.15.140:0/2143247153 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3180003930 0x7f3180003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3193fff700 1 -- 172.21.15.140:0/2143247153 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3180003930 msgr2=0x7f3180003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:54.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3193fff700 1 --2- 172.21.15.140:0/2143247153 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3180003930 0x7f3180003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3193fff700 1 -- 172.21.15.140:0/2143247153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3180004500 msgr2=0x7f3180004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:54.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3193fff700 1 --2- 172.21.15.140:0/2143247153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3180004500 0x7f3180004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3193fff700 1 -- 172.21.15.140:0/2143247153 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3180003260 con 0x7f3180004e70 2022-07-21T01:55:54.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3193fff700 1 -- 172.21.15.140:0/2143247153 reap_dead start 2022-07-21T01:55:54.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3193fff700 1 --2- 172.21.15.140:0/2143247153 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 0x7f318000b750 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f3184001430 tx=0x7f31940b11a0).ready entity=mon.0 client_cookie=2e7cccd1cce35db0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:54.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3184007e90 con 0x7f3180004e70 2022-07-21T01:55:54.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31840070d0 con 0x7f3180004e70 2022-07-21T01:55:54.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f31927fc700 1 -- 172.21.15.140:0/2143247153 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3184007430 con 0x7f3180004e70 2022-07-21T01:55:54.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3198d8a700 1 -- 172.21.15.140:0/2143247153 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 msgr2=0x7f318000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3198d8a700 1 --2- 172.21.15.140:0/2143247153 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 0x7f318000b750 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f3184001430 tx=0x7f31940b11a0).stop 2022-07-21T01:55:54.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3198d8a700 1 -- 172.21.15.140:0/2143247153 shutdown_connections 2022-07-21T01:55:54.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3198d8a700 1 --2- 172.21.15.140:0/2143247153 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 0x7f318000b750 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3198d8a700 1 -- 172.21.15.140:0/2143247153 >> 172.21.15.140:0/2143247153 conn(0x7f3194063480 msgr2=0x7f31940b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:54.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3198d8a700 1 -- 172.21.15.140:0/2143247153 shutdown_connections 2022-07-21T01:55:54.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3198d8a700 1 -- 172.21.15.140:0/2143247153 wait complete. 2022-07-21T01:55:54.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3198d8a700 1 Processor -- start 2022-07-21T01:55:54.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.654+0000 7f3198d8a700 1 -- start start 2022-07-21T01:55:54.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3198d8a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 0x7f31941569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3198d8a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3194064d70 0x7f31940667e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3198d8a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3194145350 0x7f3194063950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3198d8a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3194065210 con 0x7f3180004e70 2022-07-21T01:55:54.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3198d8a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3194067020 con 0x7f3194064d70 2022-07-21T01:55:54.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3198d8a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31940672c0 con 0x7f3194145350 2022-07-21T01:55:54.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f31937fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 0x7f31941569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f31937fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 0x7f31941569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38456/0 (socket says 172.21.15.140:38456) 2022-07-21T01:55:54.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f31937fe700 1 -- 172.21.15.140:0/2833254022 learned_addr learned my addr 172.21.15.140:0/2833254022 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:54.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3192ffd700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3194064d70 0x7f31940667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3193fff700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3194145350 0x7f3194063950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f31937fe700 1 -- 172.21.15.140:0/2833254022 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3194145350 msgr2=0x7f3194063950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f31937fe700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3194145350 0x7f3194063950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f31937fe700 1 -- 172.21.15.140:0/2833254022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3194064d70 msgr2=0x7f31940667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:54.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f31937fe700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3194064d70 0x7f31940667e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:54.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f31937fe700 1 -- 172.21.15.140:0/2833254022 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3194067710 con 0x7f3180004e70 2022-07-21T01:55:54.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f31937fe700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 0x7f31941569f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f318c00b000 tx=0x7f318c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:54.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3190ff9700 1 -- 172.21.15.140:0/2833254022 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f318c005d50 con 0x7f3180004e70 2022-07-21T01:55:54.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3190ff9700 1 -- 172.21.15.140:0/2833254022 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c00f730 con 0x7f3180004e70 2022-07-21T01:55:54.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3198d8a700 1 -- 172.21.15.140:0/2833254022 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3194066cd0 con 0x7f3180004e70 2022-07-21T01:55:54.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3190ff9700 1 -- 172.21.15.140:0/2833254022 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f318c00fa90 con 0x7f3180004e70 2022-07-21T01:55:54.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3198d8a700 1 -- 172.21.15.140:0/2833254022 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3194064060 con 0x7f3180004e70 2022-07-21T01:55:54.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.658+0000 7f3198d8a700 1 -- 172.21.15.140:0/2833254022 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3194067480 con 0x7f3180004e70 2022-07-21T01:55:54.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.662+0000 7f3190ff9700 1 -- 172.21.15.140:0/2833254022 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f318c00fbf0 con 0x7f3180004e70 2022-07-21T01:55:54.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.662+0000 7f3190ff9700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3174053aa0 0x7f3174055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:54.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.662+0000 7f3190ff9700 1 -- 172.21.15.140:0/2833254022 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f318c06ce40 con 0x7f3180004e70 2022-07-21T01:55:54.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.662+0000 7f3192ffd700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3174053aa0 0x7f3174055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:54.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.662+0000 7f3192ffd700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3174053aa0 0x7f3174055f60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f3188003000 tx=0x7f3188007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:54.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.666+0000 7f3190ff9700 1 -- 172.21.15.140:0/2833254022 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f318c043c50 con 0x7f3180004e70 2022-07-21T01:55:54.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:54.830+0000 7f3198d8a700 1 -- 172.21.15.140:0/2833254022 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f31940574e0 con 0x7f3180004e70 2022-07-21T01:55:55.069 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:55:55.070+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:55:55.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.074+0000 7f3190ff9700 1 -- 172.21.15.140:0/2833254022 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7f318c018070 con 0x7f3180004e70 2022-07-21T01:55:55.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 -- 172.21.15.140:0/2833254022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3174053aa0 msgr2=0x7f3174055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3174053aa0 0x7f3174055f60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f3188003000 tx=0x7f3188007a00).stop 2022-07-21T01:55:55.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 -- 172.21.15.140:0/2833254022 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 msgr2=0x7f31941569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 0x7f31941569f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f318c00b000 tx=0x7f318c0091a0).stop 2022-07-21T01:55:55.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 -- 172.21.15.140:0/2833254022 shutdown_connections 2022-07-21T01:55:55.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3174053aa0 0x7f3174055f60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3194145350 0x7f3194063950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3194064d70 0x7f31940667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 --2- 172.21.15.140:0/2833254022 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3180004e70 0x7f31941569f0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 -- 172.21.15.140:0/2833254022 >> 172.21.15.140:0/2833254022 conn(0x7f3194063480 msgr2=0x7f31940b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:55.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 -- 172.21.15.140:0/2833254022 shutdown_connections 2022-07-21T01:55:55.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.078+0000 7f3198d8a700 1 -- 172.21.15.140:0/2833254022 wait complete. 2022-07-21T01:55:55.087 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:55:55.099 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:55:55.104 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:55:55.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.194+0000 7fea8b18f700 1 Processor -- start 2022-07-21T01:55:55.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.194+0000 7fea8b18f700 1 -- start start 2022-07-21T01:55:55.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fea840b13e0 0x7fea840b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fea840af120 0x7fea84067260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea840b20f0 con 0x7fea840b61d0 2022-07-21T01:55:55.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8a18d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fea840b61d0 0x7fea840aea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52622/0 (socket says 172.21.15.140:52622) 2022-07-21T01:55:55.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea8413efd0 con 0x7fea840b13e0 2022-07-21T01:55:55.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8a18d700 1 -- 172.21.15.140:0/179447035 learned_addr learned my addr 172.21.15.140:0/179447035 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:55.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 -- 172.21.15.140:0/179447035 --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea840b65b0 con 0x7fea840af120 2022-07-21T01:55:55.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8998c700 1 --2- 172.21.15.140:0/179447035 >> v2:172.21.15.140:3300/0 conn(0x7fea840b13e0 0x7fea840b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8a98e700 1 --2- 172.21.15.140:0/179447035 >> v2:172.21.15.152:3300/0 conn(0x7fea840af120 0x7fea84067260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8918b700 1 -- 172.21.15.140:0/179447035 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318566772 0 0) 0x7fea840b20f0 con 0x7fea840b61d0 2022-07-21T01:55:55.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8918b700 1 -- 172.21.15.140:0/179447035 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea6c004370 con 0x7fea840b61d0 2022-07-21T01:55:55.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8998c700 1 -- 172.21.15.140:0/179447035 >> v2:172.21.15.152:3300/0 conn(0x7fea840af120 msgr2=0x7fea84067260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:55.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8998c700 1 --2- 172.21.15.140:0/179447035 >> v2:172.21.15.152:3300/0 conn(0x7fea840af120 0x7fea84067260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8998c700 1 -- 172.21.15.140:0/179447035 >> v1:172.21.15.140:6789/0 conn(0x7fea840b61d0 legacy=0x7fea840aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8998c700 1 -- 172.21.15.140:0/179447035 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840b1ad0 con 0x7fea840b13e0 2022-07-21T01:55:55.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8998c700 1 --2- 172.21.15.140:0/179447035 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea840b13e0 0x7fea840b0460 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7fea74002f70 tx=0x7fea74006df0).ready entity=mon.0 client_cookie=fefe3a99cd7d9636 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:55.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8918b700 1 -- 172.21.15.140:0/179447035 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea74014070 con 0x7fea840b13e0 2022-07-21T01:55:55.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8918b700 1 -- 172.21.15.140:0/179447035 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea74000e90 con 0x7fea840b13e0 2022-07-21T01:55:55.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8918b700 1 -- 172.21.15.140:0/179447035 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea7400fae0 con 0x7fea840b13e0 2022-07-21T01:55:55.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 -- 172.21.15.140:0/179447035 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea840b13e0 msgr2=0x7fea840b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 --2- 172.21.15.140:0/179447035 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea840b13e0 0x7fea840b0460 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7fea74002f70 tx=0x7fea74006df0).stop 2022-07-21T01:55:55.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 -- 172.21.15.140:0/179447035 shutdown_connections 2022-07-21T01:55:55.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 --2- 172.21.15.140:0/179447035 >> v2:172.21.15.152:3300/0 conn(0x7fea840af120 0x7fea84067260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 --2- 172.21.15.140:0/179447035 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea840b13e0 0x7fea840b0460 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 -- 172.21.15.140:0/179447035 >> 172.21.15.140:0/179447035 conn(0x7fea84063cf0 msgr2=0x7fea840b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:55.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 -- 172.21.15.140:0/179447035 shutdown_connections 2022-07-21T01:55:55.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 -- 172.21.15.140:0/179447035 wait complete. 2022-07-21T01:55:55.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 Processor -- start 2022-07-21T01:55:55.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.198+0000 7fea8b18f700 1 -- start start 2022-07-21T01:55:55.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8b18f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea840af120 0x7fea841458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8b18f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea840b13e0 0x7fea8413d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8b18f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea840b61d0 0x7fea8413db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8b18f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea84066e70 con 0x7fea840b13e0 2022-07-21T01:55:55.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8b18f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea84147880 con 0x7fea840af120 2022-07-21T01:55:55.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8b18f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea84147b20 con 0x7fea840b61d0 2022-07-21T01:55:55.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8998c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea840b13e0 0x7fea8413d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8998c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea840b13e0 0x7fea8413d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38484/0 (socket says 172.21.15.140:38484) 2022-07-21T01:55:55.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8998c700 1 -- 172.21.15.140:0/305082666 learned_addr learned my addr 172.21.15.140:0/305082666 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:55.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8a98e700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea840b61d0 0x7fea8413db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8a18d700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea840af120 0x7fea841458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8998c700 1 -- 172.21.15.140:0/305082666 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea840b61d0 msgr2=0x7fea8413db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8998c700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea840b61d0 0x7fea8413db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8998c700 1 -- 172.21.15.140:0/305082666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea840af120 msgr2=0x7fea841458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8998c700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea840af120 0x7fea841458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8998c700 1 -- 172.21.15.140:0/305082666 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7400b180 con 0x7fea840b13e0 2022-07-21T01:55:55.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8998c700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea840b13e0 0x7fea8413d640 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7fea74002f70 tx=0x7fea74016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:55.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea737fe700 1 -- 172.21.15.140:0/305082666 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea74010bf0 con 0x7fea840b13e0 2022-07-21T01:55:55.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea737fe700 1 -- 172.21.15.140:0/305082666 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea74010d50 con 0x7fea840b13e0 2022-07-21T01:55:55.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8b18f700 1 -- 172.21.15.140:0/305082666 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea84147dc0 con 0x7fea840b13e0 2022-07-21T01:55:55.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea737fe700 1 -- 172.21.15.140:0/305082666 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea7401c630 con 0x7fea840b13e0 2022-07-21T01:55:55.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8b18f700 1 -- 172.21.15.140:0/305082666 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea84144460 con 0x7fea840b13e0 2022-07-21T01:55:55.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.202+0000 7fea8b18f700 1 -- 172.21.15.140:0/305082666 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea84147f50 con 0x7fea840b13e0 2022-07-21T01:55:55.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.206+0000 7fea737fe700 1 -- 172.21.15.140:0/305082666 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fea74022020 con 0x7fea840b13e0 2022-07-21T01:55:55.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.206+0000 7fea737fe700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea64053aa0 0x7fea64055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.206+0000 7fea737fe700 1 -- 172.21.15.140:0/305082666 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fea7406d850 con 0x7fea840b13e0 2022-07-21T01:55:55.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.206+0000 7fea8a18d700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea64053aa0 0x7fea64055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.206+0000 7fea8a18d700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea64053aa0 0x7fea64055f60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fea8000d000 tx=0x7fea80009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:55.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.210+0000 7fea737fe700 1 -- 172.21.15.140:0/305082666 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea74044720 con 0x7fea840b13e0 2022-07-21T01:55:55.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.374+0000 7fea8b18f700 1 -- 172.21.15.140:0/305082666 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea840b1d70 con 0x7fea840b13e0 2022-07-21T01:55:55.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.374+0000 7fea737fe700 1 -- 172.21.15.140:0/305082666 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+1779 (secure 0 0 0) 0x7fea74047d40 con 0x7fea840b13e0 2022-07-21T01:55:55.378 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:55:55.379 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":43,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/518920349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":518920349},{"type":"v1","addr":"172.21.15.152:6833","nonce":518920349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:55:43.343194+0000","modified":"2022-07-21T01:55:55.070790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2022-07-21T01:55:55.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 -- 172.21.15.140:0/305082666 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea64053aa0 msgr2=0x7fea64055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea64053aa0 0x7fea64055f60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fea8000d000 tx=0x7fea80009040).stop 2022-07-21T01:55:55.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 -- 172.21.15.140:0/305082666 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea840b13e0 msgr2=0x7fea8413d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea840b13e0 0x7fea8413d640 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7fea74002f70 tx=0x7fea74016000).stop 2022-07-21T01:55:55.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 -- 172.21.15.140:0/305082666 shutdown_connections 2022-07-21T01:55:55.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea64053aa0 0x7fea64055f60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea840b61d0 0x7fea8413db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea840b13e0 0x7fea8413d640 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 --2- 172.21.15.140:0/305082666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea840af120 0x7fea841458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 -- 172.21.15.140:0/305082666 >> 172.21.15.140:0/305082666 conn(0x7fea84063cf0 msgr2=0x7fea840b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:55.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 -- 172.21.15.140:0/305082666 shutdown_connections 2022-07-21T01:55:55.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.378+0000 7fea8b18f700 1 -- 172.21.15.140:0/305082666 wait complete. 2022-07-21T01:55:55.384 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 43 2022-07-21T01:55:55.391 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:55:55.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.494+0000 7f16eb4cc700 1 Processor -- start 2022-07-21T01:55:55.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.494+0000 7f16eb4cc700 1 -- start start 2022-07-21T01:55:55.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f16e40b5a10 0x7f16e40ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16e40ae780 0x7f16e40aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e413ced0 con 0x7f16e40b5630 2022-07-21T01:55:55.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e40b0a10 con 0x7f16e40b5a10 2022-07-21T01:55:55.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40b0cb0 con 0x7f16e40ae780 2022-07-21T01:55:55.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16e9cc9700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f16e40b5a10 0x7f16e40ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16e9cc9700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f16e40b5a10 0x7f16e40ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38492/0 (socket says 172.21.15.140:38492) 2022-07-21T01:55:55.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16e9cc9700 1 -- 172.21.15.140:0/3658497340 learned_addr learned my addr 172.21.15.140:0/3658497340 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:55.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eaccb700 1 --2- 172.21.15.140:0/3658497340 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16e40ae780 0x7f16e40aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16e9cc9700 1 -- 172.21.15.140:0/3658497340 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16e40ae780 msgr2=0x7f16e40aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:55.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16e9cc9700 1 --2- 172.21.15.140:0/3658497340 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16e40ae780 0x7f16e40aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16e9cc9700 1 -- 172.21.15.140:0/3658497340 >> v1:172.21.15.152:6789/0 conn(0x7f16e40b5630 legacy=0x7f16e413c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16e9cc9700 1 -- 172.21.15.140:0/3658497340 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e40b0f50 con 0x7f16e40b5a10 2022-07-21T01:55:55.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eaccb700 1 --2- 172.21.15.140:0/3658497340 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16e40ae780 0x7f16e40aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:55.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16e9cc9700 1 --2- 172.21.15.140:0/3658497340 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16e40b5a10 0x7f16e40ae290 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f16d8002f70 tx=0x7f16d8006eb0).ready entity=mon.0 client_cookie=194df8b2b6cdd85a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:55.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16e94c8700 1 -- 172.21.15.140:0/3658497340 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16d800d5c0 con 0x7f16e40b5a10 2022-07-21T01:55:55.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16e94c8700 1 -- 172.21.15.140:0/3658497340 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8000e90 con 0x7f16e40b5a10 2022-07-21T01:55:55.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16e94c8700 1 -- 172.21.15.140:0/3658497340 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16d800d950 con 0x7f16e40b5a10 2022-07-21T01:55:55.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 -- 172.21.15.140:0/3658497340 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16e40b5a10 msgr2=0x7f16e40ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 --2- 172.21.15.140:0/3658497340 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16e40b5a10 0x7f16e40ae290 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f16d8002f70 tx=0x7f16d8006eb0).stop 2022-07-21T01:55:55.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 -- 172.21.15.140:0/3658497340 shutdown_connections 2022-07-21T01:55:55.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 --2- 172.21.15.140:0/3658497340 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16e40ae780 0x7f16e40aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 --2- 172.21.15.140:0/3658497340 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16e40b5a10 0x7f16e40ae290 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 -- 172.21.15.140:0/3658497340 >> 172.21.15.140:0/3658497340 conn(0x7f16e4063490 msgr2=0x7f16e40b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:55.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 -- 172.21.15.140:0/3658497340 shutdown_connections 2022-07-21T01:55:55.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.498+0000 7f16eb4cc700 1 -- 172.21.15.140:0/3658497340 wait complete. 2022-07-21T01:55:55.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eb4cc700 1 Processor -- start 2022-07-21T01:55:55.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eb4cc700 1 -- start start 2022-07-21T01:55:55.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eb4cc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e40ae780 0x7f16e413aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eb4cc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16e40b5630 0x7f16e4151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eb4cc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16e40b5a10 0x7f16e413fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eb4cc700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e4152130 con 0x7f16e40b5630 2022-07-21T01:55:55.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eb4cc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e41422c0 con 0x7f16e40ae780 2022-07-21T01:55:55.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eb4cc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e4145f40 con 0x7f16e40b5a10 2022-07-21T01:55:55.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e9cc9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16e40b5630 0x7f16e4151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e9cc9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16e40b5630 0x7f16e4151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38498/0 (socket says 172.21.15.140:38498) 2022-07-21T01:55:55.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e9cc9700 1 -- 172.21.15.140:0/586495302 learned_addr learned my addr 172.21.15.140:0/586495302 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:55.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eaccb700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16e40b5a10 0x7f16e413fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16ea4ca700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e40ae780 0x7f16e413aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e9cc9700 1 -- 172.21.15.140:0/586495302 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16e40b5a10 msgr2=0x7f16e413fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e9cc9700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16e40b5a10 0x7f16e413fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e9cc9700 1 -- 172.21.15.140:0/586495302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e40ae780 msgr2=0x7f16e413aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:55.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e9cc9700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e40ae780 0x7f16e413aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:55.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e9cc9700 1 -- 172.21.15.140:0/586495302 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d800b1e0 con 0x7f16e40b5630 2022-07-21T01:55:55.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e9cc9700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16e40b5630 0x7f16e4151c40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f16d8002f70 tx=0x7f16d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:55.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e37fe700 1 -- 172.21.15.140:0/586495302 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16d80117e0 con 0x7f16e40b5630 2022-07-21T01:55:55.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e37fe700 1 -- 172.21.15.140:0/586495302 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d8011940 con 0x7f16e40b5630 2022-07-21T01:55:55.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eb4cc700 1 -- 172.21.15.140:0/586495302 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e4152310 con 0x7f16e40b5630 2022-07-21T01:55:55.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16e37fe700 1 -- 172.21.15.140:0/586495302 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16d8011ca0 con 0x7f16e40b5630 2022-07-21T01:55:55.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eb4cc700 1 -- 172.21.15.140:0/586495302 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e4152850 con 0x7f16e40b5630 2022-07-21T01:55:55.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.502+0000 7f16eb4cc700 1 -- 172.21.15.140:0/586495302 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e413dbb0 con 0x7f16e40b5630 2022-07-21T01:55:55.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.506+0000 7f16e37fe700 1 -- 172.21.15.140:0/586495302 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f16d8025030 con 0x7f16e40b5630 2022-07-21T01:55:55.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.506+0000 7f16e37fe700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f16c4053b00 0x7f16c4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:55.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.506+0000 7f16e37fe700 1 -- 172.21.15.140:0/586495302 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f16d806d1e0 con 0x7f16e40b5630 2022-07-21T01:55:55.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.506+0000 7f16ea4ca700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f16c4053b00 0x7f16c4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:55.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.506+0000 7f16ea4ca700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f16c4053b00 0x7f16c4055fc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f16dc00d000 tx=0x7f16dc007f10).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:55.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.510+0000 7f16e37fe700 1 -- 172.21.15.140:0/586495302 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f16e413dbb0 con 0x7f16e40b5630 2022-07-21T01:55:55.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:55.674+0000 7f16eb4cc700 1 -- 172.21.15.140:0/586495302 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f16e413dbb0 con 0x7f16e40b5630 2022-07-21T01:55:56.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.082+0000 7f16e37fe700 1 -- 172.21.15.140:0/586495302 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f16e413dbb0 con 0x7f16e40b5630 2022-07-21T01:55:56.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 -- 172.21.15.140:0/586495302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f16c4053b00 msgr2=0x7f16c4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:56.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f16c4053b00 0x7f16c4055fc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f16dc00d000 tx=0x7f16dc007f10).stop 2022-07-21T01:55:56.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 -- 172.21.15.140:0/586495302 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16e40b5630 msgr2=0x7f16e4151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:56.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16e40b5630 0x7f16e4151c40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f16d8002f70 tx=0x7f16d8015000).stop 2022-07-21T01:55:56.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 -- 172.21.15.140:0/586495302 shutdown_connections 2022-07-21T01:55:56.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f16c4053b00 0x7f16c4055fc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:56.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16e40b5a10 0x7f16e413fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:56.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16e40b5630 0x7f16e4151c40 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:56.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 --2- 172.21.15.140:0/586495302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16e40ae780 0x7f16e413aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:56.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 -- 172.21.15.140:0/586495302 >> 172.21.15.140:0/586495302 conn(0x7f16e4063490 msgr2=0x7f16e40aee70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:56.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 -- 172.21.15.140:0/586495302 shutdown_connections 2022-07-21T01:55:56.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.090+0000 7f16eb4cc700 1 -- 172.21.15.140:0/586495302 wait complete. 2022-07-21T01:55:56.108 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:55:56.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.214+0000 7fd249804700 1 Processor -- start 2022-07-21T01:55:56.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.214+0000 7fd249804700 1 -- start start 2022-07-21T01:55:56.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2440b5990 0x7fd244146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:56.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd24405d210 0x7fd24414a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:56.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd244146c70 con 0x7fd2440af3e0 2022-07-21T01:55:56.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2440ad050 con 0x7fd2440b5990 2022-07-21T01:55:56.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440b17e0 con 0x7fd24405d210 2022-07-21T01:55:56.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd248802700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2440b5990 0x7fd244146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:56.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd248802700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2440b5990 0x7fd244146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38506/0 (socket says 172.21.15.140:38506) 2022-07-21T01:55:56.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd248802700 1 -- 172.21.15.140:0/2277744548 learned_addr learned my addr 172.21.15.140:0/2277744548 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:56.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249003700 1 --2- 172.21.15.140:0/2277744548 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd24405d210 0x7fd24414a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:56.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd248802700 1 -- 172.21.15.140:0/2277744548 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd24405d210 msgr2=0x7fd24414a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:56.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd248802700 1 --2- 172.21.15.140:0/2277744548 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd24405d210 0x7fd24414a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:56.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd2437fe700 1 -- 172.21.15.140:0/2277744548 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284654742 0 0) 0x7fd244146c70 con 0x7fd2440af3e0 2022-07-21T01:55:56.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd248802700 1 -- 172.21.15.140:0/2277744548 >> v1:172.21.15.152:6789/0 conn(0x7fd2440af3e0 legacy=0x7fd244146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:56.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd248802700 1 -- 172.21.15.140:0/2277744548 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440b5d70 con 0x7fd2440b5990 2022-07-21T01:55:56.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd248802700 1 --2- 172.21.15.140:0/2277744548 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2440b5990 0x7fd244146070 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fd22c002f70 tx=0x7fd22c006e50).ready entity=mon.0 client_cookie=92bd3a2fbf7bc3f0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:56.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd2437fe700 1 -- 172.21.15.140:0/2277744548 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd22c00d5c0 con 0x7fd2440b5990 2022-07-21T01:55:56.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd2437fe700 1 -- 172.21.15.140:0/2277744548 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c000e90 con 0x7fd2440b5990 2022-07-21T01:55:56.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd2437fe700 1 -- 172.21.15.140:0/2277744548 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd22c00d910 con 0x7fd2440b5990 2022-07-21T01:55:56.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 -- 172.21.15.140:0/2277744548 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2440b5990 msgr2=0x7fd244146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:56.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 --2- 172.21.15.140:0/2277744548 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2440b5990 0x7fd244146070 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fd22c002f70 tx=0x7fd22c006e50).stop 2022-07-21T01:55:56.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 -- 172.21.15.140:0/2277744548 shutdown_connections 2022-07-21T01:55:56.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 --2- 172.21.15.140:0/2277744548 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd24405d210 0x7fd24414a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:56.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 --2- 172.21.15.140:0/2277744548 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2440b5990 0x7fd244146070 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:56.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 -- 172.21.15.140:0/2277744548 >> 172.21.15.140:0/2277744548 conn(0x7fd244063480 msgr2=0x7fd2440b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:56.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 -- 172.21.15.140:0/2277744548 shutdown_connections 2022-07-21T01:55:56.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.218+0000 7fd249804700 1 -- 172.21.15.140:0/2277744548 wait complete. 2022-07-21T01:55:56.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.222+0000 7fd249804700 1 Processor -- start 2022-07-21T01:55:56.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.222+0000 7fd249804700 1 -- start start 2022-07-21T01:55:56.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.222+0000 7fd249804700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd24405d210 0x7fd2440b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:56.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.222+0000 7fd249804700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2440af3e0 0x7fd244143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:56.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.222+0000 7fd249804700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2440b5990 0x7fd244143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:56.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.222+0000 7fd249804700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244065a20 con 0x7fd24405d210 2022-07-21T01:55:56.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.222+0000 7fd249804700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244156230 con 0x7fd2440b5990 2022-07-21T01:55:56.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.222+0000 7fd249804700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd244067310 con 0x7fd2440af3e0 2022-07-21T01:55:56.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.222+0000 7fd248802700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd24405d210 0x7fd2440b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:56.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd248802700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd24405d210 0x7fd2440b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38512/0 (socket says 172.21.15.140:38512) 2022-07-21T01:55:56.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd248802700 1 -- 172.21.15.140:0/3726830089 learned_addr learned my addr 172.21.15.140:0/3726830089 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:56.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd243fff700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2440af3e0 0x7fd244143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:56.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd249003700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2440b5990 0x7fd244143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:56.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd248802700 1 -- 172.21.15.140:0/3726830089 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2440af3e0 msgr2=0x7fd244143030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:56.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd248802700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2440af3e0 0x7fd244143030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:56.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd248802700 1 -- 172.21.15.140:0/3726830089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2440b5990 msgr2=0x7fd244143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:56.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd248802700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2440b5990 0x7fd244143520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:56.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd248802700 1 -- 172.21.15.140:0/3726830089 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c00b1e0 con 0x7fd24405d210 2022-07-21T01:55:56.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd248802700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd24405d210 0x7fd2440b3f10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fd22c002f70 tx=0x7fd22c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:56.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd241ffb700 1 -- 172.21.15.140:0/3726830089 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd22c013e90 con 0x7fd24405d210 2022-07-21T01:55:56.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd241ffb700 1 -- 172.21.15.140:0/3726830089 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c006850 con 0x7fd24405d210 2022-07-21T01:55:56.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd249804700 1 -- 172.21.15.140:0/3726830089 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24414a430 con 0x7fd24405d210 2022-07-21T01:55:56.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd241ffb700 1 -- 172.21.15.140:0/3726830089 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd22c01e5f0 con 0x7fd24405d210 2022-07-21T01:55:56.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd249804700 1 -- 172.21.15.140:0/3726830089 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd244066ee0 con 0x7fd24405d210 2022-07-21T01:55:56.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd249804700 1 -- 172.21.15.140:0/3726830089 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd244064f20 con 0x7fd24405d210 2022-07-21T01:55:56.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.226+0000 7fd241ffb700 1 -- 172.21.15.140:0/3726830089 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd22c025030 con 0x7fd24405d210 2022-07-21T01:55:56.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.230+0000 7fd241ffb700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd228053b00 0x7fd228055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:56.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.230+0000 7fd241ffb700 1 -- 172.21.15.140:0/3726830089 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd22c06ce30 con 0x7fd24405d210 2022-07-21T01:55:56.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.230+0000 7fd243fff700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd228053b00 0x7fd228055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:56.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.230+0000 7fd243fff700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd228053b00 0x7fd228055fc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fd23400d000 tx=0x7fd234007d80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:56.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.234+0000 7fd241ffb700 1 -- 172.21.15.140:0/3726830089 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd244064f20 con 0x7fd24405d210 2022-07-21T01:55:56.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:56.358+0000 7fd249804700 1 -- 172.21.15.140:0/3726830089 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd244064f20 con 0x7fd24405d210 2022-07-21T01:55:57.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.122+0000 7fd241ffb700 1 -- 172.21.15.140:0/3726830089 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) v1 ==== 185+0+0 (secure 0 0 0) 0x7fd244064f20 con 0x7fd24405d210 2022-07-21T01:55:57.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.126+0000 7fd249804700 1 -- 172.21.15.140:0/3726830089 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd228053b00 msgr2=0x7fd228055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:57.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.126+0000 7fd249804700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd228053b00 0x7fd228055fc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fd23400d000 tx=0x7fd234007d80).stop 2022-07-21T01:55:57.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.126+0000 7fd249804700 1 -- 172.21.15.140:0/3726830089 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd24405d210 msgr2=0x7fd2440b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:57.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.126+0000 7fd249804700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd24405d210 0x7fd2440b3f10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fd22c002f70 tx=0x7fd22c014000).stop 2022-07-21T01:55:57.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.126+0000 7fd249804700 1 -- 172.21.15.140:0/3726830089 shutdown_connections 2022-07-21T01:55:57.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.126+0000 7fd249804700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd228053b00 0x7fd228055fc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:57.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.126+0000 7fd249804700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2440b5990 0x7fd244143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:57.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.126+0000 7fd249804700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2440af3e0 0x7fd244143030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:57.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.126+0000 7fd249804700 1 --2- 172.21.15.140:0/3726830089 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd24405d210 0x7fd2440b3f10 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:57.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.126+0000 7fd249804700 1 -- 172.21.15.140:0/3726830089 >> 172.21.15.140:0/3726830089 conn(0x7fd244063480 msgr2=0x7fd244141220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:57.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.130+0000 7fd249804700 1 -- 172.21.15.140:0/3726830089 shutdown_connections 2022-07-21T01:55:57.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.130+0000 7fd249804700 1 -- 172.21.15.140:0/3726830089 wait complete. 2022-07-21T01:55:57.135 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:55:57.145 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:55:57.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.250+0000 7fbc61c1b700 1 Processor -- start 2022-07-21T01:55:57.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.250+0000 7fbc61c1b700 1 -- start start 2022-07-21T01:55:57.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc61c1b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc5c0aa9d0 0x7fbc5c060b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:57.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc61c1b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5c061980 con 0x7fbc5c05bfa0 2022-07-21T01:55:57.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc61c1b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5c142d40 con 0x7fbc5c060770 2022-07-21T01:55:57.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc61c1b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0b2260 con 0x7fbc5c0aa9d0 2022-07-21T01:55:57.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc5bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fbc5c05bfa0 0x7fbc5c1438c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52670/0 (socket says 172.21.15.140:52670) 2022-07-21T01:55:57.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc5bfff700 1 -- 172.21.15.140:0/121778777 learned_addr learned my addr 172.21.15.140:0/121778777 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:57.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc52ffd700 1 --2- 172.21.15.140:0/121778777 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc5c0aa9d0 0x7fbc5c060b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:57.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc5affd700 1 -- 172.21.15.140:0/121778777 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283432666 0 0) 0x7fbc5c061980 con 0x7fbc5c05bfa0 2022-07-21T01:55:57.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc5affd700 1 -- 172.21.15.140:0/121778777 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc3c003260 con 0x7fbc5c05bfa0 2022-07-21T01:55:57.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc5affd700 1 -- 172.21.15.140:0/121778777 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017006304 0 0) 0x7fbc4c001110 con 0x7fbc5c060770 2022-07-21T01:55:57.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc5affd700 1 -- 172.21.15.140:0/121778777 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc5c061980 con 0x7fbc5c060770 2022-07-21T01:55:57.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc52ffd700 1 -- 172.21.15.140:0/121778777 >> v1:172.21.15.152:6789/0 conn(0x7fbc5c060770 legacy=0x7fbc5c0aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:57.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc5affd700 1 -- 172.21.15.140:0/121778777 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4248165604 0 0) 0x7fbc3c003260 con 0x7fbc5c05bfa0 2022-07-21T01:55:57.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc52ffd700 1 -- 172.21.15.140:0/121778777 >> v1:172.21.15.140:6789/0 conn(0x7fbc5c05bfa0 legacy=0x7fbc5c1438c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:57.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc52ffd700 1 -- 172.21.15.140:0/121778777 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c061bb0 con 0x7fbc5c0aa9d0 2022-07-21T01:55:57.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc52ffd700 1 --2- 172.21.15.140:0/121778777 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc5c0aa9d0 0x7fbc5c060b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fbc54001b80 tx=0x7fbc44006df0).ready entity=mon.2 client_cookie=90c4accd7d497bf1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:57.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc5affd700 1 -- 172.21.15.140:0/121778777 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc540019e0 con 0x7fbc5c0aa9d0 2022-07-21T01:55:57.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc5affd700 1 -- 172.21.15.140:0/121778777 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc44000e90 con 0x7fbc5c0aa9d0 2022-07-21T01:55:57.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc5affd700 1 -- 172.21.15.140:0/121778777 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc44012460 con 0x7fbc5c0aa9d0 2022-07-21T01:55:57.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc61c1b700 1 -- 172.21.15.140:0/121778777 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc5c0aa9d0 msgr2=0x7fbc5c060b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:57.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.254+0000 7fbc61c1b700 1 --2- 172.21.15.140:0/121778777 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc5c0aa9d0 0x7fbc5c060b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fbc54001b80 tx=0x7fbc44006df0).stop 2022-07-21T01:55:57.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 -- 172.21.15.140:0/121778777 shutdown_connections 2022-07-21T01:55:57.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 --2- 172.21.15.140:0/121778777 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc5c0aa9d0 0x7fbc5c060b50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:57.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 -- 172.21.15.140:0/121778777 >> 172.21.15.140:0/121778777 conn(0x7fbc5c0b5910 msgr2=0x7fbc5c0b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:57.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 -- 172.21.15.140:0/121778777 shutdown_connections 2022-07-21T01:55:57.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 -- 172.21.15.140:0/121778777 wait complete. 2022-07-21T01:55:57.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 Processor -- start 2022-07-21T01:55:57.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 -- start start 2022-07-21T01:55:57.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc5c05bfa0 0x7fbc5c144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:57.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc5c060770 0x7fbc5c0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:57.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0aa9d0 0x7fbc5c0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:57.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c1464c0 con 0x7fbc5c060770 2022-07-21T01:55:57.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c065d60 con 0x7fbc5c0aa9d0 2022-07-21T01:55:57.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc61c1b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0ada00 con 0x7fbc5c05bfa0 2022-07-21T01:55:57.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc52ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc5c060770 0x7fbc5c0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:57.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc52ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc5c060770 0x7fbc5c0ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38528/0 (socket says 172.21.15.140:38528) 2022-07-21T01:55:57.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc52ffd700 1 -- 172.21.15.140:0/2286974259 learned_addr learned my addr 172.21.15.140:0/2286974259 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:57.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc5bfff700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0aa9d0 0x7fbc5c0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:57.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.258+0000 7fbc5b7fe700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc5c05bfa0 0x7fbc5c144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:57.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc52ffd700 1 -- 172.21.15.140:0/2286974259 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc5c05bfa0 msgr2=0x7fbc5c144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:57.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc52ffd700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc5c05bfa0 0x7fbc5c144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:57.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc52ffd700 1 -- 172.21.15.140:0/2286974259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0aa9d0 msgr2=0x7fbc5c0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:57.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc52ffd700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0aa9d0 0x7fbc5c0ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:57.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc52ffd700 1 -- 172.21.15.140:0/2286974259 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4400b180 con 0x7fbc5c060770 2022-07-21T01:55:57.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc52ffd700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc5c060770 0x7fbc5c0ad020 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7fbc54001b80 tx=0x7fbc44014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:57.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc597fa700 1 -- 172.21.15.140:0/2286974259 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc4400d920 con 0x7fbc5c060770 2022-07-21T01:55:57.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc597fa700 1 -- 172.21.15.140:0/2286974259 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4400da80 con 0x7fbc5c060770 2022-07-21T01:55:57.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc61c1b700 1 -- 172.21.15.140:0/2286974259 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c066c00 con 0x7fbc5c060770 2022-07-21T01:55:57.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc597fa700 1 -- 172.21.15.140:0/2286974259 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc4400dde0 con 0x7fbc5c060770 2022-07-21T01:55:57.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc61c1b700 1 -- 172.21.15.140:0/2286974259 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c145b20 con 0x7fbc5c060770 2022-07-21T01:55:57.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc61c1b700 1 -- 172.21.15.140:0/2286974259 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c0670a0 con 0x7fbc5c060770 2022-07-21T01:55:57.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc597fa700 1 -- 172.21.15.140:0/2286974259 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbc4401c070 con 0x7fbc5c060770 2022-07-21T01:55:57.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc597fa700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc40053b00 0x7fbc40055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:57.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc597fa700 1 -- 172.21.15.140:0/2286974259 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5721+0+0 (secure 0 0 0) 0x7fbc4406c580 con 0x7fbc5c060770 2022-07-21T01:55:57.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc5b7fe700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc40053b00 0x7fbc40055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:57.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.262+0000 7fbc5b7fe700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc40053b00 0x7fbc40055fc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fbc4c00d000 tx=0x7fbc4c0027e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:57.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.270+0000 7fbc597fa700 1 -- 172.21.15.140:0/2286974259 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbc44043640 con 0x7fbc5c060770 2022-07-21T01:55:57.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:57.394+0000 7fbc61c1b700 1 -- 172.21.15.140:0/2286974259 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbc5c0acd30 con 0x7fbc5c060770 2022-07-21T01:55:58.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.138+0000 7fbc597fa700 1 -- 172.21.15.140:0/2286974259 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) v1 ==== 173+0+0 (secure 0 0 0) 0x7fbc440230d0 con 0x7fbc5c060770 2022-07-21T01:55:58.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 -- 172.21.15.140:0/2286974259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc40053b00 msgr2=0x7fbc40055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc40053b00 0x7fbc40055fc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fbc4c00d000 tx=0x7fbc4c0027e0).stop 2022-07-21T01:55:58.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 -- 172.21.15.140:0/2286974259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc5c060770 msgr2=0x7fbc5c0ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc5c060770 0x7fbc5c0ad020 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7fbc54001b80 tx=0x7fbc44014000).stop 2022-07-21T01:55:58.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 -- 172.21.15.140:0/2286974259 shutdown_connections 2022-07-21T01:55:58.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbc40053b00 0x7fbc40055fc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc5c0aa9d0 0x7fbc5c0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbc5c060770 0x7fbc5c0ad020 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 --2- 172.21.15.140:0/2286974259 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbc5c05bfa0 0x7fbc5c144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 -- 172.21.15.140:0/2286974259 >> 172.21.15.140:0/2286974259 conn(0x7fbc5c0b5910 msgr2=0x7fbc5c142ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:58.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 -- 172.21.15.140:0/2286974259 shutdown_connections 2022-07-21T01:55:58.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.142+0000 7fbc61c1b700 1 -- 172.21.15.140:0/2286974259 wait complete. 2022-07-21T01:55:58.154 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:55:58.160 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-07-21T01:55:58.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.262+0000 7efdf9bd9700 1 Processor -- start 2022-07-21T01:55:58.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.262+0000 7efdf9bd9700 1 -- start start 2022-07-21T01:55:58.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf40b5990 0x7efdf4146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:58.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efdf405d210 0x7efdf414a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:58.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf4146c70 con 0x7efdf40af3e0 2022-07-21T01:55:58.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf40ad050 con 0x7efdf405d210 2022-07-21T01:55:58.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40b17e0 con 0x7efdf40b5990 2022-07-21T01:55:58.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf93d8700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efdf405d210 0x7efdf414a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:58.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf8bd7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf40b5990 0x7efdf4146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:58.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf3fff700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7efdf40af3e0 0x7efdf4146560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:53794/0 (socket says 172.21.15.140:53794) 2022-07-21T01:55:58.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf3fff700 1 -- 172.21.15.140:0/328051294 learned_addr learned my addr 172.21.15.140:0/328051294 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:58.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf93d8700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efdf405d210 0x7efdf414a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:44278/0 (socket says 172.21.15.140:44278) 2022-07-21T01:55:58.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf8bd7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf40b5990 0x7efdf4146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:35264/0 (socket says 172.21.15.140:35264) 2022-07-21T01:55:58.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf93d8700 1 -- 172.21.15.140:0/328051294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf40b5990 msgr2=0x7efdf4146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:58.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf93d8700 1 --2- 172.21.15.140:0/328051294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf40b5990 0x7efdf4146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf93d8700 1 -- 172.21.15.140:0/328051294 >> v1:172.21.15.152:6789/0 conn(0x7efdf40af3e0 legacy=0x7efdf4146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf93d8700 1 -- 172.21.15.140:0/328051294 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf40b5d70 con 0x7efdf405d210 2022-07-21T01:55:58.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf93d8700 1 --2- 172.21.15.140:0/328051294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdf405d210 0x7efdf414a360 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7efdec0032f0 tx=0x7efdec0091d0).ready entity=mon.1 client_cookie=e20eca26e9bde9e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:58.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf37fe700 1 -- 172.21.15.140:0/328051294 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdec00b6c0 con 0x7efdf405d210 2022-07-21T01:55:58.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf37fe700 1 -- 172.21.15.140:0/328051294 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec00b820 con 0x7efdf405d210 2022-07-21T01:55:58.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf37fe700 1 -- 172.21.15.140:0/328051294 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdec00b990 con 0x7efdf405d210 2022-07-21T01:55:58.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 -- 172.21.15.140:0/328051294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdf405d210 msgr2=0x7efdf414a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 --2- 172.21.15.140:0/328051294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdf405d210 0x7efdf414a360 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7efdec0032f0 tx=0x7efdec0091d0).stop 2022-07-21T01:55:58.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 -- 172.21.15.140:0/328051294 shutdown_connections 2022-07-21T01:55:58.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 --2- 172.21.15.140:0/328051294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdf405d210 0x7efdf414a360 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 --2- 172.21.15.140:0/328051294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf40b5990 0x7efdf4146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 -- 172.21.15.140:0/328051294 >> 172.21.15.140:0/328051294 conn(0x7efdf4063480 msgr2=0x7efdf40b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:58.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 -- 172.21.15.140:0/328051294 shutdown_connections 2022-07-21T01:55:58.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.266+0000 7efdf9bd9700 1 -- 172.21.15.140:0/328051294 wait complete. 2022-07-21T01:55:58.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf9bd9700 1 Processor -- start 2022-07-21T01:55:58.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf9bd9700 1 -- start start 2022-07-21T01:55:58.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf9bd9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf405d210 0x7efdf40b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:58.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf9bd9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efdf40af3e0 0x7efdf4066a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:58.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf9bd9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdf40b5990 0x7efdf4064da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:58.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf9bd9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4065f60 con 0x7efdf40af3e0 2022-07-21T01:55:58.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf9bd9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4145340 con 0x7efdf40b5990 2022-07-21T01:55:58.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf9bd9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf41455e0 con 0x7efdf405d210 2022-07-21T01:55:58.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efdf40af3e0 0x7efdf4066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:58.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf8bd7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf405d210 0x7efdf40b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:58.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf93d8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdf40b5990 0x7efdf4064da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:58.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf8bd7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf405d210 0x7efdf40b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:35270/0 (socket says 172.21.15.140:35270) 2022-07-21T01:55:58.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efdf40af3e0 0x7efdf4066a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38542/0 (socket says 172.21.15.140:38542) 2022-07-21T01:55:58.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf8bd7700 1 -- 172.21.15.140:0/2551547572 learned_addr learned my addr 172.21.15.140:0/2551547572 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:58.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf8bd7700 1 -- 172.21.15.140:0/2551547572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdf40b5990 msgr2=0x7efdf4064da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf8bd7700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdf40b5990 0x7efdf4064da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf8bd7700 1 -- 172.21.15.140:0/2551547572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efdf40af3e0 msgr2=0x7efdf4066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf8bd7700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efdf40af3e0 0x7efdf4066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf8bd7700 1 -- 172.21.15.140:0/2551547572 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec009470 con 0x7efdf405d210 2022-07-21T01:55:58.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf93d8700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdf40b5990 0x7efdf4064da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:58.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf3fff700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efdf40af3e0 0x7efdf4066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:55:58.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf8bd7700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf405d210 0x7efdf40b22a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7efde800c000 tx=0x7efde8006d90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:58.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf1ffb700 1 -- 172.21.15.140:0/2551547572 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efde8011750 con 0x7efdf405d210 2022-07-21T01:55:58.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf1ffb700 1 -- 172.21.15.140:0/2551547572 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde80118b0 con 0x7efdf405d210 2022-07-21T01:55:58.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf9bd9700 1 -- 172.21.15.140:0/2551547572 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf4064960 con 0x7efdf405d210 2022-07-21T01:55:58.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf1ffb700 1 -- 172.21.15.140:0/2551547572 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efde8011c10 con 0x7efdf405d210 2022-07-21T01:55:58.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.270+0000 7efdf9bd9700 1 -- 172.21.15.140:0/2551547572 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf413fd00 con 0x7efdf405d210 2022-07-21T01:55:58.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.274+0000 7efdf9bd9700 1 -- 172.21.15.140:0/2551547572 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf4064630 con 0x7efdf405d210 2022-07-21T01:55:58.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.274+0000 7efdf1ffb700 1 -- 172.21.15.140:0/2551547572 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efde8011d70 con 0x7efdf405d210 2022-07-21T01:55:58.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.274+0000 7efdf1ffb700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdd4053a70 0x7efdd4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:58.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.274+0000 7efdf1ffb700 1 -- 172.21.15.140:0/2551547572 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5339+0+0 (secure 0 0 0) 0x7efde806bca0 con 0x7efdf405d210 2022-07-21T01:55:58.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.274+0000 7efdf3fff700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdd4053a70 0x7efdd4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:58.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.274+0000 7efdf3fff700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdd4053a70 0x7efdd4055f30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7efde400d000 tx=0x7efde4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:58.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.282+0000 7efdf1ffb700 1 -- 172.21.15.140:0/2551547572 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efdf4064630 con 0x7efdf405d210 2022-07-21T01:55:58.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.410+0000 7efdf9bd9700 1 -- 172.21.15.140:0/2551547572 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efdf4064630 con 0x7efdf405d210 2022-07-21T01:55:58.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf1ffb700 1 -- 172.21.15.140:0/2551547572 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7efdf4064630 con 0x7efdf405d210 2022-07-21T01:55:58.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 -- 172.21.15.140:0/2551547572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdd4053a70 msgr2=0x7efdd4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdd4053a70 0x7efdd4055f30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7efde400d000 tx=0x7efde4009040).stop 2022-07-21T01:55:58.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 -- 172.21.15.140:0/2551547572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf405d210 msgr2=0x7efdf40b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf405d210 0x7efdf40b22a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7efde800c000 tx=0x7efde8006d90).stop 2022-07-21T01:55:58.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 -- 172.21.15.140:0/2551547572 shutdown_connections 2022-07-21T01:55:58.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efdd4053a70 0x7efdd4055f30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdf40b5990 0x7efdf4064da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efdf40af3e0 0x7efdf4066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 --2- 172.21.15.140:0/2551547572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efdf405d210 0x7efdf40b22a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 -- 172.21.15.140:0/2551547572 >> 172.21.15.140:0/2551547572 conn(0x7efdf4063480 msgr2=0x7efdf414aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:58.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 -- 172.21.15.140:0/2551547572 shutdown_connections 2022-07-21T01:55:58.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.426+0000 7efdf9bd9700 1 -- 172.21.15.140:0/2551547572 wait complete. 2022-07-21T01:55:58.443 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2022-07-21T01:55:58.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.538+0000 7f28f907e700 1 Processor -- start 2022-07-21T01:55:58.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.538+0000 7f28f907e700 1 -- start start 2022-07-21T01:55:58.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f907e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40af920 0x7f28f40b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:58.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f907e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f40adf70 con 0x7f28f40b59d0 2022-07-21T01:55:58.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f907e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f40b5db0 con 0x7f28f405d0c0 2022-07-21T01:55:58.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f907e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f4146300 con 0x7f28f40af920 2022-07-21T01:55:58.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f3fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f28f40b59d0 0x7f28f40ae3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52696/0 (socket says 172.21.15.140:52696) 2022-07-21T01:55:58.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f3fff700 1 -- 172.21.15.140:0/2957148016 learned_addr learned my addr 172.21.15.140:0/2957148016 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:58.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f37fe700 1 --2- 172.21.15.140:0/2957148016 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40af920 0x7f28f40b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:58.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f2ffd700 1 -- 172.21.15.140:0/2957148016 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124742858 0 0) 0x7f28f40adf70 con 0x7f28f40b59d0 2022-07-21T01:55:58.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f2ffd700 1 -- 172.21.15.140:0/2957148016 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28d8003260 con 0x7f28f40b59d0 2022-07-21T01:55:58.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f2ffd700 1 -- 172.21.15.140:0/2957148016 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112513352 0 0) 0x7f28f40b5db0 con 0x7f28f405d0c0 2022-07-21T01:55:58.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f2ffd700 1 -- 172.21.15.140:0/2957148016 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28f40adf70 con 0x7f28f405d0c0 2022-07-21T01:55:58.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f37fe700 1 -- 172.21.15.140:0/2957148016 >> v1:172.21.15.152:6789/0 conn(0x7f28f405d0c0 legacy=0x7f28f4141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f37fe700 1 -- 172.21.15.140:0/2957148016 >> v1:172.21.15.140:6789/0 conn(0x7f28f40b59d0 legacy=0x7f28f40ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f37fe700 1 -- 172.21.15.140:0/2957148016 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f4146060 con 0x7f28f40af920 2022-07-21T01:55:58.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f37fe700 1 --2- 172.21.15.140:0/2957148016 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40af920 0x7f28f40b0040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f28f40add70 tx=0x7f28dc006df0).ready entity=mon.2 client_cookie=244b49a104715411 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:58.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f2ffd700 1 -- 172.21.15.140:0/2957148016 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28dc00d5c0 con 0x7f28f40af920 2022-07-21T01:55:58.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f2ffd700 1 -- 172.21.15.140:0/2957148016 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc000e90 con 0x7f28f40af920 2022-07-21T01:55:58.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f2ffd700 1 -- 172.21.15.140:0/2957148016 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28dc00d910 con 0x7f28f40af920 2022-07-21T01:55:58.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f907e700 1 -- 172.21.15.140:0/2957148016 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40af920 msgr2=0x7f28f40b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f907e700 1 --2- 172.21.15.140:0/2957148016 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40af920 0x7f28f40b0040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f28f40add70 tx=0x7f28dc006df0).stop 2022-07-21T01:55:58.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f907e700 1 -- 172.21.15.140:0/2957148016 shutdown_connections 2022-07-21T01:55:58.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f907e700 1 --2- 172.21.15.140:0/2957148016 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40af920 0x7f28f40b0040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f907e700 1 -- 172.21.15.140:0/2957148016 >> 172.21.15.140:0/2957148016 conn(0x7f28f40634c0 msgr2=0x7f28f40b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:58.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f907e700 1 -- 172.21.15.140:0/2957148016 shutdown_connections 2022-07-21T01:55:58.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.542+0000 7f28f907e700 1 -- 172.21.15.140:0/2957148016 wait complete. 2022-07-21T01:55:58.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f907e700 1 Processor -- start 2022-07-21T01:55:58.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f907e700 1 -- start start 2022-07-21T01:55:58.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f907e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f405d0c0 0x7f28f4155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:58.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f907e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f40af920 0x7f28f4140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:58.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f907e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f40b59d0 0x7f28f413ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:58.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f907e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f4113510 con 0x7f28f40af920 2022-07-21T01:55:58.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f907e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f4066560 con 0x7f28f40b59d0 2022-07-21T01:55:58.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f907e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f4066800 con 0x7f28f405d0c0 2022-07-21T01:55:58.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f40af920 0x7f28f4140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:58.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f40af920 0x7f28f4140d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38556/0 (socket says 172.21.15.140:38556) 2022-07-21T01:55:58.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f37fe700 1 -- 172.21.15.140:0/1781571260 learned_addr learned my addr 172.21.15.140:0/1781571260 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:58.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f3fff700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f405d0c0 0x7f28f4155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:58.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f887d700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f40b59d0 0x7f28f413ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:58.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f37fe700 1 -- 172.21.15.140:0/1781571260 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f405d0c0 msgr2=0x7f28f4155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f37fe700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f405d0c0 0x7f28f4155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f37fe700 1 -- 172.21.15.140:0/1781571260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f40b59d0 msgr2=0x7f28f413ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:58.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f37fe700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f40b59d0 0x7f28f413ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:58.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f37fe700 1 -- 172.21.15.140:0/1781571260 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc00b180 con 0x7f28f40af920 2022-07-21T01:55:58.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f37fe700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f40af920 0x7f28f4140d40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f28f40add70 tx=0x7f28dc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:58.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f17fa700 1 -- 172.21.15.140:0/1781571260 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28dc016070 con 0x7f28f40af920 2022-07-21T01:55:58.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f17fa700 1 -- 172.21.15.140:0/1781571260 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc00de40 con 0x7f28f40af920 2022-07-21T01:55:58.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f907e700 1 -- 172.21.15.140:0/1781571260 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f4066aa0 con 0x7f28f40af920 2022-07-21T01:55:58.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f17fa700 1 -- 172.21.15.140:0/1781571260 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28dc013d20 con 0x7f28f40af920 2022-07-21T01:55:58.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.546+0000 7f28f907e700 1 -- 172.21.15.140:0/1781571260 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f4112580 con 0x7f28f40af920 2022-07-21T01:55:58.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.550+0000 7f28f907e700 1 -- 172.21.15.140:0/1781571260 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f40af2d0 con 0x7f28f40af920 2022-07-21T01:55:58.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.550+0000 7f28f17fa700 1 -- 172.21.15.140:0/1781571260 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f28dc025030 con 0x7f28f40af920 2022-07-21T01:55:58.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.550+0000 7f28f17fa700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28cc053950 0x7f28cc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:58.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.550+0000 7f28f17fa700 1 -- 172.21.15.140:0/1781571260 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f28dc06caf0 con 0x7f28f40af920 2022-07-21T01:55:58.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.550+0000 7f28f3fff700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28cc053950 0x7f28cc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:58.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.550+0000 7f28f3fff700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28cc053950 0x7f28cc055e10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f28e400d000 tx=0x7f28e4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:58.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.554+0000 7f28f17fa700 1 -- 172.21.15.140:0/1781571260 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f28dc043cf0 con 0x7f28f40af920 2022-07-21T01:55:58.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:58.718+0000 7f28f907e700 1 -- 172.21.15.140:0/1781571260 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7f28f40574e0 con 0x7f28f40af920 2022-07-21T01:55:59.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.138+0000 7f28f17fa700 1 -- 172.21.15.140:0/1781571260 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v334) v1 ==== 138+0+0 (secure 0 0 0) 0x7f28dc0230d0 con 0x7f28f40af920 2022-07-21T01:55:59.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 -- 172.21.15.140:0/1781571260 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28cc053950 msgr2=0x7f28cc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:59.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28cc053950 0x7f28cc055e10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f28e400d000 tx=0x7f28e4009040).stop 2022-07-21T01:55:59.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 -- 172.21.15.140:0/1781571260 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f40af920 msgr2=0x7f28f4140d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:59.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f40af920 0x7f28f4140d40 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f28f40add70 tx=0x7f28dc014000).stop 2022-07-21T01:55:59.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 -- 172.21.15.140:0/1781571260 shutdown_connections 2022-07-21T01:55:59.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28cc053950 0x7f28cc055e10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:59.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f40b59d0 0x7f28f413ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:59.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f40af920 0x7f28f4140d40 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:59.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 --2- 172.21.15.140:0/1781571260 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f405d0c0 0x7f28f4155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:59.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 -- 172.21.15.140:0/1781571260 >> 172.21.15.140:0/1781571260 conn(0x7f28f40634c0 msgr2=0x7f28f413fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:59.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 -- 172.21.15.140:0/1781571260 shutdown_connections 2022-07-21T01:55:59.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.142+0000 7f28f907e700 1 -- 172.21.15.140:0/1781571260 wait complete. 2022-07-21T01:55:59.161 INFO:tasks.cephfs_test_runner:test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) ... ok 2022-07-21T01:55:59.161 INFO:tasks.cephfs_test_runner:Starting test: test_filtered_df (tasks.cephfs.test_misc.TestMisc) 2022-07-21T01:55:59.162 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2022-07-21T01:55:59.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 Processor -- start 2022-07-21T01:55:59.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 -- start start 2022-07-21T01:55:59.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1bfc0b5650 0x7f1bfc0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:59.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1bfc0af8e0 0x7f1bfc05bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:59.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bfc147010 con 0x7f1bfc0adff0 2022-07-21T01:55:59.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bfc1472b0 con 0x7f1bfc0b5650 2022-07-21T01:55:59.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bfc146d40 con 0x7f1bfc0af8e0 2022-07-21T01:55:59.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfa59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1bfc0b5650 0x7f1bfc0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:59.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bf9d9b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1bfc0adff0 0x7f1bfc147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52708/0 (socket says 172.21.15.140:52708) 2022-07-21T01:55:59.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bf9d9b700 1 -- 172.21.15.140:0/2609302164 learned_addr learned my addr 172.21.15.140:0/2609302164 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:55:59.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfad9d700 1 --2- 172.21.15.140:0/2609302164 >> v2:172.21.15.152:3300/0 conn(0x7f1bfc0af8e0 0x7f1bfc05bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:59.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfa59c700 1 -- 172.21.15.140:0/2609302164 >> v2:172.21.15.152:3300/0 conn(0x7f1bfc0af8e0 msgr2=0x7f1bfc05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:55:59.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfa59c700 1 --2- 172.21.15.140:0/2609302164 >> v2:172.21.15.152:3300/0 conn(0x7f1bfc0af8e0 0x7f1bfc05bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:59.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfa59c700 1 -- 172.21.15.140:0/2609302164 >> v1:172.21.15.140:6789/0 conn(0x7f1bfc0adff0 legacy=0x7f1bfc147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:59.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfa59c700 1 -- 172.21.15.140:0/2609302164 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc0acae0 con 0x7f1bfc0b5650 2022-07-21T01:55:59.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfa59c700 1 --2- 172.21.15.140:0/2609302164 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1bfc0b5650 0x7f1bfc0b5a30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f1bec002f70 tx=0x7f1bec006df0).ready entity=mon.0 client_cookie=e7a37b5b1d64f90b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:59.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bf959a700 1 -- 172.21.15.140:0/2609302164 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bec00d570 con 0x7f1bfc0b5650 2022-07-21T01:55:59.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bf959a700 1 -- 172.21.15.140:0/2609302164 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec000e90 con 0x7f1bfc0b5650 2022-07-21T01:55:59.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bf959a700 1 -- 172.21.15.140:0/2609302164 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bec00d8c0 con 0x7f1bfc0b5650 2022-07-21T01:55:59.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 -- 172.21.15.140:0/2609302164 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1bfc0b5650 msgr2=0x7f1bfc0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:59.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 --2- 172.21.15.140:0/2609302164 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1bfc0b5650 0x7f1bfc0b5a30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f1bec002f70 tx=0x7f1bec006df0).stop 2022-07-21T01:55:59.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 -- 172.21.15.140:0/2609302164 shutdown_connections 2022-07-21T01:55:59.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 --2- 172.21.15.140:0/2609302164 >> v2:172.21.15.152:3300/0 conn(0x7f1bfc0af8e0 0x7f1bfc05bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:59.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 --2- 172.21.15.140:0/2609302164 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1bfc0b5650 0x7f1bfc0b5a30 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:59.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 -- 172.21.15.140:0/2609302164 >> 172.21.15.140:0/2609302164 conn(0x7f1bfc0634e0 msgr2=0x7f1bfc0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:55:59.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 -- 172.21.15.140:0/2609302164 shutdown_connections 2022-07-21T01:55:59.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.270+0000 7f1bfb59e700 1 -- 172.21.15.140:0/2609302164 wait complete. 2022-07-21T01:55:59.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.274+0000 7f1bfb59e700 1 Processor -- start 2022-07-21T01:55:59.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.274+0000 7f1bfb59e700 1 -- start start 2022-07-21T01:55:59.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfb59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1bfc0adff0 0x7f1bfc0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:59.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfb59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1bfc0af8e0 0x7f1bfc10f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:59.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfb59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1bfc0b5650 0x7f1bfc10f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:59.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfb59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc067160 con 0x7f1bfc0adff0 2022-07-21T01:55:59.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfb59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc111140 con 0x7f1bfc0b5650 2022-07-21T01:55:59.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfb59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc1113e0 con 0x7f1bfc0af8e0 2022-07-21T01:55:59.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfa59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1bfc0adff0 0x7f1bfc0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:59.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfa59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1bfc0adff0 0x7f1bfc0b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38568/0 (socket says 172.21.15.140:38568) 2022-07-21T01:55:59.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfa59c700 1 -- 172.21.15.140:0/4293544802 learned_addr learned my addr 172.21.15.140:0/4293544802 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:55:59.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfad9d700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1bfc0b5650 0x7f1bfc10f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:59.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bf9d9b700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1bfc0af8e0 0x7f1bfc10f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:59.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfa59c700 1 -- 172.21.15.140:0/4293544802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1bfc0af8e0 msgr2=0x7f1bfc10f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:59.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfa59c700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1bfc0af8e0 0x7f1bfc10f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:59.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfa59c700 1 -- 172.21.15.140:0/4293544802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1bfc0b5650 msgr2=0x7f1bfc10f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:55:59.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfa59c700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1bfc0b5650 0x7f1bfc10f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:55:59.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfa59c700 1 -- 172.21.15.140:0/4293544802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec00b180 con 0x7f1bfc0adff0 2022-07-21T01:55:59.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfad9d700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1bfc0b5650 0x7f1bfc10f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:55:59.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfa59c700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1bfc0adff0 0x7f1bfc0b40a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f1bec002f70 tx=0x7f1bec014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:59.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1beb7fe700 1 -- 172.21.15.140:0/4293544802 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bec016070 con 0x7f1bfc0adff0 2022-07-21T01:55:59.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1beb7fe700 1 -- 172.21.15.140:0/4293544802 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec00d8f0 con 0x7f1bfc0adff0 2022-07-21T01:55:59.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfb59e700 1 -- 172.21.15.140:0/4293544802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bfc1104e0 con 0x7f1bfc0adff0 2022-07-21T01:55:59.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1beb7fe700 1 -- 172.21.15.140:0/4293544802 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bec01e450 con 0x7f1bfc0adff0 2022-07-21T01:55:59.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfb59e700 1 -- 172.21.15.140:0/4293544802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bfc066f20 con 0x7f1bfc0adff0 2022-07-21T01:55:59.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1beb7fe700 1 -- 172.21.15.140:0/4293544802 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1bec025030 con 0x7f1bfc0adff0 2022-07-21T01:55:59.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.278+0000 7f1bfb59e700 1 -- 172.21.15.140:0/4293544802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bfc0574e0 con 0x7f1bfc0adff0 2022-07-21T01:55:59.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.282+0000 7f1beb7fe700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bd8053b00 0x7f1bd8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:55:59.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.282+0000 7f1bf9d9b700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bd8053b00 0x7f1bd8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:55:59.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.282+0000 7f1beb7fe700 1 -- 172.21.15.140:0/4293544802 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f1bec06c9e0 con 0x7f1bfc0adff0 2022-07-21T01:55:59.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.282+0000 7f1bf9d9b700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bd8053b00 0x7f1bd8055fc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f1bf000b000 tx=0x7f1bf0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:55:59.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.290+0000 7f1beb7fe700 1 -- 172.21.15.140:0/4293544802 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1bec043c20 con 0x7f1bfc0adff0 2022-07-21T01:55:59.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:55:59.458+0000 7f1bfb59e700 1 -- 172.21.15.140:0/4293544802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f1bfc067ea0 con 0x7f1bfc0adff0 2022-07-21T01:56:00.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.142+0000 7f1beb7fe700 1 -- 172.21.15.140:0/4293544802 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v335) v1 ==== 130+0+0 (secure 0 0 0) 0x7f1bec023070 con 0x7f1bfc0adff0 2022-07-21T01:56:00.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.142+0000 7f1bfb59e700 1 -- 172.21.15.140:0/4293544802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bd8053b00 msgr2=0x7f1bd8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.142+0000 7f1bfb59e700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bd8053b00 0x7f1bd8055fc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f1bf000b000 tx=0x7f1bf0009040).stop 2022-07-21T01:56:00.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.142+0000 7f1bfb59e700 1 -- 172.21.15.140:0/4293544802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1bfc0adff0 msgr2=0x7f1bfc0b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.142+0000 7f1bfb59e700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1bfc0adff0 0x7f1bfc0b40a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f1bec002f70 tx=0x7f1bec014000).stop 2022-07-21T01:56:00.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.146+0000 7f1bfb59e700 1 -- 172.21.15.140:0/4293544802 shutdown_connections 2022-07-21T01:56:00.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.146+0000 7f1bfb59e700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bd8053b00 0x7f1bd8055fc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.146+0000 7f1bfb59e700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1bfc0b5650 0x7f1bfc10f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.146+0000 7f1bfb59e700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1bfc0af8e0 0x7f1bfc10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.146+0000 7f1bfb59e700 1 --2- 172.21.15.140:0/4293544802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1bfc0adff0 0x7f1bfc0b40a0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.146+0000 7f1bfb59e700 1 -- 172.21.15.140:0/4293544802 >> 172.21.15.140:0/4293544802 conn(0x7f1bfc0634e0 msgr2=0x7f1bfc0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:00.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.146+0000 7f1bfb59e700 1 -- 172.21.15.140:0/4293544802 shutdown_connections 2022-07-21T01:56:00.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.146+0000 7f1bfb59e700 1 -- 172.21.15.140:0/4293544802 wait complete. 2022-07-21T01:56:00.162 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:56:00.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.274+0000 7fb38d754700 1 Processor -- start 2022-07-21T01:56:00.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.274+0000 7fb38d754700 1 -- start start 2022-07-21T01:56:00.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.274+0000 7fb38d754700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb3880b2ec0 0x7fb3880b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:00.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.274+0000 7fb38d754700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb38805d160 0x7fb388142210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:00.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.274+0000 7fb38d754700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3880b3f10 con 0x7fb388142700 2022-07-21T01:56:00.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.274+0000 7fb38d754700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3880b4ea0 con 0x7fb3880b2ec0 2022-07-21T01:56:00.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.274+0000 7fb38d754700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb388142b30 con 0x7fb38805d160 2022-07-21T01:56:00.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.274+0000 7fb387fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb3880b2ec0 0x7fb3880b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:00.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb38cf53700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fb388142700 0x7fb3881461c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52724/0 (socket says 172.21.15.140:52724) 2022-07-21T01:56:00.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb38cf53700 1 -- 172.21.15.140:0/1299006406 learned_addr learned my addr 172.21.15.140:0/1299006406 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:00.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb3877fe700 1 --2- 172.21.15.140:0/1299006406 >> v2:172.21.15.152:3300/0 conn(0x7fb38805d160 0x7fb388142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:00.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb386ffd700 1 -- 172.21.15.140:0/1299006406 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020135433 0 0) 0x7fb3880b3f10 con 0x7fb388142700 2022-07-21T01:56:00.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb386ffd700 1 -- 172.21.15.140:0/1299006406 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb368003260 con 0x7fb388142700 2022-07-21T01:56:00.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb387fff700 1 -- 172.21.15.140:0/1299006406 >> v2:172.21.15.152:3300/0 conn(0x7fb38805d160 msgr2=0x7fb388142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:00.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb387fff700 1 --2- 172.21.15.140:0/1299006406 >> v2:172.21.15.152:3300/0 conn(0x7fb38805d160 0x7fb388142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb387fff700 1 -- 172.21.15.140:0/1299006406 >> v1:172.21.15.140:6789/0 conn(0x7fb388142700 legacy=0x7fb3881461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb387fff700 1 -- 172.21.15.140:0/1299006406 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880b3790 con 0x7fb3880b2ec0 2022-07-21T01:56:00.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb387fff700 1 --2- 172.21.15.140:0/1299006406 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880b2ec0 0x7fb3880b32a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7fb370002f70 tx=0x7fb370006df0).ready entity=mon.0 client_cookie=403dd1b23430093 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:00.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb386ffd700 1 -- 172.21.15.140:0/1299006406 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb37000d4f0 con 0x7fb3880b2ec0 2022-07-21T01:56:00.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb386ffd700 1 -- 172.21.15.140:0/1299006406 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb370000e90 con 0x7fb3880b2ec0 2022-07-21T01:56:00.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb386ffd700 1 -- 172.21.15.140:0/1299006406 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb37000d840 con 0x7fb3880b2ec0 2022-07-21T01:56:00.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb38d754700 1 -- 172.21.15.140:0/1299006406 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880b2ec0 msgr2=0x7fb3880b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb38d754700 1 --2- 172.21.15.140:0/1299006406 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880b2ec0 0x7fb3880b32a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7fb370002f70 tx=0x7fb370006df0).stop 2022-07-21T01:56:00.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb38d754700 1 -- 172.21.15.140:0/1299006406 shutdown_connections 2022-07-21T01:56:00.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb38d754700 1 --2- 172.21.15.140:0/1299006406 >> v2:172.21.15.152:3300/0 conn(0x7fb38805d160 0x7fb388142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb38d754700 1 --2- 172.21.15.140:0/1299006406 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880b2ec0 0x7fb3880b32a0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb38d754700 1 -- 172.21.15.140:0/1299006406 >> 172.21.15.140:0/1299006406 conn(0x7fb388066e20 msgr2=0x7fb388067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:00.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb38d754700 1 -- 172.21.15.140:0/1299006406 shutdown_connections 2022-07-21T01:56:00.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.278+0000 7fb38d754700 1 -- 172.21.15.140:0/1299006406 wait complete. 2022-07-21T01:56:00.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38d754700 1 Processor -- start 2022-07-21T01:56:00.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38d754700 1 -- start start 2022-07-21T01:56:00.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38d754700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb38805d160 0x7fb3881405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:00.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38d754700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3880b2ec0 0x7fb38813ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:00.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38d754700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb388142700 0x7fb38813b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:00.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38d754700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880ae610 con 0x7fb388142700 2022-07-21T01:56:00.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38d754700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880af0b0 con 0x7fb38805d160 2022-07-21T01:56:00.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38d754700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880af350 con 0x7fb3880b2ec0 2022-07-21T01:56:00.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38cf53700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb388142700 0x7fb38813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:00.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38cf53700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb388142700 0x7fb38813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38586/0 (socket says 172.21.15.140:38586) 2022-07-21T01:56:00.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38cf53700 1 -- 172.21.15.140:0/822462452 learned_addr learned my addr 172.21.15.140:0/822462452 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:00.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb3877fe700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3880b2ec0 0x7fb38813ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:00.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb387fff700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb38805d160 0x7fb3881405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:00.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38cf53700 1 -- 172.21.15.140:0/822462452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3880b2ec0 msgr2=0x7fb38813ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38cf53700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3880b2ec0 0x7fb38813ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38cf53700 1 -- 172.21.15.140:0/822462452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb38805d160 msgr2=0x7fb3881405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38cf53700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb38805d160 0x7fb3881405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38cf53700 1 -- 172.21.15.140:0/822462452 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37000b180 con 0x7fb388142700 2022-07-21T01:56:00.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb387fff700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb38805d160 0x7fb3881405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:00.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38cf53700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb388142700 0x7fb38813b160 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7fb380001490 tx=0x7fb380007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:00.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb3857fa700 1 -- 172.21.15.140:0/822462452 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb380005ba0 con 0x7fb388142700 2022-07-21T01:56:00.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb3857fa700 1 -- 172.21.15.140:0/822462452 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb380005d00 con 0x7fb388142700 2022-07-21T01:56:00.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb38d754700 1 -- 172.21.15.140:0/822462452 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3880ae370 con 0x7fb388142700 2022-07-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.282+0000 7fb3857fa700 1 -- 172.21.15.140:0/822462452 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3800072c0 con 0x7fb388142700 2022-07-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.286+0000 7fb38d754700 1 -- 172.21.15.140:0/822462452 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb388144a80 con 0x7fb388142700 2022-07-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.286+0000 7fb38d754700 1 -- 172.21.15.140:0/822462452 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb388145fd0 con 0x7fb388142700 2022-07-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.286+0000 7fb3857fa700 1 -- 172.21.15.140:0/822462452 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb38001a060 con 0x7fb388142700 2022-07-21T01:56:00.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.286+0000 7fb3857fa700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb36c053b00 0x7fb36c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:00.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.286+0000 7fb3857fa700 1 -- 172.21.15.140:0/822462452 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fb38006b6d0 con 0x7fb388142700 2022-07-21T01:56:00.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.286+0000 7fb387fff700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb36c053b00 0x7fb36c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:00.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.286+0000 7fb387fff700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb36c053b00 0x7fb36c055fc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fb370018000 tx=0x7fb370014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:00.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.290+0000 7fb3857fa700 1 -- 172.21.15.140:0/822462452 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb3800428d0 con 0x7fb388142700 2022-07-21T01:56:00.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.414+0000 7fb38d754700 1 -- 172.21.15.140:0/822462452 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb3880b2620 con 0x7fb388142700 2022-07-21T01:56:00.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb3857fa700 1 -- 172.21.15.140:0/822462452 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb380018030 con 0x7fb388142700 2022-07-21T01:56:00.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 -- 172.21.15.140:0/822462452 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb36c053b00 msgr2=0x7fb36c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb36c053b00 0x7fb36c055fc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fb370018000 tx=0x7fb370014000).stop 2022-07-21T01:56:00.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 -- 172.21.15.140:0/822462452 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb388142700 msgr2=0x7fb38813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb388142700 0x7fb38813b160 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7fb380001490 tx=0x7fb380007e00).stop 2022-07-21T01:56:00.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 -- 172.21.15.140:0/822462452 shutdown_connections 2022-07-21T01:56:00.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb36c053b00 0x7fb36c055fc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb388142700 0x7fb38813b160 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3880b2ec0 0x7fb38813ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 --2- 172.21.15.140:0/822462452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb38805d160 0x7fb3881405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 -- 172.21.15.140:0/822462452 >> 172.21.15.140:0/822462452 conn(0x7fb388066e20 msgr2=0x7fb38813c190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:00.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 -- 172.21.15.140:0/822462452 shutdown_connections 2022-07-21T01:56:00.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.422+0000 7fb38d754700 1 -- 172.21.15.140:0/822462452 wait complete. 2022-07-21T01:56:00.438 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:56:00.439 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:00.460 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:00.481 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:00.503 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:00.524 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:00.545 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:56:00.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:00.546 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:00.546 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:00.553 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:56:00.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:56:00.555 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:56:00.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:00.556 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:00.556 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:00.571 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:56:00.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:56:00.572 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:56:00.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:00.573 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:00.574 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:56:00.581 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:56:00.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:56:00.584 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:56:00.585 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:56:00.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.690+0000 7fe27f59e700 1 Processor -- start 2022-07-21T01:56:00.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.690+0000 7fe27f59e700 1 -- start start 2022-07-21T01:56:00.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.690+0000 7fe27f59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe28005fa50 0x7fe28014a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:00.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.690+0000 7fe27f59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe280060980 con 0x7fe2800ac3a0 2022-07-21T01:56:00.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.690+0000 7fe27f59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2800b22d0 con 0x7fe2800b1ef0 2022-07-21T01:56:00.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.690+0000 7fe27f59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe280098810 con 0x7fe28005fa50 2022-07-21T01:56:00.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.690+0000 7fe27ed9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe28005fa50 0x7fe28014a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:00.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.690+0000 7fe27ed9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe28005fa50 0x7fe28014a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38594/0 (socket says 172.21.15.140:38594) 2022-07-21T01:56:00.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.690+0000 7fe27ed9d700 1 -- 172.21.15.140:0/26692789 learned_addr learned my addr 172.21.15.140:0/26692789 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:00.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27ed9d700 1 -- 172.21.15.140:0/26692789 >> v1:172.21.15.152:6789/0 conn(0x7fe2800b1ef0 legacy=0x7fe280142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27ed9d700 1 -- 172.21.15.140:0/26692789 >> v1:172.21.15.140:6789/0 conn(0x7fe2800ac3a0 legacy=0x7fe280143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27ed9d700 1 -- 172.21.15.140:0/26692789 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe280060bb0 con 0x7fe28005fa50 2022-07-21T01:56:00.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27ed9d700 1 --2- 172.21.15.140:0/26692789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe28005fa50 0x7fe28014a530 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7fe2780032f0 tx=0x7fe278009290).ready entity=mon.0 client_cookie=f272a4723f4deb6d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:00.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27d59a700 1 -- 172.21.15.140:0/26692789 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe27800b8a0 con 0x7fe28005fa50 2022-07-21T01:56:00.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27d59a700 1 -- 172.21.15.140:0/26692789 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27800ba00 con 0x7fe28005fa50 2022-07-21T01:56:00.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27d59a700 1 -- 172.21.15.140:0/26692789 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe27800bb70 con 0x7fe28005fa50 2022-07-21T01:56:00.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27f59e700 1 -- 172.21.15.140:0/26692789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe28005fa50 msgr2=0x7fe28014a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27f59e700 1 --2- 172.21.15.140:0/26692789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe28005fa50 0x7fe28014a530 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7fe2780032f0 tx=0x7fe278009290).stop 2022-07-21T01:56:00.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27f59e700 1 -- 172.21.15.140:0/26692789 shutdown_connections 2022-07-21T01:56:00.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27f59e700 1 --2- 172.21.15.140:0/26692789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe28005fa50 0x7fe28014a530 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27f59e700 1 -- 172.21.15.140:0/26692789 >> 172.21.15.140:0/26692789 conn(0x7fe2800b2ca0 msgr2=0x7fe2800b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:00.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27f59e700 1 -- 172.21.15.140:0/26692789 shutdown_connections 2022-07-21T01:56:00.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.694+0000 7fe27f59e700 1 -- 172.21.15.140:0/26692789 wait complete. 2022-07-21T01:56:00.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27f59e700 1 Processor -- start 2022-07-21T01:56:00.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27f59e700 1 -- start start 2022-07-21T01:56:00.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27f59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe28005fa50 0x7fe2800af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:00.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2800ac3a0 0x7fe28009b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:00.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27f59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2800b1ef0 0x7fe28009b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:00.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27f59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe280064ca0 con 0x7fe2800b1ef0 2022-07-21T01:56:00.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27f59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28009abd0 con 0x7fe28005fa50 2022-07-21T01:56:00.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2800659d0 con 0x7fe2800ac3a0 2022-07-21T01:56:00.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2800b1ef0 0x7fe28009b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:00.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2800b1ef0 0x7fe28009b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38600/0 (socket says 172.21.15.140:38600) 2022-07-21T01:56:00.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27ed9d700 1 -- 172.21.15.140:0/4126214584 learned_addr learned my addr 172.21.15.140:0/4126214584 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:00.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27dd9b700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2800ac3a0 0x7fe28009b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:00.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27e59c700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe28005fa50 0x7fe2800af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:00.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27ed9d700 1 -- 172.21.15.140:0/4126214584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2800ac3a0 msgr2=0x7fe28009b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27ed9d700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2800ac3a0 0x7fe28009b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27ed9d700 1 -- 172.21.15.140:0/4126214584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe28005fa50 msgr2=0x7fe2800af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:00.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27ed9d700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe28005fa50 0x7fe2800af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:00.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27ed9d700 1 -- 172.21.15.140:0/4126214584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe278009530 con 0x7fe2800b1ef0 2022-07-21T01:56:00.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27ed9d700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2800b1ef0 0x7fe28009b9a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7fe28014c620 tx=0x7fe278016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:00.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe26f7fe700 1 -- 172.21.15.140:0/4126214584 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe27800b8a0 con 0x7fe2800b1ef0 2022-07-21T01:56:00.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe26f7fe700 1 -- 172.21.15.140:0/4126214584 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27800bde0 con 0x7fe2800b1ef0 2022-07-21T01:56:00.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.698+0000 7fe27f59e700 1 -- 172.21.15.140:0/4126214584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28014c700 con 0x7fe2800b1ef0 2022-07-21T01:56:00.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.702+0000 7fe26f7fe700 1 -- 172.21.15.140:0/4126214584 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe27801f5e0 con 0x7fe2800b1ef0 2022-07-21T01:56:00.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.702+0000 7fe27f59e700 1 -- 172.21.15.140:0/4126214584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe280064660 con 0x7fe2800b1ef0 2022-07-21T01:56:00.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.702+0000 7fe27f59e700 1 -- 172.21.15.140:0/4126214584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe280062270 con 0x7fe2800b1ef0 2022-07-21T01:56:00.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.702+0000 7fe26f7fe700 1 -- 172.21.15.140:0/4126214584 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe278026030 con 0x7fe2800b1ef0 2022-07-21T01:56:00.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.702+0000 7fe26f7fe700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe25c053980 0x7fe25c055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:00.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.702+0000 7fe26f7fe700 1 -- 172.21.15.140:0/4126214584 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fe27806c840 con 0x7fe2800b1ef0 2022-07-21T01:56:00.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.702+0000 7fe27e59c700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe25c053980 0x7fe25c055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:00.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.702+0000 7fe27e59c700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe25c053980 0x7fe25c055e40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fe27400d000 tx=0x7fe274009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:00.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.710+0000 7fe26f7fe700 1 -- 172.21.15.140:0/4126214584 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe278043a40 con 0x7fe2800b1ef0 2022-07-21T01:56:00.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.838+0000 7fe27f59e700 1 -- 172.21.15.140:0/4126214584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe2800574e0 con 0x7fe2800b1ef0 2022-07-21T01:56:00.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.838+0000 7fe26f7fe700 1 -- 172.21.15.140:0/4126214584 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v97) v1 ==== 81+0+28926 (secure 0 0 0) 0x7fe278024090 con 0x7fe2800b1ef0 2022-07-21T01:56:00.843 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:00.844 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:56:00.844 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 97, 2022-07-21T01:56:00.844 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:56:00.844 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:56:00.845 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:55:58.122717+0000", 2022-07-21T01:56:00.845 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:56:00.845 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:56:00.845 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:56:00.846 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:56:00.846 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:56:00.846 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:56:00.846 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:56:00.847 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:56:00.847 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:56:00.847 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:00.847 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:56:00.848 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:56:00.848 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:56:00.848 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:56:00.848 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:56:00.849 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 18, 2022-07-21T01:56:00.849 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:56:00.849 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:56:00.849 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:56:00.850 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:56:00.850 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:56:00.850 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.851 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:56:00.851 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:56:00.851 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:56:00.851 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:56:00.852 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:56:00.852 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:56:00.852 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:56:00.852 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:56:00.853 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:56:00.853 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:56:00.853 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:56:00.853 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:56:00.853 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:56:00.854 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:56:00.854 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:56:00.854 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:56:00.855 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:56:00.855 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:56:00.855 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:56:00.856 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:56:00.856 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:56:00.856 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:56:00.856 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:56:00.857 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.857 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:56:00.857 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:56:00.857 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:56:00.858 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:56:00.858 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:56:00.858 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:56:00.858 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:56:00.859 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:56:00.859 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:56:00.859 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:56:00.859 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:56:00.859 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:56:00.860 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:56:00.860 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:56:00.860 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:56:00.860 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:56:00.861 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:56:00.861 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:56:00.861 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:56:00.861 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:56:00.861 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:56:00.862 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:56:00.862 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:56:00.862 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:56:00.862 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:56:00.863 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:56:00.863 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:56:00.863 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.863 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:56:00.864 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:56:00.864 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:56:00.864 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:56:00.864 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:56:00.865 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:56:00.865 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:56:00.865 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:56:00.866 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:56:00.866 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:56:00.866 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:56:00.867 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:56:00.867 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:56:00.867 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.868 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:56:00.868 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:56:00.868 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.869 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.869 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.869 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:56:00.870 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:56:00.870 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:56:00.870 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:56:00.870 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:56:00.871 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:56:00.871 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:56:00.871 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:56:00.871 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:56:00.871 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:56:00.872 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:56:00.872 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:56:00.872 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:56:00.872 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:56:00.873 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:56:00.873 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:56:00.873 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:56:00.873 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:56:00.873 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:56:00.874 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:56:00.874 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:56:00.874 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:56:00.874 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:56:00.875 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.875 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:56:00.875 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:56:00.876 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:56:00.876 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:56:00.876 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:56:00.876 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:56:00.876 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:56:00.877 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:56:00.877 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:56:00.877 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:56:00.877 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:56:00.878 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:56:00.878 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:56:00.878 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:56:00.878 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:56:00.879 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:56:00.879 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:56:00.879 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:56:00.879 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:56:00.880 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:56:00.880 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:56:00.881 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:56:00.881 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:56:00.882 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:56:00.882 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:56:00.882 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:56:00.883 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:56:00.883 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.883 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:56:00.883 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:56:00.884 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:56:00.884 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:56:00.884 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:56:00.884 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:56:00.884 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:56:00.885 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:56:00.885 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:56:00.885 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:56:00.885 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:56:00.886 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:56:00.886 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:56:00.886 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:56:00.886 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.886 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.887 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:00.887 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:56:00.887 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.887 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:56:00.888 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:56:00.888 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:00.888 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:00.888 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:00.889 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:00.889 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:00.889 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:00.890 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:00.890 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 91, 2022-07-21T01:56:00.891 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:00.891 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:00.891 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:00.892 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.892 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.892 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.893 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:56:00.893 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:00.893 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.893 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.894 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.894 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:56:00.894 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:00.894 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.895 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.895 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.895 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:00.895 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.895 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.896 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.896 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:56:00.897 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:00.897 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.897 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.897 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.898 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:56:00.898 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:00.898 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.898 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.899 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.899 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:00.899 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.899 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.900 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.900 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:56:00.900 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:00.900 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.901 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.901 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.901 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:56:00.901 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:00.902 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.902 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.902 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.903 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:00.903 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.903 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.903 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.904 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:56:00.904 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:00.904 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.904 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.905 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.905 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:56:00.905 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:00.905 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.906 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.906 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.906 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:56:00.906 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:56:00.907 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:56:00.907 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:56:00.908 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:00.908 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:00.909 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:00.909 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.909 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.910 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.911 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:56:00.911 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:56:00.912 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:00.912 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:00.913 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:00.913 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:00.913 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:00.914 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:00.914 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:00.915 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 91, 2022-07-21T01:56:00.915 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:00.916 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:00.916 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:00.916 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.917 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.917 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.917 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:56:00.918 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:00.918 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.919 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.919 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.919 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:56:00.919 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:00.920 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.920 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.921 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.921 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:00.922 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.922 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.923 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.923 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:56:00.924 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:00.924 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.925 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.925 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.926 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:56:00.926 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:00.927 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.927 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.927 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.928 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:00.928 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.929 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.929 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.929 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:56:00.930 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:00.930 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.930 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.930 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.931 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:56:00.931 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:00.931 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.931 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.932 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.932 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:00.932 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.932 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.933 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.933 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:56:00.933 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:00.933 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.934 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.934 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.934 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:56:00.934 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:00.935 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.935 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.935 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.935 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:56:00.935 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:56:00.936 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:56:00.936 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:56:00.936 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:00.936 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:00.937 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:00.937 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.937 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.937 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.938 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:56:00.938 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:56:00.938 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:00.938 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:00.939 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:00.939 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:00.939 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:00.939 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:00.940 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:00.940 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 91, 2022-07-21T01:56:00.940 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:00.940 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:00.941 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:00.941 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.941 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.941 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.942 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:56:00.942 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:00.942 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.942 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.943 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.943 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:56:00.943 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:00.943 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.944 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.944 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.944 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:00.944 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.945 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.945 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.945 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:56:00.945 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:00.946 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.946 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.946 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.946 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:56:00.947 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:00.947 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.947 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.947 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.948 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:00.948 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.948 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.948 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.949 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:56:00.949 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:00.949 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.949 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.950 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.950 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:56:00.950 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:00.950 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.951 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.951 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.951 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:00.951 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.952 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.952 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.952 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:56:00.952 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:00.953 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.953 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.953 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.953 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:56:00.954 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:00.954 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.954 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.954 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.955 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:56:00.955 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:56:00.955 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:56:00.955 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:56:00.956 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:00.956 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:00.956 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:00.956 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.957 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.957 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.957 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:56:00.957 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:56:00.957 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:00.958 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:00.958 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:00.958 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:00.958 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:00.959 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:00.959 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:00.959 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 92, 2022-07-21T01:56:00.959 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:00.959 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:00.960 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:00.960 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.960 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.960 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.961 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:56:00.961 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:00.961 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.961 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.962 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.962 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:56:00.962 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:00.962 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.963 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.963 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.963 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:00.963 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.964 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.964 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.964 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:56:00.964 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:00.965 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.965 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.965 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.965 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:56:00.965 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:00.966 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.966 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.966 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.966 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:00.967 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.967 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.967 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.967 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:56:00.967 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:00.968 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.968 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.968 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.968 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:56:00.969 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:00.969 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.969 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.969 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.970 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:00.970 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.970 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.970 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.971 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:56:00.971 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:00.971 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.971 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.971 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.972 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:56:00.972 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:00.972 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.972 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.973 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.973 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:56:00.973 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:56:00.973 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:56:00.974 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:56:00.974 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:00.974 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:00.974 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:00.975 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.975 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.975 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.975 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:56:00.975 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:56:00.976 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:00.976 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:00.976 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:00.976 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:00.977 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:00.977 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:00.977 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:00.977 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 90, 2022-07-21T01:56:00.978 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:00.978 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:00.978 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:00.978 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.979 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.979 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.979 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:56:00.979 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:00.980 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.980 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.980 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.980 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:56:00.981 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:00.981 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.981 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.981 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.981 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:00.982 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.982 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.982 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.983 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:56:00.983 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:00.983 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.983 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.983 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.984 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:56:00.984 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:00.984 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.984 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.985 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.985 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:00.985 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.985 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.986 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.986 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:56:00.986 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:00.986 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.987 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.987 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.987 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:56:00.987 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:00.988 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.988 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.988 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.988 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:00.988 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.989 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.989 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.989 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:56:00.989 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:00.990 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.990 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.990 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.990 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:56:00.991 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:00.991 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.991 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.991 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.992 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:56:00.992 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:56:00.992 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:56:00.992 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:56:00.993 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:00.993 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:00.993 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:00.993 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:00.993 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.994 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.994 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:56:00.994 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:56:00.994 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:00.995 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:00.995 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:00.995 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:00.995 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:00.995 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:00.996 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:00.996 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 82, 2022-07-21T01:56:00.996 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:00.996 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:00.997 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:00.997 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:00.997 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.997 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:00.997 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:56:00.998 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:00.998 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:00.998 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:00.998 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:00.999 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:56:00.999 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:00.999 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:00.999 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.000 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.000 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:01.000 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:01.000 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.000 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:01.001 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:56:01.001 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:01.001 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.001 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.002 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:01.002 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:56:01.002 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:01.002 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.002 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.003 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.003 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:01.003 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:01.003 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.004 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:01.004 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:56:01.004 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:01.004 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.004 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.005 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:01.005 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:56:01.005 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:01.005 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.006 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.006 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.006 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:01.006 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:01.007 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.007 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:01.007 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:56:01.007 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:01.007 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.008 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.008 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:01.008 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:56:01.008 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:01.009 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.009 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.009 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.009 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:56:01.009 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:56:01.010 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:56:01.010 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:56:01.010 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:01.010 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:01.011 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:01.011 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.011 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.011 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.011 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:56:01.012 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:56:01.012 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:01.012 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:01.012 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:01.013 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:01.013 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:01.013 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:01.013 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:01.013 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 90, 2022-07-21T01:56:01.014 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:01.014 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:01.014 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:01.014 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:01.015 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.015 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:01.015 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:56:01.015 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:01.016 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.016 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.016 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:01.016 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:56:01.016 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:01.017 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.030 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.030 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.030 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:01.031 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:01.031 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.031 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:01.031 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:56:01.032 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:01.032 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.032 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.032 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:01.032 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:56:01.033 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:01.033 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.033 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.033 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.034 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:01.034 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:01.034 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.034 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:01.034 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:56:01.035 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:01.035 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.035 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.035 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:01.036 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:56:01.036 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:01.036 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.036 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.036 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.037 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:01.037 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:01.037 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.037 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:01.038 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:56:01.038 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:01.038 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.038 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.039 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:01.039 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:56:01.039 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:01.039 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.040 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.040 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.040 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:56:01.040 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:56:01.040 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:56:01.041 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:56:01.041 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:01.041 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:01.041 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:01.042 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.042 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.042 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.042 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:56:01.043 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:56:01.043 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:01.043 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:01.043 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:01.043 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:01.044 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:01.044 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:01.044 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:01.044 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 91, 2022-07-21T01:56:01.045 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:01.045 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:01.045 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:01.045 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:01.046 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.046 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:01.046 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:56:01.046 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:01.046 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.047 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.047 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:01.047 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:56:01.047 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:01.048 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.048 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.048 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.048 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:01.048 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:01.049 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.049 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:01.049 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:56:01.049 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:01.050 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.050 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.050 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:01.050 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:56:01.050 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:01.051 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.051 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.051 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.051 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:01.052 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:01.052 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.052 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:01.052 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:56:01.053 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:01.053 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.053 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.053 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:01.054 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:56:01.054 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:01.054 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.054 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.055 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.055 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:01.055 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:01.055 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.055 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:01.056 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:56:01.056 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:01.056 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.056 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.057 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:01.057 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:56:01.057 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:01.057 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.058 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.058 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.058 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:56:01.058 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:56:01.058 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:56:01.059 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:56:01.059 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:01.059 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:01.059 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:01.060 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:01.060 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.060 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:01.060 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:56:01.060 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.061 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:56:01.061 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:01.061 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:01.061 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:01.062 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:01.062 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:01.062 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:56:01.062 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:01.063 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.063 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.063 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:56:01.063 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:01.063 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:01.064 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:01.064 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:01.064 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:01.064 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:56:01.065 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:01.065 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.065 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.065 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:56:01.066 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:01.066 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:01.066 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:01.066 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:01.067 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:01.067 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:56:01.067 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:01.067 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.068 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.068 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:56:01.068 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:01.068 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:01.069 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:01.069 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:01.069 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:01.069 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:56:01.069 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:01.070 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.070 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.070 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:56:01.070 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:01.071 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:01.071 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:01.071 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:01.071 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:01.072 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:56:01.072 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:01.072 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.072 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.072 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:56:01.073 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:01.073 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:01.073 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:01.073 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:01.074 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:01.074 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:56:01.074 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:01.074 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.075 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.075 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:56:01.075 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:01.075 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:01.075 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:01.076 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:01.076 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:01.076 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:56:01.076 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:01.077 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.077 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:01.077 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:56:01.077 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:01.078 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:01.078 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:01.078 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:01.078 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:01.079 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:56:01.079 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:01.079 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.079 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:01.079 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:56:01.080 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:56:01.080 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:56:01.080 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:56:01.080 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:56:01.081 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6835/2478727503": "2022-07-22T01:55:54.835865+0000", 2022-07-21T01:56:01.081 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6834/2478727503": "2022-07-22T01:55:54.835865+0000" 2022-07-21T01:56:01.081 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.081 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:56:01.081 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:56:01.082 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:56:01.082 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:56:01.082 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:56:01.082 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:56:01.083 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:56:01.083 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:56:01.083 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:01.083 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:01.084 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:56:01.084 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:56:01.084 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:56:01.084 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:56:01.084 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:56:01.085 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:56:01.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 -- 172.21.15.140:0/4126214584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe25c053980 msgr2=0x7fe25c055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe25c053980 0x7fe25c055e40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fe27400d000 tx=0x7fe274009040).stop 2022-07-21T01:56:01.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 -- 172.21.15.140:0/4126214584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2800b1ef0 msgr2=0x7fe28009b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2800b1ef0 0x7fe28009b9a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7fe28014c620 tx=0x7fe278016000).stop 2022-07-21T01:56:01.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 -- 172.21.15.140:0/4126214584 shutdown_connections 2022-07-21T01:56:01.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe25c053980 0x7fe25c055e40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe2800b1ef0 0x7fe28009b9a0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2800ac3a0 0x7fe28009b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 --2- 172.21.15.140:0/4126214584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe28005fa50 0x7fe2800af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 -- 172.21.15.140:0/4126214584 >> 172.21.15.140:0/4126214584 conn(0x7fe2800b2ca0 msgr2=0x7fe2800b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:01.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 -- 172.21.15.140:0/4126214584 shutdown_connections 2022-07-21T01:56:01.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:00.842+0000 7fe27f59e700 1 -- 172.21.15.140:0/4126214584 wait complete. 2022-07-21T01:56:01.090 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:01.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.194+0000 7f3817ebe700 1 Processor -- start 2022-07-21T01:56:01.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.194+0000 7f3817ebe700 1 -- start start 2022-07-21T01:56:01.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3817ebe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 0x7f381014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3817ebe700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381005d480 con 0x7f381005c3f0 2022-07-21T01:56:01.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3817ebe700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38100b21e0 con 0x7f38100acef0 2022-07-21T01:56:01.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3817ebe700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381005cf40 con 0x7f38100b1e00 2022-07-21T01:56:01.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3815459700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f381005c3f0 0x7f38100ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52750/0 (socket says 172.21.15.140:52750) 2022-07-21T01:56:01.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3815459700 1 -- 172.21.15.140:0/793948019 learned_addr learned my addr 172.21.15.140:0/793948019 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:01.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3815c5a700 1 --2- 172.21.15.140:0/793948019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 0x7f381014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3814c58700 1 -- 172.21.15.140:0/793948019 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1793790804 0 0) 0x7f381005d480 con 0x7f381005c3f0 2022-07-21T01:56:01.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3814c58700 1 -- 172.21.15.140:0/793948019 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f8003260 con 0x7f381005c3f0 2022-07-21T01:56:01.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3814c58700 1 -- 172.21.15.140:0/793948019 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317073896 0 0) 0x7f38100b21e0 con 0x7f38100acef0 2022-07-21T01:56:01.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3814c58700 1 -- 172.21.15.140:0/793948019 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f381005d480 con 0x7f38100acef0 2022-07-21T01:56:01.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3814c58700 1 -- 172.21.15.140:0/793948019 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 674319704 0 0) 0x7f37f8003260 con 0x7f381005c3f0 2022-07-21T01:56:01.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3815c5a700 1 -- 172.21.15.140:0/793948019 >> v1:172.21.15.152:6789/0 conn(0x7f38100acef0 legacy=0x7f3810146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3815c5a700 1 -- 172.21.15.140:0/793948019 >> v1:172.21.15.140:6789/0 conn(0x7f381005c3f0 legacy=0x7f38100ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3815c5a700 1 -- 172.21.15.140:0/793948019 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381005d1e0 con 0x7f38100b1e00 2022-07-21T01:56:01.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3815c5a700 1 --2- 172.21.15.140:0/793948019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 0x7f381014a6d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f380c001800 tx=0x7f3804006df0).ready entity=mon.2 client_cookie=bedeeeabc095d2b2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:01.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3814c58700 1 -- 172.21.15.140:0/793948019 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f380c001660 con 0x7f38100b1e00 2022-07-21T01:56:01.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3814c58700 1 -- 172.21.15.140:0/793948019 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3804000e90 con 0x7f38100b1e00 2022-07-21T01:56:01.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3814c58700 1 -- 172.21.15.140:0/793948019 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3804012460 con 0x7f38100b1e00 2022-07-21T01:56:01.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3817ebe700 1 -- 172.21.15.140:0/793948019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 msgr2=0x7f381014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3817ebe700 1 --2- 172.21.15.140:0/793948019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 0x7f381014a6d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f380c001800 tx=0x7f3804006df0).stop 2022-07-21T01:56:01.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3817ebe700 1 -- 172.21.15.140:0/793948019 shutdown_connections 2022-07-21T01:56:01.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3817ebe700 1 --2- 172.21.15.140:0/793948019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 0x7f381014a6d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3817ebe700 1 -- 172.21.15.140:0/793948019 >> 172.21.15.140:0/793948019 conn(0x7f38100611c0 msgr2=0x7f3810062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:01.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3817ebe700 1 -- 172.21.15.140:0/793948019 shutdown_connections 2022-07-21T01:56:01.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.198+0000 7f3817ebe700 1 -- 172.21.15.140:0/793948019 wait complete. 2022-07-21T01:56:01.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3817ebe700 1 Processor -- start 2022-07-21T01:56:01.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3817ebe700 1 -- start start 2022-07-21T01:56:01.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3817ebe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38100acef0 0x7f38100a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3817ebe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 0x7f3810143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3817ebe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f381009e830 0x7f381009ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3817ebe700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3810149d30 con 0x7f381009e830 2022-07-21T01:56:01.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3817ebe700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3810149820 con 0x7f38100acef0 2022-07-21T01:56:01.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3817ebe700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381009f1d0 con 0x7f38100b1e00 2022-07-21T01:56:01.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f381645b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f381009e830 0x7f381009ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f381645b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f381009e830 0x7f381009ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38614/0 (socket says 172.21.15.140:38614) 2022-07-21T01:56:01.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f381645b700 1 -- 172.21.15.140:0/2558496386 learned_addr learned my addr 172.21.15.140:0/2558496386 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:01.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3815459700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 0x7f3810143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3815c5a700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38100acef0 0x7f38100a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3815459700 1 -- 172.21.15.140:0/2558496386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38100acef0 msgr2=0x7f38100a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3815459700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38100acef0 0x7f38100a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3815459700 1 -- 172.21.15.140:0/2558496386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f381009e830 msgr2=0x7f381009ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3815459700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f381009e830 0x7f381009ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3815459700 1 -- 172.21.15.140:0/2558496386 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380400b180 con 0x7f38100b1e00 2022-07-21T01:56:01.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.202+0000 7f3815459700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 0x7f3810143e90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f380c000fe0 tx=0x7f380c001680).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:01.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.206+0000 7f3802ffd700 1 -- 172.21.15.140:0/2558496386 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f380c0070d0 con 0x7f38100b1e00 2022-07-21T01:56:01.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.206+0000 7f3802ffd700 1 -- 172.21.15.140:0/2558496386 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c007230 con 0x7f38100b1e00 2022-07-21T01:56:01.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.206+0000 7f3817ebe700 1 -- 172.21.15.140:0/2558496386 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381014a340 con 0x7f38100b1e00 2022-07-21T01:56:01.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.206+0000 7f3802ffd700 1 -- 172.21.15.140:0/2558496386 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f380c005610 con 0x7f38100b1e00 2022-07-21T01:56:01.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.206+0000 7f3817ebe700 1 -- 172.21.15.140:0/2558496386 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38101429d0 con 0x7f38100b1e00 2022-07-21T01:56:01.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.206+0000 7f3817ebe700 1 -- 172.21.15.140:0/2558496386 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381009e080 con 0x7f38100b1e00 2022-07-21T01:56:01.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.206+0000 7f3802ffd700 1 -- 172.21.15.140:0/2558496386 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f380c01a060 con 0x7f38100b1e00 2022-07-21T01:56:01.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.206+0000 7f3802ffd700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37f00539e0 0x7f37f0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.206+0000 7f3802ffd700 1 -- 172.21.15.140:0/2558496386 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f380c06b2e0 con 0x7f38100b1e00 2022-07-21T01:56:01.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.206+0000 7f3815c5a700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37f00539e0 0x7f37f0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.206+0000 7f3815c5a700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37f00539e0 0x7f37f0055ea0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f380401a000 tx=0x7f3804014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:01.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.218+0000 7f3802ffd700 1 -- 172.21.15.140:0/2558496386 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f380c0424e0 con 0x7f38100b1e00 2022-07-21T01:56:01.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.378+0000 7f3817ebe700 1 -- 172.21.15.140:0/2558496386 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38100b04b0 con 0x7f38100b1e00 2022-07-21T01:56:01.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3802ffd700 1 -- 172.21.15.140:0/2558496386 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+1212 (secure 0 0 0) 0x7f380c018070 con 0x7f38100b1e00 2022-07-21T01:56:01.385 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:01.385 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/518920349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":518920349},{"type":"v1","addr":"172.21.15.152:6833","nonce":518920349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40},{"gid":7248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1224793989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1224793989},{"type":"v1","addr":"172.21.15.140:6835","nonce":1224793989}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44}],"filesystems":[]} 2022-07-21T01:56:01.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 -- 172.21.15.140:0/2558496386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37f00539e0 msgr2=0x7f37f0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37f00539e0 0x7f37f0055ea0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f380401a000 tx=0x7f3804014000).stop 2022-07-21T01:56:01.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 -- 172.21.15.140:0/2558496386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 msgr2=0x7f3810143e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 0x7f3810143e90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f380c000fe0 tx=0x7f380c001680).stop 2022-07-21T01:56:01.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 -- 172.21.15.140:0/2558496386 shutdown_connections 2022-07-21T01:56:01.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37f00539e0 0x7f37f0055ea0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f381009e830 0x7f381009ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38100b1e00 0x7f3810143e90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 --2- 172.21.15.140:0/2558496386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38100acef0 0x7f38100a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 -- 172.21.15.140:0/2558496386 >> 172.21.15.140:0/2558496386 conn(0x7f381005c3f0 msgr2=0x7f3810144fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:01.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 -- 172.21.15.140:0/2558496386 shutdown_connections 2022-07-21T01:56:01.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.382+0000 7f3817ebe700 1 -- 172.21.15.140:0/2558496386 wait complete. 2022-07-21T01:56:01.391 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 44 2022-07-21T01:56:01.400 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:01.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe60dc4a700 1 Processor -- start 2022-07-21T01:56:01.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe60dc4a700 1 -- start start 2022-07-21T01:56:01.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe60dc4a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6080b5650 0x7fe6080b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe60dc4a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe6080adff0 0x7fe608147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe60dc4a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe608147010 con 0x7fe6080af8e0 2022-07-21T01:56:01.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe60dc4a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6081472b0 con 0x7fe6080adff0 2022-07-21T01:56:01.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe60dc4a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe608146d40 con 0x7fe6080b5650 2022-07-21T01:56:01.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe607fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe6080adff0 0x7fe608147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe60cc48700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6080b5650 0x7fe6080b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe607fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe6080adff0 0x7fe608147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:44358/0 (socket says 172.21.15.140:44358) 2022-07-21T01:56:01.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe607fff700 1 -- 172.21.15.140:0/2474500168 learned_addr learned my addr 172.21.15.140:0/2474500168 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:01.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe60d449700 1 --1- 172.21.15.140:0/2474500168 >> v1:172.21.15.152:6789/0 conn(0x7fe6080af8e0 0x7fe60805bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:53882/0 (socket says 172.21.15.140:53882) 2022-07-21T01:56:01.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe607fff700 1 -- 172.21.15.140:0/2474500168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6080b5650 msgr2=0x7fe6080b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:01.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe607fff700 1 --2- 172.21.15.140:0/2474500168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6080b5650 0x7fe6080b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe6077fe700 1 -- 172.21.15.140:0/2474500168 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613227368 0 0) 0x7fe608147010 con 0x7fe6080af8e0 2022-07-21T01:56:01.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe607fff700 1 -- 172.21.15.140:0/2474500168 >> v1:172.21.15.152:6789/0 conn(0x7fe6080af8e0 legacy=0x7fe60805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe607fff700 1 -- 172.21.15.140:0/2474500168 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608147c40 con 0x7fe6080adff0 2022-07-21T01:56:01.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe60cc48700 1 --2- 172.21.15.140:0/2474500168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6080b5650 0x7fe6080b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:01.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe607fff700 1 --2- 172.21.15.140:0/2474500168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6080adff0 0x7fe608147750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fe5f00032f0 tx=0x7fe5f0009250).ready entity=mon.1 client_cookie=6d28662b39f889f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:01.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe6077fe700 1 -- 172.21.15.140:0/2474500168 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5f0009ea0 con 0x7fe6080adff0 2022-07-21T01:56:01.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe6077fe700 1 -- 172.21.15.140:0/2474500168 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f000b8a0 con 0x7fe6080adff0 2022-07-21T01:56:01.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.502+0000 7fe6077fe700 1 -- 172.21.15.140:0/2474500168 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5f000ba10 con 0x7fe6080adff0 2022-07-21T01:56:01.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 -- 172.21.15.140:0/2474500168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6080adff0 msgr2=0x7fe608147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 --2- 172.21.15.140:0/2474500168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6080adff0 0x7fe608147750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fe5f00032f0 tx=0x7fe5f0009250).stop 2022-07-21T01:56:01.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 -- 172.21.15.140:0/2474500168 shutdown_connections 2022-07-21T01:56:01.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 --2- 172.21.15.140:0/2474500168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6080adff0 0x7fe608147750 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 --2- 172.21.15.140:0/2474500168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6080b5650 0x7fe6080b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 -- 172.21.15.140:0/2474500168 >> 172.21.15.140:0/2474500168 conn(0x7fe6080634e0 msgr2=0x7fe6080b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:01.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 -- 172.21.15.140:0/2474500168 shutdown_connections 2022-07-21T01:56:01.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 -- 172.21.15.140:0/2474500168 wait complete. 2022-07-21T01:56:01.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 Processor -- start 2022-07-21T01:56:01.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 -- start start 2022-07-21T01:56:01.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe6080adff0 0x7fe6081460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6080af8e0 0x7fe608140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6080b5650 0x7fe608140880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60810e2f0 con 0x7fe6080adff0 2022-07-21T01:56:01.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6081410e0 con 0x7fe6080af8e0 2022-07-21T01:56:01.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.506+0000 7fe60dc4a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe608141380 con 0x7fe6080b5650 2022-07-21T01:56:01.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60cc48700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe6080adff0 0x7fe6081460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60cc48700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe6080adff0 0x7fe6081460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38626/0 (socket says 172.21.15.140:38626) 2022-07-21T01:56:01.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60cc48700 1 -- 172.21.15.140:0/433991858 learned_addr learned my addr 172.21.15.140:0/433991858 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:01.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60d449700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6080b5650 0x7fe608140880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe607fff700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6080af8e0 0x7fe608140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60cc48700 1 -- 172.21.15.140:0/433991858 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6080b5650 msgr2=0x7fe608140880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60cc48700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6080b5650 0x7fe608140880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60cc48700 1 -- 172.21.15.140:0/433991858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6080af8e0 msgr2=0x7fe608140390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60cc48700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6080af8e0 0x7fe608140390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60cc48700 1 -- 172.21.15.140:0/433991858 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f800b180 con 0x7fe6080adff0 2022-07-21T01:56:01.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60cc48700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe6080adff0 0x7fe6081460f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fe5f800c000 tx=0x7fe5f8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:01.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe605ffb700 1 -- 172.21.15.140:0/433991858 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5f8010780 con 0x7fe6080adff0 2022-07-21T01:56:01.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe605ffb700 1 -- 172.21.15.140:0/433991858 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f80108e0 con 0x7fe6080adff0 2022-07-21T01:56:01.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60dc4a700 1 -- 172.21.15.140:0/433991858 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f00094d0 con 0x7fe6080adff0 2022-07-21T01:56:01.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe605ffb700 1 -- 172.21.15.140:0/433991858 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5f8010c40 con 0x7fe6080adff0 2022-07-21T01:56:01.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60dc4a700 1 -- 172.21.15.140:0/433991858 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe608149570 con 0x7fe6080adff0 2022-07-21T01:56:01.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.510+0000 7fe60dc4a700 1 -- 172.21.15.140:0/433991858 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe60815a180 con 0x7fe6080adff0 2022-07-21T01:56:01.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.514+0000 7fe605ffb700 1 -- 172.21.15.140:0/433991858 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe5f801c460 con 0x7fe6080adff0 2022-07-21T01:56:01.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.514+0000 7fe605ffb700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5ec053950 0x7fe5ec055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.514+0000 7fe605ffb700 1 -- 172.21.15.140:0/433991858 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fe5f806c160 con 0x7fe6080adff0 2022-07-21T01:56:01.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.514+0000 7fe607fff700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5ec053950 0x7fe5ec055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.514+0000 7fe607fff700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5ec053950 0x7fe5ec055e10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fe5f001b000 tx=0x7fe5f0016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:01.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.522+0000 7fe605ffb700 1 -- 172.21.15.140:0/433991858 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe5f8043300 con 0x7fe6080adff0 2022-07-21T01:56:01.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.690+0000 7fe60dc4a700 1 -- 172.21.15.140:0/433991858 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe608141540 con 0x7fe6080adff0 2022-07-21T01:56:01.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.690+0000 7fe605ffb700 1 -- 172.21.15.140:0/433991858 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+1212 (secure 0 0 0) 0x7fe5f8046920 con 0x7fe6080adff0 2022-07-21T01:56:01.695 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:01.695 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/518920349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":518920349},{"type":"v1","addr":"172.21.15.152:6833","nonce":518920349}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":40},{"gid":7248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1224793989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1224793989},{"type":"v1","addr":"172.21.15.140:6835","nonce":1224793989}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":44}],"filesystems":[]} 2022-07-21T01:56:01.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 -- 172.21.15.140:0/433991858 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5ec053950 msgr2=0x7fe5ec055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5ec053950 0x7fe5ec055e10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fe5f001b000 tx=0x7fe5f0016000).stop 2022-07-21T01:56:01.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 -- 172.21.15.140:0/433991858 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe6080adff0 msgr2=0x7fe6081460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe6080adff0 0x7fe6081460f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fe5f800c000 tx=0x7fe5f8006df0).stop 2022-07-21T01:56:01.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 -- 172.21.15.140:0/433991858 shutdown_connections 2022-07-21T01:56:01.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe5ec053950 0x7fe5ec055e10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6080b5650 0x7fe608140880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6080af8e0 0x7fe608140390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 --2- 172.21.15.140:0/433991858 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe6080adff0 0x7fe6081460f0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 -- 172.21.15.140:0/433991858 >> 172.21.15.140:0/433991858 conn(0x7fe6080634e0 msgr2=0x7fe6080b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:01.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 -- 172.21.15.140:0/433991858 shutdown_connections 2022-07-21T01:56:01.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.694+0000 7fe60dc4a700 1 -- 172.21.15.140:0/433991858 wait complete. 2022-07-21T01:56:01.702 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 44 2022-07-21T01:56:01.712 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:56:01.713 INFO:tasks.ceph.mds.a:Stopping old one... 2022-07-21T01:56:01.713 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:56:01.713 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:56:01.714 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:56:01.714 INFO:tasks.ceph.mds.b:Stopping old one... 2022-07-21T01:56:01.714 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:56:01.715 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:56:01.723 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:56:01.724 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:56:01.726 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:56:01.727 DEBUG:teuthology.parallel:result is None 2022-07-21T01:56:01.730 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:56:01.730 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:56:01.733 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:56:01.733 DEBUG:teuthology.parallel:result is None 2022-07-21T01:56:01.734 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:56:01.799 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:56:01.806 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:56:01.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.830+0000 7f0f123f4700 1 Processor -- start 2022-07-21T01:56:01.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.830+0000 7f0f123f4700 1 -- start start 2022-07-21T01:56:01.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f123f4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0f0c0b6200 0x7f0f0c0b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f123f4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f0c0b08e0 0x7f0f0c146260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f123f4700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0c0b17f0 con 0x7f0f0c0b49f0 2022-07-21T01:56:01.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f123f4700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c0b4640 con 0x7f0f0c0b6200 2022-07-21T01:56:01.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f123f4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0b5190 con 0x7f0f0c0b08e0 2022-07-21T01:56:01.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f10bf1700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0f0c0b49f0 0x7f0f0c0b01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52794/0 (socket says 172.21.15.140:52794) 2022-07-21T01:56:01.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f11bf3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f0c0b08e0 0x7f0f0c146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f10bf1700 1 -- 172.21.15.140:0/3516445290 learned_addr learned my addr 172.21.15.140:0/3516445290 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:01.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f113f2700 1 --2- 172.21.15.140:0/3516445290 >> v2:172.21.15.140:3300/0 conn(0x7f0f0c0b6200 0x7f0f0c0b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f113f2700 1 -- 172.21.15.140:0/3516445290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f0c0b08e0 msgr2=0x7f0f0c146260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:01.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f113f2700 1 --2- 172.21.15.140:0/3516445290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f0c0b08e0 0x7f0f0c146260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f113f2700 1 -- 172.21.15.140:0/3516445290 >> v1:172.21.15.140:6789/0 conn(0x7f0f0c0b49f0 legacy=0x7f0f0c0b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f113f2700 1 -- 172.21.15.140:0/3516445290 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0b65e0 con 0x7f0f0c0b6200 2022-07-21T01:56:01.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f113f2700 1 --2- 172.21.15.140:0/3516445290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f0c0b6200 0x7f0f0c0b4080 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f0f08002f70 tx=0x7f0f08006df0).ready entity=mon.0 client_cookie=f2fd4b845ae06d2f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:01.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f03fff700 1 -- 172.21.15.140:0/3516445290 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f0800d570 con 0x7f0f0c0b6200 2022-07-21T01:56:01.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f03fff700 1 -- 172.21.15.140:0/3516445290 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f08000e90 con 0x7f0f0c0b6200 2022-07-21T01:56:01.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f03fff700 1 -- 172.21.15.140:0/3516445290 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f0800d8c0 con 0x7f0f0c0b6200 2022-07-21T01:56:01.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f123f4700 1 -- 172.21.15.140:0/3516445290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f0c0b6200 msgr2=0x7f0f0c0b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f123f4700 1 --2- 172.21.15.140:0/3516445290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f0c0b6200 0x7f0f0c0b4080 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f0f08002f70 tx=0x7f0f08006df0).stop 2022-07-21T01:56:01.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f123f4700 1 -- 172.21.15.140:0/3516445290 shutdown_connections 2022-07-21T01:56:01.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f123f4700 1 --2- 172.21.15.140:0/3516445290 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f0c0b08e0 0x7f0f0c146260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f123f4700 1 --2- 172.21.15.140:0/3516445290 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f0c0b6200 0x7f0f0c0b4080 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.834+0000 7f0f123f4700 1 -- 172.21.15.140:0/3516445290 >> 172.21.15.140:0/3516445290 conn(0x7f0f0c063d10 msgr2=0x7f0f0c0b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:01.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.838+0000 7f0f123f4700 1 -- 172.21.15.140:0/3516445290 shutdown_connections 2022-07-21T01:56:01.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.838+0000 7f0f123f4700 1 -- 172.21.15.140:0/3516445290 wait complete. 2022-07-21T01:56:01.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.838+0000 7f0f123f4700 1 Processor -- start 2022-07-21T01:56:01.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.838+0000 7f0f123f4700 1 -- start start 2022-07-21T01:56:01.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f123f4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f0c0b08e0 0x7f0f0c060cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f123f4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f0c0b49f0 0x7f0f0c1410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f123f4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f0c0b6200 0x7f0f0c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f123f4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c140510 con 0x7f0f0c0b6200 2022-07-21T01:56:01.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f123f4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0acda0 con 0x7f0f0c0b08e0 2022-07-21T01:56:01.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f123f4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0ae490 con 0x7f0f0c0b49f0 2022-07-21T01:56:01.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f11bf3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f0c0b6200 0x7f0f0c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f11bf3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f0c0b6200 0x7f0f0c0ae930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38654/0 (socket says 172.21.15.140:38654) 2022-07-21T01:56:01.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f113f2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f0c0b08e0 0x7f0f0c060cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f11bf3700 1 -- 172.21.15.140:0/2763413051 learned_addr learned my addr 172.21.15.140:0/2763413051 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:01.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f10bf1700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f0c0b49f0 0x7f0f0c1410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f11bf3700 1 -- 172.21.15.140:0/2763413051 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f0c0b49f0 msgr2=0x7f0f0c1410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f11bf3700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f0c0b49f0 0x7f0f0c1410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f11bf3700 1 -- 172.21.15.140:0/2763413051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f0c0b08e0 msgr2=0x7f0f0c060cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:01.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f11bf3700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f0c0b08e0 0x7f0f0c060cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:01.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f11bf3700 1 -- 172.21.15.140:0/2763413051 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0800b180 con 0x7f0f0c0b6200 2022-07-21T01:56:01.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f11bf3700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f0c0b6200 0x7f0f0c0ae930 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f0f04006e00 tx=0x7f0f040091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:01.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f027fc700 1 -- 172.21.15.140:0/2763413051 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f0400c640 con 0x7f0f0c0b6200 2022-07-21T01:56:01.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f027fc700 1 -- 172.21.15.140:0/2763413051 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0400c7a0 con 0x7f0f0c0b6200 2022-07-21T01:56:01.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f123f4700 1 -- 172.21.15.140:0/2763413051 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c1415e0 con 0x7f0f0c0b6200 2022-07-21T01:56:01.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f027fc700 1 -- 172.21.15.140:0/2763413051 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f0400cb00 con 0x7f0f0c0b6200 2022-07-21T01:56:01.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.842+0000 7f0f123f4700 1 -- 172.21.15.140:0/2763413051 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c1408a0 con 0x7f0f0c0b6200 2022-07-21T01:56:01.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.846+0000 7f0f123f4700 1 -- 172.21.15.140:0/2763413051 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c0acc00 con 0x7f0f0c0b6200 2022-07-21T01:56:01.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.846+0000 7f0f027fc700 1 -- 172.21.15.140:0/2763413051 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0f0400cc60 con 0x7f0f0c0b6200 2022-07-21T01:56:01.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.846+0000 7f0f027fc700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0ef40539e0 0x7f0ef4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:01.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.846+0000 7f0f027fc700 1 -- 172.21.15.140:0/2763413051 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f0f0406beb0 con 0x7f0f0c0b6200 2022-07-21T01:56:01.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.846+0000 7f0f113f2700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0ef40539e0 0x7f0ef4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:01.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.846+0000 7f0f113f2700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0ef40539e0 0x7f0ef4055ea0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f0f0801a000 tx=0x7f0f08014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:01.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.854+0000 7f0f027fc700 1 -- 172.21.15.140:0/2763413051 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0f0c0acc00 con 0x7f0f0c0b6200 2022-07-21T01:56:01.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:01.986+0000 7f0f123f4700 1 -- 172.21.15.140:0/2763413051 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f0f0c0acc00 con 0x7f0f0c0b6200 2022-07-21T01:56:02.211 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:56:02.214 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:56:02.216 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:56:02.219 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:56:02.221 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:56:02.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.854+0000 7f0f027fc700 1 -- 172.21.15.140:0/2763413051 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v99) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0f0c0acc00 con 0x7f0f0c0b6200 2022-07-21T01:56:02.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 -- 172.21.15.140:0/2763413051 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0ef40539e0 msgr2=0x7f0ef4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:02.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0ef40539e0 0x7f0ef4055ea0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f0f0801a000 tx=0x7f0f08014000).stop 2022-07-21T01:56:02.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 -- 172.21.15.140:0/2763413051 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f0c0b6200 msgr2=0x7f0f0c0ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:02.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f0c0b6200 0x7f0f0c0ae930 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f0f04006e00 tx=0x7f0f040091d0).stop 2022-07-21T01:56:02.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 -- 172.21.15.140:0/2763413051 shutdown_connections 2022-07-21T01:56:02.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0ef40539e0 0x7f0ef4055ea0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:02.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0f0c0b6200 0x7f0f0c0ae930 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:02.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0f0c0b49f0 0x7f0f0c1410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:02.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 --2- 172.21.15.140:0/2763413051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f0c0b08e0 0x7f0f0c060cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:02.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 -- 172.21.15.140:0/2763413051 >> 172.21.15.140:0/2763413051 conn(0x7f0f0c063d10 msgr2=0x7f0f0c0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:02.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 -- 172.21.15.140:0/2763413051 shutdown_connections 2022-07-21T01:56:02.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.858+0000 7f0f123f4700 1 -- 172.21.15.140:0/2763413051 wait complete. 2022-07-21T01:56:02.866 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:56:02.878 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:56:02.879 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:56:02.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d7476700 1 Processor -- start 2022-07-21T01:56:02.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d7476700 1 -- start start 2022-07-21T01:56:02.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d7476700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44d00b5650 0x7f44d00b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:02.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d7476700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f44d00adff0 0x7f44d0147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:02.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d7476700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f44d00af8e0 0x7f44d005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:02.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d7476700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d0147010 con 0x7f44d00adff0 2022-07-21T01:56:02.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d7476700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d01472b0 con 0x7f44d00af8e0 2022-07-21T01:56:02.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d7476700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0146d40 con 0x7f44d00b5650 2022-07-21T01:56:02.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5c73700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f44d00adff0 0x7f44d0147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:02.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5c73700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f44d00adff0 0x7f44d0147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38658/0 (socket says 172.21.15.140:38658) 2022-07-21T01:56:02.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5c73700 1 -- 172.21.15.140:0/1028208400 learned_addr learned my addr 172.21.15.140:0/1028208400 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:02.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d6474700 1 --2- 172.21.15.140:0/1028208400 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44d00b5650 0x7f44d00b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:02.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d6c75700 1 --2- 172.21.15.140:0/1028208400 >> v2:172.21.15.152:3300/0 conn(0x7f44d00af8e0 0x7f44d005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:02.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5c73700 1 -- 172.21.15.140:0/1028208400 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44d00b5650 msgr2=0x7f44d00b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:02.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5c73700 1 --2- 172.21.15.140:0/1028208400 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44d00b5650 0x7f44d00b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:02.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5c73700 1 -- 172.21.15.140:0/1028208400 >> v2:172.21.15.152:3300/0 conn(0x7f44d00af8e0 msgr2=0x7f44d005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:02.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5c73700 1 --2- 172.21.15.140:0/1028208400 >> v2:172.21.15.152:3300/0 conn(0x7f44d00af8e0 0x7f44d005bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:02.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5c73700 1 -- 172.21.15.140:0/1028208400 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d0147c40 con 0x7f44d00adff0 2022-07-21T01:56:02.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5c73700 1 --2- 172.21.15.140:0/1028208400 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44d00adff0 0x7f44d0147750 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f44cc0032f0 tx=0x7f44cc009250).ready entity=mon.0 client_cookie=f8d40f287111ec94 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:02.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5472700 1 -- 172.21.15.140:0/1028208400 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44cc00b870 con 0x7f44d00adff0 2022-07-21T01:56:02.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5472700 1 -- 172.21.15.140:0/1028208400 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc00b9d0 con 0x7f44d00adff0 2022-07-21T01:56:02.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.982+0000 7f44d5472700 1 -- 172.21.15.140:0/1028208400 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44cc00bb40 con 0x7f44d00adff0 2022-07-21T01:56:02.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.986+0000 7f44d7476700 1 -- 172.21.15.140:0/1028208400 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44d00adff0 msgr2=0x7f44d0147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:02.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.986+0000 7f44d7476700 1 --2- 172.21.15.140:0/1028208400 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44d00adff0 0x7f44d0147750 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f44cc0032f0 tx=0x7f44cc009250).stop 2022-07-21T01:56:02.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.986+0000 7f44d7476700 1 -- 172.21.15.140:0/1028208400 shutdown_connections 2022-07-21T01:56:02.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.986+0000 7f44d7476700 1 --2- 172.21.15.140:0/1028208400 >> v2:172.21.15.152:3300/0 conn(0x7f44d00af8e0 0x7f44d005bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:02.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.986+0000 7f44d7476700 1 --2- 172.21.15.140:0/1028208400 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44d00adff0 0x7f44d0147750 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:02.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.986+0000 7f44d7476700 1 --2- 172.21.15.140:0/1028208400 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44d00b5650 0x7f44d00b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:02.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.986+0000 7f44d7476700 1 -- 172.21.15.140:0/1028208400 >> 172.21.15.140:0/1028208400 conn(0x7f44d00634e0 msgr2=0x7f44d00b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:02.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.986+0000 7f44d7476700 1 -- 172.21.15.140:0/1028208400 shutdown_connections 2022-07-21T01:56:02.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.986+0000 7f44d7476700 1 -- 172.21.15.140:0/1028208400 wait complete. 2022-07-21T01:56:02.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.986+0000 7f44d7476700 1 Processor -- start 2022-07-21T01:56:02.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.986+0000 7f44d7476700 1 -- start start 2022-07-21T01:56:02.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d7476700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d00adff0 0x7f44d0110310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:02.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d7476700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44d00af8e0 0x7f44d014a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:02.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d7476700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44d00b5650 0x7f44d014aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:02.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d7476700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d014b390 con 0x7f44d00b5650 2022-07-21T01:56:02.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d7476700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0065ad0 con 0x7f44d00adff0 2022-07-21T01:56:02.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d7476700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0149370 con 0x7f44d00af8e0 2022-07-21T01:56:02.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d6c75700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44d00b5650 0x7f44d014aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:02.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d5c73700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44d00af8e0 0x7f44d014a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:02.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d6c75700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44d00b5650 0x7f44d014aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38668/0 (socket says 172.21.15.140:38668) 2022-07-21T01:56:03.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d6c75700 1 -- 172.21.15.140:0/1786722870 learned_addr learned my addr 172.21.15.140:0/1786722870 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:03.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d6474700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d00adff0 0x7f44d0110310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d6c75700 1 -- 172.21.15.140:0/1786722870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44d00af8e0 msgr2=0x7f44d014a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d6c75700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44d00af8e0 0x7f44d014a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d6c75700 1 -- 172.21.15.140:0/1786722870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d00adff0 msgr2=0x7f44d0110310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d6c75700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d00adff0 0x7f44d0110310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d6c75700 1 -- 172.21.15.140:0/1786722870 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc0094d0 con 0x7f44d00b5650 2022-07-21T01:56:03.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d6c75700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44d00b5650 0x7f44d014aa40 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f44c800a000 tx=0x7f44c8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:03.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44c37fe700 1 -- 172.21.15.140:0/1786722870 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44c8011780 con 0x7f44d00b5650 2022-07-21T01:56:03.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44c37fe700 1 -- 172.21.15.140:0/1786722870 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c80118e0 con 0x7f44d00b5650 2022-07-21T01:56:03.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44c37fe700 1 -- 172.21.15.140:0/1786722870 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f44c8011c40 con 0x7f44d00b5650 2022-07-21T01:56:03.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d7476700 1 -- 172.21.15.140:0/1786722870 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d014af30 con 0x7f44d00b5650 2022-07-21T01:56:03.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.990+0000 7f44d7476700 1 -- 172.21.15.140:0/1786722870 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d0063bb0 con 0x7f44d00b5650 2022-07-21T01:56:03.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.994+0000 7f44d7476700 1 -- 172.21.15.140:0/1786722870 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d0065920 con 0x7f44d00b5650 2022-07-21T01:56:03.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.994+0000 7f44c37fe700 1 -- 172.21.15.140:0/1786722870 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f44c8012420 con 0x7f44d00b5650 2022-07-21T01:56:03.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.994+0000 7f44c37fe700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44b0053aa0 0x7f44b0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.994+0000 7f44d6474700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44b0053aa0 0x7f44b0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.994+0000 7f44c37fe700 1 -- 172.21.15.140:0/1786722870 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f44c806b840 con 0x7f44d00b5650 2022-07-21T01:56:03.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:02.994+0000 7f44d6474700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44b0053aa0 0x7f44b0055f60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f44c400c000 tx=0x7f44c4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:03.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.002+0000 7f44c37fe700 1 -- 172.21.15.140:0/1786722870 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f44c8042a30 con 0x7f44d00b5650 2022-07-21T01:56:03.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.162+0000 7f44d7476700 1 -- 172.21.15.140:0/1786722870 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f44d00574e0 con 0x7f44d00b5650 2022-07-21T01:56:03.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44c37fe700 1 -- 172.21.15.140:0/1786722870 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f44c8046050 con 0x7f44d00b5650 2022-07-21T01:56:03.169 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:03.169 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:56:03.169 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:56:03.170 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.170 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:56:03.170 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:56:03.170 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.171 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:56:03.171 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:03.171 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:03.171 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.172 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.172 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.172 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.172 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:56:03.173 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:56:03.173 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.173 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:56:03.173 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:03.174 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:03.174 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.174 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.175 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.175 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.175 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:56:03.175 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:56:03.176 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.176 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:03.176 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:03.177 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.177 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.177 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.178 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.178 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:56:03.178 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:56:03.178 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.179 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:03.179 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:03.179 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.179 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.180 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.180 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.180 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:56:03.180 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:56:03.181 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.181 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:03.181 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:03.181 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.182 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.182 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.182 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.182 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:56:03.183 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:56:03.183 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.183 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:03.183 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:03.184 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.184 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.184 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.184 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.185 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:56:03.185 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:56:03.185 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.185 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:03.185 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:03.186 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.186 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.186 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.187 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.187 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:56:03.187 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:56:03.187 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.188 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:03.188 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:03.189 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.189 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.190 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.190 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.191 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:56:03.191 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:56:03.192 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.192 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:03.192 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:03.193 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.193 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.194 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.194 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.195 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:56:03.195 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:56:03.195 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.196 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:03.196 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:03.196 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.196 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.197 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.197 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.197 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:56:03.197 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:56:03.198 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.198 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:56:03.198 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:56:03.199 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:56:03.199 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.199 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.199 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.200 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:56:03.200 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:56:03.200 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.200 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:56:03.201 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:56:03.201 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:56:03.201 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.201 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.202 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.202 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:56:03.202 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:56:03.202 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.203 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:56:03.203 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:56:03.203 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:56:03.203 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.204 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.204 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.204 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:56:03.204 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:56:03.205 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.205 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:56:03.205 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:03.205 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:03.206 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.206 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.206 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.206 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.207 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:56:03.207 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:56:03.207 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.207 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:56:03.208 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:56:03.208 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.208 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.208 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:03.209 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:03.209 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:56:03.209 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:56:03.209 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:03.210 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:56:03.210 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:56:03.210 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:03.210 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.211 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:03.211 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:03.211 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:56:03.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 -- 172.21.15.140:0/1786722870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44b0053aa0 msgr2=0x7f44b0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44b0053aa0 0x7f44b0055f60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f44c400c000 tx=0x7f44c4008040).stop 2022-07-21T01:56:03.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 -- 172.21.15.140:0/1786722870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44d00b5650 msgr2=0x7f44d014aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44d00b5650 0x7f44d014aa40 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f44c800a000 tx=0x7f44c8008040).stop 2022-07-21T01:56:03.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 -- 172.21.15.140:0/1786722870 shutdown_connections 2022-07-21T01:56:03.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44b0053aa0 0x7f44b0055f60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44d00b5650 0x7f44d014aa40 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44d00af8e0 0x7f44d014a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 --2- 172.21.15.140:0/1786722870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d00adff0 0x7f44d0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 -- 172.21.15.140:0/1786722870 >> 172.21.15.140:0/1786722870 conn(0x7f44d00634e0 msgr2=0x7f44d00b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:03.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 -- 172.21.15.140:0/1786722870 shutdown_connections 2022-07-21T01:56:03.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.166+0000 7f44d7476700 1 -- 172.21.15.140:0/1786722870 wait complete. 2022-07-21T01:56:03.217 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:03.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e564b0700 1 Processor -- start 2022-07-21T01:56:03.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e564b0700 1 -- start start 2022-07-21T01:56:03.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e564b0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3e500b1e00 0x7f3e5014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e564b0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e500acef0 0x7f3e50146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e564b0700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5005d480 con 0x7f3e5005c3f0 2022-07-21T01:56:03.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e564b0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e500b21e0 con 0x7f3e500b1e00 2022-07-21T01:56:03.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e564b0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5005cf40 con 0x7f3e500acef0 2022-07-21T01:56:03.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3e500b1e00 0x7f3e5014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3e500b1e00 0x7f3e5014a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38676/0 (socket says 172.21.15.140:38676) 2022-07-21T01:56:03.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4ffff700 1 -- 172.21.15.140:0/1567880188 learned_addr learned my addr 172.21.15.140:0/1567880188 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:03.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e54a4d700 1 --2- 172.21.15.140:0/1567880188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e500acef0 0x7f3e50146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4ffff700 1 -- 172.21.15.140:0/1567880188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e500acef0 msgr2=0x7f3e50146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:03.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4ffff700 1 --2- 172.21.15.140:0/1567880188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e500acef0 0x7f3e50146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4ffff700 1 -- 172.21.15.140:0/1567880188 >> v1:172.21.15.152:6789/0 conn(0x7f3e5005c3f0 legacy=0x7f3e500ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4effd700 1 -- 172.21.15.140:0/1567880188 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063442642 0 0) 0x7f3e5005d480 con 0x7f3e5005c3f0 2022-07-21T01:56:03.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4ffff700 1 -- 172.21.15.140:0/1567880188 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5005d1e0 con 0x7f3e500b1e00 2022-07-21T01:56:03.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4ffff700 1 --2- 172.21.15.140:0/1567880188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e500b1e00 0x7f3e5014a6d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f3e38002f70 tx=0x7f3e38006eb0).ready entity=mon.0 client_cookie=13849331fba57820 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:03.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4effd700 1 -- 172.21.15.140:0/1567880188 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e3800d570 con 0x7f3e500b1e00 2022-07-21T01:56:03.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4effd700 1 -- 172.21.15.140:0/1567880188 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e38000e90 con 0x7f3e500b1e00 2022-07-21T01:56:03.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.322+0000 7f3e4effd700 1 -- 172.21.15.140:0/1567880188 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e3800d8c0 con 0x7f3e500b1e00 2022-07-21T01:56:03.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 -- 172.21.15.140:0/1567880188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e500b1e00 msgr2=0x7f3e5014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 --2- 172.21.15.140:0/1567880188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e500b1e00 0x7f3e5014a6d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f3e38002f70 tx=0x7f3e38006eb0).stop 2022-07-21T01:56:03.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 -- 172.21.15.140:0/1567880188 shutdown_connections 2022-07-21T01:56:03.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 --2- 172.21.15.140:0/1567880188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e500acef0 0x7f3e50146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 --2- 172.21.15.140:0/1567880188 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e500b1e00 0x7f3e5014a6d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 -- 172.21.15.140:0/1567880188 >> 172.21.15.140:0/1567880188 conn(0x7f3e500611c0 msgr2=0x7f3e50062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:03.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 -- 172.21.15.140:0/1567880188 shutdown_connections 2022-07-21T01:56:03.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 -- 172.21.15.140:0/1567880188 wait complete. 2022-07-21T01:56:03.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 Processor -- start 2022-07-21T01:56:03.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 -- start start 2022-07-21T01:56:03.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e500acef0 0x7f3e500ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e500b1e00 0x7f3e50063e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e50064320 0x7f3e50159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50143820 con 0x7f3e500b1e00 2022-07-21T01:56:03.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50065e10 con 0x7f3e500acef0 2022-07-21T01:56:03.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e564b0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50149840 con 0x7f3e50064320 2022-07-21T01:56:03.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e4f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e500b1e00 0x7f3e50063e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e4f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e500b1e00 0x7f3e50063e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38680/0 (socket says 172.21.15.140:38680) 2022-07-21T01:56:03.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e4ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e500acef0 0x7f3e500ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e4f7fe700 1 -- 172.21.15.140:0/703756196 learned_addr learned my addr 172.21.15.140:0/703756196 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:03.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.326+0000 7f3e54a4d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e50064320 0x7f3e50159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e4ffff700 1 -- 172.21.15.140:0/703756196 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e50064320 msgr2=0x7f3e50159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e4ffff700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e50064320 0x7f3e50159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e4ffff700 1 -- 172.21.15.140:0/703756196 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e500b1e00 msgr2=0x7f3e50063e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e4ffff700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e500b1e00 0x7f3e50063e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e4ffff700 1 -- 172.21.15.140:0/703756196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3800b1e0 con 0x7f3e500acef0 2022-07-21T01:56:03.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e4f7fe700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e500b1e00 0x7f3e50063e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:03.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e4ffff700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e500acef0 0x7f3e500ad900 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f3e50065cb0 tx=0x7f3e38015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:03.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e4d7fa700 1 -- 172.21.15.140:0/703756196 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e3800d8f0 con 0x7f3e500acef0 2022-07-21T01:56:03.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e4d7fa700 1 -- 172.21.15.140:0/703756196 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e38011b70 con 0x7f3e500acef0 2022-07-21T01:56:03.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e564b0700 1 -- 172.21.15.140:0/703756196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e50149a70 con 0x7f3e500acef0 2022-07-21T01:56:03.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e4d7fa700 1 -- 172.21.15.140:0/703756196 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e3801e460 con 0x7f3e500acef0 2022-07-21T01:56:03.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e564b0700 1 -- 172.21.15.140:0/703756196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e50149480 con 0x7f3e500acef0 2022-07-21T01:56:03.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e564b0700 1 -- 172.21.15.140:0/703756196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e501491c0 con 0x7f3e500acef0 2022-07-21T01:56:03.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.330+0000 7f3e4d7fa700 1 -- 172.21.15.140:0/703756196 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3e3801a040 con 0x7f3e500acef0 2022-07-21T01:56:03.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.334+0000 7f3e4d7fa700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e34053b00 0x7f3e34055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.334+0000 7f3e4d7fa700 1 -- 172.21.15.140:0/703756196 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3e3806c930 con 0x7f3e500acef0 2022-07-21T01:56:03.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.334+0000 7f3e4f7fe700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e34053b00 0x7f3e34055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.334+0000 7f3e4f7fe700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e34053b00 0x7f3e34055fc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f3e4000d000 tx=0x7f3e40007f70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:03.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.342+0000 7f3e4d7fa700 1 -- 172.21.15.140:0/703756196 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3e501491c0 con 0x7f3e500acef0 2022-07-21T01:56:03.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.466+0000 7f3e564b0700 1 -- 172.21.15.140:0/703756196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e5014a150 con 0x7f3e500acef0 2022-07-21T01:56:03.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.466+0000 7f3e4d7fa700 1 -- 172.21.15.140:0/703756196 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+13278 (secure 0 0 0) 0x7f3e38023070 con 0x7f3e500acef0 2022-07-21T01:56:03.471 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:03.471 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":99,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:02.845847+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:03.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 -- 172.21.15.140:0/703756196 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e34053b00 msgr2=0x7f3e34055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e34053b00 0x7f3e34055fc0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f3e4000d000 tx=0x7f3e40007f70).stop 2022-07-21T01:56:03.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 -- 172.21.15.140:0/703756196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e500acef0 msgr2=0x7f3e500ad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e500acef0 0x7f3e500ad900 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f3e50065cb0 tx=0x7f3e38015000).stop 2022-07-21T01:56:03.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 -- 172.21.15.140:0/703756196 shutdown_connections 2022-07-21T01:56:03.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3e34053b00 0x7f3e34055fc0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e50064320 0x7f3e50159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e500b1e00 0x7f3e50063e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 --2- 172.21.15.140:0/703756196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e500acef0 0x7f3e500ad900 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 -- 172.21.15.140:0/703756196 >> 172.21.15.140:0/703756196 conn(0x7f3e5005c3f0 msgr2=0x7f3e50144fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:03.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 -- 172.21.15.140:0/703756196 shutdown_connections 2022-07-21T01:56:03.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.470+0000 7f3e564b0700 1 -- 172.21.15.140:0/703756196 wait complete. 2022-07-21T01:56:03.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:56:03.484 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:56:03.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19b184f700 1 Processor -- start 2022-07-21T01:56:03.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19b184f700 1 -- start start 2022-07-21T01:56:03.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19b184f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 0x7f19ac0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19b184f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f19ac0b0480 0x7f19ac14a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19b184f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ac0ac5e0 con 0x7f19ac14ab60 2022-07-21T01:56:03.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19b184f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ac0b5f30 con 0x7f19ac0b0480 2022-07-21T01:56:03.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19b184f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac05cf70 con 0x7f19ac0b5b50 2022-07-21T01:56:03.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 0x7f19ac0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19aa7fc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f19ac0b0480 0x7f19ac14a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 0x7f19ac0accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:35416/0 (socket says 172.21.15.140:35416) 2022-07-21T01:56:03.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19aaffd700 1 -- 172.21.15.140:0/2665098325 learned_addr learned my addr 172.21.15.140:0/2665098325 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:03.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19aaffd700 1 -- 172.21.15.140:0/2665098325 >> v2:172.21.15.152:3300/0 conn(0x7f19ac0b0480 msgr2=0x7f19ac14a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:03.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19aaffd700 1 --2- 172.21.15.140:0/2665098325 >> v2:172.21.15.152:3300/0 conn(0x7f19ac0b0480 0x7f19ac14a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19aaffd700 1 -- 172.21.15.140:0/2665098325 >> v1:172.21.15.152:6789/0 conn(0x7f19ac14ab60 legacy=0x7f19ac144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19aaffd700 1 -- 172.21.15.140:0/2665098325 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac0ac810 con 0x7f19ac0b5b50 2022-07-21T01:56:03.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19aa7fc700 1 --2- 172.21.15.140:0/2665098325 >> v2:172.21.15.152:3300/0 conn(0x7f19ac0b0480 0x7f19ac14a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:03.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19aaffd700 1 --2- 172.21.15.140:0/2665098325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 0x7f19ac0accb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f199c002f70 tx=0x7f199c006eb0).ready entity=mon.2 client_cookie=27c2d2deb0397be6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:03.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19a9ffb700 1 -- 172.21.15.140:0/2665098325 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f199c00d570 con 0x7f19ac0b5b50 2022-07-21T01:56:03.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19a9ffb700 1 -- 172.21.15.140:0/2665098325 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c000e90 con 0x7f19ac0b5b50 2022-07-21T01:56:03.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.590+0000 7f19a9ffb700 1 -- 172.21.15.140:0/2665098325 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f199c00d900 con 0x7f19ac0b5b50 2022-07-21T01:56:03.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 -- 172.21.15.140:0/2665098325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 msgr2=0x7f19ac0accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 --2- 172.21.15.140:0/2665098325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 0x7f19ac0accb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f199c002f70 tx=0x7f199c006eb0).stop 2022-07-21T01:56:03.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 -- 172.21.15.140:0/2665098325 shutdown_connections 2022-07-21T01:56:03.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 --2- 172.21.15.140:0/2665098325 >> v2:172.21.15.152:3300/0 conn(0x7f19ac0b0480 0x7f19ac14a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 --2- 172.21.15.140:0/2665098325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 0x7f19ac0accb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 -- 172.21.15.140:0/2665098325 >> 172.21.15.140:0/2665098325 conn(0x7f19ac0635e0 msgr2=0x7f19ac0b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:03.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 -- 172.21.15.140:0/2665098325 shutdown_connections 2022-07-21T01:56:03.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 -- 172.21.15.140:0/2665098325 wait complete. 2022-07-21T01:56:03.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 Processor -- start 2022-07-21T01:56:03.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 -- start start 2022-07-21T01:56:03.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f19ac0b0480 0x7f19ac063c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 0x7f19ac1499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ac14ab60 0x7f19ac149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac156ae0 con 0x7f19ac0b0480 2022-07-21T01:56:03.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac156d80 con 0x7f19ac14ab60 2022-07-21T01:56:03.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19b184f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac157020 con 0x7f19ac0b5b50 2022-07-21T01:56:03.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f19ac0b0480 0x7f19ac063c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f19ac0b0480 0x7f19ac063c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38692/0 (socket says 172.21.15.140:38692) 2022-07-21T01:56:03.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19aaffd700 1 -- 172.21.15.140:0/2167748266 learned_addr learned my addr 172.21.15.140:0/2167748266 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:03.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19aa7fc700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 0x7f19ac1499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19ab7fe700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ac14ab60 0x7f19ac149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19aaffd700 1 -- 172.21.15.140:0/2167748266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 msgr2=0x7f19ac1499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19aaffd700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 0x7f19ac1499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19aaffd700 1 -- 172.21.15.140:0/2167748266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ac14ab60 msgr2=0x7f19ac149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19aaffd700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ac14ab60 0x7f19ac149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19aaffd700 1 -- 172.21.15.140:0/2167748266 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a000b180 con 0x7f19ac0b0480 2022-07-21T01:56:03.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19ab7fe700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ac14ab60 0x7f19ac149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:03.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.594+0000 7f19aaffd700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f19ac0b0480 0x7f19ac063c80 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f199c002f70 tx=0x7f199c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:03.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.598+0000 7f199bfff700 1 -- 172.21.15.140:0/2167748266 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f199c0117e0 con 0x7f19ac0b0480 2022-07-21T01:56:03.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.598+0000 7f199bfff700 1 -- 172.21.15.140:0/2167748266 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c011940 con 0x7f19ac0b0480 2022-07-21T01:56:03.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.598+0000 7f19b184f700 1 -- 172.21.15.140:0/2167748266 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c00b1e0 con 0x7f19ac0b0480 2022-07-21T01:56:03.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.598+0000 7f199bfff700 1 -- 172.21.15.140:0/2167748266 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f199c011ca0 con 0x7f19ac0b0480 2022-07-21T01:56:03.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.598+0000 7f19b184f700 1 -- 172.21.15.140:0/2167748266 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac148f20 con 0x7f19ac0b0480 2022-07-21T01:56:03.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.598+0000 7f19b184f700 1 -- 172.21.15.140:0/2167748266 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac1490e0 con 0x7f19ac0b0480 2022-07-21T01:56:03.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.598+0000 7f199bfff700 1 -- 172.21.15.140:0/2167748266 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f199c01a040 con 0x7f19ac0b0480 2022-07-21T01:56:03.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.598+0000 7f199bfff700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1988053b00 0x7f1988055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.598+0000 7f199bfff700 1 -- 172.21.15.140:0/2167748266 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f199c06cbc0 con 0x7f19ac0b0480 2022-07-21T01:56:03.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.598+0000 7f19aa7fc700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1988053b00 0x7f1988055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.598+0000 7f19aa7fc700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1988053b00 0x7f1988055fc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f19a000b000 tx=0x7f19a0006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:03.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.606+0000 7f199bfff700 1 -- 172.21.15.140:0/2167748266 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f199c043d70 con 0x7f19ac0b0480 2022-07-21T01:56:03.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.730+0000 7f19b184f700 1 -- 172.21.15.140:0/2167748266 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f19ac05d210 con 0x7f19ac0b0480 2022-07-21T01:56:03.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.730+0000 7f199bfff700 1 -- 172.21.15.140:0/2167748266 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v99) v1 ==== 110+0+10 (secure 0 0 0) 0x7f199c023020 con 0x7f19ac0b0480 2022-07-21T01:56:03.735 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:56:03.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 -- 172.21.15.140:0/2167748266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1988053b00 msgr2=0x7f1988055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1988053b00 0x7f1988055fc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f19a000b000 tx=0x7f19a0006df0).stop 2022-07-21T01:56:03.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 -- 172.21.15.140:0/2167748266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f19ac0b0480 msgr2=0x7f19ac063c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f19ac0b0480 0x7f19ac063c80 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f199c002f70 tx=0x7f199c015000).stop 2022-07-21T01:56:03.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 -- 172.21.15.140:0/2167748266 shutdown_connections 2022-07-21T01:56:03.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1988053b00 0x7f1988055fc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ac14ab60 0x7f19ac149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f19ac0b5b50 0x7f19ac1499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 --2- 172.21.15.140:0/2167748266 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f19ac0b0480 0x7f19ac063c80 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 -- 172.21.15.140:0/2167748266 >> 172.21.15.140:0/2167748266 conn(0x7f19ac0635e0 msgr2=0x7f19ac141870 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:03.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 -- 172.21.15.140:0/2167748266 shutdown_connections 2022-07-21T01:56:03.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.734+0000 7f19b184f700 1 -- 172.21.15.140:0/2167748266 wait complete. 2022-07-21T01:56:03.748 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:56:03.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.850+0000 7f05eaf56700 1 Processor -- start 2022-07-21T01:56:03.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.850+0000 7f05eaf56700 1 -- start start 2022-07-21T01:56:03.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.850+0000 7f05eaf56700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f05e40b5960 0x7f05e4142b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.850+0000 7f05eaf56700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05e4143060 0x7f05e409b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.850+0000 7f05eaf56700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e4147030 con 0x7f05e405d0e0 2022-07-21T01:56:03.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.850+0000 7f05eaf56700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e405d4c0 con 0x7f05e40b5960 2022-07-21T01:56:03.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.850+0000 7f05eaf56700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e41461e0 con 0x7f05e4143060 2022-07-21T01:56:03.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e3fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f05e40b5960 0x7f05e4142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e3fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f05e40b5960 0x7f05e4142b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38702/0 (socket says 172.21.15.140:38702) 2022-07-21T01:56:03.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e3fff700 1 -- 172.21.15.140:0/2535253896 learned_addr learned my addr 172.21.15.140:0/2535253896 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:03.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e94f3700 1 --2- 172.21.15.140:0/2535253896 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05e4143060 0x7f05e409b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e3fff700 1 -- 172.21.15.140:0/2535253896 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05e4143060 msgr2=0x7f05e409b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:03.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e3fff700 1 --2- 172.21.15.140:0/2535253896 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05e4143060 0x7f05e409b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e3fff700 1 -- 172.21.15.140:0/2535253896 >> v1:172.21.15.152:6789/0 conn(0x7f05e405d0e0 legacy=0x7f05e4146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e3fff700 1 -- 172.21.15.140:0/2535253896 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4146480 con 0x7f05e40b5960 2022-07-21T01:56:03.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e3fff700 1 --2- 172.21.15.140:0/2535253896 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05e40b5960 0x7f05e4142b70 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f05d4002f70 tx=0x7f05d4006eb0).ready entity=mon.0 client_cookie=eeb8c8f61043c01d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:03.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e37fe700 1 -- 172.21.15.140:0/2535253896 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05d400d570 con 0x7f05e40b5960 2022-07-21T01:56:03.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e37fe700 1 -- 172.21.15.140:0/2535253896 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d4000e90 con 0x7f05e40b5960 2022-07-21T01:56:03.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05e37fe700 1 -- 172.21.15.140:0/2535253896 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05d400d900 con 0x7f05e40b5960 2022-07-21T01:56:03.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05eaf56700 1 -- 172.21.15.140:0/2535253896 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05e40b5960 msgr2=0x7f05e4142b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05eaf56700 1 --2- 172.21.15.140:0/2535253896 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05e40b5960 0x7f05e4142b70 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f05d4002f70 tx=0x7f05d4006eb0).stop 2022-07-21T01:56:03.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05eaf56700 1 -- 172.21.15.140:0/2535253896 shutdown_connections 2022-07-21T01:56:03.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05eaf56700 1 --2- 172.21.15.140:0/2535253896 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05e4143060 0x7f05e409b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05eaf56700 1 --2- 172.21.15.140:0/2535253896 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05e40b5960 0x7f05e4142b70 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05eaf56700 1 -- 172.21.15.140:0/2535253896 >> 172.21.15.140:0/2535253896 conn(0x7f05e40b6a20 msgr2=0x7f05e4066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:03.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05eaf56700 1 -- 172.21.15.140:0/2535253896 shutdown_connections 2022-07-21T01:56:03.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05eaf56700 1 -- 172.21.15.140:0/2535253896 wait complete. 2022-07-21T01:56:03.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.854+0000 7f05eaf56700 1 Processor -- start 2022-07-21T01:56:03.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05eaf56700 1 -- start start 2022-07-21T01:56:03.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05eaf56700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05e405d0e0 0x7f05e40b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05eaf56700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05e40b5960 0x7f05e40b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05eaf56700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05e4143060 0x7f05e4152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05eaf56700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40b1ee0 con 0x7f05e4143060 2022-07-21T01:56:03.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05eaf56700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40b2180 con 0x7f05e40b5960 2022-07-21T01:56:03.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05eaf56700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40b2420 con 0x7f05e405d0e0 2022-07-21T01:56:03.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e94f3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05e4143060 0x7f05e4152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e94f3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05e4143060 0x7f05e4152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38710/0 (socket says 172.21.15.140:38710) 2022-07-21T01:56:03.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e94f3700 1 -- 172.21.15.140:0/3578732749 learned_addr learned my addr 172.21.15.140:0/3578732749 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:03.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e8cf2700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05e405d0e0 0x7f05e40b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e3fff700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05e40b5960 0x7f05e40b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e94f3700 1 -- 172.21.15.140:0/3578732749 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05e405d0e0 msgr2=0x7f05e40b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e94f3700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05e405d0e0 0x7f05e40b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e94f3700 1 -- 172.21.15.140:0/3578732749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05e40b5960 msgr2=0x7f05e40b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e94f3700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05e40b5960 0x7f05e40b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:03.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e94f3700 1 -- 172.21.15.140:0/3578732749 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d400b1e0 con 0x7f05e4143060 2022-07-21T01:56:03.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e94f3700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05e4143060 0x7f05e4152940 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f05dc00c000 tx=0x7f05dc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:03.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e1ffb700 1 -- 172.21.15.140:0/3578732749 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05dc012600 con 0x7f05e4143060 2022-07-21T01:56:03.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e1ffb700 1 -- 172.21.15.140:0/3578732749 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc012760 con 0x7f05e4143060 2022-07-21T01:56:03.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05eaf56700 1 -- 172.21.15.140:0/3578732749 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e40b2db0 con 0x7f05e4143060 2022-07-21T01:56:03.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05e1ffb700 1 -- 172.21.15.140:0/3578732749 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05dc012ac0 con 0x7f05e4143060 2022-07-21T01:56:03.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05eaf56700 1 -- 172.21.15.140:0/3578732749 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e40b3220 con 0x7f05e4143060 2022-07-21T01:56:03.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.858+0000 7f05eaf56700 1 -- 172.21.15.140:0/3578732749 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e40b25e0 con 0x7f05e4143060 2022-07-21T01:56:03.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.862+0000 7f05e1ffb700 1 -- 172.21.15.140:0/3578732749 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f05dc022070 con 0x7f05e4143060 2022-07-21T01:56:03.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.862+0000 7f05e1ffb700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f05c40539e0 0x7f05c4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:03.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.862+0000 7f05e1ffb700 1 -- 172.21.15.140:0/3578732749 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f05dc06c080 con 0x7f05e4143060 2022-07-21T01:56:03.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.862+0000 7f05e8cf2700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f05c40539e0 0x7f05c4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:03.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.862+0000 7f05e8cf2700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f05c40539e0 0x7f05c4055ea0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f05d800d000 tx=0x7f05d8000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:03.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.866+0000 7f05e1ffb700 1 -- 172.21.15.140:0/3578732749 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f05dc0432d0 con 0x7f05e4143060 2022-07-21T01:56:03.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.990+0000 7f05eaf56700 1 -- 172.21.15.140:0/3578732749 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f05e40b01d0 con 0x7f05e4143060 2022-07-21T01:56:03.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.990+0000 7f05e1ffb700 1 -- 172.21.15.140:0/3578732749 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v99) v1 ==== 92+0+10 (secure 0 0 0) 0x7f05dc016020 con 0x7f05e4143060 2022-07-21T01:56:03.996 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:03.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 -- 172.21.15.140:0/3578732749 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f05c40539e0 msgr2=0x7f05c4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f05c40539e0 0x7f05c4055ea0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f05d800d000 tx=0x7f05d8000bc0).stop 2022-07-21T01:56:03.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 -- 172.21.15.140:0/3578732749 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05e4143060 msgr2=0x7f05e4152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:03.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05e4143060 0x7f05e4152940 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f05dc00c000 tx=0x7f05dc0091d0).stop 2022-07-21T01:56:03.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 -- 172.21.15.140:0/3578732749 shutdown_connections 2022-07-21T01:56:03.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f05c40539e0 0x7f05c4055ea0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05e4143060 0x7f05e4152940 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05e40b5960 0x7f05e40b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 --2- 172.21.15.140:0/3578732749 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05e405d0e0 0x7f05e40b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 -- 172.21.15.140:0/3578732749 >> 172.21.15.140:0/3578732749 conn(0x7f05e40b6a20 msgr2=0x7f05e4145910 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:04.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 -- 172.21.15.140:0/3578732749 shutdown_connections 2022-07-21T01:56:04.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:03.994+0000 7f05eaf56700 1 -- 172.21.15.140:0/3578732749 wait complete. 2022-07-21T01:56:04.009 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:56:04.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 Processor -- start 2022-07-21T01:56:04.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 -- start start 2022-07-21T01:56:04.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f24480b5650 0x7f24480b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24480adff0 0x7f2448147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f24480af8e0 0x7f244805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2448147010 con 0x7f24480af8e0 2022-07-21T01:56:04.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24481472b0 con 0x7f24480b5650 2022-07-21T01:56:04.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2448146d40 con 0x7f24480adff0 2022-07-21T01:56:04.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244d228700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f24480af8e0 0x7f244805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244d228700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f24480af8e0 0x7f244805bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38716/0 (socket says 172.21.15.140:38716) 2022-07-21T01:56:04.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244d228700 1 -- 172.21.15.140:0/3339129487 learned_addr learned my addr 172.21.15.140:0/3339129487 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:04.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244ca27700 1 --2- 172.21.15.140:0/3339129487 >> v2:172.21.15.152:3300/0 conn(0x7f24480b5650 0x7f24480b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f2447fff700 1 --2- 172.21.15.140:0/3339129487 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24480adff0 0x7f2448147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244d228700 1 -- 172.21.15.140:0/3339129487 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24480adff0 msgr2=0x7f2448147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:04.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244d228700 1 --2- 172.21.15.140:0/3339129487 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24480adff0 0x7f2448147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244d228700 1 -- 172.21.15.140:0/3339129487 >> v2:172.21.15.152:3300/0 conn(0x7f24480b5650 msgr2=0x7f24480b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:04.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244d228700 1 --2- 172.21.15.140:0/3339129487 >> v2:172.21.15.152:3300/0 conn(0x7f24480b5650 0x7f24480b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244d228700 1 -- 172.21.15.140:0/3339129487 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2448147c40 con 0x7f24480af8e0 2022-07-21T01:56:04.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244d228700 1 --2- 172.21.15.140:0/3339129487 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24480af8e0 0x7f244805bd90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f2430002f70 tx=0x7f2430006eb0).ready entity=mon.0 client_cookie=fe10d31c0ceb865 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:04.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f24477fe700 1 -- 172.21.15.140:0/3339129487 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f243000d570 con 0x7f24480af8e0 2022-07-21T01:56:04.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f24477fe700 1 -- 172.21.15.140:0/3339129487 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430000e90 con 0x7f24480af8e0 2022-07-21T01:56:04.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f24477fe700 1 -- 172.21.15.140:0/3339129487 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f243000d900 con 0x7f24480af8e0 2022-07-21T01:56:04.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 -- 172.21.15.140:0/3339129487 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24480af8e0 msgr2=0x7f244805bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:04.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 --2- 172.21.15.140:0/3339129487 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24480af8e0 0x7f244805bd90 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f2430002f70 tx=0x7f2430006eb0).stop 2022-07-21T01:56:04.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 -- 172.21.15.140:0/3339129487 shutdown_connections 2022-07-21T01:56:04.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 --2- 172.21.15.140:0/3339129487 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24480af8e0 0x7f244805bd90 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 --2- 172.21.15.140:0/3339129487 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24480adff0 0x7f2448147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 --2- 172.21.15.140:0/3339129487 >> v2:172.21.15.152:3300/0 conn(0x7f24480b5650 0x7f24480b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 -- 172.21.15.140:0/3339129487 >> 172.21.15.140:0/3339129487 conn(0x7f24480634e0 msgr2=0x7f24480b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:04.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 -- 172.21.15.140:0/3339129487 shutdown_connections 2022-07-21T01:56:04.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.106+0000 7f244da29700 1 -- 172.21.15.140:0/3339129487 wait complete. 2022-07-21T01:56:04.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244da29700 1 Processor -- start 2022-07-21T01:56:04.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244da29700 1 -- start start 2022-07-21T01:56:04.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244da29700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24480adff0 0x7f244814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244da29700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24480af8e0 0x7f2448064e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244da29700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24480b5650 0x7f2448145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244da29700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2448148540 con 0x7f24480adff0 2022-07-21T01:56:04.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244da29700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24480654a0 con 0x7f24480af8e0 2022-07-21T01:56:04.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244da29700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244815a350 con 0x7f24480b5650 2022-07-21T01:56:04.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244ca27700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24480adff0 0x7f244814a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244ca27700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24480adff0 0x7f244814a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38720/0 (socket says 172.21.15.140:38720) 2022-07-21T01:56:04.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244ca27700 1 -- 172.21.15.140:0/3051860226 learned_addr learned my addr 172.21.15.140:0/3051860226 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:04.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244d228700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24480b5650 0x7f2448145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f2447fff700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24480af8e0 0x7f2448064e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244ca27700 1 -- 172.21.15.140:0/3051860226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24480b5650 msgr2=0x7f2448145350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:04.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244ca27700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24480b5650 0x7f2448145350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244ca27700 1 -- 172.21.15.140:0/3051860226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24480af8e0 msgr2=0x7f2448064e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:04.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244ca27700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24480af8e0 0x7f2448064e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244ca27700 1 -- 172.21.15.140:0/3051860226 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243000b1e0 con 0x7f24480adff0 2022-07-21T01:56:04.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244d228700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24480b5650 0x7f2448145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:04.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.110+0000 7f244ca27700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24480adff0 0x7f244814a170 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f2448064590 tx=0x7f2438006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:04.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.114+0000 7f2445ffb700 1 -- 172.21.15.140:0/3051860226 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2438010780 con 0x7f24480adff0 2022-07-21T01:56:04.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.114+0000 7f2445ffb700 1 -- 172.21.15.140:0/3051860226 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24380108e0 con 0x7f24480adff0 2022-07-21T01:56:04.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.114+0000 7f2445ffb700 1 -- 172.21.15.140:0/3051860226 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2438010c40 con 0x7f24480adff0 2022-07-21T01:56:04.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.114+0000 7f244da29700 1 -- 172.21.15.140:0/3051860226 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24480658d0 con 0x7f24480adff0 2022-07-21T01:56:04.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.114+0000 7f244da29700 1 -- 172.21.15.140:0/3051860226 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24481491e0 con 0x7f24480adff0 2022-07-21T01:56:04.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.114+0000 7f244da29700 1 -- 172.21.15.140:0/3051860226 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244815a190 con 0x7f24480adff0 2022-07-21T01:56:04.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.114+0000 7f2445ffb700 1 -- 172.21.15.140:0/3051860226 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f243801c460 con 0x7f24480adff0 2022-07-21T01:56:04.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.114+0000 7f2445ffb700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f242c053a70 0x7f242c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.114+0000 7f2445ffb700 1 -- 172.21.15.140:0/3051860226 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f243806bf60 con 0x7f24480adff0 2022-07-21T01:56:04.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.114+0000 7f2447fff700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f242c053a70 0x7f242c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.114+0000 7f2447fff700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f242c053a70 0x7f242c055f30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f244000d000 tx=0x7f2440008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:04.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.122+0000 7f2445ffb700 1 -- 172.21.15.140:0/3051860226 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2438043150 con 0x7f24480adff0 2022-07-21T01:56:04.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.242+0000 7f244da29700 1 -- 172.21.15.140:0/3051860226 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f24480574e0 con 0x7f24480adff0 2022-07-21T01:56:04.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.242+0000 7f2445ffb700 1 -- 172.21.15.140:0/3051860226 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+143 (secure 0 0 0) 0x7f2438046770 con 0x7f24480adff0 2022-07-21T01:56:04.247 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:04.248 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:56:04.248 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:04.248 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:56:04.248 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:56:04.249 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:04.249 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:04.249 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:56:04.249 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:56:04.250 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:04.250 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:56:04.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 -- 172.21.15.140:0/3051860226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f242c053a70 msgr2=0x7f242c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:04.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f242c053a70 0x7f242c055f30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f244000d000 tx=0x7f2440008040).stop 2022-07-21T01:56:04.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 -- 172.21.15.140:0/3051860226 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24480adff0 msgr2=0x7f244814a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:04.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24480adff0 0x7f244814a170 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f2448064590 tx=0x7f2438006df0).stop 2022-07-21T01:56:04.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 -- 172.21.15.140:0/3051860226 shutdown_connections 2022-07-21T01:56:04.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f242c053a70 0x7f242c055f30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24480b5650 0x7f2448145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24480af8e0 0x7f2448064e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 --2- 172.21.15.140:0/3051860226 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24480adff0 0x7f244814a170 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 -- 172.21.15.140:0/3051860226 >> 172.21.15.140:0/3051860226 conn(0x7f24480634e0 msgr2=0x7f244810e130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:04.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 -- 172.21.15.140:0/3051860226 shutdown_connections 2022-07-21T01:56:04.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.246+0000 7f244da29700 1 -- 172.21.15.140:0/3051860226 wait complete. 2022-07-21T01:56:04.260 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:56:04.261 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:56:04.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.362+0000 7f61220a7700 1 Processor -- start 2022-07-21T01:56:04.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.362+0000 7f61220a7700 1 -- start start 2022-07-21T01:56:04.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.362+0000 7f61220a7700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f611c0adb80 0x7f611c0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.362+0000 7f61220a7700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611c14b500 con 0x7f611c14b070 2022-07-21T01:56:04.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.362+0000 7f61220a7700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611c14a690 con 0x7f611c0b2210 2022-07-21T01:56:04.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.362+0000 7f61220a7700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611c146fd0 con 0x7f611c0adb80 2022-07-21T01:56:04.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.362+0000 7f61208a4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f611c0adb80 0x7f611c0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.362+0000 7f61218a6700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f611c14b070 0x7f611c0aebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52878/0 (socket says 172.21.15.140:52878) 2022-07-21T01:56:04.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.362+0000 7f61218a6700 1 -- 172.21.15.140:0/1632232792 learned_addr learned my addr 172.21.15.140:0/1632232792 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:04.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f61208a4700 1 -- 172.21.15.140:0/1632232792 >> v1:172.21.15.152:6789/0 conn(0x7f611c0b2210 legacy=0x7f611c067680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:04.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f6113fff700 1 -- 172.21.15.140:0/1632232792 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606297809 0 0) 0x7f611c14b500 con 0x7f611c14b070 2022-07-21T01:56:04.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f61208a4700 1 -- 172.21.15.140:0/1632232792 >> v1:172.21.15.140:6789/0 conn(0x7f611c14b070 legacy=0x7f611c0aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:04.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f61208a4700 1 -- 172.21.15.140:0/1632232792 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c14a930 con 0x7f611c0adb80 2022-07-21T01:56:04.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f61208a4700 1 --2- 172.21.15.140:0/1632232792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f611c0adb80 0x7f611c0b25f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f610c002f70 tx=0x7f610c006df0).ready entity=mon.0 client_cookie=4b2526943b7c3675 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:04.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f6113fff700 1 -- 172.21.15.140:0/1632232792 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f610c00d610 con 0x7f611c0adb80 2022-07-21T01:56:04.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f6113fff700 1 -- 172.21.15.140:0/1632232792 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610c000e90 con 0x7f611c0adb80 2022-07-21T01:56:04.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f6113fff700 1 -- 172.21.15.140:0/1632232792 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f610c00d960 con 0x7f611c0adb80 2022-07-21T01:56:04.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f61220a7700 1 -- 172.21.15.140:0/1632232792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f611c0adb80 msgr2=0x7f611c0b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:04.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f61220a7700 1 --2- 172.21.15.140:0/1632232792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f611c0adb80 0x7f611c0b25f0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f610c002f70 tx=0x7f610c006df0).stop 2022-07-21T01:56:04.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f61220a7700 1 -- 172.21.15.140:0/1632232792 shutdown_connections 2022-07-21T01:56:04.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f61220a7700 1 --2- 172.21.15.140:0/1632232792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f611c0adb80 0x7f611c0b25f0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f61220a7700 1 -- 172.21.15.140:0/1632232792 >> 172.21.15.140:0/1632232792 conn(0x7f611c0632d0 msgr2=0x7f611c0636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:04.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f61220a7700 1 -- 172.21.15.140:0/1632232792 shutdown_connections 2022-07-21T01:56:04.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.366+0000 7f61220a7700 1 -- 172.21.15.140:0/1632232792 wait complete. 2022-07-21T01:56:04.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61220a7700 1 Processor -- start 2022-07-21T01:56:04.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61220a7700 1 -- start start 2022-07-21T01:56:04.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61220a7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f611c0adb80 0x7f611c149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61220a7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f611c0b2210 0x7f611c159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61220a7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f611c14b070 0x7f611c159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61220a7700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c065a90 con 0x7f611c0b2210 2022-07-21T01:56:04.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61220a7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c066760 con 0x7f611c0adb80 2022-07-21T01:56:04.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61220a7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611c065710 con 0x7f611c14b070 2022-07-21T01:56:04.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61208a4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f611c0b2210 0x7f611c159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61208a4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f611c0b2210 0x7f611c159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38736/0 (socket says 172.21.15.140:38736) 2022-07-21T01:56:04.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61208a4700 1 -- 172.21.15.140:0/2400852673 learned_addr learned my addr 172.21.15.140:0/2400852673 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:04.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61210a5700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f611c0adb80 0x7f611c149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61218a6700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f611c14b070 0x7f611c159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61208a4700 1 -- 172.21.15.140:0/2400852673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f611c14b070 msgr2=0x7f611c159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:04.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61208a4700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f611c14b070 0x7f611c159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61208a4700 1 -- 172.21.15.140:0/2400852673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f611c0adb80 msgr2=0x7f611c149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:04.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61208a4700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f611c0adb80 0x7f611c149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:04.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61208a4700 1 -- 172.21.15.140:0/2400852673 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610c00b180 con 0x7f611c0b2210 2022-07-21T01:56:04.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61210a5700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f611c0adb80 0x7f611c149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:04.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61218a6700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f611c14b070 0x7f611c159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:04.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61208a4700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f611c0b2210 0x7f611c159a80 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f610c002f70 tx=0x7f610c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:04.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61127fc700 1 -- 172.21.15.140:0/2400852673 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f610c00de70 con 0x7f611c0b2210 2022-07-21T01:56:04.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61127fc700 1 -- 172.21.15.140:0/2400852673 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610c011c60 con 0x7f611c0b2210 2022-07-21T01:56:04.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61220a7700 1 -- 172.21.15.140:0/2400852673 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611c15a460 con 0x7f611c0b2210 2022-07-21T01:56:04.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.370+0000 7f61127fc700 1 -- 172.21.15.140:0/2400852673 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f610c01e460 con 0x7f611c0b2210 2022-07-21T01:56:04.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.374+0000 7f61220a7700 1 -- 172.21.15.140:0/2400852673 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c064770 con 0x7f611c0b2210 2022-07-21T01:56:04.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.374+0000 7f61220a7700 1 -- 172.21.15.140:0/2400852673 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611c064a50 con 0x7f611c0b2210 2022-07-21T01:56:04.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.374+0000 7f61127fc700 1 -- 172.21.15.140:0/2400852673 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f610c025030 con 0x7f611c0b2210 2022-07-21T01:56:04.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.374+0000 7f61127fc700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61040539e0 0x7f6104055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:04.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.374+0000 7f61127fc700 1 -- 172.21.15.140:0/2400852673 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f610c06d0b0 con 0x7f611c0b2210 2022-07-21T01:56:04.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.374+0000 7f61210a5700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61040539e0 0x7f6104055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:04.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.374+0000 7f61210a5700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61040539e0 0x7f6104055ea0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f611800b000 tx=0x7f6118009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:04.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.378+0000 7f61127fc700 1 -- 172.21.15.140:0/2400852673 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f610c044300 con 0x7f611c0b2210 2022-07-21T01:56:04.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:04.502+0000 7f61220a7700 1 -- 172.21.15.140:0/2400852673 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f611c1564d0 con 0x7f611c0b2210 2022-07-21T01:56:05.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.178+0000 7f61127fc700 1 -- 172.21.15.140:0/2400852673 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v100) v1 ==== 133+0+0 (secure 0 0 0) 0x7f610c023020 con 0x7f611c0b2210 2022-07-21T01:56:05.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 -- 172.21.15.140:0/2400852673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61040539e0 msgr2=0x7f6104055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:05.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61040539e0 0x7f6104055ea0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f611800b000 tx=0x7f6118009040).stop 2022-07-21T01:56:05.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 -- 172.21.15.140:0/2400852673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f611c0b2210 msgr2=0x7f611c159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:05.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f611c0b2210 0x7f611c159a80 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f610c002f70 tx=0x7f610c015000).stop 2022-07-21T01:56:05.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 -- 172.21.15.140:0/2400852673 shutdown_connections 2022-07-21T01:56:05.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61040539e0 0x7f6104055ea0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f611c14b070 0x7f611c159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f611c0b2210 0x7f611c159a80 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 --2- 172.21.15.140:0/2400852673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f611c0adb80 0x7f611c149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 -- 172.21.15.140:0/2400852673 >> 172.21.15.140:0/2400852673 conn(0x7f611c0632d0 msgr2=0x7f611c145e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:05.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 -- 172.21.15.140:0/2400852673 shutdown_connections 2022-07-21T01:56:05.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.182+0000 7f61220a7700 1 -- 172.21.15.140:0/2400852673 wait complete. 2022-07-21T01:56:05.191 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:56:05.202 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:56:05.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 Processor -- start 2022-07-21T01:56:05.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 -- start start 2022-07-21T01:56:05.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f62e41462a0 0x7f62e4146680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:05.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 0x7f62e4146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:05.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e40afdb0 con 0x7f62e41422c0 2022-07-21T01:56:05.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e40b0050 con 0x7f62e41462a0 2022-07-21T01:56:05.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e405fcd0 con 0x7f62e40988b0 2022-07-21T01:56:05.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e37fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 0x7f62e4146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:05.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e3fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f62e41462a0 0x7f62e4146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:05.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e3fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f62e41462a0 0x7f62e4146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:44482/0 (socket says 172.21.15.140:44482) 2022-07-21T01:56:05.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e37fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 0x7f62e4146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:35474/0 (socket says 172.21.15.140:35474) 2022-07-21T01:56:05.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e3fff700 1 -- 172.21.15.140:0/3254628684 learned_addr learned my addr 172.21.15.140:0/3254628684 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:05.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e3fff700 1 -- 172.21.15.140:0/3254628684 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 msgr2=0x7f62e4146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:05.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e3fff700 1 --2- 172.21.15.140:0/3254628684 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 0x7f62e4146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e3fff700 1 -- 172.21.15.140:0/3254628684 >> v1:172.21.15.152:6789/0 conn(0x7f62e41422c0 legacy=0x7f62e4060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:05.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e3fff700 1 -- 172.21.15.140:0/3254628684 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40afae0 con 0x7f62e41462a0 2022-07-21T01:56:05.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e37fe700 1 --2- 172.21.15.140:0/3254628684 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 0x7f62e4146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:05.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e3fff700 1 --2- 172.21.15.140:0/3254628684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62e41462a0 0x7f62e4146680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f62d4002f70 tx=0x7f62d4006eb0).ready entity=mon.1 client_cookie=9d3aee179ae187c3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:05.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e2ffd700 1 -- 172.21.15.140:0/3254628684 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62d400d570 con 0x7f62e41462a0 2022-07-21T01:56:05.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e2ffd700 1 -- 172.21.15.140:0/3254628684 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d4000e90 con 0x7f62e41462a0 2022-07-21T01:56:05.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62e2ffd700 1 -- 172.21.15.140:0/3254628684 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62d400d900 con 0x7f62e41462a0 2022-07-21T01:56:05.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 -- 172.21.15.140:0/3254628684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62e41462a0 msgr2=0x7f62e4146680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:05.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 --2- 172.21.15.140:0/3254628684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62e41462a0 0x7f62e4146680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f62d4002f70 tx=0x7f62d4006eb0).stop 2022-07-21T01:56:05.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 -- 172.21.15.140:0/3254628684 shutdown_connections 2022-07-21T01:56:05.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 --2- 172.21.15.140:0/3254628684 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 0x7f62e4146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 --2- 172.21.15.140:0/3254628684 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62e41462a0 0x7f62e4146680 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 -- 172.21.15.140:0/3254628684 >> 172.21.15.140:0/3254628684 conn(0x7f62e40b5f00 msgr2=0x7f62e40b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:05.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 -- 172.21.15.140:0/3254628684 shutdown_connections 2022-07-21T01:56:05.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.310+0000 7f62ea6c5700 1 -- 172.21.15.140:0/3254628684 wait complete. 2022-07-21T01:56:05.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62ea6c5700 1 Processor -- start 2022-07-21T01:56:05.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62ea6c5700 1 -- start start 2022-07-21T01:56:05.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62ea6c5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 0x7f62e409ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:05.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62ea6c5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62e41422c0 0x7f62e4155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:05.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62ea6c5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62e41462a0 0x7f62e40b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:05.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62ea6c5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e409d5a0 con 0x7f62e41462a0 2022-07-21T01:56:05.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62ea6c5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e409d050 con 0x7f62e41422c0 2022-07-21T01:56:05.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62ea6c5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e409f380 con 0x7f62e40988b0 2022-07-21T01:56:05.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e8c62700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62e41462a0 0x7f62e40b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:05.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e8c62700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62e41462a0 0x7f62e40b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38752/0 (socket says 172.21.15.140:38752) 2022-07-21T01:56:05.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e8c62700 1 -- 172.21.15.140:0/235750173 learned_addr learned my addr 172.21.15.140:0/235750173 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:05.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e3fff700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 0x7f62e409ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:05.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e37fe700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62e41422c0 0x7f62e4155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:05.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e8c62700 1 -- 172.21.15.140:0/235750173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 msgr2=0x7f62e409ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:05.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e8c62700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 0x7f62e409ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e8c62700 1 -- 172.21.15.140:0/235750173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62e41422c0 msgr2=0x7f62e4155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:05.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e8c62700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62e41422c0 0x7f62e4155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e8c62700 1 -- 172.21.15.140:0/235750173 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d800b180 con 0x7f62e41462a0 2022-07-21T01:56:05.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e8c62700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62e41462a0 0x7f62e40b1830 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f62e409f510 tx=0x7f62dc007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:05.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e17fa700 1 -- 172.21.15.140:0/235750173 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62dc00f470 con 0x7f62e41462a0 2022-07-21T01:56:05.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e17fa700 1 -- 172.21.15.140:0/235750173 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62dc00f5d0 con 0x7f62e41462a0 2022-07-21T01:56:05.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62ea6c5700 1 -- 172.21.15.140:0/235750173 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d400b1e0 con 0x7f62e41462a0 2022-07-21T01:56:05.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62e17fa700 1 -- 172.21.15.140:0/235750173 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62dc00f930 con 0x7f62e41462a0 2022-07-21T01:56:05.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.314+0000 7f62ea6c5700 1 -- 172.21.15.140:0/235750173 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e4156040 con 0x7f62e41462a0 2022-07-21T01:56:05.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.318+0000 7f62ea6c5700 1 -- 172.21.15.140:0/235750173 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e41567f0 con 0x7f62e41462a0 2022-07-21T01:56:05.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.318+0000 7f62e17fa700 1 -- 172.21.15.140:0/235750173 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f62dc019030 con 0x7f62e41462a0 2022-07-21T01:56:05.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.318+0000 7f62e17fa700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62c4053b00 0x7f62c4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:05.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.318+0000 7f62e17fa700 1 -- 172.21.15.140:0/235750173 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f62dc06b470 con 0x7f62e41462a0 2022-07-21T01:56:05.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.318+0000 7f62e3fff700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62c4053b00 0x7f62c4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:05.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.318+0000 7f62e3fff700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62c4053b00 0x7f62c4055fc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f62d401c000 tx=0x7f62d4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:05.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.322+0000 7f62e17fa700 1 -- 172.21.15.140:0/235750173 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f62dc042450 con 0x7f62e41462a0 2022-07-21T01:56:05.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.450+0000 7f62ea6c5700 1 -- 172.21.15.140:0/235750173 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f62e40574e0 con 0x7f62e41462a0 2022-07-21T01:56:05.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.849+0000 7f62e17fa700 1 -- 172.21.15.140:0/235750173 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v101) v1 ==== 125+0+0 (secure 0 0 0) 0x7f62dc045a70 con 0x7f62e41462a0 2022-07-21T01:56:05.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 -- 172.21.15.140:0/235750173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62c4053b00 msgr2=0x7f62c4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:05.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62c4053b00 0x7f62c4055fc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f62d401c000 tx=0x7f62d4015000).stop 2022-07-21T01:56:05.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 -- 172.21.15.140:0/235750173 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62e41462a0 msgr2=0x7f62e40b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:05.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62e41462a0 0x7f62e40b1830 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f62e409f510 tx=0x7f62dc007f40).stop 2022-07-21T01:56:05.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 -- 172.21.15.140:0/235750173 shutdown_connections 2022-07-21T01:56:05.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f62c4053b00 0x7f62c4055fc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f62e41462a0 0x7f62e40b1830 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62e41422c0 0x7f62e4155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 --2- 172.21.15.140:0/235750173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62e40988b0 0x7f62e409ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 -- 172.21.15.140:0/235750173 >> 172.21.15.140:0/235750173 conn(0x7f62e40b5f00 msgr2=0x7f62e40b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:05.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 -- 172.21.15.140:0/235750173 shutdown_connections 2022-07-21T01:56:05.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.857+0000 7f62ea6c5700 1 -- 172.21.15.140:0/235750173 wait complete. 2022-07-21T01:56:05.865 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:56:05.875 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:56:05.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f598ccb8700 1 Processor -- start 2022-07-21T01:56:05.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f598ccb8700 1 -- start start 2022-07-21T01:56:05.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f598ccb8700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f59880b34d0 0x7f5988061ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:05.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f598ccb8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5988142720 0x7f5988142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:05.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f598ccb8700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598805f840 con 0x7f59880ab180 2022-07-21T01:56:05.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f598ccb8700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5988142b90 con 0x7f5988142720 2022-07-21T01:56:05.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f598ccb8700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598805fa70 con 0x7f59880b34d0 2022-07-21T01:56:05.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f5986d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5988142720 0x7f5988142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:05.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f5986d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5988142720 0x7f5988142fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38758/0 (socket says 172.21.15.140:38758) 2022-07-21T01:56:05.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f5986d9d700 1 -- 172.21.15.140:0/3024241675 learned_addr learned my addr 172.21.15.140:0/3024241675 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:05.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f598659c700 1 --2- 172.21.15.140:0/3024241675 >> v2:172.21.15.152:3300/0 conn(0x7f59880b34d0 0x7f5988061ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:05.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f5986d9d700 1 -- 172.21.15.140:0/3024241675 >> v2:172.21.15.152:3300/0 conn(0x7f59880b34d0 msgr2=0x7f5988061ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:05.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f5986d9d700 1 --2- 172.21.15.140:0/3024241675 >> v2:172.21.15.152:3300/0 conn(0x7f59880b34d0 0x7f5988061ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f5986d9d700 1 -- 172.21.15.140:0/3024241675 >> v1:172.21.15.152:6789/0 conn(0x7f59880ab180 legacy=0x7f5988142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:05.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f5986d9d700 1 -- 172.21.15.140:0/3024241675 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598805fd10 con 0x7f5988142720 2022-07-21T01:56:05.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f5986d9d700 1 --2- 172.21.15.140:0/3024241675 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5988142720 0x7f5988142fd0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f5988060870 tx=0x7f5980009290).ready entity=mon.0 client_cookie=ebe288569b437099 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:05.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f598559a700 1 -- 172.21.15.140:0/3024241675 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f598000b860 con 0x7f5988142720 2022-07-21T01:56:05.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f598559a700 1 -- 172.21.15.140:0/3024241675 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598000b9c0 con 0x7f5988142720 2022-07-21T01:56:05.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.985+0000 7f598559a700 1 -- 172.21.15.140:0/3024241675 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f598000bb30 con 0x7f5988142720 2022-07-21T01:56:05.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 -- 172.21.15.140:0/3024241675 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5988142720 msgr2=0x7f5988142fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:05.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 --2- 172.21.15.140:0/3024241675 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5988142720 0x7f5988142fd0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f5988060870 tx=0x7f5980009290).stop 2022-07-21T01:56:05.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 -- 172.21.15.140:0/3024241675 shutdown_connections 2022-07-21T01:56:05.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 --2- 172.21.15.140:0/3024241675 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5988142720 0x7f5988142fd0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 --2- 172.21.15.140:0/3024241675 >> v2:172.21.15.152:3300/0 conn(0x7f59880b34d0 0x7f5988061ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:05.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 -- 172.21.15.140:0/3024241675 >> 172.21.15.140:0/3024241675 conn(0x7f59880b6c50 msgr2=0x7f59880b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:05.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 -- 172.21.15.140:0/3024241675 shutdown_connections 2022-07-21T01:56:05.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 -- 172.21.15.140:0/3024241675 wait complete. 2022-07-21T01:56:06.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 Processor -- start 2022-07-21T01:56:06.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 -- start start 2022-07-21T01:56:06.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59880ab180 0x7f5988061320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59880b34d0 0x7f5988144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5988142720 0x7f598813e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59880afdd0 con 0x7f59880ab180 2022-07-21T01:56:06.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5988145130 con 0x7f59880b34d0 2022-07-21T01:56:06.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59880b0000 con 0x7f5988142720 2022-07-21T01:56:06.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59880ab180 0x7f5988061320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59880ab180 0x7f5988061320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38764/0 (socket says 172.21.15.140:38764) 2022-07-21T01:56:06.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598659c700 1 -- 172.21.15.140:0/1504076151 learned_addr learned my addr 172.21.15.140:0/1504076151 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:06.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f5985d9b700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59880b34d0 0x7f5988144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f5986d9d700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5988142720 0x7f598813e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598659c700 1 -- 172.21.15.140:0/1504076151 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5988142720 msgr2=0x7f598813e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598659c700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5988142720 0x7f598813e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598659c700 1 -- 172.21.15.140:0/1504076151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59880b34d0 msgr2=0x7f5988144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598659c700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59880b34d0 0x7f5988144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598659c700 1 -- 172.21.15.140:0/1504076151 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5980009530 con 0x7f59880ab180 2022-07-21T01:56:06.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598659c700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59880ab180 0x7f5988061320 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f597800c000 tx=0x7f5978006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:06.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f59777fe700 1 -- 172.21.15.140:0/1504076151 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5978010780 con 0x7f59880ab180 2022-07-21T01:56:06.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f59777fe700 1 -- 172.21.15.140:0/1504076151 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59780108e0 con 0x7f59880ab180 2022-07-21T01:56:06.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 -- 172.21.15.140:0/1504076151 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59880b02a0 con 0x7f59880ab180 2022-07-21T01:56:06.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f59777fe700 1 -- 172.21.15.140:0/1504076151 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5978010c40 con 0x7f59880ab180 2022-07-21T01:56:06.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.989+0000 7f598ccb8700 1 -- 172.21.15.140:0/1504076151 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5988145580 con 0x7f59880ab180 2022-07-21T01:56:06.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.993+0000 7f598ccb8700 1 -- 172.21.15.140:0/1504076151 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598805ffb0 con 0x7f59880ab180 2022-07-21T01:56:06.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.993+0000 7f59777fe700 1 -- 172.21.15.140:0/1504076151 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5978026430 con 0x7f59880ab180 2022-07-21T01:56:06.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.993+0000 7f59777fe700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5964053980 0x7f5964055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.997+0000 7f5985d9b700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5964053980 0x7f5964055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.997+0000 7f59777fe700 1 -- 172.21.15.140:0/1504076151 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f597806cda0 con 0x7f59880ab180 2022-07-21T01:56:06.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:05.997+0000 7f5985d9b700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5964053980 0x7f5964055e40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f597c00d000 tx=0x7f597c000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:06.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.001+0000 7f59777fe700 1 -- 172.21.15.140:0/1504076151 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5978043cd0 con 0x7f59880ab180 2022-07-21T01:56:06.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.165+0000 7f598ccb8700 1 -- 172.21.15.140:0/1504076151 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f59880574e0 con 0x7f59880ab180 2022-07-21T01:56:06.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.181+0000 7f59777fe700 1 -- 172.21.15.140:0/1504076151 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 19 and data pool 20 v46) v1 ==== 189+0+0 (secure 0 0 0) 0x7f59780472f0 con 0x7f59880ab180 2022-07-21T01:56:06.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.181+0000 7f598ccb8700 1 -- 172.21.15.140:0/1504076151 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5964053980 msgr2=0x7f5964055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.181+0000 7f598ccb8700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5964053980 0x7f5964055e40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f597c00d000 tx=0x7f597c000bc0).stop 2022-07-21T01:56:06.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.185+0000 7f598ccb8700 1 -- 172.21.15.140:0/1504076151 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59880ab180 msgr2=0x7f5988061320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.185+0000 7f598ccb8700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59880ab180 0x7f5988061320 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f597800c000 tx=0x7f5978006df0).stop 2022-07-21T01:56:06.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.185+0000 7f598ccb8700 1 -- 172.21.15.140:0/1504076151 shutdown_connections 2022-07-21T01:56:06.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.185+0000 7f598ccb8700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5964053980 0x7f5964055e40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.185+0000 7f598ccb8700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5988142720 0x7f598813e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.185+0000 7f598ccb8700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59880b34d0 0x7f5988144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.185+0000 7f598ccb8700 1 --2- 172.21.15.140:0/1504076151 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f59880ab180 0x7f5988061320 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.185+0000 7f598ccb8700 1 -- 172.21.15.140:0/1504076151 >> 172.21.15.140:0/1504076151 conn(0x7f59880b6c50 msgr2=0x7f59881403f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.185+0000 7f598ccb8700 1 -- 172.21.15.140:0/1504076151 shutdown_connections 2022-07-21T01:56:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.185+0000 7f598ccb8700 1 -- 172.21.15.140:0/1504076151 wait complete. 2022-07-21T01:56:06.196 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 19 and data pool 20 2022-07-21T01:56:06.207 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:06.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.293+0000 7ffb9fb68700 1 Processor -- start 2022-07-21T01:56:06.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.293+0000 7ffb9fb68700 1 -- start start 2022-07-21T01:56:06.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ffb980b1620 0x7ffb98142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffb9813ee40 0x7ffb981499b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb981426b0 con 0x7ffb98142fd0 2022-07-21T01:56:06.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb98060b50 con 0x7ffb980b1620 2022-07-21T01:56:06.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb980b1a00 con 0x7ffb9813ee40 2022-07-21T01:56:06.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9d904700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ffb980b1620 0x7ffb98142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9d904700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ffb980b1620 0x7ffb98142ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38772/0 (socket says 172.21.15.140:38772) 2022-07-21T01:56:06.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9d904700 1 -- 172.21.15.140:0/4111937333 learned_addr learned my addr 172.21.15.140:0/4111937333 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:06.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9e105700 1 --2- 172.21.15.140:0/4111937333 >> v2:172.21.15.152:3300/0 conn(0x7ffb9813ee40 0x7ffb981499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9d904700 1 -- 172.21.15.140:0/4111937333 >> v2:172.21.15.152:3300/0 conn(0x7ffb9813ee40 msgr2=0x7ffb981499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:06.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9d904700 1 --2- 172.21.15.140:0/4111937333 >> v2:172.21.15.152:3300/0 conn(0x7ffb9813ee40 0x7ffb981499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9d904700 1 -- 172.21.15.140:0/4111937333 >> v1:172.21.15.152:6789/0 conn(0x7ffb98142fd0 legacy=0x7ffb9813e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9d904700 1 -- 172.21.15.140:0/4111937333 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9805f840 con 0x7ffb980b1620 2022-07-21T01:56:06.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9d904700 1 --2- 172.21.15.140:0/4111937333 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffb980b1620 0x7ffb98142ae0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7ffb94002f70 tx=0x7ffb94006eb0).ready entity=mon.0 client_cookie=60b6be9a785d91f6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:06.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9c902700 1 -- 172.21.15.140:0/4111937333 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb9400d610 con 0x7ffb980b1620 2022-07-21T01:56:06.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9c902700 1 -- 172.21.15.140:0/4111937333 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb94000e90 con 0x7ffb980b1620 2022-07-21T01:56:06.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9c902700 1 -- 172.21.15.140:0/4111937333 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb9400d9a0 con 0x7ffb980b1620 2022-07-21T01:56:06.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 -- 172.21.15.140:0/4111937333 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffb980b1620 msgr2=0x7ffb98142ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 --2- 172.21.15.140:0/4111937333 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffb980b1620 0x7ffb98142ae0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7ffb94002f70 tx=0x7ffb94006eb0).stop 2022-07-21T01:56:06.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 -- 172.21.15.140:0/4111937333 shutdown_connections 2022-07-21T01:56:06.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 --2- 172.21.15.140:0/4111937333 >> v2:172.21.15.152:3300/0 conn(0x7ffb9813ee40 0x7ffb981499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 --2- 172.21.15.140:0/4111937333 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffb980b1620 0x7ffb98142ae0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 -- 172.21.15.140:0/4111937333 >> 172.21.15.140:0/4111937333 conn(0x7ffb980b23d0 msgr2=0x7ffb980b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:06.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 -- 172.21.15.140:0/4111937333 shutdown_connections 2022-07-21T01:56:06.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.297+0000 7ffb9fb68700 1 -- 172.21.15.140:0/4111937333 wait complete. 2022-07-21T01:56:06.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9fb68700 1 Processor -- start 2022-07-21T01:56:06.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9fb68700 1 -- start start 2022-07-21T01:56:06.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9fb68700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffb980b1620 0x7ffb98065e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9fb68700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb9813ee40 0x7ffb9805bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9fb68700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffb98142fd0 0x7ffb9808f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9fb68700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9814f160 con 0x7ffb980b1620 2022-07-21T01:56:06.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9fb68700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb98091120 con 0x7ffb9813ee40 2022-07-21T01:56:06.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9fb68700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9814f350 con 0x7ffb98142fd0 2022-07-21T01:56:06.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9d904700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffb980b1620 0x7ffb98065e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9d904700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffb980b1620 0x7ffb98065e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38778/0 (socket says 172.21.15.140:38778) 2022-07-21T01:56:06.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9d904700 1 -- 172.21.15.140:0/697565709 learned_addr learned my addr 172.21.15.140:0/697565709 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:06.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9d103700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb9813ee40 0x7ffb9805bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.301+0000 7ffb9e105700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffb98142fd0 0x7ffb9808f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9d904700 1 -- 172.21.15.140:0/697565709 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffb98142fd0 msgr2=0x7ffb9808f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9d904700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffb98142fd0 0x7ffb9808f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9d904700 1 -- 172.21.15.140:0/697565709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb9813ee40 msgr2=0x7ffb9805bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9d904700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb9813ee40 0x7ffb9805bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9d904700 1 -- 172.21.15.140:0/697565709 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9400b1e0 con 0x7ffb980b1620 2022-07-21T01:56:06.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9e105700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffb98142fd0 0x7ffb9808f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:06.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9d904700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffb980b1620 0x7ffb98065e60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7ffb94002f70 tx=0x7ffb94015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:06.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb86ffd700 1 -- 172.21.15.140:0/697565709 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb94011860 con 0x7ffb980b1620 2022-07-21T01:56:06.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb86ffd700 1 -- 172.21.15.140:0/697565709 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb940119c0 con 0x7ffb980b1620 2022-07-21T01:56:06.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9fb68700 1 -- 172.21.15.140:0/697565709 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9805c3d0 con 0x7ffb980b1620 2022-07-21T01:56:06.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb86ffd700 1 -- 172.21.15.140:0/697565709 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb94011d20 con 0x7ffb980b1620 2022-07-21T01:56:06.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9fb68700 1 -- 172.21.15.140:0/697565709 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9809b3e0 con 0x7ffb980b1620 2022-07-21T01:56:06.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9fb68700 1 -- 172.21.15.140:0/697565709 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb98091290 con 0x7ffb980b1620 2022-07-21T01:56:06.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb86ffd700 1 -- 172.21.15.140:0/697565709 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffb94025030 con 0x7ffb980b1620 2022-07-21T01:56:06.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb86ffd700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffb780539e0 0x7ffb78055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb86ffd700 1 -- 172.21.15.140:0/697565709 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffb9406d140 con 0x7ffb980b1620 2022-07-21T01:56:06.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9d103700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffb780539e0 0x7ffb78055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.305+0000 7ffb9d103700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffb780539e0 0x7ffb78055ea0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7ffb8800d000 tx=0x7ffb88007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:06.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.313+0000 7ffb86ffd700 1 -- 172.21.15.140:0/697565709 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffb98091290 con 0x7ffb980b1620 2022-07-21T01:56:06.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.441+0000 7ffb9fb68700 1 -- 172.21.15.140:0/697565709 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb98091290 con 0x7ffb980b1620 2022-07-21T01:56:06.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.441+0000 7ffb86ffd700 1 -- 172.21.15.140:0/697565709 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16161 (secure 0 0 0) 0x7ffb98091290 con 0x7ffb980b1620 2022-07-21T01:56:06.447 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:06.448 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":102,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:06.173244+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:56:04.557618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-07-21T01:56:05.505410+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:06.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 -- 172.21.15.140:0/697565709 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffb780539e0 msgr2=0x7ffb78055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffb780539e0 0x7ffb78055ea0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7ffb8800d000 tx=0x7ffb88007b90).stop 2022-07-21T01:56:06.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 -- 172.21.15.140:0/697565709 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffb980b1620 msgr2=0x7ffb98065e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffb980b1620 0x7ffb98065e60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7ffb94002f70 tx=0x7ffb94015000).stop 2022-07-21T01:56:06.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 -- 172.21.15.140:0/697565709 shutdown_connections 2022-07-21T01:56:06.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffb780539e0 0x7ffb78055ea0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffb98142fd0 0x7ffb9808f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb9813ee40 0x7ffb9805bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 --2- 172.21.15.140:0/697565709 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffb980b1620 0x7ffb98065e60 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 -- 172.21.15.140:0/697565709 >> 172.21.15.140:0/697565709 conn(0x7ffb980b23d0 msgr2=0x7ffb9813da50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:06.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 -- 172.21.15.140:0/697565709 shutdown_connections 2022-07-21T01:56:06.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.445+0000 7ffb9fb68700 1 -- 172.21.15.140:0/697565709 wait complete. 2022-07-21T01:56:06.465 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:06.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f22107700 1 Processor -- start 2022-07-21T01:56:06.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f22107700 1 -- start start 2022-07-21T01:56:06.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f22107700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2f1c0b5650 0x7f2f1c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f22107700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c147010 con 0x7f2f1c0adff0 2022-07-21T01:56:06.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f22107700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c1472b0 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f22107700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1c146d40 con 0x7f2f1c0b5650 2022-07-21T01:56:06.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f21105700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2f1c0b5650 0x7f2f1c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f20904700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2f1c0adff0 0x7f2f1c147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52932/0 (socket says 172.21.15.140:52932) 2022-07-21T01:56:06.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f21105700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2f1c0b5650 0x7f2f1c0b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38786/0 (socket says 172.21.15.140:38786) 2022-07-21T01:56:06.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f20904700 1 -- 172.21.15.140:0/21821833 learned_addr learned my addr 172.21.15.140:0/21821833 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:06.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f13fff700 1 -- 172.21.15.140:0/21821833 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968449303 0 0) 0x7f2f1c147010 con 0x7f2f1c0adff0 2022-07-21T01:56:06.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f21105700 1 -- 172.21.15.140:0/21821833 >> v1:172.21.15.152:6789/0 conn(0x7f2f1c0af8e0 legacy=0x7f2f1c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f21105700 1 -- 172.21.15.140:0/21821833 >> v1:172.21.15.140:6789/0 conn(0x7f2f1c0adff0 legacy=0x7f2f1c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f21105700 1 -- 172.21.15.140:0/21821833 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c05c4a0 con 0x7f2f1c0b5650 2022-07-21T01:56:06.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f21105700 1 --2- 172.21.15.140:0/21821833 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2f1c0b5650 0x7f2f1c0b5a30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f2f14002f70 tx=0x7f2f14006df0).ready entity=mon.0 client_cookie=7aabfb23773b88d6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:06.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f13fff700 1 -- 172.21.15.140:0/21821833 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f1400d610 con 0x7f2f1c0b5650 2022-07-21T01:56:06.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f13fff700 1 -- 172.21.15.140:0/21821833 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f14000e90 con 0x7f2f1c0b5650 2022-07-21T01:56:06.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.565+0000 7f2f13fff700 1 -- 172.21.15.140:0/21821833 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f1400d960 con 0x7f2f1c0b5650 2022-07-21T01:56:06.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 -- 172.21.15.140:0/21821833 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2f1c0b5650 msgr2=0x7f2f1c0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 --2- 172.21.15.140:0/21821833 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2f1c0b5650 0x7f2f1c0b5a30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f2f14002f70 tx=0x7f2f14006df0).stop 2022-07-21T01:56:06.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 -- 172.21.15.140:0/21821833 shutdown_connections 2022-07-21T01:56:06.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 --2- 172.21.15.140:0/21821833 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2f1c0b5650 0x7f2f1c0b5a30 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 -- 172.21.15.140:0/21821833 >> 172.21.15.140:0/21821833 conn(0x7f2f1c0634e0 msgr2=0x7f2f1c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:06.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 -- 172.21.15.140:0/21821833 shutdown_connections 2022-07-21T01:56:06.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 -- 172.21.15.140:0/21821833 wait complete. 2022-07-21T01:56:06.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 Processor -- start 2022-07-21T01:56:06.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 -- start start 2022-07-21T01:56:06.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f1c0adff0 0x7f2f1c159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2f1c0af8e0 0x7f2f1c13c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.569+0000 7f2f22107700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2f1c0b5650 0x7f2f1c10f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f22107700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0653b0 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f22107700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0641c0 con 0x7f2f1c0adff0 2022-07-21T01:56:06.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f22107700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c064460 con 0x7f2f1c0b5650 2022-07-21T01:56:06.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f20904700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2f1c0af8e0 0x7f2f1c13c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f20904700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2f1c0af8e0 0x7f2f1c13c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38794/0 (socket says 172.21.15.140:38794) 2022-07-21T01:56:06.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f20904700 1 -- 172.21.15.140:0/712925969 learned_addr learned my addr 172.21.15.140:0/712925969 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:06.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f21105700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f1c0adff0 0x7f2f1c159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f21906700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2f1c0b5650 0x7f2f1c10f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f20904700 1 -- 172.21.15.140:0/712925969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2f1c0b5650 msgr2=0x7f2f1c10f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f20904700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2f1c0b5650 0x7f2f1c10f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f20904700 1 -- 172.21.15.140:0/712925969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f1c0adff0 msgr2=0x7f2f1c159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f20904700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f1c0adff0 0x7f2f1c159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f20904700 1 -- 172.21.15.140:0/712925969 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1400b180 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f21906700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2f1c0b5650 0x7f2f1c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:06.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f20904700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2f1c0af8e0 0x7f2f1c13c600 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f2f180013d0 tx=0x7f2f18007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:06.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f127fc700 1 -- 172.21.15.140:0/712925969 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f18005a70 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f127fc700 1 -- 172.21.15.140:0/712925969 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f18005bd0 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f22107700 1 -- 172.21.15.140:0/712925969 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c0650a0 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f127fc700 1 -- 172.21.15.140:0/712925969 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2f180070d0 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f22107700 1 -- 172.21.15.140:0/712925969 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c0676c0 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f22107700 1 -- 172.21.15.140:0/712925969 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c064620 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f127fc700 1 -- 172.21.15.140:0/712925969 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2f1801a060 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f127fc700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2efc053aa0 0x7f2efc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f127fc700 1 -- 172.21.15.140:0/712925969 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2f1806b940 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.573+0000 7f2f21105700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2efc053aa0 0x7f2efc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.577+0000 7f2f21105700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2efc053aa0 0x7f2efc055f60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f2f1401c000 tx=0x7f2f14015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:06.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.581+0000 7f2f127fc700 1 -- 172.21.15.140:0/712925969 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2f18042860 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.705+0000 7f2f22107700 1 -- 172.21.15.140:0/712925969 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f1c0574e0 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f127fc700 1 -- 172.21.15.140:0/712925969 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f2f18018020 con 0x7f2f1c0af8e0 2022-07-21T01:56:06.713 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:06.714 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":102,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:06.173244+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:56:04.557618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-07-21T01:56:05.505410+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:06.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 -- 172.21.15.140:0/712925969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2efc053aa0 msgr2=0x7f2efc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2efc053aa0 0x7f2efc055f60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f2f1401c000 tx=0x7f2f14015000).stop 2022-07-21T01:56:06.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 -- 172.21.15.140:0/712925969 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2f1c0af8e0 msgr2=0x7f2f1c13c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2f1c0af8e0 0x7f2f1c13c600 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f2f180013d0 tx=0x7f2f18007c30).stop 2022-07-21T01:56:06.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 -- 172.21.15.140:0/712925969 shutdown_connections 2022-07-21T01:56:06.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2efc053aa0 0x7f2efc055f60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2f1c0b5650 0x7f2f1c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2f1c0af8e0 0x7f2f1c13c600 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 --2- 172.21.15.140:0/712925969 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f1c0adff0 0x7f2f1c159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 -- 172.21.15.140:0/712925969 >> 172.21.15.140:0/712925969 conn(0x7f2f1c0634e0 msgr2=0x7f2f1c0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:06.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 -- 172.21.15.140:0/712925969 shutdown_connections 2022-07-21T01:56:06.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.709+0000 7f2f22107700 1 -- 172.21.15.140:0/712925969 wait complete. 2022-07-21T01:56:06.726 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:56:06.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.829+0000 7f5e83fff700 1 Processor -- start 2022-07-21T01:56:06.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.829+0000 7f5e83fff700 1 -- start start 2022-07-21T01:56:06.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5e8413eb20 0x7f5e840b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e840ae720 0x7f5e840aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8413f970 con 0x7f5e840b1080 2022-07-21T01:56:06.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8413fbe0 con 0x7f5e8413eb20 2022-07-21T01:56:06.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8413fe80 con 0x7f5e840ae720 2022-07-21T01:56:06.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e827fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5e8413eb20 0x7f5e840b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e827fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5e8413eb20 0x7f5e840b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38802/0 (socket says 172.21.15.140:38802) 2022-07-21T01:56:06.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e827fc700 1 -- 172.21.15.140:0/3123597920 learned_addr learned my addr 172.21.15.140:0/3123597920 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:06.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e837fe700 1 --2- 172.21.15.140:0/3123597920 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e840ae720 0x7f5e840aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e827fc700 1 -- 172.21.15.140:0/3123597920 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e840ae720 msgr2=0x7f5e840aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:06.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e827fc700 1 --2- 172.21.15.140:0/3123597920 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e840ae720 0x7f5e840aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e827fc700 1 -- 172.21.15.140:0/3123597920 >> v1:172.21.15.140:6789/0 conn(0x7f5e840b1080 legacy=0x7f5e8413e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e827fc700 1 -- 172.21.15.140:0/3123597920 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8405bd90 con 0x7f5e8413eb20 2022-07-21T01:56:06.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e827fc700 1 --2- 172.21.15.140:0/3123597920 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e8413eb20 0x7f5e840b1460 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f5e6c0032f0 tx=0x7f5e6c009290).ready entity=mon.0 client_cookie=6424381bfb303b0f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:06.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e81ffb700 1 -- 172.21.15.140:0/3123597920 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e6c00b8a0 con 0x7f5e8413eb20 2022-07-21T01:56:06.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e81ffb700 1 -- 172.21.15.140:0/3123597920 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c00ba00 con 0x7f5e8413eb20 2022-07-21T01:56:06.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e81ffb700 1 -- 172.21.15.140:0/3123597920 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e6c00bb70 con 0x7f5e8413eb20 2022-07-21T01:56:06.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 -- 172.21.15.140:0/3123597920 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e8413eb20 msgr2=0x7f5e840b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 --2- 172.21.15.140:0/3123597920 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e8413eb20 0x7f5e840b1460 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7f5e6c0032f0 tx=0x7f5e6c009290).stop 2022-07-21T01:56:06.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 -- 172.21.15.140:0/3123597920 shutdown_connections 2022-07-21T01:56:06.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 --2- 172.21.15.140:0/3123597920 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e840ae720 0x7f5e840aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 --2- 172.21.15.140:0/3123597920 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e8413eb20 0x7f5e840b1460 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 -- 172.21.15.140:0/3123597920 >> 172.21.15.140:0/3123597920 conn(0x7f5e84063480 msgr2=0x7f5e840b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:06.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 -- 172.21.15.140:0/3123597920 shutdown_connections 2022-07-21T01:56:06.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.833+0000 7f5e83fff700 1 -- 172.21.15.140:0/3123597920 wait complete. 2022-07-21T01:56:06.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e83fff700 1 Processor -- start 2022-07-21T01:56:06.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e83fff700 1 -- start start 2022-07-21T01:56:06.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e840ae720 0x7f5e841369d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e840b1080 0x7f5e8413ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e83fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e8413eb20 0x7f5e8414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e83fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e84150300 con 0x7f5e840b1080 2022-07-21T01:56:06.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e83fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e841534d0 con 0x7f5e840ae720 2022-07-21T01:56:06.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e83fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e84137000 con 0x7f5e8413eb20 2022-07-21T01:56:06.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e827fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e840b1080 0x7f5e8413ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e827fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e840b1080 0x7f5e8413ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38808/0 (socket says 172.21.15.140:38808) 2022-07-21T01:56:06.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e827fc700 1 -- 172.21.15.140:0/3087388019 learned_addr learned my addr 172.21.15.140:0/3087388019 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:06.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e82ffd700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e840ae720 0x7f5e841369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e837fe700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e8413eb20 0x7f5e8414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e827fc700 1 -- 172.21.15.140:0/3087388019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e8413eb20 msgr2=0x7f5e8414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e827fc700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e8413eb20 0x7f5e8414fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e827fc700 1 -- 172.21.15.140:0/3087388019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e840ae720 msgr2=0x7f5e841369d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:06.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e827fc700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e840ae720 0x7f5e841369d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:06.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e827fc700 1 -- 172.21.15.140:0/3087388019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c009530 con 0x7f5e840b1080 2022-07-21T01:56:06.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e827fc700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e840b1080 0x7f5e8413ae70 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f5e6c0032f0 tx=0x7f5e6c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:06.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e7bfff700 1 -- 172.21.15.140:0/3087388019 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e6c021440 con 0x7f5e840b1080 2022-07-21T01:56:06.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e7bfff700 1 -- 172.21.15.140:0/3087388019 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c0215a0 con 0x7f5e840b1080 2022-07-21T01:56:06.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e83fff700 1 -- 172.21.15.140:0/3087388019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e841372a0 con 0x7f5e840b1080 2022-07-21T01:56:06.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e7bfff700 1 -- 172.21.15.140:0/3087388019 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e6c021900 con 0x7f5e840b1080 2022-07-21T01:56:06.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.837+0000 7f5e83fff700 1 -- 172.21.15.140:0/3087388019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e84142c70 con 0x7f5e840b1080 2022-07-21T01:56:06.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.841+0000 7f5e83fff700 1 -- 172.21.15.140:0/3087388019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e840acae0 con 0x7f5e840b1080 2022-07-21T01:56:06.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.841+0000 7f5e7bfff700 1 -- 172.21.15.140:0/3087388019 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e6c027020 con 0x7f5e840b1080 2022-07-21T01:56:06.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.841+0000 7f5e7bfff700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e680539e0 0x7f5e68055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:06.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.841+0000 7f5e7bfff700 1 -- 172.21.15.140:0/3087388019 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e6c06d200 con 0x7f5e840b1080 2022-07-21T01:56:06.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.841+0000 7f5e82ffd700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e680539e0 0x7f5e68055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:06.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.841+0000 7f5e82ffd700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e680539e0 0x7f5e68055ea0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f5e7c00d000 tx=0x7f5e7c007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:06.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:06.849+0000 7f5e7bfff700 1 -- 172.21.15.140:0/3087388019 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e6c044120 con 0x7f5e840b1080 2022-07-21T01:56:07.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.013+0000 7f5e83fff700 1 -- 172.21.15.140:0/3087388019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5e840574e0 con 0x7f5e840b1080 2022-07-21T01:56:07.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.201+0000 7f5e7bfff700 1 -- 172.21.15.140:0/3087388019 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5e6c015020 con 0x7f5e840b1080 2022-07-21T01:56:07.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 -- 172.21.15.140:0/3087388019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e680539e0 msgr2=0x7f5e68055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e680539e0 0x7f5e68055ea0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f5e7c00d000 tx=0x7f5e7c007b90).stop 2022-07-21T01:56:07.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 -- 172.21.15.140:0/3087388019 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e840b1080 msgr2=0x7f5e8413ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e840b1080 0x7f5e8413ae70 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7f5e6c0032f0 tx=0x7f5e6c016000).stop 2022-07-21T01:56:07.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 -- 172.21.15.140:0/3087388019 shutdown_connections 2022-07-21T01:56:07.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e680539e0 0x7f5e68055ea0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e8413eb20 0x7f5e8414fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e840b1080 0x7f5e8413ae70 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 --2- 172.21.15.140:0/3087388019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e840ae720 0x7f5e841369d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 -- 172.21.15.140:0/3087388019 >> 172.21.15.140:0/3087388019 conn(0x7f5e84063480 msgr2=0x7f5e840aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:07.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 -- 172.21.15.140:0/3087388019 shutdown_connections 2022-07-21T01:56:07.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.205+0000 7f5e83fff700 1 -- 172.21.15.140:0/3087388019 wait complete. 2022-07-21T01:56:07.223 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:07.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.329+0000 7f3e03fff700 1 Processor -- start 2022-07-21T01:56:07.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.329+0000 7f3e03fff700 1 -- start start 2022-07-21T01:56:07.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e03fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3e040ac3a0 0x7f3e04143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e03fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e04060980 con 0x7f3e0405fa50 2022-07-21T01:56:07.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e03fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e040b22d0 con 0x7f3e040b1ef0 2022-07-21T01:56:07.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e03fff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e04098810 con 0x7f3e040ac3a0 2022-07-21T01:56:07.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e037fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f3e0405fa50 0x7f3e0413e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:52964/0 (socket says 172.21.15.140:52964) 2022-07-21T01:56:07.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e037fe700 1 -- 172.21.15.140:0/395900932 learned_addr learned my addr 172.21.15.140:0/395900932 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:07.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e027fc700 1 --2- 172.21.15.140:0/395900932 >> v2:172.21.15.152:3300/0 conn(0x7f3e040ac3a0 0x7f3e04143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e027fc700 1 -- 172.21.15.140:0/395900932 >> v1:172.21.15.152:6789/0 conn(0x7f3e040b1ef0 legacy=0x7f3e04142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e027fc700 1 -- 172.21.15.140:0/395900932 >> v1:172.21.15.140:6789/0 conn(0x7f3e0405fa50 legacy=0x7f3e0413e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e027fc700 1 -- 172.21.15.140:0/395900932 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e04060bb0 con 0x7f3e040ac3a0 2022-07-21T01:56:07.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e01ffb700 1 -- 172.21.15.140:0/395900932 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202830696 0 0) 0x7f3e04060980 con 0x7f3e0405fa50 2022-07-21T01:56:07.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e027fc700 1 --2- 172.21.15.140:0/395900932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e040ac3a0 0x7f3e04143690 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f3df8002f70 tx=0x7f3df8006df0).ready entity=mon.1 client_cookie=bb56a87cff47787c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:07.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e01ffb700 1 -- 172.21.15.140:0/395900932 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3df800d610 con 0x7f3e040ac3a0 2022-07-21T01:56:07.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e01ffb700 1 -- 172.21.15.140:0/395900932 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df8000e90 con 0x7f3e040ac3a0 2022-07-21T01:56:07.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.333+0000 7f3e01ffb700 1 -- 172.21.15.140:0/395900932 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3df800d960 con 0x7f3e040ac3a0 2022-07-21T01:56:07.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 -- 172.21.15.140:0/395900932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e040ac3a0 msgr2=0x7f3e04143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 --2- 172.21.15.140:0/395900932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e040ac3a0 0x7f3e04143690 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f3df8002f70 tx=0x7f3df8006df0).stop 2022-07-21T01:56:07.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 -- 172.21.15.140:0/395900932 shutdown_connections 2022-07-21T01:56:07.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 --2- 172.21.15.140:0/395900932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e040ac3a0 0x7f3e04143690 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 -- 172.21.15.140:0/395900932 >> 172.21.15.140:0/395900932 conn(0x7f3e040b2ca0 msgr2=0x7f3e040b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:07.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 -- 172.21.15.140:0/395900932 shutdown_connections 2022-07-21T01:56:07.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 -- 172.21.15.140:0/395900932 wait complete. 2022-07-21T01:56:07.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 Processor -- start 2022-07-21T01:56:07.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 -- start start 2022-07-21T01:56:07.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e0405fa50 0x7f3e04064d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e040ac3a0 0x7f3e041541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e040b1ef0 0x7f3e041429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04099a50 con 0x7f3e040b1ef0 2022-07-21T01:56:07.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e041546a0 con 0x7f3e0405fa50 2022-07-21T01:56:07.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e03fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e041537d0 con 0x7f3e040ac3a0 2022-07-21T01:56:07.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e037fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e040b1ef0 0x7f3e041429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e037fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e040b1ef0 0x7f3e041429a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38824/0 (socket says 172.21.15.140:38824) 2022-07-21T01:56:07.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e037fe700 1 -- 172.21.15.140:0/1605334238 learned_addr learned my addr 172.21.15.140:0/1605334238 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:07.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e027fc700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e040ac3a0 0x7f3e041541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e02ffd700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e0405fa50 0x7f3e04064d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.337+0000 7f3e037fe700 1 -- 172.21.15.140:0/1605334238 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e040ac3a0 msgr2=0x7f3e041541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3e037fe700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e040ac3a0 0x7f3e041541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3e037fe700 1 -- 172.21.15.140:0/1605334238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e0405fa50 msgr2=0x7f3e04064d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3e037fe700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e0405fa50 0x7f3e04064d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3e037fe700 1 -- 172.21.15.140:0/1605334238 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df800b180 con 0x7f3e040b1ef0 2022-07-21T01:56:07.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3e037fe700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e040b1ef0 0x7f3e041429a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7f3dfc001250 tx=0x7f3dfc007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:07.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3df3fff700 1 -- 172.21.15.140:0/1605334238 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dfc005ba0 con 0x7f3e040b1ef0 2022-07-21T01:56:07.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3e03fff700 1 -- 172.21.15.140:0/1605334238 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e04153a70 con 0x7f3e040b1ef0 2022-07-21T01:56:07.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3df3fff700 1 -- 172.21.15.140:0/1605334238 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc005d00 con 0x7f3e040b1ef0 2022-07-21T01:56:07.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3df3fff700 1 -- 172.21.15.140:0/1605334238 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dfc004a90 con 0x7f3e040b1ef0 2022-07-21T01:56:07.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3e03fff700 1 -- 172.21.15.140:0/1605334238 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e04142e90 con 0x7f3e040b1ef0 2022-07-21T01:56:07.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3e03fff700 1 -- 172.21.15.140:0/1605334238 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e04062270 con 0x7f3e040b1ef0 2022-07-21T01:56:07.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3df3fff700 1 -- 172.21.15.140:0/1605334238 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3dfc004bf0 con 0x7f3e040b1ef0 2022-07-21T01:56:07.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3df3fff700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3de00539e0 0x7f3de0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3df3fff700 1 -- 172.21.15.140:0/1605334238 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3dfc06baf0 con 0x7f3e040b1ef0 2022-07-21T01:56:07.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.341+0000 7f3e02ffd700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3de00539e0 0x7f3de0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.345+0000 7f3e02ffd700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3de00539e0 0x7f3de0055ea0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f3df400d000 tx=0x7f3df4007be0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:07.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.349+0000 7f3df3fff700 1 -- 172.21.15.140:0/1605334238 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3dfc042970 con 0x7f3e040b1ef0 2022-07-21T01:56:07.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.513+0000 7f3e03fff700 1 -- 172.21.15.140:0/1605334238 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e040574e0 con 0x7f3e040b1ef0 2022-07-21T01:56:07.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.513+0000 7f3df3fff700 1 -- 172.21.15.140:0/1605334238 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f3dfc019030 con 0x7f3e040b1ef0 2022-07-21T01:56:07.518 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:07.518 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":47,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/3024179649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":3024179649},{"type":"v1","addr":"172.21.15.140:6835","nonce":3024179649}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:56:06.173419+0000","modified":"2022-07-21T01:56:07.189859+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/727644549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":727644549},{"type":"v1","addr":"172.21.15.152:6833","nonce":727644549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-07-21T01:56:07.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.513+0000 7f3e03fff700 1 -- 172.21.15.140:0/1605334238 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3de00539e0 msgr2=0x7f3de0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.513+0000 7f3e03fff700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3de00539e0 0x7f3de0055ea0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f3df400d000 tx=0x7f3df4007be0).stop 2022-07-21T01:56:07.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.513+0000 7f3e03fff700 1 -- 172.21.15.140:0/1605334238 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e040b1ef0 msgr2=0x7f3e041429a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.513+0000 7f3e03fff700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e040b1ef0 0x7f3e041429a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7f3dfc001250 tx=0x7f3dfc007dc0).stop 2022-07-21T01:56:07.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.513+0000 7f3e03fff700 1 -- 172.21.15.140:0/1605334238 shutdown_connections 2022-07-21T01:56:07.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.517+0000 7f3e03fff700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3de00539e0 0x7f3de0055ea0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.517+0000 7f3e03fff700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3e040b1ef0 0x7f3e041429a0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.517+0000 7f3e03fff700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3e040ac3a0 0x7f3e041541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.517+0000 7f3e03fff700 1 --2- 172.21.15.140:0/1605334238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e0405fa50 0x7f3e04064d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.517+0000 7f3e03fff700 1 -- 172.21.15.140:0/1605334238 >> 172.21.15.140:0/1605334238 conn(0x7f3e040b2ca0 msgr2=0x7f3e0413ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:07.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.517+0000 7f3e03fff700 1 -- 172.21.15.140:0/1605334238 shutdown_connections 2022-07-21T01:56:07.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.517+0000 7f3e03fff700 1 -- 172.21.15.140:0/1605334238 wait complete. 2022-07-21T01:56:07.524 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 47 2022-07-21T01:56:07.531 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:07.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.625+0000 7fbea4899700 1 Processor -- start 2022-07-21T01:56:07.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.625+0000 7fbea4899700 1 -- start start 2022-07-21T01:56:07.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.625+0000 7fbea4899700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbea00b5650 0x7fbea00b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.625+0000 7fbea4899700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbea00b1200 0x7fbea005c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbea005bd90 0x7fbea005c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea0142970 con 0x7fbea00b1200 2022-07-21T01:56:07.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea00aea30 con 0x7fbea00b5650 2022-07-21T01:56:07.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00aec60 con 0x7fbea005bd90 2022-07-21T01:56:07.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe97fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbea00b1200 0x7fbea005c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe97fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbea00b1200 0x7fbea005c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38830/0 (socket says 172.21.15.140:38830) 2022-07-21T01:56:07.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe97fff700 1 -- 172.21.15.140:0/1682189990 learned_addr learned my addr 172.21.15.140:0/1682189990 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:07.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe9f7fe700 1 --2- 172.21.15.140:0/1682189990 >> v2:172.21.15.152:3300/0 conn(0x7fbea00b5650 0x7fbea00b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe9ffff700 1 --2- 172.21.15.140:0/1682189990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbea005bd90 0x7fbea005c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe97fff700 1 -- 172.21.15.140:0/1682189990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbea005bd90 msgr2=0x7fbea005c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:07.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe97fff700 1 --2- 172.21.15.140:0/1682189990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbea005bd90 0x7fbea005c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe97fff700 1 -- 172.21.15.140:0/1682189990 >> v2:172.21.15.152:3300/0 conn(0x7fbea00b5650 msgr2=0x7fbea00b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:07.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe97fff700 1 --2- 172.21.15.140:0/1682189990 >> v2:172.21.15.152:3300/0 conn(0x7fbea00b5650 0x7fbea00b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe97fff700 1 -- 172.21.15.140:0/1682189990 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00af180 con 0x7fbea00b1200 2022-07-21T01:56:07.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe97fff700 1 --2- 172.21.15.140:0/1682189990 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbea00b1200 0x7fbea005c4f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7fbe88002f70 tx=0x7fbe88006eb0).ready entity=mon.0 client_cookie=7cae27bba8ae787b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:07.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe9effd700 1 -- 172.21.15.140:0/1682189990 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe8800d570 con 0x7fbea00b1200 2022-07-21T01:56:07.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe9effd700 1 -- 172.21.15.140:0/1682189990 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe88000e90 con 0x7fbea00b1200 2022-07-21T01:56:07.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbe9effd700 1 -- 172.21.15.140:0/1682189990 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe8800d900 con 0x7fbea00b1200 2022-07-21T01:56:07.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 -- 172.21.15.140:0/1682189990 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbea00b1200 msgr2=0x7fbea005c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 --2- 172.21.15.140:0/1682189990 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbea00b1200 0x7fbea005c4f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7fbe88002f70 tx=0x7fbe88006eb0).stop 2022-07-21T01:56:07.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 -- 172.21.15.140:0/1682189990 shutdown_connections 2022-07-21T01:56:07.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 --2- 172.21.15.140:0/1682189990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbea005bd90 0x7fbea005c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 --2- 172.21.15.140:0/1682189990 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbea00b1200 0x7fbea005c4f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 --2- 172.21.15.140:0/1682189990 >> v2:172.21.15.152:3300/0 conn(0x7fbea00b5650 0x7fbea00b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 -- 172.21.15.140:0/1682189990 >> 172.21.15.140:0/1682189990 conn(0x7fbea00634e0 msgr2=0x7fbea00b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:07.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 -- 172.21.15.140:0/1682189990 shutdown_connections 2022-07-21T01:56:07.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 -- 172.21.15.140:0/1682189990 wait complete. 2022-07-21T01:56:07.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.629+0000 7fbea4899700 1 Processor -- start 2022-07-21T01:56:07.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbea4899700 1 -- start start 2022-07-21T01:56:07.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbea4899700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbea005bd90 0x7fbea0145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbea4899700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbea00b1200 0x7fbea01521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbea4899700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbea00b5650 0x7fbea01526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbea4899700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea0064830 con 0x7fbea00b1200 2022-07-21T01:56:07.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbea4899700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea0064590 con 0x7fbea00b5650 2022-07-21T01:56:07.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbea4899700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea0141a10 con 0x7fbea005bd90 2022-07-21T01:56:07.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe97fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbea00b1200 0x7fbea01521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe9f7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbea005bd90 0x7fbea0145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe97fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbea00b1200 0x7fbea01521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38836/0 (socket says 172.21.15.140:38836) 2022-07-21T01:56:07.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe97fff700 1 -- 172.21.15.140:0/3980528006 learned_addr learned my addr 172.21.15.140:0/3980528006 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:07.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe9f7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbea005bd90 0x7fbea0145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:35564/0 (socket says 172.21.15.140:35564) 2022-07-21T01:56:07.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe9ffff700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbea00b5650 0x7fbea01526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe97fff700 1 -- 172.21.15.140:0/3980528006 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbea005bd90 msgr2=0x7fbea0145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe97fff700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbea005bd90 0x7fbea0145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe97fff700 1 -- 172.21.15.140:0/3980528006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbea00b5650 msgr2=0x7fbea01526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe97fff700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbea00b5650 0x7fbea01526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe97fff700 1 -- 172.21.15.140:0/3980528006 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8800b1e0 con 0x7fbea00b1200 2022-07-21T01:56:07.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe97fff700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbea00b1200 0x7fbea01521f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7fbe88002f70 tx=0x7fbe88014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:07.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe9d7fa700 1 -- 172.21.15.140:0/3980528006 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe88013de0 con 0x7fbea00b1200 2022-07-21T01:56:07.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe9d7fa700 1 -- 172.21.15.140:0/3980528006 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe880068b0 con 0x7fbea00b1200 2022-07-21T01:56:07.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbe9d7fa700 1 -- 172.21.15.140:0/3980528006 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe8801e5f0 con 0x7fbea00b1200 2022-07-21T01:56:07.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbea4899700 1 -- 172.21.15.140:0/3980528006 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea0152bd0 con 0x7fbea00b1200 2022-07-21T01:56:07.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbea4899700 1 -- 172.21.15.140:0/3980528006 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea01555c0 con 0x7fbea00b1200 2022-07-21T01:56:07.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.633+0000 7fbea4899700 1 -- 172.21.15.140:0/3980528006 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea00574e0 con 0x7fbea00b1200 2022-07-21T01:56:07.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.637+0000 7fbe9d7fa700 1 -- 172.21.15.140:0/3980528006 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbe8801c070 con 0x7fbea00b1200 2022-07-21T01:56:07.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.637+0000 7fbe9d7fa700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbe84053a70 0x7fbe84055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.637+0000 7fbe9d7fa700 1 -- 172.21.15.140:0/3980528006 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbe8806c910 con 0x7fbea00b1200 2022-07-21T01:56:07.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.637+0000 7fbe9f7fe700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbe84053a70 0x7fbe84055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.637+0000 7fbe9f7fe700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbe84053a70 0x7fbe84055f30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fbe9000b000 tx=0x7fbe90006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:07.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.645+0000 7fbe9d7fa700 1 -- 172.21.15.140:0/3980528006 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbe88043720 con 0x7fbea00b1200 2022-07-21T01:56:07.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.769+0000 7fbea4899700 1 -- 172.21.15.140:0/3980528006 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbea0067ea0 con 0x7fbea00b1200 2022-07-21T01:56:07.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbe9d7fa700 1 -- 172.21.15.140:0/3980528006 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16148 (secure 0 0 0) 0x7fbe88023020 con 0x7fbea00b1200 2022-07-21T01:56:07.776 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:07.777 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":103,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:07.189453+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:56:04.557618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-07-21T01:56:05.505410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:07.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 -- 172.21.15.140:0/3980528006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbe84053a70 msgr2=0x7fbe84055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbe84053a70 0x7fbe84055f30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fbe9000b000 tx=0x7fbe90006df0).stop 2022-07-21T01:56:07.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 -- 172.21.15.140:0/3980528006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbea00b1200 msgr2=0x7fbea01521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbea00b1200 0x7fbea01521f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7fbe88002f70 tx=0x7fbe88014000).stop 2022-07-21T01:56:07.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 -- 172.21.15.140:0/3980528006 shutdown_connections 2022-07-21T01:56:07.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbe84053a70 0x7fbe84055f30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbea00b5650 0x7fbea01526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbea00b1200 0x7fbea01521f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 --2- 172.21.15.140:0/3980528006 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbea005bd90 0x7fbea0145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 -- 172.21.15.140:0/3980528006 >> 172.21.15.140:0/3980528006 conn(0x7fbea00634e0 msgr2=0x7fbea0142170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:07.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 -- 172.21.15.140:0/3980528006 shutdown_connections 2022-07-21T01:56:07.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.773+0000 7fbea4899700 1 -- 172.21.15.140:0/3980528006 wait complete. 2022-07-21T01:56:07.790 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:56:07.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c970ff700 1 Processor -- start 2022-07-21T01:56:07.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c970ff700 1 -- start start 2022-07-21T01:56:07.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c970ff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c900b6800 0x7f4c900aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c970ff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4c900ad0e0 0x7f4c901418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c970ff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c900ae280 con 0x7f4c900b3cb0 2022-07-21T01:56:07.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c970ff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c900b6be0 con 0x7f4c900ad0e0 2022-07-21T01:56:07.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c970ff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900ae4b0 con 0x7f4c900b6800 2022-07-21T01:56:07.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c968fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4c900ad0e0 0x7f4c901418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c968fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4c900ad0e0 0x7f4c901418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38846/0 (socket says 172.21.15.140:38846) 2022-07-21T01:56:07.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c968fe700 1 -- 172.21.15.140:0/258734158 learned_addr learned my addr 172.21.15.140:0/258734158 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:07.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c960fd700 1 --2- 172.21.15.140:0/258734158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c900b6800 0x7f4c900aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c968fe700 1 -- 172.21.15.140:0/258734158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c900b6800 msgr2=0x7f4c900aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:07.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c968fe700 1 --2- 172.21.15.140:0/258734158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c900b6800 0x7f4c900aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c968fe700 1 -- 172.21.15.140:0/258734158 >> v1:172.21.15.152:6789/0 conn(0x7f4c900b3cb0 legacy=0x7f4c90145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c968fe700 1 -- 172.21.15.140:0/258734158 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900ae750 con 0x7f4c900ad0e0 2022-07-21T01:56:07.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c968fe700 1 --2- 172.21.15.140:0/258734158 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c900ad0e0 0x7f4c901418f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f4c880032f0 tx=0x7f4c88009290).ready entity=mon.0 client_cookie=a4d4aa45f89dc080 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:07.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c950fb700 1 -- 172.21.15.140:0/258734158 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c8800b8a0 con 0x7f4c900ad0e0 2022-07-21T01:56:07.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c950fb700 1 -- 172.21.15.140:0/258734158 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8800ba00 con 0x7f4c900ad0e0 2022-07-21T01:56:07.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.901+0000 7f4c950fb700 1 -- 172.21.15.140:0/258734158 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c8800bb70 con 0x7f4c900ad0e0 2022-07-21T01:56:07.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.905+0000 7f4c970ff700 1 -- 172.21.15.140:0/258734158 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c900ad0e0 msgr2=0x7f4c901418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.905+0000 7f4c970ff700 1 --2- 172.21.15.140:0/258734158 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c900ad0e0 0x7f4c901418f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f4c880032f0 tx=0x7f4c88009290).stop 2022-07-21T01:56:07.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.905+0000 7f4c970ff700 1 -- 172.21.15.140:0/258734158 shutdown_connections 2022-07-21T01:56:07.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.905+0000 7f4c970ff700 1 --2- 172.21.15.140:0/258734158 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c900ad0e0 0x7f4c901418f0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.905+0000 7f4c970ff700 1 --2- 172.21.15.140:0/258734158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c900b6800 0x7f4c900aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.905+0000 7f4c970ff700 1 -- 172.21.15.140:0/258734158 >> 172.21.15.140:0/258734158 conn(0x7f4c900b9dd0 msgr2=0x7f4c900b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:07.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.905+0000 7f4c970ff700 1 -- 172.21.15.140:0/258734158 shutdown_connections 2022-07-21T01:56:07.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.905+0000 7f4c970ff700 1 -- 172.21.15.140:0/258734158 wait complete. 2022-07-21T01:56:07.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.905+0000 7f4c970ff700 1 Processor -- start 2022-07-21T01:56:07.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.905+0000 7f4c970ff700 1 -- start start 2022-07-21T01:56:07.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.905+0000 7f4c970ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c900ad0e0 0x7f4c900b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c970ff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c900b3cb0 0x7f4c90151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c970ff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c900b6800 0x7f4c90115520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c970ff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9013fb90 con 0x7f4c900b3cb0 2022-07-21T01:56:07.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c970ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c901148a0 con 0x7f4c900ad0e0 2022-07-21T01:56:07.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c970ff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c901529b0 con 0x7f4c900b6800 2022-07-21T01:56:07.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c958fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c900b3cb0 0x7f4c90151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c958fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c900b3cb0 0x7f4c90151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38850/0 (socket says 172.21.15.140:38850) 2022-07-21T01:56:07.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c958fc700 1 -- 172.21.15.140:0/2234293585 learned_addr learned my addr 172.21.15.140:0/2234293585 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:07.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c960fd700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c900ad0e0 0x7f4c900b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c968fe700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c900b6800 0x7f4c90115520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c958fc700 1 -- 172.21.15.140:0/2234293585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c900b6800 msgr2=0x7f4c90115520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c958fc700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c900b6800 0x7f4c90115520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c958fc700 1 -- 172.21.15.140:0/2234293585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c900ad0e0 msgr2=0x7f4c900b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:07.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c958fc700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c900ad0e0 0x7f4c900b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:07.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c958fc700 1 -- 172.21.15.140:0/2234293585 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c88009530 con 0x7f4c900b3cb0 2022-07-21T01:56:07.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c958fc700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c900b3cb0 0x7f4c90151f70 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f4c90066600 tx=0x7f4c8c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:07.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c7f7fe700 1 -- 172.21.15.140:0/2234293585 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c8c012400 con 0x7f4c900b3cb0 2022-07-21T01:56:07.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c7f7fe700 1 -- 172.21.15.140:0/2234293585 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c012560 con 0x7f4c900b3cb0 2022-07-21T01:56:07.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c970ff700 1 -- 172.21.15.140:0/2234293585 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c90152c50 con 0x7f4c900b3cb0 2022-07-21T01:56:07.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c7f7fe700 1 -- 172.21.15.140:0/2234293585 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c8c0128c0 con 0x7f4c900b3cb0 2022-07-21T01:56:07.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c970ff700 1 -- 172.21.15.140:0/2234293585 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c900b1b00 con 0x7f4c900b3cb0 2022-07-21T01:56:07.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.909+0000 7f4c970ff700 1 -- 172.21.15.140:0/2234293585 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c900b5530 con 0x7f4c900b3cb0 2022-07-21T01:56:07.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.913+0000 7f4c7f7fe700 1 -- 172.21.15.140:0/2234293585 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4c8c019030 con 0x7f4c900b3cb0 2022-07-21T01:56:07.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.913+0000 7f4c7f7fe700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 0x7f4c70055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:07.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.913+0000 7f4c7f7fe700 1 -- 172.21.15.140:0/2234293585 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c8c06b340 con 0x7f4c900b3cb0 2022-07-21T01:56:07.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.913+0000 7f4c960fd700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 0x7f4c70055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:07.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.913+0000 7f4c960fd700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 0x7f4c70055fc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f4c8000b000 tx=0x7f4c80006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:07.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:07.917+0000 7f4c7f7fe700 1 -- 172.21.15.140:0/2234293585 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4c8c0420f0 con 0x7f4c900b3cb0 2022-07-21T01:56:08.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.081+0000 7f4c970ff700 1 -- 172.21.15.140:0/2234293585 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4c901527a0 con 0x7f4c900b3cb0 2022-07-21T01:56:08.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.089+0000 7f4c7f7fe700 1 -- 172.21.15.140:0/2234293585 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v36) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4c8c045710 con 0x7f4c900b3cb0 2022-07-21T01:56:08.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 -- 172.21.15.140:0/2234293585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 msgr2=0x7f4c70055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 0x7f4c70055fc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f4c8000b000 tx=0x7f4c80006df0).stop 2022-07-21T01:56:08.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 -- 172.21.15.140:0/2234293585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c900b3cb0 msgr2=0x7f4c90151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c900b3cb0 0x7f4c90151f70 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f4c90066600 tx=0x7f4c8c007c50).stop 2022-07-21T01:56:08.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 -- 172.21.15.140:0/2234293585 shutdown_connections 2022-07-21T01:56:08.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 0x7f4c70055fc0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c900b6800 0x7f4c90115520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c900b3cb0 0x7f4c90151f70 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 --2- 172.21.15.140:0/2234293585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c900ad0e0 0x7f4c900b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 -- 172.21.15.140:0/2234293585 >> 172.21.15.140:0/2234293585 conn(0x7f4c900b9dd0 msgr2=0x7f4c90141ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:08.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 -- 172.21.15.140:0/2234293585 shutdown_connections 2022-07-21T01:56:08.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.093+0000 7f4c970ff700 1 -- 172.21.15.140:0/2234293585 wait complete. 2022-07-21T01:56:08.103 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:56:08.109 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:56:08.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.213+0000 7fc18cb08700 1 Processor -- start 2022-07-21T01:56:08.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.213+0000 7fc18cb08700 1 -- start start 2022-07-21T01:56:08.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18cb08700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc1880af2c0 0x7fc1880af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18cb08700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc188147120 con 0x7fc1880b5830 2022-07-21T01:56:08.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18cb08700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1880b0130 con 0x7fc18805cf40 2022-07-21T01:56:08.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18cb08700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1881462d0 con 0x7fc1880af2c0 2022-07-21T01:56:08.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc185d9b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc1880b5830 0x7fc188142c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53004/0 (socket says 172.21.15.140:53004) 2022-07-21T01:56:08.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc185d9b700 1 -- 172.21.15.140:0/2190526470 learned_addr learned my addr 172.21.15.140:0/2190526470 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:08.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc186d9d700 1 --2- 172.21.15.140:0/2190526470 >> v2:172.21.15.152:3300/0 conn(0x7fc1880af2c0 0x7fc1880af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18559a700 1 -- 172.21.15.140:0/2190526470 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723115927 0 0) 0x7fc188147120 con 0x7fc1880b5830 2022-07-21T01:56:08.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18559a700 1 -- 172.21.15.140:0/2190526470 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc168003260 con 0x7fc1880b5830 2022-07-21T01:56:08.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc186d9d700 1 -- 172.21.15.140:0/2190526470 >> v1:172.21.15.152:6789/0 conn(0x7fc18805cf40 legacy=0x7fc188146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc186d9d700 1 -- 172.21.15.140:0/2190526470 >> v1:172.21.15.140:6789/0 conn(0x7fc1880b5830 legacy=0x7fc188142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc186d9d700 1 -- 172.21.15.140:0/2190526470 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc188146570 con 0x7fc1880af2c0 2022-07-21T01:56:08.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc186d9d700 1 --2- 172.21.15.140:0/2190526470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1880af2c0 0x7fc1880af6f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fc1800032f0 tx=0x7fc1800091d0).ready entity=mon.1 client_cookie=f9b3a86f1d251af server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:08.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18559a700 1 -- 172.21.15.140:0/2190526470 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc18000b6c0 con 0x7fc1880af2c0 2022-07-21T01:56:08.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18559a700 1 -- 172.21.15.140:0/2190526470 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18000b820 con 0x7fc1880af2c0 2022-07-21T01:56:08.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18559a700 1 -- 172.21.15.140:0/2190526470 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc18000b990 con 0x7fc1880af2c0 2022-07-21T01:56:08.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18cb08700 1 -- 172.21.15.140:0/2190526470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1880af2c0 msgr2=0x7fc1880af6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18cb08700 1 --2- 172.21.15.140:0/2190526470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1880af2c0 0x7fc1880af6f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fc1800032f0 tx=0x7fc1800091d0).stop 2022-07-21T01:56:08.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18cb08700 1 -- 172.21.15.140:0/2190526470 shutdown_connections 2022-07-21T01:56:08.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18cb08700 1 --2- 172.21.15.140:0/2190526470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1880af2c0 0x7fc1880af6f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18cb08700 1 -- 172.21.15.140:0/2190526470 >> 172.21.15.140:0/2190526470 conn(0x7fc1880b69e0 msgr2=0x7fc188066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:08.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18cb08700 1 -- 172.21.15.140:0/2190526470 shutdown_connections 2022-07-21T01:56:08.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.217+0000 7fc18cb08700 1 -- 172.21.15.140:0/2190526470 wait complete. 2022-07-21T01:56:08.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.221+0000 7fc18cb08700 1 Processor -- start 2022-07-21T01:56:08.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.221+0000 7fc18cb08700 1 -- start start 2022-07-21T01:56:08.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.221+0000 7fc18cb08700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc18805cf40 0x7fc1880b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.221+0000 7fc18cb08700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1880af2c0 0x7fc18813c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.221+0000 7fc18cb08700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1880b5830 0x7fc18813cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.221+0000 7fc18cb08700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc188145e40 con 0x7fc18805cf40 2022-07-21T01:56:08.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.221+0000 7fc18cb08700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18805fed0 con 0x7fc1880b5830 2022-07-21T01:56:08.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.221+0000 7fc18cb08700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc188060170 con 0x7fc1880af2c0 2022-07-21T01:56:08.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.221+0000 7fc18659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc18805cf40 0x7fc1880b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.221+0000 7fc18659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc18805cf40 0x7fc1880b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38862/0 (socket says 172.21.15.140:38862) 2022-07-21T01:56:08.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.221+0000 7fc18659c700 1 -- 172.21.15.140:0/3095360713 learned_addr learned my addr 172.21.15.140:0/3095360713 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:08.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc186d9d700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1880b5830 0x7fc18813cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc185d9b700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1880af2c0 0x7fc18813c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc18659c700 1 -- 172.21.15.140:0/3095360713 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1880af2c0 msgr2=0x7fc18813c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc18659c700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1880af2c0 0x7fc18813c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc18659c700 1 -- 172.21.15.140:0/3095360713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1880b5830 msgr2=0x7fc18813cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc18659c700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1880b5830 0x7fc18813cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc18659c700 1 -- 172.21.15.140:0/3095360713 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc180009470 con 0x7fc18805cf40 2022-07-21T01:56:08.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc18659c700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc18805cf40 0x7fc1880b0bf0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7fc18806ad40 tx=0x7fc178007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:08.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc17f7fe700 1 -- 172.21.15.140:0/3095360713 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc17800ce30 con 0x7fc18805cf40 2022-07-21T01:56:08.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc17f7fe700 1 -- 172.21.15.140:0/3095360713 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17800f470 con 0x7fc18805cf40 2022-07-21T01:56:08.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc18cb08700 1 -- 172.21.15.140:0/3095360713 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc188060410 con 0x7fc18805cf40 2022-07-21T01:56:08.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc17f7fe700 1 -- 172.21.15.140:0/3095360713 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc17800f7d0 con 0x7fc18805cf40 2022-07-21T01:56:08.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc18cb08700 1 -- 172.21.15.140:0/3095360713 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18809d680 con 0x7fc18805cf40 2022-07-21T01:56:08.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc18cb08700 1 -- 172.21.15.140:0/3095360713 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc188143340 con 0x7fc18805cf40 2022-07-21T01:56:08.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc17f7fe700 1 -- 172.21.15.140:0/3095360713 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc178019030 con 0x7fc18805cf40 2022-07-21T01:56:08.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc17f7fe700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc16c053b00 0x7fc16c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc185d9b700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc16c053b00 0x7fc16c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.225+0000 7fc17f7fe700 1 -- 172.21.15.140:0/3095360713 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc17806b310 con 0x7fc18805cf40 2022-07-21T01:56:08.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.229+0000 7fc185d9b700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc16c053b00 0x7fc16c055fc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fc17000b000 tx=0x7fc170009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:08.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.233+0000 7fc17f7fe700 1 -- 172.21.15.140:0/3095360713 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc178042230 con 0x7fc18805cf40 2022-07-21T01:56:08.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.401+0000 7fc18cb08700 1 -- 172.21.15.140:0/3095360713 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fc1880574e0 con 0x7fc18805cf40 2022-07-21T01:56:08.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.409+0000 7fc17f7fe700 1 -- 172.21.15.140:0/3095360713 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v37) v1 ==== 184+0+0 (secure 0 0 0) 0x7fc178045850 con 0x7fc18805cf40 2022-07-21T01:56:08.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.409+0000 7fc18cb08700 1 -- 172.21.15.140:0/3095360713 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc16c053b00 msgr2=0x7fc16c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.409+0000 7fc18cb08700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc16c053b00 0x7fc16c055fc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fc17000b000 tx=0x7fc170009040).stop 2022-07-21T01:56:08.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.409+0000 7fc18cb08700 1 -- 172.21.15.140:0/3095360713 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc18805cf40 msgr2=0x7fc1880b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.409+0000 7fc18cb08700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc18805cf40 0x7fc1880b0bf0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7fc18806ad40 tx=0x7fc178007c50).stop 2022-07-21T01:56:08.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.413+0000 7fc18cb08700 1 -- 172.21.15.140:0/3095360713 shutdown_connections 2022-07-21T01:56:08.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.413+0000 7fc18cb08700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc16c053b00 0x7fc16c055fc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.413+0000 7fc18cb08700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1880b5830 0x7fc18813cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.413+0000 7fc18cb08700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1880af2c0 0x7fc18813c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.413+0000 7fc18cb08700 1 --2- 172.21.15.140:0/3095360713 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc18805cf40 0x7fc1880b0bf0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.413+0000 7fc18cb08700 1 -- 172.21.15.140:0/3095360713 >> 172.21.15.140:0/3095360713 conn(0x7fc1880b69e0 msgr2=0x7fc1880b8590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:08.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.413+0000 7fc18cb08700 1 -- 172.21.15.140:0/3095360713 shutdown_connections 2022-07-21T01:56:08.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.413+0000 7fc18cb08700 1 -- 172.21.15.140:0/3095360713 wait complete. 2022-07-21T01:56:08.419 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:56:08.431 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:56:08.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.537+0000 7faf82f5e700 1 Processor -- start 2022-07-21T01:56:08.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.537+0000 7faf82f5e700 1 -- start start 2022-07-21T01:56:08.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.537+0000 7faf82f5e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faf7404ae80 0x7faf7404b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.537+0000 7faf82f5e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faf74041c90 0x7faf740d7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.537+0000 7faf82f5e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf740d7570 con 0x7faf740456e0 2022-07-21T01:56:08.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.537+0000 7faf82f5e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf740d4670 con 0x7faf7404ae80 2022-07-21T01:56:08.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.537+0000 7faf82f5e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf740423a0 con 0x7faf74041c90 2022-07-21T01:56:08.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.537+0000 7faf81f5c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faf7404ae80 0x7faf7404b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.537+0000 7faf81f5c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faf7404ae80 0x7faf7404b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38872/0 (socket says 172.21.15.140:38872) 2022-07-21T01:56:08.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.537+0000 7faf81f5c700 1 -- 172.21.15.140:0/2818340996 learned_addr learned my addr 172.21.15.140:0/2818340996 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:08.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.537+0000 7faf8275d700 1 --2- 172.21.15.140:0/2818340996 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faf74041c90 0x7faf740d7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf81f5c700 1 -- 172.21.15.140:0/2818340996 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faf74041c90 msgr2=0x7faf740d7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:08.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf81f5c700 1 --2- 172.21.15.140:0/2818340996 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faf74041c90 0x7faf740d7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf81f5c700 1 -- 172.21.15.140:0/2818340996 >> v1:172.21.15.152:6789/0 conn(0x7faf740456e0 legacy=0x7faf740d3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf81f5c700 1 -- 172.21.15.140:0/2818340996 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf740d77a0 con 0x7faf7404ae80 2022-07-21T01:56:08.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf81f5c700 1 --2- 172.21.15.140:0/2818340996 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faf7404ae80 0x7faf7404b260 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7faf6c002f70 tx=0x7faf6c006eb0).ready entity=mon.0 client_cookie=cc9adb5ee30868d3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:08.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf80f5a700 1 -- 172.21.15.140:0/2818340996 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf6c00d570 con 0x7faf7404ae80 2022-07-21T01:56:08.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf80f5a700 1 -- 172.21.15.140:0/2818340996 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c000e90 con 0x7faf7404ae80 2022-07-21T01:56:08.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf80f5a700 1 -- 172.21.15.140:0/2818340996 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf6c00d900 con 0x7faf7404ae80 2022-07-21T01:56:08.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf82f5e700 1 -- 172.21.15.140:0/2818340996 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faf7404ae80 msgr2=0x7faf7404b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf82f5e700 1 --2- 172.21.15.140:0/2818340996 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faf7404ae80 0x7faf7404b260 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7faf6c002f70 tx=0x7faf6c006eb0).stop 2022-07-21T01:56:08.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf82f5e700 1 -- 172.21.15.140:0/2818340996 shutdown_connections 2022-07-21T01:56:08.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf82f5e700 1 --2- 172.21.15.140:0/2818340996 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faf74041c90 0x7faf740d7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf82f5e700 1 --2- 172.21.15.140:0/2818340996 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faf7404ae80 0x7faf7404b260 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf82f5e700 1 -- 172.21.15.140:0/2818340996 >> 172.21.15.140:0/2818340996 conn(0x7faf74011570 msgr2=0x7faf7404a770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:08.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf82f5e700 1 -- 172.21.15.140:0/2818340996 shutdown_connections 2022-07-21T01:56:08.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf82f5e700 1 -- 172.21.15.140:0/2818340996 wait complete. 2022-07-21T01:56:08.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf82f5e700 1 Processor -- start 2022-07-21T01:56:08.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.541+0000 7faf82f5e700 1 -- start start 2022-07-21T01:56:08.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf82f5e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faf74041c90 0x7faf740d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf82f5e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faf740456e0 0x7faf740d65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf82f5e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf7404ae80 0x7faf74040ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf82f5e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740d2060 con 0x7faf740456e0 2022-07-21T01:56:08.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf82f5e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740d3980 con 0x7faf7404ae80 2022-07-21T01:56:08.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf82f5e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf740d3c20 con 0x7faf74041c90 2022-07-21T01:56:08.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf8175b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faf740456e0 0x7faf740d65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf8175b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faf740456e0 0x7faf740d65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38878/0 (socket says 172.21.15.140:38878) 2022-07-21T01:56:08.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf8175b700 1 -- 172.21.15.140:0/2825292983 learned_addr learned my addr 172.21.15.140:0/2825292983 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:08.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf8275d700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf7404ae80 0x7faf74040ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf81f5c700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faf74041c90 0x7faf740d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf8175b700 1 -- 172.21.15.140:0/2825292983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faf74041c90 msgr2=0x7faf740d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf8175b700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faf74041c90 0x7faf740d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf8175b700 1 -- 172.21.15.140:0/2825292983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf7404ae80 msgr2=0x7faf74040ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf8175b700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf7404ae80 0x7faf74040ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf8175b700 1 -- 172.21.15.140:0/2825292983 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c00b1e0 con 0x7faf740456e0 2022-07-21T01:56:08.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf81f5c700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faf74041c90 0x7faf740d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:08.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf8175b700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faf740456e0 0x7faf740d65a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7faf74011a90 tx=0x7faf78007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:08.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf6affd700 1 -- 172.21.15.140:0/2825292983 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf78012400 con 0x7faf740456e0 2022-07-21T01:56:08.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf6affd700 1 -- 172.21.15.140:0/2825292983 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf78012560 con 0x7faf740456e0 2022-07-21T01:56:08.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf82f5e700 1 -- 172.21.15.140:0/2825292983 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf740d3ec0 con 0x7faf740456e0 2022-07-21T01:56:08.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf6affd700 1 -- 172.21.15.140:0/2825292983 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faf780128c0 con 0x7faf740456e0 2022-07-21T01:56:08.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf82f5e700 1 -- 172.21.15.140:0/2825292983 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7403da30 con 0x7faf740456e0 2022-07-21T01:56:08.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf82f5e700 1 -- 172.21.15.140:0/2825292983 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7403f360 con 0x7faf740456e0 2022-07-21T01:56:08.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf6affd700 1 -- 172.21.15.140:0/2825292983 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faf78019030 con 0x7faf740456e0 2022-07-21T01:56:08.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf6affd700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf5c0539e0 0x7faf5c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.545+0000 7faf6affd700 1 -- 172.21.15.140:0/2825292983 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faf7806b3f0 con 0x7faf740456e0 2022-07-21T01:56:08.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.549+0000 7faf81f5c700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf5c0539e0 0x7faf5c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.549+0000 7faf81f5c700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf5c0539e0 0x7faf5c055ea0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7faf6c01a000 tx=0x7faf6c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:08.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.553+0000 7faf6affd700 1 -- 172.21.15.140:0/2825292983 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faf7403f360 con 0x7faf740456e0 2022-07-21T01:56:08.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.713+0000 7faf82f5e700 1 -- 172.21.15.140:0/2825292983 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7faf7403f360 con 0x7faf740456e0 2022-07-21T01:56:08.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.721+0000 7faf6affd700 1 -- 172.21.15.140:0/2825292983 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v38) v1 ==== 184+0+0 (secure 0 0 0) 0x7faf7403f360 con 0x7faf740456e0 2022-07-21T01:56:08.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 -- 172.21.15.140:0/2825292983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf5c0539e0 msgr2=0x7faf5c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf5c0539e0 0x7faf5c055ea0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7faf6c01a000 tx=0x7faf6c015000).stop 2022-07-21T01:56:08.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 -- 172.21.15.140:0/2825292983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faf740456e0 msgr2=0x7faf740d65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faf740456e0 0x7faf740d65a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7faf74011a90 tx=0x7faf78007c50).stop 2022-07-21T01:56:08.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 -- 172.21.15.140:0/2825292983 shutdown_connections 2022-07-21T01:56:08.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faf5c0539e0 0x7faf5c055ea0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf7404ae80 0x7faf74040ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faf740456e0 0x7faf740d65a0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 --2- 172.21.15.140:0/2825292983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faf74041c90 0x7faf740d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 -- 172.21.15.140:0/2825292983 >> 172.21.15.140:0/2825292983 conn(0x7faf74011570 msgr2=0x7faf74047960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:08.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 -- 172.21.15.140:0/2825292983 shutdown_connections 2022-07-21T01:56:08.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.725+0000 7faf82f5e700 1 -- 172.21.15.140:0/2825292983 wait complete. 2022-07-21T01:56:08.734 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:56:08.742 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:08.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.837+0000 7f82e878b700 1 Processor -- start 2022-07-21T01:56:08.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.837+0000 7f82e878b700 1 -- start start 2022-07-21T01:56:08.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e878b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f82e005c3f0 0x7f82e014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e878b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e005d480 con 0x7f82e00acef0 2022-07-21T01:56:08.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e878b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e00b21e0 con 0x7f82e00b1e00 2022-07-21T01:56:08.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e878b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e005cf40 con 0x7f82e005c3f0 2022-07-21T01:56:08.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e5d26700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f82e005c3f0 0x7f82e014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e5d26700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f82e005c3f0 0x7f82e014a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38884/0 (socket says 172.21.15.140:38884) 2022-07-21T01:56:08.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e5d26700 1 -- 172.21.15.140:0/2825231674 learned_addr learned my addr 172.21.15.140:0/2825231674 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:08.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e5d26700 1 -- 172.21.15.140:0/2825231674 >> v1:172.21.15.152:6789/0 conn(0x7f82e00b1e00 legacy=0x7f82e00ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:08.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e5d26700 1 -- 172.21.15.140:0/2825231674 >> v1:172.21.15.140:6789/0 conn(0x7f82e00acef0 legacy=0x7f82e0146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e5d26700 1 -- 172.21.15.140:0/2825231674 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e005d1e0 con 0x7f82e005c3f0 2022-07-21T01:56:08.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e5d26700 1 --2- 172.21.15.140:0/2825231674 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82e005c3f0 0x7f82e014a6d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f82e005f3e0 tx=0x7f82d0009290).ready entity=mon.0 client_cookie=477b01f59b908162 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:08.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e5525700 1 -- 172.21.15.140:0/2825231674 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82d000b860 con 0x7f82e005c3f0 2022-07-21T01:56:08.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e5525700 1 -- 172.21.15.140:0/2825231674 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d000b9c0 con 0x7f82e005c3f0 2022-07-21T01:56:08.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e5525700 1 -- 172.21.15.140:0/2825231674 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82d000bb30 con 0x7f82e005c3f0 2022-07-21T01:56:08.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e878b700 1 -- 172.21.15.140:0/2825231674 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82e005c3f0 msgr2=0x7f82e014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e878b700 1 --2- 172.21.15.140:0/2825231674 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82e005c3f0 0x7f82e014a6d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f82e005f3e0 tx=0x7f82d0009290).stop 2022-07-21T01:56:08.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e878b700 1 -- 172.21.15.140:0/2825231674 shutdown_connections 2022-07-21T01:56:08.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e878b700 1 --2- 172.21.15.140:0/2825231674 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82e005c3f0 0x7f82e014a6d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e878b700 1 -- 172.21.15.140:0/2825231674 >> 172.21.15.140:0/2825231674 conn(0x7f82e00611c0 msgr2=0x7f82e0062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:08.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e878b700 1 -- 172.21.15.140:0/2825231674 shutdown_connections 2022-07-21T01:56:08.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.841+0000 7f82e878b700 1 -- 172.21.15.140:0/2825231674 wait complete. 2022-07-21T01:56:08.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e878b700 1 Processor -- start 2022-07-21T01:56:08.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e878b700 1 -- start start 2022-07-21T01:56:08.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e878b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82e00acef0 0x7f82e009e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e878b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82e00b1e00 0x7f82e015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e878b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82e0143bd0 0x7f82e015a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e878b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e00b5ae0 con 0x7f82e00b1e00 2022-07-21T01:56:08.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e878b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e00b5440 con 0x7f82e00acef0 2022-07-21T01:56:08.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e878b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e00b5600 con 0x7f82e0143bd0 2022-07-21T01:56:08.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e5d26700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82e00b1e00 0x7f82e015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e5d26700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82e00b1e00 0x7f82e015a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38892/0 (socket says 172.21.15.140:38892) 2022-07-21T01:56:08.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e5d26700 1 -- 172.21.15.140:0/3811249067 learned_addr learned my addr 172.21.15.140:0/3811249067 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:08.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e6527700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82e00acef0 0x7f82e009e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e6d28700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82e0143bd0 0x7f82e015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e5d26700 1 -- 172.21.15.140:0/3811249067 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82e0143bd0 msgr2=0x7f82e015a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e5d26700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82e0143bd0 0x7f82e015a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e5d26700 1 -- 172.21.15.140:0/3811249067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82e00acef0 msgr2=0x7f82e009e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:08.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e5d26700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82e00acef0 0x7f82e009e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:08.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e5d26700 1 -- 172.21.15.140:0/3811249067 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d0009530 con 0x7f82e00b1e00 2022-07-21T01:56:08.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e6d28700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82e0143bd0 0x7f82e015a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:08.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e5d26700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82e00b1e00 0x7f82e015a170 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f82e00b3860 tx=0x7f82d0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:08.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82df7fe700 1 -- 172.21.15.140:0/3811249067 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82d0006b00 con 0x7f82e00b1e00 2022-07-21T01:56:08.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82df7fe700 1 -- 172.21.15.140:0/3811249067 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d000bda0 con 0x7f82e00b1e00 2022-07-21T01:56:08.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82e878b700 1 -- 172.21.15.140:0/3811249067 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e00b3f50 con 0x7f82e00b1e00 2022-07-21T01:56:08.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.845+0000 7f82df7fe700 1 -- 172.21.15.140:0/3811249067 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82d001f5e0 con 0x7f82e00b1e00 2022-07-21T01:56:08.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.849+0000 7f82e878b700 1 -- 172.21.15.140:0/3811249067 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e015acf0 con 0x7f82e00b1e00 2022-07-21T01:56:08.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.849+0000 7f82e878b700 1 -- 172.21.15.140:0/3811249067 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e00b5740 con 0x7f82e00b1e00 2022-07-21T01:56:08.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.849+0000 7f82df7fe700 1 -- 172.21.15.140:0/3811249067 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f82d0026030 con 0x7f82e00b1e00 2022-07-21T01:56:08.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.849+0000 7f82df7fe700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f82c0053b00 0x7f82c0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:08.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.849+0000 7f82df7fe700 1 -- 172.21.15.140:0/3811249067 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f82d006cb10 con 0x7f82e00b1e00 2022-07-21T01:56:08.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.849+0000 7f82e6527700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f82c0053b00 0x7f82c0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:08.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.849+0000 7f82e6527700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f82c0053b00 0x7f82c0055fc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f82d400d000 tx=0x7f82d4007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:08.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:08.853+0000 7f82df7fe700 1 -- 172.21.15.140:0/3811249067 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f82e00b5740 con 0x7f82e00b1e00 2022-07-21T01:56:09.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.013+0000 7f82e878b700 1 -- 172.21.15.140:0/3811249067 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82e00b5740 con 0x7f82e00b1e00 2022-07-21T01:56:09.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.013+0000 7f82df7fe700 1 -- 172.21.15.140:0/3811249067 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f82e00b5740 con 0x7f82e00b1e00 2022-07-21T01:56:09.019 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:09.020 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":47,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/3024179649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":3024179649},{"type":"v1","addr":"172.21.15.140:6835","nonce":3024179649}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:56:06.173419+0000","modified":"2022-07-21T01:56:07.189859+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/727644549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":727644549},{"type":"v1","addr":"172.21.15.152:6833","nonce":727644549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-07-21T01:56:09.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 -- 172.21.15.140:0/3811249067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f82c0053b00 msgr2=0x7f82c0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f82c0053b00 0x7f82c0055fc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f82d400d000 tx=0x7f82d4007b70).stop 2022-07-21T01:56:09.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 -- 172.21.15.140:0/3811249067 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82e00b1e00 msgr2=0x7f82e015a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82e00b1e00 0x7f82e015a170 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f82e00b3860 tx=0x7f82d0016000).stop 2022-07-21T01:56:09.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 -- 172.21.15.140:0/3811249067 shutdown_connections 2022-07-21T01:56:09.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f82c0053b00 0x7f82c0055fc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82e0143bd0 0x7f82e015a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82e00b1e00 0x7f82e015a170 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 --2- 172.21.15.140:0/3811249067 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82e00acef0 0x7f82e009e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 -- 172.21.15.140:0/3811249067 >> 172.21.15.140:0/3811249067 conn(0x7f82e005c3f0 msgr2=0x7f82e0145630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:09.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 -- 172.21.15.140:0/3811249067 shutdown_connections 2022-07-21T01:56:09.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.017+0000 7f82e878b700 1 -- 172.21.15.140:0/3811249067 wait complete. 2022-07-21T01:56:09.024 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 47 2022-07-21T01:56:09.235 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:56:06.173419+0000', 'modified': '2022-07-21T01:56:07.189859+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7335}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7335': {'gid': 7335, 'name': 'b', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/727644549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 727644549}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 727644549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:56:09.235 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:56:09.236 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:56:09.354 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":7335,"want_state":"up:active","state":"up:active","rank_uptime":3.168005145,"mdsmap_epoch":47,"osdmap_epoch":103,"osdmap_epoch_barrier":102,"uptime":7.5353286910000001} 2022-07-21T01:56:09.355 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 7335, "mdsmap_epoch": 47, "osdmap_epoch": 103, "osdmap_epoch_barrier": 102, "rank_uptime": 3.168005145, "state": "up:active", "uptime": 7.535328691, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:56:09.356 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:09.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.461+0000 7f973c541700 1 Processor -- start 2022-07-21T01:56:09.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.461+0000 7f973c541700 1 -- start start 2022-07-21T01:56:09.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f97340afa10 0x7f97340b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f97340b3de0 0x7f97341452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97340b42a0 con 0x7f97340608b0 2022-07-21T01:56:09.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9734060c90 con 0x7f97340b3de0 2022-07-21T01:56:09.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97340af410 con 0x7f97340afa10 2022-07-21T01:56:09.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973aade700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f97340b3de0 0x7f97341452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973aade700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f97340b3de0 0x7f97341452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38902/0 (socket says 172.21.15.140:38902) 2022-07-21T01:56:09.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973aade700 1 -- 172.21.15.140:0/2727148439 learned_addr learned my addr 172.21.15.140:0/2727148439 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:09.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f9739adc700 1 --2- 172.21.15.140:0/2727148439 >> v2:172.21.15.152:3300/0 conn(0x7f97340afa10 0x7f97340b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973aade700 1 -- 172.21.15.140:0/2727148439 >> v2:172.21.15.152:3300/0 conn(0x7f97340afa10 msgr2=0x7f97340b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:09.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973aade700 1 --2- 172.21.15.140:0/2727148439 >> v2:172.21.15.152:3300/0 conn(0x7f97340afa10 0x7f97340b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973aade700 1 -- 172.21.15.140:0/2727148439 >> v1:172.21.15.152:6789/0 conn(0x7f97340608b0 legacy=0x7f973405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973aade700 1 -- 172.21.15.140:0/2727148439 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97340af6b0 con 0x7f97340b3de0 2022-07-21T01:56:09.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973aade700 1 --2- 172.21.15.140:0/2727148439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97340b3de0 0x7f97341452d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f9730002f70 tx=0x7f9730006eb0).ready entity=mon.0 client_cookie=fea4a42f5b9f9b91 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:09.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f97392db700 1 -- 172.21.15.140:0/2727148439 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f973000d5c0 con 0x7f97340b3de0 2022-07-21T01:56:09.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f97392db700 1 -- 172.21.15.140:0/2727148439 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9730000e90 con 0x7f97340b3de0 2022-07-21T01:56:09.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f97392db700 1 -- 172.21.15.140:0/2727148439 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f973000d950 con 0x7f97340b3de0 2022-07-21T01:56:09.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 -- 172.21.15.140:0/2727148439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97340b3de0 msgr2=0x7f97341452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 --2- 172.21.15.140:0/2727148439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97340b3de0 0x7f97341452d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f9730002f70 tx=0x7f9730006eb0).stop 2022-07-21T01:56:09.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 -- 172.21.15.140:0/2727148439 shutdown_connections 2022-07-21T01:56:09.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 --2- 172.21.15.140:0/2727148439 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97340b3de0 0x7f97341452d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 --2- 172.21.15.140:0/2727148439 >> v2:172.21.15.152:3300/0 conn(0x7f97340afa10 0x7f97340b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 -- 172.21.15.140:0/2727148439 >> 172.21.15.140:0/2727148439 conn(0x7f97340b5ee0 msgr2=0x7f97340b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:09.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 -- 172.21.15.140:0/2727148439 shutdown_connections 2022-07-21T01:56:09.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.465+0000 7f973c541700 1 -- 172.21.15.140:0/2727148439 wait complete. 2022-07-21T01:56:09.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973c541700 1 Processor -- start 2022-07-21T01:56:09.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973c541700 1 -- start start 2022-07-21T01:56:09.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973c541700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97340608b0 0x7f97340b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973c541700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97340afa10 0x7f973409d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973c541700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97340b3de0 0x7f9734143280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973c541700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973409dcb0 con 0x7f97340608b0 2022-07-21T01:56:09.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973c541700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9734152080 con 0x7f97340afa10 2022-07-21T01:56:09.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973c541700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9734152320 con 0x7f97340b3de0 2022-07-21T01:56:09.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973a2dd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97340608b0 0x7f97340b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f9739adc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97340afa10 0x7f973409d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973a2dd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97340608b0 0x7f97340b75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38904/0 (socket says 172.21.15.140:38904) 2022-07-21T01:56:09.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973a2dd700 1 -- 172.21.15.140:0/2986387378 learned_addr learned my addr 172.21.15.140:0/2986387378 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:09.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973aade700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97340b3de0 0x7f9734143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973a2dd700 1 -- 172.21.15.140:0/2986387378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97340b3de0 msgr2=0x7f9734143280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973a2dd700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97340b3de0 0x7f9734143280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973a2dd700 1 -- 172.21.15.140:0/2986387378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97340afa10 msgr2=0x7f973409d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973a2dd700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97340afa10 0x7f973409d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973a2dd700 1 -- 172.21.15.140:0/2986387378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973000b1e0 con 0x7f97340608b0 2022-07-21T01:56:09.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973a2dd700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97340608b0 0x7f97340b75b0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f973406ad80 tx=0x7f972c0014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:09.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f97237fe700 1 -- 172.21.15.140:0/2986387378 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f972c00e4a0 con 0x7f97340608b0 2022-07-21T01:56:09.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f97237fe700 1 -- 172.21.15.140:0/2986387378 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c00e600 con 0x7f97340608b0 2022-07-21T01:56:09.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973c541700 1 -- 172.21.15.140:0/2986387378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9734151db0 con 0x7f97340608b0 2022-07-21T01:56:09.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f97237fe700 1 -- 172.21.15.140:0/2986387378 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f972c00e960 con 0x7f97340608b0 2022-07-21T01:56:09.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.469+0000 7f973c541700 1 -- 172.21.15.140:0/2986387378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973409e190 con 0x7f97340608b0 2022-07-21T01:56:09.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.473+0000 7f973c541700 1 -- 172.21.15.140:0/2986387378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97340574e0 con 0x7f97340608b0 2022-07-21T01:56:09.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.473+0000 7f97237fe700 1 -- 172.21.15.140:0/2986387378 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f972c019030 con 0x7f97340608b0 2022-07-21T01:56:09.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.473+0000 7f97237fe700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9714053a70 0x7f9714055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.473+0000 7f97237fe700 1 -- 172.21.15.140:0/2986387378 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f972c06b650 con 0x7f97340608b0 2022-07-21T01:56:09.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.473+0000 7f9739adc700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9714053a70 0x7f9714055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.473+0000 7f9739adc700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9714053a70 0x7f9714055f30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f972400b000 tx=0x7f9724006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:09.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.481+0000 7f97237fe700 1 -- 172.21.15.140:0/2986387378 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f972c042470 con 0x7f97340608b0 2022-07-21T01:56:09.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.605+0000 7f973c541700 1 -- 172.21.15.140:0/2986387378 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9734069a60 con 0x7f97340608b0 2022-07-21T01:56:09.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.609+0000 7f97237fe700 1 -- 172.21.15.140:0/2986387378 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f972c045a90 con 0x7f97340608b0 2022-07-21T01:56:09.614 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:09.614 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":103,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:07.189453+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:56:04.557618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-07-21T01:56:05.505410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:09.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 -- 172.21.15.140:0/2986387378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9714053a70 msgr2=0x7f9714055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9714053a70 0x7f9714055f30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f972400b000 tx=0x7f9724006df0).stop 2022-07-21T01:56:09.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 -- 172.21.15.140:0/2986387378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97340608b0 msgr2=0x7f97340b75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97340608b0 0x7f97340b75b0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7f973406ad80 tx=0x7f972c0014a0).stop 2022-07-21T01:56:09.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 -- 172.21.15.140:0/2986387378 shutdown_connections 2022-07-21T01:56:09.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9714053a70 0x7f9714055f30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97340b3de0 0x7f9734143280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97340afa10 0x7f973409d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 --2- 172.21.15.140:0/2986387378 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97340608b0 0x7f97340b75b0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 -- 172.21.15.140:0/2986387378 >> 172.21.15.140:0/2986387378 conn(0x7f97340b5ee0 msgr2=0x7f97340aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:09.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 -- 172.21.15.140:0/2986387378 shutdown_connections 2022-07-21T01:56:09.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.613+0000 7f973c541700 1 -- 172.21.15.140:0/2986387378 wait complete. 2022-07-21T01:56:09.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:56:04.557618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:56:05.505410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:56:09.631 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:56:09.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 Processor -- start 2022-07-21T01:56:09.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 -- start start 2022-07-21T01:56:09.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fab8c0af940 0x7fab8c0b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab8c147ab0 0x7fab8c0b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab8c05cc70 con 0x7fab8c0b59f0 2022-07-21T01:56:09.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab8c0b5dd0 con 0x7fab8c0af940 2022-07-21T01:56:09.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c05c760 con 0x7fab8c147ab0 2022-07-21T01:56:09.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab83fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fab8c0af940 0x7fab8c0b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab83fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fab8c0af940 0x7fab8c0b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38914/0 (socket says 172.21.15.140:38914) 2022-07-21T01:56:09.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab83fff700 1 -- 172.21.15.140:0/2101876167 learned_addr learned my addr 172.21.15.140:0/2101876167 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:09.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8b7fe700 1 --2- 172.21.15.140:0/2101876167 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab8c147ab0 0x7fab8c0b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab83fff700 1 -- 172.21.15.140:0/2101876167 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab8c147ab0 msgr2=0x7fab8c0b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:09.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab83fff700 1 --2- 172.21.15.140:0/2101876167 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab8c147ab0 0x7fab8c0b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab83fff700 1 -- 172.21.15.140:0/2101876167 >> v1:172.21.15.140:6789/0 conn(0x7fab8c0b59f0 legacy=0x7fab8c1473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab83fff700 1 -- 172.21.15.140:0/2101876167 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c05c9d0 con 0x7fab8c0af940 2022-07-21T01:56:09.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab83fff700 1 --2- 172.21.15.140:0/2101876167 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab8c0af940 0x7fab8c0b0060 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7fab74002f70 tx=0x7fab74006eb0).ready entity=mon.0 client_cookie=ea3af020ddee0ef0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:09.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8a7fc700 1 -- 172.21.15.140:0/2101876167 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab7400d5c0 con 0x7fab8c0af940 2022-07-21T01:56:09.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8a7fc700 1 -- 172.21.15.140:0/2101876167 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab74000e90 con 0x7fab8c0af940 2022-07-21T01:56:09.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8a7fc700 1 -- 172.21.15.140:0/2101876167 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab7400d950 con 0x7fab8c0af940 2022-07-21T01:56:09.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 -- 172.21.15.140:0/2101876167 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab8c0af940 msgr2=0x7fab8c0b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 --2- 172.21.15.140:0/2101876167 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab8c0af940 0x7fab8c0b0060 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7fab74002f70 tx=0x7fab74006eb0).stop 2022-07-21T01:56:09.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 -- 172.21.15.140:0/2101876167 shutdown_connections 2022-07-21T01:56:09.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 --2- 172.21.15.140:0/2101876167 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab8c147ab0 0x7fab8c0b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 --2- 172.21.15.140:0/2101876167 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab8c0af940 0x7fab8c0b0060 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 -- 172.21.15.140:0/2101876167 >> 172.21.15.140:0/2101876167 conn(0x7fab8c063480 msgr2=0x7fab8c0b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:09.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 -- 172.21.15.140:0/2101876167 shutdown_connections 2022-07-21T01:56:09.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.741+0000 7fab8bfff700 1 -- 172.21.15.140:0/2101876167 wait complete. 2022-07-21T01:56:09.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8bfff700 1 Processor -- start 2022-07-21T01:56:09.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8bfff700 1 -- start start 2022-07-21T01:56:09.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab8c0af940 0x7fab8c066e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab8c0b59f0 0x7fab8c067370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c147ab0 0x7fab8c064d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8bfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c063ba0 con 0x7fab8c0b59f0 2022-07-21T01:56:09.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8bfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c067860 con 0x7fab8c147ab0 2022-07-21T01:56:09.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8c1570d0 con 0x7fab8c0af940 2022-07-21T01:56:09.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab83fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab8c0b59f0 0x7fab8c067370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab83fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab8c0b59f0 0x7fab8c067370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38920/0 (socket says 172.21.15.140:38920) 2022-07-21T01:56:09.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab83fff700 1 -- 172.21.15.140:0/1769619394 learned_addr learned my addr 172.21.15.140:0/1769619394 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:09.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8affd700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab8c0af940 0x7fab8c066e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8b7fe700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c147ab0 0x7fab8c064d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab83fff700 1 -- 172.21.15.140:0/1769619394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab8c0af940 msgr2=0x7fab8c066e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab83fff700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab8c0af940 0x7fab8c066e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab83fff700 1 -- 172.21.15.140:0/1769619394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c147ab0 msgr2=0x7fab8c064d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab83fff700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c147ab0 0x7fab8c064d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab83fff700 1 -- 172.21.15.140:0/1769619394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7400b1e0 con 0x7fab8c0b59f0 2022-07-21T01:56:09.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8affd700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab8c0af940 0x7fab8c066e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:09.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8b7fe700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c147ab0 0x7fab8c064d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:09.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab83fff700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab8c0b59f0 0x7fab8c067370 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7fab8c064a60 tx=0x7fab74015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:09.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab88ff9700 1 -- 172.21.15.140:0/1769619394 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab74006b40 con 0x7fab8c0b59f0 2022-07-21T01:56:09.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8bfff700 1 -- 172.21.15.140:0/1769619394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8c157370 con 0x7fab8c0b59f0 2022-07-21T01:56:09.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab88ff9700 1 -- 172.21.15.140:0/1769619394 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7400d940 con 0x7fab8c0b59f0 2022-07-21T01:56:09.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab88ff9700 1 -- 172.21.15.140:0/1769619394 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab74011e50 con 0x7fab8c0b59f0 2022-07-21T01:56:09.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.745+0000 7fab8bfff700 1 -- 172.21.15.140:0/1769619394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8c1420e0 con 0x7fab8c0b59f0 2022-07-21T01:56:09.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.749+0000 7fab8bfff700 1 -- 172.21.15.140:0/1769619394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8c146f00 con 0x7fab8c0b59f0 2022-07-21T01:56:09.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.749+0000 7fab88ff9700 1 -- 172.21.15.140:0/1769619394 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fab74025030 con 0x7fab8c0b59f0 2022-07-21T01:56:09.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.749+0000 7fab88ff9700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fab700539e0 0x7fab70055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:09.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.749+0000 7fab88ff9700 1 -- 172.21.15.140:0/1769619394 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fab7406d350 con 0x7fab8c0b59f0 2022-07-21T01:56:09.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.749+0000 7fab8affd700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fab700539e0 0x7fab70055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:09.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.749+0000 7fab8affd700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fab700539e0 0x7fab70055ea0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fab7c00b000 tx=0x7fab7c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:09.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.757+0000 7fab88ff9700 1 -- 172.21.15.140:0/1769619394 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fab740440e0 con 0x7fab8c0b59f0 2022-07-21T01:56:09.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.881+0000 7fab8bfff700 1 -- 172.21.15.140:0/1769619394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fab8c0574e0 con 0x7fab8c0b59f0 2022-07-21T01:56:09.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.881+0000 7fab88ff9700 1 -- 172.21.15.140:0/1769619394 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v103) v1 ==== 110+0+10 (secure 0 0 0) 0x7fab74023020 con 0x7fab8c0b59f0 2022-07-21T01:56:09.888 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:56:09.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 -- 172.21.15.140:0/1769619394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fab700539e0 msgr2=0x7fab70055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fab700539e0 0x7fab70055ea0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fab7c00b000 tx=0x7fab7c009040).stop 2022-07-21T01:56:09.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 -- 172.21.15.140:0/1769619394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab8c0b59f0 msgr2=0x7fab8c067370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:09.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab8c0b59f0 0x7fab8c067370 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7fab8c064a60 tx=0x7fab74015000).stop 2022-07-21T01:56:09.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 -- 172.21.15.140:0/1769619394 shutdown_connections 2022-07-21T01:56:09.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fab700539e0 0x7fab70055ea0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fab8c147ab0 0x7fab8c064d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fab8c0b59f0 0x7fab8c067370 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 --2- 172.21.15.140:0/1769619394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fab8c0af940 0x7fab8c066e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:09.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 -- 172.21.15.140:0/1769619394 >> 172.21.15.140:0/1769619394 conn(0x7fab8c063480 msgr2=0x7fab8c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:09.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 -- 172.21.15.140:0/1769619394 shutdown_connections 2022-07-21T01:56:09.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:09.885+0000 7fab8bfff700 1 -- 172.21.15.140:0/1769619394 wait complete. 2022-07-21T01:56:09.900 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:56:10.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 Processor -- start 2022-07-21T01:56:10.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 -- start start 2022-07-21T01:56:10.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f539c05d0c0 0x7f539c0b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f539c0adf70 con 0x7f539c0b59d0 2022-07-21T01:56:10.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f539c0b5db0 con 0x7f539c0af920 2022-07-21T01:56:10.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539c146300 con 0x7f539c05d0c0 2022-07-21T01:56:10.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f539bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f539c0b59d0 0x7f539c0ae3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53072/0 (socket says 172.21.15.140:53072) 2022-07-21T01:56:10.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f539bfff700 1 -- 172.21.15.140:0/4034891919 learned_addr learned my addr 172.21.15.140:0/4034891919 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:10.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a09be700 1 --2- 172.21.15.140:0/4034891919 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f539c05d0c0 0x7f539c0b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f539affd700 1 -- 172.21.15.140:0/4034891919 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918990789 0 0) 0x7f539c0b5db0 con 0x7f539c0af920 2022-07-21T01:56:10.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f539affd700 1 -- 172.21.15.140:0/4034891919 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f537c003260 con 0x7f539c0af920 2022-07-21T01:56:10.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f539affd700 1 -- 172.21.15.140:0/4034891919 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110316970 0 0) 0x7f539c0adf70 con 0x7f539c0b59d0 2022-07-21T01:56:10.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f539affd700 1 -- 172.21.15.140:0/4034891919 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f539c0b5db0 con 0x7f539c0b59d0 2022-07-21T01:56:10.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a09be700 1 -- 172.21.15.140:0/4034891919 >> v1:172.21.15.152:6789/0 conn(0x7f539c0af920 legacy=0x7f539c141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a09be700 1 -- 172.21.15.140:0/4034891919 >> v1:172.21.15.140:6789/0 conn(0x7f539c0b59d0 legacy=0x7f539c0ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a09be700 1 -- 172.21.15.140:0/4034891919 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c146060 con 0x7f539c05d0c0 2022-07-21T01:56:10.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a09be700 1 --2- 172.21.15.140:0/4034891919 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f539c05d0c0 0x7f539c0b0040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f53940032f0 tx=0x7f53940091d0).ready entity=mon.2 client_cookie=2cae1520ff22afab server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f539affd700 1 -- 172.21.15.140:0/4034891919 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f539400b640 con 0x7f539c05d0c0 2022-07-21T01:56:10.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f539affd700 1 -- 172.21.15.140:0/4034891919 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400b7a0 con 0x7f539c05d0c0 2022-07-21T01:56:10.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f539affd700 1 -- 172.21.15.140:0/4034891919 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f539400b910 con 0x7f539c05d0c0 2022-07-21T01:56:10.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 -- 172.21.15.140:0/4034891919 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f539c05d0c0 msgr2=0x7f539c0b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 --2- 172.21.15.140:0/4034891919 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f539c05d0c0 0x7f539c0b0040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f53940032f0 tx=0x7f53940091d0).stop 2022-07-21T01:56:10.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 -- 172.21.15.140:0/4034891919 shutdown_connections 2022-07-21T01:56:10.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 --2- 172.21.15.140:0/4034891919 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f539c05d0c0 0x7f539c0b0040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 -- 172.21.15.140:0/4034891919 >> 172.21.15.140:0/4034891919 conn(0x7f539c0634c0 msgr2=0x7f539c0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:10.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 -- 172.21.15.140:0/4034891919 shutdown_connections 2022-07-21T01:56:10.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.009+0000 7f53a11bf700 1 -- 172.21.15.140:0/4034891919 wait complete. 2022-07-21T01:56:10.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a11bf700 1 Processor -- start 2022-07-21T01:56:10.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a11bf700 1 -- start start 2022-07-21T01:56:10.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a11bf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f539c05d0c0 0x7f539c13d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a11bf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f539c0af920 0x7f539c155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a11bf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f539c0b59d0 0x7f539c155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a11bf700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c148c20 con 0x7f539c05d0c0 2022-07-21T01:56:10.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a11bf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539c1560e0 con 0x7f539c0b59d0 2022-07-21T01:56:10.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a11bf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539c064d10 con 0x7f539c0af920 2022-07-21T01:56:10.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f539bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f539c05d0c0 0x7f539c13d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f539bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f539c05d0c0 0x7f539c13d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38932/0 (socket says 172.21.15.140:38932) 2022-07-21T01:56:10.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f539bfff700 1 -- 172.21.15.140:0/2981664557 learned_addr learned my addr 172.21.15.140:0/2981664557 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:10.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a09be700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f539c0b59d0 0x7f539c155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f539b7fe700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f539c0af920 0x7f539c155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a09be700 1 -- 172.21.15.140:0/2981664557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f539c0af920 msgr2=0x7f539c155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a09be700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f539c0af920 0x7f539c155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a09be700 1 -- 172.21.15.140:0/2981664557 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f539c05d0c0 msgr2=0x7f539c13d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a09be700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f539c05d0c0 0x7f539c13d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a09be700 1 -- 172.21.15.140:0/2981664557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5394009470 con 0x7f539c0b59d0 2022-07-21T01:56:10.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a09be700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f539c0b59d0 0x7f539c155bf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f539c156220 tx=0x7f5394016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53997fa700 1 -- 172.21.15.140:0/2981664557 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f539400b910 con 0x7f539c0b59d0 2022-07-21T01:56:10.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53997fa700 1 -- 172.21.15.140:0/2981664557 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539401f3f0 con 0x7f539c0b59d0 2022-07-21T01:56:10.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a11bf700 1 -- 172.21.15.140:0/2981664557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539c064a70 con 0x7f539c0b59d0 2022-07-21T01:56:10.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53997fa700 1 -- 172.21.15.140:0/2981664557 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f539400b640 con 0x7f539c0b59d0 2022-07-21T01:56:10.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.013+0000 7f53a11bf700 1 -- 172.21.15.140:0/2981664557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539c13dc20 con 0x7f539c0b59d0 2022-07-21T01:56:10.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.017+0000 7f53997fa700 1 -- 172.21.15.140:0/2981664557 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f539400f070 con 0x7f539c0b59d0 2022-07-21T01:56:10.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.017+0000 7f53a11bf700 1 -- 172.21.15.140:0/2981664557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539c0af2d0 con 0x7f539c0b59d0 2022-07-21T01:56:10.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.017+0000 7f53997fa700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5380053980 0x7f5380055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.017+0000 7f53997fa700 1 -- 172.21.15.140:0/2981664557 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f539406c970 con 0x7f539c0b59d0 2022-07-21T01:56:10.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.017+0000 7f539bfff700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5380053980 0x7f5380055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.017+0000 7f539bfff700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5380053980 0x7f5380055e40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f538400d000 tx=0x7f5384009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.021+0000 7f53997fa700 1 -- 172.21.15.140:0/2981664557 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5394043780 con 0x7f539c0b59d0 2022-07-21T01:56:10.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.145+0000 7f53a11bf700 1 -- 172.21.15.140:0/2981664557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f539c0574e0 con 0x7f539c0b59d0 2022-07-21T01:56:10.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.145+0000 7f53997fa700 1 -- 172.21.15.140:0/2981664557 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7f53940150a0 con 0x7f539c0b59d0 2022-07-21T01:56:10.152 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:10.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 -- 172.21.15.140:0/2981664557 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5380053980 msgr2=0x7f5380055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5380053980 0x7f5380055e40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f538400d000 tx=0x7f5384009040).stop 2022-07-21T01:56:10.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 -- 172.21.15.140:0/2981664557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f539c0b59d0 msgr2=0x7f539c155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f539c0b59d0 0x7f539c155bf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f539c156220 tx=0x7f5394016000).stop 2022-07-21T01:56:10.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 -- 172.21.15.140:0/2981664557 shutdown_connections 2022-07-21T01:56:10.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5380053980 0x7f5380055e40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f539c0b59d0 0x7f539c155bf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f539c0af920 0x7f539c155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 --2- 172.21.15.140:0/2981664557 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f539c05d0c0 0x7f539c13d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 -- 172.21.15.140:0/2981664557 >> 172.21.15.140:0/2981664557 conn(0x7f539c0634c0 msgr2=0x7f539c0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:10.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 -- 172.21.15.140:0/2981664557 shutdown_connections 2022-07-21T01:56:10.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.149+0000 7f53a11bf700 1 -- 172.21.15.140:0/2981664557 wait complete. 2022-07-21T01:56:10.165 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:56:10.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 Processor -- start 2022-07-21T01:56:10.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 -- start start 2022-07-21T01:56:10.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faef40b0480 0x7faef414a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faef414ab60 0x7faef4144320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef40ac5e0 con 0x7faef40b5b50 2022-07-21T01:56:10.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef40ad3c0 con 0x7faef414ab60 2022-07-21T01:56:10.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef40b5f30 con 0x7faef40b0480 2022-07-21T01:56:10.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef37fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7faef40b5b50 0x7faef40accb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53086/0 (socket says 172.21.15.140:53086) 2022-07-21T01:56:10.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef37fe700 1 -- 172.21.15.140:0/2615714578 learned_addr learned my addr 172.21.15.140:0/2615714578 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:10.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef3fff700 1 --2- 172.21.15.140:0/2615714578 >> v2:172.21.15.140:3300/0 conn(0x7faef414ab60 0x7faef4144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef2ffd700 1 --2- 172.21.15.140:0/2615714578 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faef40b0480 0x7faef414a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef27fc700 1 -- 172.21.15.140:0/2615714578 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691714038 0 0) 0x7faef40ac5e0 con 0x7faef40b5b50 2022-07-21T01:56:10.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef27fc700 1 -- 172.21.15.140:0/2615714578 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee0003260 con 0x7faef40b5b50 2022-07-21T01:56:10.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef3fff700 1 -- 172.21.15.140:0/2615714578 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faef40b0480 msgr2=0x7faef414a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:10.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef3fff700 1 --2- 172.21.15.140:0/2615714578 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faef40b0480 0x7faef414a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef3fff700 1 -- 172.21.15.140:0/2615714578 >> v1:172.21.15.140:6789/0 conn(0x7faef40b5b50 legacy=0x7faef40accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef3fff700 1 -- 172.21.15.140:0/2615714578 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef405cf70 con 0x7faef414ab60 2022-07-21T01:56:10.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef3fff700 1 --2- 172.21.15.140:0/2615714578 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faef414ab60 0x7faef4144320 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7faeec0032f0 tx=0x7faeec0091d0).ready entity=mon.0 client_cookie=9b4e60c6002fb615 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef27fc700 1 -- 172.21.15.140:0/2615714578 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faeec00b6c0 con 0x7faef414ab60 2022-07-21T01:56:10.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef27fc700 1 -- 172.21.15.140:0/2615714578 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec00b820 con 0x7faef414ab60 2022-07-21T01:56:10.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef27fc700 1 -- 172.21.15.140:0/2615714578 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faeec00b990 con 0x7faef414ab60 2022-07-21T01:56:10.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 -- 172.21.15.140:0/2615714578 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faef414ab60 msgr2=0x7faef4144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 --2- 172.21.15.140:0/2615714578 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faef414ab60 0x7faef4144320 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7faeec0032f0 tx=0x7faeec0091d0).stop 2022-07-21T01:56:10.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 -- 172.21.15.140:0/2615714578 shutdown_connections 2022-07-21T01:56:10.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 --2- 172.21.15.140:0/2615714578 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faef414ab60 0x7faef4144320 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 --2- 172.21.15.140:0/2615714578 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faef40b0480 0x7faef414a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 -- 172.21.15.140:0/2615714578 >> 172.21.15.140:0/2615714578 conn(0x7faef40635e0 msgr2=0x7faef40b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:10.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.269+0000 7faef9ac2700 1 -- 172.21.15.140:0/2615714578 shutdown_connections 2022-07-21T01:56:10.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef9ac2700 1 -- 172.21.15.140:0/2615714578 wait complete. 2022-07-21T01:56:10.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef9ac2700 1 Processor -- start 2022-07-21T01:56:10.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef9ac2700 1 -- start start 2022-07-21T01:56:10.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef9ac2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faef40b0480 0x7faef4147060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef9ac2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faef40b5b50 0x7faef40af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef9ac2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faef414ab60 0x7faef4145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef9ac2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef4147c40 con 0x7faef40b5b50 2022-07-21T01:56:10.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef9ac2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef41463e0 con 0x7faef40b0480 2022-07-21T01:56:10.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef9ac2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef4146990 con 0x7faef414ab60 2022-07-21T01:56:10.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef2ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faef40b5b50 0x7faef40af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef2ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faef40b5b50 0x7faef40af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38948/0 (socket says 172.21.15.140:38948) 2022-07-21T01:56:10.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef2ffd700 1 -- 172.21.15.140:0/4255230931 learned_addr learned my addr 172.21.15.140:0/4255230931 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:10.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef37fe700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faef40b0480 0x7faef4147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef3fff700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faef414ab60 0x7faef4145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef2ffd700 1 -- 172.21.15.140:0/4255230931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faef414ab60 msgr2=0x7faef4145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef2ffd700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faef414ab60 0x7faef4145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef2ffd700 1 -- 172.21.15.140:0/4255230931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faef40b0480 msgr2=0x7faef4147060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef2ffd700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faef40b0480 0x7faef4147060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef2ffd700 1 -- 172.21.15.140:0/4255230931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec009470 con 0x7faef40b5b50 2022-07-21T01:56:10.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef2ffd700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faef40b5b50 0x7faef40af040 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7faef414e4f0 tx=0x7faee4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.273+0000 7faef0ff9700 1 -- 172.21.15.140:0/4255230931 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faee4011780 con 0x7faef40b5b50 2022-07-21T01:56:10.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.277+0000 7faef0ff9700 1 -- 172.21.15.140:0/4255230931 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee400ded0 con 0x7faef40b5b50 2022-07-21T01:56:10.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.277+0000 7faef0ff9700 1 -- 172.21.15.140:0/4255230931 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faee4011ad0 con 0x7faef40b5b50 2022-07-21T01:56:10.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.277+0000 7faef9ac2700 1 -- 172.21.15.140:0/4255230931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef4146600 con 0x7faef40b5b50 2022-07-21T01:56:10.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.277+0000 7faef9ac2700 1 -- 172.21.15.140:0/4255230931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef4067690 con 0x7faef40b5b50 2022-07-21T01:56:10.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.277+0000 7faef9ac2700 1 -- 172.21.15.140:0/4255230931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef40ac810 con 0x7faef40b5b50 2022-07-21T01:56:10.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.277+0000 7faef0ff9700 1 -- 172.21.15.140:0/4255230931 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faee4011c30 con 0x7faef40b5b50 2022-07-21T01:56:10.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.277+0000 7faef0ff9700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faed4053a70 0x7faed4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.277+0000 7faef0ff9700 1 -- 172.21.15.140:0/4255230931 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faee406bdf0 con 0x7faef40b5b50 2022-07-21T01:56:10.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.277+0000 7faef37fe700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faed4053a70 0x7faed4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.281+0000 7faef37fe700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faed4053a70 0x7faed4055f30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7faee800d000 tx=0x7faee8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.285+0000 7faef0ff9700 1 -- 172.21.15.140:0/4255230931 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faee4042cb0 con 0x7faef40b5b50 2022-07-21T01:56:10.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.409+0000 7faef9ac2700 1 -- 172.21.15.140:0/4255230931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faef40574e0 con 0x7faef40b5b50 2022-07-21T01:56:10.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.409+0000 7faef0ff9700 1 -- 172.21.15.140:0/4255230931 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+10 (secure 0 0 0) 0x7faee40462d0 con 0x7faef40b5b50 2022-07-21T01:56:10.415 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:10.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 -- 172.21.15.140:0/4255230931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faed4053a70 msgr2=0x7faed4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faed4053a70 0x7faed4055f30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7faee800d000 tx=0x7faee8009040).stop 2022-07-21T01:56:10.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 -- 172.21.15.140:0/4255230931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faef40b5b50 msgr2=0x7faef40af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faef40b5b50 0x7faef40af040 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7faef414e4f0 tx=0x7faee4008040).stop 2022-07-21T01:56:10.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 -- 172.21.15.140:0/4255230931 shutdown_connections 2022-07-21T01:56:10.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faed4053a70 0x7faed4055f30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faef414ab60 0x7faef4145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faef40b5b50 0x7faef40af040 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 --2- 172.21.15.140:0/4255230931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faef40b0480 0x7faef4147060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 -- 172.21.15.140:0/4255230931 >> 172.21.15.140:0/4255230931 conn(0x7faef40635e0 msgr2=0x7faef40b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:10.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 -- 172.21.15.140:0/4255230931 shutdown_connections 2022-07-21T01:56:10.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.413+0000 7faef9ac2700 1 -- 172.21.15.140:0/4255230931 wait complete. 2022-07-21T01:56:10.428 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:56:10.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.529+0000 7f5e5cc4b700 1 Processor -- start 2022-07-21T01:56:10.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.529+0000 7f5e5cc4b700 1 -- start start 2022-07-21T01:56:10.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5e58061d90 0x7f5e58143620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5e580b1df0 0x7f5e58143b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e58067f30 con 0x7f5e580b2270 2022-07-21T01:56:10.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e58062170 con 0x7f5e58061d90 2022-07-21T01:56:10.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e580abb10 con 0x7f5e580b1df0 2022-07-21T01:56:10.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e577fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5e58061d90 0x7f5e58143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e56ffd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5e580b1df0 0x7f5e58143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e577fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5e58061d90 0x7f5e58143620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38954/0 (socket says 172.21.15.140:38954) 2022-07-21T01:56:10.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e577fe700 1 -- 172.21.15.140:0/4032512482 learned_addr learned my addr 172.21.15.140:0/4032512482 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:10.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e577fe700 1 -- 172.21.15.140:0/4032512482 >> v2:172.21.15.152:3300/0 conn(0x7f5e580b1df0 msgr2=0x7f5e58143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:10.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e577fe700 1 --2- 172.21.15.140:0/4032512482 >> v2:172.21.15.152:3300/0 conn(0x7f5e580b1df0 0x7f5e58143b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e577fe700 1 -- 172.21.15.140:0/4032512482 >> v1:172.21.15.152:6789/0 conn(0x7f5e580b2270 legacy=0x7f5e580993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e577fe700 1 -- 172.21.15.140:0/4032512482 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e580abd90 con 0x7f5e58061d90 2022-07-21T01:56:10.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e577fe700 1 --2- 172.21.15.140:0/4032512482 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e58061d90 0x7f5e58143620 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f5e40002f70 tx=0x7f5e40006eb0).ready entity=mon.0 client_cookie=90248f245fd36d8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e567fc700 1 -- 172.21.15.140:0/4032512482 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e4000d5c0 con 0x7f5e58061d90 2022-07-21T01:56:10.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e567fc700 1 -- 172.21.15.140:0/4032512482 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e40000e90 con 0x7f5e58061d90 2022-07-21T01:56:10.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e567fc700 1 -- 172.21.15.140:0/4032512482 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e4000d950 con 0x7f5e58061d90 2022-07-21T01:56:10.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/4032512482 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e58061d90 msgr2=0x7f5e58143620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 --2- 172.21.15.140:0/4032512482 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e58061d90 0x7f5e58143620 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f5e40002f70 tx=0x7f5e40006eb0).stop 2022-07-21T01:56:10.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/4032512482 shutdown_connections 2022-07-21T01:56:10.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 --2- 172.21.15.140:0/4032512482 >> v2:172.21.15.152:3300/0 conn(0x7f5e580b1df0 0x7f5e58143b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 --2- 172.21.15.140:0/4032512482 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e58061d90 0x7f5e58143620 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/4032512482 >> 172.21.15.140:0/4032512482 conn(0x7f5e580b61f0 msgr2=0x7f5e580b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:10.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/4032512482 shutdown_connections 2022-07-21T01:56:10.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.533+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/4032512482 wait complete. 2022-07-21T01:56:10.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.537+0000 7f5e5cc4b700 1 Processor -- start 2022-07-21T01:56:10.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.537+0000 7f5e5cc4b700 1 -- start start 2022-07-21T01:56:10.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.537+0000 7f5e5cc4b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e58061d90 0x7f5e58144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.537+0000 7f5e5cc4b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e580b2270 0x7f5e5813f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e5cc4b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e580ae3a0 0x7f5e5813f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e5cc4b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5814fda0 con 0x7f5e58061d90 2022-07-21T01:56:10.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e5cc4b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e58065710 con 0x7f5e580ae3a0 2022-07-21T01:56:10.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e5cc4b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5813fa30 con 0x7f5e580b2270 2022-07-21T01:56:10.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e577fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e58061d90 0x7f5e58144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e577fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e58061d90 0x7f5e58144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38960/0 (socket says 172.21.15.140:38960) 2022-07-21T01:56:10.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e577fe700 1 -- 172.21.15.140:0/1717332560 learned_addr learned my addr 172.21.15.140:0/1717332560 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:10.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e56ffd700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e580b2270 0x7f5e5813f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e57fff700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e580ae3a0 0x7f5e5813f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e577fe700 1 -- 172.21.15.140:0/1717332560 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e580b2270 msgr2=0x7f5e5813f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e577fe700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e580b2270 0x7f5e5813f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e577fe700 1 -- 172.21.15.140:0/1717332560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e580ae3a0 msgr2=0x7f5e5813f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e577fe700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e580ae3a0 0x7f5e5813f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e577fe700 1 -- 172.21.15.140:0/1717332560 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4000b1e0 con 0x7f5e58061d90 2022-07-21T01:56:10.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e577fe700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e58061d90 0x7f5e58144520 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7f5e40002f70 tx=0x7f5e40015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e54ff9700 1 -- 172.21.15.140:0/1717332560 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e40011860 con 0x7f5e58061d90 2022-07-21T01:56:10.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e54ff9700 1 -- 172.21.15.140:0/1717332560 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e400119c0 con 0x7f5e58061d90 2022-07-21T01:56:10.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/1717332560 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e58144a10 con 0x7f5e58061d90 2022-07-21T01:56:10.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e54ff9700 1 -- 172.21.15.140:0/1717332560 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e40011d20 con 0x7f5e58061d90 2022-07-21T01:56:10.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/1717332560 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e58147130 con 0x7f5e58061d90 2022-07-21T01:56:10.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/1717332560 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e580658d0 con 0x7f5e58061d90 2022-07-21T01:56:10.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e54ff9700 1 -- 172.21.15.140:0/1717332560 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5e40025030 con 0x7f5e58061d90 2022-07-21T01:56:10.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e54ff9700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e3c053b00 0x7f5e3c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e54ff9700 1 -- 172.21.15.140:0/1717332560 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5e4006d000 con 0x7f5e58061d90 2022-07-21T01:56:10.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.541+0000 7f5e56ffd700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e3c053b00 0x7f5e3c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.545+0000 7f5e56ffd700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e3c053b00 0x7f5e3c055fc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f5e4800b000 tx=0x7f5e48006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.549+0000 7f5e54ff9700 1 -- 172.21.15.140:0/1717332560 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5e40043f20 con 0x7f5e58061d90 2022-07-21T01:56:10.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.673+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/1717332560 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e580b1950 con 0x7f5e58061d90 2022-07-21T01:56:10.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e54ff9700 1 -- 172.21.15.140:0/1717332560 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5e40023020 con 0x7f5e58061d90 2022-07-21T01:56:10.681 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:10.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/1717332560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e3c053b00 msgr2=0x7f5e3c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e3c053b00 0x7f5e3c055fc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f5e4800b000 tx=0x7f5e48006df0).stop 2022-07-21T01:56:10.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/1717332560 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e58061d90 msgr2=0x7f5e58144520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e58061d90 0x7f5e58144520 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7f5e40002f70 tx=0x7f5e40015000).stop 2022-07-21T01:56:10.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/1717332560 shutdown_connections 2022-07-21T01:56:10.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5e3c053b00 0x7f5e3c055fc0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e580ae3a0 0x7f5e5813f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5e580b2270 0x7f5e5813f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 --2- 172.21.15.140:0/1717332560 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5e58061d90 0x7f5e58144520 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/1717332560 >> 172.21.15.140:0/1717332560 conn(0x7f5e580b61f0 msgr2=0x7f5e580b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:10.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/1717332560 shutdown_connections 2022-07-21T01:56:10.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.677+0000 7f5e5cc4b700 1 -- 172.21.15.140:0/1717332560 wait complete. 2022-07-21T01:56:10.693 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:56:10.694 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:10.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.789+0000 7f1bed865700 1 Processor -- start 2022-07-21T01:56:10.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.789+0000 7f1bed865700 1 -- start start 2022-07-21T01:56:10.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.789+0000 7f1bed865700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1be80adff0 0x7f1be80b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.789+0000 7f1bed865700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be8147010 con 0x7f1be80af8e0 2022-07-21T01:56:10.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.789+0000 7f1bed865700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be81472b0 con 0x7f1be80b5650 2022-07-21T01:56:10.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.789+0000 7f1bed865700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be8146d40 con 0x7f1be80adff0 2022-07-21T01:56:10.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.789+0000 7f1bed064700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1be80af8e0 0x7f1be805bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53116/0 (socket says 172.21.15.140:53116) 2022-07-21T01:56:10.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.789+0000 7f1bed064700 1 -- 172.21.15.140:0/3823189214 learned_addr learned my addr 172.21.15.140:0/3823189214 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:10.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.789+0000 7f1be7fff700 1 --2- 172.21.15.140:0/3823189214 >> v2:172.21.15.152:3300/0 conn(0x7f1be80adff0 0x7f1be80b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be77fe700 1 -- 172.21.15.140:0/3823189214 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911426742 0 0) 0x7f1be8147010 con 0x7f1be80af8e0 2022-07-21T01:56:10.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be77fe700 1 -- 172.21.15.140:0/3823189214 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bd4003260 con 0x7f1be80af8e0 2022-07-21T01:56:10.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be77fe700 1 -- 172.21.15.140:0/3823189214 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320735051 0 0) 0x7f1be81472b0 con 0x7f1be80b5650 2022-07-21T01:56:10.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be77fe700 1 -- 172.21.15.140:0/3823189214 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1be8147010 con 0x7f1be80b5650 2022-07-21T01:56:10.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be7fff700 1 -- 172.21.15.140:0/3823189214 >> v1:172.21.15.152:6789/0 conn(0x7f1be80b5650 legacy=0x7f1be8147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be7fff700 1 -- 172.21.15.140:0/3823189214 >> v1:172.21.15.140:6789/0 conn(0x7f1be80af8e0 legacy=0x7f1be805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be77fe700 1 -- 172.21.15.140:0/3823189214 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2549797030 0 0) 0x7f1bd4003260 con 0x7f1be80af8e0 2022-07-21T01:56:10.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be7fff700 1 -- 172.21.15.140:0/3823189214 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be805c4a0 con 0x7f1be80adff0 2022-07-21T01:56:10.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be7fff700 1 --2- 172.21.15.140:0/3823189214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be80adff0 0x7f1be80b5a30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f1bd8001800 tx=0x7f1bdc006d90).ready entity=mon.1 client_cookie=a7e90455c695baa3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be77fe700 1 -- 172.21.15.140:0/3823189214 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bd8001660 con 0x7f1be80adff0 2022-07-21T01:56:10.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be77fe700 1 -- 172.21.15.140:0/3823189214 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bdc000e90 con 0x7f1be80adff0 2022-07-21T01:56:10.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1be77fe700 1 -- 172.21.15.140:0/3823189214 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bdc00beb0 con 0x7f1be80adff0 2022-07-21T01:56:10.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1bed865700 1 -- 172.21.15.140:0/3823189214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be80adff0 msgr2=0x7f1be80b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1bed865700 1 --2- 172.21.15.140:0/3823189214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be80adff0 0x7f1be80b5a30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f1bd8001800 tx=0x7f1bdc006d90).stop 2022-07-21T01:56:10.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1bed865700 1 -- 172.21.15.140:0/3823189214 shutdown_connections 2022-07-21T01:56:10.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1bed865700 1 --2- 172.21.15.140:0/3823189214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be80adff0 0x7f1be80b5a30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1bed865700 1 -- 172.21.15.140:0/3823189214 >> 172.21.15.140:0/3823189214 conn(0x7f1be80634e0 msgr2=0x7f1be80b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:10.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1bed865700 1 -- 172.21.15.140:0/3823189214 shutdown_connections 2022-07-21T01:56:10.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1bed865700 1 -- 172.21.15.140:0/3823189214 wait complete. 2022-07-21T01:56:10.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1bed865700 1 Processor -- start 2022-07-21T01:56:10.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.793+0000 7f1bed865700 1 -- start start 2022-07-21T01:56:10.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bed865700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1be80adff0 0x7f1be8066750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bed865700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be80af8e0 0x7f1be81119f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bed865700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1be80b5650 0x7f1be8064eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bed865700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be8065c40 con 0x7f1be80adff0 2022-07-21T01:56:10.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bed865700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80679c0 con 0x7f1be80af8e0 2022-07-21T01:56:10.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bed865700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be80644d0 con 0x7f1be80b5650 2022-07-21T01:56:10.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bec863700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1be80adff0 0x7f1be8066750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bec863700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1be80adff0 0x7f1be8066750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38974/0 (socket says 172.21.15.140:38974) 2022-07-21T01:56:10.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bec863700 1 -- 172.21.15.140:0/3843527520 learned_addr learned my addr 172.21.15.140:0/3843527520 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:10.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bed064700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1be80b5650 0x7f1be8064eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1be7fff700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be80af8e0 0x7f1be81119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bec863700 1 -- 172.21.15.140:0/3843527520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1be80b5650 msgr2=0x7f1be8064eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bec863700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1be80b5650 0x7f1be8064eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bec863700 1 -- 172.21.15.140:0/3843527520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be80af8e0 msgr2=0x7f1be81119f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bec863700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be80af8e0 0x7f1be81119f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bec863700 1 -- 172.21.15.140:0/3843527520 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc00b120 con 0x7f1be80adff0 2022-07-21T01:56:10.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bed064700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1be80b5650 0x7f1be8064eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:56:10.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1be7fff700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be80af8e0 0x7f1be81119f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:56:10.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bec863700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1be80adff0 0x7f1be8066750 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f1be8066620 tx=0x7f1be0007e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1be5ffb700 1 -- 172.21.15.140:0/3843527520 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1be0002030 con 0x7f1be80adff0 2022-07-21T01:56:10.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1be5ffb700 1 -- 172.21.15.140:0/3843527520 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be0002190 con 0x7f1be80adff0 2022-07-21T01:56:10.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bed865700 1 -- 172.21.15.140:0/3843527520 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be8064770 con 0x7f1be80adff0 2022-07-21T01:56:10.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1be5ffb700 1 -- 172.21.15.140:0/3843527520 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1be00024f0 con 0x7f1be80adff0 2022-07-21T01:56:10.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bed865700 1 -- 172.21.15.140:0/3843527520 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be8063e10 con 0x7f1be80adff0 2022-07-21T01:56:10.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1bed865700 1 -- 172.21.15.140:0/3843527520 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be80574e0 con 0x7f1be80adff0 2022-07-21T01:56:10.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1be5ffb700 1 -- 172.21.15.140:0/3843527520 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1be0010070 con 0x7f1be80adff0 2022-07-21T01:56:10.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.797+0000 7f1be5ffb700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bc80539e0 0x7f1bc8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:10.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.801+0000 7f1be7fff700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bc80539e0 0x7f1bc8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:10.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.801+0000 7f1be5ffb700 1 -- 172.21.15.140:0/3843527520 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1be006b550 con 0x7f1be80adff0 2022-07-21T01:56:10.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.801+0000 7f1be7fff700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bc80539e0 0x7f1bc8055ea0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f1bdc015000 tx=0x7f1bdc014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:10.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.805+0000 7f1be5ffb700 1 -- 172.21.15.140:0/3843527520 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1be0042470 con 0x7f1be80adff0 2022-07-21T01:56:10.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.961+0000 7f1bed865700 1 -- 172.21.15.140:0/3843527520 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1be80adc10 con 0x7f1be80adff0 2022-07-21T01:56:10.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1be5ffb700 1 -- 172.21.15.140:0/3843527520 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f1be0045a90 con 0x7f1be80adff0 2022-07-21T01:56:10.968 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:10.969 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":47,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/3024179649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":3024179649},{"type":"v1","addr":"172.21.15.140:6835","nonce":3024179649}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:56:06.173419+0000","modified":"2022-07-21T01:56:07.189859+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/727644549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":727644549},{"type":"v1","addr":"172.21.15.152:6833","nonce":727644549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-07-21T01:56:10.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 -- 172.21.15.140:0/3843527520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bc80539e0 msgr2=0x7f1bc8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bc80539e0 0x7f1bc8055ea0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f1bdc015000 tx=0x7f1bdc014000).stop 2022-07-21T01:56:10.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 -- 172.21.15.140:0/3843527520 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1be80adff0 msgr2=0x7f1be8066750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:10.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1be80adff0 0x7f1be8066750 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f1be8066620 tx=0x7f1be0007e60).stop 2022-07-21T01:56:10.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 -- 172.21.15.140:0/3843527520 shutdown_connections 2022-07-21T01:56:10.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1bc80539e0 0x7f1bc8055ea0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1be80b5650 0x7f1be8064eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be80af8e0 0x7f1be81119f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 --2- 172.21.15.140:0/3843527520 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1be80adff0 0x7f1be8066750 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:10.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 -- 172.21.15.140:0/3843527520 >> 172.21.15.140:0/3843527520 conn(0x7f1be80634e0 msgr2=0x7f1be8146340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:10.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 -- 172.21.15.140:0/3843527520 shutdown_connections 2022-07-21T01:56:10.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:10.965+0000 7f1bed865700 1 -- 172.21.15.140:0/3843527520 wait complete. 2022-07-21T01:56:10.974 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 47 2022-07-21T01:56:11.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:56:06.173419+0000', 'modified': '2022-07-21T01:56:07.189859+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7335}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7335': {'gid': 7335, 'name': 'b', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/727644549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 727644549}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 727644549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:56:11.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:56:11.184 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:56:11.306 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":7335,"want_state":"up:active","state":"up:active","rank_uptime":5.1132887699999996,"mdsmap_epoch":47,"osdmap_epoch":103,"osdmap_epoch_barrier":102,"uptime":9.4806192459999998} 2022-07-21T01:56:11.307 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 7335, "mdsmap_epoch": 47, "osdmap_epoch": 103, "osdmap_epoch_barrier": 102, "rank_uptime": 5.11328877, "state": "up:active", "uptime": 9.480619246, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:56:11.307 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:56:11.308 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:56:11.308 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:56:11.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:11.309 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:11.318 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:56:11.376 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:56:11.377 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:56:11.387 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:56:11.388 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:56:11.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:11.391 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:56:11.397 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:56:11.398 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:11.401 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:56:11.419 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:56:11.420 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:56:11.477 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:56:11.472+0000 7f03d7bc4080 -1 init, newargv = 0x5611aee33070 newargc=9ceph-fuse[ 2022-07-21T01:56:11.478 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:23126]: starting ceph client 2022-07-21T01:56:11.481 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:56:11.482 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:56:11.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:11.484 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:56:11.487 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23126]: starting fuse 2022-07-21T01:56:11.492 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:56:11.492 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:56:11.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:11.493 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:56:11.494 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:56:11.494 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:56:11.495 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:56:11.496 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:56:11.496 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:56:11.497 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:56:11.497 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:56:11.498 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:56:11.498 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:56:11.498 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:56:11.499 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:56:11.499 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:56:11.500 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:56:11.500 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:56:11.501 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:56:11.501 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:56:11.502 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:56:11.502 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:56:11.502 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:56:11.503 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:56:11.503 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:56:11.504 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:56:11.541 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.23126.asok 2022-07-21T01:56:11.545 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23126.asok 2022-07-21T01:56:11.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:11.546 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23126.asok status 2022-07-21T01:56:11.681 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:56:11.682 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:56:11.682 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:56:11.682 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:56:11.682 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:56:11.683 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:56:11.683 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:56:11.683 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "23126", 2022-07-21T01:56:11.683 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:56:11.684 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:56:11.684 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:56:11.684 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:56:11.684 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 7378, 2022-07-21T01:56:11.685 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:56:11.685 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:56:11.685 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:56:11.685 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 7378 2022-07-21T01:56:11.686 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:56:11.686 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:56:11.686 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:56:11.687 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:56:11.687 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 561005812 2022-07-21T01:56:11.687 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:56:11.687 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:56:11.688 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:56:11.688 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:56:11.688 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:56:11.688 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 561005812 2022-07-21T01:56:11.689 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:56:11.689 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.7378 172.21.15.191:0/561005812", 2022-07-21T01:56:11.689 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/561005812", 2022-07-21T01:56:11.689 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:56:11.690 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 47, 2022-07-21T01:56:11.690 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 103, 2022-07-21T01:56:11.690 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:56:11.690 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:56:11.691 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:56:11.691 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:56:11.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:11.693 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:11.693 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:11.701 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:56:11.702 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:56:11.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:11.703 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:11.720 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:11.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.825+0000 7ffa5a9ce700 1 Processor -- start 2022-07-21T01:56:11.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.825+0000 7ffa5a9ce700 1 -- start start 2022-07-21T01:56:11.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa540b0570 0x7ffa540b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:11.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffa540aea10 0x7ffa54141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:11.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa54142750 con 0x7ffa541422c0 2022-07-21T01:56:11.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa54143b10 con 0x7ffa540aea10 2022-07-21T01:56:11.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa541433d0 con 0x7ffa540b0570 2022-07-21T01:56:11.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a1cd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ffa541422c0 0x7ffa540ad350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53132/0 (socket says 172.21.15.140:53132) 2022-07-21T01:56:11.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a1cd700 1 -- 172.21.15.140:0/4291157640 learned_addr learned my addr 172.21.15.140:0/4291157640 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:11.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa599cc700 1 --2- 172.21.15.140:0/4291157640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa540b0570 0x7ffa540b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:11.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa591cb700 1 --2- 172.21.15.140:0/4291157640 >> v2:172.21.15.152:3300/0 conn(0x7ffa540aea10 0x7ffa54141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:11.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa589ca700 1 -- 172.21.15.140:0/4291157640 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822944601 0 0) 0x7ffa54142750 con 0x7ffa541422c0 2022-07-21T01:56:11.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa589ca700 1 -- 172.21.15.140:0/4291157640 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa3c003260 con 0x7ffa541422c0 2022-07-21T01:56:11.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa591cb700 1 -- 172.21.15.140:0/4291157640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa540b0570 msgr2=0x7ffa540b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:11.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa591cb700 1 --2- 172.21.15.140:0/4291157640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa540b0570 0x7ffa540b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:11.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa591cb700 1 -- 172.21.15.140:0/4291157640 >> v1:172.21.15.140:6789/0 conn(0x7ffa541422c0 legacy=0x7ffa540ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:11.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa591cb700 1 -- 172.21.15.140:0/4291157640 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa54143670 con 0x7ffa540aea10 2022-07-21T01:56:11.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa591cb700 1 --2- 172.21.15.140:0/4291157640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa540aea10 0x7ffa54141dd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7ffa50002f70 tx=0x7ffa50006df0).ready entity=mon.1 client_cookie=e1455464183d4a1d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:11.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa589ca700 1 -- 172.21.15.140:0/4291157640 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa5000d4a0 con 0x7ffa540aea10 2022-07-21T01:56:11.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa589ca700 1 -- 172.21.15.140:0/4291157640 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa50000e90 con 0x7ffa540aea10 2022-07-21T01:56:11.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa589ca700 1 -- 172.21.15.140:0/4291157640 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa5000d7f0 con 0x7ffa540aea10 2022-07-21T01:56:11.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/4291157640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa540aea10 msgr2=0x7ffa54141dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:11.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 --2- 172.21.15.140:0/4291157640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa540aea10 0x7ffa54141dd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7ffa50002f70 tx=0x7ffa50006df0).stop 2022-07-21T01:56:11.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/4291157640 shutdown_connections 2022-07-21T01:56:11.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 --2- 172.21.15.140:0/4291157640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa540aea10 0x7ffa54141dd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:11.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 --2- 172.21.15.140:0/4291157640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa540b0570 0x7ffa540b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:11.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/4291157640 >> 172.21.15.140:0/4291157640 conn(0x7ffa540b9360 msgr2=0x7ffa540b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:11.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/4291157640 shutdown_connections 2022-07-21T01:56:11.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.829+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/4291157640 wait complete. 2022-07-21T01:56:11.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a9ce700 1 Processor -- start 2022-07-21T01:56:11.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a9ce700 1 -- start start 2022-07-21T01:56:11.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a9ce700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa540aea10 0x7ffa54147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:11.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a9ce700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa540b0570 0x7ffa54151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:11.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a9ce700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa541422c0 0x7ffa541557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:11.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a9ce700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa541136a0 con 0x7ffa541422c0 2022-07-21T01:56:11.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a9ce700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa541550d0 con 0x7ffa540b0570 2022-07-21T01:56:11.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a9ce700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa54113900 con 0x7ffa540aea10 2022-07-21T01:56:11.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a1cd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa541422c0 0x7ffa541557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:11.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a1cd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa541422c0 0x7ffa541557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38992/0 (socket says 172.21.15.140:38992) 2022-07-21T01:56:11.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a1cd700 1 -- 172.21.15.140:0/2350666824 learned_addr learned my addr 172.21.15.140:0/2350666824 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:11.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa599cc700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa540aea10 0x7ffa54147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:11.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa591cb700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa540b0570 0x7ffa54151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:11.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a1cd700 1 -- 172.21.15.140:0/2350666824 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa540aea10 msgr2=0x7ffa54147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:11.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a1cd700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa540aea10 0x7ffa54147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:11.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a1cd700 1 -- 172.21.15.140:0/2350666824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa540b0570 msgr2=0x7ffa54151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:11.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a1cd700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa540b0570 0x7ffa54151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:11.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a1cd700 1 -- 172.21.15.140:0/2350666824 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5000b180 con 0x7ffa541422c0 2022-07-21T01:56:11.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a1cd700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa541422c0 0x7ffa541557d0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7ffa4c001490 tx=0x7ffa4c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:11.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa42ffd700 1 -- 172.21.15.140:0/2350666824 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa4c005ba0 con 0x7ffa541422c0 2022-07-21T01:56:11.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa42ffd700 1 -- 172.21.15.140:0/2350666824 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c005d00 con 0x7ffa541422c0 2022-07-21T01:56:11.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/2350666824 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa54155330 con 0x7ffa541422c0 2022-07-21T01:56:11.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa42ffd700 1 -- 172.21.15.140:0/2350666824 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa4c0072c0 con 0x7ffa541422c0 2022-07-21T01:56:11.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.833+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/2350666824 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa540b2940 con 0x7ffa541422c0 2022-07-21T01:56:11.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.837+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/2350666824 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa541554c0 con 0x7ffa541422c0 2022-07-21T01:56:11.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.837+0000 7ffa42ffd700 1 -- 172.21.15.140:0/2350666824 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ffa4c01a060 con 0x7ffa541422c0 2022-07-21T01:56:11.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.837+0000 7ffa42ffd700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffa340539e0 0x7ffa34055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:11.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.837+0000 7ffa599cc700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffa340539e0 0x7ffa34055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:11.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.837+0000 7ffa42ffd700 1 -- 172.21.15.140:0/2350666824 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffa4c06bb00 con 0x7ffa541422c0 2022-07-21T01:56:11.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.837+0000 7ffa599cc700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffa340539e0 0x7ffa34055ea0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7ffa4400b000 tx=0x7ffa44006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:11.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.845+0000 7ffa42ffd700 1 -- 172.21.15.140:0/2350666824 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ffa4c06de60 con 0x7ffa541422c0 2022-07-21T01:56:11.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.969+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/2350666824 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa540574e0 con 0x7ffa541422c0 2022-07-21T01:56:11.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa42ffd700 1 -- 172.21.15.140:0/2350666824 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16148 (secure 0 0 0) 0x7ffa4c018030 con 0x7ffa541422c0 2022-07-21T01:56:11.978 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:11.979 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":103,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:07.189453+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:56:04.557618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2022-07-21T01:56:05.505410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:11.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/2350666824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffa340539e0 msgr2=0x7ffa34055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:11.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffa340539e0 0x7ffa34055ea0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7ffa4400b000 tx=0x7ffa44006df0).stop 2022-07-21T01:56:11.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/2350666824 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa541422c0 msgr2=0x7ffa541557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:11.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa541422c0 0x7ffa541557d0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7ffa4c001490 tx=0x7ffa4c007e00).stop 2022-07-21T01:56:11.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/2350666824 shutdown_connections 2022-07-21T01:56:11.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ffa340539e0 0x7ffa34055ea0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:11.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ffa541422c0 0x7ffa541557d0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:11.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa540b0570 0x7ffa54151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:11.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 --2- 172.21.15.140:0/2350666824 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffa540aea10 0x7ffa54147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:11.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/2350666824 >> 172.21.15.140:0/2350666824 conn(0x7ffa540b9360 msgr2=0x7ffa5405bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:11.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/2350666824 shutdown_connections 2022-07-21T01:56:11.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:11.973+0000 7ffa5a9ce700 1 -- 172.21.15.140:0/2350666824 wait complete. 2022-07-21T01:56:11.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:56:04.557618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:56:05.505410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:56:11.992 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:56:12.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.093+0000 7f366a1a5700 1 Processor -- start 2022-07-21T01:56:12.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.093+0000 7f366a1a5700 1 -- start start 2022-07-21T01:56:12.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.093+0000 7f366a1a5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f36640b3cf0 0x7f3664142a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.093+0000 7f366a1a5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3664142f40 0x7f36641419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.093+0000 7f366a1a5700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36640ba180 con 0x7f36640b9da0 2022-07-21T01:56:12.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.093+0000 7f366a1a5700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36640ad5b0 con 0x7f3664142f40 2022-07-21T01:56:12.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.093+0000 7f366a1a5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36641461b0 con 0x7f36640b3cf0 2022-07-21T01:56:12.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.093+0000 7f36699a4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3664142f40 0x7f36641419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.093+0000 7f36699a4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3664142f40 0x7f36641419a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:38998/0 (socket says 172.21.15.140:38998) 2022-07-21T01:56:12.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.093+0000 7f36699a4700 1 -- 172.21.15.140:0/3321032746 learned_addr learned my addr 172.21.15.140:0/3321032746 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:12.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.093+0000 7f36689a2700 1 --2- 172.21.15.140:0/3321032746 >> v2:172.21.15.152:3300/0 conn(0x7f36640b3cf0 0x7f3664142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f36699a4700 1 -- 172.21.15.140:0/3321032746 >> v2:172.21.15.152:3300/0 conn(0x7f36640b3cf0 msgr2=0x7f3664142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:12.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f36699a4700 1 --2- 172.21.15.140:0/3321032746 >> v2:172.21.15.152:3300/0 conn(0x7f36640b3cf0 0x7f3664142a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f36699a4700 1 -- 172.21.15.140:0/3321032746 >> v1:172.21.15.152:6789/0 conn(0x7f36640b9da0 legacy=0x7f36641468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f36699a4700 1 -- 172.21.15.140:0/3321032746 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3664146450 con 0x7f3664142f40 2022-07-21T01:56:12.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f36699a4700 1 --2- 172.21.15.140:0/3321032746 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3664142f40 0x7f36641419a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f365c0032f0 tx=0x7f365c009290).ready entity=mon.0 client_cookie=7d3a8b0428114580 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f365bfff700 1 -- 172.21.15.140:0/3321032746 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f365c00b870 con 0x7f3664142f40 2022-07-21T01:56:12.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f365bfff700 1 -- 172.21.15.140:0/3321032746 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c00b9d0 con 0x7f3664142f40 2022-07-21T01:56:12.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f365bfff700 1 -- 172.21.15.140:0/3321032746 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f365c00bb40 con 0x7f3664142f40 2022-07-21T01:56:12.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f366a1a5700 1 -- 172.21.15.140:0/3321032746 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3664142f40 msgr2=0x7f36641419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f366a1a5700 1 --2- 172.21.15.140:0/3321032746 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3664142f40 0x7f36641419a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f365c0032f0 tx=0x7f365c009290).stop 2022-07-21T01:56:12.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f366a1a5700 1 -- 172.21.15.140:0/3321032746 shutdown_connections 2022-07-21T01:56:12.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f366a1a5700 1 --2- 172.21.15.140:0/3321032746 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3664142f40 0x7f36641419a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f366a1a5700 1 --2- 172.21.15.140:0/3321032746 >> v2:172.21.15.152:3300/0 conn(0x7f36640b3cf0 0x7f3664142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f366a1a5700 1 -- 172.21.15.140:0/3321032746 >> 172.21.15.140:0/3321032746 conn(0x7f36640678c0 msgr2=0x7f36640b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:12.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f366a1a5700 1 -- 172.21.15.140:0/3321032746 shutdown_connections 2022-07-21T01:56:12.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.097+0000 7f366a1a5700 1 -- 172.21.15.140:0/3321032746 wait complete. 2022-07-21T01:56:12.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f366a1a5700 1 Processor -- start 2022-07-21T01:56:12.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f366a1a5700 1 -- start start 2022-07-21T01:56:12.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f366a1a5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36640b3cf0 0x7f36640b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f366a1a5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36640b9da0 0x7f3664144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f366a1a5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3664142f40 0x7f3664151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f366a1a5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640b22f0 con 0x7f36640b9da0 2022-07-21T01:56:12.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f366a1a5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3664144040 con 0x7f36640b3cf0 2022-07-21T01:56:12.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f366a1a5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36641442e0 con 0x7f3664142f40 2022-07-21T01:56:12.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f36689a2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36640b9da0 0x7f3664144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f36689a2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36640b9da0 0x7f3664144780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39004/0 (socket says 172.21.15.140:39004) 2022-07-21T01:56:12.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f36689a2700 1 -- 172.21.15.140:0/35956115 learned_addr learned my addr 172.21.15.140:0/35956115 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:12.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f36691a3700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36640b3cf0 0x7f36640b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f36699a4700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3664142f40 0x7f3664151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f36689a2700 1 -- 172.21.15.140:0/35956115 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3664142f40 msgr2=0x7f3664151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f36689a2700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3664142f40 0x7f3664151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f36689a2700 1 -- 172.21.15.140:0/35956115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36640b3cf0 msgr2=0x7f36640b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f36689a2700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36640b3cf0 0x7f36640b7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f36689a2700 1 -- 172.21.15.140:0/35956115 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c009530 con 0x7f36640b9da0 2022-07-21T01:56:12.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f36689a2700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36640b9da0 0x7f3664144780 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f365400c000 tx=0x7f3654006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f365a7fc700 1 -- 172.21.15.140:0/35956115 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f365400b3b0 con 0x7f36640b9da0 2022-07-21T01:56:12.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f365a7fc700 1 -- 172.21.15.140:0/35956115 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3654010590 con 0x7f36640b9da0 2022-07-21T01:56:12.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f365a7fc700 1 -- 172.21.15.140:0/35956115 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36540108f0 con 0x7f36640b9da0 2022-07-21T01:56:12.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f366a1a5700 1 -- 172.21.15.140:0/35956115 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36640b7e60 con 0x7f36640b9da0 2022-07-21T01:56:12.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.101+0000 7f366a1a5700 1 -- 172.21.15.140:0/35956115 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36641454d0 con 0x7f36640b9da0 2022-07-21T01:56:12.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.105+0000 7f365a7fc700 1 -- 172.21.15.140:0/35956115 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3654010a50 con 0x7f36640b9da0 2022-07-21T01:56:12.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.105+0000 7f366a1a5700 1 -- 172.21.15.140:0/35956115 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36640b2160 con 0x7f36640b9da0 2022-07-21T01:56:12.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.105+0000 7f365a7fc700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f364c053a70 0x7f364c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.105+0000 7f365a7fc700 1 -- 172.21.15.140:0/35956115 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f365406bf40 con 0x7f36640b9da0 2022-07-21T01:56:12.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.105+0000 7f36691a3700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f364c053a70 0x7f364c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.105+0000 7f36691a3700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f364c053a70 0x7f364c055f30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f366000d000 tx=0x7f3660007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.109+0000 7f365a7fc700 1 -- 172.21.15.140:0/35956115 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3654042d60 con 0x7f36640b9da0 2022-07-21T01:56:12.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.233+0000 7f366a1a5700 1 -- 172.21.15.140:0/35956115 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f36641444a0 con 0x7f36640b9da0 2022-07-21T01:56:12.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.233+0000 7f365a7fc700 1 -- 172.21.15.140:0/35956115 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v103) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3654046380 con 0x7f36640b9da0 2022-07-21T01:56:12.239 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:56:12.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 -- 172.21.15.140:0/35956115 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f364c053a70 msgr2=0x7f364c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f364c053a70 0x7f364c055f30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f366000d000 tx=0x7f3660007b90).stop 2022-07-21T01:56:12.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 -- 172.21.15.140:0/35956115 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36640b9da0 msgr2=0x7f3664144780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36640b9da0 0x7f3664144780 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f365400c000 tx=0x7f3654006df0).stop 2022-07-21T01:56:12.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 -- 172.21.15.140:0/35956115 shutdown_connections 2022-07-21T01:56:12.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f364c053a70 0x7f364c055f30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3664142f40 0x7f3664151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36640b9da0 0x7f3664144780 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 --2- 172.21.15.140:0/35956115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36640b3cf0 0x7f36640b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 -- 172.21.15.140:0/35956115 >> 172.21.15.140:0/35956115 conn(0x7f36640678c0 msgr2=0x7f3664140ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:12.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 -- 172.21.15.140:0/35956115 shutdown_connections 2022-07-21T01:56:12.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.237+0000 7f366a1a5700 1 -- 172.21.15.140:0/35956115 wait complete. 2022-07-21T01:56:12.252 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:56:12.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.353+0000 7f00a63f8700 1 Processor -- start 2022-07-21T01:56:12.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.353+0000 7f00a63f8700 1 -- start start 2022-07-21T01:56:12.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00a00b2ec0 0x7f00a00b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f00a005d160 0x7f00a0142210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f00a0142700 0x7f00a01461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a0142b90 con 0x7f00a005d160 2022-07-21T01:56:12.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a00b4ea0 con 0x7f00a0142700 2022-07-21T01:56:12.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a00b3f10 con 0x7f00a00b2ec0 2022-07-21T01:56:12.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a4bf5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f00a005d160 0x7f00a0142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a4bf5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f00a005d160 0x7f00a0142210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39010/0 (socket says 172.21.15.140:39010) 2022-07-21T01:56:12.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a4bf5700 1 -- 172.21.15.140:0/259937587 learned_addr learned my addr 172.21.15.140:0/259937587 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:12.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a5bf7700 1 --2- 172.21.15.140:0/259937587 >> v2:172.21.15.152:3300/0 conn(0x7f00a0142700 0x7f00a01461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a53f6700 1 --2- 172.21.15.140:0/259937587 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00a00b2ec0 0x7f00a00b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a4bf5700 1 -- 172.21.15.140:0/259937587 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00a00b2ec0 msgr2=0x7f00a00b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:12.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a4bf5700 1 --2- 172.21.15.140:0/259937587 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00a00b2ec0 0x7f00a00b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a4bf5700 1 -- 172.21.15.140:0/259937587 >> v2:172.21.15.152:3300/0 conn(0x7f00a0142700 msgr2=0x7f00a01461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:12.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a4bf5700 1 --2- 172.21.15.140:0/259937587 >> v2:172.21.15.152:3300/0 conn(0x7f00a0142700 0x7f00a01461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a4bf5700 1 -- 172.21.15.140:0/259937587 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a00b3790 con 0x7f00a005d160 2022-07-21T01:56:12.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a53f6700 1 --2- 172.21.15.140:0/259937587 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00a00b2ec0 0x7f00a00b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:12.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a4bf5700 1 --2- 172.21.15.140:0/259937587 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00a005d160 0x7f00a0142210 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f00a00b4fe0 tx=0x7f009c006eb0).ready entity=mon.0 client_cookie=24170549365c8a6b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f0093fff700 1 -- 172.21.15.140:0/259937587 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f009c00d570 con 0x7f00a005d160 2022-07-21T01:56:12.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f0093fff700 1 -- 172.21.15.140:0/259937587 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c000e90 con 0x7f00a005d160 2022-07-21T01:56:12.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f0093fff700 1 -- 172.21.15.140:0/259937587 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f009c00d900 con 0x7f00a005d160 2022-07-21T01:56:12.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 -- 172.21.15.140:0/259937587 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00a005d160 msgr2=0x7f00a0142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 --2- 172.21.15.140:0/259937587 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00a005d160 0x7f00a0142210 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f00a00b4fe0 tx=0x7f009c006eb0).stop 2022-07-21T01:56:12.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 -- 172.21.15.140:0/259937587 shutdown_connections 2022-07-21T01:56:12.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 --2- 172.21.15.140:0/259937587 >> v2:172.21.15.152:3300/0 conn(0x7f00a0142700 0x7f00a01461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 --2- 172.21.15.140:0/259937587 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00a005d160 0x7f00a0142210 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 --2- 172.21.15.140:0/259937587 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00a00b2ec0 0x7f00a00b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 -- 172.21.15.140:0/259937587 >> 172.21.15.140:0/259937587 conn(0x7f00a0066e20 msgr2=0x7f00a0067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:12.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 -- 172.21.15.140:0/259937587 shutdown_connections 2022-07-21T01:56:12.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.357+0000 7f00a63f8700 1 -- 172.21.15.140:0/259937587 wait complete. 2022-07-21T01:56:12.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a63f8700 1 Processor -- start 2022-07-21T01:56:12.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a63f8700 1 -- start start 2022-07-21T01:56:12.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a63f8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00a005d160 0x7f00a01469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a63f8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00a00b2ec0 0x7f00a00afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a63f8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00a0142700 0x7f00a01449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a63f8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a00b90f0 con 0x7f00a005d160 2022-07-21T01:56:12.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a63f8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a00af750 con 0x7f00a0142700 2022-07-21T01:56:12.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a63f8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a00b8e50 con 0x7f00a00b2ec0 2022-07-21T01:56:12.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a53f6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00a005d160 0x7f00a01469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a53f6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00a005d160 0x7f00a01469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39016/0 (socket says 172.21.15.140:39016) 2022-07-21T01:56:12.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a53f6700 1 -- 172.21.15.140:0/3478239019 learned_addr learned my addr 172.21.15.140:0/3478239019 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:12.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a4bf5700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00a00b2ec0 0x7f00a00afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.361+0000 7f00a5bf7700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00a0142700 0x7f00a01449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00a53f6700 1 -- 172.21.15.140:0/3478239019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00a00b2ec0 msgr2=0x7f00a00afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00a53f6700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00a00b2ec0 0x7f00a00afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00a53f6700 1 -- 172.21.15.140:0/3478239019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00a0142700 msgr2=0x7f00a01449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00a53f6700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00a0142700 0x7f00a01449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00a53f6700 1 -- 172.21.15.140:0/3478239019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c00b1e0 con 0x7f00a005d160 2022-07-21T01:56:12.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00a53f6700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00a005d160 0x7f00a01469d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f0094006940 tx=0x7f0094008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00927fc700 1 -- 172.21.15.140:0/3478239019 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0094011780 con 0x7f00a005d160 2022-07-21T01:56:12.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00927fc700 1 -- 172.21.15.140:0/3478239019 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00940118e0 con 0x7f00a005d160 2022-07-21T01:56:12.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00a63f8700 1 -- 172.21.15.140:0/3478239019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a01463b0 con 0x7f00a005d160 2022-07-21T01:56:12.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00927fc700 1 -- 172.21.15.140:0/3478239019 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0094011c40 con 0x7f00a005d160 2022-07-21T01:56:12.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00a63f8700 1 -- 172.21.15.140:0/3478239019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a00ad150 con 0x7f00a005d160 2022-07-21T01:56:12.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00a63f8700 1 -- 172.21.15.140:0/3478239019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a00b2620 con 0x7f00a005d160 2022-07-21T01:56:12.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00927fc700 1 -- 172.21.15.140:0/3478239019 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0094012420 con 0x7f00a005d160 2022-07-21T01:56:12.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00927fc700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0088053a10 0x7f0088055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00927fc700 1 -- 172.21.15.140:0/3478239019 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f009406bcd0 con 0x7f00a005d160 2022-07-21T01:56:12.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.365+0000 7f00a4bf5700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0088053a10 0x7f0088055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.369+0000 7f00a4bf5700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0088053a10 0x7f0088055ed0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f009c01b000 tx=0x7f009c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.373+0000 7f00927fc700 1 -- 172.21.15.140:0/3478239019 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0094042b90 con 0x7f00a005d160 2022-07-21T01:56:12.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.497+0000 7f00a63f8700 1 -- 172.21.15.140:0/3478239019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f00a00574e0 con 0x7f00a005d160 2022-07-21T01:56:12.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.497+0000 7f00927fc700 1 -- 172.21.15.140:0/3478239019 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7f00940461b0 con 0x7f00a005d160 2022-07-21T01:56:12.503 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:12.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 -- 172.21.15.140:0/3478239019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0088053a10 msgr2=0x7f0088055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0088053a10 0x7f0088055ed0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f009c01b000 tx=0x7f009c015000).stop 2022-07-21T01:56:12.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 -- 172.21.15.140:0/3478239019 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00a005d160 msgr2=0x7f00a01469d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00a005d160 0x7f00a01469d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f0094006940 tx=0x7f0094008040).stop 2022-07-21T01:56:12.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 -- 172.21.15.140:0/3478239019 shutdown_connections 2022-07-21T01:56:12.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0088053a10 0x7f0088055ed0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00a0142700 0x7f00a01449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f00a00b2ec0 0x7f00a00afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 --2- 172.21.15.140:0/3478239019 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f00a005d160 0x7f00a01469d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 -- 172.21.15.140:0/3478239019 >> 172.21.15.140:0/3478239019 conn(0x7f00a0066e20 msgr2=0x7f00a00b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:12.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 -- 172.21.15.140:0/3478239019 shutdown_connections 2022-07-21T01:56:12.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.501+0000 7f00a63f8700 1 -- 172.21.15.140:0/3478239019 wait complete. 2022-07-21T01:56:12.515 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:56:12.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ffac6e700 1 Processor -- start 2022-07-21T01:56:12.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ffac6e700 1 -- start start 2022-07-21T01:56:12.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ffac6e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0ff40b9da0 0x7f0ff41468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ffac6e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ff41419a0 0x7f0ff4141e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ffac6e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff40ba180 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ffac6e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff40ad5b0 con 0x7f0ff40b9da0 2022-07-21T01:56:12.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ffac6e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41461b0 con 0x7f0ff41419a0 2022-07-21T01:56:12.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff9c6c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0ff40b9da0 0x7f0ff41468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff9c6c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0ff40b9da0 0x7f0ff41468f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39024/0 (socket says 172.21.15.140:39024) 2022-07-21T01:56:12.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff9c6c700 1 -- 172.21.15.140:0/3633700868 learned_addr learned my addr 172.21.15.140:0/3633700868 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:12.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ffa46d700 1 --2- 172.21.15.140:0/3633700868 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ff41419a0 0x7f0ff4141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff9c6c700 1 -- 172.21.15.140:0/3633700868 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ff41419a0 msgr2=0x7f0ff4141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:12.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff9c6c700 1 --2- 172.21.15.140:0/3633700868 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ff41419a0 0x7f0ff4141e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff9c6c700 1 -- 172.21.15.140:0/3633700868 >> v1:172.21.15.140:6789/0 conn(0x7f0ff40b3cf0 legacy=0x7f0ff4142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff9c6c700 1 -- 172.21.15.140:0/3633700868 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff4146450 con 0x7f0ff40b9da0 2022-07-21T01:56:12.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff8c6a700 1 -- 172.21.15.140:0/3633700868 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067868373 0 0) 0x7f0ff40ba180 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff9c6c700 1 --2- 172.21.15.140:0/3633700868 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ff40b9da0 0x7f0ff41468f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f0fe8002f70 tx=0x7f0fe8006eb0).ready entity=mon.0 client_cookie=ee9428b347ab653f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff8c6a700 1 -- 172.21.15.140:0/3633700868 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fe800d570 con 0x7f0ff40b9da0 2022-07-21T01:56:12.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff8c6a700 1 -- 172.21.15.140:0/3633700868 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe8000e90 con 0x7f0ff40b9da0 2022-07-21T01:56:12.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ff8c6a700 1 -- 172.21.15.140:0/3633700868 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fe800d8c0 con 0x7f0ff40b9da0 2022-07-21T01:56:12.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.621+0000 7f0ffac6e700 1 -- 172.21.15.140:0/3633700868 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ff40b9da0 msgr2=0x7f0ff41468f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 --2- 172.21.15.140:0/3633700868 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ff40b9da0 0x7f0ff41468f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f0fe8002f70 tx=0x7f0fe8006eb0).stop 2022-07-21T01:56:12.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 -- 172.21.15.140:0/3633700868 shutdown_connections 2022-07-21T01:56:12.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 --2- 172.21.15.140:0/3633700868 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ff41419a0 0x7f0ff4141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 --2- 172.21.15.140:0/3633700868 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ff40b9da0 0x7f0ff41468f0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 -- 172.21.15.140:0/3633700868 >> 172.21.15.140:0/3633700868 conn(0x7f0ff40678c0 msgr2=0x7f0ff40b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:12.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 -- 172.21.15.140:0/3633700868 shutdown_connections 2022-07-21T01:56:12.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 -- 172.21.15.140:0/3633700868 wait complete. 2022-07-21T01:56:12.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 Processor -- start 2022-07-21T01:56:12.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 -- start start 2022-07-21T01:56:12.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ff40b3cf0 0x7f0ff413f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ff40b9da0 0x7f0ff413d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ff41419a0 0x7f0ff4151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4148ea0 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40b19a0 con 0x7f0ff40b9da0 2022-07-21T01:56:12.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ffac6e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40b1c40 con 0x7f0ff41419a0 2022-07-21T01:56:12.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ff9c6c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ff40b3cf0 0x7f0ff413f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ff9c6c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ff40b3cf0 0x7f0ff413f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39030/0 (socket says 172.21.15.140:39030) 2022-07-21T01:56:12.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.625+0000 7f0ff9c6c700 1 -- 172.21.15.140:0/638212201 learned_addr learned my addr 172.21.15.140:0/638212201 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:12.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ff946b700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ff40b9da0 0x7f0ff413d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ffa46d700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ff41419a0 0x7f0ff4151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ff9c6c700 1 -- 172.21.15.140:0/638212201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ff41419a0 msgr2=0x7f0ff4151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ff9c6c700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ff41419a0 0x7f0ff4151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ff9c6c700 1 -- 172.21.15.140:0/638212201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ff40b9da0 msgr2=0x7f0ff413d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ff9c6c700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ff40b9da0 0x7f0ff413d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ff9c6c700 1 -- 172.21.15.140:0/638212201 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe800b1e0 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ff9c6c700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ff40b3cf0 0x7f0ff413f7a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f0fe8002f70 tx=0x7f0fe8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0fe6ffd700 1 -- 172.21.15.140:0/638212201 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fe8011760 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0fe6ffd700 1 -- 172.21.15.140:0/638212201 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe80118c0 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ffac6e700 1 -- 172.21.15.140:0/638212201 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff413fc90 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0fe6ffd700 1 -- 172.21.15.140:0/638212201 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fe8011c20 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ffac6e700 1 -- 172.21.15.140:0/638212201 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff4141210 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ffac6e700 1 -- 172.21.15.140:0/638212201 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff4151b20 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0fe6ffd700 1 -- 172.21.15.140:0/638212201 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0fe8025030 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0fe6ffd700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fdc053a70 0x7f0fdc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0fe6ffd700 1 -- 172.21.15.140:0/638212201 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0fe806cfb0 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.629+0000 7f0ff946b700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fdc053a70 0x7f0fdc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.633+0000 7f0ff946b700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fdc053a70 0x7f0fdc055f30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f0ff000d000 tx=0x7f0ff0007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.637+0000 7f0fe6ffd700 1 -- 172.21.15.140:0/638212201 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0ff4151b20 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0ffac6e700 1 -- 172.21.15.140:0/638212201 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ff4151b20 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0fe6ffd700 1 -- 172.21.15.140:0/638212201 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0ff4151b20 con 0x7f0ff40b3cf0 2022-07-21T01:56:12.767 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:12.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0ffac6e700 1 -- 172.21.15.140:0/638212201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fdc053a70 msgr2=0x7f0fdc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0ffac6e700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fdc053a70 0x7f0fdc055f30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f0ff000d000 tx=0x7f0ff0007a00).stop 2022-07-21T01:56:12.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0ffac6e700 1 -- 172.21.15.140:0/638212201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ff40b3cf0 msgr2=0x7f0ff413f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0ffac6e700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ff40b3cf0 0x7f0ff413f7a0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f0fe8002f70 tx=0x7f0fe8015000).stop 2022-07-21T01:56:12.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0ffac6e700 1 -- 172.21.15.140:0/638212201 shutdown_connections 2022-07-21T01:56:12.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0ffac6e700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0fdc053a70 0x7f0fdc055f30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0ffac6e700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ff41419a0 0x7f0ff4151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0ffac6e700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ff40b9da0 0x7f0ff413d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0ffac6e700 1 --2- 172.21.15.140:0/638212201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0ff40b3cf0 0x7f0ff413f7a0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.761+0000 7f0ffac6e700 1 -- 172.21.15.140:0/638212201 >> 172.21.15.140:0/638212201 conn(0x7f0ff40678c0 msgr2=0x7f0ff40b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:12.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.765+0000 7f0ffac6e700 1 -- 172.21.15.140:0/638212201 shutdown_connections 2022-07-21T01:56:12.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.765+0000 7f0ffac6e700 1 -- 172.21.15.140:0/638212201 wait complete. 2022-07-21T01:56:12.779 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:56:12.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.881+0000 7f52d1797700 1 Processor -- start 2022-07-21T01:56:12.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.881+0000 7f52d1797700 1 -- start start 2022-07-21T01:56:12.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.881+0000 7f52d1797700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f52cc05c0d0 0x7f52cc0b3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.881+0000 7f52d1797700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f52cc1439c0 0x7f52cc142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.881+0000 7f52d1797700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc05c4b0 con 0x7f52cc05d200 2022-07-21T01:56:12.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.881+0000 7f52d1797700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc0b3da0 con 0x7f52cc1439c0 2022-07-21T01:56:12.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.881+0000 7f52d1797700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0b4040 con 0x7f52cc05c0d0 2022-07-21T01:56:12.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.881+0000 7f52d0f96700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f52cc1439c0 0x7f52cc142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.881+0000 7f52d0f96700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f52cc1439c0 0x7f52cc142ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39042/0 (socket says 172.21.15.140:39042) 2022-07-21T01:56:12.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.881+0000 7f52d0f96700 1 -- 172.21.15.140:0/2162777647 learned_addr learned my addr 172.21.15.140:0/2162777647 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:12.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.881+0000 7f52cbfff700 1 --2- 172.21.15.140:0/2162777647 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f52cc05c0d0 0x7f52cc0b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d0f96700 1 -- 172.21.15.140:0/2162777647 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f52cc05c0d0 msgr2=0x7f52cc0b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:12.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d0f96700 1 --2- 172.21.15.140:0/2162777647 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f52cc05c0d0 0x7f52cc0b3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d0f96700 1 -- 172.21.15.140:0/2162777647 >> v1:172.21.15.152:6789/0 conn(0x7f52cc05d200 legacy=0x7f52cc1432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d0f96700 1 -- 172.21.15.140:0/2162777647 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc146440 con 0x7f52cc1439c0 2022-07-21T01:56:12.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52cbfff700 1 --2- 172.21.15.140:0/2162777647 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f52cc05c0d0 0x7f52cc0b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:12.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d0f96700 1 --2- 172.21.15.140:0/2162777647 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f52cc1439c0 0x7f52cc142ad0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7f52c40032f0 tx=0x7f52c4009290).ready entity=mon.0 client_cookie=64858ea5669b9bc server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52caffd700 1 -- 172.21.15.140:0/2162777647 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52c400b8a0 con 0x7f52cc1439c0 2022-07-21T01:56:12.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52caffd700 1 -- 172.21.15.140:0/2162777647 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c400ba00 con 0x7f52cc1439c0 2022-07-21T01:56:12.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52caffd700 1 -- 172.21.15.140:0/2162777647 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52c400bb70 con 0x7f52cc1439c0 2022-07-21T01:56:12.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d1797700 1 -- 172.21.15.140:0/2162777647 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f52cc1439c0 msgr2=0x7f52cc142ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d1797700 1 --2- 172.21.15.140:0/2162777647 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f52cc1439c0 0x7f52cc142ad0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7f52c40032f0 tx=0x7f52c4009290).stop 2022-07-21T01:56:12.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d1797700 1 -- 172.21.15.140:0/2162777647 shutdown_connections 2022-07-21T01:56:12.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d1797700 1 --2- 172.21.15.140:0/2162777647 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f52cc1439c0 0x7f52cc142ad0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d1797700 1 --2- 172.21.15.140:0/2162777647 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f52cc05c0d0 0x7f52cc0b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d1797700 1 -- 172.21.15.140:0/2162777647 >> 172.21.15.140:0/2162777647 conn(0x7f52cc064b90 msgr2=0x7f52cc064f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:12.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d1797700 1 -- 172.21.15.140:0/2162777647 shutdown_connections 2022-07-21T01:56:12.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.885+0000 7f52d1797700 1 -- 172.21.15.140:0/2162777647 wait complete. 2022-07-21T01:56:12.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d1797700 1 Processor -- start 2022-07-21T01:56:12.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d1797700 1 -- start start 2022-07-21T01:56:12.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d1797700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f52cc05c0d0 0x7f52cc152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d1797700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f52cc05d200 0x7f52cc0b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d1797700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52cc1439c0 0x7f52cc143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d1797700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0afe40 con 0x7f52cc05d200 2022-07-21T01:56:12.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d1797700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc152f00 con 0x7f52cc1439c0 2022-07-21T01:56:12.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d1797700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0aefa0 con 0x7f52cc05c0d0 2022-07-21T01:56:12.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52cb7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f52cc05d200 0x7f52cc0b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52cb7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f52cc05d200 0x7f52cc0b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39046/0 (socket says 172.21.15.140:39046) 2022-07-21T01:56:12.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52cb7fe700 1 -- 172.21.15.140:0/2446945302 learned_addr learned my addr 172.21.15.140:0/2446945302 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:12.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52cbfff700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f52cc05c0d0 0x7f52cc152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d0f96700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52cc1439c0 0x7f52cc143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52cb7fe700 1 -- 172.21.15.140:0/2446945302 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f52cc05c0d0 msgr2=0x7f52cc152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52cb7fe700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f52cc05c0d0 0x7f52cc152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52cb7fe700 1 -- 172.21.15.140:0/2446945302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52cc1439c0 msgr2=0x7f52cc143fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:12.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52cb7fe700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52cc1439c0 0x7f52cc143fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:12.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52cb7fe700 1 -- 172.21.15.140:0/2446945302 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009530 con 0x7f52cc05d200 2022-07-21T01:56:12.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d0f96700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52cc1439c0 0x7f52cc143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:12.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52cb7fe700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f52cc05d200 0x7f52cc0b0920 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f52cc065830 tx=0x7f52bc007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52c97fa700 1 -- 172.21.15.140:0/2446945302 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52bc012400 con 0x7f52cc05d200 2022-07-21T01:56:12.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52c97fa700 1 -- 172.21.15.140:0/2446945302 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc012560 con 0x7f52cc05d200 2022-07-21T01:56:12.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d1797700 1 -- 172.21.15.140:0/2446945302 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc0b0070 con 0x7f52cc05d200 2022-07-21T01:56:12.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52c97fa700 1 -- 172.21.15.140:0/2446945302 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f52bc0128c0 con 0x7f52cc05d200 2022-07-21T01:56:12.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.889+0000 7f52d1797700 1 -- 172.21.15.140:0/2446945302 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc0aea20 con 0x7f52cc05d200 2022-07-21T01:56:12.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.893+0000 7f52d1797700 1 -- 172.21.15.140:0/2446945302 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc0574e0 con 0x7f52cc05d200 2022-07-21T01:56:12.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.893+0000 7f52c97fa700 1 -- 172.21.15.140:0/2446945302 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f52bc010070 con 0x7f52cc05d200 2022-07-21T01:56:12.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.893+0000 7f52c97fa700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f52b00539e0 0x7f52b0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:12.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.893+0000 7f52cbfff700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f52b00539e0 0x7f52b0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:12.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.893+0000 7f52c97fa700 1 -- 172.21.15.140:0/2446945302 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f52bc06b040 con 0x7f52cc05d200 2022-07-21T01:56:12.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.893+0000 7f52cbfff700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f52b00539e0 0x7f52b0055ea0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f52b400c000 tx=0x7f52b4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:12.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:12.897+0000 7f52c97fa700 1 -- 172.21.15.140:0/2446945302 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f52bc041e50 con 0x7f52cc05d200 2022-07-21T01:56:13.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.021+0000 7f52d1797700 1 -- 172.21.15.140:0/2446945302 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52cc0699b0 con 0x7f52cc05d200 2022-07-21T01:56:13.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.021+0000 7f52c97fa700 1 -- 172.21.15.140:0/2446945302 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+10 (secure 0 0 0) 0x7f52bc045470 con 0x7f52cc05d200 2022-07-21T01:56:13.029 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:13.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 -- 172.21.15.140:0/2446945302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f52b00539e0 msgr2=0x7f52b0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:13.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f52b00539e0 0x7f52b0055ea0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f52b400c000 tx=0x7f52b4008040).stop 2022-07-21T01:56:13.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 -- 172.21.15.140:0/2446945302 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f52cc05d200 msgr2=0x7f52cc0b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:13.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f52cc05d200 0x7f52cc0b0920 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f52cc065830 tx=0x7f52bc007c50).stop 2022-07-21T01:56:13.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 -- 172.21.15.140:0/2446945302 shutdown_connections 2022-07-21T01:56:13.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f52b00539e0 0x7f52b0055ea0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52cc1439c0 0x7f52cc143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f52cc05d200 0x7f52cc0b0920 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 --2- 172.21.15.140:0/2446945302 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f52cc05c0d0 0x7f52cc152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 -- 172.21.15.140:0/2446945302 >> 172.21.15.140:0/2446945302 conn(0x7f52cc064b90 msgr2=0x7f52cc065910 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:13.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 -- 172.21.15.140:0/2446945302 shutdown_connections 2022-07-21T01:56:13.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.025+0000 7f52d1797700 1 -- 172.21.15.140:0/2446945302 wait complete. 2022-07-21T01:56:13.046 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:56:13.047 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:13.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.145+0000 7f9bf759e700 1 Processor -- start 2022-07-21T01:56:13.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.145+0000 7f9bf759e700 1 -- start start 2022-07-21T01:56:13.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9bf80b5650 0x7f9bf80b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:13.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9bf80adff0 0x7f9bf8147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:13.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf8147010 con 0x7f9bf80af8e0 2022-07-21T01:56:13.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf81472b0 con 0x7f9bf80b5650 2022-07-21T01:56:13.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf8146d40 con 0x7f9bf80adff0 2022-07-21T01:56:13.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf659c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9bf80b5650 0x7f9bf80b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:13.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf659c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9bf80b5650 0x7f9bf80b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39052/0 (socket says 172.21.15.140:39052) 2022-07-21T01:56:13.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf659c700 1 -- 172.21.15.140:0/3863298447 learned_addr learned my addr 172.21.15.140:0/3863298447 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:13.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf5d9b700 1 --2- 172.21.15.140:0/3863298447 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9bf80adff0 0x7f9bf8147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:13.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf559a700 1 -- 172.21.15.140:0/3863298447 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795265512 0 0) 0x7f9bf8147010 con 0x7f9bf80af8e0 2022-07-21T01:56:13.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf559a700 1 -- 172.21.15.140:0/3863298447 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bdc003260 con 0x7f9bf80af8e0 2022-07-21T01:56:13.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf659c700 1 -- 172.21.15.140:0/3863298447 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9bf80adff0 msgr2=0x7f9bf8147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:13.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf659c700 1 --2- 172.21.15.140:0/3863298447 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9bf80adff0 0x7f9bf8147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf659c700 1 -- 172.21.15.140:0/3863298447 >> v1:172.21.15.152:6789/0 conn(0x7f9bf80af8e0 legacy=0x7f9bf805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:13.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf659c700 1 -- 172.21.15.140:0/3863298447 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf8147c40 con 0x7f9bf80b5650 2022-07-21T01:56:13.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf659c700 1 --2- 172.21.15.140:0/3863298447 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9bf80b5650 0x7f9bf80b5a30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f9be8002f70 tx=0x7f9be8006eb0).ready entity=mon.0 client_cookie=f526f723cdbeaa02 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:13.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf559a700 1 -- 172.21.15.140:0/3863298447 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9be800d5c0 con 0x7f9bf80b5650 2022-07-21T01:56:13.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf559a700 1 -- 172.21.15.140:0/3863298447 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8000e90 con 0x7f9bf80b5650 2022-07-21T01:56:13.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf559a700 1 -- 172.21.15.140:0/3863298447 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9be800d950 con 0x7f9bf80b5650 2022-07-21T01:56:13.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 -- 172.21.15.140:0/3863298447 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9bf80b5650 msgr2=0x7f9bf80b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:13.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 --2- 172.21.15.140:0/3863298447 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9bf80b5650 0x7f9bf80b5a30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f9be8002f70 tx=0x7f9be8006eb0).stop 2022-07-21T01:56:13.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 -- 172.21.15.140:0/3863298447 shutdown_connections 2022-07-21T01:56:13.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 --2- 172.21.15.140:0/3863298447 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9bf80adff0 0x7f9bf8147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 --2- 172.21.15.140:0/3863298447 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9bf80b5650 0x7f9bf80b5a30 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 -- 172.21.15.140:0/3863298447 >> 172.21.15.140:0/3863298447 conn(0x7f9bf80634e0 msgr2=0x7f9bf80b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:13.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 -- 172.21.15.140:0/3863298447 shutdown_connections 2022-07-21T01:56:13.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 -- 172.21.15.140:0/3863298447 wait complete. 2022-07-21T01:56:13.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 Processor -- start 2022-07-21T01:56:13.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.149+0000 7f9bf759e700 1 -- start start 2022-07-21T01:56:13.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9bf80adff0 0x7f9bf8148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:13.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9bf80af8e0 0x7f9bf8110e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:13.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bf80b5650 0x7f9bf8159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:13.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf8064170 con 0x7f9bf80af8e0 2022-07-21T01:56:13.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf8156df0 con 0x7f9bf80b5650 2022-07-21T01:56:13.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf8157090 con 0x7f9bf80adff0 2022-07-21T01:56:13.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf5d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9bf80af8e0 0x7f9bf8110e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:13.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf5d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9bf80af8e0 0x7f9bf8110e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39058/0 (socket says 172.21.15.140:39058) 2022-07-21T01:56:13.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf5d9b700 1 -- 172.21.15.140:0/3532129039 learned_addr learned my addr 172.21.15.140:0/3532129039 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:13.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf6d9d700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bf80b5650 0x7f9bf8159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:13.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf659c700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9bf80adff0 0x7f9bf8148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:13.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf5d9b700 1 -- 172.21.15.140:0/3532129039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9bf80adff0 msgr2=0x7f9bf8148a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:13.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf5d9b700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9bf80adff0 0x7f9bf8148a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf5d9b700 1 -- 172.21.15.140:0/3532129039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bf80b5650 msgr2=0x7f9bf8159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:13.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf5d9b700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bf80b5650 0x7f9bf8159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf5d9b700 1 -- 172.21.15.140:0/3532129039 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be800b1e0 con 0x7f9bf80af8e0 2022-07-21T01:56:13.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf5d9b700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9bf80af8e0 0x7f9bf8110e70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f9be000c000 tx=0x7f9be0006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:13.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bef7fe700 1 -- 172.21.15.140:0/3532129039 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9be000be70 con 0x7f9bf80af8e0 2022-07-21T01:56:13.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bef7fe700 1 -- 172.21.15.140:0/3532129039 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be00104c0 con 0x7f9bf80af8e0 2022-07-21T01:56:13.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf759e700 1 -- 172.21.15.140:0/3532129039 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf81496b0 con 0x7f9bf80af8e0 2022-07-21T01:56:13.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bef7fe700 1 -- 172.21.15.140:0/3532129039 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9be0010820 con 0x7f9bf80af8e0 2022-07-21T01:56:13.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf759e700 1 -- 172.21.15.140:0/3532129039 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf8111360 con 0x7f9bf80af8e0 2022-07-21T01:56:13.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.153+0000 7f9bf759e700 1 -- 172.21.15.140:0/3532129039 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf8110c10 con 0x7f9bf80af8e0 2022-07-21T01:56:13.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.157+0000 7f9bef7fe700 1 -- 172.21.15.140:0/3532129039 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9be001a030 con 0x7f9bf80af8e0 2022-07-21T01:56:13.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.157+0000 7f9bef7fe700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9bd0053a70 0x7f9bd0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:13.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.157+0000 7f9bef7fe700 1 -- 172.21.15.140:0/3532129039 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9be006b9b0 con 0x7f9bf80af8e0 2022-07-21T01:56:13.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.157+0000 7f9bf659c700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9bd0053a70 0x7f9bd0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:13.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.157+0000 7f9bf659c700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9bd0053a70 0x7f9bd0055f30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f9be801c000 tx=0x7f9be8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:13.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.161+0000 7f9bef7fe700 1 -- 172.21.15.140:0/3532129039 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9be0042920 con 0x7f9bf80af8e0 2022-07-21T01:56:13.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.329+0000 7f9bf759e700 1 -- 172.21.15.140:0/3532129039 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bf805c4a0 con 0x7f9bf80af8e0 2022-07-21T01:56:13.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.329+0000 7f9bef7fe700 1 -- 172.21.15.140:0/3532129039 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f9be0045f40 con 0x7f9bf80af8e0 2022-07-21T01:56:13.335 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:13.335 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":47,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/3024179649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":3024179649},{"type":"v1","addr":"172.21.15.140:6835","nonce":3024179649}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:56:06.173419+0000","modified":"2022-07-21T01:56:07.189859+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/727644549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":727644549},{"type":"v1","addr":"172.21.15.152:6833","nonce":727644549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-07-21T01:56:13.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 -- 172.21.15.140:0/3532129039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9bd0053a70 msgr2=0x7f9bd0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:13.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9bd0053a70 0x7f9bd0055f30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f9be801c000 tx=0x7f9be8015000).stop 2022-07-21T01:56:13.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 -- 172.21.15.140:0/3532129039 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9bf80af8e0 msgr2=0x7f9bf8110e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:13.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9bf80af8e0 0x7f9bf8110e70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f9be000c000 tx=0x7f9be0006d90).stop 2022-07-21T01:56:13.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 -- 172.21.15.140:0/3532129039 shutdown_connections 2022-07-21T01:56:13.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9bd0053a70 0x7f9bd0055f30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bf80b5650 0x7f9bf8159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9bf80af8e0 0x7f9bf8110e70 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 --2- 172.21.15.140:0/3532129039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9bf80adff0 0x7f9bf8148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:13.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 -- 172.21.15.140:0/3532129039 >> 172.21.15.140:0/3532129039 conn(0x7f9bf80634e0 msgr2=0x7f9bf80ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:13.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 -- 172.21.15.140:0/3532129039 shutdown_connections 2022-07-21T01:56:13.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:13.333+0000 7f9bf759e700 1 -- 172.21.15.140:0/3532129039 wait complete. 2022-07-21T01:56:13.342 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 47 2022-07-21T01:56:13.550 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:56:06.173419+0000', 'modified': '2022-07-21T01:56:07.189859+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7335}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7335': {'gid': 7335, 'name': 'b', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/727644549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 727644549}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 727644549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:56:13.550 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:56:13.551 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:56:13.667 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":7335,"want_state":"up:active","state":"up:active","rank_uptime":7.4808803780000002,"mdsmap_epoch":47,"osdmap_epoch":103,"osdmap_epoch_barrier":102,"uptime":11.848206615} 2022-07-21T01:56:13.667 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 7335, "mdsmap_epoch": 47, "osdmap_epoch": 103, "osdmap_epoch_barrier": 102, "rank_uptime": 7.480880378, "state": "up:active", "uptime": 11.848206615, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:56:13.668 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:56:13.669 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-07-21T01:56:13.669 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-07-21T01:56:13.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:13.670 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:13.680 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:56:13.750 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:56:13.751 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:56:13.765 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:56:13.766 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:56:13.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:13.768 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:56:13.776 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:56:13.776 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:13.823 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:56:13.843 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:56:13.844 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:56:13.859 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:56:13.860 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:56:13.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:13.862 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:56:13.899 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2022-07-21T01:56:13.894+0000 7f86eeffe080 -1 init, newargv = 0x55ee68bb5510 newargc=9ceph-fuse[22651]: starting ceph client 2022-07-21T01:56:13.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr: 2022-07-21T01:56:13.908 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22651]: starting fuse 2022-07-21T01:56:15.008 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:56:15.027 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:56:15.028 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:56:15.082 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:56:15.083 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:56:15.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:15.084 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:56:15.090 INFO:teuthology.orchestra.run.smithi192.stdout:52 2022-07-21T01:56:15.091 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:56:15.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:15.093 DEBUG:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2022-07-21T01:56:15.094 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2022-07-21T01:56:15.095 DEBUG:teuthology.orchestra.run.smithi192:> import re 2022-07-21T01:56:15.096 DEBUG:teuthology.orchestra.run.smithi192:> import os 2022-07-21T01:56:15.096 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2022-07-21T01:56:15.098 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:56:15.098 DEBUG:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2022-07-21T01:56:15.099 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-07-21T01:56:15.099 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2022-07-21T01:56:15.100 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:56:15.100 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2022-07-21T01:56:15.101 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2022-07-21T01:56:15.101 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2022-07-21T01:56:15.102 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2022-07-21T01:56:15.102 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:56:15.103 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2022-07-21T01:56:15.103 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:56:15.104 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:56:15.104 DEBUG:teuthology.orchestra.run.smithi192:> return f 2022-07-21T01:56:15.105 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:56:15.105 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:56:15.105 DEBUG:teuthology.orchestra.run.smithi192:> print(find_socket("client.1")) 2022-07-21T01:56:15.106 DEBUG:teuthology.orchestra.run.smithi192:> ' 2022-07-21T01:56:15.151 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.22651.asok 2022-07-21T01:56:15.156 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22651.asok 2022-07-21T01:56:15.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:15.157 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22651.asok status 2022-07-21T01:56:15.286 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2022-07-21T01:56:15.287 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2022-07-21T01:56:15.287 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:56:15.287 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:56:15.288 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2022-07-21T01:56:15.288 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2022-07-21T01:56:15.288 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-07-21T01:56:15.289 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "22651", 2022-07-21T01:56:15.289 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2022-07-21T01:56:15.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:56:15.289 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2022-07-21T01:56:15.290 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2022-07-21T01:56:15.290 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 6626, 2022-07-21T01:56:15.290 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2022-07-21T01:56:15.290 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2022-07-21T01:56:15.291 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2022-07-21T01:56:15.291 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 6626 2022-07-21T01:56:15.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:56:15.291 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:56:15.291 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:56:15.292 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:56:15.292 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2058455205 2022-07-21T01:56:15.292 INFO:teuthology.orchestra.run.smithi192.stdout: } 2022-07-21T01:56:15.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:56:15.293 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:56:15.293 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:56:15.293 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:56:15.293 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2058455205 2022-07-21T01:56:15.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:56:15.294 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.6626 172.21.15.192:0/2058455205", 2022-07-21T01:56:15.294 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/2058455205", 2022-07-21T01:56:15.294 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2022-07-21T01:56:15.294 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 47, 2022-07-21T01:56:15.295 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 103, 2022-07-21T01:56:15.295 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:56:15.295 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false, 2022-07-21T01:56:15.295 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2022-07-21T01:56:15.296 INFO:teuthology.orchestra.run.smithi192.stdout:} 2022-07-21T01:56:15.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:15.297 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:15.298 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:15.306 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:56:15.307 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:56:15.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:15.308 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:15.325 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-07-21T01:56:15.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.425+0000 7fd5af59e700 1 Processor -- start 2022-07-21T01:56:15.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.425+0000 7fd5af59e700 1 -- start start 2022-07-21T01:56:15.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.425+0000 7fd5af59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00b0570 0x7fd5b00b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.425+0000 7fd5af59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd5b00aea10 0x7fd5b0141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.425+0000 7fd5af59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b0142750 con 0x7fd5b01422c0 2022-07-21T01:56:15.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.425+0000 7fd5af59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b0143b10 con 0x7fd5b00aea10 2022-07-21T01:56:15.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.425+0000 7fd5af59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b01433d0 con 0x7fd5b00b0570 2022-07-21T01:56:15.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.425+0000 7fd5aed9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd5b01422c0 0x7fd5b00ad350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53216/0 (socket says 172.21.15.140:53216) 2022-07-21T01:56:15.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.425+0000 7fd5aed9d700 1 -- 172.21.15.140:0/2367854711 learned_addr learned my addr 172.21.15.140:0/2367854711 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:15.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.425+0000 7fd5add9b700 1 --2- 172.21.15.140:0/2367854711 >> v2:172.21.15.152:3300/0 conn(0x7fd5b00aea10 0x7fd5b0141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.425+0000 7fd5ae59c700 1 --2- 172.21.15.140:0/2367854711 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00b0570 0x7fd5b00b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5ad59a700 1 -- 172.21.15.140:0/2367854711 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052821942 0 0) 0x7fd5b0142750 con 0x7fd5b01422c0 2022-07-21T01:56:15.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5ad59a700 1 -- 172.21.15.140:0/2367854711 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd590003260 con 0x7fd5b01422c0 2022-07-21T01:56:15.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5add9b700 1 -- 172.21.15.140:0/2367854711 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00b0570 msgr2=0x7fd5b00b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:15.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5add9b700 1 --2- 172.21.15.140:0/2367854711 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00b0570 0x7fd5b00b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5add9b700 1 -- 172.21.15.140:0/2367854711 >> v1:172.21.15.140:6789/0 conn(0x7fd5b01422c0 legacy=0x7fd5b00ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:15.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5add9b700 1 -- 172.21.15.140:0/2367854711 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b0143670 con 0x7fd5b00aea10 2022-07-21T01:56:15.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5add9b700 1 --2- 172.21.15.140:0/2367854711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5b00aea10 0x7fd5b0141dd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fd5a0002f70 tx=0x7fd5a0006df0).ready entity=mon.1 client_cookie=209f2b2ee11bebb3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:15.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5ad59a700 1 -- 172.21.15.140:0/2367854711 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5a000d4a0 con 0x7fd5b00aea10 2022-07-21T01:56:15.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5ad59a700 1 -- 172.21.15.140:0/2367854711 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a0000e90 con 0x7fd5b00aea10 2022-07-21T01:56:15.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5ad59a700 1 -- 172.21.15.140:0/2367854711 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5a000d7f0 con 0x7fd5b00aea10 2022-07-21T01:56:15.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5af59e700 1 -- 172.21.15.140:0/2367854711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5b00aea10 msgr2=0x7fd5b0141dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:15.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5af59e700 1 --2- 172.21.15.140:0/2367854711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5b00aea10 0x7fd5b0141dd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fd5a0002f70 tx=0x7fd5a0006df0).stop 2022-07-21T01:56:15.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5af59e700 1 -- 172.21.15.140:0/2367854711 shutdown_connections 2022-07-21T01:56:15.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5af59e700 1 --2- 172.21.15.140:0/2367854711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5b00aea10 0x7fd5b0141dd0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5af59e700 1 --2- 172.21.15.140:0/2367854711 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00b0570 0x7fd5b00b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5af59e700 1 -- 172.21.15.140:0/2367854711 >> 172.21.15.140:0/2367854711 conn(0x7fd5b00b9360 msgr2=0x7fd5b00b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:15.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5af59e700 1 -- 172.21.15.140:0/2367854711 shutdown_connections 2022-07-21T01:56:15.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.429+0000 7fd5af59e700 1 -- 172.21.15.140:0/2367854711 wait complete. 2022-07-21T01:56:15.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5af59e700 1 Processor -- start 2022-07-21T01:56:15.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5af59e700 1 -- start start 2022-07-21T01:56:15.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5af59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00aea10 0x7fd5b00b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5af59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5b00b0570 0x7fd5b013fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5af59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5b01422c0 0x7fd5b0140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5af59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b01135c0 con 0x7fd5b01422c0 2022-07-21T01:56:15.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5af59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b0113100 con 0x7fd5b00b0570 2022-07-21T01:56:15.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5af59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b0113320 con 0x7fd5b00aea10 2022-07-21T01:56:15.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5aed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5b01422c0 0x7fd5b0140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5ae59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00aea10 0x7fd5b00b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5ae59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00aea10 0x7fd5b00b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:35802/0 (socket says 172.21.15.140:35802) 2022-07-21T01:56:15.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5ae59c700 1 -- 172.21.15.140:0/654921626 learned_addr learned my addr 172.21.15.140:0/654921626 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:15.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5aed9d700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5b01422c0 0x7fd5b0140140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39076/0 (socket says 172.21.15.140:39076) 2022-07-21T01:56:15.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5add9b700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5b00b0570 0x7fd5b013fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5ae59c700 1 -- 172.21.15.140:0/654921626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5b00b0570 msgr2=0x7fd5b013fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:15.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5ae59c700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5b00b0570 0x7fd5b013fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5ae59c700 1 -- 172.21.15.140:0/654921626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5b01422c0 msgr2=0x7fd5b0140140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:15.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5ae59c700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5b01422c0 0x7fd5b0140140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.433+0000 7fd5ae59c700 1 -- 172.21.15.140:0/654921626 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a000b180 con 0x7fd5b00aea10 2022-07-21T01:56:15.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5ae59c700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00aea10 0x7fd5b00b38b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fd5b0113240 tx=0x7fd598006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:15.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5a77fe700 1 -- 172.21.15.140:0/654921626 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd598010780 con 0x7fd5b00aea10 2022-07-21T01:56:15.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5a77fe700 1 -- 172.21.15.140:0/654921626 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5980108e0 con 0x7fd5b00aea10 2022-07-21T01:56:15.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5af59e700 1 -- 172.21.15.140:0/654921626 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b00b22b0 con 0x7fd5b00aea10 2022-07-21T01:56:15.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5a77fe700 1 -- 172.21.15.140:0/654921626 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd598010c40 con 0x7fd5b00aea10 2022-07-21T01:56:15.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5af59e700 1 -- 172.21.15.140:0/654921626 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b00b1d30 con 0x7fd5b00aea10 2022-07-21T01:56:15.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5a77fe700 1 -- 172.21.15.140:0/654921626 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd59801c460 con 0x7fd5b00aea10 2022-07-21T01:56:15.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5af59e700 1 -- 172.21.15.140:0/654921626 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd588003410 con 0x7fd5b00aea10 2022-07-21T01:56:15.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5a77fe700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd594053b00 0x7fd594055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5a77fe700 1 -- 172.21.15.140:0/654921626 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd59806c480 con 0x7fd5b00aea10 2022-07-21T01:56:15.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5add9b700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd594053b00 0x7fd594055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.437+0000 7fd5add9b700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd594053b00 0x7fd594055fc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fd5a001a000 tx=0x7fd5a0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:15.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.449+0000 7fd5a77fe700 1 -- 172.21.15.140:0/654921626 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd5980431f0 con 0x7fd5b00aea10 2022-07-21T01:56:15.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.617+0000 7fd5af59e700 1 -- 172.21.15.140:0/654921626 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7fd5880036c0 con 0x7fd5b00aea10 2022-07-21T01:56:15.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5a77fe700 1 -- 172.21.15.140:0/654921626 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1940 (secure 0 0 0) 0x7fd598046810 con 0x7fd5b00aea10 2022-07-21T01:56:15.624 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:15.625 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:56:15.625 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:15.625 INFO:teuthology.orchestra.run.smithi140.stdout: "total_bytes": 767585320960, 2022-07-21T01:56:15.626 INFO:teuthology.orchestra.run.smithi140.stdout: "total_avail_bytes": 761053749248, 2022-07-21T01:56:15.626 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_bytes": 6531530752, 2022-07-21T01:56:15.626 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_bytes": 6531571712, 2022-07-21T01:56:15.627 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_ratio": 0.0085092457011342049, 2022-07-21T01:56:15.627 INFO:teuthology.orchestra.run.smithi140.stdout: "num_osds": 8, 2022-07-21T01:56:15.627 INFO:teuthology.orchestra.run.smithi140.stdout: "num_per_pool_osds": 0, 2022-07-21T01:56:15.628 INFO:teuthology.orchestra.run.smithi140.stdout: "num_per_pool_omap_osds": 0 2022-07-21T01:56:15.628 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:15.628 INFO:teuthology.orchestra.run.smithi140.stdout: "stats_by_class": { 2022-07-21T01:56:15.628 INFO:teuthology.orchestra.run.smithi140.stdout: "ssd": { 2022-07-21T01:56:15.629 INFO:teuthology.orchestra.run.smithi140.stdout: "total_bytes": 767585320960, 2022-07-21T01:56:15.629 INFO:teuthology.orchestra.run.smithi140.stdout: "total_avail_bytes": 761053749248, 2022-07-21T01:56:15.629 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_bytes": 6531530752, 2022-07-21T01:56:15.629 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_bytes": 6531571712, 2022-07-21T01:56:15.630 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_ratio": 0.0085092457011342049 2022-07-21T01:56:15.630 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:15.630 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:15.630 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:56:15.631 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:15.631 INFO:teuthology.orchestra.run.smithi140.stdout: "name": "device_health_metrics", 2022-07-21T01:56:15.631 INFO:teuthology.orchestra.run.smithi140.stdout: "id": 1, 2022-07-21T01:56:15.631 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:15.631 INFO:teuthology.orchestra.run.smithi140.stdout: "stored": 0, 2022-07-21T01:56:15.632 INFO:teuthology.orchestra.run.smithi140.stdout: "objects": 2, 2022-07-21T01:56:15.632 INFO:teuthology.orchestra.run.smithi140.stdout: "kb_used": 0, 2022-07-21T01:56:15.632 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes_used": 0, 2022-07-21T01:56:15.632 INFO:teuthology.orchestra.run.smithi140.stdout: "percent_used": 0, 2022-07-21T01:56:15.633 INFO:teuthology.orchestra.run.smithi140.stdout: "max_avail": 361336832000 2022-07-21T01:56:15.633 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:15.633 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:15.633 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:15.633 INFO:teuthology.orchestra.run.smithi140.stdout: "name": "rbd", 2022-07-21T01:56:15.634 INFO:teuthology.orchestra.run.smithi140.stdout: "id": 2, 2022-07-21T01:56:15.634 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:15.634 INFO:teuthology.orchestra.run.smithi140.stdout: "stored": 0, 2022-07-21T01:56:15.634 INFO:teuthology.orchestra.run.smithi140.stdout: "objects": 0, 2022-07-21T01:56:15.635 INFO:teuthology.orchestra.run.smithi140.stdout: "kb_used": 0, 2022-07-21T01:56:15.635 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes_used": 0, 2022-07-21T01:56:15.635 INFO:teuthology.orchestra.run.smithi140.stdout: "percent_used": 0, 2022-07-21T01:56:15.635 INFO:teuthology.orchestra.run.smithi140.stdout: "max_avail": 361336832000 2022-07-21T01:56:15.636 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:15.636 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:15.636 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:15.636 INFO:teuthology.orchestra.run.smithi140.stdout: "name": "cephfs_metadata", 2022-07-21T01:56:15.637 INFO:teuthology.orchestra.run.smithi140.stdout: "id": 19, 2022-07-21T01:56:15.637 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:15.637 INFO:teuthology.orchestra.run.smithi140.stdout: "stored": 3524, 2022-07-21T01:56:15.637 INFO:teuthology.orchestra.run.smithi140.stdout: "objects": 22, 2022-07-21T01:56:15.638 INFO:teuthology.orchestra.run.smithi140.stdout: "kb_used": 4, 2022-07-21T01:56:15.638 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes_used": 3524, 2022-07-21T01:56:15.638 INFO:teuthology.orchestra.run.smithi140.stdout: "percent_used": 4.8763366677917475e-09, 2022-07-21T01:56:15.639 INFO:teuthology.orchestra.run.smithi140.stdout: "max_avail": 361336832000 2022-07-21T01:56:15.639 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:15.639 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:15.639 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:15.640 INFO:teuthology.orchestra.run.smithi140.stdout: "name": "cephfs_data", 2022-07-21T01:56:15.640 INFO:teuthology.orchestra.run.smithi140.stdout: "id": 20, 2022-07-21T01:56:15.640 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:15.640 INFO:teuthology.orchestra.run.smithi140.stdout: "stored": 0, 2022-07-21T01:56:15.640 INFO:teuthology.orchestra.run.smithi140.stdout: "objects": 0, 2022-07-21T01:56:15.641 INFO:teuthology.orchestra.run.smithi140.stdout: "kb_used": 0, 2022-07-21T01:56:15.641 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes_used": 0, 2022-07-21T01:56:15.641 INFO:teuthology.orchestra.run.smithi140.stdout: "percent_used": 0, 2022-07-21T01:56:15.641 INFO:teuthology.orchestra.run.smithi140.stdout: "max_avail": 361336832000 2022-07-21T01:56:15.642 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:15.642 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:15.642 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:15.642 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:56:15.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 -- 172.21.15.140:0/654921626 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd594053b00 msgr2=0x7fd594055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:15.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd594053b00 0x7fd594055fc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fd5a001a000 tx=0x7fd5a0014000).stop 2022-07-21T01:56:15.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 -- 172.21.15.140:0/654921626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00aea10 msgr2=0x7fd5b00b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:15.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00aea10 0x7fd5b00b38b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fd5b0113240 tx=0x7fd598006df0).stop 2022-07-21T01:56:15.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 -- 172.21.15.140:0/654921626 shutdown_connections 2022-07-21T01:56:15.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd594053b00 0x7fd594055fc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd5b01422c0 0x7fd5b0140140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5b00b0570 0x7fd5b013fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 --2- 172.21.15.140:0/654921626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5b00aea10 0x7fd5b00b38b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 -- 172.21.15.140:0/654921626 >> 172.21.15.140:0/654921626 conn(0x7fd5b00b9360 msgr2=0x7fd5b00b8490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:15.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 -- 172.21.15.140:0/654921626 shutdown_connections 2022-07-21T01:56:15.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.621+0000 7fd5af59e700 1 -- 172.21.15.140:0/654921626 wait complete. 2022-07-21T01:56:15.647 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data size -f json-pretty 2022-07-21T01:56:15.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.745+0000 7fdeff267700 1 Processor -- start 2022-07-21T01:56:15.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.745+0000 7fdeff267700 1 -- start start 2022-07-21T01:56:15.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.745+0000 7fdeff267700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdef805c960 0x7fdef81465e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.745+0000 7fdeff267700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdef80b2230 0x7fdef80618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.745+0000 7fdeff267700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdef8061db0 0x7fdef80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef805cd40 con 0x7fdef8061db0 2022-07-21T01:56:15.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef8060400 con 0x7fdef80b2230 2022-07-21T01:56:15.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8061440 con 0x7fdef805c960 2022-07-21T01:56:15.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefea66700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdef8061db0 0x7fdef80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefea66700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdef8061db0 0x7fdef80b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39084/0 (socket says 172.21.15.140:39084) 2022-07-21T01:56:15.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefea66700 1 -- 172.21.15.140:0/893195519 learned_addr learned my addr 172.21.15.140:0/893195519 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:15.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefe265700 1 --2- 172.21.15.140:0/893195519 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdef805c960 0x7fdef81465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefda64700 1 --2- 172.21.15.140:0/893195519 >> v2:172.21.15.152:3300/0 conn(0x7fdef80b2230 0x7fdef80618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefea66700 1 -- 172.21.15.140:0/893195519 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdef805c960 msgr2=0x7fdef81465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:15.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefea66700 1 --2- 172.21.15.140:0/893195519 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdef805c960 0x7fdef81465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefea66700 1 -- 172.21.15.140:0/893195519 >> v2:172.21.15.152:3300/0 conn(0x7fdef80b2230 msgr2=0x7fdef80618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:15.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefea66700 1 --2- 172.21.15.140:0/893195519 >> v2:172.21.15.152:3300/0 conn(0x7fdef80b2230 0x7fdef80618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefea66700 1 -- 172.21.15.140:0/893195519 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8060680 con 0x7fdef8061db0 2022-07-21T01:56:15.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefda64700 1 --2- 172.21.15.140:0/893195519 >> v2:172.21.15.152:3300/0 conn(0x7fdef80b2230 0x7fdef80618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:15.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefea66700 1 --2- 172.21.15.140:0/893195519 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdef8061db0 0x7fdef80b0b90 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fdef00032f0 tx=0x7fdef0009290).ready entity=mon.0 client_cookie=10282815b81ee8f0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:15.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefd263700 1 -- 172.21.15.140:0/893195519 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdef000b870 con 0x7fdef8061db0 2022-07-21T01:56:15.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefd263700 1 -- 172.21.15.140:0/893195519 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef000b9d0 con 0x7fdef8061db0 2022-07-21T01:56:15.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdefd263700 1 -- 172.21.15.140:0/893195519 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdef000bb40 con 0x7fdef8061db0 2022-07-21T01:56:15.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 -- 172.21.15.140:0/893195519 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdef8061db0 msgr2=0x7fdef80b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:15.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 --2- 172.21.15.140:0/893195519 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdef8061db0 0x7fdef80b0b90 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fdef00032f0 tx=0x7fdef0009290).stop 2022-07-21T01:56:15.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 -- 172.21.15.140:0/893195519 shutdown_connections 2022-07-21T01:56:15.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 --2- 172.21.15.140:0/893195519 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdef8061db0 0x7fdef80b0b90 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 --2- 172.21.15.140:0/893195519 >> v2:172.21.15.152:3300/0 conn(0x7fdef80b2230 0x7fdef80618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 --2- 172.21.15.140:0/893195519 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdef805c960 0x7fdef81465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 -- 172.21.15.140:0/893195519 >> 172.21.15.140:0/893195519 conn(0x7fdef80b61f0 msgr2=0x7fdef80b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:15.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 -- 172.21.15.140:0/893195519 shutdown_connections 2022-07-21T01:56:15.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 -- 172.21.15.140:0/893195519 wait complete. 2022-07-21T01:56:15.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 Processor -- start 2022-07-21T01:56:15.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.749+0000 7fdeff267700 1 -- start start 2022-07-21T01:56:15.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeff267700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdef805c960 0x7fdef8067310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeff267700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef8061db0 0x7fdef8067800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeff267700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdef80b2230 0x7fdef8065490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeff267700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8067cf0 con 0x7fdef80b2230 2022-07-21T01:56:15.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeff267700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8152d00 con 0x7fdef8061db0 2022-07-21T01:56:15.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeff267700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8152fa0 con 0x7fdef805c960 2022-07-21T01:56:15.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdefea66700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdef80b2230 0x7fdef8065490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdefea66700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdef80b2230 0x7fdef8065490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39088/0 (socket says 172.21.15.140:39088) 2022-07-21T01:56:15.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdefea66700 1 -- 172.21.15.140:0/250626429 learned_addr learned my addr 172.21.15.140:0/250626429 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:15.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdefe265700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdef805c960 0x7fdef8067310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdefda64700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef8061db0 0x7fdef8067800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdefea66700 1 -- 172.21.15.140:0/250626429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdef805c960 msgr2=0x7fdef8067310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:15.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdefea66700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdef805c960 0x7fdef8067310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdefea66700 1 -- 172.21.15.140:0/250626429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef8061db0 msgr2=0x7fdef8067800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:15.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdefea66700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef8061db0 0x7fdef8067800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdefea66700 1 -- 172.21.15.140:0/250626429 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef0009530 con 0x7fdef80b2230 2022-07-21T01:56:15.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdefea66700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdef80b2230 0x7fdef8065490 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7fdef00032f0 tx=0x7fdef0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:15.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeeb7fe700 1 -- 172.21.15.140:0/250626429 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdef000b870 con 0x7fdef80b2230 2022-07-21T01:56:15.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeeb7fe700 1 -- 172.21.15.140:0/250626429 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef0020430 con 0x7fdef80b2230 2022-07-21T01:56:15.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeff267700 1 -- 172.21.15.140:0/250626429 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef8152a20 con 0x7fdef80b2230 2022-07-21T01:56:15.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeeb7fe700 1 -- 172.21.15.140:0/250626429 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdef0020790 con 0x7fdef80b2230 2022-07-21T01:56:15.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeff267700 1 -- 172.21.15.140:0/250626429 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef80ad850 con 0x7fdef80b2230 2022-07-21T01:56:15.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.753+0000 7fdeff267700 1 -- 172.21.15.140:0/250626429 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef8153160 con 0x7fdef80b2230 2022-07-21T01:56:15.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.757+0000 7fdeeb7fe700 1 -- 172.21.15.140:0/250626429 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdef000f070 con 0x7fdef80b2230 2022-07-21T01:56:15.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.757+0000 7fdeeb7fe700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fded80539e0 0x7fded8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:15.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.757+0000 7fdeeb7fe700 1 -- 172.21.15.140:0/250626429 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdef006d120 con 0x7fdef80b2230 2022-07-21T01:56:15.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.757+0000 7fdefe265700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fded80539e0 0x7fded8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:15.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.757+0000 7fdefe265700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fded80539e0 0x7fded8055ea0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fdeec00b000 tx=0x7fdeec006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:15.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.761+0000 7fdeeb7fe700 1 -- 172.21.15.140:0/250626429 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdef0043f30 con 0x7fdef80b2230 2022-07-21T01:56:15.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.885+0000 7fdeff267700 1 -- 172.21.15.140:0/250626429 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"} v 0) v1 -- 0x7fdef8062260 con 0x7fdef80b2230 2022-07-21T01:56:15.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeeb7fe700 1 -- 172.21.15.140:0/250626429 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"}]=0 v103) v1 ==== 123+0+64 (secure 0 0 0) 0x7fdef0015070 con 0x7fdef80b2230 2022-07-21T01:56:15.893 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:15.894 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:56:15.894 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": "cephfs_data", 2022-07-21T01:56:15.895 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_id": 20, 2022-07-21T01:56:15.895 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2 2022-07-21T01:56:15.896 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:56:15.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 -- 172.21.15.140:0/250626429 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fded80539e0 msgr2=0x7fded8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:15.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fded80539e0 0x7fded8055ea0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fdeec00b000 tx=0x7fdeec006df0).stop 2022-07-21T01:56:15.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 -- 172.21.15.140:0/250626429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdef80b2230 msgr2=0x7fdef8065490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:15.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdef80b2230 0x7fdef8065490 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7fdef00032f0 tx=0x7fdef0016000).stop 2022-07-21T01:56:15.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 -- 172.21.15.140:0/250626429 shutdown_connections 2022-07-21T01:56:15.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fded80539e0 0x7fded8055ea0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdef80b2230 0x7fdef8065490 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef8061db0 0x7fdef8067800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 --2- 172.21.15.140:0/250626429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdef805c960 0x7fdef8067310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:15.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 -- 172.21.15.140:0/250626429 >> 172.21.15.140:0/250626429 conn(0x7fdef80b61f0 msgr2=0x7fdef80b5170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:15.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 -- 172.21.15.140:0/250626429 shutdown_connections 2022-07-21T01:56:15.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:15.889+0000 7fdeff267700 1 -- 172.21.15.140:0/250626429 wait complete. 2022-07-21T01:56:15.906 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-07-21T01:56:15.907 DEBUG:teuthology.orchestra.run.smithi191:> cd /home/ubuntu/cephtest/mnt.0 && sudo df . 2022-07-21T01:56:15.942 INFO:teuthology.orchestra.run.smithi191.stdout:Filesystem 1K-blocks Used Available Use% Mounted on 2022-07-21T01:56:15.942 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-fuse 352866304 0 352866304 0% /home/ubuntu/cephtest/mnt.0 2022-07-21T01:56:15.944 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:56:15.944 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:15.960 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:15.982 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:16.003 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:16.025 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:16.047 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:56:16.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:16.048 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:16.049 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:16.066 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:56:16.068 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:56:16.069 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:56:16.069 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:56:16.070 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:16.126 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23126]: fuse finished with error 0 and tester_r 0 2022-07-21T01:56:16.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:16.130 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:16.130 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:16.136 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:56:16.137 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:56:16.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:56:16.255 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:16.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:16.303 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:16.304 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:16.312 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:56:16.313 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:56:16.314 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2022-07-21T01:56:16.314 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:56:16.315 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:16.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22651]: fuse finished with error 0 and tester_r 0 2022-07-21T01:56:16.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:16.333 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:16.333 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:16.383 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2022-07-21T01:56:16.384 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:56:16.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:56:16.496 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:16.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:16.504 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:16.505 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:56:16.512 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:56:16.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:56:16.513 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:56:16.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:56:16.515 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:56:16.521 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:56:16.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:56:16.523 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:56:16.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024ffff700 1 Processor -- start 2022-07-21T01:56:16.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024ffff700 1 -- start start 2022-07-21T01:56:16.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f02500b5650 0x7f02500b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:16.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02500af8e0 0x7f025005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:16.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024ffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0250147010 con 0x7f02500adff0 2022-07-21T01:56:16.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024ffff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02501472b0 con 0x7f02500b5650 2022-07-21T01:56:16.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0250146d40 con 0x7f02500af8e0 2022-07-21T01:56:16.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024e7fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f02500adff0 0x7f0250147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53240/0 (socket says 172.21.15.140:53240) 2022-07-21T01:56:16.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024e7fc700 1 -- 172.21.15.140:0/218984320 learned_addr learned my addr 172.21.15.140:0/218984320 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:16.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024effd700 1 --2- 172.21.15.140:0/218984320 >> v2:172.21.15.152:3300/0 conn(0x7f02500b5650 0x7f02500b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:16.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024f7fe700 1 --2- 172.21.15.140:0/218984320 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02500af8e0 0x7f025005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:16.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024dffb700 1 -- 172.21.15.140:0/218984320 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391849905 0 0) 0x7f0250147010 con 0x7f02500adff0 2022-07-21T01:56:16.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024dffb700 1 -- 172.21.15.140:0/218984320 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0230003260 con 0x7f02500adff0 2022-07-21T01:56:16.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024effd700 1 -- 172.21.15.140:0/218984320 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02500af8e0 msgr2=0x7f025005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:16.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024effd700 1 --2- 172.21.15.140:0/218984320 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02500af8e0 0x7f025005bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:16.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024effd700 1 -- 172.21.15.140:0/218984320 >> v1:172.21.15.140:6789/0 conn(0x7f02500adff0 legacy=0x7f0250147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:16.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024dffb700 1 -- 172.21.15.140:0/218984320 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1580005808 0 0) 0x7f0230003260 con 0x7f02500adff0 2022-07-21T01:56:16.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.629+0000 7f024effd700 1 -- 172.21.15.140:0/218984320 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02500acae0 con 0x7f02500b5650 2022-07-21T01:56:16.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024effd700 1 --2- 172.21.15.140:0/218984320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02500b5650 0x7f02500b5a30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f0238001980 tx=0x7f0240006df0).ready entity=mon.1 client_cookie=b517967c6ad5fc6f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:16.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024dffb700 1 -- 172.21.15.140:0/218984320 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02380017e0 con 0x7f02500b5650 2022-07-21T01:56:16.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024dffb700 1 -- 172.21.15.140:0/218984320 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0240000e90 con 0x7f02500b5650 2022-07-21T01:56:16.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024dffb700 1 -- 172.21.15.140:0/218984320 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0240012460 con 0x7f02500b5650 2022-07-21T01:56:16.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 -- 172.21.15.140:0/218984320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02500b5650 msgr2=0x7f02500b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:16.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 --2- 172.21.15.140:0/218984320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02500b5650 0x7f02500b5a30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f0238001980 tx=0x7f0240006df0).stop 2022-07-21T01:56:16.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 -- 172.21.15.140:0/218984320 shutdown_connections 2022-07-21T01:56:16.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 --2- 172.21.15.140:0/218984320 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02500af8e0 0x7f025005bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:16.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 --2- 172.21.15.140:0/218984320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02500b5650 0x7f02500b5a30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:16.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 -- 172.21.15.140:0/218984320 >> 172.21.15.140:0/218984320 conn(0x7f02500634e0 msgr2=0x7f02500b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:16.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 -- 172.21.15.140:0/218984320 shutdown_connections 2022-07-21T01:56:16.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 -- 172.21.15.140:0/218984320 wait complete. 2022-07-21T01:56:16.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 Processor -- start 2022-07-21T01:56:16.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 -- start start 2022-07-21T01:56:16.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02500adff0 0x7f0250149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:16.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02500af8e0 0x7f0250156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:16.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02500b5650 0x7f0250159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:16.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02501564d0 con 0x7f02500adff0 2022-07-21T01:56:16.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0250156700 con 0x7f02500b5650 2022-07-21T01:56:16.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0250064f10 con 0x7f02500af8e0 2022-07-21T01:56:16.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02500adff0 0x7f0250149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:16.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024effd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02500adff0 0x7f0250149510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39100/0 (socket says 172.21.15.140:39100) 2022-07-21T01:56:16.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024effd700 1 -- 172.21.15.140:0/1962059037 learned_addr learned my addr 172.21.15.140:0/1962059037 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:16.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024e7fc700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02500af8e0 0x7f0250156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:16.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.633+0000 7f024f7fe700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02500b5650 0x7f0250159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:16.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f024effd700 1 -- 172.21.15.140:0/1962059037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02500af8e0 msgr2=0x7f0250156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:16.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f024effd700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02500af8e0 0x7f0250156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:16.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f024effd700 1 -- 172.21.15.140:0/1962059037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02500b5650 msgr2=0x7f0250159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:16.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f024effd700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02500b5650 0x7f0250159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:16.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f024effd700 1 -- 172.21.15.140:0/1962059037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024000b180 con 0x7f02500adff0 2022-07-21T01:56:16.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f024effd700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02500adff0 0x7f0250149510 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f0238001980 tx=0x7f0240014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:16.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f0247fff700 1 -- 172.21.15.140:0/1962059037 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f024000d8a0 con 0x7f02500adff0 2022-07-21T01:56:16.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f024ffff700 1 -- 172.21.15.140:0/1962059037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0250063b20 con 0x7f02500adff0 2022-07-21T01:56:16.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f0247fff700 1 -- 172.21.15.140:0/1962059037 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024000da00 con 0x7f02500adff0 2022-07-21T01:56:16.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f0247fff700 1 -- 172.21.15.140:0/1962059037 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f024000dd60 con 0x7f02500adff0 2022-07-21T01:56:16.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f024ffff700 1 -- 172.21.15.140:0/1962059037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025010f8e0 con 0x7f02500adff0 2022-07-21T01:56:16.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f024ffff700 1 -- 172.21.15.140:0/1962059037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025005c280 con 0x7f02500adff0 2022-07-21T01:56:16.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f0247fff700 1 -- 172.21.15.140:0/1962059037 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0240025030 con 0x7f02500adff0 2022-07-21T01:56:16.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.637+0000 7f0247fff700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0234053950 0x7f0234055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:16.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.641+0000 7f0247fff700 1 -- 172.21.15.140:0/1962059037 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f024006cab0 con 0x7f02500adff0 2022-07-21T01:56:16.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.641+0000 7f024e7fc700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0234053950 0x7f0234055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:16.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.641+0000 7f024e7fc700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0234053950 0x7f0234055e10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f023800d000 tx=0x7f02380013b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:16.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.645+0000 7f0247fff700 1 -- 172.21.15.140:0/1962059037 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0240043930 con 0x7f02500adff0 2022-07-21T01:56:16.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.769+0000 7f024ffff700 1 -- 172.21.15.140:0/1962059037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f02500574e0 con 0x7f02500adff0 2022-07-21T01:56:16.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.773+0000 7f0247fff700 1 -- 172.21.15.140:0/1962059037 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) v1 ==== 81+0+33755 (secure 0 0 0) 0x7f0240023020 con 0x7f02500adff0 2022-07-21T01:56:16.779 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:16.779 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:56:16.779 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 103, 2022-07-21T01:56:16.779 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:56:16.780 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:56:16.780 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:56:07.189453+0000", 2022-07-21T01:56:16.780 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:56:16.780 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:56:16.781 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:56:16.781 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:56:16.781 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:56:16.781 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:56:16.782 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:56:16.782 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:56:16.782 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:56:16.782 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:16.783 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:56:16.783 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:56:16.783 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:56:16.783 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:56:16.784 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:56:16.784 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 20, 2022-07-21T01:56:16.784 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:56:16.784 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:56:16.785 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:56:16.785 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:56:16.785 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:56:16.785 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.786 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:56:16.786 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:56:16.786 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:56:16.786 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:56:16.787 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:56:16.787 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:56:16.787 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:56:16.787 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:56:16.787 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:56:16.788 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:56:16.788 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:56:16.788 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:56:16.789 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:56:16.789 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:56:16.789 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:56:16.789 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:56:16.790 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:56:16.790 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:56:16.790 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:56:16.790 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:56:16.791 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:56:16.791 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:56:16.791 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:56:16.791 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.791 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:56:16.792 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:56:16.792 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:56:16.792 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:56:16.792 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:56:16.793 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:56:16.793 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:56:16.793 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:56:16.793 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:56:16.794 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:56:16.794 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:56:16.794 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:56:16.794 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:56:16.794 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:56:16.795 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:56:16.795 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:56:16.795 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:56:16.795 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:56:16.796 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:56:16.796 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:56:16.796 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:56:16.796 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:56:16.797 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:56:16.797 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:56:16.797 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:56:16.797 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:56:16.797 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:56:16.798 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.798 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:56:16.798 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:56:16.798 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:56:16.799 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:56:16.799 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:56:16.799 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:56:16.799 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:56:16.800 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:56:16.800 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:56:16.800 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:56:16.800 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:56:16.801 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:56:16.801 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:56:16.801 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.801 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:56:16.801 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:56:16.802 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.802 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.802 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.802 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:56:16.803 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:56:16.803 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:56:16.803 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:56:16.803 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:56:16.804 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:56:16.804 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:56:16.804 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:56:16.804 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:56:16.805 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:56:16.805 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:56:16.805 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:56:16.805 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:56:16.805 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:56:16.806 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:56:16.806 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:56:16.806 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:56:16.806 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:56:16.807 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:56:16.807 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:56:16.807 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:56:16.807 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:56:16.808 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:56:16.808 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.808 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:56:16.808 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:56:16.808 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:56:16.809 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:56:16.809 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:56:16.809 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:56:16.809 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:56:16.810 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:56:16.810 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:56:16.810 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:56:16.810 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:56:16.811 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:56:16.811 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:56:16.811 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:56:16.811 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:56:16.812 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:56:16.812 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:56:16.812 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:56:16.812 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:56:16.813 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:56:16.813 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:56:16.813 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:56:16.813 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:56:16.814 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:56:16.814 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:56:16.814 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:56:16.814 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:56:16.814 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.815 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:56:16.815 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:56:16.815 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:56:16.815 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:56:16.816 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:56:16.816 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:56:16.816 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:56:16.816 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:56:16.816 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:56:16.817 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:56:16.817 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:56:16.817 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:56:16.817 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:56:16.818 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:56:16.818 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.818 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.818 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.819 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 19, 2022-07-21T01:56:16.819 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:56:16.819 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:56:04.557618+0000", 2022-07-21T01:56:16.819 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:56:16.820 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:56:16.820 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:56:16.820 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:56:16.820 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:56:16.821 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:56:16.821 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:56:16.821 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:56:16.821 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:56:16.822 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:56:16.822 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:56:16.822 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:56:16.822 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:56:16.823 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:56:16.823 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:56:16.823 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:56:16.823 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:56:16.823 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:56:16.824 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:56:16.824 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:56:16.824 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.824 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "102", 2022-07-21T01:56:16.825 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:56:16.825 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:56:16.825 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:56:16.825 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:56:16.826 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:56:16.826 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:56:16.826 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:56:16.826 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:56:16.827 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:56:16.827 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:56:16.827 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:56:16.827 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:56:16.828 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:56:16.828 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:56:16.828 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:56:16.828 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:56:16.829 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:56:16.829 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:56:16.829 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:56:16.829 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:56:16.829 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:56:16.830 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:56:16.830 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:56:16.830 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:56:16.830 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:56:16.831 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:56:16.831 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.831 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:56:16.831 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:56:16.832 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:56:16.832 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:56:16.832 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:56:16.832 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:56:16.832 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:56:16.833 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:56:16.833 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:56:16.833 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:56:16.833 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:56:16.834 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:56:16.834 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:56:16.834 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:56:16.834 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:56:16.834 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.835 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:56:16.835 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:56:16.835 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:56:16.835 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.836 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.836 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.836 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.836 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 20, 2022-07-21T01:56:16.837 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:56:16.837 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:56:05.505410+0000", 2022-07-21T01:56:16.837 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:56:16.837 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:56:16.838 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:56:16.838 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:56:16.838 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:56:16.838 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:56:16.839 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:56:16.839 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:56:16.839 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:56:16.839 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:56:16.840 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:56:16.840 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:56:16.840 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:56:16.840 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:56:16.840 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:56:16.841 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:56:16.841 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:56:16.841 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:56:16.841 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:56:16.842 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:56:16.842 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.842 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "103", 2022-07-21T01:56:16.842 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:56:16.843 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:56:16.843 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:56:16.843 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:56:16.843 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:56:16.843 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:56:16.844 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:56:16.844 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:56:16.844 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:56:16.844 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:56:16.845 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:56:16.845 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:56:16.845 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:56:16.845 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:56:16.845 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:56:16.846 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:56:16.846 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:56:16.846 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:56:16.846 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:56:16.847 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:56:16.847 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:56:16.847 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:56:16.847 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:56:16.848 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:56:16.848 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:56:16.848 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:56:16.848 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.849 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:56:16.849 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:56:16.849 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:56:16.849 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:56:16.850 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:56:16.850 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:56:16.850 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:56:16.851 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:56:16.851 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:56:16.851 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:56:16.851 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:56:16.851 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:56:16.852 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:56:16.852 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:56:16.852 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:56:16.853 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.853 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.853 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.853 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:16.853 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:56:16.854 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.854 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:56:16.854 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:56:16.855 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:16.855 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:16.855 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:16.855 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:16.856 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:16.856 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:16.856 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:16.856 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 101, 2022-07-21T01:56:16.857 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:16.857 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:16.857 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:16.857 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.858 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.858 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.858 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:56:16.858 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:16.859 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.859 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.859 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.859 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:56:16.859 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:16.860 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.860 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.860 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.860 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:16.861 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.861 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.861 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.861 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:56:16.862 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:16.862 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.862 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.862 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.863 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:56:16.863 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:16.863 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.863 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.863 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.864 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:16.864 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.864 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.864 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.865 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:56:16.865 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:16.865 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.865 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.865 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.866 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:56:16.866 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:16.866 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.866 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.867 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.867 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:16.867 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.867 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.868 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.868 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:56:16.868 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:16.868 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.868 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.869 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.869 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:56:16.869 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:16.869 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.870 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.870 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.870 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:56:16.870 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:56:16.871 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:56:16.871 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:56:16.871 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:16.871 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:16.871 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:16.872 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.872 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.872 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.872 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:56:16.873 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:56:16.873 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:16.873 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:16.873 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:16.874 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:16.874 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:16.874 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:16.874 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:16.875 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 101, 2022-07-21T01:56:16.875 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:16.875 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:16.875 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:16.876 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.876 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.876 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.876 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:56:16.876 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:16.877 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.877 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.877 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.877 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:56:16.877 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:16.878 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.878 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.878 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.878 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:16.879 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.879 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.879 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.879 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:56:16.879 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:16.880 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.880 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.880 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.880 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:56:16.881 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:16.881 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.881 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.881 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.882 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:16.882 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.882 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.882 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.882 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:56:16.883 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:16.883 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.883 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.883 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.884 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:56:16.884 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:16.884 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.885 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.885 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.885 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:16.885 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.885 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.886 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.886 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:56:16.886 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:16.886 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.887 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.887 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.887 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:56:16.887 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:16.888 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.888 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.888 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.888 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:56:16.889 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:56:16.889 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:56:16.889 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:56:16.889 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:16.890 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:16.890 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:16.890 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.890 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.891 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.891 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:56:16.891 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:56:16.891 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:16.891 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:16.892 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:16.892 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:16.892 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:16.892 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:16.893 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:16.893 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 100, 2022-07-21T01:56:16.893 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:16.893 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:16.894 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:16.894 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.894 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.894 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.895 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:56:16.895 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:16.895 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.895 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.896 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.896 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:56:16.896 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:16.896 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.897 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.897 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.897 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:16.897 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.898 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.898 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.898 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:56:16.898 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:16.899 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.899 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.899 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.899 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:56:16.900 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:16.900 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.900 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.900 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.900 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:16.901 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.901 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.901 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.901 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:56:16.902 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:16.902 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.902 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.902 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.903 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:56:16.903 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:16.903 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.903 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.904 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.904 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:16.904 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.904 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.905 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.905 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:56:16.905 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:16.905 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.905 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.906 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.906 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:56:16.906 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:16.906 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.907 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.907 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.907 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:56:16.907 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:56:16.908 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:56:16.908 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:56:16.908 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:16.908 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:16.909 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:16.909 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.909 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.909 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.910 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:56:16.910 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:56:16.910 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:16.910 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:16.911 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:16.911 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:16.911 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:16.912 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:16.912 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:16.912 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 101, 2022-07-21T01:56:16.912 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:16.913 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:16.913 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:16.913 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.913 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.914 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.914 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:56:16.914 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:16.915 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.915 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.915 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.915 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:56:16.916 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:16.916 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.916 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.916 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.917 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:16.917 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.917 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.917 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.918 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:56:16.918 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:16.918 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.918 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.919 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.919 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:56:16.919 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:16.919 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.920 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.920 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.920 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:16.920 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.921 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.921 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.921 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:56:16.921 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:16.921 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.922 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.922 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.922 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:56:16.922 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:16.923 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.923 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.923 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.923 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:16.924 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.924 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.924 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.924 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:56:16.925 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:16.925 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.925 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.925 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.926 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:56:16.926 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:16.926 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.926 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.927 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.927 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:56:16.927 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:56:16.927 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:56:16.928 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:56:16.928 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:16.928 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:16.928 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:16.929 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.929 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.929 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.929 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:56:16.930 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:56:16.930 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:16.930 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:16.930 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:16.931 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:16.931 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:16.931 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:16.931 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:16.932 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 100, 2022-07-21T01:56:16.932 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:16.932 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:16.932 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:16.933 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.933 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.933 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.933 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:56:16.934 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:16.934 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.934 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.934 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.935 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:56:16.935 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:16.935 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.935 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.936 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.936 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:16.936 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.936 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.937 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.937 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:56:16.937 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:16.937 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.938 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.938 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.938 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:56:16.938 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:16.938 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.939 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.939 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.939 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:16.939 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.940 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.940 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.940 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:56:16.940 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:16.941 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.941 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.941 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.941 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:56:16.942 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:16.942 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.942 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.942 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.943 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:16.943 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.943 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.944 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.944 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:56:16.944 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:16.944 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.945 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.945 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.945 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:56:16.945 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:16.946 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.946 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.946 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.946 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:56:16.947 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:56:16.947 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:56:16.947 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:56:16.947 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:16.948 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:16.948 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:16.948 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.948 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.948 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.949 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:56:16.949 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:56:16.949 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:16.949 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:16.950 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:16.950 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:16.950 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:16.950 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:16.951 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:16.951 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 100, 2022-07-21T01:56:16.951 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:16.951 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.952 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.953 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:56:16.953 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:16.953 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.953 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.954 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.954 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:56:16.954 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:16.954 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.955 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.955 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.955 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:16.955 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.955 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.956 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.956 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:56:16.956 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:16.956 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.957 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.957 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.957 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:56:16.957 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:16.957 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.958 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.958 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.958 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:16.958 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.959 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.959 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.959 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:56:16.959 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:16.960 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.960 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.960 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.960 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:56:16.961 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:16.961 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.961 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.961 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.962 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:16.962 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.962 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.962 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.962 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:56:16.963 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:16.963 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.963 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.963 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.964 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:56:16.964 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:16.964 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.964 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.965 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.965 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:56:16.965 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:56:16.965 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:56:16.966 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:56:16.966 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:16.966 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:16.966 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:16.966 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.967 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.967 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.967 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:56:16.967 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:56:16.968 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:16.968 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:16.968 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:16.968 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:16.969 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:16.969 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:16.969 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:16.969 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 101, 2022-07-21T01:56:16.970 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:16.970 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:16.970 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:16.970 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.971 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.971 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.971 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:56:16.971 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:16.972 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.972 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.972 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.972 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:56:16.973 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:16.973 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.973 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.974 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.974 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:16.974 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.974 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.975 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.975 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:56:16.975 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:16.976 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.976 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.976 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.976 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:56:16.977 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:16.977 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.977 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.977 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.978 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:16.978 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.978 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.978 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.979 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:56:16.979 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:16.979 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.979 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.980 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.980 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:56:16.980 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:16.980 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.980 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.981 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.981 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:16.981 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.981 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.982 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.982 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:56:16.982 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:16.982 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.983 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.983 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.983 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:56:16.983 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:16.984 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.984 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.984 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.984 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:56:16.984 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:56:16.985 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:56:16.985 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:56:16.985 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:16.985 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:16.986 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:16.986 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.986 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.986 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.986 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:56:16.987 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:56:16.987 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:16.987 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:16.987 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:16.988 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:16.988 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:16.988 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:16.988 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:16.989 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 101, 2022-07-21T01:56:16.989 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:16.989 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:16.989 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:16.990 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.990 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.990 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.990 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:56:16.991 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:16.991 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.991 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.991 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.991 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:56:16.992 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:16.992 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.992 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.992 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.993 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:16.993 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.993 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.993 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.994 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:56:16.994 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:16.994 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.994 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.995 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.995 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:56:16.995 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:16.995 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.996 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.996 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.996 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:16.996 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:16.997 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.997 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:16.997 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:56:16.997 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:16.998 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:16.998 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:16.998 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:16.998 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:56:16.999 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:16.999 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:16.999 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:16.999 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:17.000 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:17.000 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:17.000 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:17.000 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:17.000 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:56:17.001 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:17.001 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:17.001 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:17.001 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:17.002 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:56:17.002 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:17.002 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:17.002 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:17.003 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:17.003 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:56:17.003 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:56:17.003 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:56:17.004 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:56:17.004 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:17.004 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:17.004 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:17.005 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:17.005 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:17.005 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:17.005 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:56:17.006 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:17.006 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:56:17.006 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:17.007 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:17.007 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:17.007 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:17.007 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:17.008 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:56:17.008 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:17.008 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:17.008 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:17.009 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:56:17.009 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:17.009 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:17.009 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:17.010 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:17.010 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:17.010 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:56:17.010 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:17.011 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:17.011 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:17.011 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:56:17.011 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:17.011 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:17.012 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:17.012 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:17.012 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:17.012 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:56:17.013 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:17.013 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:17.013 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:17.013 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:56:17.014 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:17.014 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:17.014 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:17.014 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:17.015 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:17.015 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:56:17.015 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:17.015 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:17.016 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:17.016 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:56:17.016 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:17.016 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:17.017 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:17.017 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:17.017 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:17.017 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:56:17.017 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:17.018 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:17.018 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:17.018 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:56:17.018 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:17.019 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:17.019 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:17.019 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:17.019 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:17.020 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:56:17.020 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:17.020 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:17.020 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:17.021 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:56:17.021 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:17.021 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:17.021 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:17.022 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:17.022 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:17.022 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:56:17.022 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:17.023 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:17.023 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:17.023 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:56:17.023 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:17.024 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:17.024 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:17.024 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:17.024 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:17.025 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:56:17.025 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:17.025 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:17.025 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:17.026 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:56:17.026 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:56:17.026 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:56:17.026 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:56:17.027 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": {}, 2022-07-21T01:56:17.027 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:56:17.027 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:56:17.027 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:56:17.028 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:56:17.028 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:56:17.028 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:56:17.028 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:56:17.028 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:56:17.029 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:17.029 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:17.029 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:56:17.029 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:56:17.030 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:56:17.030 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:56:17.030 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:56:17.030 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:56:17.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.773+0000 7f024ffff700 1 -- 172.21.15.140:0/1962059037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0234053950 msgr2=0x7f0234055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.773+0000 7f024ffff700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0234053950 0x7f0234055e10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f023800d000 tx=0x7f02380013b0).stop 2022-07-21T01:56:17.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.773+0000 7f024ffff700 1 -- 172.21.15.140:0/1962059037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02500adff0 msgr2=0x7f0250149510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.773+0000 7f024ffff700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02500adff0 0x7f0250149510 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f0238001980 tx=0x7f0240014000).stop 2022-07-21T01:56:17.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.777+0000 7f024ffff700 1 -- 172.21.15.140:0/1962059037 shutdown_connections 2022-07-21T01:56:17.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.777+0000 7f024ffff700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0234053950 0x7f0234055e10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.777+0000 7f024ffff700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02500b5650 0x7f0250159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.777+0000 7f024ffff700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f02500af8e0 0x7f0250156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.777+0000 7f024ffff700 1 --2- 172.21.15.140:0/1962059037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f02500adff0 0x7f0250149510 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.777+0000 7f024ffff700 1 -- 172.21.15.140:0/1962059037 >> 172.21.15.140:0/1962059037 conn(0x7f02500634e0 msgr2=0x7f025014da30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:17.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.777+0000 7f024ffff700 1 -- 172.21.15.140:0/1962059037 shutdown_connections 2022-07-21T01:56:17.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:16.777+0000 7f024ffff700 1 -- 172.21.15.140:0/1962059037 wait complete. 2022-07-21T01:56:17.036 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:17.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7759e700 1 Processor -- start 2022-07-21T01:56:17.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7759e700 1 -- start start 2022-07-21T01:56:17.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7759e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9c780b3ca0 0x7f9c780b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7759e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9c7813cb70 0x7f9c78143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7759e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c78142ba0 con 0x7f9c7813c790 2022-07-21T01:56:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7759e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c781429b0 con 0x7f9c780b3ca0 2022-07-21T01:56:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7759e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c78146330 con 0x7f9c7813cb70 2022-07-21T01:56:17.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c75d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9c780b3ca0 0x7f9c780b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c76d9d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9c7813cb70 0x7f9c78143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c75d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9c780b3ca0 0x7f9c780b4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39110/0 (socket says 172.21.15.140:39110) 2022-07-21T01:56:17.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c75d9b700 1 -- 172.21.15.140:0/2635027946 learned_addr learned my addr 172.21.15.140:0/2635027946 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:17.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c76d9d700 1 -- 172.21.15.140:0/2635027946 >> v2:172.21.15.140:3300/0 conn(0x7f9c780b3ca0 msgr2=0x7f9c780b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:17.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c76d9d700 1 --2- 172.21.15.140:0/2635027946 >> v2:172.21.15.140:3300/0 conn(0x7f9c780b3ca0 0x7f9c780b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c76d9d700 1 -- 172.21.15.140:0/2635027946 >> v1:172.21.15.152:6789/0 conn(0x7f9c7813c790 legacy=0x7f9c780b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7559a700 1 -- 172.21.15.140:0/2635027946 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460054642 0 0) 0x7f9c78142ba0 con 0x7f9c7813c790 2022-07-21T01:56:17.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c76d9d700 1 -- 172.21.15.140:0/2635027946 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c780ba120 con 0x7f9c7813cb70 2022-07-21T01:56:17.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c75d9b700 1 --2- 172.21.15.140:0/2635027946 >> v2:172.21.15.140:3300/0 conn(0x7f9c780b3ca0 0x7f9c780b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:17.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c76d9d700 1 --2- 172.21.15.140:0/2635027946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7813cb70 0x7f9c78143000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f9c70002f70 tx=0x7f9c70006df0).ready entity=mon.1 client_cookie=e63858f9e1d522e7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:17.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7559a700 1 -- 172.21.15.140:0/2635027946 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c7000d570 con 0x7f9c7813cb70 2022-07-21T01:56:17.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7559a700 1 -- 172.21.15.140:0/2635027946 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c70000e90 con 0x7f9c7813cb70 2022-07-21T01:56:17.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7559a700 1 -- 172.21.15.140:0/2635027946 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c7000d8c0 con 0x7f9c7813cb70 2022-07-21T01:56:17.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7759e700 1 -- 172.21.15.140:0/2635027946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7813cb70 msgr2=0x7f9c78143000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.141+0000 7f9c7759e700 1 --2- 172.21.15.140:0/2635027946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7813cb70 0x7f9c78143000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f9c70002f70 tx=0x7f9c70006df0).stop 2022-07-21T01:56:17.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 -- 172.21.15.140:0/2635027946 shutdown_connections 2022-07-21T01:56:17.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 --2- 172.21.15.140:0/2635027946 >> v2:172.21.15.140:3300/0 conn(0x7f9c780b3ca0 0x7f9c780b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 --2- 172.21.15.140:0/2635027946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7813cb70 0x7f9c78143000 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 -- 172.21.15.140:0/2635027946 >> 172.21.15.140:0/2635027946 conn(0x7f9c780678b0 msgr2=0x7f9c780b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:17.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 -- 172.21.15.140:0/2635027946 shutdown_connections 2022-07-21T01:56:17.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 -- 172.21.15.140:0/2635027946 wait complete. 2022-07-21T01:56:17.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 Processor -- start 2022-07-21T01:56:17.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 -- start start 2022-07-21T01:56:17.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c780b3ca0 0x7f9c780ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c7813c790 0x7f9c780aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7813cb70 0x7f9c780af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7813a810 con 0x7f9c780b3ca0 2022-07-21T01:56:17.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780b1710 con 0x7f9c7813cb70 2022-07-21T01:56:17.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c780b1940 con 0x7f9c7813c790 2022-07-21T01:56:17.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c780b3ca0 0x7f9c780ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c780b3ca0 0x7f9c780ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39114/0 (socket says 172.21.15.140:39114) 2022-07-21T01:56:17.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7659c700 1 -- 172.21.15.140:0/1854203215 learned_addr learned my addr 172.21.15.140:0/1854203215 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:17.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c75d9b700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c7813c790 0x7f9c780aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c76d9d700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7813cb70 0x7f9c780af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7659c700 1 -- 172.21.15.140:0/1854203215 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c7813c790 msgr2=0x7f9c780aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7659c700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c7813c790 0x7f9c780aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7659c700 1 -- 172.21.15.140:0/1854203215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7813cb70 msgr2=0x7f9c780af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7659c700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7813cb70 0x7f9c780af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7659c700 1 -- 172.21.15.140:0/1854203215 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6800b1e0 con 0x7f9c780b3ca0 2022-07-21T01:56:17.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7659c700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c780b3ca0 0x7f9c780ae740 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f9c60001250 tx=0x7f9c60007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:17.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c6f7fe700 1 -- 172.21.15.140:0/1854203215 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c6000e4a0 con 0x7f9c780b3ca0 2022-07-21T01:56:17.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c6f7fe700 1 -- 172.21.15.140:0/1854203215 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6000e600 con 0x7f9c780b3ca0 2022-07-21T01:56:17.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c6f7fe700 1 -- 172.21.15.140:0/1854203215 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c6000e960 con 0x7f9c780b3ca0 2022-07-21T01:56:17.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 -- 172.21.15.140:0/1854203215 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7000b180 con 0x7f9c780b3ca0 2022-07-21T01:56:17.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.145+0000 7f9c7759e700 1 -- 172.21.15.140:0/1854203215 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7813fb20 con 0x7f9c780b3ca0 2022-07-21T01:56:17.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.149+0000 7f9c6f7fe700 1 -- 172.21.15.140:0/1854203215 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9c60019030 con 0x7f9c780b3ca0 2022-07-21T01:56:17.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.149+0000 7f9c6f7fe700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c5c053aa0 0x7f9c5c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.149+0000 7f9c6f7fe700 1 -- 172.21.15.140:0/1854203215 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9c6006ba50 con 0x7f9c780b3ca0 2022-07-21T01:56:17.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.149+0000 7f9c75d9b700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c5c053aa0 0x7f9c5c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.149+0000 7f9c7759e700 1 -- 172.21.15.140:0/1854203215 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7813fe30 con 0x7f9c780b3ca0 2022-07-21T01:56:17.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.149+0000 7f9c75d9b700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c5c053aa0 0x7f9c5c055f60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f9c6800b000 tx=0x7f9c68006eb0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:17.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.153+0000 7f9c6f7fe700 1 -- 172.21.15.140:0/1854203215 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c60042860 con 0x7f9c780b3ca0 2022-07-21T01:56:17.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.313+0000 7f9c7759e700 1 -- 172.21.15.140:0/1854203215 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c780b1b10 con 0x7f9c780b3ca0 2022-07-21T01:56:17.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.313+0000 7f9c6f7fe700 1 -- 172.21.15.140:0/1854203215 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+2143 (secure 0 0 0) 0x7f9c60045e80 con 0x7f9c780b3ca0 2022-07-21T01:56:17.320 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:17.321 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":47,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/3024179649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":3024179649},{"type":"v1","addr":"172.21.15.140:6835","nonce":3024179649}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:56:06.173419+0000","modified":"2022-07-21T01:56:07.189859+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7335},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7335":{"gid":7335,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/727644549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":727644549},{"type":"v1","addr":"172.21.15.152:6833","nonce":727644549}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-07-21T01:56:17.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 -- 172.21.15.140:0/1854203215 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c5c053aa0 msgr2=0x7f9c5c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c5c053aa0 0x7f9c5c055f60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f9c6800b000 tx=0x7f9c68006eb0).stop 2022-07-21T01:56:17.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 -- 172.21.15.140:0/1854203215 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c780b3ca0 msgr2=0x7f9c780ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c780b3ca0 0x7f9c780ae740 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f9c60001250 tx=0x7f9c60007e40).stop 2022-07-21T01:56:17.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 -- 172.21.15.140:0/1854203215 shutdown_connections 2022-07-21T01:56:17.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c5c053aa0 0x7f9c5c055f60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c7813cb70 0x7f9c780af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c7813c790 0x7f9c780aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 --2- 172.21.15.140:0/1854203215 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c780b3ca0 0x7f9c780ae740 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 -- 172.21.15.140:0/1854203215 >> 172.21.15.140:0/1854203215 conn(0x7f9c780678b0 msgr2=0x7f9c7813b310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:17.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 -- 172.21.15.140:0/1854203215 shutdown_connections 2022-07-21T01:56:17.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.317+0000 7f9c7759e700 1 -- 172.21.15.140:0/1854203215 wait complete. 2022-07-21T01:56:17.328 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 47 2022-07-21T01:56:17.337 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:56:17.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.433+0000 7f841dae3700 1 Processor -- start 2022-07-21T01:56:17.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.433+0000 7f841dae3700 1 -- start start 2022-07-21T01:56:17.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b5860 0x7f84180acae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8418146770 0x7f841814c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84180b46e0 con 0x7f84180b4f80 2022-07-21T01:56:17.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84180ad990 con 0x7f8418146770 2022-07-21T01:56:17.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84180b4b10 con 0x7f84180b5860 2022-07-21T01:56:17.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841cae1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b5860 0x7f84180acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841d2e2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8418146770 0x7f841814c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841cae1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b5860 0x7f84180acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:35854/0 (socket says 172.21.15.140:35854) 2022-07-21T01:56:17.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841cae1700 1 -- 172.21.15.140:0/2733912670 learned_addr learned my addr 172.21.15.140:0/2733912670 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:17.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841d2e2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8418146770 0x7f841814c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:44866/0 (socket says 172.21.15.140:44866) 2022-07-21T01:56:17.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f8417fff700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f84180b4f80 0x7f8418146060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:54382/0 (socket says 172.21.15.140:54382) 2022-07-21T01:56:17.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841cae1700 1 -- 172.21.15.140:0/2733912670 >> v2:172.21.15.152:3300/0 conn(0x7f8418146770 msgr2=0x7f841814c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:17.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841cae1700 1 --2- 172.21.15.140:0/2733912670 >> v2:172.21.15.152:3300/0 conn(0x7f8418146770 0x7f841814c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841cae1700 1 -- 172.21.15.140:0/2733912670 >> v1:172.21.15.152:6789/0 conn(0x7f84180b4f80 legacy=0x7f8418146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841cae1700 1 -- 172.21.15.140:0/2733912670 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84180ad3d0 con 0x7f84180b5860 2022-07-21T01:56:17.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841d2e2700 1 --2- 172.21.15.140:0/2733912670 >> v2:172.21.15.152:3300/0 conn(0x7f8418146770 0x7f841814c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:17.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841cae1700 1 --2- 172.21.15.140:0/2733912670 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b5860 0x7f84180acae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f8408002f70 tx=0x7f8408006eb0).ready entity=mon.2 client_cookie=3f6af6a19120f52e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:17.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f84177fe700 1 -- 172.21.15.140:0/2733912670 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840800d570 con 0x7f84180b5860 2022-07-21T01:56:17.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f84177fe700 1 -- 172.21.15.140:0/2733912670 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8408000e90 con 0x7f84180b5860 2022-07-21T01:56:17.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f84177fe700 1 -- 172.21.15.140:0/2733912670 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840800d900 con 0x7f84180b5860 2022-07-21T01:56:17.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 -- 172.21.15.140:0/2733912670 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b5860 msgr2=0x7f84180acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 --2- 172.21.15.140:0/2733912670 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b5860 0x7f84180acae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f8408002f70 tx=0x7f8408006eb0).stop 2022-07-21T01:56:17.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 -- 172.21.15.140:0/2733912670 shutdown_connections 2022-07-21T01:56:17.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 --2- 172.21.15.140:0/2733912670 >> v2:172.21.15.152:3300/0 conn(0x7f8418146770 0x7f841814c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 --2- 172.21.15.140:0/2733912670 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b5860 0x7f84180acae0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 -- 172.21.15.140:0/2733912670 >> 172.21.15.140:0/2733912670 conn(0x7f84180b5e10 msgr2=0x7f84180b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:17.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 -- 172.21.15.140:0/2733912670 shutdown_connections 2022-07-21T01:56:17.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.437+0000 7f841dae3700 1 -- 172.21.15.140:0/2733912670 wait complete. 2022-07-21T01:56:17.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841dae3700 1 Processor -- start 2022-07-21T01:56:17.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841dae3700 1 -- start start 2022-07-21T01:56:17.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841dae3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b4f80 0x7f8418112930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841dae3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84180b5860 0x7f8418151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841dae3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418146770 0x7f84181523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841dae3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84180b3110 con 0x7f84180b5860 2022-07-21T01:56:17.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841dae3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8418115240 con 0x7f8418146770 2022-07-21T01:56:17.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841dae3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84180b1e10 con 0x7f84180b4f80 2022-07-21T01:56:17.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8417fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84180b5860 0x7f8418151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8417fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84180b5860 0x7f8418151ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39130/0 (socket says 172.21.15.140:39130) 2022-07-21T01:56:17.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8417fff700 1 -- 172.21.15.140:0/2818315310 learned_addr learned my addr 172.21.15.140:0/2818315310 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:17.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841d2e2700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418146770 0x7f84181523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841cae1700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b4f80 0x7f8418112930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8417fff700 1 -- 172.21.15.140:0/2818315310 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b4f80 msgr2=0x7f8418112930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8417fff700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b4f80 0x7f8418112930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8417fff700 1 -- 172.21.15.140:0/2818315310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418146770 msgr2=0x7f84181523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:17.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8417fff700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418146770 0x7f84181523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:17.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8417fff700 1 -- 172.21.15.140:0/2818315310 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8410009470 con 0x7f84180b5860 2022-07-21T01:56:17.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8417fff700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84180b5860 0x7f8418151ed0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f84180666e0 tx=0x7f840c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:17.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8415ffb700 1 -- 172.21.15.140:0/2818315310 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840c0070d0 con 0x7f84180b5860 2022-07-21T01:56:17.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8415ffb700 1 -- 172.21.15.140:0/2818315310 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c007230 con 0x7f84180b5860 2022-07-21T01:56:17.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841dae3700 1 -- 172.21.15.140:0/2818315310 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840800b1e0 con 0x7f84180b5860 2022-07-21T01:56:17.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f8415ffb700 1 -- 172.21.15.140:0/2818315310 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840c005740 con 0x7f84180b5860 2022-07-21T01:56:17.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.441+0000 7f841dae3700 1 -- 172.21.15.140:0/2818315310 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84180b1bb0 con 0x7f84180b5860 2022-07-21T01:56:17.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.445+0000 7f841dae3700 1 -- 172.21.15.140:0/2818315310 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84180af0f0 con 0x7f84180b5860 2022-07-21T01:56:17.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.445+0000 7f8415ffb700 1 -- 172.21.15.140:0/2818315310 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f840c010070 con 0x7f84180b5860 2022-07-21T01:56:17.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.445+0000 7f8415ffb700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f8053a70 0x7f83f8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:17.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.445+0000 7f8415ffb700 1 -- 172.21.15.140:0/2818315310 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f840c06b490 con 0x7f84180b5860 2022-07-21T01:56:17.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.445+0000 7f841cae1700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f8053a70 0x7f83f8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:17.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.445+0000 7f841cae1700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f8053a70 0x7f83f8055f30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f840801c000 tx=0x7f8408015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:17.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.449+0000 7f8415ffb700 1 -- 172.21.15.140:0/2818315310 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f840c042310 con 0x7f84180b5860 2022-07-21T01:56:17.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:17.609+0000 7f841dae3700 1 -- 172.21.15.140:0/2818315310 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f84180574e0 con 0x7f84180b5860 2022-07-21T01:56:18.260 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:56:18.257+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:56:18.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.269+0000 7f8415ffb700 1 -- 172.21.15.140:0/2818315310 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7f840c018070 con 0x7f84180b5860 2022-07-21T01:56:18.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 -- 172.21.15.140:0/2818315310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f8053a70 msgr2=0x7f83f8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f8053a70 0x7f83f8055f30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f840801c000 tx=0x7f8408015000).stop 2022-07-21T01:56:18.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 -- 172.21.15.140:0/2818315310 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84180b5860 msgr2=0x7f8418151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84180b5860 0x7f8418151ed0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f84180666e0 tx=0x7f840c007c00).stop 2022-07-21T01:56:18.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 -- 172.21.15.140:0/2818315310 shutdown_connections 2022-07-21T01:56:18.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f8053a70 0x7f83f8055f30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8418146770 0x7f84181523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f84180b5860 0x7f8418151ed0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 --2- 172.21.15.140:0/2818315310 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84180b4f80 0x7f8418112930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 -- 172.21.15.140:0/2818315310 >> 172.21.15.140:0/2818315310 conn(0x7f84180b5e10 msgr2=0x7f84180ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:18.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 -- 172.21.15.140:0/2818315310 shutdown_connections 2022-07-21T01:56:18.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.273+0000 7f841dae3700 1 -- 172.21.15.140:0/2818315310 wait complete. 2022-07-21T01:56:18.281 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:56:18.292 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:18.300 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:56:18.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.389+0000 7fc59f59e700 1 Processor -- start 2022-07-21T01:56:18.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.389+0000 7fc59f59e700 1 -- start start 2022-07-21T01:56:18.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b61d0 0x7fc5a00ae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59f59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc5a00b01a0 0x7fc5a00b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59f59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a00b1580 con 0x7fc5a00b1dd0 2022-07-21T01:56:18.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59f59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a00b65b0 con 0x7fc5a00b01a0 2022-07-21T01:56:18.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a0060a10 con 0x7fc5a00b61d0 2022-07-21T01:56:18.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59ed9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc5a00b1dd0 0x7fc5a005f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53286/0 (socket says 172.21.15.140:53286) 2022-07-21T01:56:18.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59ed9d700 1 -- 172.21.15.140:0/1709391429 learned_addr learned my addr 172.21.15.140:0/1709391429 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:18.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59e59c700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b61d0 0x7fc5a00ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59dd9b700 1 --2- 172.21.15.140:0/1709391429 >> v2:172.21.15.152:3300/0 conn(0x7fc5a00b01a0 0x7fc5a00b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181202959 0 0) 0x7fc5a00b1580 con 0x7fc5a00b1dd0 2022-07-21T01:56:18.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc580003260 con 0x7fc5a00b1dd0 2022-07-21T01:56:18.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4259824574 0 0) 0x7fc580003260 con 0x7fc5a00b1dd0 2022-07-21T01:56:18.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b61d0 msgr2=0x7fc5a00ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:18.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b61d0 0x7fc5a00ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 >> v2:172.21.15.152:3300/0 conn(0x7fc5a00b01a0 msgr2=0x7fc5a00b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:18.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 --2- 172.21.15.140:0/1709391429 >> v2:172.21.15.152:3300/0 conn(0x7fc5a00b01a0 0x7fc5a00b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a0060cb0 con 0x7fc5a00b1dd0 2022-07-21T01:56:18.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fc5980019e0 con 0x7fc5a00b1dd0 2022-07-21T01:56:18.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 >> v1:172.21.15.140:6789/0 conn(0x7fc5a00b1dd0 legacy=0x7fc5a005f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc580003750 0x7fc580003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b61d0 0x7fc580004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5800049f0 0x7fc580004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a00b65b0 con 0x7fc5800049f0 2022-07-21T01:56:18.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a0060a10 con 0x7fc580003750 2022-07-21T01:56:18.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58000b750 con 0x7fc5a00b61d0 2022-07-21T01:56:18.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59ed9d700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5800049f0 0x7fc580004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59e59c700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc580003750 0x7fc580003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59dd9b700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b61d0 0x7fc580004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59ed9d700 1 -- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b61d0 msgr2=0x7fc580004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:18.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59ed9d700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b61d0 0x7fc580004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59ed9d700 1 -- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc580003750 msgr2=0x7fc580003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:18.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59ed9d700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc580003750 0x7fc580003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59ed9d700 1 -- 172.21.15.140:0/1709391429 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc580003260 con 0x7fc5800049f0 2022-07-21T01:56:18.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59dd9b700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b61d0 0x7fc580004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:56:18.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59ed9d700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5800049f0 0x7fc580004e70 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fc5980016f0 tx=0x7fc5a0112bc0).ready entity=mon.0 client_cookie=d8553730a3e3758f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:18.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc59800cd70 con 0x7fc5800049f0 2022-07-21T01:56:18.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598005990 con 0x7fc5800049f0 2022-07-21T01:56:18.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59d59a700 1 -- 172.21.15.140:0/1709391429 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc598005cf0 con 0x7fc5800049f0 2022-07-21T01:56:18.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59f59e700 1 -- 172.21.15.140:0/1709391429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5800049f0 msgr2=0x7fc580004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59f59e700 1 --2- 172.21.15.140:0/1709391429 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5800049f0 0x7fc580004e70 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fc5980016f0 tx=0x7fc5a0112bc0).stop 2022-07-21T01:56:18.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59ed9d700 1 -- 172.21.15.140:0/1709391429 reap_dead start 2022-07-21T01:56:18.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59f59e700 1 -- 172.21.15.140:0/1709391429 shutdown_connections 2022-07-21T01:56:18.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59f59e700 1 -- 172.21.15.140:0/1709391429 >> 172.21.15.140:0/1709391429 conn(0x7fc5a0063cf0 msgr2=0x7fc5a00b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:18.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59f59e700 1 -- 172.21.15.140:0/1709391429 shutdown_connections 2022-07-21T01:56:18.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.393+0000 7fc59f59e700 1 -- 172.21.15.140:0/1709391429 wait complete. 2022-07-21T01:56:18.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59f59e700 1 Processor -- start 2022-07-21T01:56:18.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59f59e700 1 -- start start 2022-07-21T01:56:18.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b3f20 0x7fc5a0156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59f59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5a01563f0 0x7fc5a0156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59f59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5a0064610 0x7fc5a0159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59f59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a010f920 con 0x7fc5a01563f0 2022-07-21T01:56:18.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59f59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a010fdc0 con 0x7fc5a0064610 2022-07-21T01:56:18.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a00b4300 con 0x7fc5a00b3f20 2022-07-21T01:56:18.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59dd9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5a01563f0 0x7fc5a0156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59ed9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5a0064610 0x7fc5a0159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59dd9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5a01563f0 0x7fc5a0156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39150/0 (socket says 172.21.15.140:39150) 2022-07-21T01:56:18.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59dd9b700 1 -- 172.21.15.140:0/2052484613 learned_addr learned my addr 172.21.15.140:0/2052484613 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:18.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59e59c700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b3f20 0x7fc5a0156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59ed9d700 1 -- 172.21.15.140:0/2052484613 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b3f20 msgr2=0x7fc5a0156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59ed9d700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b3f20 0x7fc5a0156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59ed9d700 1 -- 172.21.15.140:0/2052484613 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5a01563f0 msgr2=0x7fc5a0156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59ed9d700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5a01563f0 0x7fc5a0156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59ed9d700 1 -- 172.21.15.140:0/2052484613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a010fb20 con 0x7fc5a0064610 2022-07-21T01:56:18.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59ed9d700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5a0064610 0x7fc5a0159c50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fc5a0066d10 tx=0x7fc598013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:18.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc5977fe700 1 -- 172.21.15.140:0/2052484613 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5980072e0 con 0x7fc5a0064610 2022-07-21T01:56:18.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59f59e700 1 -- 172.21.15.140:0/2052484613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a0148bd0 con 0x7fc5a0064610 2022-07-21T01:56:18.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc5977fe700 1 -- 172.21.15.140:0/2052484613 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59800b030 con 0x7fc5a0064610 2022-07-21T01:56:18.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc59f59e700 1 -- 172.21.15.140:0/2052484613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a01125f0 con 0x7fc5a0064610 2022-07-21T01:56:18.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.397+0000 7fc5977fe700 1 -- 172.21.15.140:0/2052484613 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc598007630 con 0x7fc5a0064610 2022-07-21T01:56:18.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.401+0000 7fc59f59e700 1 -- 172.21.15.140:0/2052484613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a0159940 con 0x7fc5a0064610 2022-07-21T01:56:18.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.401+0000 7fc5977fe700 1 -- 172.21.15.140:0/2052484613 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc59801c020 con 0x7fc5a0064610 2022-07-21T01:56:18.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.401+0000 7fc5977fe700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc584053950 0x7fc584055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.401+0000 7fc5977fe700 1 -- 172.21.15.140:0/2052484613 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc59806c230 con 0x7fc5a0064610 2022-07-21T01:56:18.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.401+0000 7fc59e59c700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc584053950 0x7fc584055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.401+0000 7fc59e59c700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc584053950 0x7fc584055e10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fc59000b000 tx=0x7fc590008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:18.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.409+0000 7fc5977fe700 1 -- 172.21.15.140:0/2052484613 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc598043100 con 0x7fc5a0064610 2022-07-21T01:56:18.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.569+0000 7fc59f59e700 1 -- 172.21.15.140:0/2052484613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5a0112900 con 0x7fc5a0064610 2022-07-21T01:56:18.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc5977fe700 1 -- 172.21.15.140:0/2052484613 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+1783 (secure 0 0 0) 0x7fc598046720 con 0x7fc5a0064610 2022-07-21T01:56:18.578 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:18.578 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":48,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/3024179649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":3024179649},{"type":"v1","addr":"172.21.15.140:6835","nonce":3024179649}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46}],"filesystems":[{"mdsmap":{"epoch":48,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:56:06.173419+0000","modified":"2022-07-21T01:56:18.260670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2022-07-21T01:56:18.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 -- 172.21.15.140:0/2052484613 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc584053950 msgr2=0x7fc584055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc584053950 0x7fc584055e10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fc59000b000 tx=0x7fc590008040).stop 2022-07-21T01:56:18.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 -- 172.21.15.140:0/2052484613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5a0064610 msgr2=0x7fc5a0159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5a0064610 0x7fc5a0159c50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7fc5a0066d10 tx=0x7fc598013000).stop 2022-07-21T01:56:18.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 -- 172.21.15.140:0/2052484613 shutdown_connections 2022-07-21T01:56:18.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc584053950 0x7fc584055e10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5a0064610 0x7fc5a0159c50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5a01563f0 0x7fc5a0156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 --2- 172.21.15.140:0/2052484613 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5a00b3f20 0x7fc5a0156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 -- 172.21.15.140:0/2052484613 >> 172.21.15.140:0/2052484613 conn(0x7fc5a0063cf0 msgr2=0x7fc5a0149c10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:18.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 -- 172.21.15.140:0/2052484613 shutdown_connections 2022-07-21T01:56:18.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.573+0000 7fc59f59e700 1 -- 172.21.15.140:0/2052484613 wait complete. 2022-07-21T01:56:18.583 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 48 2022-07-21T01:56:18.590 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:56:18.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cffff700 1 Processor -- start 2022-07-21T01:56:18.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cffff700 1 -- start start 2022-07-21T01:56:18.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f61d00b2170 0x7f61d00b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61d00b2550 0x7f61d00aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d00b5020 con 0x7f61d00b3d40 2022-07-21T01:56:18.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cffff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d00b3640 con 0x7f61d00b2170 2022-07-21T01:56:18.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00add00 con 0x7f61d00b2550 2022-07-21T01:56:18.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cf7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f61d00b3d40 0x7f61d013e5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53306/0 (socket says 172.21.15.140:53306) 2022-07-21T01:56:18.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cf7fe700 1 -- 172.21.15.140:0/313019945 learned_addr learned my addr 172.21.15.140:0/313019945 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:18.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61ceffd700 1 --2- 172.21.15.140:0/313019945 >> v2:172.21.15.152:3300/0 conn(0x7f61d00b2170 0x7f61d00b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61ce7fc700 1 --2- 172.21.15.140:0/313019945 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61d00b2550 0x7f61d00aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cdffb700 1 -- 172.21.15.140:0/313019945 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435338315 0 0) 0x7f61d00b5020 con 0x7f61d00b3d40 2022-07-21T01:56:18.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cdffb700 1 -- 172.21.15.140:0/313019945 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b4003260 con 0x7f61d00b3d40 2022-07-21T01:56:18.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61ceffd700 1 -- 172.21.15.140:0/313019945 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61d00b2550 msgr2=0x7f61d00aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:18.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61ceffd700 1 --2- 172.21.15.140:0/313019945 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61d00b2550 0x7f61d00aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61ceffd700 1 -- 172.21.15.140:0/313019945 >> v1:172.21.15.140:6789/0 conn(0x7f61d00b3d40 legacy=0x7f61d013e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61ceffd700 1 -- 172.21.15.140:0/313019945 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00adad0 con 0x7f61d00b2170 2022-07-21T01:56:18.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61ce7fc700 1 --2- 172.21.15.140:0/313019945 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61d00b2550 0x7f61d00aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:18.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61ceffd700 1 --2- 172.21.15.140:0/313019945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61d00b2170 0x7f61d00b51e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f61c4002f70 tx=0x7f61c4006df0).ready entity=mon.1 client_cookie=f40b0c563285a0f8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:18.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cdffb700 1 -- 172.21.15.140:0/313019945 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c400d4a0 con 0x7f61d00b2170 2022-07-21T01:56:18.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cdffb700 1 -- 172.21.15.140:0/313019945 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c4000e90 con 0x7f61d00b2170 2022-07-21T01:56:18.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.693+0000 7f61cdffb700 1 -- 172.21.15.140:0/313019945 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c400d7f0 con 0x7f61d00b2170 2022-07-21T01:56:18.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 -- 172.21.15.140:0/313019945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61d00b2170 msgr2=0x7f61d00b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 --2- 172.21.15.140:0/313019945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61d00b2170 0x7f61d00b51e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f61c4002f70 tx=0x7f61c4006df0).stop 2022-07-21T01:56:18.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 -- 172.21.15.140:0/313019945 shutdown_connections 2022-07-21T01:56:18.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 --2- 172.21.15.140:0/313019945 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61d00b2550 0x7f61d00aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 --2- 172.21.15.140:0/313019945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61d00b2170 0x7f61d00b51e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 -- 172.21.15.140:0/313019945 >> 172.21.15.140:0/313019945 conn(0x7f61d0063150 msgr2=0x7f61d0063530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:18.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 -- 172.21.15.140:0/313019945 shutdown_connections 2022-07-21T01:56:18.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 -- 172.21.15.140:0/313019945 wait complete. 2022-07-21T01:56:18.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 Processor -- start 2022-07-21T01:56:18.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 -- start start 2022-07-21T01:56:18.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61d00b2170 0x7f61d013c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61d00b2550 0x7f61d0142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61d00b3d40 0x7f61d01429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0064ea0 con 0x7f61d00b2550 2022-07-21T01:56:18.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0064af0 con 0x7f61d00b2170 2022-07-21T01:56:18.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d0141850 con 0x7f61d00b3d40 2022-07-21T01:56:18.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61ce7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61d00b2550 0x7f61d0142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61ce7fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61d00b2550 0x7f61d0142500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39164/0 (socket says 172.21.15.140:39164) 2022-07-21T01:56:18.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61ce7fc700 1 -- 172.21.15.140:0/334912376 learned_addr learned my addr 172.21.15.140:0/334912376 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:18.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61ceffd700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61d00b2170 0x7f61d013c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.697+0000 7f61cf7fe700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61d00b3d40 0x7f61d01429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61ce7fc700 1 -- 172.21.15.140:0/334912376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61d00b3d40 msgr2=0x7f61d01429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61ce7fc700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61d00b3d40 0x7f61d01429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61ce7fc700 1 -- 172.21.15.140:0/334912376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61d00b2170 msgr2=0x7f61d013c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:18.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61ce7fc700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61d00b2170 0x7f61d013c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:18.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61ce7fc700 1 -- 172.21.15.140:0/334912376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c8009470 con 0x7f61d00b2550 2022-07-21T01:56:18.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61ce7fc700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61d00b2550 0x7f61d0142500 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f61c800c000 tx=0x7f61c80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:18.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61bffff700 1 -- 172.21.15.140:0/334912376 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c800c850 con 0x7f61d00b2550 2022-07-21T01:56:18.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61bffff700 1 -- 172.21.15.140:0/334912376 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c800c9b0 con 0x7f61d00b2550 2022-07-21T01:56:18.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61cffff700 1 -- 172.21.15.140:0/334912376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c400b180 con 0x7f61d00b2550 2022-07-21T01:56:18.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61bffff700 1 -- 172.21.15.140:0/334912376 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c800cd10 con 0x7f61d00b2550 2022-07-21T01:56:18.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61cffff700 1 -- 172.21.15.140:0/334912376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d0142ee0 con 0x7f61d00b2550 2022-07-21T01:56:18.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61cffff700 1 -- 172.21.15.140:0/334912376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d00b38a0 con 0x7f61d00b2550 2022-07-21T01:56:18.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61bffff700 1 -- 172.21.15.140:0/334912376 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f61c801b400 con 0x7f61d00b2550 2022-07-21T01:56:18.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61bffff700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61ac053b00 0x7f61ac055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:18.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61bffff700 1 -- 172.21.15.140:0/334912376 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f61c806ca00 con 0x7f61d00b2550 2022-07-21T01:56:18.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61ceffd700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61ac053b00 0x7f61ac055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:18.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.701+0000 7f61ceffd700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61ac053b00 0x7f61ac055fc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f61c401a000 tx=0x7f61c4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:18.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.709+0000 7f61bffff700 1 -- 172.21.15.140:0/334912376 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f61c80437d0 con 0x7f61d00b2550 2022-07-21T01:56:18.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:18.869+0000 7f61cffff700 1 -- 172.21.15.140:0/334912376 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f61d0141990 con 0x7f61d00b2550 2022-07-21T01:56:19.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.273+0000 7f61bffff700 1 -- 172.21.15.140:0/334912376 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f61c8046df0 con 0x7f61d00b2550 2022-07-21T01:56:19.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 -- 172.21.15.140:0/334912376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61ac053b00 msgr2=0x7f61ac055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:19.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61ac053b00 0x7f61ac055fc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f61c401a000 tx=0x7f61c4014000).stop 2022-07-21T01:56:19.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 -- 172.21.15.140:0/334912376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61d00b2550 msgr2=0x7f61d0142500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:19.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61d00b2550 0x7f61d0142500 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f61c800c000 tx=0x7f61c80091d0).stop 2022-07-21T01:56:19.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 -- 172.21.15.140:0/334912376 shutdown_connections 2022-07-21T01:56:19.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f61ac053b00 0x7f61ac055fc0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:19.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61d00b3d40 0x7f61d01429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:19.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f61d00b2550 0x7f61d0142500 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:19.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 --2- 172.21.15.140:0/334912376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61d00b2170 0x7f61d013c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:19.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 -- 172.21.15.140:0/334912376 >> 172.21.15.140:0/334912376 conn(0x7f61d0063150 msgr2=0x7f61d00adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:19.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 -- 172.21.15.140:0/334912376 shutdown_connections 2022-07-21T01:56:19.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.277+0000 7f61cffff700 1 -- 172.21.15.140:0/334912376 wait complete. 2022-07-21T01:56:19.296 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:56:19.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.409+0000 7fa505be3700 1 Processor -- start 2022-07-21T01:56:19.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.409+0000 7fa505be3700 1 -- start start 2022-07-21T01:56:19.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.409+0000 7fa505be3700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa5000b5990 0x7fa500146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:19.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.409+0000 7fa505be3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa5000af3e0 0x7fa500146560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:19.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.409+0000 7fa505be3700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5000b5d70 con 0x7fa50005d210 2022-07-21T01:56:19.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.409+0000 7fa505be3700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5000ad050 con 0x7fa5000af3e0 2022-07-21T01:56:19.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.409+0000 7fa505be3700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5000b17e0 con 0x7fa5000b5990 2022-07-21T01:56:19.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.409+0000 7fa4fffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa5000af3e0 0x7fa500146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:19.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa5053e2700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa50005d210 0x7fa50014a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53318/0 (socket says 172.21.15.140:53318) 2022-07-21T01:56:19.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa5053e2700 1 -- 172.21.15.140:0/3911091260 learned_addr learned my addr 172.21.15.140:0/3911091260 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:19.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa504be1700 1 --2- 172.21.15.140:0/3911091260 >> v2:172.21.15.152:3300/0 conn(0x7fa5000b5990 0x7fa500146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:19.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4fffff700 1 -- 172.21.15.140:0/3911091260 >> v2:172.21.15.152:3300/0 conn(0x7fa5000b5990 msgr2=0x7fa500146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:19.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4fffff700 1 --2- 172.21.15.140:0/3911091260 >> v2:172.21.15.152:3300/0 conn(0x7fa5000b5990 0x7fa500146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:19.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4fffff700 1 -- 172.21.15.140:0/3911091260 >> v1:172.21.15.140:6789/0 conn(0x7fa50005d210 legacy=0x7fa50014a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:19.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4fffff700 1 -- 172.21.15.140:0/3911091260 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5000b1010 con 0x7fa5000af3e0 2022-07-21T01:56:19.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4ff7fe700 1 -- 172.21.15.140:0/3911091260 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456441085 0 0) 0x7fa5000b5d70 con 0x7fa50005d210 2022-07-21T01:56:19.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4fffff700 1 --2- 172.21.15.140:0/3911091260 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5000af3e0 0x7fa500146560 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7fa4f00032f0 tx=0x7fa4f00091d0).ready entity=mon.0 client_cookie=7113a7ff3ea80a97 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:19.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4ff7fe700 1 -- 172.21.15.140:0/3911091260 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f000b680 con 0x7fa5000af3e0 2022-07-21T01:56:19.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4ff7fe700 1 -- 172.21.15.140:0/3911091260 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f000b7e0 con 0x7fa5000af3e0 2022-07-21T01:56:19.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4ff7fe700 1 -- 172.21.15.140:0/3911091260 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f000b950 con 0x7fa5000af3e0 2022-07-21T01:56:19.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 -- 172.21.15.140:0/3911091260 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5000af3e0 msgr2=0x7fa500146560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:19.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 --2- 172.21.15.140:0/3911091260 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5000af3e0 0x7fa500146560 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7fa4f00032f0 tx=0x7fa4f00091d0).stop 2022-07-21T01:56:19.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 -- 172.21.15.140:0/3911091260 shutdown_connections 2022-07-21T01:56:19.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 --2- 172.21.15.140:0/3911091260 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5000af3e0 0x7fa500146560 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:19.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 --2- 172.21.15.140:0/3911091260 >> v2:172.21.15.152:3300/0 conn(0x7fa5000b5990 0x7fa500146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:19.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 -- 172.21.15.140:0/3911091260 >> 172.21.15.140:0/3911091260 conn(0x7fa500063480 msgr2=0x7fa5000b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:19.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 -- 172.21.15.140:0/3911091260 shutdown_connections 2022-07-21T01:56:19.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 -- 172.21.15.140:0/3911091260 wait complete. 2022-07-21T01:56:19.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 Processor -- start 2022-07-21T01:56:19.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 -- start start 2022-07-21T01:56:19.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa50005d210 0x7fa50015a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:19.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5000af3e0 0x7fa500111d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:19.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5000b5990 0x7fa500065110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:19.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa500064e70 con 0x7fa5000af3e0 2022-07-21T01:56:19.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa500159ce0 con 0x7fa50005d210 2022-07-21T01:56:19.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa505be3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa500159f80 con 0x7fa5000b5990 2022-07-21T01:56:19.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4fffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5000af3e0 0x7fa500111d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:19.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4fffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5000af3e0 0x7fa500111d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39178/0 (socket says 172.21.15.140:39178) 2022-07-21T01:56:19.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa4fffff700 1 -- 172.21.15.140:0/1559096994 learned_addr learned my addr 172.21.15.140:0/1559096994 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:19.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa504be1700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa50005d210 0x7fa50015a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:19.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.413+0000 7fa5053e2700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5000b5990 0x7fa500065110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:19.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fffff700 1 -- 172.21.15.140:0/1559096994 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5000b5990 msgr2=0x7fa500065110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:19.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fffff700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5000b5990 0x7fa500065110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:19.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fffff700 1 -- 172.21.15.140:0/1559096994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa50005d210 msgr2=0x7fa50015a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:19.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fffff700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa50005d210 0x7fa50015a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:19.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fffff700 1 -- 172.21.15.140:0/1559096994 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f0009470 con 0x7fa5000af3e0 2022-07-21T01:56:19.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fffff700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5000af3e0 0x7fa500111d80 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7fa4f00032f0 tx=0x7fa4f0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:19.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fdffb700 1 -- 172.21.15.140:0/1559096994 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f000b950 con 0x7fa5000af3e0 2022-07-21T01:56:19.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fdffb700 1 -- 172.21.15.140:0/1559096994 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f0021630 con 0x7fa5000af3e0 2022-07-21T01:56:19.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fdffb700 1 -- 172.21.15.140:0/1559096994 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f000b680 con 0x7fa5000af3e0 2022-07-21T01:56:19.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa505be3700 1 -- 172.21.15.140:0/1559096994 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50015a220 con 0x7fa5000af3e0 2022-07-21T01:56:19.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa505be3700 1 -- 172.21.15.140:0/1559096994 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50014ae30 con 0x7fa5000af3e0 2022-07-21T01:56:19.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fdffb700 1 -- 172.21.15.140:0/1559096994 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa4f0027020 con 0x7fa5000af3e0 2022-07-21T01:56:19.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa505be3700 1 -- 172.21.15.140:0/1559096994 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50014a9a0 con 0x7fa5000af3e0 2022-07-21T01:56:19.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fdffb700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4e4053b30 0x7fa4e4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:19.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa4fdffb700 1 -- 172.21.15.140:0/1559096994 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa4f006d400 con 0x7fa5000af3e0 2022-07-21T01:56:19.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.417+0000 7fa504be1700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4e4053b30 0x7fa4e4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:19.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.421+0000 7fa504be1700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4e4053b30 0x7fa4e4055ff0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fa4e8006000 tx=0x7fa4e8006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:19.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.425+0000 7fa4fdffb700 1 -- 172.21.15.140:0/1559096994 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa50014a9a0 con 0x7fa5000af3e0 2022-07-21T01:56:19.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:19.549+0000 7fa505be3700 1 -- 172.21.15.140:0/1559096994 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa50014a9a0 con 0x7fa5000af3e0 2022-07-21T01:56:20.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.281+0000 7fa4fdffb700 1 -- 172.21.15.140:0/1559096994 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 185+0+0 (secure 0 0 0) 0x7fa50014a9a0 con 0x7fa5000af3e0 2022-07-21T01:56:20.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 -- 172.21.15.140:0/1559096994 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4e4053b30 msgr2=0x7fa4e4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:20.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4e4053b30 0x7fa4e4055ff0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fa4e8006000 tx=0x7fa4e8006d90).stop 2022-07-21T01:56:20.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 -- 172.21.15.140:0/1559096994 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5000af3e0 msgr2=0x7fa500111d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:20.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5000af3e0 0x7fa500111d80 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7fa4f00032f0 tx=0x7fa4f0016000).stop 2022-07-21T01:56:20.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 -- 172.21.15.140:0/1559096994 shutdown_connections 2022-07-21T01:56:20.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa4e4053b30 0x7fa4e4055ff0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:20.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5000b5990 0x7fa500065110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:20.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5000af3e0 0x7fa500111d80 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:20.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 --2- 172.21.15.140:0/1559096994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa50005d210 0x7fa50015a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:20.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 -- 172.21.15.140:0/1559096994 >> 172.21.15.140:0/1559096994 conn(0x7fa500063480 msgr2=0x7fa5000b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:20.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 -- 172.21.15.140:0/1559096994 shutdown_connections 2022-07-21T01:56:20.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.285+0000 7fa505be3700 1 -- 172.21.15.140:0/1559096994 wait complete. 2022-07-21T01:56:20.296 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:56:20.303 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:56:20.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.409+0000 7f0bb4cec700 1 Processor -- start 2022-07-21T01:56:20.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.409+0000 7f0bb4cec700 1 -- start start 2022-07-21T01:56:20.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.409+0000 7f0bb4cec700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0bb00b6690 0x7f0bb00b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:20.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.409+0000 7f0bb4cec700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0bb00b59a0 0x7f0bb013c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:20.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.409+0000 7f0bb4cec700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb00acae0 con 0x7f0bb013ce60 2022-07-21T01:56:20.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.409+0000 7f0bb4cec700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb0147100 con 0x7f0bb00b59a0 2022-07-21T01:56:20.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.409+0000 7f0bb4cec700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb00acd10 con 0x7f0bb00b6690 2022-07-21T01:56:20.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0badd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0bb00b59a0 0x7f0bb013c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:20.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0badd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0bb00b59a0 0x7f0bb013c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39186/0 (socket says 172.21.15.140:39186) 2022-07-21T01:56:20.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0badd9b700 1 -- 172.21.15.140:0/765799789 learned_addr learned my addr 172.21.15.140:0/765799789 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:20.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bae59c700 1 --2- 172.21.15.140:0/765799789 >> v2:172.21.15.152:3300/0 conn(0x7f0bb00b6690 0x7f0bb00b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:20.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0badd9b700 1 -- 172.21.15.140:0/765799789 >> v2:172.21.15.152:3300/0 conn(0x7f0bb00b6690 msgr2=0x7f0bb00b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:20.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0badd9b700 1 --2- 172.21.15.140:0/765799789 >> v2:172.21.15.152:3300/0 conn(0x7f0bb00b6690 0x7f0bb00b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:20.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0badd9b700 1 -- 172.21.15.140:0/765799789 >> v1:172.21.15.152:6789/0 conn(0x7f0bb013ce60 legacy=0x7f0bb00ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:20.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0badd9b700 1 -- 172.21.15.140:0/765799789 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00b5df0 con 0x7f0bb00b59a0 2022-07-21T01:56:20.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0badd9b700 1 --2- 172.21.15.140:0/765799789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00b59a0 0x7f0bb013c970 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f0ba0002f70 tx=0x7f0ba0006eb0).ready entity=mon.0 client_cookie=50edec2924a3290 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:20.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bad59a700 1 -- 172.21.15.140:0/765799789 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ba000d610 con 0x7f0bb00b59a0 2022-07-21T01:56:20.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bad59a700 1 -- 172.21.15.140:0/765799789 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba0000e90 con 0x7f0bb00b59a0 2022-07-21T01:56:20.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bad59a700 1 -- 172.21.15.140:0/765799789 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ba000d9a0 con 0x7f0bb00b59a0 2022-07-21T01:56:20.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 -- 172.21.15.140:0/765799789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00b59a0 msgr2=0x7f0bb013c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:20.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 --2- 172.21.15.140:0/765799789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00b59a0 0x7f0bb013c970 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f0ba0002f70 tx=0x7f0ba0006eb0).stop 2022-07-21T01:56:20.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 -- 172.21.15.140:0/765799789 shutdown_connections 2022-07-21T01:56:20.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 --2- 172.21.15.140:0/765799789 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00b59a0 0x7f0bb013c970 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:20.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 --2- 172.21.15.140:0/765799789 >> v2:172.21.15.152:3300/0 conn(0x7f0bb00b6690 0x7f0bb00b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:20.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 -- 172.21.15.140:0/765799789 >> 172.21.15.140:0/765799789 conn(0x7f0bb00b9f70 msgr2=0x7f0bb00b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:20.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 -- 172.21.15.140:0/765799789 shutdown_connections 2022-07-21T01:56:20.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 -- 172.21.15.140:0/765799789 wait complete. 2022-07-21T01:56:20.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 Processor -- start 2022-07-21T01:56:20.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 -- start start 2022-07-21T01:56:20.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00b59a0 0x7f0bb00b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:20.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bb00b6690 0x7f0bb0140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:20.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bb013ce60 0x7f0bb01561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:20.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0155ac0 con 0x7f0bb00b59a0 2022-07-21T01:56:20.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb013f7d0 con 0x7f0bb013ce60 2022-07-21T01:56:20.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bb4cec700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0155d20 con 0x7f0bb00b6690 2022-07-21T01:56:20.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bae59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00b59a0 0x7f0bb00b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:20.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bae59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00b59a0 0x7f0bb00b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39192/0 (socket says 172.21.15.140:39192) 2022-07-21T01:56:20.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.413+0000 7f0bae59c700 1 -- 172.21.15.140:0/1130020621 learned_addr learned my addr 172.21.15.140:0/1130020621 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:20.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0badd9b700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bb00b6690 0x7f0bb0140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:20.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0baed9d700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bb013ce60 0x7f0bb01561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:20.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0bae59c700 1 -- 172.21.15.140:0/1130020621 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bb00b6690 msgr2=0x7f0bb0140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:20.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0bae59c700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bb00b6690 0x7f0bb0140290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:20.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0bae59c700 1 -- 172.21.15.140:0/1130020621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bb013ce60 msgr2=0x7f0bb01561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:20.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0bae59c700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bb013ce60 0x7f0bb01561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:20.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0bae59c700 1 -- 172.21.15.140:0/1130020621 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba000b1e0 con 0x7f0bb00b59a0 2022-07-21T01:56:20.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0bae59c700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00b59a0 0x7f0bb00b2a60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f0b980030b0 tx=0x7f0b98008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:20.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0ba77fe700 1 -- 172.21.15.140:0/1130020621 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b98011780 con 0x7f0bb00b59a0 2022-07-21T01:56:20.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0ba77fe700 1 -- 172.21.15.140:0/1130020621 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b980118e0 con 0x7f0bb00b59a0 2022-07-21T01:56:20.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0ba77fe700 1 -- 172.21.15.140:0/1130020621 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b98011c40 con 0x7f0bb00b59a0 2022-07-21T01:56:20.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0bb4cec700 1 -- 172.21.15.140:0/1130020621 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb0113110 con 0x7f0bb00b59a0 2022-07-21T01:56:20.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0bb4cec700 1 -- 172.21.15.140:0/1130020621 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb013ee70 con 0x7f0bb00b59a0 2022-07-21T01:56:20.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0bb4cec700 1 -- 172.21.15.140:0/1130020621 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb0155ee0 con 0x7f0bb00b59a0 2022-07-21T01:56:20.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0ba77fe700 1 -- 172.21.15.140:0/1130020621 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0b98012420 con 0x7f0bb00b59a0 2022-07-21T01:56:20.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0ba77fe700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b94053a70 0x7f0b94055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:20.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0ba77fe700 1 -- 172.21.15.140:0/1130020621 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f0b9806bb30 con 0x7f0bb00b59a0 2022-07-21T01:56:20.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.417+0000 7f0badd9b700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b94053a70 0x7f0b94055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:20.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.421+0000 7f0badd9b700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b94053a70 0x7f0b94055f30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f0ba001c000 tx=0x7f0ba0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:20.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.425+0000 7f0ba77fe700 1 -- 172.21.15.140:0/1130020621 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0b98042b50 con 0x7f0bb00b59a0 2022-07-21T01:56:20.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:20.549+0000 7f0bb4cec700 1 -- 172.21.15.140:0/1130020621 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0bb00acfb0 con 0x7f0bb00b59a0 2022-07-21T01:56:21.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.285+0000 7f0ba77fe700 1 -- 172.21.15.140:0/1130020621 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0b98046170 con 0x7f0bb00b59a0 2022-07-21T01:56:21.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.285+0000 7f0bb4cec700 1 -- 172.21.15.140:0/1130020621 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b94053a70 msgr2=0x7f0b94055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.285+0000 7f0bb4cec700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b94053a70 0x7f0b94055f30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f0ba001c000 tx=0x7f0ba0015000).stop 2022-07-21T01:56:21.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.285+0000 7f0bb4cec700 1 -- 172.21.15.140:0/1130020621 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00b59a0 msgr2=0x7f0bb00b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.285+0000 7f0bb4cec700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00b59a0 0x7f0bb00b2a60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f0b980030b0 tx=0x7f0b98008040).stop 2022-07-21T01:56:21.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.289+0000 7f0bb4cec700 1 -- 172.21.15.140:0/1130020621 shutdown_connections 2022-07-21T01:56:21.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.289+0000 7f0bb4cec700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0b94053a70 0x7f0b94055f30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.289+0000 7f0bb4cec700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bb013ce60 0x7f0bb01561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.289+0000 7f0bb4cec700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0bb00b6690 0x7f0bb0140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.289+0000 7f0bb4cec700 1 --2- 172.21.15.140:0/1130020621 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0bb00b59a0 0x7f0bb00b2a60 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.289+0000 7f0bb4cec700 1 -- 172.21.15.140:0/1130020621 >> 172.21.15.140:0/1130020621 conn(0x7f0bb00b9f70 msgr2=0x7f0bb00b8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:21.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.289+0000 7f0bb4cec700 1 -- 172.21.15.140:0/1130020621 shutdown_connections 2022-07-21T01:56:21.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.289+0000 7f0bb4cec700 1 -- 172.21.15.140:0/1130020621 wait complete. 2022-07-21T01:56:21.295 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:56:21.306 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-07-21T01:56:21.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb477fff700 1 Processor -- start 2022-07-21T01:56:21.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb477fff700 1 -- start start 2022-07-21T01:56:21.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb477fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb47805c7c0 0x7fb478141490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:21.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb477fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb4780b2130 0x7fb4780b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:21.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb477fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb478140870 con 0x7fb4780b0ee0 2022-07-21T01:56:21.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb477fff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb478141b50 con 0x7fb4780b2130 2022-07-21T01:56:21.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb477fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb478061760 con 0x7fb47805c7c0 2022-07-21T01:56:21.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb4777fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fb4780b0ee0 0x7fb478099290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53348/0 (socket says 172.21.15.140:53348) 2022-07-21T01:56:21.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb4777fe700 1 -- 172.21.15.140:0/2279207833 learned_addr learned my addr 172.21.15.140:0/2279207833 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:21.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb4767fc700 1 --2- 172.21.15.140:0/2279207833 >> v2:172.21.15.152:3300/0 conn(0x7fb4780b2130 0x7fb4780b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:21.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb476ffd700 1 --2- 172.21.15.140:0/2279207833 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb47805c7c0 0x7fb478141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:21.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb475ffb700 1 -- 172.21.15.140:0/2279207833 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517267887 0 0) 0x7fb470001490 con 0x7fb4780b0ee0 2022-07-21T01:56:21.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb475ffb700 1 -- 172.21.15.140:0/2279207833 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb45c003260 con 0x7fb4780b0ee0 2022-07-21T01:56:21.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb4767fc700 1 -- 172.21.15.140:0/2279207833 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb47805c7c0 msgr2=0x7fb478141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:21.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb4767fc700 1 --2- 172.21.15.140:0/2279207833 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb47805c7c0 0x7fb478141490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb4767fc700 1 -- 172.21.15.140:0/2279207833 >> v1:172.21.15.140:6789/0 conn(0x7fb4780b0ee0 legacy=0x7fb478099290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb4767fc700 1 -- 172.21.15.140:0/2279207833 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb478140ab0 con 0x7fb4780b2130 2022-07-21T01:56:21.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.409+0000 7fb4767fc700 1 --2- 172.21.15.140:0/2279207833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4780b2130 0x7fb4780b09f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fb46c002f70 tx=0x7fb46c006df0).ready entity=mon.1 client_cookie=987e041490861363 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:21.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb475ffb700 1 -- 172.21.15.140:0/2279207833 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46c00d4a0 con 0x7fb4780b2130 2022-07-21T01:56:21.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb475ffb700 1 -- 172.21.15.140:0/2279207833 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c000e90 con 0x7fb4780b2130 2022-07-21T01:56:21.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb475ffb700 1 -- 172.21.15.140:0/2279207833 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46c00d7f0 con 0x7fb4780b2130 2022-07-21T01:56:21.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 -- 172.21.15.140:0/2279207833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4780b2130 msgr2=0x7fb4780b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 --2- 172.21.15.140:0/2279207833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4780b2130 0x7fb4780b09f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7fb46c002f70 tx=0x7fb46c006df0).stop 2022-07-21T01:56:21.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 -- 172.21.15.140:0/2279207833 shutdown_connections 2022-07-21T01:56:21.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 --2- 172.21.15.140:0/2279207833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4780b2130 0x7fb4780b09f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 --2- 172.21.15.140:0/2279207833 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb47805c7c0 0x7fb478141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 -- 172.21.15.140:0/2279207833 >> 172.21.15.140:0/2279207833 conn(0x7fb4780b6050 msgr2=0x7fb4780b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:21.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 -- 172.21.15.140:0/2279207833 shutdown_connections 2022-07-21T01:56:21.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 -- 172.21.15.140:0/2279207833 wait complete. 2022-07-21T01:56:21.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 Processor -- start 2022-07-21T01:56:21.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 -- start start 2022-07-21T01:56:21.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb47805c7c0 0x7fb47813b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:21.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4780b0ee0 0x7fb4780667a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:21.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4780b2130 0x7fb4780ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:21.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47809b480 con 0x7fb47805c7c0 2022-07-21T01:56:21.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780ad030 con 0x7fb4780b0ee0 2022-07-21T01:56:21.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb477fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47813af10 con 0x7fb4780b2130 2022-07-21T01:56:21.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb476ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb47805c7c0 0x7fb47813b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:21.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb476ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb47805c7c0 0x7fb47813b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39204/0 (socket says 172.21.15.140:39204) 2022-07-21T01:56:21.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb476ffd700 1 -- 172.21.15.140:0/912538763 learned_addr learned my addr 172.21.15.140:0/912538763 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:21.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb4777fe700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4780b2130 0x7fb4780ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:21.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.413+0000 7fb4767fc700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4780b0ee0 0x7fb4780667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:21.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb476ffd700 1 -- 172.21.15.140:0/912538763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4780b2130 msgr2=0x7fb4780ac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb476ffd700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4780b2130 0x7fb4780ac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb476ffd700 1 -- 172.21.15.140:0/912538763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4780b0ee0 msgr2=0x7fb4780667a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb476ffd700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4780b0ee0 0x7fb4780667a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb476ffd700 1 -- 172.21.15.140:0/912538763 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c00b180 con 0x7fb47805c7c0 2022-07-21T01:56:21.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb476ffd700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb47805c7c0 0x7fb47813b650 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fb46800c000 tx=0x7fb468006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:21.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb467fff700 1 -- 172.21.15.140:0/912538763 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb468010780 con 0x7fb47805c7c0 2022-07-21T01:56:21.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb467fff700 1 -- 172.21.15.140:0/912538763 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4680108e0 con 0x7fb47805c7c0 2022-07-21T01:56:21.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb477fff700 1 -- 172.21.15.140:0/912538763 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47813b1b0 con 0x7fb47805c7c0 2022-07-21T01:56:21.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb467fff700 1 -- 172.21.15.140:0/912538763 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb468010c40 con 0x7fb47805c7c0 2022-07-21T01:56:21.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb477fff700 1 -- 172.21.15.140:0/912538763 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47809c170 con 0x7fb47805c7c0 2022-07-21T01:56:21.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb477fff700 1 -- 172.21.15.140:0/912538763 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb478140d50 con 0x7fb47805c7c0 2022-07-21T01:56:21.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb467fff700 1 -- 172.21.15.140:0/912538763 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb46801c460 con 0x7fb47805c7c0 2022-07-21T01:56:21.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb467fff700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb454053950 0x7fb454055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:21.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb467fff700 1 -- 172.21.15.140:0/912538763 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fb46806c040 con 0x7fb47805c7c0 2022-07-21T01:56:21.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.417+0000 7fb4767fc700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb454053950 0x7fb454055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:21.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.421+0000 7fb4767fc700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb454053950 0x7fb454055e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fb46c01a000 tx=0x7fb46c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:21.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.425+0000 7fb467fff700 1 -- 172.21.15.140:0/912538763 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb468043140 con 0x7fb47805c7c0 2022-07-21T01:56:21.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.549+0000 7fb477fff700 1 -- 172.21.15.140:0/912538763 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb4780574e0 con 0x7fb47805c7c0 2022-07-21T01:56:21.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb467fff700 1 -- 172.21.15.140:0/912538763 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb468046760 con 0x7fb47805c7c0 2022-07-21T01:56:21.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb477fff700 1 -- 172.21.15.140:0/912538763 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb454053950 msgr2=0x7fb454055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb477fff700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb454053950 0x7fb454055e10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fb46c01a000 tx=0x7fb46c014000).stop 2022-07-21T01:56:21.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb477fff700 1 -- 172.21.15.140:0/912538763 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb47805c7c0 msgr2=0x7fb47813b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb477fff700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb47805c7c0 0x7fb47813b650 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fb46800c000 tx=0x7fb468006df0).stop 2022-07-21T01:56:21.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb477fff700 1 -- 172.21.15.140:0/912538763 shutdown_connections 2022-07-21T01:56:21.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb477fff700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb454053950 0x7fb454055e10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb477fff700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb4780b2130 0x7fb4780ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb477fff700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb4780b0ee0 0x7fb4780667a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb477fff700 1 --2- 172.21.15.140:0/912538763 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb47805c7c0 0x7fb47813b650 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb477fff700 1 -- 172.21.15.140:0/912538763 >> 172.21.15.140:0/912538763 conn(0x7fb4780b6050 msgr2=0x7fb47805fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:21.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.557+0000 7fb477fff700 1 -- 172.21.15.140:0/912538763 shutdown_connections 2022-07-21T01:56:21.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.561+0000 7fb477fff700 1 -- 172.21.15.140:0/912538763 wait complete. 2022-07-21T01:56:21.576 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2022-07-21T01:56:21.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 Processor -- start 2022-07-21T01:56:21.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 -- start start 2022-07-21T01:56:21.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f89180b5b30 0x7f89180aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:21.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89180b5f10 con 0x7f89180b13e0 2022-07-21T01:56:21.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89180afc40 con 0x7f89180ace90 2022-07-21T01:56:21.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89180afee0 con 0x7f89180b5b30 2022-07-21T01:56:21.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891d56d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f89180b5b30 0x7f89180aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:21.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891d56d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f89180b5b30 0x7f89180aefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39212/0 (socket says 172.21.15.140:39212) 2022-07-21T01:56:21.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891d56d700 1 -- 172.21.15.140:0/1179612480 learned_addr learned my addr 172.21.15.140:0/1179612480 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:21.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891d56d700 1 -- 172.21.15.140:0/1179612480 >> v1:172.21.15.152:6789/0 conn(0x7f89180ace90 legacy=0x7f89180b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891d56d700 1 -- 172.21.15.140:0/1179612480 >> v1:172.21.15.140:6789/0 conn(0x7f89180b13e0 legacy=0x7f89181463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f890ffff700 1 -- 172.21.15.140:0/1179612480 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388746814 0 0) 0x7f89180b5f10 con 0x7f89180b13e0 2022-07-21T01:56:21.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891d56d700 1 -- 172.21.15.140:0/1179612480 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89180ad490 con 0x7f89180b5b30 2022-07-21T01:56:21.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891d56d700 1 --2- 172.21.15.140:0/1179612480 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89180b5b30 0x7f89180aefb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7f89100032f0 tx=0x7f8910009290).ready entity=mon.0 client_cookie=f54d7afd0e58d1e3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:21.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f890ffff700 1 -- 172.21.15.140:0/1179612480 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f891000b8a0 con 0x7f89180b5b30 2022-07-21T01:56:21.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f890ffff700 1 -- 172.21.15.140:0/1179612480 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891000ba00 con 0x7f89180b5b30 2022-07-21T01:56:21.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f890ffff700 1 -- 172.21.15.140:0/1179612480 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f891000bb70 con 0x7f89180b5b30 2022-07-21T01:56:21.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 -- 172.21.15.140:0/1179612480 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89180b5b30 msgr2=0x7f89180aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 --2- 172.21.15.140:0/1179612480 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89180b5b30 0x7f89180aefb0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7f89100032f0 tx=0x7f8910009290).stop 2022-07-21T01:56:21.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 -- 172.21.15.140:0/1179612480 shutdown_connections 2022-07-21T01:56:21.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 --2- 172.21.15.140:0/1179612480 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89180b5b30 0x7f89180aefb0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 -- 172.21.15.140:0/1179612480 >> 172.21.15.140:0/1179612480 conn(0x7f89180635b0 msgr2=0x7f89180b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:21.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 -- 172.21.15.140:0/1179612480 shutdown_connections 2022-07-21T01:56:21.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.673+0000 7f891f7d1700 1 -- 172.21.15.140:0/1179612480 wait complete. 2022-07-21T01:56:21.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891f7d1700 1 Processor -- start 2022-07-21T01:56:21.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891f7d1700 1 -- start start 2022-07-21T01:56:21.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891f7d1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89180ace90 0x7f891814a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:21.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891f7d1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89180b13e0 0x7f8918064990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:21.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891f7d1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89180b5b30 0x7f89180b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:21.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891f7d1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89180656a0 con 0x7f89180ace90 2022-07-21T01:56:21.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891f7d1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891814b0c0 con 0x7f89180b13e0 2022-07-21T01:56:21.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891f7d1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891814ae20 con 0x7f89180b5b30 2022-07-21T01:56:21.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891d56d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89180ace90 0x7f891814a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:21.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891d56d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89180ace90 0x7f891814a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39218/0 (socket says 172.21.15.140:39218) 2022-07-21T01:56:21.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891d56d700 1 -- 172.21.15.140:0/3182678691 learned_addr learned my addr 172.21.15.140:0/3182678691 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:21.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891cd6c700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89180b13e0 0x7f8918064990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:21.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891d56d700 1 -- 172.21.15.140:0/3182678691 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89180b5b30 msgr2=0x7f89180b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891dd6e700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89180b5b30 0x7f89180b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:21.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891d56d700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89180b5b30 0x7f89180b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891d56d700 1 -- 172.21.15.140:0/3182678691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89180b13e0 msgr2=0x7f8918064990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:21.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891d56d700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89180b13e0 0x7f8918064990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:21.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891d56d700 1 -- 172.21.15.140:0/3182678691 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8910009530 con 0x7f89180ace90 2022-07-21T01:56:21.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891cd6c700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89180b13e0 0x7f8918064990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:56:21.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891dd6e700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89180b5b30 0x7f89180b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:56:21.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891d56d700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89180ace90 0x7f891814a700 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f89100032f0 tx=0x7f8910016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:21.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f890e7fc700 1 -- 172.21.15.140:0/3182678691 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f891001f3f0 con 0x7f89180ace90 2022-07-21T01:56:21.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f890e7fc700 1 -- 172.21.15.140:0/3182678691 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891001f550 con 0x7f89180ace90 2022-07-21T01:56:21.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891f7d1700 1 -- 172.21.15.140:0/3182678691 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8918156590 con 0x7f89180ace90 2022-07-21T01:56:21.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f890e7fc700 1 -- 172.21.15.140:0/3182678691 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f891001f8b0 con 0x7f89180ace90 2022-07-21T01:56:21.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.677+0000 7f891f7d1700 1 -- 172.21.15.140:0/3182678691 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8918149520 con 0x7f89180ace90 2022-07-21T01:56:21.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.681+0000 7f891f7d1700 1 -- 172.21.15.140:0/3182678691 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89181563e0 con 0x7f89180ace90 2022-07-21T01:56:21.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.681+0000 7f890e7fc700 1 -- 172.21.15.140:0/3182678691 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8910026030 con 0x7f89180ace90 2022-07-21T01:56:21.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.681+0000 7f890e7fc700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89000539e0 0x7f8900055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:21.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.681+0000 7f890e7fc700 1 -- 172.21.15.140:0/3182678691 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f891006cb80 con 0x7f89180ace90 2022-07-21T01:56:21.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.681+0000 7f891cd6c700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89000539e0 0x7f8900055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:21.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.681+0000 7f891cd6c700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89000539e0 0x7f8900055ea0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f8908008000 tx=0x7f8908009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:21.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.685+0000 7f890e7fc700 1 -- 172.21.15.140:0/3182678691 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8910043c70 con 0x7f89180ace90 2022-07-21T01:56:21.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:21.853+0000 7f891f7d1700 1 -- 172.21.15.140:0/3182678691 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7f8918156720 con 0x7f89180ace90 2022-07-21T01:56:22.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.297+0000 7f890e7fc700 1 -- 172.21.15.140:0/3182678691 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v357) v1 ==== 127+0+0 (secure 0 0 0) 0x7f891006ce60 con 0x7f89180ace90 2022-07-21T01:56:22.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 -- 172.21.15.140:0/3182678691 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89000539e0 msgr2=0x7f8900055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:22.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89000539e0 0x7f8900055ea0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f8908008000 tx=0x7f8908009040).stop 2022-07-21T01:56:22.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 -- 172.21.15.140:0/3182678691 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89180ace90 msgr2=0x7f891814a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:22.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89180ace90 0x7f891814a700 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f89100032f0 tx=0x7f8910016000).stop 2022-07-21T01:56:22.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 -- 172.21.15.140:0/3182678691 shutdown_connections 2022-07-21T01:56:22.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89000539e0 0x7f8900055ea0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:22.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89180b5b30 0x7f89180b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:22.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89180b13e0 0x7f8918064990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:22.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 --2- 172.21.15.140:0/3182678691 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89180ace90 0x7f891814a700 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:22.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 -- 172.21.15.140:0/3182678691 >> 172.21.15.140:0/3182678691 conn(0x7f89180635b0 msgr2=0x7f89180ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:22.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 -- 172.21.15.140:0/3182678691 shutdown_connections 2022-07-21T01:56:22.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.301+0000 7f891f7d1700 1 -- 172.21.15.140:0/3182678691 wait complete. 2022-07-21T01:56:22.319 INFO:tasks.cephfs_test_runner:test_filtered_df (tasks.cephfs.test_misc.TestMisc) ... ok 2022-07-21T01:56:22.320 INFO:tasks.cephfs_test_runner:Starting test: test_fs_new (tasks.cephfs.test_misc.TestMisc) 2022-07-21T01:56:22.321 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2022-07-21T01:56:22.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901c6a9700 1 Processor -- start 2022-07-21T01:56:22.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901c6a9700 1 -- start start 2022-07-21T01:56:22.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901c6a9700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f90140608b0 0x7f901405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:22.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901c6a9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140afa10 0x7f90140b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:22.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901c6a9700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90140b42a0 con 0x7f90140b3de0 2022-07-21T01:56:22.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901c6a9700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9014060c90 con 0x7f90140608b0 2022-07-21T01:56:22.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901c6a9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90140af410 con 0x7f90140afa10 2022-07-21T01:56:22.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901ac46700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f90140b3de0 0x7f9014146350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53372/0 (socket says 172.21.15.140:53372) 2022-07-21T01:56:22.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901ac46700 1 -- 172.21.15.140:0/4187196009 learned_addr learned my addr 172.21.15.140:0/4187196009 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:22.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901a445700 1 --2- 172.21.15.140:0/4187196009 >> v2:172.21.15.152:3300/0 conn(0x7f90140608b0 0x7f901405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:22.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019c44700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140afa10 0x7f90140b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:22.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426546421 0 0) 0x7f90140b42a0 con 0x7f90140b3de0 2022-07-21T01:56:22.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9000003260 con 0x7f90140b3de0 2022-07-21T01:56:22.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4061044670 0 0) 0x7f9000003260 con 0x7f90140b3de0 2022-07-21T01:56:22.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140afa10 msgr2=0x7f90140b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:22.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140afa10 0x7f90140b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:22.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 >> v2:172.21.15.152:3300/0 conn(0x7f90140608b0 msgr2=0x7f901405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:22.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 --2- 172.21.15.140:0/4187196009 >> v2:172.21.15.152:3300/0 conn(0x7f90140608b0 0x7f901405bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:22.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90140af6b0 con 0x7f90140b3de0 2022-07-21T01:56:22.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f900c001660 con 0x7f90140b3de0 2022-07-21T01:56:22.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 >> v1:172.21.15.140:6789/0 conn(0x7f90140b3de0 legacy=0x7f9014146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:22.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9000003750 0x7f9000003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:22.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140afa10 0x7f9000004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:22.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90000049f0 0x7f9000004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:22.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9014060c90 con 0x7f90000049f0 2022-07-21T01:56:22.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90140af410 con 0x7f9000003750 2022-07-21T01:56:22.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900000b750 con 0x7f90140afa10 2022-07-21T01:56:22.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901ac46700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90000049f0 0x7f9000004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:22.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901a445700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9000003750 0x7f9000003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:22.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f9019c44700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140afa10 0x7f9000004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:22.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901ac46700 1 -- 172.21.15.140:0/4187196009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140afa10 msgr2=0x7f9000004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:22.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901ac46700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140afa10 0x7f9000004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:22.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901ac46700 1 -- 172.21.15.140:0/4187196009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9000003750 msgr2=0x7f9000003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:22.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901ac46700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9000003750 0x7f9000003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:22.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.429+0000 7f901ac46700 1 -- 172.21.15.140:0/4187196009 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9000003260 con 0x7f90000049f0 2022-07-21T01:56:22.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901ac46700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90000049f0 0x7f9000004e70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f900c001370 tx=0x7f90140b5a50).ready entity=mon.0 client_cookie=6046d4ff1379b9b6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:22.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f900c0071a0 con 0x7f90000049f0 2022-07-21T01:56:22.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c007300 con 0x7f90000049f0 2022-07-21T01:56:22.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f9019443700 1 -- 172.21.15.140:0/4187196009 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f900c00cbb0 con 0x7f90000049f0 2022-07-21T01:56:22.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 -- 172.21.15.140:0/4187196009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90000049f0 msgr2=0x7f9000004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:22.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 --2- 172.21.15.140:0/4187196009 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90000049f0 0x7f9000004e70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f900c001370 tx=0x7f90140b5a50).stop 2022-07-21T01:56:22.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901ac46700 1 -- 172.21.15.140:0/4187196009 reap_dead start 2022-07-21T01:56:22.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 -- 172.21.15.140:0/4187196009 shutdown_connections 2022-07-21T01:56:22.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 -- 172.21.15.140:0/4187196009 >> 172.21.15.140:0/4187196009 conn(0x7f90140b5ee0 msgr2=0x7f90140b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:22.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 -- 172.21.15.140:0/4187196009 shutdown_connections 2022-07-21T01:56:22.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 -- 172.21.15.140:0/4187196009 wait complete. 2022-07-21T01:56:22.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 Processor -- start 2022-07-21T01:56:22.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 -- start start 2022-07-21T01:56:22.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90000049f0 0x7f9014151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:22.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b1c20 0x7f9014143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:22.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140b0810 0x7f901409d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:22.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90140b20c0 con 0x7f90140b1c20 2022-07-21T01:56:22.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901409cfb0 con 0x7f90000049f0 2022-07-21T01:56:22.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f901c6a9700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90140b0e00 con 0x7f90140b0810 2022-07-21T01:56:22.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.433+0000 7f9019c44700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b1c20 0x7f9014143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:22.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f901a445700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90000049f0 0x7f9014151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:22.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f9019c44700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b1c20 0x7f9014143c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39240/0 (socket says 172.21.15.140:39240) 2022-07-21T01:56:22.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f9019c44700 1 -- 172.21.15.140:0/445358663 learned_addr learned my addr 172.21.15.140:0/445358663 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:22.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f901ac46700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140b0810 0x7f901409d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:22.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f901a445700 1 -- 172.21.15.140:0/445358663 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140b0810 msgr2=0x7f901409d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:22.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f901a445700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140b0810 0x7f901409d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:22.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f901a445700 1 -- 172.21.15.140:0/445358663 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b1c20 msgr2=0x7f9014143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:22.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f901a445700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b1c20 0x7f9014143c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:22.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f901a445700 1 -- 172.21.15.140:0/445358663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901409d940 con 0x7f90000049f0 2022-07-21T01:56:22.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f9019c44700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b1c20 0x7f9014143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:22.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f901a445700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90000049f0 0x7f9014151fa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f901000b000 tx=0x7f90100091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:22.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f900b7fe700 1 -- 172.21.15.140:0/445358663 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f901000f830 con 0x7f90000049f0 2022-07-21T01:56:22.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f900b7fe700 1 -- 172.21.15.140:0/445358663 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901000f990 con 0x7f90000049f0 2022-07-21T01:56:22.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f900b7fe700 1 -- 172.21.15.140:0/445358663 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f901000fcf0 con 0x7f90000049f0 2022-07-21T01:56:22.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f901c6a9700 1 -- 172.21.15.140:0/445358663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901409dbf0 con 0x7f90000049f0 2022-07-21T01:56:22.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f901c6a9700 1 -- 172.21.15.140:0/445358663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901409df00 con 0x7f90000049f0 2022-07-21T01:56:22.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f901c6a9700 1 -- 172.21.15.140:0/445358663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901409d170 con 0x7f90000049f0 2022-07-21T01:56:22.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.437+0000 7f900b7fe700 1 -- 172.21.15.140:0/445358663 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9010023070 con 0x7f90000049f0 2022-07-21T01:56:22.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.441+0000 7f900b7fe700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 0x7f8ff4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:22.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.441+0000 7f900b7fe700 1 -- 172.21.15.140:0/445358663 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f901006cc00 con 0x7f90000049f0 2022-07-21T01:56:22.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.441+0000 7f9019c44700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 0x7f8ff4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:22.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.441+0000 7f9019c44700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 0x7f8ff4055f30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f900400b000 tx=0x7f9004006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:22.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.449+0000 7f900b7fe700 1 -- 172.21.15.140:0/445358663 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9010072020 con 0x7f90000049f0 2022-07-21T01:56:22.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:22.617+0000 7f901c6a9700 1 -- 172.21.15.140:0/445358663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f901405c280 con 0x7f90000049f0 2022-07-21T01:56:23.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.301+0000 7f900b7fe700 1 -- 172.21.15.140:0/445358663 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v358) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9010043e00 con 0x7f90000049f0 2022-07-21T01:56:23.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.301+0000 7f901c6a9700 1 -- 172.21.15.140:0/445358663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 msgr2=0x7f8ff4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.301+0000 7f901c6a9700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 0x7f8ff4055f30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f900400b000 tx=0x7f9004006df0).stop 2022-07-21T01:56:23.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.305+0000 7f901c6a9700 1 -- 172.21.15.140:0/445358663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90000049f0 msgr2=0x7f9014151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.305+0000 7f901c6a9700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90000049f0 0x7f9014151fa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f901000b000 tx=0x7f90100091d0).stop 2022-07-21T01:56:23.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.305+0000 7f901c6a9700 1 -- 172.21.15.140:0/445358663 shutdown_connections 2022-07-21T01:56:23.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.305+0000 7f901c6a9700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 0x7f8ff4055f30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.305+0000 7f901c6a9700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90140b0810 0x7f901409d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.305+0000 7f901c6a9700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90140b1c20 0x7f9014143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.305+0000 7f901c6a9700 1 --2- 172.21.15.140:0/445358663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90000049f0 0x7f9014151fa0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.305+0000 7f901c6a9700 1 -- 172.21.15.140:0/445358663 >> 172.21.15.140:0/445358663 conn(0x7f90140b5ee0 msgr2=0x7f9014145f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:23.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.305+0000 7f901c6a9700 1 -- 172.21.15.140:0/445358663 shutdown_connections 2022-07-21T01:56:23.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.305+0000 7f901c6a9700 1 -- 172.21.15.140:0/445358663 wait complete. 2022-07-21T01:56:23.323 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:56:23.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.433+0000 7ff0aaf4b700 1 Processor -- start 2022-07-21T01:56:23.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.433+0000 7ff0aaf4b700 1 -- start start 2022-07-21T01:56:23.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff0a40b2ec0 0x7ff0a40b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:23.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff0a41461c0 0x7ff0a41465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:23.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a40b3f10 con 0x7ff0a405d160 2022-07-21T01:56:23.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a40b4ea0 con 0x7ff0a40b2ec0 2022-07-21T01:56:23.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a40b3790 con 0x7ff0a41461c0 2022-07-21T01:56:23.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0a9f49700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff0a40b2ec0 0x7ff0a40b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:23.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0a9f49700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff0a40b2ec0 0x7ff0a40b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39246/0 (socket says 172.21.15.140:39246) 2022-07-21T01:56:23.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0a9f49700 1 -- 172.21.15.140:0/2176910573 learned_addr learned my addr 172.21.15.140:0/2176910573 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:23.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aa74a700 1 --2- 172.21.15.140:0/2176910573 >> v2:172.21.15.152:3300/0 conn(0x7ff0a41461c0 0x7ff0a41465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:23.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0a9f49700 1 -- 172.21.15.140:0/2176910573 >> v2:172.21.15.152:3300/0 conn(0x7ff0a41461c0 msgr2=0x7ff0a41465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:23.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0a9f49700 1 --2- 172.21.15.140:0/2176910573 >> v2:172.21.15.152:3300/0 conn(0x7ff0a41461c0 0x7ff0a41465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0a9f49700 1 -- 172.21.15.140:0/2176910573 >> v1:172.21.15.152:6789/0 conn(0x7ff0a405d160 legacy=0x7ff0a4142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0a9f49700 1 -- 172.21.15.140:0/2176910573 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a40b2620 con 0x7ff0a40b2ec0 2022-07-21T01:56:23.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0a9f49700 1 --2- 172.21.15.140:0/2176910573 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0a40b2ec0 0x7ff0a40b32a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7ff0a0002f70 tx=0x7ff0a0006eb0).ready entity=mon.0 client_cookie=a99e49c68784a20e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:23.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0a8f47700 1 -- 172.21.15.140:0/2176910573 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0a000d5c0 con 0x7ff0a40b2ec0 2022-07-21T01:56:23.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0a8f47700 1 -- 172.21.15.140:0/2176910573 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a0000e90 con 0x7ff0a40b2ec0 2022-07-21T01:56:23.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0a8f47700 1 -- 172.21.15.140:0/2176910573 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0a000d950 con 0x7ff0a40b2ec0 2022-07-21T01:56:23.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/2176910573 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0a40b2ec0 msgr2=0x7ff0a40b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 --2- 172.21.15.140:0/2176910573 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0a40b2ec0 0x7ff0a40b32a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7ff0a0002f70 tx=0x7ff0a0006eb0).stop 2022-07-21T01:56:23.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/2176910573 shutdown_connections 2022-07-21T01:56:23.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 --2- 172.21.15.140:0/2176910573 >> v2:172.21.15.152:3300/0 conn(0x7ff0a41461c0 0x7ff0a41465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 --2- 172.21.15.140:0/2176910573 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0a40b2ec0 0x7ff0a40b32a0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/2176910573 >> 172.21.15.140:0/2176910573 conn(0x7ff0a4066e20 msgr2=0x7ff0a4067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:23.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/2176910573 shutdown_connections 2022-07-21T01:56:23.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.437+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/2176910573 wait complete. 2022-07-21T01:56:23.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aaf4b700 1 Processor -- start 2022-07-21T01:56:23.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aaf4b700 1 -- start start 2022-07-21T01:56:23.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aaf4b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0a405d160 0x7ff0a4144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:23.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aaf4b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0a40b2ec0 0x7ff0a4152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:23.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aaf4b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0a41461c0 0x7ff0a4152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:23.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aaf4b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a40b9020 con 0x7ff0a405d160 2022-07-21T01:56:23.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aaf4b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a40ada70 con 0x7ff0a41461c0 2022-07-21T01:56:23.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aaf4b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a40aefc0 con 0x7ff0a40b2ec0 2022-07-21T01:56:23.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0a9f49700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0a405d160 0x7ff0a4144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:23.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0a9f49700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0a405d160 0x7ff0a4144400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39252/0 (socket says 172.21.15.140:39252) 2022-07-21T01:56:23.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aa74a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0a41461c0 0x7ff0a4152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:23.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0a9f49700 1 -- 172.21.15.140:0/451213640 learned_addr learned my addr 172.21.15.140:0/451213640 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:23.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0a9748700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0a40b2ec0 0x7ff0a4152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:23.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0a9f49700 1 -- 172.21.15.140:0/451213640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0a40b2ec0 msgr2=0x7ff0a4152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0a9f49700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0a40b2ec0 0x7ff0a4152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0a9f49700 1 -- 172.21.15.140:0/451213640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0a41461c0 msgr2=0x7ff0a4152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0a9f49700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0a41461c0 0x7ff0a4152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0a9f49700 1 -- 172.21.15.140:0/451213640 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a000b1e0 con 0x7ff0a405d160 2022-07-21T01:56:23.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0a9748700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0a40b2ec0 0x7ff0a4152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:23.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aa74a700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0a41461c0 0x7ff0a4152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:23.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0a9f49700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0a405d160 0x7ff0a4144400 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7ff0a0002f70 tx=0x7ff0a0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:23.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff092ffd700 1 -- 172.21.15.140:0/451213640 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0a0011760 con 0x7ff0a405d160 2022-07-21T01:56:23.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/451213640 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a40af260 con 0x7ff0a405d160 2022-07-21T01:56:23.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff092ffd700 1 -- 172.21.15.140:0/451213640 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a00118c0 con 0x7ff0a405d160 2022-07-21T01:56:23.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff092ffd700 1 -- 172.21.15.140:0/451213640 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0a0011c20 con 0x7ff0a405d160 2022-07-21T01:56:23.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.441+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/451213640 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a40af920 con 0x7ff0a405d160 2022-07-21T01:56:23.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.445+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/451213640 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a40b28c0 con 0x7ff0a405d160 2022-07-21T01:56:23.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.445+0000 7ff092ffd700 1 -- 172.21.15.140:0/451213640 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff0a001a040 con 0x7ff0a405d160 2022-07-21T01:56:23.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.445+0000 7ff092ffd700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff084053a70 0x7ff084055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:23.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.445+0000 7ff092ffd700 1 -- 172.21.15.140:0/451213640 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5339+0+0 (secure 0 0 0) 0x7ff0a006cac0 con 0x7ff0a405d160 2022-07-21T01:56:23.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.445+0000 7ff0a9748700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff084053a70 0x7ff084055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:23.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.445+0000 7ff0a9748700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff084053a70 0x7ff084055f30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7ff09400d000 tx=0x7ff094009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:23.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.453+0000 7ff092ffd700 1 -- 172.21.15.140:0/451213640 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff0a0043ca0 con 0x7ff0a405d160 2022-07-21T01:56:23.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.577+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/451213640 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff0a40574e0 con 0x7ff0a405d160 2022-07-21T01:56:23.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff092ffd700 1 -- 172.21.15.140:0/451213640 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff0a0023020 con 0x7ff0a405d160 2022-07-21T01:56:23.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/451213640 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff084053a70 msgr2=0x7ff084055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff084053a70 0x7ff084055f30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7ff09400d000 tx=0x7ff094009040).stop 2022-07-21T01:56:23.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/451213640 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0a405d160 msgr2=0x7ff0a4144400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0a405d160 0x7ff0a4144400 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7ff0a0002f70 tx=0x7ff0a0015000).stop 2022-07-21T01:56:23.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/451213640 shutdown_connections 2022-07-21T01:56:23.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff084053a70 0x7ff084055f30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0a41461c0 0x7ff0a4152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff0a40b2ec0 0x7ff0a4152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 --2- 172.21.15.140:0/451213640 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff0a405d160 0x7ff0a4144400 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/451213640 >> 172.21.15.140:0/451213640 conn(0x7ff0a4066e20 msgr2=0x7ff0a40b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:23.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/451213640 shutdown_connections 2022-07-21T01:56:23.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.585+0000 7ff0aaf4b700 1 -- 172.21.15.140:0/451213640 wait complete. 2022-07-21T01:56:23.603 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:56:23.604 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:23.624 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:23.646 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:23.669 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:23.691 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:56:23.713 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:56:23.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:23.715 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:23.716 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:23.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:56:23.726 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:56:23.726 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:56:23.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:23.728 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:23.728 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:23.736 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:56:23.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:56:23.738 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:56:23.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:23.739 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:23.739 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:56:23.746 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:56:23.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:56:23.749 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:56:23.750 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:56:23.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.853+0000 7fe74b484700 1 Processor -- start 2022-07-21T01:56:23.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.853+0000 7fe74b484700 1 -- start start 2022-07-21T01:56:23.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74b484700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe74405bd90 0x7fe7440b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:23.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74b484700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7440aea30 con 0x7fe7440b5650 2022-07-21T01:56:23.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74b484700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7440aec60 con 0x7fe7440b1200 2022-07-21T01:56:23.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74b484700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe744142970 con 0x7fe74405bd90 2022-07-21T01:56:23.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74a482700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe7440b5650 0x7fe74405c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53408/0 (socket says 172.21.15.140:53408) 2022-07-21T01:56:23.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74a482700 1 -- 172.21.15.140:0/3311523761 learned_addr learned my addr 172.21.15.140:0/3311523761 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:23.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74ac83700 1 --2- 172.21.15.140:0/3311523761 >> v2:172.21.15.152:3300/0 conn(0x7fe74405bd90 0x7fe7440b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:23.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe749480700 1 -- 172.21.15.140:0/3311523761 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963192443 0 0) 0x7fe7440aea30 con 0x7fe7440b5650 2022-07-21T01:56:23.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe749480700 1 -- 172.21.15.140:0/3311523761 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe72c003260 con 0x7fe7440b5650 2022-07-21T01:56:23.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74ac83700 1 -- 172.21.15.140:0/3311523761 >> v1:172.21.15.152:6789/0 conn(0x7fe7440b1200 legacy=0x7fe744142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74ac83700 1 -- 172.21.15.140:0/3311523761 >> v1:172.21.15.140:6789/0 conn(0x7fe7440b5650 legacy=0x7fe74405c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74ac83700 1 -- 172.21.15.140:0/3311523761 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440af180 con 0x7fe74405bd90 2022-07-21T01:56:23.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74ac83700 1 --2- 172.21.15.140:0/3311523761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74405bd90 0x7fe7440b5a30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fe7440adec0 tx=0x7fe73c006df0).ready entity=mon.1 client_cookie=e432a96fe35cc4f0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:23.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe749480700 1 -- 172.21.15.140:0/3311523761 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe73c00d610 con 0x7fe74405bd90 2022-07-21T01:56:23.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe749480700 1 -- 172.21.15.140:0/3311523761 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c000e90 con 0x7fe74405bd90 2022-07-21T01:56:23.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe749480700 1 -- 172.21.15.140:0/3311523761 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe73c00d960 con 0x7fe74405bd90 2022-07-21T01:56:23.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74b484700 1 -- 172.21.15.140:0/3311523761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74405bd90 msgr2=0x7fe7440b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74b484700 1 --2- 172.21.15.140:0/3311523761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74405bd90 0x7fe7440b5a30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fe7440adec0 tx=0x7fe73c006df0).stop 2022-07-21T01:56:23.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74b484700 1 -- 172.21.15.140:0/3311523761 shutdown_connections 2022-07-21T01:56:23.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74b484700 1 --2- 172.21.15.140:0/3311523761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74405bd90 0x7fe7440b5a30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74b484700 1 -- 172.21.15.140:0/3311523761 >> 172.21.15.140:0/3311523761 conn(0x7fe7440634e0 msgr2=0x7fe7440b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:23.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74b484700 1 -- 172.21.15.140:0/3311523761 shutdown_connections 2022-07-21T01:56:23.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.857+0000 7fe74b484700 1 -- 172.21.15.140:0/3311523761 wait complete. 2022-07-21T01:56:23.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74b484700 1 Processor -- start 2022-07-21T01:56:23.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74b484700 1 -- start start 2022-07-21T01:56:23.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74b484700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe74405bd90 0x7fe7441555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:23.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74b484700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7440b1200 0x7fe7441406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:23.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74b484700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe7440b5650 0x7fe7440640d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:23.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74b484700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe744063ca0 con 0x7fe7440b5650 2022-07-21T01:56:23.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74b484700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74413d390 con 0x7fe7440b1200 2022-07-21T01:56:23.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74b484700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74413d630 con 0x7fe74405bd90 2022-07-21T01:56:23.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74ac83700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe7440b5650 0x7fe7440640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:23.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74ac83700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe7440b5650 0x7fe7440640d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39270/0 (socket says 172.21.15.140:39270) 2022-07-21T01:56:23.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74ac83700 1 -- 172.21.15.140:0/3525432804 learned_addr learned my addr 172.21.15.140:0/3525432804 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:23.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe749c81700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7440b1200 0x7fe7441406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:23.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74a482700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe74405bd90 0x7fe7441555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:23.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74ac83700 1 -- 172.21.15.140:0/3525432804 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe74405bd90 msgr2=0x7fe7441555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74ac83700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe74405bd90 0x7fe7441555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74ac83700 1 -- 172.21.15.140:0/3525432804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7440b1200 msgr2=0x7fe7441406e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:23.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74ac83700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7440b1200 0x7fe7441406e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:23.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74ac83700 1 -- 172.21.15.140:0/3525432804 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c00b180 con 0x7fe7440b5650 2022-07-21T01:56:23.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe749c81700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7440b1200 0x7fe7441406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:23.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74a482700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe74405bd90 0x7fe7441555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:23.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74ac83700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe7440b5650 0x7fe7440640d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7fe7440adec0 tx=0x7fe73c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:23.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe7337fe700 1 -- 172.21.15.140:0/3525432804 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe73c016070 con 0x7fe7440b5650 2022-07-21T01:56:23.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe7337fe700 1 -- 172.21.15.140:0/3525432804 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c00de70 con 0x7fe7440b5650 2022-07-21T01:56:23.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe7337fe700 1 -- 172.21.15.140:0/3525432804 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe73c013e90 con 0x7fe7440b5650 2022-07-21T01:56:23.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74b484700 1 -- 172.21.15.140:0/3525432804 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe744140bd0 con 0x7fe7440b5650 2022-07-21T01:56:23.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.861+0000 7fe74b484700 1 -- 172.21.15.140:0/3525432804 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74413cf60 con 0x7fe7440b5650 2022-07-21T01:56:23.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.865+0000 7fe74b484700 1 -- 172.21.15.140:0/3525432804 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74413e3d0 con 0x7fe7440b5650 2022-07-21T01:56:23.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.865+0000 7fe7337fe700 1 -- 172.21.15.140:0/3525432804 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe73c00f030 con 0x7fe7440b5650 2022-07-21T01:56:23.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.865+0000 7fe7337fe700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe724053950 0x7fe724055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:23.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.865+0000 7fe7337fe700 1 -- 172.21.15.140:0/3525432804 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fe73c06c9c0 con 0x7fe7440b5650 2022-07-21T01:56:23.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.865+0000 7fe74a482700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe724053950 0x7fe724055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:23.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.865+0000 7fe74a482700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe724053950 0x7fe724055e10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fe73400b000 tx=0x7fe734009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:23.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:23.873+0000 7fe7337fe700 1 -- 172.21.15.140:0/3525432804 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe74413e3d0 con 0x7fe7440b5650 2022-07-21T01:56:24.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.001+0000 7fe74b484700 1 -- 172.21.15.140:0/3525432804 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe74413e3d0 con 0x7fe7440b5650 2022-07-21T01:56:24.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe7337fe700 1 -- 172.21.15.140:0/3525432804 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v106) v1 ==== 81+0+28933 (secure 0 0 0) 0x7fe74413e3d0 con 0x7fe7440b5650 2022-07-21T01:56:24.009 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:24.009 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:56:24.009 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 106, 2022-07-21T01:56:24.010 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:56:24.010 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:56:24.010 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:56:21.278098+0000", 2022-07-21T01:56:24.010 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:56:24.011 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:56:24.011 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:56:24.011 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:56:24.011 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:56:24.011 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:56:24.012 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:56:24.012 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:56:24.012 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:56:24.012 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:24.013 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:56:24.013 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:56:24.013 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:56:24.013 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:56:24.013 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:56:24.014 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 20, 2022-07-21T01:56:24.014 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:56:24.014 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:56:24.014 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:56:24.015 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:56:24.015 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:56:24.015 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.015 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:56:24.015 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:56:24.016 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:56:24.016 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:56:24.016 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:56:24.016 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:56:24.017 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:56:24.017 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:56:24.017 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:56:24.017 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:56:24.017 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:56:24.018 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:56:24.018 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:56:24.018 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:56:24.018 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:56:24.019 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:56:24.019 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:56:24.019 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:56:24.019 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:56:24.019 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:56:24.020 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:56:24.020 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:56:24.020 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:56:24.020 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.021 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:56:24.021 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:56:24.021 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:56:24.021 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:56:24.021 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:56:24.022 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:56:24.022 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:56:24.022 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:56:24.022 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:56:24.023 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:56:24.023 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:56:24.023 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:56:24.023 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:56:24.023 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:56:24.024 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:56:24.024 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:56:24.024 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:56:24.024 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:56:24.024 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:56:24.025 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:56:24.025 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:56:24.025 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:56:24.025 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:56:24.026 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:56:24.026 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:56:24.026 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:56:24.026 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:56:24.027 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.027 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:56:24.027 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:56:24.027 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:56:24.027 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:56:24.028 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:56:24.028 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:56:24.028 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:56:24.028 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:56:24.029 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:56:24.029 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:56:24.029 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:56:24.029 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:56:24.029 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:56:24.030 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.030 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:56:24.030 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:56:24.030 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.031 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.031 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.031 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:56:24.031 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:56:24.032 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:56:24.032 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:56:24.032 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:56:24.032 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:56:24.032 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:56:24.033 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:56:24.033 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:56:24.033 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:56:24.033 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:56:24.034 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:56:24.034 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:56:24.034 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:56:24.034 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:56:24.035 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:56:24.035 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:56:24.035 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:56:24.035 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:56:24.035 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:56:24.036 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:56:24.036 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:56:24.036 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:56:24.036 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.037 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:56:24.037 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:56:24.037 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:56:24.037 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:56:24.037 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:56:24.038 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:56:24.038 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:56:24.038 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:56:24.038 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:56:24.039 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:56:24.039 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:56:24.039 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:56:24.039 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:56:24.039 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:56:24.040 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:56:24.040 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:56:24.040 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:56:24.040 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:56:24.041 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:56:24.041 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:56:24.041 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:56:24.041 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:56:24.041 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:56:24.042 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:56:24.042 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:56:24.042 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:56:24.042 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:56:24.043 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.043 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:56:24.043 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:56:24.043 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:56:24.043 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:56:24.044 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:56:24.044 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:56:24.044 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:56:24.044 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:56:24.044 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:56:24.045 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:56:24.045 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:56:24.045 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:56:24.045 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:56:24.046 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:56:24.046 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.046 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.046 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:24.047 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:56:24.047 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.047 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:56:24.047 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:56:24.047 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:24.048 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:24.048 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:24.048 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:24.048 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:24.049 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:24.049 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:24.049 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 101, 2022-07-21T01:56:24.049 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:24.049 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:24.050 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:24.050 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.050 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.050 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.051 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:56:24.051 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:24.051 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.051 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.051 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.052 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:56:24.052 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:24.052 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.052 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.052 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.053 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:24.053 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.053 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.053 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.054 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:56:24.054 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:24.054 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.054 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.054 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.055 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:56:24.055 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:24.055 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.055 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.056 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.056 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:24.056 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.056 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.056 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.057 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:56:24.057 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:24.057 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.057 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.058 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.058 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:56:24.058 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:24.058 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.058 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.059 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.059 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:24.059 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.059 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.060 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.060 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:56:24.060 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:24.060 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.060 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.061 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.061 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:56:24.061 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:56:24.061 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.061 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.062 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.062 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:56:24.062 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:56:24.062 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:56:24.063 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:56:24.063 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:24.063 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:24.063 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:24.063 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.064 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.064 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.064 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:56:24.064 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:56:24.065 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:24.065 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:24.065 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:24.065 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:24.065 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:24.066 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:24.066 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:24.066 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 101, 2022-07-21T01:56:24.066 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:24.067 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:24.067 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:24.067 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.067 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.067 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.068 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:56:24.068 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:24.068 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.068 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.069 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.069 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:56:24.069 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:24.069 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.069 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.070 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.070 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:24.070 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.070 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.071 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.071 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:56:24.071 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:24.071 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.071 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.072 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.072 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:56:24.072 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:24.072 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.072 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.073 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.073 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:24.073 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.073 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.074 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.074 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:56:24.074 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:24.074 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.074 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.075 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.075 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:56:24.075 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:24.075 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.075 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.076 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.076 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:24.076 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.076 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.077 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.077 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:56:24.077 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:24.077 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.077 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.078 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.078 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:56:24.078 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:56:24.078 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.079 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.079 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.079 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:56:24.079 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:56:24.079 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:56:24.080 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:56:24.080 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:24.080 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:24.080 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:24.081 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.081 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.081 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.081 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:56:24.081 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:56:24.082 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:24.082 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:24.082 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:24.082 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:24.083 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:24.083 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:24.083 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:24.083 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 100, 2022-07-21T01:56:24.084 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:24.084 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:24.084 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:24.085 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.085 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.085 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.085 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:56:24.085 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:24.086 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.086 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.086 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.086 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:56:24.087 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:24.087 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.087 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.087 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.087 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:24.088 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.088 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.088 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.088 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:56:24.088 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:24.089 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.089 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.089 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.089 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:56:24.090 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:24.090 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.090 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.090 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.091 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:24.091 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.091 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.091 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.091 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:56:24.092 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:24.092 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.092 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.092 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.092 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:56:24.093 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:24.093 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.093 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.094 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.094 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:24.094 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.095 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.095 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.095 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:56:24.095 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:24.096 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.096 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.096 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.096 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:56:24.096 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:56:24.097 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.097 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.097 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.097 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:56:24.098 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:56:24.098 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:56:24.098 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:56:24.098 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:24.098 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:24.099 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:24.099 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.099 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.099 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.100 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:56:24.100 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:56:24.100 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:24.100 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:24.100 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:24.101 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:24.101 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:24.101 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:24.101 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:24.102 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 101, 2022-07-21T01:56:24.102 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:24.102 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:24.103 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:24.103 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.103 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.103 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.104 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:56:24.104 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:24.104 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.104 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.104 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.105 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:56:24.105 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:24.105 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.105 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.106 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.106 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:24.106 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.106 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.106 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.107 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:56:24.107 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:24.107 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.107 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.108 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.108 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:56:24.108 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:24.108 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.108 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.109 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.109 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:24.109 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.109 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.109 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.110 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:56:24.110 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:24.110 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.110 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.111 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.111 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:56:24.111 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:24.111 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.111 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.112 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.112 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:24.112 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.113 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.113 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.113 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:56:24.113 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:24.114 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.114 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.114 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.114 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:56:24.115 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:56:24.115 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.115 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.115 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.116 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:56:24.116 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:56:24.116 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:56:24.116 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:56:24.116 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:24.117 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:24.117 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:24.117 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.117 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.118 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.118 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:56:24.118 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:56:24.118 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:24.118 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:24.119 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:24.119 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:24.119 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:24.119 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:24.120 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:24.120 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 100, 2022-07-21T01:56:24.120 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:24.120 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:24.120 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:24.121 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.121 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.121 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.121 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:56:24.121 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:24.122 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.122 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.122 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.122 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:56:24.123 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:24.123 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.123 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.123 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.124 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:24.124 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.124 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.124 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.124 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:56:24.125 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:24.125 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.125 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.125 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.125 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:56:24.126 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:24.126 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.126 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.126 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.127 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:24.127 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.127 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.128 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.128 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:56:24.128 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:24.128 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.129 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.129 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.129 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:56:24.129 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:24.130 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.130 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.130 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.130 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:24.130 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.131 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.131 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.131 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:56:24.131 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:24.132 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.132 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.132 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.132 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:56:24.132 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:56:24.133 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.133 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.133 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.133 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:56:24.134 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:56:24.134 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:56:24.134 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:56:24.134 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:24.135 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:24.135 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:24.135 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.135 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.135 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.136 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:56:24.136 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:56:24.136 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:24.136 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:24.137 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:24.137 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:24.137 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:24.137 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:24.138 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:24.138 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 100, 2022-07-21T01:56:24.138 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:24.139 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:24.139 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:24.139 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.139 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.140 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.140 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:56:24.140 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:24.140 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.140 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.141 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.141 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:56:24.141 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:24.141 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.142 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.142 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.142 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:24.142 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.142 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.143 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.143 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:56:24.143 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:24.143 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.144 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.144 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.144 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:56:24.144 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:24.144 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.145 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.145 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.145 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:24.145 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.145 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.146 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.146 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:56:24.146 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:24.146 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.147 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.147 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.147 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:56:24.147 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:24.147 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.148 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.148 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.148 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:24.148 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.148 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.149 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.149 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:56:24.149 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:24.149 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.150 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.150 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.150 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:56:24.150 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:56:24.150 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.151 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.151 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.151 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:56:24.151 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:56:24.152 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:56:24.152 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:56:24.152 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:24.152 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:24.152 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:24.153 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.153 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.153 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.153 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:56:24.153 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:56:24.154 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:24.154 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:24.154 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:24.154 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:24.155 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:24.155 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:24.155 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:24.155 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 101, 2022-07-21T01:56:24.155 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:24.156 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:24.156 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:24.156 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.156 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.156 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.157 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:56:24.157 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:24.157 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.157 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.158 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.158 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:56:24.158 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:24.158 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.158 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.159 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.159 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:24.159 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.159 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.160 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.160 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:56:24.160 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:24.160 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.160 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.161 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.161 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:56:24.161 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:24.161 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.161 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.162 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.162 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:24.162 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.162 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.163 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.163 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:56:24.163 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:24.163 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.163 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.164 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.164 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:56:24.164 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:24.164 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.164 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.165 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.165 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:24.165 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.165 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.166 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.166 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:56:24.166 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:24.166 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.166 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.167 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.167 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:56:24.167 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:56:24.167 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.168 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.168 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.168 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:56:24.168 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:56:24.168 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:56:24.169 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:56:24.169 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:24.169 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:24.169 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:24.169 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.170 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.170 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.170 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:56:24.170 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:56:24.171 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:56:24.171 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:56:24.171 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:56:24.171 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:56:24.171 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:56:24.172 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:56:24.172 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:56:24.172 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 101, 2022-07-21T01:56:24.172 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:56:24.173 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:56:24.173 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:56:24.173 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.173 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.173 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.174 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:56:24.174 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:24.174 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.174 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.174 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.175 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:56:24.175 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:24.175 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.175 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.176 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.176 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:56:24.176 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.176 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.176 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.177 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:56:24.177 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:24.177 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.177 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.177 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.178 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:56:24.178 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:24.178 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.178 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.179 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.179 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:56:24.179 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.179 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.180 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.180 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:56:24.180 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:24.180 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.180 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.181 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.181 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:56:24.181 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:24.181 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.181 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.182 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.182 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:56:24.182 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:56:24.182 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.183 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:56:24.183 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:56:24.183 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:24.183 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.183 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.184 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:56:24.184 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:56:24.184 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:56:24.184 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.185 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.185 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.185 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:56:24.185 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:56:24.185 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:56:24.186 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:56:24.186 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:56:24.186 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:56:24.186 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:56:24.187 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:24.187 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.187 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:24.187 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:56:24.187 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.188 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:56:24.188 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:24.188 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:24.188 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:24.188 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:24.189 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:24.189 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:56:24.189 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:24.189 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.190 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.190 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:56:24.190 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:24.190 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:24.191 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:24.191 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:24.191 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:24.191 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:56:24.191 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:24.192 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.192 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.192 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:56:24.192 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:24.192 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:24.193 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:24.193 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:24.193 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:24.193 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:56:24.193 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:24.194 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.194 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.194 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:56:24.194 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:24.195 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:24.195 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:24.195 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:24.195 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:24.196 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:56:24.196 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:24.196 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.196 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.196 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:56:24.197 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:24.197 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:24.197 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:24.197 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:24.198 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:24.198 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:56:24.198 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:24.198 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.198 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.199 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:56:24.199 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:24.199 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:24.200 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:24.200 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:24.200 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:24.200 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:56:24.201 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:24.202 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.203 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.203 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:56:24.204 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:24.204 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:24.204 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:24.204 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:24.205 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:24.205 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:56:24.205 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:24.205 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.206 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:24.206 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:56:24.206 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:56:24.206 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:56:24.207 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:56:24.207 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:56:24.207 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:56:24.207 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:56:24.208 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:56:24.208 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.209 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:56:24.209 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:56:24.209 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:56:24.209 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:56:24.210 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:56:24.210 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:56:24.210 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6833/727644549": "2022-07-22T01:56:17.616317+0000", 2022-07-21T01:56:24.211 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6832/727644549": "2022-07-22T01:56:17.616317+0000" 2022-07-21T01:56:24.211 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.211 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:56:24.211 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:56:24.212 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:56:24.212 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:56:24.212 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:56:24.212 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:56:24.213 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:56:24.213 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:56:24.213 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:24.213 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:24.214 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:56:24.214 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:56:24.214 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:56:24.214 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:56:24.215 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:56:24.215 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:56:24.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 -- 172.21.15.140:0/3525432804 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe724053950 msgr2=0x7fe724055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe724053950 0x7fe724055e10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fe73400b000 tx=0x7fe734009040).stop 2022-07-21T01:56:24.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 -- 172.21.15.140:0/3525432804 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe7440b5650 msgr2=0x7fe7440640d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe7440b5650 0x7fe7440640d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7fe7440adec0 tx=0x7fe73c014000).stop 2022-07-21T01:56:24.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 -- 172.21.15.140:0/3525432804 shutdown_connections 2022-07-21T01:56:24.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe724053950 0x7fe724055e10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe7440b5650 0x7fe7440640d0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7440b1200 0x7fe7441406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 --2- 172.21.15.140:0/3525432804 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe74405bd90 0x7fe7441555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 -- 172.21.15.140:0/3525432804 >> 172.21.15.140:0/3525432804 conn(0x7fe7440634e0 msgr2=0x7fe74405c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:24.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 -- 172.21.15.140:0/3525432804 shutdown_connections 2022-07-21T01:56:24.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.005+0000 7fe74b484700 1 -- 172.21.15.140:0/3525432804 wait complete. 2022-07-21T01:56:24.220 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:24.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 Processor -- start 2022-07-21T01:56:24.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 -- start start 2022-07-21T01:56:24.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb6b80afa10 0x7fb6b80b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6b80b3de0 0x7fb6b81452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b80b42a0 con 0x7fb6b80608b0 2022-07-21T01:56:24.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b8060c90 con 0x7fb6b80afa10 2022-07-21T01:56:24.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80af410 con 0x7fb6b80b3de0 2022-07-21T01:56:24.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6b7fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb6b80afa10 0x7fb6b80b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6b7fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb6b80afa10 0x7fb6b80b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:45014/0 (socket says 172.21.15.140:45014) 2022-07-21T01:56:24.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bd636700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6b80b3de0 0x7fb6b81452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6b7fff700 1 -- 172.21.15.140:0/3110880867 learned_addr learned my addr 172.21.15.140:0/3110880867 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:24.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6b7fff700 1 -- 172.21.15.140:0/3110880867 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6b80b3de0 msgr2=0x7fb6b81452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:24.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6b7fff700 1 --2- 172.21.15.140:0/3110880867 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6b80b3de0 0x7fb6b81452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6b7fff700 1 -- 172.21.15.140:0/3110880867 >> v1:172.21.15.152:6789/0 conn(0x7fb6b80608b0 legacy=0x7fb6b805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6b7fff700 1 -- 172.21.15.140:0/3110880867 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b80af6b0 con 0x7fb6b80afa10 2022-07-21T01:56:24.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bd636700 1 --2- 172.21.15.140:0/3110880867 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6b80b3de0 0x7fb6b81452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:24.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6b7fff700 1 --2- 172.21.15.140:0/3110880867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6b80afa10 0x7fb6b80b46d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fb6ac002f70 tx=0x7fb6ac006eb0).ready entity=mon.1 client_cookie=608552929fcc8119 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:24.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6b77fe700 1 -- 172.21.15.140:0/3110880867 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6ac00d570 con 0x7fb6b80afa10 2022-07-21T01:56:24.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6b77fe700 1 -- 172.21.15.140:0/3110880867 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac000e90 con 0x7fb6b80afa10 2022-07-21T01:56:24.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6b77fe700 1 -- 172.21.15.140:0/3110880867 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6ac00d900 con 0x7fb6b80afa10 2022-07-21T01:56:24.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 -- 172.21.15.140:0/3110880867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6b80afa10 msgr2=0x7fb6b80b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 --2- 172.21.15.140:0/3110880867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6b80afa10 0x7fb6b80b46d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fb6ac002f70 tx=0x7fb6ac006eb0).stop 2022-07-21T01:56:24.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 -- 172.21.15.140:0/3110880867 shutdown_connections 2022-07-21T01:56:24.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 --2- 172.21.15.140:0/3110880867 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6b80b3de0 0x7fb6b81452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 --2- 172.21.15.140:0/3110880867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6b80afa10 0x7fb6b80b46d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 -- 172.21.15.140:0/3110880867 >> 172.21.15.140:0/3110880867 conn(0x7fb6b80b5ee0 msgr2=0x7fb6b80b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:24.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 -- 172.21.15.140:0/3110880867 shutdown_connections 2022-07-21T01:56:24.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.325+0000 7fb6bf099700 1 -- 172.21.15.140:0/3110880867 wait complete. 2022-07-21T01:56:24.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bf099700 1 Processor -- start 2022-07-21T01:56:24.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bf099700 1 -- start start 2022-07-21T01:56:24.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bf099700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6b80608b0 0x7fb6b8141b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bf099700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6b80afa10 0x7fb6b8152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bf099700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6b80b3de0 0x7fb6b8152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bf099700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80b39d0 con 0x7fb6b80b3de0 2022-07-21T01:56:24.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bf099700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80b1fa0 con 0x7fb6b80afa10 2022-07-21T01:56:24.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bf099700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b80b2240 con 0x7fb6b80608b0 2022-07-21T01:56:24.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bd636700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6b80b3de0 0x7fb6b8152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6b80afa10 0x7fb6b8152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bd636700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6b80b3de0 0x7fb6b8152600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39284/0 (socket says 172.21.15.140:39284) 2022-07-21T01:56:24.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bd636700 1 -- 172.21.15.140:0/1785980790 learned_addr learned my addr 172.21.15.140:0/1785980790 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:24.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bce35700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6b80608b0 0x7fb6b8141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bd636700 1 -- 172.21.15.140:0/1785980790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6b80608b0 msgr2=0x7fb6b8141b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bd636700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6b80608b0 0x7fb6b8141b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bd636700 1 -- 172.21.15.140:0/1785980790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6b80afa10 msgr2=0x7fb6b8152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bd636700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6b80afa10 0x7fb6b8152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bd636700 1 -- 172.21.15.140:0/1785980790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b0009470 con 0x7fb6b80b3de0 2022-07-21T01:56:24.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6b7fff700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6b80afa10 0x7fb6b8152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:24.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6bd636700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6b80b3de0 0x7fb6b8152600 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7fb6b000c000 tx=0x7fb6b00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:24.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.329+0000 7fb6b5ffb700 1 -- 172.21.15.140:0/1785980790 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6b000c6a0 con 0x7fb6b80b3de0 2022-07-21T01:56:24.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.333+0000 7fb6b5ffb700 1 -- 172.21.15.140:0/1785980790 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b000c800 con 0x7fb6b80b3de0 2022-07-21T01:56:24.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.333+0000 7fb6bf099700 1 -- 172.21.15.140:0/1785980790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac00b1e0 con 0x7fb6b80b3de0 2022-07-21T01:56:24.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.333+0000 7fb6b5ffb700 1 -- 172.21.15.140:0/1785980790 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6b000cb60 con 0x7fb6b80b3de0 2022-07-21T01:56:24.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.333+0000 7fb6bf099700 1 -- 172.21.15.140:0/1785980790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b8152af0 con 0x7fb6b80b3de0 2022-07-21T01:56:24.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.333+0000 7fb6bf099700 1 -- 172.21.15.140:0/1785980790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b80574e0 con 0x7fb6b80b3de0 2022-07-21T01:56:24.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.333+0000 7fb6b5ffb700 1 -- 172.21.15.140:0/1785980790 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb6b0022070 con 0x7fb6b80b3de0 2022-07-21T01:56:24.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.333+0000 7fb6b5ffb700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb698053a10 0x7fb698055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.333+0000 7fb6b5ffb700 1 -- 172.21.15.140:0/1785980790 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fb6b006c350 con 0x7fb6b80b3de0 2022-07-21T01:56:24.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.333+0000 7fb6bce35700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb698053a10 0x7fb698055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.333+0000 7fb6bce35700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb698053a10 0x7fb698055ed0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fb6a400d000 tx=0x7fb6a4000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:24.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.337+0000 7fb6b5ffb700 1 -- 172.21.15.140:0/1785980790 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb6b0043440 con 0x7fb6b80b3de0 2022-07-21T01:56:24.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.501+0000 7fb6bf099700 1 -- 172.21.15.140:0/1785980790 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6b8069a60 con 0x7fb6b80b3de0 2022-07-21T01:56:24.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.501+0000 7fb6b5ffb700 1 -- 172.21.15.140:0/1785980790 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+1212 (secure 0 0 0) 0x7fb6b0016090 con 0x7fb6b80b3de0 2022-07-21T01:56:24.509 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:24.509 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/3024179649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":3024179649},{"type":"v1","addr":"172.21.15.140:6835","nonce":3024179649}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46},{"gid":7578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/165663159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":165663159},{"type":"v1","addr":"172.21.15.152:6833","nonce":165663159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[]} 2022-07-21T01:56:24.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 -- 172.21.15.140:0/1785980790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb698053a10 msgr2=0x7fb698055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb698053a10 0x7fb698055ed0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fb6a400d000 tx=0x7fb6a4000bc0).stop 2022-07-21T01:56:24.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 -- 172.21.15.140:0/1785980790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6b80b3de0 msgr2=0x7fb6b8152600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6b80b3de0 0x7fb6b8152600 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7fb6b000c000 tx=0x7fb6b00091d0).stop 2022-07-21T01:56:24.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 -- 172.21.15.140:0/1785980790 shutdown_connections 2022-07-21T01:56:24.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb698053a10 0x7fb698055ed0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6b80b3de0 0x7fb6b8152600 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6b80afa10 0x7fb6b8152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 --2- 172.21.15.140:0/1785980790 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb6b80608b0 0x7fb6b8141b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 -- 172.21.15.140:0/1785980790 >> 172.21.15.140:0/1785980790 conn(0x7fb6b80b5ee0 msgr2=0x7fb6b80b62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:24.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 -- 172.21.15.140:0/1785980790 shutdown_connections 2022-07-21T01:56:24.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.505+0000 7fb6bf099700 1 -- 172.21.15.140:0/1785980790 wait complete. 2022-07-21T01:56:24.516 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 49 2022-07-21T01:56:24.520 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:24.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45d1709700 1 Processor -- start 2022-07-21T01:56:24.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45d1709700 1 -- start start 2022-07-21T01:56:24.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45d1709700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45cc0b31e0 0x7f45cc13df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45d1709700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f45cc066160 0x7f45cc0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45d1709700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45cc0b7f50 con 0x7f45cc0b7b70 2022-07-21T01:56:24.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45d1709700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45cc0b2800 con 0x7f45cc066160 2022-07-21T01:56:24.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45d1709700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc0b2aa0 con 0x7f45cc0b31e0 2022-07-21T01:56:24.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45d0f08700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f45cc066160 0x7f45cc0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45cbfff700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f45cc0b7b70 0x7f45cc13d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:54548/0 (socket says 172.21.15.140:54548) 2022-07-21T01:56:24.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45cbfff700 1 -- 172.21.15.140:0/987370857 learned_addr learned my addr 172.21.15.140:0/987370857 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:24.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45c3fff700 1 --2- 172.21.15.140:0/987370857 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45cc0b31e0 0x7f45cc13df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45cb7fe700 1 -- 172.21.15.140:0/987370857 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159203044 0 0) 0x7f45cc0b7f50 con 0x7f45cc0b7b70 2022-07-21T01:56:24.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45cb7fe700 1 -- 172.21.15.140:0/987370857 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45ac003260 con 0x7f45cc0b7b70 2022-07-21T01:56:24.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45c3fff700 1 -- 172.21.15.140:0/987370857 >> v2:172.21.15.140:3300/0 conn(0x7f45cc066160 msgr2=0x7f45cc0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:24.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45c3fff700 1 --2- 172.21.15.140:0/987370857 >> v2:172.21.15.140:3300/0 conn(0x7f45cc066160 0x7f45cc0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45c3fff700 1 -- 172.21.15.140:0/987370857 >> v1:172.21.15.152:6789/0 conn(0x7f45cc0b7b70 legacy=0x7f45cc13d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45c3fff700 1 -- 172.21.15.140:0/987370857 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc0b2d40 con 0x7f45cc0b31e0 2022-07-21T01:56:24.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45c3fff700 1 --2- 172.21.15.140:0/987370857 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45cc0b31e0 0x7f45cc13df20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f45b4002f70 tx=0x7f45b4006df0).ready entity=mon.2 client_cookie=c74155bb87c51793 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:24.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45cb7fe700 1 -- 172.21.15.140:0/987370857 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b400d4a0 con 0x7f45cc0b31e0 2022-07-21T01:56:24.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45cb7fe700 1 -- 172.21.15.140:0/987370857 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b4000e90 con 0x7f45cc0b31e0 2022-07-21T01:56:24.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.617+0000 7f45cb7fe700 1 -- 172.21.15.140:0/987370857 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b400d7f0 con 0x7f45cc0b31e0 2022-07-21T01:56:24.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.621+0000 7f45d1709700 1 -- 172.21.15.140:0/987370857 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45cc0b31e0 msgr2=0x7f45cc13df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.621+0000 7f45d1709700 1 --2- 172.21.15.140:0/987370857 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45cc0b31e0 0x7f45cc13df20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f45b4002f70 tx=0x7f45b4006df0).stop 2022-07-21T01:56:24.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.621+0000 7f45d1709700 1 -- 172.21.15.140:0/987370857 shutdown_connections 2022-07-21T01:56:24.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.621+0000 7f45d1709700 1 --2- 172.21.15.140:0/987370857 >> v2:172.21.15.140:3300/0 conn(0x7f45cc066160 0x7f45cc0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.621+0000 7f45d1709700 1 --2- 172.21.15.140:0/987370857 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45cc0b31e0 0x7f45cc13df20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.621+0000 7f45d1709700 1 -- 172.21.15.140:0/987370857 >> 172.21.15.140:0/987370857 conn(0x7f45cc0656b0 msgr2=0x7f45cc0b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:24.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.621+0000 7f45d1709700 1 -- 172.21.15.140:0/987370857 shutdown_connections 2022-07-21T01:56:24.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.621+0000 7f45d1709700 1 -- 172.21.15.140:0/987370857 wait complete. 2022-07-21T01:56:24.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.621+0000 7f45d1709700 1 Processor -- start 2022-07-21T01:56:24.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.621+0000 7f45d1709700 1 -- start start 2022-07-21T01:56:24.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45d1709700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45cc066160 0x7f45cc13edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45d1709700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45cc0b31e0 0x7f45cc153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45d1709700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45cc0b7b70 0x7f45cc14fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45d1709700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc111340 con 0x7f45cc066160 2022-07-21T01:56:24.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45d1709700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc1532e0 con 0x7f45cc0b7b70 2022-07-21T01:56:24.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45d1709700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc153580 con 0x7f45cc0b31e0 2022-07-21T01:56:24.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45cbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45cc066160 0x7f45cc13edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45cbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45cc066160 0x7f45cc13edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39294/0 (socket says 172.21.15.140:39294) 2022-07-21T01:56:24.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45cbfff700 1 -- 172.21.15.140:0/2090398931 learned_addr learned my addr 172.21.15.140:0/2090398931 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:24.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45d0f08700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45cc0b7b70 0x7f45cc14fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45c3fff700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45cc0b31e0 0x7f45cc153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45cbfff700 1 -- 172.21.15.140:0/2090398931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45cc0b31e0 msgr2=0x7f45cc153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45cbfff700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45cc0b31e0 0x7f45cc153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45cbfff700 1 -- 172.21.15.140:0/2090398931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45cc0b7b70 msgr2=0x7f45cc14fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45cbfff700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45cc0b7b70 0x7f45cc14fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45cbfff700 1 -- 172.21.15.140:0/2090398931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b400b180 con 0x7f45cc066160 2022-07-21T01:56:24.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45cbfff700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45cc066160 0x7f45cc13edc0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7f45bc001110 tx=0x7f45bc007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:24.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45c9ffb700 1 -- 172.21.15.140:0/2090398931 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45bc0070d0 con 0x7f45cc066160 2022-07-21T01:56:24.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45c9ffb700 1 -- 172.21.15.140:0/2090398931 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc007230 con 0x7f45cc066160 2022-07-21T01:56:24.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45c9ffb700 1 -- 172.21.15.140:0/2090398931 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45bc005610 con 0x7f45cc066160 2022-07-21T01:56:24.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45d1709700 1 -- 172.21.15.140:0/2090398931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc153f10 con 0x7f45cc066160 2022-07-21T01:56:24.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45d1709700 1 -- 172.21.15.140:0/2090398931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc0ade70 con 0x7f45cc066160 2022-07-21T01:56:24.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.625+0000 7f45c9ffb700 1 -- 172.21.15.140:0/2090398931 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f45bc010070 con 0x7f45cc066160 2022-07-21T01:56:24.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.629+0000 7f45d1709700 1 -- 172.21.15.140:0/2090398931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45cc153740 con 0x7f45cc066160 2022-07-21T01:56:24.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.629+0000 7f45c9ffb700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45b0053950 0x7f45b0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.629+0000 7f45c9ffb700 1 -- 172.21.15.140:0/2090398931 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f45bc06b200 con 0x7f45cc066160 2022-07-21T01:56:24.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.629+0000 7f45c3fff700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45b0053950 0x7f45b0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.629+0000 7f45c3fff700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45b0053950 0x7f45b0055e10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f45b401a000 tx=0x7f45b4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:24.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.633+0000 7f45c9ffb700 1 -- 172.21.15.140:0/2090398931 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f45bc042360 con 0x7f45cc066160 2022-07-21T01:56:24.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.793+0000 7f45d1709700 1 -- 172.21.15.140:0/2090398931 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45cc0574e0 con 0x7f45cc066160 2022-07-21T01:56:24.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.793+0000 7f45c9ffb700 1 -- 172.21.15.140:0/2090398931 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+1212 (secure 0 0 0) 0x7f45bc045980 con 0x7f45cc066160 2022-07-21T01:56:24.798 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:24.798 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6470,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.140:6835/3024179649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":3024179649},{"type":"v1","addr":"172.21.15.140:6835","nonce":3024179649}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":46},{"gid":7578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/165663159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":165663159},{"type":"v1","addr":"172.21.15.152:6833","nonce":165663159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49}],"filesystems":[]} 2022-07-21T01:56:24.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 -- 172.21.15.140:0/2090398931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45b0053950 msgr2=0x7f45b0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45b0053950 0x7f45b0055e10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f45b401a000 tx=0x7f45b4014000).stop 2022-07-21T01:56:24.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 -- 172.21.15.140:0/2090398931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45cc066160 msgr2=0x7f45cc13edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45cc066160 0x7f45cc13edc0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7f45bc001110 tx=0x7f45bc007ad0).stop 2022-07-21T01:56:24.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 -- 172.21.15.140:0/2090398931 shutdown_connections 2022-07-21T01:56:24.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45b0053950 0x7f45b0055e10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45cc0b7b70 0x7f45cc14fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45cc0b31e0 0x7f45cc153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 --2- 172.21.15.140:0/2090398931 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45cc066160 0x7f45cc13edc0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 -- 172.21.15.140:0/2090398931 >> 172.21.15.140:0/2090398931 conn(0x7f45cc0656b0 msgr2=0x7f45cc0b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:24.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 -- 172.21.15.140:0/2090398931 shutdown_connections 2022-07-21T01:56:24.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.797+0000 7f45d1709700 1 -- 172.21.15.140:0/2090398931 wait complete. 2022-07-21T01:56:24.805 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 49 2022-07-21T01:56:24.815 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:56:24.815 INFO:tasks.ceph.mds.a:Stopping old one... 2022-07-21T01:56:24.816 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:56:24.816 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:56:24.817 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:56:24.817 INFO:tasks.ceph.mds.b:Stopping old one... 2022-07-21T01:56:24.818 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:56:24.818 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:56:24.826 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:56:24.827 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:56:24.829 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:56:24.830 DEBUG:teuthology.parallel:result is None 2022-07-21T01:56:24.834 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:56:24.835 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:56:24.837 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:56:24.838 DEBUG:teuthology.parallel:result is None 2022-07-21T01:56:24.838 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:56:24.907 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:56:24.908 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:56:24.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3addde700 1 Processor -- start 2022-07-21T01:56:24.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3addde700 1 -- start start 2022-07-21T01:56:24.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3addde700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 0x7fe3a8143750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3addde700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a813fd40 con 0x7fe3a813f8b0 2022-07-21T01:56:24.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3addde700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a80b19e0 con 0x7fe3a805bd90 2022-07-21T01:56:24.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3addde700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8143010 con 0x7fe3a80b1600 2022-07-21T01:56:24.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe39effd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe3a813f8b0 0x7fe3a805f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53462/0 (socket says 172.21.15.140:53462) 2022-07-21T01:56:24.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe39effd700 1 -- 172.21.15.140:0/3172423791 learned_addr learned my addr 172.21.15.140:0/3172423791 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:24.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3a77fe700 1 --2- 172.21.15.140:0/3172423791 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 0x7fe3a8143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3a6ffd700 1 -- 172.21.15.140:0/3172423791 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989869493 0 0) 0x7fe3a813fd40 con 0x7fe3a813f8b0 2022-07-21T01:56:24.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3a6ffd700 1 -- 172.21.15.140:0/3172423791 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe388003260 con 0x7fe3a813f8b0 2022-07-21T01:56:24.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3a77fe700 1 -- 172.21.15.140:0/3172423791 >> v1:172.21.15.152:6789/0 conn(0x7fe3a805bd90 legacy=0x7fe3a813e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3a77fe700 1 -- 172.21.15.140:0/3172423791 >> v1:172.21.15.140:6789/0 conn(0x7fe3a813f8b0 legacy=0x7fe3a805f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3a77fe700 1 -- 172.21.15.140:0/3172423791 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a81432b0 con 0x7fe3a80b1600 2022-07-21T01:56:24.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3a77fe700 1 --2- 172.21.15.140:0/3172423791 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 0x7fe3a8143750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fe398002f70 tx=0x7fe398006df0).ready entity=mon.2 client_cookie=d3ac0ca23a363816 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:24.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3a6ffd700 1 -- 172.21.15.140:0/3172423791 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39800d570 con 0x7fe3a80b1600 2022-07-21T01:56:24.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3a6ffd700 1 -- 172.21.15.140:0/3172423791 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe398000e90 con 0x7fe3a80b1600 2022-07-21T01:56:24.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.933+0000 7fe3a6ffd700 1 -- 172.21.15.140:0/3172423791 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39800d8c0 con 0x7fe3a80b1600 2022-07-21T01:56:24.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 -- 172.21.15.140:0/3172423791 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 msgr2=0x7fe3a8143750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 --2- 172.21.15.140:0/3172423791 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 0x7fe3a8143750 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fe398002f70 tx=0x7fe398006df0).stop 2022-07-21T01:56:24.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 -- 172.21.15.140:0/3172423791 shutdown_connections 2022-07-21T01:56:24.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 --2- 172.21.15.140:0/3172423791 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 0x7fe3a8143750 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 -- 172.21.15.140:0/3172423791 >> 172.21.15.140:0/3172423791 conn(0x7fe3a80b23b0 msgr2=0x7fe3a80b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:24.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 -- 172.21.15.140:0/3172423791 shutdown_connections 2022-07-21T01:56:24.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 -- 172.21.15.140:0/3172423791 wait complete. 2022-07-21T01:56:24.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 Processor -- start 2022-07-21T01:56:24.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 -- start start 2022-07-21T01:56:24.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe3a805bd90 0x7fe3a808f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 0x7fe3a80aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a8066c10 0x7fe3a8153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a813c880 con 0x7fe3a805bd90 2022-07-21T01:56:24.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a808fc40 con 0x7fe3a8066c10 2022-07-21T01:56:24.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3addde700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a808fee0 con 0x7fe3a80b1600 2022-07-21T01:56:24.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3a77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe3a805bd90 0x7fe3a808f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3a77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe3a805bd90 0x7fe3a808f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39320/0 (socket says 172.21.15.140:39320) 2022-07-21T01:56:24.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3a77fe700 1 -- 172.21.15.140:0/1511911249 learned_addr learned my addr 172.21.15.140:0/1511911249 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:24.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe39effd700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 0x7fe3a80aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3a7fff700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a8066c10 0x7fe3a8153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3a77fe700 1 -- 172.21.15.140:0/1511911249 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 msgr2=0x7fe3a80aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3a77fe700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 0x7fe3a80aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3a77fe700 1 -- 172.21.15.140:0/1511911249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a8066c10 msgr2=0x7fe3a8153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:24.957 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3a77fe700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a8066c10 0x7fe3a8153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:24.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3a77fe700 1 -- 172.21.15.140:0/1511911249 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39800b180 con 0x7fe3a805bd90 2022-07-21T01:56:24.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe39effd700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 0x7fe3a80aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:24.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3a7fff700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a8066c10 0x7fe3a8153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:24.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.937+0000 7fe3a77fe700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe3a805bd90 0x7fe3a808f610 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7fe398002f70 tx=0x7fe398014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:24.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.941+0000 7fe3a57fa700 1 -- 172.21.15.140:0/1511911249 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39800dea0 con 0x7fe3a805bd90 2022-07-21T01:56:24.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.941+0000 7fe3a57fa700 1 -- 172.21.15.140:0/1511911249 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe398013de0 con 0x7fe3a805bd90 2022-07-21T01:56:24.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.941+0000 7fe3addde700 1 -- 172.21.15.140:0/1511911249 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a8090180 con 0x7fe3a805bd90 2022-07-21T01:56:24.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.941+0000 7fe3a57fa700 1 -- 172.21.15.140:0/1511911249 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39801e5f0 con 0x7fe3a805bd90 2022-07-21T01:56:24.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.941+0000 7fe3addde700 1 -- 172.21.15.140:0/1511911249 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a81540b0 con 0x7fe3a805bd90 2022-07-21T01:56:24.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.941+0000 7fe3a57fa700 1 -- 172.21.15.140:0/1511911249 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe398025030 con 0x7fe3a805bd90 2022-07-21T01:56:24.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.941+0000 7fe3addde700 1 -- 172.21.15.140:0/1511911249 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a8154530 con 0x7fe3a805bd90 2022-07-21T01:56:24.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.941+0000 7fe3a57fa700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe38c053b00 0x7fe38c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:24.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.941+0000 7fe3a57fa700 1 -- 172.21.15.140:0/1511911249 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fe39806ca30 con 0x7fe3a805bd90 2022-07-21T01:56:24.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.941+0000 7fe39effd700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe38c053b00 0x7fe38c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:24.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.945+0000 7fe39effd700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe38c053b00 0x7fe38c055fc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fe39000d000 tx=0x7fe390009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:24.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:24.949+0000 7fe3a57fa700 1 -- 172.21.15.140:0/1511911249 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe3a8154530 con 0x7fe3a805bd90 2022-07-21T01:56:25.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:25.089+0000 7fe3addde700 1 -- 172.21.15.140:0/1511911249 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fe3a8154530 con 0x7fe3a805bd90 2022-07-21T01:56:26.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.017+0000 7fe3a57fa700 1 -- 172.21.15.140:0/1511911249 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v109) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe3a8154530 con 0x7fe3a805bd90 2022-07-21T01:56:26.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 -- 172.21.15.140:0/1511911249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe38c053b00 msgr2=0x7fe38c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe38c053b00 0x7fe38c055fc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fe39000d000 tx=0x7fe390009040).stop 2022-07-21T01:56:26.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 -- 172.21.15.140:0/1511911249 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe3a805bd90 msgr2=0x7fe3a808f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe3a805bd90 0x7fe3a808f610 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7fe398002f70 tx=0x7fe398014000).stop 2022-07-21T01:56:26.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 -- 172.21.15.140:0/1511911249 shutdown_connections 2022-07-21T01:56:26.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe38c053b00 0x7fe38c055fc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3a8066c10 0x7fe3a8153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3a80b1600 0x7fe3a80aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 --2- 172.21.15.140:0/1511911249 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe3a805bd90 0x7fe3a808f610 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 -- 172.21.15.140:0/1511911249 >> 172.21.15.140:0/1511911249 conn(0x7fe3a80b23b0 msgr2=0x7fe3a80abf60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:26.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 -- 172.21.15.140:0/1511911249 shutdown_connections 2022-07-21T01:56:26.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.021+0000 7fe3addde700 1 -- 172.21.15.140:0/1511911249 wait complete. 2022-07-21T01:56:26.033 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:56:26.041 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:56:26.042 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:56:26.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.145+0000 7fe0d8e26700 1 Processor -- start 2022-07-21T01:56:26.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.145+0000 7fe0d8e26700 1 -- start start 2022-07-21T01:56:26.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d8e26700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe0d40af660 0x7fe0d40afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d8e26700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d4140ba0 con 0x7fe0d405bd90 2022-07-21T01:56:26.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d8e26700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d4140010 con 0x7fe0d40b1480 2022-07-21T01:56:26.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d8e26700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d40ae410 con 0x7fe0d40af660 2022-07-21T01:56:26.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0c9d9b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe0d405bd90 0x7fe0d41436e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53478/0 (socket says 172.21.15.140:53478) 2022-07-21T01:56:26.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0c9d9b700 1 -- 172.21.15.140:0/1399431485 learned_addr learned my addr 172.21.15.140:0/1399431485 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:26.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d2d9d700 1 --2- 172.21.15.140:0/1399431485 >> v2:172.21.15.152:3300/0 conn(0x7fe0d40af660 0x7fe0d40afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d1d9b700 1 -- 172.21.15.140:0/1399431485 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562623981 0 0) 0x7fe0d4140ba0 con 0x7fe0d405bd90 2022-07-21T01:56:26.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d1d9b700 1 -- 172.21.15.140:0/1399431485 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0b4003260 con 0x7fe0d405bd90 2022-07-21T01:56:26.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d2d9d700 1 -- 172.21.15.140:0/1399431485 >> v1:172.21.15.152:6789/0 conn(0x7fe0d40b1480 legacy=0x7fe0d413f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d2d9d700 1 -- 172.21.15.140:0/1399431485 >> v1:172.21.15.140:6789/0 conn(0x7fe0d405bd90 legacy=0x7fe0d41436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d2d9d700 1 -- 172.21.15.140:0/1399431485 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d4142fa0 con 0x7fe0d40af660 2022-07-21T01:56:26.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d2d9d700 1 --2- 172.21.15.140:0/1399431485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d40af660 0x7fe0d40afaf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7fe0d40ad4e0 tx=0x7fe0cc0091d0).ready entity=mon.1 client_cookie=f115901c38fdc585 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:26.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d1d9b700 1 -- 172.21.15.140:0/1399431485 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0cc00b680 con 0x7fe0d40af660 2022-07-21T01:56:26.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d1d9b700 1 -- 172.21.15.140:0/1399431485 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0cc00b7e0 con 0x7fe0d40af660 2022-07-21T01:56:26.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d1d9b700 1 -- 172.21.15.140:0/1399431485 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0cc00b950 con 0x7fe0d40af660 2022-07-21T01:56:26.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1399431485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d40af660 msgr2=0x7fe0d40afaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d8e26700 1 --2- 172.21.15.140:0/1399431485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d40af660 0x7fe0d40afaf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7fe0d40ad4e0 tx=0x7fe0cc0091d0).stop 2022-07-21T01:56:26.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1399431485 shutdown_connections 2022-07-21T01:56:26.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d8e26700 1 --2- 172.21.15.140:0/1399431485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d40af660 0x7fe0d40afaf0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1399431485 >> 172.21.15.140:0/1399431485 conn(0x7fe0d40b23d0 msgr2=0x7fe0d40b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:26.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1399431485 shutdown_connections 2022-07-21T01:56:26.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.149+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1399431485 wait complete. 2022-07-21T01:56:26.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d8e26700 1 Processor -- start 2022-07-21T01:56:26.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d8e26700 1 -- start start 2022-07-21T01:56:26.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d8e26700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe0d405bd90 0x7fe0d40b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d8e26700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe0d40af660 0x7fe0d414fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d8e26700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d40b1480 0x7fe0d4153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d8e26700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d4091c20 con 0x7fe0d405bd90 2022-07-21T01:56:26.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d8e26700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d40930b0 con 0x7fe0d40b1480 2022-07-21T01:56:26.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d8e26700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d4093310 con 0x7fe0d40af660 2022-07-21T01:56:26.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe0d405bd90 0x7fe0d40b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe0d405bd90 0x7fe0d40b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39336/0 (socket says 172.21.15.140:39336) 2022-07-21T01:56:26.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d259c700 1 -- 172.21.15.140:0/1769658724 learned_addr learned my addr 172.21.15.140:0/1769658724 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:26.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0c9d9b700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe0d40af660 0x7fe0d414fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d2d9d700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d40b1480 0x7fe0d4153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d259c700 1 -- 172.21.15.140:0/1769658724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe0d40af660 msgr2=0x7fe0d414fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d259c700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe0d40af660 0x7fe0d414fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d259c700 1 -- 172.21.15.140:0/1769658724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d40b1480 msgr2=0x7fe0d4153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d259c700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d40b1480 0x7fe0d4153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d259c700 1 -- 172.21.15.140:0/1769658724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0cc009470 con 0x7fe0d405bd90 2022-07-21T01:56:26.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d2d9d700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d40b1480 0x7fe0d4153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:26.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0c9d9b700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe0d40af660 0x7fe0d414fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:26.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d259c700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe0d405bd90 0x7fe0d40b49d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fe0d4067f50 tx=0x7fe0c4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:26.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0cbfff700 1 -- 172.21.15.140:0/1769658724 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0c400ce30 con 0x7fe0d405bd90 2022-07-21T01:56:26.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0cbfff700 1 -- 172.21.15.140:0/1769658724 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c400f470 con 0x7fe0d405bd90 2022-07-21T01:56:26.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0cbfff700 1 -- 172.21.15.140:0/1769658724 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe0c400f7d0 con 0x7fe0d405bd90 2022-07-21T01:56:26.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1769658724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d4153260 con 0x7fe0d405bd90 2022-07-21T01:56:26.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.153+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1769658724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d408f610 con 0x7fe0d405bd90 2022-07-21T01:56:26.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.157+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1769658724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d40ac440 con 0x7fe0d405bd90 2022-07-21T01:56:26.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.157+0000 7fe0cbfff700 1 -- 172.21.15.140:0/1769658724 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe0c4019030 con 0x7fe0d405bd90 2022-07-21T01:56:26.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.157+0000 7fe0cbfff700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe0b8053950 0x7fe0b8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.157+0000 7fe0cbfff700 1 -- 172.21.15.140:0/1769658724 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe0c406afb0 con 0x7fe0d405bd90 2022-07-21T01:56:26.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.157+0000 7fe0c9d9b700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe0b8053950 0x7fe0b8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.157+0000 7fe0c9d9b700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe0b8053950 0x7fe0b8055e10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fe0bc00b000 tx=0x7fe0bc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:26.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.165+0000 7fe0cbfff700 1 -- 172.21.15.140:0/1769658724 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe0d40ac440 con 0x7fe0d405bd90 2022-07-21T01:56:26.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.329+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1769658724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe0d40ac440 con 0x7fe0d405bd90 2022-07-21T01:56:26.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0cbfff700 1 -- 172.21.15.140:0/1769658724 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v39) v1 ==== 80+0+4242 (secure 0 0 0) 0x7fe0d40ac440 con 0x7fe0d405bd90 2022-07-21T01:56:26.337 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:26.337 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:56:26.337 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:56:26.337 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.338 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:56:26.338 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:56:26.338 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.338 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:56:26.339 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:26.339 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:26.339 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.339 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.339 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.340 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.340 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:56:26.340 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:56:26.340 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.340 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:56:26.341 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:26.341 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:26.341 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.341 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.342 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.342 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.342 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:56:26.342 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:56:26.343 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.343 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:26.343 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:26.343 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.344 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.344 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.344 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.344 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:56:26.345 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:56:26.345 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.345 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:26.346 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:26.346 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.346 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.347 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.347 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.347 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:56:26.348 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:56:26.348 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.348 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:26.348 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:26.348 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.349 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.349 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.349 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.349 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:56:26.350 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:56:26.350 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.350 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:26.350 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:26.351 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.351 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.351 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.351 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.351 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:56:26.352 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:56:26.352 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.352 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:26.353 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:26.353 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.353 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.354 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.354 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.354 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:56:26.354 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:56:26.355 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.355 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:26.355 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:26.355 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.356 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.356 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.356 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.356 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:56:26.357 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:56:26.357 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.357 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:26.357 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:26.358 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.358 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.358 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.358 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.359 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:56:26.359 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:56:26.359 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.359 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:26.360 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:26.360 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.360 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.361 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.361 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.361 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:56:26.362 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:56:26.362 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.362 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:56:26.362 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:56:26.363 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:56:26.363 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.363 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.363 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.364 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:56:26.364 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:56:26.364 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.364 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:56:26.364 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:56:26.365 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:56:26.365 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.365 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.365 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.366 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:56:26.366 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:56:26.366 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.366 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:56:26.367 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:56:26.367 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:56:26.367 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.368 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.368 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.368 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:56:26.369 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:56:26.369 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.369 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:56:26.369 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:56:26.370 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:56:26.370 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.370 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.370 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.371 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.371 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:56:26.371 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:56:26.371 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.372 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:56:26.372 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:56:26.372 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.372 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.373 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:26.373 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:26.373 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:56:26.374 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:56:26.374 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:56:26.374 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:56:26.375 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:56:26.375 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:56:26.375 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.375 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:26.376 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:26.376 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:56:26.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1769658724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe0b8053950 msgr2=0x7fe0b8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe0b8053950 0x7fe0b8055e10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fe0bc00b000 tx=0x7fe0bc009040).stop 2022-07-21T01:56:26.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1769658724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe0d405bd90 msgr2=0x7fe0d40b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe0d405bd90 0x7fe0d40b49d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fe0d4067f50 tx=0x7fe0c4007c50).stop 2022-07-21T01:56:26.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1769658724 shutdown_connections 2022-07-21T01:56:26.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe0b8053950 0x7fe0b8055e10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0d40b1480 0x7fe0d4153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe0d40af660 0x7fe0d414fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 --2- 172.21.15.140:0/1769658724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe0d405bd90 0x7fe0d40b49d0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1769658724 >> 172.21.15.140:0/1769658724 conn(0x7fe0d40b23d0 msgr2=0x7fe0d40b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:26.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1769658724 shutdown_connections 2022-07-21T01:56:26.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.333+0000 7fe0d8e26700 1 -- 172.21.15.140:0/1769658724 wait complete. 2022-07-21T01:56:26.383 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:26.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8e11f700 1 Processor -- start 2022-07-21T01:56:26.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8e11f700 1 -- start start 2022-07-21T01:56:26.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8e11f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4c8813c790 0x7f4c880b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8e11f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4c8813cb70 0x7f4c88143000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8e11f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c88142ba0 con 0x7f4c880b3ca0 2022-07-21T01:56:26.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8e11f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c881429b0 con 0x7f4c8813cb70 2022-07-21T01:56:26.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8e11f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c88146330 con 0x7f4c8813c790 2022-07-21T01:56:26.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8d91e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4c8813cb70 0x7f4c88143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8d91e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4c8813cb70 0x7f4c88143000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39346/0 (socket says 172.21.15.140:39346) 2022-07-21T01:56:26.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8d91e700 1 -- 172.21.15.140:0/2864808396 learned_addr learned my addr 172.21.15.140:0/2864808396 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:26.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8d11d700 1 --2- 172.21.15.140:0/2864808396 >> v2:172.21.15.152:3300/0 conn(0x7f4c8813c790 0x7f4c880b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8d91e700 1 -- 172.21.15.140:0/2864808396 >> v2:172.21.15.152:3300/0 conn(0x7f4c8813c790 msgr2=0x7f4c880b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:26.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8d91e700 1 --2- 172.21.15.140:0/2864808396 >> v2:172.21.15.152:3300/0 conn(0x7f4c8813c790 0x7f4c880b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8d91e700 1 -- 172.21.15.140:0/2864808396 >> v1:172.21.15.152:6789/0 conn(0x7f4c880b3ca0 legacy=0x7f4c880b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:26.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8d91e700 1 -- 172.21.15.140:0/2864808396 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c880b4c00 con 0x7f4c8813cb70 2022-07-21T01:56:26.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.481+0000 7f4c8d91e700 1 --2- 172.21.15.140:0/2864808396 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c8813cb70 0x7f4c88143000 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f4c880608b0 tx=0x7f4c80009290).ready entity=mon.0 client_cookie=7a4a7a0875fb65a4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:26.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.485+0000 7f4c7ffff700 1 -- 172.21.15.140:0/2864808396 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c8000b830 con 0x7f4c8813cb70 2022-07-21T01:56:26.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.485+0000 7f4c7ffff700 1 -- 172.21.15.140:0/2864808396 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8000b990 con 0x7f4c8813cb70 2022-07-21T01:56:26.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.485+0000 7f4c7ffff700 1 -- 172.21.15.140:0/2864808396 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c8000bb40 con 0x7f4c8813cb70 2022-07-21T01:56:26.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.485+0000 7f4c8e11f700 1 -- 172.21.15.140:0/2864808396 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c8813cb70 msgr2=0x7f4c88143000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.485+0000 7f4c8e11f700 1 --2- 172.21.15.140:0/2864808396 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c8813cb70 0x7f4c88143000 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f4c880608b0 tx=0x7f4c80009290).stop 2022-07-21T01:56:26.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.485+0000 7f4c8e11f700 1 -- 172.21.15.140:0/2864808396 shutdown_connections 2022-07-21T01:56:26.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.485+0000 7f4c8e11f700 1 --2- 172.21.15.140:0/2864808396 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c8813cb70 0x7f4c88143000 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.485+0000 7f4c8e11f700 1 --2- 172.21.15.140:0/2864808396 >> v2:172.21.15.152:3300/0 conn(0x7f4c8813c790 0x7f4c880b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.485+0000 7f4c8e11f700 1 -- 172.21.15.140:0/2864808396 >> 172.21.15.140:0/2864808396 conn(0x7f4c880678b0 msgr2=0x7f4c880b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:26.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.485+0000 7f4c8e11f700 1 -- 172.21.15.140:0/2864808396 shutdown_connections 2022-07-21T01:56:26.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.485+0000 7f4c8e11f700 1 -- 172.21.15.140:0/2864808396 wait complete. 2022-07-21T01:56:26.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8e11f700 1 Processor -- start 2022-07-21T01:56:26.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8e11f700 1 -- start start 2022-07-21T01:56:26.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8e11f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c880b3ca0 0x7f4c88151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8e11f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c8813c790 0x7f4c880b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8e11f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c8813cb70 0x7f4c88155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8e11f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88152430 con 0x7f4c8813cb70 2022-07-21T01:56:26.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8e11f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c881555e0 con 0x7f4c880b3ca0 2022-07-21T01:56:26.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8e11f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c881407e0 con 0x7f4c8813c790 2022-07-21T01:56:26.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8d91e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c8813cb70 0x7f4c88155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8d11d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c880b3ca0 0x7f4c88151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8c91c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c8813c790 0x7f4c880b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8d91e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c8813cb70 0x7f4c88155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39352/0 (socket says 172.21.15.140:39352) 2022-07-21T01:56:26.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8d11d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c880b3ca0 0x7f4c88151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:45088/0 (socket says 172.21.15.140:45088) 2022-07-21T01:56:26.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8d91e700 1 -- 172.21.15.140:0/3859842200 learned_addr learned my addr 172.21.15.140:0/3859842200 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:26.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8d91e700 1 -- 172.21.15.140:0/3859842200 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c8813c790 msgr2=0x7f4c880b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8d91e700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c8813c790 0x7f4c880b7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8d91e700 1 -- 172.21.15.140:0/3859842200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c880b3ca0 msgr2=0x7f4c88151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8d91e700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c880b3ca0 0x7f4c88151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8d91e700 1 -- 172.21.15.140:0/3859842200 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c80009530 con 0x7f4c8813cb70 2022-07-21T01:56:26.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8d91e700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c8813cb70 0x7f4c88155a40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7f4c880608b0 tx=0x7f4c80016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:26.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c7e7fc700 1 -- 172.21.15.140:0/3859842200 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c800032f0 con 0x7f4c8813cb70 2022-07-21T01:56:26.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c7e7fc700 1 -- 172.21.15.140:0/3859842200 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80021440 con 0x7f4c8813cb70 2022-07-21T01:56:26.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c7e7fc700 1 -- 172.21.15.140:0/3859842200 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c800217a0 con 0x7f4c8813cb70 2022-07-21T01:56:26.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8e11f700 1 -- 172.21.15.140:0/3859842200 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c880b8030 con 0x7f4c8813cb70 2022-07-21T01:56:26.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.489+0000 7f4c8e11f700 1 -- 172.21.15.140:0/3859842200 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c880b2b90 con 0x7f4c8813cb70 2022-07-21T01:56:26.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.493+0000 7f4c8e11f700 1 -- 172.21.15.140:0/3859842200 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8813d340 con 0x7f4c8813cb70 2022-07-21T01:56:26.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.493+0000 7f4c7e7fc700 1 -- 172.21.15.140:0/3859842200 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4c8000f070 con 0x7f4c8813cb70 2022-07-21T01:56:26.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.493+0000 7f4c7e7fc700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 0x7f4c70055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.493+0000 7f4c7e7fc700 1 -- 172.21.15.140:0/3859842200 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f4c8006c6b0 con 0x7f4c8813cb70 2022-07-21T01:56:26.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.493+0000 7f4c8d11d700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 0x7f4c70055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.493+0000 7f4c8d11d700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 0x7f4c70055fc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f4c8400c000 tx=0x7f4c84008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:26.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.501+0000 7f4c7e7fc700 1 -- 172.21.15.140:0/3859842200 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4c8813d340 con 0x7f4c8813cb70 2022-07-21T01:56:26.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.625+0000 7f4c8e11f700 1 -- 172.21.15.140:0/3859842200 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c8813d340 con 0x7f4c8813cb70 2022-07-21T01:56:26.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c7e7fc700 1 -- 172.21.15.140:0/3859842200 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+13287 (secure 0 0 0) 0x7f4c8813d340 con 0x7f4c8813cb70 2022-07-21T01:56:26.633 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:26.633 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":109,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:26.014448+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:26.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 -- 172.21.15.140:0/3859842200 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 msgr2=0x7f4c70055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 0x7f4c70055fc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f4c8400c000 tx=0x7f4c84008040).stop 2022-07-21T01:56:26.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 -- 172.21.15.140:0/3859842200 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c8813cb70 msgr2=0x7f4c88155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c8813cb70 0x7f4c88155a40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7f4c880608b0 tx=0x7f4c80016000).stop 2022-07-21T01:56:26.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 -- 172.21.15.140:0/3859842200 shutdown_connections 2022-07-21T01:56:26.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c70053b00 0x7f4c70055fc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c8813cb70 0x7f4c88155a40 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c8813c790 0x7f4c880b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 --2- 172.21.15.140:0/3859842200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c880b3ca0 0x7f4c88151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 -- 172.21.15.140:0/3859842200 >> 172.21.15.140:0/3859842200 conn(0x7f4c880678b0 msgr2=0x7f4c88146470 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:26.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 -- 172.21.15.140:0/3859842200 shutdown_connections 2022-07-21T01:56:26.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.629+0000 7f4c8e11f700 1 -- 172.21.15.140:0/3859842200 wait complete. 2022-07-21T01:56:26.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:56:26.646 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:56:26.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.745+0000 7f912759e700 1 Processor -- start 2022-07-21T01:56:26.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.745+0000 7f912759e700 1 -- start start 2022-07-21T01:56:26.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f91280aef30 0x7f91281445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9128144a90 0x7f9128129340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91280b5dd0 con 0x7f91280b59f0 2022-07-21T01:56:26.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91280b1750 con 0x7f91280aef30 2022-07-21T01:56:26.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9128140ef0 con 0x7f9128144a90 2022-07-21T01:56:26.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912659c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f91280b59f0 0x7f91281407e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53502/0 (socket says 172.21.15.140:53502) 2022-07-21T01:56:26.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912659c700 1 -- 172.21.15.140:0/3875967856 learned_addr learned my addr 172.21.15.140:0/3875967856 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:26.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f9126d9d700 1 --2- 172.21.15.140:0/3875967856 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9128144a90 0x7f9128129340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f9125d9b700 1 --2- 172.21.15.140:0/3875967856 >> v2:172.21.15.152:3300/0 conn(0x7f91280aef30 0x7f91281445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912559a700 1 -- 172.21.15.140:0/3875967856 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911640061 0 0) 0x7f91280b5dd0 con 0x7f91280b59f0 2022-07-21T01:56:26.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912559a700 1 -- 172.21.15.140:0/3875967856 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9108003260 con 0x7f91280b59f0 2022-07-21T01:56:26.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f9126d9d700 1 -- 172.21.15.140:0/3875967856 >> v2:172.21.15.152:3300/0 conn(0x7f91280aef30 msgr2=0x7f91281445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:26.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f9126d9d700 1 --2- 172.21.15.140:0/3875967856 >> v2:172.21.15.152:3300/0 conn(0x7f91280aef30 0x7f91281445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f9126d9d700 1 -- 172.21.15.140:0/3875967856 >> v1:172.21.15.140:6789/0 conn(0x7f91280b59f0 legacy=0x7f91281407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f9126d9d700 1 -- 172.21.15.140:0/3875967856 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280652a0 con 0x7f9128144a90 2022-07-21T01:56:26.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f9126d9d700 1 --2- 172.21.15.140:0/3875967856 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9128144a90 0x7f9128129340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f9120002f70 tx=0x7f9120006df0).ready entity=mon.2 client_cookie=74e85e053349be41 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:26.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912559a700 1 -- 172.21.15.140:0/3875967856 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f912000d4a0 con 0x7f9128144a90 2022-07-21T01:56:26.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912559a700 1 -- 172.21.15.140:0/3875967856 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9120000e90 con 0x7f9128144a90 2022-07-21T01:56:26.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912559a700 1 -- 172.21.15.140:0/3875967856 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f912000d7f0 con 0x7f9128144a90 2022-07-21T01:56:26.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 -- 172.21.15.140:0/3875967856 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9128144a90 msgr2=0x7f9128129340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 --2- 172.21.15.140:0/3875967856 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9128144a90 0x7f9128129340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f9120002f70 tx=0x7f9120006df0).stop 2022-07-21T01:56:26.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 -- 172.21.15.140:0/3875967856 shutdown_connections 2022-07-21T01:56:26.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 --2- 172.21.15.140:0/3875967856 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9128144a90 0x7f9128129340 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 --2- 172.21.15.140:0/3875967856 >> v2:172.21.15.152:3300/0 conn(0x7f91280aef30 0x7f91281445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 -- 172.21.15.140:0/3875967856 >> 172.21.15.140:0/3875967856 conn(0x7f9128063480 msgr2=0x7f91280b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:26.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 -- 172.21.15.140:0/3875967856 shutdown_connections 2022-07-21T01:56:26.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 -- 172.21.15.140:0/3875967856 wait complete. 2022-07-21T01:56:26.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 Processor -- start 2022-07-21T01:56:26.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.749+0000 7f912759e700 1 -- start start 2022-07-21T01:56:26.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f912759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91280aef30 0x7f91280b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f912759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91280b59f0 0x7f91280acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f912759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9128144a90 0x7f91281524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f912759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9128130440 con 0x7f91280b59f0 2022-07-21T01:56:26.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f912759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9128151de0 con 0x7f9128144a90 2022-07-21T01:56:26.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f912759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91280acfd0 con 0x7f91280aef30 2022-07-21T01:56:26.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f9125d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91280b59f0 0x7f91280acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f9125d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91280b59f0 0x7f91280acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39362/0 (socket says 172.21.15.140:39362) 2022-07-21T01:56:26.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f9125d9b700 1 -- 172.21.15.140:0/2875080343 learned_addr learned my addr 172.21.15.140:0/2875080343 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:26.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f9126d9d700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9128144a90 0x7f91281524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f912659c700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91280aef30 0x7f91280b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f9125d9b700 1 -- 172.21.15.140:0/2875080343 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91280aef30 msgr2=0x7f91280b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f9125d9b700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91280aef30 0x7f91280b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f9125d9b700 1 -- 172.21.15.140:0/2875080343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9128144a90 msgr2=0x7f91281524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f9125d9b700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9128144a90 0x7f91281524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f9125d9b700 1 -- 172.21.15.140:0/2875080343 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912000b180 con 0x7f91280b59f0 2022-07-21T01:56:26.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f9125d9b700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91280b59f0 0x7f91280acae0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f911000c000 tx=0x7f91100091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:26.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f911f7fe700 1 -- 172.21.15.140:0/2875080343 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f911000c6a0 con 0x7f91280b59f0 2022-07-21T01:56:26.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f911f7fe700 1 -- 172.21.15.140:0/2875080343 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911000c800 con 0x7f91280b59f0 2022-07-21T01:56:26.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f912759e700 1 -- 172.21.15.140:0/2875080343 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91280ae9a0 con 0x7f91280b59f0 2022-07-21T01:56:26.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f911f7fe700 1 -- 172.21.15.140:0/2875080343 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f911000cb60 con 0x7f91280b59f0 2022-07-21T01:56:26.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f912759e700 1 -- 172.21.15.140:0/2875080343 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91280ae690 con 0x7f91280b59f0 2022-07-21T01:56:26.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.753+0000 7f912759e700 1 -- 172.21.15.140:0/2875080343 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9128143e60 con 0x7f91280b59f0 2022-07-21T01:56:26.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.757+0000 7f911f7fe700 1 -- 172.21.15.140:0/2875080343 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9110022070 con 0x7f91280b59f0 2022-07-21T01:56:26.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.757+0000 7f911f7fe700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f910c053b30 0x7f910c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:26.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.757+0000 7f911f7fe700 1 -- 172.21.15.140:0/2875080343 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f911006c8e0 con 0x7f91280b59f0 2022-07-21T01:56:26.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.757+0000 7f912659c700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f910c053b30 0x7f910c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:26.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.757+0000 7f912659c700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f910c053b30 0x7f910c055ff0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f911800d000 tx=0x7f9118009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:26.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.761+0000 7f911f7fe700 1 -- 172.21.15.140:0/2875080343 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9110043a20 con 0x7f91280b59f0 2022-07-21T01:56:26.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 -- 172.21.15.140:0/2875080343 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f91280aeb30 con 0x7f91280b59f0 2022-07-21T01:56:26.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f911f7fe700 1 -- 172.21.15.140:0/2875080343 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v109) v1 ==== 110+0+10 (secure 0 0 0) 0x7f91100290b0 con 0x7f91280b59f0 2022-07-21T01:56:26.893 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:56:26.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 -- 172.21.15.140:0/2875080343 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f910c053b30 msgr2=0x7f910c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f910c053b30 0x7f910c055ff0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f911800d000 tx=0x7f9118009040).stop 2022-07-21T01:56:26.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 -- 172.21.15.140:0/2875080343 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91280b59f0 msgr2=0x7f91280acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:26.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91280b59f0 0x7f91280acae0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f911000c000 tx=0x7f91100091d0).stop 2022-07-21T01:56:26.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 -- 172.21.15.140:0/2875080343 shutdown_connections 2022-07-21T01:56:26.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f910c053b30 0x7f910c055ff0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9128144a90 0x7f91281524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91280b59f0 0x7f91280acae0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 --2- 172.21.15.140:0/2875080343 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91280aef30 0x7f91280b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:26.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 -- 172.21.15.140:0/2875080343 >> 172.21.15.140:0/2875080343 conn(0x7f9128063480 msgr2=0x7f9128065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:26.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 -- 172.21.15.140:0/2875080343 shutdown_connections 2022-07-21T01:56:26.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:26.889+0000 7f912759e700 1 -- 172.21.15.140:0/2875080343 wait complete. 2022-07-21T01:56:26.906 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:56:27.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.009+0000 7ff3fffff700 1 Processor -- start 2022-07-21T01:56:27.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.009+0000 7ff3fffff700 1 -- start start 2022-07-21T01:56:27.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff400061fd0 0x7ff4001465e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40005cd40 con 0x7ff40005c960 2022-07-21T01:56:27.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fffff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff400060400 con 0x7ff4000b2230 2022-07-21T01:56:27.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fffff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff400061440 con 0x7ff400061fd0 2022-07-21T01:56:27.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3feffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff40005c960 0x7ff4000618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53516/0 (socket says 172.21.15.140:53516) 2022-07-21T01:56:27.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3feffd700 1 -- 172.21.15.140:0/3679421809 learned_addr learned my addr 172.21.15.140:0/3679421809 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:27.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3ff7fe700 1 --2- 172.21.15.140:0/3679421809 >> v2:172.21.15.152:3300/0 conn(0x7ff400061fd0 0x7ff4001465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fe7fc700 1 -- 172.21.15.140:0/3679421809 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595135360 0 0) 0x7ff40005cd40 con 0x7ff40005c960 2022-07-21T01:56:27.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3ff7fe700 1 -- 172.21.15.140:0/3679421809 >> v1:172.21.15.152:6789/0 conn(0x7ff4000b2230 legacy=0x7ff400142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3ff7fe700 1 -- 172.21.15.140:0/3679421809 >> v1:172.21.15.140:6789/0 conn(0x7ff40005c960 legacy=0x7ff4000618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3ff7fe700 1 -- 172.21.15.140:0/3679421809 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400060680 con 0x7ff400061fd0 2022-07-21T01:56:27.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3ff7fe700 1 --2- 172.21.15.140:0/3679421809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff400061fd0 0x7ff4001465e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7ff3f80032f0 tx=0x7ff3f80091d0).ready entity=mon.1 client_cookie=e151d8398351a661 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:27.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fe7fc700 1 -- 172.21.15.140:0/3679421809 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3f8009ea0 con 0x7ff400061fd0 2022-07-21T01:56:27.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fe7fc700 1 -- 172.21.15.140:0/3679421809 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f800b8a0 con 0x7ff400061fd0 2022-07-21T01:56:27.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fe7fc700 1 -- 172.21.15.140:0/3679421809 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3f800ba10 con 0x7ff400061fd0 2022-07-21T01:56:27.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fffff700 1 -- 172.21.15.140:0/3679421809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff400061fd0 msgr2=0x7ff4001465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fffff700 1 --2- 172.21.15.140:0/3679421809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff400061fd0 0x7ff4001465e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7ff3f80032f0 tx=0x7ff3f80091d0).stop 2022-07-21T01:56:27.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fffff700 1 -- 172.21.15.140:0/3679421809 shutdown_connections 2022-07-21T01:56:27.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fffff700 1 --2- 172.21.15.140:0/3679421809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff400061fd0 0x7ff4001465e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fffff700 1 -- 172.21.15.140:0/3679421809 >> 172.21.15.140:0/3679421809 conn(0x7ff4000b61f0 msgr2=0x7ff4000b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:27.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fffff700 1 -- 172.21.15.140:0/3679421809 shutdown_connections 2022-07-21T01:56:27.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.013+0000 7ff3fffff700 1 -- 172.21.15.140:0/3679421809 wait complete. 2022-07-21T01:56:27.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fffff700 1 Processor -- start 2022-07-21T01:56:27.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fffff700 1 -- start start 2022-07-21T01:56:27.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff40005c960 0x7ff400149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff400061fd0 0x7ff400141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4000b2230 0x7ff400142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4000991c0 con 0x7ff400061fd0 2022-07-21T01:56:27.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4000ada60 con 0x7ff40005c960 2022-07-21T01:56:27.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4000add00 con 0x7ff4000b2230 2022-07-21T01:56:27.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff400061fd0 0x7ff400141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3f67fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff400061fd0 0x7ff400141ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39376/0 (socket says 172.21.15.140:39376) 2022-07-21T01:56:27.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3f67fc700 1 -- 172.21.15.140:0/936376810 learned_addr learned my addr 172.21.15.140:0/936376810 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:27.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3ff7fe700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4000b2230 0x7ff400142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3feffd700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff40005c960 0x7ff400149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3f67fc700 1 -- 172.21.15.140:0/936376810 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4000b2230 msgr2=0x7ff400142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3f67fc700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4000b2230 0x7ff400142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3f67fc700 1 -- 172.21.15.140:0/936376810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff40005c960 msgr2=0x7ff400149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3f67fc700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff40005c960 0x7ff400149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3f67fc700 1 -- 172.21.15.140:0/936376810 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f8009470 con 0x7ff400061fd0 2022-07-21T01:56:27.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3f67fc700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff400061fd0 0x7ff400141ba0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7ff4000b6e40 tx=0x7ff3e8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:27.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fcff9700 1 -- 172.21.15.140:0/936376810 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3e8012400 con 0x7ff400061fd0 2022-07-21T01:56:27.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fcff9700 1 -- 172.21.15.140:0/936376810 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e8012560 con 0x7ff400061fd0 2022-07-21T01:56:27.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fcff9700 1 -- 172.21.15.140:0/936376810 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3e80128c0 con 0x7ff400061fd0 2022-07-21T01:56:27.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fffff700 1 -- 172.21.15.140:0/936376810 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40009be90 con 0x7ff400061fd0 2022-07-21T01:56:27.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fffff700 1 -- 172.21.15.140:0/936376810 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff400142720 con 0x7ff400061fd0 2022-07-21T01:56:27.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.017+0000 7ff3fffff700 1 -- 172.21.15.140:0/936376810 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4000b2840 con 0x7ff400061fd0 2022-07-21T01:56:27.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.021+0000 7ff3fcff9700 1 -- 172.21.15.140:0/936376810 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff3e8019030 con 0x7ff400061fd0 2022-07-21T01:56:27.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.021+0000 7ff3fcff9700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3e4053aa0 0x7ff3e4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.021+0000 7ff3fcff9700 1 -- 172.21.15.140:0/936376810 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff3e806acd0 con 0x7ff400061fd0 2022-07-21T01:56:27.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.021+0000 7ff3feffd700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3e4053aa0 0x7ff3e4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.021+0000 7ff3feffd700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3e4053aa0 0x7ff3e4055f60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff3f000d000 tx=0x7ff3f00011a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:27.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.025+0000 7ff3fcff9700 1 -- 172.21.15.140:0/936376810 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff3e8041e80 con 0x7ff400061fd0 2022-07-21T01:56:27.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.153+0000 7ff3fffff700 1 -- 172.21.15.140:0/936376810 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff400142720 con 0x7ff400061fd0 2022-07-21T01:56:27.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.153+0000 7ff3fcff9700 1 -- 172.21.15.140:0/936376810 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v109) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff400142720 con 0x7ff400061fd0 2022-07-21T01:56:27.159 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:27.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 -- 172.21.15.140:0/936376810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3e4053aa0 msgr2=0x7ff3e4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3e4053aa0 0x7ff3e4055f60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff3f000d000 tx=0x7ff3f00011a0).stop 2022-07-21T01:56:27.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 -- 172.21.15.140:0/936376810 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff400061fd0 msgr2=0x7ff400141ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff400061fd0 0x7ff400141ba0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7ff4000b6e40 tx=0x7ff3e8007c50).stop 2022-07-21T01:56:27.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 -- 172.21.15.140:0/936376810 shutdown_connections 2022-07-21T01:56:27.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff3e4053aa0 0x7ff3e4055f60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4000b2230 0x7ff400142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff400061fd0 0x7ff400141ba0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 --2- 172.21.15.140:0/936376810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff40005c960 0x7ff400149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 -- 172.21.15.140:0/936376810 >> 172.21.15.140:0/936376810 conn(0x7ff4000b61f0 msgr2=0x7ff4000b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:27.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 -- 172.21.15.140:0/936376810 shutdown_connections 2022-07-21T01:56:27.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.157+0000 7ff3fffff700 1 -- 172.21.15.140:0/936376810 wait complete. 2022-07-21T01:56:27.178 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:56:27.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4526ba0700 1 Processor -- start 2022-07-21T01:56:27.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4526ba0700 1 -- start start 2022-07-21T01:56:27.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4526ba0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f45200b61d0 0x7f45200b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4526ba0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45200b13e0 0x7f452005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4526ba0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f45200aea10 0x7f45200aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4526ba0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452013efd0 con 0x7f45200aea10 2022-07-21T01:56:27.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4526ba0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45200b20f0 con 0x7f45200b61d0 2022-07-21T01:56:27.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4526ba0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45200b65b0 con 0x7f45200b13e0 2022-07-21T01:56:27.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f452639f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f45200aea10 0x7f45200aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4525b9e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f45200b61d0 0x7f45200b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f452639f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f45200aea10 0x7f45200aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39388/0 (socket says 172.21.15.140:39388) 2022-07-21T01:56:27.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f452539d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45200b13e0 0x7f452005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f452639f700 1 -- 172.21.15.140:0/1810009449 learned_addr learned my addr 172.21.15.140:0/1810009449 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:27.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f452639f700 1 -- 172.21.15.140:0/1810009449 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45200b13e0 msgr2=0x7f452005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:27.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f452639f700 1 --2- 172.21.15.140:0/1810009449 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45200b13e0 0x7f452005bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f452639f700 1 -- 172.21.15.140:0/1810009449 >> v2:172.21.15.152:3300/0 conn(0x7f45200b61d0 msgr2=0x7f45200b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:27.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f452639f700 1 --2- 172.21.15.140:0/1810009449 >> v2:172.21.15.152:3300/0 conn(0x7f45200b61d0 0x7f45200b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f452639f700 1 -- 172.21.15.140:0/1810009449 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45200b1ad0 con 0x7f45200aea10 2022-07-21T01:56:27.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4525b9e700 1 --2- 172.21.15.140:0/1810009449 >> v2:172.21.15.152:3300/0 conn(0x7f45200b61d0 0x7f45200b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:27.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f452539d700 1 --2- 172.21.15.140:0/1810009449 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45200b13e0 0x7f452005bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:27.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f452639f700 1 --2- 172.21.15.140:0/1810009449 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45200aea10 0x7f45200aee40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f4510002f70 tx=0x7f4510006eb0).ready entity=mon.0 client_cookie=6fa8092ac3456235 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:27.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4524b9c700 1 -- 172.21.15.140:0/1810009449 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f451000d570 con 0x7f45200aea10 2022-07-21T01:56:27.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4524b9c700 1 -- 172.21.15.140:0/1810009449 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4510000e90 con 0x7f45200aea10 2022-07-21T01:56:27.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.281+0000 7f4524b9c700 1 -- 172.21.15.140:0/1810009449 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f451000d900 con 0x7f45200aea10 2022-07-21T01:56:27.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.285+0000 7f4526ba0700 1 -- 172.21.15.140:0/1810009449 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45200aea10 msgr2=0x7f45200aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.285+0000 7f4526ba0700 1 --2- 172.21.15.140:0/1810009449 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45200aea10 0x7f45200aee40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f4510002f70 tx=0x7f4510006eb0).stop 2022-07-21T01:56:27.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.285+0000 7f4526ba0700 1 -- 172.21.15.140:0/1810009449 shutdown_connections 2022-07-21T01:56:27.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.285+0000 7f4526ba0700 1 --2- 172.21.15.140:0/1810009449 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45200aea10 0x7f45200aee40 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.285+0000 7f4526ba0700 1 --2- 172.21.15.140:0/1810009449 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45200b13e0 0x7f452005bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.285+0000 7f4526ba0700 1 --2- 172.21.15.140:0/1810009449 >> v2:172.21.15.152:3300/0 conn(0x7f45200b61d0 0x7f45200b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.285+0000 7f4526ba0700 1 -- 172.21.15.140:0/1810009449 >> 172.21.15.140:0/1810009449 conn(0x7f4520063cf0 msgr2=0x7f45200b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:27.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.285+0000 7f4526ba0700 1 -- 172.21.15.140:0/1810009449 shutdown_connections 2022-07-21T01:56:27.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.285+0000 7f4526ba0700 1 -- 172.21.15.140:0/1810009449 wait complete. 2022-07-21T01:56:27.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.285+0000 7f4526ba0700 1 Processor -- start 2022-07-21T01:56:27.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.285+0000 7f4526ba0700 1 -- start start 2022-07-21T01:56:27.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f4526ba0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45200aea10 0x7f4520066670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f4526ba0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45200b13e0 0x7f452013dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f4526ba0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45200b61d0 0x7f4520151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f4526ba0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4520155220 con 0x7f45200b61d0 2022-07-21T01:56:27.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f4526ba0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4520066ca0 con 0x7f45200aea10 2022-07-21T01:56:27.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f4526ba0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4520066f40 con 0x7f45200b13e0 2022-07-21T01:56:27.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f452639f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45200b61d0 0x7f4520151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f452639f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45200b61d0 0x7f4520151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39394/0 (socket says 172.21.15.140:39394) 2022-07-21T01:56:27.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f452639f700 1 -- 172.21.15.140:0/664055756 learned_addr learned my addr 172.21.15.140:0/664055756 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:27.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f4525b9e700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45200aea10 0x7f4520066670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f452539d700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45200b13e0 0x7f452013dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f452639f700 1 -- 172.21.15.140:0/664055756 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45200b13e0 msgr2=0x7f452013dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f452639f700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45200b13e0 0x7f452013dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f452639f700 1 -- 172.21.15.140:0/664055756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45200aea10 msgr2=0x7f4520066670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f452639f700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45200aea10 0x7f4520066670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f452639f700 1 -- 172.21.15.140:0/664055756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451800b180 con 0x7f45200b61d0 2022-07-21T01:56:27.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f452639f700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45200b61d0 0x7f4520151c60 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f4510002f70 tx=0x7f4510015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:27.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f451effd700 1 -- 172.21.15.140:0/664055756 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45100116e0 con 0x7f45200b61d0 2022-07-21T01:56:27.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f451effd700 1 -- 172.21.15.140:0/664055756 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4510011840 con 0x7f45200b61d0 2022-07-21T01:56:27.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f4526ba0700 1 -- 172.21.15.140:0/664055756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4508009470 con 0x7f45200b61d0 2022-07-21T01:56:27.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f451effd700 1 -- 172.21.15.140:0/664055756 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4510011ba0 con 0x7f45200b61d0 2022-07-21T01:56:27.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f4526ba0700 1 -- 172.21.15.140:0/664055756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451000b1e0 con 0x7f45200b61d0 2022-07-21T01:56:27.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.289+0000 7f4526ba0700 1 -- 172.21.15.140:0/664055756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45201553e0 con 0x7f45200b61d0 2022-07-21T01:56:27.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.293+0000 7f451effd700 1 -- 172.21.15.140:0/664055756 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4510025030 con 0x7f45200b61d0 2022-07-21T01:56:27.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.293+0000 7f451effd700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45040539e0 0x7f4504055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.293+0000 7f451effd700 1 -- 172.21.15.140:0/664055756 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f451006cad0 con 0x7f45200b61d0 2022-07-21T01:56:27.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.293+0000 7f4525b9e700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45040539e0 0x7f4504055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.293+0000 7f4525b9e700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45040539e0 0x7f4504055ea0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f450800b000 tx=0x7f45080091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:27.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.297+0000 7f451effd700 1 -- 172.21.15.140:0/664055756 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f45201553e0 con 0x7f45200b61d0 2022-07-21T01:56:27.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.421+0000 7f4526ba0700 1 -- 172.21.15.140:0/664055756 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f45200b1d70 con 0x7f45200b61d0 2022-07-21T01:56:27.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.421+0000 7f451effd700 1 -- 172.21.15.140:0/664055756 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v109) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4510023020 con 0x7f45200b61d0 2022-07-21T01:56:27.428 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:27.428 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:56:27.428 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:27.429 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:56:27.429 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:56:27.429 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:27.429 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:27.429 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:56:27.430 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:56:27.430 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:27.430 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:56:27.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 -- 172.21.15.140:0/664055756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45040539e0 msgr2=0x7f4504055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45040539e0 0x7f4504055ea0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f450800b000 tx=0x7f45080091d0).stop 2022-07-21T01:56:27.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 -- 172.21.15.140:0/664055756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45200b61d0 msgr2=0x7f4520151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45200b61d0 0x7f4520151c60 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f4510002f70 tx=0x7f4510015000).stop 2022-07-21T01:56:27.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 -- 172.21.15.140:0/664055756 shutdown_connections 2022-07-21T01:56:27.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f45040539e0 0x7f4504055ea0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45200b61d0 0x7f4520151c60 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45200b13e0 0x7f452013dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 --2- 172.21.15.140:0/664055756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45200aea10 0x7f4520066670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 -- 172.21.15.140:0/664055756 >> 172.21.15.140:0/664055756 conn(0x7f4520063cf0 msgr2=0x7f45200b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:27.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 -- 172.21.15.140:0/664055756 shutdown_connections 2022-07-21T01:56:27.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.425+0000 7f4526ba0700 1 -- 172.21.15.140:0/664055756 wait complete. 2022-07-21T01:56:27.441 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:56:27.441 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:56:27.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.541+0000 7fb8d5e7c700 1 Processor -- start 2022-07-21T01:56:27.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d5e7c700 1 -- start start 2022-07-21T01:56:27.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d5e7c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8d00af940 0x7fb8d005c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d5e7c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d005d2f0 con 0x7fb8d00b59f0 2022-07-21T01:56:27.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d5e7c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d00b5dd0 con 0x7fb8d00addd0 2022-07-21T01:56:27.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d5e7c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d005ccd0 con 0x7fb8d00af940 2022-07-21T01:56:27.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d4e7a700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fb8d00b59f0 0x7fb8d005c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53544/0 (socket says 172.21.15.140:53544) 2022-07-21T01:56:27.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d4e7a700 1 -- 172.21.15.140:0/3884024643 learned_addr learned my addr 172.21.15.140:0/3884024643 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:27.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cffff700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8d00af940 0x7fb8d005c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424832793 0 0) 0x7fb8d005d2f0 con 0x7fb8d00b59f0 2022-07-21T01:56:27.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8b0003260 con 0x7fb8d00b59f0 2022-07-21T01:56:27.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871559054 0 0) 0x7fb8d00b5dd0 con 0x7fb8d00addd0 2022-07-21T01:56:27.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8d005d2f0 con 0x7fb8d00addd0 2022-07-21T01:56:27.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2002777459 0 0) 0x7fb8b0003260 con 0x7fb8d00b59f0 2022-07-21T01:56:27.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8d00af940 msgr2=0x7fb8d005c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:27.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8d00af940 0x7fb8d005c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 >> v1:172.21.15.152:6789/0 conn(0x7fb8d00addd0 legacy=0x7fb8d0142d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d01460b0 con 0x7fb8d00b59f0 2022-07-21T01:56:27.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fb8c0001660 con 0x7fb8d00b59f0 2022-07-21T01:56:27.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 >> v1:172.21.15.140:6789/0 conn(0x7fb8d00b59f0 legacy=0x7fb8d005c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b0003910 0x7fb8b0003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8d00af940 0x7fb8b0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8b00049f0 0x7fb8b0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b000b800 con 0x7fb8b00049f0 2022-07-21T01:56:27.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b000b940 con 0x7fb8b0003910 2022-07-21T01:56:27.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b000bab0 con 0x7fb8d00af940 2022-07-21T01:56:27.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d567b700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8b00049f0 0x7fb8b0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cffff700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8d00af940 0x7fb8b0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d4e7a700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b0003910 0x7fb8b0003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d567b700 1 -- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8d00af940 msgr2=0x7fb8b0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:27.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d567b700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8d00af940 0x7fb8b0004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d567b700 1 -- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b0003910 msgr2=0x7fb8b0003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:27.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d567b700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b0003910 0x7fb8b0003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d567b700 1 -- 172.21.15.140:0/3884024643 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b0003260 con 0x7fb8b00049f0 2022-07-21T01:56:27.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d4e7a700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b0003910 0x7fb8b0003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:27.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8d567b700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8b00049f0 0x7fb8b0004e70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7fb8c8001490 tx=0x7fb8c8007d80).ready entity=mon.0 client_cookie=5684a5ece3419cda server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:27.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c8001780 con 0x7fb8b00049f0 2022-07-21T01:56:27.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c8006890 con 0x7fb8b00049f0 2022-07-21T01:56:27.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.545+0000 7fb8cf7fe700 1 -- 172.21.15.140:0/3884024643 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c8006bf0 con 0x7fb8b00049f0 2022-07-21T01:56:27.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3884024643 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8b00049f0 msgr2=0x7fb8b0004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 --2- 172.21.15.140:0/3884024643 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8b00049f0 0x7fb8b0004e70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7fb8c8001490 tx=0x7fb8c8007d80).stop 2022-07-21T01:56:27.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d567b700 1 -- 172.21.15.140:0/3884024643 reap_dead start 2022-07-21T01:56:27.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3884024643 shutdown_connections 2022-07-21T01:56:27.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3884024643 >> 172.21.15.140:0/3884024643 conn(0x7fb8d0063480 msgr2=0x7fb8d00b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:27.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3884024643 shutdown_connections 2022-07-21T01:56:27.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3884024643 wait complete. 2022-07-21T01:56:27.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 Processor -- start 2022-07-21T01:56:27.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 -- start start 2022-07-21T01:56:27.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8b00049f0 0x7fb8d0065d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d00657b0 0x7fb8d0066620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8d0112a10 0x7fb8d0066b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0067000 con 0x7fb8d0112a10 2022-07-21T01:56:27.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0067220 con 0x7fb8d00657b0 2022-07-21T01:56:27.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d5e7c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0114860 con 0x7fb8b00049f0 2022-07-21T01:56:27.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d567b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8d0112a10 0x7fb8d0066b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d567b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8d0112a10 0x7fb8d0066b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39414/0 (socket says 172.21.15.140:39414) 2022-07-21T01:56:27.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d567b700 1 -- 172.21.15.140:0/3217417245 learned_addr learned my addr 172.21.15.140:0/3217417245 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:27.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8cffff700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d00657b0 0x7fb8d0066620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d4e7a700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8b00049f0 0x7fb8d0065d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d567b700 1 -- 172.21.15.140:0/3217417245 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8b00049f0 msgr2=0x7fb8d0065d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d567b700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8b00049f0 0x7fb8d0065d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d567b700 1 -- 172.21.15.140:0/3217417245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d00657b0 msgr2=0x7fb8d0066620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:27.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d567b700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d00657b0 0x7fb8d0066620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:27.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d567b700 1 -- 172.21.15.140:0/3217417245 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0114b00 con 0x7fb8d0112a10 2022-07-21T01:56:27.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8d567b700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8d0112a10 0x7fb8d0066b10 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fb8d00add00 tx=0x7fb8c8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:27.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.549+0000 7fb8cdffb700 1 -- 172.21.15.140:0/3217417245 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c8001780 con 0x7fb8d0112a10 2022-07-21T01:56:27.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.553+0000 7fb8cdffb700 1 -- 172.21.15.140:0/3217417245 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c800c9e0 con 0x7fb8d0112a10 2022-07-21T01:56:27.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.553+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3217417245 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d0155320 con 0x7fb8d0112a10 2022-07-21T01:56:27.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.553+0000 7fb8cdffb700 1 -- 172.21.15.140:0/3217417245 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8c800cd40 con 0x7fb8d0112a10 2022-07-21T01:56:27.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.553+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3217417245 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d0155850 con 0x7fb8d0112a10 2022-07-21T01:56:27.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.553+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3217417245 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d005cfe0 con 0x7fb8d0112a10 2022-07-21T01:56:27.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.553+0000 7fb8cdffb700 1 -- 172.21.15.140:0/3217417245 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb8c8025030 con 0x7fb8d0112a10 2022-07-21T01:56:27.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.553+0000 7fb8cdffb700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb8b4053a70 0x7fb8b4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:27.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.553+0000 7fb8cdffb700 1 -- 172.21.15.140:0/3217417245 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb8c806cc30 con 0x7fb8d0112a10 2022-07-21T01:56:27.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.553+0000 7fb8d4e7a700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb8b4053a70 0x7fb8b4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:27.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.553+0000 7fb8d4e7a700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb8b4053a70 0x7fb8b4055f30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fb8c000d000 tx=0x7fb8c0007040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:27.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.557+0000 7fb8cdffb700 1 -- 172.21.15.140:0/3217417245 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb8c8043d70 con 0x7fb8d0112a10 2022-07-21T01:56:27.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:27.681+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3217417245 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb8d00574e0 con 0x7fb8d0112a10 2022-07-21T01:56:28.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.604+0000 7fb8cdffb700 1 -- 172.21.15.140:0/3217417245 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v110) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb8c8023070 con 0x7fb8d0112a10 2022-07-21T01:56:28.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.608+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3217417245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb8b4053a70 msgr2=0x7fb8b4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:28.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.608+0000 7fb8d5e7c700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb8b4053a70 0x7fb8b4055f30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fb8c000d000 tx=0x7fb8c0007040).stop 2022-07-21T01:56:28.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.608+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3217417245 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8d0112a10 msgr2=0x7fb8d0066b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:28.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.608+0000 7fb8d5e7c700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8d0112a10 0x7fb8d0066b10 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fb8d00add00 tx=0x7fb8c8016000).stop 2022-07-21T01:56:28.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.612+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3217417245 shutdown_connections 2022-07-21T01:56:28.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.612+0000 7fb8d5e7c700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb8b4053a70 0x7fb8b4055f30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:28.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.612+0000 7fb8d5e7c700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb8d0112a10 0x7fb8d0066b10 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:28.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.612+0000 7fb8d5e7c700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d00657b0 0x7fb8d0066620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:28.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.612+0000 7fb8d5e7c700 1 --2- 172.21.15.140:0/3217417245 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb8b00049f0 0x7fb8d0065d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:28.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.612+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3217417245 >> 172.21.15.140:0/3217417245 conn(0x7fb8d0063480 msgr2=0x7fb8d013ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:28.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.612+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3217417245 shutdown_connections 2022-07-21T01:56:28.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.612+0000 7fb8d5e7c700 1 -- 172.21.15.140:0/3217417245 wait complete. 2022-07-21T01:56:28.620 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:56:28.631 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:56:28.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.732+0000 7fa5f1db5700 1 Processor -- start 2022-07-21T01:56:28.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.732+0000 7fa5f1db5700 1 -- start start 2022-07-21T01:56:28.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f1db5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa5ec0b5690 0x7fa5ec140730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:28.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f1db5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5ec140c20 0x7fa5ec1296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:28.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f1db5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec05cc80 con 0x7fa5ec0af030 2022-07-21T01:56:28.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f1db5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec0af410 con 0x7fa5ec0b5690 2022-07-21T01:56:28.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f1db5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec144050 con 0x7fa5ec140c20 2022-07-21T01:56:28.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f0db3700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa5ec0af030 0x7fa5ec05c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53564/0 (socket says 172.21.15.140:53564) 2022-07-21T01:56:28.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f0db3700 1 -- 172.21.15.140:0/1189185377 learned_addr learned my addr 172.21.15.140:0/1189185377 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:28.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f15b4700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5ec140c20 0x7fa5ec1296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:28.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5ebfff700 1 --2- 172.21.15.140:0/1189185377 >> v2:172.21.15.152:3300/0 conn(0x7fa5ec0b5690 0x7fa5ec140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:28.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455610976 0 0) 0x7fa5ec05cc80 con 0x7fa5ec0af030 2022-07-21T01:56:28.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5d8003260 con 0x7fa5ec0af030 2022-07-21T01:56:28.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2218542113 0 0) 0x7fa5d8003260 con 0x7fa5ec0af030 2022-07-21T01:56:28.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5ec140c20 msgr2=0x7fa5ec1296f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:28.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5ec140c20 0x7fa5ec1296f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:28.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 >> v2:172.21.15.152:3300/0 conn(0x7fa5ec0b5690 msgr2=0x7fa5ec140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:28.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 --2- 172.21.15.140:0/1189185377 >> v2:172.21.15.152:3300/0 conn(0x7fa5ec0b5690 0x7fa5ec140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:28.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec05be30 con 0x7fa5ec0af030 2022-07-21T01:56:28.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fa5e40019e0 con 0x7fa5ec0af030 2022-07-21T01:56:28.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 >> v1:172.21.15.140:6789/0 conn(0x7fa5ec0af030 legacy=0x7fa5ec05c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:28.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d8003750 0x7fa5d8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:28.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5d8004500 0x7fa5d8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:28.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5ec140c20 0x7fa5d8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:28.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0af410 con 0x7fa5d8004500 2022-07-21T01:56:28.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec144050 con 0x7fa5d8003750 2022-07-21T01:56:28.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d800b750 con 0x7fa5ec140c20 2022-07-21T01:56:28.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f0db3700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d8003750 0x7fa5d8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:28.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5ebfff700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5d8004500 0x7fa5d8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:28.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f15b4700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5ec140c20 0x7fa5d8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:28.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f0db3700 1 -- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5ec140c20 msgr2=0x7fa5d8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:28.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f0db3700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5ec140c20 0x7fa5d8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:28.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f0db3700 1 -- 172.21.15.140:0/1189185377 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5d8004500 msgr2=0x7fa5d8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:28.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f0db3700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5d8004500 0x7fa5d8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:28.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f0db3700 1 -- 172.21.15.140:0/1189185377 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d8003260 con 0x7fa5d8003750 2022-07-21T01:56:28.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5f0db3700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d8003750 0x7fa5d8003b70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fa5e40016f0 tx=0x7fa5ec064280).ready entity=mon.1 client_cookie=1749cc257ecc8ce1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:28.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5e400cda0 con 0x7fa5d8003750 2022-07-21T01:56:28.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e4005990 con 0x7fa5d8003750 2022-07-21T01:56:28.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.736+0000 7fa5eb7fe700 1 -- 172.21.15.140:0/1189185377 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5e4005cf0 con 0x7fa5d8003750 2022-07-21T01:56:28.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 -- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d8003750 msgr2=0x7fa5d8003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:28.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 --2- 172.21.15.140:0/1189185377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d8003750 0x7fa5d8003b70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fa5e40016f0 tx=0x7fa5ec064280).stop 2022-07-21T01:56:28.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f15b4700 1 -- 172.21.15.140:0/1189185377 reap_dead start 2022-07-21T01:56:28.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 -- 172.21.15.140:0/1189185377 shutdown_connections 2022-07-21T01:56:28.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 -- 172.21.15.140:0/1189185377 >> 172.21.15.140:0/1189185377 conn(0x7fa5ec0634c0 msgr2=0x7fa5ec0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:28.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 -- 172.21.15.140:0/1189185377 shutdown_connections 2022-07-21T01:56:28.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 -- 172.21.15.140:0/1189185377 wait complete. 2022-07-21T01:56:28.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 Processor -- start 2022-07-21T01:56:28.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 -- start start 2022-07-21T01:56:28.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5d8003750 0x7fa5ec129800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:28.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5ec12b510 0x7fa5ec14ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:28.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5ec12a9f0 0x7fa5ec150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:28.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec14fb20 con 0x7fa5ec12b510 2022-07-21T01:56:28.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec154040 con 0x7fa5ec12a9f0 2022-07-21T01:56:28.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f1db5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec153da0 con 0x7fa5d8003750 2022-07-21T01:56:28.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5ebfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5ec12b510 0x7fa5ec14ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:28.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f0db3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5d8003750 0x7fa5ec129800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:28.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f0db3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5d8003750 0x7fa5ec129800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36160/0 (socket says 172.21.15.140:36160) 2022-07-21T01:56:28.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f0db3700 1 -- 172.21.15.140:0/2394096100 learned_addr learned my addr 172.21.15.140:0/2394096100 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:28.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5ebfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5ec12b510 0x7fa5ec14ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39432/0 (socket says 172.21.15.140:39432) 2022-07-21T01:56:28.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.740+0000 7fa5f15b4700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5ec12a9f0 0x7fa5ec150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:28.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5f0db3700 1 -- 172.21.15.140:0/2394096100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5ec12a9f0 msgr2=0x7fa5ec150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:28.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5f0db3700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5ec12a9f0 0x7fa5ec150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:28.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5f0db3700 1 -- 172.21.15.140:0/2394096100 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5ec12b510 msgr2=0x7fa5ec14ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:28.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5f0db3700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5ec12b510 0x7fa5ec14ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:28.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5f0db3700 1 -- 172.21.15.140:0/2394096100 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec12b050 con 0x7fa5d8003750 2022-07-21T01:56:28.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5f0db3700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5d8003750 0x7fa5ec129800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fa5ec14fdf0 tx=0x7fa5e4013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:28.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5e9ffb700 1 -- 172.21.15.140:0/2394096100 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5e4001df0 con 0x7fa5d8003750 2022-07-21T01:56:28.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5e9ffb700 1 -- 172.21.15.140:0/2394096100 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e40072e0 con 0x7fa5d8003750 2022-07-21T01:56:28.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5f1db5700 1 -- 172.21.15.140:0/2394096100 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec153a30 con 0x7fa5d8003750 2022-07-21T01:56:28.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5e9ffb700 1 -- 172.21.15.140:0/2394096100 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa5e4007640 con 0x7fa5d8003750 2022-07-21T01:56:28.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5f1db5700 1 -- 172.21.15.140:0/2394096100 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec12cb20 con 0x7fa5d8003750 2022-07-21T01:56:28.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5f1db5700 1 -- 172.21.15.140:0/2394096100 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ec05c0d0 con 0x7fa5d8003750 2022-07-21T01:56:28.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5e9ffb700 1 -- 172.21.15.140:0/2394096100 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa5e4027070 con 0x7fa5d8003750 2022-07-21T01:56:28.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5e9ffb700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa5cc053b00 0x7fa5cc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:28.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5e9ffb700 1 -- 172.21.15.140:0/2394096100 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fa5e406c710 con 0x7fa5d8003750 2022-07-21T01:56:28.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.744+0000 7fa5ebfff700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa5cc053b00 0x7fa5cc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:28.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.748+0000 7fa5ebfff700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa5cc053b00 0x7fa5cc055fc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fa5e000b000 tx=0x7fa5e0006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:28.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.752+0000 7fa5e9ffb700 1 -- 172.21.15.140:0/2394096100 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa5e4043670 con 0x7fa5d8003750 2022-07-21T01:56:28.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:28.880+0000 7fa5f1db5700 1 -- 172.21.15.140:0/2394096100 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fa5ec0574e0 con 0x7fa5d8003750 2022-07-21T01:56:29.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.620+0000 7fa5e9ffb700 1 -- 172.21.15.140:0/2394096100 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v111) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa5e4046c90 con 0x7fa5d8003750 2022-07-21T01:56:29.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 -- 172.21.15.140:0/2394096100 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa5cc053b00 msgr2=0x7fa5cc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:29.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa5cc053b00 0x7fa5cc055fc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fa5e000b000 tx=0x7fa5e0006df0).stop 2022-07-21T01:56:29.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 -- 172.21.15.140:0/2394096100 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5d8003750 msgr2=0x7fa5ec129800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:29.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5d8003750 0x7fa5ec129800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fa5ec14fdf0 tx=0x7fa5e4013000).stop 2022-07-21T01:56:29.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 -- 172.21.15.140:0/2394096100 shutdown_connections 2022-07-21T01:56:29.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa5cc053b00 0x7fa5cc055fc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5ec12a9f0 0x7fa5ec150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa5ec12b510 0x7fa5ec14ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 --2- 172.21.15.140:0/2394096100 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa5d8003750 0x7fa5ec129800 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 -- 172.21.15.140:0/2394096100 >> 172.21.15.140:0/2394096100 conn(0x7fa5ec0634c0 msgr2=0x7fa5ec126630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:29.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 -- 172.21.15.140:0/2394096100 shutdown_connections 2022-07-21T01:56:29.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.624+0000 7fa5f1db5700 1 -- 172.21.15.140:0/2394096100 wait complete. 2022-07-21T01:56:29.634 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:56:29.640 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:56:29.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.744+0000 7ff23348d700 1 Processor -- start 2022-07-21T01:56:29.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.744+0000 7ff23348d700 1 -- start start 2022-07-21T01:56:29.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff22c067370 0x7ff22c067750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:29.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff22c0b2130 0x7ff22c143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:29.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22c13ad00 con 0x7ff22c0acbc0 2022-07-21T01:56:29.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c0b3c60 con 0x7ff22c0b2130 2022-07-21T01:56:29.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0b3f00 con 0x7ff22c067370 2022-07-21T01:56:29.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff232c8c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff22c0b2130 0x7ff22c143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:29.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff232c8c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff22c0b2130 0x7ff22c143dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39442/0 (socket says 172.21.15.140:39442) 2022-07-21T01:56:29.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff232c8c700 1 -- 172.21.15.140:0/367743354 learned_addr learned my addr 172.21.15.140:0/367743354 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:29.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23248b700 1 --2- 172.21.15.140:0/367743354 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff22c067370 0x7ff22c067750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:29.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff232c8c700 1 -- 172.21.15.140:0/367743354 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff22c067370 msgr2=0x7ff22c067750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:29.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff232c8c700 1 --2- 172.21.15.140:0/367743354 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff22c067370 0x7ff22c067750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff232c8c700 1 -- 172.21.15.140:0/367743354 >> v1:172.21.15.152:6789/0 conn(0x7ff22c0acbc0 legacy=0x7ff22c13a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:29.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff232c8c700 1 -- 172.21.15.140:0/367743354 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c0b41a0 con 0x7ff22c0b2130 2022-07-21T01:56:29.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff232c8c700 1 --2- 172.21.15.140:0/367743354 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff22c0b2130 0x7ff22c143dc0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7ff2240032f0 tx=0x7ff224009290).ready entity=mon.0 client_cookie=458369f65736d11 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:29.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff231489700 1 -- 172.21.15.140:0/367743354 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff22400b8a0 con 0x7ff22c0b2130 2022-07-21T01:56:29.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff231489700 1 -- 172.21.15.140:0/367743354 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22400ba00 con 0x7ff22c0b2130 2022-07-21T01:56:29.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff231489700 1 -- 172.21.15.140:0/367743354 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff22400bb70 con 0x7ff22c0b2130 2022-07-21T01:56:29.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 -- 172.21.15.140:0/367743354 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff22c0b2130 msgr2=0x7ff22c143dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:29.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 --2- 172.21.15.140:0/367743354 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff22c0b2130 0x7ff22c143dc0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7ff2240032f0 tx=0x7ff224009290).stop 2022-07-21T01:56:29.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 -- 172.21.15.140:0/367743354 shutdown_connections 2022-07-21T01:56:29.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 --2- 172.21.15.140:0/367743354 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff22c0b2130 0x7ff22c143dc0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 --2- 172.21.15.140:0/367743354 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff22c067370 0x7ff22c067750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 -- 172.21.15.140:0/367743354 >> 172.21.15.140:0/367743354 conn(0x7ff22c064c00 msgr2=0x7ff22c064fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:29.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 -- 172.21.15.140:0/367743354 shutdown_connections 2022-07-21T01:56:29.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.748+0000 7ff23348d700 1 -- 172.21.15.140:0/367743354 wait complete. 2022-07-21T01:56:29.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23348d700 1 Processor -- start 2022-07-21T01:56:29.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23348d700 1 -- start start 2022-07-21T01:56:29.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23348d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff22c067370 0x7ff22c13f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:29.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23348d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff22c0acbc0 0x7ff22c1523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:29.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23348d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff22c0b2130 0x7ff22c1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:29.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23348d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0b0250 con 0x7ff22c0acbc0 2022-07-21T01:56:29.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23348d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c152dc0 con 0x7ff22c067370 2022-07-21T01:56:29.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23348d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c060dd0 con 0x7ff22c0b2130 2022-07-21T01:56:29.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff231c8a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff22c0acbc0 0x7ff22c1523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:29.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23248b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff22c067370 0x7ff22c13f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:29.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff232c8c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff22c0b2130 0x7ff22c1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:29.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff231c8a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff22c0acbc0 0x7ff22c1523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39446/0 (socket says 172.21.15.140:39446) 2022-07-21T01:56:29.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23248b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff22c067370 0x7ff22c13f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:45184/0 (socket says 172.21.15.140:45184) 2022-07-21T01:56:29.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff231c8a700 1 -- 172.21.15.140:0/3997092648 learned_addr learned my addr 172.21.15.140:0/3997092648 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:29.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff232c8c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff22c0b2130 0x7ff22c1528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36178/0 (socket says 172.21.15.140:36178) 2022-07-21T01:56:29.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff231c8a700 1 -- 172.21.15.140:0/3997092648 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff22c0b2130 msgr2=0x7ff22c1528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:29.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff231c8a700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff22c0b2130 0x7ff22c1528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff231c8a700 1 -- 172.21.15.140:0/3997092648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff22c067370 msgr2=0x7ff22c13f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:29.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff231c8a700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff22c067370 0x7ff22c13f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff231c8a700 1 -- 172.21.15.140:0/3997092648 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224009530 con 0x7ff22c0acbc0 2022-07-21T01:56:29.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff231c8a700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff22c0acbc0 0x7ff22c1523e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7ff22c152f00 tx=0x7ff228007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:29.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff21b7fe700 1 -- 172.21.15.140:0/3997092648 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff22800cea0 con 0x7ff22c0acbc0 2022-07-21T01:56:29.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff21b7fe700 1 -- 172.21.15.140:0/3997092648 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff228007030 con 0x7ff22c0acbc0 2022-07-21T01:56:29.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23348d700 1 -- 172.21.15.140:0/3997092648 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c061070 con 0x7ff22c0acbc0 2022-07-21T01:56:29.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff21b7fe700 1 -- 172.21.15.140:0/3997092648 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff228007390 con 0x7ff22c0acbc0 2022-07-21T01:56:29.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.752+0000 7ff23348d700 1 -- 172.21.15.140:0/3997092648 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22c13ffc0 con 0x7ff22c0acbc0 2022-07-21T01:56:29.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.756+0000 7ff23348d700 1 -- 172.21.15.140:0/3997092648 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22c061200 con 0x7ff22c0acbc0 2022-07-21T01:56:29.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.756+0000 7ff21b7fe700 1 -- 172.21.15.140:0/3997092648 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff228006510 con 0x7ff22c0acbc0 2022-07-21T01:56:29.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.756+0000 7ff21b7fe700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff20c053a70 0x7ff20c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:29.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.756+0000 7ff21b7fe700 1 -- 172.21.15.140:0/3997092648 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5957+0+0 (secure 0 0 0) 0x7ff22806b800 con 0x7ff22c0acbc0 2022-07-21T01:56:29.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.756+0000 7ff23248b700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff20c053a70 0x7ff20c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:29.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.756+0000 7ff23248b700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff20c053a70 0x7ff20c055f30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7ff21c00c000 tx=0x7ff21c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:29.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.760+0000 7ff21b7fe700 1 -- 172.21.15.140:0/3997092648 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff228042690 con 0x7ff22c0acbc0 2022-07-21T01:56:29.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.928+0000 7ff23348d700 1 -- 172.21.15.140:0/3997092648 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7ff22c0574e0 con 0x7ff22c0acbc0 2022-07-21T01:56:29.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.936+0000 7ff21b7fe700 1 -- 172.21.15.140:0/3997092648 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 21 and data pool 22 v51) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff228045cb0 con 0x7ff22c0acbc0 2022-07-21T01:56:29.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 -- 172.21.15.140:0/3997092648 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff20c053a70 msgr2=0x7ff20c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:29.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff20c053a70 0x7ff20c055f30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7ff21c00c000 tx=0x7ff21c008040).stop 2022-07-21T01:56:29.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 -- 172.21.15.140:0/3997092648 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff22c0acbc0 msgr2=0x7ff22c1523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:29.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff22c0acbc0 0x7ff22c1523e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7ff22c152f00 tx=0x7ff228007be0).stop 2022-07-21T01:56:29.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 -- 172.21.15.140:0/3997092648 shutdown_connections 2022-07-21T01:56:29.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff20c053a70 0x7ff20c055f30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff22c0b2130 0x7ff22c1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff22c0acbc0 0x7ff22c1523e0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 --2- 172.21.15.140:0/3997092648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff22c067370 0x7ff22c13f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:29.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 -- 172.21.15.140:0/3997092648 >> 172.21.15.140:0/3997092648 conn(0x7ff22c064c00 msgr2=0x7ff22c138a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:29.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 -- 172.21.15.140:0/3997092648 shutdown_connections 2022-07-21T01:56:29.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:29.940+0000 7ff23348d700 1 -- 172.21.15.140:0/3997092648 wait complete. 2022-07-21T01:56:29.946 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 21 and data pool 22 2022-07-21T01:56:29.955 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:30.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fbfff700 1 Processor -- start 2022-07-21T01:56:30.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fbfff700 1 -- start start 2022-07-21T01:56:30.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11fc0b1ef0 0x7f11fc0605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fbfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f11fc13f980 0x7f11fc0abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fbfff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc140940 con 0x7f11fc0ac4e0 2022-07-21T01:56:30.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fbfff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc0b22d0 con 0x7f11fc13f980 2022-07-21T01:56:30.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fbfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc05fbc0 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fa7fc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f11fc13f980 0x7f11fc0abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11faffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11fc0b1ef0 0x7f11fc0605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fa7fc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f11fc13f980 0x7f11fc0abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:45194/0 (socket says 172.21.15.140:45194) 2022-07-21T01:56:30.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11faffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11fc0b1ef0 0x7f11fc0605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36182/0 (socket says 172.21.15.140:36182) 2022-07-21T01:56:30.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fb7fe700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f11fc0ac4e0 0x7f11fc0ab180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:54716/0 (socket says 172.21.15.140:54716) 2022-07-21T01:56:30.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fa7fc700 1 -- 172.21.15.140:0/3688284293 learned_addr learned my addr 172.21.15.140:0/3688284293 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:30.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fa7fc700 1 -- 172.21.15.140:0/3688284293 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11fc0b1ef0 msgr2=0x7f11fc0605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:30.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fa7fc700 1 --2- 172.21.15.140:0/3688284293 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11fc0b1ef0 0x7f11fc0605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fa7fc700 1 -- 172.21.15.140:0/3688284293 >> v1:172.21.15.152:6789/0 conn(0x7f11fc0ac4e0 legacy=0x7f11fc0ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fa7fc700 1 -- 172.21.15.140:0/3688284293 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc05fe60 con 0x7f11fc13f980 2022-07-21T01:56:30.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11f9ffb700 1 -- 172.21.15.140:0/3688284293 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431201457 0 0) 0x7f11fc140940 con 0x7f11fc0ac4e0 2022-07-21T01:56:30.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11fa7fc700 1 --2- 172.21.15.140:0/3688284293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11fc13f980 0x7f11fc0abff0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f11ec0032f0 tx=0x7f11ec00b530).ready entity=mon.1 client_cookie=88e93507698e82a6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:30.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11f9ffb700 1 -- 172.21.15.140:0/3688284293 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11ec00de20 con 0x7f11fc13f980 2022-07-21T01:56:30.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11f9ffb700 1 -- 172.21.15.140:0/3688284293 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec010470 con 0x7f11fc13f980 2022-07-21T01:56:30.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.044+0000 7f11f9ffb700 1 -- 172.21.15.140:0/3688284293 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11ec0107d0 con 0x7f11fc13f980 2022-07-21T01:56:30.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 -- 172.21.15.140:0/3688284293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11fc13f980 msgr2=0x7f11fc0abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 --2- 172.21.15.140:0/3688284293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11fc13f980 0x7f11fc0abff0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f11ec0032f0 tx=0x7f11ec00b530).stop 2022-07-21T01:56:30.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 -- 172.21.15.140:0/3688284293 shutdown_connections 2022-07-21T01:56:30.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 --2- 172.21.15.140:0/3688284293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11fc13f980 0x7f11fc0abff0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 --2- 172.21.15.140:0/3688284293 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11fc0b1ef0 0x7f11fc0605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 -- 172.21.15.140:0/3688284293 >> 172.21.15.140:0/3688284293 conn(0x7f11fc0b2ca0 msgr2=0x7f11fc0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:30.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 -- 172.21.15.140:0/3688284293 shutdown_connections 2022-07-21T01:56:30.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 -- 172.21.15.140:0/3688284293 wait complete. 2022-07-21T01:56:30.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 Processor -- start 2022-07-21T01:56:30.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 -- start start 2022-07-21T01:56:30.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11fc0ac4e0 0x7f11fc0aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11fc0b1ef0 0x7f11fc05c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11fc13f980 0x7f11fc13a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc141eb0 con 0x7f11fc0ac4e0 2022-07-21T01:56:30.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc05cfa0 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fbfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0afd90 con 0x7f11fc13f980 2022-07-21T01:56:30.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11faffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11fc0ac4e0 0x7f11fc0aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11faffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11fc0ac4e0 0x7f11fc0aeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39458/0 (socket says 172.21.15.140:39458) 2022-07-21T01:56:30.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fa7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11fc0b1ef0 0x7f11fc05c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11faffd700 1 -- 172.21.15.140:0/3155641716 learned_addr learned my addr 172.21.15.140:0/3155641716 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:30.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.048+0000 7f11fb7fe700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11fc13f980 0x7f11fc13a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11fa7fc700 1 -- 172.21.15.140:0/3155641716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11fc13f980 msgr2=0x7f11fc13a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11fa7fc700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11fc13f980 0x7f11fc13a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11fa7fc700 1 -- 172.21.15.140:0/3155641716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11fc0ac4e0 msgr2=0x7f11fc0aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11fa7fc700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11fc0ac4e0 0x7f11fc0aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11fa7fc700 1 -- 172.21.15.140:0/3155641716 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec00b7d0 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11faffd700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11fc0ac4e0 0x7f11fc0aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:30.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11fa7fc700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11fc0b1ef0 0x7f11fc05c940 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f11ec0032f0 tx=0x7f11fc0b2950).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:30.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11f3fff700 1 -- 172.21.15.140:0/3155641716 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11ec018610 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11f3fff700 1 -- 172.21.15.140:0/3155641716 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec018770 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11f3fff700 1 -- 172.21.15.140:0/3155641716 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11ec018ad0 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11fbfff700 1 -- 172.21.15.140:0/3155641716 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc0b0480 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11fbfff700 1 -- 172.21.15.140:0/3155641716 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc0affe0 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11fbfff700 1 -- 172.21.15.140:0/3155641716 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc0b01a0 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11f3fff700 1 -- 172.21.15.140:0/3155641716 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f11ec021020 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11f3fff700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11e0053950 0x7f11e0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11f3fff700 1 -- 172.21.15.140:0/3155641716 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f11ec06fe30 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.052+0000 7f11faffd700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11e0053950 0x7f11e0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.056+0000 7f11faffd700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11e0053950 0x7f11e0055e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f11e400b000 tx=0x7f11e4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:30.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.064+0000 7f11f3fff700 1 -- 172.21.15.140:0/3155641716 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f11ec046cb0 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.192+0000 7f11fbfff700 1 -- 172.21.15.140:0/3155641716 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11fc060100 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11f3fff700 1 -- 172.21.15.140:0/3155641716 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f11ec04a2d0 con 0x7f11fc0b1ef0 2022-07-21T01:56:30.198 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:30.198 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":112,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:29.930257+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:56:27.736358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-07-21T01:56:28.943049+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:30.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 -- 172.21.15.140:0/3155641716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11e0053950 msgr2=0x7f11e0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11e0053950 0x7f11e0055e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f11e400b000 tx=0x7f11e4006df0).stop 2022-07-21T01:56:30.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 -- 172.21.15.140:0/3155641716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11fc0b1ef0 msgr2=0x7f11fc05c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11fc0b1ef0 0x7f11fc05c940 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f11ec0032f0 tx=0x7f11fc0b2950).stop 2022-07-21T01:56:30.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 -- 172.21.15.140:0/3155641716 shutdown_connections 2022-07-21T01:56:30.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f11e0053950 0x7f11e0055e10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11fc13f980 0x7f11fc13a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11fc0b1ef0 0x7f11fc05c940 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 --2- 172.21.15.140:0/3155641716 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11fc0ac4e0 0x7f11fc0aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 -- 172.21.15.140:0/3155641716 >> 172.21.15.140:0/3155641716 conn(0x7f11fc0b2ca0 msgr2=0x7f11fc13ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:30.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 -- 172.21.15.140:0/3155641716 shutdown_connections 2022-07-21T01:56:30.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.196+0000 7f11fbfff700 1 -- 172.21.15.140:0/3155641716 wait complete. 2022-07-21T01:56:30.211 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:30.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b2a15700 1 Processor -- start 2022-07-21T01:56:30.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b2a15700 1 -- start start 2022-07-21T01:56:30.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b2a15700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f17ac0af8e0 0x7f17ac0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b2a15700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ac147010 con 0x7f17ac0b5650 2022-07-21T01:56:30.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b2a15700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ac1472b0 con 0x7f17ac0adff0 2022-07-21T01:56:30.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b2a15700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17ac146d40 con 0x7f17ac0af8e0 2022-07-21T01:56:30.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b1a13700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f17ac0b5650 0x7f17ac147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53614/0 (socket says 172.21.15.140:53614) 2022-07-21T01:56:30.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b1a13700 1 -- 172.21.15.140:0/1781150769 learned_addr learned my addr 172.21.15.140:0/1781150769 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:30.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b2214700 1 --2- 172.21.15.140:0/1781150769 >> v2:172.21.15.152:3300/0 conn(0x7f17ac0af8e0 0x7f17ac0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b0a11700 1 -- 172.21.15.140:0/1781150769 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794112664 0 0) 0x7f17ac147010 con 0x7f17ac0b5650 2022-07-21T01:56:30.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b0a11700 1 -- 172.21.15.140:0/1781150769 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1798003260 con 0x7f17ac0b5650 2022-07-21T01:56:30.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b2214700 1 -- 172.21.15.140:0/1781150769 >> v1:172.21.15.152:6789/0 conn(0x7f17ac0adff0 legacy=0x7f17ac05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b2214700 1 -- 172.21.15.140:0/1781150769 >> v1:172.21.15.140:6789/0 conn(0x7f17ac0b5650 legacy=0x7f17ac147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b0a11700 1 -- 172.21.15.140:0/1781150769 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1238561598 0 0) 0x7f1798003260 con 0x7f17ac0b5650 2022-07-21T01:56:30.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.312+0000 7f17b2214700 1 -- 172.21.15.140:0/1781150769 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac05c4a0 con 0x7f17ac0af8e0 2022-07-21T01:56:30.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b2214700 1 --2- 172.21.15.140:0/1781150769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17ac0af8e0 0x7f17ac0b5a30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f179c001800 tx=0x7f17a4006df0).ready entity=mon.1 client_cookie=746b4e2590038700 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:30.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b0a11700 1 -- 172.21.15.140:0/1781150769 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f179c001660 con 0x7f17ac0af8e0 2022-07-21T01:56:30.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b0a11700 1 -- 172.21.15.140:0/1781150769 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a4000e90 con 0x7f17ac0af8e0 2022-07-21T01:56:30.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b0a11700 1 -- 172.21.15.140:0/1781150769 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17a400d4f0 con 0x7f17ac0af8e0 2022-07-21T01:56:30.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b2a15700 1 -- 172.21.15.140:0/1781150769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17ac0af8e0 msgr2=0x7f17ac0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b2a15700 1 --2- 172.21.15.140:0/1781150769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17ac0af8e0 0x7f17ac0b5a30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f179c001800 tx=0x7f17a4006df0).stop 2022-07-21T01:56:30.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b2a15700 1 -- 172.21.15.140:0/1781150769 shutdown_connections 2022-07-21T01:56:30.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b2a15700 1 --2- 172.21.15.140:0/1781150769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17ac0af8e0 0x7f17ac0b5a30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b2a15700 1 -- 172.21.15.140:0/1781150769 >> 172.21.15.140:0/1781150769 conn(0x7f17ac0634e0 msgr2=0x7f17ac0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:30.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b2a15700 1 -- 172.21.15.140:0/1781150769 shutdown_connections 2022-07-21T01:56:30.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b2a15700 1 -- 172.21.15.140:0/1781150769 wait complete. 2022-07-21T01:56:30.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b2a15700 1 Processor -- start 2022-07-21T01:56:30.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.316+0000 7f17b2a15700 1 -- start start 2022-07-21T01:56:30.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2a15700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17ac0adff0 0x7f17ac149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2a15700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17ac0af8e0 0x7f17ac142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2a15700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17ac0b5650 0x7f17ac142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2a15700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac146260 con 0x7f17ac0b5650 2022-07-21T01:56:30.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2a15700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac111670 con 0x7f17ac0adff0 2022-07-21T01:56:30.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2a15700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ac111090 con 0x7f17ac0af8e0 2022-07-21T01:56:30.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2214700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17ac0b5650 0x7f17ac142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2214700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17ac0b5650 0x7f17ac142810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39476/0 (socket says 172.21.15.140:39476) 2022-07-21T01:56:30.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2214700 1 -- 172.21.15.140:0/2479364945 learned_addr learned my addr 172.21.15.140:0/2479364945 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:30.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b1212700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17ac0af8e0 0x7f17ac142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b1a13700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17ac0adff0 0x7f17ac149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2214700 1 -- 172.21.15.140:0/2479364945 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17ac0af8e0 msgr2=0x7f17ac142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2214700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17ac0af8e0 0x7f17ac142320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2214700 1 -- 172.21.15.140:0/2479364945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17ac0adff0 msgr2=0x7f17ac149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2214700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17ac0adff0 0x7f17ac149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2214700 1 -- 172.21.15.140:0/2479364945 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a400b180 con 0x7f17ac0b5650 2022-07-21T01:56:30.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2214700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17ac0b5650 0x7f17ac142810 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f179c001800 tx=0x7f17a4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:30.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17a2ffd700 1 -- 172.21.15.140:0/2479364945 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17a4016070 con 0x7f17ac0b5650 2022-07-21T01:56:30.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17a2ffd700 1 -- 172.21.15.140:0/2479364945 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a4013b70 con 0x7f17ac0b5650 2022-07-21T01:56:30.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2a15700 1 -- 172.21.15.140:0/2479364945 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ac111330 con 0x7f17ac0b5650 2022-07-21T01:56:30.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17a2ffd700 1 -- 172.21.15.140:0/2479364945 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f17a400d4f0 con 0x7f17ac0b5650 2022-07-21T01:56:30.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2a15700 1 -- 172.21.15.140:0/2479364945 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ac142d00 con 0x7f17ac0b5650 2022-07-21T01:56:30.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.320+0000 7f17b2a15700 1 -- 172.21.15.140:0/2479364945 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ac142ec0 con 0x7f17ac0b5650 2022-07-21T01:56:30.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.324+0000 7f17a2ffd700 1 -- 172.21.15.140:0/2479364945 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f17a400f030 con 0x7f17ac0b5650 2022-07-21T01:56:30.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.324+0000 7f17a2ffd700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f178c053a70 0x7f178c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.324+0000 7f17a2ffd700 1 -- 172.21.15.140:0/2479364945 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f17a406cad0 con 0x7f17ac0b5650 2022-07-21T01:56:30.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.324+0000 7f17b1a13700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f178c053a70 0x7f178c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.324+0000 7f17b1a13700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f178c053a70 0x7f178c055f30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f179c00d000 tx=0x7f179c000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:30.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.328+0000 7f17a2ffd700 1 -- 172.21.15.140:0/2479364945 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f17a40439f0 con 0x7f17ac0b5650 2022-07-21T01:56:30.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.456+0000 7f17b2a15700 1 -- 172.21.15.140:0/2479364945 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17ac0574e0 con 0x7f17ac0b5650 2022-07-21T01:56:30.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.456+0000 7f17a2ffd700 1 -- 172.21.15.140:0/2479364945 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f17a4047010 con 0x7f17ac0b5650 2022-07-21T01:56:30.459 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:30.459 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":112,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:29.930257+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:56:27.736358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-07-21T01:56:28.943049+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:30.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.456+0000 7f17b2a15700 1 -- 172.21.15.140:0/2479364945 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f178c053a70 msgr2=0x7f178c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.456+0000 7f17b2a15700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f178c053a70 0x7f178c055f30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f179c00d000 tx=0x7f179c000bc0).stop 2022-07-21T01:56:30.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.456+0000 7f17b2a15700 1 -- 172.21.15.140:0/2479364945 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17ac0b5650 msgr2=0x7f17ac142810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.456+0000 7f17b2a15700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17ac0b5650 0x7f17ac142810 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f179c001800 tx=0x7f17a4014000).stop 2022-07-21T01:56:30.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.460+0000 7f17b2a15700 1 -- 172.21.15.140:0/2479364945 shutdown_connections 2022-07-21T01:56:30.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.460+0000 7f17b2a15700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f178c053a70 0x7f178c055f30 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.460+0000 7f17b2a15700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17ac0b5650 0x7f17ac142810 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.460+0000 7f17b2a15700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17ac0af8e0 0x7f17ac142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.460+0000 7f17b2a15700 1 --2- 172.21.15.140:0/2479364945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17ac0adff0 0x7f17ac149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.460+0000 7f17b2a15700 1 -- 172.21.15.140:0/2479364945 >> 172.21.15.140:0/2479364945 conn(0x7f17ac0634e0 msgr2=0x7f17ac0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:30.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.460+0000 7f17b2a15700 1 -- 172.21.15.140:0/2479364945 shutdown_connections 2022-07-21T01:56:30.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.460+0000 7f17b2a15700 1 -- 172.21.15.140:0/2479364945 wait complete. 2022-07-21T01:56:30.473 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:56:30.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c7f0cc700 1 Processor -- start 2022-07-21T01:56:30.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c7f0cc700 1 -- start start 2022-07-21T01:56:30.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c7f0cc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c780b4d60 0x7f8c78145d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c7f0cc700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c780607e0 con 0x7f8c78060ee0 2022-07-21T01:56:30.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c7f0cc700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c78060a80 con 0x7f8c7809df40 2022-07-21T01:56:30.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c7f0cc700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c78060d20 con 0x7f8c780b4d60 2022-07-21T01:56:30.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c77fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c780b4d60 0x7f8c78145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c77fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c780b4d60 0x7f8c78145d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39482/0 (socket says 172.21.15.140:39482) 2022-07-21T01:56:30.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c77fff700 1 -- 172.21.15.140:0/2980527958 learned_addr learned my addr 172.21.15.140:0/2980527958 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:30.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c77fff700 1 -- 172.21.15.140:0/2980527958 >> v1:172.21.15.152:6789/0 conn(0x7f8c7809df40 legacy=0x7f8c78145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c77fff700 1 -- 172.21.15.140:0/2980527958 >> v1:172.21.15.140:6789/0 conn(0x7f8c78060ee0 legacy=0x7f8c7809cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c777fe700 1 -- 172.21.15.140:0/2980527958 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304096077 0 0) 0x7f8c780607e0 con 0x7f8c78060ee0 2022-07-21T01:56:30.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c77fff700 1 -- 172.21.15.140:0/2980527958 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7809e320 con 0x7f8c780b4d60 2022-07-21T01:56:30.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c77fff700 1 --2- 172.21.15.140:0/2980527958 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c780b4d60 0x7f8c78145d90 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7f8c68002f70 tx=0x7f8c68006eb0).ready entity=mon.0 client_cookie=a0c66467febcf02b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:30.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c777fe700 1 -- 172.21.15.140:0/2980527958 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c6800d660 con 0x7f8c780b4d60 2022-07-21T01:56:30.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c777fe700 1 -- 172.21.15.140:0/2980527958 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c68000e90 con 0x7f8c780b4d60 2022-07-21T01:56:30.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.576+0000 7f8c777fe700 1 -- 172.21.15.140:0/2980527958 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c6800d9b0 con 0x7f8c780b4d60 2022-07-21T01:56:30.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/2980527958 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c780b4d60 msgr2=0x7f8c78145d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 --2- 172.21.15.140:0/2980527958 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c780b4d60 0x7f8c78145d90 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7f8c68002f70 tx=0x7f8c68006eb0).stop 2022-07-21T01:56:30.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/2980527958 shutdown_connections 2022-07-21T01:56:30.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 --2- 172.21.15.140:0/2980527958 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c780b4d60 0x7f8c78145d90 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/2980527958 >> 172.21.15.140:0/2980527958 conn(0x7f8c780b5f30 msgr2=0x7f8c780b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:30.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/2980527958 shutdown_connections 2022-07-21T01:56:30.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/2980527958 wait complete. 2022-07-21T01:56:30.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 Processor -- start 2022-07-21T01:56:30.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 -- start start 2022-07-21T01:56:30.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c78060ee0 0x7f8c78148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c7809df40 0x7f8c78152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c780b4d60 0x7f8c78155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c781556b0 con 0x7f8c780b4d60 2022-07-21T01:56:30.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c780b26b0 con 0x7f8c78060ee0 2022-07-21T01:56:30.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7f0cc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c780b2410 con 0x7f8c7809df40 2022-07-21T01:56:30.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7d669700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c780b4d60 0x7f8c78155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7d669700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c780b4d60 0x7f8c78155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39490/0 (socket says 172.21.15.140:39490) 2022-07-21T01:56:30.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7d669700 1 -- 172.21.15.140:0/939352207 learned_addr learned my addr 172.21.15.140:0/939352207 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:30.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c77fff700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c7809df40 0x7f8c78152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.580+0000 7f8c7ce68700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c78060ee0 0x7f8c78148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c7d669700 1 -- 172.21.15.140:0/939352207 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c7809df40 msgr2=0x7f8c78152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c7d669700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c7809df40 0x7f8c78152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c7d669700 1 -- 172.21.15.140:0/939352207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c78060ee0 msgr2=0x7f8c78148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:30.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c7d669700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c78060ee0 0x7f8c78148c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:30.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c7d669700 1 -- 172.21.15.140:0/939352207 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6800b1e0 con 0x7f8c780b4d60 2022-07-21T01:56:30.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c7d669700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c780b4d60 0x7f8c78155b10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7f8c70000f40 tx=0x7f8c70007d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:30.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c75ffb700 1 -- 172.21.15.140:0/939352207 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c7000f470 con 0x7f8c780b4d60 2022-07-21T01:56:30.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c75ffb700 1 -- 172.21.15.140:0/939352207 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7000f5d0 con 0x7f8c780b4d60 2022-07-21T01:56:30.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/939352207 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c780b0ea0 con 0x7f8c780b4d60 2022-07-21T01:56:30.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c75ffb700 1 -- 172.21.15.140:0/939352207 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c7000f930 con 0x7f8c780b4d60 2022-07-21T01:56:30.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/939352207 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c780b1410 con 0x7f8c780b4d60 2022-07-21T01:56:30.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c75ffb700 1 -- 172.21.15.140:0/939352207 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8c70010070 con 0x7f8c780b4d60 2022-07-21T01:56:30.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/939352207 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c780574e0 con 0x7f8c780b4d60 2022-07-21T01:56:30.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c75ffb700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c58053a70 0x7f8c58055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:30.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c75ffb700 1 -- 172.21.15.140:0/939352207 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8c7006b2d0 con 0x7f8c780b4d60 2022-07-21T01:56:30.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c7ce68700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c58053a70 0x7f8c58055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:30.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.584+0000 7f8c7ce68700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c58053a70 0x7f8c58055f30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f8c6c00d000 tx=0x7f8c6c007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:30.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.592+0000 7f8c75ffb700 1 -- 172.21.15.140:0/939352207 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c70042080 con 0x7f8c780b4d60 2022-07-21T01:56:30.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:30.760+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/939352207 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8c78061810 con 0x7f8c780b4d60 2022-07-21T01:56:31.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.644+0000 7f8c75ffb700 1 -- 172.21.15.140:0/939352207 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8c700456a0 con 0x7f8c780b4d60 2022-07-21T01:56:31.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/939352207 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c58053a70 msgr2=0x7f8c58055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:31.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c58053a70 0x7f8c58055f30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f8c6c00d000 tx=0x7f8c6c007b80).stop 2022-07-21T01:56:31.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/939352207 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c780b4d60 msgr2=0x7f8c78155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:31.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c780b4d60 0x7f8c78155b10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7f8c70000f40 tx=0x7f8c70007d60).stop 2022-07-21T01:56:31.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/939352207 shutdown_connections 2022-07-21T01:56:31.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c58053a70 0x7f8c58055f30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c780b4d60 0x7f8c78155b10 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c7809df40 0x7f8c78152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 --2- 172.21.15.140:0/939352207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c78060ee0 0x7f8c78148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/939352207 >> 172.21.15.140:0/939352207 conn(0x7f8c780b5f30 msgr2=0x7f8c7805be00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:31.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/939352207 shutdown_connections 2022-07-21T01:56:31.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.648+0000 7f8c7f0cc700 1 -- 172.21.15.140:0/939352207 wait complete. 2022-07-21T01:56:31.665 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:31.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7ba156b700 1 Processor -- start 2022-07-21T01:56:31.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7ba156b700 1 -- start start 2022-07-21T01:56:31.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7ba156b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0af8e0 0x7f7b9c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:31.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7ba156b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c147010 con 0x7f7b9c0adff0 2022-07-21T01:56:31.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7ba156b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c1472b0 con 0x7f7b9c0b5650 2022-07-21T01:56:31.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7ba156b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9c146d40 con 0x7f7b9c0af8e0 2022-07-21T01:56:31.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f7b9c0adff0 0x7f7b9c05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53640/0 (socket says 172.21.15.140:53640) 2022-07-21T01:56:31.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9b7fe700 1 -- 172.21.15.140:0/3775141053 learned_addr learned my addr 172.21.15.140:0/3775141053 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:31.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7ba0d6a700 1 --2- 172.21.15.140:0/3775141053 >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0af8e0 0x7f7b9c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:31.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205525453 0 0) 0x7f7b9c147010 con 0x7f7b9c0adff0 2022-07-21T01:56:31.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b7c003260 con 0x7f7b9c0adff0 2022-07-21T01:56:31.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3973611162 0 0) 0x7f7b7c003260 con 0x7f7b9c0adff0 2022-07-21T01:56:31.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0af8e0 msgr2=0x7f7b9c0b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:31.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 --2- 172.21.15.140:0/3775141053 >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0af8e0 0x7f7b9c0b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 >> v1:172.21.15.152:6789/0 conn(0x7f7b9c0b5650 legacy=0x7f7b9c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:31.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c05c4a0 con 0x7f7b9c0adff0 2022-07-21T01:56:31.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f7b94001b60 con 0x7f7b9c0adff0 2022-07-21T01:56:31.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 >> v1:172.21.15.140:6789/0 conn(0x7f7b9c0adff0 legacy=0x7f7b9c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:31.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c003750 0x7f7b7c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:31.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b7c004500 0x7f7b7c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:31.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b7c004e70 0x7f7b7c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:31.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c146d40 con 0x7f7b7c003750 2022-07-21T01:56:31.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c00bdd0 con 0x7f7b7c004500 2022-07-21T01:56:31.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c00bf60 con 0x7f7b7c004e70 2022-07-21T01:56:31.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9bfff700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c003750 0x7f7b7c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:31.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9b7fe700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b7c004500 0x7f7b7c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:31.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7ba0d6a700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b7c004e70 0x7f7b7c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:31.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9b7fe700 1 -- 172.21.15.140:0/3775141053 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b7c004e70 msgr2=0x7f7b7c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:31.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9b7fe700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b7c004e70 0x7f7b7c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9b7fe700 1 -- 172.21.15.140:0/3775141053 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c003750 msgr2=0x7f7b7c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:31.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9b7fe700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c003750 0x7f7b7c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9b7fe700 1 -- 172.21.15.140:0/3775141053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c003260 con 0x7f7b7c004500 2022-07-21T01:56:31.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7b9bfff700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c003750 0x7f7b7c003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:31.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.768+0000 7f7ba0d6a700 1 -- 172.21.15.140:0/3775141053 reap_dead start 2022-07-21T01:56:31.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7b9b7fe700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b7c004500 0x7f7b7c004980 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f7b94001870 tx=0x7f7b94007910).ready entity=mon.1 client_cookie=4a966a3887d9c32 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:31.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b9400cc70 con 0x7f7b7c004500 2022-07-21T01:56:31.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9400cdd0 con 0x7f7b7c004500 2022-07-21T01:56:31.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7b9affd700 1 -- 172.21.15.140:0/3775141053 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b94005a50 con 0x7f7b7c004500 2022-07-21T01:56:31.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 -- 172.21.15.140:0/3775141053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b7c004500 msgr2=0x7f7b7c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:31.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b7c004500 0x7f7b7c004980 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f7b94001870 tx=0x7f7b94007910).stop 2022-07-21T01:56:31.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 -- 172.21.15.140:0/3775141053 shutdown_connections 2022-07-21T01:56:31.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 --2- 172.21.15.140:0/3775141053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b7c004500 0x7f7b7c004980 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 -- 172.21.15.140:0/3775141053 >> 172.21.15.140:0/3775141053 conn(0x7f7b9c0634e0 msgr2=0x7f7b9c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:31.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 -- 172.21.15.140:0/3775141053 shutdown_connections 2022-07-21T01:56:31.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 -- 172.21.15.140:0/3775141053 wait complete. 2022-07-21T01:56:31.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 Processor -- start 2022-07-21T01:56:31.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 -- start start 2022-07-21T01:56:31.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c004500 0x7f7b9c0648a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:31.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c144fa0 0x7f7b9c143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:31.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c1435f0 0x7f7b9c142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:31.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c145400 con 0x7f7b7c004500 2022-07-21T01:56:31.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c067a20 con 0x7f7b9c1435f0 2022-07-21T01:56:31.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba156b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c111430 con 0x7f7b9c144fa0 2022-07-21T01:56:31.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7b9bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c004500 0x7f7b9c0648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:31.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7b9bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c004500 0x7f7b9c0648a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39506/0 (socket says 172.21.15.140:39506) 2022-07-21T01:56:31.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7b9bfff700 1 -- 172.21.15.140:0/2518034887 learned_addr learned my addr 172.21.15.140:0/2518034887 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:31.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7b9b7fe700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c144fa0 0x7f7b9c143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:31.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.772+0000 7f7ba0d6a700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c1435f0 0x7f7b9c142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:31.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b9b7fe700 1 -- 172.21.15.140:0/2518034887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c1435f0 msgr2=0x7f7b9c142420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:31.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b9b7fe700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c1435f0 0x7f7b9c142420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b9b7fe700 1 -- 172.21.15.140:0/2518034887 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c004500 msgr2=0x7f7b9c0648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:31.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b9b7fe700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c004500 0x7f7b9c0648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b9b7fe700 1 -- 172.21.15.140:0/2518034887 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c111190 con 0x7f7b9c144fa0 2022-07-21T01:56:31.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7ba0d6a700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c1435f0 0x7f7b9c142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:31.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b9bfff700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c004500 0x7f7b9c0648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:31.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b9b7fe700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c144fa0 0x7f7b9c143100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f7b94001870 tx=0x7f7b94015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:31.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b997fa700 1 -- 172.21.15.140:0/2518034887 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b9400cc70 con 0x7f7b9c144fa0 2022-07-21T01:56:31.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b997fa700 1 -- 172.21.15.140:0/2518034887 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b94007b20 con 0x7f7b9c144fa0 2022-07-21T01:56:31.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b997fa700 1 -- 172.21.15.140:0/2518034887 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b940072c0 con 0x7f7b9c144fa0 2022-07-21T01:56:31.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7ba156b700 1 -- 172.21.15.140:0/2518034887 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c067490 con 0x7f7b9c144fa0 2022-07-21T01:56:31.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7ba156b700 1 -- 172.21.15.140:0/2518034887 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c110d30 con 0x7f7b9c144fa0 2022-07-21T01:56:31.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7ba156b700 1 -- 172.21.15.140:0/2518034887 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c067620 con 0x7f7b9c144fa0 2022-07-21T01:56:31.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b997fa700 1 -- 172.21.15.140:0/2518034887 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7b94026070 con 0x7f7b9c144fa0 2022-07-21T01:56:31.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b997fa700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 0x7f7b80055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:31.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b997fa700 1 -- 172.21.15.140:0/2518034887 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7b9406c720 con 0x7f7b9c144fa0 2022-07-21T01:56:31.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b9bfff700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 0x7f7b80055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:31.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.776+0000 7f7b9bfff700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 0x7f7b80055fc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f7b84009000 tx=0x7f7b84009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:31.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.784+0000 7f7b997fa700 1 -- 172.21.15.140:0/2518034887 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7b94043530 con 0x7f7b9c144fa0 2022-07-21T01:56:31.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.948+0000 7f7ba156b700 1 -- 172.21.15.140:0/2518034887 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b9c0574e0 con 0x7f7b9c144fa0 2022-07-21T01:56:31.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.948+0000 7f7b997fa700 1 -- 172.21.15.140:0/2518034887 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f7b94046b50 con 0x7f7b9c144fa0 2022-07-21T01:56:31.951 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:31.951 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":53,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/223182184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":223182184},{"type":"v1","addr":"172.21.15.140:6835","nonce":223182184}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:56:29.930618+0000","modified":"2022-07-21T01:56:31.631480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7662},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7662":{"gid":7662,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/456689269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":456689269},{"type":"v1","addr":"172.21.15.152:6833","nonce":456689269}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-07-21T01:56:31.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.948+0000 7f7ba156b700 1 -- 172.21.15.140:0/2518034887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 msgr2=0x7f7b80055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:31.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.948+0000 7f7ba156b700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 0x7f7b80055fc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f7b84009000 tx=0x7f7b84009040).stop 2022-07-21T01:56:31.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.948+0000 7f7ba156b700 1 -- 172.21.15.140:0/2518034887 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c144fa0 msgr2=0x7f7b9c143100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:31.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.948+0000 7f7ba156b700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c144fa0 0x7f7b9c143100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f7b94001870 tx=0x7f7b94015000).stop 2022-07-21T01:56:31.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.952+0000 7f7ba156b700 1 -- 172.21.15.140:0/2518034887 shutdown_connections 2022-07-21T01:56:31.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.952+0000 7f7ba156b700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 0x7f7b80055fc0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.952+0000 7f7ba156b700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c1435f0 0x7f7b9c142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.952+0000 7f7ba156b700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c144fa0 0x7f7b9c143100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.952+0000 7f7ba156b700 1 --2- 172.21.15.140:0/2518034887 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b7c004500 0x7f7b9c0648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:31.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.952+0000 7f7ba156b700 1 -- 172.21.15.140:0/2518034887 >> 172.21.15.140:0/2518034887 conn(0x7f7b9c0634e0 msgr2=0x7f7b9c144400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:31.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.952+0000 7f7ba156b700 1 -- 172.21.15.140:0/2518034887 shutdown_connections 2022-07-21T01:56:31.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:31.952+0000 7f7ba156b700 1 -- 172.21.15.140:0/2518034887 wait complete. 2022-07-21T01:56:31.957 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 53 2022-07-21T01:56:31.964 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:32.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897759e700 1 Processor -- start 2022-07-21T01:56:32.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897759e700 1 -- start start 2022-07-21T01:56:32.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897759e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f89780b61d0 0x7f89780b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897759e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f89780af120 0x7f8978067260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897759e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89780b20f0 con 0x7f89780b13e0 2022-07-21T01:56:32.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897759e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897813efd0 con 0x7f89780b61d0 2022-07-21T01:56:32.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897759e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89780b65b0 con 0x7f89780af120 2022-07-21T01:56:32.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897659c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f89780b61d0 0x7f89780b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897659c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f89780b61d0 0x7f89780b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39514/0 (socket says 172.21.15.140:39514) 2022-07-21T01:56:32.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897659c700 1 -- 172.21.15.140:0/3436110845 learned_addr learned my addr 172.21.15.140:0/3436110845 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:32.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f8975d9b700 1 --1- 172.21.15.140:0/3436110845 >> v1:172.21.15.140:6789/0 conn(0x7f89780b13e0 0x7f89780aea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53664/0 (socket says 172.21.15.140:53664) 2022-07-21T01:56:32.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f8976d9d700 1 --2- 172.21.15.140:0/3436110845 >> v2:172.21.15.152:3300/0 conn(0x7f89780af120 0x7f8978067260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897659c700 1 -- 172.21.15.140:0/3436110845 >> v2:172.21.15.152:3300/0 conn(0x7f89780af120 msgr2=0x7f8978067260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:32.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897659c700 1 --2- 172.21.15.140:0/3436110845 >> v2:172.21.15.152:3300/0 conn(0x7f89780af120 0x7f8978067260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897659c700 1 -- 172.21.15.140:0/3436110845 >> v1:172.21.15.140:6789/0 conn(0x7f89780b13e0 legacy=0x7f89780aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897659c700 1 -- 172.21.15.140:0/3436110845 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780b1ad0 con 0x7f89780b61d0 2022-07-21T01:56:32.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f8976d9d700 1 --2- 172.21.15.140:0/3436110845 >> v2:172.21.15.152:3300/0 conn(0x7f89780af120 0x7f8978067260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:56:32.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897659c700 1 --2- 172.21.15.140:0/3436110845 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89780b61d0 0x7f89780b0460 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f8968002f70 tx=0x7f8968006e40).ready entity=mon.0 client_cookie=ae4d24185bc7b890 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897559a700 1 -- 172.21.15.140:0/3436110845 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896800bcf0 con 0x7f89780b61d0 2022-07-21T01:56:32.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897559a700 1 -- 172.21.15.140:0/3436110845 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8968000e90 con 0x7f89780b61d0 2022-07-21T01:56:32.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.064+0000 7f897559a700 1 -- 172.21.15.140:0/3436110845 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89680155e0 con 0x7f89780b61d0 2022-07-21T01:56:32.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.068+0000 7f897759e700 1 -- 172.21.15.140:0/3436110845 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89780b61d0 msgr2=0x7f89780b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.068+0000 7f897759e700 1 --2- 172.21.15.140:0/3436110845 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89780b61d0 0x7f89780b0460 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f8968002f70 tx=0x7f8968006e40).stop 2022-07-21T01:56:32.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.068+0000 7f897759e700 1 -- 172.21.15.140:0/3436110845 shutdown_connections 2022-07-21T01:56:32.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.068+0000 7f897759e700 1 --2- 172.21.15.140:0/3436110845 >> v2:172.21.15.152:3300/0 conn(0x7f89780af120 0x7f8978067260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.068+0000 7f897759e700 1 --2- 172.21.15.140:0/3436110845 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89780b61d0 0x7f89780b0460 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.068+0000 7f897759e700 1 -- 172.21.15.140:0/3436110845 >> 172.21.15.140:0/3436110845 conn(0x7f8978063cf0 msgr2=0x7f89780b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:32.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.068+0000 7f897759e700 1 -- 172.21.15.140:0/3436110845 shutdown_connections 2022-07-21T01:56:32.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.068+0000 7f897759e700 1 -- 172.21.15.140:0/3436110845 wait complete. 2022-07-21T01:56:32.091 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.068+0000 7f897759e700 1 Processor -- start 2022-07-21T01:56:32.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.068+0000 7f897759e700 1 -- start start 2022-07-21T01:56:32.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f897759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89780af120 0x7f8978110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f897759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89780b13e0 0x7f897814dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.093 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f897759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89780b61d0 0x7f897814e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f897759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978066ce0 con 0x7f89780b13e0 2022-07-21T01:56:32.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f897759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897810eb30 con 0x7f89780af120 2022-07-21T01:56:32.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f897759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897810fab0 con 0x7f89780b61d0 2022-07-21T01:56:32.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f8975d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89780b13e0 0x7f897814dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f8975d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89780b13e0 0x7f897814dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39522/0 (socket says 172.21.15.140:39522) 2022-07-21T01:56:32.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f8975d9b700 1 -- 172.21.15.140:0/2242805564 learned_addr learned my addr 172.21.15.140:0/2242805564 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:32.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f897659c700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89780af120 0x7f8978110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f8976d9d700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89780b61d0 0x7f897814e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f8975d9b700 1 -- 172.21.15.140:0/2242805564 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89780b61d0 msgr2=0x7f897814e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f8975d9b700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89780b61d0 0x7f897814e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f8975d9b700 1 -- 172.21.15.140:0/2242805564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89780af120 msgr2=0x7f8978110120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f8975d9b700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89780af120 0x7f8978110120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f8975d9b700 1 -- 172.21.15.140:0/2242805564 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896800d4d0 con 0x7f89780b13e0 2022-07-21T01:56:32.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f8975d9b700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89780b13e0 0x7f897814dbe0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f89780640d0 tx=0x7f896c007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f89677fe700 1 -- 172.21.15.140:0/2242805564 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896c0070d0 con 0x7f89780b13e0 2022-07-21T01:56:32.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f89677fe700 1 -- 172.21.15.140:0/2242805564 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c007230 con 0x7f89780b13e0 2022-07-21T01:56:32.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f89677fe700 1 -- 172.21.15.140:0/2242805564 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f896c005550 con 0x7f89780b13e0 2022-07-21T01:56:32.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f897759e700 1 -- 172.21.15.140:0/2242805564 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897814e5c0 con 0x7f89780b13e0 2022-07-21T01:56:32.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f897759e700 1 -- 172.21.15.140:0/2242805564 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8978110830 con 0x7f89780b13e0 2022-07-21T01:56:32.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.072+0000 7f897759e700 1 -- 172.21.15.140:0/2242805564 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897810e970 con 0x7f89780b13e0 2022-07-21T01:56:32.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.076+0000 7f89677fe700 1 -- 172.21.15.140:0/2242805564 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f896c01a070 con 0x7f89780b13e0 2022-07-21T01:56:32.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.076+0000 7f89677fe700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89540539e0 0x7f8954055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.076+0000 7f89677fe700 1 -- 172.21.15.140:0/2242805564 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f896c06b190 con 0x7f89780b13e0 2022-07-21T01:56:32.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.076+0000 7f897659c700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89540539e0 0x7f8954055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.076+0000 7f897659c700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89540539e0 0x7f8954055ea0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f896801a000 tx=0x7f8968018000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.080+0000 7f89677fe700 1 -- 172.21.15.140:0/2242805564 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f896c041fe0 con 0x7f89780b13e0 2022-07-21T01:56:32.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.204+0000 7f897759e700 1 -- 172.21.15.140:0/2242805564 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f897814e7b0 con 0x7f89780b13e0 2022-07-21T01:56:32.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.204+0000 7f89677fe700 1 -- 172.21.15.140:0/2242805564 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f896c045600 con 0x7f89780b13e0 2022-07-21T01:56:32.208 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:32.209 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":113,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:30.933163+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:56:27.736358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-07-21T01:56:28.943049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:32.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 -- 172.21.15.140:0/2242805564 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89540539e0 msgr2=0x7f8954055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89540539e0 0x7f8954055ea0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f896801a000 tx=0x7f8968018000).stop 2022-07-21T01:56:32.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 -- 172.21.15.140:0/2242805564 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89780b13e0 msgr2=0x7f897814dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89780b13e0 0x7f897814dbe0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f89780640d0 tx=0x7f896c007c10).stop 2022-07-21T01:56:32.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 -- 172.21.15.140:0/2242805564 shutdown_connections 2022-07-21T01:56:32.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89540539e0 0x7f8954055ea0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89780b61d0 0x7f897814e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89780b13e0 0x7f897814dbe0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 --2- 172.21.15.140:0/2242805564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89780af120 0x7f8978110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 -- 172.21.15.140:0/2242805564 >> 172.21.15.140:0/2242805564 conn(0x7f8978063cf0 msgr2=0x7f89780acef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:32.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 -- 172.21.15.140:0/2242805564 shutdown_connections 2022-07-21T01:56:32.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.208+0000 7f897759e700 1 -- 172.21.15.140:0/2242805564 wait complete. 2022-07-21T01:56:32.222 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:56:32.275 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:56:32.277 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:56:32.280 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:56:32.283 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:56:32.286 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:56:32.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee7227700 1 Processor -- start 2022-07-21T01:56:32.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee7227700 1 -- start start 2022-07-21T01:56:32.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee7227700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7faee00b4a90 0x7faee00b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee7227700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faee00ae350 0x7faee013e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee7227700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee00b0bc0 con 0x7faee00b0270 2022-07-21T01:56:32.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee7227700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee00b4e70 con 0x7faee00ae350 2022-07-21T01:56:32.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee7227700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee00af470 con 0x7faee00b4a90 2022-07-21T01:56:32.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee6a26700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faee00ae350 0x7faee013e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee6a26700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7faee00ae350 0x7faee013e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39530/0 (socket says 172.21.15.140:39530) 2022-07-21T01:56:32.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee6a26700 1 -- 172.21.15.140:0/3135034646 learned_addr learned my addr 172.21.15.140:0/3135034646 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:32.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee6225700 1 --2- 172.21.15.140:0/3135034646 >> v2:172.21.15.152:3300/0 conn(0x7faee00b4a90 0x7faee00b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee6a26700 1 -- 172.21.15.140:0/3135034646 >> v2:172.21.15.152:3300/0 conn(0x7faee00b4a90 msgr2=0x7faee00b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:32.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee6a26700 1 --2- 172.21.15.140:0/3135034646 >> v2:172.21.15.152:3300/0 conn(0x7faee00b4a90 0x7faee00b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee6a26700 1 -- 172.21.15.140:0/3135034646 >> v1:172.21.15.152:6789/0 conn(0x7faee00b0270 legacy=0x7faee00acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee6a26700 1 -- 172.21.15.140:0/3135034646 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee00af710 con 0x7faee00ae350 2022-07-21T01:56:32.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.332+0000 7faee6a26700 1 --2- 172.21.15.140:0/3135034646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faee00ae350 0x7faee013e860 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7faed80032f0 tx=0x7faed8009290).ready entity=mon.0 client_cookie=dcb1a5d433391fcb server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee5223700 1 -- 172.21.15.140:0/3135034646 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faed800b870 con 0x7faee00ae350 2022-07-21T01:56:32.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee5223700 1 -- 172.21.15.140:0/3135034646 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed800b9d0 con 0x7faee00ae350 2022-07-21T01:56:32.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee5223700 1 -- 172.21.15.140:0/3135034646 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faed800bb40 con 0x7faee00ae350 2022-07-21T01:56:32.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 -- 172.21.15.140:0/3135034646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faee00ae350 msgr2=0x7faee013e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 --2- 172.21.15.140:0/3135034646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faee00ae350 0x7faee013e860 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7faed80032f0 tx=0x7faed8009290).stop 2022-07-21T01:56:32.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 -- 172.21.15.140:0/3135034646 shutdown_connections 2022-07-21T01:56:32.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 --2- 172.21.15.140:0/3135034646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faee00ae350 0x7faee013e860 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 --2- 172.21.15.140:0/3135034646 >> v2:172.21.15.152:3300/0 conn(0x7faee00b4a90 0x7faee00b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 -- 172.21.15.140:0/3135034646 >> 172.21.15.140:0/3135034646 conn(0x7faee0063150 msgr2=0x7faee0063530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:32.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 -- 172.21.15.140:0/3135034646 shutdown_connections 2022-07-21T01:56:32.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 -- 172.21.15.140:0/3135034646 wait complete. 2022-07-21T01:56:32.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 Processor -- start 2022-07-21T01:56:32.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 -- start start 2022-07-21T01:56:32.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faee00ae350 0x7faee00b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faee00b0270 0x7faee0065030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faee00b4a90 0x7faee0065520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee0141920 con 0x7faee00b0270 2022-07-21T01:56:32.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.336+0000 7faee7227700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee00a9e70 con 0x7faee00b4a90 2022-07-21T01:56:32.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee7227700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee0140dd0 con 0x7faee00ae350 2022-07-21T01:56:32.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee6a26700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faee00b4a90 0x7faee0065520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee6a26700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faee00b4a90 0x7faee0065520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:45278/0 (socket says 172.21.15.140:45278) 2022-07-21T01:56:32.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee6225700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faee00ae350 0x7faee00b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee6a26700 1 -- 172.21.15.140:0/4171607629 learned_addr learned my addr 172.21.15.140:0/4171607629 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:32.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee5a24700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faee00b0270 0x7faee0065030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee6a26700 1 -- 172.21.15.140:0/4171607629 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faee00ae350 msgr2=0x7faee00b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee6a26700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faee00ae350 0x7faee00b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee6a26700 1 -- 172.21.15.140:0/4171607629 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faee00b0270 msgr2=0x7faee0065030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee6a26700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faee00b0270 0x7faee0065030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee6a26700 1 -- 172.21.15.140:0/4171607629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed8009530 con 0x7faee00b4a90 2022-07-21T01:56:32.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee6a26700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faee00b4a90 0x7faee0065520 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7faee00a9b50 tx=0x7faed8016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faed37fe700 1 -- 172.21.15.140:0/4171607629 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faed800b870 con 0x7faee00b4a90 2022-07-21T01:56:32.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faed37fe700 1 -- 172.21.15.140:0/4171607629 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed8021670 con 0x7faee00b4a90 2022-07-21T01:56:32.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faed37fe700 1 -- 172.21.15.140:0/4171607629 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faed80219d0 con 0x7faee00b4a90 2022-07-21T01:56:32.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee7227700 1 -- 172.21.15.140:0/4171607629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee0066d00 con 0x7faee00b4a90 2022-07-21T01:56:32.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee7227700 1 -- 172.21.15.140:0/4171607629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee01346a0 con 0x7faee00b4a90 2022-07-21T01:56:32.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faee7227700 1 -- 172.21.15.140:0/4171607629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee00a9cb0 con 0x7faee00b4a90 2022-07-21T01:56:32.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faed37fe700 1 -- 172.21.15.140:0/4171607629 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7faed800f070 con 0x7faee00b4a90 2022-07-21T01:56:32.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.340+0000 7faed37fe700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faec0053950 0x7faec0055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.344+0000 7faed37fe700 1 -- 172.21.15.140:0/4171607629 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faed806d030 con 0x7faee00b4a90 2022-07-21T01:56:32.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.344+0000 7faee6225700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faec0053950 0x7faec0055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.344+0000 7faee6225700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faec0053950 0x7faec0055e10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7faed400c000 tx=0x7faed4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.352+0000 7faed37fe700 1 -- 172.21.15.140:0/4171607629 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7faed8043f50 con 0x7faee00b4a90 2022-07-21T01:56:32.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.516+0000 7faee7227700 1 -- 172.21.15.140:0/4171607629 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7faee00af9b0 con 0x7faee00b4a90 2022-07-21T01:56:32.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faed37fe700 1 -- 172.21.15.140:0/4171607629 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v40) v1 ==== 184+0+0 (secure 0 0 0) 0x7faed8026030 con 0x7faee00b4a90 2022-07-21T01:56:32.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 -- 172.21.15.140:0/4171607629 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faec0053950 msgr2=0x7faec0055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faec0053950 0x7faec0055e10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7faed400c000 tx=0x7faed4008040).stop 2022-07-21T01:56:32.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 -- 172.21.15.140:0/4171607629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faee00b4a90 msgr2=0x7faee0065520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faee00b4a90 0x7faee0065520 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7faee00a9b50 tx=0x7faed8016000).stop 2022-07-21T01:56:32.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 -- 172.21.15.140:0/4171607629 shutdown_connections 2022-07-21T01:56:32.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7faec0053950 0x7faec0055e10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faee00b4a90 0x7faee0065520 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7faee00b0270 0x7faee0065030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 --2- 172.21.15.140:0/4171607629 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7faee00ae350 0x7faee00b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 -- 172.21.15.140:0/4171607629 >> 172.21.15.140:0/4171607629 conn(0x7faee0063150 msgr2=0x7faee00603e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:32.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 -- 172.21.15.140:0/4171607629 shutdown_connections 2022-07-21T01:56:32.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.528+0000 7faee7227700 1 -- 172.21.15.140:0/4171607629 wait complete. 2022-07-21T01:56:32.539 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:56:32.546 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:56:32.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.656+0000 7f4c6de50700 1 Processor -- start 2022-07-21T01:56:32.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 -- start start 2022-07-21T01:56:32.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4c680b1c30 0x7f4c68144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c68066cb0 0x7f4c68141100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c680649d0 con 0x7f4c6805cf40 2022-07-21T01:56:32.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c681442c0 con 0x7f4c680b1c30 2022-07-21T01:56:32.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68144560 con 0x7f4c68066cb0 2022-07-21T01:56:32.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c66ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4c680b1c30 0x7f4c68144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c66ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4c680b1c30 0x7f4c68144ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39546/0 (socket says 172.21.15.140:39546) 2022-07-21T01:56:32.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c66ffd700 1 -- 172.21.15.140:0/4141696720 learned_addr learned my addr 172.21.15.140:0/4141696720 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:32.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c67fff700 1 --2- 172.21.15.140:0/4141696720 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c68066cb0 0x7f4c68141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c66ffd700 1 -- 172.21.15.140:0/4141696720 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c68066cb0 msgr2=0x7f4c68141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:32.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c66ffd700 1 --2- 172.21.15.140:0/4141696720 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c68066cb0 0x7f4c68141100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c66ffd700 1 -- 172.21.15.140:0/4141696720 >> v1:172.21.15.152:6789/0 conn(0x7f4c6805cf40 legacy=0x7f4c68061da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c66ffd700 1 -- 172.21.15.140:0/4141696720 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c68144800 con 0x7f4c680b1c30 2022-07-21T01:56:32.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c66ffd700 1 --2- 172.21.15.140:0/4141696720 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c680b1c30 0x7f4c68144ca0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f4c50002f70 tx=0x7f4c50006e50).ready entity=mon.0 client_cookie=7fbe48334bf60b5 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c667fc700 1 -- 172.21.15.140:0/4141696720 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c5000d5c0 con 0x7f4c680b1c30 2022-07-21T01:56:32.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c667fc700 1 -- 172.21.15.140:0/4141696720 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c50000e90 con 0x7f4c680b1c30 2022-07-21T01:56:32.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c667fc700 1 -- 172.21.15.140:0/4141696720 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c5000d950 con 0x7f4c680b1c30 2022-07-21T01:56:32.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 -- 172.21.15.140:0/4141696720 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c680b1c30 msgr2=0x7f4c68144ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 --2- 172.21.15.140:0/4141696720 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c680b1c30 0x7f4c68144ca0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f4c50002f70 tx=0x7f4c50006e50).stop 2022-07-21T01:56:32.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 -- 172.21.15.140:0/4141696720 shutdown_connections 2022-07-21T01:56:32.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 --2- 172.21.15.140:0/4141696720 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c68066cb0 0x7f4c68141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 --2- 172.21.15.140:0/4141696720 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c680b1c30 0x7f4c68144ca0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 -- 172.21.15.140:0/4141696720 >> 172.21.15.140:0/4141696720 conn(0x7f4c680b7d60 msgr2=0x7f4c680b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:32.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 -- 172.21.15.140:0/4141696720 shutdown_connections 2022-07-21T01:56:32.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.660+0000 7f4c6de50700 1 -- 172.21.15.140:0/4141696720 wait complete. 2022-07-21T01:56:32.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c6de50700 1 Processor -- start 2022-07-21T01:56:32.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c6de50700 1 -- start start 2022-07-21T01:56:32.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c6de50700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c6805cf40 0x7f4c68094080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c6de50700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c68066cb0 0x7f4c68155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c6de50700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c680b1c30 0x7f4c68155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c6de50700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680613b0 con 0x7f4c680b1c30 2022-07-21T01:56:32.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c6de50700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680b0470 con 0x7f4c68066cb0 2022-07-21T01:56:32.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c6de50700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68152610 con 0x7f4c6805cf40 2022-07-21T01:56:32.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c67fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c680b1c30 0x7f4c68155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c677fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c6805cf40 0x7f4c68094080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c66ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c68066cb0 0x7f4c68155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c677fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c6805cf40 0x7f4c68094080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36278/0 (socket says 172.21.15.140:36278) 2022-07-21T01:56:32.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c677fe700 1 -- 172.21.15.140:0/463067239 learned_addr learned my addr 172.21.15.140:0/463067239 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:32.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c67fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c680b1c30 0x7f4c68155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39552/0 (socket says 172.21.15.140:39552) 2022-07-21T01:56:32.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c67fff700 1 -- 172.21.15.140:0/463067239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c6805cf40 msgr2=0x7f4c68094080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c67fff700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c6805cf40 0x7f4c68094080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c67fff700 1 -- 172.21.15.140:0/463067239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c68066cb0 msgr2=0x7f4c68155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c67fff700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c68066cb0 0x7f4c68155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c67fff700 1 -- 172.21.15.140:0/463067239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5000b1e0 con 0x7f4c680b1c30 2022-07-21T01:56:32.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c677fe700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c6805cf40 0x7f4c68094080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:32.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c67fff700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c680b1c30 0x7f4c68155a30 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f4c6000c000 tx=0x7f4c60006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c64ff9700 1 -- 172.21.15.140:0/463067239 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c60011750 con 0x7f4c680b1c30 2022-07-21T01:56:32.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c64ff9700 1 -- 172.21.15.140:0/463067239 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c600118b0 con 0x7f4c680b1c30 2022-07-21T01:56:32.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c6de50700 1 -- 172.21.15.140:0/463067239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c681528b0 con 0x7f4c680b1c30 2022-07-21T01:56:32.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c64ff9700 1 -- 172.21.15.140:0/463067239 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c60011c10 con 0x7f4c680b1c30 2022-07-21T01:56:32.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.664+0000 7f4c6de50700 1 -- 172.21.15.140:0/463067239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c680af690 con 0x7f4c680b1c30 2022-07-21T01:56:32.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.668+0000 7f4c6de50700 1 -- 172.21.15.140:0/463067239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c68152d50 con 0x7f4c680b1c30 2022-07-21T01:56:32.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.668+0000 7f4c64ff9700 1 -- 172.21.15.140:0/463067239 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4c60011d70 con 0x7f4c680b1c30 2022-07-21T01:56:32.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.668+0000 7f4c64ff9700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c4c053b00 0x7f4c4c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.668+0000 7f4c64ff9700 1 -- 172.21.15.140:0/463067239 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c6006c2a0 con 0x7f4c680b1c30 2022-07-21T01:56:32.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.668+0000 7f4c677fe700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c4c053b00 0x7f4c4c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.668+0000 7f4c677fe700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c4c053b00 0x7f4c4c055fc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f4c5800d000 tx=0x7f4c58009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.676+0000 7f4c64ff9700 1 -- 172.21.15.140:0/463067239 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4c68152d50 con 0x7f4c680b1c30 2022-07-21T01:56:32.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.840+0000 7f4c6de50700 1 -- 172.21.15.140:0/463067239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f4c68152d50 con 0x7f4c680b1c30 2022-07-21T01:56:32.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c64ff9700 1 -- 172.21.15.140:0/463067239 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v41) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4c68152d50 con 0x7f4c680b1c30 2022-07-21T01:56:32.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 -- 172.21.15.140:0/463067239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c4c053b00 msgr2=0x7f4c4c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c4c053b00 0x7f4c4c055fc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f4c5800d000 tx=0x7f4c58009040).stop 2022-07-21T01:56:32.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 -- 172.21.15.140:0/463067239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c680b1c30 msgr2=0x7f4c68155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c680b1c30 0x7f4c68155a30 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f4c6000c000 tx=0x7f4c60006df0).stop 2022-07-21T01:56:32.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 -- 172.21.15.140:0/463067239 shutdown_connections 2022-07-21T01:56:32.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4c4c053b00 0x7f4c4c055fc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4c680b1c30 0x7f4c68155a30 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c68066cb0 0x7f4c68155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 --2- 172.21.15.140:0/463067239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4c6805cf40 0x7f4c68094080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 -- 172.21.15.140:0/463067239 >> 172.21.15.140:0/463067239 conn(0x7f4c680b7d60 msgr2=0x7f4c68140b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:32.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 -- 172.21.15.140:0/463067239 shutdown_connections 2022-07-21T01:56:32.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.848+0000 7f4c6de50700 1 -- 172.21.15.140:0/463067239 wait complete. 2022-07-21T01:56:32.855 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:56:32.863 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:56:32.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.968+0000 7f8b51ec2700 1 Processor -- start 2022-07-21T01:56:32.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.968+0000 7f8b51ec2700 1 -- start start 2022-07-21T01:56:32.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b4c05cf40 0x7f8b4c146a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8b4c0b5830 0x7f8b4c142c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8b4c143120 0x7f8b4c0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4c0b0130 con 0x7f8b4c0b5830 2022-07-21T01:56:32.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4c1462d0 con 0x7f8b4c143120 2022-07-21T01:56:32.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c146570 con 0x7f8b4c05cf40 2022-07-21T01:56:32.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b43fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8b4c0b5830 0x7f8b4c142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b43fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8b4c0b5830 0x7f8b4c142c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39556/0 (socket says 172.21.15.140:39556) 2022-07-21T01:56:32.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b43fff700 1 -- 172.21.15.140:0/3024718783 learned_addr learned my addr 172.21.15.140:0/3024718783 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:32.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b4bfff700 1 --2- 172.21.15.140:0/3024718783 >> v2:172.21.15.152:3300/0 conn(0x7f8b4c143120 0x7f8b4c0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b4b7fe700 1 --2- 172.21.15.140:0/3024718783 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b4c05cf40 0x7f8b4c146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b43fff700 1 -- 172.21.15.140:0/3024718783 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b4c05cf40 msgr2=0x7f8b4c146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:32.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b43fff700 1 --2- 172.21.15.140:0/3024718783 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b4c05cf40 0x7f8b4c146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b43fff700 1 -- 172.21.15.140:0/3024718783 >> v2:172.21.15.152:3300/0 conn(0x7f8b4c143120 msgr2=0x7f8b4c0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:32.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b43fff700 1 --2- 172.21.15.140:0/3024718783 >> v2:172.21.15.152:3300/0 conn(0x7f8b4c143120 0x7f8b4c0af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b43fff700 1 -- 172.21.15.140:0/3024718783 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c061280 con 0x7f8b4c0b5830 2022-07-21T01:56:32.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b43fff700 1 --2- 172.21.15.140:0/3024718783 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b4c0b5830 0x7f8b4c142c30 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f8b34002f70 tx=0x7f8b34006eb0).ready entity=mon.0 client_cookie=2f4c17109bccd015 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b4affd700 1 -- 172.21.15.140:0/3024718783 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b3400d570 con 0x7f8b4c0b5830 2022-07-21T01:56:32.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b4affd700 1 -- 172.21.15.140:0/3024718783 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b34000e90 con 0x7f8b4c0b5830 2022-07-21T01:56:32.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b4affd700 1 -- 172.21.15.140:0/3024718783 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b3400d900 con 0x7f8b4c0b5830 2022-07-21T01:56:32.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 -- 172.21.15.140:0/3024718783 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b4c0b5830 msgr2=0x7f8b4c142c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 --2- 172.21.15.140:0/3024718783 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b4c0b5830 0x7f8b4c142c30 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f8b34002f70 tx=0x7f8b34006eb0).stop 2022-07-21T01:56:32.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 -- 172.21.15.140:0/3024718783 shutdown_connections 2022-07-21T01:56:32.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 --2- 172.21.15.140:0/3024718783 >> v2:172.21.15.152:3300/0 conn(0x7f8b4c143120 0x7f8b4c0af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 --2- 172.21.15.140:0/3024718783 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b4c0b5830 0x7f8b4c142c30 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 --2- 172.21.15.140:0/3024718783 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b4c05cf40 0x7f8b4c146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 -- 172.21.15.140:0/3024718783 >> 172.21.15.140:0/3024718783 conn(0x7f8b4c0b69e0 msgr2=0x7f8b4c066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:32.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 -- 172.21.15.140:0/3024718783 shutdown_connections 2022-07-21T01:56:32.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.972+0000 7f8b51ec2700 1 -- 172.21.15.140:0/3024718783 wait complete. 2022-07-21T01:56:32.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b51ec2700 1 Processor -- start 2022-07-21T01:56:32.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b51ec2700 1 -- start start 2022-07-21T01:56:32.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b51ec2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b4c05cf40 0x7f8b4c099d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b51ec2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b4c0b5830 0x7f8b4c09c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b51ec2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b4c143120 0x7f8b4c152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b51ec2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c13d9c0 con 0x7f8b4c143120 2022-07-21T01:56:32.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b51ec2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c13dc60 con 0x7f8b4c0b5830 2022-07-21T01:56:32.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b51ec2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c13df00 con 0x7f8b4c05cf40 2022-07-21T01:56:32.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b4bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b4c143120 0x7f8b4c152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b4bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b4c143120 0x7f8b4c152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39566/0 (socket says 172.21.15.140:39566) 2022-07-21T01:56:32.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b4bfff700 1 -- 172.21.15.140:0/2194793937 learned_addr learned my addr 172.21.15.140:0/2194793937 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:32.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b4b7fe700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b4c05cf40 0x7f8b4c099d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b43fff700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b4c0b5830 0x7f8b4c09c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b4bfff700 1 -- 172.21.15.140:0/2194793937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b4c05cf40 msgr2=0x7f8b4c099d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b4bfff700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b4c05cf40 0x7f8b4c099d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b4bfff700 1 -- 172.21.15.140:0/2194793937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b4c0b5830 msgr2=0x7f8b4c09c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:32.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b4bfff700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b4c0b5830 0x7f8b4c09c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:32.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b4bfff700 1 -- 172.21.15.140:0/2194793937 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3400b1e0 con 0x7f8b4c143120 2022-07-21T01:56:32.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b4bfff700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b4c143120 0x7f8b4c152420 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f8b44006e00 tx=0x7f8b440091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b497fa700 1 -- 172.21.15.140:0/2194793937 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b4400c670 con 0x7f8b4c143120 2022-07-21T01:56:32.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.976+0000 7f8b497fa700 1 -- 172.21.15.140:0/2194793937 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4400c7d0 con 0x7f8b4c143120 2022-07-21T01:56:32.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.980+0000 7f8b51ec2700 1 -- 172.21.15.140:0/2194793937 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c151e20 con 0x7f8b4c143120 2022-07-21T01:56:32.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.980+0000 7f8b497fa700 1 -- 172.21.15.140:0/2194793937 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b4400cb30 con 0x7f8b4c143120 2022-07-21T01:56:32.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.980+0000 7f8b51ec2700 1 -- 172.21.15.140:0/2194793937 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c13c7f0 con 0x7f8b4c143120 2022-07-21T01:56:32.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.980+0000 7f8b497fa700 1 -- 172.21.15.140:0/2194793937 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8b4400cc90 con 0x7f8b4c143120 2022-07-21T01:56:32.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.980+0000 7f8b51ec2700 1 -- 172.21.15.140:0/2194793937 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4c152010 con 0x7f8b4c143120 2022-07-21T01:56:32.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.980+0000 7f8b497fa700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8b30053950 0x7f8b30055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:32.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.980+0000 7f8b497fa700 1 -- 172.21.15.140:0/2194793937 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8b4406c280 con 0x7f8b4c143120 2022-07-21T01:56:32.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.980+0000 7f8b4b7fe700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8b30053950 0x7f8b30055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:32.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.980+0000 7f8b4b7fe700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8b30053950 0x7f8b30055e10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f8b3c00b000 tx=0x7f8b3c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:32.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:32.984+0000 7f8b497fa700 1 -- 172.21.15.140:0/2194793937 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8b44043010 con 0x7f8b4c143120 2022-07-21T01:56:33.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.144+0000 7f8b51ec2700 1 -- 172.21.15.140:0/2194793937 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8b4c05d320 con 0x7f8b4c143120 2022-07-21T01:56:33.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.152+0000 7f8b497fa700 1 -- 172.21.15.140:0/2194793937 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v42) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8b44046630 con 0x7f8b4c143120 2022-07-21T01:56:33.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 -- 172.21.15.140:0/2194793937 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8b30053950 msgr2=0x7f8b30055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8b30053950 0x7f8b30055e10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f8b3c00b000 tx=0x7f8b3c006df0).stop 2022-07-21T01:56:33.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 -- 172.21.15.140:0/2194793937 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b4c143120 msgr2=0x7f8b4c152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b4c143120 0x7f8b4c152420 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f8b44006e00 tx=0x7f8b440091d0).stop 2022-07-21T01:56:33.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 -- 172.21.15.140:0/2194793937 shutdown_connections 2022-07-21T01:56:33.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8b30053950 0x7f8b30055e10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b4c143120 0x7f8b4c152420 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b4c0b5830 0x7f8b4c09c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 --2- 172.21.15.140:0/2194793937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b4c05cf40 0x7f8b4c099d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 -- 172.21.15.140:0/2194793937 >> 172.21.15.140:0/2194793937 conn(0x7f8b4c0b69e0 msgr2=0x7f8b4c0af280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:33.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 -- 172.21.15.140:0/2194793937 shutdown_connections 2022-07-21T01:56:33.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.156+0000 7f8b51ec2700 1 -- 172.21.15.140:0/2194793937 wait complete. 2022-07-21T01:56:33.160 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:56:33.170 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:33.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 Processor -- start 2022-07-21T01:56:33.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 -- start start 2022-07-21T01:56:33.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f35e805cda0 0x7f35e805d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:33.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e80ab6f0 con 0x7f35e813e1f0 2022-07-21T01:56:33.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e80ab990 con 0x7f35e80ab010 2022-07-21T01:56:33.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e8061000 con 0x7f35e805cda0 2022-07-21T01:56:33.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e7fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f35e813e1f0 0x7f35e8060060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53720/0 (socket says 172.21.15.140:53720) 2022-07-21T01:56:33.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e7fff700 1 -- 172.21.15.140:0/3445859484 learned_addr learned my addr 172.21.15.140:0/3445859484 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:33.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e77fe700 1 --2- 172.21.15.140:0/3445859484 >> v2:172.21.15.152:3300/0 conn(0x7f35e805cda0 0x7f35e805d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:33.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e67fc700 1 -- 172.21.15.140:0/3445859484 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092918965 0 0) 0x7f35e80ab6f0 con 0x7f35e813e1f0 2022-07-21T01:56:33.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e67fc700 1 -- 172.21.15.140:0/3445859484 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d4003260 con 0x7f35e813e1f0 2022-07-21T01:56:33.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e77fe700 1 -- 172.21.15.140:0/3445859484 >> v1:172.21.15.152:6789/0 conn(0x7f35e80ab010 legacy=0x7f35e813f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e77fe700 1 -- 172.21.15.140:0/3445859484 >> v1:172.21.15.140:6789/0 conn(0x7f35e813e1f0 legacy=0x7f35e8060060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e77fe700 1 -- 172.21.15.140:0/3445859484 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e80abc30 con 0x7f35e805cda0 2022-07-21T01:56:33.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e77fe700 1 --2- 172.21.15.140:0/3445859484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e805cda0 0x7f35e805d180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f35d80032f0 tx=0x7f35d80091d0).ready entity=mon.1 client_cookie=2b539dd5cff08ff4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:33.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e67fc700 1 -- 172.21.15.140:0/3445859484 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d800b8b0 con 0x7f35e805cda0 2022-07-21T01:56:33.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e67fc700 1 -- 172.21.15.140:0/3445859484 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d800ba10 con 0x7f35e805cda0 2022-07-21T01:56:33.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35e67fc700 1 -- 172.21.15.140:0/3445859484 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d800bd70 con 0x7f35e805cda0 2022-07-21T01:56:33.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 -- 172.21.15.140:0/3445859484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e805cda0 msgr2=0x7f35e805d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 --2- 172.21.15.140:0/3445859484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e805cda0 0x7f35e805d180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f35d80032f0 tx=0x7f35d80091d0).stop 2022-07-21T01:56:33.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 -- 172.21.15.140:0/3445859484 shutdown_connections 2022-07-21T01:56:33.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 --2- 172.21.15.140:0/3445859484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e805cda0 0x7f35e805d180 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 -- 172.21.15.140:0/3445859484 >> 172.21.15.140:0/3445859484 conn(0x7f35e80b5e60 msgr2=0x7f35e80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:33.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 -- 172.21.15.140:0/3445859484 shutdown_connections 2022-07-21T01:56:33.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.272+0000 7f35ec9d6700 1 -- 172.21.15.140:0/3445859484 wait complete. 2022-07-21T01:56:33.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35ec9d6700 1 Processor -- start 2022-07-21T01:56:33.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35ec9d6700 1 -- start start 2022-07-21T01:56:33.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35ec9d6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35e805cda0 0x7f35e8093d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:33.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35ec9d6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f35e80ab010 0x7f35e8064960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:33.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35ec9d6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e813e1f0 0x7f35e8064e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:33.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35ec9d6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e8151d90 con 0x7f35e805cda0 2022-07-21T01:56:33.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35ec9d6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e8099720 con 0x7f35e813e1f0 2022-07-21T01:56:33.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35ec9d6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e80999c0 con 0x7f35e80ab010 2022-07-21T01:56:33.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35e805cda0 0x7f35e8093d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:33.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35e805cda0 0x7f35e8093d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39576/0 (socket says 172.21.15.140:39576) 2022-07-21T01:56:33.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e77fe700 1 -- 172.21.15.140:0/242663533 learned_addr learned my addr 172.21.15.140:0/242663533 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:33.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e7fff700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e813e1f0 0x7f35e8064e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:33.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e6ffd700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f35e80ab010 0x7f35e8064960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:33.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e77fe700 1 -- 172.21.15.140:0/242663533 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f35e80ab010 msgr2=0x7f35e8064960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e77fe700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f35e80ab010 0x7f35e8064960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e77fe700 1 -- 172.21.15.140:0/242663533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e813e1f0 msgr2=0x7f35e8064e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e77fe700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e813e1f0 0x7f35e8064e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e77fe700 1 -- 172.21.15.140:0/242663533 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d8009470 con 0x7f35e805cda0 2022-07-21T01:56:33.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e7fff700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e813e1f0 0x7f35e8064e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:56:33.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e77fe700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35e805cda0 0x7f35e8093d60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f35d80032f0 tx=0x7f35d8018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:33.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e4ff9700 1 -- 172.21.15.140:0/242663533 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d801d070 con 0x7f35e805cda0 2022-07-21T01:56:33.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e4ff9700 1 -- 172.21.15.140:0/242663533 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d8022420 con 0x7f35e805cda0 2022-07-21T01:56:33.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35ec9d6700 1 -- 172.21.15.140:0/242663533 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e8099c60 con 0x7f35e805cda0 2022-07-21T01:56:33.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35e4ff9700 1 -- 172.21.15.140:0/242663533 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d8022780 con 0x7f35e805cda0 2022-07-21T01:56:33.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.276+0000 7f35ec9d6700 1 -- 172.21.15.140:0/242663533 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e809a880 con 0x7f35e805cda0 2022-07-21T01:56:33.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.280+0000 7f35ec9d6700 1 -- 172.21.15.140:0/242663533 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e80574e0 con 0x7f35e805cda0 2022-07-21T01:56:33.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.280+0000 7f35e4ff9700 1 -- 172.21.15.140:0/242663533 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f35d80228e0 con 0x7f35e805cda0 2022-07-21T01:56:33.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.280+0000 7f35e4ff9700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f35c8053a70 0x7f35c8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:33.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.280+0000 7f35e4ff9700 1 -- 172.21.15.140:0/242663533 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f35d806d9b0 con 0x7f35e805cda0 2022-07-21T01:56:33.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.280+0000 7f35e6ffd700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f35c8053a70 0x7f35c8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:33.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.280+0000 7f35e6ffd700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f35c8053a70 0x7f35c8055f30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f35dc008000 tx=0x7f35dc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:33.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.284+0000 7f35e4ff9700 1 -- 172.21.15.140:0/242663533 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f35d80448d0 con 0x7f35e805cda0 2022-07-21T01:56:33.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.444+0000 7f35ec9d6700 1 -- 172.21.15.140:0/242663533 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35e80afa10 con 0x7f35e805cda0 2022-07-21T01:56:33.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.444+0000 7f35e4ff9700 1 -- 172.21.15.140:0/242663533 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f35d8027070 con 0x7f35e805cda0 2022-07-21T01:56:33.448 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:33.448 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":53,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/223182184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":223182184},{"type":"v1","addr":"172.21.15.140:6835","nonce":223182184}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:56:29.930618+0000","modified":"2022-07-21T01:56:31.631480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7662},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7662":{"gid":7662,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/456689269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":456689269},{"type":"v1","addr":"172.21.15.152:6833","nonce":456689269}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-07-21T01:56:33.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 -- 172.21.15.140:0/242663533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f35c8053a70 msgr2=0x7f35c8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f35c8053a70 0x7f35c8055f30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f35dc008000 tx=0x7f35dc009040).stop 2022-07-21T01:56:33.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 -- 172.21.15.140:0/242663533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35e805cda0 msgr2=0x7f35e8093d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35e805cda0 0x7f35e8093d60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f35d80032f0 tx=0x7f35d8018000).stop 2022-07-21T01:56:33.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 -- 172.21.15.140:0/242663533 shutdown_connections 2022-07-21T01:56:33.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f35c8053a70 0x7f35c8055f30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35e813e1f0 0x7f35e8064e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f35e80ab010 0x7f35e8064960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 --2- 172.21.15.140:0/242663533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f35e805cda0 0x7f35e8093d60 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 -- 172.21.15.140:0/242663533 >> 172.21.15.140:0/242663533 conn(0x7f35e80b5e60 msgr2=0x7f35e80b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:33.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 -- 172.21.15.140:0/242663533 shutdown_connections 2022-07-21T01:56:33.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.448+0000 7f35ec9d6700 1 -- 172.21.15.140:0/242663533 wait complete. 2022-07-21T01:56:33.455 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 53 2022-07-21T01:56:33.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:56:29.930618+0000', 'modified': '2022-07-21T01:56:31.631480+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7662}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7662': {'gid': 7662, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/456689269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 456689269}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 456689269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:56:33.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:56:33.667 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:56:33.781 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":7662,"want_state":"up:active","state":"up:active","rank_uptime":3.839101286,"mdsmap_epoch":53,"osdmap_epoch":113,"osdmap_epoch_barrier":113,"uptime":8.8623653559999997} 2022-07-21T01:56:33.782 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 7662, "mdsmap_epoch": 53, "osdmap_epoch": 113, "osdmap_epoch_barrier": 113, "rank_uptime": 3.839101286, "state": "up:active", "uptime": 8.862365356, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:56:33.783 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:33.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.888+0000 7f9767fab700 1 Processor -- start 2022-07-21T01:56:33.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.888+0000 7f9767fab700 1 -- start start 2022-07-21T01:56:33.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97600b1e00 0x7f976014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:33.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f976005c3f0 0x7f97600ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:33.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976005d480 con 0x7f976005bd90 2022-07-21T01:56:33.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97600b21e0 con 0x7f976005c3f0 2022-07-21T01:56:33.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976005cf40 con 0x7f97600b1e00 2022-07-21T01:56:33.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9765546700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f976005c3f0 0x7f97600ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:33.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9766548700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f976005bd90 0x7f9760146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53734/0 (socket says 172.21.15.140:53734) 2022-07-21T01:56:33.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9766548700 1 -- 172.21.15.140:0/4006285820 learned_addr learned my addr 172.21.15.140:0/4006285820 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:33.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9765d47700 1 --2- 172.21.15.140:0/4006285820 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97600b1e00 0x7f976014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:33.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9765546700 1 -- 172.21.15.140:0/4006285820 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97600b1e00 msgr2=0x7f976014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:33.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9765546700 1 --2- 172.21.15.140:0/4006285820 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97600b1e00 0x7f976014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9765546700 1 -- 172.21.15.140:0/4006285820 >> v1:172.21.15.140:6789/0 conn(0x7f976005bd90 legacy=0x7f9760146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9765546700 1 -- 172.21.15.140:0/4006285820 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976005d1e0 con 0x7f976005c3f0 2022-07-21T01:56:33.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9765546700 1 --2- 172.21.15.140:0/4006285820 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f976005c3f0 0x7f97600ac7e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f97500032f0 tx=0x7f97500091d0).ready entity=mon.0 client_cookie=2dc1ead3f5390f84 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:33.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9764d45700 1 -- 172.21.15.140:0/4006285820 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9750009ea0 con 0x7f976005c3f0 2022-07-21T01:56:33.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9764d45700 1 -- 172.21.15.140:0/4006285820 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975000b8a0 con 0x7f976005c3f0 2022-07-21T01:56:33.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9764d45700 1 -- 172.21.15.140:0/4006285820 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f975000ba10 con 0x7f976005c3f0 2022-07-21T01:56:33.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 -- 172.21.15.140:0/4006285820 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f976005c3f0 msgr2=0x7f97600ac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 --2- 172.21.15.140:0/4006285820 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f976005c3f0 0x7f97600ac7e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f97500032f0 tx=0x7f97500091d0).stop 2022-07-21T01:56:33.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 -- 172.21.15.140:0/4006285820 shutdown_connections 2022-07-21T01:56:33.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 --2- 172.21.15.140:0/4006285820 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f976005c3f0 0x7f97600ac7e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 --2- 172.21.15.140:0/4006285820 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97600b1e00 0x7f976014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 -- 172.21.15.140:0/4006285820 >> 172.21.15.140:0/4006285820 conn(0x7f97600611c0 msgr2=0x7f9760062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:33.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 -- 172.21.15.140:0/4006285820 shutdown_connections 2022-07-21T01:56:33.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 -- 172.21.15.140:0/4006285820 wait complete. 2022-07-21T01:56:33.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 Processor -- start 2022-07-21T01:56:33.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.892+0000 7f9767fab700 1 -- start start 2022-07-21T01:56:33.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9767fab700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f976005c3f0 0x7f9760144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:33.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9767fab700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97600b1e00 0x7f97600a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:33.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9767fab700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f976009e110 0x7f97600a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:33.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9767fab700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976009e5b0 con 0x7f976005c3f0 2022-07-21T01:56:33.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9767fab700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9760143400 con 0x7f976009e110 2022-07-21T01:56:33.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9767fab700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976009e9c0 con 0x7f97600b1e00 2022-07-21T01:56:33.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9765d47700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f976005c3f0 0x7f9760144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:33.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9765d47700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f976005c3f0 0x7f9760144660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39592/0 (socket says 172.21.15.140:39592) 2022-07-21T01:56:33.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9765d47700 1 -- 172.21.15.140:0/3729766220 learned_addr learned my addr 172.21.15.140:0/3729766220 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:33.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9765546700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97600b1e00 0x7f97600a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:33.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9766548700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f976009e110 0x7f97600a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:33.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9765d47700 1 -- 172.21.15.140:0/3729766220 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97600b1e00 msgr2=0x7f97600a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9765d47700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97600b1e00 0x7f97600a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9765d47700 1 -- 172.21.15.140:0/3729766220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f976009e110 msgr2=0x7f97600a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:33.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9765d47700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f976009e110 0x7f97600a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:33.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9765d47700 1 -- 172.21.15.140:0/3729766220 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750009470 con 0x7f976005c3f0 2022-07-21T01:56:33.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9766548700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f976009e110 0x7f97600a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:33.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9765546700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97600b1e00 0x7f97600a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:33.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9765d47700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f976005c3f0 0x7f9760144660 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f97540030b0 tx=0x7f9754008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:33.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f975effd700 1 -- 172.21.15.140:0/3729766220 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9754011780 con 0x7f976005c3f0 2022-07-21T01:56:33.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f975effd700 1 -- 172.21.15.140:0/3729766220 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97540118e0 con 0x7f976005c3f0 2022-07-21T01:56:33.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9767fab700 1 -- 172.21.15.140:0/3729766220 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976009ec60 con 0x7f976005c3f0 2022-07-21T01:56:33.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f975effd700 1 -- 172.21.15.140:0/3729766220 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9754011c40 con 0x7f976005c3f0 2022-07-21T01:56:33.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.896+0000 7f9767fab700 1 -- 172.21.15.140:0/3729766220 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97600a1210 con 0x7f976005c3f0 2022-07-21T01:56:33.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.900+0000 7f9767fab700 1 -- 172.21.15.140:0/3729766220 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97600b04b0 con 0x7f976005c3f0 2022-07-21T01:56:33.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.900+0000 7f975effd700 1 -- 172.21.15.140:0/3729766220 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9754012420 con 0x7f976005c3f0 2022-07-21T01:56:33.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.900+0000 7f975effd700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9740053a70 0x7f9740055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:33.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.900+0000 7f9765546700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9740053a70 0x7f9740055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:33.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.900+0000 7f975effd700 1 -- 172.21.15.140:0/3729766220 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f975406bcd0 con 0x7f976005c3f0 2022-07-21T01:56:33.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.900+0000 7f9765546700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9740053a70 0x7f9740055f30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f975001b000 tx=0x7f9750016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:33.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:33.908+0000 7f975effd700 1 -- 172.21.15.140:0/3729766220 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9754042a80 con 0x7f976005c3f0 2022-07-21T01:56:34.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.028+0000 7f9767fab700 1 -- 172.21.15.140:0/3729766220 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97600574e0 con 0x7f976005c3f0 2022-07-21T01:56:34.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.028+0000 7f975effd700 1 -- 172.21.15.140:0/3729766220 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f97540460a0 con 0x7f976005c3f0 2022-07-21T01:56:34.033 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:34.034 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":113,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:30.933163+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:56:27.736358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-07-21T01:56:28.943049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:34.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 -- 172.21.15.140:0/3729766220 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9740053a70 msgr2=0x7f9740055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9740053a70 0x7f9740055f30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f975001b000 tx=0x7f9750016000).stop 2022-07-21T01:56:34.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 -- 172.21.15.140:0/3729766220 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f976005c3f0 msgr2=0x7f9760144660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f976005c3f0 0x7f9760144660 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f97540030b0 tx=0x7f9754008040).stop 2022-07-21T01:56:34.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 -- 172.21.15.140:0/3729766220 shutdown_connections 2022-07-21T01:56:34.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9740053a70 0x7f9740055f30 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f976009e110 0x7f97600a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97600b1e00 0x7f97600a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 --2- 172.21.15.140:0/3729766220 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f976005c3f0 0x7f9760144660 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 -- 172.21.15.140:0/3729766220 >> 172.21.15.140:0/3729766220 conn(0x7f976005bd90 msgr2=0x7f976014b810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:34.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 -- 172.21.15.140:0/3729766220 shutdown_connections 2022-07-21T01:56:34.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.032+0000 7f9767fab700 1 -- 172.21.15.140:0/3729766220 wait complete. 2022-07-21T01:56:34.046 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:56:27.736358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:56:28.943049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:56:34.047 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:56:34.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.152+0000 7f37fec10700 1 Processor -- start 2022-07-21T01:56:34.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.152+0000 7f37fec10700 1 -- start start 2022-07-21T01:56:34.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f37f809df40 0x7f37f8145680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f37f8060ee0 0x7f37f809cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f80607e0 con 0x7f37f80b4d60 2022-07-21T01:56:34.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f8060d20 con 0x7f37f809df40 2022-07-21T01:56:34.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f8060a80 con 0x7f37f8060ee0 2022-07-21T01:56:34.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fc9ac700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f37f809df40 0x7f37f8145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37f7fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f37f80b4d60 0x7f37f8145b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53744/0 (socket says 172.21.15.140:53744) 2022-07-21T01:56:34.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37f7fff700 1 -- 172.21.15.140:0/976417501 learned_addr learned my addr 172.21.15.140:0/976417501 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:34.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fd1ad700 1 --2- 172.21.15.140:0/976417501 >> v2:172.21.15.152:3300/0 conn(0x7f37f8060ee0 0x7f37f809cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37f77fe700 1 -- 172.21.15.140:0/976417501 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413660315 0 0) 0x7f37f80607e0 con 0x7f37f80b4d60 2022-07-21T01:56:34.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37f77fe700 1 -- 172.21.15.140:0/976417501 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e4003260 con 0x7f37f80b4d60 2022-07-21T01:56:34.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fc9ac700 1 -- 172.21.15.140:0/976417501 >> v2:172.21.15.152:3300/0 conn(0x7f37f8060ee0 msgr2=0x7f37f809cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:34.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fc9ac700 1 --2- 172.21.15.140:0/976417501 >> v2:172.21.15.152:3300/0 conn(0x7f37f8060ee0 0x7f37f809cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fc9ac700 1 -- 172.21.15.140:0/976417501 >> v1:172.21.15.140:6789/0 conn(0x7f37f80b4d60 legacy=0x7f37f8145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fc9ac700 1 -- 172.21.15.140:0/976417501 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f809e320 con 0x7f37f809df40 2022-07-21T01:56:34.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fc9ac700 1 --2- 172.21.15.140:0/976417501 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37f809df40 0x7f37f8145680 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f37e8002f70 tx=0x7f37e8006df0).ready entity=mon.0 client_cookie=aa7602dda996e891 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37f77fe700 1 -- 172.21.15.140:0/976417501 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37e800e470 con 0x7f37f809df40 2022-07-21T01:56:34.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37f77fe700 1 -- 172.21.15.140:0/976417501 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e8000e90 con 0x7f37f809df40 2022-07-21T01:56:34.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37f77fe700 1 -- 172.21.15.140:0/976417501 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37e800e7c0 con 0x7f37f809df40 2022-07-21T01:56:34.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 -- 172.21.15.140:0/976417501 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37f809df40 msgr2=0x7f37f8145680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 --2- 172.21.15.140:0/976417501 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37f809df40 0x7f37f8145680 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f37e8002f70 tx=0x7f37e8006df0).stop 2022-07-21T01:56:34.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 -- 172.21.15.140:0/976417501 shutdown_connections 2022-07-21T01:56:34.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 --2- 172.21.15.140:0/976417501 >> v2:172.21.15.152:3300/0 conn(0x7f37f8060ee0 0x7f37f809cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 --2- 172.21.15.140:0/976417501 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37f809df40 0x7f37f8145680 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 -- 172.21.15.140:0/976417501 >> 172.21.15.140:0/976417501 conn(0x7f37f80b5f30 msgr2=0x7f37f80b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:34.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 -- 172.21.15.140:0/976417501 shutdown_connections 2022-07-21T01:56:34.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.156+0000 7f37fec10700 1 -- 172.21.15.140:0/976417501 wait complete. 2022-07-21T01:56:34.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.160+0000 7f37fec10700 1 Processor -- start 2022-07-21T01:56:34.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.160+0000 7f37fec10700 1 -- start start 2022-07-21T01:56:34.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.160+0000 7f37fec10700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37f8060ee0 0x7f37f80b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.160+0000 7f37fec10700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37f809df40 0x7f37f8148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.160+0000 7f37fec10700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37f80b4d60 0x7f37f809b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.160+0000 7f37fec10700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f80b1420 con 0x7f37f8060ee0 2022-07-21T01:56:34.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.160+0000 7f37fec10700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f80b2a20 con 0x7f37f809df40 2022-07-21T01:56:34.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.160+0000 7f37fec10700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f80b24e0 con 0x7f37f80b4d60 2022-07-21T01:56:34.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fc9ac700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37f8060ee0 0x7f37f80b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fc9ac700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37f8060ee0 0x7f37f80b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39604/0 (socket says 172.21.15.140:39604) 2022-07-21T01:56:34.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fc9ac700 1 -- 172.21.15.140:0/323705087 learned_addr learned my addr 172.21.15.140:0/323705087 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:34.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37f7fff700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37f809df40 0x7f37f8148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fd1ad700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37f80b4d60 0x7f37f809b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fc9ac700 1 -- 172.21.15.140:0/323705087 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37f80b4d60 msgr2=0x7f37f809b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fc9ac700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37f80b4d60 0x7f37f809b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fc9ac700 1 -- 172.21.15.140:0/323705087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37f809df40 msgr2=0x7f37f8148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fc9ac700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37f809df40 0x7f37f8148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fc9ac700 1 -- 172.21.15.140:0/323705087 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e800b120 con 0x7f37f8060ee0 2022-07-21T01:56:34.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fc9ac700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37f8060ee0 0x7f37f80b2c10 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f37e8002f70 tx=0x7f37e8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37f5ffb700 1 -- 172.21.15.140:0/323705087 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37e800ee20 con 0x7f37f8060ee0 2022-07-21T01:56:34.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37f5ffb700 1 -- 172.21.15.140:0/323705087 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e8011930 con 0x7f37f8060ee0 2022-07-21T01:56:34.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fec10700 1 -- 172.21.15.140:0/323705087 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f80b2780 con 0x7f37f8060ee0 2022-07-21T01:56:34.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37f5ffb700 1 -- 172.21.15.140:0/323705087 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f37e8011c90 con 0x7f37f8060ee0 2022-07-21T01:56:34.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fec10700 1 -- 172.21.15.140:0/323705087 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f81558a0 con 0x7f37f8060ee0 2022-07-21T01:56:34.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37fec10700 1 -- 172.21.15.140:0/323705087 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f80b19d0 con 0x7f37f8060ee0 2022-07-21T01:56:34.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37f5ffb700 1 -- 172.21.15.140:0/323705087 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f37e800f070 con 0x7f37f8060ee0 2022-07-21T01:56:34.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37f5ffb700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37d8053a70 0x7f37d8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37f5ffb700 1 -- 172.21.15.140:0/323705087 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f37e806d0c0 con 0x7f37f8060ee0 2022-07-21T01:56:34.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.164+0000 7f37f7fff700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37d8053a70 0x7f37d8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.168+0000 7f37f7fff700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37d8053a70 0x7f37d8055f30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f37ec00d000 tx=0x7f37ec009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.172+0000 7f37f5ffb700 1 -- 172.21.15.140:0/323705087 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f37f80b19d0 con 0x7f37f8060ee0 2022-07-21T01:56:34.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.300+0000 7f37fec10700 1 -- 172.21.15.140:0/323705087 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f37f80b19d0 con 0x7f37f8060ee0 2022-07-21T01:56:34.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.300+0000 7f37f5ffb700 1 -- 172.21.15.140:0/323705087 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v113) v1 ==== 110+0+10 (secure 0 0 0) 0x7f37f80b19d0 con 0x7f37f8060ee0 2022-07-21T01:56:34.303 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:56:34.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 -- 172.21.15.140:0/323705087 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37d8053a70 msgr2=0x7f37d8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37d8053a70 0x7f37d8055f30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f37ec00d000 tx=0x7f37ec009040).stop 2022-07-21T01:56:34.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 -- 172.21.15.140:0/323705087 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37f8060ee0 msgr2=0x7f37f80b2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37f8060ee0 0x7f37f80b2c10 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f37e8002f70 tx=0x7f37e8014000).stop 2022-07-21T01:56:34.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 -- 172.21.15.140:0/323705087 shutdown_connections 2022-07-21T01:56:34.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f37d8053a70 0x7f37d8055f30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37f80b4d60 0x7f37f809b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37f809df40 0x7f37f8148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 --2- 172.21.15.140:0/323705087 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f37f8060ee0 0x7f37f80b2c10 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 -- 172.21.15.140:0/323705087 >> 172.21.15.140:0/323705087 conn(0x7f37f80b5f30 msgr2=0x7f37f80b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:34.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 -- 172.21.15.140:0/323705087 shutdown_connections 2022-07-21T01:56:34.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.304+0000 7f37fec10700 1 -- 172.21.15.140:0/323705087 wait complete. 2022-07-21T01:56:34.316 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:56:34.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.420+0000 7fb9e6d0f700 1 Processor -- start 2022-07-21T01:56:34.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.420+0000 7fb9e6d0f700 1 -- start start 2022-07-21T01:56:34.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e6d0f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9e00b59d0 0x7fb9e00b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e6d0f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e00adf70 con 0x7fb9e005d0c0 2022-07-21T01:56:34.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e6d0f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e00b5db0 con 0x7fb9e00af920 2022-07-21T01:56:34.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e6d0f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e0146300 con 0x7fb9e00b59d0 2022-07-21T01:56:34.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e650e700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fb9e005d0c0 0x7fb9e0141900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53758/0 (socket says 172.21.15.140:53758) 2022-07-21T01:56:34.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e650e700 1 -- 172.21.15.140:0/1263572329 learned_addr learned my addr 172.21.15.140:0/1263572329 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:34.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e5d0d700 1 --2- 172.21.15.140:0/1263572329 >> v2:172.21.15.152:3300/0 conn(0x7fb9e00b59d0 0x7fb9e00b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787623520 0 0) 0x7fb9e00adf70 con 0x7fb9e005d0c0 2022-07-21T01:56:34.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c0003260 con 0x7fb9e005d0c0 2022-07-21T01:56:34.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3616415749 0 0) 0x7fb9c0003260 con 0x7fb9e005d0c0 2022-07-21T01:56:34.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 >> v2:172.21.15.152:3300/0 conn(0x7fb9e00b59d0 msgr2=0x7fb9e00b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:34.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 --2- 172.21.15.140:0/1263572329 >> v2:172.21.15.152:3300/0 conn(0x7fb9e00b59d0 0x7fb9e00b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 >> v1:172.21.15.152:6789/0 conn(0x7fb9e00af920 legacy=0x7fb9e00ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e0146060 con 0x7fb9e005d0c0 2022-07-21T01:56:34.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fb9d8001660 con 0x7fb9e005d0c0 2022-07-21T01:56:34.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 >> v1:172.21.15.140:6789/0 conn(0x7fb9e005d0c0 legacy=0x7fb9e0141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9c0003750 0x7fb9c0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9c0004500 0x7fb9c0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9c0004e70 0x7fb9c000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0146300 con 0x7fb9c0003750 2022-07-21T01:56:34.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c000bd20 con 0x7fb9c0004e70 2022-07-21T01:56:34.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c000bee0 con 0x7fb9c0004500 2022-07-21T01:56:34.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e5d0d700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9c0003750 0x7fb9c0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e650e700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9c0004e70 0x7fb9c000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e550c700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9c0004500 0x7fb9c0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e650e700 1 -- 172.21.15.140:0/1263572329 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9c0004500 msgr2=0x7fb9c0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:34.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e650e700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9c0004500 0x7fb9c0004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e650e700 1 -- 172.21.15.140:0/1263572329 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9c0003750 msgr2=0x7fb9c0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:34.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e650e700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9c0003750 0x7fb9c0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e650e700 1 -- 172.21.15.140:0/1263572329 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c0003260 con 0x7fb9c0004e70 2022-07-21T01:56:34.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e550c700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9c0004500 0x7fb9c0004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:34.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e650e700 1 -- 172.21.15.140:0/1263572329 reap_dead start 2022-07-21T01:56:34.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e650e700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9c0004e70 0x7fb9c000b750 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fb9d8001370 tx=0x7fb9e01461f0).ready entity=mon.1 client_cookie=250c45dcfd4cf22f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d80070d0 con 0x7fb9c0004e70 2022-07-21T01:56:34.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d8007230 con 0x7fb9c0004e70 2022-07-21T01:56:34.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.424+0000 7fb9e4d0b700 1 -- 172.21.15.140:0/1263572329 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d800c8f0 con 0x7fb9c0004e70 2022-07-21T01:56:34.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.428+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/1263572329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9c0004e70 msgr2=0x7fb9c000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.428+0000 7fb9e6d0f700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9c0004e70 0x7fb9c000b750 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fb9d8001370 tx=0x7fb9e01461f0).stop 2022-07-21T01:56:34.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.428+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/1263572329 shutdown_connections 2022-07-21T01:56:34.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.428+0000 7fb9e6d0f700 1 --2- 172.21.15.140:0/1263572329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9c0004e70 0x7fb9c000b750 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.428+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/1263572329 >> 172.21.15.140:0/1263572329 conn(0x7fb9e00634c0 msgr2=0x7fb9e00b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:34.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.428+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/1263572329 shutdown_connections 2022-07-21T01:56:34.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.428+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/1263572329 wait complete. 2022-07-21T01:56:34.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.428+0000 7fb9e6d0f700 1 Processor -- start 2022-07-21T01:56:34.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.428+0000 7fb9e6d0f700 1 -- start start 2022-07-21T01:56:34.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e6d0f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9c0004e70 0x7fb9e0142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e6d0f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9e00b3cf0 0x7fb9e01556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e6d0f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9e01551f0 0x7fb9e0155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e6d0f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e013cc10 con 0x7fb9e01551f0 2022-07-21T01:56:34.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e6d0f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0151c40 con 0x7fb9c0004e70 2022-07-21T01:56:34.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e6d0f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00649a0 con 0x7fb9e00b3cf0 2022-07-21T01:56:34.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e650e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9e01551f0 0x7fb9e0155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e650e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9e01551f0 0x7fb9e0155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39626/0 (socket says 172.21.15.140:39626) 2022-07-21T01:56:34.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e650e700 1 -- 172.21.15.140:0/2560442654 learned_addr learned my addr 172.21.15.140:0/2560442654 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:34.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e550c700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9e00b3cf0 0x7fb9e01556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e5d0d700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9c0004e70 0x7fb9e0142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e650e700 1 -- 172.21.15.140:0/2560442654 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9e00b3cf0 msgr2=0x7fb9e01556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e650e700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9e00b3cf0 0x7fb9e01556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e650e700 1 -- 172.21.15.140:0/2560442654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9c0004e70 msgr2=0x7fb9e0142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e650e700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9c0004e70 0x7fb9e0142cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e650e700 1 -- 172.21.15.140:0/2560442654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e0064c00 con 0x7fb9e01551f0 2022-07-21T01:56:34.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e650e700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9e01551f0 0x7fb9e0155be0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7fb9d8001370 tx=0x7fb9d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9deffd700 1 -- 172.21.15.140:0/2560442654 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d80070d0 con 0x7fb9e01551f0 2022-07-21T01:56:34.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9deffd700 1 -- 172.21.15.140:0/2560442654 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d8004710 con 0x7fb9e01551f0 2022-07-21T01:56:34.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/2560442654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e01560d0 con 0x7fb9e01551f0 2022-07-21T01:56:34.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9deffd700 1 -- 172.21.15.140:0/2560442654 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9d8004a70 con 0x7fb9e01551f0 2022-07-21T01:56:34.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/2560442654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e0143900 con 0x7fb9e01551f0 2022-07-21T01:56:34.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/2560442654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e0151db0 con 0x7fb9e01551f0 2022-07-21T01:56:34.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.432+0000 7fb9deffd700 1 -- 172.21.15.140:0/2560442654 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb9d801d070 con 0x7fb9e01551f0 2022-07-21T01:56:34.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.436+0000 7fb9deffd700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb9c4053a10 0x7fb9c4055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.436+0000 7fb9deffd700 1 -- 172.21.15.140:0/2560442654 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb9d806c780 con 0x7fb9e01551f0 2022-07-21T01:56:34.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.436+0000 7fb9e5d0d700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb9c4053a10 0x7fb9c4055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.436+0000 7fb9e5d0d700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb9c4053a10 0x7fb9c4055ed0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fb9d000b000 tx=0x7fb9d0006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.440+0000 7fb9deffd700 1 -- 172.21.15.140:0/2560442654 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb9e0151db0 con 0x7fb9e01551f0 2022-07-21T01:56:34.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.564+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/2560442654 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb9e0151db0 con 0x7fb9e01551f0 2022-07-21T01:56:34.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9deffd700 1 -- 172.21.15.140:0/2560442654 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb9e0151db0 con 0x7fb9e01551f0 2022-07-21T01:56:34.569 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:34.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/2560442654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb9c4053a10 msgr2=0x7fb9c4055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb9c4053a10 0x7fb9c4055ed0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fb9d000b000 tx=0x7fb9d0006df0).stop 2022-07-21T01:56:34.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/2560442654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9e01551f0 msgr2=0x7fb9e0155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9e01551f0 0x7fb9e0155be0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7fb9d8001370 tx=0x7fb9d8015000).stop 2022-07-21T01:56:34.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/2560442654 shutdown_connections 2022-07-21T01:56:34.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb9c4053a10 0x7fb9c4055ed0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9e01551f0 0x7fb9e0155be0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9e00b3cf0 0x7fb9e01556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 --2- 172.21.15.140:0/2560442654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9c0004e70 0x7fb9e0142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/2560442654 >> 172.21.15.140:0/2560442654 conn(0x7fb9e00634c0 msgr2=0x7fb9e0147230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:34.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/2560442654 shutdown_connections 2022-07-21T01:56:34.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.568+0000 7fb9e6d0f700 1 -- 172.21.15.140:0/2560442654 wait complete. 2022-07-21T01:56:34.582 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:56:34.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.676+0000 7f364bfff700 1 Processor -- start 2022-07-21T01:56:34.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.676+0000 7f364bfff700 1 -- start start 2022-07-21T01:56:34.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f364c0b3cf0 0x7f364c142a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364bfff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c0ba180 con 0x7f364c0b9da0 2022-07-21T01:56:34.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364bfff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f364c1461b0 con 0x7f364c142f40 2022-07-21T01:56:34.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364bfff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364c0ad5b0 con 0x7f364c0b3cf0 2022-07-21T01:56:34.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364a7fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f364c0b3cf0 0x7f364c142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364affd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f364c0b9da0 0x7f364c1468f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53778/0 (socket says 172.21.15.140:53778) 2022-07-21T01:56:34.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364affd700 1 -- 172.21.15.140:0/2209191607 learned_addr learned my addr 172.21.15.140:0/2209191607 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:34.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364a7fc700 1 -- 172.21.15.140:0/2209191607 >> v1:172.21.15.152:6789/0 conn(0x7f364c142f40 legacy=0x7f364c1419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:34.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364a7fc700 1 -- 172.21.15.140:0/2209191607 >> v1:172.21.15.140:6789/0 conn(0x7f364c0b9da0 legacy=0x7f364c1468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f3649ffb700 1 -- 172.21.15.140:0/2209191607 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841835515 0 0) 0x7f364c0ba180 con 0x7f364c0b9da0 2022-07-21T01:56:34.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364a7fc700 1 -- 172.21.15.140:0/2209191607 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c146450 con 0x7f364c0b3cf0 2022-07-21T01:56:34.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364a7fc700 1 --2- 172.21.15.140:0/2209191607 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f364c0b3cf0 0x7f364c142a50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f3634002f70 tx=0x7f3634006df0).ready entity=mon.0 client_cookie=4f7415415e700188 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f3649ffb700 1 -- 172.21.15.140:0/2209191607 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f363400d610 con 0x7f364c0b3cf0 2022-07-21T01:56:34.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f3649ffb700 1 -- 172.21.15.140:0/2209191607 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3634000e90 con 0x7f364c0b3cf0 2022-07-21T01:56:34.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f3649ffb700 1 -- 172.21.15.140:0/2209191607 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f363400d960 con 0x7f364c0b3cf0 2022-07-21T01:56:34.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364bfff700 1 -- 172.21.15.140:0/2209191607 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f364c0b3cf0 msgr2=0x7f364c142a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364bfff700 1 --2- 172.21.15.140:0/2209191607 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f364c0b3cf0 0x7f364c142a50 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f3634002f70 tx=0x7f3634006df0).stop 2022-07-21T01:56:34.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364bfff700 1 -- 172.21.15.140:0/2209191607 shutdown_connections 2022-07-21T01:56:34.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364bfff700 1 --2- 172.21.15.140:0/2209191607 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f364c0b3cf0 0x7f364c142a50 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364bfff700 1 -- 172.21.15.140:0/2209191607 >> 172.21.15.140:0/2209191607 conn(0x7f364c0678c0 msgr2=0x7f364c0b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:34.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364bfff700 1 -- 172.21.15.140:0/2209191607 shutdown_connections 2022-07-21T01:56:34.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.680+0000 7f364bfff700 1 -- 172.21.15.140:0/2209191607 wait complete. 2022-07-21T01:56:34.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364bfff700 1 Processor -- start 2022-07-21T01:56:34.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364bfff700 1 -- start start 2022-07-21T01:56:34.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f364c0b3cf0 0x7f364c0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f364c0b9da0 0x7f364c149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f364c142f40 0x7f364c0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364bfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c113920 con 0x7f364c142f40 2022-07-21T01:56:34.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364bfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0afd60 con 0x7f364c0b3cf0 2022-07-21T01:56:34.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364c0b1a80 con 0x7f364c0b9da0 2022-07-21T01:56:34.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f364c142f40 0x7f364c0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364affd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f364c0b3cf0 0x7f364c0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f364c142f40 0x7f364c0aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39640/0 (socket says 172.21.15.140:39640) 2022-07-21T01:56:34.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364a7fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f364c0b9da0 0x7f364c149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364b7fe700 1 -- 172.21.15.140:0/1425324394 learned_addr learned my addr 172.21.15.140:0/1425324394 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:34.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364b7fe700 1 -- 172.21.15.140:0/1425324394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f364c0b9da0 msgr2=0x7f364c149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364b7fe700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f364c0b9da0 0x7f364c149810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364b7fe700 1 -- 172.21.15.140:0/1425324394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f364c0b3cf0 msgr2=0x7f364c0b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364b7fe700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f364c0b3cf0 0x7f364c0b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364b7fe700 1 -- 172.21.15.140:0/1425324394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363400b180 con 0x7f364c142f40 2022-07-21T01:56:34.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364b7fe700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f364c142f40 0x7f364c0aea90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7f364c1467d0 tx=0x7f363c007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f3643fff700 1 -- 172.21.15.140:0/1425324394 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f363c007030 con 0x7f364c142f40 2022-07-21T01:56:34.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f3643fff700 1 -- 172.21.15.140:0/1425324394 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c007190 con 0x7f364c142f40 2022-07-21T01:56:34.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364bfff700 1 -- 172.21.15.140:0/1425324394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c0b1d20 con 0x7f364c142f40 2022-07-21T01:56:34.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f3643fff700 1 -- 172.21.15.140:0/1425324394 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f363c006510 con 0x7f364c142f40 2022-07-21T01:56:34.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.684+0000 7f364bfff700 1 -- 172.21.15.140:0/1425324394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c113620 con 0x7f364c142f40 2022-07-21T01:56:34.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.688+0000 7f364bfff700 1 -- 172.21.15.140:0/1425324394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c0b1eb0 con 0x7f364c142f40 2022-07-21T01:56:34.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.688+0000 7f3643fff700 1 -- 172.21.15.140:0/1425324394 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f363c010030 con 0x7f364c142f40 2022-07-21T01:56:34.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.688+0000 7f3643fff700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36300539e0 0x7f3630055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.688+0000 7f364affd700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36300539e0 0x7f3630055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.688+0000 7f3643fff700 1 -- 172.21.15.140:0/1425324394 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f363c06ba50 con 0x7f364c142f40 2022-07-21T01:56:34.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.688+0000 7f364affd700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36300539e0 0x7f3630055ea0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f364400d000 tx=0x7f3644001520).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.692+0000 7f3643fff700 1 -- 172.21.15.140:0/1425324394 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f363c042970 con 0x7f364c142f40 2022-07-21T01:56:34.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.816+0000 7f364bfff700 1 -- 172.21.15.140:0/1425324394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f364c0574e0 con 0x7f364c142f40 2022-07-21T01:56:34.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.816+0000 7f3643fff700 1 -- 172.21.15.140:0/1425324394 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+10 (secure 0 0 0) 0x7f363c045f90 con 0x7f364c142f40 2022-07-21T01:56:34.820 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:34.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 -- 172.21.15.140:0/1425324394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36300539e0 msgr2=0x7f3630055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36300539e0 0x7f3630055ea0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f364400d000 tx=0x7f3644001520).stop 2022-07-21T01:56:34.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 -- 172.21.15.140:0/1425324394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f364c142f40 msgr2=0x7f364c0aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f364c142f40 0x7f364c0aea90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7f364c1467d0 tx=0x7f363c007c20).stop 2022-07-21T01:56:34.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 -- 172.21.15.140:0/1425324394 shutdown_connections 2022-07-21T01:56:34.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36300539e0 0x7f3630055ea0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f364c142f40 0x7f364c0aea90 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f364c0b9da0 0x7f364c149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 --2- 172.21.15.140:0/1425324394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f364c0b3cf0 0x7f364c0b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 -- 172.21.15.140:0/1425324394 >> 172.21.15.140:0/1425324394 conn(0x7f364c0678c0 msgr2=0x7f364c0b9330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:34.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 -- 172.21.15.140:0/1425324394 shutdown_connections 2022-07-21T01:56:34.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.820+0000 7f364bfff700 1 -- 172.21.15.140:0/1425324394 wait complete. 2022-07-21T01:56:34.833 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:56:34.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.932+0000 7f2ef36b1700 1 Processor -- start 2022-07-21T01:56:34.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 -- start start 2022-07-21T01:56:34.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2eec0b1220 0x7f2eec0b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eec1472b0 con 0x7f2eec05cf40 2022-07-21T01:56:34.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eec147550 con 0x7f2eec147c90 2022-07-21T01:56:34.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0b0920 con 0x7f2eec0b1220 2022-07-21T01:56:34.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef144d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f2eec05cf40 0x7f2eec0afd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53792/0 (socket says 172.21.15.140:53792) 2022-07-21T01:56:34.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef144d700 1 -- 172.21.15.140:0/1886421479 learned_addr learned my addr 172.21.15.140:0/1886421479 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:34.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef1c4e700 1 --2- 172.21.15.140:0/1886421479 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2eec0b1220 0x7f2eec0b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ee3fff700 1 -- 172.21.15.140:0/1886421479 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503788911 0 0) 0x7f2eec1472b0 con 0x7f2eec05cf40 2022-07-21T01:56:34.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ee3fff700 1 -- 172.21.15.140:0/1886421479 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ed0003260 con 0x7f2eec05cf40 2022-07-21T01:56:34.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ee3fff700 1 -- 172.21.15.140:0/1886421479 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561837528 0 0) 0x7f2eec147550 con 0x7f2eec147c90 2022-07-21T01:56:34.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ee3fff700 1 -- 172.21.15.140:0/1886421479 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eec1472b0 con 0x7f2eec147c90 2022-07-21T01:56:34.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef1c4e700 1 -- 172.21.15.140:0/1886421479 >> v1:172.21.15.152:6789/0 conn(0x7f2eec147c90 legacy=0x7f2eec14a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ee3fff700 1 -- 172.21.15.140:0/1886421479 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1588203834 0 0) 0x7f2ed0003260 con 0x7f2eec05cf40 2022-07-21T01:56:34.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef1c4e700 1 -- 172.21.15.140:0/1886421479 >> v1:172.21.15.140:6789/0 conn(0x7f2eec05cf40 legacy=0x7f2eec0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef1c4e700 1 -- 172.21.15.140:0/1886421479 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec1477f0 con 0x7f2eec0b1220 2022-07-21T01:56:34.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef1c4e700 1 --2- 172.21.15.140:0/1886421479 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2eec0b1220 0x7f2eec0b0430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f2edc001110 tx=0x7f2ee40091d0).ready entity=mon.2 client_cookie=63750c5e252b5b6f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ee3fff700 1 -- 172.21.15.140:0/1886421479 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2edc001660 con 0x7f2eec0b1220 2022-07-21T01:56:34.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ee3fff700 1 -- 172.21.15.140:0/1886421479 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee400f510 con 0x7f2eec0b1220 2022-07-21T01:56:34.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ee3fff700 1 -- 172.21.15.140:0/1886421479 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ee400f680 con 0x7f2eec0b1220 2022-07-21T01:56:34.958 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 -- 172.21.15.140:0/1886421479 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2eec0b1220 msgr2=0x7f2eec0b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 --2- 172.21.15.140:0/1886421479 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2eec0b1220 0x7f2eec0b0430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f2edc001110 tx=0x7f2ee40091d0).stop 2022-07-21T01:56:34.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 -- 172.21.15.140:0/1886421479 shutdown_connections 2022-07-21T01:56:34.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 --2- 172.21.15.140:0/1886421479 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2eec0b1220 0x7f2eec0b0430 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 -- 172.21.15.140:0/1886421479 >> 172.21.15.140:0/1886421479 conn(0x7f2eec0b5b70 msgr2=0x7f2eec0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:34.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 -- 172.21.15.140:0/1886421479 shutdown_connections 2022-07-21T01:56:34.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.936+0000 7f2ef36b1700 1 -- 172.21.15.140:0/1886421479 wait complete. 2022-07-21T01:56:34.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef36b1700 1 Processor -- start 2022-07-21T01:56:34.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef36b1700 1 -- start start 2022-07-21T01:56:34.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef36b1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2eec05cf40 0x7f2eec0b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef36b1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2eec0b1220 0x7f2eec13f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef36b1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2eec147c90 0x7f2eec13f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef36b1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0ae2a0 con 0x7f2eec147c90 2022-07-21T01:56:34.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef36b1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0ad8c0 con 0x7f2eec0b1220 2022-07-21T01:56:34.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef36b1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0adb60 con 0x7f2eec05cf40 2022-07-21T01:56:34.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef1c4e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2eec147c90 0x7f2eec13f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef1c4e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2eec147c90 0x7f2eec13f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39654/0 (socket says 172.21.15.140:39654) 2022-07-21T01:56:34.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef1c4e700 1 -- 172.21.15.140:0/217639467 learned_addr learned my addr 172.21.15.140:0/217639467 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:34.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef144d700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2eec05cf40 0x7f2eec0b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef0c4c700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2eec0b1220 0x7f2eec13f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef1c4e700 1 -- 172.21.15.140:0/217639467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2eec05cf40 msgr2=0x7f2eec0b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef1c4e700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2eec05cf40 0x7f2eec0b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef1c4e700 1 -- 172.21.15.140:0/217639467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2eec0b1220 msgr2=0x7f2eec13f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:34.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef1c4e700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2eec0b1220 0x7f2eec13f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:34.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef1c4e700 1 -- 172.21.15.140:0/217639467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee4009470 con 0x7f2eec147c90 2022-07-21T01:56:34.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef0c4c700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2eec0b1220 0x7f2eec13f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:34.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef1c4e700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2eec147c90 0x7f2eec13f840 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f2edc001110 tx=0x7f2ee4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.971 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ee27fc700 1 -- 172.21.15.140:0/217639467 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ee40032f0 con 0x7f2eec147c90 2022-07-21T01:56:34.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ee27fc700 1 -- 172.21.15.140:0/217639467 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee4011c30 con 0x7f2eec147c90 2022-07-21T01:56:34.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ef36b1700 1 -- 172.21.15.140:0/217639467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eec0ade00 con 0x7f2eec147c90 2022-07-21T01:56:34.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.940+0000 7f2ee27fc700 1 -- 172.21.15.140:0/217639467 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ee4021440 con 0x7f2eec147c90 2022-07-21T01:56:34.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.944+0000 7f2ef36b1700 1 -- 172.21.15.140:0/217639467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eec13fd30 con 0x7f2eec147c90 2022-07-21T01:56:34.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.944+0000 7f2ef36b1700 1 -- 172.21.15.140:0/217639467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eec146fa0 con 0x7f2eec147c90 2022-07-21T01:56:34.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.944+0000 7f2ee27fc700 1 -- 172.21.15.140:0/217639467 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2ee4027020 con 0x7f2eec147c90 2022-07-21T01:56:34.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.944+0000 7f2ee27fc700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2ed4053950 0x7f2ed4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:34.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.944+0000 7f2ee27fc700 1 -- 172.21.15.140:0/217639467 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2ee406d150 con 0x7f2eec147c90 2022-07-21T01:56:34.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.944+0000 7f2ef144d700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2ed4053950 0x7f2ed4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:34.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.944+0000 7f2ef144d700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2ed4053950 0x7f2ed4055e10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f2edc00d000 tx=0x7f2edc000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:34.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:34.952+0000 7f2ee27fc700 1 -- 172.21.15.140:0/217639467 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2ee4044070 con 0x7f2eec147c90 2022-07-21T01:56:35.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.076+0000 7f2ef36b1700 1 -- 172.21.15.140:0/217639467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2eec069a60 con 0x7f2eec147c90 2022-07-21T01:56:35.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.076+0000 7f2ee27fc700 1 -- 172.21.15.140:0/217639467 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2eec069a60 con 0x7f2eec147c90 2022-07-21T01:56:35.080 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:35.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 -- 172.21.15.140:0/217639467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2ed4053950 msgr2=0x7f2ed4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:35.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2ed4053950 0x7f2ed4055e10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f2edc00d000 tx=0x7f2edc000bc0).stop 2022-07-21T01:56:35.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 -- 172.21.15.140:0/217639467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2eec147c90 msgr2=0x7f2eec13f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:35.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2eec147c90 0x7f2eec13f840 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f2edc001110 tx=0x7f2ee4016000).stop 2022-07-21T01:56:35.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 -- 172.21.15.140:0/217639467 shutdown_connections 2022-07-21T01:56:35.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2ed4053950 0x7f2ed4055e10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2eec147c90 0x7f2eec13f840 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2eec0b1220 0x7f2eec13f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 --2- 172.21.15.140:0/217639467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2eec05cf40 0x7f2eec0b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 -- 172.21.15.140:0/217639467 >> 172.21.15.140:0/217639467 conn(0x7f2eec0b5b70 msgr2=0x7f2eec14ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:35.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 -- 172.21.15.140:0/217639467 shutdown_connections 2022-07-21T01:56:35.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.080+0000 7f2ef36b1700 1 -- 172.21.15.140:0/217639467 wait complete. 2022-07-21T01:56:35.097 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:56:35.097 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:35.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1ca998700 1 Processor -- start 2022-07-21T01:56:35.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1ca998700 1 -- start start 2022-07-21T01:56:35.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1ca998700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1c40b1e00 0x7fe1c414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:35.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1ca998700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe1c40acef0 0x7fe1c4146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:35.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1ca998700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c405d480 con 0x7fe1c405c3f0 2022-07-21T01:56:35.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1ca998700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c40b21e0 con 0x7fe1c40acef0 2022-07-21T01:56:35.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1ca998700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c405cf40 con 0x7fe1c40b1e00 2022-07-21T01:56:35.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c8f35700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe1c40acef0 0x7fe1c4146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:35.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c3fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1c40b1e00 0x7fe1c414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:35.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c8f35700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe1c40acef0 0x7fe1c4146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:45404/0 (socket says 172.21.15.140:45404) 2022-07-21T01:56:35.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c8f35700 1 -- 172.21.15.140:0/2765296869 learned_addr learned my addr 172.21.15.140:0/2765296869 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:35.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c8f35700 1 -- 172.21.15.140:0/2765296869 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1c40b1e00 msgr2=0x7fe1c414a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:35.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c8f35700 1 --2- 172.21.15.140:0/2765296869 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1c40b1e00 0x7fe1c414a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c8f35700 1 -- 172.21.15.140:0/2765296869 >> v1:172.21.15.152:6789/0 conn(0x7fe1c405c3f0 legacy=0x7fe1c40ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:35.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c8f35700 1 -- 172.21.15.140:0/2765296869 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c405d1e0 con 0x7fe1c40acef0 2022-07-21T01:56:35.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c8f35700 1 --2- 172.21.15.140:0/2765296869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1c40acef0 0x7fe1c4146410 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fe1bc002f70 tx=0x7fe1bc006eb0).ready entity=mon.1 client_cookie=4fc774be2066515a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:35.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c2ffd700 1 -- 172.21.15.140:0/2765296869 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1bc00d570 con 0x7fe1c40acef0 2022-07-21T01:56:35.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c2ffd700 1 -- 172.21.15.140:0/2765296869 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc000e90 con 0x7fe1c40acef0 2022-07-21T01:56:35.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.196+0000 7fe1c2ffd700 1 -- 172.21.15.140:0/2765296869 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1bc00d900 con 0x7fe1c40acef0 2022-07-21T01:56:35.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 -- 172.21.15.140:0/2765296869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1c40acef0 msgr2=0x7fe1c4146410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:35.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 --2- 172.21.15.140:0/2765296869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1c40acef0 0x7fe1c4146410 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fe1bc002f70 tx=0x7fe1bc006eb0).stop 2022-07-21T01:56:35.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 -- 172.21.15.140:0/2765296869 shutdown_connections 2022-07-21T01:56:35.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 --2- 172.21.15.140:0/2765296869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1c40acef0 0x7fe1c4146410 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 --2- 172.21.15.140:0/2765296869 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1c40b1e00 0x7fe1c414a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 -- 172.21.15.140:0/2765296869 >> 172.21.15.140:0/2765296869 conn(0x7fe1c40611c0 msgr2=0x7fe1c4062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:35.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 -- 172.21.15.140:0/2765296869 shutdown_connections 2022-07-21T01:56:35.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 -- 172.21.15.140:0/2765296869 wait complete. 2022-07-21T01:56:35.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 Processor -- start 2022-07-21T01:56:35.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 -- start start 2022-07-21T01:56:35.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1c40acef0 0x7fe1c4156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:35.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1c40b1e00 0x7fe1c41485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:35.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1c4148ac0 0x7fe1c415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:35.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c409eab0 con 0x7fe1c40acef0 2022-07-21T01:56:35.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4159e20 con 0x7fe1c40b1e00 2022-07-21T01:56:35.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1ca998700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40a2010 con 0x7fe1c4148ac0 2022-07-21T01:56:35.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1c3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1c40acef0 0x7fe1c4156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:35.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1c37fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1c40b1e00 0x7fe1c41485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:35.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1c3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1c40acef0 0x7fe1c4156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39666/0 (socket says 172.21.15.140:39666) 2022-07-21T01:56:35.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1c3fff700 1 -- 172.21.15.140:0/760215759 learned_addr learned my addr 172.21.15.140:0/760215759 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:35.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1c8f35700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1c4148ac0 0x7fe1c415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:35.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1c3fff700 1 -- 172.21.15.140:0/760215759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1c4148ac0 msgr2=0x7fe1c415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:35.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1c3fff700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1c4148ac0 0x7fe1c415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1c3fff700 1 -- 172.21.15.140:0/760215759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1c40b1e00 msgr2=0x7fe1c41485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:35.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1c3fff700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1c40b1e00 0x7fe1c41485d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.200+0000 7fe1c3fff700 1 -- 172.21.15.140:0/760215759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc00b1e0 con 0x7fe1c40acef0 2022-07-21T01:56:35.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.204+0000 7fe1c3fff700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1c40acef0 0x7fe1c4156690 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7fe1ac00c000 tx=0x7fe1ac006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:35.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.204+0000 7fe1c17fa700 1 -- 172.21.15.140:0/760215759 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1ac011750 con 0x7fe1c40acef0 2022-07-21T01:56:35.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.204+0000 7fe1c17fa700 1 -- 172.21.15.140:0/760215759 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac0118b0 con 0x7fe1c40acef0 2022-07-21T01:56:35.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.204+0000 7fe1ca998700 1 -- 172.21.15.140:0/760215759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c4156b80 con 0x7fe1c40acef0 2022-07-21T01:56:35.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.204+0000 7fe1c17fa700 1 -- 172.21.15.140:0/760215759 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe1ac011c10 con 0x7fe1c40acef0 2022-07-21T01:56:35.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.204+0000 7fe1ca998700 1 -- 172.21.15.140:0/760215759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c409e6d0 con 0x7fe1c40acef0 2022-07-21T01:56:35.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.204+0000 7fe1ca998700 1 -- 172.21.15.140:0/760215759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c409e920 con 0x7fe1c40acef0 2022-07-21T01:56:35.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.204+0000 7fe1c17fa700 1 -- 172.21.15.140:0/760215759 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe1ac011d70 con 0x7fe1c40acef0 2022-07-21T01:56:35.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.204+0000 7fe1c17fa700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe1a8053aa0 0x7fe1a8055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:35.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.204+0000 7fe1c17fa700 1 -- 172.21.15.140:0/760215759 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe1ac06c2b0 con 0x7fe1c40acef0 2022-07-21T01:56:35.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.204+0000 7fe1c37fe700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe1a8053aa0 0x7fe1a8055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:35.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.208+0000 7fe1c37fe700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe1a8053aa0 0x7fe1a8055f60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fe1b400d000 tx=0x7fe1b4000f40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:35.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.212+0000 7fe1c17fa700 1 -- 172.21.15.140:0/760215759 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1ac043170 con 0x7fe1c40acef0 2022-07-21T01:56:35.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.372+0000 7fe1ca998700 1 -- 172.21.15.140:0/760215759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1c409ecb0 con 0x7fe1c40acef0 2022-07-21T01:56:35.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.372+0000 7fe1c17fa700 1 -- 172.21.15.140:0/760215759 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fe1ac046790 con 0x7fe1c40acef0 2022-07-21T01:56:35.376 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:35.377 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":53,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/223182184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":223182184},{"type":"v1","addr":"172.21.15.140:6835","nonce":223182184}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:56:29.930618+0000","modified":"2022-07-21T01:56:31.631480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7662},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7662":{"gid":7662,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/456689269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":456689269},{"type":"v1","addr":"172.21.15.152:6833","nonce":456689269}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-07-21T01:56:35.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 -- 172.21.15.140:0/760215759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe1a8053aa0 msgr2=0x7fe1a8055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:35.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe1a8053aa0 0x7fe1a8055f60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fe1b400d000 tx=0x7fe1b4000f40).stop 2022-07-21T01:56:35.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 -- 172.21.15.140:0/760215759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1c40acef0 msgr2=0x7fe1c4156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:35.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1c40acef0 0x7fe1c4156690 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7fe1ac00c000 tx=0x7fe1ac006df0).stop 2022-07-21T01:56:35.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 -- 172.21.15.140:0/760215759 shutdown_connections 2022-07-21T01:56:35.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe1a8053aa0 0x7fe1a8055f60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe1c4148ac0 0x7fe1c415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1c40b1e00 0x7fe1c41485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 --2- 172.21.15.140:0/760215759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe1c40acef0 0x7fe1c4156690 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:35.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 -- 172.21.15.140:0/760215759 >> 172.21.15.140:0/760215759 conn(0x7fe1c405c3f0 msgr2=0x7fe1c41454e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:35.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 -- 172.21.15.140:0/760215759 shutdown_connections 2022-07-21T01:56:35.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:35.376+0000 7fe1ca998700 1 -- 172.21.15.140:0/760215759 wait complete. 2022-07-21T01:56:35.381 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 53 2022-07-21T01:56:35.590 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:56:29.930618+0000', 'modified': '2022-07-21T01:56:31.631480+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7662}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7662': {'gid': 7662, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/456689269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 456689269}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 456689269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:56:35.591 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:56:35.591 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:56:35.705 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":7662,"want_state":"up:active","state":"up:active","rank_uptime":5.7626918180000004,"mdsmap_epoch":53,"osdmap_epoch":113,"osdmap_epoch_barrier":113,"uptime":10.785951990999999} 2022-07-21T01:56:35.705 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 7662, "mdsmap_epoch": 53, "osdmap_epoch": 113, "osdmap_epoch_barrier": 113, "rank_uptime": 5.762691818, "state": "up:active", "uptime": 10.785951991, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:56:35.706 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:56:35.707 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:56:35.708 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:56:35.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:35.709 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:35.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:56:35.736 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:56:35.736 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:56:35.752 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:56:35.753 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:56:35.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:35.755 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:56:35.803 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:56:35.804 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:35.806 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:56:35.825 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:56:35.825 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:56:35.836 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:56:35.837 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:56:35.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:35.839 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:56:35.883 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:56:35.874+0000 7f784e582080 -1 init, newargv = 0x5580475d7310 newargc=9 2022-07-21T01:56:35.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23211]: starting ceph client 2022-07-21T01:56:35.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23211]: starting fuse 2022-07-21T01:56:36.991 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:56:37.010 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:56:37.011 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:56:37.068 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:56:37.069 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:56:37.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:37.070 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:56:37.076 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:56:37.076 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:56:37.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:37.078 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:56:37.079 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:56:37.079 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:56:37.080 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:56:37.080 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:56:37.080 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:56:37.081 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:56:37.081 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:56:37.081 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:56:37.082 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:56:37.082 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:56:37.083 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:56:37.083 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:56:37.084 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:56:37.084 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:56:37.084 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:56:37.085 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:56:37.085 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:56:37.085 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:56:37.086 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:56:37.086 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:56:37.087 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:56:37.087 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:56:37.132 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.23211.asok 2022-07-21T01:56:37.137 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23211.asok 2022-07-21T01:56:37.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:37.138 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23211.asok status 2022-07-21T01:56:37.226 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:56:37.227 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:56:37.227 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:56:37.227 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:56:37.227 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:56:37.227 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:56:37.228 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:56:37.228 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "23211", 2022-07-21T01:56:37.228 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:56:37.228 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:56:37.229 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:56:37.229 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:56:37.229 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 6824, 2022-07-21T01:56:37.229 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:56:37.229 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:56:37.230 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:56:37.230 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 6824 2022-07-21T01:56:37.230 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:56:37.230 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:56:37.231 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:56:37.231 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:56:37.231 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 629132972 2022-07-21T01:56:37.231 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:56:37.231 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:56:37.232 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:56:37.232 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:56:37.232 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:56:37.232 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 629132972 2022-07-21T01:56:37.233 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:56:37.233 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.6824 172.21.15.191:0/629132972", 2022-07-21T01:56:37.233 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/629132972", 2022-07-21T01:56:37.233 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:56:37.233 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 53, 2022-07-21T01:56:37.234 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 113, 2022-07-21T01:56:37.234 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:56:37.234 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:56:37.234 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:56:37.234 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:56:37.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:37.236 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:37.237 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:37.244 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:56:37.245 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:56:37.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:37.246 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:37.263 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:56:37.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.368+0000 7fec3cccb700 1 Processor -- start 2022-07-21T01:56:37.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.368+0000 7fec3cccb700 1 -- start start 2022-07-21T01:56:37.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fec3805c3f0 0x7fec3814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3805d480 con 0x7fec380b1e00 2022-07-21T01:56:37.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec380b21e0 con 0x7fec380acef0 2022-07-21T01:56:37.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3805cf40 con 0x7fec3805c3f0 2022-07-21T01:56:37.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec2dd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fec3805c3f0 0x7fec3814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec2dd9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fec3805c3f0 0x7fec3814a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39674/0 (socket says 172.21.15.140:39674) 2022-07-21T01:56:37.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec2dd9b700 1 -- 172.21.15.140:0/4267024702 learned_addr learned my addr 172.21.15.140:0/4267024702 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:37.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3659c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fec380b1e00 0x7fec380ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53820/0 (socket says 172.21.15.140:53820) 2022-07-21T01:56:37.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec2dd9b700 1 -- 172.21.15.140:0/4267024702 >> v1:172.21.15.152:6789/0 conn(0x7fec380acef0 legacy=0x7fec38146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:37.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec2dd9b700 1 -- 172.21.15.140:0/4267024702 >> v1:172.21.15.140:6789/0 conn(0x7fec380b1e00 legacy=0x7fec380ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec2dd9b700 1 -- 172.21.15.140:0/4267024702 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3805d1e0 con 0x7fec3805c3f0 2022-07-21T01:56:37.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec2dd9b700 1 --2- 172.21.15.140:0/4267024702 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec3805c3f0 0x7fec3814a6d0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fec3805f3e0 tx=0x7fec28006eb0).ready entity=mon.0 client_cookie=bf1deb628e1c84d9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:37.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec35d9b700 1 -- 172.21.15.140:0/4267024702 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec2800d610 con 0x7fec3805c3f0 2022-07-21T01:56:37.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec35d9b700 1 -- 172.21.15.140:0/4267024702 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec28000e90 con 0x7fec3805c3f0 2022-07-21T01:56:37.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec35d9b700 1 -- 172.21.15.140:0/4267024702 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec2800d960 con 0x7fec3805c3f0 2022-07-21T01:56:37.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 -- 172.21.15.140:0/4267024702 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec3805c3f0 msgr2=0x7fec3814a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 --2- 172.21.15.140:0/4267024702 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec3805c3f0 0x7fec3814a6d0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fec3805f3e0 tx=0x7fec28006eb0).stop 2022-07-21T01:56:37.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 -- 172.21.15.140:0/4267024702 shutdown_connections 2022-07-21T01:56:37.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 --2- 172.21.15.140:0/4267024702 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec3805c3f0 0x7fec3814a6d0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 -- 172.21.15.140:0/4267024702 >> 172.21.15.140:0/4267024702 conn(0x7fec380611c0 msgr2=0x7fec38062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:37.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 -- 172.21.15.140:0/4267024702 shutdown_connections 2022-07-21T01:56:37.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 -- 172.21.15.140:0/4267024702 wait complete. 2022-07-21T01:56:37.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 Processor -- start 2022-07-21T01:56:37.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.372+0000 7fec3cccb700 1 -- start start 2022-07-21T01:56:37.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec3cccb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec380acef0 0x7fec380b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec3cccb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec380b1e00 0x7fec380ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec3cccb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec380b5850 0x7fec3809f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec3cccb700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380a1f60 con 0x7fec380b5850 2022-07-21T01:56:37.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec3cccb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380b4940 con 0x7fec380b1e00 2022-07-21T01:56:37.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec3cccb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec380b4be0 con 0x7fec380acef0 2022-07-21T01:56:37.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec36d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec380b5850 0x7fec3809f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec36d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec380b5850 0x7fec3809f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39682/0 (socket says 172.21.15.140:39682) 2022-07-21T01:56:37.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec36d9d700 1 -- 172.21.15.140:0/408883323 learned_addr learned my addr 172.21.15.140:0/408883323 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:37.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec2dd9b700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec380b1e00 0x7fec380ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec3659c700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec380acef0 0x7fec380b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec36d9d700 1 -- 172.21.15.140:0/408883323 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec380acef0 msgr2=0x7fec380b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec36d9d700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec380acef0 0x7fec380b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec36d9d700 1 -- 172.21.15.140:0/408883323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec380b1e00 msgr2=0x7fec380ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec36d9d700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec380b1e00 0x7fec380ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec36d9d700 1 -- 172.21.15.140:0/408883323 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2800b1e0 con 0x7fec380b5850 2022-07-21T01:56:37.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec36d9d700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec380b5850 0x7fec3809f820 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7fec30002320 tx=0x7fec300023f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:37.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec2ffff700 1 -- 172.21.15.140:0/408883323 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec30006aa0 con 0x7fec380b5850 2022-07-21T01:56:37.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec2ffff700 1 -- 172.21.15.140:0/408883323 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec30006c00 con 0x7fec380b5850 2022-07-21T01:56:37.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec3cccb700 1 -- 172.21.15.140:0/408883323 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec380a1be0 con 0x7fec380b5850 2022-07-21T01:56:37.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec2ffff700 1 -- 172.21.15.140:0/408883323 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec30007030 con 0x7fec380b5850 2022-07-21T01:56:37.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec3cccb700 1 -- 172.21.15.140:0/408883323 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec380b43a0 con 0x7fec380b5850 2022-07-21T01:56:37.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.376+0000 7fec3cccb700 1 -- 172.21.15.140:0/408883323 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec380a1dd0 con 0x7fec380b5850 2022-07-21T01:56:37.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.380+0000 7fec2ffff700 1 -- 172.21.15.140:0/408883323 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fec30010030 con 0x7fec380b5850 2022-07-21T01:56:37.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.380+0000 7fec2ffff700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec1c053b00 0x7fec1c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.380+0000 7fec3659c700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec1c053b00 0x7fec1c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.380+0000 7fec2ffff700 1 -- 172.21.15.140:0/408883323 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec3006bf40 con 0x7fec380b5850 2022-07-21T01:56:37.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.380+0000 7fec3659c700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec1c053b00 0x7fec1c055fc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fec2000b000 tx=0x7fec20007160).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:37.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.384+0000 7fec2ffff700 1 -- 172.21.15.140:0/408883323 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fec30042d50 con 0x7fec380b5850 2022-07-21T01:56:37.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.508+0000 7fec3cccb700 1 -- 172.21.15.140:0/408883323 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec380b04b0 con 0x7fec380b5850 2022-07-21T01:56:37.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.508+0000 7fec2ffff700 1 -- 172.21.15.140:0/408883323 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16148 (secure 0 0 0) 0x7fec30046370 con 0x7fec380b5850 2022-07-21T01:56:37.512 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:37.513 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":113,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:56:30.933163+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:56:27.736358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2022-07-21T01:56:28.943049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:56:37.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 -- 172.21.15.140:0/408883323 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec1c053b00 msgr2=0x7fec1c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec1c053b00 0x7fec1c055fc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fec2000b000 tx=0x7fec20007160).stop 2022-07-21T01:56:37.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 -- 172.21.15.140:0/408883323 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec380b5850 msgr2=0x7fec3809f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec380b5850 0x7fec3809f820 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7fec30002320 tx=0x7fec300023f0).stop 2022-07-21T01:56:37.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 -- 172.21.15.140:0/408883323 shutdown_connections 2022-07-21T01:56:37.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec1c053b00 0x7fec1c055fc0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec380b5850 0x7fec3809f820 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec380b1e00 0x7fec380ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 --2- 172.21.15.140:0/408883323 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec380acef0 0x7fec380b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 -- 172.21.15.140:0/408883323 >> 172.21.15.140:0/408883323 conn(0x7fec3805c3f0 msgr2=0x7fec380afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:37.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 -- 172.21.15.140:0/408883323 shutdown_connections 2022-07-21T01:56:37.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.512+0000 7fec3cccb700 1 -- 172.21.15.140:0/408883323 wait complete. 2022-07-21T01:56:37.525 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:56:27.736358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:56:28.943049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:56:37.526 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:56:37.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 Processor -- start 2022-07-21T01:56:37.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 -- start start 2022-07-21T01:56:37.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fef4c0afc30 0x7fef4c147a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fef4c148b60 0x7fef4c145e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef4c148190 con 0x7fef4c0b1590 2022-07-21T01:56:37.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c098670 con 0x7fef4c148b60 2022-07-21T01:56:37.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c147f70 con 0x7fef4c0afc30 2022-07-21T01:56:37.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef4ad9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fef4c148b60 0x7fef4c145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef4ad9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fef4c148b60 0x7fef4c145e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39692/0 (socket says 172.21.15.140:39692) 2022-07-21T01:56:37.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef4ad9d700 1 -- 172.21.15.140:0/298685974 learned_addr learned my addr 172.21.15.140:0/298685974 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:37.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef49d9b700 1 --2- 172.21.15.140:0/298685974 >> v2:172.21.15.152:3300/0 conn(0x7fef4c0afc30 0x7fef4c147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef4ad9d700 1 -- 172.21.15.140:0/298685974 >> v2:172.21.15.152:3300/0 conn(0x7fef4c0afc30 msgr2=0x7fef4c147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:37.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef4ad9d700 1 --2- 172.21.15.140:0/298685974 >> v2:172.21.15.152:3300/0 conn(0x7fef4c0afc30 0x7fef4c147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef4ad9d700 1 -- 172.21.15.140:0/298685974 >> v1:172.21.15.152:6789/0 conn(0x7fef4c0b1590 legacy=0x7fef4c148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef4ad9d700 1 -- 172.21.15.140:0/298685974 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c0b1970 con 0x7fef4c148b60 2022-07-21T01:56:37.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef4ad9d700 1 --2- 172.21.15.140:0/298685974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef4c148b60 0x7fef4c145e90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7fef4c0987b0 tx=0x7fef3c006eb0).ready entity=mon.0 client_cookie=ef92de88e858367b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:37.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef4959a700 1 -- 172.21.15.140:0/298685974 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef3c00d610 con 0x7fef4c148b60 2022-07-21T01:56:37.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef4959a700 1 -- 172.21.15.140:0/298685974 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c000e90 con 0x7fef4c148b60 2022-07-21T01:56:37.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef4959a700 1 -- 172.21.15.140:0/298685974 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef3c00d9a0 con 0x7fef4c148b60 2022-07-21T01:56:37.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 -- 172.21.15.140:0/298685974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef4c148b60 msgr2=0x7fef4c145e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 --2- 172.21.15.140:0/298685974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef4c148b60 0x7fef4c145e90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7fef4c0987b0 tx=0x7fef3c006eb0).stop 2022-07-21T01:56:37.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 -- 172.21.15.140:0/298685974 shutdown_connections 2022-07-21T01:56:37.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 --2- 172.21.15.140:0/298685974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef4c148b60 0x7fef4c145e90 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 --2- 172.21.15.140:0/298685974 >> v2:172.21.15.152:3300/0 conn(0x7fef4c0afc30 0x7fef4c147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 -- 172.21.15.140:0/298685974 >> 172.21.15.140:0/298685974 conn(0x7fef4c0b2340 msgr2=0x7fef4c0b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:37.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 -- 172.21.15.140:0/298685974 shutdown_connections 2022-07-21T01:56:37.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.636+0000 7fef50e7e700 1 -- 172.21.15.140:0/298685974 wait complete. 2022-07-21T01:56:37.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef50e7e700 1 Processor -- start 2022-07-21T01:56:37.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef50e7e700 1 -- start start 2022-07-21T01:56:37.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef50e7e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef4c0afc30 0x7fef4c064190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef50e7e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef4c0b1590 0x7fef4c0b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef50e7e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef4c148b60 0x7fef4c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef50e7e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c064680 con 0x7fef4c148b60 2022-07-21T01:56:37.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef50e7e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1422e0 con 0x7fef4c0b1590 2022-07-21T01:56:37.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef50e7e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c142580 con 0x7fef4c0afc30 2022-07-21T01:56:37.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef4ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef4c148b60 0x7fef4c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef4ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef4c148b60 0x7fef4c159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39698/0 (socket says 172.21.15.140:39698) 2022-07-21T01:56:37.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef4ad9d700 1 -- 172.21.15.140:0/766341039 learned_addr learned my addr 172.21.15.140:0/766341039 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:37.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef49d9b700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef4c0b1590 0x7fef4c0b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef4a59c700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef4c0afc30 0x7fef4c064190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef4ad9d700 1 -- 172.21.15.140:0/766341039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef4c0afc30 msgr2=0x7fef4c064190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef4ad9d700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef4c0afc30 0x7fef4c064190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef4ad9d700 1 -- 172.21.15.140:0/766341039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef4c0b1590 msgr2=0x7fef4c0b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef4ad9d700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef4c0b1590 0x7fef4c0b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef4ad9d700 1 -- 172.21.15.140:0/766341039 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c00b1e0 con 0x7fef4c148b60 2022-07-21T01:56:37.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef4ad9d700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef4c148b60 0x7fef4c159ce0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7fef4c0987b0 tx=0x7fef3c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:37.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef437fe700 1 -- 172.21.15.140:0/766341039 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef3c021070 con 0x7fef4c148b60 2022-07-21T01:56:37.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef437fe700 1 -- 172.21.15.140:0/766341039 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c0116e0 con 0x7fef4c148b60 2022-07-21T01:56:37.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef50e7e700 1 -- 172.21.15.140:0/766341039 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4c159840 con 0x7fef4c148b60 2022-07-21T01:56:37.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef437fe700 1 -- 172.21.15.140:0/766341039 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fef3c011a40 con 0x7fef4c148b60 2022-07-21T01:56:37.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.640+0000 7fef50e7e700 1 -- 172.21.15.140:0/766341039 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c141d60 con 0x7fef4c148b60 2022-07-21T01:56:37.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.644+0000 7fef50e7e700 1 -- 172.21.15.140:0/766341039 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c1599d0 con 0x7fef4c148b60 2022-07-21T01:56:37.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.644+0000 7fef437fe700 1 -- 172.21.15.140:0/766341039 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fef3c00f070 con 0x7fef4c148b60 2022-07-21T01:56:37.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.644+0000 7fef437fe700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef30053b00 0x7fef30055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.644+0000 7fef437fe700 1 -- 172.21.15.140:0/766341039 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fef3c06d1c0 con 0x7fef4c148b60 2022-07-21T01:56:37.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.644+0000 7fef4a59c700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef30053b00 0x7fef30055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.644+0000 7fef4a59c700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef30053b00 0x7fef30055fc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fef3400d000 tx=0x7fef34007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:37.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.648+0000 7fef437fe700 1 -- 172.21.15.140:0/766341039 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fef3c044190 con 0x7fef4c148b60 2022-07-21T01:56:37.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.772+0000 7fef50e7e700 1 -- 172.21.15.140:0/766341039 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fef4c097f30 con 0x7fef4c148b60 2022-07-21T01:56:37.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.772+0000 7fef437fe700 1 -- 172.21.15.140:0/766341039 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v113) v1 ==== 110+0+10 (secure 0 0 0) 0x7fef3c0477b0 con 0x7fef4c148b60 2022-07-21T01:56:37.777 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:56:37.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 -- 172.21.15.140:0/766341039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef30053b00 msgr2=0x7fef30055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef30053b00 0x7fef30055fc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fef3400d000 tx=0x7fef34007b90).stop 2022-07-21T01:56:37.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 -- 172.21.15.140:0/766341039 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef4c148b60 msgr2=0x7fef4c159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef4c148b60 0x7fef4c159ce0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7fef4c0987b0 tx=0x7fef3c015000).stop 2022-07-21T01:56:37.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 -- 172.21.15.140:0/766341039 shutdown_connections 2022-07-21T01:56:37.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fef30053b00 0x7fef30055fc0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fef4c148b60 0x7fef4c159ce0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef4c0b1590 0x7fef4c0b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 --2- 172.21.15.140:0/766341039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fef4c0afc30 0x7fef4c064190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 -- 172.21.15.140:0/766341039 >> 172.21.15.140:0/766341039 conn(0x7fef4c0b2340 msgr2=0x7fef4c1404c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:37.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 -- 172.21.15.140:0/766341039 shutdown_connections 2022-07-21T01:56:37.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.776+0000 7fef50e7e700 1 -- 172.21.15.140:0/766341039 wait complete. 2022-07-21T01:56:37.797 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:56:37.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc767fff700 1 Processor -- start 2022-07-21T01:56:37.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc767fff700 1 -- start start 2022-07-21T01:56:37.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc767fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7680b2ec0 0x7fc7680b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc767fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc76805d160 0x7fc768142210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc767fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7680b3f10 con 0x7fc768142700 2022-07-21T01:56:37.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc767fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc768142b90 con 0x7fc76805d160 2022-07-21T01:56:37.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc767fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680b4ea0 con 0x7fc7680b2ec0 2022-07-21T01:56:37.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc7667fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc76805d160 0x7fc768142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc7667fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc76805d160 0x7fc768142210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39704/0 (socket says 172.21.15.140:39704) 2022-07-21T01:56:37.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc7667fc700 1 -- 172.21.15.140:0/3711231704 learned_addr learned my addr 172.21.15.140:0/3711231704 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:37.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc766ffd700 1 --2- 172.21.15.140:0/3711231704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7680b2ec0 0x7fc7680b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc7667fc700 1 -- 172.21.15.140:0/3711231704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7680b2ec0 msgr2=0x7fc7680b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:37.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc7667fc700 1 --2- 172.21.15.140:0/3711231704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7680b2ec0 0x7fc7680b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc7667fc700 1 -- 172.21.15.140:0/3711231704 >> v1:172.21.15.152:6789/0 conn(0x7fc768142700 legacy=0x7fc7681461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc7667fc700 1 -- 172.21.15.140:0/3711231704 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7680b3790 con 0x7fc76805d160 2022-07-21T01:56:37.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc7667fc700 1 --2- 172.21.15.140:0/3711231704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc76805d160 0x7fc768142210 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7fc7680b4fe0 tx=0x7fc750006eb0).ready entity=mon.0 client_cookie=8542eae4438ce624 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:37.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc765ffb700 1 -- 172.21.15.140:0/3711231704 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc75000d5c0 con 0x7fc76805d160 2022-07-21T01:56:37.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc765ffb700 1 -- 172.21.15.140:0/3711231704 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc750000e90 con 0x7fc76805d160 2022-07-21T01:56:37.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.904+0000 7fc765ffb700 1 -- 172.21.15.140:0/3711231704 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc75000d950 con 0x7fc76805d160 2022-07-21T01:56:37.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 -- 172.21.15.140:0/3711231704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc76805d160 msgr2=0x7fc768142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 --2- 172.21.15.140:0/3711231704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc76805d160 0x7fc768142210 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7fc7680b4fe0 tx=0x7fc750006eb0).stop 2022-07-21T01:56:37.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 -- 172.21.15.140:0/3711231704 shutdown_connections 2022-07-21T01:56:37.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 --2- 172.21.15.140:0/3711231704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc76805d160 0x7fc768142210 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 --2- 172.21.15.140:0/3711231704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc7680b2ec0 0x7fc7680b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 -- 172.21.15.140:0/3711231704 >> 172.21.15.140:0/3711231704 conn(0x7fc768066e20 msgr2=0x7fc768067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:37.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 -- 172.21.15.140:0/3711231704 shutdown_connections 2022-07-21T01:56:37.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 -- 172.21.15.140:0/3711231704 wait complete. 2022-07-21T01:56:37.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 Processor -- start 2022-07-21T01:56:37.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 -- start start 2022-07-21T01:56:37.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc76805d160 0x7fc7681444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7680b2ec0 0x7fc768155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc768142700 0x7fc768112800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680aded0 con 0x7fc768142700 2022-07-21T01:56:37.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680acd80 con 0x7fc7680b2ec0 2022-07-21T01:56:37.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc767fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680acae0 con 0x7fc76805d160 2022-07-21T01:56:37.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc7677fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc768142700 0x7fc768112800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc7677fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc768142700 0x7fc768112800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39712/0 (socket says 172.21.15.140:39712) 2022-07-21T01:56:37.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc7677fe700 1 -- 172.21.15.140:0/899625786 learned_addr learned my addr 172.21.15.140:0/899625786 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:37.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc7667fc700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7680b2ec0 0x7fc768155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc766ffd700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc76805d160 0x7fc7681444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc7677fe700 1 -- 172.21.15.140:0/899625786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc76805d160 msgr2=0x7fc7681444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc7677fe700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc76805d160 0x7fc7681444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc7677fe700 1 -- 172.21.15.140:0/899625786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7680b2ec0 msgr2=0x7fc768155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:37.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc7677fe700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7680b2ec0 0x7fc768155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:37.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc7677fe700 1 -- 172.21.15.140:0/899625786 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75000b1e0 con 0x7fc768142700 2022-07-21T01:56:37.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.908+0000 7fc7667fc700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7680b2ec0 0x7fc768155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:37.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc7677fe700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc768142700 0x7fc768112800 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7fc7680b6e70 tx=0x7fc760007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:37.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc75ffff700 1 -- 172.21.15.140:0/899625786 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc76000e4a0 con 0x7fc768142700 2022-07-21T01:56:37.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc75ffff700 1 -- 172.21.15.140:0/899625786 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76000e600 con 0x7fc768142700 2022-07-21T01:56:37.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc767fff700 1 -- 172.21.15.140:0/899625786 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7680adc30 con 0x7fc768142700 2022-07-21T01:56:37.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc75ffff700 1 -- 172.21.15.140:0/899625786 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc76000e960 con 0x7fc768142700 2022-07-21T01:56:37.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc767fff700 1 -- 172.21.15.140:0/899625786 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc768155fd0 con 0x7fc768142700 2022-07-21T01:56:37.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc767fff700 1 -- 172.21.15.140:0/899625786 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7680b2620 con 0x7fc768142700 2022-07-21T01:56:37.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc75ffff700 1 -- 172.21.15.140:0/899625786 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc760019030 con 0x7fc768142700 2022-07-21T01:56:37.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc75ffff700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc74c053a70 0x7fc74c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:37.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc75ffff700 1 -- 172.21.15.140:0/899625786 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc76006b670 con 0x7fc768142700 2022-07-21T01:56:37.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc766ffd700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc74c053a70 0x7fc74c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:37.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.912+0000 7fc766ffd700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc74c053a70 0x7fc74c055f30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fc75800c000 tx=0x7fc758008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:37.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:37.924+0000 7fc75ffff700 1 -- 172.21.15.140:0/899625786 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc760042590 con 0x7fc768142700 2022-07-21T01:56:38.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.048+0000 7fc767fff700 1 -- 172.21.15.140:0/899625786 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc7680574e0 con 0x7fc768142700 2022-07-21T01:56:38.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.048+0000 7fc75ffff700 1 -- 172.21.15.140:0/899625786 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc760045bb0 con 0x7fc768142700 2022-07-21T01:56:38.052 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:38.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 -- 172.21.15.140:0/899625786 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc74c053a70 msgr2=0x7fc74c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc74c053a70 0x7fc74c055f30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fc75800c000 tx=0x7fc758008040).stop 2022-07-21T01:56:38.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 -- 172.21.15.140:0/899625786 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc768142700 msgr2=0x7fc768112800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc768142700 0x7fc768112800 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7fc7680b6e70 tx=0x7fc760007f40).stop 2022-07-21T01:56:38.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 -- 172.21.15.140:0/899625786 shutdown_connections 2022-07-21T01:56:38.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc74c053a70 0x7fc74c055f30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc768142700 0x7fc768112800 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc7680b2ec0 0x7fc768155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 --2- 172.21.15.140:0/899625786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc76805d160 0x7fc7681444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 -- 172.21.15.140:0/899625786 >> 172.21.15.140:0/899625786 conn(0x7fc768066e20 msgr2=0x7fc7680b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:38.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 -- 172.21.15.140:0/899625786 shutdown_connections 2022-07-21T01:56:38.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.052+0000 7fc767fff700 1 -- 172.21.15.140:0/899625786 wait complete. 2022-07-21T01:56:38.066 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:56:38.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40e6b0700 1 Processor -- start 2022-07-21T01:56:38.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40e6b0700 1 -- start start 2022-07-21T01:56:38.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40e6b0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe408147ab0 0x7fe4080b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40e6b0700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40805cc70 con 0x7fe4080af940 2022-07-21T01:56:38.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40e6b0700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4080b5dd0 con 0x7fe4080b59f0 2022-07-21T01:56:38.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40e6b0700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40805c760 con 0x7fe408147ab0 2022-07-21T01:56:38.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40deaf700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe408147ab0 0x7fe4080b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40cead700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe4080af940 0x7fe4080b0060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53862/0 (socket says 172.21.15.140:53862) 2022-07-21T01:56:38.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40cead700 1 -- 172.21.15.140:0/223206580 learned_addr learned my addr 172.21.15.140:0/223206580 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:38.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe3fbfff700 1 -- 172.21.15.140:0/223206580 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643877392 0 0) 0x7fe40805cc70 con 0x7fe4080af940 2022-07-21T01:56:38.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40deaf700 1 -- 172.21.15.140:0/223206580 >> v1:172.21.15.152:6789/0 conn(0x7fe4080b59f0 legacy=0x7fe4081473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40deaf700 1 -- 172.21.15.140:0/223206580 >> v1:172.21.15.140:6789/0 conn(0x7fe4080af940 legacy=0x7fe4080b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40deaf700 1 -- 172.21.15.140:0/223206580 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40805c9d0 con 0x7fe408147ab0 2022-07-21T01:56:38.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe40deaf700 1 --2- 172.21.15.140:0/223206580 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe408147ab0 0x7fe4080b0770 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7fe400002f70 tx=0x7fe400006df0).ready entity=mon.0 client_cookie=604c40265adfa801 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:38.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe3fbfff700 1 -- 172.21.15.140:0/223206580 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe40000d610 con 0x7fe408147ab0 2022-07-21T01:56:38.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe3fbfff700 1 -- 172.21.15.140:0/223206580 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400000e90 con 0x7fe408147ab0 2022-07-21T01:56:38.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.172+0000 7fe3fbfff700 1 -- 172.21.15.140:0/223206580 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe40000d960 con 0x7fe408147ab0 2022-07-21T01:56:38.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 -- 172.21.15.140:0/223206580 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe408147ab0 msgr2=0x7fe4080b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 --2- 172.21.15.140:0/223206580 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe408147ab0 0x7fe4080b0770 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7fe400002f70 tx=0x7fe400006df0).stop 2022-07-21T01:56:38.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 -- 172.21.15.140:0/223206580 shutdown_connections 2022-07-21T01:56:38.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 --2- 172.21.15.140:0/223206580 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe408147ab0 0x7fe4080b0770 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 -- 172.21.15.140:0/223206580 >> 172.21.15.140:0/223206580 conn(0x7fe408063480 msgr2=0x7fe4080b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:38.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 -- 172.21.15.140:0/223206580 shutdown_connections 2022-07-21T01:56:38.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 -- 172.21.15.140:0/223206580 wait complete. 2022-07-21T01:56:38.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 Processor -- start 2022-07-21T01:56:38.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 -- start start 2022-07-21T01:56:38.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4080af940 0x7fe408156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4080b59f0 0x7fe40810f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe408147ab0 0x7fe4081555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40810e440 con 0x7fe408147ab0 2022-07-21T01:56:38.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe408157200 con 0x7fe4080b59f0 2022-07-21T01:56:38.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40e6b0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40810f750 con 0x7fe4080af940 2022-07-21T01:56:38.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40deaf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe408147ab0 0x7fe4081555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40deaf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe408147ab0 0x7fe4081555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39726/0 (socket says 172.21.15.140:39726) 2022-07-21T01:56:38.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40deaf700 1 -- 172.21.15.140:0/1138457304 learned_addr learned my addr 172.21.15.140:0/1138457304 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:38.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40cead700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4080b59f0 0x7fe40810f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40d6ae700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4080af940 0x7fe408156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40deaf700 1 -- 172.21.15.140:0/1138457304 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4080af940 msgr2=0x7fe408156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40deaf700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4080af940 0x7fe408156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40deaf700 1 -- 172.21.15.140:0/1138457304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4080b59f0 msgr2=0x7fe40810f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40deaf700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4080b59f0 0x7fe40810f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.176+0000 7fe40deaf700 1 -- 172.21.15.140:0/1138457304 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40000b180 con 0x7fe408147ab0 2022-07-21T01:56:38.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.180+0000 7fe40deaf700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe408147ab0 0x7fe4081555a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7fe408157340 tx=0x7fe400015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:38.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.180+0000 7fe3fa7fc700 1 -- 172.21.15.140:0/1138457304 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe40000d990 con 0x7fe408147ab0 2022-07-21T01:56:38.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.180+0000 7fe40e6b0700 1 -- 172.21.15.140:0/1138457304 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40810f990 con 0x7fe408147ab0 2022-07-21T01:56:38.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.180+0000 7fe3fa7fc700 1 -- 172.21.15.140:0/1138457304 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400011b70 con 0x7fe408147ab0 2022-07-21T01:56:38.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.180+0000 7fe3fa7fc700 1 -- 172.21.15.140:0/1138457304 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe40001e460 con 0x7fe408147ab0 2022-07-21T01:56:38.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.180+0000 7fe40e6b0700 1 -- 172.21.15.140:0/1138457304 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4081421a0 con 0x7fe408147ab0 2022-07-21T01:56:38.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.180+0000 7fe40e6b0700 1 -- 172.21.15.140:0/1138457304 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe408146f00 con 0x7fe408147ab0 2022-07-21T01:56:38.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.180+0000 7fe3fa7fc700 1 -- 172.21.15.140:0/1138457304 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe400025030 con 0x7fe408147ab0 2022-07-21T01:56:38.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.180+0000 7fe3fa7fc700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe3f0053a70 0x7fe3f0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.180+0000 7fe40d6ae700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe3f0053a70 0x7fe3f0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.180+0000 7fe3fa7fc700 1 -- 172.21.15.140:0/1138457304 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe40006d300 con 0x7fe408147ab0 2022-07-21T01:56:38.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.184+0000 7fe40d6ae700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe3f0053a70 0x7fe3f0055f30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fe40400d000 tx=0x7fe404009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:38.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.188+0000 7fe3fa7fc700 1 -- 172.21.15.140:0/1138457304 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe400044110 con 0x7fe408147ab0 2022-07-21T01:56:38.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.312+0000 7fe40e6b0700 1 -- 172.21.15.140:0/1138457304 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe408142ca0 con 0x7fe408147ab0 2022-07-21T01:56:38.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.312+0000 7fe3fa7fc700 1 -- 172.21.15.140:0/1138457304 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe4000230a0 con 0x7fe408147ab0 2022-07-21T01:56:38.316 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:38.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 -- 172.21.15.140:0/1138457304 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe3f0053a70 msgr2=0x7fe3f0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe3f0053a70 0x7fe3f0055f30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fe40400d000 tx=0x7fe404009040).stop 2022-07-21T01:56:38.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 -- 172.21.15.140:0/1138457304 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe408147ab0 msgr2=0x7fe4081555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe408147ab0 0x7fe4081555a0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7fe408157340 tx=0x7fe400015000).stop 2022-07-21T01:56:38.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 -- 172.21.15.140:0/1138457304 shutdown_connections 2022-07-21T01:56:38.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe3f0053a70 0x7fe3f0055f30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe408147ab0 0x7fe4081555a0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4080b59f0 0x7fe40810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 --2- 172.21.15.140:0/1138457304 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4080af940 0x7fe408156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 -- 172.21.15.140:0/1138457304 >> 172.21.15.140:0/1138457304 conn(0x7fe408063480 msgr2=0x7fe40813e780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:38.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 -- 172.21.15.140:0/1138457304 shutdown_connections 2022-07-21T01:56:38.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.316+0000 7fe40e6b0700 1 -- 172.21.15.140:0/1138457304 wait complete. 2022-07-21T01:56:38.328 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:56:38.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.436+0000 7fd817fff700 1 Processor -- start 2022-07-21T01:56:38.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.436+0000 7fd817fff700 1 -- start start 2022-07-21T01:56:38.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.436+0000 7fd817fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8180b59d0 0x7fd8180b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.436+0000 7fd817fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8180afbc0 con 0x7fd81814ada0 2022-07-21T01:56:38.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.436+0000 7fd817fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8180b00e0 con 0x7fd8180b14f0 2022-07-21T01:56:38.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd817fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd818148080 con 0x7fd8180b59d0 2022-07-21T01:56:38.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd8167fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd81814ada0 0x7fd81805bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53880/0 (socket says 172.21.15.140:53880) 2022-07-21T01:56:38.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd8167fc700 1 -- 172.21.15.140:0/1557972836 learned_addr learned my addr 172.21.15.140:0/1557972836 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:38.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd816ffd700 1 --2- 172.21.15.140:0/1557972836 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8180b59d0 0x7fd8180b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd815ffb700 1 -- 172.21.15.140:0/1557972836 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856464545 0 0) 0x7fd8180afbc0 con 0x7fd81814ada0 2022-07-21T01:56:38.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd815ffb700 1 -- 172.21.15.140:0/1557972836 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7fc003260 con 0x7fd81814ada0 2022-07-21T01:56:38.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd815ffb700 1 -- 172.21.15.140:0/1557972836 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263538701 0 0) 0x7fd8180b00e0 con 0x7fd8180b14f0 2022-07-21T01:56:38.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd815ffb700 1 -- 172.21.15.140:0/1557972836 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8180afbc0 con 0x7fd8180b14f0 2022-07-21T01:56:38.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd816ffd700 1 -- 172.21.15.140:0/1557972836 >> v1:172.21.15.152:6789/0 conn(0x7fd8180b14f0 legacy=0x7fd81805c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd816ffd700 1 -- 172.21.15.140:0/1557972836 >> v1:172.21.15.140:6789/0 conn(0x7fd81814ada0 legacy=0x7fd81805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd816ffd700 1 -- 172.21.15.140:0/1557972836 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8180b5db0 con 0x7fd8180b59d0 2022-07-21T01:56:38.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd816ffd700 1 --2- 172.21.15.140:0/1557972836 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8180b59d0 0x7fd8180b07e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fd8100032f0 tx=0x7fd8100091d0).ready entity=mon.2 client_cookie=52004549d78a401a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:38.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd815ffb700 1 -- 172.21.15.140:0/1557972836 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd81000b600 con 0x7fd8180b59d0 2022-07-21T01:56:38.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd815ffb700 1 -- 172.21.15.140:0/1557972836 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81000b760 con 0x7fd8180b59d0 2022-07-21T01:56:38.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd815ffb700 1 -- 172.21.15.140:0/1557972836 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd81000b8d0 con 0x7fd8180b59d0 2022-07-21T01:56:38.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd817fff700 1 -- 172.21.15.140:0/1557972836 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8180b59d0 msgr2=0x7fd8180b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd817fff700 1 --2- 172.21.15.140:0/1557972836 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8180b59d0 0x7fd8180b07e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fd8100032f0 tx=0x7fd8100091d0).stop 2022-07-21T01:56:38.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd817fff700 1 -- 172.21.15.140:0/1557972836 shutdown_connections 2022-07-21T01:56:38.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd817fff700 1 --2- 172.21.15.140:0/1557972836 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8180b59d0 0x7fd8180b07e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd817fff700 1 -- 172.21.15.140:0/1557972836 >> 172.21.15.140:0/1557972836 conn(0x7fd8180634c0 msgr2=0x7fd8180b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:38.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd817fff700 1 -- 172.21.15.140:0/1557972836 shutdown_connections 2022-07-21T01:56:38.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd817fff700 1 -- 172.21.15.140:0/1557972836 wait complete. 2022-07-21T01:56:38.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.440+0000 7fd817fff700 1 Processor -- start 2022-07-21T01:56:38.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd817fff700 1 -- start start 2022-07-21T01:56:38.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd817fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8180b14f0 0x7fd81810f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd817fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8180b59d0 0x7fd81814c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd817fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd81814ada0 0x7fd8181566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd817fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd818065190 con 0x7fd8180b59d0 2022-07-21T01:56:38.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd817fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd818064c50 con 0x7fd81814ada0 2022-07-21T01:56:38.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd817fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd818064ef0 con 0x7fd8180b14f0 2022-07-21T01:56:38.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd8167fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8180b59d0 0x7fd81814c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd8167fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8180b59d0 0x7fd81814c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39738/0 (socket says 172.21.15.140:39738) 2022-07-21T01:56:38.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd8167fc700 1 -- 172.21.15.140:0/297244288 learned_addr learned my addr 172.21.15.140:0/297244288 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:38.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd8177fe700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd81814ada0 0x7fd8181566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd816ffd700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8180b14f0 0x7fd81810f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd8167fc700 1 -- 172.21.15.140:0/297244288 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8180b14f0 msgr2=0x7fd81810f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd8167fc700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8180b14f0 0x7fd81810f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd8167fc700 1 -- 172.21.15.140:0/297244288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd81814ada0 msgr2=0x7fd8181566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd8167fc700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd81814ada0 0x7fd8181566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd8167fc700 1 -- 172.21.15.140:0/297244288 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd810009470 con 0x7fd8180b59d0 2022-07-21T01:56:38.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd8167fc700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8180b59d0 0x7fd81814c370 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7fd80c001110 tx=0x7fd80c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:38.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd807fff700 1 -- 172.21.15.140:0/297244288 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd80c0070d0 con 0x7fd8180b59d0 2022-07-21T01:56:38.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd807fff700 1 -- 172.21.15.140:0/297244288 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c007230 con 0x7fd8180b59d0 2022-07-21T01:56:38.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd817fff700 1 -- 172.21.15.140:0/297244288 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81814c860 con 0x7fd8180b59d0 2022-07-21T01:56:38.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd807fff700 1 -- 172.21.15.140:0/297244288 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd80c005610 con 0x7fd8180b59d0 2022-07-21T01:56:38.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd817fff700 1 -- 172.21.15.140:0/297244288 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd818111f50 con 0x7fd8180b59d0 2022-07-21T01:56:38.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.444+0000 7fd817fff700 1 -- 172.21.15.140:0/297244288 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8180b0340 con 0x7fd8180b59d0 2022-07-21T01:56:38.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.448+0000 7fd807fff700 1 -- 172.21.15.140:0/297244288 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd80c01a060 con 0x7fd8180b59d0 2022-07-21T01:56:38.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.448+0000 7fd807fff700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7f4053a70 0x7fd7f4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.448+0000 7fd807fff700 1 -- 172.21.15.140:0/297244288 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd80c06b150 con 0x7fd8180b59d0 2022-07-21T01:56:38.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.448+0000 7fd816ffd700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7f4053a70 0x7fd7f4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.448+0000 7fd816ffd700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7f4053a70 0x7fd7f4055f30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fd81001b000 tx=0x7fd810016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:38.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.452+0000 7fd807fff700 1 -- 172.21.15.140:0/297244288 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd80c042070 con 0x7fd8180b59d0 2022-07-21T01:56:38.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.576+0000 7fd817fff700 1 -- 172.21.15.140:0/297244288 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8180574e0 con 0x7fd8180b59d0 2022-07-21T01:56:38.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd807fff700 1 -- 172.21.15.140:0/297244288 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd80c018020 con 0x7fd8180b59d0 2022-07-21T01:56:38.581 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:56:38.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 -- 172.21.15.140:0/297244288 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7f4053a70 msgr2=0x7fd7f4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7f4053a70 0x7fd7f4055f30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fd81001b000 tx=0x7fd810016000).stop 2022-07-21T01:56:38.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 -- 172.21.15.140:0/297244288 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8180b59d0 msgr2=0x7fd81814c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8180b59d0 0x7fd81814c370 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7fd80c001110 tx=0x7fd80c007ad0).stop 2022-07-21T01:56:38.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 -- 172.21.15.140:0/297244288 shutdown_connections 2022-07-21T01:56:38.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7f4053a70 0x7fd7f4055f30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd81814ada0 0x7fd8181566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8180b59d0 0x7fd81814c370 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 --2- 172.21.15.140:0/297244288 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8180b14f0 0x7fd81810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 -- 172.21.15.140:0/297244288 >> 172.21.15.140:0/297244288 conn(0x7fd8180634c0 msgr2=0x7fd81814d810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:38.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 -- 172.21.15.140:0/297244288 shutdown_connections 2022-07-21T01:56:38.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.580+0000 7fd817fff700 1 -- 172.21.15.140:0/297244288 wait complete. 2022-07-21T01:56:38.594 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:56:38.595 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:56:38.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.692+0000 7f990a5e5700 1 Processor -- start 2022-07-21T01:56:38.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.692+0000 7f990a5e5700 1 -- start start 2022-07-21T01:56:38.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f990a5e5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9904098ae0 0x7f990409a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f990a5e5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f99040ac850 0x7f990413d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f990a5e5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9904062170 con 0x7f9904061d90 2022-07-21T01:56:38.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f990a5e5700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990405d530 con 0x7f99040ac850 2022-07-21T01:56:38.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f990a5e5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99040abe70 con 0x7f9904098ae0 2022-07-21T01:56:38.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f9909de4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f99040ac850 0x7f990413d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f9909de4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f99040ac850 0x7f990413d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39748/0 (socket says 172.21.15.140:39748) 2022-07-21T01:56:38.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f9909de4700 1 -- 172.21.15.140:0/1742256133 learned_addr learned my addr 172.21.15.140:0/1742256133 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:38.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f99095e3700 1 --1- 172.21.15.140:0/1742256133 >> v1:172.21.15.140:6789/0 conn(0x7f9904061d90 0x7f9904099fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53890/0 (socket says 172.21.15.140:53890) 2022-07-21T01:56:38.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f9908de2700 1 --2- 172.21.15.140:0/1742256133 >> v2:172.21.15.152:3300/0 conn(0x7f9904098ae0 0x7f990409a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f9909de4700 1 -- 172.21.15.140:0/1742256133 >> v2:172.21.15.152:3300/0 conn(0x7f9904098ae0 msgr2=0x7f990409a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:38.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f9909de4700 1 --2- 172.21.15.140:0/1742256133 >> v2:172.21.15.152:3300/0 conn(0x7f9904098ae0 0x7f990409a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f9909de4700 1 -- 172.21.15.140:0/1742256133 >> v1:172.21.15.140:6789/0 conn(0x7f9904061d90 legacy=0x7f9904099fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f9909de4700 1 -- 172.21.15.140:0/1742256133 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99040ac110 con 0x7f99040ac850 2022-07-21T01:56:38.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f9909de4700 1 --2- 172.21.15.140:0/1742256133 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99040ac850 0x7f990413d4f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f98fc0032f0 tx=0x7f98fc009290).ready entity=mon.0 client_cookie=afe0059bf9a334b0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:38.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f98f3fff700 1 -- 172.21.15.140:0/1742256133 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98fc00b870 con 0x7f99040ac850 2022-07-21T01:56:38.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f98f3fff700 1 -- 172.21.15.140:0/1742256133 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc00b9d0 con 0x7f99040ac850 2022-07-21T01:56:38.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.696+0000 7f98f3fff700 1 -- 172.21.15.140:0/1742256133 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98fc00bb40 con 0x7f99040ac850 2022-07-21T01:56:38.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.700+0000 7f990a5e5700 1 -- 172.21.15.140:0/1742256133 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99040ac850 msgr2=0x7f990413d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.700+0000 7f990a5e5700 1 --2- 172.21.15.140:0/1742256133 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99040ac850 0x7f990413d4f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f98fc0032f0 tx=0x7f98fc009290).stop 2022-07-21T01:56:38.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.700+0000 7f990a5e5700 1 -- 172.21.15.140:0/1742256133 shutdown_connections 2022-07-21T01:56:38.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.700+0000 7f990a5e5700 1 --2- 172.21.15.140:0/1742256133 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99040ac850 0x7f990413d4f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.700+0000 7f990a5e5700 1 --2- 172.21.15.140:0/1742256133 >> v2:172.21.15.152:3300/0 conn(0x7f9904098ae0 0x7f990409a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.700+0000 7f990a5e5700 1 -- 172.21.15.140:0/1742256133 >> 172.21.15.140:0/1742256133 conn(0x7f99040b2ca0 msgr2=0x7f99040b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:38.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.700+0000 7f990a5e5700 1 -- 172.21.15.140:0/1742256133 shutdown_connections 2022-07-21T01:56:38.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.700+0000 7f990a5e5700 1 -- 172.21.15.140:0/1742256133 wait complete. 2022-07-21T01:56:38.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.700+0000 7f990a5e5700 1 Processor -- start 2022-07-21T01:56:38.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.700+0000 7f990a5e5700 1 -- start start 2022-07-21T01:56:38.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f990a5e5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9904061d90 0x7f99040afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f990a5e5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9904098ae0 0x7f9904153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f990a5e5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99040ac850 0x7f9904153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f990a5e5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99040aec80 con 0x7f99040ac850 2022-07-21T01:56:38.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f990a5e5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990409c1f0 con 0x7f9904061d90 2022-07-21T01:56:38.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f990a5e5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990409bf90 con 0x7f9904098ae0 2022-07-21T01:56:38.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f9909de4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99040ac850 0x7f9904153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f9909de4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99040ac850 0x7f9904153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39754/0 (socket says 172.21.15.140:39754) 2022-07-21T01:56:38.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f9909de4700 1 -- 172.21.15.140:0/976936915 learned_addr learned my addr 172.21.15.140:0/976936915 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:38.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f99095e3700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9904061d90 0x7f99040afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f9908de2700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9904098ae0 0x7f9904153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f9909de4700 1 -- 172.21.15.140:0/976936915 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9904098ae0 msgr2=0x7f9904153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f9909de4700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9904098ae0 0x7f9904153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f9909de4700 1 -- 172.21.15.140:0/976936915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9904061d90 msgr2=0x7f99040afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f9909de4700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9904061d90 0x7f99040afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f9909de4700 1 -- 172.21.15.140:0/976936915 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc009530 con 0x7f99040ac850 2022-07-21T01:56:38.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f9909de4700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99040ac850 0x7f9904153730 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7f99040ae2f0 tx=0x7f98fc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:38.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f98f27fc700 1 -- 172.21.15.140:0/976936915 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98fc00b870 con 0x7f99040ac850 2022-07-21T01:56:38.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f98f27fc700 1 -- 172.21.15.140:0/976936915 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc021440 con 0x7f99040ac850 2022-07-21T01:56:38.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f990a5e5700 1 -- 172.21.15.140:0/976936915 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990409aeb0 con 0x7f99040ac850 2022-07-21T01:56:38.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f98f27fc700 1 -- 172.21.15.140:0/976936915 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f98fc0217a0 con 0x7f99040ac850 2022-07-21T01:56:38.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.704+0000 7f990a5e5700 1 -- 172.21.15.140:0/976936915 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99040b03a0 con 0x7f99040ac850 2022-07-21T01:56:38.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.708+0000 7f98f27fc700 1 -- 172.21.15.140:0/976936915 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f98fc00f070 con 0x7f99040ac850 2022-07-21T01:56:38.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.708+0000 7f990a5e5700 1 -- 172.21.15.140:0/976936915 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99040ac3b0 con 0x7f99040ac850 2022-07-21T01:56:38.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.708+0000 7f98f27fc700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98e4053aa0 0x7f98e4055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:38.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.708+0000 7f98f27fc700 1 -- 172.21.15.140:0/976936915 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f98fc06d670 con 0x7f99040ac850 2022-07-21T01:56:38.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.708+0000 7f99095e3700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98e4053aa0 0x7f98e4055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:38.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.708+0000 7f99095e3700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98e4053aa0 0x7f98e4055f60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f990000d000 tx=0x7f9900007ae0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:38.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.712+0000 7f98f27fc700 1 -- 172.21.15.140:0/976936915 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f98fc044590 con 0x7f99040ac850 2022-07-21T01:56:38.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 -- 172.21.15.140:0/976936915 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99040574e0 con 0x7f99040ac850 2022-07-21T01:56:38.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f98f27fc700 1 -- 172.21.15.140:0/976936915 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f98fc015090 con 0x7f99040ac850 2022-07-21T01:56:38.874 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:38.874 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":53,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/223182184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":223182184},{"type":"v1","addr":"172.21.15.140:6835","nonce":223182184}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":50}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:56:29.930618+0000","modified":"2022-07-21T01:56:31.631480+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7662},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7662":{"gid":7662,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.152:6833/456689269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":456689269},{"type":"v1","addr":"172.21.15.152:6833","nonce":456689269}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2022-07-21T01:56:38.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 -- 172.21.15.140:0/976936915 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98e4053aa0 msgr2=0x7f98e4055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98e4053aa0 0x7f98e4055f60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f990000d000 tx=0x7f9900007ae0).stop 2022-07-21T01:56:38.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 -- 172.21.15.140:0/976936915 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99040ac850 msgr2=0x7f9904153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:38.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99040ac850 0x7f9904153730 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7f99040ae2f0 tx=0x7f98fc016000).stop 2022-07-21T01:56:38.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 -- 172.21.15.140:0/976936915 shutdown_connections 2022-07-21T01:56:38.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f98e4053aa0 0x7f98e4055f60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f99040ac850 0x7f9904153730 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9904098ae0 0x7f9904153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 --2- 172.21.15.140:0/976936915 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9904061d90 0x7f99040afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:38.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 -- 172.21.15.140:0/976936915 >> 172.21.15.140:0/976936915 conn(0x7f99040b2ca0 msgr2=0x7f990413c910 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:38.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 -- 172.21.15.140:0/976936915 shutdown_connections 2022-07-21T01:56:38.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:38.872+0000 7f990a5e5700 1 -- 172.21.15.140:0/976936915 wait complete. 2022-07-21T01:56:38.882 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 53 2022-07-21T01:56:39.089 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:56:29.930618+0000', 'modified': '2022-07-21T01:56:31.631480+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7662}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7662': {'gid': 7662, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.152:6833/456689269', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6832', 'nonce': 456689269}, {'type': 'v1', 'addr': '172.21.15.152:6833', 'nonce': 456689269}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:56:39.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:56:39.122 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2022-07-21T01:56:39.238 INFO:teuthology.orchestra.run.smithi152.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":7662,"want_state":"up:active","state":"up:active","rank_uptime":9.296150184,"mdsmap_epoch":53,"osdmap_epoch":113,"osdmap_epoch_barrier":113,"uptime":14.319414331000001} 2022-07-21T01:56:39.239 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 7662, "mdsmap_epoch": 53, "osdmap_epoch": 113, "osdmap_epoch_barrier": 113, "rank_uptime": 9.296150184, "state": "up:active", "uptime": 14.319414331, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:56:39.239 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:56:39.240 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-07-21T01:56:39.240 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-07-21T01:56:39.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:39.241 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:39.254 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:56:39.314 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:56:39.314 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:56:39.327 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:56:39.328 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:56:39.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:39.330 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:56:39.335 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:56:39.335 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:39.338 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:56:39.355 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:56:39.355 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:56:39.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2022-07-21T01:56:39.405+0000 7f974b26e080 -1 init, newargv = 0x5625f2500200 newargc=9ceph-fuse[22734]: starting ceph client 2022-07-21T01:56:39.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr: 2022-07-21T01:56:39.415 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:56:39.416 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:56:39.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:39.418 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:56:39.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22734]: starting fuse 2022-07-21T01:56:39.424 INFO:teuthology.orchestra.run.smithi192.stdout:52 2022-07-21T01:56:39.424 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:56:39.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:39.426 DEBUG:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2022-07-21T01:56:39.427 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2022-07-21T01:56:39.428 DEBUG:teuthology.orchestra.run.smithi192:> import re 2022-07-21T01:56:39.429 DEBUG:teuthology.orchestra.run.smithi192:> import os 2022-07-21T01:56:39.430 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2022-07-21T01:56:39.431 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:56:39.432 DEBUG:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2022-07-21T01:56:39.433 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-07-21T01:56:39.433 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2022-07-21T01:56:39.434 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:56:39.435 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2022-07-21T01:56:39.436 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2022-07-21T01:56:39.437 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2022-07-21T01:56:39.437 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2022-07-21T01:56:39.439 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:56:39.439 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2022-07-21T01:56:39.440 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:56:39.440 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:56:39.440 DEBUG:teuthology.orchestra.run.smithi192:> return f 2022-07-21T01:56:39.441 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:56:39.441 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:56:39.442 DEBUG:teuthology.orchestra.run.smithi192:> print(find_socket("client.1")) 2022-07-21T01:56:39.442 DEBUG:teuthology.orchestra.run.smithi192:> ' 2022-07-21T01:56:39.485 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.22734.asok 2022-07-21T01:56:39.490 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22734.asok 2022-07-21T01:56:39.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:39.491 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22734.asok status 2022-07-21T01:56:39.626 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2022-07-21T01:56:39.626 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2022-07-21T01:56:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:56:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:56:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2022-07-21T01:56:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2022-07-21T01:56:39.627 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-07-21T01:56:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "22734", 2022-07-21T01:56:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2022-07-21T01:56:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:56:39.628 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2022-07-21T01:56:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2022-07-21T01:56:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 7872, 2022-07-21T01:56:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2022-07-21T01:56:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2022-07-21T01:56:39.629 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2022-07-21T01:56:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 7872 2022-07-21T01:56:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:56:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:56:39.630 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:56:39.631 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:56:39.631 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1037557468 2022-07-21T01:56:39.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2022-07-21T01:56:39.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:56:39.632 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:56:39.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:56:39.632 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:56:39.632 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1037557468 2022-07-21T01:56:39.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:56:39.633 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.7872 172.21.15.192:0/1037557468", 2022-07-21T01:56:39.633 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/1037557468", 2022-07-21T01:56:39.633 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2022-07-21T01:56:39.633 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 53, 2022-07-21T01:56:39.634 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 113, 2022-07-21T01:56:39.634 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:56:39.634 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false, 2022-07-21T01:56:39.634 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2022-07-21T01:56:39.634 INFO:teuthology.orchestra.run.smithi192.stdout:} 2022-07-21T01:56:39.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:39.636 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:39.637 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:39.643 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:56:39.644 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:56:39.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:39.645 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:39.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:39.660 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:39.660 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:39.668 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:56:39.670 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:56:39.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:39.671 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:39.671 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:39.679 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:56:39.680 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:56:39.681 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:56:39.681 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:56:39.681 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:39.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23211]: fuse finished with error 0 and tester_r 0 2022-07-21T01:56:39.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:39.700 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:39.700 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:39.707 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:56:39.708 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:56:39.708 INFO:teuthology.orchestra.run:waiting for 900 2022-07-21T01:56:45.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:56:45.811 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:56:45.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:45.821 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:45.821 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:45.830 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:56:45.831 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:56:45.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:45.832 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:45.832 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:45.840 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:56:45.841 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:56:45.842 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2022-07-21T01:56:45.842 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:56:45.843 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:45.899 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22734]: fuse finished with error 0 and tester_r 0 2022-07-21T01:56:45.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:56:45.902 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:56:45.902 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:45.908 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2022-07-21T01:56:45.910 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:56:45.910 INFO:teuthology.orchestra.run:waiting for 900 2022-07-21T01:56:46.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:56:46.011 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:56:46.019 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:56:46.020 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:56:46.020 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:56:46.021 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:56:46.035 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:56:46.036 DEBUG:teuthology.parallel:result is None 2022-07-21T01:56:46.037 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:56:46.037 DEBUG:teuthology.parallel:result is None 2022-07-21T01:56:46.038 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2022-07-21T01:56:46.039 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2022-07-21T01:56:46.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.176+0000 7f7d216ed700 1 Processor -- start 2022-07-21T01:56:46.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.176+0000 7f7d216ed700 1 -- start start 2022-07-21T01:56:46.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7d1c0b3700 0x7f7d1c098a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7d1c0abd10 0x7f7d1c096290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d1c0b3ae0 con 0x7f7d1c062230 2022-07-21T01:56:46.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1c098070 con 0x7f7d1c0abd10 2022-07-21T01:56:46.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1c098310 con 0x7f7d1c0b3700 2022-07-21T01:56:46.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d20eec700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7d1c0abd10 0x7f7d1c096290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d20eec700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7d1c0abd10 0x7f7d1c096290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39762/0 (socket says 172.21.15.140:39762) 2022-07-21T01:56:46.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d20eec700 1 -- 172.21.15.140:0/67108953 learned_addr learned my addr 172.21.15.140:0/67108953 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:46.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d1bfff700 1 --2- 172.21.15.140:0/67108953 >> v2:172.21.15.152:3300/0 conn(0x7f7d1c0b3700 0x7f7d1c098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ffcb94700 1 Processor -- start 2022-07-21T01:56:46.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ffcb94700 1 -- start start 2022-07-21T01:56:46.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d20eec700 1 -- 172.21.15.140:0/67108953 >> v2:172.21.15.152:3300/0 conn(0x7f7d1c0b3700 msgr2=0x7f7d1c098a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:46.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d20eec700 1 --2- 172.21.15.140:0/67108953 >> v2:172.21.15.152:3300/0 conn(0x7f7d1c0b3700 0x7f7d1c098a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:46.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d20eec700 1 -- 172.21.15.140:0/67108953 >> v1:172.21.15.152:6789/0 conn(0x7f7d1c062230 legacy=0x7f7d1c095b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:46.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d20eec700 1 -- 172.21.15.140:0/67108953 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c0985b0 con 0x7f7d1c0abd10 2022-07-21T01:56:46.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d20eec700 1 --2- 172.21.15.140:0/67108953 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d1c0abd10 0x7f7d1c096290 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f7d140032f0 tx=0x7f7d1400b530).ready entity=mon.0 client_cookie=c73ee646c80a4fa server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:46.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d1affd700 1 -- 172.21.15.140:0/67108953 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d1400ddf0 con 0x7f7d1c0abd10 2022-07-21T01:56:46.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d1affd700 1 -- 172.21.15.140:0/67108953 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d14010470 con 0x7f7d1c0abd10 2022-07-21T01:56:46.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d1affd700 1 -- 172.21.15.140:0/67108953 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d140107d0 con 0x7f7d1c0abd10 2022-07-21T01:56:46.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ffcb94700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80b5970 0x7f1ff813c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ffcb94700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1ff80b1060 0x7f1ff813cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ffcb94700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff80ada70 con 0x7f1ff80acae0 2022-07-21T01:56:46.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ffcb94700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff80b5d50 con 0x7f1ff80b1060 2022-07-21T01:56:46.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ffcb94700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80af8c0 con 0x7f1ff80b5970 2022-07-21T01:56:46.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 -- 172.21.15.140:0/67108953 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d1c0abd10 msgr2=0x7f7d1c096290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:46.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 --2- 172.21.15.140:0/67108953 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d1c0abd10 0x7f7d1c096290 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f7d140032f0 tx=0x7f7d1400b530).stop 2022-07-21T01:56:46.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 -- 172.21.15.140:0/67108953 shutdown_connections 2022-07-21T01:56:46.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff77fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80b5970 0x7f1ff813c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff6ffd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1ff80b1060 0x7f1ff813cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 --2- 172.21.15.140:0/67108953 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d1c0abd10 0x7f7d1c096290 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:46.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 --2- 172.21.15.140:0/67108953 >> v2:172.21.15.152:3300/0 conn(0x7f7d1c0b3700 0x7f7d1c098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:46.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 -- 172.21.15.140:0/67108953 >> 172.21.15.140:0/67108953 conn(0x7f7d1c0b6ce0 msgr2=0x7f7d1c0b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:46.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff77fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80b5970 0x7f1ff813c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36494/0 (socket says 172.21.15.140:36494) 2022-07-21T01:56:46.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff77fe700 1 -- 172.21.15.140:0/2453000960 learned_addr learned my addr 172.21.15.140:0/2453000960 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:46.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff7fff700 1 --1- 172.21.15.140:0/2453000960 >> v1:172.21.15.152:6789/0 conn(0x7f1ff80acae0 0x7f1ff813f8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:55028/0 (socket says 172.21.15.140:55028) 2022-07-21T01:56:46.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 -- 172.21.15.140:0/67108953 shutdown_connections 2022-07-21T01:56:46.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f7d216ed700 1 -- 172.21.15.140:0/67108953 wait complete. 2022-07-21T01:56:46.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff77fe700 1 -- 172.21.15.140:0/2453000960 >> v2:172.21.15.152:3300/0 conn(0x7f1ff80b1060 msgr2=0x7f1ff813cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:46.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff77fe700 1 --2- 172.21.15.140:0/2453000960 >> v2:172.21.15.152:3300/0 conn(0x7f1ff80b1060 0x7f1ff813cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:46.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff77fe700 1 -- 172.21.15.140:0/2453000960 >> v1:172.21.15.152:6789/0 conn(0x7f1ff80acae0 legacy=0x7f1ff813f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:46.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff77fe700 1 -- 172.21.15.140:0/2453000960 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff80adcd0 con 0x7f1ff80b5970 2022-07-21T01:56:46.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff77fe700 1 --2- 172.21.15.140:0/2453000960 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80b5970 0x7f1ff813c830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f1fe8002f70 tx=0x7f1fe8006eb0).ready entity=mon.2 client_cookie=963333729e22dc35 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:46.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff67fc700 1 -- 172.21.15.140:0/2453000960 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fe800d570 con 0x7f1ff80b5970 2022-07-21T01:56:46.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff67fc700 1 -- 172.21.15.140:0/2453000960 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe8000e90 con 0x7f1ff80b5970 2022-07-21T01:56:46.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.180+0000 7f1ff67fc700 1 -- 172.21.15.140:0/2453000960 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1fe800d900 con 0x7f1ff80b5970 2022-07-21T01:56:46.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d216ed700 1 Processor -- start 2022-07-21T01:56:46.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d216ed700 1 -- start start 2022-07-21T01:56:46.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 -- 172.21.15.140:0/2453000960 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80b5970 msgr2=0x7f1ff813c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:46.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 --2- 172.21.15.140:0/2453000960 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80b5970 0x7f1ff813c830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f1fe8002f70 tx=0x7f1fe8006eb0).stop 2022-07-21T01:56:46.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 -- 172.21.15.140:0/2453000960 shutdown_connections 2022-07-21T01:56:46.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 --2- 172.21.15.140:0/2453000960 >> v2:172.21.15.152:3300/0 conn(0x7f1ff80b1060 0x7f1ff813cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:46.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 --2- 172.21.15.140:0/2453000960 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80b5970 0x7f1ff813c830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:46.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 -- 172.21.15.140:0/2453000960 >> 172.21.15.140:0/2453000960 conn(0x7f1ff8063490 msgr2=0x7f1ff80b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:46.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 -- 172.21.15.140:0/2453000960 shutdown_connections 2022-07-21T01:56:46.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 -- 172.21.15.140:0/2453000960 wait complete. 2022-07-21T01:56:46.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d216ed700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d1c062230 0x7f7d1c14e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d216ed700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d1c0abd10 0x7f7d1c0ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d216ed700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c0b3700 0x7f7d1c0aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d216ed700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c099d50 con 0x7f7d1c0abd10 2022-07-21T01:56:46.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d216ed700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c099810 con 0x7f7d1c0b3700 2022-07-21T01:56:46.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d216ed700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c099ab0 con 0x7f7d1c062230 2022-07-21T01:56:46.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d1b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d1c0abd10 0x7f7d1c0ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d1b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d1c0abd10 0x7f7d1c0ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39772/0 (socket says 172.21.15.140:39772) 2022-07-21T01:56:46.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d1b7fe700 1 -- 172.21.15.140:0/495783450 learned_addr learned my addr 172.21.15.140:0/495783450 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:46.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d1bfff700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d1c062230 0x7f7d1c14e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d20eec700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c0b3700 0x7f7d1c0aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d1b7fe700 1 -- 172.21.15.140:0/495783450 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d1c062230 msgr2=0x7f7d1c14e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:46.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d1b7fe700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d1c062230 0x7f7d1c14e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:46.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d1b7fe700 1 -- 172.21.15.140:0/495783450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c0b3700 msgr2=0x7f7d1c0aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:46.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d1b7fe700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c0b3700 0x7f7d1c0aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:46.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d1b7fe700 1 -- 172.21.15.140:0/495783450 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1400b7d0 con 0x7f7d1c0abd10 2022-07-21T01:56:46.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d1b7fe700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d1c0abd10 0x7f7d1c0ae690 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f7d1c097070 tx=0x7f7d10007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:46.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d197fa700 1 -- 172.21.15.140:0/495783450 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d10007030 con 0x7f7d1c0abd10 2022-07-21T01:56:46.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d197fa700 1 -- 172.21.15.140:0/495783450 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d10007190 con 0x7f7d1c0abd10 2022-07-21T01:56:46.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d197fa700 1 -- 172.21.15.140:0/495783450 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d10006510 con 0x7f7d1c0abd10 2022-07-21T01:56:46.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d216ed700 1 -- 172.21.15.140:0/495783450 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c066d70 con 0x7f7d1c0abd10 2022-07-21T01:56:46.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d216ed700 1 -- 172.21.15.140:0/495783450 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c067270 con 0x7f7d1c0abd10 2022-07-21T01:56:46.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 Processor -- start 2022-07-21T01:56:46.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 -- start start 2022-07-21T01:56:46.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f7d216ed700 1 -- 172.21.15.140:0/495783450 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c09a450 con 0x7f7d1c0abd10 2022-07-21T01:56:46.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80acae0 0x7f1ff8141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ff80b1060 0x7f1ff8063ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff80b5970 0x7f1ff81410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff813e570 con 0x7f1ff80b5970 2022-07-21T01:56:46.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8142b60 con 0x7f1ff80b1060 2022-07-21T01:56:46.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.184+0000 7f1ffcb94700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8064190 con 0x7f1ff80acae0 2022-07-21T01:56:46.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f1ff7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff80b5970 0x7f1ff81410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f1ff7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff80b5970 0x7f1ff81410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39780/0 (socket says 172.21.15.140:39780) 2022-07-21T01:56:46.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f1ff7fff700 1 -- 172.21.15.140:0/3084406195 learned_addr learned my addr 172.21.15.140:0/3084406195 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:46.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f1ff77fe700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80acae0 0x7f1ff8141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f1ff6ffd700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ff80b1060 0x7f1ff8063ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f1ff7fff700 1 -- 172.21.15.140:0/3084406195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80acae0 msgr2=0x7f1ff8141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:46.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f1ff7fff700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80acae0 0x7f1ff8141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:46.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f1ff7fff700 1 -- 172.21.15.140:0/3084406195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ff80b1060 msgr2=0x7f1ff8063ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:46.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f1ff7fff700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ff80b1060 0x7f1ff8063ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:46.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f1ff7fff700 1 -- 172.21.15.140:0/3084406195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe800b1e0 con 0x7f1ff80b5970 2022-07-21T01:56:46.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f7d197fa700 1 -- 172.21.15.140:0/495783450 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7d10010030 con 0x7f7d1c0abd10 2022-07-21T01:56:46.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f1ff7fff700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff80b5970 0x7f1ff81410b0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f1ff8062550 tx=0x7f1ff0007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:46.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f7d197fa700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cfc0539e0 0x7f7cfc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f7d197fa700 1 -- 172.21.15.140:0/495783450 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7d1006ba30 con 0x7f7d1c0abd10 2022-07-21T01:56:46.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f7d1bfff700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cfc0539e0 0x7f7cfc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.188+0000 7f7d1bfff700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cfc0539e0 0x7f7cfc055ea0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f7d0c00b000 tx=0x7f7d0c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:46.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.192+0000 7f1ff4ff9700 1 -- 172.21.15.140:0/3084406195 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ff0004ab0 con 0x7f1ff80b5970 2022-07-21T01:56:46.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.192+0000 7f1ff4ff9700 1 -- 172.21.15.140:0/3084406195 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff0004c10 con 0x7f1ff80b5970 2022-07-21T01:56:46.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.192+0000 7f7d197fa700 1 -- 172.21.15.140:0/495783450 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7d10042850 con 0x7f7d1c0abd10 2022-07-21T01:56:46.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.192+0000 7f1ffcb94700 1 -- 172.21.15.140:0/3084406195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff8142620 con 0x7f1ff80b5970 2022-07-21T01:56:46.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.192+0000 7f1ff4ff9700 1 -- 172.21.15.140:0/3084406195 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ff0007170 con 0x7f1ff80b5970 2022-07-21T01:56:46.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.192+0000 7f1ffcb94700 1 -- 172.21.15.140:0/3084406195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff8141620 con 0x7f1ff80b5970 2022-07-21T01:56:46.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.192+0000 7f1ffcb94700 1 -- 172.21.15.140:0/3084406195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff813f360 con 0x7f1ff80b5970 2022-07-21T01:56:46.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.192+0000 7f1ff4ff9700 1 -- 172.21.15.140:0/3084406195 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1ff00072d0 con 0x7f1ff80b5970 2022-07-21T01:56:46.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.192+0000 7f1ff4ff9700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053ac0 0x7f1fd8055f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:46.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.196+0000 7f1ff4ff9700 1 -- 172.21.15.140:0/3084406195 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1ff006bb30 con 0x7f1ff80b5970 2022-07-21T01:56:46.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.196+0000 7f1ff77fe700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053ac0 0x7f1fd8055f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:46.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.196+0000 7f1ff77fe700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053ac0 0x7f1fd8055f80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f1fe801c000 tx=0x7f1fe8015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:46.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.200+0000 7f1ff4ff9700 1 -- 172.21.15.140:0/3084406195 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1ff813f360 con 0x7f1ff80b5970 2022-07-21T01:56:46.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.356+0000 7f7d216ed700 1 -- 172.21.15.140:0/495783450 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f7d1c0af420 con 0x7f7d1c0abd10 2022-07-21T01:56:46.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:46.368+0000 7f1ffcb94700 1 -- 172.21.15.140:0/3084406195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f1ff813f360 con 0x7f1ff80b5970 2022-07-21T01:56:47.107 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:56:47.104+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:56:47.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.112+0000 7f1ff4ff9700 1 -- 172.21.15.140:0/3084406195 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7659 v54) v1 ==== 95+0+0 (secure 0 0 0) 0x7f1ff813f360 con 0x7f1ff80b5970 2022-07-21T01:56:47.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.112+0000 7f7d197fa700 1 -- 172.21.15.140:0/495783450 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 7662 v54) v1 ==== 95+0+0 (secure 0 0 0) 0x7f7d10045e70 con 0x7f7d1c0abd10 2022-07-21T01:56:47.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 -- 172.21.15.140:0/495783450 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cfc0539e0 msgr2=0x7f7cfc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:47.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cfc0539e0 0x7f7cfc055ea0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f7d0c00b000 tx=0x7f7d0c006df0).stop 2022-07-21T01:56:47.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 -- 172.21.15.140:0/495783450 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d1c0abd10 msgr2=0x7f7d1c0ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:47.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d1c0abd10 0x7f7d1c0ae690 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f7d1c097070 tx=0x7f7d10007c10).stop 2022-07-21T01:56:47.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 -- 172.21.15.140:0/495783450 shutdown_connections 2022-07-21T01:56:47.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7cfc0539e0 0x7f7cfc055ea0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c0b3700 0x7f7d1c0aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7d1c0abd10 0x7f7d1c0ae690 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 --2- 172.21.15.140:0/495783450 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d1c062230 0x7f7d1c14e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 -- 172.21.15.140:0/495783450 >> 172.21.15.140:0/495783450 conn(0x7f7d1c0b6ce0 msgr2=0x7f7d1c141b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:47.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f1ffcb94700 1 -- 172.21.15.140:0/3084406195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053ac0 msgr2=0x7f1fd8055f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:47.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f1ffcb94700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053ac0 0x7f1fd8055f80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f1fe801c000 tx=0x7f1fe8015000).stop 2022-07-21T01:56:47.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 -- 172.21.15.140:0/495783450 shutdown_connections 2022-07-21T01:56:47.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f1ffcb94700 1 -- 172.21.15.140:0/3084406195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff80b5970 msgr2=0x7f1ff81410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:47.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f1ffcb94700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff80b5970 0x7f1ff81410b0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f1ff8062550 tx=0x7f1ff0007f30).stop 2022-07-21T01:56:47.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.116+0000 7f7d216ed700 1 -- 172.21.15.140:0/495783450 wait complete. 2022-07-21T01:56:47.125 INFO:teuthology.orchestra.run.smithi140.stderr:failed mds gid 7662 2022-07-21T01:56:47.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.120+0000 7f1ffcb94700 1 -- 172.21.15.140:0/3084406195 shutdown_connections 2022-07-21T01:56:47.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.120+0000 7f1ffcb94700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1fd8053ac0 0x7f1fd8055f80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.120+0000 7f1ffcb94700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1ff80b5970 0x7f1ff81410b0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.120+0000 7f1ffcb94700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1ff80b1060 0x7f1ff8063ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.120+0000 7f1ffcb94700 1 --2- 172.21.15.140:0/3084406195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1ff80acae0 0x7f1ff8141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.120+0000 7f1ffcb94700 1 -- 172.21.15.140:0/3084406195 >> 172.21.15.140:0/3084406195 conn(0x7f1ff8063490 msgr2=0x7f1ff80b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:47.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.120+0000 7f1ffcb94700 1 -- 172.21.15.140:0/3084406195 shutdown_connections 2022-07-21T01:56:47.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.120+0000 7f1ffcb94700 1 -- 172.21.15.140:0/3084406195 wait complete. 2022-07-21T01:56:47.128 INFO:teuthology.orchestra.run.smithi140.stderr:failed mds gid 7659 2022-07-21T01:56:47.136 DEBUG:teuthology.parallel:result is '' 2022-07-21T01:56:47.137 DEBUG:teuthology.parallel:result is '' 2022-07-21T01:56:47.138 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:56:47.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.236+0000 7f824cfe2700 1 Processor -- start 2022-07-21T01:56:47.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.236+0000 7f824cfe2700 1 -- start start 2022-07-21T01:56:47.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f82480ac040 0x7f824813f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:47.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f82480acd70 0x7f82480ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:47.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82480ac6e0 con 0x7f82480b1480 2022-07-21T01:56:47.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824805ff50 con 0x7f82480ac040 2022-07-21T01:56:47.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82480ac480 con 0x7f82480acd70 2022-07-21T01:56:47.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f8245d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f82480ac040 0x7f824813f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:47.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f8245d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f82480ac040 0x7f824813f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39790/0 (socket says 172.21.15.140:39790) 2022-07-21T01:56:47.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f8245d9b700 1 -- 172.21.15.140:0/3612197956 learned_addr learned my addr 172.21.15.140:0/3612197956 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:47.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f8246d9d700 1 --2- 172.21.15.140:0/3612197956 >> v2:172.21.15.152:3300/0 conn(0x7f82480acd70 0x7f82480ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:47.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f8245d9b700 1 -- 172.21.15.140:0/3612197956 >> v2:172.21.15.152:3300/0 conn(0x7f82480acd70 msgr2=0x7f82480ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:47.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f8245d9b700 1 --2- 172.21.15.140:0/3612197956 >> v2:172.21.15.152:3300/0 conn(0x7f82480acd70 0x7f82480ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f8245d9b700 1 -- 172.21.15.140:0/3612197956 >> v1:172.21.15.152:6789/0 conn(0x7f82480b1480 legacy=0x7f824805f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:47.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f8245d9b700 1 -- 172.21.15.140:0/3612197956 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824813f230 con 0x7f82480ac040 2022-07-21T01:56:47.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f8245d9b700 1 --2- 172.21.15.140:0/3612197956 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82480ac040 0x7f824813f970 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f823c002f70 tx=0x7f823c006eb0).ready entity=mon.0 client_cookie=cac16ce396a5032 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:47.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824559a700 1 -- 172.21.15.140:0/3612197956 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f823c014070 con 0x7f82480ac040 2022-07-21T01:56:47.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824559a700 1 -- 172.21.15.140:0/3612197956 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823c000e90 con 0x7f82480ac040 2022-07-21T01:56:47.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824559a700 1 -- 172.21.15.140:0/3612197956 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f823c00e930 con 0x7f82480ac040 2022-07-21T01:56:47.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 -- 172.21.15.140:0/3612197956 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82480ac040 msgr2=0x7f824813f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:47.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 --2- 172.21.15.140:0/3612197956 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82480ac040 0x7f824813f970 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f823c002f70 tx=0x7f823c006eb0).stop 2022-07-21T01:56:47.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 -- 172.21.15.140:0/3612197956 shutdown_connections 2022-07-21T01:56:47.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 --2- 172.21.15.140:0/3612197956 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82480ac040 0x7f824813f970 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 --2- 172.21.15.140:0/3612197956 >> v2:172.21.15.152:3300/0 conn(0x7f82480acd70 0x7f82480ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 -- 172.21.15.140:0/3612197956 >> 172.21.15.140:0/3612197956 conn(0x7f82480b23d0 msgr2=0x7f82480b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:47.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 -- 172.21.15.140:0/3612197956 shutdown_connections 2022-07-21T01:56:47.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.240+0000 7f824cfe2700 1 -- 172.21.15.140:0/3612197956 wait complete. 2022-07-21T01:56:47.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f824cfe2700 1 Processor -- start 2022-07-21T01:56:47.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f824cfe2700 1 -- start start 2022-07-21T01:56:47.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f824cfe2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82480ac040 0x7f82480631c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:47.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f824cfe2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82480acd70 0x7f824813af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:47.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f824cfe2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82480b1480 0x7f824814e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:47.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f824cfe2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8248151d50 con 0x7f82480b1480 2022-07-21T01:56:47.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f824cfe2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824814da70 con 0x7f82480ac040 2022-07-21T01:56:47.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f824cfe2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824814dd10 con 0x7f82480acd70 2022-07-21T01:56:47.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f8246d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82480b1480 0x7f824814e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:47.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f8246d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82480b1480 0x7f824814e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39796/0 (socket says 172.21.15.140:39796) 2022-07-21T01:56:47.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f8246d9d700 1 -- 172.21.15.140:0/1170943529 learned_addr learned my addr 172.21.15.140:0/1170943529 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:47.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f824659c700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82480ac040 0x7f82480631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:47.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f8245d9b700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82480acd70 0x7f824813af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:47.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f8246d9d700 1 -- 172.21.15.140:0/1170943529 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82480acd70 msgr2=0x7f824813af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:47.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f8246d9d700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82480acd70 0x7f824813af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f8246d9d700 1 -- 172.21.15.140:0/1170943529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82480ac040 msgr2=0x7f82480631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:47.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.244+0000 7f8246d9d700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82480ac040 0x7f82480631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:47.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f8246d9d700 1 -- 172.21.15.140:0/1170943529 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c00c2f0 con 0x7f82480b1480 2022-07-21T01:56:47.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f824659c700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82480ac040 0x7f82480631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:47.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f8246d9d700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82480b1480 0x7f824814e1b0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f8240006e00 tx=0x7f82400091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:47.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f82377fe700 1 -- 172.21.15.140:0/1170943529 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f824000ed20 con 0x7f82480b1480 2022-07-21T01:56:47.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f82377fe700 1 -- 172.21.15.140:0/1170943529 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824000ee80 con 0x7f82480b1480 2022-07-21T01:56:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f824cfe2700 1 -- 172.21.15.140:0/1170943529 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82481519d0 con 0x7f82480b1480 2022-07-21T01:56:47.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f82377fe700 1 -- 172.21.15.140:0/1170943529 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f824001a660 con 0x7f82480b1480 2022-07-21T01:56:47.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f824cfe2700 1 -- 172.21.15.140:0/1170943529 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824813c9c0 con 0x7f82480b1480 2022-07-21T01:56:47.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f824cfe2700 1 -- 172.21.15.140:0/1170943529 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824813f4d0 con 0x7f82480b1480 2022-07-21T01:56:47.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f82377fe700 1 -- 172.21.15.140:0/1170943529 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f824001a7c0 con 0x7f82480b1480 2022-07-21T01:56:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f82377fe700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8224053980 0x7f8224055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:47.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f82377fe700 1 -- 172.21.15.140:0/1170943529 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f824006ea60 con 0x7f82480b1480 2022-07-21T01:56:47.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f824659c700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8224053980 0x7f8224055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:47.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.248+0000 7f824659c700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8224053980 0x7f8224055e40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f823800d000 tx=0x7f8238009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:47.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.256+0000 7f82377fe700 1 -- 172.21.15.140:0/1170943529 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8240045890 con 0x7f82480b1480 2022-07-21T01:56:47.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:47.416+0000 7f824cfe2700 1 -- 172.21.15.140:0/1170943529 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f824814ded0 con 0x7f82480b1480 2022-07-21T01:56:48.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.128+0000 7f82377fe700 1 -- 172.21.15.140:0/1170943529 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8240048eb0 con 0x7f82480b1480 2022-07-21T01:56:48.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.132+0000 7f824cfe2700 1 -- 172.21.15.140:0/1170943529 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8224053980 msgr2=0x7f8224055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:48.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.132+0000 7f824cfe2700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8224053980 0x7f8224055e40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f823800d000 tx=0x7f8238009040).stop 2022-07-21T01:56:48.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.132+0000 7f824cfe2700 1 -- 172.21.15.140:0/1170943529 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82480b1480 msgr2=0x7f824814e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:48.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.132+0000 7f824cfe2700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82480b1480 0x7f824814e1b0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f8240006e00 tx=0x7f82400091d0).stop 2022-07-21T01:56:48.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.132+0000 7f824cfe2700 1 -- 172.21.15.140:0/1170943529 shutdown_connections 2022-07-21T01:56:48.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.132+0000 7f824cfe2700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8224053980 0x7f8224055e40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:48.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.132+0000 7f824cfe2700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f82480b1480 0x7f824814e1b0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:48.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.132+0000 7f824cfe2700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82480acd70 0x7f824813af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:48.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.132+0000 7f824cfe2700 1 --2- 172.21.15.140:0/1170943529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82480ac040 0x7f82480631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:48.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.132+0000 7f824cfe2700 1 -- 172.21.15.140:0/1170943529 >> 172.21.15.140:0/1170943529 conn(0x7f82480b23d0 msgr2=0x7f824813f890 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:48.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.136+0000 7f824cfe2700 1 -- 172.21.15.140:0/1170943529 shutdown_connections 2022-07-21T01:56:48.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.136+0000 7f824cfe2700 1 -- 172.21.15.140:0/1170943529 wait complete. 2022-07-21T01:56:48.151 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:56:48.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.260+0000 7f8b1759e700 1 Processor -- start 2022-07-21T01:56:48.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.260+0000 7f8b1759e700 1 -- start start 2022-07-21T01:56:48.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8b180b6840 0x7f8b180b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:48.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b4bc0 0x7f8b18141a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:48.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b18066b10 con 0x7f8b180b3cf0 2022-07-21T01:56:48.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b180b6c20 con 0x7f8b180b6840 2022-07-21T01:56:48.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180aead0 con 0x7f8b180b4bc0 2022-07-21T01:56:48.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b16d9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b4bc0 0x7f8b18141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:48.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1659c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8b180b6840 0x7f8b180b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:48.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b16d9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b4bc0 0x7f8b18141a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36534/0 (socket says 172.21.15.140:36534) 2022-07-21T01:56:48.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b16d9d700 1 -- 172.21.15.140:0/2479403050 learned_addr learned my addr 172.21.15.140:0/2479403050 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:48.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b15d9b700 1 --1- 172.21.15.140:0/2479403050 >> v1:172.21.15.152:6789/0 conn(0x7f8b180b3cf0 0x7f8b18146040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:55060/0 (socket says 172.21.15.140:55060) 2022-07-21T01:56:48.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b16d9d700 1 -- 172.21.15.140:0/2479403050 >> v2:172.21.15.152:3300/0 conn(0x7f8b180b6840 msgr2=0x7f8b180b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:48.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b16d9d700 1 --2- 172.21.15.140:0/2479403050 >> v2:172.21.15.152:3300/0 conn(0x7f8b180b6840 0x7f8b180b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:48.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b16d9d700 1 -- 172.21.15.140:0/2479403050 >> v1:172.21.15.152:6789/0 conn(0x7f8b180b3cf0 legacy=0x7f8b18146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:48.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1559a700 1 -- 172.21.15.140:0/2479403050 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720839038 0 0) 0x7f8b18066b10 con 0x7f8b180b3cf0 2022-07-21T01:56:48.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b16d9d700 1 -- 172.21.15.140:0/2479403050 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b180aed70 con 0x7f8b180b4bc0 2022-07-21T01:56:48.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b16d9d700 1 --2- 172.21.15.140:0/2479403050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b4bc0 0x7f8b18141a90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f8b100032f0 tx=0x7f8b10009290).ready entity=mon.2 client_cookie=9fa6adc4f11c7940 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:48.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1559a700 1 -- 172.21.15.140:0/2479403050 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b1000b8a0 con 0x7f8b180b4bc0 2022-07-21T01:56:48.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1559a700 1 -- 172.21.15.140:0/2479403050 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1000ba00 con 0x7f8b180b4bc0 2022-07-21T01:56:48.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1559a700 1 -- 172.21.15.140:0/2479403050 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b1000bb70 con 0x7f8b180b4bc0 2022-07-21T01:56:48.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 -- 172.21.15.140:0/2479403050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b4bc0 msgr2=0x7f8b18141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:48.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 --2- 172.21.15.140:0/2479403050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b4bc0 0x7f8b18141a90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f8b100032f0 tx=0x7f8b10009290).stop 2022-07-21T01:56:48.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 -- 172.21.15.140:0/2479403050 shutdown_connections 2022-07-21T01:56:48.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 --2- 172.21.15.140:0/2479403050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b4bc0 0x7f8b18141a90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:48.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 --2- 172.21.15.140:0/2479403050 >> v2:172.21.15.152:3300/0 conn(0x7f8b180b6840 0x7f8b180b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:48.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 -- 172.21.15.140:0/2479403050 >> 172.21.15.140:0/2479403050 conn(0x7f8b180b9e10 msgr2=0x7f8b180b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:48.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 -- 172.21.15.140:0/2479403050 shutdown_connections 2022-07-21T01:56:48.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.264+0000 7f8b1759e700 1 -- 172.21.15.140:0/2479403050 wait complete. 2022-07-21T01:56:48.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1759e700 1 Processor -- start 2022-07-21T01:56:48.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1759e700 1 -- start start 2022-07-21T01:56:48.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b3cf0 0x7f8b18061bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:48.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b180b4bc0 0x7f8b1813df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:48.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b180b6840 0x7f8b1813e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180647d0 con 0x7f8b180b6840 2022-07-21T01:56:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180642f0 con 0x7f8b180b4bc0 2022-07-21T01:56:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b18062620 con 0x7f8b180b3cf0 2022-07-21T01:56:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b16d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b180b6840 0x7f8b1813e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:48.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b16d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b180b6840 0x7f8b1813e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39810/0 (socket says 172.21.15.140:39810) 2022-07-21T01:56:48.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b16d9d700 1 -- 172.21.15.140:0/337929808 learned_addr learned my addr 172.21.15.140:0/337929808 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:48.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1659c700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b3cf0 0x7f8b18061bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:48.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b15d9b700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b180b4bc0 0x7f8b1813df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:48.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1659c700 1 -- 172.21.15.140:0/337929808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b180b4bc0 msgr2=0x7f8b1813df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:48.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1659c700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b180b4bc0 0x7f8b1813df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:48.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1659c700 1 -- 172.21.15.140:0/337929808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b180b6840 msgr2=0x7f8b1813e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:48.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1659c700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b180b6840 0x7f8b1813e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:48.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b1659c700 1 -- 172.21.15.140:0/337929808 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b10009530 con 0x7f8b180b3cf0 2022-07-21T01:56:48.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.268+0000 7f8b16d9d700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b180b6840 0x7f8b1813e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:48.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.272+0000 7f8b1659c700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b3cf0 0x7f8b18061bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f8b0000c000 tx=0x7f8b00006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:48.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.272+0000 7f8b0f7fe700 1 -- 172.21.15.140:0/337929808 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b00011750 con 0x7f8b180b3cf0 2022-07-21T01:56:48.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.272+0000 7f8b0f7fe700 1 -- 172.21.15.140:0/337929808 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b000118b0 con 0x7f8b180b3cf0 2022-07-21T01:56:48.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.272+0000 7f8b1759e700 1 -- 172.21.15.140:0/337929808 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b180b1540 con 0x7f8b180b3cf0 2022-07-21T01:56:48.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.272+0000 7f8b0f7fe700 1 -- 172.21.15.140:0/337929808 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b00011c10 con 0x7f8b180b3cf0 2022-07-21T01:56:48.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.272+0000 7f8b1759e700 1 -- 172.21.15.140:0/337929808 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1813e960 con 0x7f8b180b3cf0 2022-07-21T01:56:48.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.272+0000 7f8b1759e700 1 -- 172.21.15.140:0/337929808 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b180b16d0 con 0x7f8b180b3cf0 2022-07-21T01:56:48.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.272+0000 7f8b0f7fe700 1 -- 172.21.15.140:0/337929808 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8b00011d70 con 0x7f8b180b3cf0 2022-07-21T01:56:48.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.272+0000 7f8b0f7fe700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8afc053950 0x7f8afc055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:48.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.272+0000 7f8b0f7fe700 1 -- 172.21.15.140:0/337929808 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f8b0006c2c0 con 0x7f8b180b3cf0 2022-07-21T01:56:48.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.272+0000 7f8b15d9b700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8afc053950 0x7f8afc055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:48.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.276+0000 7f8b15d9b700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8afc053950 0x7f8afc055e10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f8b0800d000 tx=0x7f8b08007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:48.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.280+0000 7f8b0f7fe700 1 -- 172.21.15.140:0/337929808 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8b000430c0 con 0x7f8b180b3cf0 2022-07-21T01:56:48.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:48.408+0000 7f8b1759e700 1 -- 172.21.15.140:0/337929808 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b180b50e0 con 0x7f8b180b3cf0 2022-07-21T01:56:49.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.144+0000 7f8b0f7fe700 1 -- 172.21.15.140:0/337929808 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8b000466e0 con 0x7f8b180b3cf0 2022-07-21T01:56:49.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 -- 172.21.15.140:0/337929808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8afc053950 msgr2=0x7f8afc055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:49.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8afc053950 0x7f8afc055e10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f8b0800d000 tx=0x7f8b08007a00).stop 2022-07-21T01:56:49.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 -- 172.21.15.140:0/337929808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b3cf0 msgr2=0x7f8b18061bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:49.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b3cf0 0x7f8b18061bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f8b0000c000 tx=0x7f8b00006df0).stop 2022-07-21T01:56:49.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 -- 172.21.15.140:0/337929808 shutdown_connections 2022-07-21T01:56:49.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8afc053950 0x7f8afc055e10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:49.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b180b6840 0x7f8b1813e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:49.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b180b4bc0 0x7f8b1813df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:49.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 --2- 172.21.15.140:0/337929808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b180b3cf0 0x7f8b18061bf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:49.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 -- 172.21.15.140:0/337929808 >> 172.21.15.140:0/337929808 conn(0x7f8b180b9e10 msgr2=0x7f8b1805c580 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:49.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 -- 172.21.15.140:0/337929808 shutdown_connections 2022-07-21T01:56:49.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.148+0000 7f8b1759e700 1 -- 172.21.15.140:0/337929808 wait complete. 2022-07-21T01:56:49.154 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:56:49.167 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:56:49.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9ac57af700 1 Processor -- start 2022-07-21T01:56:49.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9ac57af700 1 -- start start 2022-07-21T01:56:49.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9ac57af700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac009df40 0x7f9ac0145680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:49.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9ac57af700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9ac0060ee0 0x7f9ac009cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:49.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9ac57af700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac00607e0 con 0x7f9ac00b4d60 2022-07-21T01:56:49.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9ac57af700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac0060d20 con 0x7f9ac0060ee0 2022-07-21T01:56:49.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9ac57af700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac0060a80 con 0x7f9ac009df40 2022-07-21T01:56:49.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9abf7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9ac0060ee0 0x7f9ac009cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:49.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9abeffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac009df40 0x7f9ac0145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:49.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9abeffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac009df40 0x7f9ac0145680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36544/0 (socket says 172.21.15.140:36544) 2022-07-21T01:56:49.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9abf7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9ac0060ee0 0x7f9ac009cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:45558/0 (socket says 172.21.15.140:45558) 2022-07-21T01:56:49.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9abeffd700 1 -- 172.21.15.140:0/370098421 learned_addr learned my addr 172.21.15.140:0/370098421 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:49.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.276+0000 7f9abe7fc700 1 --1- 172.21.15.140:0/370098421 >> v1:172.21.15.152:6789/0 conn(0x7f9ac00b4d60 0x7f9ac0145b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:55076/0 (socket says 172.21.15.140:55076) 2022-07-21T01:56:49.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9abeffd700 1 -- 172.21.15.140:0/370098421 >> v2:172.21.15.152:3300/0 conn(0x7f9ac0060ee0 msgr2=0x7f9ac009cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:49.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9abeffd700 1 --2- 172.21.15.140:0/370098421 >> v2:172.21.15.152:3300/0 conn(0x7f9ac0060ee0 0x7f9ac009cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:49.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9abeffd700 1 -- 172.21.15.140:0/370098421 >> v1:172.21.15.152:6789/0 conn(0x7f9ac00b4d60 legacy=0x7f9ac0145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:49.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9abeffd700 1 -- 172.21.15.140:0/370098421 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac009e320 con 0x7f9ac009df40 2022-07-21T01:56:49.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9abf7fe700 1 --2- 172.21.15.140:0/370098421 >> v2:172.21.15.152:3300/0 conn(0x7f9ac0060ee0 0x7f9ac009cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:49.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9abeffd700 1 --2- 172.21.15.140:0/370098421 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac009df40 0x7f9ac0145680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f9ab4002f70 tx=0x7f9ab4006e50).ready entity=mon.2 client_cookie=3cf8bf8c69ed05f4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:49.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9abdffb700 1 -- 172.21.15.140:0/370098421 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ab400d5c0 con 0x7f9ac009df40 2022-07-21T01:56:49.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9abdffb700 1 -- 172.21.15.140:0/370098421 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab4000e90 con 0x7f9ac009df40 2022-07-21T01:56:49.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9abdffb700 1 -- 172.21.15.140:0/370098421 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ab400d950 con 0x7f9ac009df40 2022-07-21T01:56:49.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9ac57af700 1 -- 172.21.15.140:0/370098421 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac009df40 msgr2=0x7f9ac0145680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:49.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9ac57af700 1 --2- 172.21.15.140:0/370098421 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac009df40 0x7f9ac0145680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f9ab4002f70 tx=0x7f9ab4006e50).stop 2022-07-21T01:56:49.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9ac57af700 1 -- 172.21.15.140:0/370098421 shutdown_connections 2022-07-21T01:56:49.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9ac57af700 1 --2- 172.21.15.140:0/370098421 >> v2:172.21.15.152:3300/0 conn(0x7f9ac0060ee0 0x7f9ac009cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:49.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9ac57af700 1 --2- 172.21.15.140:0/370098421 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac009df40 0x7f9ac0145680 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:49.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9ac57af700 1 -- 172.21.15.140:0/370098421 >> 172.21.15.140:0/370098421 conn(0x7f9ac00b5f30 msgr2=0x7f9ac00b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:49.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9ac57af700 1 -- 172.21.15.140:0/370098421 shutdown_connections 2022-07-21T01:56:49.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.280+0000 7f9ac57af700 1 -- 172.21.15.140:0/370098421 wait complete. 2022-07-21T01:56:49.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9ac57af700 1 Processor -- start 2022-07-21T01:56:49.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9ac57af700 1 -- start start 2022-07-21T01:56:49.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9ac57af700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ac0060ee0 0x7f9ac00b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:49.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9ac57af700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ac009df40 0x7f9ac00b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:49.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9ac57af700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac00b4d60 0x7f9ac01522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:49.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9ac57af700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac00b2f60 con 0x7f9ac0060ee0 2022-07-21T01:56:49.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9ac57af700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac0156470 con 0x7f9ac009df40 2022-07-21T01:56:49.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9ac57af700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac01561c0 con 0x7f9ac00b4d60 2022-07-21T01:56:49.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9abeffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ac0060ee0 0x7f9ac00b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:49.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9abeffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ac0060ee0 0x7f9ac00b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39822/0 (socket says 172.21.15.140:39822) 2022-07-21T01:56:49.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9abeffd700 1 -- 172.21.15.140:0/3958091871 learned_addr learned my addr 172.21.15.140:0/3958091871 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:49.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9abe7fc700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ac009df40 0x7f9ac00b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:49.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9abf7fe700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac00b4d60 0x7f9ac01522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:49.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9abeffd700 1 -- 172.21.15.140:0/3958091871 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac00b4d60 msgr2=0x7f9ac01522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:49.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9abeffd700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac00b4d60 0x7f9ac01522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:49.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9abeffd700 1 -- 172.21.15.140:0/3958091871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ac009df40 msgr2=0x7f9ac00b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:49.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9abeffd700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ac009df40 0x7f9ac00b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:49.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9abeffd700 1 -- 172.21.15.140:0/3958091871 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab8009470 con 0x7f9ac0060ee0 2022-07-21T01:56:49.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9abeffd700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ac0060ee0 0x7f9ac00b03f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7f9ab4002f70 tx=0x7f9ab4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:49.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9aaffff700 1 -- 172.21.15.140:0/3958091871 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ab4013e90 con 0x7f9ac0060ee0 2022-07-21T01:56:49.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9ac57af700 1 -- 172.21.15.140:0/3958091871 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab400b1e0 con 0x7f9ac0060ee0 2022-07-21T01:56:49.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9ac57af700 1 -- 172.21.15.140:0/3958091871 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac00b1c50 con 0x7f9ac0060ee0 2022-07-21T01:56:49.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9aaffff700 1 -- 172.21.15.140:0/3958091871 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab401e400 con 0x7f9ac0060ee0 2022-07-21T01:56:49.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.284+0000 7f9aaffff700 1 -- 172.21.15.140:0/3958091871 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ab401e760 con 0x7f9ac0060ee0 2022-07-21T01:56:49.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.288+0000 7f9ac57af700 1 -- 172.21.15.140:0/3958091871 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac00574e0 con 0x7f9ac0060ee0 2022-07-21T01:56:49.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.288+0000 7f9aaffff700 1 -- 172.21.15.140:0/3958091871 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9ab4025030 con 0x7f9ac0060ee0 2022-07-21T01:56:49.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.288+0000 7f9aaffff700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a9c053aa0 0x7f9a9c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:49.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.288+0000 7f9aaffff700 1 -- 172.21.15.140:0/3958091871 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f9ab406cc60 con 0x7f9ac0060ee0 2022-07-21T01:56:49.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.288+0000 7f9abe7fc700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a9c053aa0 0x7f9a9c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:49.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.288+0000 7f9abe7fc700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a9c053aa0 0x7f9a9c055f60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f9ab000d000 tx=0x7f9ab0007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:49.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.296+0000 7f9aaffff700 1 -- 172.21.15.140:0/3958091871 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9ab4043c40 con 0x7f9ac0060ee0 2022-07-21T01:56:49.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:49.420+0000 7f9ac57af700 1 -- 172.21.15.140:0/3958091871 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f9ac0156630 con 0x7f9ac0060ee0 2022-07-21T01:56:50.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.156+0000 7f9aaffff700 1 -- 172.21.15.140:0/3958091871 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v117) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9ab4023070 con 0x7f9ac0060ee0 2022-07-21T01:56:50.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.156+0000 7f9ac57af700 1 -- 172.21.15.140:0/3958091871 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a9c053aa0 msgr2=0x7f9a9c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:50.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.156+0000 7f9ac57af700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a9c053aa0 0x7f9a9c055f60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f9ab000d000 tx=0x7f9ab0007b80).stop 2022-07-21T01:56:50.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.156+0000 7f9ac57af700 1 -- 172.21.15.140:0/3958091871 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ac0060ee0 msgr2=0x7f9ac00b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:50.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.156+0000 7f9ac57af700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ac0060ee0 0x7f9ac00b03f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7f9ab4002f70 tx=0x7f9ab4014000).stop 2022-07-21T01:56:50.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.160+0000 7f9ac57af700 1 -- 172.21.15.140:0/3958091871 shutdown_connections 2022-07-21T01:56:50.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.160+0000 7f9ac57af700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a9c053aa0 0x7f9a9c055f60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:50.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.160+0000 7f9ac57af700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9ac00b4d60 0x7f9ac01522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:50.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.160+0000 7f9ac57af700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9ac009df40 0x7f9ac00b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:50.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.160+0000 7f9ac57af700 1 --2- 172.21.15.140:0/3958091871 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9ac0060ee0 0x7f9ac00b03f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:50.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.160+0000 7f9ac57af700 1 -- 172.21.15.140:0/3958091871 >> 172.21.15.140:0/3958091871 conn(0x7f9ac00b5f30 msgr2=0x7f9ac0143f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:50.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.160+0000 7f9ac57af700 1 -- 172.21.15.140:0/3958091871 shutdown_connections 2022-07-21T01:56:50.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:50.160+0000 7f9ac57af700 1 -- 172.21.15.140:0/3958091871 wait complete. 2022-07-21T01:56:50.168 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:56:50.173 DEBUG:teuthology.orchestra.run.smithi191:> sudo rados -p cephfs_metadata put key /etc/fstab 2022-07-21T01:56:50.227 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.206+0000 7fdd06538b80 1 Processor -- start 2022-07-21T01:56:50.228 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.206+0000 7fdd06538b80 1 -- start start 2022-07-21T01:56:50.229 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.206+0000 7fdd06538b80 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x559a163ca9d0 0x559a163d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:50.229 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.206+0000 7fdd06538b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a163cb2a0 0x559a163d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:50.229 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.206+0000 7fdd06538b80 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cfd90 0x559a163decb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:50.230 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.206+0000 7fdd06538b80 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a163cd9e0 con 0x559a163cfd90 2022-07-21T01:56:50.231 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.206+0000 7fdd06538b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a163d4050 con 0x559a163cb2a0 2022-07-21T01:56:50.231 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.206+0000 7fdd06538b80 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a163df1a0 con 0x559a163ca9d0 2022-07-21T01:56:50.232 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd05176700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cfd90 0x559a163decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:50.232 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd04174700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a163cb2a0 0x559a163d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:50.233 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd04975700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x559a163ca9d0 0x559a163d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:50.233 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd05176700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cfd90 0x559a163decb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.191:58194/0 (socket says 172.21.15.191:58194) 2022-07-21T01:56:50.234 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd05176700 1 -- 172.21.15.191:0/211726906 learned_addr learned my addr 172.21.15.191:0/211726906 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-07-21T01:56:50.234 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd05176700 1 -- 172.21.15.191:0/211726906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x559a163ca9d0 msgr2=0x559a163d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:50.234 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd05176700 1 --2- 172.21.15.191:0/211726906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x559a163ca9d0 0x559a163d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:50.235 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd05176700 1 -- 172.21.15.191:0/211726906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a163cb2a0 msgr2=0x559a163d3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:50.235 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd05176700 1 --2- 172.21.15.191:0/211726906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a163cb2a0 0x559a163d3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:50.236 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd05176700 1 -- 172.21.15.191:0/211726906 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf400a0e0 con 0x559a163cfd90 2022-07-21T01:56:50.236 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd04174700 1 --2- 172.21.15.191:0/211726906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a163cb2a0 0x559a163d3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:50.237 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd05176700 1 --2- 172.21.15.191:0/211726906 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cfd90 0x559a163decb0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7fdcf400b000 tx=0x7fdcf4007040).ready entity=mon.0 client_cookie=536794f39d2b0509 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:50.237 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd02971700 1 -- 172.21.15.191:0/211726906 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcf4011600 con 0x559a163cfd90 2022-07-21T01:56:50.238 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd02971700 1 -- 172.21.15.191:0/211726906 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf4011760 con 0x559a163cfd90 2022-07-21T01:56:50.238 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd02971700 1 -- 172.21.15.191:0/211726906 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcf4011ac0 con 0x559a163cfd90 2022-07-21T01:56:50.239 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 -- 172.21.15.191:0/211726906 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cfd90 msgr2=0x559a163decb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:50.239 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 --2- 172.21.15.191:0/211726906 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cfd90 0x559a163decb0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7fdcf400b000 tx=0x7fdcf4007040).stop 2022-07-21T01:56:50.240 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 -- 172.21.15.191:0/211726906 shutdown_connections 2022-07-21T01:56:50.240 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 --2- 172.21.15.191:0/211726906 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cfd90 0x559a163decb0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:50.241 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 --2- 172.21.15.191:0/211726906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a163cb2a0 0x559a163d3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:50.241 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 --2- 172.21.15.191:0/211726906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x559a163ca9d0 0x559a163d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:50.241 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 -- 172.21.15.191:0/211726906 >> 172.21.15.191:0/211726906 conn(0x559a163c41a0 msgr2=0x559a163cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:50.242 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 -- 172.21.15.191:0/211726906 shutdown_connections 2022-07-21T01:56:50.242 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 -- 172.21.15.191:0/211726906 wait complete. 2022-07-21T01:56:50.243 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 Processor -- start 2022-07-21T01:56:50.243 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 -- start start 2022-07-21T01:56:50.244 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a163ca9d0 0x559a163deb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:50.244 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cb2a0 0x559a163d7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:50.244 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x559a163cfd90 0x559a163d8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:50.245 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a163d7b30 con 0x559a163cb2a0 2022-07-21T01:56:50.245 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a163d88d0 con 0x559a163ca9d0 2022-07-21T01:56:50.246 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd06538b80 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a163d8a90 con 0x559a163cfd90 2022-07-21T01:56:50.246 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd04174700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cb2a0 0x559a163d7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:50.247 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd05176700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x559a163cfd90 0x559a163d8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:50.247 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd04975700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a163ca9d0 0x559a163deb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:50.248 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd04174700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cb2a0 0x559a163d7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.191:58198/0 (socket says 172.21.15.191:58198) 2022-07-21T01:56:50.248 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd04174700 1 -- 172.21.15.191:0/579567599 learned_addr learned my addr 172.21.15.191:0/579567599 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-07-21T01:56:50.249 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.210+0000 7fdd05176700 1 -- 172.21.15.191:0/579567599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a163ca9d0 msgr2=0x559a163deb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:50.249 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.214+0000 7fdd05176700 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a163ca9d0 0x559a163deb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:50.249 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.214+0000 7fdd05176700 1 -- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cb2a0 msgr2=0x559a163d7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:50.250 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.214+0000 7fdd05176700 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x559a163cb2a0 0x559a163d7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:50.250 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.214+0000 7fdd05176700 1 -- 172.21.15.191:0/579567599 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a162daef0 con 0x559a163cfd90 2022-07-21T01:56:50.251 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.214+0000 7fdd05176700 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x559a163cfd90 0x559a163d8160 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fdcf400b000 tx=0x7fdcf4017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:50.251 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.214+0000 7fdd02170700 1 -- 172.21.15.191:0/579567599 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcf4011e40 con 0x559a163cfd90 2022-07-21T01:56:50.252 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.214+0000 7fdd02170700 1 -- 172.21.15.191:0/579567599 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf40193b0 con 0x559a163cfd90 2022-07-21T01:56:50.253 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.214+0000 7fdd06538b80 1 -- 172.21.15.191:0/579567599 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf400a550 con 0x559a163cfd90 2022-07-21T01:56:50.253 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.214+0000 7fdd02170700 1 -- 172.21.15.191:0/579567599 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdcf4019710 con 0x559a163cfd90 2022-07-21T01:56:50.254 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.214+0000 7fdd06538b80 1 -- 172.21.15.191:0/579567599 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec005d90 con 0x559a163cfd90 2022-07-21T01:56:50.254 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.214+0000 7fdd02170700 1 -- 172.21.15.191:0/579567599 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdcf4019870 con 0x559a163cfd90 2022-07-21T01:56:50.254 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.218+0000 7fdd02170700 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdce40525f0 0x7fdce4054ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:50.255 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.218+0000 7fdd02170700 1 -- 172.21.15.191:0/579567599 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6075+0+0 (secure 0 0 0) 0x7fdcf406c620 con 0x559a163cfd90 2022-07-21T01:56:50.256 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.218+0000 7fdd04975700 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdce40525f0 0x7fdce4054ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:50.256 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.218+0000 7fdd06538b80 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x559a163e5310 0x559a163e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:50.257 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.218+0000 7fdd06538b80 1 -- 172.21.15.191:0/579567599 --> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] -- osd_op(unknown.0.0:1 23.0 23:0620e6f9:::key:head [writefull 0~617 in=617b] snapc 0=[] ondisk+write+known_if_redirected e117) v8 -- 0x559a163e7bf0 con 0x559a163e5310 2022-07-21T01:56:50.257 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.218+0000 7fdd04174700 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x559a163e5310 0x559a163e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:50.257 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.218+0000 7fdd04975700 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdce40525f0 0x7fdce4054ab0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fdcfc00a000 tx=0x7fdcfc00c040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:50.258 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:50.218+0000 7fdd04174700 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x559a163e5310 0x559a163e7700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:51.220 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.214+0000 7fdd02170700 1 -- 172.21.15.191:0/579567599 <== osd.0 v2:172.21.15.140:6802/22710 1 ==== osd_map(118..118 src has 1..118) v4 ==== 320+0+0 (crc 0 0 0) 0x7fdcec008850 con 0x559a163e5310 2022-07-21T01:56:51.236 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd04174700 1 -- 172.21.15.191:0/579567599 <== osd.0 v2:172.21.15.140:6802/22710 2 ==== osd_op_reply(1 key [writefull 0~617] v118'1 uv1 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7fdcec008a10 con 0x559a163e5310 2022-07-21T01:56:51.237 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd06538b80 1 -- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x559a163e5310 msgr2=0x559a163e7700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:51.237 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd06538b80 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x559a163e5310 0x559a163e7700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.238 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd06538b80 1 -- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdce40525f0 msgr2=0x7fdce4054ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:51.238 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd06538b80 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdce40525f0 0x7fdce4054ab0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fdcfc00a000 tx=0x7fdcfc00c040).stop 2022-07-21T01:56:51.239 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd06538b80 1 -- 172.21.15.191:0/579567599 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x559a163cfd90 msgr2=0x559a163d8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:51.239 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd06538b80 1 --2- 172.21.15.191:0/579567599 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x559a163cfd90 0x559a163d8160 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fdcf400b000 tx=0x7fdcf4017000).stop 2022-07-21T01:56:51.239 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd05176700 1 -- 172.21.15.191:0/579567599 reap_dead start 2022-07-21T01:56:51.240 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd06538b80 1 -- 172.21.15.191:0/579567599 shutdown_connections 2022-07-21T01:56:51.240 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd06538b80 1 -- 172.21.15.191:0/579567599 >> 172.21.15.191:0/579567599 conn(0x559a163c41a0 msgr2=0x559a163d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:51.240 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd06538b80 1 -- 172.21.15.191:0/579567599 shutdown_connections 2022-07-21T01:56:51.241 INFO:teuthology.orchestra.run.smithi191.stderr:2022-07-21T01:56:51.230+0000 7fdd06538b80 1 -- 172.21.15.191:0/579567599 wait complete. 2022-07-21T01:56:51.243 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-07-21T01:56:51.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.348+0000 7fec3157d700 1 Processor -- start 2022-07-21T01:56:51.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.348+0000 7fec3157d700 1 -- start start 2022-07-21T01:56:51.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fec2c0b5650 0x7fec2c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:51.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fec2c0b1200 0x7fec2c05c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:51.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec2c05bd90 0x7fec2c05c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:51.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c142970 con 0x7fec2c0b1200 2022-07-21T01:56:51.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c0aea30 con 0x7fec2c0b5650 2022-07-21T01:56:51.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0aec60 con 0x7fec2c05bd90 2022-07-21T01:56:51.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fec2c0b1200 0x7fec2c05c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:51.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fec2c0b1200 0x7fec2c05c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39828/0 (socket says 172.21.15.140:39828) 2022-07-21T01:56:51.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2b7fe700 1 -- 172.21.15.140:0/3456300871 learned_addr learned my addr 172.21.15.140:0/3456300871 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:51.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec30d7c700 1 --2- 172.21.15.140:0/3456300871 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec2c05bd90 0x7fec2c05c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:51.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2bfff700 1 --2- 172.21.15.140:0/3456300871 >> v2:172.21.15.152:3300/0 conn(0x7fec2c0b5650 0x7fec2c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:51.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2b7fe700 1 -- 172.21.15.140:0/3456300871 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec2c05bd90 msgr2=0x7fec2c05c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:51.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2b7fe700 1 --2- 172.21.15.140:0/3456300871 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec2c05bd90 0x7fec2c05c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2b7fe700 1 -- 172.21.15.140:0/3456300871 >> v2:172.21.15.152:3300/0 conn(0x7fec2c0b5650 msgr2=0x7fec2c0b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:51.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2b7fe700 1 --2- 172.21.15.140:0/3456300871 >> v2:172.21.15.152:3300/0 conn(0x7fec2c0b5650 0x7fec2c0b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2b7fe700 1 -- 172.21.15.140:0/3456300871 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c0af180 con 0x7fec2c0b1200 2022-07-21T01:56:51.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2b7fe700 1 --2- 172.21.15.140:0/3456300871 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec2c0b1200 0x7fec2c05c4f0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7fec1c0032f0 tx=0x7fec1c009250).ready entity=mon.0 client_cookie=2b72cf51cf4751ec server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:51.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2affd700 1 -- 172.21.15.140:0/3456300871 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec1c00b870 con 0x7fec2c0b1200 2022-07-21T01:56:51.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2affd700 1 -- 172.21.15.140:0/3456300871 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c00b9d0 con 0x7fec2c0b1200 2022-07-21T01:56:51.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec2affd700 1 -- 172.21.15.140:0/3456300871 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec1c00bb40 con 0x7fec2c0b1200 2022-07-21T01:56:51.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 -- 172.21.15.140:0/3456300871 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec2c0b1200 msgr2=0x7fec2c05c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:51.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 --2- 172.21.15.140:0/3456300871 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec2c0b1200 0x7fec2c05c4f0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7fec1c0032f0 tx=0x7fec1c009250).stop 2022-07-21T01:56:51.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 -- 172.21.15.140:0/3456300871 shutdown_connections 2022-07-21T01:56:51.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 --2- 172.21.15.140:0/3456300871 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec2c05bd90 0x7fec2c05c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 --2- 172.21.15.140:0/3456300871 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec2c0b1200 0x7fec2c05c4f0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 --2- 172.21.15.140:0/3456300871 >> v2:172.21.15.152:3300/0 conn(0x7fec2c0b5650 0x7fec2c0b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 -- 172.21.15.140:0/3456300871 >> 172.21.15.140:0/3456300871 conn(0x7fec2c0634e0 msgr2=0x7fec2c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:51.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 -- 172.21.15.140:0/3456300871 shutdown_connections 2022-07-21T01:56:51.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 -- 172.21.15.140:0/3456300871 wait complete. 2022-07-21T01:56:51.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.352+0000 7fec3157d700 1 Processor -- start 2022-07-21T01:56:51.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec3157d700 1 -- start start 2022-07-21T01:56:51.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec3157d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec2c05bd90 0x7fec2c149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:51.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec3157d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec2c0b1200 0x7fec2c146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:51.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec3157d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec2c0b5650 0x7fec2c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:51.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec3157d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c141d90 con 0x7fec2c0b1200 2022-07-21T01:56:51.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec3157d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c142030 con 0x7fec2c0b5650 2022-07-21T01:56:51.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec3157d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c146690 con 0x7fec2c05bd90 2022-07-21T01:56:51.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec2b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec2c0b1200 0x7fec2c146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:51.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec2b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec2c0b1200 0x7fec2c146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39836/0 (socket says 172.21.15.140:39836) 2022-07-21T01:56:51.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec2b7fe700 1 -- 172.21.15.140:0/311908733 learned_addr learned my addr 172.21.15.140:0/311908733 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:51.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec2bfff700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec2c05bd90 0x7fec2c149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:51.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec30d7c700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec2c0b5650 0x7fec2c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:51.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec2b7fe700 1 -- 172.21.15.140:0/311908733 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec2c05bd90 msgr2=0x7fec2c149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:51.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec2b7fe700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec2c05bd90 0x7fec2c149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec2b7fe700 1 -- 172.21.15.140:0/311908733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec2c0b5650 msgr2=0x7fec2c155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:51.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec2b7fe700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec2c0b5650 0x7fec2c155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec2b7fe700 1 -- 172.21.15.140:0/311908733 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c0094d0 con 0x7fec2c0b1200 2022-07-21T01:56:51.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec30d7c700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec2c0b5650 0x7fec2c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:51.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec2b7fe700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec2c0b1200 0x7fec2c146b30 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fec1c0032f0 tx=0x7fec1c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:51.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec297fa700 1 -- 172.21.15.140:0/311908733 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec1c006690 con 0x7fec2c0b1200 2022-07-21T01:56:51.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec297fa700 1 -- 172.21.15.140:0/311908733 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c0067f0 con 0x7fec2c0b1200 2022-07-21T01:56:51.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec3157d700 1 -- 172.21.15.140:0/311908733 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2c144e70 con 0x7fec2c0b1200 2022-07-21T01:56:51.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec297fa700 1 -- 172.21.15.140:0/311908733 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec1c021670 con 0x7fec2c0b1200 2022-07-21T01:56:51.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.356+0000 7fec3157d700 1 -- 172.21.15.140:0/311908733 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c144050 con 0x7fec2c0b1200 2022-07-21T01:56:51.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.360+0000 7fec3157d700 1 -- 172.21.15.140:0/311908733 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2c145000 con 0x7fec2c0b1200 2022-07-21T01:56:51.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.360+0000 7fec297fa700 1 -- 172.21.15.140:0/311908733 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fec1c028070 con 0x7fec2c0b1200 2022-07-21T01:56:51.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.360+0000 7fec297fa700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec10053a10 0x7fec10055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:51.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.360+0000 7fec297fa700 1 -- 172.21.15.140:0/311908733 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6075+0+0 (secure 0 0 0) 0x7fec1c06d050 con 0x7fec2c0b1200 2022-07-21T01:56:51.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.360+0000 7fec2bfff700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec10053a10 0x7fec10055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:51.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.360+0000 7fec2bfff700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec10053a10 0x7fec10055ed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fec1400c000 tx=0x7fec14008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:51.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.368+0000 7fec297fa700 1 -- 172.21.15.140:0/311908733 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fec1c043ed0 con 0x7fec2c0b1200 2022-07-21T01:56:51.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.532+0000 7fec3157d700 1 -- 172.21.15.140:0/311908733 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7fec2c0574e0 con 0x7fec2c0b1200 2022-07-21T01:56:51.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec297fa700 1 -- 172.21.15.140:0/311908733 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1594 (secure 0 0 0) 0x7fec1c015030 con 0x7fec2c0b1200 2022-07-21T01:56:51.537 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:51.537 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:56:51.537 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:51.538 INFO:teuthology.orchestra.run.smithi140.stdout: "total_bytes": 767585320960, 2022-07-21T01:56:51.538 INFO:teuthology.orchestra.run.smithi140.stdout: "total_avail_bytes": 761051537408, 2022-07-21T01:56:51.538 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_bytes": 6533636096, 2022-07-21T01:56:51.538 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_bytes": 6533783552, 2022-07-21T01:56:51.539 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_ratio": 0.0085121272131800652, 2022-07-21T01:56:51.539 INFO:teuthology.orchestra.run.smithi140.stdout: "num_osds": 8, 2022-07-21T01:56:51.539 INFO:teuthology.orchestra.run.smithi140.stdout: "num_per_pool_osds": 0, 2022-07-21T01:56:51.539 INFO:teuthology.orchestra.run.smithi140.stdout: "num_per_pool_omap_osds": 0 2022-07-21T01:56:51.540 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:51.540 INFO:teuthology.orchestra.run.smithi140.stdout: "stats_by_class": { 2022-07-21T01:56:51.540 INFO:teuthology.orchestra.run.smithi140.stdout: "ssd": { 2022-07-21T01:56:51.540 INFO:teuthology.orchestra.run.smithi140.stdout: "total_bytes": 767585320960, 2022-07-21T01:56:51.541 INFO:teuthology.orchestra.run.smithi140.stdout: "total_avail_bytes": 761051537408, 2022-07-21T01:56:51.541 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_bytes": 6533636096, 2022-07-21T01:56:51.541 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_bytes": 6533783552, 2022-07-21T01:56:51.541 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_ratio": 0.0085121272131800652 2022-07-21T01:56:51.542 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:51.542 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:51.542 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:56:51.542 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:51.543 INFO:teuthology.orchestra.run.smithi140.stdout: "name": "device_health_metrics", 2022-07-21T01:56:51.543 INFO:teuthology.orchestra.run.smithi140.stdout: "id": 1, 2022-07-21T01:56:51.543 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:51.543 INFO:teuthology.orchestra.run.smithi140.stdout: "stored": 0, 2022-07-21T01:56:51.544 INFO:teuthology.orchestra.run.smithi140.stdout: "objects": 2, 2022-07-21T01:56:51.544 INFO:teuthology.orchestra.run.smithi140.stdout: "kb_used": 0, 2022-07-21T01:56:51.544 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes_used": 0, 2022-07-21T01:56:51.544 INFO:teuthology.orchestra.run.smithi140.stdout: "percent_used": 0, 2022-07-21T01:56:51.544 INFO:teuthology.orchestra.run.smithi140.stdout: "max_avail": 361335685120 2022-07-21T01:56:51.545 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:51.545 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:51.545 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:51.545 INFO:teuthology.orchestra.run.smithi140.stdout: "name": "rbd", 2022-07-21T01:56:51.546 INFO:teuthology.orchestra.run.smithi140.stdout: "id": 2, 2022-07-21T01:56:51.546 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:51.546 INFO:teuthology.orchestra.run.smithi140.stdout: "stored": 0, 2022-07-21T01:56:51.546 INFO:teuthology.orchestra.run.smithi140.stdout: "objects": 0, 2022-07-21T01:56:51.547 INFO:teuthology.orchestra.run.smithi140.stdout: "kb_used": 0, 2022-07-21T01:56:51.547 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes_used": 0, 2022-07-21T01:56:51.547 INFO:teuthology.orchestra.run.smithi140.stdout: "percent_used": 0, 2022-07-21T01:56:51.547 INFO:teuthology.orchestra.run.smithi140.stdout: "max_avail": 361335685120 2022-07-21T01:56:51.547 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:51.548 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:51.548 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:51.548 INFO:teuthology.orchestra.run.smithi140.stdout: "name": "cephfs_data", 2022-07-21T01:56:51.548 INFO:teuthology.orchestra.run.smithi140.stdout: "id": 22, 2022-07-21T01:56:51.549 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:51.549 INFO:teuthology.orchestra.run.smithi140.stdout: "stored": 0, 2022-07-21T01:56:51.549 INFO:teuthology.orchestra.run.smithi140.stdout: "objects": 0, 2022-07-21T01:56:51.549 INFO:teuthology.orchestra.run.smithi140.stdout: "kb_used": 0, 2022-07-21T01:56:51.550 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes_used": 0, 2022-07-21T01:56:51.550 INFO:teuthology.orchestra.run.smithi140.stdout: "percent_used": 0, 2022-07-21T01:56:51.550 INFO:teuthology.orchestra.run.smithi140.stdout: "max_avail": 361335685120 2022-07-21T01:56:51.550 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:51.551 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:51.551 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:51.551 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:56:51.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 -- 172.21.15.140:0/311908733 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec10053a10 msgr2=0x7fec10055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:51.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec10053a10 0x7fec10055ed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fec1400c000 tx=0x7fec14008040).stop 2022-07-21T01:56:51.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 -- 172.21.15.140:0/311908733 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec2c0b1200 msgr2=0x7fec2c146b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:51.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec2c0b1200 0x7fec2c146b30 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fec1c0032f0 tx=0x7fec1c016000).stop 2022-07-21T01:56:51.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 -- 172.21.15.140:0/311908733 shutdown_connections 2022-07-21T01:56:51.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec10053a10 0x7fec10055ed0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec2c0b5650 0x7fec2c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec2c0b1200 0x7fec2c146b30 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 --2- 172.21.15.140:0/311908733 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec2c05bd90 0x7fec2c149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:51.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 -- 172.21.15.140:0/311908733 >> 172.21.15.140:0/311908733 conn(0x7fec2c0634e0 msgr2=0x7fec2c143520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:51.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 -- 172.21.15.140:0/311908733 shutdown_connections 2022-07-21T01:56:51.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:51.536+0000 7fec3157d700 1 -- 172.21.15.140:0/311908733 wait complete. 2022-07-21T01:56:51.556 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2022-07-21T01:56:56.558 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2022-07-21T01:56:56.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63c66c4700 1 Processor -- start 2022-07-21T01:56:56.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63c66c4700 1 -- start start 2022-07-21T01:56:56.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63c66c4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f63c00adc90 0x7f63c005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:56.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63c66c4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63c00b21e0 0x7f63c0141b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:56.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63c66c4700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c00ae070 con 0x7f63c00ad430 2022-07-21T01:56:56.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63c66c4700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c00ace10 con 0x7f63c00adc90 2022-07-21T01:56:56.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63c66c4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00ad0b0 con 0x7f63c00b21e0 2022-07-21T01:56:56.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63bf7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f63c00ad430 0x7f63c005c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:53990/0 (socket says 172.21.15.140:53990) 2022-07-21T01:56:56.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63bf7fe700 1 -- 172.21.15.140:0/2007936252 learned_addr learned my addr 172.21.15.140:0/2007936252 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:56.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63bffff700 1 --2- 172.21.15.140:0/2007936252 >> v2:172.21.15.152:3300/0 conn(0x7f63c00adc90 0x7f63c005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:56.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63c4c61700 1 --2- 172.21.15.140:0/2007936252 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63c00b21e0 0x7f63c0141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:56.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63beffd700 1 -- 172.21.15.140:0/2007936252 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228620295 0 0) 0x7f63c00ae070 con 0x7f63c00ad430 2022-07-21T01:56:56.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63beffd700 1 -- 172.21.15.140:0/2007936252 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63ac003260 con 0x7f63c00ad430 2022-07-21T01:56:56.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63bffff700 1 -- 172.21.15.140:0/2007936252 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63c00b21e0 msgr2=0x7f63c0141b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:56.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63bffff700 1 --2- 172.21.15.140:0/2007936252 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63c00b21e0 0x7f63c0141b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:56.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63bffff700 1 -- 172.21.15.140:0/2007936252 >> v1:172.21.15.140:6789/0 conn(0x7f63c00ad430 legacy=0x7f63c005c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:56.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63beffd700 1 -- 172.21.15.140:0/2007936252 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1316622109 0 0) 0x7f63ac003260 con 0x7f63c00ad430 2022-07-21T01:56:56.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.668+0000 7f63bffff700 1 -- 172.21.15.140:0/2007936252 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c0143c10 con 0x7f63c00adc90 2022-07-21T01:56:56.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63bffff700 1 --2- 172.21.15.140:0/2007936252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c00adc90 0x7f63c005bd90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f63b4001800 tx=0x7f63b80091d0).ready entity=mon.1 client_cookie=cd0b2964fb6c0902 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:56.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63beffd700 1 -- 172.21.15.140:0/2007936252 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63b4001660 con 0x7f63c00adc90 2022-07-21T01:56:56.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63beffd700 1 -- 172.21.15.140:0/2007936252 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b800f510 con 0x7f63c00adc90 2022-07-21T01:56:56.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63beffd700 1 -- 172.21.15.140:0/2007936252 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63b800f680 con 0x7f63c00adc90 2022-07-21T01:56:56.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 -- 172.21.15.140:0/2007936252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c00adc90 msgr2=0x7f63c005bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:56.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 --2- 172.21.15.140:0/2007936252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c00adc90 0x7f63c005bd90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f63b4001800 tx=0x7f63b80091d0).stop 2022-07-21T01:56:56.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 -- 172.21.15.140:0/2007936252 shutdown_connections 2022-07-21T01:56:56.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 --2- 172.21.15.140:0/2007936252 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63c00b21e0 0x7f63c0141b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:56.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 --2- 172.21.15.140:0/2007936252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c00adc90 0x7f63c005bd90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:56.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 -- 172.21.15.140:0/2007936252 >> 172.21.15.140:0/2007936252 conn(0x7f63c00b72c0 msgr2=0x7f63c00b76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:56.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 -- 172.21.15.140:0/2007936252 shutdown_connections 2022-07-21T01:56:56.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 -- 172.21.15.140:0/2007936252 wait complete. 2022-07-21T01:56:56.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 Processor -- start 2022-07-21T01:56:56.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 -- start start 2022-07-21T01:56:56.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63c00ad430 0x7f63c0152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:56.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63c00adc90 0x7f63c0141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:56.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c00b21e0 0x7f63c0155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:56.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00af1f0 con 0x7f63c00ad430 2022-07-21T01:56:56.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c0155710 con 0x7f63c00b21e0 2022-07-21T01:56:56.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c66c4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00af450 con 0x7f63c00adc90 2022-07-21T01:56:56.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63bffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63c00ad430 0x7f63c0152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:56.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63bffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63c00ad430 0x7f63c0152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39848/0 (socket says 172.21.15.140:39848) 2022-07-21T01:56:56.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63bffff700 1 -- 172.21.15.140:0/2423084805 learned_addr learned my addr 172.21.15.140:0/2423084805 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:56.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63bf7fe700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63c00adc90 0x7f63c0141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:56.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.672+0000 7f63c4c61700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c00b21e0 0x7f63c0155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:56.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bffff700 1 -- 172.21.15.140:0/2423084805 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63c00adc90 msgr2=0x7f63c0141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:56.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bffff700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63c00adc90 0x7f63c0141220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:56.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bffff700 1 -- 172.21.15.140:0/2423084805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c00b21e0 msgr2=0x7f63c0155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:56.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bffff700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c00b21e0 0x7f63c0155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:56.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bffff700 1 -- 172.21.15.140:0/2423084805 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b8009470 con 0x7f63c00ad430 2022-07-21T01:56:56.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bffff700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63c00ad430 0x7f63c0152140 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f63b4001800 tx=0x7f63b8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:56.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bd7fa700 1 -- 172.21.15.140:0/2423084805 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63b800f680 con 0x7f63c00ad430 2022-07-21T01:56:56.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bd7fa700 1 -- 172.21.15.140:0/2423084805 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b80032f0 con 0x7f63c00ad430 2022-07-21T01:56:56.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63c66c4700 1 -- 172.21.15.140:0/2423084805 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c0155970 con 0x7f63c00ad430 2022-07-21T01:56:56.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bd7fa700 1 -- 172.21.15.140:0/2423084805 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f63b8021670 con 0x7f63c00ad430 2022-07-21T01:56:56.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63c66c4700 1 -- 172.21.15.140:0/2423084805 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c0142110 con 0x7f63c00ad430 2022-07-21T01:56:56.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bd7fa700 1 -- 172.21.15.140:0/2423084805 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f63b800f070 con 0x7f63c00ad430 2022-07-21T01:56:56.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63c66c4700 1 -- 172.21.15.140:0/2423084805 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c00574e0 con 0x7f63c00ad430 2022-07-21T01:56:56.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bd7fa700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63a0053b00 0x7f63a0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:56.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bd7fa700 1 -- 172.21.15.140:0/2423084805 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6075+0+0 (secure 0 0 0) 0x7f63b806d1e0 con 0x7f63c00ad430 2022-07-21T01:56:56.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.676+0000 7f63bf7fe700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63a0053b00 0x7f63a0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:56.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.680+0000 7f63bf7fe700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63a0053b00 0x7f63a0055fc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f63b400d000 tx=0x7f63b4001a80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:56.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.684+0000 7f63bd7fa700 1 -- 172.21.15.140:0/2423084805 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f63b8044100 con 0x7f63c00ad430 2022-07-21T01:56:56.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.852+0000 7f63c66c4700 1 -- 172.21.15.140:0/2423084805 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f63c005f4e0 con 0x7f63c00ad430 2022-07-21T01:56:56.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.852+0000 7f63bd7fa700 1 -- 172.21.15.140:0/2423084805 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1935 (secure 0 0 0) 0x7f63b8015020 con 0x7f63c00ad430 2022-07-21T01:56:56.855 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:56:56.855 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:56:56.855 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:56.856 INFO:teuthology.orchestra.run.smithi140.stdout: "total_bytes": 767585320960, 2022-07-21T01:56:56.856 INFO:teuthology.orchestra.run.smithi140.stdout: "total_avail_bytes": 761049374720, 2022-07-21T01:56:56.856 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_bytes": 6535122944, 2022-07-21T01:56:56.856 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_bytes": 6535946240, 2022-07-21T01:56:56.857 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_ratio": 0.008514944463968277, 2022-07-21T01:56:56.857 INFO:teuthology.orchestra.run.smithi140.stdout: "num_osds": 8, 2022-07-21T01:56:56.857 INFO:teuthology.orchestra.run.smithi140.stdout: "num_per_pool_osds": 0, 2022-07-21T01:56:56.857 INFO:teuthology.orchestra.run.smithi140.stdout: "num_per_pool_omap_osds": 0 2022-07-21T01:56:56.857 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:56.858 INFO:teuthology.orchestra.run.smithi140.stdout: "stats_by_class": { 2022-07-21T01:56:56.858 INFO:teuthology.orchestra.run.smithi140.stdout: "ssd": { 2022-07-21T01:56:56.858 INFO:teuthology.orchestra.run.smithi140.stdout: "total_bytes": 767585320960, 2022-07-21T01:56:56.858 INFO:teuthology.orchestra.run.smithi140.stdout: "total_avail_bytes": 761049374720, 2022-07-21T01:56:56.859 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_bytes": 6535122944, 2022-07-21T01:56:56.859 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_bytes": 6535946240, 2022-07-21T01:56:56.859 INFO:teuthology.orchestra.run.smithi140.stdout: "total_used_raw_ratio": 0.008514944463968277 2022-07-21T01:56:56.859 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:56.860 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:56.860 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:56:56.860 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:56.860 INFO:teuthology.orchestra.run.smithi140.stdout: "name": "device_health_metrics", 2022-07-21T01:56:56.861 INFO:teuthology.orchestra.run.smithi140.stdout: "id": 1, 2022-07-21T01:56:56.861 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:56.861 INFO:teuthology.orchestra.run.smithi140.stdout: "stored": 0, 2022-07-21T01:56:56.861 INFO:teuthology.orchestra.run.smithi140.stdout: "objects": 2, 2022-07-21T01:56:56.861 INFO:teuthology.orchestra.run.smithi140.stdout: "kb_used": 0, 2022-07-21T01:56:56.862 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes_used": 0, 2022-07-21T01:56:56.862 INFO:teuthology.orchestra.run.smithi140.stdout: "percent_used": 0, 2022-07-21T01:56:56.862 INFO:teuthology.orchestra.run.smithi140.stdout: "max_avail": 361334898688 2022-07-21T01:56:56.862 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:56.863 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:56.863 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:56.863 INFO:teuthology.orchestra.run.smithi140.stdout: "name": "rbd", 2022-07-21T01:56:56.863 INFO:teuthology.orchestra.run.smithi140.stdout: "id": 2, 2022-07-21T01:56:56.863 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:56.864 INFO:teuthology.orchestra.run.smithi140.stdout: "stored": 0, 2022-07-21T01:56:56.864 INFO:teuthology.orchestra.run.smithi140.stdout: "objects": 0, 2022-07-21T01:56:56.864 INFO:teuthology.orchestra.run.smithi140.stdout: "kb_used": 0, 2022-07-21T01:56:56.864 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes_used": 0, 2022-07-21T01:56:56.865 INFO:teuthology.orchestra.run.smithi140.stdout: "percent_used": 0, 2022-07-21T01:56:56.865 INFO:teuthology.orchestra.run.smithi140.stdout: "max_avail": 361334898688 2022-07-21T01:56:56.865 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:56.865 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:56.865 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:56.866 INFO:teuthology.orchestra.run.smithi140.stdout: "name": "cephfs_data", 2022-07-21T01:56:56.866 INFO:teuthology.orchestra.run.smithi140.stdout: "id": 22, 2022-07-21T01:56:56.866 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:56.866 INFO:teuthology.orchestra.run.smithi140.stdout: "stored": 0, 2022-07-21T01:56:56.867 INFO:teuthology.orchestra.run.smithi140.stdout: "objects": 0, 2022-07-21T01:56:56.867 INFO:teuthology.orchestra.run.smithi140.stdout: "kb_used": 0, 2022-07-21T01:56:56.867 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes_used": 0, 2022-07-21T01:56:56.867 INFO:teuthology.orchestra.run.smithi140.stdout: "percent_used": 0, 2022-07-21T01:56:56.867 INFO:teuthology.orchestra.run.smithi140.stdout: "max_avail": 361334898688 2022-07-21T01:56:56.868 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:56.868 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:56:56.868 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:56:56.868 INFO:teuthology.orchestra.run.smithi140.stdout: "name": "cephfs_metadata", 2022-07-21T01:56:56.868 INFO:teuthology.orchestra.run.smithi140.stdout: "id": 23, 2022-07-21T01:56:56.869 INFO:teuthology.orchestra.run.smithi140.stdout: "stats": { 2022-07-21T01:56:56.869 INFO:teuthology.orchestra.run.smithi140.stdout: "stored": 617, 2022-07-21T01:56:56.869 INFO:teuthology.orchestra.run.smithi140.stdout: "objects": 1, 2022-07-21T01:56:56.869 INFO:teuthology.orchestra.run.smithi140.stdout: "kb_used": 1, 2022-07-21T01:56:56.870 INFO:teuthology.orchestra.run.smithi140.stdout: "bytes_used": 617, 2022-07-21T01:56:56.870 INFO:teuthology.orchestra.run.smithi140.stdout: "percent_used": 8.5377860337132461e-10, 2022-07-21T01:56:56.870 INFO:teuthology.orchestra.run.smithi140.stdout: "max_avail": 361334898688 2022-07-21T01:56:56.870 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:56.870 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:56:56.871 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:56:56.871 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:56:56.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.852+0000 7f63c66c4700 1 -- 172.21.15.140:0/2423084805 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63a0053b00 msgr2=0x7f63a0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:56.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.852+0000 7f63c66c4700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63a0053b00 0x7f63a0055fc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f63b400d000 tx=0x7f63b4001a80).stop 2022-07-21T01:56:56.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.852+0000 7f63c66c4700 1 -- 172.21.15.140:0/2423084805 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63c00ad430 msgr2=0x7f63c0152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:56.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.852+0000 7f63c66c4700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63c00ad430 0x7f63c0152140 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7f63b4001800 tx=0x7f63b8016000).stop 2022-07-21T01:56:56.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.856+0000 7f63c66c4700 1 -- 172.21.15.140:0/2423084805 shutdown_connections 2022-07-21T01:56:56.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.856+0000 7f63c66c4700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f63a0053b00 0x7f63a0055fc0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:56.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.856+0000 7f63c66c4700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f63c00b21e0 0x7f63c0155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:56.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.856+0000 7f63c66c4700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f63c00adc90 0x7f63c0141220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:56.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.856+0000 7f63c66c4700 1 --2- 172.21.15.140:0/2423084805 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f63c00ad430 0x7f63c0152140 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:56.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.856+0000 7f63c66c4700 1 -- 172.21.15.140:0/2423084805 >> 172.21.15.140:0/2423084805 conn(0x7f63c00b72c0 msgr2=0x7f63c00b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:56.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.856+0000 7f63c66c4700 1 -- 172.21.15.140:0/2423084805 shutdown_connections 2022-07-21T01:56:56.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.856+0000 7f63c66c4700 1 -- 172.21.15.140:0/2423084805 wait complete. 2022-07-21T01:56:56.876 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s 2022-07-21T01:56:56.877 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-07-21T01:56:56.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.984+0000 7f077ee1a700 1 Processor -- start 2022-07-21T01:56:56.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.984+0000 7f077ee1a700 1 -- start start 2022-07-21T01:56:56.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.984+0000 7f077ee1a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0778060ee0 0x7f0778145d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:56.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.984+0000 7f077ee1a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07780607e0 con 0x7f077809df40 2022-07-21T01:56:56.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.984+0000 7f077ee1a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0778060a80 con 0x7f07780b4d60 2022-07-21T01:56:56.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.984+0000 7f077ee1a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0778060d20 con 0x7f0778060ee0 2022-07-21T01:56:56.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.984+0000 7f077cbb6700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f077809df40 0x7f0778145680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54002/0 (socket says 172.21.15.140:54002) 2022-07-21T01:56:56.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.984+0000 7f077cbb6700 1 -- 172.21.15.140:0/38987130 learned_addr learned my addr 172.21.15.140:0/38987130 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:56.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.984+0000 7f077d3b7700 1 --2- 172.21.15.140:0/38987130 >> v2:172.21.15.152:3300/0 conn(0x7f0778060ee0 0x7f0778145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:56.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.984+0000 7f07777fe700 1 -- 172.21.15.140:0/38987130 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159940197 0 0) 0x7f07780607e0 con 0x7f077809df40 2022-07-21T01:56:56.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.984+0000 7f07777fe700 1 -- 172.21.15.140:0/38987130 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0758003260 con 0x7f077809df40 2022-07-21T01:56:56.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077d3b7700 1 -- 172.21.15.140:0/38987130 >> v1:172.21.15.152:6789/0 conn(0x7f07780b4d60 legacy=0x7f077809cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:56.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077d3b7700 1 -- 172.21.15.140:0/38987130 >> v1:172.21.15.140:6789/0 conn(0x7f077809df40 legacy=0x7f0778145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:56.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077d3b7700 1 -- 172.21.15.140:0/38987130 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077809e320 con 0x7f0778060ee0 2022-07-21T01:56:56.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077d3b7700 1 --2- 172.21.15.140:0/38987130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0778060ee0 0x7f0778145d90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f07700032f0 tx=0x7f07700091d0).ready entity=mon.1 client_cookie=99e20e63a8b7be4c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:56.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f07777fe700 1 -- 172.21.15.140:0/38987130 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f077000b6c0 con 0x7f0778060ee0 2022-07-21T01:56:56.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f07777fe700 1 -- 172.21.15.140:0/38987130 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077000b820 con 0x7f0778060ee0 2022-07-21T01:56:56.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f07777fe700 1 -- 172.21.15.140:0/38987130 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f077000b990 con 0x7f0778060ee0 2022-07-21T01:56:56.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077ee1a700 1 -- 172.21.15.140:0/38987130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0778060ee0 msgr2=0x7f0778145d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:56.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077ee1a700 1 --2- 172.21.15.140:0/38987130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0778060ee0 0x7f0778145d90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f07700032f0 tx=0x7f07700091d0).stop 2022-07-21T01:56:56.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077ee1a700 1 -- 172.21.15.140:0/38987130 shutdown_connections 2022-07-21T01:56:56.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077ee1a700 1 --2- 172.21.15.140:0/38987130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0778060ee0 0x7f0778145d90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:56.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077ee1a700 1 -- 172.21.15.140:0/38987130 >> 172.21.15.140:0/38987130 conn(0x7f07780b5f30 msgr2=0x7f07780b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:56.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077ee1a700 1 -- 172.21.15.140:0/38987130 shutdown_connections 2022-07-21T01:56:56.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077ee1a700 1 -- 172.21.15.140:0/38987130 wait complete. 2022-07-21T01:56:56.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077ee1a700 1 Processor -- start 2022-07-21T01:56:56.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.988+0000 7f077ee1a700 1 -- start start 2022-07-21T01:56:56.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077ee1a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0778060ee0 0x7f07781555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:56.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077ee1a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f077809df40 0x7f0778145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:56.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077ee1a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07780b4d60 0x7f0778092430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:56.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077ee1a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07780900f0 con 0x7f07780b4d60 2022-07-21T01:56:56.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077ee1a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0778090800 con 0x7f077809df40 2022-07-21T01:56:56.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077ee1a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0778090aa0 con 0x7f0778060ee0 2022-07-21T01:56:56.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077d3b7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07780b4d60 0x7f0778092430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077d3b7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07780b4d60 0x7f0778092430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39864/0 (socket says 172.21.15.140:39864) 2022-07-21T01:56:57.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077d3b7700 1 -- 172.21.15.140:0/2988175635 learned_addr learned my addr 172.21.15.140:0/2988175635 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:57.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077cbb6700 1 --2- 172.21.15.140:0/2988175635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0778060ee0 0x7f07781555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f0777fff700 1 --2- 172.21.15.140:0/2988175635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f077809df40 0x7f0778145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077d3b7700 1 -- 172.21.15.140:0/2988175635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0778060ee0 msgr2=0x7f07781555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:57.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077d3b7700 1 --2- 172.21.15.140:0/2988175635 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0778060ee0 0x7f07781555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077d3b7700 1 -- 172.21.15.140:0/2988175635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f077809df40 msgr2=0x7f0778145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:57.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077d3b7700 1 --2- 172.21.15.140:0/2988175635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f077809df40 0x7f0778145d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077d3b7700 1 -- 172.21.15.140:0/2988175635 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0770009470 con 0x7f07780b4d60 2022-07-21T01:56:57.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f0777fff700 1 --2- 172.21.15.140:0/2988175635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f077809df40 0x7f0778145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:57.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077d3b7700 1 --2- 172.21.15.140:0/2988175635 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f07780b4d60 0x7f0778092430 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f07700032f0 tx=0x7f0770016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:57.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f0775ffb700 1 -- 172.21.15.140:0/2988175635 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f077000b6c0 con 0x7f07780b4d60 2022-07-21T01:56:57.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f0775ffb700 1 -- 172.21.15.140:0/2988175635 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07700107a0 con 0x7f07780b4d60 2022-07-21T01:56:57.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077ee1a700 1 -- 172.21.15.140:0/2988175635 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07781461f0 con 0x7f07780b4d60 2022-07-21T01:56:57.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f0775ffb700 1 -- 172.21.15.140:0/2988175635 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0770010b00 con 0x7f07780b4d60 2022-07-21T01:56:57.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077ee1a700 1 -- 172.21.15.140:0/2988175635 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07780904a0 con 0x7f07780b4d60 2022-07-21T01:56:57.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f0775ffb700 1 -- 172.21.15.140:0/2988175635 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0770029070 con 0x7f07780b4d60 2022-07-21T01:56:57.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.992+0000 7f077ee1a700 1 -- 172.21.15.140:0/2988175635 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077808fd20 con 0x7f07780b4d60 2022-07-21T01:56:57.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.996+0000 7f0775ffb700 1 --2- 172.21.15.140:0/2988175635 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f075c053a70 0x7f075c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.996+0000 7f0775ffb700 1 -- 172.21.15.140:0/2988175635 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6075+0+0 (secure 0 0 0) 0x7f077006d080 con 0x7f07780b4d60 2022-07-21T01:56:57.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.996+0000 7f077cbb6700 1 --2- 172.21.15.140:0/2988175635 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f075c053a70 0x7f075c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:56.996+0000 7f077cbb6700 1 --2- 172.21.15.140:0/2988175635 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f075c053a70 0x7f075c055f30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f076000b000 tx=0x7f0760009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:57.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.000+0000 7f0775ffb700 1 -- 172.21.15.140:0/2988175635 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f077808fd20 con 0x7f07780b4d60 2022-07-21T01:56:57.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.164+0000 7f077ee1a700 1 -- 172.21.15.140:0/2988175635 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f077808fd20 con 0x7f07780b4d60 2022-07-21T01:56:57.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.164+0000 7f0775ffb700 1 -- 172.21.15.140:0/2988175635 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=-22 pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. v55) v1 ==== 209+0+0 (secure 0 0 0) 0x7f077808fd20 con 0x7f07780b4d60 2022-07-21T01:56:57.168 INFO:teuthology.orchestra.run.smithi140.stderr:Error EINVAL: pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. 2022-07-21T01:56:57.171 DEBUG:teuthology.orchestra.run:got remote process result: 22 2022-07-21T01:56:57.173 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:56:57.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e08fbc700 1 Processor -- start 2022-07-21T01:56:57.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e08fbc700 1 -- start start 2022-07-21T01:56:57.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e08fbc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2e040b5990 0x7f2e04146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e08fbc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2e040af3e0 0x7f2e04146560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e08fbc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e0405d210 0x7f2e0414a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e08fbc700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e040ad050 con 0x7f2e040af3e0 2022-07-21T01:56:57.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e08fbc700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e040b17e0 con 0x7f2e040b5990 2022-07-21T01:56:57.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e08fbc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e040b5d70 con 0x7f2e0405d210 2022-07-21T01:56:57.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e02ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2e040af3e0 0x7f2e04146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e02ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2e040af3e0 0x7f2e04146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39874/0 (socket says 172.21.15.140:39874) 2022-07-21T01:56:57.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e02ffd700 1 -- 172.21.15.140:0/2755175378 learned_addr learned my addr 172.21.15.140:0/2755175378 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:57.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e03fff700 1 --2- 172.21.15.140:0/2755175378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e0405d210 0x7f2e0414a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.280+0000 7f2e037fe700 1 --2- 172.21.15.140:0/2755175378 >> v2:172.21.15.152:3300/0 conn(0x7f2e040b5990 0x7f2e04146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e037fe700 1 -- 172.21.15.140:0/2755175378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e0405d210 msgr2=0x7f2e0414a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:57.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e037fe700 1 --2- 172.21.15.140:0/2755175378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e0405d210 0x7f2e0414a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e037fe700 1 -- 172.21.15.140:0/2755175378 >> v2:172.21.15.140:3300/0 conn(0x7f2e040af3e0 msgr2=0x7f2e04146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:57.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e037fe700 1 --2- 172.21.15.140:0/2755175378 >> v2:172.21.15.140:3300/0 conn(0x7f2e040af3e0 0x7f2e04146560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e037fe700 1 -- 172.21.15.140:0/2755175378 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e040b1010 con 0x7f2e040b5990 2022-07-21T01:56:57.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e037fe700 1 --2- 172.21.15.140:0/2755175378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e040b5990 0x7f2e04146070 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f2df4002f70 tx=0x7f2df4006d90).ready entity=mon.1 client_cookie=6dd40aab40c397a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:57.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e027fc700 1 -- 172.21.15.140:0/2755175378 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2df400e470 con 0x7f2e040b5990 2022-07-21T01:56:57.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e027fc700 1 -- 172.21.15.140:0/2755175378 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df4000e90 con 0x7f2e040b5990 2022-07-21T01:56:57.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e027fc700 1 -- 172.21.15.140:0/2755175378 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2df400e7c0 con 0x7f2e040b5990 2022-07-21T01:56:57.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 -- 172.21.15.140:0/2755175378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e040b5990 msgr2=0x7f2e04146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:57.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 --2- 172.21.15.140:0/2755175378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e040b5990 0x7f2e04146070 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f2df4002f70 tx=0x7f2df4006d90).stop 2022-07-21T01:56:57.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 -- 172.21.15.140:0/2755175378 shutdown_connections 2022-07-21T01:56:57.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 --2- 172.21.15.140:0/2755175378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e0405d210 0x7f2e0414a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 --2- 172.21.15.140:0/2755175378 >> v2:172.21.15.140:3300/0 conn(0x7f2e040af3e0 0x7f2e04146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 --2- 172.21.15.140:0/2755175378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e040b5990 0x7f2e04146070 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 -- 172.21.15.140:0/2755175378 >> 172.21.15.140:0/2755175378 conn(0x7f2e04063480 msgr2=0x7f2e040b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:57.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 -- 172.21.15.140:0/2755175378 shutdown_connections 2022-07-21T01:56:57.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 -- 172.21.15.140:0/2755175378 wait complete. 2022-07-21T01:56:57.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 Processor -- start 2022-07-21T01:56:57.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 -- start start 2022-07-21T01:56:57.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0405d210 0x7f2e0410ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e040af3e0 0x7f2e0410f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e040b5990 0x7f2e041554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04144fb0 con 0x7f2e040b5990 2022-07-21T01:56:57.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0410f850 con 0x7f2e0405d210 2022-07-21T01:56:57.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.284+0000 7f2e08fbc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04156a70 con 0x7f2e040af3e0 2022-07-21T01:56:57.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e03fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e040b5990 0x7f2e041554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e03fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e040b5990 0x7f2e041554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39878/0 (socket says 172.21.15.140:39878) 2022-07-21T01:56:57.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e03fff700 1 -- 172.21.15.140:0/1130619329 learned_addr learned my addr 172.21.15.140:0/1130619329 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:57.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e037fe700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0405d210 0x7f2e0410ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e02ffd700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e040af3e0 0x7f2e0410f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e03fff700 1 -- 172.21.15.140:0/1130619329 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e040af3e0 msgr2=0x7f2e0410f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:57.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e03fff700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e040af3e0 0x7f2e0410f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e03fff700 1 -- 172.21.15.140:0/1130619329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0405d210 msgr2=0x7f2e0410ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:57.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e03fff700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0405d210 0x7f2e0410ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e03fff700 1 -- 172.21.15.140:0/1130619329 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df400b120 con 0x7f2e040b5990 2022-07-21T01:56:57.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e03fff700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e040b5990 0x7f2e041554c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f2e0410eba0 tx=0x7f2dfc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:57.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e00ff9700 1 -- 172.21.15.140:0/1130619329 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dfc00c7f0 con 0x7f2e040b5990 2022-07-21T01:56:57.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e00ff9700 1 -- 172.21.15.140:0/1130619329 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dfc00c950 con 0x7f2e040b5990 2022-07-21T01:56:57.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e08fbc700 1 -- 172.21.15.140:0/1130619329 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e04141d70 con 0x7f2e040b5990 2022-07-21T01:56:57.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e00ff9700 1 -- 172.21.15.140:0/1130619329 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dfc00ccb0 con 0x7f2e040b5990 2022-07-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e08fbc700 1 -- 172.21.15.140:0/1130619329 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e04142be0 con 0x7f2e040b5990 2022-07-21T01:56:57.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e08fbc700 1 -- 172.21.15.140:0/1130619329 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0413e500 con 0x7f2e040b5990 2022-07-21T01:56:57.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e00ff9700 1 -- 172.21.15.140:0/1130619329 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2dfc0123f0 con 0x7f2e040b5990 2022-07-21T01:56:57.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e00ff9700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2de4053980 0x7f2de4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e037fe700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2de4053980 0x7f2de4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e00ff9700 1 -- 172.21.15.140:0/1130619329 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6075+0+0 (secure 0 0 0) 0x7f2dfc06c250 con 0x7f2e040b5990 2022-07-21T01:56:57.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.288+0000 7f2e037fe700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2de4053980 0x7f2de4055e40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f2df401a000 tx=0x7f2df4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:57.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.296+0000 7f2e00ff9700 1 -- 172.21.15.140:0/1130619329 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2e0413e500 con 0x7f2e040b5990 2022-07-21T01:56:57.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.456+0000 7f2e08fbc700 1 -- 172.21.15.140:0/1130619329 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f2e0413e500 con 0x7f2e040b5990 2022-07-21T01:56:57.460 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:56:57.456+0000 7f72a1df4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:56:57.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.464+0000 7f2e00ff9700 1 -- 172.21.15.140:0/1130619329 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 23 and data pool 22 v56) v1 ==== 189+0+0 (secure 0 0 0) 0x7f2e0413e500 con 0x7f2e040b5990 2022-07-21T01:56:57.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 -- 172.21.15.140:0/1130619329 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2de4053980 msgr2=0x7f2de4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:57.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2de4053980 0x7f2de4055e40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f2df401a000 tx=0x7f2df4014000).stop 2022-07-21T01:56:57.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 -- 172.21.15.140:0/1130619329 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e040b5990 msgr2=0x7f2e041554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:57.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e040b5990 0x7f2e041554c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f2e0410eba0 tx=0x7f2dfc0091d0).stop 2022-07-21T01:56:57.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 -- 172.21.15.140:0/1130619329 shutdown_connections 2022-07-21T01:56:57.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2de4053980 0x7f2de4055e40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2e040b5990 0x7f2e041554c0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2e040af3e0 0x7f2e0410f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 --2- 172.21.15.140:0/1130619329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2e0405d210 0x7f2e0410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 -- 172.21.15.140:0/1130619329 >> 172.21.15.140:0/1130619329 conn(0x7f2e04063480 msgr2=0x7f2e0414a950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:57.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 -- 172.21.15.140:0/1130619329 shutdown_connections 2022-07-21T01:56:57.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.468+0000 7f2e08fbc700 1 -- 172.21.15.140:0/1130619329 wait complete. 2022-07-21T01:56:57.474 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 23 and data pool 22 2022-07-21T01:56:57.482 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:56:57.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4ac6f700 1 Processor -- start 2022-07-21T01:56:57.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4ac6f700 1 -- start start 2022-07-21T01:56:57.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4ac6f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f440b5970 0x7f5f440ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4ac6f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5f440af8c0 0x7f5f440ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4ac6f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5f440aeae0 0x7f5f4405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4ac6f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f440b0b80 con 0x7f5f440aeae0 2022-07-21T01:56:57.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4ac6f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4405d4d0 con 0x7f5f440af8c0 2022-07-21T01:56:57.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4ac6f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f440b5d50 con 0x7f5f440b5970 2022-07-21T01:56:57.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4a46e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5f440aeae0 0x7f5f4405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4a46e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5f440aeae0 0x7f5f4405bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39888/0 (socket says 172.21.15.140:39888) 2022-07-21T01:56:57.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4a46e700 1 -- 172.21.15.140:0/593913100 learned_addr learned my addr 172.21.15.140:0/593913100 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:57.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4946c700 1 --2- 172.21.15.140:0/593913100 >> v2:172.21.15.152:3300/0 conn(0x7f5f440af8c0 0x7f5f440ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f49c6d700 1 --2- 172.21.15.140:0/593913100 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f440b5970 0x7f5f440ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4a46e700 1 -- 172.21.15.140:0/593913100 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f440b5970 msgr2=0x7f5f440ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:57.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4a46e700 1 --2- 172.21.15.140:0/593913100 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f440b5970 0x7f5f440ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4a46e700 1 -- 172.21.15.140:0/593913100 >> v2:172.21.15.152:3300/0 conn(0x7f5f440af8c0 msgr2=0x7f5f440ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:57.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4a46e700 1 --2- 172.21.15.140:0/593913100 >> v2:172.21.15.152:3300/0 conn(0x7f5f440af8c0 0x7f5f440ae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4a46e700 1 -- 172.21.15.140:0/593913100 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f440b0db0 con 0x7f5f440aeae0 2022-07-21T01:56:57.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f4a46e700 1 --2- 172.21.15.140:0/593913100 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f440aeae0 0x7f5f4405bd90 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f5f3c002f70 tx=0x7f5f3c006eb0).ready entity=mon.0 client_cookie=7e583f782dfecc2d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:57.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f48c6b700 1 -- 172.21.15.140:0/593913100 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f3c00d570 con 0x7f5f440aeae0 2022-07-21T01:56:57.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f48c6b700 1 -- 172.21.15.140:0/593913100 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c000e90 con 0x7f5f440aeae0 2022-07-21T01:56:57.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.584+0000 7f5f48c6b700 1 -- 172.21.15.140:0/593913100 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f3c00d900 con 0x7f5f440aeae0 2022-07-21T01:56:57.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/593913100 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f440aeae0 msgr2=0x7f5f4405bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:57.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 --2- 172.21.15.140:0/593913100 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f440aeae0 0x7f5f4405bd90 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f5f3c002f70 tx=0x7f5f3c006eb0).stop 2022-07-21T01:56:57.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/593913100 shutdown_connections 2022-07-21T01:56:57.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 --2- 172.21.15.140:0/593913100 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f440aeae0 0x7f5f4405bd90 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 --2- 172.21.15.140:0/593913100 >> v2:172.21.15.152:3300/0 conn(0x7f5f440af8c0 0x7f5f440ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 --2- 172.21.15.140:0/593913100 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f440b5970 0x7f5f440ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/593913100 >> 172.21.15.140:0/593913100 conn(0x7f5f44063490 msgr2=0x7f5f440b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:57.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/593913100 shutdown_connections 2022-07-21T01:56:57.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/593913100 wait complete. 2022-07-21T01:56:57.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 Processor -- start 2022-07-21T01:56:57.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 -- start start 2022-07-21T01:56:57.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f440aeae0 0x7f5f4414bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f440af8c0 0x7f5f44156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f440b5970 0x7f5f44156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f44141550 con 0x7f5f440b5970 2022-07-21T01:56:57.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f441403a0 con 0x7f5f440af8c0 2022-07-21T01:56:57.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4ac6f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f441560d0 con 0x7f5f440aeae0 2022-07-21T01:56:57.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4a46e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f440b5970 0x7f5f44156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4946c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f440af8c0 0x7f5f44156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f49c6d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f440aeae0 0x7f5f4414bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4946c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f440af8c0 0x7f5f44156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:45630/0 (socket says 172.21.15.140:45630) 2022-07-21T01:56:57.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4946c700 1 -- 172.21.15.140:0/1653863802 learned_addr learned my addr 172.21.15.140:0/1653863802 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:57.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4a46e700 1 -- 172.21.15.140:0/1653863802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f440aeae0 msgr2=0x7f5f4414bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:57.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4a46e700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f440aeae0 0x7f5f4414bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4a46e700 1 -- 172.21.15.140:0/1653863802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f440af8c0 msgr2=0x7f5f44156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:57.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4a46e700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f440af8c0 0x7f5f44156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:57.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4a46e700 1 -- 172.21.15.140:0/1653863802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c00b1e0 con 0x7f5f440b5970 2022-07-21T01:56:57.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.588+0000 7f5f4a46e700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f440b5970 0x7f5f44156a00 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f5f3c002f70 tx=0x7f5f3c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:57.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.592+0000 7f5f36ffd700 1 -- 172.21.15.140:0/1653863802 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f3c0116e0 con 0x7f5f440b5970 2022-07-21T01:56:57.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.592+0000 7f5f36ffd700 1 -- 172.21.15.140:0/1653863802 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c011840 con 0x7f5f440b5970 2022-07-21T01:56:57.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.592+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/1653863802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f44144e50 con 0x7f5f440b5970 2022-07-21T01:56:57.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.592+0000 7f5f36ffd700 1 -- 172.21.15.140:0/1653863802 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f3c011ba0 con 0x7f5f440b5970 2022-07-21T01:56:57.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.592+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/1653863802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f44157090 con 0x7f5f440b5970 2022-07-21T01:56:57.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.592+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/1653863802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f44140560 con 0x7f5f440b5970 2022-07-21T01:56:57.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.592+0000 7f5f36ffd700 1 -- 172.21.15.140:0/1653863802 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5f3c01a040 con 0x7f5f440b5970 2022-07-21T01:56:57.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.592+0000 7f5f36ffd700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f24053a70 0x7f5f24055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:57.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.592+0000 7f5f36ffd700 1 -- 172.21.15.140:0/1653863802 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5f3c06ca60 con 0x7f5f440b5970 2022-07-21T01:56:57.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.592+0000 7f5f49c6d700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f24053a70 0x7f5f24055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:57.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.596+0000 7f5f49c6d700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f24053a70 0x7f5f24055f30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f5f4000b000 tx=0x7f5f40006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:57.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.600+0000 7f5f36ffd700 1 -- 172.21.15.140:0/1653863802 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5f3c043930 con 0x7f5f440b5970 2022-07-21T01:56:57.766 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:57.764+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/1653863802 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5f44157090 con 0x7f5f440b5970 2022-07-21T01:56:58.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.472+0000 7f5f36ffd700 1 -- 172.21.15.140:0/1653863802 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5f44157090 con 0x7f5f440b5970 2022-07-21T01:56:58.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/1653863802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f24053a70 msgr2=0x7f5f24055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:58.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f24053a70 0x7f5f24055f30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f5f4000b000 tx=0x7f5f40006df0).stop 2022-07-21T01:56:58.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/1653863802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f440b5970 msgr2=0x7f5f44156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:58.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f440b5970 0x7f5f44156a00 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f5f3c002f70 tx=0x7f5f3c015000).stop 2022-07-21T01:56:58.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/1653863802 shutdown_connections 2022-07-21T01:56:58.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f24053a70 0x7f5f24055f30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:58.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f440b5970 0x7f5f44156a00 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:58.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f440af8c0 0x7f5f44156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:58.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 --2- 172.21.15.140:0/1653863802 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f440aeae0 0x7f5f4414bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:58.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/1653863802 >> 172.21.15.140:0/1653863802 conn(0x7f5f44063490 msgr2=0x7f5f440b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:58.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/1653863802 shutdown_connections 2022-07-21T01:56:58.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.476+0000 7f5f4ac6f700 1 -- 172.21.15.140:0/1653863802 wait complete. 2022-07-21T01:56:58.495 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:56:58.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.612+0000 7fa81c6a3700 1 Processor -- start 2022-07-21T01:56:58.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.612+0000 7fa81c6a3700 1 -- start start 2022-07-21T01:56:58.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.612+0000 7fa81c6a3700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa81405cf40 0x7fa814146200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:58.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.612+0000 7fa81c6a3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa8140b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:58.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.612+0000 7fa81c6a3700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8140b2370 con 0x7fa8140620a0 2022-07-21T01:56:58.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.612+0000 7fa81c6a3700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8140b1f00 con 0x7fa81405cf40 2022-07-21T01:56:58.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.612+0000 7fa81c6a3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b2610 con 0x7fa814066650 2022-07-21T01:56:58.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.612+0000 7fa81ac40700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa8140620a0 0x7fa8141420d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54048/0 (socket says 172.21.15.140:54048) 2022-07-21T01:56:58.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.612+0000 7fa81ac40700 1 -- 172.21.15.140:0/3888966622 learned_addr learned my addr 172.21.15.140:0/3888966622 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:56:58.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.612+0000 7fa81a43f700 1 --2- 172.21.15.140:0/3888966622 >> v2:172.21.15.152:3300/0 conn(0x7fa81405cf40 0x7fa814146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:58.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.612+0000 7fa819c3e700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa8140b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:58.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636038563 0 0) 0x7fa8140b2370 con 0x7fa8140620a0 2022-07-21T01:56:58.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7fc003260 con 0x7fa8140620a0 2022-07-21T01:56:58.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2651971280 0 0) 0x7fa7fc003260 con 0x7fa8140620a0 2022-07-21T01:56:58.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 msgr2=0x7fa8140b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:58.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa8140b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:58.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 >> v2:172.21.15.152:3300/0 conn(0x7fa81405cf40 msgr2=0x7fa814146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:58.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 --2- 172.21.15.140:0/3888966622 >> v2:172.21.15.152:3300/0 conn(0x7fa81405cf40 0x7fa814146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:58.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140b0780 con 0x7fa8140620a0 2022-07-21T01:56:58.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fa80c0019e0 con 0x7fa8140620a0 2022-07-21T01:56:58.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 >> v1:172.21.15.140:6789/0 conn(0x7fa8140620a0 legacy=0x7fa8141420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:58.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7fc003750 0x7fa7fc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:58.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa7fc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:58.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7fc0049f0 0x7fa7fc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:58.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b2610 con 0x7fa7fc0049f0 2022-07-21T01:56:58.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc004360 con 0x7fa7fc003750 2022-07-21T01:56:58.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc00b7d0 con 0x7fa814066650 2022-07-21T01:56:58.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa819c3e700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa7fc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:58.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81a43f700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7fc003750 0x7fa7fc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:58.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81ac40700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7fc0049f0 0x7fa7fc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:58.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa819c3e700 1 -- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7fc003750 msgr2=0x7fa7fc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:58.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa819c3e700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7fc003750 0x7fa7fc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:58.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa819c3e700 1 -- 172.21.15.140:0/3888966622 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7fc0049f0 msgr2=0x7fa7fc004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:58.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa819c3e700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa7fc0049f0 0x7fa7fc004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:58.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa819c3e700 1 -- 172.21.15.140:0/3888966622 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc003260 con 0x7fa814066650 2022-07-21T01:56:58.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81a43f700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa7fc003750 0x7fa7fc003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:56:58.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa819c3e700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa7fc004500 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fa81000c000 tx=0x7fa810006df0).ready entity=mon.2 client_cookie=13e29abbb4af4465 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:58.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa81000e830 con 0x7fa814066650 2022-07-21T01:56:58.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81000e990 con 0x7fa814066650 2022-07-21T01:56:58.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81943d700 1 -- 172.21.15.140:0/3888966622 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa81000ecf0 con 0x7fa814066650 2022-07-21T01:56:58.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81c6a3700 1 -- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 msgr2=0x7fa7fc004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:58.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81c6a3700 1 --2- 172.21.15.140:0/3888966622 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa7fc004500 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fa81000c000 tx=0x7fa810006df0).stop 2022-07-21T01:56:58.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81ac40700 1 -- 172.21.15.140:0/3888966622 reap_dead start 2022-07-21T01:56:58.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81c6a3700 1 -- 172.21.15.140:0/3888966622 shutdown_connections 2022-07-21T01:56:58.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81c6a3700 1 -- 172.21.15.140:0/3888966622 >> 172.21.15.140:0/3888966622 conn(0x7fa8140b7cc0 msgr2=0x7fa8140b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:58.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81c6a3700 1 -- 172.21.15.140:0/3888966622 shutdown_connections 2022-07-21T01:56:58.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81c6a3700 1 -- 172.21.15.140:0/3888966622 wait complete. 2022-07-21T01:56:58.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81c6a3700 1 Processor -- start 2022-07-21T01:56:58.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.616+0000 7fa81c6a3700 1 -- start start 2022-07-21T01:56:58.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81c6a3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa814097de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:58.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81c6a3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa81409c680 0x7fa81409b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:58.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81c6a3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa81409b870 0x7fa814151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:58.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81c6a3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81409a9b0 con 0x7fa81409c680 2022-07-21T01:56:58.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81c6a3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81409a690 con 0x7fa81409b870 2022-07-21T01:56:58.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81c6a3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140976a0 con 0x7fa814066650 2022-07-21T01:56:58.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa819c3e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa81409c680 0x7fa81409b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:58.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa819c3e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa81409c680 0x7fa81409b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39912/0 (socket says 172.21.15.140:39912) 2022-07-21T01:56:58.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa819c3e700 1 -- 172.21.15.140:0/2384526017 learned_addr learned my addr 172.21.15.140:0/2384526017 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:58.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81a43f700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa814097de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:58.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81ac40700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa81409b870 0x7fa814151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:58.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa819c3e700 1 -- 172.21.15.140:0/2384526017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 msgr2=0x7fa814097de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:58.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa819c3e700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa814097de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:58.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa819c3e700 1 -- 172.21.15.140:0/2384526017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa81409b870 msgr2=0x7fa814151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:58.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa819c3e700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa81409b870 0x7fa814151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:58.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa819c3e700 1 -- 172.21.15.140:0/2384526017 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa814097940 con 0x7fa81409c680 2022-07-21T01:56:58.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81a43f700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa814097de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:58.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa819c3e700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa81409c680 0x7fa81409b380 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7fa81000c000 tx=0x7fa810019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:58.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa8037fe700 1 -- 172.21.15.140:0/2384526017 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa81001b820 con 0x7fa81409c680 2022-07-21T01:56:58.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa8037fe700 1 -- 172.21.15.140:0/2384526017 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa810000e90 con 0x7fa81409c680 2022-07-21T01:56:58.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa8037fe700 1 -- 172.21.15.140:0/2384526017 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa81001bb70 con 0x7fa81409c680 2022-07-21T01:56:58.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81c6a3700 1 -- 172.21.15.140:0/2384526017 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81409a380 con 0x7fa81409c680 2022-07-21T01:56:58.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81c6a3700 1 -- 172.21.15.140:0/2384526017 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8141521c0 con 0x7fa81409c680 2022-07-21T01:56:58.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa8037fe700 1 -- 172.21.15.140:0/2384526017 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa81001bcd0 con 0x7fa81409c680 2022-07-21T01:56:58.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa81c6a3700 1 -- 172.21.15.140:0/2384526017 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa814097ad0 con 0x7fa81409c680 2022-07-21T01:56:58.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.620+0000 7fa8037fe700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7f4053b00 0x7fa7f4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:58.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.624+0000 7fa8037fe700 1 -- 172.21.15.140:0/2384526017 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa81006d420 con 0x7fa81409c680 2022-07-21T01:56:58.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.624+0000 7fa81a43f700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7f4053b00 0x7fa7f4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:58.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.624+0000 7fa81a43f700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7f4053b00 0x7fa7f4055fc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fa80400b000 tx=0x7fa804008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:58.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.628+0000 7fa8037fe700 1 -- 172.21.15.140:0/2384526017 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa810044300 con 0x7fa81409c680 2022-07-21T01:56:58.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:58.756+0000 7fa81c6a3700 1 -- 172.21.15.140:0/2384526017 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa8140574e0 con 0x7fa81409c680 2022-07-21T01:56:59.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.476+0000 7fa8037fe700 1 -- 172.21.15.140:0/2384526017 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v121) v1 ==== 185+0+0 (secure 0 0 0) 0x7fa810047920 con 0x7fa81409c680 2022-07-21T01:56:59.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 -- 172.21.15.140:0/2384526017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7f4053b00 msgr2=0x7fa7f4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:59.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7f4053b00 0x7fa7f4055fc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fa80400b000 tx=0x7fa804008040).stop 2022-07-21T01:56:59.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 -- 172.21.15.140:0/2384526017 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa81409c680 msgr2=0x7fa81409b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:59.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa81409c680 0x7fa81409b380 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7fa81000c000 tx=0x7fa810019000).stop 2022-07-21T01:56:59.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 -- 172.21.15.140:0/2384526017 shutdown_connections 2022-07-21T01:56:59.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa7f4053b00 0x7fa7f4055fc0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:59.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa81409b870 0x7fa814151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:59.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa81409c680 0x7fa81409b380 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:59.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 --2- 172.21.15.140:0/2384526017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa814066650 0x7fa814097de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:59.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 -- 172.21.15.140:0/2384526017 >> 172.21.15.140:0/2384526017 conn(0x7fa8140b7cc0 msgr2=0x7fa814066f30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:59.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 -- 172.21.15.140:0/2384526017 shutdown_connections 2022-07-21T01:56:59.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.480+0000 7fa81c6a3700 1 -- 172.21.15.140:0/2384526017 wait complete. 2022-07-21T01:56:59.486 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:56:59.496 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:56:59.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6f25c700 1 Processor -- start 2022-07-21T01:56:59.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6f25c700 1 -- start start 2022-07-21T01:56:59.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6f25c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcc680af7f0 0x7fcc68142db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:59.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6f25c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcc68067540 0x7fcc68067990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:59.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6f25c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc68060340 con 0x7fcc6805fb80 2022-07-21T01:56:59.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6f25c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc680aec80 con 0x7fcc68067540 2022-07-21T01:56:59.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6f25c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6805ff60 con 0x7fcc680af7f0 2022-07-21T01:56:59.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6ea5b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcc68067540 0x7fcc68067990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:59.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6ea5b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcc68067540 0x7fcc68067990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39922/0 (socket says 172.21.15.140:39922) 2022-07-21T01:56:59.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6ea5b700 1 -- 172.21.15.140:0/3953687021 learned_addr learned my addr 172.21.15.140:0/3953687021 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:59.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6da59700 1 --2- 172.21.15.140:0/3953687021 >> v2:172.21.15.152:3300/0 conn(0x7fcc680af7f0 0x7fcc68142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:59.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6ea5b700 1 -- 172.21.15.140:0/3953687021 >> v2:172.21.15.152:3300/0 conn(0x7fcc680af7f0 msgr2=0x7fcc68142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:56:59.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6ea5b700 1 --2- 172.21.15.140:0/3953687021 >> v2:172.21.15.152:3300/0 conn(0x7fcc680af7f0 0x7fcc68142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:59.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6ea5b700 1 -- 172.21.15.140:0/3953687021 >> v1:172.21.15.152:6789/0 conn(0x7fcc6805fb80 legacy=0x7fcc680af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:59.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6d258700 1 -- 172.21.15.140:0/3953687021 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255514049 0 0) 0x7fcc68060340 con 0x7fcc6805fb80 2022-07-21T01:56:59.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6ea5b700 1 -- 172.21.15.140:0/3953687021 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68060570 con 0x7fcc68067540 2022-07-21T01:56:59.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6ea5b700 1 --2- 172.21.15.140:0/3953687021 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcc68067540 0x7fcc68067990 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7fcc60002f70 tx=0x7fcc60006eb0).ready entity=mon.0 client_cookie=27bc6d6be8769147 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:59.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6d258700 1 -- 172.21.15.140:0/3953687021 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6000d5c0 con 0x7fcc68067540 2022-07-21T01:56:59.617 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6d258700 1 -- 172.21.15.140:0/3953687021 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc60000e90 con 0x7fcc68067540 2022-07-21T01:56:59.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.608+0000 7fcc6d258700 1 -- 172.21.15.140:0/3953687021 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6000d910 con 0x7fcc68067540 2022-07-21T01:56:59.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 -- 172.21.15.140:0/3953687021 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcc68067540 msgr2=0x7fcc68067990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:59.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 --2- 172.21.15.140:0/3953687021 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcc68067540 0x7fcc68067990 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7fcc60002f70 tx=0x7fcc60006eb0).stop 2022-07-21T01:56:59.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 -- 172.21.15.140:0/3953687021 shutdown_connections 2022-07-21T01:56:59.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 --2- 172.21.15.140:0/3953687021 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcc68067540 0x7fcc68067990 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:59.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 --2- 172.21.15.140:0/3953687021 >> v2:172.21.15.152:3300/0 conn(0x7fcc680af7f0 0x7fcc68142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:59.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 -- 172.21.15.140:0/3953687021 >> 172.21.15.140:0/3953687021 conn(0x7fcc680632d0 msgr2=0x7fcc680636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:56:59.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 -- 172.21.15.140:0/3953687021 shutdown_connections 2022-07-21T01:56:59.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 -- 172.21.15.140:0/3953687021 wait complete. 2022-07-21T01:56:59.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 Processor -- start 2022-07-21T01:56:59.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 -- start start 2022-07-21T01:56:59.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc6805fb80 0x7fcc681441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:59.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc68067540 0x7fcc6813e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:59.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcc680af7f0 0x7fcc68139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:59.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6813e370 con 0x7fcc680af7f0 2022-07-21T01:56:59.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6813cca0 con 0x7fcc6805fb80 2022-07-21T01:56:59.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6f25c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68140540 con 0x7fcc68067540 2022-07-21T01:56:59.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6ea5b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcc680af7f0 0x7fcc68139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:59.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6ea5b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcc680af7f0 0x7fcc68139e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39928/0 (socket says 172.21.15.140:39928) 2022-07-21T01:56:59.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6ea5b700 1 -- 172.21.15.140:0/678998949 learned_addr learned my addr 172.21.15.140:0/678998949 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:56:59.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6da59700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc68067540 0x7fcc6813e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:59.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6e25a700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc6805fb80 0x7fcc681441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:59.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6ea5b700 1 -- 172.21.15.140:0/678998949 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc68067540 msgr2=0x7fcc6813e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:59.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6ea5b700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc68067540 0x7fcc6813e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:59.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6ea5b700 1 -- 172.21.15.140:0/678998949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc6805fb80 msgr2=0x7fcc681441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:56:59.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6ea5b700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc6805fb80 0x7fcc681441b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:56:59.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6ea5b700 1 -- 172.21.15.140:0/678998949 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6000b1e0 con 0x7fcc680af7f0 2022-07-21T01:56:59.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6e25a700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc6805fb80 0x7fcc681441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:59.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6da59700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc68067540 0x7fcc6813e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:56:59.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc6ea5b700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcc680af7f0 0x7fcc68139e30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7fcc60002f70 tx=0x7fcc60015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:59.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc677fe700 1 -- 172.21.15.140:0/678998949 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6000de40 con 0x7fcc680af7f0 2022-07-21T01:56:59.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.612+0000 7fcc677fe700 1 -- 172.21.15.140:0/678998949 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc60011ce0 con 0x7fcc680af7f0 2022-07-21T01:56:59.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.616+0000 7fcc6f25c700 1 -- 172.21.15.140:0/678998949 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6813c870 con 0x7fcc680af7f0 2022-07-21T01:56:59.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.616+0000 7fcc677fe700 1 -- 172.21.15.140:0/678998949 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc6001e650 con 0x7fcc680af7f0 2022-07-21T01:56:59.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.616+0000 7fcc6f25c700 1 -- 172.21.15.140:0/678998949 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6813c560 con 0x7fcc680af7f0 2022-07-21T01:56:59.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.616+0000 7fcc6f25c700 1 -- 172.21.15.140:0/678998949 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6813e4b0 con 0x7fcc680af7f0 2022-07-21T01:56:59.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.616+0000 7fcc677fe700 1 -- 172.21.15.140:0/678998949 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcc60025030 con 0x7fcc680af7f0 2022-07-21T01:56:59.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.616+0000 7fcc677fe700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcc4c053b00 0x7fcc4c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:56:59.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.616+0000 7fcc677fe700 1 -- 172.21.15.140:0/678998949 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5721+0+0 (secure 0 0 0) 0x7fcc6006cf50 con 0x7fcc680af7f0 2022-07-21T01:56:59.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.616+0000 7fcc6e25a700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcc4c053b00 0x7fcc4c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:56:59.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.616+0000 7fcc6e25a700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcc4c053b00 0x7fcc4c055fc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fcc5000b000 tx=0x7fcc50009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:56:59.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.624+0000 7fcc677fe700 1 -- 172.21.15.140:0/678998949 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcc60043fd0 con 0x7fcc680af7f0 2022-07-21T01:56:59.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:56:59.748+0000 7fcc6f25c700 1 -- 172.21.15.140:0/678998949 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fcc680574e0 con 0x7fcc680af7f0 2022-07-21T01:57:00.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.488+0000 7fcc677fe700 1 -- 172.21.15.140:0/678998949 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v122) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcc60023070 con 0x7fcc680af7f0 2022-07-21T01:57:00.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 -- 172.21.15.140:0/678998949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcc4c053b00 msgr2=0x7fcc4c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:00.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcc4c053b00 0x7fcc4c055fc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fcc5000b000 tx=0x7fcc50009040).stop 2022-07-21T01:57:00.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 -- 172.21.15.140:0/678998949 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcc680af7f0 msgr2=0x7fcc68139e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:00.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcc680af7f0 0x7fcc68139e30 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7fcc60002f70 tx=0x7fcc60015000).stop 2022-07-21T01:57:00.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 -- 172.21.15.140:0/678998949 shutdown_connections 2022-07-21T01:57:00.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcc4c053b00 0x7fcc4c055fc0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcc680af7f0 0x7fcc68139e30 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc68067540 0x7fcc6813e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 --2- 172.21.15.140:0/678998949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc6805fb80 0x7fcc681441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 -- 172.21.15.140:0/678998949 >> 172.21.15.140:0/678998949 conn(0x7fcc680632d0 msgr2=0x7fcc680b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:00.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 -- 172.21.15.140:0/678998949 shutdown_connections 2022-07-21T01:57:00.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.492+0000 7fcc6f25c700 1 -- 172.21.15.140:0/678998949 wait complete. 2022-07-21T01:57:00.499 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:57:00.508 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-07-21T01:57:00.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd928cb1700 1 Processor -- start 2022-07-21T01:57:00.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd928cb1700 1 -- start start 2022-07-21T01:57:00.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd928cb1700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd92409df40 0x7fd924145680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:00.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd928cb1700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd924060ee0 0x7fd92409cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:00.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd928cb1700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9240607e0 con 0x7fd9240b4d60 2022-07-21T01:57:00.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd928cb1700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd924060d20 con 0x7fd924060ee0 2022-07-21T01:57:00.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd928cb1700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd924060a80 con 0x7fd92409df40 2022-07-21T01:57:00.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd922d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd924060ee0 0x7fd92409cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:00.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd922d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd924060ee0 0x7fd92409cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39936/0 (socket says 172.21.15.140:39936) 2022-07-21T01:57:00.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd92259c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd92409df40 0x7fd924145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:00.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd922d9d700 1 -- 172.21.15.140:0/2114822741 learned_addr learned my addr 172.21.15.140:0/2114822741 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:00.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd922d9d700 1 -- 172.21.15.140:0/2114822741 >> v2:172.21.15.152:3300/0 conn(0x7fd92409df40 msgr2=0x7fd924145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:00.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd922d9d700 1 --2- 172.21.15.140:0/2114822741 >> v2:172.21.15.152:3300/0 conn(0x7fd92409df40 0x7fd924145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd922d9d700 1 -- 172.21.15.140:0/2114822741 >> v1:172.21.15.152:6789/0 conn(0x7fd9240b4d60 legacy=0x7fd924145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:00.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd922d9d700 1 -- 172.21.15.140:0/2114822741 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92409e320 con 0x7fd924060ee0 2022-07-21T01:57:00.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd922d9d700 1 --2- 172.21.15.140:0/2114822741 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd924060ee0 0x7fd92409cba0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7fd91c0032f0 tx=0x7fd91c009290).ready entity=mon.0 client_cookie=fa510e94d5e848f8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:00.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd921d9b700 1 -- 172.21.15.140:0/2114822741 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd91c00b870 con 0x7fd924060ee0 2022-07-21T01:57:00.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd921d9b700 1 -- 172.21.15.140:0/2114822741 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c00b9d0 con 0x7fd924060ee0 2022-07-21T01:57:00.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd921d9b700 1 -- 172.21.15.140:0/2114822741 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd91c00bb40 con 0x7fd924060ee0 2022-07-21T01:57:00.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd928cb1700 1 -- 172.21.15.140:0/2114822741 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd924060ee0 msgr2=0x7fd92409cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:00.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.620+0000 7fd928cb1700 1 --2- 172.21.15.140:0/2114822741 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd924060ee0 0x7fd92409cba0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7fd91c0032f0 tx=0x7fd91c009290).stop 2022-07-21T01:57:00.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.624+0000 7fd928cb1700 1 -- 172.21.15.140:0/2114822741 shutdown_connections 2022-07-21T01:57:00.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.624+0000 7fd928cb1700 1 --2- 172.21.15.140:0/2114822741 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd924060ee0 0x7fd92409cba0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.624+0000 7fd928cb1700 1 --2- 172.21.15.140:0/2114822741 >> v2:172.21.15.152:3300/0 conn(0x7fd92409df40 0x7fd924145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.624+0000 7fd928cb1700 1 -- 172.21.15.140:0/2114822741 >> 172.21.15.140:0/2114822741 conn(0x7fd9240b5f30 msgr2=0x7fd9240b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:00.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.624+0000 7fd928cb1700 1 -- 172.21.15.140:0/2114822741 shutdown_connections 2022-07-21T01:57:00.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.624+0000 7fd928cb1700 1 -- 172.21.15.140:0/2114822741 wait complete. 2022-07-21T01:57:00.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.624+0000 7fd928cb1700 1 Processor -- start 2022-07-21T01:57:00.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.624+0000 7fd928cb1700 1 -- start start 2022-07-21T01:57:00.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd928cb1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd924060ee0 0x7fd924155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:00.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd928cb1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd92409df40 0x7fd924149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:00.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd928cb1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9240b4d60 0x7fd924092090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:00.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd928cb1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd924091c50 con 0x7fd92409df40 2022-07-21T01:57:00.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd928cb1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240b14b0 con 0x7fd924060ee0 2022-07-21T01:57:00.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd928cb1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240b1750 con 0x7fd9240b4d60 2022-07-21T01:57:00.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd919d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd92409df40 0x7fd924149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:00.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd919d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd92409df40 0x7fd924149060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39942/0 (socket says 172.21.15.140:39942) 2022-07-21T01:57:00.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd919d9b700 1 -- 172.21.15.140:0/1590640601 learned_addr learned my addr 172.21.15.140:0/1590640601 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:00.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd922d9d700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9240b4d60 0x7fd924092090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:00.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd92259c700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd924060ee0 0x7fd924155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:00.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd919d9b700 1 -- 172.21.15.140:0/1590640601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9240b4d60 msgr2=0x7fd924092090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:00.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd919d9b700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9240b4d60 0x7fd924092090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd919d9b700 1 -- 172.21.15.140:0/1590640601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd924060ee0 msgr2=0x7fd924155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:00.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd919d9b700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd924060ee0 0x7fd924155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd919d9b700 1 -- 172.21.15.140:0/1590640601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c009530 con 0x7fd92409df40 2022-07-21T01:57:00.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd919d9b700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd92409df40 0x7fd924149060 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7fd924067ed0 tx=0x7fd90c007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:00.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd91bfff700 1 -- 172.21.15.140:0/1590640601 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd90c00ccd0 con 0x7fd92409df40 2022-07-21T01:57:00.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd91bfff700 1 -- 172.21.15.140:0/1590640601 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c00ce30 con 0x7fd92409df40 2022-07-21T01:57:00.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd928cb1700 1 -- 172.21.15.140:0/1590640601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9240b19f0 con 0x7fd92409df40 2022-07-21T01:57:00.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd91bfff700 1 -- 172.21.15.140:0/1590640601 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd90c0125f0 con 0x7fd92409df40 2022-07-21T01:57:00.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd928cb1700 1 -- 172.21.15.140:0/1590640601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd924090830 con 0x7fd92409df40 2022-07-21T01:57:00.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.628+0000 7fd928cb1700 1 -- 172.21.15.140:0/1590640601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9240b2610 con 0x7fd92409df40 2022-07-21T01:57:00.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.632+0000 7fd91bfff700 1 -- 172.21.15.140:0/1590640601 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd90c019030 con 0x7fd92409df40 2022-07-21T01:57:00.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.632+0000 7fd91bfff700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd908053950 0x7fd908055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:00.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.632+0000 7fd91bfff700 1 -- 172.21.15.140:0/1590640601 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6075+0+0 (secure 0 0 0) 0x7fd90c06b180 con 0x7fd92409df40 2022-07-21T01:57:00.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.632+0000 7fd92259c700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd908053950 0x7fd908055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:00.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.632+0000 7fd92259c700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd908053950 0x7fd908055e10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7fd914006000 tx=0x7fd914006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:00.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.636+0000 7fd91bfff700 1 -- 172.21.15.140:0/1590640601 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd9240b2610 con 0x7fd92409df40 2022-07-21T01:57:00.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.804+0000 7fd928cb1700 1 -- 172.21.15.140:0/1590640601 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd9240b2610 con 0x7fd92409df40 2022-07-21T01:57:00.807 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:57:00.804+0000 7f72a1df4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:57:00.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.812+0000 7fd91bfff700 1 -- 172.21.15.140:0/1590640601 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 22 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd9240b2610 con 0x7fd92409df40 2022-07-21T01:57:00.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 -- 172.21.15.140:0/1590640601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd908053950 msgr2=0x7fd908055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:00.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd908053950 0x7fd908055e10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7fd914006000 tx=0x7fd914006d90).stop 2022-07-21T01:57:00.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 -- 172.21.15.140:0/1590640601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd92409df40 msgr2=0x7fd924149060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:00.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd92409df40 0x7fd924149060 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7fd924067ed0 tx=0x7fd90c007c20).stop 2022-07-21T01:57:00.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 -- 172.21.15.140:0/1590640601 shutdown_connections 2022-07-21T01:57:00.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd908053950 0x7fd908055e10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd9240b4d60 0x7fd924092090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd92409df40 0x7fd924149060 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 --2- 172.21.15.140:0/1590640601 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd924060ee0 0x7fd924155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:00.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 -- 172.21.15.140:0/1590640601 >> 172.21.15.140:0/1590640601 conn(0x7fd9240b5f30 msgr2=0x7fd9240b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:00.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 -- 172.21.15.140:0/1590640601 shutdown_connections 2022-07-21T01:57:00.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:00.816+0000 7fd928cb1700 1 -- 172.21.15.140:0/1590640601 wait complete. 2022-07-21T01:57:00.821 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 24 and data pool 22 2022-07-21T01:57:00.832 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:57:00.832 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:00.845 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:00.866 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:00.888 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:00.910 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:00.932 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:57:00.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:00.933 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:00.934 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:00.941 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:57:00.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:00.943 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:57:00.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:57:00.944 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:00.990 INFO:teuthology.orchestra.run.smithi191.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:57:00.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:00.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:00.992 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:00.993 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:01.000 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:57:01.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:01.003 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:57:01.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:57:01.005 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:01.012 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:57:01.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:01.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:01.016 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:01.017 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:57:01.025 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:57:01.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:01.026 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:57:01.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:57:01.027 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:57:01.034 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:57:01.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:01.036 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:57:01.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 Processor -- start 2022-07-21T01:57:01.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 -- start start 2022-07-21T01:57:01.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fc80b0080 0x7f6fc813f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc80acdb0 con 0x7f6fc80ac9d0 2022-07-21T01:57:01.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc80b1450 con 0x7f6fc80aedc0 2022-07-21T01:57:01.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80af260 con 0x7f6fc80b0080 2022-07-21T01:57:01.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc6ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f6fc80ac9d0 0x7f6fc813e780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54094/0 (socket says 172.21.15.140:54094) 2022-07-21T01:57:01.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc6ffd700 1 -- 172.21.15.140:0/718034607 learned_addr learned my addr 172.21.15.140:0/718034607 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:01.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc77fe700 1 --2- 172.21.15.140:0/718034607 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fc80b0080 0x7f6fc813f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc67fc700 1 -- 172.21.15.140:0/718034607 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033742817 0 0) 0x7f6fc80acdb0 con 0x7f6fc80ac9d0 2022-07-21T01:57:01.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc67fc700 1 -- 172.21.15.140:0/718034607 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa8004370 con 0x7f6fc80ac9d0 2022-07-21T01:57:01.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc67fc700 1 -- 172.21.15.140:0/718034607 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853242126 0 0) 0x7f6fc80b1450 con 0x7f6fc80aedc0 2022-07-21T01:57:01.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc67fc700 1 -- 172.21.15.140:0/718034607 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc80acdb0 con 0x7f6fc80aedc0 2022-07-21T01:57:01.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc77fe700 1 -- 172.21.15.140:0/718034607 >> v1:172.21.15.152:6789/0 conn(0x7f6fc80aedc0 legacy=0x7f6fc813f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc77fe700 1 -- 172.21.15.140:0/718034607 >> v1:172.21.15.140:6789/0 conn(0x7f6fc80ac9d0 legacy=0x7f6fc813e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc67fc700 1 -- 172.21.15.140:0/718034607 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4162579609 0 0) 0x7f6fa8004370 con 0x7f6fc80ac9d0 2022-07-21T01:57:01.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc77fe700 1 -- 172.21.15.140:0/718034607 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc80af450 con 0x7f6fc80b0080 2022-07-21T01:57:01.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc77fe700 1 --2- 172.21.15.140:0/718034607 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fc80b0080 0x7f6fc813f9b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f6fb0001110 tx=0x7f6fc00091d0).ready entity=mon.2 client_cookie=694198a3fe13bf9e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:01.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc67fc700 1 -- 172.21.15.140:0/718034607 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fc000f8a0 con 0x7f6fc80b0080 2022-07-21T01:57:01.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc67fc700 1 -- 172.21.15.140:0/718034607 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc000fa00 con 0x7f6fc80b0080 2022-07-21T01:57:01.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fc67fc700 1 -- 172.21.15.140:0/718034607 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fc000fd60 con 0x7f6fc80b0080 2022-07-21T01:57:01.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 -- 172.21.15.140:0/718034607 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fc80b0080 msgr2=0x7f6fc813f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 --2- 172.21.15.140:0/718034607 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fc80b0080 0x7f6fc813f9b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f6fb0001110 tx=0x7f6fc00091d0).stop 2022-07-21T01:57:01.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 -- 172.21.15.140:0/718034607 shutdown_connections 2022-07-21T01:57:01.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 --2- 172.21.15.140:0/718034607 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fc80b0080 0x7f6fc813f9b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 -- 172.21.15.140:0/718034607 >> 172.21.15.140:0/718034607 conn(0x7f6fc80611c0 msgr2=0x7f6fc8062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:01.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 -- 172.21.15.140:0/718034607 shutdown_connections 2022-07-21T01:57:01.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.148+0000 7f6fcda00700 1 -- 172.21.15.140:0/718034607 wait complete. 2022-07-21T01:57:01.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fcda00700 1 Processor -- start 2022-07-21T01:57:01.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fcda00700 1 -- start start 2022-07-21T01:57:01.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fcda00700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fc80aedc0 0x7f6fc80a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fcda00700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fc80b0080 0x7f6fc80a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fcda00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fc80a4900 0x7f6fc80abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fcda00700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80a74e0 con 0x7f6fc80b0080 2022-07-21T01:57:01.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fcda00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80a6c70 con 0x7f6fc80a4900 2022-07-21T01:57:01.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fcda00700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80a56c0 con 0x7f6fc80aedc0 2022-07-21T01:57:01.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fbffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fc80b0080 0x7f6fc80a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fbffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fc80b0080 0x7f6fc80a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39954/0 (socket says 172.21.15.140:39954) 2022-07-21T01:57:01.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fbffff700 1 -- 172.21.15.140:0/2128037226 learned_addr learned my addr 172.21.15.140:0/2128037226 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:01.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fc6ffd700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fc80aedc0 0x7f6fc80a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fc77fe700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fc80a4900 0x7f6fc80abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fbffff700 1 -- 172.21.15.140:0/2128037226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fc80aedc0 msgr2=0x7f6fc80a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fbffff700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fc80aedc0 0x7f6fc80a89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fbffff700 1 -- 172.21.15.140:0/2128037226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fc80a4900 msgr2=0x7f6fc80abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fbffff700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fc80a4900 0x7f6fc80abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fbffff700 1 -- 172.21.15.140:0/2128037226 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc0009470 con 0x7f6fc80b0080 2022-07-21T01:57:01.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fbffff700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fc80b0080 0x7f6fc80a5c40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f6fb0000bc0 tx=0x7f6fb0007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:01.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fc4ff9700 1 -- 172.21.15.140:0/2128037226 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fb0012400 con 0x7f6fc80b0080 2022-07-21T01:57:01.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fc4ff9700 1 -- 172.21.15.140:0/2128037226 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb0012560 con 0x7f6fc80b0080 2022-07-21T01:57:01.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fcda00700 1 -- 172.21.15.140:0/2128037226 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc80a5960 con 0x7f6fc80b0080 2022-07-21T01:57:01.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fc4ff9700 1 -- 172.21.15.140:0/2128037226 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fb00128c0 con 0x7f6fc80b0080 2022-07-21T01:57:01.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.152+0000 7f6fcda00700 1 -- 172.21.15.140:0/2128037226 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc80ac160 con 0x7f6fc80b0080 2022-07-21T01:57:01.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.156+0000 7f6fcda00700 1 -- 172.21.15.140:0/2128037226 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc80574e0 con 0x7f6fc80b0080 2022-07-21T01:57:01.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.156+0000 7f6fc4ff9700 1 -- 172.21.15.140:0/2128037226 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6fb0019030 con 0x7f6fc80b0080 2022-07-21T01:57:01.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.156+0000 7f6fc4ff9700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fac053b00 0x7f6fac055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.156+0000 7f6fc4ff9700 1 -- 172.21.15.140:0/2128037226 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f6fb006b160 con 0x7f6fc80b0080 2022-07-21T01:57:01.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.156+0000 7f6fc6ffd700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fac053b00 0x7f6fac055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.160+0000 7f6fc6ffd700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fac053b00 0x7f6fac055fc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f6fb800d000 tx=0x7f6fb8000f90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:01.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.164+0000 7f6fc4ff9700 1 -- 172.21.15.140:0/2128037226 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6fb00420e0 con 0x7f6fc80b0080 2022-07-21T01:57:01.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.292+0000 7f6fcda00700 1 -- 172.21.15.140:0/2128037226 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6fc8062fe0 con 0x7f6fc80b0080 2022-07-21T01:57:01.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.292+0000 7f6fc4ff9700 1 -- 172.21.15.140:0/2128037226 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v123) v1 ==== 81+0+33922 (secure 0 0 0) 0x7f6fb00294a0 con 0x7f6fc80b0080 2022-07-21T01:57:01.296 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:01.296 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:57:01.296 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 123, 2022-07-21T01:57:01.297 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:57:01.297 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:57:01.298 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:57:00.807346+0000", 2022-07-21T01:57:01.298 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:01.298 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:01.299 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:57:01.299 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:57:01.300 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:57:01.300 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:57:01.301 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:57:01.301 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:57:01.302 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:57:01.302 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:01.303 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:57:01.303 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:57:01.304 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:57:01.304 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:57:01.304 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:57:01.305 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 24, 2022-07-21T01:57:01.305 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:57:01.306 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:57:01.306 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:57:01.307 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:57:01.307 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:57:01.308 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.308 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:57:01.309 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:57:01.309 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:57:01.310 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:01.310 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:01.311 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:01.311 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:01.312 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:01.312 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:01.312 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:01.313 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:01.314 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:57:01.314 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:57:01.314 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:57:01.315 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:57:01.315 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:57:01.316 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:01.316 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:01.317 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:01.317 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:01.318 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:01.318 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:01.319 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:01.319 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.320 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:57:01.320 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:01.321 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:01.321 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:01.322 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:01.322 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:01.323 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:01.323 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:01.324 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:01.324 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:01.324 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:01.325 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:01.325 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:01.326 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:01.326 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:01.327 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:01.327 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:01.328 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:01.328 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:01.328 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:01.329 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:01.329 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:01.329 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:01.330 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:01.330 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:01.331 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:01.331 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:01.331 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.332 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:01.332 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:01.332 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:01.333 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:01.333 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:01.333 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:01.334 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:01.334 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:01.334 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:01.334 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:01.335 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:01.335 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:57:01.335 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:57:01.335 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.336 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:01.336 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:57:01.336 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.336 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.337 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.337 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:57:01.337 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:57:01.337 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:57:01.337 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:01.338 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:01.338 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:01.338 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:01.338 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:01.339 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:01.339 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:01.339 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:01.339 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:01.339 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:01.340 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:01.340 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:01.340 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:01.340 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:01.341 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:01.341 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:01.341 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:01.341 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:01.341 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:01.342 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:01.342 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.342 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:57:01.343 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:01.343 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:01.343 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:01.343 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:01.343 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:01.344 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:01.344 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:01.344 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:01.344 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:01.345 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:01.345 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:01.345 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:01.345 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:01.345 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:01.346 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:01.346 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:01.346 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:01.346 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:01.347 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:01.347 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:01.347 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:01.347 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:01.348 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:01.348 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:01.348 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:01.348 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:01.348 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.349 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:01.349 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:01.349 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:01.349 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:01.350 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:01.350 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:01.350 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:01.350 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:01.350 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:01.351 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:01.351 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:01.351 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:57:01.351 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:01.352 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:57:01.352 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.352 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.352 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.352 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 22, 2022-07-21T01:57:01.353 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:57:01.353 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:56:28.943049+0000", 2022-07-21T01:57:01.353 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:01.353 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:01.354 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:01.354 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:01.354 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:01.354 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:01.354 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:01.355 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:01.355 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:01.355 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:01.355 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:01.356 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:01.356 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:01.356 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:01.356 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:01.356 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:01.357 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:01.357 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:01.357 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:01.357 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:01.358 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.358 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "123", 2022-07-21T01:57:01.358 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:01.358 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:01.359 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:01.359 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:01.359 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:01.359 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:01.359 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:01.360 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:01.360 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:01.360 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:01.360 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:01.361 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:01.361 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:01.361 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:01.361 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:01.361 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:01.362 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:01.362 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:01.362 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:01.362 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:01.363 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:01.363 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:01.363 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:01.363 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:01.363 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:01.364 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:01.364 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.364 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:01.364 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:01.365 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:01.365 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:01.365 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:01.365 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:01.365 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:01.366 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:01.366 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:01.366 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:01.366 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:01.367 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:57:01.367 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:01.367 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:57:01.367 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:57:01.368 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.368 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.368 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.368 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.368 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 24, 2022-07-21T01:57:01.369 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:57:01.369 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:56:59.801219+0000", 2022-07-21T01:57:01.369 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 32769, 2022-07-21T01:57:01.369 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool,creating", 2022-07-21T01:57:01.370 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:01.370 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:01.370 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:01.370 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:01.370 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:01.371 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:01.371 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:01.371 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:01.371 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:01.372 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:01.372 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:01.372 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:01.372 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:01.372 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:01.373 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:01.373 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:01.373 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:01.373 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:01.374 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.374 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "123", 2022-07-21T01:57:01.374 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:01.374 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:01.375 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:01.375 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:01.375 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:01.375 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:01.375 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:01.376 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:01.376 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:01.376 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:01.376 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:01.377 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:01.377 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:01.377 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:01.377 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:01.377 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:01.378 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:01.378 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:01.378 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:01.378 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:01.379 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:01.379 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:01.379 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:01.379 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:01.379 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:01.380 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:01.380 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.380 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:01.380 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:01.381 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:01.381 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:01.381 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:01.381 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:01.381 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:01.382 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:01.382 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:01.382 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:01.382 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:01.383 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:57:01.383 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:57:01.383 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:57:01.383 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:57:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:57:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:57:01.384 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.385 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.385 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.385 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:01.385 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:57:01.386 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.386 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:57:01.386 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:57:01.386 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:01.387 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:01.387 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:01.387 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:01.387 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:01.387 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:01.388 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:01.388 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:01.388 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:01.388 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.389 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:57:01.390 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:01.390 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.390 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:57:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:01.391 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.392 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.392 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.392 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:01.392 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.393 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.393 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.393 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:57:01.393 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:01.393 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.394 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.394 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.394 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:57:01.394 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:01.395 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:57:01.396 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:57:01.397 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.398 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.399 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:57:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.400 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.401 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:57:01.401 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:01.401 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:57:01.402 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:57:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:57:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:57:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:01.403 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.404 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:57:01.405 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:57:01.405 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:01.405 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:01.405 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:01.406 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:01.407 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:57:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:01.408 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:57:01.409 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.410 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.411 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:57:01.412 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:01.412 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.412 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.412 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.413 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:57:01.413 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:01.413 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.413 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.414 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.415 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.415 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:57:01.415 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:01.415 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.416 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.416 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.416 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:57:01.416 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:01.417 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.417 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.417 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.417 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:01.417 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.418 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.418 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.418 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:57:01.418 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:01.419 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.419 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.419 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.419 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:57:01.420 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:01.420 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.420 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.421 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.421 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:57:01.421 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:57:01.421 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:57:01.422 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:57:01.422 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:01.422 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:01.423 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:01.423 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.423 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.423 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.423 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:57:01.424 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:57:01.424 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:01.424 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:01.424 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:01.425 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:01.425 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:01.425 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:01.425 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:01.426 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 117, 2022-07-21T01:57:01.426 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:01.426 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:01.426 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:01.427 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.427 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.427 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.427 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:57:01.428 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:01.428 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.428 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.428 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.429 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:57:01.429 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:01.429 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.429 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.429 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.430 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:01.430 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.430 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.430 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.431 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:57:01.431 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:01.431 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.432 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.432 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.432 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:57:01.432 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:01.432 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.433 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.433 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.433 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:01.433 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.434 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.434 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.434 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:57:01.434 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:01.435 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.435 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.435 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.435 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:57:01.435 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:01.436 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.436 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.436 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.436 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:01.437 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.437 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.437 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.437 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:57:01.437 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:01.438 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.438 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.438 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.438 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:57:01.439 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:01.439 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.439 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.439 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.440 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:57:01.440 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:57:01.440 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:57:01.440 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:57:01.440 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:01.441 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:01.441 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:01.441 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.441 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.442 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.442 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:57:01.442 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:57:01.442 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:01.443 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:01.443 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:01.443 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:01.443 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:01.443 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:01.444 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:01.444 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:01.444 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:01.444 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:01.445 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:01.445 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.445 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.445 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.446 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:57:01.446 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:01.446 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.446 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.447 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.447 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:57:01.447 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:01.447 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.448 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.448 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.448 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:01.448 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.448 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.449 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.449 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:57:01.449 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:01.449 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.450 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.450 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.450 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:57:01.450 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:01.451 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.451 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.451 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.451 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:01.451 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.452 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.452 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.452 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:57:01.453 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:01.453 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.453 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.453 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.454 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:57:01.454 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:01.454 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.454 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.455 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.455 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:01.455 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.456 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.456 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.456 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:57:01.456 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:01.457 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.457 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.457 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.458 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:57:01.458 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:01.458 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.458 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.459 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.459 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:57:01.459 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:57:01.459 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:57:01.460 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:57:01.460 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:01.460 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:01.460 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:01.461 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.461 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.461 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.461 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:57:01.461 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:57:01.462 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:01.462 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:01.462 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:01.462 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:01.463 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:01.463 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:01.463 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:01.463 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 117, 2022-07-21T01:57:01.464 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:01.464 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:01.464 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:01.464 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.465 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.465 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.465 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:57:01.465 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:01.465 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.466 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.466 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.466 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:57:01.466 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:01.467 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.467 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.467 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.467 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:01.468 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.468 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.468 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.468 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:57:01.469 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:01.469 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.469 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.469 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.470 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:57:01.470 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:01.470 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.470 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.471 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.471 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:01.471 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.471 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.472 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.472 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:57:01.472 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:01.472 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.473 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.473 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.473 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:57:01.473 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:01.473 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.474 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.474 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.474 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:01.474 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.475 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.475 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.475 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:57:01.476 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:01.476 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.476 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.476 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.477 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:57:01.477 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:01.477 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.477 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.478 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.478 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:57:01.478 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:57:01.478 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:57:01.478 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:57:01.479 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:01.479 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:01.479 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:01.479 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.480 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.480 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.480 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:57:01.480 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:57:01.480 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:01.481 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:01.481 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:01.481 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:01.481 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:01.482 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:01.482 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:01.482 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:01.482 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:01.483 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:01.483 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:01.483 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.483 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.483 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.484 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:57:01.484 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:01.484 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.485 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.485 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.485 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:57:01.485 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:01.486 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.486 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.486 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.486 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:01.487 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.487 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.487 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.487 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:57:01.488 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:01.488 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.488 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.488 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.489 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:57:01.489 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:01.489 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.489 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.489 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.490 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:01.490 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.490 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.490 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.491 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:57:01.491 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:01.491 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.491 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.492 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.492 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:57:01.492 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:01.492 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.492 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.493 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.493 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:01.493 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.493 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.494 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.494 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:57:01.494 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:01.494 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.494 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.495 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.495 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:57:01.495 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:01.495 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.496 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.496 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.496 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:57:01.496 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:57:01.496 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:57:01.497 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:57:01.497 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:01.497 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:01.497 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:01.498 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.498 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.498 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.498 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:57:01.499 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:57:01.499 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:01.499 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:01.499 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:01.499 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:01.500 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:01.500 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:01.500 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:01.500 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:01.501 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:01.501 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:01.501 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:01.501 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.501 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.502 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.502 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:57:01.502 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:01.502 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.503 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.503 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.503 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:57:01.503 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:01.503 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.504 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.504 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.504 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:01.504 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.505 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.505 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.505 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:57:01.505 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:01.505 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.506 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.506 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.506 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:57:01.506 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:01.507 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.507 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.507 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.507 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:01.508 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.508 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.508 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.508 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:57:01.508 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:01.509 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.509 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.509 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.509 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:57:01.510 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:01.510 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.510 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.510 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.510 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:01.511 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.511 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.511 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.511 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:57:01.512 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:01.512 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.512 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.512 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.512 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:57:01.513 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:01.513 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.513 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.513 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.514 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:57:01.514 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:57:01.514 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:57:01.514 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:57:01.514 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:01.515 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:01.515 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:01.515 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.515 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.516 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.516 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:57:01.516 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:57:01.516 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:01.516 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:01.517 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:01.517 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:01.517 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:01.517 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:01.518 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:01.518 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 110, 2022-07-21T01:57:01.518 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:01.518 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:01.519 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:01.519 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.519 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.519 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.519 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:57:01.520 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:01.520 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.520 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.520 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.521 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:57:01.521 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:01.521 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.521 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.521 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.522 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:01.522 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.522 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.522 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.523 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:57:01.523 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:01.523 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.523 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.523 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.524 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:57:01.524 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:01.524 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.524 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.525 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.525 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:01.525 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.525 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.526 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.526 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:57:01.526 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:01.526 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.527 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.527 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.527 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:57:01.527 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:01.528 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.528 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.528 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.528 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:01.528 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:01.529 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.529 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:01.529 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:57:01.529 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:01.530 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.530 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.530 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:01.530 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:57:01.531 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:01.531 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.531 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.531 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.531 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:57:01.532 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:57:01.532 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:57:01.532 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:57:01.532 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:01.532 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:01.533 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:01.533 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:01.533 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.533 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:01.534 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:57:01.534 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.534 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:57:01.534 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:01.535 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:01.535 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:01.535 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:01.535 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:01.535 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:57:01.536 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:01.536 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.536 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.536 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:57:01.537 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:01.537 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:01.537 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:01.537 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:01.537 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:01.538 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:57:01.538 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:01.538 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.538 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.539 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:57:01.539 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:01.539 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:01.539 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:01.539 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:01.540 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:01.540 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:57:01.540 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:01.540 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.541 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.541 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:57:01.541 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:01.541 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:01.541 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:01.542 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:01.542 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:01.542 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:57:01.542 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:01.543 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.543 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.543 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:57:01.543 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:01.543 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:01.544 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:01.544 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:01.544 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:01.544 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:57:01.545 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:01.545 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.545 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.545 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:57:01.545 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:01.546 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:01.546 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:01.546 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:01.546 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:01.547 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:57:01.547 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:01.547 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.547 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.547 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:57:01.548 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:01.548 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:01.548 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:01.548 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:01.549 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:01.549 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:57:01.549 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:01.549 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.550 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:01.550 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:57:01.550 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:01.550 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:01.551 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:01.551 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:01.551 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:01.551 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:57:01.551 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:01.552 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.552 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:01.552 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:57:01.552 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:57:01.552 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:57:01.553 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:57:01.553 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:57:01.553 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6833/456689269": "2022-07-22T01:56:46.358661+0000", 2022-07-21T01:57:01.553 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6832/456689269": "2022-07-22T01:56:46.358661+0000" 2022-07-21T01:57:01.554 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.554 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:57:01.554 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:57:01.554 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:57:01.555 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:57:01.555 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:57:01.555 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:57:01.555 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:57:01.555 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:57:01.556 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:01.556 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:01.556 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:57:01.556 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:57:01.557 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:57:01.557 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:57:01.557 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:57:01.557 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:57:01.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 -- 172.21.15.140:0/2128037226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fac053b00 msgr2=0x7f6fac055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fac053b00 0x7f6fac055fc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f6fb800d000 tx=0x7f6fb8000f90).stop 2022-07-21T01:57:01.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 -- 172.21.15.140:0/2128037226 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fc80b0080 msgr2=0x7f6fc80a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fc80b0080 0x7f6fc80a5c40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f6fb0000bc0 tx=0x7f6fb0007aa0).stop 2022-07-21T01:57:01.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 -- 172.21.15.140:0/2128037226 shutdown_connections 2022-07-21T01:57:01.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fac053b00 0x7f6fac055fc0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fc80a4900 0x7f6fc80abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fc80b0080 0x7f6fc80a5c40 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 --2- 172.21.15.140:0/2128037226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fc80aedc0 0x7f6fc80a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 -- 172.21.15.140:0/2128037226 >> 172.21.15.140:0/2128037226 conn(0x7f6fc80ac9d0 msgr2=0x7f6fc80b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:01.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 -- 172.21.15.140:0/2128037226 shutdown_connections 2022-07-21T01:57:01.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.296+0000 7f6fcda00700 1 -- 172.21.15.140:0/2128037226 wait complete. 2022-07-21T01:57:01.564 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:01.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21359e700 1 Processor -- start 2022-07-21T01:57:01.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21359e700 1 -- start start 2022-07-21T01:57:01.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21359e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2140b5650 0x7fd2140b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2140b1200 0x7fd21405c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21359e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd21405bd90 0x7fd21405c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21359e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd214142970 con 0x7fd2140b5650 2022-07-21T01:57:01.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21359e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2140aea30 con 0x7fd21405bd90 2022-07-21T01:57:01.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2140aec60 con 0x7fd2140b1200 2022-07-21T01:57:01.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21259c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2140b5650 0x7fd2140b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21259c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd2140b5650 0x7fd2140b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39962/0 (socket says 172.21.15.140:39962) 2022-07-21T01:57:01.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21259c700 1 -- 172.21.15.140:0/3447835183 learned_addr learned my addr 172.21.15.140:0/3447835183 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:01.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd212d9d700 1 --2- 172.21.15.140:0/3447835183 >> v2:172.21.15.152:3300/0 conn(0x7fd21405bd90 0x7fd21405c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd211d9b700 1 --2- 172.21.15.140:0/3447835183 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2140b1200 0x7fd21405c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21259c700 1 -- 172.21.15.140:0/3447835183 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2140b1200 msgr2=0x7fd21405c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:01.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21259c700 1 --2- 172.21.15.140:0/3447835183 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2140b1200 0x7fd21405c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21259c700 1 -- 172.21.15.140:0/3447835183 >> v2:172.21.15.152:3300/0 conn(0x7fd21405bd90 msgr2=0x7fd21405c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:01.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.672+0000 7fd21259c700 1 --2- 172.21.15.140:0/3447835183 >> v2:172.21.15.152:3300/0 conn(0x7fd21405bd90 0x7fd21405c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21259c700 1 -- 172.21.15.140:0/3447835183 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2140af180 con 0x7fd2140b5650 2022-07-21T01:57:01.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd212d9d700 1 --2- 172.21.15.140:0/3447835183 >> v2:172.21.15.152:3300/0 conn(0x7fd21405bd90 0x7fd21405c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:01.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21259c700 1 --2- 172.21.15.140:0/3447835183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2140b5650 0x7fd2140b5a30 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7fd2140b16d0 tx=0x7fd1fc006eb0).ready entity=mon.0 client_cookie=b9822d4b3cd1c78 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:01.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21159a700 1 -- 172.21.15.140:0/3447835183 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1fc00e610 con 0x7fd2140b5650 2022-07-21T01:57:01.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21159a700 1 -- 172.21.15.140:0/3447835183 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc000e90 con 0x7fd2140b5650 2022-07-21T01:57:01.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21159a700 1 -- 172.21.15.140:0/3447835183 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1fc00e9a0 con 0x7fd2140b5650 2022-07-21T01:57:01.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 -- 172.21.15.140:0/3447835183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2140b5650 msgr2=0x7fd2140b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 --2- 172.21.15.140:0/3447835183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2140b5650 0x7fd2140b5a30 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7fd2140b16d0 tx=0x7fd1fc006eb0).stop 2022-07-21T01:57:01.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 -- 172.21.15.140:0/3447835183 shutdown_connections 2022-07-21T01:57:01.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 --2- 172.21.15.140:0/3447835183 >> v2:172.21.15.152:3300/0 conn(0x7fd21405bd90 0x7fd21405c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 --2- 172.21.15.140:0/3447835183 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2140b1200 0x7fd21405c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 --2- 172.21.15.140:0/3447835183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd2140b5650 0x7fd2140b5a30 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 -- 172.21.15.140:0/3447835183 >> 172.21.15.140:0/3447835183 conn(0x7fd2140634e0 msgr2=0x7fd2140b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:01.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 -- 172.21.15.140:0/3447835183 shutdown_connections 2022-07-21T01:57:01.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 -- 172.21.15.140:0/3447835183 wait complete. 2022-07-21T01:57:01.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 Processor -- start 2022-07-21T01:57:01.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 -- start start 2022-07-21T01:57:01.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.676+0000 7fd21359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd21405bd90 0x7fd214134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2140b1200 0x7fd21414eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21359e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2140b5650 0x7fd21414f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21359e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd214153000 con 0x7fd21405bd90 2022-07-21T01:57:01.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21359e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd214152690 con 0x7fd2140b5650 2022-07-21T01:57:01.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd214134920 con 0x7fd2140b1200 2022-07-21T01:57:01.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd21405bd90 0x7fd214134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd21405bd90 0x7fd214134e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39966/0 (socket says 172.21.15.140:39966) 2022-07-21T01:57:01.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21259c700 1 -- 172.21.15.140:0/2226616483 learned_addr learned my addr 172.21.15.140:0/2226616483 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:01.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd212d9d700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2140b5650 0x7fd21414f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd211d9b700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2140b1200 0x7fd21414eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21259c700 1 -- 172.21.15.140:0/2226616483 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2140b1200 msgr2=0x7fd21414eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21259c700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2140b1200 0x7fd21414eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21259c700 1 -- 172.21.15.140:0/2226616483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2140b5650 msgr2=0x7fd21414f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21259c700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2140b5650 0x7fd21414f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21259c700 1 -- 172.21.15.140:0/2226616483 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20c00b180 con 0x7fd21405bd90 2022-07-21T01:57:01.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd212d9d700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2140b5650 0x7fd21414f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:01.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21259c700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd21405bd90 0x7fd214134e00 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7fd214151db0 tx=0x7fd1fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:01.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd20b7fe700 1 -- 172.21.15.140:0/2226616483 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1fc021070 con 0x7fd21405bd90 2022-07-21T01:57:01.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd20b7fe700 1 -- 172.21.15.140:0/2226616483 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc002f50 con 0x7fd21405bd90 2022-07-21T01:57:01.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd20b7fe700 1 -- 172.21.15.140:0/2226616483 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1fc011bf0 con 0x7fd21405bd90 2022-07-21T01:57:01.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21359e700 1 -- 172.21.15.140:0/2226616483 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1fc00b1e0 con 0x7fd21405bd90 2022-07-21T01:57:01.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21359e700 1 -- 172.21.15.140:0/2226616483 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd214064120 con 0x7fd21405bd90 2022-07-21T01:57:01.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd21359e700 1 -- 172.21.15.140:0/2226616483 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2140574e0 con 0x7fd21405bd90 2022-07-21T01:57:01.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.680+0000 7fd20b7fe700 1 -- 172.21.15.140:0/2226616483 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd1fc00f070 con 0x7fd21405bd90 2022-07-21T01:57:01.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.684+0000 7fd20b7fe700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd1f80539e0 0x7fd1f8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.684+0000 7fd20b7fe700 1 -- 172.21.15.140:0/2226616483 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd1fc06d450 con 0x7fd21405bd90 2022-07-21T01:57:01.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.684+0000 7fd211d9b700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd1f80539e0 0x7fd1f8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.684+0000 7fd211d9b700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd1f80539e0 0x7fd1f8055ea0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fd20400b000 tx=0x7fd2040091a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:01.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.692+0000 7fd20b7fe700 1 -- 172.21.15.140:0/2226616483 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd1fc044210 con 0x7fd21405bd90 2022-07-21T01:57:01.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.856+0000 7fd21359e700 1 -- 172.21.15.140:0/2226616483 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd214067ea0 con 0x7fd21405bd90 2022-07-21T01:57:01.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.856+0000 7fd20b7fe700 1 -- 172.21.15.140:0/2226616483 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+1426 (secure 0 0 0) 0x7fd1fc047830 con 0x7fd21405bd90 2022-07-21T01:57:01.859 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:01.859 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":58,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:00.807686+0000","modified":"2022-07-21T01:57:00.807807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2022-07-21T01:57:01.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 -- 172.21.15.140:0/2226616483 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd1f80539e0 msgr2=0x7fd1f8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd1f80539e0 0x7fd1f8055ea0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7fd20400b000 tx=0x7fd2040091a0).stop 2022-07-21T01:57:01.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 -- 172.21.15.140:0/2226616483 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd21405bd90 msgr2=0x7fd214134e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd21405bd90 0x7fd214134e00 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7fd214151db0 tx=0x7fd1fc015000).stop 2022-07-21T01:57:01.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 -- 172.21.15.140:0/2226616483 shutdown_connections 2022-07-21T01:57:01.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd1f80539e0 0x7fd1f8055ea0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2140b5650 0x7fd21414f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd2140b1200 0x7fd21414eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 --2- 172.21.15.140:0/2226616483 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd21405bd90 0x7fd214134e00 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 -- 172.21.15.140:0/2226616483 >> 172.21.15.140:0/2226616483 conn(0x7fd2140634e0 msgr2=0x7fd2140b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:01.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 -- 172.21.15.140:0/2226616483 shutdown_connections 2022-07-21T01:57:01.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.860+0000 7fd21359e700 1 -- 172.21.15.140:0/2226616483 wait complete. 2022-07-21T01:57:01.865 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 58 2022-07-21T01:57:01.876 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:57:01.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515b59e700 1 Processor -- start 2022-07-21T01:57:01.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515b59e700 1 -- start start 2022-07-21T01:57:01.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515b59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f515c0acae0 0x7f515c14d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515b59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515c0b46e0 con 0x7f515c0b5860 2022-07-21T01:57:01.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515b59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515c0b4b10 con 0x7f515c0b4f80 2022-07-21T01:57:01.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515b59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515c0ad990 con 0x7f515c0acae0 2022-07-21T01:57:01.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515a59c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f515c0b5860 0x7f515c146060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54120/0 (socket says 172.21.15.140:54120) 2022-07-21T01:57:01.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515a59c700 1 -- 172.21.15.140:0/405269997 learned_addr learned my addr 172.21.15.140:0/405269997 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:01.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515ad9d700 1 --2- 172.21.15.140:0/405269997 >> v2:172.21.15.140:3300/0 conn(0x7f515c0acae0 0x7f515c14d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515959a700 1 -- 172.21.15.140:0/405269997 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572639802 0 0) 0x7f515c0b46e0 con 0x7f515c0b5860 2022-07-21T01:57:01.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515959a700 1 -- 172.21.15.140:0/405269997 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5140003260 con 0x7f515c0b5860 2022-07-21T01:57:01.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515959a700 1 -- 172.21.15.140:0/405269997 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1709214377 0 0) 0x7f5140003260 con 0x7f515c0b5860 2022-07-21T01:57:01.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515ad9d700 1 -- 172.21.15.140:0/405269997 >> v1:172.21.15.152:6789/0 conn(0x7f515c0b4f80 legacy=0x7f515c146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515ad9d700 1 -- 172.21.15.140:0/405269997 >> v1:172.21.15.140:6789/0 conn(0x7f515c0b5860 legacy=0x7f515c146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515ad9d700 1 -- 172.21.15.140:0/405269997 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c0ad3d0 con 0x7f515c0acae0 2022-07-21T01:57:01.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515ad9d700 1 --2- 172.21.15.140:0/405269997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f515c0acae0 0x7f515c14d680 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f5144001800 tx=0x7f5154006df0).ready entity=mon.0 client_cookie=8e75164c9843f548 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:01.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515959a700 1 -- 172.21.15.140:0/405269997 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5144001660 con 0x7f515c0acae0 2022-07-21T01:57:01.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515959a700 1 -- 172.21.15.140:0/405269997 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5154000e90 con 0x7f515c0acae0 2022-07-21T01:57:01.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.972+0000 7f515959a700 1 -- 172.21.15.140:0/405269997 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f515400d4f0 con 0x7f515c0acae0 2022-07-21T01:57:01.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 -- 172.21.15.140:0/405269997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f515c0acae0 msgr2=0x7f515c14d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 --2- 172.21.15.140:0/405269997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f515c0acae0 0x7f515c14d680 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f5144001800 tx=0x7f5154006df0).stop 2022-07-21T01:57:01.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 -- 172.21.15.140:0/405269997 shutdown_connections 2022-07-21T01:57:01.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 --2- 172.21.15.140:0/405269997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f515c0acae0 0x7f515c14d680 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 -- 172.21.15.140:0/405269997 >> 172.21.15.140:0/405269997 conn(0x7f515c0b5e10 msgr2=0x7f515c0b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:01.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 -- 172.21.15.140:0/405269997 shutdown_connections 2022-07-21T01:57:01.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 -- 172.21.15.140:0/405269997 wait complete. 2022-07-21T01:57:01.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 Processor -- start 2022-07-21T01:57:01.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 -- start start 2022-07-21T01:57:01.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f515c0acae0 0x7f515c143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f515c0b4f80 0x7f515c13e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f515c0b5860 0x7f515c13ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c13f320 con 0x7f515c0acae0 2022-07-21T01:57:01.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515c0b1c70 con 0x7f515c0b4f80 2022-07-21T01:57:01.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515c0b00f0 con 0x7f515c0b5860 2022-07-21T01:57:01.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f515c0acae0 0x7f515c143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f515c0acae0 0x7f515c143300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:39982/0 (socket says 172.21.15.140:39982) 2022-07-21T01:57:01.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515a59c700 1 -- 172.21.15.140:0/1819538195 learned_addr learned my addr 172.21.15.140:0/1819538195 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:01.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515ad9d700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f515c0b5860 0x7f515c13ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f5159d9b700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f515c0b4f80 0x7f515c13e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515a59c700 1 -- 172.21.15.140:0/1819538195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f515c0b5860 msgr2=0x7f515c13ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515a59c700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f515c0b5860 0x7f515c13ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515a59c700 1 -- 172.21.15.140:0/1819538195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f515c0b4f80 msgr2=0x7f515c13e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:01.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515a59c700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f515c0b4f80 0x7f515c13e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:01.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515a59c700 1 -- 172.21.15.140:0/1819538195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515400b180 con 0x7f515c0acae0 2022-07-21T01:57:01.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.976+0000 7f515a59c700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f515c0acae0 0x7f515c143300 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f515c0b3450 tx=0x7f5144001840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:01.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.980+0000 7f51537fe700 1 -- 172.21.15.140:0/1819538195 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5144007170 con 0x7f515c0acae0 2022-07-21T01:57:01.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.980+0000 7f51537fe700 1 -- 172.21.15.140:0/1819538195 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51440072d0 con 0x7f515c0acae0 2022-07-21T01:57:01.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.980+0000 7f515b59e700 1 -- 172.21.15.140:0/1819538195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515c0b1ea0 con 0x7f515c0acae0 2022-07-21T01:57:01.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.980+0000 7f51537fe700 1 -- 172.21.15.140:0/1819538195 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5144004710 con 0x7f515c0acae0 2022-07-21T01:57:01.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.980+0000 7f515b59e700 1 -- 172.21.15.140:0/1819538195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515c05d2d0 con 0x7f515c0acae0 2022-07-21T01:57:01.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.980+0000 7f515b59e700 1 -- 172.21.15.140:0/1819538195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515c112630 con 0x7f515c0acae0 2022-07-21T01:57:01.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.980+0000 7f51537fe700 1 -- 172.21.15.140:0/1819538195 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5144004870 con 0x7f515c0acae0 2022-07-21T01:57:01.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.980+0000 7f51537fe700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5134053b00 0x7f5134055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:01.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.984+0000 7f51537fe700 1 -- 172.21.15.140:0/1819538195 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f514406b950 con 0x7f515c0acae0 2022-07-21T01:57:01.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.984+0000 7f5159d9b700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5134053b00 0x7f5134055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:01.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.984+0000 7f5159d9b700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5134053b00 0x7f5134055fc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f514c00d000 tx=0x7f514c007f10).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:01.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:01.988+0000 7f51537fe700 1 -- 172.21.15.140:0/1819538195 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f515c112630 con 0x7f515c0acae0 2022-07-21T01:57:02.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.148+0000 7f515b59e700 1 -- 172.21.15.140:0/1819538195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f515c112630 con 0x7f515c0acae0 2022-07-21T01:57:02.354 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:57:02.357 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:57:02.360 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:57:02.363 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:57:02.365 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:57:02.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.832+0000 7f51537fe700 1 -- 172.21.15.140:0/1819538195 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7f515c112630 con 0x7f515c0acae0 2022-07-21T01:57:02.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 -- 172.21.15.140:0/1819538195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5134053b00 msgr2=0x7f5134055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:02.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5134053b00 0x7f5134055fc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f514c00d000 tx=0x7f514c007f10).stop 2022-07-21T01:57:02.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 -- 172.21.15.140:0/1819538195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f515c0acae0 msgr2=0x7f515c143300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:02.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f515c0acae0 0x7f515c143300 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f515c0b3450 tx=0x7f5144001840).stop 2022-07-21T01:57:02.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 -- 172.21.15.140:0/1819538195 shutdown_connections 2022-07-21T01:57:02.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5134053b00 0x7f5134055fc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:02.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f515c0b5860 0x7f515c13ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:02.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f515c0b4f80 0x7f515c13e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:02.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 --2- 172.21.15.140:0/1819538195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f515c0acae0 0x7f515c143300 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:02.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 -- 172.21.15.140:0/1819538195 >> 172.21.15.140:0/1819538195 conn(0x7f515c0b5e10 msgr2=0x7f515c140960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:02.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 -- 172.21.15.140:0/1819538195 shutdown_connections 2022-07-21T01:57:02.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.836+0000 7f515b59e700 1 -- 172.21.15.140:0/1819538195 wait complete. 2022-07-21T01:57:02.842 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:57:02.856 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:02.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.960+0000 7fd66a520700 1 Processor -- start 2022-07-21T01:57:02.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.960+0000 7fd66a520700 1 -- start start 2022-07-21T01:57:02.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd66a520700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aecf0 0x7fd664140240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:02.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd66a520700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6640b5d70 con 0x7fd6640aff80 2022-07-21T01:57:02.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd66a520700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6640651c0 con 0x7fd6640b5990 2022-07-21T01:57:02.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd66a520700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640b0690 con 0x7fd6640aecf0 2022-07-21T01:57:02.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd669d1f700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd6640aff80 0x7fd66405bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54138/0 (socket says 172.21.15.140:54138) 2022-07-21T01:57:02.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd669d1f700 1 -- 172.21.15.140:0/4085761104 learned_addr learned my addr 172.21.15.140:0/4085761104 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:02.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd668d1d700 1 --2- 172.21.15.140:0/4085761104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aecf0 0x7fd664140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:02.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd653fff700 1 -- 172.21.15.140:0/4085761104 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749274335 0 0) 0x7fd6640b5d70 con 0x7fd6640aff80 2022-07-21T01:57:02.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd653fff700 1 -- 172.21.15.140:0/4085761104 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd64c003260 con 0x7fd6640aff80 2022-07-21T01:57:02.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd668d1d700 1 -- 172.21.15.140:0/4085761104 >> v1:172.21.15.152:6789/0 conn(0x7fd6640b5990 legacy=0x7fd66413fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:02.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd668d1d700 1 -- 172.21.15.140:0/4085761104 >> v1:172.21.15.140:6789/0 conn(0x7fd6640aff80 legacy=0x7fd66405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:02.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd668d1d700 1 -- 172.21.15.140:0/4085761104 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6640b0930 con 0x7fd6640aecf0 2022-07-21T01:57:02.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd668d1d700 1 --2- 172.21.15.140:0/4085761104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aecf0 0x7fd664140240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fd654002f70 tx=0x7fd654006d90).ready entity=mon.2 client_cookie=97014000ca1f69ba server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:02.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd653fff700 1 -- 172.21.15.140:0/4085761104 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd65400d4a0 con 0x7fd6640aecf0 2022-07-21T01:57:02.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd653fff700 1 -- 172.21.15.140:0/4085761104 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd654000e90 con 0x7fd6640aecf0 2022-07-21T01:57:02.970 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd653fff700 1 -- 172.21.15.140:0/4085761104 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd65400d7f0 con 0x7fd6640aecf0 2022-07-21T01:57:02.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd66a520700 1 -- 172.21.15.140:0/4085761104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aecf0 msgr2=0x7fd664140240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:02.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd66a520700 1 --2- 172.21.15.140:0/4085761104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aecf0 0x7fd664140240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fd654002f70 tx=0x7fd654006d90).stop 2022-07-21T01:57:02.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd66a520700 1 -- 172.21.15.140:0/4085761104 shutdown_connections 2022-07-21T01:57:02.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd66a520700 1 --2- 172.21.15.140:0/4085761104 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aecf0 0x7fd664140240 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:02.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd66a520700 1 -- 172.21.15.140:0/4085761104 >> 172.21.15.140:0/4085761104 conn(0x7fd6640634e0 msgr2=0x7fd6640b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:02.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd66a520700 1 -- 172.21.15.140:0/4085761104 shutdown_connections 2022-07-21T01:57:02.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.964+0000 7fd66a520700 1 -- 172.21.15.140:0/4085761104 wait complete. 2022-07-21T01:57:02.974 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd66a520700 1 Processor -- start 2022-07-21T01:57:02.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd66a520700 1 -- start start 2022-07-21T01:57:02.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd66a520700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6640aecf0 0x7fd664144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:02.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd66a520700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aff80 0x7fd664143000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:02.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd66a520700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6640b5990 0x7fd664114920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:02.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd66a520700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6641141e0 con 0x7fd6640b5990 2022-07-21T01:57:02.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd66a520700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd664114480 con 0x7fd6640aecf0 2022-07-21T01:57:02.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd66a520700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd664144bb0 con 0x7fd6640aff80 2022-07-21T01:57:02.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd669d1f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6640b5990 0x7fd664114920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:02.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd668d1d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aff80 0x7fd664143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:02.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd668d1d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aff80 0x7fd664143000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36726/0 (socket says 172.21.15.140:36726) 2022-07-21T01:57:02.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd668d1d700 1 -- 172.21.15.140:0/1808260096 learned_addr learned my addr 172.21.15.140:0/1808260096 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:02.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd66951e700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6640aecf0 0x7fd664144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:02.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd669d1f700 1 -- 172.21.15.140:0/1808260096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aff80 msgr2=0x7fd664143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:02.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd669d1f700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aff80 0x7fd664143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:02.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd669d1f700 1 -- 172.21.15.140:0/1808260096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6640aecf0 msgr2=0x7fd664144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:02.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd669d1f700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6640aecf0 0x7fd664144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:02.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd669d1f700 1 -- 172.21.15.140:0/1808260096 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65400b120 con 0x7fd6640b5990 2022-07-21T01:57:02.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd668d1d700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aff80 0x7fd664143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:02.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd669d1f700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6640b5990 0x7fd664114920 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7fd65c001490 tx=0x7fd65c007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:02.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd6527fc700 1 -- 172.21.15.140:0/1808260096 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd65c005ba0 con 0x7fd6640b5990 2022-07-21T01:57:02.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd6527fc700 1 -- 172.21.15.140:0/1808260096 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c005d00 con 0x7fd6640b5990 2022-07-21T01:57:02.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd66a520700 1 -- 172.21.15.140:0/1808260096 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6641436a0 con 0x7fd6640b5990 2022-07-21T01:57:02.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd6527fc700 1 -- 172.21.15.140:0/1808260096 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd65c0072c0 con 0x7fd6640b5990 2022-07-21T01:57:02.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.968+0000 7fd66a520700 1 -- 172.21.15.140:0/1808260096 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6640ae550 con 0x7fd6640b5990 2022-07-21T01:57:02.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.972+0000 7fd66a520700 1 -- 172.21.15.140:0/1808260096 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6641434f0 con 0x7fd6640b5990 2022-07-21T01:57:02.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.972+0000 7fd6527fc700 1 -- 172.21.15.140:0/1808260096 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd65c01a060 con 0x7fd6640b5990 2022-07-21T01:57:02.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.972+0000 7fd6527fc700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd644053aa0 0x7fd644055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:02.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.972+0000 7fd66951e700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd644053aa0 0x7fd644055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:02.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.972+0000 7fd6527fc700 1 -- 172.21.15.140:0/1808260096 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd65c06b8a0 con 0x7fd6640b5990 2022-07-21T01:57:02.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.972+0000 7fd66951e700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd644053aa0 0x7fd644055f60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fd66000d000 tx=0x7fd660009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:02.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:02.980+0000 7fd6527fc700 1 -- 172.21.15.140:0/1808260096 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd65c042770 con 0x7fd6640b5990 2022-07-21T01:57:03.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.140+0000 7fd66a520700 1 -- 172.21.15.140:0/1808260096 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd664114640 con 0x7fd6640b5990 2022-07-21T01:57:03.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd6527fc700 1 -- 172.21.15.140:0/1808260096 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+1426 (secure 0 0 0) 0x7fd65c018020 con 0x7fd6640b5990 2022-07-21T01:57:03.145 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:03.145 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":59,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":59,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:00.807686+0000","modified":"2022-07-21T01:57:02.824298+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2022-07-21T01:57:03.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 -- 172.21.15.140:0/1808260096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd644053aa0 msgr2=0x7fd644055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd644053aa0 0x7fd644055f60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fd66000d000 tx=0x7fd660009040).stop 2022-07-21T01:57:03.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 -- 172.21.15.140:0/1808260096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6640b5990 msgr2=0x7fd664114920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6640b5990 0x7fd664114920 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7fd65c001490 tx=0x7fd65c007d40).stop 2022-07-21T01:57:03.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 -- 172.21.15.140:0/1808260096 shutdown_connections 2022-07-21T01:57:03.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd644053aa0 0x7fd644055f60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6640b5990 0x7fd664114920 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6640aff80 0x7fd664143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 --2- 172.21.15.140:0/1808260096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6640aecf0 0x7fd664144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 -- 172.21.15.140:0/1808260096 >> 172.21.15.140:0/1808260096 conn(0x7fd6640634e0 msgr2=0x7fd66405bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:03.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 -- 172.21.15.140:0/1808260096 shutdown_connections 2022-07-21T01:57:03.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.144+0000 7fd66a520700 1 -- 172.21.15.140:0/1808260096 wait complete. 2022-07-21T01:57:03.151 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 59 2022-07-21T01:57:03.158 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:57:03.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67ffff700 1 Processor -- start 2022-07-21T01:57:03.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67ffff700 1 -- start start 2022-07-21T01:57:03.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd68014f9d0 0x7fd680151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:03.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67ffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6800b0b80 con 0x7fd6800af8c0 2022-07-21T01:57:03.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67ffff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd68005d4d0 con 0x7fd6800b5970 2022-07-21T01:57:03.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800b5d50 con 0x7fd68014f9d0 2022-07-21T01:57:03.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67e7fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd6800af8c0 0x7fd6800ae810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54148/0 (socket says 172.21.15.140:54148) 2022-07-21T01:57:03.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67e7fc700 1 -- 172.21.15.140:0/2312826168 learned_addr learned my addr 172.21.15.140:0/2312826168 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:03.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67f7fe700 1 --2- 172.21.15.140:0/2312826168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd68014f9d0 0x7fd680151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:03.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67dffb700 1 -- 172.21.15.140:0/2312826168 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103399730 0 0) 0x7fd6800b0b80 con 0x7fd6800af8c0 2022-07-21T01:57:03.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67dffb700 1 -- 172.21.15.140:0/2312826168 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd664003260 con 0x7fd6800af8c0 2022-07-21T01:57:03.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67dffb700 1 -- 172.21.15.140:0/2312826168 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1642533589 0 0) 0x7fd664003260 con 0x7fd6800af8c0 2022-07-21T01:57:03.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67f7fe700 1 -- 172.21.15.140:0/2312826168 >> v1:172.21.15.152:6789/0 conn(0x7fd6800b5970 legacy=0x7fd6800ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67f7fe700 1 -- 172.21.15.140:0/2312826168 >> v1:172.21.15.140:6789/0 conn(0x7fd6800af8c0 legacy=0x7fd6800ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.264+0000 7fd67f7fe700 1 -- 172.21.15.140:0/2312826168 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6800b0db0 con 0x7fd68014f9d0 2022-07-21T01:57:03.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67f7fe700 1 --2- 172.21.15.140:0/2312826168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd68014f9d0 0x7fd680151dc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fd670001800 tx=0x7fd6780091d0).ready entity=mon.2 client_cookie=e5de6eef45141c02 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:03.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67dffb700 1 -- 172.21.15.140:0/2312826168 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd670001660 con 0x7fd68014f9d0 2022-07-21T01:57:03.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67dffb700 1 -- 172.21.15.140:0/2312826168 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67800b580 con 0x7fd68014f9d0 2022-07-21T01:57:03.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67dffb700 1 -- 172.21.15.140:0/2312826168 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd67800b6f0 con 0x7fd68014f9d0 2022-07-21T01:57:03.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 -- 172.21.15.140:0/2312826168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd68014f9d0 msgr2=0x7fd680151dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 --2- 172.21.15.140:0/2312826168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd68014f9d0 0x7fd680151dc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fd670001800 tx=0x7fd6780091d0).stop 2022-07-21T01:57:03.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 -- 172.21.15.140:0/2312826168 shutdown_connections 2022-07-21T01:57:03.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 --2- 172.21.15.140:0/2312826168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd68014f9d0 0x7fd680151dc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 -- 172.21.15.140:0/2312826168 >> 172.21.15.140:0/2312826168 conn(0x7fd680063490 msgr2=0x7fd6800b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:03.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 -- 172.21.15.140:0/2312826168 shutdown_connections 2022-07-21T01:57:03.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 -- 172.21.15.140:0/2312826168 wait complete. 2022-07-21T01:57:03.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 Processor -- start 2022-07-21T01:57:03.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 -- start start 2022-07-21T01:57:03.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6800af8c0 0x7fd6801426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:03.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6800b5970 0x7fd680063cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:03.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd68014f9d0 0x7fd68010ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:03.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68013f490 con 0x7fd68014f9d0 2022-07-21T01:57:03.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68010e110 con 0x7fd6800af8c0 2022-07-21T01:57:03.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68010e3b0 con 0x7fd6800b5970 2022-07-21T01:57:03.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd68014f9d0 0x7fd68010ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:03.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd68014f9d0 0x7fd68010ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40012/0 (socket says 172.21.15.140:40012) 2022-07-21T01:57:03.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67f7fe700 1 -- 172.21.15.140:0/685288060 learned_addr learned my addr 172.21.15.140:0/685288060 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:03.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67e7fc700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6800b5970 0x7fd680063cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:03.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.268+0000 7fd67effd700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6800af8c0 0x7fd6801426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:03.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd67f7fe700 1 -- 172.21.15.140:0/685288060 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6800b5970 msgr2=0x7fd680063cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd67f7fe700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6800b5970 0x7fd680063cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd67f7fe700 1 -- 172.21.15.140:0/685288060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6800af8c0 msgr2=0x7fd6801426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd67f7fe700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6800af8c0 0x7fd6801426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd67f7fe700 1 -- 172.21.15.140:0/685288060 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd678009470 con 0x7fd68014f9d0 2022-07-21T01:57:03.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd67f7fe700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd68014f9d0 0x7fd68010ed90 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7fd670001800 tx=0x7fd678016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:03.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd66ffff700 1 -- 172.21.15.140:0/685288060 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6780032d0 con 0x7fd68014f9d0 2022-07-21T01:57:03.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd66ffff700 1 -- 172.21.15.140:0/685288060 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6780119e0 con 0x7fd68014f9d0 2022-07-21T01:57:03.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd67ffff700 1 -- 172.21.15.140:0/685288060 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68010e650 con 0x7fd68014f9d0 2022-07-21T01:57:03.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd66ffff700 1 -- 172.21.15.140:0/685288060 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd678011d40 con 0x7fd68014f9d0 2022-07-21T01:57:03.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd67ffff700 1 -- 172.21.15.140:0/685288060 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd680142c50 con 0x7fd68014f9d0 2022-07-21T01:57:03.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd67ffff700 1 -- 172.21.15.140:0/685288060 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6800acae0 con 0x7fd68014f9d0 2022-07-21T01:57:03.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd66ffff700 1 -- 172.21.15.140:0/685288060 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd678026030 con 0x7fd68014f9d0 2022-07-21T01:57:03.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd66ffff700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd65c053b00 0x7fd65c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:03.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd66ffff700 1 -- 172.21.15.140:0/685288060 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd67806c8f0 con 0x7fd68014f9d0 2022-07-21T01:57:03.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd67effd700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd65c053b00 0x7fd65c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:03.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.272+0000 7fd67effd700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd65c053b00 0x7fd65c055fc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fd67400b000 tx=0x7fd674009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:03.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.280+0000 7fd66ffff700 1 -- 172.21.15.140:0/685288060 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd678047190 con 0x7fd68014f9d0 2022-07-21T01:57:03.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.444+0000 7fd67ffff700 1 -- 172.21.15.140:0/685288060 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd6800574e0 con 0x7fd68014f9d0 2022-07-21T01:57:03.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.836+0000 7fd66ffff700 1 -- 172.21.15.140:0/685288060 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd678046d80 con 0x7fd68014f9d0 2022-07-21T01:57:03.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 -- 172.21.15.140:0/685288060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd65c053b00 msgr2=0x7fd65c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd65c053b00 0x7fd65c055fc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fd67400b000 tx=0x7fd674009040).stop 2022-07-21T01:57:03.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 -- 172.21.15.140:0/685288060 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd68014f9d0 msgr2=0x7fd68010ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd68014f9d0 0x7fd68010ed90 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7fd670001800 tx=0x7fd678016000).stop 2022-07-21T01:57:03.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 -- 172.21.15.140:0/685288060 shutdown_connections 2022-07-21T01:57:03.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd65c053b00 0x7fd65c055fc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd68014f9d0 0x7fd68010ed90 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6800b5970 0x7fd680063cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 --2- 172.21.15.140:0/685288060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6800af8c0 0x7fd6801426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 -- 172.21.15.140:0/685288060 >> 172.21.15.140:0/685288060 conn(0x7fd680063490 msgr2=0x7fd6801395f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:03.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 -- 172.21.15.140:0/685288060 shutdown_connections 2022-07-21T01:57:03.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.840+0000 7fd67ffff700 1 -- 172.21.15.140:0/685288060 wait complete. 2022-07-21T01:57:03.858 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:57:03.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.972+0000 7fdfb30c8700 1 Processor -- start 2022-07-21T01:57:03.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.972+0000 7fdfb30c8700 1 -- start start 2022-07-21T01:57:03.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb30c8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdfac0b6a70 0x7fdfac0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:03.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb30c8700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfac0acae0 con 0x7fdfac0ae670 2022-07-21T01:57:03.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb30c8700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfac0acd10 con 0x7fdfac0b6690 2022-07-21T01:57:03.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb30c8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac147100 con 0x7fdfac0b6a70 2022-07-21T01:57:03.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb1665700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fdfac0ae670 0x7fdfac0aeb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54166/0 (socket says 172.21.15.140:54166) 2022-07-21T01:57:03.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb1665700 1 -- 172.21.15.140:0/1961367072 learned_addr learned my addr 172.21.15.140:0/1961367072 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:03.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfabfff700 1 --2- 172.21.15.140:0/1961367072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdfac0b6a70 0x7fdfac0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:03.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfab7fe700 1 -- 172.21.15.140:0/1961367072 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300680231 0 0) 0x7fdfac0acae0 con 0x7fdfac0ae670 2022-07-21T01:57:03.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfab7fe700 1 -- 172.21.15.140:0/1961367072 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf98003260 con 0x7fdfac0ae670 2022-07-21T01:57:03.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfab7fe700 1 -- 172.21.15.140:0/1961367072 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607833768 0 0) 0x7fdfac0acd10 con 0x7fdfac0b6690 2022-07-21T01:57:03.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfab7fe700 1 -- 172.21.15.140:0/1961367072 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfac0acae0 con 0x7fdfac0b6690 2022-07-21T01:57:03.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfabfff700 1 -- 172.21.15.140:0/1961367072 >> v1:172.21.15.152:6789/0 conn(0x7fdfac0b6690 legacy=0x7fdfac13c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfabfff700 1 -- 172.21.15.140:0/1961367072 >> v1:172.21.15.140:6789/0 conn(0x7fdfac0ae670 legacy=0x7fdfac0aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfab7fe700 1 -- 172.21.15.140:0/1961367072 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4106539440 0 0) 0x7fdf98003260 con 0x7fdfac0ae670 2022-07-21T01:57:03.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfabfff700 1 -- 172.21.15.140:0/1961367072 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac0acfb0 con 0x7fdfac0b6a70 2022-07-21T01:57:03.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfabfff700 1 --2- 172.21.15.140:0/1961367072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdfac0b6a70 0x7fdfac0b59a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fdfa4001490 tx=0x7fdfa0006df0).ready entity=mon.2 client_cookie=4a1a22d6cb63281c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:03.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfab7fe700 1 -- 172.21.15.140:0/1961367072 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfa40019e0 con 0x7fdfac0b6a70 2022-07-21T01:57:03.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfab7fe700 1 -- 172.21.15.140:0/1961367072 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa0000e90 con 0x7fdfac0b6a70 2022-07-21T01:57:03.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfab7fe700 1 -- 172.21.15.140:0/1961367072 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdfa0012460 con 0x7fdfac0b6a70 2022-07-21T01:57:03.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1961367072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdfac0b6a70 msgr2=0x7fdfac0b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:03.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb30c8700 1 --2- 172.21.15.140:0/1961367072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdfac0b6a70 0x7fdfac0b59a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fdfa4001490 tx=0x7fdfa0006df0).stop 2022-07-21T01:57:03.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1961367072 shutdown_connections 2022-07-21T01:57:03.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb30c8700 1 --2- 172.21.15.140:0/1961367072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdfac0b6a70 0x7fdfac0b59a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:03.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1961367072 >> 172.21.15.140:0/1961367072 conn(0x7fdfac0b9f70 msgr2=0x7fdfac0b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:03.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1961367072 shutdown_connections 2022-07-21T01:57:03.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.976+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1961367072 wait complete. 2022-07-21T01:57:03.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb30c8700 1 Processor -- start 2022-07-21T01:57:03.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb30c8700 1 -- start start 2022-07-21T01:57:04.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb30c8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdfac0ae670 0x7fdfac112da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:04.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb30c8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfac0b6690 0x7fdfac1523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:04.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb30c8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdfac0b6a70 0x7fdfac1528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:04.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb30c8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac116870 con 0x7fdfac0ae670 2022-07-21T01:57:04.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb30c8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac116b10 con 0x7fdfac0b6690 2022-07-21T01:57:04.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb30c8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac116db0 con 0x7fdfac0b6a70 2022-07-21T01:57:04.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb0e64700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdfac0ae670 0x7fdfac112da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:04.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb0e64700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdfac0ae670 0x7fdfac112da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40022/0 (socket says 172.21.15.140:40022) 2022-07-21T01:57:04.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb0e64700 1 -- 172.21.15.140:0/1374809570 learned_addr learned my addr 172.21.15.140:0/1374809570 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:04.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfabfff700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfac0b6690 0x7fdfac1523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:04.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.980+0000 7fdfb1665700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdfac0b6a70 0x7fdfac1528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:04.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfb0e64700 1 -- 172.21.15.140:0/1374809570 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdfac0b6a70 msgr2=0x7fdfac1528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:04.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfb0e64700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdfac0b6a70 0x7fdfac1528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:04.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfb0e64700 1 -- 172.21.15.140:0/1374809570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfac0b6690 msgr2=0x7fdfac1523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:04.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfb0e64700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfac0b6690 0x7fdfac1523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:04.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfb0e64700 1 -- 172.21.15.140:0/1374809570 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa000b180 con 0x7fdfac0ae670 2022-07-21T01:57:04.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfabfff700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfac0b6690 0x7fdfac1523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:04.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfb0e64700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdfac0ae670 0x7fdfac112da0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7fdf9c001110 tx=0x7fdf9c007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:04.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfa9ffb700 1 -- 172.21.15.140:0/1374809570 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9c012400 con 0x7fdfac0ae670 2022-07-21T01:57:04.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfa9ffb700 1 -- 172.21.15.140:0/1374809570 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c012560 con 0x7fdfac0ae670 2022-07-21T01:57:04.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1374809570 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfac117250 con 0x7fdfac0ae670 2022-07-21T01:57:04.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfa9ffb700 1 -- 172.21.15.140:0/1374809570 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf9c0128c0 con 0x7fdfac0ae670 2022-07-21T01:57:04.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1374809570 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac113290 con 0x7fdfac0ae670 2022-07-21T01:57:04.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1374809570 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfac1166e0 con 0x7fdfac0ae670 2022-07-21T01:57:04.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfa9ffb700 1 -- 172.21.15.140:0/1374809570 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdf9c019030 con 0x7fdfac0ae670 2022-07-21T01:57:04.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfa9ffb700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdf8c053b00 0x7fdf8c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:04.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfa9ffb700 1 -- 172.21.15.140:0/1374809570 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fdf9c06b330 con 0x7fdfac0ae670 2022-07-21T01:57:04.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.984+0000 7fdfabfff700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdf8c053b00 0x7fdf8c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:04.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.988+0000 7fdfabfff700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdf8c053b00 0x7fdf8c055fc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fdfa001a000 tx=0x7fdfa0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:04.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:03.992+0000 7fdfa9ffb700 1 -- 172.21.15.140:0/1374809570 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdf9c042200 con 0x7fdfac0ae670 2022-07-21T01:57:04.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.116+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1374809570 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdfac116f70 con 0x7fdfac0ae670 2022-07-21T01:57:04.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.856+0000 7fdfa9ffb700 1 -- 172.21.15.140:0/1374809570 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) v1 ==== 185+0+0 (secure 0 0 0) 0x7fdf9c045820 con 0x7fdfac0ae670 2022-07-21T01:57:04.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1374809570 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdf8c053b00 msgr2=0x7fdf8c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:04.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdf8c053b00 0x7fdf8c055fc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7fdfa001a000 tx=0x7fdfa0014000).stop 2022-07-21T01:57:04.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1374809570 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdfac0ae670 msgr2=0x7fdfac112da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:04.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdfac0ae670 0x7fdfac112da0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7fdf9c001110 tx=0x7fdf9c007ae0).stop 2022-07-21T01:57:04.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1374809570 shutdown_connections 2022-07-21T01:57:04.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdf8c053b00 0x7fdf8c055fc0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:04.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdfac0b6a70 0x7fdfac1528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:04.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdfac0b6690 0x7fdfac1523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:04.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 --2- 172.21.15.140:0/1374809570 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdfac0ae670 0x7fdfac112da0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:04.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1374809570 >> 172.21.15.140:0/1374809570 conn(0x7fdfac0b9f70 msgr2=0x7fdfac141550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:04.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1374809570 shutdown_connections 2022-07-21T01:57:04.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.860+0000 7fdfb30c8700 1 -- 172.21.15.140:0/1374809570 wait complete. 2022-07-21T01:57:04.867 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:57:04.880 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:57:04.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.992+0000 7f042667c700 1 Processor -- start 2022-07-21T01:57:04.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.992+0000 7f042667c700 1 -- start start 2022-07-21T01:57:05.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.992+0000 7f042667c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04200b6840 0x7f04200b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:05.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.992+0000 7f042667c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f04200b4bc0 0x7f0420141a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:05.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.992+0000 7f042667c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0420066b10 con 0x7f04200b3cf0 2022-07-21T01:57:05.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.992+0000 7f042667c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04200b6c20 con 0x7f04200b4bc0 2022-07-21T01:57:05.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f042667c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04200aead0 con 0x7f04200b6840 2022-07-21T01:57:05.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f0425e7b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f04200b4bc0 0x7f0420141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:05.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f0424e79700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f04200b3cf0 0x7f0420146040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54178/0 (socket says 172.21.15.140:54178) 2022-07-21T01:57:05.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f0424e79700 1 -- 172.21.15.140:0/2603855814 learned_addr learned my addr 172.21.15.140:0/2603855814 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:05.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f042567a700 1 --2- 172.21.15.140:0/2603855814 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04200b6840 0x7f04200b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:05.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f0425e7b700 1 -- 172.21.15.140:0/2603855814 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04200b6840 msgr2=0x7f04200b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:05.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f0425e7b700 1 --2- 172.21.15.140:0/2603855814 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04200b6840 0x7f04200b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:05.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f0425e7b700 1 -- 172.21.15.140:0/2603855814 >> v1:172.21.15.140:6789/0 conn(0x7f04200b3cf0 legacy=0x7f0420146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:05.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f0425e7b700 1 -- 172.21.15.140:0/2603855814 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04200aed70 con 0x7f04200b4bc0 2022-07-21T01:57:05.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f0425e7b700 1 --2- 172.21.15.140:0/2603855814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04200b4bc0 0x7f0420141a90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f04180032f0 tx=0x7f04180091d0).ready entity=mon.0 client_cookie=ce8fb6ca401798c2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:05.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f040bfff700 1 -- 172.21.15.140:0/2603855814 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0418009ea0 con 0x7f04200b4bc0 2022-07-21T01:57:05.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f040bfff700 1 -- 172.21.15.140:0/2603855814 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041800b8a0 con 0x7f04200b4bc0 2022-07-21T01:57:05.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f040bfff700 1 -- 172.21.15.140:0/2603855814 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f041800ba10 con 0x7f04200b4bc0 2022-07-21T01:57:05.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f042667c700 1 -- 172.21.15.140:0/2603855814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04200b4bc0 msgr2=0x7f0420141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:05.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f042667c700 1 --2- 172.21.15.140:0/2603855814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04200b4bc0 0x7f0420141a90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f04180032f0 tx=0x7f04180091d0).stop 2022-07-21T01:57:05.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f042667c700 1 -- 172.21.15.140:0/2603855814 shutdown_connections 2022-07-21T01:57:05.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f042667c700 1 --2- 172.21.15.140:0/2603855814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04200b4bc0 0x7f0420141a90 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:05.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f042667c700 1 --2- 172.21.15.140:0/2603855814 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04200b6840 0x7f04200b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:05.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f042667c700 1 -- 172.21.15.140:0/2603855814 >> 172.21.15.140:0/2603855814 conn(0x7f04200b9e10 msgr2=0x7f04200b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:05.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f042667c700 1 -- 172.21.15.140:0/2603855814 shutdown_connections 2022-07-21T01:57:05.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:04.996+0000 7f042667c700 1 -- 172.21.15.140:0/2603855814 wait complete. 2022-07-21T01:57:05.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f042667c700 1 Processor -- start 2022-07-21T01:57:05.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f042667c700 1 -- start start 2022-07-21T01:57:05.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f042667c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04200b3cf0 0x7f0420155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:05.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f042667c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04200b4bc0 0x7f0420140710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:05.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f042667c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04200b6840 0x7f0420134240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:05.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f042667c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0420132cd0 con 0x7f04200b4bc0 2022-07-21T01:57:05.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f042667c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04200b2400 con 0x7f04200b6840 2022-07-21T01:57:05.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f042667c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0420140c00 con 0x7f04200b3cf0 2022-07-21T01:57:05.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0424e79700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04200b4bc0 0x7f0420140710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:05.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0424e79700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04200b4bc0 0x7f0420140710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40038/0 (socket says 172.21.15.140:40038) 2022-07-21T01:57:05.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0424e79700 1 -- 172.21.15.140:0/3765072665 learned_addr learned my addr 172.21.15.140:0/3765072665 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:05.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0425e7b700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04200b6840 0x7f0420134240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:05.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f042567a700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04200b3cf0 0x7f0420155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:05.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0424e79700 1 -- 172.21.15.140:0/3765072665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04200b3cf0 msgr2=0x7f0420155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:05.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0424e79700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04200b3cf0 0x7f0420155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:05.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0424e79700 1 -- 172.21.15.140:0/3765072665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04200b6840 msgr2=0x7f0420134240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:05.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0424e79700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04200b6840 0x7f0420134240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:05.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0424e79700 1 -- 172.21.15.140:0/3765072665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0418009470 con 0x7f04200b4bc0 2022-07-21T01:57:05.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0424e79700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04200b4bc0 0x7f0420140710 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f0420067e80 tx=0x7f0414007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:05.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0412ffd700 1 -- 172.21.15.140:0/3765072665 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04140070d0 con 0x7f04200b4bc0 2022-07-21T01:57:05.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0412ffd700 1 -- 172.21.15.140:0/3765072665 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0414007230 con 0x7f04200b4bc0 2022-07-21T01:57:05.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f042667c700 1 -- 172.21.15.140:0/3765072665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0420133a90 con 0x7f04200b4bc0 2022-07-21T01:57:05.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f0412ffd700 1 -- 172.21.15.140:0/3765072665 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0414005610 con 0x7f04200b4bc0 2022-07-21T01:57:05.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.000+0000 7f042667c700 1 -- 172.21.15.140:0/3765072665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04200b20b0 con 0x7f04200b4bc0 2022-07-21T01:57:05.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.004+0000 7f042667c700 1 -- 172.21.15.140:0/3765072665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04200b2fe0 con 0x7f04200b4bc0 2022-07-21T01:57:05.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.004+0000 7f0412ffd700 1 -- 172.21.15.140:0/3765072665 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0414010070 con 0x7f04200b4bc0 2022-07-21T01:57:05.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.004+0000 7f0412ffd700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f03fc053a70 0x7f03fc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:05.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.004+0000 7f0412ffd700 1 -- 172.21.15.140:0/3765072665 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f041406b220 con 0x7f04200b4bc0 2022-07-21T01:57:05.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.004+0000 7f042567a700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f03fc053a70 0x7f03fc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:05.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.004+0000 7f042567a700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f03fc053a70 0x7f03fc055f30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f041c00c000 tx=0x7f041c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:05.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.008+0000 7f0412ffd700 1 -- 172.21.15.140:0/3765072665 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f04200b2fe0 con 0x7f04200b4bc0 2022-07-21T01:57:05.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:05.136+0000 7f042667c700 1 -- 172.21.15.140:0/3765072665 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f04200b2fe0 con 0x7f04200b4bc0 2022-07-21T01:57:06.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.052+0000 7f0412ffd700 1 -- 172.21.15.140:0/3765072665 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v127) v1 ==== 173+0+0 (secure 0 0 0) 0x7f04200b2fe0 con 0x7f04200b4bc0 2022-07-21T01:57:06.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 -- 172.21.15.140:0/3765072665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f03fc053a70 msgr2=0x7f03fc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f03fc053a70 0x7f03fc055f30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f041c00c000 tx=0x7f041c008040).stop 2022-07-21T01:57:06.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 -- 172.21.15.140:0/3765072665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04200b4bc0 msgr2=0x7f0420140710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04200b4bc0 0x7f0420140710 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f0420067e80 tx=0x7f0414007c10).stop 2022-07-21T01:57:06.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 -- 172.21.15.140:0/3765072665 shutdown_connections 2022-07-21T01:57:06.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f03fc053a70 0x7f03fc055f30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04200b6840 0x7f0420134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04200b4bc0 0x7f0420140710 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 --2- 172.21.15.140:0/3765072665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04200b3cf0 0x7f0420155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 -- 172.21.15.140:0/3765072665 >> 172.21.15.140:0/3765072665 conn(0x7f04200b9e10 msgr2=0x7f0420066d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:06.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 -- 172.21.15.140:0/3765072665 shutdown_connections 2022-07-21T01:57:06.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.056+0000 7f042667c700 1 -- 172.21.15.140:0/3765072665 wait complete. 2022-07-21T01:57:06.061 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:57:06.072 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-07-21T01:57:06.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f8951562700 1 Processor -- start 2022-07-21T01:57:06.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f8951562700 1 -- start start 2022-07-21T01:57:06.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f8951562700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f894c0b5990 0x7f894c0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:06.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f8951562700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f894c0af8e0 0x7f894c146960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:06.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f8951562700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894c142a90 con 0x7f894c142ef0 2022-07-21T01:57:06.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f8951562700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894c146e50 con 0x7f894c0af8e0 2022-07-21T01:57:06.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f8951562700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0b5d70 con 0x7f894c0b5990 2022-07-21T01:57:06.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f894b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f894c0af8e0 0x7f894c146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:06.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f894b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f894c0af8e0 0x7f894c146960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40044/0 (socket says 172.21.15.140:40044) 2022-07-21T01:57:06.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f894b7fe700 1 -- 172.21.15.140:0/887382857 learned_addr learned my addr 172.21.15.140:0/887382857 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:06.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f894bfff700 1 --2- 172.21.15.140:0/887382857 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f894c0b5990 0x7f894c0ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:06.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f894b7fe700 1 -- 172.21.15.140:0/887382857 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f894c0b5990 msgr2=0x7f894c0ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:06.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f894b7fe700 1 --2- 172.21.15.140:0/887382857 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f894c0b5990 0x7f894c0ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f894b7fe700 1 -- 172.21.15.140:0/887382857 >> v1:172.21.15.152:6789/0 conn(0x7f894c142ef0 legacy=0x7f894c1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.180+0000 7f894b7fe700 1 -- 172.21.15.140:0/887382857 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c0afdf0 con 0x7f894c0af8e0 2022-07-21T01:57:06.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894b7fe700 1 --2- 172.21.15.140:0/887382857 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f894c0af8e0 0x7f894c146960 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f893c0032f0 tx=0x7f893c009290).ready entity=mon.0 client_cookie=8337404db3e2910a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:06.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894affd700 1 -- 172.21.15.140:0/887382857 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f893c00b8a0 con 0x7f894c0af8e0 2022-07-21T01:57:06.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894affd700 1 -- 172.21.15.140:0/887382857 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893c00ba00 con 0x7f894c0af8e0 2022-07-21T01:57:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894affd700 1 -- 172.21.15.140:0/887382857 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f893c00bb70 con 0x7f894c0af8e0 2022-07-21T01:57:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 -- 172.21.15.140:0/887382857 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f894c0af8e0 msgr2=0x7f894c146960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 --2- 172.21.15.140:0/887382857 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f894c0af8e0 0x7f894c146960 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f893c0032f0 tx=0x7f893c009290).stop 2022-07-21T01:57:06.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 -- 172.21.15.140:0/887382857 shutdown_connections 2022-07-21T01:57:06.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 --2- 172.21.15.140:0/887382857 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f894c0af8e0 0x7f894c146960 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 --2- 172.21.15.140:0/887382857 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f894c0b5990 0x7f894c0ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 -- 172.21.15.140:0/887382857 >> 172.21.15.140:0/887382857 conn(0x7f894c063480 msgr2=0x7f894c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:06.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 -- 172.21.15.140:0/887382857 shutdown_connections 2022-07-21T01:57:06.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 -- 172.21.15.140:0/887382857 wait complete. 2022-07-21T01:57:06.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 Processor -- start 2022-07-21T01:57:06.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 -- start start 2022-07-21T01:57:06.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f894c0af8e0 0x7f894c0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:06.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f894c0b5990 0x7f894c065cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:06.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f894c142ef0 0x7f894c0661e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:06.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c1140c0 con 0x7f894c0b5990 2022-07-21T01:57:06.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c113120 con 0x7f894c142ef0 2022-07-21T01:57:06.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8951562700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0670f0 con 0x7f894c0af8e0 2022-07-21T01:57:06.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f894c0b5990 0x7f894c065cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:06.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f894c0b5990 0x7f894c065cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40052/0 (socket says 172.21.15.140:40052) 2022-07-21T01:57:06.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894b7fe700 1 -- 172.21.15.140:0/1708983567 learned_addr learned my addr 172.21.15.140:0/1708983567 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:06.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894bfff700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f894c0af8e0 0x7f894c0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:06.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f8950d61700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f894c142ef0 0x7f894c0661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:06.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894b7fe700 1 -- 172.21.15.140:0/1708983567 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f894c0af8e0 msgr2=0x7f894c0b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894b7fe700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f894c0af8e0 0x7f894c0b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894b7fe700 1 -- 172.21.15.140:0/1708983567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f894c142ef0 msgr2=0x7f894c0661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894b7fe700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f894c142ef0 0x7f894c0661e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894b7fe700 1 -- 172.21.15.140:0/1708983567 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c009530 con 0x7f894c0b5990 2022-07-21T01:57:06.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f894b7fe700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f894c0b5990 0x7f894c065cf0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f893c0032f0 tx=0x7f893c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:06.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.184+0000 7f89497fa700 1 -- 172.21.15.140:0/1708983567 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f893c00b8a0 con 0x7f894c0b5990 2022-07-21T01:57:06.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.188+0000 7f89497fa700 1 -- 172.21.15.140:0/1708983567 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893c021440 con 0x7f894c0b5990 2022-07-21T01:57:06.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.188+0000 7f8951562700 1 -- 172.21.15.140:0/1708983567 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894c112540 con 0x7f894c0b5990 2022-07-21T01:57:06.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.188+0000 7f89497fa700 1 -- 172.21.15.140:0/1708983567 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f893c0217a0 con 0x7f894c0b5990 2022-07-21T01:57:06.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.188+0000 7f8951562700 1 -- 172.21.15.140:0/1708983567 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c066eb0 con 0x7f894c0b5990 2022-07-21T01:57:06.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.188+0000 7f8951562700 1 -- 172.21.15.140:0/1708983567 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894c0673d0 con 0x7f894c0b5990 2022-07-21T01:57:06.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.188+0000 7f89497fa700 1 -- 172.21.15.140:0/1708983567 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f893c027020 con 0x7f894c0b5990 2022-07-21T01:57:06.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.188+0000 7f89497fa700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89300539e0 0x7f8930055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:06.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.188+0000 7f894bfff700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89300539e0 0x7f8930055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:06.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.188+0000 7f89497fa700 1 -- 172.21.15.140:0/1708983567 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f893c06d200 con 0x7f894c0b5990 2022-07-21T01:57:06.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.188+0000 7f894bfff700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89300539e0 0x7f8930055ea0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f8934006000 tx=0x7f8934006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:06.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.192+0000 7f89497fa700 1 -- 172.21.15.140:0/1708983567 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f893c0442f0 con 0x7f894c0b5990 2022-07-21T01:57:06.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.316+0000 7f8951562700 1 -- 172.21.15.140:0/1708983567 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f894c0b0090 con 0x7f894c0b5990 2022-07-21T01:57:06.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.324+0000 7f89497fa700 1 -- 172.21.15.140:0/1708983567 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f893c0150a0 con 0x7f894c0b5990 2022-07-21T01:57:06.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 -- 172.21.15.140:0/1708983567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89300539e0 msgr2=0x7f8930055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89300539e0 0x7f8930055ea0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f8934006000 tx=0x7f8934006d90).stop 2022-07-21T01:57:06.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 -- 172.21.15.140:0/1708983567 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f894c0b5990 msgr2=0x7f894c065cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f894c0b5990 0x7f894c065cf0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f893c0032f0 tx=0x7f893c016000).stop 2022-07-21T01:57:06.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 -- 172.21.15.140:0/1708983567 shutdown_connections 2022-07-21T01:57:06.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89300539e0 0x7f8930055ea0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f894c142ef0 0x7f894c0661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f894c0b5990 0x7f894c065cf0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 --2- 172.21.15.140:0/1708983567 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f894c0af8e0 0x7f894c0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 -- 172.21.15.140:0/1708983567 >> 172.21.15.140:0/1708983567 conn(0x7f894c063480 msgr2=0x7f894c1433a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:06.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 -- 172.21.15.140:0/1708983567 shutdown_connections 2022-07-21T01:57:06.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.328+0000 7f8951562700 1 -- 172.21.15.140:0/1708983567 wait complete. 2022-07-21T01:57:06.345 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2022-07-21T01:57:06.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.448+0000 7f5515633700 1 Processor -- start 2022-07-21T01:57:06.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.448+0000 7f5515633700 1 -- start start 2022-07-21T01:57:06.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.448+0000 7f5515633700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5510147130 0x7f55100aac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:06.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.448+0000 7f5515633700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5510147640 con 0x7f551005cda0 2022-07-21T01:57:06.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.448+0000 7f5515633700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55101478e0 con 0x7f5510148020 2022-07-21T01:57:06.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.448+0000 7f5515633700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5510147b80 con 0x7f5510147130 2022-07-21T01:57:06.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.448+0000 7f550ffff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f551005cda0 0x7f55100ab1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54204/0 (socket says 172.21.15.140:54204) 2022-07-21T01:57:06.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.448+0000 7f550ffff700 1 -- 172.21.15.140:0/2762372497 learned_addr learned my addr 172.21.15.140:0/2762372497 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:06.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.448+0000 7f550f7fe700 1 --2- 172.21.15.140:0/2762372497 >> v2:172.21.15.152:3300/0 conn(0x7f5510147130 0x7f55100aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:06.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f550effd700 1 -- 172.21.15.140:0/2762372497 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414402897 0 0) 0x7f5510147640 con 0x7f551005cda0 2022-07-21T01:57:06.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f550effd700 1 -- 172.21.15.140:0/2762372497 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f4003260 con 0x7f551005cda0 2022-07-21T01:57:06.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f550f7fe700 1 -- 172.21.15.140:0/2762372497 >> v1:172.21.15.152:6789/0 conn(0x7f5510148020 legacy=0x7f551005ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f550f7fe700 1 -- 172.21.15.140:0/2762372497 >> v1:172.21.15.140:6789/0 conn(0x7f551005cda0 legacy=0x7f55100ab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f550f7fe700 1 -- 172.21.15.140:0/2762372497 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55100ac0f0 con 0x7f5510147130 2022-07-21T01:57:06.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f550f7fe700 1 --2- 172.21.15.140:0/2762372497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5510147130 0x7f55100aac30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f54f8002f70 tx=0x7f54f8006df0).ready entity=mon.1 client_cookie=4568763304af9189 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:06.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f550effd700 1 -- 172.21.15.140:0/2762372497 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54f800d610 con 0x7f5510147130 2022-07-21T01:57:06.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f550effd700 1 -- 172.21.15.140:0/2762372497 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f8000e90 con 0x7f5510147130 2022-07-21T01:57:06.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f550effd700 1 -- 172.21.15.140:0/2762372497 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54f800d960 con 0x7f5510147130 2022-07-21T01:57:06.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f5515633700 1 -- 172.21.15.140:0/2762372497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5510147130 msgr2=0x7f55100aac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f5515633700 1 --2- 172.21.15.140:0/2762372497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5510147130 0x7f55100aac30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f54f8002f70 tx=0x7f54f8006df0).stop 2022-07-21T01:57:06.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f5515633700 1 -- 172.21.15.140:0/2762372497 shutdown_connections 2022-07-21T01:57:06.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f5515633700 1 --2- 172.21.15.140:0/2762372497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5510147130 0x7f55100aac30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f5515633700 1 -- 172.21.15.140:0/2762372497 >> 172.21.15.140:0/2762372497 conn(0x7f55100b5e60 msgr2=0x7f55100b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:06.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f5515633700 1 -- 172.21.15.140:0/2762372497 shutdown_connections 2022-07-21T01:57:06.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f5515633700 1 -- 172.21.15.140:0/2762372497 wait complete. 2022-07-21T01:57:06.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f5515633700 1 Processor -- start 2022-07-21T01:57:06.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.452+0000 7f5515633700 1 -- start start 2022-07-21T01:57:06.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f5515633700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f551005cda0 0x7f551014af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:06.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f5515633700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5510147130 0x7f551009ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:06.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f5515633700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5510148020 0x7f5510159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:06.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f5515633700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5510159620 con 0x7f5510148020 2022-07-21T01:57:06.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f5515633700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55101598c0 con 0x7f551005cda0 2022-07-21T01:57:06.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f5515633700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551009b2c0 con 0x7f5510147130 2022-07-21T01:57:06.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550f7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5510147130 0x7f551009ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:06.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f5514e32700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5510148020 0x7f5510159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:06.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550f7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5510147130 0x7f551009ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36796/0 (socket says 172.21.15.140:36796) 2022-07-21T01:57:06.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550f7fe700 1 -- 172.21.15.140:0/2800731218 learned_addr learned my addr 172.21.15.140:0/2800731218 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:06.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550ffff700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f551005cda0 0x7f551014af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:06.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550f7fe700 1 -- 172.21.15.140:0/2800731218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f551005cda0 msgr2=0x7f551014af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550f7fe700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f551005cda0 0x7f551014af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550f7fe700 1 -- 172.21.15.140:0/2800731218 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5510148020 msgr2=0x7f5510159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:06.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550f7fe700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5510148020 0x7f5510159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:06.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550f7fe700 1 -- 172.21.15.140:0/2800731218 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f800b180 con 0x7f5510147130 2022-07-21T01:57:06.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550ffff700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f551005cda0 0x7f551014af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:06.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f5514e32700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5510148020 0x7f5510159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:06.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550f7fe700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5510147130 0x7f551009ac60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f54f8002f70 tx=0x7f54f8014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:06.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550d7fa700 1 -- 172.21.15.140:0/2800731218 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54f8016070 con 0x7f5510147130 2022-07-21T01:57:06.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550d7fa700 1 -- 172.21.15.140:0/2800731218 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f800de90 con 0x7f5510147130 2022-07-21T01:57:06.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f5515633700 1 -- 172.21.15.140:0/2800731218 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551015a250 con 0x7f5510147130 2022-07-21T01:57:06.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550d7fa700 1 -- 172.21.15.140:0/2800731218 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54f8013da0 con 0x7f5510147130 2022-07-21T01:57:06.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f5515633700 1 -- 172.21.15.140:0/2800731218 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5510094a90 con 0x7f5510147130 2022-07-21T01:57:06.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f5515633700 1 -- 172.21.15.140:0/2800731218 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55100574e0 con 0x7f5510147130 2022-07-21T01:57:06.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.456+0000 7f550d7fa700 1 -- 172.21.15.140:0/2800731218 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f54f800f030 con 0x7f5510147130 2022-07-21T01:57:06.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.460+0000 7f550d7fa700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54e8053950 0x7f54e8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:06.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.460+0000 7f550d7fa700 1 -- 172.21.15.140:0/2800731218 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f54f806c5b0 con 0x7f5510147130 2022-07-21T01:57:06.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.460+0000 7f550ffff700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54e8053950 0x7f54e8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:06.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.460+0000 7f550ffff700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54e8053950 0x7f54e8055e10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f550800d000 tx=0x7f5508009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:06.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.468+0000 7f550d7fa700 1 -- 172.21.15.140:0/2800731218 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f54f8043620 con 0x7f5510147130 2022-07-21T01:57:06.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:06.632+0000 7f5515633700 1 -- 172.21.15.140:0/2800731218 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f55100ab8c0 con 0x7f5510147130 2022-07-21T01:57:07.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.060+0000 7f550d7fa700 1 -- 172.21.15.140:0/2800731218 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v396) v1 ==== 122+0+0 (secure 0 0 0) 0x7f54f8022020 con 0x7f5510147130 2022-07-21T01:57:07.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.064+0000 7f5515633700 1 -- 172.21.15.140:0/2800731218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54e8053950 msgr2=0x7f54e8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:07.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.064+0000 7f5515633700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54e8053950 0x7f54e8055e10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f550800d000 tx=0x7f5508009040).stop 2022-07-21T01:57:07.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.064+0000 7f5515633700 1 -- 172.21.15.140:0/2800731218 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5510147130 msgr2=0x7f551009ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:07.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.064+0000 7f5515633700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5510147130 0x7f551009ac60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f54f8002f70 tx=0x7f54f8014000).stop 2022-07-21T01:57:07.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.068+0000 7f5515633700 1 -- 172.21.15.140:0/2800731218 shutdown_connections 2022-07-21T01:57:07.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.068+0000 7f5515633700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f54e8053950 0x7f54e8055e10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:07.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.068+0000 7f5515633700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5510148020 0x7f5510159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:07.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.068+0000 7f5515633700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5510147130 0x7f551009ac60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:07.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.068+0000 7f5515633700 1 --2- 172.21.15.140:0/2800731218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f551005cda0 0x7f551014af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:07.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.068+0000 7f5515633700 1 -- 172.21.15.140:0/2800731218 >> 172.21.15.140:0/2800731218 conn(0x7f55100b5e60 msgr2=0x7f55100afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:07.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.068+0000 7f5515633700 1 -- 172.21.15.140:0/2800731218 shutdown_connections 2022-07-21T01:57:07.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.068+0000 7f5515633700 1 -- 172.21.15.140:0/2800731218 wait complete. 2022-07-21T01:57:07.084 INFO:tasks.cephfs_test_runner:test_fs_new (tasks.cephfs.test_misc.TestMisc) ... ok 2022-07-21T01:57:07.084 INFO:tasks.cephfs_test_runner:Starting test: test_getattr_caps (tasks.cephfs.test_misc.TestMisc) 2022-07-21T01:57:07.085 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2022-07-21T01:57:07.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.188+0000 7fd107fff700 1 Processor -- start 2022-07-21T01:57:07.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.188+0000 7fd107fff700 1 -- start start 2022-07-21T01:57:07.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1080b5650 0x7fd1080b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:07.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd1080b1200 0x7fd10805c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:07.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1080aea30 con 0x7fd10805bd90 2022-07-21T01:57:07.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd108142970 con 0x7fd1080b1200 2022-07-21T01:57:07.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1080aec60 con 0x7fd1080b5650 2022-07-21T01:57:07.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd1067fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd1080b1200 0x7fd10805c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:07.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd1067fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd1080b1200 0x7fd10805c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40072/0 (socket says 172.21.15.140:40072) 2022-07-21T01:57:07.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd1077fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd10805bd90 0x7fd108142d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54222/0 (socket says 172.21.15.140:54222) 2022-07-21T01:57:07.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd1067fc700 1 -- 172.21.15.140:0/2365688102 learned_addr learned my addr 172.21.15.140:0/2365688102 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:07.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd106ffd700 1 --2- 172.21.15.140:0/2365688102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1080b5650 0x7fd1080b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:07.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd1067fc700 1 -- 172.21.15.140:0/2365688102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1080b5650 msgr2=0x7fd1080b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:07.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd1067fc700 1 --2- 172.21.15.140:0/2365688102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1080b5650 0x7fd1080b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:07.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd1067fc700 1 -- 172.21.15.140:0/2365688102 >> v1:172.21.15.140:6789/0 conn(0x7fd10805bd90 legacy=0x7fd108142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:07.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd105ffb700 1 -- 172.21.15.140:0/2365688102 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127515395 0 0) 0x7fd1080aea30 con 0x7fd10805bd90 2022-07-21T01:57:07.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd1067fc700 1 -- 172.21.15.140:0/2365688102 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1080af180 con 0x7fd1080b1200 2022-07-21T01:57:07.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd106ffd700 1 --2- 172.21.15.140:0/2365688102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1080b5650 0x7fd1080b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:07.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd1067fc700 1 --2- 172.21.15.140:0/2365688102 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1080b1200 0x7fd10805c4f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7fd0f8002f70 tx=0x7fd0f8006e50).ready entity=mon.0 client_cookie=e5340b4362d2479b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:07.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd105ffb700 1 -- 172.21.15.140:0/2365688102 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0f800d5c0 con 0x7fd1080b1200 2022-07-21T01:57:07.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd105ffb700 1 -- 172.21.15.140:0/2365688102 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f8000e90 con 0x7fd1080b1200 2022-07-21T01:57:07.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd105ffb700 1 -- 172.21.15.140:0/2365688102 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0f800d910 con 0x7fd1080b1200 2022-07-21T01:57:07.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 -- 172.21.15.140:0/2365688102 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1080b1200 msgr2=0x7fd10805c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:07.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 --2- 172.21.15.140:0/2365688102 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1080b1200 0x7fd10805c4f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7fd0f8002f70 tx=0x7fd0f8006e50).stop 2022-07-21T01:57:07.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 -- 172.21.15.140:0/2365688102 shutdown_connections 2022-07-21T01:57:07.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 --2- 172.21.15.140:0/2365688102 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1080b1200 0x7fd10805c4f0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:07.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 --2- 172.21.15.140:0/2365688102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1080b5650 0x7fd1080b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:07.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 -- 172.21.15.140:0/2365688102 >> 172.21.15.140:0/2365688102 conn(0x7fd1080634e0 msgr2=0x7fd1080b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:07.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 -- 172.21.15.140:0/2365688102 shutdown_connections 2022-07-21T01:57:07.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.192+0000 7fd107fff700 1 -- 172.21.15.140:0/2365688102 wait complete. 2022-07-21T01:57:07.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd107fff700 1 Processor -- start 2022-07-21T01:57:07.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd107fff700 1 -- start start 2022-07-21T01:57:07.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd107fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd10805bd90 0x7fd108155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:07.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd107fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1080b1200 0x7fd1080b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:07.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd107fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1080b5650 0x7fd108063f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:07.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd107fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd108063b50 con 0x7fd1080b1200 2022-07-21T01:57:07.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd107fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10813ea80 con 0x7fd10805bd90 2022-07-21T01:57:07.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd107fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1080b0780 con 0x7fd1080b5650 2022-07-21T01:57:07.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd1067fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1080b1200 0x7fd1080b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:07.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd1067fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1080b1200 0x7fd1080b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40080/0 (socket says 172.21.15.140:40080) 2022-07-21T01:57:07.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd1067fc700 1 -- 172.21.15.140:0/2653064391 learned_addr learned my addr 172.21.15.140:0/2653064391 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:07.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd1077fe700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1080b5650 0x7fd108063f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:07.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd106ffd700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd10805bd90 0x7fd108155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:07.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd1067fc700 1 -- 172.21.15.140:0/2653064391 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1080b5650 msgr2=0x7fd108063f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:07.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd1067fc700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1080b5650 0x7fd108063f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:07.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd1067fc700 1 -- 172.21.15.140:0/2653064391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd10805bd90 msgr2=0x7fd108155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:07.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd1067fc700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd10805bd90 0x7fd108155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:07.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd1067fc700 1 -- 172.21.15.140:0/2653064391 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc00b180 con 0x7fd1080b1200 2022-07-21T01:57:07.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd106ffd700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd10805bd90 0x7fd108155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:57:07.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd1067fc700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1080b1200 0x7fd1080b0290 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7fd0f8002f70 tx=0x7fd0f8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:07.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd0f7fff700 1 -- 172.21.15.140:0/2653064391 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0f8016070 con 0x7fd1080b1200 2022-07-21T01:57:07.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd0f7fff700 1 -- 172.21.15.140:0/2653064391 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f8013a00 con 0x7fd1080b1200 2022-07-21T01:57:07.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd0f7fff700 1 -- 172.21.15.140:0/2653064391 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd0f8013d60 con 0x7fd1080b1200 2022-07-21T01:57:07.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd107fff700 1 -- 172.21.15.140:0/2653064391 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f800b1e0 con 0x7fd1080b1200 2022-07-21T01:57:07.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.196+0000 7fd107fff700 1 -- 172.21.15.140:0/2653064391 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd108145c30 con 0x7fd1080b1200 2022-07-21T01:57:07.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.200+0000 7fd107fff700 1 -- 172.21.15.140:0/2653064391 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd108063cc0 con 0x7fd1080b1200 2022-07-21T01:57:07.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.200+0000 7fd0f7fff700 1 -- 172.21.15.140:0/2653064391 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd0f8025030 con 0x7fd1080b1200 2022-07-21T01:57:07.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.200+0000 7fd0f7fff700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd0e4053b30 0x7fd0e4055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:07.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.200+0000 7fd0f7fff700 1 -- 172.21.15.140:0/2653064391 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fd0f806cce0 con 0x7fd1080b1200 2022-07-21T01:57:07.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.200+0000 7fd106ffd700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd0e4053b30 0x7fd0e4055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:07.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.200+0000 7fd106ffd700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd0e4053b30 0x7fd0e4055ff0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fd0fc00b000 tx=0x7fd0fc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:07.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.204+0000 7fd0f7fff700 1 -- 172.21.15.140:0/2653064391 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd108063cc0 con 0x7fd1080b1200 2022-07-21T01:57:07.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:07.372+0000 7fd107fff700 1 -- 172.21.15.140:0/2653064391 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7fd108063c90 con 0x7fd1080b1200 2022-07-21T01:57:08.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.068+0000 7fd0f7fff700 1 -- 172.21.15.140:0/2653064391 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v397) v1 ==== 131+0+0 (secure 0 0 0) 0x7fd0f80230c0 con 0x7fd1080b1200 2022-07-21T01:57:08.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 -- 172.21.15.140:0/2653064391 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd0e4053b30 msgr2=0x7fd0e4055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd0e4053b30 0x7fd0e4055ff0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fd0fc00b000 tx=0x7fd0fc008040).stop 2022-07-21T01:57:08.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 -- 172.21.15.140:0/2653064391 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1080b1200 msgr2=0x7fd1080b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1080b1200 0x7fd1080b0290 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7fd0f8002f70 tx=0x7fd0f8014000).stop 2022-07-21T01:57:08.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 -- 172.21.15.140:0/2653064391 shutdown_connections 2022-07-21T01:57:08.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd0e4053b30 0x7fd0e4055ff0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.077 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1080b5650 0x7fd108063f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1080b1200 0x7fd1080b0290 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 --2- 172.21.15.140:0/2653064391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd10805bd90 0x7fd108155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 -- 172.21.15.140:0/2653064391 >> 172.21.15.140:0/2653064391 conn(0x7fd1080634e0 msgr2=0x7fd1080b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:08.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 -- 172.21.15.140:0/2653064391 shutdown_connections 2022-07-21T01:57:08.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.072+0000 7fd107fff700 1 -- 172.21.15.140:0/2653064391 wait complete. 2022-07-21T01:57:08.092 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:57:08.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.200+0000 7f8899892700 1 Processor -- start 2022-07-21T01:57:08.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.200+0000 7f8899892700 1 -- start start 2022-07-21T01:57:08.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.200+0000 7f8899892700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f88940b59d0 0x7f88940b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:08.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.200+0000 7f8899892700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f88940af920 0x7f88940ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:08.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.200+0000 7f8899892700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88940adf70 con 0x7f889405d0c0 2022-07-21T01:57:08.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.200+0000 7f8899892700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88940b5db0 con 0x7f88940af920 2022-07-21T01:57:08.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.200+0000 7f8899892700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8894146300 con 0x7f88940b59d0 2022-07-21T01:57:08.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8893fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f88940af920 0x7f88940ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:08.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8893fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f88940af920 0x7f88940ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40086/0 (socket says 172.21.15.140:40086) 2022-07-21T01:57:08.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8893fff700 1 -- 172.21.15.140:0/3488185677 learned_addr learned my addr 172.21.15.140:0/3488185677 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:08.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8898890700 1 --2- 172.21.15.140:0/3488185677 >> v2:172.21.15.152:3300/0 conn(0x7f88940b59d0 0x7f88940b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:08.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8893fff700 1 -- 172.21.15.140:0/3488185677 >> v2:172.21.15.152:3300/0 conn(0x7f88940b59d0 msgr2=0x7f88940b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:08.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8893fff700 1 --2- 172.21.15.140:0/3488185677 >> v2:172.21.15.152:3300/0 conn(0x7f88940b59d0 0x7f88940b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8893fff700 1 -- 172.21.15.140:0/3488185677 >> v1:172.21.15.152:6789/0 conn(0x7f889405d0c0 legacy=0x7f88940ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8893fff700 1 -- 172.21.15.140:0/3488185677 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894146060 con 0x7f88940af920 2022-07-21T01:57:08.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8893fff700 1 --2- 172.21.15.140:0/3488185677 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940af920 0x7f88940ae3d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f887c002f70 tx=0x7f887c006eb0).ready entity=mon.0 client_cookie=44937891495459da server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:08.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f88937fe700 1 -- 172.21.15.140:0/3488185677 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f887c00d610 con 0x7f88940af920 2022-07-21T01:57:08.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f88937fe700 1 -- 172.21.15.140:0/3488185677 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c000e90 con 0x7f88940af920 2022-07-21T01:57:08.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f88937fe700 1 -- 172.21.15.140:0/3488185677 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f887c00d9a0 con 0x7f88940af920 2022-07-21T01:57:08.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 -- 172.21.15.140:0/3488185677 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940af920 msgr2=0x7f88940ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 --2- 172.21.15.140:0/3488185677 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940af920 0x7f88940ae3d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f887c002f70 tx=0x7f887c006eb0).stop 2022-07-21T01:57:08.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 -- 172.21.15.140:0/3488185677 shutdown_connections 2022-07-21T01:57:08.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 --2- 172.21.15.140:0/3488185677 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940af920 0x7f88940ae3d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 --2- 172.21.15.140:0/3488185677 >> v2:172.21.15.152:3300/0 conn(0x7f88940b59d0 0x7f88940b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 -- 172.21.15.140:0/3488185677 >> 172.21.15.140:0/3488185677 conn(0x7f88940634c0 msgr2=0x7f88940b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:08.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 -- 172.21.15.140:0/3488185677 shutdown_connections 2022-07-21T01:57:08.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 -- 172.21.15.140:0/3488185677 wait complete. 2022-07-21T01:57:08.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 Processor -- start 2022-07-21T01:57:08.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 -- start start 2022-07-21T01:57:08.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f889405d0c0 0x7f8894115520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:08.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88940af920 0x7f8894151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:08.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940b59d0 0x7f8894064970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:08.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8894115f00 con 0x7f88940b59d0 2022-07-21T01:57:08.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8894115a10 con 0x7f88940af920 2022-07-21T01:57:08.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899892700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8894115c60 con 0x7f889405d0c0 2022-07-21T01:57:08.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899091700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940b59d0 0x7f8894064970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:08.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8898890700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f889405d0c0 0x7f8894115520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:08.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899091700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940b59d0 0x7f8894064970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40096/0 (socket says 172.21.15.140:40096) 2022-07-21T01:57:08.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.204+0000 7f8899091700 1 -- 172.21.15.140:0/24922134 learned_addr learned my addr 172.21.15.140:0/24922134 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:08.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8893fff700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88940af920 0x7f8894151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:08.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8899091700 1 -- 172.21.15.140:0/24922134 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f889405d0c0 msgr2=0x7f8894115520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8899091700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f889405d0c0 0x7f8894115520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8899091700 1 -- 172.21.15.140:0/24922134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88940af920 msgr2=0x7f8894151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8899091700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88940af920 0x7f8894151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8899091700 1 -- 172.21.15.140:0/24922134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c00b1e0 con 0x7f88940b59d0 2022-07-21T01:57:08.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8899091700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940b59d0 0x7f8894064970 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f88940639a0 tx=0x7f888c007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:08.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8891ffb700 1 -- 172.21.15.140:0/24922134 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f888c00f470 con 0x7f88940b59d0 2022-07-21T01:57:08.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8891ffb700 1 -- 172.21.15.140:0/24922134 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c00f5d0 con 0x7f88940b59d0 2022-07-21T01:57:08.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8899892700 1 -- 172.21.15.140:0/24922134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8894155440 con 0x7f88940b59d0 2022-07-21T01:57:08.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8891ffb700 1 -- 172.21.15.140:0/24922134 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f888c00f930 con 0x7f88940b59d0 2022-07-21T01:57:08.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8899892700 1 -- 172.21.15.140:0/24922134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88940671a0 con 0x7f88940b59d0 2022-07-21T01:57:08.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.208+0000 7f8899892700 1 -- 172.21.15.140:0/24922134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88941555d0 con 0x7f88940b59d0 2022-07-21T01:57:08.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.212+0000 7f8891ffb700 1 -- 172.21.15.140:0/24922134 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f888c019030 con 0x7f88940b59d0 2022-07-21T01:57:08.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.212+0000 7f8891ffb700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88780539e0 0x7f8878055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:08.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.212+0000 7f8891ffb700 1 -- 172.21.15.140:0/24922134 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f888c06b1e0 con 0x7f88940b59d0 2022-07-21T01:57:08.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.212+0000 7f8898890700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88780539e0 0x7f8878055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:08.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.212+0000 7f8898890700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88780539e0 0x7f8878055ea0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f888400c000 tx=0x7f8884008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:08.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.216+0000 7f8891ffb700 1 -- 172.21.15.140:0/24922134 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f888c0423e0 con 0x7f88940b59d0 2022-07-21T01:57:08.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.344+0000 7f8899892700 1 -- 172.21.15.140:0/24922134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f88940af2d0 con 0x7f88940b59d0 2022-07-21T01:57:08.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.352+0000 7f8891ffb700 1 -- 172.21.15.140:0/24922134 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f888c017020 con 0x7f88940b59d0 2022-07-21T01:57:08.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 -- 172.21.15.140:0/24922134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88780539e0 msgr2=0x7f8878055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88780539e0 0x7f8878055ea0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f888400c000 tx=0x7f8884008040).stop 2022-07-21T01:57:08.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 -- 172.21.15.140:0/24922134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940b59d0 msgr2=0x7f8894064970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940b59d0 0x7f8894064970 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f88940639a0 tx=0x7f888c007f00).stop 2022-07-21T01:57:08.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 -- 172.21.15.140:0/24922134 shutdown_connections 2022-07-21T01:57:08.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88780539e0 0x7f8878055ea0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88940b59d0 0x7f8894064970 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88940af920 0x7f8894151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 --2- 172.21.15.140:0/24922134 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f889405d0c0 0x7f8894115520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 -- 172.21.15.140:0/24922134 >> 172.21.15.140:0/24922134 conn(0x7f88940634c0 msgr2=0x7f88940b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:08.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 -- 172.21.15.140:0/24922134 shutdown_connections 2022-07-21T01:57:08.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.356+0000 7f8899892700 1 -- 172.21.15.140:0/24922134 wait complete. 2022-07-21T01:57:08.370 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:57:08.370 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:08.390 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:08.412 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:08.434 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:08.456 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:08.477 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:57:08.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:08.478 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:08.478 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:08.486 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:57:08.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:08.488 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:57:08.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:08.489 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:08.490 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:08.503 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:57:08.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:08.506 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:57:08.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:08.508 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:08.509 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:57:08.517 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:57:08.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:08.520 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:57:08.521 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:57:08.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db8f65700 1 Processor -- start 2022-07-21T01:57:08.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db8f65700 1 -- start start 2022-07-21T01:57:08.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db8f65700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3db40aedc0 0x7f3db413f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:08.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db8f65700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db40acdb0 con 0x7f3db40b0080 2022-07-21T01:57:08.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db8f65700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db40b1450 con 0x7f3db40ac9d0 2022-07-21T01:57:08.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db8f65700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40af260 con 0x7f3db40aedc0 2022-07-21T01:57:08.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db2d9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f3db40b0080 0x7f3db413f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54250/0 (socket says 172.21.15.140:54250) 2022-07-21T01:57:08.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db2d9d700 1 -- 172.21.15.140:0/250592644 learned_addr learned my addr 172.21.15.140:0/250592644 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:08.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3da9d9b700 1 --2- 172.21.15.140:0/250592644 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3db40aedc0 0x7f3db413f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:08.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db1d9b700 1 -- 172.21.15.140:0/250592644 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085642463 0 0) 0x7f3db40acdb0 con 0x7f3db40b0080 2022-07-21T01:57:08.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db1d9b700 1 -- 172.21.15.140:0/250592644 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d94003260 con 0x7f3db40b0080 2022-07-21T01:57:08.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db1d9b700 1 -- 172.21.15.140:0/250592644 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282005170 0 0) 0x7f3db40b1450 con 0x7f3db40ac9d0 2022-07-21T01:57:08.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3da9d9b700 1 -- 172.21.15.140:0/250592644 >> v1:172.21.15.152:6789/0 conn(0x7f3db40ac9d0 legacy=0x7f3db413e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3da9d9b700 1 -- 172.21.15.140:0/250592644 >> v1:172.21.15.140:6789/0 conn(0x7f3db40b0080 legacy=0x7f3db413f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3da9d9b700 1 -- 172.21.15.140:0/250592644 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db40af450 con 0x7f3db40aedc0 2022-07-21T01:57:08.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3da9d9b700 1 --2- 172.21.15.140:0/250592644 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3db40aedc0 0x7f3db413f2a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f3d9c002f70 tx=0x7f3d9c006df0).ready entity=mon.2 client_cookie=5ceb8a8d63fddb50 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:08.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db1d9b700 1 -- 172.21.15.140:0/250592644 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d9c00d570 con 0x7f3db40aedc0 2022-07-21T01:57:08.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db1d9b700 1 -- 172.21.15.140:0/250592644 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c000e90 con 0x7f3db40aedc0 2022-07-21T01:57:08.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db1d9b700 1 -- 172.21.15.140:0/250592644 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d9c00d8c0 con 0x7f3db40aedc0 2022-07-21T01:57:08.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db8f65700 1 -- 172.21.15.140:0/250592644 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3db40aedc0 msgr2=0x7f3db413f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db8f65700 1 --2- 172.21.15.140:0/250592644 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3db40aedc0 0x7f3db413f2a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f3d9c002f70 tx=0x7f3d9c006df0).stop 2022-07-21T01:57:08.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db8f65700 1 -- 172.21.15.140:0/250592644 shutdown_connections 2022-07-21T01:57:08.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db8f65700 1 --2- 172.21.15.140:0/250592644 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3db40aedc0 0x7f3db413f2a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.632+0000 7f3db8f65700 1 -- 172.21.15.140:0/250592644 >> 172.21.15.140:0/250592644 conn(0x7f3db40611c0 msgr2=0x7f3db4062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:08.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.636+0000 7f3db8f65700 1 -- 172.21.15.140:0/250592644 shutdown_connections 2022-07-21T01:57:08.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.636+0000 7f3db8f65700 1 -- 172.21.15.140:0/250592644 wait complete. 2022-07-21T01:57:08.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.636+0000 7f3db8f65700 1 Processor -- start 2022-07-21T01:57:08.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.636+0000 7f3db8f65700 1 -- start start 2022-07-21T01:57:08.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3db8f65700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3db40aedc0 0x7f3db4137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:08.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3db8f65700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3db40b0080 0x7f3db40669b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:08.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3db8f65700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3db4135b20 0x7f3db4136060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:08.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3db8f65700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4130c20 con 0x7f3db40b0080 2022-07-21T01:57:08.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3db8f65700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4066ea0 con 0x7f3db4135b20 2022-07-21T01:57:08.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3db8f65700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db414d8c0 con 0x7f3db40aedc0 2022-07-21T01:57:08.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3da9d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3db40b0080 0x7f3db40669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:08.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3da9d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3db40b0080 0x7f3db40669b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40108/0 (socket says 172.21.15.140:40108) 2022-07-21T01:57:08.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3da9d9b700 1 -- 172.21.15.140:0/4290918473 learned_addr learned my addr 172.21.15.140:0/4290918473 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:08.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3db259c700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3db40aedc0 0x7f3db4137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:08.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3db2d9d700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3db4135b20 0x7f3db4136060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:08.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3da9d9b700 1 -- 172.21.15.140:0/4290918473 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3db40aedc0 msgr2=0x7f3db4137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3da9d9b700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3db40aedc0 0x7f3db4137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3da9d9b700 1 -- 172.21.15.140:0/4290918473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3db4135b20 msgr2=0x7f3db4136060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:08.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3da9d9b700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3db4135b20 0x7f3db4136060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:08.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3da9d9b700 1 -- 172.21.15.140:0/4290918473 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c00b180 con 0x7f3db40b0080 2022-07-21T01:57:08.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3da9d9b700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3db40b0080 0x7f3db40669b0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7f3d9c002f70 tx=0x7f3d9c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:08.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3dabfff700 1 -- 172.21.15.140:0/4290918473 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d9c016070 con 0x7f3db40b0080 2022-07-21T01:57:08.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3db8f65700 1 -- 172.21.15.140:0/4290918473 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db41301a0 con 0x7f3db40b0080 2022-07-21T01:57:08.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3db8f65700 1 -- 172.21.15.140:0/4290918473 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db41366f0 con 0x7f3db40b0080 2022-07-21T01:57:08.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3dabfff700 1 -- 172.21.15.140:0/4290918473 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c013700 con 0x7f3db40b0080 2022-07-21T01:57:08.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.640+0000 7f3dabfff700 1 -- 172.21.15.140:0/4290918473 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d9c013870 con 0x7f3db40b0080 2022-07-21T01:57:08.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.644+0000 7f3dabfff700 1 -- 172.21.15.140:0/4290918473 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3d9c00f030 con 0x7f3db40b0080 2022-07-21T01:57:08.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.644+0000 7f3dabfff700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d98053b00 0x7f3d98055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:08.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.644+0000 7f3db8f65700 1 -- 172.21.15.140:0/4290918473 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db40676b0 con 0x7f3db40b0080 2022-07-21T01:57:08.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.644+0000 7f3dabfff700 1 -- 172.21.15.140:0/4290918473 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f3d9c06c670 con 0x7f3db40b0080 2022-07-21T01:57:08.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.644+0000 7f3db259c700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d98053b00 0x7f3d98055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:08.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.644+0000 7f3db259c700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d98053b00 0x7f3d98055fc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f3da400d000 tx=0x7f3da40079b0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:08.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.648+0000 7f3dabfff700 1 -- 172.21.15.140:0/4290918473 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3d9c0472b0 con 0x7f3db40b0080 2022-07-21T01:57:08.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.768+0000 7f3db8f65700 1 -- 172.21.15.140:0/4290918473 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3db40574e0 con 0x7f3db40b0080 2022-07-21T01:57:08.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.768+0000 7f3dabfff700 1 -- 172.21.15.140:0/4290918473 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v127) v1 ==== 81+0+28933 (secure 0 0 0) 0x7f3d9c05b120 con 0x7f3db40b0080 2022-07-21T01:57:08.772 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:08.772 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:57:08.773 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 127, 2022-07-21T01:57:08.773 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:57:08.773 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:57:08.773 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:57:06.043420+0000", 2022-07-21T01:57:08.774 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:08.774 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:08.774 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:57:08.775 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:57:08.775 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:57:08.775 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:57:08.775 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:57:08.775 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:57:08.776 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:57:08.776 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:08.776 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:57:08.777 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:57:08.777 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:57:08.778 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:57:08.778 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:57:08.778 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 24, 2022-07-21T01:57:08.779 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:57:08.779 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:57:08.779 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:57:08.779 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:57:08.780 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:57:08.780 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.780 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:57:08.781 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:57:08.781 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:57:08.781 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:08.781 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:08.782 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:08.782 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:08.782 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:08.782 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:08.783 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:08.783 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:08.783 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:57:08.783 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:57:08.783 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:57:08.784 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:57:08.784 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:57:08.784 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:08.784 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:08.785 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:08.785 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:08.785 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:08.785 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:08.785 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:08.786 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.786 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:57:08.786 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:08.787 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:08.787 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:08.787 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:08.787 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:08.788 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:08.788 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:08.788 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:08.788 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:08.789 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:08.789 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:08.789 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:08.790 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:08.790 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:08.791 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:08.791 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:08.791 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:08.792 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:08.792 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:08.792 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:08.792 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:08.793 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:08.793 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:08.793 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:08.793 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:08.793 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:08.794 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.794 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:08.794 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:08.794 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:08.795 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:08.795 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:08.795 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:08.795 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:08.796 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:08.796 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:08.796 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:08.797 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:08.797 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:57:08.797 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:57:08.797 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.797 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:08.798 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:57:08.798 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.798 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.798 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.799 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:57:08.799 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:57:08.799 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:57:08.800 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:08.800 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:08.801 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:08.801 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:08.802 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:08.802 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:08.803 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:08.803 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:08.804 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:08.804 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:08.805 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:08.805 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:08.806 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:08.807 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:08.807 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:08.808 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:08.808 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:08.809 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:08.809 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:08.810 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:08.810 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.811 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:57:08.812 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:08.813 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:08.813 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:08.814 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:08.814 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:08.815 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:08.815 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:08.816 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:08.817 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:08.817 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:08.818 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:08.818 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:08.819 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:08.819 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:08.820 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:08.820 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:08.821 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:08.821 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:08.822 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:08.822 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:08.823 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:08.824 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:08.824 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:08.825 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:08.825 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:08.826 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:08.826 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.827 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:08.827 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:08.828 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:08.829 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:08.829 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:08.830 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:08.830 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:08.831 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:08.831 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:08.832 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:08.832 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:08.833 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:57:08.833 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:08.834 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:57:08.834 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.835 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.835 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:08.835 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:57:08.836 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.836 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:57:08.836 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:57:08.836 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:08.837 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:08.837 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:08.837 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:08.837 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:08.838 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:08.838 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:08.838 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:08.838 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:08.839 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:08.839 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:08.839 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.839 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.839 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.840 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:57:08.840 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:08.840 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.840 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.841 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.841 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:57:08.841 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:08.841 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.841 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.842 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.842 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:08.842 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.842 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.843 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.843 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:57:08.843 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:08.843 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.844 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.844 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.844 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:57:08.844 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:08.844 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.845 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.845 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.846 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:08.846 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.846 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.847 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.847 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:57:08.848 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:08.848 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.848 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.849 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.849 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:57:08.850 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:08.850 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.851 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.851 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.852 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:08.852 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.852 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.853 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.853 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:57:08.854 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:08.854 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.855 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.855 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.856 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:57:08.856 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:08.856 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.857 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.857 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.858 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:57:08.858 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:57:08.859 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:57:08.859 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:57:08.860 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:08.860 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:08.860 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:08.861 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.861 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.862 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.862 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:57:08.863 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:57:08.863 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:08.864 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:08.864 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:08.865 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:08.865 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:08.865 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:08.866 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:08.866 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:08.867 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:08.867 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:08.868 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:08.868 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.869 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.869 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.870 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:57:08.870 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:08.870 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.871 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.871 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.872 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:57:08.872 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:08.873 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.873 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.873 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.874 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:08.874 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.874 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.874 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.875 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:57:08.875 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:08.875 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.875 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.876 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.876 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:57:08.876 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:08.876 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.876 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.877 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.877 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:08.877 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.877 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.878 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.878 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:57:08.878 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:08.878 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.879 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.879 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.879 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:57:08.879 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:08.879 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.880 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.880 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.880 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:08.880 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.881 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.881 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.881 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:57:08.881 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:08.881 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.882 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.882 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.882 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:57:08.882 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:08.883 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.883 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.883 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.883 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:57:08.884 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:57:08.884 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:57:08.884 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:57:08.884 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:08.885 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:08.885 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:08.885 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.885 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.885 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.886 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:57:08.886 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:57:08.886 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:08.886 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:08.887 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:08.887 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:08.887 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:08.887 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:08.887 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:08.888 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 117, 2022-07-21T01:57:08.888 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:08.888 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:08.888 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:08.889 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.889 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.889 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.889 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:57:08.890 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:08.890 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.890 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.890 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.890 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:57:08.891 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:08.891 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.891 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.891 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.892 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:08.892 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.892 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.892 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.892 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:57:08.893 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:08.893 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.893 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.893 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.894 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:57:08.894 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:08.894 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.894 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.895 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.895 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:08.895 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.895 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.895 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.896 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:57:08.896 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:08.896 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.896 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.897 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.897 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:57:08.897 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:08.897 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.897 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.898 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.898 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:08.898 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.898 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.899 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.899 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:57:08.899 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:08.899 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.900 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.900 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.900 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:57:08.900 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:08.901 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.901 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.901 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.901 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:57:08.902 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:57:08.902 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:57:08.902 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:57:08.902 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:08.902 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:08.903 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:08.903 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.903 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.903 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.904 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:57:08.904 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:57:08.904 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:08.904 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:08.905 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:08.905 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:08.905 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:08.905 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:08.905 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:08.906 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:08.906 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:08.906 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:08.906 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:08.907 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.907 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.907 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.907 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:57:08.908 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:08.908 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.908 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.908 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.908 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:57:08.909 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:08.909 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.909 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.909 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.910 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:08.910 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.910 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.910 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.910 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:57:08.911 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:08.911 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.911 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.911 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.912 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:57:08.912 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:08.912 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.912 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.913 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.913 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:08.913 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.913 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.913 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.914 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:57:08.914 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:08.914 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.914 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.915 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.915 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:57:08.915 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:08.915 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.915 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.916 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.916 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:08.916 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.916 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.917 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.917 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:57:08.917 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:08.917 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.917 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.918 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.918 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:57:08.918 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:08.918 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.919 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.919 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.919 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:57:08.919 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:57:08.920 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:57:08.920 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:57:08.920 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:08.920 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:08.920 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:08.921 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.921 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.921 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.921 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:57:08.922 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:57:08.922 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:08.922 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:08.922 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:08.922 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:08.923 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:08.923 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:08.923 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:08.923 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 117, 2022-07-21T01:57:08.924 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:08.924 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:08.924 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:08.924 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.925 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.925 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.925 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:57:08.925 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:08.925 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.926 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.926 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.926 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:57:08.926 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:08.927 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.927 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.927 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.927 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:08.928 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.928 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.928 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.928 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:57:08.928 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:08.929 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.929 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.929 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.929 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:57:08.930 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:08.930 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.930 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.930 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.930 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:08.931 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.931 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.931 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.931 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:57:08.932 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:08.932 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.932 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.932 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.933 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:57:08.933 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:08.933 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.933 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.933 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.934 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:08.934 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.934 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.934 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.935 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:57:08.935 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:08.935 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.935 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.936 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.936 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:57:08.936 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:08.936 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.937 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.937 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.937 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:57:08.937 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:57:08.937 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:57:08.938 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:57:08.938 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:08.938 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:08.938 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:08.939 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.939 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.939 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.939 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:57:08.939 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:57:08.940 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:08.940 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:08.940 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:08.940 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:08.941 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:08.941 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:08.941 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:08.941 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:08.942 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:08.942 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:08.942 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:08.942 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.943 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.943 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.943 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:57:08.943 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:08.943 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.944 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.944 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.944 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:57:08.944 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:08.945 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.945 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.945 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.945 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:08.945 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.946 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.946 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.946 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:57:08.946 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:08.947 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.947 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.947 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.947 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:57:08.948 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:08.948 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.948 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.948 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.949 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:08.949 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.949 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.949 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.949 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:57:08.950 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:08.950 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.950 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.950 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.951 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:57:08.951 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:08.951 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.951 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.951 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.952 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:08.952 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.952 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.952 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.953 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:57:08.953 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:08.953 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.953 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.954 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.954 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:57:08.954 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:08.954 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.955 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.955 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.955 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:57:08.955 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:57:08.955 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:57:08.956 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:57:08.956 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:08.956 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:08.956 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:08.957 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.957 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.957 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.957 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:57:08.957 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:57:08.958 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:08.958 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:08.958 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:08.958 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:08.959 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:08.959 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:08.959 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:08.959 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:08.960 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:08.960 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:08.960 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:08.960 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.960 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.961 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.961 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:57:08.961 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:08.961 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.962 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.962 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.962 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:57:08.962 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:08.962 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.963 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.963 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.963 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:08.963 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.964 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.964 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.964 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:57:08.964 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:08.965 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.965 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.965 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.965 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:57:08.965 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:08.966 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.966 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.966 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.966 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:08.967 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.967 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.967 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.967 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:57:08.968 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:08.968 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.968 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.968 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.968 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:57:08.969 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:08.969 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.969 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.969 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.970 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:08.970 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.970 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.970 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.970 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:57:08.971 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:08.971 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.971 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.971 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.972 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:57:08.972 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:08.972 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.972 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.973 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.973 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:57:08.973 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:57:08.973 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:57:08.973 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:57:08.974 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:08.974 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:08.974 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:08.974 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.975 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.975 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.975 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:57:08.975 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:57:08.976 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:08.976 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:08.976 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:08.976 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:08.976 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:08.977 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:08.977 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:08.977 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 123, 2022-07-21T01:57:08.977 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:08.978 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:08.978 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:08.978 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.978 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.979 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.979 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:57:08.979 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:08.979 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.980 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.980 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.980 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:57:08.980 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:08.980 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.981 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.981 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.981 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:08.981 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.982 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.982 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.982 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:57:08.982 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:08.983 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.983 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.983 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.983 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:57:08.983 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:08.984 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.984 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.984 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.984 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:08.985 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.985 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.985 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.985 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:57:08.985 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:08.986 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.986 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.986 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.986 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:57:08.987 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:08.987 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.987 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.987 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.988 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:08.988 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:08.988 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.988 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:08.989 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:57:08.989 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:08.989 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.989 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.989 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:08.990 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:57:08.990 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:08.990 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.990 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.991 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.991 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:57:08.991 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:57:08.991 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:57:08.992 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:57:08.992 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:08.992 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:08.992 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:08.992 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:08.993 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:08.993 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:08.993 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:57:08.993 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.994 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:57:08.994 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:08.994 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:08.994 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:08.994 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:08.995 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:08.995 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:57:08.995 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:08.995 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.996 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.996 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:57:08.996 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:08.996 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:08.996 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:08.997 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:08.997 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:08.997 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:57:08.997 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:08.998 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:08.998 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:08.998 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:57:08.998 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:08.998 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:08.999 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:08.999 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:08.999 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:08.999 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:57:09.000 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:09.000 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:09.000 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:09.000 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:57:09.001 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:09.001 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:09.001 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:09.001 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:09.002 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:09.002 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:57:09.002 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:09.002 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:09.002 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:09.003 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:57:09.003 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:09.003 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:09.003 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:09.004 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:09.004 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:09.004 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:57:09.004 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:09.004 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:09.005 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:09.005 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:57:09.005 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:09.005 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:09.006 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:09.006 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:09.006 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:09.006 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:57:09.007 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:09.007 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:09.007 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:09.007 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:57:09.007 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:09.008 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:09.008 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:09.008 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:09.008 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:09.009 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:57:09.009 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:09.009 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:09.009 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:09.010 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:57:09.010 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:09.010 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:09.010 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:09.010 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:09.011 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:09.011 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:57:09.011 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:09.011 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:09.012 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:09.012 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:57:09.012 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:57:09.012 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:57:09.012 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:57:09.013 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:57:09.013 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6833/456689269": "2022-07-22T01:56:46.358661+0000", 2022-07-21T01:57:09.013 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.152:6832/456689269": "2022-07-22T01:56:46.358661+0000" 2022-07-21T01:57:09.013 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:09.014 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:57:09.014 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:57:09.014 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:57:09.014 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:57:09.014 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:57:09.015 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:57:09.015 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:57:09.015 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:57:09.015 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:09.016 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:09.016 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:57:09.016 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:57:09.016 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:57:09.017 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:57:09.017 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:57:09.017 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:57:09.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 -- 172.21.15.140:0/4290918473 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d98053b00 msgr2=0x7f3d98055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d98053b00 0x7f3d98055fc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f3da400d000 tx=0x7f3da40079b0).stop 2022-07-21T01:57:09.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 -- 172.21.15.140:0/4290918473 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3db40b0080 msgr2=0x7f3db40669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3db40b0080 0x7f3db40669b0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7f3d9c002f70 tx=0x7f3d9c014000).stop 2022-07-21T01:57:09.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 -- 172.21.15.140:0/4290918473 shutdown_connections 2022-07-21T01:57:09.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3d98053b00 0x7f3d98055fc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3db4135b20 0x7f3db4136060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3db40b0080 0x7f3db40669b0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 --2- 172.21.15.140:0/4290918473 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3db40aedc0 0x7f3db4137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 -- 172.21.15.140:0/4290918473 >> 172.21.15.140:0/4290918473 conn(0x7f3db40ac9d0 msgr2=0x7f3db413e550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:09.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 -- 172.21.15.140:0/4290918473 shutdown_connections 2022-07-21T01:57:09.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:08.772+0000 7f3db8f65700 1 -- 172.21.15.140:0/4290918473 wait complete. 2022-07-21T01:57:09.023 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:09.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.132+0000 7f343159f700 1 Processor -- start 2022-07-21T01:57:09.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.132+0000 7f343159f700 1 -- start start 2022-07-21T01:57:09.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.132+0000 7f343159f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f342c0aff80 0x7f342c140240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.132+0000 7f343159f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0b5d70 con 0x7f342c0aecf0 2022-07-21T01:57:09.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.132+0000 7f343159f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0651c0 con 0x7f342c0b5990 2022-07-21T01:57:09.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.132+0000 7f343159f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c0b0690 con 0x7f342c0aff80 2022-07-21T01:57:09.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.132+0000 7f342b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f342c0aecf0 0x7f342c05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54260/0 (socket says 172.21.15.140:54260) 2022-07-21T01:57:09.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.132+0000 7f342b7fe700 1 -- 172.21.15.140:0/1117732606 learned_addr learned my addr 172.21.15.140:0/1117732606 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:09.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.132+0000 7f3430d9e700 1 --2- 172.21.15.140:0/1117732606 >> v2:172.21.15.152:3300/0 conn(0x7f342c0aff80 0x7f342c140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.132+0000 7f342affd700 1 -- 172.21.15.140:0/1117732606 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433344322 0 0) 0x7f342c0b5d70 con 0x7f342c0aecf0 2022-07-21T01:57:09.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.132+0000 7f342affd700 1 -- 172.21.15.140:0/1117732606 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3410003260 con 0x7f342c0aecf0 2022-07-21T01:57:09.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f3430d9e700 1 -- 172.21.15.140:0/1117732606 >> v1:172.21.15.152:6789/0 conn(0x7f342c0b5990 legacy=0x7f342c13fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f342affd700 1 -- 172.21.15.140:0/1117732606 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297215481 0 0) 0x7f342c0651c0 con 0x7f342c0b5990 2022-07-21T01:57:09.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f3430d9e700 1 -- 172.21.15.140:0/1117732606 >> v1:172.21.15.140:6789/0 conn(0x7f342c0aecf0 legacy=0x7f342c05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f3430d9e700 1 -- 172.21.15.140:0/1117732606 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0b0930 con 0x7f342c0aff80 2022-07-21T01:57:09.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f3430d9e700 1 --2- 172.21.15.140:0/1117732606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342c0aff80 0x7f342c140240 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f34240032f0 tx=0x7f34240091d0).ready entity=mon.1 client_cookie=70917d3e7d26eb47 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:09.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f342affd700 1 -- 172.21.15.140:0/1117732606 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f342400b6c0 con 0x7f342c0aff80 2022-07-21T01:57:09.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f342affd700 1 -- 172.21.15.140:0/1117732606 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342400b820 con 0x7f342c0aff80 2022-07-21T01:57:09.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f342affd700 1 -- 172.21.15.140:0/1117732606 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f342400b990 con 0x7f342c0aff80 2022-07-21T01:57:09.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f343159f700 1 -- 172.21.15.140:0/1117732606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342c0aff80 msgr2=0x7f342c140240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f343159f700 1 --2- 172.21.15.140:0/1117732606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342c0aff80 0x7f342c140240 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f34240032f0 tx=0x7f34240091d0).stop 2022-07-21T01:57:09.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f343159f700 1 -- 172.21.15.140:0/1117732606 shutdown_connections 2022-07-21T01:57:09.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f343159f700 1 --2- 172.21.15.140:0/1117732606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342c0aff80 0x7f342c140240 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f343159f700 1 -- 172.21.15.140:0/1117732606 >> 172.21.15.140:0/1117732606 conn(0x7f342c0634e0 msgr2=0x7f342c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:09.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f343159f700 1 -- 172.21.15.140:0/1117732606 shutdown_connections 2022-07-21T01:57:09.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f343159f700 1 -- 172.21.15.140:0/1117732606 wait complete. 2022-07-21T01:57:09.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f343159f700 1 Processor -- start 2022-07-21T01:57:09.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.136+0000 7f343159f700 1 -- start start 2022-07-21T01:57:09.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f343159f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f342c0aecf0 0x7f342c1422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f343159f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342c0aff80 0x7f342c13cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f343159f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f342c0b5990 0x7f342c13d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f343159f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0662d0 con 0x7f342c0aecf0 2022-07-21T01:57:09.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f343159f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0667b0 con 0x7f342c0aff80 2022-07-21T01:57:09.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f343159f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c10f350 con 0x7f342c0b5990 2022-07-21T01:57:09.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f342bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f342c0aecf0 0x7f342c1422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f342bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f342c0aecf0 0x7f342c1422a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40120/0 (socket says 172.21.15.140:40120) 2022-07-21T01:57:09.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f342bfff700 1 -- 172.21.15.140:0/2328035990 learned_addr learned my addr 172.21.15.140:0/2328035990 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:09.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f342b7fe700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342c0aff80 0x7f342c13cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f3430d9e700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f342c0b5990 0x7f342c13d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f342bfff700 1 -- 172.21.15.140:0/2328035990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f342c0b5990 msgr2=0x7f342c13d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f342bfff700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f342c0b5990 0x7f342c13d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f342bfff700 1 -- 172.21.15.140:0/2328035990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342c0aff80 msgr2=0x7f342c13cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f342bfff700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342c0aff80 0x7f342c13cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f342bfff700 1 -- 172.21.15.140:0/2328035990 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424009470 con 0x7f342c0aecf0 2022-07-21T01:57:09.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f3430d9e700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f342c0b5990 0x7f342c13d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:09.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f342b7fe700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342c0aff80 0x7f342c13cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:09.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f342bfff700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f342c0aecf0 0x7f342c1422a0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7f341c000ed0 tx=0x7f341c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:09.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f34297fa700 1 -- 172.21.15.140:0/2328035990 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f341c00ce80 con 0x7f342c0aecf0 2022-07-21T01:57:09.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f34297fa700 1 -- 172.21.15.140:0/2328035990 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c00f470 con 0x7f342c0aecf0 2022-07-21T01:57:09.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f343159f700 1 -- 172.21.15.140:0/2328035990 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c142790 con 0x7f342c0aecf0 2022-07-21T01:57:09.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f34297fa700 1 -- 172.21.15.140:0/2328035990 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f341c00f7d0 con 0x7f342c0aecf0 2022-07-21T01:57:09.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f343159f700 1 -- 172.21.15.140:0/2328035990 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c10f5f0 con 0x7f342c0aecf0 2022-07-21T01:57:09.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.140+0000 7f343159f700 1 -- 172.21.15.140:0/2328035990 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c0574e0 con 0x7f342c0aecf0 2022-07-21T01:57:09.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.144+0000 7f34297fa700 1 -- 172.21.15.140:0/2328035990 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f341c019030 con 0x7f342c0aecf0 2022-07-21T01:57:09.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.144+0000 7f34297fa700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3404053b00 0x7f3404055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.144+0000 7f34297fa700 1 -- 172.21.15.140:0/2328035990 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f341c06b190 con 0x7f342c0aecf0 2022-07-21T01:57:09.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.144+0000 7f342b7fe700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3404053b00 0x7f3404055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.144+0000 7f342b7fe700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3404053b00 0x7f3404055fc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f341400b000 tx=0x7f3414009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:09.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.148+0000 7f34297fa700 1 -- 172.21.15.140:0/2328035990 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f341c068e90 con 0x7f342c0aecf0 2022-07-21T01:57:09.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.308+0000 7f343159f700 1 -- 172.21.15.140:0/2328035990 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f342c141190 con 0x7f342c0aecf0 2022-07-21T01:57:09.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f34297fa700 1 -- 172.21.15.140:0/2328035990 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+504 (secure 0 0 0) 0x7f341c042280 con 0x7f342c0aecf0 2022-07-21T01:57:09.313 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:09.313 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2022-07-21T01:57:09.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 -- 172.21.15.140:0/2328035990 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3404053b00 msgr2=0x7f3404055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3404053b00 0x7f3404055fc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f341400b000 tx=0x7f3414009040).stop 2022-07-21T01:57:09.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 -- 172.21.15.140:0/2328035990 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f342c0aecf0 msgr2=0x7f342c1422a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f342c0aecf0 0x7f342c1422a0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7f341c000ed0 tx=0x7f341c007c50).stop 2022-07-21T01:57:09.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 -- 172.21.15.140:0/2328035990 shutdown_connections 2022-07-21T01:57:09.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3404053b00 0x7f3404055fc0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f342c0b5990 0x7f342c13d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f342c0aff80 0x7f342c13cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 --2- 172.21.15.140:0/2328035990 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f342c0aecf0 0x7f342c1422a0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 -- 172.21.15.140:0/2328035990 >> 172.21.15.140:0/2328035990 conn(0x7f342c0634e0 msgr2=0x7f342c0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:09.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 -- 172.21.15.140:0/2328035990 shutdown_connections 2022-07-21T01:57:09.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.312+0000 7f343159f700 1 -- 172.21.15.140:0/2328035990 wait complete. 2022-07-21T01:57:09.322 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 60 2022-07-21T01:57:09.332 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:09.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.432+0000 7f44689ef700 1 Processor -- start 2022-07-21T01:57:09.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.432+0000 7f44689ef700 1 -- start start 2022-07-21T01:57:09.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f44689ef700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44640b61d0 0x7f44640adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f44689ef700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44640b1d10 con 0x7f44640b08c0 2022-07-21T01:57:09.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f44689ef700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44640b65b0 con 0x7f4464060d80 2022-07-21T01:57:09.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f44689ef700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44640ad480 con 0x7f44640b61d0 2022-07-21T01:57:09.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f445bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f44640b08c0 0x7f4464145400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54274/0 (socket says 172.21.15.140:54274) 2022-07-21T01:57:09.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f445bfff700 1 -- 172.21.15.140:0/3526597748 learned_addr learned my addr 172.21.15.140:0/3526597748 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:09.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f4462d9d700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44640b61d0 0x7f44640adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965843150 0 0) 0x7f44640b1d10 con 0x7f44640b08c0 2022-07-21T01:57:09.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4444003260 con 0x7f44640b08c0 2022-07-21T01:57:09.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276180772 0 0) 0x7f44640b65b0 con 0x7f4464060d80 2022-07-21T01:57:09.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44640b1d10 con 0x7f4464060d80 2022-07-21T01:57:09.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 769735522 0 0) 0x7f4444003260 con 0x7f44640b08c0 2022-07-21T01:57:09.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44640b61d0 msgr2=0x7f44640adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:09.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44640b61d0 0x7f44640adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 >> v1:172.21.15.152:6789/0 conn(0x7f4464060d80 legacy=0x7f446414d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44640b1f70 con 0x7f44640b08c0 2022-07-21T01:57:09.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f444c001660 con 0x7f44640b08c0 2022-07-21T01:57:09.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 >> v1:172.21.15.140:6789/0 conn(0x7f44640b08c0 legacy=0x7f4464145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44640b61d0 0x7f4444003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4444003e20 0x7f4444004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 0x7f4444004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44640ad480 con 0x7f44440049f0 2022-07-21T01:57:09.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444400b800 con 0x7f4444003e20 2022-07-21T01:57:09.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444400b9c0 con 0x7f44640b61d0 2022-07-21T01:57:09.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446359e700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 0x7f4444004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f4462d9d700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44640b61d0 0x7f4444003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f445bfff700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4444003e20 0x7f4444004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446359e700 1 -- 172.21.15.140:0/3526597748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44640b61d0 msgr2=0x7f4444003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:09.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446359e700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f44640b61d0 0x7f4444003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446359e700 1 -- 172.21.15.140:0/3526597748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4444003e20 msgr2=0x7f4444004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:09.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446359e700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4444003e20 0x7f4444004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446359e700 1 -- 172.21.15.140:0/3526597748 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4444003260 con 0x7f44440049f0 2022-07-21T01:57:09.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446359e700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 0x7f4444004e70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f445c001490 tx=0x7f445c007d80).ready entity=mon.0 client_cookie=66e82ee48f1e2d5a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:09.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f445c001780 con 0x7f44440049f0 2022-07-21T01:57:09.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c006890 con 0x7f44440049f0 2022-07-21T01:57:09.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446259c700 1 -- 172.21.15.140:0/3526597748 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f445c006bf0 con 0x7f44440049f0 2022-07-21T01:57:09.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f44689ef700 1 -- 172.21.15.140:0/3526597748 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 msgr2=0x7f4444004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f44689ef700 1 --2- 172.21.15.140:0/3526597748 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 0x7f4444004e70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f445c001490 tx=0x7f445c007d80).stop 2022-07-21T01:57:09.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.436+0000 7f446359e700 1 -- 172.21.15.140:0/3526597748 reap_dead start 2022-07-21T01:57:09.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 -- 172.21.15.140:0/3526597748 shutdown_connections 2022-07-21T01:57:09.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 -- 172.21.15.140:0/3526597748 >> 172.21.15.140:0/3526597748 conn(0x7f4464063cf0 msgr2=0x7f44640b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:09.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 -- 172.21.15.140:0/3526597748 shutdown_connections 2022-07-21T01:57:09.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 -- 172.21.15.140:0/3526597748 wait complete. 2022-07-21T01:57:09.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 Processor -- start 2022-07-21T01:57:09.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 -- start start 2022-07-21T01:57:09.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 0x7f446413ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4464144700 0x7f4464155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4464152010 0x7f4464152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4464156190 con 0x7f44440049f0 2022-07-21T01:57:09.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446413f360 con 0x7f4464152010 2022-07-21T01:57:09.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f44689ef700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446413f600 con 0x7f4464144700 2022-07-21T01:57:09.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f4462d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 0x7f446413ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f4462d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 0x7f446413ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40140/0 (socket says 172.21.15.140:40140) 2022-07-21T01:57:09.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f4462d9d700 1 -- 172.21.15.140:0/2927025872 learned_addr learned my addr 172.21.15.140:0/2927025872 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:09.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f445bfff700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4464144700 0x7f4464155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.440+0000 7f446359e700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4464152010 0x7f4464152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4462d9d700 1 -- 172.21.15.140:0/2927025872 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4464144700 msgr2=0x7f4464155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4462d9d700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4464144700 0x7f4464155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4462d9d700 1 -- 172.21.15.140:0/2927025872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4464152010 msgr2=0x7f4464152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4462d9d700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4464152010 0x7f4464152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4462d9d700 1 -- 172.21.15.140:0/2927025872 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446413f8a0 con 0x7f44440049f0 2022-07-21T01:57:09.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4462d9d700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 0x7f446413ea40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f445400b000 tx=0x7f4454006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:09.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4460d99700 1 -- 172.21.15.140:0/2927025872 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f445400f760 con 0x7f44440049f0 2022-07-21T01:57:09.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4460d99700 1 -- 172.21.15.140:0/2927025872 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445400f8c0 con 0x7f44440049f0 2022-07-21T01:57:09.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f44689ef700 1 -- 172.21.15.140:0/2927025872 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446413fc10 con 0x7f44440049f0 2022-07-21T01:57:09.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4460d99700 1 -- 172.21.15.140:0/2927025872 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f445400fc20 con 0x7f44440049f0 2022-07-21T01:57:09.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f44689ef700 1 -- 172.21.15.140:0/2927025872 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4464152b20 con 0x7f44440049f0 2022-07-21T01:57:09.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f44689ef700 1 -- 172.21.15.140:0/2927025872 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44640b01a0 con 0x7f44440049f0 2022-07-21T01:57:09.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4460d99700 1 -- 172.21.15.140:0/2927025872 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4454014440 con 0x7f44440049f0 2022-07-21T01:57:09.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4460d99700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44480539e0 0x7f4448055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.444+0000 7f4460d99700 1 -- 172.21.15.140:0/2927025872 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f445406c5e0 con 0x7f44440049f0 2022-07-21T01:57:09.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.448+0000 7f445bfff700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44480539e0 0x7f4448055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.448+0000 7f445bfff700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44480539e0 0x7f4448055ea0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f444c005000 tx=0x7f44640addc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:09.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.452+0000 7f4460d99700 1 -- 172.21.15.140:0/2927025872 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4454043780 con 0x7f44440049f0 2022-07-21T01:57:09.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.616+0000 7f44689ef700 1 -- 172.21.15.140:0/2927025872 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44640574e0 con 0x7f44440049f0 2022-07-21T01:57:09.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f4460d99700 1 -- 172.21.15.140:0/2927025872 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+504 (secure 0 0 0) 0x7f4454018020 con 0x7f44440049f0 2022-07-21T01:57:09.622 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:09.622 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2022-07-21T01:57:09.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 -- 172.21.15.140:0/2927025872 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44480539e0 msgr2=0x7f4448055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44480539e0 0x7f4448055ea0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f444c005000 tx=0x7f44640addc0).stop 2022-07-21T01:57:09.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 -- 172.21.15.140:0/2927025872 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 msgr2=0x7f446413ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 0x7f446413ea40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f445400b000 tx=0x7f4454006df0).stop 2022-07-21T01:57:09.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 -- 172.21.15.140:0/2927025872 shutdown_connections 2022-07-21T01:57:09.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f44480539e0 0x7f4448055ea0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4464152010 0x7f4464152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4464144700 0x7f4464155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 --2- 172.21.15.140:0/2927025872 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f44440049f0 0x7f446413ea40 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 -- 172.21.15.140:0/2927025872 >> 172.21.15.140:0/2927025872 conn(0x7f4464063cf0 msgr2=0x7f4464148b30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:09.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 -- 172.21.15.140:0/2927025872 shutdown_connections 2022-07-21T01:57:09.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.620+0000 7f44689ef700 1 -- 172.21.15.140:0/2927025872 wait complete. 2022-07-21T01:57:09.630 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 60 2022-07-21T01:57:09.638 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:57:09.639 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:57:09.639 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:57:09.639 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:57:09.642 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:57:09.643 DEBUG:teuthology.parallel:result is None 2022-07-21T01:57:09.643 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:57:09.644 DEBUG:teuthology.parallel:result is None 2022-07-21T01:57:09.644 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:57:09.721 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:57:09.728 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:57:09.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.740+0000 7f3f2495b700 1 Processor -- start 2022-07-21T01:57:09.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.740+0000 7f3f2495b700 1 -- start start 2022-07-21T01:57:09.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3f200aec90 0x7f3f200acae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f200affa0 0x7f3f2013f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f200b0ff0 con 0x7f3f2013e430 2022-07-21T01:57:09.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f200af070 con 0x7f3f200aec90 2022-07-21T01:57:09.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200b1250 con 0x7f3f200affa0 2022-07-21T01:57:09.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3f200aec90 0x7f3f200acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3f200aec90 0x7f3f200acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40160/0 (socket says 172.21.15.140:40160) 2022-07-21T01:57:09.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1f7fe700 1 -- 172.21.15.140:0/3702802593 learned_addr learned my addr 172.21.15.140:0/3702802593 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:09.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1ffff700 1 --1- 172.21.15.140:0/3702802593 >> v1:172.21.15.140:6789/0 conn(0x7f3f2013e430 0x7f3f20142460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54310/0 (socket says 172.21.15.140:54310) 2022-07-21T01:57:09.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1effd700 1 --2- 172.21.15.140:0/3702802593 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f200affa0 0x7f3f2013f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1e7fc700 1 -- 172.21.15.140:0/3702802593 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586829709 0 0) 0x7f3f200b0ff0 con 0x7f3f2013e430 2022-07-21T01:57:09.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1f7fe700 1 -- 172.21.15.140:0/3702802593 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f200affa0 msgr2=0x7f3f2013f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:09.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1f7fe700 1 --2- 172.21.15.140:0/3702802593 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f200affa0 0x7f3f2013f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1f7fe700 1 -- 172.21.15.140:0/3702802593 >> v1:172.21.15.140:6789/0 conn(0x7f3f2013e430 legacy=0x7f3f20142460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1f7fe700 1 -- 172.21.15.140:0/3702802593 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f200af2e0 con 0x7f3f200aec90 2022-07-21T01:57:09.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1f7fe700 1 --2- 172.21.15.140:0/3702802593 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f200aec90 0x7f3f200acae0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f3f20062890 tx=0x7f3f14006e50).ready entity=mon.0 client_cookie=960aa910539ab5b8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:09.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1e7fc700 1 -- 172.21.15.140:0/3702802593 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f1400f9c0 con 0x7f3f200aec90 2022-07-21T01:57:09.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1e7fc700 1 -- 172.21.15.140:0/3702802593 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f14000e90 con 0x7f3f200aec90 2022-07-21T01:57:09.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f1e7fc700 1 -- 172.21.15.140:0/3702802593 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f1400fd10 con 0x7f3f200aec90 2022-07-21T01:57:09.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 -- 172.21.15.140:0/3702802593 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f200aec90 msgr2=0x7f3f200acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 --2- 172.21.15.140:0/3702802593 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f200aec90 0x7f3f200acae0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7f3f20062890 tx=0x7f3f14006e50).stop 2022-07-21T01:57:09.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 -- 172.21.15.140:0/3702802593 shutdown_connections 2022-07-21T01:57:09.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 --2- 172.21.15.140:0/3702802593 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f200affa0 0x7f3f2013f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 --2- 172.21.15.140:0/3702802593 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f200aec90 0x7f3f200acae0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 -- 172.21.15.140:0/3702802593 >> 172.21.15.140:0/3702802593 conn(0x7f3f20063480 msgr2=0x7f3f200b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:09.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 -- 172.21.15.140:0/3702802593 shutdown_connections 2022-07-21T01:57:09.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 -- 172.21.15.140:0/3702802593 wait complete. 2022-07-21T01:57:09.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 Processor -- start 2022-07-21T01:57:09.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.744+0000 7f3f2495b700 1 -- start start 2022-07-21T01:57:09.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f2495b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f200aec90 0x7f3f20065e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f2495b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f200affa0 0x7f3f20139a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f2495b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f2013e430 0x7f3f201449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f2495b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f201431b0 con 0x7f3f200aec90 2022-07-21T01:57:09.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f2495b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f20139f70 con 0x7f3f200affa0 2022-07-21T01:57:09.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f2495b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f20140840 con 0x7f3f2013e430 2022-07-21T01:57:09.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f200aec90 0x7f3f20065e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f200aec90 0x7f3f20065e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40166/0 (socket says 172.21.15.140:40166) 2022-07-21T01:57:09.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1f7fe700 1 -- 172.21.15.140:0/4206111672 learned_addr learned my addr 172.21.15.140:0/4206111672 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:09.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1ffff700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f2013e430 0x7f3f201449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1effd700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f200affa0 0x7f3f20139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1f7fe700 1 -- 172.21.15.140:0/4206111672 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f2013e430 msgr2=0x7f3f201449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1f7fe700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f2013e430 0x7f3f201449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1f7fe700 1 -- 172.21.15.140:0/4206111672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f200affa0 msgr2=0x7f3f20139a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:09.761 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1f7fe700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f200affa0 0x7f3f20139a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:09.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1f7fe700 1 -- 172.21.15.140:0/4206111672 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1400b1e0 con 0x7f3f200aec90 2022-07-21T01:57:09.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1f7fe700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f200aec90 0x7f3f20065e70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f3f20062890 tx=0x7f3f14017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:09.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1cff9700 1 -- 172.21.15.140:0/4206111672 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f14006850 con 0x7f3f200aec90 2022-07-21T01:57:09.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1cff9700 1 -- 172.21.15.140:0/4206111672 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f14002f50 con 0x7f3f200aec90 2022-07-21T01:57:09.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f2495b700 1 -- 172.21.15.140:0/4206111672 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f20140ae0 con 0x7f3f200aec90 2022-07-21T01:57:09.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f1cff9700 1 -- 172.21.15.140:0/4206111672 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f1401a670 con 0x7f3f200aec90 2022-07-21T01:57:09.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f2495b700 1 -- 172.21.15.140:0/4206111672 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f20140360 con 0x7f3f200aec90 2022-07-21T01:57:09.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.748+0000 7f3f2495b700 1 -- 172.21.15.140:0/4206111672 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f200b03e0 con 0x7f3f200aec90 2022-07-21T01:57:09.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.752+0000 7f3f1cff9700 1 -- 172.21.15.140:0/4206111672 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3f1401a7d0 con 0x7f3f200aec90 2022-07-21T01:57:09.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.752+0000 7f3f1cff9700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f00053b30 0x7f3f00055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:09.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.752+0000 7f3f1cff9700 1 -- 172.21.15.140:0/4206111672 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f3f1406d080 con 0x7f3f200aec90 2022-07-21T01:57:09.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.752+0000 7f3f1effd700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f00053b30 0x7f3f00055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:09.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.752+0000 7f3f1effd700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f00053b30 0x7f3f00055ff0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f3f1000c000 tx=0x7f3f10008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:09.765 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.756+0000 7f3f1cff9700 1 -- 172.21.15.140:0/4206111672 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3f1406ae90 con 0x7f3f200aec90 2022-07-21T01:57:09.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:09.892+0000 7f3f2495b700 1 -- 172.21.15.140:0/4206111672 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f3f200574e0 con 0x7f3f200aec90 2022-07-21T01:57:10.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.748+0000 7f3f1cff9700 1 -- 172.21.15.140:0/4206111672 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v128) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3f14044280 con 0x7f3f200aec90 2022-07-21T01:57:10.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.748+0000 7f3f2495b700 1 -- 172.21.15.140:0/4206111672 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f00053b30 msgr2=0x7f3f00055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:10.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.748+0000 7f3f2495b700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f00053b30 0x7f3f00055ff0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f3f1000c000 tx=0x7f3f10008040).stop 2022-07-21T01:57:10.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.748+0000 7f3f2495b700 1 -- 172.21.15.140:0/4206111672 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f200aec90 msgr2=0x7f3f20065e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:10.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.748+0000 7f3f2495b700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f200aec90 0x7f3f20065e70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f3f20062890 tx=0x7f3f14017000).stop 2022-07-21T01:57:10.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.748+0000 7f3f2495b700 1 -- 172.21.15.140:0/4206111672 shutdown_connections 2022-07-21T01:57:10.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.748+0000 7f3f2495b700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3f00053b30 0x7f3f00055ff0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:10.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.752+0000 7f3f2495b700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3f2013e430 0x7f3f201449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:10.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.752+0000 7f3f2495b700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3f200affa0 0x7f3f20139a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:10.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.752+0000 7f3f2495b700 1 --2- 172.21.15.140:0/4206111672 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3f200aec90 0x7f3f20065e70 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:10.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.752+0000 7f3f2495b700 1 -- 172.21.15.140:0/4206111672 >> 172.21.15.140:0/4206111672 conn(0x7f3f20063480 msgr2=0x7f3f200b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:10.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.752+0000 7f3f2495b700 1 -- 172.21.15.140:0/4206111672 shutdown_connections 2022-07-21T01:57:10.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.752+0000 7f3f2495b700 1 -- 172.21.15.140:0/4206111672 wait complete. 2022-07-21T01:57:10.755 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:57:10.767 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:57:10.768 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:57:10.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.872+0000 7f05b89c5700 1 Processor -- start 2022-07-21T01:57:10.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.872+0000 7f05b89c5700 1 -- start start 2022-07-21T01:57:10.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b89c5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ab890 0x7f05b413f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:10.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b89c5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b4140940 con 0x7f05b40605a0 2022-07-21T01:57:10.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b89c5700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b40b22d0 con 0x7f05b40b1ef0 2022-07-21T01:57:10.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b89c5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b405fbc0 con 0x7f05b40ab890 2022-07-21T01:57:10.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b259c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f05b40605a0 0x7f05b40ab180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54324/0 (socket says 172.21.15.140:54324) 2022-07-21T01:57:10.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b259c700 1 -- 172.21.15.140:0/947944405 learned_addr learned my addr 172.21.15.140:0/947944405 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:10.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b359e700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ab890 0x7f05b413f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:10.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226346041 0 0) 0x7f05b4140940 con 0x7f05b40605a0 2022-07-21T01:57:10.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0598003260 con 0x7f05b40605a0 2022-07-21T01:57:10.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1704356079 0 0) 0x7f0598003260 con 0x7f05b40605a0 2022-07-21T01:57:10.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ab890 msgr2=0x7f05b413f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:10.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ab890 0x7f05b413f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:10.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 >> v1:172.21.15.152:6789/0 conn(0x7f05b40b1ef0 legacy=0x7f05b40abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:10.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b405fe60 con 0x7f05b40605a0 2022-07-21T01:57:10.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f059c001660 con 0x7f05b40605a0 2022-07-21T01:57:10.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 >> v1:172.21.15.140:6789/0 conn(0x7f05b40605a0 legacy=0x7f05b40ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:10.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0598003780 0x7f0598003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:10.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0598004500 0x7f0598004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:10.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ab890 0x7f0598004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:10.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b405fbc0 con 0x7f0598004500 2022-07-21T01:57:10.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059800d990 con 0x7f0598003780 2022-07-21T01:57:10.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059800db50 con 0x7f05b40ab890 2022-07-21T01:57:10.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b259c700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0598004500 0x7f0598004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:10.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b359e700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ab890 0x7f0598004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:10.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b2d9d700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0598003780 0x7f0598003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:10.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b359e700 1 -- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0598003780 msgr2=0x7f0598003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:10.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b359e700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0598003780 0x7f0598003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:10.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b359e700 1 -- 172.21.15.140:0/947944405 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0598004500 msgr2=0x7f0598004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:10.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b359e700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0598004500 0x7f0598004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:10.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b359e700 1 -- 172.21.15.140:0/947944405 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598003260 con 0x7f05b40ab890 2022-07-21T01:57:10.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b259c700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0598004500 0x7f0598004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:10.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b359e700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ab890 0x7f0598004e70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f05ac00c000 tx=0x7f05ac0091d0).ready entity=mon.2 client_cookie=7c194ed4d2a9d1d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:10.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05ac00e7e0 con 0x7f05b40ab890 2022-07-21T01:57:10.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac00e940 con 0x7f05b40ab890 2022-07-21T01:57:10.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.876+0000 7f05b1d9b700 1 -- 172.21.15.140:0/947944405 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05ac00eca0 con 0x7f05b40ab890 2022-07-21T01:57:10.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.880+0000 7f05b89c5700 1 -- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ab890 msgr2=0x7f0598004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:10.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.880+0000 7f05b89c5700 1 --2- 172.21.15.140:0/947944405 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ab890 0x7f0598004e70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f05ac00c000 tx=0x7f05ac0091d0).stop 2022-07-21T01:57:10.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.880+0000 7f05b359e700 1 -- 172.21.15.140:0/947944405 reap_dead start 2022-07-21T01:57:10.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.880+0000 7f05b89c5700 1 -- 172.21.15.140:0/947944405 shutdown_connections 2022-07-21T01:57:10.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.880+0000 7f05b89c5700 1 -- 172.21.15.140:0/947944405 >> 172.21.15.140:0/947944405 conn(0x7f05b40b2ca0 msgr2=0x7f05b40b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:10.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.880+0000 7f05b89c5700 1 -- 172.21.15.140:0/947944405 shutdown_connections 2022-07-21T01:57:10.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.880+0000 7f05b89c5700 1 -- 172.21.15.140:0/947944405 wait complete. 2022-07-21T01:57:10.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.880+0000 7f05b89c5700 1 Processor -- start 2022-07-21T01:57:10.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.880+0000 7f05b89c5700 1 -- start start 2022-07-21T01:57:10.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.880+0000 7f05b89c5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05b40ab890 0x7f05b4143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:10.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.880+0000 7f05b89c5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ae620 0x7f05b4151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:10.904 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b89c5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05b4142830 0x7f05b4142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:10.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b89c5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4143ee0 con 0x7f05b40ab890 2022-07-21T01:57:10.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b89c5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b41431a0 con 0x7f05b4142830 2022-07-21T01:57:10.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b89c5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4151840 con 0x7f05b40ae620 2022-07-21T01:57:10.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b2d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05b40ab890 0x7f05b4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:10.905 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b2d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05b40ab890 0x7f05b4143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40188/0 (socket says 172.21.15.140:40188) 2022-07-21T01:57:10.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b2d9d700 1 -- 172.21.15.140:0/649598467 learned_addr learned my addr 172.21.15.140:0/649598467 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:10.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b259c700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ae620 0x7f05b4151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:10.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b359e700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05b4142830 0x7f05b4142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:10.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b2d9d700 1 -- 172.21.15.140:0/649598467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ae620 msgr2=0x7f05b4151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:10.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b2d9d700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ae620 0x7f05b4151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:10.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b2d9d700 1 -- 172.21.15.140:0/649598467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05b4142830 msgr2=0x7f05b4142cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:10.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b2d9d700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05b4142830 0x7f05b4142cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:10.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b2d9d700 1 -- 172.21.15.140:0/649598467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b40988b0 con 0x7f05b40ab890 2022-07-21T01:57:10.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b2d9d700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05b40ab890 0x7f05b4143640 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f05b4143c90 tx=0x7f05a4007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:10.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05abfff700 1 -- 172.21.15.140:0/649598467 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05a4002e40 con 0x7f05b40ab890 2022-07-21T01:57:10.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05abfff700 1 -- 172.21.15.140:0/649598467 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a40114c0 con 0x7f05b40ab890 2022-07-21T01:57:10.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b89c5700 1 -- 172.21.15.140:0/649598467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b4098b50 con 0x7f05b40ab890 2022-07-21T01:57:10.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05abfff700 1 -- 172.21.15.140:0/649598467 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05a4011820 con 0x7f05b40ab890 2022-07-21T01:57:10.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b89c5700 1 -- 172.21.15.140:0/649598467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b4142390 con 0x7f05b40ab890 2022-07-21T01:57:10.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05b89c5700 1 -- 172.21.15.140:0/649598467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b4142520 con 0x7f05b40ab890 2022-07-21T01:57:10.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.884+0000 7f05abfff700 1 -- 172.21.15.140:0/649598467 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f05a401a030 con 0x7f05b40ab890 2022-07-21T01:57:10.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.888+0000 7f05abfff700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f058c053a10 0x7f058c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:10.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.888+0000 7f05abfff700 1 -- 172.21.15.140:0/649598467 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f05a406b570 con 0x7f05b40ab890 2022-07-21T01:57:10.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.888+0000 7f05b259c700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f058c053a10 0x7f058c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:10.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.888+0000 7f05b259c700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f058c053a10 0x7f058c055ed0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f059c005000 tx=0x7f05b4062870).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:10.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:10.892+0000 7f05abfff700 1 -- 172.21.15.140:0/649598467 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f05a40426e0 con 0x7f05b40ab890 2022-07-21T01:57:11.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.060+0000 7f05b89c5700 1 -- 172.21.15.140:0/649598467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f05b4060100 con 0x7f05b40ab890 2022-07-21T01:57:11.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.060+0000 7f05abfff700 1 -- 172.21.15.140:0/649598467 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v42) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f05a4018070 con 0x7f05b40ab890 2022-07-21T01:57:11.061 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:11.061 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:57:11.062 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:57:11.062 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.062 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:57:11.062 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:57:11.063 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.063 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:57:11.063 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:11.063 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:11.064 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.064 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.064 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.064 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.064 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:57:11.065 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:57:11.065 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.065 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:57:11.065 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:11.066 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:11.066 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.066 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.066 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.066 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.067 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:57:11.067 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:57:11.067 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.067 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:11.067 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:11.068 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.068 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.068 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.068 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.069 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:57:11.069 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:57:11.069 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.069 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:11.069 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:11.070 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.070 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.070 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.070 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.071 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:57:11.071 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:57:11.071 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.071 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:11.071 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:11.072 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.072 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.072 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.072 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.073 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:57:11.073 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:57:11.073 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.073 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:11.073 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:11.074 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.074 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.074 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.074 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.075 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:57:11.075 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:57:11.075 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.075 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:11.076 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:11.076 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.076 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.076 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.076 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.077 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:57:11.077 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:57:11.077 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.077 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:11.078 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:11.078 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.078 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.078 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.078 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.079 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:57:11.079 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:57:11.079 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.079 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:11.080 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:11.080 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.080 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.080 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.080 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.081 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:57:11.081 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:57:11.081 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.081 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:11.082 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:11.082 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.082 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.082 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.082 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.083 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:57:11.083 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:57:11.083 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.083 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:57:11.083 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:57:11.084 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:57:11.084 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.084 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.084 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.085 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:57:11.085 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:57:11.085 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.085 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:57:11.085 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:57:11.086 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:57:11.086 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.086 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.086 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.087 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:57:11.087 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:57:11.087 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.087 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:57:11.087 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:57:11.088 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:57:11.088 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.088 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.088 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.088 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:57:11.089 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:57:11.089 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.089 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:57:11.089 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:11.090 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:11.090 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.090 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.090 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.090 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.091 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:57:11.091 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:57:11.091 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.091 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:57:11.091 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:57:11.092 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.092 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.092 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:11.092 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:11.093 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:57:11.093 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:57:11.093 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:11.093 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:57:11.093 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:57:11.094 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:11.094 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.094 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:11.094 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:11.094 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:57:11.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 -- 172.21.15.140:0/649598467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f058c053a10 msgr2=0x7f058c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f058c053a10 0x7f058c055ed0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f059c005000 tx=0x7f05b4062870).stop 2022-07-21T01:57:11.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 -- 172.21.15.140:0/649598467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05b40ab890 msgr2=0x7f05b4143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05b40ab890 0x7f05b4143640 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f05b4143c90 tx=0x7f05a4007a00).stop 2022-07-21T01:57:11.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 -- 172.21.15.140:0/649598467 shutdown_connections 2022-07-21T01:57:11.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f058c053a10 0x7f058c055ed0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05b4142830 0x7f05b4142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05b40ae620 0x7f05b4151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 --2- 172.21.15.140:0/649598467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05b40ab890 0x7f05b4143640 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 -- 172.21.15.140:0/649598467 >> 172.21.15.140:0/649598467 conn(0x7f05b40b2ca0 msgr2=0x7f05b413c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:11.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 -- 172.21.15.140:0/649598467 shutdown_connections 2022-07-21T01:57:11.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.064+0000 7f05b89c5700 1 -- 172.21.15.140:0/649598467 wait complete. 2022-07-21T01:57:11.100 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:11.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.196+0000 7f680f05a700 1 Processor -- start 2022-07-21T01:57:11.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.196+0000 7f680f05a700 1 -- start start 2022-07-21T01:57:11.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f680814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f68080acef0 0x7f6808146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680805d480 con 0x7f680805c3f0 2022-07-21T01:57:11.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68080b21e0 con 0x7f68080acef0 2022-07-21T01:57:11.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680805cf40 con 0x7f68080b1e00 2022-07-21T01:57:11.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680cdf6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f680814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680d5f7700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f68080acef0 0x7f6808146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680cdf6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f680814a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36928/0 (socket says 172.21.15.140:36928) 2022-07-21T01:57:11.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680cdf6700 1 -- 172.21.15.140:0/1964589632 learned_addr learned my addr 172.21.15.140:0/1964589632 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:11.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680cdf6700 1 -- 172.21.15.140:0/1964589632 >> v2:172.21.15.152:3300/0 conn(0x7f68080acef0 msgr2=0x7f6808146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:11.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680cdf6700 1 --2- 172.21.15.140:0/1964589632 >> v2:172.21.15.152:3300/0 conn(0x7f68080acef0 0x7f6808146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680cdf6700 1 -- 172.21.15.140:0/1964589632 >> v1:172.21.15.152:6789/0 conn(0x7f680805c3f0 legacy=0x7f68080ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680cdf6700 1 -- 172.21.15.140:0/1964589632 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680805d1e0 con 0x7f68080b1e00 2022-07-21T01:57:11.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680cdf6700 1 --2- 172.21.15.140:0/1964589632 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f680814a6d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f68000032f0 tx=0x7f6800009290).ready entity=mon.2 client_cookie=a892f21bec610a33 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:11.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f68077fe700 1 -- 172.21.15.140:0/1964589632 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f680000b8a0 con 0x7f68080b1e00 2022-07-21T01:57:11.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f68077fe700 1 -- 172.21.15.140:0/1964589632 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680000ba00 con 0x7f68080b1e00 2022-07-21T01:57:11.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f68077fe700 1 -- 172.21.15.140:0/1964589632 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f680000bb70 con 0x7f68080b1e00 2022-07-21T01:57:11.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 -- 172.21.15.140:0/1964589632 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 msgr2=0x7f680814a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 --2- 172.21.15.140:0/1964589632 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f680814a6d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f68000032f0 tx=0x7f6800009290).stop 2022-07-21T01:57:11.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 -- 172.21.15.140:0/1964589632 shutdown_connections 2022-07-21T01:57:11.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 --2- 172.21.15.140:0/1964589632 >> v2:172.21.15.152:3300/0 conn(0x7f68080acef0 0x7f6808146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 --2- 172.21.15.140:0/1964589632 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f680814a6d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 -- 172.21.15.140:0/1964589632 >> 172.21.15.140:0/1964589632 conn(0x7f68080611c0 msgr2=0x7f6808062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:11.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 -- 172.21.15.140:0/1964589632 shutdown_connections 2022-07-21T01:57:11.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 -- 172.21.15.140:0/1964589632 wait complete. 2022-07-21T01:57:11.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 Processor -- start 2022-07-21T01:57:11.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.200+0000 7f680f05a700 1 -- start start 2022-07-21T01:57:11.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680f05a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68080acef0 0x7f68081443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680f05a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f68081570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680f05a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f680809e3a0 0x7f6808149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680f05a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680814a5e0 con 0x7f68080acef0 2022-07-21T01:57:11.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680f05a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68081566c0 con 0x7f680809e3a0 2022-07-21T01:57:11.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680f05a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6808156960 con 0x7f68080b1e00 2022-07-21T01:57:11.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680cdf6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68080acef0 0x7f68081443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680d5f7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f680809e3a0 0x7f6808149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f6807fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f68081570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680d5f7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f680809e3a0 0x7f6808149560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:45946/0 (socket says 172.21.15.140:45946) 2022-07-21T01:57:11.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680d5f7700 1 -- 172.21.15.140:0/1195554017 learned_addr learned my addr 172.21.15.140:0/1195554017 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:11.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680cdf6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68080acef0 0x7f68081443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40204/0 (socket says 172.21.15.140:40204) 2022-07-21T01:57:11.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f6807fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f68081570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36938/0 (socket says 172.21.15.140:36938) 2022-07-21T01:57:11.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f6807fff700 1 -- 172.21.15.140:0/1195554017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f680809e3a0 msgr2=0x7f6808149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f6807fff700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f680809e3a0 0x7f6808149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f6807fff700 1 -- 172.21.15.140:0/1195554017 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68080acef0 msgr2=0x7f68081443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f6807fff700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68080acef0 0x7f68081443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f6807fff700 1 -- 172.21.15.140:0/1195554017 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800009530 con 0x7f68080b1e00 2022-07-21T01:57:11.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f6807fff700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f68081570a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f68080ac6c0 tx=0x7f67fc007b50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:11.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f6805ffb700 1 -- 172.21.15.140:0/1195554017 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67fc007030 con 0x7f68080b1e00 2022-07-21T01:57:11.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f6805ffb700 1 -- 172.21.15.140:0/1195554017 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc007190 con 0x7f68080b1e00 2022-07-21T01:57:11.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680f05a700 1 -- 172.21.15.140:0/1195554017 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6808156c00 con 0x7f68080b1e00 2022-07-21T01:57:11.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f6805ffb700 1 -- 172.21.15.140:0/1195554017 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67fc006510 con 0x7f68080b1e00 2022-07-21T01:57:11.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680f05a700 1 -- 172.21.15.140:0/1195554017 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6808157590 con 0x7f68080b1e00 2022-07-21T01:57:11.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.204+0000 7f680f05a700 1 -- 172.21.15.140:0/1195554017 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6808156d90 con 0x7f68080b1e00 2022-07-21T01:57:11.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.208+0000 7f6805ffb700 1 -- 172.21.15.140:0/1195554017 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f67fc010030 con 0x7f68080b1e00 2022-07-21T01:57:11.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.208+0000 7f6805ffb700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67e8053a70 0x7f67e8055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.208+0000 7f6805ffb700 1 -- 172.21.15.140:0/1195554017 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f67fc06b780 con 0x7f68080b1e00 2022-07-21T01:57:11.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.208+0000 7f680cdf6700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67e8053a70 0x7f67e8055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.208+0000 7f680cdf6700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67e8053a70 0x7f67e8055f30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f680001b000 tx=0x7f6800016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:11.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.216+0000 7f6805ffb700 1 -- 172.21.15.140:0/1195554017 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f67fc0428c0 con 0x7f68080b1e00 2022-07-21T01:57:11.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.340+0000 7f680f05a700 1 -- 172.21.15.140:0/1195554017 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68080574e0 con 0x7f68080b1e00 2022-07-21T01:57:11.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.344+0000 7f6805ffb700 1 -- 172.21.15.140:0/1195554017 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+13287 (secure 0 0 0) 0x7f67fc045ee0 con 0x7f68080b1e00 2022-07-21T01:57:11.346 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:11.346 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":128,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:10.739002+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:11.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 -- 172.21.15.140:0/1195554017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67e8053a70 msgr2=0x7f67e8055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67e8053a70 0x7f67e8055f30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f680001b000 tx=0x7f6800016000).stop 2022-07-21T01:57:11.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 -- 172.21.15.140:0/1195554017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 msgr2=0x7f68081570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f68081570a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f68080ac6c0 tx=0x7f67fc007b50).stop 2022-07-21T01:57:11.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 -- 172.21.15.140:0/1195554017 shutdown_connections 2022-07-21T01:57:11.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67e8053a70 0x7f67e8055f30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f680809e3a0 0x7f6808149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68080b1e00 0x7f68081570a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 --2- 172.21.15.140:0/1195554017 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68080acef0 0x7f68081443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 -- 172.21.15.140:0/1195554017 >> 172.21.15.140:0/1195554017 conn(0x7f680805c3f0 msgr2=0x7f6808069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:11.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 -- 172.21.15.140:0/1195554017 shutdown_connections 2022-07-21T01:57:11.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.348+0000 7f680f05a700 1 -- 172.21.15.140:0/1195554017 wait complete. 2022-07-21T01:57:11.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:57:11.366 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:57:11.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679f573700 1 Processor -- start 2022-07-21T01:57:11.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679f573700 1 -- start start 2022-07-21T01:57:11.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679f573700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f67980b0480 0x7f679814a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679f573700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f679814ab60 0x7f6798144320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679f573700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67980ac5e0 con 0x7f67980b5b50 2022-07-21T01:57:11.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679f573700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67980ad3c0 con 0x7f679814ab60 2022-07-21T01:57:11.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679f573700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67980b5f30 con 0x7f67980b0480 2022-07-21T01:57:11.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679db10700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f679814ab60 0x7f6798144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679db10700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f679814ab60 0x7f6798144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40216/0 (socket says 172.21.15.140:40216) 2022-07-21T01:57:11.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679db10700 1 -- 172.21.15.140:0/2763765926 learned_addr learned my addr 172.21.15.140:0/2763765926 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:11.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679cb0e700 1 --2- 172.21.15.140:0/2763765926 >> v2:172.21.15.152:3300/0 conn(0x7f67980b0480 0x7f679814a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679db10700 1 -- 172.21.15.140:0/2763765926 >> v2:172.21.15.152:3300/0 conn(0x7f67980b0480 msgr2=0x7f679814a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:11.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679db10700 1 --2- 172.21.15.140:0/2763765926 >> v2:172.21.15.152:3300/0 conn(0x7f67980b0480 0x7f679814a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679db10700 1 -- 172.21.15.140:0/2763765926 >> v1:172.21.15.152:6789/0 conn(0x7f67980b5b50 legacy=0x7f67980accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679db10700 1 -- 172.21.15.140:0/2763765926 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679805cf70 con 0x7f679814ab60 2022-07-21T01:57:11.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f679db10700 1 --2- 172.21.15.140:0/2763765926 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f679814ab60 0x7f6798144320 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f67900032f0 tx=0x7f6790009290).ready entity=mon.0 client_cookie=4bd41782f358293f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:11.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f678bfff700 1 -- 172.21.15.140:0/2763765926 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679000b870 con 0x7f679814ab60 2022-07-21T01:57:11.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f678bfff700 1 -- 172.21.15.140:0/2763765926 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679000b9d0 con 0x7f679814ab60 2022-07-21T01:57:11.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.472+0000 7f678bfff700 1 -- 172.21.15.140:0/2763765926 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679000bb40 con 0x7f679814ab60 2022-07-21T01:57:11.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 -- 172.21.15.140:0/2763765926 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f679814ab60 msgr2=0x7f6798144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 --2- 172.21.15.140:0/2763765926 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f679814ab60 0x7f6798144320 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f67900032f0 tx=0x7f6790009290).stop 2022-07-21T01:57:11.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 -- 172.21.15.140:0/2763765926 shutdown_connections 2022-07-21T01:57:11.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 --2- 172.21.15.140:0/2763765926 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f679814ab60 0x7f6798144320 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 --2- 172.21.15.140:0/2763765926 >> v2:172.21.15.152:3300/0 conn(0x7f67980b0480 0x7f679814a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 -- 172.21.15.140:0/2763765926 >> 172.21.15.140:0/2763765926 conn(0x7f67980635e0 msgr2=0x7f67980b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:11.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 -- 172.21.15.140:0/2763765926 shutdown_connections 2022-07-21T01:57:11.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 -- 172.21.15.140:0/2763765926 wait complete. 2022-07-21T01:57:11.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 Processor -- start 2022-07-21T01:57:11.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 -- start start 2022-07-21T01:57:11.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67980b0480 0x7f67980af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67980b5b50 0x7f67980ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f679814ab60 0x7f6798140780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6798063d40 con 0x7f67980b0480 2022-07-21T01:57:11.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6798063fe0 con 0x7f679814ab60 2022-07-21T01:57:11.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679f573700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6798064280 con 0x7f67980b5b50 2022-07-21T01:57:11.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679d30f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67980b0480 0x7f67980af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679d30f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67980b0480 0x7f67980af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40218/0 (socket says 172.21.15.140:40218) 2022-07-21T01:57:11.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679d30f700 1 -- 172.21.15.140:0/1498185762 learned_addr learned my addr 172.21.15.140:0/1498185762 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:11.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679cb0e700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67980b5b50 0x7f67980ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679db10700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f679814ab60 0x7f6798140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679d30f700 1 -- 172.21.15.140:0/1498185762 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67980b5b50 msgr2=0x7f67980ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679d30f700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67980b5b50 0x7f67980ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679d30f700 1 -- 172.21.15.140:0/1498185762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f679814ab60 msgr2=0x7f6798140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679d30f700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f679814ab60 0x7f6798140780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679d30f700 1 -- 172.21.15.140:0/1498185762 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6790009530 con 0x7f67980b0480 2022-07-21T01:57:11.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679db10700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f679814ab60 0x7f6798140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:11.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.476+0000 7f679d30f700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67980b0480 0x7f67980af090 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f67980627e0 tx=0x7f6794007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:11.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.480+0000 7f678a7fc700 1 -- 172.21.15.140:0/1498185762 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679400ce30 con 0x7f67980b0480 2022-07-21T01:57:11.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.480+0000 7f678a7fc700 1 -- 172.21.15.140:0/1498185762 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679400f470 con 0x7f67980b0480 2022-07-21T01:57:11.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.480+0000 7f678a7fc700 1 -- 172.21.15.140:0/1498185762 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f679400f7d0 con 0x7f67980b0480 2022-07-21T01:57:11.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.480+0000 7f679f573700 1 -- 172.21.15.140:0/1498185762 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6798148790 con 0x7f67980b0480 2022-07-21T01:57:11.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.480+0000 7f679f573700 1 -- 172.21.15.140:0/1498185762 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6798067740 con 0x7f67980b0480 2022-07-21T01:57:11.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.480+0000 7f678a7fc700 1 -- 172.21.15.140:0/1498185762 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6794019030 con 0x7f67980b0480 2022-07-21T01:57:11.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.480+0000 7f679f573700 1 -- 172.21.15.140:0/1498185762 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6798148920 con 0x7f67980b0480 2022-07-21T01:57:11.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.480+0000 7f678a7fc700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6780053b00 0x7f6780055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.480+0000 7f678a7fc700 1 -- 172.21.15.140:0/1498185762 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f679406b130 con 0x7f67980b0480 2022-07-21T01:57:11.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.480+0000 7f679cb0e700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6780053b00 0x7f6780055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.480+0000 7f679cb0e700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6780053b00 0x7f6780055fc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f678c00b000 tx=0x7f678c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:11.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.488+0000 7f678a7fc700 1 -- 172.21.15.140:0/1498185762 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6794042270 con 0x7f67980b0480 2022-07-21T01:57:11.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.612+0000 7f679f573700 1 -- 172.21.15.140:0/1498185762 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f67980ac810 con 0x7f67980b0480 2022-07-21T01:57:11.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.612+0000 7f678a7fc700 1 -- 172.21.15.140:0/1498185762 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v128) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6794045890 con 0x7f67980b0480 2022-07-21T01:57:11.615 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:57:11.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 -- 172.21.15.140:0/1498185762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6780053b00 msgr2=0x7f6780055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6780053b00 0x7f6780055fc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f678c00b000 tx=0x7f678c006df0).stop 2022-07-21T01:57:11.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 -- 172.21.15.140:0/1498185762 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67980b0480 msgr2=0x7f67980af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67980b0480 0x7f67980af090 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f67980627e0 tx=0x7f6794007c50).stop 2022-07-21T01:57:11.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 -- 172.21.15.140:0/1498185762 shutdown_connections 2022-07-21T01:57:11.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6780053b00 0x7f6780055fc0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f679814ab60 0x7f6798140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67980b5b50 0x7f67980ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 --2- 172.21.15.140:0/1498185762 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67980b0480 0x7f67980af090 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 -- 172.21.15.140:0/1498185762 >> 172.21.15.140:0/1498185762 conn(0x7f67980635e0 msgr2=0x7f679805c120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:11.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 -- 172.21.15.140:0/1498185762 shutdown_connections 2022-07-21T01:57:11.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.616+0000 7f679f573700 1 -- 172.21.15.140:0/1498185762 wait complete. 2022-07-21T01:57:11.629 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:57:11.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.736+0000 7f615236a700 1 Processor -- start 2022-07-21T01:57:11.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.736+0000 7f615236a700 1 -- start start 2022-07-21T01:57:11.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f614c0af030 0x7f614c05c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614c0af410 con 0x7f614c1296f0 2022-07-21T01:57:11.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614c144050 con 0x7f614c0b5690 2022-07-21T01:57:11.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614c05be30 con 0x7f614c0af030 2022-07-21T01:57:11.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f6151b69700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f614c1296f0 0x7f614c129ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54374/0 (socket says 172.21.15.140:54374) 2022-07-21T01:57:11.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f6151368700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f614c0af030 0x7f614c05c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f6151b69700 1 -- 172.21.15.140:0/2038297271 learned_addr learned my addr 172.21.15.140:0/2038297271 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:11.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f613bfff700 1 -- 172.21.15.140:0/2038297271 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259204003 0 0) 0x7f614c0af410 con 0x7f614c1296f0 2022-07-21T01:57:11.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f613bfff700 1 -- 172.21.15.140:0/2038297271 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6134003260 con 0x7f614c1296f0 2022-07-21T01:57:11.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f613bfff700 1 -- 172.21.15.140:0/2038297271 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669753147 0 0) 0x7f614c144050 con 0x7f614c0b5690 2022-07-21T01:57:11.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f613bfff700 1 -- 172.21.15.140:0/2038297271 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f614c0af410 con 0x7f614c0b5690 2022-07-21T01:57:11.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f6151368700 1 -- 172.21.15.140:0/2038297271 >> v1:172.21.15.152:6789/0 conn(0x7f614c0b5690 legacy=0x7f614c140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f6151368700 1 -- 172.21.15.140:0/2038297271 >> v1:172.21.15.140:6789/0 conn(0x7f614c1296f0 legacy=0x7f614c129ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f6151368700 1 -- 172.21.15.140:0/2038297271 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c05c0d0 con 0x7f614c0af030 2022-07-21T01:57:11.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f6151368700 1 --2- 172.21.15.140:0/2038297271 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f614c0af030 0x7f614c05c570 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f613c001110 tx=0x7f6148006df0).ready entity=mon.0 client_cookie=d05302e7b27eb398 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:11.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f613bfff700 1 -- 172.21.15.140:0/2038297271 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f614800d4f0 con 0x7f614c0af030 2022-07-21T01:57:11.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f613bfff700 1 -- 172.21.15.140:0/2038297271 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6148000e90 con 0x7f614c0af030 2022-07-21T01:57:11.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f613bfff700 1 -- 172.21.15.140:0/2038297271 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f614800d840 con 0x7f614c0af030 2022-07-21T01:57:11.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 -- 172.21.15.140:0/2038297271 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f614c0af030 msgr2=0x7f614c05c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 --2- 172.21.15.140:0/2038297271 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f614c0af030 0x7f614c05c570 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f613c001110 tx=0x7f6148006df0).stop 2022-07-21T01:57:11.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 -- 172.21.15.140:0/2038297271 shutdown_connections 2022-07-21T01:57:11.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 --2- 172.21.15.140:0/2038297271 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f614c0af030 0x7f614c05c570 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 -- 172.21.15.140:0/2038297271 >> 172.21.15.140:0/2038297271 conn(0x7f614c0634c0 msgr2=0x7f614c0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:11.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 -- 172.21.15.140:0/2038297271 shutdown_connections 2022-07-21T01:57:11.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 -- 172.21.15.140:0/2038297271 wait complete. 2022-07-21T01:57:11.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.740+0000 7f615236a700 1 Processor -- start 2022-07-21T01:57:11.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f615236a700 1 -- start start 2022-07-21T01:57:11.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f615236a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f614c0af030 0x7f614c065ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f615236a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f614c0b5690 0x7f614c064ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f615236a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f614c1296f0 0x7f614c1402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f615236a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c12ef80 con 0x7f614c1296f0 2022-07-21T01:57:11.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f615236a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c12f1e0 con 0x7f614c0af030 2022-07-21T01:57:11.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f615236a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c12f480 con 0x7f614c0b5690 2022-07-21T01:57:11.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f6151b69700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f614c1296f0 0x7f614c1402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f6151368700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f614c0af030 0x7f614c065ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f6151b69700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f614c1296f0 0x7f614c1402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40234/0 (socket says 172.21.15.140:40234) 2022-07-21T01:57:11.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f6150b67700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f614c0b5690 0x7f614c064ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f6151b69700 1 -- 172.21.15.140:0/3446147435 learned_addr learned my addr 172.21.15.140:0/3446147435 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:11.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f6151b69700 1 -- 172.21.15.140:0/3446147435 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f614c0b5690 msgr2=0x7f614c064ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f6151b69700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f614c0b5690 0x7f614c064ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f6151b69700 1 -- 172.21.15.140:0/3446147435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f614c0af030 msgr2=0x7f614c065ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f6151b69700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f614c0af030 0x7f614c065ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f6151b69700 1 -- 172.21.15.140:0/3446147435 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614800b180 con 0x7f614c1296f0 2022-07-21T01:57:11.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f6151b69700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f614c1296f0 0x7f614c1402f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f61440013c0 tx=0x7f6144007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:11.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f613a7fc700 1 -- 172.21.15.140:0/3446147435 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6144005ba0 con 0x7f614c1296f0 2022-07-21T01:57:11.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f613a7fc700 1 -- 172.21.15.140:0/3446147435 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6144005d00 con 0x7f614c1296f0 2022-07-21T01:57:11.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f615236a700 1 -- 172.21.15.140:0/3446147435 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614c12f720 con 0x7f614c1296f0 2022-07-21T01:57:11.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f613a7fc700 1 -- 172.21.15.140:0/3446147435 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61440072c0 con 0x7f614c1296f0 2022-07-21T01:57:11.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f615236a700 1 -- 172.21.15.140:0/3446147435 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614c064720 con 0x7f614c1296f0 2022-07-21T01:57:11.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f615236a700 1 -- 172.21.15.140:0/3446147435 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614c12f8b0 con 0x7f614c1296f0 2022-07-21T01:57:11.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.744+0000 7f613a7fc700 1 -- 172.21.15.140:0/3446147435 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f614401a070 con 0x7f614c1296f0 2022-07-21T01:57:11.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.748+0000 7f613a7fc700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f612c053a70 0x7f612c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.748+0000 7f613a7fc700 1 -- 172.21.15.140:0/3446147435 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f614406b190 con 0x7f614c1296f0 2022-07-21T01:57:11.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.748+0000 7f6151368700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f612c053a70 0x7f612c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.748+0000 7f6151368700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f612c053a70 0x7f612c055f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f6148018000 tx=0x7f6148014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:11.760 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.748+0000 7f613a7fc700 1 -- 172.21.15.140:0/3446147435 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f61440422d0 con 0x7f614c1296f0 2022-07-21T01:57:11.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.872+0000 7f615236a700 1 -- 172.21.15.140:0/3446147435 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f614c0574e0 con 0x7f614c1296f0 2022-07-21T01:57:11.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f613a7fc700 1 -- 172.21.15.140:0/3446147435 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v128) v1 ==== 92+0+10 (secure 0 0 0) 0x7f61440458f0 con 0x7f614c1296f0 2022-07-21T01:57:11.876 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:11.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 -- 172.21.15.140:0/3446147435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f612c053a70 msgr2=0x7f612c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f612c053a70 0x7f612c055f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f6148018000 tx=0x7f6148014000).stop 2022-07-21T01:57:11.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 -- 172.21.15.140:0/3446147435 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f614c1296f0 msgr2=0x7f614c1402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:11.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f614c1296f0 0x7f614c1402f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f61440013c0 tx=0x7f6144007e50).stop 2022-07-21T01:57:11.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 -- 172.21.15.140:0/3446147435 shutdown_connections 2022-07-21T01:57:11.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f612c053a70 0x7f612c055f30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f614c1296f0 0x7f614c1402f0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f614c0b5690 0x7f614c064ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 --2- 172.21.15.140:0/3446147435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f614c0af030 0x7f614c065ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 -- 172.21.15.140:0/3446147435 >> 172.21.15.140:0/3446147435 conn(0x7f614c0634c0 msgr2=0x7f614c142ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:11.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 -- 172.21.15.140:0/3446147435 shutdown_connections 2022-07-21T01:57:11.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.876+0000 7f615236a700 1 -- 172.21.15.140:0/3446147435 wait complete. 2022-07-21T01:57:11.890 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:57:11.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.988+0000 7fbd1707c700 1 Processor -- start 2022-07-21T01:57:11.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.988+0000 7fbd1707c700 1 -- start start 2022-07-21T01:57:11.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd100b5650 0x7fbd100b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 0x7fbd1005c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbd1005bd90 0x7fbd1005c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:11.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd10142970 con 0x7fbd1005bd90 2022-07-21T01:57:11.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd100aea30 con 0x7fbd100b5650 2022-07-21T01:57:11.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100aec60 con 0x7fbd100b1200 2022-07-21T01:57:11.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1687b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbd1005bd90 0x7fbd1005c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15879700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 0x7fbd1005c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1607a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbd100b5650 0x7fbd100b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:11.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1687b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbd1005bd90 0x7fbd1005c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40242/0 (socket says 172.21.15.140:40242) 2022-07-21T01:57:11.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15879700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 0x7fbd1005c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36968/0 (socket says 172.21.15.140:36968) 2022-07-21T01:57:11.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15879700 1 -- 172.21.15.140:0/2929946537 learned_addr learned my addr 172.21.15.140:0/2929946537 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:11.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15879700 1 -- 172.21.15.140:0/2929946537 >> v2:172.21.15.152:3300/0 conn(0x7fbd100b5650 msgr2=0x7fbd100b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:11.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15879700 1 --2- 172.21.15.140:0/2929946537 >> v2:172.21.15.152:3300/0 conn(0x7fbd100b5650 0x7fbd100b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15879700 1 -- 172.21.15.140:0/2929946537 >> v2:172.21.15.140:3300/0 conn(0x7fbd1005bd90 msgr2=0x7fbd1005c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:11.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15879700 1 --2- 172.21.15.140:0/2929946537 >> v2:172.21.15.140:3300/0 conn(0x7fbd1005bd90 0x7fbd1005c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:11.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15879700 1 -- 172.21.15.140:0/2929946537 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd100af180 con 0x7fbd100b1200 2022-07-21T01:57:11.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15879700 1 --2- 172.21.15.140:0/2929946537 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 0x7fbd1005c4f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fbd100b16d0 tx=0x7fbd0c009250).ready entity=mon.2 client_cookie=b80570ad7a3e249a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:11.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15078700 1 -- 172.21.15.140:0/2929946537 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd0c00b680 con 0x7fbd100b1200 2022-07-21T01:57:11.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15078700 1 -- 172.21.15.140:0/2929946537 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c00b7e0 con 0x7fbd100b1200 2022-07-21T01:57:11.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd15078700 1 -- 172.21.15.140:0/2929946537 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd0c00b950 con 0x7fbd100b1200 2022-07-21T01:57:12.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 -- 172.21.15.140:0/2929946537 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 msgr2=0x7fbd1005c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:12.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 --2- 172.21.15.140:0/2929946537 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 0x7fbd1005c4f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fbd100b16d0 tx=0x7fbd0c009250).stop 2022-07-21T01:57:12.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 -- 172.21.15.140:0/2929946537 shutdown_connections 2022-07-21T01:57:12.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 --2- 172.21.15.140:0/2929946537 >> v2:172.21.15.140:3300/0 conn(0x7fbd1005bd90 0x7fbd1005c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 --2- 172.21.15.140:0/2929946537 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 0x7fbd1005c4f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 --2- 172.21.15.140:0/2929946537 >> v2:172.21.15.152:3300/0 conn(0x7fbd100b5650 0x7fbd100b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 -- 172.21.15.140:0/2929946537 >> 172.21.15.140:0/2929946537 conn(0x7fbd100634e0 msgr2=0x7fbd100b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:12.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 -- 172.21.15.140:0/2929946537 shutdown_connections 2022-07-21T01:57:12.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.992+0000 7fbd1707c700 1 -- 172.21.15.140:0/2929946537 wait complete. 2022-07-21T01:57:12.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1707c700 1 Processor -- start 2022-07-21T01:57:12.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1707c700 1 -- start start 2022-07-21T01:57:12.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1707c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1005bd90 0x7fbd101555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1707c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 0x7fbd101450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1707c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbd100b5650 0x7fbd100641a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1707c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1013f210 con 0x7fbd100b5650 2022-07-21T01:57:12.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1707c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd10063d40 con 0x7fbd1005bd90 2022-07-21T01:57:12.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1707c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1013e390 con 0x7fbd100b1200 2022-07-21T01:57:12.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1687b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbd100b5650 0x7fbd100641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1687b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbd100b5650 0x7fbd100641a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40248/0 (socket says 172.21.15.140:40248) 2022-07-21T01:57:12.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1687b700 1 -- 172.21.15.140:0/3849253425 learned_addr learned my addr 172.21.15.140:0/3849253425 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:12.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd1607a700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1005bd90 0x7fbd101555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:11.996+0000 7fbd15879700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 0x7fbd101450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd1687b700 1 -- 172.21.15.140:0/3849253425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 msgr2=0x7fbd101450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:12.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd1687b700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 0x7fbd101450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd1687b700 1 -- 172.21.15.140:0/3849253425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1005bd90 msgr2=0x7fbd101555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:12.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd1687b700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1005bd90 0x7fbd101555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd1687b700 1 -- 172.21.15.140:0/3849253425 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0094d0 con 0x7fbd100b5650 2022-07-21T01:57:12.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd1687b700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbd100b5650 0x7fbd100641a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7fbd0000c000 tx=0x7fbd00006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:12.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd077fe700 1 -- 172.21.15.140:0/3849253425 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd00011750 con 0x7fbd100b5650 2022-07-21T01:57:12.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd077fe700 1 -- 172.21.15.140:0/3849253425 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd000118b0 con 0x7fbd100b5650 2022-07-21T01:57:12.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd1707c700 1 -- 172.21.15.140:0/3849253425 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1013e0f0 con 0x7fbd100b5650 2022-07-21T01:57:12.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd077fe700 1 -- 172.21.15.140:0/3849253425 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd00011c10 con 0x7fbd100b5650 2022-07-21T01:57:12.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd1707c700 1 -- 172.21.15.140:0/3849253425 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd10145640 con 0x7fbd100b5650 2022-07-21T01:57:12.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd1707c700 1 -- 172.21.15.140:0/3849253425 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd10063eb0 con 0x7fbd100b5650 2022-07-21T01:57:12.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd077fe700 1 -- 172.21.15.140:0/3849253425 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbd00011d70 con 0x7fbd100b5650 2022-07-21T01:57:12.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.000+0000 7fbd077fe700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbcf0053aa0 0x7fbcf0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.004+0000 7fbd077fe700 1 -- 172.21.15.140:0/3849253425 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fbd0006be80 con 0x7fbd100b5650 2022-07-21T01:57:12.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.004+0000 7fbd1607a700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbcf0053aa0 0x7fbcf0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.004+0000 7fbd1607a700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbcf0053aa0 0x7fbcf0055f60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fbd0800b000 tx=0x7fbd08006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:12.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.008+0000 7fbd077fe700 1 -- 172.21.15.140:0/3849253425 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbd000430d0 con 0x7fbd100b5650 2022-07-21T01:57:12.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.128+0000 7fbd1707c700 1 -- 172.21.15.140:0/3849253425 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbd100574e0 con 0x7fbd100b5650 2022-07-21T01:57:12.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd077fe700 1 -- 172.21.15.140:0/3849253425 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) v1 ==== 84+0+143 (secure 0 0 0) 0x7fbd000466f0 con 0x7fbd100b5650 2022-07-21T01:57:12.133 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:12.133 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:57:12.133 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:12.133 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:57:12.134 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:57:12.134 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:12.134 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:12.134 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:57:12.135 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:57:12.135 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:12.135 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:57:12.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 -- 172.21.15.140:0/3849253425 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbcf0053aa0 msgr2=0x7fbcf0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:12.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbcf0053aa0 0x7fbcf0055f60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fbd0800b000 tx=0x7fbd08006df0).stop 2022-07-21T01:57:12.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 -- 172.21.15.140:0/3849253425 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbd100b5650 msgr2=0x7fbd100641a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:12.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbd100b5650 0x7fbd100641a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7fbd0000c000 tx=0x7fbd00006df0).stop 2022-07-21T01:57:12.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 -- 172.21.15.140:0/3849253425 shutdown_connections 2022-07-21T01:57:12.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbcf0053aa0 0x7fbcf0055f60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbd100b5650 0x7fbd100641a0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbd100b1200 0x7fbd101450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 --2- 172.21.15.140:0/3849253425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd1005bd90 0x7fbd101555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 -- 172.21.15.140:0/3849253425 >> 172.21.15.140:0/3849253425 conn(0x7fbd100634e0 msgr2=0x7fbd101421b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:12.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 -- 172.21.15.140:0/3849253425 shutdown_connections 2022-07-21T01:57:12.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.132+0000 7fbd1707c700 1 -- 172.21.15.140:0/3849253425 wait complete. 2022-07-21T01:57:12.145 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:57:12.146 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:57:12.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6547baf700 1 Processor -- start 2022-07-21T01:57:12.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6547baf700 1 -- start start 2022-07-21T01:57:12.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6547baf700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f654009df40 0x7f6540145680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6547baf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65400b4d60 0x7f6540145b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6547baf700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65400607e0 con 0x7f6540060ee0 2022-07-21T01:57:12.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6547baf700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6540060d20 con 0x7f654009df40 2022-07-21T01:57:12.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6547baf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6540060a80 con 0x7f65400b4d60 2022-07-21T01:57:12.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654614c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f6540060ee0 0x7f6540146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54402/0 (socket says 172.21.15.140:54402) 2022-07-21T01:57:12.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654614c700 1 -- 172.21.15.140:0/3009504840 learned_addr learned my addr 172.21.15.140:0/3009504840 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:12.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654514a700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65400b4d60 0x7f6540145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654594b700 1 --2- 172.21.15.140:0/3009504840 >> v2:172.21.15.152:3300/0 conn(0x7f654009df40 0x7f6540145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561575875 0 0) 0x7f65400607e0 con 0x7f6540060ee0 2022-07-21T01:57:12.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6528003260 con 0x7f6540060ee0 2022-07-21T01:57:12.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4230272487 0 0) 0x7f6528003260 con 0x7f6540060ee0 2022-07-21T01:57:12.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65400b4d60 msgr2=0x7f6540145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:12.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65400b4d60 0x7f6540145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 >> v2:172.21.15.152:3300/0 conn(0x7f654009df40 msgr2=0x7f6540145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:12.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 --2- 172.21.15.140:0/3009504840 >> v2:172.21.15.152:3300/0 conn(0x7f654009df40 0x7f6540145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654009e320 con 0x7f6540060ee0 2022-07-21T01:57:12.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f65380019e0 con 0x7f6540060ee0 2022-07-21T01:57:12.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654594b700 1 --2- 172.21.15.140:0/3009504840 >> v2:172.21.15.152:3300/0 conn(0x7f654009df40 0x7f6540145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:12.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 >> v1:172.21.15.140:6789/0 conn(0x7f6540060ee0 legacy=0x7f6540146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:12.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65400b4d60 0x7f6528003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6528003c80 0x7f6528004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65280049f0 0x7f6528004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6540060d20 con 0x7f6528003c80 2022-07-21T01:57:12.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6540060a80 con 0x7f65280049f0 2022-07-21T01:57:12.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652800b750 con 0x7f65400b4d60 2022-07-21T01:57:12.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654514a700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6528003c80 0x7f6528004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654594b700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65400b4d60 0x7f6528003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654614c700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65280049f0 0x7f6528004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654514a700 1 -- 172.21.15.140:0/3009504840 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65400b4d60 msgr2=0x7f6528003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:12.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654514a700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65400b4d60 0x7f6528003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654514a700 1 -- 172.21.15.140:0/3009504840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65280049f0 msgr2=0x7f6528004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:12.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654514a700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65280049f0 0x7f6528004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.252+0000 7f654514a700 1 -- 172.21.15.140:0/3009504840 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6528003260 con 0x7f6528003c80 2022-07-21T01:57:12.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654514a700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6528003c80 0x7f6528004500 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f65280054d0 tx=0x7f6530006df0).ready entity=mon.0 client_cookie=2987be29f78d240a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:12.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f653000e830 con 0x7f6528003c80 2022-07-21T01:57:12.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653000e990 con 0x7f6528003c80 2022-07-21T01:57:12.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6544949700 1 -- 172.21.15.140:0/3009504840 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f653000ecf0 con 0x7f6528003c80 2022-07-21T01:57:12.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 -- 172.21.15.140:0/3009504840 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6528003c80 msgr2=0x7f6528004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:12.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 --2- 172.21.15.140:0/3009504840 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6528003c80 0x7f6528004500 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f65280054d0 tx=0x7f6530006df0).stop 2022-07-21T01:57:12.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654614c700 1 -- 172.21.15.140:0/3009504840 reap_dead start 2022-07-21T01:57:12.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 -- 172.21.15.140:0/3009504840 shutdown_connections 2022-07-21T01:57:12.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 -- 172.21.15.140:0/3009504840 >> 172.21.15.140:0/3009504840 conn(0x7f65400b5f30 msgr2=0x7f65400b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:12.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 -- 172.21.15.140:0/3009504840 shutdown_connections 2022-07-21T01:57:12.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 -- 172.21.15.140:0/3009504840 wait complete. 2022-07-21T01:57:12.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 Processor -- start 2022-07-21T01:57:12.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 -- start start 2022-07-21T01:57:12.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6528003c80 0x7f65400b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65400b2af0 0x7f65400b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f654009b0f0 0x7f654009b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654008fe20 con 0x7f654009b0f0 2022-07-21T01:57:12.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65400916a0 con 0x7f65400b2af0 2022-07-21T01:57:12.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f6547baf700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6540091940 con 0x7f6528003c80 2022-07-21T01:57:12.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654614c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f654009b0f0 0x7f654009b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654594b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6528003c80 0x7f65400b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654614c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f654009b0f0 0x7f654009b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40268/0 (socket says 172.21.15.140:40268) 2022-07-21T01:57:12.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654594b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6528003c80 0x7f65400b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:36994/0 (socket says 172.21.15.140:36994) 2022-07-21T01:57:12.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654614c700 1 -- 172.21.15.140:0/2643271794 learned_addr learned my addr 172.21.15.140:0/2643271794 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:12.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654514a700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65400b2af0 0x7f65400b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654614c700 1 -- 172.21.15.140:0/2643271794 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6528003c80 msgr2=0x7f65400b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:12.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654614c700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6528003c80 0x7f65400b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654614c700 1 -- 172.21.15.140:0/2643271794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65400b2af0 msgr2=0x7f65400b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:12.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654614c700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65400b2af0 0x7f65400b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:12.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654614c700 1 -- 172.21.15.140:0/2643271794 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6540091be0 con 0x7f654009b0f0 2022-07-21T01:57:12.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.256+0000 7f654614c700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f654009b0f0 0x7f654009b570 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f6538000b60 tx=0x7f6540060360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:12.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.260+0000 7f652effd700 1 -- 172.21.15.140:0/2643271794 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6538011780 con 0x7f654009b0f0 2022-07-21T01:57:12.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.260+0000 7f652effd700 1 -- 172.21.15.140:0/2643271794 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65380118e0 con 0x7f654009b0f0 2022-07-21T01:57:12.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.260+0000 7f6547baf700 1 -- 172.21.15.140:0/2643271794 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654009ba60 con 0x7f654009b0f0 2022-07-21T01:57:12.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.260+0000 7f652effd700 1 -- 172.21.15.140:0/2643271794 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6538011c40 con 0x7f654009b0f0 2022-07-21T01:57:12.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.260+0000 7f6547baf700 1 -- 172.21.15.140:0/2643271794 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65400b1230 con 0x7f654009b0f0 2022-07-21T01:57:12.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.260+0000 7f6547baf700 1 -- 172.21.15.140:0/2643271794 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6540145b70 con 0x7f654009b0f0 2022-07-21T01:57:12.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.260+0000 7f652effd700 1 -- 172.21.15.140:0/2643271794 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f653801c360 con 0x7f654009b0f0 2022-07-21T01:57:12.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.260+0000 7f652effd700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6520053a10 0x7f6520055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:12.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.260+0000 7f654594b700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6520053a10 0x7f6520055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:12.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.260+0000 7f652effd700 1 -- 172.21.15.140:0/2643271794 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f653806b690 con 0x7f654009b0f0 2022-07-21T01:57:12.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.264+0000 7f654594b700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6520053a10 0x7f6520055ed0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f653c008000 tx=0x7f653c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:12.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.268+0000 7f652effd700 1 -- 172.21.15.140:0/2643271794 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6538042800 con 0x7f654009b0f0 2022-07-21T01:57:12.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:12.384+0000 7f6547baf700 1 -- 172.21.15.140:0/2643271794 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f65400574e0 con 0x7f654009b0f0 2022-07-21T01:57:13.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.104+0000 7f652effd700 1 -- 172.21.15.140:0/2643271794 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v129) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6538019020 con 0x7f654009b0f0 2022-07-21T01:57:13.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.108+0000 7f6547baf700 1 -- 172.21.15.140:0/2643271794 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6520053a10 msgr2=0x7f6520055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:13.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.108+0000 7f6547baf700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6520053a10 0x7f6520055ed0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f653c008000 tx=0x7f653c008040).stop 2022-07-21T01:57:13.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.108+0000 7f6547baf700 1 -- 172.21.15.140:0/2643271794 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f654009b0f0 msgr2=0x7f654009b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:13.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.108+0000 7f6547baf700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f654009b0f0 0x7f654009b570 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f6538000b60 tx=0x7f6540060360).stop 2022-07-21T01:57:13.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.108+0000 7f6547baf700 1 -- 172.21.15.140:0/2643271794 shutdown_connections 2022-07-21T01:57:13.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.108+0000 7f6547baf700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6520053a10 0x7f6520055ed0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:13.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.108+0000 7f6547baf700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f654009b0f0 0x7f654009b570 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:13.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.108+0000 7f6547baf700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65400b2af0 0x7f65400b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:13.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.108+0000 7f6547baf700 1 --2- 172.21.15.140:0/2643271794 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6528003c80 0x7f65400b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:13.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.108+0000 7f6547baf700 1 -- 172.21.15.140:0/2643271794 >> 172.21.15.140:0/2643271794 conn(0x7f65400b5f30 msgr2=0x7f654005cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:13.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.112+0000 7f6547baf700 1 -- 172.21.15.140:0/2643271794 shutdown_connections 2022-07-21T01:57:13.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.112+0000 7f6547baf700 1 -- 172.21.15.140:0/2643271794 wait complete. 2022-07-21T01:57:13.115 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:57:13.127 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:57:13.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.228+0000 7f4b50f5a700 1 Processor -- start 2022-07-21T01:57:13.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.228+0000 7f4b50f5a700 1 -- start start 2022-07-21T01:57:13.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.228+0000 7f4b50f5a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4b4c0b1600 0x7f4b4c05fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:13.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.228+0000 7f4b50f5a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b4c05bd90 0x7f4b4c061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:13.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.228+0000 7f4b50f5a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4c0b19e0 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.228+0000 7f4b50f5a700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4c061af0 con 0x7f4b4c0b1600 2022-07-21T01:57:13.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.228+0000 7f4b50f5a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c0b1f40 con 0x7f4b4c05bd90 2022-07-21T01:57:13.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b4a59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4b4c0b1600 0x7f4b4c05fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:13.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b4a59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f4b4c0b1600 0x7f4b4c05fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40272/0 (socket says 172.21.15.140:40272) 2022-07-21T01:57:13.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b4a59c700 1 -- 172.21.15.140:0/1069701495 learned_addr learned my addr 172.21.15.140:0/1069701495 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:13.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b43fff700 1 --2- 172.21.15.140:0/1069701495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b4c05bd90 0x7f4b4c061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:13.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b4a59c700 1 -- 172.21.15.140:0/1069701495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b4c05bd90 msgr2=0x7f4b4c061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:13.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b4a59c700 1 --2- 172.21.15.140:0/1069701495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b4c05bd90 0x7f4b4c061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:13.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b4a59c700 1 -- 172.21.15.140:0/1069701495 >> v1:172.21.15.152:6789/0 conn(0x7f4b4c0aa9d0 legacy=0x7f4b4c1420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:13.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b4a59c700 1 -- 172.21.15.140:0/1069701495 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c061160 con 0x7f4b4c0b1600 2022-07-21T01:57:13.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b4a59c700 1 --2- 172.21.15.140:0/1069701495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4b4c0b1600 0x7f4b4c05fe10 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f4b3c002f70 tx=0x7f4b3c006eb0).ready entity=mon.0 client_cookie=a8ca81c36af34757 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:13.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b49d9b700 1 -- 172.21.15.140:0/1069701495 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b3c00d570 con 0x7f4b4c0b1600 2022-07-21T01:57:13.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b49d9b700 1 -- 172.21.15.140:0/1069701495 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c000e90 con 0x7f4b4c0b1600 2022-07-21T01:57:13.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b49d9b700 1 -- 172.21.15.140:0/1069701495 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b3c00d900 con 0x7f4b4c0b1600 2022-07-21T01:57:13.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b50f5a700 1 -- 172.21.15.140:0/1069701495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4b4c0b1600 msgr2=0x7f4b4c05fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:13.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b50f5a700 1 --2- 172.21.15.140:0/1069701495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4b4c0b1600 0x7f4b4c05fe10 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f4b3c002f70 tx=0x7f4b3c006eb0).stop 2022-07-21T01:57:13.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b50f5a700 1 -- 172.21.15.140:0/1069701495 shutdown_connections 2022-07-21T01:57:13.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b50f5a700 1 --2- 172.21.15.140:0/1069701495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b4c05bd90 0x7f4b4c061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:13.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b50f5a700 1 --2- 172.21.15.140:0/1069701495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4b4c0b1600 0x7f4b4c05fe10 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:13.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b50f5a700 1 -- 172.21.15.140:0/1069701495 >> 172.21.15.140:0/1069701495 conn(0x7f4b4c0b23b0 msgr2=0x7f4b4c0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:13.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b50f5a700 1 -- 172.21.15.140:0/1069701495 shutdown_connections 2022-07-21T01:57:13.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b50f5a700 1 -- 172.21.15.140:0/1069701495 wait complete. 2022-07-21T01:57:13.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b50f5a700 1 Processor -- start 2022-07-21T01:57:13.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.232+0000 7f4b50f5a700 1 -- start start 2022-07-21T01:57:13.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b50f5a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b4c05bd90 0x7f4b4c0acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:13.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b50f5a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4b4c0aa9d0 0x7f4b4c0ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:13.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b50f5a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c0b1600 0x7f4b4c152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:13.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b50f5a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c151f30 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b50f5a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c152160 con 0x7f4b4c0b1600 2022-07-21T01:57:13.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b50f5a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4c0ad400 con 0x7f4b4c05bd90 2022-07-21T01:57:13.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b43fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4b4c0aa9d0 0x7f4b4c0ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:13.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b43fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4b4c0aa9d0 0x7f4b4c0ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40280/0 (socket says 172.21.15.140:40280) 2022-07-21T01:57:13.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b43fff700 1 -- 172.21.15.140:0/2584516358 learned_addr learned my addr 172.21.15.140:0/2584516358 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:13.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b4ad9d700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c0b1600 0x7f4b4c152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:13.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b4a59c700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b4c05bd90 0x7f4b4c0acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:13.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b43fff700 1 -- 172.21.15.140:0/2584516358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b4c05bd90 msgr2=0x7f4b4c0acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:13.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b43fff700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b4c05bd90 0x7f4b4c0acf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:13.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b43fff700 1 -- 172.21.15.140:0/2584516358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c0b1600 msgr2=0x7f4b4c152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:13.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b43fff700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c0b1600 0x7f4b4c152600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:13.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b43fff700 1 -- 172.21.15.140:0/2584516358 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c00b1e0 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b43fff700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4b4c0aa9d0 0x7f4b4c0ae820 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f4b3400c000 tx=0x7f4b34006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:13.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b437fe700 1 -- 172.21.15.140:0/2584516358 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b34010780 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b437fe700 1 -- 172.21.15.140:0/2584516358 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b340108e0 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b50f5a700 1 -- 172.21.15.140:0/2584516358 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4c144990 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b437fe700 1 -- 172.21.15.140:0/2584516358 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b34010c40 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b50f5a700 1 -- 172.21.15.140:0/2584516358 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4c146170 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.236+0000 7f4b50f5a700 1 -- 172.21.15.140:0/2584516358 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4c0aeeb0 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.240+0000 7f4b437fe700 1 -- 172.21.15.140:0/2584516358 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4b3401c460 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.240+0000 7f4b437fe700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b30053a70 0x7f4b30055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:13.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.240+0000 7f4b4a59c700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b30053a70 0x7f4b30055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:13.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.240+0000 7f4b437fe700 1 -- 172.21.15.140:0/2584516358 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f4b3406c270 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.240+0000 7f4b4a59c700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b30053a70 0x7f4b30055f30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f4b3c01a000 tx=0x7f4b3c015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:13.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.244+0000 7f4b437fe700 1 -- 172.21.15.140:0/2584516358 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4b34043260 con 0x7f4b4c0aa9d0 2022-07-21T01:57:13.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:13.368+0000 7f4b50f5a700 1 -- 172.21.15.140:0/2584516358 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f4b4c152320 con 0x7f4b4c0aa9d0 2022-07-21T01:57:14.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.112+0000 7f4b437fe700 1 -- 172.21.15.140:0/2584516358 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v130) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4b34046880 con 0x7f4b4c0aa9d0 2022-07-21T01:57:14.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.120+0000 7f4b50f5a700 1 -- 172.21.15.140:0/2584516358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b30053a70 msgr2=0x7f4b30055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.120+0000 7f4b50f5a700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b30053a70 0x7f4b30055f30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f4b3c01a000 tx=0x7f4b3c015000).stop 2022-07-21T01:57:14.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.120+0000 7f4b50f5a700 1 -- 172.21.15.140:0/2584516358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4b4c0aa9d0 msgr2=0x7f4b4c0ae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.120+0000 7f4b50f5a700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4b4c0aa9d0 0x7f4b4c0ae820 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f4b3400c000 tx=0x7f4b34006df0).stop 2022-07-21T01:57:14.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.120+0000 7f4b50f5a700 1 -- 172.21.15.140:0/2584516358 shutdown_connections 2022-07-21T01:57:14.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.120+0000 7f4b50f5a700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4b30053a70 0x7f4b30055f30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.120+0000 7f4b50f5a700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4c0b1600 0x7f4b4c152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.120+0000 7f4b50f5a700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4b4c0aa9d0 0x7f4b4c0ae820 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.120+0000 7f4b50f5a700 1 --2- 172.21.15.140:0/2584516358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b4c05bd90 0x7f4b4c0acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.120+0000 7f4b50f5a700 1 -- 172.21.15.140:0/2584516358 >> 172.21.15.140:0/2584516358 conn(0x7f4b4c0b23b0 msgr2=0x7f4b4c0aafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:14.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.124+0000 7f4b50f5a700 1 -- 172.21.15.140:0/2584516358 shutdown_connections 2022-07-21T01:57:14.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.124+0000 7f4b50f5a700 1 -- 172.21.15.140:0/2584516358 wait complete. 2022-07-21T01:57:14.126 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:57:14.136 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:57:14.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf21d6d700 1 Processor -- start 2022-07-21T01:57:14.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf21d6d700 1 -- start start 2022-07-21T01:57:14.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf21d6d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcf1c05d2c0 0x7fcf1c144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf21d6d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcf1c144e60 0x7fcf1c143010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf21d6d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1c05fd60 con 0x7fcf1c0b3560 2022-07-21T01:57:14.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf21d6d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1c05ccb0 con 0x7fcf1c144e60 2022-07-21T01:57:14.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf21d6d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1c0b3940 con 0x7fcf1c05d2c0 2022-07-21T01:57:14.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf2156c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcf1c144e60 0x7fcf1c143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf2156c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcf1c144e60 0x7fcf1c143010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40288/0 (socket says 172.21.15.140:40288) 2022-07-21T01:57:14.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf2156c700 1 -- 172.21.15.140:0/1579745189 learned_addr learned my addr 172.21.15.140:0/1579745189 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:14.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf1bfff700 1 --2- 172.21.15.140:0/1579745189 >> v2:172.21.15.152:3300/0 conn(0x7fcf1c05d2c0 0x7fcf1c144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf2156c700 1 -- 172.21.15.140:0/1579745189 >> v2:172.21.15.152:3300/0 conn(0x7fcf1c05d2c0 msgr2=0x7fcf1c144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:14.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf2156c700 1 --2- 172.21.15.140:0/1579745189 >> v2:172.21.15.152:3300/0 conn(0x7fcf1c05d2c0 0x7fcf1c144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf2156c700 1 -- 172.21.15.140:0/1579745189 >> v1:172.21.15.140:6789/0 conn(0x7fcf1c0b3560 legacy=0x7fcf1c144260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf2156c700 1 -- 172.21.15.140:0/1579745189 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c099df0 con 0x7fcf1c144e60 2022-07-21T01:57:14.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf2156c700 1 --2- 172.21.15.140:0/1579745189 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1c144e60 0x7fcf1c143010 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7fcf140032f0 tx=0x7fcf14009290).ready entity=mon.0 client_cookie=7babd8342f5123c7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:14.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.240+0000 7fcf1b7fe700 1 -- 172.21.15.140:0/1579745189 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf1400b870 con 0x7fcf1c144e60 2022-07-21T01:57:14.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf1b7fe700 1 -- 172.21.15.140:0/1579745189 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1400b9d0 con 0x7fcf1c144e60 2022-07-21T01:57:14.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf1b7fe700 1 -- 172.21.15.140:0/1579745189 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf1400bb40 con 0x7fcf1c144e60 2022-07-21T01:57:14.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 -- 172.21.15.140:0/1579745189 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1c144e60 msgr2=0x7fcf1c143010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 --2- 172.21.15.140:0/1579745189 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1c144e60 0x7fcf1c143010 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7fcf140032f0 tx=0x7fcf14009290).stop 2022-07-21T01:57:14.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 -- 172.21.15.140:0/1579745189 shutdown_connections 2022-07-21T01:57:14.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 --2- 172.21.15.140:0/1579745189 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1c144e60 0x7fcf1c143010 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 --2- 172.21.15.140:0/1579745189 >> v2:172.21.15.152:3300/0 conn(0x7fcf1c05d2c0 0x7fcf1c144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 -- 172.21.15.140:0/1579745189 >> 172.21.15.140:0/1579745189 conn(0x7fcf1c0b6b20 msgr2=0x7fcf1c0b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:14.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 -- 172.21.15.140:0/1579745189 shutdown_connections 2022-07-21T01:57:14.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 -- 172.21.15.140:0/1579745189 wait complete. 2022-07-21T01:57:14.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 Processor -- start 2022-07-21T01:57:14.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 -- start start 2022-07-21T01:57:14.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1c05d2c0 0x7fcf1c146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1c0b3560 0x7fcf1c0ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf1c0b15e0 0x7fcf1c05fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0b01d0 con 0x7fcf1c0b3560 2022-07-21T01:57:14.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0ab6d0 con 0x7fcf1c0b15e0 2022-07-21T01:57:14.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf21d6d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c0b0ef0 con 0x7fcf1c05d2c0 2022-07-21T01:57:14.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf1bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1c0b3560 0x7fcf1c0ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf1bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1c0b3560 0x7fcf1c0ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40296/0 (socket says 172.21.15.140:40296) 2022-07-21T01:57:14.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf20d6b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1c05d2c0 0x7fcf1c146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf1bfff700 1 -- 172.21.15.140:0/2547600297 learned_addr learned my addr 172.21.15.140:0/2547600297 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:14.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf2156c700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf1c0b15e0 0x7fcf1c05fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf1bfff700 1 -- 172.21.15.140:0/2547600297 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1c05d2c0 msgr2=0x7fcf1c146d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf1bfff700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1c05d2c0 0x7fcf1c146d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf1bfff700 1 -- 172.21.15.140:0/2547600297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf1c0b15e0 msgr2=0x7fcf1c05fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf1bfff700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf1c0b15e0 0x7fcf1c05fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf1bfff700 1 -- 172.21.15.140:0/2547600297 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf14009530 con 0x7fcf1c0b3560 2022-07-21T01:57:14.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.244+0000 7fcf20d6b700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1c05d2c0 0x7fcf1c146d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:14.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.248+0000 7fcf1bfff700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1c0b3560 0x7fcf1c0ac140 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7fcf0c00c000 tx=0x7fcf0c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:14.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.248+0000 7fcf19ffb700 1 -- 172.21.15.140:0/2547600297 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf0c010780 con 0x7fcf1c0b3560 2022-07-21T01:57:14.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.248+0000 7fcf19ffb700 1 -- 172.21.15.140:0/2547600297 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c0108e0 con 0x7fcf1c0b3560 2022-07-21T01:57:14.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.248+0000 7fcf21d6d700 1 -- 172.21.15.140:0/2547600297 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c0b1190 con 0x7fcf1c0b3560 2022-07-21T01:57:14.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.248+0000 7fcf19ffb700 1 -- 172.21.15.140:0/2547600297 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf0c010c40 con 0x7fcf1c0b3560 2022-07-21T01:57:14.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.248+0000 7fcf21d6d700 1 -- 172.21.15.140:0/2547600297 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c0afe50 con 0x7fcf1c0b3560 2022-07-21T01:57:14.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.248+0000 7fcf21d6d700 1 -- 172.21.15.140:0/2547600297 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1c0b1320 con 0x7fcf1c0b3560 2022-07-21T01:57:14.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.248+0000 7fcf19ffb700 1 -- 172.21.15.140:0/2547600297 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcf0c01c460 con 0x7fcf1c0b3560 2022-07-21T01:57:14.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.252+0000 7fcf19ffb700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcefc053b00 0x7fcefc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.252+0000 7fcf19ffb700 1 -- 172.21.15.140:0/2547600297 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fcf0c06c390 con 0x7fcf1c0b3560 2022-07-21T01:57:14.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.252+0000 7fcf20d6b700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcefc053b00 0x7fcefc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.252+0000 7fcf20d6b700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcefc053b00 0x7fcefc055fc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fcf1000d000 tx=0x7fcf10009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:14.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.256+0000 7fcf19ffb700 1 -- 172.21.15.140:0/2547600297 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcf0c0431b0 con 0x7fcf1c0b3560 2022-07-21T01:57:14.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.420+0000 7fcf21d6d700 1 -- 172.21.15.140:0/2547600297 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fcf1c05cea0 con 0x7fcf1c0b3560 2022-07-21T01:57:14.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.428+0000 7fcf19ffb700 1 -- 172.21.15.140:0/2547600297 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 25 and data pool 26 v62) v1 ==== 189+0+0 (secure 0 0 0) 0x7fcf0c0467d0 con 0x7fcf1c0b3560 2022-07-21T01:57:14.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 -- 172.21.15.140:0/2547600297 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcefc053b00 msgr2=0x7fcefc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcefc053b00 0x7fcefc055fc0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fcf1000d000 tx=0x7fcf10009040).stop 2022-07-21T01:57:14.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 -- 172.21.15.140:0/2547600297 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1c0b3560 msgr2=0x7fcf1c0ac140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1c0b3560 0x7fcf1c0ac140 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7fcf0c00c000 tx=0x7fcf0c006df0).stop 2022-07-21T01:57:14.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 -- 172.21.15.140:0/2547600297 shutdown_connections 2022-07-21T01:57:14.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcefc053b00 0x7fcefc055fc0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf1c0b15e0 0x7fcf1c05fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf1c0b3560 0x7fcf1c0ac140 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 --2- 172.21.15.140:0/2547600297 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf1c05d2c0 0x7fcf1c146d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 -- 172.21.15.140:0/2547600297 >> 172.21.15.140:0/2547600297 conn(0x7fcf1c0b6b20 msgr2=0x7fcf1c0b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:14.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 -- 172.21.15.140:0/2547600297 shutdown_connections 2022-07-21T01:57:14.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.440+0000 7fcf21d6d700 1 -- 172.21.15.140:0/2547600297 wait complete. 2022-07-21T01:57:14.447 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 25 and data pool 26 2022-07-21T01:57:14.454 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:14.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.544+0000 7f6609b6d700 1 Processor -- start 2022-07-21T01:57:14.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.544+0000 7f6609b6d700 1 -- start start 2022-07-21T01:57:14.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 0x7f660414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f660405c3f0 0x7f66040ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660405d480 con 0x7f660405bd90 2022-07-21T01:57:14.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66040b21e0 con 0x7f660405c3f0 2022-07-21T01:57:14.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660405cf40 con 0x7f66040b1e00 2022-07-21T01:57:14.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6603fff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f660405bd90 0x7f6604146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54458/0 (socket says 172.21.15.140:54458) 2022-07-21T01:57:14.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6603fff700 1 -- 172.21.15.140:0/2134982255 learned_addr learned my addr 172.21.15.140:0/2134982255 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:14.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6602ffd700 1 --2- 172.21.15.140:0/2134982255 >> v2:172.21.15.152:3300/0 conn(0x7f660405c3f0 0x7f66040ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f66037fe700 1 --2- 172.21.15.140:0/2134982255 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 0x7f660414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f66027fc700 1 -- 172.21.15.140:0/2134982255 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44647947 0 0) 0x7f660405d480 con 0x7f660405bd90 2022-07-21T01:57:14.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f66027fc700 1 -- 172.21.15.140:0/2134982255 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65f0003260 con 0x7f660405bd90 2022-07-21T01:57:14.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f66037fe700 1 -- 172.21.15.140:0/2134982255 >> v2:172.21.15.152:3300/0 conn(0x7f660405c3f0 msgr2=0x7f66040ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:14.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f66037fe700 1 --2- 172.21.15.140:0/2134982255 >> v2:172.21.15.152:3300/0 conn(0x7f660405c3f0 0x7f66040ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f66037fe700 1 -- 172.21.15.140:0/2134982255 >> v1:172.21.15.140:6789/0 conn(0x7f660405bd90 legacy=0x7f6604146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f66037fe700 1 -- 172.21.15.140:0/2134982255 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660405d1e0 con 0x7f66040b1e00 2022-07-21T01:57:14.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f66037fe700 1 --2- 172.21.15.140:0/2134982255 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 0x7f660414a6d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f65fc0032f0 tx=0x7f65fc0091d0).ready entity=mon.2 client_cookie=55c02f09a1a0d3cd server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:14.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f66027fc700 1 -- 172.21.15.140:0/2134982255 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65fc00b600 con 0x7f66040b1e00 2022-07-21T01:57:14.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f66027fc700 1 -- 172.21.15.140:0/2134982255 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc00b760 con 0x7f66040b1e00 2022-07-21T01:57:14.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f66027fc700 1 -- 172.21.15.140:0/2134982255 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65fc00b8d0 con 0x7f66040b1e00 2022-07-21T01:57:14.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 -- 172.21.15.140:0/2134982255 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 msgr2=0x7f660414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 --2- 172.21.15.140:0/2134982255 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 0x7f660414a6d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f65fc0032f0 tx=0x7f65fc0091d0).stop 2022-07-21T01:57:14.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 -- 172.21.15.140:0/2134982255 shutdown_connections 2022-07-21T01:57:14.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 --2- 172.21.15.140:0/2134982255 >> v2:172.21.15.152:3300/0 conn(0x7f660405c3f0 0x7f66040ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 --2- 172.21.15.140:0/2134982255 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 0x7f660414a6d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 -- 172.21.15.140:0/2134982255 >> 172.21.15.140:0/2134982255 conn(0x7f66040611c0 msgr2=0x7f6604062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:14.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 -- 172.21.15.140:0/2134982255 shutdown_connections 2022-07-21T01:57:14.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.548+0000 7f6609b6d700 1 -- 172.21.15.140:0/2134982255 wait complete. 2022-07-21T01:57:14.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6609b6d700 1 Processor -- start 2022-07-21T01:57:14.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6609b6d700 1 -- start start 2022-07-21T01:57:14.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6609b6d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f660405c3f0 0x7f66040b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6609b6d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 0x7f660409fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6609b6d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f660409f7d0 0x7f6604063be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6609b6d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660409e520 con 0x7f660409f7d0 2022-07-21T01:57:14.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6609b6d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660409e7c0 con 0x7f660405c3f0 2022-07-21T01:57:14.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6609b6d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660409ea60 con 0x7f66040b1e00 2022-07-21T01:57:14.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6603fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f660409f7d0 0x7f6604063be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f66037fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f660405c3f0 0x7f66040b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6603fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f660409f7d0 0x7f6604063be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40320/0 (socket says 172.21.15.140:40320) 2022-07-21T01:57:14.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6602ffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 0x7f660409fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6603fff700 1 -- 172.21.15.140:0/340897884 learned_addr learned my addr 172.21.15.140:0/340897884 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:14.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6603fff700 1 -- 172.21.15.140:0/340897884 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 msgr2=0x7f660409fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6603fff700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 0x7f660409fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6603fff700 1 -- 172.21.15.140:0/340897884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f660405c3f0 msgr2=0x7f66040b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6603fff700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f660405c3f0 0x7f66040b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6603fff700 1 -- 172.21.15.140:0/340897884 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc009470 con 0x7f660409f7d0 2022-07-21T01:57:14.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f66037fe700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f660405c3f0 0x7f66040b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:14.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6602ffd700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 0x7f660409fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:14.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6603fff700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f660409f7d0 0x7f6604063be0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f65f4001110 tx=0x7f65f4007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:14.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6600ff9700 1 -- 172.21.15.140:0/340897884 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65f40070d0 con 0x7f660409f7d0 2022-07-21T01:57:14.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6600ff9700 1 -- 172.21.15.140:0/340897884 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f4007230 con 0x7f660409f7d0 2022-07-21T01:57:14.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6609b6d700 1 -- 172.21.15.140:0/340897884 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66040ae880 con 0x7f660409f7d0 2022-07-21T01:57:14.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6600ff9700 1 -- 172.21.15.140:0/340897884 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65f4005610 con 0x7f660409f7d0 2022-07-21T01:57:14.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.552+0000 7f6609b6d700 1 -- 172.21.15.140:0/340897884 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6604144e20 con 0x7f660409f7d0 2022-07-21T01:57:14.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.556+0000 7f6609b6d700 1 -- 172.21.15.140:0/340897884 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66040b04b0 con 0x7f660409f7d0 2022-07-21T01:57:14.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.556+0000 7f6600ff9700 1 -- 172.21.15.140:0/340897884 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f65f4010070 con 0x7f660409f7d0 2022-07-21T01:57:14.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.556+0000 7f6600ff9700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65e40539e0 0x7f65e4055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.556+0000 7f6600ff9700 1 -- 172.21.15.140:0/340897884 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f65f406b6e0 con 0x7f660409f7d0 2022-07-21T01:57:14.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.556+0000 7f66037fe700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65e40539e0 0x7f65e4055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.556+0000 7f66037fe700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65e40539e0 0x7f65e4055ea0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f65fc01b000 tx=0x7f65fc016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:14.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.560+0000 7f6600ff9700 1 -- 172.21.15.140:0/340897884 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f65f40424f0 con 0x7f660409f7d0 2022-07-21T01:57:14.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.684+0000 7f6609b6d700 1 -- 172.21.15.140:0/340897884 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66040574e0 con 0x7f660409f7d0 2022-07-21T01:57:14.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.688+0000 7f6600ff9700 1 -- 172.21.15.140:0/340897884 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f65f4045b10 con 0x7f660409f7d0 2022-07-21T01:57:14.690 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:14.691 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":131,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:14.423472+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:57:12.443397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-07-21T01:57:13.427432+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:14.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 -- 172.21.15.140:0/340897884 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65e40539e0 msgr2=0x7f65e4055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65e40539e0 0x7f65e4055ea0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f65fc01b000 tx=0x7f65fc016000).stop 2022-07-21T01:57:14.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 -- 172.21.15.140:0/340897884 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f660409f7d0 msgr2=0x7f6604063be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f660409f7d0 0x7f6604063be0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f65f4001110 tx=0x7f65f4007a80).stop 2022-07-21T01:57:14.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 -- 172.21.15.140:0/340897884 shutdown_connections 2022-07-21T01:57:14.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65e40539e0 0x7f65e4055ea0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f660409f7d0 0x7f6604063be0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f66040b1e00 0x7f660409fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 --2- 172.21.15.140:0/340897884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f660405c3f0 0x7f66040b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 -- 172.21.15.140:0/340897884 >> 172.21.15.140:0/340897884 conn(0x7f660405bd90 msgr2=0x7f66040b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:14.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 -- 172.21.15.140:0/340897884 shutdown_connections 2022-07-21T01:57:14.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.692+0000 7f6609b6d700 1 -- 172.21.15.140:0/340897884 wait complete. 2022-07-21T01:57:14.704 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:14.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.804+0000 7f2b65494700 1 Processor -- start 2022-07-21T01:57:14.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.804+0000 7f2b65494700 1 -- start start 2022-07-21T01:57:14.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.804+0000 7f2b65494700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2b600b1520 0x7f2b60066870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.804+0000 7f2b65494700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b60066d60 0x7f2b6012aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.804+0000 7f2b65494700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b600b7f50 con 0x7f2b600b7b70 2022-07-21T01:57:14.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.804+0000 7f2b65494700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b60146120 con 0x7f2b600b1520 2022-07-21T01:57:14.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.804+0000 7f2b65494700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b601463c0 con 0x7f2b60066d60 2022-07-21T01:57:14.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b5f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2b600b1520 0x7f2b60066870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b5f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2b600b1520 0x7f2b60066870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40324/0 (socket says 172.21.15.140:40324) 2022-07-21T01:57:14.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b5f7fe700 1 -- 172.21.15.140:0/1608317506 learned_addr learned my addr 172.21.15.140:0/1608317506 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:14.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b64c93700 1 --2- 172.21.15.140:0/1608317506 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b60066d60 0x7f2b6012aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b5f7fe700 1 -- 172.21.15.140:0/1608317506 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b60066d60 msgr2=0x7f2b6012aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:14.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b5f7fe700 1 --2- 172.21.15.140:0/1608317506 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b60066d60 0x7f2b6012aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b5f7fe700 1 -- 172.21.15.140:0/1608317506 >> v1:172.21.15.152:6789/0 conn(0x7f2b600b7b70 legacy=0x7f2b60146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b5f7fe700 1 -- 172.21.15.140:0/1608317506 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6012c130 con 0x7f2b600b1520 2022-07-21T01:57:14.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b64c93700 1 --2- 172.21.15.140:0/1608317506 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b60066d60 0x7f2b6012aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:14.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b5f7fe700 1 --2- 172.21.15.140:0/1608317506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b600b1520 0x7f2b60066870 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f2b580032f0 tx=0x7f2b58009290).ready entity=mon.0 client_cookie=31a29b548e7f227 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:14.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b5effd700 1 -- 172.21.15.140:0/1608317506 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b5800b8a0 con 0x7f2b600b1520 2022-07-21T01:57:14.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b5effd700 1 -- 172.21.15.140:0/1608317506 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5800ba00 con 0x7f2b600b1520 2022-07-21T01:57:14.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b5effd700 1 -- 172.21.15.140:0/1608317506 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b5800bb70 con 0x7f2b600b1520 2022-07-21T01:57:14.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b65494700 1 -- 172.21.15.140:0/1608317506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b600b1520 msgr2=0x7f2b60066870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b65494700 1 --2- 172.21.15.140:0/1608317506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b600b1520 0x7f2b60066870 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f2b580032f0 tx=0x7f2b58009290).stop 2022-07-21T01:57:14.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b65494700 1 -- 172.21.15.140:0/1608317506 shutdown_connections 2022-07-21T01:57:14.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b65494700 1 --2- 172.21.15.140:0/1608317506 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b60066d60 0x7f2b6012aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b65494700 1 --2- 172.21.15.140:0/1608317506 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b600b1520 0x7f2b60066870 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b65494700 1 -- 172.21.15.140:0/1608317506 >> 172.21.15.140:0/1608317506 conn(0x7f2b600656b0 msgr2=0x7f2b600b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:14.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b65494700 1 -- 172.21.15.140:0/1608317506 shutdown_connections 2022-07-21T01:57:14.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.808+0000 7f2b65494700 1 -- 172.21.15.140:0/1608317506 wait complete. 2022-07-21T01:57:14.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b65494700 1 Processor -- start 2022-07-21T01:57:14.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b65494700 1 -- start start 2022-07-21T01:57:14.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b65494700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b60066d60 0x7f2b6012e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b65494700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b600b1520 0x7f2b600b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b65494700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b600b7b70 0x7f2b60127de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b65494700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6012e880 con 0x7f2b600b1520 2022-07-21T01:57:14.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b65494700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6012ead0 con 0x7f2b600b7b70 2022-07-21T01:57:14.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b65494700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6012ed70 con 0x7f2b60066d60 2022-07-21T01:57:14.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b5f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b600b1520 0x7f2b600b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b5f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b600b1520 0x7f2b600b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40332/0 (socket says 172.21.15.140:40332) 2022-07-21T01:57:14.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b64c93700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b600b7b70 0x7f2b60127de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b5f7fe700 1 -- 172.21.15.140:0/2246753913 learned_addr learned my addr 172.21.15.140:0/2246753913 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:14.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b5ffff700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b60066d60 0x7f2b6012e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b5f7fe700 1 -- 172.21.15.140:0/2246753913 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b60066d60 msgr2=0x7f2b6012e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b5f7fe700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b60066d60 0x7f2b6012e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b5f7fe700 1 -- 172.21.15.140:0/2246753913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b600b7b70 msgr2=0x7f2b60127de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b5f7fe700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b600b7b70 0x7f2b60127de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b5f7fe700 1 -- 172.21.15.140:0/2246753913 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58009530 con 0x7f2b600b1520 2022-07-21T01:57:14.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.812+0000 7f2b5f7fe700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b600b1520 0x7f2b600b1c60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f2b580032f0 tx=0x7f2b58016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:14.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.816+0000 7f2b5d7fa700 1 -- 172.21.15.140:0/2246753913 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b5800b8a0 con 0x7f2b600b1520 2022-07-21T01:57:14.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.816+0000 7f2b5d7fa700 1 -- 172.21.15.140:0/2246753913 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5800bb70 con 0x7f2b600b1520 2022-07-21T01:57:14.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.816+0000 7f2b65494700 1 -- 172.21.15.140:0/2246753913 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6012f010 con 0x7f2b600b1520 2022-07-21T01:57:14.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.816+0000 7f2b5d7fa700 1 -- 172.21.15.140:0/2246753913 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2b58020620 con 0x7f2b600b1520 2022-07-21T01:57:14.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.816+0000 7f2b65494700 1 -- 172.21.15.140:0/2246753913 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b601278a0 con 0x7f2b600b1520 2022-07-21T01:57:14.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.816+0000 7f2b65494700 1 -- 172.21.15.140:0/2246753913 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b601555b0 con 0x7f2b600b1520 2022-07-21T01:57:14.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.816+0000 7f2b5d7fa700 1 -- 172.21.15.140:0/2246753913 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2b58026020 con 0x7f2b600b1520 2022-07-21T01:57:14.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.816+0000 7f2b5d7fa700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2b44053a10 0x7f2b44055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:14.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.816+0000 7f2b5d7fa700 1 -- 172.21.15.140:0/2246753913 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2b5806cd00 con 0x7f2b600b1520 2022-07-21T01:57:14.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.816+0000 7f2b5ffff700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2b44053a10 0x7f2b44055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:14.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.816+0000 7f2b5ffff700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2b44053a10 0x7f2b44055ed0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f2b5000d000 tx=0x7f2b50007b70).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:14.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.824+0000 7f2b5d7fa700 1 -- 172.21.15.140:0/2246753913 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2b58043c20 con 0x7f2b600b1520 2022-07-21T01:57:14.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.948+0000 7f2b65494700 1 -- 172.21.15.140:0/2246753913 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b60127100 con 0x7f2b600b1520 2022-07-21T01:57:14.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.948+0000 7f2b5d7fa700 1 -- 172.21.15.140:0/2246753913 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f2b60127100 con 0x7f2b600b1520 2022-07-21T01:57:14.950 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:14.950 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":131,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:14.423472+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:57:12.443397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-07-21T01:57:13.427432+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:14.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.948+0000 7f2b65494700 1 -- 172.21.15.140:0/2246753913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2b44053a10 msgr2=0x7f2b44055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.948+0000 7f2b65494700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2b44053a10 0x7f2b44055ed0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f2b5000d000 tx=0x7f2b50007b70).stop 2022-07-21T01:57:14.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.948+0000 7f2b65494700 1 -- 172.21.15.140:0/2246753913 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b600b1520 msgr2=0x7f2b600b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:14.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.948+0000 7f2b65494700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b600b1520 0x7f2b600b1c60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f2b580032f0 tx=0x7f2b58016000).stop 2022-07-21T01:57:14.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.948+0000 7f2b65494700 1 -- 172.21.15.140:0/2246753913 shutdown_connections 2022-07-21T01:57:14.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.952+0000 7f2b65494700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2b44053a10 0x7f2b44055ed0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.952+0000 7f2b65494700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b600b7b70 0x7f2b60127de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.952+0000 7f2b65494700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2b600b1520 0x7f2b600b1c60 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.952+0000 7f2b65494700 1 --2- 172.21.15.140:0/2246753913 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2b60066d60 0x7f2b6012e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:14.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.952+0000 7f2b65494700 1 -- 172.21.15.140:0/2246753913 >> 172.21.15.140:0/2246753913 conn(0x7f2b600656b0 msgr2=0x7f2b60067210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:14.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.952+0000 7f2b65494700 1 -- 172.21.15.140:0/2246753913 shutdown_connections 2022-07-21T01:57:14.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:14.952+0000 7f2b65494700 1 -- 172.21.15.140:0/2246753913 wait complete. 2022-07-21T01:57:14.963 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:57:15.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36de3f5700 1 Processor -- start 2022-07-21T01:57:15.067 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36de3f5700 1 -- start start 2022-07-21T01:57:15.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36de3f5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f36d80b59f0 0x7f36d81473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:15.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36de3f5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d805cc70 con 0x7f36d80af940 2022-07-21T01:57:15.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36de3f5700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d80b5dd0 con 0x7f36d80b1400 2022-07-21T01:57:15.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36de3f5700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d80b1070 con 0x7f36d80b59f0 2022-07-21T01:57:15.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36dd3f3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f36d80b59f0 0x7f36d81473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:15.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36dcbf2700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f36d80af940 0x7f36d814a3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54486/0 (socket says 172.21.15.140:54486) 2022-07-21T01:57:15.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36dcbf2700 1 -- 172.21.15.140:0/116526658 learned_addr learned my addr 172.21.15.140:0/116526658 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:15.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36dd3f3700 1 -- 172.21.15.140:0/116526658 >> v1:172.21.15.152:6789/0 conn(0x7f36d80b1400 legacy=0x7f36d814aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:15.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36dd3f3700 1 -- 172.21.15.140:0/116526658 >> v1:172.21.15.140:6789/0 conn(0x7f36d80af940 legacy=0x7f36d814a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:15.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36dd3f3700 1 -- 172.21.15.140:0/116526658 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d805c760 con 0x7f36d80b59f0 2022-07-21T01:57:15.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36cffff700 1 -- 172.21.15.140:0/116526658 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143140056 0 0) 0x7f36d805cc70 con 0x7f36d80af940 2022-07-21T01:57:15.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36dd3f3700 1 --2- 172.21.15.140:0/116526658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36d80b59f0 0x7f36d81473a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f36d4002f70 tx=0x7f36d4006df0).ready entity=mon.0 client_cookie=4945f2dbf980195a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:15.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.064+0000 7f36cffff700 1 -- 172.21.15.140:0/116526658 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36d400d610 con 0x7f36d80b59f0 2022-07-21T01:57:15.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36cffff700 1 -- 172.21.15.140:0/116526658 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d4000e90 con 0x7f36d80b59f0 2022-07-21T01:57:15.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36cffff700 1 -- 172.21.15.140:0/116526658 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36d400d960 con 0x7f36d80b59f0 2022-07-21T01:57:15.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 -- 172.21.15.140:0/116526658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36d80b59f0 msgr2=0x7f36d81473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:15.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 --2- 172.21.15.140:0/116526658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36d80b59f0 0x7f36d81473a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f36d4002f70 tx=0x7f36d4006df0).stop 2022-07-21T01:57:15.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 -- 172.21.15.140:0/116526658 shutdown_connections 2022-07-21T01:57:15.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 --2- 172.21.15.140:0/116526658 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36d80b59f0 0x7f36d81473a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:15.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 -- 172.21.15.140:0/116526658 >> 172.21.15.140:0/116526658 conn(0x7f36d8063480 msgr2=0x7f36d80b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:15.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 -- 172.21.15.140:0/116526658 shutdown_connections 2022-07-21T01:57:15.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 -- 172.21.15.140:0/116526658 wait complete. 2022-07-21T01:57:15.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 Processor -- start 2022-07-21T01:57:15.078 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 -- start start 2022-07-21T01:57:15.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36d80af940 0x7f36d814a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:15.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36d80b1400 0x7f36d8156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:15.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36d80b59f0 0x7f36d815a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:15.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d810e820 con 0x7f36d80b1400 2022-07-21T01:57:15.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d8159bc0 con 0x7f36d80af940 2022-07-21T01:57:15.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36de3f5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d8156920 con 0x7f36d80b59f0 2022-07-21T01:57:15.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36dcbf2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36d80b1400 0x7f36d8156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:15.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36dcbf2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36d80b1400 0x7f36d8156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40346/0 (socket says 172.21.15.140:40346) 2022-07-21T01:57:15.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36dcbf2700 1 -- 172.21.15.140:0/3068257032 learned_addr learned my addr 172.21.15.140:0/3068257032 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:15.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36dd3f3700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36d80af940 0x7f36d814a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:15.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36ddbf4700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36d80b59f0 0x7f36d815a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:15.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.068+0000 7f36dcbf2700 1 -- 172.21.15.140:0/3068257032 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36d80b59f0 msgr2=0x7f36d815a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:15.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36dcbf2700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36d80b59f0 0x7f36d815a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:15.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36dcbf2700 1 -- 172.21.15.140:0/3068257032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36d80af940 msgr2=0x7f36d814a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:15.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36dcbf2700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36d80af940 0x7f36d814a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:15.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36dcbf2700 1 -- 172.21.15.140:0/3068257032 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d400b180 con 0x7f36d80b1400 2022-07-21T01:57:15.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36dcbf2700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36d80b1400 0x7f36d8156430 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f36c8000bc0 tx=0x7f36c8007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:15.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36ce7fc700 1 -- 172.21.15.140:0/3068257032 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36c80070d0 con 0x7f36d80b1400 2022-07-21T01:57:15.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36ce7fc700 1 -- 172.21.15.140:0/3068257032 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c8007230 con 0x7f36d80b1400 2022-07-21T01:57:15.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36de3f5700 1 -- 172.21.15.140:0/3068257032 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d8066c50 con 0x7f36d80b1400 2022-07-21T01:57:15.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36ce7fc700 1 -- 172.21.15.140:0/3068257032 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f36c8005610 con 0x7f36d80b1400 2022-07-21T01:57:15.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36de3f5700 1 -- 172.21.15.140:0/3068257032 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d81121c0 con 0x7f36d80b1400 2022-07-21T01:57:15.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36de3f5700 1 -- 172.21.15.140:0/3068257032 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d810e990 con 0x7f36d80b1400 2022-07-21T01:57:15.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36ce7fc700 1 -- 172.21.15.140:0/3068257032 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f36c8010070 con 0x7f36d80b1400 2022-07-21T01:57:15.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36ce7fc700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36c0053b00 0x7f36c0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:15.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36ce7fc700 1 -- 172.21.15.140:0/3068257032 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f36c806b5a0 con 0x7f36d80b1400 2022-07-21T01:57:15.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36dd3f3700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36c0053b00 0x7f36c0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:15.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.072+0000 7f36dd3f3700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36c0053b00 0x7f36c0055fc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f36d401c000 tx=0x7f36d4015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:15.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.080+0000 7f36ce7fc700 1 -- 172.21.15.140:0/3068257032 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f36d810e990 con 0x7f36d80b1400 2022-07-21T01:57:15.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:15.236+0000 7f36de3f5700 1 -- 172.21.15.140:0/3068257032 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f36d810e960 con 0x7f36d80b1400 2022-07-21T01:57:16.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.136+0000 7f36ce7fc700 1 -- 172.21.15.140:0/3068257032 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) v1 ==== 118+0+0 (secure 0 0 0) 0x7f36c80424c0 con 0x7f36d80b1400 2022-07-21T01:57:16.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 -- 172.21.15.140:0/3068257032 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36c0053b00 msgr2=0x7f36c0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36c0053b00 0x7f36c0055fc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f36d401c000 tx=0x7f36d4015000).stop 2022-07-21T01:57:16.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 -- 172.21.15.140:0/3068257032 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36d80b1400 msgr2=0x7f36d8156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36d80b1400 0x7f36d8156430 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f36c8000bc0 tx=0x7f36c8007a60).stop 2022-07-21T01:57:16.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 -- 172.21.15.140:0/3068257032 shutdown_connections 2022-07-21T01:57:16.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36c0053b00 0x7f36c0055fc0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36d80b59f0 0x7f36d815a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f36d80b1400 0x7f36d8156430 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 --2- 172.21.15.140:0/3068257032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36d80af940 0x7f36d814a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 -- 172.21.15.140:0/3068257032 >> 172.21.15.140:0/3068257032 conn(0x7f36d8063480 msgr2=0x7f36d813fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:16.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 -- 172.21.15.140:0/3068257032 shutdown_connections 2022-07-21T01:57:16.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.140+0000 7f36de3f5700 1 -- 172.21.15.140:0/3068257032 wait complete. 2022-07-21T01:57:16.162 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:16.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 Processor -- start 2022-07-21T01:57:16.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 -- start start 2022-07-21T01:57:16.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd6fc0b1e00 0x7fd6fc14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd6fc05c3f0 0x7fd6fc0ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc05d480 con 0x7fd6fc05bd90 2022-07-21T01:57:16.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc0b21e0 con 0x7fd6fc0b1e00 2022-07-21T01:57:16.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc05cf40 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd700f0d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd6fc0b1e00 0x7fd6fc14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd700f0d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd6fc0b1e00 0x7fd6fc14a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40352/0 (socket says 172.21.15.140:40352) 2022-07-21T01:57:16.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd700f0d700 1 -- 172.21.15.140:0/1580602869 learned_addr learned my addr 172.21.15.140:0/1580602869 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:16.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd6fbfff700 1 --2- 172.21.15.140:0/1580602869 >> v2:172.21.15.152:3300/0 conn(0x7fd6fc05c3f0 0x7fd6fc0ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd700f0d700 1 -- 172.21.15.140:0/1580602869 >> v2:172.21.15.152:3300/0 conn(0x7fd6fc05c3f0 msgr2=0x7fd6fc0ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:16.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd700f0d700 1 --2- 172.21.15.140:0/1580602869 >> v2:172.21.15.152:3300/0 conn(0x7fd6fc05c3f0 0x7fd6fc0ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd700f0d700 1 -- 172.21.15.140:0/1580602869 >> v1:172.21.15.140:6789/0 conn(0x7fd6fc05bd90 legacy=0x7fd6fc146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd700f0d700 1 -- 172.21.15.140:0/1580602869 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc05d1e0 con 0x7fd6fc0b1e00 2022-07-21T01:57:16.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd700f0d700 1 --2- 172.21.15.140:0/1580602869 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6fc0b1e00 0x7fd6fc14a6d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7fd6e4002f70 tx=0x7fd6e4006eb0).ready entity=mon.0 client_cookie=29f0178a7b735bda server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:16.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd6fb7fe700 1 -- 172.21.15.140:0/1580602869 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6e400d610 con 0x7fd6fc0b1e00 2022-07-21T01:57:16.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd6fb7fe700 1 -- 172.21.15.140:0/1580602869 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e4000e90 con 0x7fd6fc0b1e00 2022-07-21T01:57:16.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd6fb7fe700 1 -- 172.21.15.140:0/1580602869 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6e400d9a0 con 0x7fd6fc0b1e00 2022-07-21T01:57:16.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 -- 172.21.15.140:0/1580602869 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6fc0b1e00 msgr2=0x7fd6fc14a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 --2- 172.21.15.140:0/1580602869 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6fc0b1e00 0x7fd6fc14a6d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7fd6e4002f70 tx=0x7fd6e4006eb0).stop 2022-07-21T01:57:16.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 -- 172.21.15.140:0/1580602869 shutdown_connections 2022-07-21T01:57:16.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 --2- 172.21.15.140:0/1580602869 >> v2:172.21.15.152:3300/0 conn(0x7fd6fc05c3f0 0x7fd6fc0ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 --2- 172.21.15.140:0/1580602869 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6fc0b1e00 0x7fd6fc14a6d0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 -- 172.21.15.140:0/1580602869 >> 172.21.15.140:0/1580602869 conn(0x7fd6fc0611c0 msgr2=0x7fd6fc062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:16.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.268+0000 7fd703171700 1 -- 172.21.15.140:0/1580602869 shutdown_connections 2022-07-21T01:57:16.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.272+0000 7fd703171700 1 -- 172.21.15.140:0/1580602869 wait complete. 2022-07-21T01:57:16.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.272+0000 7fd703171700 1 Processor -- start 2022-07-21T01:57:16.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.272+0000 7fd703171700 1 -- start start 2022-07-21T01:57:16.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd703171700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6fc05c3f0 0x7fd6fc09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd703171700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6fc0b1e00 0x7fd6fc157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd703171700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6fc066cf0 0x7fd6fc159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd703171700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc157700 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd703171700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc156ad0 con 0x7fd6fc0b1e00 2022-07-21T01:57:16.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd703171700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc156d70 con 0x7fd6fc066cf0 2022-07-21T01:57:16.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd700f0d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6fc05c3f0 0x7fd6fc09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd700f0d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6fc05c3f0 0x7fd6fc09e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40358/0 (socket says 172.21.15.140:40358) 2022-07-21T01:57:16.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd700f0d700 1 -- 172.21.15.140:0/219986495 learned_addr learned my addr 172.21.15.140:0/219986495 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:16.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd70170e700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6fc066cf0 0x7fd6fc159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd6fbfff700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6fc0b1e00 0x7fd6fc157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd700f0d700 1 -- 172.21.15.140:0/219986495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6fc066cf0 msgr2=0x7fd6fc159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd700f0d700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6fc066cf0 0x7fd6fc159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd700f0d700 1 -- 172.21.15.140:0/219986495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6fc0b1e00 msgr2=0x7fd6fc157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd700f0d700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6fc0b1e00 0x7fd6fc157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd700f0d700 1 -- 172.21.15.140:0/219986495 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e400b1e0 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd70170e700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6fc066cf0 0x7fd6fc159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:16.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd700f0d700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6fc05c3f0 0x7fd6fc09e5d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7fd6e4002f70 tx=0x7fd6e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:16.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd6f9ffb700 1 -- 172.21.15.140:0/219986495 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6e400d290 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd6f9ffb700 1 -- 172.21.15.140:0/219986495 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e400d9b0 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd703171700 1 -- 172.21.15.140:0/219986495 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc066010 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd6f9ffb700 1 -- 172.21.15.140:0/219986495 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6e401e460 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd703171700 1 -- 172.21.15.140:0/219986495 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc0663a0 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.276+0000 7fd703171700 1 -- 172.21.15.140:0/219986495 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc156f30 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.280+0000 7fd6f9ffb700 1 -- 172.21.15.140:0/219986495 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd6e4025030 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.280+0000 7fd6f9ffb700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6e0053a70 0x7fd6e0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.280+0000 7fd6f9ffb700 1 -- 172.21.15.140:0/219986495 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd6e406d120 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.280+0000 7fd6fbfff700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6e0053a70 0x7fd6e0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.280+0000 7fd6fbfff700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6e0053a70 0x7fd6e0055f30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fd6f400d000 tx=0x7fd6f4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:16.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.288+0000 7fd6f9ffb700 1 -- 172.21.15.140:0/219986495 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd6e4043f30 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 -- 172.21.15.140:0/219986495 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6fc0574e0 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd6f9ffb700 1 -- 172.21.15.140:0/219986495 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+2145 (secure 0 0 0) 0x7fd6e4023090 con 0x7fd6fc05c3f0 2022-07-21T01:57:16.445 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:16.446 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":64,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/558475048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":558475048},{"type":"v1","addr":"172.21.15.152:6833","nonce":558475048}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:14.423640+0000","modified":"2022-07-21T01:57:16.123489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2209871450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2209871450},{"type":"v1","addr":"172.21.15.140:6835","nonce":2209871450}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-07-21T01:57:16.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 -- 172.21.15.140:0/219986495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6e0053a70 msgr2=0x7fd6e0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6e0053a70 0x7fd6e0055f30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fd6f400d000 tx=0x7fd6f4008040).stop 2022-07-21T01:57:16.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 -- 172.21.15.140:0/219986495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6fc05c3f0 msgr2=0x7fd6fc09e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6fc05c3f0 0x7fd6fc09e5d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7fd6e4002f70 tx=0x7fd6e4015000).stop 2022-07-21T01:57:16.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 -- 172.21.15.140:0/219986495 shutdown_connections 2022-07-21T01:57:16.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd6e0053a70 0x7fd6e0055f30 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd6fc066cf0 0x7fd6fc159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6fc0b1e00 0x7fd6fc157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 --2- 172.21.15.140:0/219986495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd6fc05c3f0 0x7fd6fc09e5d0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 -- 172.21.15.140:0/219986495 >> 172.21.15.140:0/219986495 conn(0x7fd6fc05bd90 msgr2=0x7fd6fc149f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:16.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 -- 172.21.15.140:0/219986495 shutdown_connections 2022-07-21T01:57:16.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.444+0000 7fd703171700 1 -- 172.21.15.140:0/219986495 wait complete. 2022-07-21T01:57:16.450 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 64 2022-07-21T01:57:16.462 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:16.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.560+0000 7fcf3f75a700 1 Processor -- start 2022-07-21T01:57:16.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.560+0000 7fcf3f75a700 1 -- start start 2022-07-21T01:57:16.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3f75a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcf3805c3f0 0x7fcf3814a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3f75a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3805d480 con 0x7fcf380b1e00 2022-07-21T01:57:16.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3f75a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf380b21e0 con 0x7fcf380acef0 2022-07-21T01:57:16.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3f75a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3805cf40 con 0x7fcf3805c3f0 2022-07-21T01:57:16.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3d4f6700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fcf380b1e00 0x7fcf380ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54514/0 (socket says 172.21.15.140:54514) 2022-07-21T01:57:16.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3d4f6700 1 -- 172.21.15.140:0/889610351 learned_addr learned my addr 172.21.15.140:0/889610351 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:16.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3ccf5700 1 --2- 172.21.15.140:0/889610351 >> v2:172.21.15.152:3300/0 conn(0x7fcf3805c3f0 0x7fcf3814a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf37fff700 1 -- 172.21.15.140:0/889610351 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926940656 0 0) 0x7fcf380b21e0 con 0x7fcf380acef0 2022-07-21T01:57:16.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf37fff700 1 -- 172.21.15.140:0/889610351 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf24003260 con 0x7fcf380acef0 2022-07-21T01:57:16.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf37fff700 1 -- 172.21.15.140:0/889610351 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778432402 0 0) 0x7fcf3805d480 con 0x7fcf380b1e00 2022-07-21T01:57:16.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf37fff700 1 -- 172.21.15.140:0/889610351 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf380b21e0 con 0x7fcf380b1e00 2022-07-21T01:57:16.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3ccf5700 1 -- 172.21.15.140:0/889610351 >> v1:172.21.15.152:6789/0 conn(0x7fcf380acef0 legacy=0x7fcf38146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf37fff700 1 -- 172.21.15.140:0/889610351 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2962453719 0 0) 0x7fcf380b21e0 con 0x7fcf380b1e00 2022-07-21T01:57:16.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3ccf5700 1 -- 172.21.15.140:0/889610351 >> v1:172.21.15.140:6789/0 conn(0x7fcf380b1e00 legacy=0x7fcf380ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3ccf5700 1 -- 172.21.15.140:0/889610351 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3805d1e0 con 0x7fcf3805c3f0 2022-07-21T01:57:16.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3ccf5700 1 --2- 172.21.15.140:0/889610351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf3805c3f0 0x7fcf3814a6d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fcf2c001110 tx=0x7fcf300091d0).ready entity=mon.1 client_cookie=1a493ae7124f9863 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:16.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf37fff700 1 -- 172.21.15.140:0/889610351 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf2c001660 con 0x7fcf3805c3f0 2022-07-21T01:57:16.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf37fff700 1 -- 172.21.15.140:0/889610351 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3000f510 con 0x7fcf3805c3f0 2022-07-21T01:57:16.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf37fff700 1 -- 172.21.15.140:0/889610351 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf3000f680 con 0x7fcf3805c3f0 2022-07-21T01:57:16.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3f75a700 1 -- 172.21.15.140:0/889610351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf3805c3f0 msgr2=0x7fcf3814a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3f75a700 1 --2- 172.21.15.140:0/889610351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf3805c3f0 0x7fcf3814a6d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7fcf2c001110 tx=0x7fcf300091d0).stop 2022-07-21T01:57:16.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3f75a700 1 -- 172.21.15.140:0/889610351 shutdown_connections 2022-07-21T01:57:16.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3f75a700 1 --2- 172.21.15.140:0/889610351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf3805c3f0 0x7fcf3814a6d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3f75a700 1 -- 172.21.15.140:0/889610351 >> 172.21.15.140:0/889610351 conn(0x7fcf380611c0 msgr2=0x7fcf38062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:16.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3f75a700 1 -- 172.21.15.140:0/889610351 shutdown_connections 2022-07-21T01:57:16.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.564+0000 7fcf3f75a700 1 -- 172.21.15.140:0/889610351 wait complete. 2022-07-21T01:57:16.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3f75a700 1 Processor -- start 2022-07-21T01:57:16.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3f75a700 1 -- start start 2022-07-21T01:57:16.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3f75a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf380acef0 0x7fcf381488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3f75a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf380b1e00 0x7fcf38067020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3f75a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf380a1c70 0x7fcf380653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3f75a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3815ab20 con 0x7fcf380acef0 2022-07-21T01:57:16.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3f75a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf38148dd0 con 0x7fcf380b1e00 2022-07-21T01:57:16.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3f75a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3815a880 con 0x7fcf380a1c70 2022-07-21T01:57:16.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3d4f6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf380acef0 0x7fcf381488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3d4f6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf380acef0 0x7fcf381488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40372/0 (socket says 172.21.15.140:40372) 2022-07-21T01:57:16.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3d4f6700 1 -- 172.21.15.140:0/825022324 learned_addr learned my addr 172.21.15.140:0/825022324 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:16.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3ccf5700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf380b1e00 0x7fcf38067020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3dcf7700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf380a1c70 0x7fcf380653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3d4f6700 1 -- 172.21.15.140:0/825022324 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf380a1c70 msgr2=0x7fcf380653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3d4f6700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf380a1c70 0x7fcf380653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3d4f6700 1 -- 172.21.15.140:0/825022324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf380b1e00 msgr2=0x7fcf38067020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3d4f6700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf380b1e00 0x7fcf38067020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3d4f6700 1 -- 172.21.15.140:0/825022324 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf30009470 con 0x7fcf380acef0 2022-07-21T01:57:16.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3d4f6700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf380acef0 0x7fcf381488e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7fcf2c001840 tx=0x7fcf2c001a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:16.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf367fc700 1 -- 172.21.15.140:0/825022324 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf2c007170 con 0x7fcf380acef0 2022-07-21T01:57:16.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf367fc700 1 -- 172.21.15.140:0/825022324 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c0072d0 con 0x7fcf380acef0 2022-07-21T01:57:16.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.568+0000 7fcf3f75a700 1 -- 172.21.15.140:0/825022324 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3815a5a0 con 0x7fcf380acef0 2022-07-21T01:57:16.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.572+0000 7fcf367fc700 1 -- 172.21.15.140:0/825022324 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf2c004920 con 0x7fcf380acef0 2022-07-21T01:57:16.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.572+0000 7fcf3f75a700 1 -- 172.21.15.140:0/825022324 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf381493e0 con 0x7fcf380acef0 2022-07-21T01:57:16.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.572+0000 7fcf3f75a700 1 -- 172.21.15.140:0/825022324 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf380b04b0 con 0x7fcf380acef0 2022-07-21T01:57:16.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.572+0000 7fcf367fc700 1 -- 172.21.15.140:0/825022324 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcf2c004a80 con 0x7fcf380acef0 2022-07-21T01:57:16.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.572+0000 7fcf367fc700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf180539e0 0x7fcf18055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.572+0000 7fcf367fc700 1 -- 172.21.15.140:0/825022324 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcf2c06b9e0 con 0x7fcf380acef0 2022-07-21T01:57:16.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.572+0000 7fcf3ccf5700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf180539e0 0x7fcf18055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.572+0000 7fcf3ccf5700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf180539e0 0x7fcf18055ea0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fcf3001b000 tx=0x7fcf30016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:16.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.580+0000 7fcf367fc700 1 -- 172.21.15.140:0/825022324 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcf2c0428a0 con 0x7fcf380acef0 2022-07-21T01:57:16.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.704+0000 7fcf3f75a700 1 -- 172.21.15.140:0/825022324 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf38063ca0 con 0x7fcf380acef0 2022-07-21T01:57:16.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.704+0000 7fcf367fc700 1 -- 172.21.15.140:0/825022324 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16148 (secure 0 0 0) 0x7fcf2c045ec0 con 0x7fcf380acef0 2022-07-21T01:57:16.707 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:16.707 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":132,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:15.426209+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:57:12.443397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-07-21T01:57:13.427432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:16.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 -- 172.21.15.140:0/825022324 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf180539e0 msgr2=0x7fcf18055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf180539e0 0x7fcf18055ea0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fcf3001b000 tx=0x7fcf30016000).stop 2022-07-21T01:57:16.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 -- 172.21.15.140:0/825022324 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf380acef0 msgr2=0x7fcf381488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf380acef0 0x7fcf381488e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7fcf2c001840 tx=0x7fcf2c001a40).stop 2022-07-21T01:57:16.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 -- 172.21.15.140:0/825022324 shutdown_connections 2022-07-21T01:57:16.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcf180539e0 0x7fcf18055ea0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcf380a1c70 0x7fcf380653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf380b1e00 0x7fcf38067020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 --2- 172.21.15.140:0/825022324 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcf380acef0 0x7fcf381488e0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 -- 172.21.15.140:0/825022324 >> 172.21.15.140:0/825022324 conn(0x7fcf3805c3f0 msgr2=0x7fcf3814a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:16.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 -- 172.21.15.140:0/825022324 shutdown_connections 2022-07-21T01:57:16.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.708+0000 7fcf3f75a700 1 -- 172.21.15.140:0/825022324 wait complete. 2022-07-21T01:57:16.721 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:57:16.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a703a700 1 Processor -- start 2022-07-21T01:57:16.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a703a700 1 -- start start 2022-07-21T01:57:16.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a703a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f91a00b24b0 0x7f91a00b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a703a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f91a0146b60 0x7f91a0143400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a703a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91a01438f0 0x7f91a00b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a703a700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a00b1820 con 0x7f91a0146b60 2022-07-21T01:57:16.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a703a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a00b2890 con 0x7f91a00b24b0 2022-07-21T01:57:16.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a703a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a0147030 con 0x7f91a01438f0 2022-07-21T01:57:16.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5837700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f91a0146b60 0x7f91a0143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5837700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f91a0146b60 0x7f91a0143400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40382/0 (socket says 172.21.15.140:40382) 2022-07-21T01:57:16.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5837700 1 -- 172.21.15.140:0/1501626585 learned_addr learned my addr 172.21.15.140:0/1501626585 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:16.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a6839700 1 --2- 172.21.15.140:0/1501626585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91a01438f0 0x7f91a00b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a6038700 1 --2- 172.21.15.140:0/1501626585 >> v2:172.21.15.152:3300/0 conn(0x7f91a00b24b0 0x7f91a00b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5837700 1 -- 172.21.15.140:0/1501626585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91a01438f0 msgr2=0x7f91a00b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:16.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5837700 1 --2- 172.21.15.140:0/1501626585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91a01438f0 0x7f91a00b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5837700 1 -- 172.21.15.140:0/1501626585 >> v2:172.21.15.152:3300/0 conn(0x7f91a00b24b0 msgr2=0x7f91a00b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:16.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5837700 1 --2- 172.21.15.140:0/1501626585 >> v2:172.21.15.152:3300/0 conn(0x7f91a00b24b0 0x7f91a00b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5837700 1 -- 172.21.15.140:0/1501626585 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a0142cc0 con 0x7f91a0146b60 2022-07-21T01:57:16.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5837700 1 --2- 172.21.15.140:0/1501626585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91a0146b60 0x7f91a0143400 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f919c002f70 tx=0x7f919c006eb0).ready entity=mon.0 client_cookie=ee098973301b5a8b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:16.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5036700 1 -- 172.21.15.140:0/1501626585 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f919c00d570 con 0x7f91a0146b60 2022-07-21T01:57:16.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5036700 1 -- 172.21.15.140:0/1501626585 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c000e90 con 0x7f91a0146b60 2022-07-21T01:57:16.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.832+0000 7f91a5036700 1 -- 172.21.15.140:0/1501626585 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f919c00d900 con 0x7f91a0146b60 2022-07-21T01:57:16.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 -- 172.21.15.140:0/1501626585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91a0146b60 msgr2=0x7f91a0143400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 --2- 172.21.15.140:0/1501626585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91a0146b60 0x7f91a0143400 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f919c002f70 tx=0x7f919c006eb0).stop 2022-07-21T01:57:16.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 -- 172.21.15.140:0/1501626585 shutdown_connections 2022-07-21T01:57:16.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 --2- 172.21.15.140:0/1501626585 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91a01438f0 0x7f91a00b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 --2- 172.21.15.140:0/1501626585 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91a0146b60 0x7f91a0143400 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 --2- 172.21.15.140:0/1501626585 >> v2:172.21.15.152:3300/0 conn(0x7f91a00b24b0 0x7f91a00b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 -- 172.21.15.140:0/1501626585 >> 172.21.15.140:0/1501626585 conn(0x7f91a0063150 msgr2=0x7f91a0063530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:16.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 -- 172.21.15.140:0/1501626585 shutdown_connections 2022-07-21T01:57:16.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 -- 172.21.15.140:0/1501626585 wait complete. 2022-07-21T01:57:16.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 Processor -- start 2022-07-21T01:57:16.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 -- start start 2022-07-21T01:57:16.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91a00b24b0 0x7f91a0064810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91a01438f0 0x7f91a0064d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0146b60 0x7f91a00664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00abb30 con 0x7f91a01438f0 2022-07-21T01:57:16.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00a9690 con 0x7f91a0146b60 2022-07-21T01:57:16.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a703a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a0152be0 con 0x7f91a00b24b0 2022-07-21T01:57:16.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a5837700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91a01438f0 0x7f91a0064d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a5837700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91a01438f0 0x7f91a0064d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40386/0 (socket says 172.21.15.140:40386) 2022-07-21T01:57:16.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a5837700 1 -- 172.21.15.140:0/41874285 learned_addr learned my addr 172.21.15.140:0/41874285 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:16.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a6839700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0146b60 0x7f91a00664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a6038700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91a00b24b0 0x7f91a0064810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a5837700 1 -- 172.21.15.140:0/41874285 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91a00b24b0 msgr2=0x7f91a0064810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a5837700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91a00b24b0 0x7f91a0064810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a5837700 1 -- 172.21.15.140:0/41874285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0146b60 msgr2=0x7f91a00664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:16.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a5837700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0146b60 0x7f91a00664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:16.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a5837700 1 -- 172.21.15.140:0/41874285 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919c00b1e0 con 0x7f91a01438f0 2022-07-21T01:57:16.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a6839700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0146b60 0x7f91a00664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:16.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.836+0000 7f91a5837700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91a01438f0 0x7f91a0064d00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f919c002f70 tx=0x7f919c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:16.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.840+0000 7f918f7fe700 1 -- 172.21.15.140:0/41874285 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f919c011760 con 0x7f91a01438f0 2022-07-21T01:57:16.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.840+0000 7f918f7fe700 1 -- 172.21.15.140:0/41874285 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c0118c0 con 0x7f91a01438f0 2022-07-21T01:57:16.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.840+0000 7f91a703a700 1 -- 172.21.15.140:0/41874285 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a00a98c0 con 0x7f91a01438f0 2022-07-21T01:57:16.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.840+0000 7f918f7fe700 1 -- 172.21.15.140:0/41874285 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f919c011c20 con 0x7f91a01438f0 2022-07-21T01:57:16.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.840+0000 7f91a703a700 1 -- 172.21.15.140:0/41874285 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a01526e0 con 0x7f91a01438f0 2022-07-21T01:57:16.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.840+0000 7f91a703a700 1 -- 172.21.15.140:0/41874285 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a00a9a50 con 0x7f91a01438f0 2022-07-21T01:57:16.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.840+0000 7f918f7fe700 1 -- 172.21.15.140:0/41874285 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f919c01a040 con 0x7f91a01438f0 2022-07-21T01:57:16.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.840+0000 7f918f7fe700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9180053a70 0x7f9180055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:16.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.840+0000 7f918f7fe700 1 -- 172.21.15.140:0/41874285 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f919c06ce70 con 0x7f91a01438f0 2022-07-21T01:57:16.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.840+0000 7f91a6038700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9180053a70 0x7f9180055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:16.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.844+0000 7f91a6038700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9180053a70 0x7f9180055f30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f919000c000 tx=0x7f9190008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:16.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:16.848+0000 7f918f7fe700 1 -- 172.21.15.140:0/41874285 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f919c043d90 con 0x7f91a01438f0 2022-07-21T01:57:17.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.008+0000 7f91a703a700 1 -- 172.21.15.140:0/41874285 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f91a0142f60 con 0x7f91a01438f0 2022-07-21T01:57:17.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.016+0000 7f918f7fe700 1 -- 172.21.15.140:0/41874285 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v43) v1 ==== 184+0+0 (secure 0 0 0) 0x7f919c023020 con 0x7f91a01438f0 2022-07-21T01:57:17.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.016+0000 7f91a703a700 1 -- 172.21.15.140:0/41874285 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9180053a70 msgr2=0x7f9180055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.016+0000 7f91a703a700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9180053a70 0x7f9180055f30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f919000c000 tx=0x7f9190008040).stop 2022-07-21T01:57:17.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.016+0000 7f91a703a700 1 -- 172.21.15.140:0/41874285 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91a01438f0 msgr2=0x7f91a0064d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.016+0000 7f91a703a700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91a01438f0 0x7f91a0064d00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f919c002f70 tx=0x7f919c015000).stop 2022-07-21T01:57:17.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.020+0000 7f91a703a700 1 -- 172.21.15.140:0/41874285 shutdown_connections 2022-07-21T01:57:17.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.020+0000 7f91a703a700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9180053a70 0x7f9180055f30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.020+0000 7f91a703a700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91a0146b60 0x7f91a00664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.020+0000 7f91a703a700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f91a01438f0 0x7f91a0064d00 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.020+0000 7f91a703a700 1 --2- 172.21.15.140:0/41874285 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f91a00b24b0 0x7f91a0064810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.020+0000 7f91a703a700 1 -- 172.21.15.140:0/41874285 >> 172.21.15.140:0/41874285 conn(0x7f91a0063150 msgr2=0x7f91a00b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:17.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.020+0000 7f91a703a700 1 -- 172.21.15.140:0/41874285 shutdown_connections 2022-07-21T01:57:17.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.020+0000 7f91a703a700 1 -- 172.21.15.140:0/41874285 wait complete. 2022-07-21T01:57:17.024 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:57:17.036 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:57:17.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.148+0000 7fbbaa4ce700 1 Processor -- start 2022-07-21T01:57:17.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.148+0000 7fbbaa4ce700 1 -- start start 2022-07-21T01:57:17.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.148+0000 7fbbaa4ce700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbba40af0d0 0x7fbba40abe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.148+0000 7fbbaa4ce700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbba40b4c20 0x7fbba41409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.148+0000 7fbbaa4ce700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba40b4290 con 0x7fbba40a9690 2022-07-21T01:57:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.148+0000 7fbbaa4ce700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba40af4b0 con 0x7fbba40af0d0 2022-07-21T01:57:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.148+0000 7fbbaa4ce700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40a9340 con 0x7fbba40b4c20 2022-07-21T01:57:17.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.148+0000 7fbba94cc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbba40af0d0 0x7fbba40abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.148+0000 7fbba94cc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbba40af0d0 0x7fbba40abe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40394/0 (socket says 172.21.15.140:40394) 2022-07-21T01:57:17.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.148+0000 7fbba94cc700 1 -- 172.21.15.140:0/1527875781 learned_addr learned my addr 172.21.15.140:0/1527875781 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:17.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.148+0000 7fbba9ccd700 1 --2- 172.21.15.140:0/1527875781 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbba40b4c20 0x7fbba41409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbba94cc700 1 -- 172.21.15.140:0/1527875781 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbba40b4c20 msgr2=0x7fbba41409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:17.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbba94cc700 1 --2- 172.21.15.140:0/1527875781 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbba40b4c20 0x7fbba41409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbba94cc700 1 -- 172.21.15.140:0/1527875781 >> v1:172.21.15.152:6789/0 conn(0x7fbba40a9690 legacy=0x7fbba40ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbba94cc700 1 -- 172.21.15.140:0/1527875781 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40b44e0 con 0x7fbba40af0d0 2022-07-21T01:57:17.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbba94cc700 1 --2- 172.21.15.140:0/1527875781 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbba40af0d0 0x7fbba40abe00 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7fbba0002f70 tx=0x7fbba0006eb0).ready entity=mon.0 client_cookie=7c5d06808d396a74 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:17.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbb97fff700 1 -- 172.21.15.140:0/1527875781 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbba000d570 con 0x7fbba40af0d0 2022-07-21T01:57:17.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbb97fff700 1 -- 172.21.15.140:0/1527875781 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba0000e90 con 0x7fbba40af0d0 2022-07-21T01:57:17.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbb97fff700 1 -- 172.21.15.140:0/1527875781 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbba000d900 con 0x7fbba40af0d0 2022-07-21T01:57:17.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1527875781 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbba40af0d0 msgr2=0x7fbba40abe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbbaa4ce700 1 --2- 172.21.15.140:0/1527875781 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbba40af0d0 0x7fbba40abe00 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7fbba0002f70 tx=0x7fbba0006eb0).stop 2022-07-21T01:57:17.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1527875781 shutdown_connections 2022-07-21T01:57:17.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbbaa4ce700 1 --2- 172.21.15.140:0/1527875781 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbba40b4c20 0x7fbba41409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbbaa4ce700 1 --2- 172.21.15.140:0/1527875781 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbba40af0d0 0x7fbba40abe00 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1527875781 >> 172.21.15.140:0/1527875781 conn(0x7fbba4063150 msgr2=0x7fbba4063530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:17.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1527875781 shutdown_connections 2022-07-21T01:57:17.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1527875781 wait complete. 2022-07-21T01:57:17.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.152+0000 7fbbaa4ce700 1 Processor -- start 2022-07-21T01:57:17.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbbaa4ce700 1 -- start start 2022-07-21T01:57:17.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbbaa4ce700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbba40a9690 0x7fbba409e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbbaa4ce700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbba40af0d0 0x7fbba40a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbbaa4ce700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbba40b4c20 0x7fbba40a89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbbaa4ce700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba4065d30 con 0x7fbba40a9690 2022-07-21T01:57:17.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbbaa4ce700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba409eba0 con 0x7fbba40af0d0 2022-07-21T01:57:17.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbbaa4ce700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba409ee40 con 0x7fbba40b4c20 2022-07-21T01:57:17.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbba94cc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbba40a9690 0x7fbba409e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbba94cc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbba40a9690 0x7fbba409e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40400/0 (socket says 172.21.15.140:40400) 2022-07-21T01:57:17.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbba8ccb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbba40af0d0 0x7fbba40a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbba94cc700 1 -- 172.21.15.140:0/1046373813 learned_addr learned my addr 172.21.15.140:0/1046373813 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:17.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbba9ccd700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbba40b4c20 0x7fbba40a89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.165 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbba94cc700 1 -- 172.21.15.140:0/1046373813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbba40b4c20 msgr2=0x7fbba40a89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbba94cc700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbba40b4c20 0x7fbba40a89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbba94cc700 1 -- 172.21.15.140:0/1046373813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbba40af0d0 msgr2=0x7fbba40a84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbba94cc700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbba40af0d0 0x7fbba40a84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.166 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbba94cc700 1 -- 172.21.15.140:0/1046373813 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba000b1e0 con 0x7fbba40a9690 2022-07-21T01:57:17.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbba94cc700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbba40a9690 0x7fbba409e550 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7fbba0002f70 tx=0x7fbba0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:17.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbb967fc700 1 -- 172.21.15.140:0/1046373813 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbba0011760 con 0x7fbba40a9690 2022-07-21T01:57:17.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbb967fc700 1 -- 172.21.15.140:0/1046373813 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba00118c0 con 0x7fbba40a9690 2022-07-21T01:57:17.167 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1046373813 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba40a8ec0 con 0x7fbba40a9690 2022-07-21T01:57:17.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbb967fc700 1 -- 172.21.15.140:0/1046373813 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbba0011c20 con 0x7fbba40a9690 2022-07-21T01:57:17.169 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.156+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1046373813 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba4150d70 con 0x7fbba40a9690 2022-07-21T01:57:17.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.160+0000 7fbb967fc700 1 -- 172.21.15.140:0/1046373813 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbba0025030 con 0x7fbba40a9690 2022-07-21T01:57:17.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.160+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1046373813 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba40b4780 con 0x7fbba40a9690 2022-07-21T01:57:17.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.160+0000 7fbb967fc700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb8c053b00 0x7fbb8c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.160+0000 7fbba8ccb700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb8c053b00 0x7fbb8c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.160+0000 7fbb967fc700 1 -- 172.21.15.140:0/1046373813 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbba006ced0 con 0x7fbba40a9690 2022-07-21T01:57:17.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.160+0000 7fbba8ccb700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb8c053b00 0x7fbb8c055fc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fbb9800d000 tx=0x7fbb98007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:17.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.168+0000 7fbb967fc700 1 -- 172.21.15.140:0/1046373813 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbba0043df0 con 0x7fbba40a9690 2022-07-21T01:57:17.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.328+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1046373813 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fbba4141b80 con 0x7fbba40a9690 2022-07-21T01:57:17.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.336+0000 7fbb967fc700 1 -- 172.21.15.140:0/1046373813 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v44) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbba0023020 con 0x7fbba40a9690 2022-07-21T01:57:17.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1046373813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb8c053b00 msgr2=0x7fbb8c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb8c053b00 0x7fbb8c055fc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7fbb9800d000 tx=0x7fbb98007b90).stop 2022-07-21T01:57:17.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1046373813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbba40a9690 msgr2=0x7fbba409e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbba40a9690 0x7fbba409e550 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7fbba0002f70 tx=0x7fbba0015000).stop 2022-07-21T01:57:17.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1046373813 shutdown_connections 2022-07-21T01:57:17.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbb8c053b00 0x7fbb8c055fc0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbba40b4c20 0x7fbba40a89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbba40af0d0 0x7fbba40a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 --2- 172.21.15.140:0/1046373813 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbba40a9690 0x7fbba409e550 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1046373813 >> 172.21.15.140:0/1046373813 conn(0x7fbba4063150 msgr2=0x7fbba4145030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:17.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1046373813 shutdown_connections 2022-07-21T01:57:17.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.340+0000 7fbbaa4ce700 1 -- 172.21.15.140:0/1046373813 wait complete. 2022-07-21T01:57:17.345 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:57:17.352 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:57:17.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.456+0000 7fbf0bfff700 1 Processor -- start 2022-07-21T01:57:17.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.456+0000 7fbf0bfff700 1 -- start start 2022-07-21T01:57:17.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbf0c067830 0x7fbf0c140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf0c140c60 0x7fbf0c1410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0c0b4340 con 0x7fbf0c13a770 2022-07-21T01:57:17.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0c066680 con 0x7fbf0c067830 2022-07-21T01:57:17.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c0b1d80 con 0x7fbf0c140c60 2022-07-21T01:57:17.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf027fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbf0c067830 0x7fbf0c140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf027fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fbf0c067830 0x7fbf0c140770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40410/0 (socket says 172.21.15.140:40410) 2022-07-21T01:57:17.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf027fc700 1 -- 172.21.15.140:0/1030139355 learned_addr learned my addr 172.21.15.140:0/1030139355 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:17.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0b7fe700 1 --2- 172.21.15.140:0/1030139355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf0c140c60 0x7fbf0c1410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf027fc700 1 -- 172.21.15.140:0/1030139355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf0c140c60 msgr2=0x7fbf0c1410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:17.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf027fc700 1 --2- 172.21.15.140:0/1030139355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf0c140c60 0x7fbf0c1410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf027fc700 1 -- 172.21.15.140:0/1030139355 >> v1:172.21.15.152:6789/0 conn(0x7fbf0c13a770 legacy=0x7fbf0c067120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0a7fc700 1 -- 172.21.15.140:0/1030139355 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204600740 0 0) 0x7fbf0c0b4340 con 0x7fbf0c13a770 2022-07-21T01:57:17.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf027fc700 1 -- 172.21.15.140:0/1030139355 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c0b1ae0 con 0x7fbf0c067830 2022-07-21T01:57:17.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf027fc700 1 --2- 172.21.15.140:0/1030139355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf0c067830 0x7fbf0c140770 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7fbef4002f70 tx=0x7fbef4006eb0).ready entity=mon.0 client_cookie=f027667194d3fd2 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:17.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0a7fc700 1 -- 172.21.15.140:0/1030139355 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef400d570 con 0x7fbf0c067830 2022-07-21T01:57:17.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0a7fc700 1 -- 172.21.15.140:0/1030139355 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef4000e90 con 0x7fbf0c067830 2022-07-21T01:57:17.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0a7fc700 1 -- 172.21.15.140:0/1030139355 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef400d8c0 con 0x7fbf0c067830 2022-07-21T01:57:17.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 -- 172.21.15.140:0/1030139355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf0c067830 msgr2=0x7fbf0c140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 --2- 172.21.15.140:0/1030139355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf0c067830 0x7fbf0c140770 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7fbef4002f70 tx=0x7fbef4006eb0).stop 2022-07-21T01:57:17.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 -- 172.21.15.140:0/1030139355 shutdown_connections 2022-07-21T01:57:17.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 --2- 172.21.15.140:0/1030139355 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf0c140c60 0x7fbf0c1410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 --2- 172.21.15.140:0/1030139355 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf0c067830 0x7fbf0c140770 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 -- 172.21.15.140:0/1030139355 >> 172.21.15.140:0/1030139355 conn(0x7fbf0c064c00 msgr2=0x7fbf0c064fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:17.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 -- 172.21.15.140:0/1030139355 shutdown_connections 2022-07-21T01:57:17.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.460+0000 7fbf0bfff700 1 -- 172.21.15.140:0/1030139355 wait complete. 2022-07-21T01:57:17.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf0bfff700 1 Processor -- start 2022-07-21T01:57:17.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf0bfff700 1 -- start start 2022-07-21T01:57:17.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf0bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf0c067830 0x7fbf0c0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf0bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf0c13a770 0x7fbf0c0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf0bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf0c140c60 0x7fbf0c1532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf0bfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c150860 con 0x7fbf0c13a770 2022-07-21T01:57:17.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf0bfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c13d980 con 0x7fbf0c140c60 2022-07-21T01:57:17.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf0bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c13d6e0 con 0x7fbf0c067830 2022-07-21T01:57:17.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf027fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf0c13a770 0x7fbf0c0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf027fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf0c13a770 0x7fbf0c0aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40416/0 (socket says 172.21.15.140:40416) 2022-07-21T01:57:17.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf027fc700 1 -- 172.21.15.140:0/3665597353 learned_addr learned my addr 172.21.15.140:0/3665597353 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:17.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf0b7fe700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf0c140c60 0x7fbf0c1532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf0affd700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf0c067830 0x7fbf0c0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf027fc700 1 -- 172.21.15.140:0/3665597353 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf0c067830 msgr2=0x7fbf0c0afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf027fc700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf0c067830 0x7fbf0c0afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf027fc700 1 -- 172.21.15.140:0/3665597353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf0c140c60 msgr2=0x7fbf0c1532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf027fc700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf0c140c60 0x7fbf0c1532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.464+0000 7fbf027fc700 1 -- 172.21.15.140:0/3665597353 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef400b1e0 con 0x7fbf0c13a770 2022-07-21T01:57:17.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.468+0000 7fbf027fc700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf0c13a770 0x7fbf0c0aff10 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7fbef4002f70 tx=0x7fbef4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:17.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.468+0000 7fbf08ff9700 1 -- 172.21.15.140:0/3665597353 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef40116e0 con 0x7fbf0c13a770 2022-07-21T01:57:17.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.468+0000 7fbf08ff9700 1 -- 172.21.15.140:0/3665597353 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef4011840 con 0x7fbf0c13a770 2022-07-21T01:57:17.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.468+0000 7fbf0bfff700 1 -- 172.21.15.140:0/3665597353 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0c13d2f0 con 0x7fbf0c13a770 2022-07-21T01:57:17.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.468+0000 7fbf08ff9700 1 -- 172.21.15.140:0/3665597353 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef4011ba0 con 0x7fbf0c13a770 2022-07-21T01:57:17.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.468+0000 7fbf0bfff700 1 -- 172.21.15.140:0/3665597353 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0c13e100 con 0x7fbf0c13a770 2022-07-21T01:57:17.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.468+0000 7fbf0bfff700 1 -- 172.21.15.140:0/3665597353 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0c1506b0 con 0x7fbf0c13a770 2022-07-21T01:57:17.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.468+0000 7fbf08ff9700 1 -- 172.21.15.140:0/3665597353 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fbef4025030 con 0x7fbf0c13a770 2022-07-21T01:57:17.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.468+0000 7fbf08ff9700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbef0053a70 0x7fbef0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.468+0000 7fbf08ff9700 1 -- 172.21.15.140:0/3665597353 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbef406d1c0 con 0x7fbf0c13a770 2022-07-21T01:57:17.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.468+0000 7fbf0affd700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbef0053a70 0x7fbef0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.472+0000 7fbf0affd700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbef0053a70 0x7fbef0055f30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fbefc00d000 tx=0x7fbefc007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:17.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.476+0000 7fbf08ff9700 1 -- 172.21.15.140:0/3665597353 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fbef4043fd0 con 0x7fbf0c13a770 2022-07-21T01:57:17.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.640+0000 7fbf0bfff700 1 -- 172.21.15.140:0/3665597353 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fbf0c05d270 con 0x7fbf0c13a770 2022-07-21T01:57:17.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf08ff9700 1 -- 172.21.15.140:0/3665597353 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v45) v1 ==== 184+0+0 (secure 0 0 0) 0x7fbef40230e0 con 0x7fbf0c13a770 2022-07-21T01:57:17.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 -- 172.21.15.140:0/3665597353 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbef0053a70 msgr2=0x7fbef0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbef0053a70 0x7fbef0055f30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fbefc00d000 tx=0x7fbefc007a00).stop 2022-07-21T01:57:17.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 -- 172.21.15.140:0/3665597353 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf0c13a770 msgr2=0x7fbf0c0aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf0c13a770 0x7fbf0c0aff10 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7fbef4002f70 tx=0x7fbef4015000).stop 2022-07-21T01:57:17.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 -- 172.21.15.140:0/3665597353 shutdown_connections 2022-07-21T01:57:17.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fbef0053a70 0x7fbef0055f30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf0c140c60 0x7fbf0c1532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fbf0c13a770 0x7fbf0c0aff10 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 --2- 172.21.15.140:0/3665597353 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbf0c067830 0x7fbf0c0afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 -- 172.21.15.140:0/3665597353 >> 172.21.15.140:0/3665597353 conn(0x7fbf0c064c00 msgr2=0x7fbf0c0b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:17.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 -- 172.21.15.140:0/3665597353 shutdown_connections 2022-07-21T01:57:17.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.648+0000 7fbf0bfff700 1 -- 172.21.15.140:0/3665597353 wait complete. 2022-07-21T01:57:17.655 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:57:17.662 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:17.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.764+0000 7ff85b3e0700 1 Processor -- start 2022-07-21T01:57:17.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.764+0000 7ff85b3e0700 1 -- start start 2022-07-21T01:57:17.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85b3e0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b1200 0x7ff8540b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85b3e0700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8540aea30 con 0x7ff85405bd90 2022-07-21T01:57:17.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85b3e0700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8540aec60 con 0x7ff8540b5650 2022-07-21T01:57:17.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85b3e0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff854142970 con 0x7ff8540b1200 2022-07-21T01:57:17.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85abdf700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff85405bd90 0x7ff854142d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54572/0 (socket says 172.21.15.140:54572) 2022-07-21T01:57:17.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85abdf700 1 -- 172.21.15.140:0/3614209264 learned_addr learned my addr 172.21.15.140:0/3614209264 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:17.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff859bdd700 1 --2- 172.21.15.140:0/3614209264 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b1200 0x7ff8540b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff8593dc700 1 -- 172.21.15.140:0/3614209264 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747058741 0 0) 0x7ff8540aea30 con 0x7ff85405bd90 2022-07-21T01:57:17.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff8593dc700 1 -- 172.21.15.140:0/3614209264 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff840003260 con 0x7ff85405bd90 2022-07-21T01:57:17.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff859bdd700 1 -- 172.21.15.140:0/3614209264 >> v1:172.21.15.152:6789/0 conn(0x7ff8540b5650 legacy=0x7ff85405c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff859bdd700 1 -- 172.21.15.140:0/3614209264 >> v1:172.21.15.140:6789/0 conn(0x7ff85405bd90 legacy=0x7ff854142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff859bdd700 1 -- 172.21.15.140:0/3614209264 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8540af180 con 0x7ff8540b1200 2022-07-21T01:57:17.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff859bdd700 1 --2- 172.21.15.140:0/3614209264 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b1200 0x7ff8540b5a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7ff8480032f0 tx=0x7ff8480091a0).ready entity=mon.2 client_cookie=da9c89a44bb267c8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:17.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff8593dc700 1 -- 172.21.15.140:0/3614209264 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff84800b6c0 con 0x7ff8540b1200 2022-07-21T01:57:17.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff8593dc700 1 -- 172.21.15.140:0/3614209264 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84800b820 con 0x7ff8540b1200 2022-07-21T01:57:17.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff8593dc700 1 -- 172.21.15.140:0/3614209264 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff84800b990 con 0x7ff8540b1200 2022-07-21T01:57:17.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3614209264 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b1200 msgr2=0x7ff8540b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85b3e0700 1 --2- 172.21.15.140:0/3614209264 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b1200 0x7ff8540b5a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7ff8480032f0 tx=0x7ff8480091a0).stop 2022-07-21T01:57:17.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3614209264 shutdown_connections 2022-07-21T01:57:17.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85b3e0700 1 --2- 172.21.15.140:0/3614209264 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b1200 0x7ff8540b5a30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3614209264 >> 172.21.15.140:0/3614209264 conn(0x7ff8540634e0 msgr2=0x7ff8540b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:17.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3614209264 shutdown_connections 2022-07-21T01:57:17.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.768+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3614209264 wait complete. 2022-07-21T01:57:17.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85b3e0700 1 Processor -- start 2022-07-21T01:57:17.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85b3e0700 1 -- start start 2022-07-21T01:57:17.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85b3e0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff85405bd90 0x7ff854145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85b3e0700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8540b1200 0x7ff854151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85b3e0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b5650 0x7ff854155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85b3e0700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85413ee70 con 0x7ff8540b1200 2022-07-21T01:57:17.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85b3e0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8541553e0 con 0x7ff85405bd90 2022-07-21T01:57:17.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85b3e0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85413f0a0 con 0x7ff8540b5650 2022-07-21T01:57:17.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff859bdd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8540b1200 0x7ff854151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85abdf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b5650 0x7ff854155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85abdf700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b5650 0x7ff854155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:37160/0 (socket says 172.21.15.140:37160) 2022-07-21T01:57:17.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85a3de700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff85405bd90 0x7ff854145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85abdf700 1 -- 172.21.15.140:0/3033208291 learned_addr learned my addr 172.21.15.140:0/3033208291 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:17.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff859bdd700 1 -- 172.21.15.140:0/3033208291 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b5650 msgr2=0x7ff854155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff859bdd700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b5650 0x7ff854155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff859bdd700 1 -- 172.21.15.140:0/3033208291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff85405bd90 msgr2=0x7ff854145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff859bdd700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff85405bd90 0x7ff854145ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff859bdd700 1 -- 172.21.15.140:0/3033208291 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff848009420 con 0x7ff8540b1200 2022-07-21T01:57:17.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85abdf700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b5650 0x7ff854155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:17.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff859bdd700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8540b1200 0x7ff854151e60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7ff8480032f0 tx=0x7ff848016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:17.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff8537fe700 1 -- 172.21.15.140:0/3033208291 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff84800b6c0 con 0x7ff8540b1200 2022-07-21T01:57:17.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff8537fe700 1 -- 172.21.15.140:0/3033208291 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff848005c60 con 0x7ff8540b1200 2022-07-21T01:57:17.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3033208291 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff854142690 con 0x7ff8540b1200 2022-07-21T01:57:17.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff8537fe700 1 -- 172.21.15.140:0/3033208291 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff848006020 con 0x7ff8540b1200 2022-07-21T01:57:17.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3033208291 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85413d0f0 con 0x7ff8540b1200 2022-07-21T01:57:17.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.772+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3033208291 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85413e280 con 0x7ff8540b1200 2022-07-21T01:57:17.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.776+0000 7ff8537fe700 1 -- 172.21.15.140:0/3033208291 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff84800f070 con 0x7ff8540b1200 2022-07-21T01:57:17.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.776+0000 7ff8537fe700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff834053a10 0x7ff834055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:17.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.776+0000 7ff8537fe700 1 -- 172.21.15.140:0/3033208291 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff84806caf0 con 0x7ff8540b1200 2022-07-21T01:57:17.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.776+0000 7ff85a3de700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff834053a10 0x7ff834055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:17.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.776+0000 7ff85a3de700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff834053a10 0x7ff834055ed0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7ff84400d000 tx=0x7ff844009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:17.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.780+0000 7ff8537fe700 1 -- 172.21.15.140:0/3033208291 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff85413e280 con 0x7ff8540b1200 2022-07-21T01:57:17.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.940+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3033208291 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff85413e280 con 0x7ff8540b1200 2022-07-21T01:57:17.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff8537fe700 1 -- 172.21.15.140:0/3033208291 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+2145 (secure 0 0 0) 0x7ff85413e280 con 0x7ff8540b1200 2022-07-21T01:57:17.944 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:17.945 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":64,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/558475048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":558475048},{"type":"v1","addr":"172.21.15.152:6833","nonce":558475048}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:14.423640+0000","modified":"2022-07-21T01:57:16.123489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2209871450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2209871450},{"type":"v1","addr":"172.21.15.140:6835","nonce":2209871450}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-07-21T01:57:17.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3033208291 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff834053a10 msgr2=0x7ff834055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff834053a10 0x7ff834055ed0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7ff84400d000 tx=0x7ff844009040).stop 2022-07-21T01:57:17.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3033208291 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8540b1200 msgr2=0x7ff854151e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:17.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8540b1200 0x7ff854151e60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7ff8480032f0 tx=0x7ff848016000).stop 2022-07-21T01:57:17.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3033208291 shutdown_connections 2022-07-21T01:57:17.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff834053a10 0x7ff834055ed0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff8540b5650 0x7ff854155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff8540b1200 0x7ff854151e60 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 --2- 172.21.15.140:0/3033208291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff85405bd90 0x7ff854145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:17.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3033208291 >> 172.21.15.140:0/3033208291 conn(0x7ff8540634e0 msgr2=0x7ff8540adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:17.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3033208291 shutdown_connections 2022-07-21T01:57:17.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:17.944+0000 7ff85b3e0700 1 -- 172.21.15.140:0/3033208291 wait complete. 2022-07-21T01:57:17.949 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 64 2022-07-21T01:57:18.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:57:14.423640+0000', 'modified': '2022-07-21T01:57:16.123489+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8061}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8061': {'gid': 8061, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/2209871450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 2209871450}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 2209871450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:57:18.159 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:57:18.160 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:57:18.274 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":8061,"want_state":"up:active","state":"up:active","rank_uptime":3.8382965590000002,"mdsmap_epoch":64,"osdmap_epoch":132,"osdmap_epoch_barrier":132,"uptime":8.5322604290000008} 2022-07-21T01:57:18.274 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 8061, "mdsmap_epoch": 64, "osdmap_epoch": 132, "osdmap_epoch_barrier": 132, "rank_uptime": 3.838296559, "state": "up:active", "uptime": 8.532260429, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:57:18.275 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:18.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 Processor -- start 2022-07-21T01:57:18.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 -- start start 2022-07-21T01:57:18.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f155c0b0570 0x7f155c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f155c0aea10 0x7f155c141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f155c1422c0 0x7f155c0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155c143b10 con 0x7f155c0b0570 2022-07-21T01:57:18.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155c142750 con 0x7f155c1422c0 2022-07-21T01:57:18.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155c1433d0 con 0x7f155c0aea10 2022-07-21T01:57:18.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155a59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f155c0b0570 0x7f155c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155a59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f155c0b0570 0x7f155c0b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40438/0 (socket says 172.21.15.140:40438) 2022-07-21T01:57:18.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155a59c700 1 -- 172.21.15.140:0/1958222856 learned_addr learned my addr 172.21.15.140:0/1958222856 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:18.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155ad9d700 1 --2- 172.21.15.140:0/1958222856 >> v2:172.21.15.152:3300/0 conn(0x7f155c1422c0 0x7f155c0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f1559d9b700 1 --2- 172.21.15.140:0/1958222856 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f155c0aea10 0x7f155c141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155a59c700 1 -- 172.21.15.140:0/1958222856 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f155c0aea10 msgr2=0x7f155c141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:18.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155a59c700 1 --2- 172.21.15.140:0/1958222856 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f155c0aea10 0x7f155c141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155a59c700 1 -- 172.21.15.140:0/1958222856 >> v2:172.21.15.152:3300/0 conn(0x7f155c1422c0 msgr2=0x7f155c0ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:18.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155a59c700 1 --2- 172.21.15.140:0/1958222856 >> v2:172.21.15.152:3300/0 conn(0x7f155c1422c0 0x7f155c0ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155a59c700 1 -- 172.21.15.140:0/1958222856 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c143670 con 0x7f155c0b0570 2022-07-21T01:57:18.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155a59c700 1 --2- 172.21.15.140:0/1958222856 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f155c0b0570 0x7f155c0b0950 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f154c002f70 tx=0x7f154c006eb0).ready entity=mon.0 client_cookie=5e59f911d0779c95 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:18.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155959a700 1 -- 172.21.15.140:0/1958222856 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154c00d570 con 0x7f155c0b0570 2022-07-21T01:57:18.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155959a700 1 -- 172.21.15.140:0/1958222856 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c000e90 con 0x7f155c0b0570 2022-07-21T01:57:18.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155959a700 1 -- 172.21.15.140:0/1958222856 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154c00d900 con 0x7f155c0b0570 2022-07-21T01:57:18.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 -- 172.21.15.140:0/1958222856 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f155c0b0570 msgr2=0x7f155c0b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 --2- 172.21.15.140:0/1958222856 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f155c0b0570 0x7f155c0b0950 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f154c002f70 tx=0x7f154c006eb0).stop 2022-07-21T01:57:18.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 -- 172.21.15.140:0/1958222856 shutdown_connections 2022-07-21T01:57:18.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 --2- 172.21.15.140:0/1958222856 >> v2:172.21.15.152:3300/0 conn(0x7f155c1422c0 0x7f155c0ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 --2- 172.21.15.140:0/1958222856 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f155c0aea10 0x7f155c141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 --2- 172.21.15.140:0/1958222856 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f155c0b0570 0x7f155c0b0950 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.380+0000 7f155b59e700 1 -- 172.21.15.140:0/1958222856 >> 172.21.15.140:0/1958222856 conn(0x7f155c0b9360 msgr2=0x7f155c0b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:18.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155b59e700 1 -- 172.21.15.140:0/1958222856 shutdown_connections 2022-07-21T01:57:18.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155b59e700 1 -- 172.21.15.140:0/1958222856 wait complete. 2022-07-21T01:57:18.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155b59e700 1 Processor -- start 2022-07-21T01:57:18.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155b59e700 1 -- start start 2022-07-21T01:57:18.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f155c0aea10 0x7f155c146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155c0b0570 0x7f155c13ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f155c1422c0 0x7f155c13ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c145330 con 0x7f155c0aea10 2022-07-21T01:57:18.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c0b38f0 con 0x7f155c0b0570 2022-07-21T01:57:18.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155c1401b0 con 0x7f155c1422c0 2022-07-21T01:57:18.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f155c0aea10 0x7f155c146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f155c0aea10 0x7f155c146000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40442/0 (socket says 172.21.15.140:40442) 2022-07-21T01:57:18.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155a59c700 1 -- 172.21.15.140:0/1034023983 learned_addr learned my addr 172.21.15.140:0/1034023983 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:18.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f1559d9b700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155c0b0570 0x7f155c13ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155ad9d700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f155c1422c0 0x7f155c13ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155a59c700 1 -- 172.21.15.140:0/1034023983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f155c1422c0 msgr2=0x7f155c13ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155a59c700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f155c1422c0 0x7f155c13ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155a59c700 1 -- 172.21.15.140:0/1034023983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155c0b0570 msgr2=0x7f155c13ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155a59c700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155c0b0570 0x7f155c13ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155a59c700 1 -- 172.21.15.140:0/1034023983 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c00b1e0 con 0x7f155c0aea10 2022-07-21T01:57:18.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f155a59c700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f155c0aea10 0x7f155c146000 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f154c002f70 tx=0x7f154c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:18.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.384+0000 7f154b7fe700 1 -- 172.21.15.140:0/1034023983 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154c011760 con 0x7f155c0aea10 2022-07-21T01:57:18.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.388+0000 7f155b59e700 1 -- 172.21.15.140:0/1034023983 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155c140920 con 0x7f155c0aea10 2022-07-21T01:57:18.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.388+0000 7f155b59e700 1 -- 172.21.15.140:0/1034023983 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155c0b26c0 con 0x7f155c0aea10 2022-07-21T01:57:18.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.388+0000 7f154b7fe700 1 -- 172.21.15.140:0/1034023983 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c0118c0 con 0x7f155c0aea10 2022-07-21T01:57:18.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.388+0000 7f154b7fe700 1 -- 172.21.15.140:0/1034023983 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154c011c20 con 0x7f155c0aea10 2022-07-21T01:57:18.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.388+0000 7f155b59e700 1 -- 172.21.15.140:0/1034023983 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155c1404d0 con 0x7f155c0aea10 2022-07-21T01:57:18.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.388+0000 7f154b7fe700 1 -- 172.21.15.140:0/1034023983 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f154c01a040 con 0x7f155c0aea10 2022-07-21T01:57:18.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.388+0000 7f154b7fe700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1538053b00 0x7f1538055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.388+0000 7f154b7fe700 1 -- 172.21.15.140:0/1034023983 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f154c06ccd0 con 0x7f155c0aea10 2022-07-21T01:57:18.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.388+0000 7f1559d9b700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1538053b00 0x7f1538055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.388+0000 7f1559d9b700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1538053b00 0x7f1538055fc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f155000b000 tx=0x7f1550006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:18.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.396+0000 7f154b7fe700 1 -- 172.21.15.140:0/1034023983 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f154c043b50 con 0x7f155c0aea10 2022-07-21T01:57:18.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.520+0000 7f155b59e700 1 -- 172.21.15.140:0/1034023983 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f155c1402f0 con 0x7f155c0aea10 2022-07-21T01:57:18.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f154b7fe700 1 -- 172.21.15.140:0/1034023983 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f154c023020 con 0x7f155c0aea10 2022-07-21T01:57:18.525 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:18.526 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":132,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:15.426209+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:57:12.443397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-07-21T01:57:13.427432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:18.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 -- 172.21.15.140:0/1034023983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1538053b00 msgr2=0x7f1538055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1538053b00 0x7f1538055fc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f155000b000 tx=0x7f1550006df0).stop 2022-07-21T01:57:18.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 -- 172.21.15.140:0/1034023983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f155c0aea10 msgr2=0x7f155c146000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f155c0aea10 0x7f155c146000 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f154c002f70 tx=0x7f154c015000).stop 2022-07-21T01:57:18.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 -- 172.21.15.140:0/1034023983 shutdown_connections 2022-07-21T01:57:18.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1538053b00 0x7f1538055fc0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f155c1422c0 0x7f155c13ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f155c0b0570 0x7f155c13ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 --2- 172.21.15.140:0/1034023983 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f155c0aea10 0x7f155c146000 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 -- 172.21.15.140:0/1034023983 >> 172.21.15.140:0/1034023983 conn(0x7f155c0b9360 msgr2=0x7f155c0b86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:18.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 -- 172.21.15.140:0/1034023983 shutdown_connections 2022-07-21T01:57:18.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.524+0000 7f155b59e700 1 -- 172.21.15.140:0/1034023983 wait complete. 2022-07-21T01:57:18.539 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:57:12.443397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:57:13.427432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:57:18.540 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:57:18.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.648+0000 7fe51638f700 1 Processor -- start 2022-07-21T01:57:18.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.648+0000 7fe51638f700 1 -- start start 2022-07-21T01:57:18.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.648+0000 7fe51638f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe5100ab360 0x7fe51014d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.648+0000 7fe51638f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5101460e0 con 0x7fe5100acbe0 2022-07-21T01:57:18.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.648+0000 7fe51638f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe510146380 con 0x7fe51005f840 2022-07-21T01:57:18.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.648+0000 7fe51638f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe510146620 con 0x7fe5100ab360 2022-07-21T01:57:18.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.648+0000 7fe51538d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe5100acbe0 0x7fe510146ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54596/0 (socket says 172.21.15.140:54596) 2022-07-21T01:57:18.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.648+0000 7fe51538d700 1 -- 172.21.15.140:0/929320821 learned_addr learned my addr 172.21.15.140:0/929320821 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:18.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.648+0000 7fe515b8e700 1 --2- 172.21.15.140:0/929320821 >> v2:172.21.15.140:3300/0 conn(0x7fe5100ab360 0x7fe51014d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.648+0000 7fe4fffff700 1 -- 172.21.15.140:0/929320821 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390301942 0 0) 0x7fe5101460e0 con 0x7fe5100acbe0 2022-07-21T01:57:18.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.648+0000 7fe4fffff700 1 -- 172.21.15.140:0/929320821 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4f8003260 con 0x7fe5100acbe0 2022-07-21T01:57:18.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe515b8e700 1 -- 172.21.15.140:0/929320821 >> v1:172.21.15.152:6789/0 conn(0x7fe51005f840 legacy=0x7fe510141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe515b8e700 1 -- 172.21.15.140:0/929320821 >> v1:172.21.15.140:6789/0 conn(0x7fe5100acbe0 legacy=0x7fe510146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe515b8e700 1 -- 172.21.15.140:0/929320821 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5100ab7b0 con 0x7fe5100ab360 2022-07-21T01:57:18.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe515b8e700 1 --2- 172.21.15.140:0/929320821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5100ab360 0x7fe51014d480 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7fe5080032f0 tx=0x7fe5080091d0).ready entity=mon.0 client_cookie=d81d64af030815c1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:18.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe4fffff700 1 -- 172.21.15.140:0/929320821 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe50800b6c0 con 0x7fe5100ab360 2022-07-21T01:57:18.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe4fffff700 1 -- 172.21.15.140:0/929320821 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50800b820 con 0x7fe5100ab360 2022-07-21T01:57:18.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe4fffff700 1 -- 172.21.15.140:0/929320821 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe50800b990 con 0x7fe5100ab360 2022-07-21T01:57:18.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe51638f700 1 -- 172.21.15.140:0/929320821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5100ab360 msgr2=0x7fe51014d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe51638f700 1 --2- 172.21.15.140:0/929320821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5100ab360 0x7fe51014d480 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7fe5080032f0 tx=0x7fe5080091d0).stop 2022-07-21T01:57:18.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe51638f700 1 -- 172.21.15.140:0/929320821 shutdown_connections 2022-07-21T01:57:18.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe51638f700 1 --2- 172.21.15.140:0/929320821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe5100ab360 0x7fe51014d480 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe51638f700 1 -- 172.21.15.140:0/929320821 >> 172.21.15.140:0/929320821 conn(0x7fe5100b2af0 msgr2=0x7fe5100b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:18.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe51638f700 1 -- 172.21.15.140:0/929320821 shutdown_connections 2022-07-21T01:57:18.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.652+0000 7fe51638f700 1 -- 172.21.15.140:0/929320821 wait complete. 2022-07-21T01:57:18.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51638f700 1 Processor -- start 2022-07-21T01:57:18.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51638f700 1 -- start start 2022-07-21T01:57:18.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51638f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe51005f840 0x7fe5100aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51638f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5100ab360 0x7fe5100aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51638f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5100acbe0 0x7fe51013ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51638f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe510061150 con 0x7fe51005f840 2022-07-21T01:57:18.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51638f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51013a9f0 con 0x7fe5100acbe0 2022-07-21T01:57:18.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51638f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5100af0f0 con 0x7fe5100ab360 2022-07-21T01:57:18.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51538d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe51005f840 0x7fe5100aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51538d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe51005f840 0x7fe5100aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40456/0 (socket says 172.21.15.140:40456) 2022-07-21T01:57:18.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51538d700 1 -- 172.21.15.140:0/182580233 learned_addr learned my addr 172.21.15.140:0/182580233 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:18.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe515b8e700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5100acbe0 0x7fe51013ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe514b8c700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5100ab360 0x7fe5100aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51538d700 1 -- 172.21.15.140:0/182580233 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5100ab360 msgr2=0x7fe5100aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51538d700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5100ab360 0x7fe5100aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51538d700 1 -- 172.21.15.140:0/182580233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5100acbe0 msgr2=0x7fe51013ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51538d700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5100acbe0 0x7fe51013ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51538d700 1 -- 172.21.15.140:0/182580233 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe508009470 con 0x7fe51005f840 2022-07-21T01:57:18.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe514b8c700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5100ab360 0x7fe5100aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:18.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51538d700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe51005f840 0x7fe5100aff50 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7fe500001290 tx=0x7fe500007950).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:18.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe4fe7fc700 1 -- 172.21.15.140:0/182580233 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe50000ce80 con 0x7fe51005f840 2022-07-21T01:57:18.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe4fe7fc700 1 -- 172.21.15.140:0/182580233 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5000070d0 con 0x7fe51005f840 2022-07-21T01:57:18.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51638f700 1 -- 172.21.15.140:0/182580233 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe510138600 con 0x7fe51005f840 2022-07-21T01:57:18.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe4fe7fc700 1 -- 172.21.15.140:0/182580233 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5000054e0 con 0x7fe51005f840 2022-07-21T01:57:18.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.656+0000 7fe51638f700 1 -- 172.21.15.140:0/182580233 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe510060c40 con 0x7fe51005f840 2022-07-21T01:57:18.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.660+0000 7fe51638f700 1 -- 172.21.15.140:0/182580233 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51013abb0 con 0x7fe51005f840 2022-07-21T01:57:18.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.660+0000 7fe4fe7fc700 1 -- 172.21.15.140:0/182580233 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe50001a060 con 0x7fe51005f840 2022-07-21T01:57:18.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.660+0000 7fe4fe7fc700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4f0053a70 0x7fe4f0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.660+0000 7fe514b8c700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4f0053a70 0x7fe4f0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.660+0000 7fe4fe7fc700 1 -- 172.21.15.140:0/182580233 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe50006b650 con 0x7fe51005f840 2022-07-21T01:57:18.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.660+0000 7fe514b8c700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4f0053a70 0x7fe4f0055f30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fe50c00d000 tx=0x7fe50c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:18.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.664+0000 7fe4fe7fc700 1 -- 172.21.15.140:0/182580233 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe500042460 con 0x7fe51005f840 2022-07-21T01:57:18.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 -- 172.21.15.140:0/182580233 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe5100574e0 con 0x7fe51005f840 2022-07-21T01:57:18.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe4fe7fc700 1 -- 172.21.15.140:0/182580233 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v132) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe500018070 con 0x7fe51005f840 2022-07-21T01:57:18.794 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:57:18.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 -- 172.21.15.140:0/182580233 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4f0053a70 msgr2=0x7fe4f0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4f0053a70 0x7fe4f0055f30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fe50c00d000 tx=0x7fe50c009040).stop 2022-07-21T01:57:18.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 -- 172.21.15.140:0/182580233 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe51005f840 msgr2=0x7fe5100aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe51005f840 0x7fe5100aff50 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7fe500001290 tx=0x7fe500007950).stop 2022-07-21T01:57:18.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 -- 172.21.15.140:0/182580233 shutdown_connections 2022-07-21T01:57:18.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe4f0053a70 0x7fe4f0055f30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5100acbe0 0x7fe51013ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe5100ab360 0x7fe5100aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 --2- 172.21.15.140:0/182580233 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe51005f840 0x7fe5100aff50 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 -- 172.21.15.140:0/182580233 >> 172.21.15.140:0/182580233 conn(0x7fe5100b2af0 msgr2=0x7fe510095410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:18.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 -- 172.21.15.140:0/182580233 shutdown_connections 2022-07-21T01:57:18.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.792+0000 7fe51638f700 1 -- 172.21.15.140:0/182580233 wait complete. 2022-07-21T01:57:18.807 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:57:18.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.912+0000 7fa17692e700 1 Processor -- start 2022-07-21T01:57:18.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.912+0000 7fa17692e700 1 -- start start 2022-07-21T01:57:18.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.912+0000 7fa17692e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa170140c30 0x7fa1700aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.912+0000 7fa17692e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1700aced0 con 0x7fa1700b0b90 2022-07-21T01:57:18.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.912+0000 7fa17692e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa170143c80 con 0x7fa1700ac610 2022-07-21T01:57:18.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.912+0000 7fa17692e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa170143f20 con 0x7fa170140c30 2022-07-21T01:57:18.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17612d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa170140c30 0x7fa1700aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17512b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa1700b0b90 0x7fa170140520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54612/0 (socket says 172.21.15.140:54612) 2022-07-21T01:57:18.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17612d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa170140c30 0x7fa1700aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40468/0 (socket says 172.21.15.140:40468) 2022-07-21T01:57:18.918 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17512b700 1 -- 172.21.15.140:0/962909814 learned_addr learned my addr 172.21.15.140:0/962909814 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:18.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17612d700 1 -- 172.21.15.140:0/962909814 >> v1:172.21.15.152:6789/0 conn(0x7fa1700ac610 legacy=0x7fa17005fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17612d700 1 -- 172.21.15.140:0/962909814 >> v1:172.21.15.140:6789/0 conn(0x7fa1700b0b90 legacy=0x7fa170140520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17492a700 1 -- 172.21.15.140:0/962909814 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699005841 0 0) 0x7fa1700aced0 con 0x7fa1700b0b90 2022-07-21T01:57:18.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17612d700 1 -- 172.21.15.140:0/962909814 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1700ac9f0 con 0x7fa170140c30 2022-07-21T01:57:18.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17612d700 1 --2- 172.21.15.140:0/962909814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa170140c30 0x7fa1700aba70 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7fa1680032f0 tx=0x7fa1680091d0).ready entity=mon.0 client_cookie=8b0a309fe7d6486a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:18.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17492a700 1 -- 172.21.15.140:0/962909814 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa168009ea0 con 0x7fa170140c30 2022-07-21T01:57:18.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17492a700 1 -- 172.21.15.140:0/962909814 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16800b8a0 con 0x7fa170140c30 2022-07-21T01:57:18.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17492a700 1 -- 172.21.15.140:0/962909814 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa16800ba10 con 0x7fa170140c30 2022-07-21T01:57:18.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 -- 172.21.15.140:0/962909814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa170140c30 msgr2=0x7fa1700aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 --2- 172.21.15.140:0/962909814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa170140c30 0x7fa1700aba70 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7fa1680032f0 tx=0x7fa1680091d0).stop 2022-07-21T01:57:18.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 -- 172.21.15.140:0/962909814 shutdown_connections 2022-07-21T01:57:18.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 --2- 172.21.15.140:0/962909814 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa170140c30 0x7fa1700aba70 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 -- 172.21.15.140:0/962909814 >> 172.21.15.140:0/962909814 conn(0x7fa1700b61f0 msgr2=0x7fa1700b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:18.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 -- 172.21.15.140:0/962909814 shutdown_connections 2022-07-21T01:57:18.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 -- 172.21.15.140:0/962909814 wait complete. 2022-07-21T01:57:18.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 Processor -- start 2022-07-21T01:57:18.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 -- start start 2022-07-21T01:57:18.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1700ac610 0x7fa1700b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1700b0b90 0x7fa1701526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa170140c30 0x7fa170152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa170066320 con 0x7fa1700b0b90 2022-07-21T01:57:18.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa170066550 con 0x7fa1700ac610 2022-07-21T01:57:18.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17692e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1700adb40 con 0x7fa170140c30 2022-07-21T01:57:18.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17512b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1700b0b90 0x7fa1701526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17512b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1700b0b90 0x7fa1701526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40470/0 (socket says 172.21.15.140:40470) 2022-07-21T01:57:18.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17512b700 1 -- 172.21.15.140:0/2068010140 learned_addr learned my addr 172.21.15.140:0/2068010140 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:18.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17612d700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa170140c30 0x7fa170152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.916+0000 7fa17592c700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1700ac610 0x7fa1700b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa17512b700 1 -- 172.21.15.140:0/2068010140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa170140c30 msgr2=0x7fa170152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa17512b700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa170140c30 0x7fa170152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa17512b700 1 -- 172.21.15.140:0/2068010140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1700ac610 msgr2=0x7fa1700b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:18.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa17512b700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1700ac610 0x7fa1700b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:18.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa17512b700 1 -- 172.21.15.140:0/2068010140 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168009470 con 0x7fa1700b0b90 2022-07-21T01:57:18.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa17592c700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1700ac610 0x7fa1700b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:18.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa17512b700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1700b0b90 0x7fa1701526e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7fa164000bc0 tx=0x7fa164007a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:18.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa162ffd700 1 -- 172.21.15.140:0/2068010140 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1640070d0 con 0x7fa1700b0b90 2022-07-21T01:57:18.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa162ffd700 1 -- 172.21.15.140:0/2068010140 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa164007230 con 0x7fa1700b0b90 2022-07-21T01:57:18.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa17692e700 1 -- 172.21.15.140:0/2068010140 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa170098060 con 0x7fa1700b0b90 2022-07-21T01:57:18.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa162ffd700 1 -- 172.21.15.140:0/2068010140 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa164005610 con 0x7fa1700b0b90 2022-07-21T01:57:18.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa17692e700 1 -- 172.21.15.140:0/2068010140 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1700676d0 con 0x7fa1700b0b90 2022-07-21T01:57:18.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa17692e700 1 -- 172.21.15.140:0/2068010140 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1700add00 con 0x7fa1700b0b90 2022-07-21T01:57:18.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa162ffd700 1 -- 172.21.15.140:0/2068010140 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa164010070 con 0x7fa1700b0b90 2022-07-21T01:57:18.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa162ffd700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1500539e0 0x7fa150055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:18.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.920+0000 7fa162ffd700 1 -- 172.21.15.140:0/2068010140 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa16406b770 con 0x7fa1700b0b90 2022-07-21T01:57:18.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.924+0000 7fa17592c700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1500539e0 0x7fa150055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:18.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.924+0000 7fa17592c700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1500539e0 0x7fa150055ea0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fa16c00b000 tx=0x7fa16c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:18.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:18.928+0000 7fa162ffd700 1 -- 172.21.15.140:0/2068010140 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa164042500 con 0x7fa1700b0b90 2022-07-21T01:57:19.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.056+0000 7fa17692e700 1 -- 172.21.15.140:0/2068010140 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa170153260 con 0x7fa1700b0b90 2022-07-21T01:57:19.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.056+0000 7fa162ffd700 1 -- 172.21.15.140:0/2068010140 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa164045b20 con 0x7fa1700b0b90 2022-07-21T01:57:19.057 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:19.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 -- 172.21.15.140:0/2068010140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1500539e0 msgr2=0x7fa150055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1500539e0 0x7fa150055ea0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fa16c00b000 tx=0x7fa16c009040).stop 2022-07-21T01:57:19.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 -- 172.21.15.140:0/2068010140 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1700b0b90 msgr2=0x7fa1701526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1700b0b90 0x7fa1701526e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7fa164000bc0 tx=0x7fa164007a40).stop 2022-07-21T01:57:19.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 -- 172.21.15.140:0/2068010140 shutdown_connections 2022-07-21T01:57:19.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa1500539e0 0x7fa150055ea0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa170140c30 0x7fa170152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1700b0b90 0x7fa1701526e0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 --2- 172.21.15.140:0/2068010140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1700ac610 0x7fa1700b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 -- 172.21.15.140:0/2068010140 >> 172.21.15.140:0/2068010140 conn(0x7fa1700b61f0 msgr2=0x7fa1700abba0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:19.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 -- 172.21.15.140:0/2068010140 shutdown_connections 2022-07-21T01:57:19.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.060+0000 7fa17692e700 1 -- 172.21.15.140:0/2068010140 wait complete. 2022-07-21T01:57:19.076 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:57:19.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c9c922700 1 Processor -- start 2022-07-21T01:57:19.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c9c922700 1 -- start start 2022-07-21T01:57:19.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c9c922700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9c980b59f0 0x7f9c981473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c9c922700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9805cc70 con 0x7f9c980b1400 2022-07-21T01:57:19.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c9c922700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c980b5dd0 con 0x7f9c980af940 2022-07-21T01:57:19.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c9c922700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c980b1070 con 0x7f9c980b59f0 2022-07-21T01:57:19.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c96d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9c980b59f0 0x7f9c981473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c9759e700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f9c980b1400 0x7f9c9814aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54628/0 (socket says 172.21.15.140:54628) 2022-07-21T01:57:19.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c9759e700 1 -- 172.21.15.140:0/677669791 learned_addr learned my addr 172.21.15.140:0/677669791 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:19.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c95d9b700 1 -- 172.21.15.140:0/677669791 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709625889 0 0) 0x7f9c980b5dd0 con 0x7f9c980af940 2022-07-21T01:57:19.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c96d9d700 1 -- 172.21.15.140:0/677669791 >> v1:172.21.15.152:6789/0 conn(0x7f9c980af940 legacy=0x7f9c9814a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c96d9d700 1 -- 172.21.15.140:0/677669791 >> v1:172.21.15.140:6789/0 conn(0x7f9c980b1400 legacy=0x7f9c9814aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c96d9d700 1 -- 172.21.15.140:0/677669791 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9805c760 con 0x7f9c980b59f0 2022-07-21T01:57:19.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c96d9d700 1 --2- 172.21.15.140:0/677669791 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c980b59f0 0x7f9c981473a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7f9c90001110 tx=0x7f9c80006df0).ready entity=mon.0 client_cookie=26b2b81b23dfb3cc server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:19.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c95d9b700 1 -- 172.21.15.140:0/677669791 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c8000d5c0 con 0x7f9c980b59f0 2022-07-21T01:57:19.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c95d9b700 1 -- 172.21.15.140:0/677669791 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c80000e90 con 0x7f9c980b59f0 2022-07-21T01:57:19.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.184+0000 7f9c95d9b700 1 -- 172.21.15.140:0/677669791 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c8000d910 con 0x7f9c980b59f0 2022-07-21T01:57:19.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 -- 172.21.15.140:0/677669791 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c980b59f0 msgr2=0x7f9c981473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 --2- 172.21.15.140:0/677669791 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c980b59f0 0x7f9c981473a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7f9c90001110 tx=0x7f9c80006df0).stop 2022-07-21T01:57:19.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 -- 172.21.15.140:0/677669791 shutdown_connections 2022-07-21T01:57:19.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 --2- 172.21.15.140:0/677669791 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c980b59f0 0x7f9c981473a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 -- 172.21.15.140:0/677669791 >> 172.21.15.140:0/677669791 conn(0x7f9c98063480 msgr2=0x7f9c980b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:19.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 -- 172.21.15.140:0/677669791 shutdown_connections 2022-07-21T01:57:19.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 -- 172.21.15.140:0/677669791 wait complete. 2022-07-21T01:57:19.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 Processor -- start 2022-07-21T01:57:19.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 -- start start 2022-07-21T01:57:19.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c980af940 0x7f9c9814a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c980b1400 0x7f9c9810fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c980b59f0 0x7f9c98065b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9810f520 con 0x7f9c980b59f0 2022-07-21T01:57:19.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9810f900 con 0x7f9c980b1400 2022-07-21T01:57:19.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9c922700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9810f280 con 0x7f9c980af940 2022-07-21T01:57:19.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c980b59f0 0x7f9c98065b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c980b59f0 0x7f9c98065b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40488/0 (socket says 172.21.15.140:40488) 2022-07-21T01:57:19.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.188+0000 7f9c9759e700 1 -- 172.21.15.140:0/556595946 learned_addr learned my addr 172.21.15.140:0/556595946 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:19.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c96d9d700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c980af940 0x7f9c9814a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c9659c700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c980b1400 0x7f9c9810fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c9759e700 1 -- 172.21.15.140:0/556595946 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c980af940 msgr2=0x7f9c9814a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c9759e700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c980af940 0x7f9c9814a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c9759e700 1 -- 172.21.15.140:0/556595946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c980b1400 msgr2=0x7f9c9810fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c9759e700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c980b1400 0x7f9c9810fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c9759e700 1 -- 172.21.15.140:0/556595946 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8000b180 con 0x7f9c980b59f0 2022-07-21T01:57:19.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c9659c700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c980b1400 0x7f9c9810fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:19.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c9759e700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c980b59f0 0x7f9c98065b10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f9c900011d0 tx=0x7f9c90007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:19.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c8ffff700 1 -- 172.21.15.140:0/556595946 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c9000ce20 con 0x7f9c980b59f0 2022-07-21T01:57:19.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c8ffff700 1 -- 172.21.15.140:0/556595946 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c900072e0 con 0x7f9c980b59f0 2022-07-21T01:57:19.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c9c922700 1 -- 172.21.15.140:0/556595946 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9810e830 con 0x7f9c980b59f0 2022-07-21T01:57:19.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c8ffff700 1 -- 172.21.15.140:0/556595946 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c90005800 con 0x7f9c980b59f0 2022-07-21T01:57:19.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c9c922700 1 -- 172.21.15.140:0/556595946 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c98067520 con 0x7f9c980b59f0 2022-07-21T01:57:19.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c9c922700 1 -- 172.21.15.140:0/556595946 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9805c9d0 con 0x7f9c980b59f0 2022-07-21T01:57:19.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c8ffff700 1 -- 172.21.15.140:0/556595946 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9c90010070 con 0x7f9c980b59f0 2022-07-21T01:57:19.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c8ffff700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c70053950 0x7f9c70055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c8ffff700 1 -- 172.21.15.140:0/556595946 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9c9006b660 con 0x7f9c980b59f0 2022-07-21T01:57:19.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.192+0000 7f9c96d9d700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c70053950 0x7f9c70055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.196+0000 7f9c96d9d700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c70053950 0x7f9c70055e10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f9c8001a000 tx=0x7f9c80014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:19.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.200+0000 7f9c8ffff700 1 -- 172.21.15.140:0/556595946 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c900424e0 con 0x7f9c980b59f0 2022-07-21T01:57:19.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.328+0000 7f9c9c922700 1 -- 172.21.15.140:0/556595946 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c980574e0 con 0x7f9c980b59f0 2022-07-21T01:57:19.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.328+0000 7f9c8ffff700 1 -- 172.21.15.140:0/556595946 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9c90018020 con 0x7f9c980b59f0 2022-07-21T01:57:19.331 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:19.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.328+0000 7f9c9c922700 1 -- 172.21.15.140:0/556595946 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c70053950 msgr2=0x7f9c70055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.328+0000 7f9c9c922700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c70053950 0x7f9c70055e10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f9c8001a000 tx=0x7f9c80014000).stop 2022-07-21T01:57:19.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.328+0000 7f9c9c922700 1 -- 172.21.15.140:0/556595946 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c980b59f0 msgr2=0x7f9c98065b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.332+0000 7f9c9c922700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c980b59f0 0x7f9c98065b10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f9c900011d0 tx=0x7f9c90007b50).stop 2022-07-21T01:57:19.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.332+0000 7f9c9c922700 1 -- 172.21.15.140:0/556595946 shutdown_connections 2022-07-21T01:57:19.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.332+0000 7f9c9c922700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c70053950 0x7f9c70055e10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.332+0000 7f9c9c922700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c980b59f0 0x7f9c98065b10 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.332+0000 7f9c9c922700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c980b1400 0x7f9c9810fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.332+0000 7f9c9c922700 1 --2- 172.21.15.140:0/556595946 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c980af940 0x7f9c9814a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.332+0000 7f9c9c922700 1 -- 172.21.15.140:0/556595946 >> 172.21.15.140:0/556595946 conn(0x7f9c98063480 msgr2=0x7f9c98140740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:19.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.332+0000 7f9c9c922700 1 -- 172.21.15.140:0/556595946 shutdown_connections 2022-07-21T01:57:19.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.332+0000 7f9c9c922700 1 -- 172.21.15.140:0/556595946 wait complete. 2022-07-21T01:57:19.344 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:57:19.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.448+0000 7ff344e60700 1 Processor -- start 2022-07-21T01:57:19.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff344e60700 1 -- start start 2022-07-21T01:57:19.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff344e60700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3401462a0 0x7ff340146680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff344e60700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff3400988b0 0x7ff340146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff344e60700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3400afdb0 con 0x7ff3401422c0 2022-07-21T01:57:19.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff344e60700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3400b0050 con 0x7ff3401462a0 2022-07-21T01:57:19.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff344e60700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34005fcd0 con 0x7ff3400988b0 2022-07-21T01:57:19.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33e59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3401462a0 0x7ff340146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33e59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff3401462a0 0x7ff340146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40496/0 (socket says 172.21.15.140:40496) 2022-07-21T01:57:19.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33e59c700 1 -- 172.21.15.140:0/2258328068 learned_addr learned my addr 172.21.15.140:0/2258328068 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:19.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33dd9b700 1 --2- 172.21.15.140:0/2258328068 >> v2:172.21.15.152:3300/0 conn(0x7ff3400988b0 0x7ff340146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33ed9d700 1 --1- 172.21.15.140:0/2258328068 >> v1:172.21.15.152:6789/0 conn(0x7ff3401422c0 0x7ff340060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:55752/0 (socket says 172.21.15.140:55752) 2022-07-21T01:57:19.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33d59a700 1 -- 172.21.15.140:0/2258328068 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76120253 0 0) 0x7ff3400afdb0 con 0x7ff3401422c0 2022-07-21T01:57:19.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33e59c700 1 -- 172.21.15.140:0/2258328068 >> v2:172.21.15.152:3300/0 conn(0x7ff3400988b0 msgr2=0x7ff340146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:19.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33e59c700 1 --2- 172.21.15.140:0/2258328068 >> v2:172.21.15.152:3300/0 conn(0x7ff3400988b0 0x7ff340146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33e59c700 1 -- 172.21.15.140:0/2258328068 >> v1:172.21.15.152:6789/0 conn(0x7ff3401422c0 legacy=0x7ff340060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33e59c700 1 -- 172.21.15.140:0/2258328068 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400afae0 con 0x7ff3401462a0 2022-07-21T01:57:19.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33dd9b700 1 --2- 172.21.15.140:0/2258328068 >> v2:172.21.15.152:3300/0 conn(0x7ff3400988b0 0x7ff340146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:19.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33e59c700 1 --2- 172.21.15.140:0/2258328068 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3401462a0 0x7ff340146680 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7ff334002f70 tx=0x7ff334006eb0).ready entity=mon.0 client_cookie=e534ea6f55464e54 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:19.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33d59a700 1 -- 172.21.15.140:0/2258328068 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff33400d5c0 con 0x7ff3401462a0 2022-07-21T01:57:19.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33d59a700 1 -- 172.21.15.140:0/2258328068 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff334000e90 con 0x7ff3401462a0 2022-07-21T01:57:19.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.452+0000 7ff33d59a700 1 -- 172.21.15.140:0/2258328068 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff33400d910 con 0x7ff3401462a0 2022-07-21T01:57:19.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 -- 172.21.15.140:0/2258328068 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3401462a0 msgr2=0x7ff340146680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 --2- 172.21.15.140:0/2258328068 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3401462a0 0x7ff340146680 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7ff334002f70 tx=0x7ff334006eb0).stop 2022-07-21T01:57:19.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 -- 172.21.15.140:0/2258328068 shutdown_connections 2022-07-21T01:57:19.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 --2- 172.21.15.140:0/2258328068 >> v2:172.21.15.152:3300/0 conn(0x7ff3400988b0 0x7ff340146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 --2- 172.21.15.140:0/2258328068 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3401462a0 0x7ff340146680 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 -- 172.21.15.140:0/2258328068 >> 172.21.15.140:0/2258328068 conn(0x7ff3400b5f00 msgr2=0x7ff3400b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:19.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 -- 172.21.15.140:0/2258328068 shutdown_connections 2022-07-21T01:57:19.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 -- 172.21.15.140:0/2258328068 wait complete. 2022-07-21T01:57:19.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 Processor -- start 2022-07-21T01:57:19.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 -- start start 2022-07-21T01:57:19.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3400988b0 0x7ff340099f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3401422c0 0x7ff340151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3401462a0 0x7ff340151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34009cc20 con 0x7ff3401462a0 2022-07-21T01:57:19.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340152c10 con 0x7ff3400988b0 2022-07-21T01:57:19.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff344e60700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34009ce50 con 0x7ff3401422c0 2022-07-21T01:57:19.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff33ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3401462a0 0x7ff340151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff33ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3401462a0 0x7ff340151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40500/0 (socket says 172.21.15.140:40500) 2022-07-21T01:57:19.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff33ed9d700 1 -- 172.21.15.140:0/2478173916 learned_addr learned my addr 172.21.15.140:0/2478173916 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:19.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff33e59c700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3400988b0 0x7ff340099f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff33dd9b700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3401422c0 0x7ff340151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff33ed9d700 1 -- 172.21.15.140:0/2478173916 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3401422c0 msgr2=0x7ff340151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff33ed9d700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3401422c0 0x7ff340151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff33ed9d700 1 -- 172.21.15.140:0/2478173916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3400988b0 msgr2=0x7ff340099f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff33ed9d700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3400988b0 0x7ff340099f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.456+0000 7ff33ed9d700 1 -- 172.21.15.140:0/2478173916 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33000b180 con 0x7ff3401462a0 2022-07-21T01:57:19.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff33ed9d700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3401462a0 0x7ff340151c30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7ff338000f40 tx=0x7ff338007d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:19.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff32f7fe700 1 -- 172.21.15.140:0/2478173916 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff338005ae0 con 0x7ff3401462a0 2022-07-21T01:57:19.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff32f7fe700 1 -- 172.21.15.140:0/2478173916 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff338005c40 con 0x7ff3401462a0 2022-07-21T01:57:19.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff344e60700 1 -- 172.21.15.140:0/2478173916 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33400b1e0 con 0x7ff3401462a0 2022-07-21T01:57:19.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff32f7fe700 1 -- 172.21.15.140:0/2478173916 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3380070d0 con 0x7ff3401462a0 2022-07-21T01:57:19.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff344e60700 1 -- 172.21.15.140:0/2478173916 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34009d0f0 con 0x7ff3401462a0 2022-07-21T01:57:19.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff344e60700 1 -- 172.21.15.140:0/2478173916 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34009e080 con 0x7ff3401462a0 2022-07-21T01:57:19.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff32f7fe700 1 -- 172.21.15.140:0/2478173916 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff33801a060 con 0x7ff3401462a0 2022-07-21T01:57:19.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff32f7fe700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff31c053a70 0x7ff31c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff32f7fe700 1 -- 172.21.15.140:0/2478173916 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff33806b8f0 con 0x7ff3401462a0 2022-07-21T01:57:19.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff33e59c700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff31c053a70 0x7ff31c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.460+0000 7ff33e59c700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff31c053a70 0x7ff31c055f30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7ff33401c000 tx=0x7ff334015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:19.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.468+0000 7ff32f7fe700 1 -- 172.21.15.140:0/2478173916 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff34009e080 con 0x7ff3401462a0 2022-07-21T01:57:19.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.592+0000 7ff344e60700 1 -- 172.21.15.140:0/2478173916 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff34009e080 con 0x7ff3401462a0 2022-07-21T01:57:19.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff32f7fe700 1 -- 172.21.15.140:0/2478173916 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff34009e080 con 0x7ff3401462a0 2022-07-21T01:57:19.597 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:19.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 -- 172.21.15.140:0/2478173916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff31c053a70 msgr2=0x7ff31c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff31c053a70 0x7ff31c055f30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7ff33401c000 tx=0x7ff334015000).stop 2022-07-21T01:57:19.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 -- 172.21.15.140:0/2478173916 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3401462a0 msgr2=0x7ff340151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3401462a0 0x7ff340151c30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7ff338000f40 tx=0x7ff338007d60).stop 2022-07-21T01:57:19.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 -- 172.21.15.140:0/2478173916 shutdown_connections 2022-07-21T01:57:19.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff31c053a70 0x7ff31c055f30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3401462a0 0x7ff340151c30 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3401422c0 0x7ff340151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 --2- 172.21.15.140:0/2478173916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3400988b0 0x7ff340099f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 -- 172.21.15.140:0/2478173916 >> 172.21.15.140:0/2478173916 conn(0x7ff3400b5f00 msgr2=0x7ff340140d60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:19.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 -- 172.21.15.140:0/2478173916 shutdown_connections 2022-07-21T01:57:19.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.596+0000 7ff344e60700 1 -- 172.21.15.140:0/2478173916 wait complete. 2022-07-21T01:57:19.611 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:57:19.612 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:19.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.708+0000 7f883c4a1700 1 Processor -- start 2022-07-21T01:57:19.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.708+0000 7f883c4a1700 1 -- start start 2022-07-21T01:57:19.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.708+0000 7f883c4a1700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f88340b1e00 0x7f883414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.708+0000 7f883c4a1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883405d480 con 0x7f883405c3f0 2022-07-21T01:57:19.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.708+0000 7f883c4a1700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88340b21e0 con 0x7f88340acef0 2022-07-21T01:57:19.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.708+0000 7f883c4a1700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883405cf40 con 0x7f88340b1e00 2022-07-21T01:57:19.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f8839a3c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f883405c3f0 0x7f88340ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54656/0 (socket says 172.21.15.140:54656) 2022-07-21T01:57:19.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f8839a3c700 1 -- 172.21.15.140:0/3992232396 learned_addr learned my addr 172.21.15.140:0/3992232396 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:19.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883a23d700 1 --2- 172.21.15.140:0/3992232396 >> v2:172.21.15.152:3300/0 conn(0x7f88340b1e00 0x7f883414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883923b700 1 -- 172.21.15.140:0/3992232396 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297049194 0 0) 0x7f883405d480 con 0x7f883405c3f0 2022-07-21T01:57:19.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883923b700 1 -- 172.21.15.140:0/3992232396 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f881c003260 con 0x7f883405c3f0 2022-07-21T01:57:19.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883a23d700 1 -- 172.21.15.140:0/3992232396 >> v1:172.21.15.152:6789/0 conn(0x7f88340acef0 legacy=0x7f8834146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883a23d700 1 -- 172.21.15.140:0/3992232396 >> v1:172.21.15.140:6789/0 conn(0x7f883405c3f0 legacy=0x7f88340ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883a23d700 1 -- 172.21.15.140:0/3992232396 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883405d1e0 con 0x7f88340b1e00 2022-07-21T01:57:19.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883a23d700 1 --2- 172.21.15.140:0/3992232396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88340b1e00 0x7f883414a6d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f882c0032f0 tx=0x7f882c0091d0).ready entity=mon.1 client_cookie=784d43724fdc643 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:19.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883923b700 1 -- 172.21.15.140:0/3992232396 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f882c00b6c0 con 0x7f88340b1e00 2022-07-21T01:57:19.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883923b700 1 -- 172.21.15.140:0/3992232396 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c00b820 con 0x7f88340b1e00 2022-07-21T01:57:19.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883923b700 1 -- 172.21.15.140:0/3992232396 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f882c00b990 con 0x7f88340b1e00 2022-07-21T01:57:19.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883c4a1700 1 -- 172.21.15.140:0/3992232396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88340b1e00 msgr2=0x7f883414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883c4a1700 1 --2- 172.21.15.140:0/3992232396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88340b1e00 0x7f883414a6d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f882c0032f0 tx=0x7f882c0091d0).stop 2022-07-21T01:57:19.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883c4a1700 1 -- 172.21.15.140:0/3992232396 shutdown_connections 2022-07-21T01:57:19.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883c4a1700 1 --2- 172.21.15.140:0/3992232396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88340b1e00 0x7f883414a6d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883c4a1700 1 -- 172.21.15.140:0/3992232396 >> 172.21.15.140:0/3992232396 conn(0x7f88340611c0 msgr2=0x7f8834062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:19.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883c4a1700 1 -- 172.21.15.140:0/3992232396 shutdown_connections 2022-07-21T01:57:19.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883c4a1700 1 -- 172.21.15.140:0/3992232396 wait complete. 2022-07-21T01:57:19.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883c4a1700 1 Processor -- start 2022-07-21T01:57:19.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.712+0000 7f883c4a1700 1 -- start start 2022-07-21T01:57:19.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883c4a1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88340acef0 0x7f88340b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883c4a1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88340b1e00 0x7f883413d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883c4a1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8834149670 0x7f883413d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883c4a1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88340a1a10 con 0x7f88340acef0 2022-07-21T01:57:19.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883c4a1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88340a2020 con 0x7f88340b1e00 2022-07-21T01:57:19.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883c4a1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88340a12d0 con 0x7f8834149670 2022-07-21T01:57:19.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883a23d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88340acef0 0x7f88340b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883a23d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88340acef0 0x7f88340b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40512/0 (socket says 172.21.15.140:40512) 2022-07-21T01:57:19.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883a23d700 1 -- 172.21.15.140:0/1792307453 learned_addr learned my addr 172.21.15.140:0/1792307453 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:19.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f8839a3c700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88340b1e00 0x7f883413d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883aa3e700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8834149670 0x7f883413d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883a23d700 1 -- 172.21.15.140:0/1792307453 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8834149670 msgr2=0x7f883413d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883a23d700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8834149670 0x7f883413d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883a23d700 1 -- 172.21.15.140:0/1792307453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88340b1e00 msgr2=0x7f883413d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883a23d700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88340b1e00 0x7f883413d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883a23d700 1 -- 172.21.15.140:0/1792307453 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c009470 con 0x7f88340acef0 2022-07-21T01:57:19.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883a23d700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88340acef0 0x7f88340b3330 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f882c0032f0 tx=0x7f882c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:19.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f88237fe700 1 -- 172.21.15.140:0/1792307453 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f882c0107a0 con 0x7f88340acef0 2022-07-21T01:57:19.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f88237fe700 1 -- 172.21.15.140:0/1792307453 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c010900 con 0x7f88340acef0 2022-07-21T01:57:19.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883c4a1700 1 -- 172.21.15.140:0/1792307453 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88340a1570 con 0x7f88340acef0 2022-07-21T01:57:19.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f88237fe700 1 -- 172.21.15.140:0/1792307453 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f882c010c60 con 0x7f88340acef0 2022-07-21T01:57:19.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883c4a1700 1 -- 172.21.15.140:0/1792307453 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883413dbc0 con 0x7f88340acef0 2022-07-21T01:57:19.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f883c4a1700 1 -- 172.21.15.140:0/1792307453 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88340b04b0 con 0x7f88340acef0 2022-07-21T01:57:19.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f88237fe700 1 -- 172.21.15.140:0/1792307453 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f882c027020 con 0x7f88340acef0 2022-07-21T01:57:19.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.716+0000 7f88237fe700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8814053aa0 0x7f8814055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:19.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.720+0000 7f8839a3c700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8814053aa0 0x7f8814055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:19.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.720+0000 7f88237fe700 1 -- 172.21.15.140:0/1792307453 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f882c06d3e0 con 0x7f88340acef0 2022-07-21T01:57:19.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.720+0000 7f8839a3c700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8814053aa0 0x7f8814055f60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f882400b000 tx=0x7f8824009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:19.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.724+0000 7f88237fe700 1 -- 172.21.15.140:0/1792307453 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f882c06ae90 con 0x7f88340acef0 2022-07-21T01:57:19.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.880+0000 7f883c4a1700 1 -- 172.21.15.140:0/1792307453 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8834069a60 con 0x7f88340acef0 2022-07-21T01:57:19.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.880+0000 7f88237fe700 1 -- 172.21.15.140:0/1792307453 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+2145 (secure 0 0 0) 0x7f882c015090 con 0x7f88340acef0 2022-07-21T01:57:19.883 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:19.883 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":64,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/558475048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":558475048},{"type":"v1","addr":"172.21.15.152:6833","nonce":558475048}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:14.423640+0000","modified":"2022-07-21T01:57:16.123489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2209871450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2209871450},{"type":"v1","addr":"172.21.15.140:6835","nonce":2209871450}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-07-21T01:57:19.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.880+0000 7f883c4a1700 1 -- 172.21.15.140:0/1792307453 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8814053aa0 msgr2=0x7f8814055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.880+0000 7f883c4a1700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8814053aa0 0x7f8814055f60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f882400b000 tx=0x7f8824009040).stop 2022-07-21T01:57:19.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.880+0000 7f883c4a1700 1 -- 172.21.15.140:0/1792307453 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88340acef0 msgr2=0x7f88340b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:19.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.880+0000 7f883c4a1700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88340acef0 0x7f88340b3330 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f882c0032f0 tx=0x7f882c016000).stop 2022-07-21T01:57:19.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.884+0000 7f883c4a1700 1 -- 172.21.15.140:0/1792307453 shutdown_connections 2022-07-21T01:57:19.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.884+0000 7f883c4a1700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8814053aa0 0x7f8814055f60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.884+0000 7f883c4a1700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8834149670 0x7f883413d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.884+0000 7f883c4a1700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88340b1e00 0x7f883413d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.884+0000 7f883c4a1700 1 --2- 172.21.15.140:0/1792307453 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88340acef0 0x7f88340b3330 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:19.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.884+0000 7f883c4a1700 1 -- 172.21.15.140:0/1792307453 >> 172.21.15.140:0/1792307453 conn(0x7f883405c3f0 msgr2=0x7f88340afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:19.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.884+0000 7f883c4a1700 1 -- 172.21.15.140:0/1792307453 shutdown_connections 2022-07-21T01:57:19.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:19.884+0000 7f883c4a1700 1 -- 172.21.15.140:0/1792307453 wait complete. 2022-07-21T01:57:19.888 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 64 2022-07-21T01:57:20.097 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:57:14.423640+0000', 'modified': '2022-07-21T01:57:16.123489+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8061}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8061': {'gid': 8061, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/2209871450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 2209871450}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 2209871450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:57:20.098 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:57:20.099 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:57:20.216 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":8061,"want_state":"up:active","state":"up:active","rank_uptime":5.7808818310000003,"mdsmap_epoch":64,"osdmap_epoch":132,"osdmap_epoch_barrier":132,"uptime":10.474847112000001} 2022-07-21T01:57:20.217 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 8061, "mdsmap_epoch": 64, "osdmap_epoch": 132, "osdmap_epoch_barrier": 132, "rank_uptime": 5.780881831, "state": "up:active", "uptime": 10.474847112, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:57:20.218 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:57:20.219 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:57:20.219 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:57:20.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:20.221 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:20.231 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:57:20.289 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:20.290 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:20.307 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:20.308 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:20.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:20.309 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:57:20.315 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:57:20.316 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:20.319 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:57:20.335 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:20.336 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:20.392 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:20.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:57:20.386+0000 7f87ccae3080 -1 init, newargv = 0x56069b8f5220 newargc=9 2022-07-21T01:57:20.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23325]: starting ceph client 2022-07-21T01:57:20.395 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:20.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:20.397 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:57:20.402 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23325]: starting fuse 2022-07-21T01:57:20.404 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:57:20.404 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:57:20.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:20.406 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:57:20.407 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:57:20.408 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:57:20.409 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:57:20.409 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:57:20.421 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:57:20.421 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:57:20.422 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:57:20.423 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:57:20.424 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:57:20.425 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:57:20.426 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:57:20.427 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:57:20.428 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:57:20.429 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:57:20.429 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:57:20.430 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:57:20.431 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:57:20.432 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:57:20.433 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:57:20.433 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:57:20.434 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:57:20.435 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:57:20.493 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.23325.asok 2022-07-21T01:57:20.499 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23325.asok 2022-07-21T01:57:20.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:20.500 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23325.asok status 2022-07-21T01:57:20.591 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:57:20.591 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:57:20.592 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:57:20.592 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:57:20.592 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:57:20.593 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:57:20.593 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:57:20.594 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "23325", 2022-07-21T01:57:20.594 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:57:20.595 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:57:20.595 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:57:20.595 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:57:20.595 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 7124, 2022-07-21T01:57:20.596 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:57:20.596 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:57:20.596 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:57:20.596 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 7124 2022-07-21T01:57:20.596 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:57:20.597 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:57:20.597 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:57:20.597 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:57:20.597 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 184548146 2022-07-21T01:57:20.598 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:57:20.598 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:57:20.598 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:57:20.598 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:57:20.599 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:57:20.599 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 184548146 2022-07-21T01:57:20.599 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:57:20.599 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.7124 172.21.15.191:0/184548146", 2022-07-21T01:57:20.599 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/184548146", 2022-07-21T01:57:20.600 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:57:20.600 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 64, 2022-07-21T01:57:20.600 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 132, 2022-07-21T01:57:20.600 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:57:20.601 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:57:20.601 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:57:20.601 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:57:20.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:20.603 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:20.604 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:20.615 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:57:20.617 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:57:20.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:20.618 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:20.685 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:20.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 Processor -- start 2022-07-21T01:57:20.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 -- start start 2022-07-21T01:57:20.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f728005c3f0 0x7f728014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:20.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728005d480 con 0x7f72800acef0 2022-07-21T01:57:20.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72800b21e0 con 0x7f72800b1e00 2022-07-21T01:57:20.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728005cf40 con 0x7f728005c3f0 2022-07-21T01:57:20.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7286826700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f72800acef0 0x7f7280146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54668/0 (socket says 172.21.15.140:54668) 2022-07-21T01:57:20.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7286826700 1 -- 172.21.15.140:0/2359594195 learned_addr learned my addr 172.21.15.140:0/2359594195 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:20.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285824700 1 --2- 172.21.15.140:0/2359594195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f728005c3f0 0x7f728014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:20.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285023700 1 -- 172.21.15.140:0/2359594195 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356016821 0 0) 0x7f728005d480 con 0x7f72800acef0 2022-07-21T01:57:20.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285023700 1 -- 172.21.15.140:0/2359594195 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7268003260 con 0x7f72800acef0 2022-07-21T01:57:20.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285023700 1 -- 172.21.15.140:0/2359594195 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506010841 0 0) 0x7f72800b21e0 con 0x7f72800b1e00 2022-07-21T01:57:20.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285023700 1 -- 172.21.15.140:0/2359594195 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f728005d480 con 0x7f72800b1e00 2022-07-21T01:57:20.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285824700 1 -- 172.21.15.140:0/2359594195 >> v1:172.21.15.152:6789/0 conn(0x7f72800b1e00 legacy=0x7f72800ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:20.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285824700 1 -- 172.21.15.140:0/2359594195 >> v1:172.21.15.140:6789/0 conn(0x7f72800acef0 legacy=0x7f7280146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:20.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285824700 1 -- 172.21.15.140:0/2359594195 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728005d1e0 con 0x7f728005c3f0 2022-07-21T01:57:20.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285824700 1 --2- 172.21.15.140:0/2359594195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f728005c3f0 0x7f728014a6d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f7270002f70 tx=0x7f7270006df0).ready entity=mon.2 client_cookie=57bf4cfdc1b39c07 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:20.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285023700 1 -- 172.21.15.140:0/2359594195 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f727000d5c0 con 0x7f728005c3f0 2022-07-21T01:57:20.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285023700 1 -- 172.21.15.140:0/2359594195 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7270000e90 con 0x7f728005c3f0 2022-07-21T01:57:20.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7285023700 1 -- 172.21.15.140:0/2359594195 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f727000d910 con 0x7f728005c3f0 2022-07-21T01:57:20.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 -- 172.21.15.140:0/2359594195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f728005c3f0 msgr2=0x7f728014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:20.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 --2- 172.21.15.140:0/2359594195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f728005c3f0 0x7f728014a6d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f7270002f70 tx=0x7f7270006df0).stop 2022-07-21T01:57:20.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 -- 172.21.15.140:0/2359594195 shutdown_connections 2022-07-21T01:57:20.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 --2- 172.21.15.140:0/2359594195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f728005c3f0 0x7f728014a6d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:20.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 -- 172.21.15.140:0/2359594195 >> 172.21.15.140:0/2359594195 conn(0x7f72800611c0 msgr2=0x7f7280062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:20.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 -- 172.21.15.140:0/2359594195 shutdown_connections 2022-07-21T01:57:20.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.800+0000 7f7288289700 1 -- 172.21.15.140:0/2359594195 wait complete. 2022-07-21T01:57:20.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7288289700 1 Processor -- start 2022-07-21T01:57:20.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7288289700 1 -- start start 2022-07-21T01:57:20.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7288289700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72800acef0 0x7f728014e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:20.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7288289700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72800b1e00 0x7f728009ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:20.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7288289700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728009e080 0x7f728009f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:20.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7288289700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7280063e00 con 0x7f728009e080 2022-07-21T01:57:20.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7288289700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72800679f0 con 0x7f72800acef0 2022-07-21T01:57:20.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7288289700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7280067750 con 0x7f72800b1e00 2022-07-21T01:57:20.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7286826700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728009e080 0x7f728009f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:20.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7285824700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72800b1e00 0x7f728009ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:20.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7286025700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72800acef0 0x7f728014e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:20.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7286826700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728009e080 0x7f728009f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40530/0 (socket says 172.21.15.140:40530) 2022-07-21T01:57:20.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7286826700 1 -- 172.21.15.140:0/2018814530 learned_addr learned my addr 172.21.15.140:0/2018814530 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:20.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7286826700 1 -- 172.21.15.140:0/2018814530 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72800b1e00 msgr2=0x7f728009ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:20.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7286826700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72800b1e00 0x7f728009ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:20.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7286826700 1 -- 172.21.15.140:0/2018814530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72800acef0 msgr2=0x7f728014e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:20.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7286826700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72800acef0 0x7f728014e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:20.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.804+0000 7f7286826700 1 -- 172.21.15.140:0/2018814530 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727000b180 con 0x7f728009e080 2022-07-21T01:57:20.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.808+0000 7f7286826700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728009e080 0x7f728009f410 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f727c001110 tx=0x7f727c007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:20.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.808+0000 7f726f7fe700 1 -- 172.21.15.140:0/2018814530 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f727c00ce80 con 0x7f728009e080 2022-07-21T01:57:20.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.808+0000 7f726f7fe700 1 -- 172.21.15.140:0/2018814530 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c0070d0 con 0x7f728009e080 2022-07-21T01:57:20.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.808+0000 7f7288289700 1 -- 172.21.15.140:0/2018814530 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7280063a80 con 0x7f728009e080 2022-07-21T01:57:20.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.808+0000 7f726f7fe700 1 -- 172.21.15.140:0/2018814530 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f727c007430 con 0x7f728009e080 2022-07-21T01:57:20.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.808+0000 7f7288289700 1 -- 172.21.15.140:0/2018814530 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728009e820 con 0x7f728009e080 2022-07-21T01:57:20.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.808+0000 7f7288289700 1 -- 172.21.15.140:0/2018814530 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7280063c70 con 0x7f728009e080 2022-07-21T01:57:20.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.808+0000 7f726f7fe700 1 -- 172.21.15.140:0/2018814530 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f727c01a060 con 0x7f728009e080 2022-07-21T01:57:20.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.808+0000 7f726f7fe700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7260053a70 0x7f7260055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:20.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.812+0000 7f726f7fe700 1 -- 172.21.15.140:0/2018814530 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f727c06b430 con 0x7f728009e080 2022-07-21T01:57:20.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.812+0000 7f7286025700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7260053a70 0x7f7260055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:20.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.812+0000 7f7286025700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7260053a70 0x7f7260055f30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f727800d000 tx=0x7f7278007da0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:20.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.816+0000 7f726f7fe700 1 -- 172.21.15.140:0/2018814530 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f727c0422d0 con 0x7f728009e080 2022-07-21T01:57:20.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.940+0000 7f7288289700 1 -- 172.21.15.140:0/2018814530 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f728009e820 con 0x7f728009e080 2022-07-21T01:57:20.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.940+0000 7f726f7fe700 1 -- 172.21.15.140:0/2018814530 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f728009e820 con 0x7f728009e080 2022-07-21T01:57:20.944 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:20.944 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":132,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:15.426209+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:57:12.443397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-07-21T01:57:13.427432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:20.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 -- 172.21.15.140:0/2018814530 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7260053a70 msgr2=0x7f7260055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:20.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7260053a70 0x7f7260055f30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f727800d000 tx=0x7f7278007da0).stop 2022-07-21T01:57:20.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 -- 172.21.15.140:0/2018814530 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728009e080 msgr2=0x7f728009f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:20.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728009e080 0x7f728009f410 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f727c001110 tx=0x7f727c007a60).stop 2022-07-21T01:57:20.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 -- 172.21.15.140:0/2018814530 shutdown_connections 2022-07-21T01:57:20.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7260053a70 0x7f7260055f30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:20.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f728009e080 0x7f728009f410 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:20.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72800b1e00 0x7f728009ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:20.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 --2- 172.21.15.140:0/2018814530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72800acef0 0x7f728014e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:20.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 -- 172.21.15.140:0/2018814530 >> 172.21.15.140:0/2018814530 conn(0x7f728005c3f0 msgr2=0x7f72800ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:20.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 -- 172.21.15.140:0/2018814530 shutdown_connections 2022-07-21T01:57:20.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:20.944+0000 7f7288289700 1 -- 172.21.15.140:0/2018814530 wait complete. 2022-07-21T01:57:20.961 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:57:12.443397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:57:13.427432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:57:20.963 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:57:21.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.072+0000 7f412759e700 1 Processor -- start 2022-07-21T01:57:21.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.072+0000 7f412759e700 1 -- start start 2022-07-21T01:57:21.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.072+0000 7f412759e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f41280b9da0 0x7f41281468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.072+0000 7f412759e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f41281419a0 0x7f4128141e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.095 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.072+0000 7f412759e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41280ba180 con 0x7f41280b3cf0 2022-07-21T01:57:21.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.072+0000 7f412759e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41280ad5b0 con 0x7f41281419a0 2022-07-21T01:57:21.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.072+0000 7f412759e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41281461b0 con 0x7f41280b9da0 2022-07-21T01:57:21.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f4126d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f41281419a0 0x7f4128141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f4126d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f41281419a0 0x7f4128141e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40538/0 (socket says 172.21.15.140:40538) 2022-07-21T01:57:21.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f4126d9d700 1 -- 172.21.15.140:0/3004637394 learned_addr learned my addr 172.21.15.140:0/3004637394 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:21.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f4125d9b700 1 --1- 172.21.15.140:0/3004637394 >> v1:172.21.15.140:6789/0 conn(0x7f41280b3cf0 0x7f4128142a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54682/0 (socket says 172.21.15.140:54682) 2022-07-21T01:57:21.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412659c700 1 --2- 172.21.15.140:0/3004637394 >> v2:172.21.15.152:3300/0 conn(0x7f41280b9da0 0x7f41281468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412559a700 1 -- 172.21.15.140:0/3004637394 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239937059 0 0) 0x7f41280ba180 con 0x7f41280b3cf0 2022-07-21T01:57:21.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f4126d9d700 1 -- 172.21.15.140:0/3004637394 >> v2:172.21.15.152:3300/0 conn(0x7f41280b9da0 msgr2=0x7f41281468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:21.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f4126d9d700 1 --2- 172.21.15.140:0/3004637394 >> v2:172.21.15.152:3300/0 conn(0x7f41280b9da0 0x7f41281468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f4126d9d700 1 -- 172.21.15.140:0/3004637394 >> v1:172.21.15.140:6789/0 conn(0x7f41280b3cf0 legacy=0x7f4128142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f4126d9d700 1 -- 172.21.15.140:0/3004637394 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128146450 con 0x7f41281419a0 2022-07-21T01:57:21.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f4126d9d700 1 --2- 172.21.15.140:0/3004637394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41281419a0 0x7f4128141e50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f4120002f70 tx=0x7f4120006eb0).ready entity=mon.0 client_cookie=f352e782c272c098 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412559a700 1 -- 172.21.15.140:0/3004637394 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f412000d5c0 con 0x7f41281419a0 2022-07-21T01:57:21.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412559a700 1 -- 172.21.15.140:0/3004637394 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4120000e90 con 0x7f41281419a0 2022-07-21T01:57:21.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412559a700 1 -- 172.21.15.140:0/3004637394 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f412000d910 con 0x7f41281419a0 2022-07-21T01:57:21.106 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412759e700 1 -- 172.21.15.140:0/3004637394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41281419a0 msgr2=0x7f4128141e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412759e700 1 --2- 172.21.15.140:0/3004637394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41281419a0 0x7f4128141e50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f4120002f70 tx=0x7f4120006eb0).stop 2022-07-21T01:57:21.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412759e700 1 -- 172.21.15.140:0/3004637394 shutdown_connections 2022-07-21T01:57:21.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412759e700 1 --2- 172.21.15.140:0/3004637394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41281419a0 0x7f4128141e50 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412759e700 1 --2- 172.21.15.140:0/3004637394 >> v2:172.21.15.152:3300/0 conn(0x7f41280b9da0 0x7f41281468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412759e700 1 -- 172.21.15.140:0/3004637394 >> 172.21.15.140:0/3004637394 conn(0x7f41280678c0 msgr2=0x7f41280b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:21.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412759e700 1 -- 172.21.15.140:0/3004637394 shutdown_connections 2022-07-21T01:57:21.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412759e700 1 -- 172.21.15.140:0/3004637394 wait complete. 2022-07-21T01:57:21.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412759e700 1 Processor -- start 2022-07-21T01:57:21.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.076+0000 7f412759e700 1 -- start start 2022-07-21T01:57:21.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f412759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41280b3cf0 0x7f41280af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f412759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41280b9da0 0x7f412813f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f412759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f41281419a0 0x7f4128152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f412759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4128144c00 con 0x7f41280b9da0 2022-07-21T01:57:21.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f412759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4128151990 con 0x7f41280b3cf0 2022-07-21T01:57:21.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f412759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412813fce0 con 0x7f41281419a0 2022-07-21T01:57:21.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f4125d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41280b9da0 0x7f412813f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f4125d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41280b9da0 0x7f412813f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40542/0 (socket says 172.21.15.140:40542) 2022-07-21T01:57:21.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f4125d9b700 1 -- 172.21.15.140:0/1848889911 learned_addr learned my addr 172.21.15.140:0/1848889911 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:21.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f412659c700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41280b3cf0 0x7f41280af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f4126d9d700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f41281419a0 0x7f4128152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f4125d9b700 1 -- 172.21.15.140:0/1848889911 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f41281419a0 msgr2=0x7f4128152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f4125d9b700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f41281419a0 0x7f4128152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f4125d9b700 1 -- 172.21.15.140:0/1848889911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41280b3cf0 msgr2=0x7f41280af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f4125d9b700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41280b3cf0 0x7f41280af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f4125d9b700 1 -- 172.21.15.140:0/1848889911 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412000b1e0 con 0x7f41280b9da0 2022-07-21T01:57:21.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f4126d9d700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f41281419a0 0x7f4128152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:57:21.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f412659c700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41280b3cf0 0x7f41280af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:21.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f4125d9b700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41280b9da0 0x7f412813f7f0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f41280afcc0 tx=0x7f411c007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f41177fe700 1 -- 172.21.15.140:0/1848889911 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f411c0072e0 con 0x7f41280b9da0 2022-07-21T01:57:21.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f41177fe700 1 -- 172.21.15.140:0/1848889911 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c005550 con 0x7f41280b9da0 2022-07-21T01:57:21.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f412759e700 1 -- 172.21.15.140:0/1848889911 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4128155bd0 con 0x7f41280b9da0 2022-07-21T01:57:21.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f41177fe700 1 -- 172.21.15.140:0/1848889911 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f411c0058b0 con 0x7f41280b9da0 2022-07-21T01:57:21.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f412759e700 1 -- 172.21.15.140:0/1848889911 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41281558f0 con 0x7f41280b9da0 2022-07-21T01:57:21.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.080+0000 7f412759e700 1 -- 172.21.15.140:0/1848889911 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41280574e0 con 0x7f41280b9da0 2022-07-21T01:57:21.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.084+0000 7f41177fe700 1 -- 172.21.15.140:0/1848889911 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f411c010070 con 0x7f41280b9da0 2022-07-21T01:57:21.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.084+0000 7f41177fe700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4104053b00 0x7f4104055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.084+0000 7f41177fe700 1 -- 172.21.15.140:0/1848889911 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f411c06b8a0 con 0x7f41280b9da0 2022-07-21T01:57:21.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.084+0000 7f412659c700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4104053b00 0x7f4104055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.084+0000 7f412659c700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4104053b00 0x7f4104055fc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f411800b000 tx=0x7f4118008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.088+0000 7f41177fe700 1 -- 172.21.15.140:0/1848889911 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f411c042630 con 0x7f41280b9da0 2022-07-21T01:57:21.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.212+0000 7f412759e700 1 -- 172.21.15.140:0/1848889911 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f41280699b0 con 0x7f41280b9da0 2022-07-21T01:57:21.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.212+0000 7f41177fe700 1 -- 172.21.15.140:0/1848889911 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v132) v1 ==== 110+0+10 (secure 0 0 0) 0x7f411c045c50 con 0x7f41280b9da0 2022-07-21T01:57:21.216 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:57:21.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 -- 172.21.15.140:0/1848889911 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4104053b00 msgr2=0x7f4104055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4104053b00 0x7f4104055fc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f411800b000 tx=0x7f4118008040).stop 2022-07-21T01:57:21.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 -- 172.21.15.140:0/1848889911 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41280b9da0 msgr2=0x7f412813f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41280b9da0 0x7f412813f7f0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f41280afcc0 tx=0x7f411c007a90).stop 2022-07-21T01:57:21.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 -- 172.21.15.140:0/1848889911 shutdown_connections 2022-07-21T01:57:21.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4104053b00 0x7f4104055fc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f41281419a0 0x7f4128152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f41280b9da0 0x7f412813f7f0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 --2- 172.21.15.140:0/1848889911 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f41280b3cf0 0x7f41280af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 -- 172.21.15.140:0/1848889911 >> 172.21.15.140:0/1848889911 conn(0x7f41280678c0 msgr2=0x7f41280b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:21.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 -- 172.21.15.140:0/1848889911 shutdown_connections 2022-07-21T01:57:21.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.216+0000 7f412759e700 1 -- 172.21.15.140:0/1848889911 wait complete. 2022-07-21T01:57:21.229 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:57:21.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.336+0000 7f4e25998700 1 Processor -- start 2022-07-21T01:57:21.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.336+0000 7f4e25998700 1 -- start start 2022-07-21T01:57:21.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.336+0000 7f4e25998700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4e200af180 0x7f4e2005d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.336+0000 7f4e25998700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e200b5db0 con 0x7f4e200b59d0 2022-07-21T01:57:21.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.336+0000 7f4e25998700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e200ad800 con 0x7f4e20128d70 2022-07-21T01:57:21.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.336+0000 7f4e25998700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e200adaa0 con 0x7f4e200af180 2022-07-21T01:57:21.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e24996700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f4e200b59d0 0x7f4e20128660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54694/0 (socket says 172.21.15.140:54694) 2022-07-21T01:57:21.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e24996700 1 -- 172.21.15.140:0/1206565593 learned_addr learned my addr 172.21.15.140:0/1206565593 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:21.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e1ffff700 1 --2- 172.21.15.140:0/1206565593 >> v2:172.21.15.152:3300/0 conn(0x7f4e200af180 0x7f4e2005d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e1f7fe700 1 -- 172.21.15.140:0/1206565593 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2637701050 0 0) 0x7f4e200b5db0 con 0x7f4e200b59d0 2022-07-21T01:57:21.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e1f7fe700 1 -- 172.21.15.140:0/1206565593 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e04003260 con 0x7f4e200b59d0 2022-07-21T01:57:21.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e1ffff700 1 -- 172.21.15.140:0/1206565593 >> v1:172.21.15.152:6789/0 conn(0x7f4e20128d70 legacy=0x7f4e2013fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e1ffff700 1 -- 172.21.15.140:0/1206565593 >> v1:172.21.15.140:6789/0 conn(0x7f4e200b59d0 legacy=0x7f4e20128660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e1ffff700 1 -- 172.21.15.140:0/1206565593 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e200ad4f0 con 0x7f4e200af180 2022-07-21T01:57:21.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e1ffff700 1 --2- 172.21.15.140:0/1206565593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e200af180 0x7f4e2005d1c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f4e0c002f70 tx=0x7f4e0c006df0).ready entity=mon.1 client_cookie=cd853c119643b4dc server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e1f7fe700 1 -- 172.21.15.140:0/1206565593 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e0c014070 con 0x7f4e200af180 2022-07-21T01:57:21.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e1f7fe700 1 -- 172.21.15.140:0/1206565593 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c000e90 con 0x7f4e200af180 2022-07-21T01:57:21.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e1f7fe700 1 -- 172.21.15.140:0/1206565593 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e0c00fc00 con 0x7f4e200af180 2022-07-21T01:57:21.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e25998700 1 -- 172.21.15.140:0/1206565593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e200af180 msgr2=0x7f4e2005d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e25998700 1 --2- 172.21.15.140:0/1206565593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e200af180 0x7f4e2005d1c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f4e0c002f70 tx=0x7f4e0c006df0).stop 2022-07-21T01:57:21.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e25998700 1 -- 172.21.15.140:0/1206565593 shutdown_connections 2022-07-21T01:57:21.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e25998700 1 --2- 172.21.15.140:0/1206565593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e200af180 0x7f4e2005d1c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e25998700 1 -- 172.21.15.140:0/1206565593 >> 172.21.15.140:0/1206565593 conn(0x7f4e200634c0 msgr2=0x7f4e200b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:21.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e25998700 1 -- 172.21.15.140:0/1206565593 shutdown_connections 2022-07-21T01:57:21.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.340+0000 7f4e25998700 1 -- 172.21.15.140:0/1206565593 wait complete. 2022-07-21T01:57:21.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e25998700 1 Processor -- start 2022-07-21T01:57:21.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e25998700 1 -- start start 2022-07-21T01:57:21.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e25998700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e200af180 0x7f4e201226e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e25998700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4e200b59d0 0x7f4e201510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e25998700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4e20128d70 0x7f4e201267f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e25998700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e201515e0 con 0x7f4e200b59d0 2022-07-21T01:57:21.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e25998700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e20125010 con 0x7f4e200af180 2022-07-21T01:57:21.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e25998700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e20151820 con 0x7f4e20128d70 2022-07-21T01:57:21.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4e200b59d0 0x7f4e201510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4e200b59d0 0x7f4e201510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40556/0 (socket says 172.21.15.140:40556) 2022-07-21T01:57:21.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1ffff700 1 -- 172.21.15.140:0/1868696722 learned_addr learned my addr 172.21.15.140:0/1868696722 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:21.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e24996700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e200af180 0x7f4e201226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e25197700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4e20128d70 0x7f4e201267f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1ffff700 1 -- 172.21.15.140:0/1868696722 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4e20128d70 msgr2=0x7f4e201267f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1ffff700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4e20128d70 0x7f4e201267f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1ffff700 1 -- 172.21.15.140:0/1868696722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e200af180 msgr2=0x7f4e201226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1ffff700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e200af180 0x7f4e201226e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1ffff700 1 -- 172.21.15.140:0/1868696722 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c00b180 con 0x7f4e200b59d0 2022-07-21T01:57:21.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1ffff700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4e200b59d0 0x7f4e201510f0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7f4e0c002f70 tx=0x7f4e0c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1dffb700 1 -- 172.21.15.140:0/1868696722 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e0c01d640 con 0x7f4e200b59d0 2022-07-21T01:57:21.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1dffb700 1 -- 172.21.15.140:0/1868696722 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c00f4a0 con 0x7f4e200b59d0 2022-07-21T01:57:21.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e25998700 1 -- 172.21.15.140:0/1868696722 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e201249b0 con 0x7f4e200b59d0 2022-07-21T01:57:21.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e1dffb700 1 -- 172.21.15.140:0/1868696722 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e0c01d990 con 0x7f4e200b59d0 2022-07-21T01:57:21.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.344+0000 7f4e25998700 1 -- 172.21.15.140:0/1868696722 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e20067720 con 0x7f4e200b59d0 2022-07-21T01:57:21.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.348+0000 7f4e25998700 1 -- 172.21.15.140:0/1868696722 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e201446d0 con 0x7f4e200b59d0 2022-07-21T01:57:21.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.348+0000 7f4e1dffb700 1 -- 172.21.15.140:0/1868696722 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4e0c01daf0 con 0x7f4e200b59d0 2022-07-21T01:57:21.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.348+0000 7f4e1dffb700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e00053aa0 0x7f4e00055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.348+0000 7f4e1dffb700 1 -- 172.21.15.140:0/1868696722 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4e0c070050 con 0x7f4e200b59d0 2022-07-21T01:57:21.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.348+0000 7f4e24996700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e00053aa0 0x7f4e00055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.348+0000 7f4e24996700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e00053aa0 0x7f4e00055f60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f4e1400d000 tx=0x7f4e14009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.352+0000 7f4e1dffb700 1 -- 172.21.15.140:0/1868696722 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4e0c04ae60 con 0x7f4e200b59d0 2022-07-21T01:57:21.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 -- 172.21.15.140:0/1868696722 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4e20124e70 con 0x7f4e200b59d0 2022-07-21T01:57:21.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e1dffb700 1 -- 172.21.15.140:0/1868696722 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4e20124e70 con 0x7f4e200b59d0 2022-07-21T01:57:21.486 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:21.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 -- 172.21.15.140:0/1868696722 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e00053aa0 msgr2=0x7f4e00055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e00053aa0 0x7f4e00055f60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f4e1400d000 tx=0x7f4e14009040).stop 2022-07-21T01:57:21.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 -- 172.21.15.140:0/1868696722 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4e200b59d0 msgr2=0x7f4e201510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4e200b59d0 0x7f4e201510f0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7f4e0c002f70 tx=0x7f4e0c016000).stop 2022-07-21T01:57:21.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 -- 172.21.15.140:0/1868696722 shutdown_connections 2022-07-21T01:57:21.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4e00053aa0 0x7f4e00055f60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4e20128d70 0x7f4e201267f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f4e200b59d0 0x7f4e201510f0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 --2- 172.21.15.140:0/1868696722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e200af180 0x7f4e201226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 -- 172.21.15.140:0/1868696722 >> 172.21.15.140:0/1868696722 conn(0x7f4e200634c0 msgr2=0x7f4e200b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:21.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 -- 172.21.15.140:0/1868696722 shutdown_connections 2022-07-21T01:57:21.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.480+0000 7f4e25998700 1 -- 172.21.15.140:0/1868696722 wait complete. 2022-07-21T01:57:21.495 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:57:21.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bea92700 1 Processor -- start 2022-07-21T01:57:21.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bea92700 1 -- start start 2022-07-21T01:57:21.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bea92700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd8b805d0e0 0x7fd8b8062f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bea92700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8b8141030 0x7fd8b813d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bea92700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b805d4c0 con 0x7fd8b8099a10 2022-07-21T01:57:21.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bea92700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b80b3aa0 con 0x7fd8b805d0e0 2022-07-21T01:57:21.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bea92700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b81449f0 con 0x7fd8b8141030 2022-07-21T01:57:21.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bc82e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd8b805d0e0 0x7fd8b8062f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bc82e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fd8b805d0e0 0x7fd8b8062f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40562/0 (socket says 172.21.15.140:40562) 2022-07-21T01:57:21.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bc82e700 1 -- 172.21.15.140:0/305487101 learned_addr learned my addr 172.21.15.140:0/305487101 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:21.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bd02f700 1 --2- 172.21.15.140:0/305487101 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8b8141030 0x7fd8b813d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bc82e700 1 -- 172.21.15.140:0/305487101 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8b8141030 msgr2=0x7fd8b813d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:21.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bc82e700 1 --2- 172.21.15.140:0/305487101 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8b8141030 0x7fd8b813d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bc82e700 1 -- 172.21.15.140:0/305487101 >> v1:172.21.15.152:6789/0 conn(0x7fd8b8099a10 legacy=0x7fd8b8140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bc82e700 1 -- 172.21.15.140:0/305487101 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b8144c90 con 0x7fd8b805d0e0 2022-07-21T01:57:21.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8bc82e700 1 --2- 172.21.15.140:0/305487101 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8b805d0e0 0x7fd8b8062f30 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fd8a0002f70 tx=0x7fd8a0006eb0).ready entity=mon.0 client_cookie=4c4c55ff1e13ad92 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8b77fe700 1 -- 172.21.15.140:0/305487101 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a000d5c0 con 0x7fd8b805d0e0 2022-07-21T01:57:21.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8b77fe700 1 -- 172.21.15.140:0/305487101 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a0000e90 con 0x7fd8b805d0e0 2022-07-21T01:57:21.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.604+0000 7fd8b77fe700 1 -- 172.21.15.140:0/305487101 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a000d950 con 0x7fd8b805d0e0 2022-07-21T01:57:21.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 -- 172.21.15.140:0/305487101 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8b805d0e0 msgr2=0x7fd8b8062f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.613 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 --2- 172.21.15.140:0/305487101 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8b805d0e0 0x7fd8b8062f30 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fd8a0002f70 tx=0x7fd8a0006eb0).stop 2022-07-21T01:57:21.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 -- 172.21.15.140:0/305487101 shutdown_connections 2022-07-21T01:57:21.614 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 --2- 172.21.15.140:0/305487101 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8b8141030 0x7fd8b813d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 --2- 172.21.15.140:0/305487101 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8b805d0e0 0x7fd8b8062f30 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 -- 172.21.15.140:0/305487101 >> 172.21.15.140:0/305487101 conn(0x7fd8b80b7cd0 msgr2=0x7fd8b80b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:21.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 -- 172.21.15.140:0/305487101 shutdown_connections 2022-07-21T01:57:21.615 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 -- 172.21.15.140:0/305487101 wait complete. 2022-07-21T01:57:21.618 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 Processor -- start 2022-07-21T01:57:21.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 -- start start 2022-07-21T01:57:21.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8b805d0e0 0x7fd8b814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8b8099a10 0x7fd8b809a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8b8141030 0x7fd8b81504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b813f490 con 0x7fd8b8099a10 2022-07-21T01:57:21.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b809c8b0 con 0x7fd8b805d0e0 2022-07-21T01:57:21.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b809c640 con 0x7fd8b8141030 2022-07-21T01:57:21.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8b8099a10 0x7fd8b809a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b7fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8b8099a10 0x7fd8b809a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40570/0 (socket says 172.21.15.140:40570) 2022-07-21T01:57:21.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b7fff700 1 -- 172.21.15.140:0/713082859 learned_addr learned my addr 172.21.15.140:0/713082859 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:21.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bd02f700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8b8141030 0x7fd8b81504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bc82e700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8b805d0e0 0x7fd8b814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b7fff700 1 -- 172.21.15.140:0/713082859 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8b8141030 msgr2=0x7fd8b81504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b7fff700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8b8141030 0x7fd8b81504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b7fff700 1 -- 172.21.15.140:0/713082859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8b805d0e0 msgr2=0x7fd8b814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b7fff700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8b805d0e0 0x7fd8b814fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b7fff700 1 -- 172.21.15.140:0/713082859 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a000b1e0 con 0x7fd8b8099a10 2022-07-21T01:57:21.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b7fff700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8b8099a10 0x7fd8b809a5f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7fd8b80b4e40 tx=0x7fd8a8007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b5ffb700 1 -- 172.21.15.140:0/713082859 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a8007030 con 0x7fd8b8099a10 2022-07-21T01:57:21.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 -- 172.21.15.140:0/713082859 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b809cb10 con 0x7fd8b8099a10 2022-07-21T01:57:21.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b5ffb700 1 -- 172.21.15.140:0/713082859 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8007190 con 0x7fd8b8099a10 2022-07-21T01:57:21.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8b5ffb700 1 -- 172.21.15.140:0/713082859 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd8a8006510 con 0x7fd8b8099a10 2022-07-21T01:57:21.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.608+0000 7fd8bea92700 1 -- 172.21.15.140:0/713082859 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b813eeb0 con 0x7fd8b8099a10 2022-07-21T01:57:21.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.612+0000 7fd8bea92700 1 -- 172.21.15.140:0/713082859 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b81408b0 con 0x7fd8b8099a10 2022-07-21T01:57:21.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.612+0000 7fd8b5ffb700 1 -- 172.21.15.140:0/713082859 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd8a8010030 con 0x7fd8b8099a10 2022-07-21T01:57:21.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.612+0000 7fd8b5ffb700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd89c053950 0x7fd89c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.612+0000 7fd8b5ffb700 1 -- 172.21.15.140:0/713082859 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd8a806b9c0 con 0x7fd8b8099a10 2022-07-21T01:57:21.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.612+0000 7fd8bc82e700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd89c053950 0x7fd89c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.612+0000 7fd8bc82e700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd89c053950 0x7fd89c055e10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fd8a001c000 tx=0x7fd8a0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.620+0000 7fd8b5ffb700 1 -- 172.21.15.140:0/713082859 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd8a80428e0 con 0x7fd8b8099a10 2022-07-21T01:57:21.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.736+0000 7fd8bea92700 1 -- 172.21.15.140:0/713082859 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd8b8099e70 con 0x7fd8b8099a10 2022-07-21T01:57:21.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8b5ffb700 1 -- 172.21.15.140:0/713082859 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd8a8045f00 con 0x7fd8b8099a10 2022-07-21T01:57:21.741 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:21.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 -- 172.21.15.140:0/713082859 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd89c053950 msgr2=0x7fd89c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd89c053950 0x7fd89c055e10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fd8a001c000 tx=0x7fd8a0015000).stop 2022-07-21T01:57:21.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 -- 172.21.15.140:0/713082859 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8b8099a10 msgr2=0x7fd8b809a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8b8099a10 0x7fd8b809a5f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7fd8b80b4e40 tx=0x7fd8a8007be0).stop 2022-07-21T01:57:21.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 -- 172.21.15.140:0/713082859 shutdown_connections 2022-07-21T01:57:21.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd89c053950 0x7fd89c055e10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8b8141030 0x7fd8b81504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8b8099a10 0x7fd8b809a5f0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 --2- 172.21.15.140:0/713082859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8b805d0e0 0x7fd8b814fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 -- 172.21.15.140:0/713082859 >> 172.21.15.140:0/713082859 conn(0x7fd8b80b7cd0 msgr2=0x7fd8b80b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:21.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 -- 172.21.15.140:0/713082859 shutdown_connections 2022-07-21T01:57:21.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.740+0000 7fd8bea92700 1 -- 172.21.15.140:0/713082859 wait complete. 2022-07-21T01:57:21.760 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:57:21.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc29b0d5700 1 Processor -- start 2022-07-21T01:57:21.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc29b0d5700 1 -- start start 2022-07-21T01:57:21.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc29b0d5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc2940b59d0 0x7fc2940b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc29b0d5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2940adf70 con 0x7fc2940af920 2022-07-21T01:57:21.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc29b0d5700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2940b5db0 con 0x7fc29405d0c0 2022-07-21T01:57:21.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc29b0d5700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc294146300 con 0x7fc2940b59d0 2022-07-21T01:57:21.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc29a0d3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc2940b59d0 0x7fc2940b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc2998d2700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc2940af920 0x7fc2940ae3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54724/0 (socket says 172.21.15.140:54724) 2022-07-21T01:57:21.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc2998d2700 1 -- 172.21.15.140:0/2429269156 learned_addr learned my addr 172.21.15.140:0/2429269156 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:21.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc29a0d3700 1 -- 172.21.15.140:0/2429269156 >> v1:172.21.15.152:6789/0 conn(0x7fc29405d0c0 legacy=0x7fc294141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc29a0d3700 1 -- 172.21.15.140:0/2429269156 >> v1:172.21.15.140:6789/0 conn(0x7fc2940af920 legacy=0x7fc2940ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc2990d1700 1 -- 172.21.15.140:0/2429269156 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191596876 0 0) 0x7fc2940adf70 con 0x7fc2940af920 2022-07-21T01:57:21.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc29a0d3700 1 -- 172.21.15.140:0/2429269156 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc294146060 con 0x7fc2940b59d0 2022-07-21T01:57:21.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc29a0d3700 1 --2- 172.21.15.140:0/2429269156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2940b59d0 0x7fc2940b0040 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7fc290002f70 tx=0x7fc290006df0).ready entity=mon.0 client_cookie=606b0cf32693c263 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc2990d1700 1 -- 172.21.15.140:0/2429269156 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc29000d610 con 0x7fc2940b59d0 2022-07-21T01:57:21.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc2990d1700 1 -- 172.21.15.140:0/2429269156 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc290000e90 con 0x7fc2940b59d0 2022-07-21T01:57:21.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.868+0000 7fc2990d1700 1 -- 172.21.15.140:0/2429269156 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc29000d960 con 0x7fc2940b59d0 2022-07-21T01:57:21.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.872+0000 7fc29b0d5700 1 -- 172.21.15.140:0/2429269156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2940b59d0 msgr2=0x7fc2940b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.882 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.872+0000 7fc29b0d5700 1 --2- 172.21.15.140:0/2429269156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2940b59d0 0x7fc2940b0040 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7fc290002f70 tx=0x7fc290006df0).stop 2022-07-21T01:57:21.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.872+0000 7fc29b0d5700 1 -- 172.21.15.140:0/2429269156 shutdown_connections 2022-07-21T01:57:21.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.872+0000 7fc29b0d5700 1 --2- 172.21.15.140:0/2429269156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2940b59d0 0x7fc2940b0040 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.872+0000 7fc29b0d5700 1 -- 172.21.15.140:0/2429269156 >> 172.21.15.140:0/2429269156 conn(0x7fc2940634c0 msgr2=0x7fc2940b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:21.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.872+0000 7fc29b0d5700 1 -- 172.21.15.140:0/2429269156 shutdown_connections 2022-07-21T01:57:21.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.872+0000 7fc29b0d5700 1 -- 172.21.15.140:0/2429269156 wait complete. 2022-07-21T01:57:21.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.872+0000 7fc29b0d5700 1 Processor -- start 2022-07-21T01:57:21.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.872+0000 7fc29b0d5700 1 -- start start 2022-07-21T01:57:21.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29b0d5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc29405d0c0 0x7fc294067160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29b0d5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2940af920 0x7fc2941406f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29b0d5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2940b59d0 0x7fc29413e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29b0d5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2941456c0 con 0x7fc2940b59d0 2022-07-21T01:57:21.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29b0d5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc294145960 con 0x7fc2940af920 2022-07-21T01:57:21.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29b0d5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc294145c00 con 0x7fc29405d0c0 2022-07-21T01:57:21.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29a8d4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2940b59d0 0x7fc29413e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29a8d4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2940b59d0 0x7fc29413e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40586/0 (socket says 172.21.15.140:40586) 2022-07-21T01:57:21.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29a8d4700 1 -- 172.21.15.140:0/1533240239 learned_addr learned my addr 172.21.15.140:0/1533240239 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:21.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc2998d2700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2940af920 0x7fc2941406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29a0d3700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc29405d0c0 0x7fc294067160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29a8d4700 1 -- 172.21.15.140:0/1533240239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc29405d0c0 msgr2=0x7fc294067160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29a8d4700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc29405d0c0 0x7fc294067160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29a8d4700 1 -- 172.21.15.140:0/1533240239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2940af920 msgr2=0x7fc2941406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:21.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29a8d4700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2940af920 0x7fc2941406f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:21.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29a8d4700 1 -- 172.21.15.140:0/1533240239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29000b180 con 0x7fc2940b59d0 2022-07-21T01:57:21.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29a8d4700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2940b59d0 0x7fc29413e840 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7fc2940b2d00 tx=0x7fc28c007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc2877fe700 1 -- 172.21.15.140:0/1533240239 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc28c00f470 con 0x7fc2940b59d0 2022-07-21T01:57:21.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc2877fe700 1 -- 172.21.15.140:0/1533240239 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c00f5d0 con 0x7fc2940b59d0 2022-07-21T01:57:21.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29b0d5700 1 -- 172.21.15.140:0/1533240239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc294140ff0 con 0x7fc2940b59d0 2022-07-21T01:57:21.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc2877fe700 1 -- 172.21.15.140:0/1533240239 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc28c00f930 con 0x7fc2940b59d0 2022-07-21T01:57:21.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29b0d5700 1 -- 172.21.15.140:0/1533240239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc294140be0 con 0x7fc2940b59d0 2022-07-21T01:57:21.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc29b0d5700 1 -- 172.21.15.140:0/1533240239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2940af2d0 con 0x7fc2940b59d0 2022-07-21T01:57:21.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.876+0000 7fc2877fe700 1 -- 172.21.15.140:0/1533240239 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc28c019030 con 0x7fc2940b59d0 2022-07-21T01:57:21.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.880+0000 7fc2877fe700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc274053aa0 0x7fc274055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:21.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.880+0000 7fc2877fe700 1 -- 172.21.15.140:0/1533240239 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc28c06b650 con 0x7fc2940b59d0 2022-07-21T01:57:21.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.880+0000 7fc29a0d3700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc274053aa0 0x7fc274055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:21.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.880+0000 7fc29a0d3700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc274053aa0 0x7fc274055f60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fc29001c000 tx=0x7fc290015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:21.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:21.884+0000 7fc2877fe700 1 -- 172.21.15.140:0/1533240239 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc28c042380 con 0x7fc2940b59d0 2022-07-21T01:57:22.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 -- 172.21.15.140:0/1533240239 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2940574e0 con 0x7fc2940b59d0 2022-07-21T01:57:22.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc2877fe700 1 -- 172.21.15.140:0/1533240239 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc28c0459a0 con 0x7fc2940b59d0 2022-07-21T01:57:22.013 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:22.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 -- 172.21.15.140:0/1533240239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc274053aa0 msgr2=0x7fc274055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:22.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc274053aa0 0x7fc274055f60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fc29001c000 tx=0x7fc290015000).stop 2022-07-21T01:57:22.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 -- 172.21.15.140:0/1533240239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2940b59d0 msgr2=0x7fc29413e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:22.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2940b59d0 0x7fc29413e840 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7fc2940b2d00 tx=0x7fc28c007f40).stop 2022-07-21T01:57:22.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 -- 172.21.15.140:0/1533240239 shutdown_connections 2022-07-21T01:57:22.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc274053aa0 0x7fc274055f60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc2940b59d0 0x7fc29413e840 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc2940af920 0x7fc2941406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 --2- 172.21.15.140:0/1533240239 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc29405d0c0 0x7fc294067160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 -- 172.21.15.140:0/1533240239 >> 172.21.15.140:0/1533240239 conn(0x7fc2940634c0 msgr2=0x7fc2940ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:22.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 -- 172.21.15.140:0/1533240239 shutdown_connections 2022-07-21T01:57:22.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.012+0000 7fc29b0d5700 1 -- 172.21.15.140:0/1533240239 wait complete. 2022-07-21T01:57:22.029 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:57:22.030 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:22.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73bd213700 1 Processor -- start 2022-07-21T01:57:22.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73bd213700 1 -- start start 2022-07-21T01:57:22.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73bd213700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73b004b1e0 0x7f73b0046d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:22.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73bd213700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f73b00421b0 0x7f73b00d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:22.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73bd213700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b004b5c0 con 0x7f73b00d7b80 2022-07-21T01:57:22.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73bd213700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b0045700 con 0x7f73b00421b0 2022-07-21T01:57:22.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73bd213700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00459a0 con 0x7f73b004b1e0 2022-07-21T01:57:22.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b77fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f73b00421b0 0x7f73b00d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:22.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b77fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f73b00421b0 0x7f73b00d3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:46330/0 (socket says 172.21.15.140:46330) 2022-07-21T01:57:22.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b77fe700 1 -- 172.21.15.140:0/2883024603 learned_addr learned my addr 172.21.15.140:0/2883024603 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:22.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73bca12700 1 --1- 172.21.15.140:0/2883024603 >> v1:172.21.15.152:6789/0 conn(0x7f73b00d7b80 0x7f73b00d0c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:55854/0 (socket says 172.21.15.140:55854) 2022-07-21T01:57:22.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b7fff700 1 --2- 172.21.15.140:0/2883024603 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73b004b1e0 0x7f73b0046d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:22.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b77fe700 1 -- 172.21.15.140:0/2883024603 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73b004b1e0 msgr2=0x7f73b0046d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:22.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b77fe700 1 --2- 172.21.15.140:0/2883024603 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73b004b1e0 0x7f73b0046d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b77fe700 1 -- 172.21.15.140:0/2883024603 >> v1:172.21.15.152:6789/0 conn(0x7f73b00d7b80 legacy=0x7f73b00d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:22.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b77fe700 1 -- 172.21.15.140:0/2883024603 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b00d76e0 con 0x7f73b00421b0 2022-07-21T01:57:22.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b77fe700 1 --2- 172.21.15.140:0/2883024603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73b00421b0 0x7f73b00d3f90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f73a8002f70 tx=0x7f73a8006eb0).ready entity=mon.1 client_cookie=eba807569a57265d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:22.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b6ffd700 1 -- 172.21.15.140:0/2883024603 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73a800d570 con 0x7f73b00421b0 2022-07-21T01:57:22.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b6ffd700 1 -- 172.21.15.140:0/2883024603 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a8000e90 con 0x7f73b00421b0 2022-07-21T01:57:22.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.128+0000 7f73b6ffd700 1 -- 172.21.15.140:0/2883024603 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73a800d900 con 0x7f73b00421b0 2022-07-21T01:57:22.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.132+0000 7f73bd213700 1 -- 172.21.15.140:0/2883024603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73b00421b0 msgr2=0x7f73b00d3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:22.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.132+0000 7f73bd213700 1 --2- 172.21.15.140:0/2883024603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73b00421b0 0x7f73b00d3f90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f73a8002f70 tx=0x7f73a8006eb0).stop 2022-07-21T01:57:22.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.132+0000 7f73bd213700 1 -- 172.21.15.140:0/2883024603 shutdown_connections 2022-07-21T01:57:22.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.132+0000 7f73bd213700 1 --2- 172.21.15.140:0/2883024603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73b00421b0 0x7f73b00d3f90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.132+0000 7f73bd213700 1 --2- 172.21.15.140:0/2883024603 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73b004b1e0 0x7f73b0046d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.132+0000 7f73bd213700 1 -- 172.21.15.140:0/2883024603 >> 172.21.15.140:0/2883024603 conn(0x7f73b00115d0 msgr2=0x7f73b004a790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:22.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.132+0000 7f73bd213700 1 -- 172.21.15.140:0/2883024603 shutdown_connections 2022-07-21T01:57:22.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.132+0000 7f73bd213700 1 -- 172.21.15.140:0/2883024603 wait complete. 2022-07-21T01:57:22.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.132+0000 7f73bd213700 1 Processor -- start 2022-07-21T01:57:22.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.132+0000 7f73bd213700 1 -- start start 2022-07-21T01:57:22.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73bd213700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73b00421b0 0x7f73b00e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:22.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73bd213700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73b004b1e0 0x7f73b00a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:22.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73bd213700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73b00d7b80 0x7f73b00d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:22.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73bd213700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00a3f50 con 0x7f73b004b1e0 2022-07-21T01:57:22.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73bd213700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00a72a0 con 0x7f73b00421b0 2022-07-21T01:57:22.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73bd213700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b00a4170 con 0x7f73b00d7b80 2022-07-21T01:57:22.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73b004b1e0 0x7f73b00a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:22.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b77fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73b004b1e0 0x7f73b00a3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40600/0 (socket says 172.21.15.140:40600) 2022-07-21T01:57:22.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b77fe700 1 -- 172.21.15.140:0/3724518800 learned_addr learned my addr 172.21.15.140:0/3724518800 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:22.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b7fff700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73b00421b0 0x7f73b00e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:22.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73bca12700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73b00d7b80 0x7f73b00d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:22.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b77fe700 1 -- 172.21.15.140:0/3724518800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73b00d7b80 msgr2=0x7f73b00d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:22.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b77fe700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73b00d7b80 0x7f73b00d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b77fe700 1 -- 172.21.15.140:0/3724518800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73b00421b0 msgr2=0x7f73b00e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:22.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b77fe700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73b00421b0 0x7f73b00e69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b77fe700 1 -- 172.21.15.140:0/3724518800 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a800b1e0 con 0x7f73b004b1e0 2022-07-21T01:57:22.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b77fe700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73b004b1e0 0x7f73b00a3a60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f73a8002f70 tx=0x7f73a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:22.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b57fa700 1 -- 172.21.15.140:0/3724518800 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73a800d8f0 con 0x7f73b004b1e0 2022-07-21T01:57:22.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b57fa700 1 -- 172.21.15.140:0/3724518800 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a8011bf0 con 0x7f73b004b1e0 2022-07-21T01:57:22.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73bd213700 1 -- 172.21.15.140:0/3724518800 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b00a4410 con 0x7f73b004b1e0 2022-07-21T01:57:22.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b57fa700 1 -- 172.21.15.140:0/3724518800 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73a801e460 con 0x7f73b004b1e0 2022-07-21T01:57:22.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73bd213700 1 -- 172.21.15.140:0/3724518800 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b00d2ef0 con 0x7f73b004b1e0 2022-07-21T01:57:22.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73b57fa700 1 -- 172.21.15.140:0/3724518800 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f73a8025030 con 0x7f73b004b1e0 2022-07-21T01:57:22.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.136+0000 7f73bd213700 1 -- 172.21.15.140:0/3724518800 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b00a5130 con 0x7f73b004b1e0 2022-07-21T01:57:22.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.140+0000 7f73b57fa700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7398053a10 0x7f7398055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:22.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.140+0000 7f73b7fff700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7398053a10 0x7f7398055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:22.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.140+0000 7f73b57fa700 1 -- 172.21.15.140:0/3724518800 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f73a806d1f0 con 0x7f73b004b1e0 2022-07-21T01:57:22.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.140+0000 7f73b7fff700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7398053a10 0x7f7398055ed0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f73ac00b000 tx=0x7f73ac006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:22.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.148+0000 7f73b57fa700 1 -- 172.21.15.140:0/3724518800 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f73b00a5130 con 0x7f73b004b1e0 2022-07-21T01:57:22.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.304+0000 7f73bd213700 1 -- 172.21.15.140:0/3724518800 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73b00a5130 con 0x7f73b004b1e0 2022-07-21T01:57:22.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.304+0000 7f73b57fa700 1 -- 172.21.15.140:0/3724518800 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+2145 (secure 0 0 0) 0x7f73b00a5130 con 0x7f73b004b1e0 2022-07-21T01:57:22.308 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:22.308 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":64,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/558475048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":558475048},{"type":"v1","addr":"172.21.15.152:6833","nonce":558475048}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:14.423640+0000","modified":"2022-07-21T01:57:16.123489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2209871450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2209871450},{"type":"v1","addr":"172.21.15.140:6835","nonce":2209871450}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-07-21T01:57:22.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 -- 172.21.15.140:0/3724518800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7398053a10 msgr2=0x7f7398055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:22.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7398053a10 0x7f7398055ed0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f73ac00b000 tx=0x7f73ac006df0).stop 2022-07-21T01:57:22.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 -- 172.21.15.140:0/3724518800 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73b004b1e0 msgr2=0x7f73b00a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:22.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73b004b1e0 0x7f73b00a3a60 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f73a8002f70 tx=0x7f73a8015000).stop 2022-07-21T01:57:22.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 -- 172.21.15.140:0/3724518800 shutdown_connections 2022-07-21T01:57:22.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7398053a10 0x7f7398055ed0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f73b00d7b80 0x7f73b00d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f73b004b1e0 0x7f73b00a3a60 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 --2- 172.21.15.140:0/3724518800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f73b00421b0 0x7f73b00e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:22.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 -- 172.21.15.140:0/3724518800 >> 172.21.15.140:0/3724518800 conn(0x7f73b00115d0 msgr2=0x7f73b00d64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:22.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 -- 172.21.15.140:0/3724518800 shutdown_connections 2022-07-21T01:57:22.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:22.308+0000 7f73bd213700 1 -- 172.21.15.140:0/3724518800 wait complete. 2022-07-21T01:57:22.314 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 64 2022-07-21T01:57:22.520 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:57:14.423640+0000', 'modified': '2022-07-21T01:57:16.123489+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8061}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8061': {'gid': 8061, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/2209871450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 2209871450}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 2209871450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:57:22.521 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:57:22.522 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:57:22.636 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":8061,"want_state":"up:active","state":"up:active","rank_uptime":8.2013406549999992,"mdsmap_epoch":64,"osdmap_epoch":132,"osdmap_epoch_barrier":132,"uptime":12.895306149} 2022-07-21T01:57:22.637 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 8061, "mdsmap_epoch": 64, "osdmap_epoch": 132, "osdmap_epoch_barrier": 132, "rank_uptime": 8.201340655, "state": "up:active", "uptime": 12.895306149, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:57:22.637 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:57:22.638 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-07-21T01:57:22.638 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-07-21T01:57:22.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:22.639 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:22.648 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:57:22.705 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:22.706 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:22.722 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:22.723 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:22.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:22.724 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:57:22.731 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:57:22.732 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:22.735 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:57:22.751 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:22.752 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:22.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2022-07-21T01:57:22.800+0000 7fdaa8fb6080 -1 init, newargv = 0x55bd4372f460 newargc=9 2022-07-21T01:57:22.805 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22821]: starting ceph client 2022-07-21T01:57:22.809 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:22.810 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:22.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:22.811 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:57:22.814 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22821]: starting fuse 2022-07-21T01:57:22.817 INFO:teuthology.orchestra.run.smithi192.stdout:52 2022-07-21T01:57:22.817 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:57:22.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:22.818 DEBUG:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2022-07-21T01:57:22.818 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2022-07-21T01:57:22.819 DEBUG:teuthology.orchestra.run.smithi192:> import re 2022-07-21T01:57:22.819 DEBUG:teuthology.orchestra.run.smithi192:> import os 2022-07-21T01:57:22.820 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2022-07-21T01:57:22.820 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:22.821 DEBUG:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2022-07-21T01:57:22.821 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-07-21T01:57:22.821 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2022-07-21T01:57:22.822 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:22.822 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2022-07-21T01:57:22.823 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2022-07-21T01:57:22.823 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2022-07-21T01:57:22.823 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2022-07-21T01:57:22.824 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:22.824 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2022-07-21T01:57:22.825 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:57:22.825 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:57:22.825 DEBUG:teuthology.orchestra.run.smithi192:> return f 2022-07-21T01:57:22.826 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:57:22.826 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:22.827 DEBUG:teuthology.orchestra.run.smithi192:> print(find_socket("client.1")) 2022-07-21T01:57:22.827 DEBUG:teuthology.orchestra.run.smithi192:> ' 2022-07-21T01:57:22.865 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.22821.asok 2022-07-21T01:57:22.869 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22821.asok 2022-07-21T01:57:22.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:22.870 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22821.asok status 2022-07-21T01:57:22.958 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2022-07-21T01:57:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2022-07-21T01:57:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:57:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:57:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2022-07-21T01:57:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2022-07-21T01:57:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-07-21T01:57:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "22821", 2022-07-21T01:57:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2022-07-21T01:57:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2022-07-21T01:57:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2022-07-21T01:57:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 8152, 2022-07-21T01:57:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2022-07-21T01:57:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2022-07-21T01:57:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2022-07-21T01:57:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 8152 2022-07-21T01:57:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:57:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:57:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:57:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 611661595 2022-07-21T01:57:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2022-07-21T01:57:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:57:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:57:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:57:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 611661595 2022-07-21T01:57:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.8152 172.21.15.192:0/611661595", 2022-07-21T01:57:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/611661595", 2022-07-21T01:57:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2022-07-21T01:57:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 64, 2022-07-21T01:57:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 132, 2022-07-21T01:57:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:57:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false, 2022-07-21T01:57:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2022-07-21T01:57:22.967 INFO:teuthology.orchestra.run.smithi192.stdout:} 2022-07-21T01:57:22.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:22.968 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:22.969 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:22.976 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:57:22.978 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:57:22.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:22.979 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:23.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:23.039 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:23.039 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:23.046 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:57:23.047 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:57:23.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:23.048 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:23.048 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:23.056 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:57:23.057 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:57:23.058 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2022-07-21T01:57:23.058 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:57:23.059 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:23.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22821]: fuse finished with error 0 and tester_r 0 2022-07-21T01:57:23.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:23.117 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:23.118 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:23.123 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2022-07-21T01:57:23.124 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:57:23.125 INFO:teuthology.orchestra.run:waiting for 900 2022-07-21T01:57:23.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:57:23.197 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:23.205 DEBUG:teuthology.orchestra.run.smithi140:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:57:23.208 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:57:23.211 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:57:23.213 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:57:23.216 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:57:23.243 INFO:teuthology.orchestra.run.smithi140.stdout:[global] 2022-07-21T01:57:23.243 INFO:teuthology.orchestra.run.smithi140.stdout: chdir = "" 2022-07-21T01:57:23.244 INFO:teuthology.orchestra.run.smithi140.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:57:23.244 INFO:teuthology.orchestra.run.smithi140.stdout: auth supported = cephx 2022-07-21T01:57:23.244 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.244 INFO:teuthology.orchestra.run.smithi140.stdout: filestore xattr use omap = true 2022-07-21T01:57:23.245 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.245 INFO:teuthology.orchestra.run.smithi140.stdout: mon clock drift allowed = 1.000 2022-07-21T01:57:23.245 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.245 INFO:teuthology.orchestra.run.smithi140.stdout: osd crush chooseleaf type = 0 2022-07-21T01:57:23.246 INFO:teuthology.orchestra.run.smithi140.stdout: auth debug = true 2022-07-21T01:57:23.246 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.246 INFO:teuthology.orchestra.run.smithi140.stdout: ms die on old message = true 2022-07-21T01:57:23.246 INFO:teuthology.orchestra.run.smithi140.stdout: ms die on bug = true 2022-07-21T01:57:23.246 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.247 INFO:teuthology.orchestra.run.smithi140.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:57:23.247 INFO:teuthology.orchestra.run.smithi140.stdout: mon pg warn max object skew = 0 2022-07-21T01:57:23.247 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.247 INFO:teuthology.orchestra.run.smithi140.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:57:23.248 INFO:teuthology.orchestra.run.smithi140.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:57:23.248 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.248 INFO:teuthology.orchestra.run.smithi140.stdout: osd pool default size = 2 2022-07-21T01:57:23.248 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.249 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd allow primary affinity = true 2022-07-21T01:57:23.249 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd allow pg remap = true 2022-07-21T01:57:23.249 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:57:23.249 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:57:23.249 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on no sortbitwise = false 2022-07-21T01:57:23.250 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:57:23.250 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on too few osds = false 2022-07-21T01:57:23.250 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:57:23.250 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:57:23.251 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.251 INFO:teuthology.orchestra.run.smithi140.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:57:23.251 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.251 INFO:teuthology.orchestra.run.smithi140.stdout: osd default data pool replay window = 5 2022-07-21T01:57:23.252 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.252 INFO:teuthology.orchestra.run.smithi140.stdout: mon allow pool delete = true 2022-07-21T01:57:23.252 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.252 INFO:teuthology.orchestra.run.smithi140.stdout: mon cluster log file level = debug 2022-07-21T01:57:23.252 INFO:teuthology.orchestra.run.smithi140.stdout: debug asserts on shutdown = true 2022-07-21T01:57:23.253 INFO:teuthology.orchestra.run.smithi140.stdout: mon health detail to clog = false 2022-07-21T01:57:23.253 INFO:teuthology.orchestra.run.smithi140.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:57:23.253 INFO:teuthology.orchestra.run.smithi140.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:57:23.253 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.254 INFO:teuthology.orchestra.run.smithi140.stdout:[osd] 2022-07-21T01:57:23.254 INFO:teuthology.orchestra.run.smithi140.stdout: osd journal size = 100 2022-07-21T01:57:23.254 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.254 INFO:teuthology.orchestra.run.smithi140.stdout: osd scrub load threshold = 5.0 2022-07-21T01:57:23.254 INFO:teuthology.orchestra.run.smithi140.stdout: osd scrub max interval = 600 2022-07-21T01:57:23.255 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.255 INFO:teuthology.orchestra.run.smithi140.stdout: osd recover clone overlap = true 2022-07-21T01:57:23.255 INFO:teuthology.orchestra.run.smithi140.stdout: osd recovery max chunk = 1048576 2022-07-21T01:57:23.255 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.256 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug shutdown = true 2022-07-21T01:57:23.256 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug op order = true 2022-07-21T01:57:23.256 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug verify stray on activate = true 2022-07-21T01:57:23.256 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.256 INFO:teuthology.orchestra.run.smithi140.stdout: osd open classes on start = true 2022-07-21T01:57:23.257 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug pg log writeout = true 2022-07-21T01:57:23.257 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.257 INFO:teuthology.orchestra.run.smithi140.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:57:23.257 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.258 INFO:teuthology.orchestra.run.smithi140.stdout: osd map max advance = 10 2022-07-21T01:57:23.258 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.258 INFO:teuthology.orchestra.run.smithi140.stdout: journal zero on create = true 2022-07-21T01:57:23.258 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.259 INFO:teuthology.orchestra.run.smithi140.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:57:23.259 INFO:teuthology.orchestra.run.smithi140.stdout: filestore apply finisher threads = 3 2022-07-21T01:57:23.259 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.259 INFO:teuthology.orchestra.run.smithi140.stdout: bdev debug aio = true 2022-07-21T01:57:23.259 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug misdirected ops = true 2022-07-21T01:57:23.260 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:57:23.260 INFO:teuthology.orchestra.run.smithi140.stdout: debug osd = 20 2022-07-21T01:57:23.260 INFO:teuthology.orchestra.run.smithi140.stdout: osd objectstore = filestore 2022-07-21T01:57:23.260 INFO:teuthology.orchestra.run.smithi140.stdout: osd op complaint time = 180 2022-07-21T01:57:23.261 INFO:teuthology.orchestra.run.smithi140.stdout: osd sloppy crc = True 2022-07-21T01:57:23.261 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.261 INFO:teuthology.orchestra.run.smithi140.stdout:[mgr] 2022-07-21T01:57:23.261 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:57:23.261 INFO:teuthology.orchestra.run.smithi140.stdout: debug mgr = 20 2022-07-21T01:57:23.262 INFO:teuthology.orchestra.run.smithi140.stdout: debug mon = 20 2022-07-21T01:57:23.262 INFO:teuthology.orchestra.run.smithi140.stdout: debug auth = 20 2022-07-21T01:57:23.262 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:23.262 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:23.263 INFO:teuthology.orchestra.run.smithi140.stdout: mgr/telemetry/nag = false 2022-07-21T01:57:23.263 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.263 INFO:teuthology.orchestra.run.smithi140.stdout:[mon] 2022-07-21T01:57:23.263 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:57:23.263 INFO:teuthology.orchestra.run.smithi140.stdout: debug mon = 20 2022-07-21T01:57:23.264 INFO:teuthology.orchestra.run.smithi140.stdout: debug paxos = 20 2022-07-21T01:57:23.264 INFO:teuthology.orchestra.run.smithi140.stdout: debug auth = 20 2022-07-21T01:57:23.264 INFO:teuthology.orchestra.run.smithi140.stdout: mon data avail warn = 5 2022-07-21T01:57:23.264 INFO:teuthology.orchestra.run.smithi140.stdout: mon mgr mkfs grace = 240 2022-07-21T01:57:23.265 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:23.265 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd reporter subtree level = osd 2022-07-21T01:57:23.265 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd prime pg temp = true 2022-07-21T01:57:23.265 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:23.265 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.266 INFO:teuthology.orchestra.run.smithi140.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:57:23.266 INFO:teuthology.orchestra.run.smithi140.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:57:23.266 INFO:teuthology.orchestra.run.smithi140.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:57:23.266 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.267 INFO:teuthology.orchestra.run.smithi140.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:57:23.267 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:57:23.267 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:57:23.267 INFO:teuthology.orchestra.run.smithi140.stdout: mon op complaint time = 120 2022-07-21T01:57:23.268 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.268 INFO:teuthology.orchestra.run.smithi140.stdout:[client] 2022-07-21T01:57:23.268 INFO:teuthology.orchestra.run.smithi140.stdout: rgw cache enabled = true 2022-07-21T01:57:23.268 INFO:teuthology.orchestra.run.smithi140.stdout: rgw enable ops log = true 2022-07-21T01:57:23.268 INFO:teuthology.orchestra.run.smithi140.stdout: rgw enable usage log = true 2022-07-21T01:57:23.269 INFO:teuthology.orchestra.run.smithi140.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:57:23.269 INFO:teuthology.orchestra.run.smithi140.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:57:23.269 INFO:teuthology.orchestra.run.smithi140.stdout: client mount timeout = 600 2022-07-21T01:57:23.269 INFO:teuthology.orchestra.run.smithi140.stdout: debug client = 20 2022-07-21T01:57:23.270 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:57:23.270 INFO:teuthology.orchestra.run.smithi140.stdout: rados mon op timeout = 15m 2022-07-21T01:57:23.270 INFO:teuthology.orchestra.run.smithi140.stdout: rados osd op timeout = 15m 2022-07-21T01:57:23.270 INFO:teuthology.orchestra.run.smithi140.stdout: client debug getattr caps = true 2022-07-21T01:57:23.271 INFO:teuthology.orchestra.run.smithi140.stdout:[mon.a] 2022-07-21T01:57:23.271 INFO:teuthology.orchestra.run.smithi140.stdout:[mon.b] 2022-07-21T01:57:23.271 INFO:teuthology.orchestra.run.smithi140.stdout:[mon.c] 2022-07-21T01:57:23.271 INFO:teuthology.orchestra.run.smithi140.stdout:[mds.a] 2022-07-21T01:57:23.271 INFO:teuthology.orchestra.run.smithi140.stdout:[mds.b] 2022-07-21T01:57:23.272 INFO:teuthology.orchestra.run.smithi140.stdout:[mds] 2022-07-21T01:57:23.272 INFO:teuthology.orchestra.run.smithi140.stdout: debug mds = 20 2022-07-21T01:57:23.272 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:57:23.272 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal frag = True 2022-07-21T01:57:23.273 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal fragment size max = 10000 2022-07-21T01:57:23.273 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal merge size = 5 2022-07-21T01:57:23.273 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal split bits = 3 2022-07-21T01:57:23.273 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal split size = 100 2022-07-21T01:57:23.274 INFO:teuthology.orchestra.run.smithi140.stdout: mds debug frag = True 2022-07-21T01:57:23.274 INFO:teuthology.orchestra.run.smithi140.stdout: mds debug scatterstat = True 2022-07-21T01:57:23.274 INFO:teuthology.orchestra.run.smithi140.stdout: mds op complaint time = 180 2022-07-21T01:57:23.274 INFO:teuthology.orchestra.run.smithi140.stdout: mds verify scatter = True 2022-07-21T01:57:23.274 INFO:teuthology.orchestra.run.smithi140.stdout: osd op complaint time = 180 2022-07-21T01:57:23.275 INFO:teuthology.orchestra.run.smithi140.stdout: rados mon op timeout = 15m 2022-07-21T01:57:23.275 INFO:teuthology.orchestra.run.smithi140.stdout: rados osd op timeout = 15m 2022-07-21T01:57:23.279 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2022-07-21T01:57:23.279 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2022-07-21T01:57:23.279 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:57:23.280 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2022-07-21T01:57:23.280 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.280 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2022-07-21T01:57:23.280 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.281 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2022-07-21T01:57:23.281 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.281 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2022-07-21T01:57:23.281 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2022-07-21T01:57:23.282 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.282 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2022-07-21T01:57:23.282 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2022-07-21T01:57:23.282 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.282 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:57:23.283 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2022-07-21T01:57:23.283 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.283 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:57:23.283 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:57:23.284 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.284 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2022-07-21T01:57:23.284 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.284 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2022-07-21T01:57:23.285 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2022-07-21T01:57:23.285 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:57:23.285 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:57:23.285 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2022-07-21T01:57:23.285 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:57:23.286 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2022-07-21T01:57:23.286 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:57:23.286 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:57:23.286 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.287 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:57:23.287 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.287 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2022-07-21T01:57:23.287 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.287 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2022-07-21T01:57:23.288 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.288 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2022-07-21T01:57:23.288 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2022-07-21T01:57:23.288 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2022-07-21T01:57:23.289 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:57:23.289 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:57:23.289 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.289 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2022-07-21T01:57:23.289 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2022-07-21T01:57:23.290 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.290 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2022-07-21T01:57:23.290 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2022-07-21T01:57:23.290 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.291 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2022-07-21T01:57:23.291 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2022-07-21T01:57:23.291 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.291 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2022-07-21T01:57:23.292 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2022-07-21T01:57:23.292 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2022-07-21T01:57:23.292 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.292 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2022-07-21T01:57:23.292 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2022-07-21T01:57:23.293 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.293 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:57:23.293 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.293 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2022-07-21T01:57:23.294 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.294 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2022-07-21T01:57:23.294 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.294 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:57:23.295 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2022-07-21T01:57:23.295 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.295 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2022-07-21T01:57:23.295 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2022-07-21T01:57:23.295 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:57:23.296 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2022-07-21T01:57:23.296 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = filestore 2022-07-21T01:57:23.296 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2022-07-21T01:57:23.296 INFO:teuthology.orchestra.run.smithi195.stdout: osd sloppy crc = True 2022-07-21T01:57:23.297 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.297 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2022-07-21T01:57:23.297 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:57:23.297 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2022-07-21T01:57:23.298 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2022-07-21T01:57:23.298 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2022-07-21T01:57:23.298 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:23.298 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:23.298 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2022-07-21T01:57:23.299 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.299 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2022-07-21T01:57:23.299 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:57:23.299 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2022-07-21T01:57:23.300 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2022-07-21T01:57:23.300 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2022-07-21T01:57:23.300 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2022-07-21T01:57:23.300 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2022-07-21T01:57:23.301 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:23.301 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2022-07-21T01:57:23.301 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2022-07-21T01:57:23.301 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:23.301 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.302 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:57:23.302 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:57:23.302 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:57:23.302 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.303 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:57:23.303 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:57:23.303 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:57:23.303 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2022-07-21T01:57:23.304 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:23.304 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2022-07-21T01:57:23.304 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2022-07-21T01:57:23.304 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2022-07-21T01:57:23.304 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2022-07-21T01:57:23.305 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:57:23.305 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:57:23.305 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2022-07-21T01:57:23.305 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2022-07-21T01:57:23.306 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:57:23.306 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 15m 2022-07-21T01:57:23.306 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 15m 2022-07-21T01:57:23.306 INFO:teuthology.orchestra.run.smithi195.stdout: client debug getattr caps = true 2022-07-21T01:57:23.307 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2022-07-21T01:57:23.307 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2022-07-21T01:57:23.307 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2022-07-21T01:57:23.307 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2022-07-21T01:57:23.308 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2022-07-21T01:57:23.308 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2022-07-21T01:57:23.308 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2022-07-21T01:57:23.308 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:57:23.308 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal frag = True 2022-07-21T01:57:23.309 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal fragment size max = 10000 2022-07-21T01:57:23.309 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal merge size = 5 2022-07-21T01:57:23.309 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal split bits = 3 2022-07-21T01:57:23.309 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal split size = 100 2022-07-21T01:57:23.310 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2022-07-21T01:57:23.310 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = True 2022-07-21T01:57:23.310 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2022-07-21T01:57:23.310 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = True 2022-07-21T01:57:23.310 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2022-07-21T01:57:23.311 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 15m 2022-07-21T01:57:23.311 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 15m 2022-07-21T01:57:23.311 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2022-07-21T01:57:23.312 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2022-07-21T01:57:23.312 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:57:23.312 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2022-07-21T01:57:23.312 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.312 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2022-07-21T01:57:23.313 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.313 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2022-07-21T01:57:23.313 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.313 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2022-07-21T01:57:23.314 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2022-07-21T01:57:23.314 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.314 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2022-07-21T01:57:23.314 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2022-07-21T01:57:23.314 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.315 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:57:23.315 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2022-07-21T01:57:23.315 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.315 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:57:23.316 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:57:23.316 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.316 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2022-07-21T01:57:23.316 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.316 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2022-07-21T01:57:23.317 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2022-07-21T01:57:23.317 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:57:23.317 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:57:23.317 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2022-07-21T01:57:23.318 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:57:23.318 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2022-07-21T01:57:23.318 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:57:23.318 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:57:23.319 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.319 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:57:23.319 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.319 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2022-07-21T01:57:23.319 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.320 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2022-07-21T01:57:23.320 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.320 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2022-07-21T01:57:23.320 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2022-07-21T01:57:23.321 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2022-07-21T01:57:23.321 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:57:23.321 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:57:23.321 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.321 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2022-07-21T01:57:23.322 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2022-07-21T01:57:23.322 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.322 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2022-07-21T01:57:23.322 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2022-07-21T01:57:23.323 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.323 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2022-07-21T01:57:23.323 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2022-07-21T01:57:23.323 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.324 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2022-07-21T01:57:23.324 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2022-07-21T01:57:23.324 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2022-07-21T01:57:23.324 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.324 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2022-07-21T01:57:23.325 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2022-07-21T01:57:23.325 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.325 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:57:23.325 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.326 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2022-07-21T01:57:23.326 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.326 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2022-07-21T01:57:23.326 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.327 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:57:23.327 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2022-07-21T01:57:23.327 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.327 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2022-07-21T01:57:23.327 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2022-07-21T01:57:23.328 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:57:23.328 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2022-07-21T01:57:23.328 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = filestore 2022-07-21T01:57:23.328 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2022-07-21T01:57:23.329 INFO:teuthology.orchestra.run.smithi192.stdout: osd sloppy crc = True 2022-07-21T01:57:23.329 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.329 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2022-07-21T01:57:23.329 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:57:23.329 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2022-07-21T01:57:23.330 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2022-07-21T01:57:23.330 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2022-07-21T01:57:23.330 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:23.330 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:23.331 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2022-07-21T01:57:23.331 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.331 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2022-07-21T01:57:23.331 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:57:23.331 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2022-07-21T01:57:23.332 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2022-07-21T01:57:23.332 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2022-07-21T01:57:23.332 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2022-07-21T01:57:23.332 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2022-07-21T01:57:23.333 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:23.333 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2022-07-21T01:57:23.333 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2022-07-21T01:57:23.333 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:23.334 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.334 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:57:23.334 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:57:23.334 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:57:23.334 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.335 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:57:23.335 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:57:23.335 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:57:23.335 INFO:teuthology.orchestra.run.smithi192.stdout: mon op complaint time = 120 2022-07-21T01:57:23.336 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:23.336 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2022-07-21T01:57:23.336 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2022-07-21T01:57:23.336 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2022-07-21T01:57:23.337 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2022-07-21T01:57:23.337 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:57:23.337 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:57:23.337 INFO:teuthology.orchestra.run.smithi192.stdout: client mount timeout = 600 2022-07-21T01:57:23.337 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2022-07-21T01:57:23.338 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:57:23.338 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2022-07-21T01:57:23.338 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2022-07-21T01:57:23.338 INFO:teuthology.orchestra.run.smithi192.stdout: client debug getattr caps = true 2022-07-21T01:57:23.339 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2022-07-21T01:57:23.339 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2022-07-21T01:57:23.339 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2022-07-21T01:57:23.339 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2022-07-21T01:57:23.340 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2022-07-21T01:57:23.340 INFO:teuthology.orchestra.run.smithi192.stdout:[mds] 2022-07-21T01:57:23.340 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds = 20 2022-07-21T01:57:23.340 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:57:23.340 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal frag = True 2022-07-21T01:57:23.341 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal fragment size max = 10000 2022-07-21T01:57:23.341 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal merge size = 5 2022-07-21T01:57:23.341 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split bits = 3 2022-07-21T01:57:23.341 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split size = 100 2022-07-21T01:57:23.342 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug frag = True 2022-07-21T01:57:23.342 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug scatterstat = True 2022-07-21T01:57:23.342 INFO:teuthology.orchestra.run.smithi192.stdout: mds op complaint time = 180 2022-07-21T01:57:23.342 INFO:teuthology.orchestra.run.smithi192.stdout: mds verify scatter = True 2022-07-21T01:57:23.343 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2022-07-21T01:57:23.343 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2022-07-21T01:57:23.343 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2022-07-21T01:57:23.343 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2022-07-21T01:57:23.344 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2022-07-21T01:57:23.344 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:57:23.344 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2022-07-21T01:57:23.344 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.344 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2022-07-21T01:57:23.345 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.345 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2022-07-21T01:57:23.345 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.345 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2022-07-21T01:57:23.346 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2022-07-21T01:57:23.346 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.346 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2022-07-21T01:57:23.346 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2022-07-21T01:57:23.347 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.347 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:57:23.347 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2022-07-21T01:57:23.347 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.347 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:57:23.348 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:57:23.348 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.348 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2022-07-21T01:57:23.348 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.349 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2022-07-21T01:57:23.349 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2022-07-21T01:57:23.349 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:57:23.349 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:57:23.349 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2022-07-21T01:57:23.350 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:57:23.350 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2022-07-21T01:57:23.350 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:57:23.350 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:57:23.351 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.351 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:57:23.351 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.351 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2022-07-21T01:57:23.351 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.352 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2022-07-21T01:57:23.352 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.352 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2022-07-21T01:57:23.352 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2022-07-21T01:57:23.353 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2022-07-21T01:57:23.353 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:57:23.353 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:57:23.353 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2022-07-21T01:57:23.354 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2022-07-21T01:57:23.354 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.354 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2022-07-21T01:57:23.354 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2022-07-21T01:57:23.355 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.355 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2022-07-21T01:57:23.355 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2022-07-21T01:57:23.355 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.355 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2022-07-21T01:57:23.356 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2022-07-21T01:57:23.356 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2022-07-21T01:57:23.356 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.356 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2022-07-21T01:57:23.357 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2022-07-21T01:57:23.357 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.357 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:57:23.357 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.357 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2022-07-21T01:57:23.358 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.358 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2022-07-21T01:57:23.358 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.358 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:57:23.359 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2022-07-21T01:57:23.359 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.359 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2022-07-21T01:57:23.359 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2022-07-21T01:57:23.359 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:57:23.360 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2022-07-21T01:57:23.360 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = filestore 2022-07-21T01:57:23.360 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2022-07-21T01:57:23.360 INFO:teuthology.orchestra.run.smithi191.stdout: osd sloppy crc = True 2022-07-21T01:57:23.361 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.361 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2022-07-21T01:57:23.361 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:57:23.361 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2022-07-21T01:57:23.361 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2022-07-21T01:57:23.362 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2022-07-21T01:57:23.362 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:23.362 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:23.362 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2022-07-21T01:57:23.363 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.363 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2022-07-21T01:57:23.363 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:57:23.363 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2022-07-21T01:57:23.363 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2022-07-21T01:57:23.364 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2022-07-21T01:57:23.364 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2022-07-21T01:57:23.364 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2022-07-21T01:57:23.364 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:23.365 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2022-07-21T01:57:23.365 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2022-07-21T01:57:23.365 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:23.365 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.366 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:57:23.366 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:57:23.366 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:57:23.366 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.367 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:57:23.367 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:57:23.367 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:57:23.367 INFO:teuthology.orchestra.run.smithi191.stdout: mon op complaint time = 120 2022-07-21T01:57:23.367 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:23.368 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2022-07-21T01:57:23.368 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2022-07-21T01:57:23.368 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2022-07-21T01:57:23.368 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2022-07-21T01:57:23.369 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:57:23.369 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:57:23.369 INFO:teuthology.orchestra.run.smithi191.stdout: client mount timeout = 600 2022-07-21T01:57:23.369 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2022-07-21T01:57:23.369 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:57:23.370 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 15m 2022-07-21T01:57:23.370 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 15m 2022-07-21T01:57:23.370 INFO:teuthology.orchestra.run.smithi191.stdout: client debug getattr caps = true 2022-07-21T01:57:23.370 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2022-07-21T01:57:23.371 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.b] 2022-07-21T01:57:23.371 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.c] 2022-07-21T01:57:23.371 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2022-07-21T01:57:23.371 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2022-07-21T01:57:23.371 INFO:teuthology.orchestra.run.smithi191.stdout:[mds] 2022-07-21T01:57:23.372 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds = 20 2022-07-21T01:57:23.372 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:57:23.372 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal frag = True 2022-07-21T01:57:23.372 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal fragment size max = 10000 2022-07-21T01:57:23.373 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal merge size = 5 2022-07-21T01:57:23.373 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal split bits = 3 2022-07-21T01:57:23.373 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal split size = 100 2022-07-21T01:57:23.373 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug frag = True 2022-07-21T01:57:23.373 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug scatterstat = True 2022-07-21T01:57:23.374 INFO:teuthology.orchestra.run.smithi191.stdout: mds op complaint time = 180 2022-07-21T01:57:23.374 INFO:teuthology.orchestra.run.smithi191.stdout: mds verify scatter = True 2022-07-21T01:57:23.374 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2022-07-21T01:57:23.374 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 15m 2022-07-21T01:57:23.375 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 15m 2022-07-21T01:57:23.375 INFO:teuthology.orchestra.run.smithi152.stdout:[global] 2022-07-21T01:57:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: chdir = "" 2022-07-21T01:57:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:57:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: auth supported = cephx 2022-07-21T01:57:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: filestore xattr use omap = true 2022-07-21T01:57:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: mon clock drift allowed = 1.000 2022-07-21T01:57:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: osd crush chooseleaf type = 0 2022-07-21T01:57:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: auth debug = true 2022-07-21T01:57:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on old message = true 2022-07-21T01:57:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on bug = true 2022-07-21T01:57:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:57:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: mon pg warn max object skew = 0 2022-07-21T01:57:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:57:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:57:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default size = 2 2022-07-21T01:57:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow primary affinity = true 2022-07-21T01:57:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow pg remap = true 2022-07-21T01:57:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:57:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:57:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on no sortbitwise = false 2022-07-21T01:57:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:57:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on too few osds = false 2022-07-21T01:57:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:57:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:57:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:57:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: osd default data pool replay window = 5 2022-07-21T01:57:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: mon allow pool delete = true 2022-07-21T01:57:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: mon cluster log file level = debug 2022-07-21T01:57:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: debug asserts on shutdown = true 2022-07-21T01:57:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: mon health detail to clog = false 2022-07-21T01:57:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:57:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:57:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.385 INFO:teuthology.orchestra.run.smithi152.stdout:[osd] 2022-07-21T01:57:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: osd journal size = 100 2022-07-21T01:57:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub load threshold = 5.0 2022-07-21T01:57:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub max interval = 600 2022-07-21T01:57:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: osd recover clone overlap = true 2022-07-21T01:57:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: osd recovery max chunk = 1048576 2022-07-21T01:57:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug shutdown = true 2022-07-21T01:57:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug op order = true 2022-07-21T01:57:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify stray on activate = true 2022-07-21T01:57:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: osd open classes on start = true 2022-07-21T01:57:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug pg log writeout = true 2022-07-21T01:57:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:57:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: osd map max advance = 10 2022-07-21T01:57:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: journal zero on create = true 2022-07-21T01:57:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:57:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: filestore apply finisher threads = 3 2022-07-21T01:57:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: bdev debug aio = true 2022-07-21T01:57:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug misdirected ops = true 2022-07-21T01:57:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:57:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: debug osd = 20 2022-07-21T01:57:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: osd objectstore = filestore 2022-07-21T01:57:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2022-07-21T01:57:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: osd sloppy crc = True 2022-07-21T01:57:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.392 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr] 2022-07-21T01:57:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:57:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: debug mgr = 20 2022-07-21T01:57:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2022-07-21T01:57:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2022-07-21T01:57:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: mgr/telemetry/nag = false 2022-07-21T01:57:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.395 INFO:teuthology.orchestra.run.smithi152.stdout:[mon] 2022-07-21T01:57:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:57:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2022-07-21T01:57:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: debug paxos = 20 2022-07-21T01:57:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2022-07-21T01:57:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: mon data avail warn = 5 2022-07-21T01:57:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: mon mgr mkfs grace = 240 2022-07-21T01:57:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd reporter subtree level = osd 2022-07-21T01:57:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd prime pg temp = true 2022-07-21T01:57:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:57:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:57:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:57:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:57:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:57:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:57:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: mon op complaint time = 120 2022-07-21T01:57:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:23.399 INFO:teuthology.orchestra.run.smithi152.stdout:[client] 2022-07-21T01:57:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: rgw cache enabled = true 2022-07-21T01:57:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable ops log = true 2022-07-21T01:57:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable usage log = true 2022-07-21T01:57:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:57:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:57:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: client mount timeout = 600 2022-07-21T01:57:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: debug client = 20 2022-07-21T01:57:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:57:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 15m 2022-07-21T01:57:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 15m 2022-07-21T01:57:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: client debug getattr caps = true 2022-07-21T01:57:23.402 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.a] 2022-07-21T01:57:23.402 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.b] 2022-07-21T01:57:23.402 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.c] 2022-07-21T01:57:23.403 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.a] 2022-07-21T01:57:23.403 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.b] 2022-07-21T01:57:23.403 INFO:teuthology.orchestra.run.smithi152.stdout:[mds] 2022-07-21T01:57:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: debug mds = 20 2022-07-21T01:57:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:57:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal frag = True 2022-07-21T01:57:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal fragment size max = 10000 2022-07-21T01:57:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal merge size = 5 2022-07-21T01:57:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal split bits = 3 2022-07-21T01:57:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal split size = 100 2022-07-21T01:57:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug frag = True 2022-07-21T01:57:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug scatterstat = True 2022-07-21T01:57:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: mds op complaint time = 180 2022-07-21T01:57:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: mds verify scatter = True 2022-07-21T01:57:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2022-07-21T01:57:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 15m 2022-07-21T01:57:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 15m 2022-07-21T01:57:23.407 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:23.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230f4c4700 1 Processor -- start 2022-07-21T01:57:23.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230f4c4700 1 -- start start 2022-07-21T01:57:23.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230f4c4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23080b5270 0x7f23080ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230f4c4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f23080afbd0 0x7f23080ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230f4c4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f230814a530 0x7f230814a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230f4c4700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23080b5650 con 0x7f230814a530 2022-07-21T01:57:23.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230f4c4700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23080ad9b0 con 0x7f23080afbd0 2022-07-21T01:57:23.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230f4c4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23080adc50 con 0x7f23080b5270 2022-07-21T01:57:23.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230ecc3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f230814a530 0x7f230814a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230ecc3700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f230814a530 0x7f230814a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40608/0 (socket says 172.21.15.140:40608) 2022-07-21T01:57:23.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230ecc3700 1 -- 172.21.15.140:0/1261934851 learned_addr learned my addr 172.21.15.140:0/1261934851 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:23.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230dcc1700 1 --2- 172.21.15.140:0/1261934851 >> v2:172.21.15.152:3300/0 conn(0x7f23080afbd0 0x7f23080ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230e4c2700 1 --2- 172.21.15.140:0/1261934851 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23080b5270 0x7f23080ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230ecc3700 1 -- 172.21.15.140:0/1261934851 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23080b5270 msgr2=0x7f23080ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:23.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230ecc3700 1 --2- 172.21.15.140:0/1261934851 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23080b5270 0x7f23080ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230ecc3700 1 -- 172.21.15.140:0/1261934851 >> v2:172.21.15.152:3300/0 conn(0x7f23080afbd0 msgr2=0x7f23080ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:23.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.516+0000 7f230ecc3700 1 --2- 172.21.15.140:0/1261934851 >> v2:172.21.15.152:3300/0 conn(0x7f23080afbd0 0x7f23080ac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230ecc3700 1 -- 172.21.15.140:0/1261934851 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23080adef0 con 0x7f230814a530 2022-07-21T01:57:23.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230ecc3700 1 --2- 172.21.15.140:0/1261934851 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f230814a530 0x7f230814a960 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f2300002f70 tx=0x7f2300006eb0).ready entity=mon.0 client_cookie=a1273d5129badee8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:23.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230d4c0700 1 -- 172.21.15.140:0/1261934851 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f230000d570 con 0x7f230814a530 2022-07-21T01:57:23.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230d4c0700 1 -- 172.21.15.140:0/1261934851 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300000e90 con 0x7f230814a530 2022-07-21T01:57:23.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230d4c0700 1 -- 172.21.15.140:0/1261934851 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f230000d900 con 0x7f230814a530 2022-07-21T01:57:23.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230f4c4700 1 -- 172.21.15.140:0/1261934851 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f230814a530 msgr2=0x7f230814a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:23.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230f4c4700 1 --2- 172.21.15.140:0/1261934851 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f230814a530 0x7f230814a960 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f2300002f70 tx=0x7f2300006eb0).stop 2022-07-21T01:57:23.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230f4c4700 1 -- 172.21.15.140:0/1261934851 shutdown_connections 2022-07-21T01:57:23.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230f4c4700 1 --2- 172.21.15.140:0/1261934851 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f230814a530 0x7f230814a960 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230f4c4700 1 --2- 172.21.15.140:0/1261934851 >> v2:172.21.15.152:3300/0 conn(0x7f23080afbd0 0x7f23080ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230f4c4700 1 --2- 172.21.15.140:0/1261934851 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23080b5270 0x7f23080ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230f4c4700 1 -- 172.21.15.140:0/1261934851 >> 172.21.15.140:0/1261934851 conn(0x7f2308062940 msgr2=0x7f2308062d20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:23.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230f4c4700 1 -- 172.21.15.140:0/1261934851 shutdown_connections 2022-07-21T01:57:23.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.520+0000 7f230f4c4700 1 -- 172.21.15.140:0/1261934851 wait complete. 2022-07-21T01:57:23.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230f4c4700 1 Processor -- start 2022-07-21T01:57:23.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230f4c4700 1 -- start start 2022-07-21T01:57:23.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230f4c4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23080afbd0 0x7f23081578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230f4c4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23080b5270 0x7f23080a9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230f4c4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230814a530 0x7f2308156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230f4c4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23080673a0 con 0x7f23080b5270 2022-07-21T01:57:23.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230f4c4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23080663a0 con 0x7f230814a530 2022-07-21T01:57:23.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230f4c4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2308066ef0 con 0x7f23080afbd0 2022-07-21T01:57:23.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230dcc1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23080b5270 0x7f23080a9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230dcc1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23080b5270 0x7f23080a9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40612/0 (socket says 172.21.15.140:40612) 2022-07-21T01:57:23.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230dcc1700 1 -- 172.21.15.140:0/264018140 learned_addr learned my addr 172.21.15.140:0/264018140 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:23.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230e4c2700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23080afbd0 0x7f23081578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230ecc3700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230814a530 0x7f2308156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230dcc1700 1 -- 172.21.15.140:0/264018140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23080afbd0 msgr2=0x7f23081578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:23.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230dcc1700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23080afbd0 0x7f23081578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230dcc1700 1 -- 172.21.15.140:0/264018140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230814a530 msgr2=0x7f2308156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:23.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230dcc1700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230814a530 0x7f2308156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230dcc1700 1 -- 172.21.15.140:0/264018140 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230000b1e0 con 0x7f23080b5270 2022-07-21T01:57:23.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f230dcc1700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23080b5270 0x7f23080a9b70 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f230400c000 tx=0x7f23040091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:23.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f22f77fe700 1 -- 172.21.15.140:0/264018140 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f230400c850 con 0x7f23080b5270 2022-07-21T01:57:23.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.524+0000 7f22f77fe700 1 -- 172.21.15.140:0/264018140 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230400c9b0 con 0x7f23080b5270 2022-07-21T01:57:23.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.528+0000 7f230f4c4700 1 -- 172.21.15.140:0/264018140 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23080aa060 con 0x7f23080b5270 2022-07-21T01:57:23.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.528+0000 7f22f77fe700 1 -- 172.21.15.140:0/264018140 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f230400cd10 con 0x7f23080b5270 2022-07-21T01:57:23.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.528+0000 7f230f4c4700 1 -- 172.21.15.140:0/264018140 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2308144600 con 0x7f23080b5270 2022-07-21T01:57:23.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.528+0000 7f230f4c4700 1 -- 172.21.15.140:0/264018140 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23080574e0 con 0x7f23080b5270 2022-07-21T01:57:23.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.528+0000 7f22f77fe700 1 -- 172.21.15.140:0/264018140 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2304022070 con 0x7f23080b5270 2022-07-21T01:57:23.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.528+0000 7f22f77fe700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f22e80539e0 0x7f22e8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.528+0000 7f22f77fe700 1 -- 172.21.15.140:0/264018140 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f230406c6d0 con 0x7f23080b5270 2022-07-21T01:57:23.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.528+0000 7f230e4c2700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f22e80539e0 0x7f22e8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.528+0000 7f230e4c2700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f22e80539e0 0x7f22e8055ea0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f22f800b000 tx=0x7f22f8006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:23.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.532+0000 7f22f77fe700 1 -- 172.21.15.140:0/264018140 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f23040434e0 con 0x7f23080b5270 2022-07-21T01:57:23.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.660+0000 7f230f4c4700 1 -- 172.21.15.140:0/264018140 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f230805cc20 con 0x7f23080b5270 2022-07-21T01:57:23.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f22f77fe700 1 -- 172.21.15.140:0/264018140 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f2304046b00 con 0x7f23080b5270 2022-07-21T01:57:23.664 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:23.665 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":132,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:15.426209+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:57:12.443397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2022-07-21T01:57:13.427432+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:23.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 -- 172.21.15.140:0/264018140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f22e80539e0 msgr2=0x7f22e8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:23.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f22e80539e0 0x7f22e8055ea0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f22f800b000 tx=0x7f22f8006df0).stop 2022-07-21T01:57:23.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 -- 172.21.15.140:0/264018140 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23080b5270 msgr2=0x7f23080a9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:23.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23080b5270 0x7f23080a9b70 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f230400c000 tx=0x7f23040091d0).stop 2022-07-21T01:57:23.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 -- 172.21.15.140:0/264018140 shutdown_connections 2022-07-21T01:57:23.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f22e80539e0 0x7f22e8055ea0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f230814a530 0x7f2308156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23080b5270 0x7f23080a9b70 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 --2- 172.21.15.140:0/264018140 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23080afbd0 0x7f23081578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 -- 172.21.15.140:0/264018140 >> 172.21.15.140:0/264018140 conn(0x7f2308062940 msgr2=0x7f23081477b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:23.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 -- 172.21.15.140:0/264018140 shutdown_connections 2022-07-21T01:57:23.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.664+0000 7f230f4c4700 1 -- 172.21.15.140:0/264018140 wait complete. 2022-07-21T01:57:23.678 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:57:12.443397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:57:13.427432+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:57:23.679 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:57:23.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.784+0000 7fa156dee700 1 Processor -- start 2022-07-21T01:57:23.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.784+0000 7fa156dee700 1 -- start start 2022-07-21T01:57:23.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.784+0000 7fa156dee700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa1500b0430 0x7fa15013cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.784+0000 7fa156dee700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa150142cd0 0x7fa150143160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.784+0000 7fa156dee700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15005cc60 con 0x7fa15005c880 2022-07-21T01:57:23.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.784+0000 7fa156dee700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1500ad5f0 con 0x7fa1500b0430 2022-07-21T01:57:23.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.784+0000 7fa156dee700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500ae970 con 0x7fa150142cd0 2022-07-21T01:57:23.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.784+0000 7fa14ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa1500b0430 0x7fa15013cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.784+0000 7fa154b8a700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa15005c880 0x7fa15013c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54764/0 (socket says 172.21.15.140:54764) 2022-07-21T01:57:23.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.784+0000 7fa14ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa1500b0430 0x7fa15013cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40620/0 (socket says 172.21.15.140:40620) 2022-07-21T01:57:23.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.784+0000 7fa154b8a700 1 -- 172.21.15.140:0/3763070646 learned_addr learned my addr 172.21.15.140:0/3763070646 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:23.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa15538b700 1 --2- 172.21.15.140:0/3763070646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa150142cd0 0x7fa150143160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa14ffff700 1 -- 172.21.15.140:0/3763070646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa150142cd0 msgr2=0x7fa150143160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:23.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa14ffff700 1 --2- 172.21.15.140:0/3763070646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa150142cd0 0x7fa150143160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa14ffff700 1 -- 172.21.15.140:0/3763070646 >> v1:172.21.15.140:6789/0 conn(0x7fa15005c880 legacy=0x7fa15013c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:23.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa14ffff700 1 -- 172.21.15.140:0/3763070646 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1500aec10 con 0x7fa1500b0430 2022-07-21T01:57:23.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa14f7fe700 1 -- 172.21.15.140:0/3763070646 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277811488 0 0) 0x7fa15005cc60 con 0x7fa15005c880 2022-07-21T01:57:23.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa14ffff700 1 --2- 172.21.15.140:0/3763070646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1500b0430 0x7fa15013cd60 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7fa1380032f0 tx=0x7fa1380091d0).ready entity=mon.0 client_cookie=1fd31d75bb1a4868 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:23.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa14f7fe700 1 -- 172.21.15.140:0/3763070646 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa13800b6c0 con 0x7fa1500b0430 2022-07-21T01:57:23.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa14f7fe700 1 -- 172.21.15.140:0/3763070646 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13800b820 con 0x7fa1500b0430 2022-07-21T01:57:23.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa14f7fe700 1 -- 172.21.15.140:0/3763070646 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa13800b990 con 0x7fa1500b0430 2022-07-21T01:57:23.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa156dee700 1 -- 172.21.15.140:0/3763070646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1500b0430 msgr2=0x7fa15013cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:23.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa156dee700 1 --2- 172.21.15.140:0/3763070646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1500b0430 0x7fa15013cd60 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7fa1380032f0 tx=0x7fa1380091d0).stop 2022-07-21T01:57:23.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa156dee700 1 -- 172.21.15.140:0/3763070646 shutdown_connections 2022-07-21T01:57:23.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa156dee700 1 --2- 172.21.15.140:0/3763070646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa150142cd0 0x7fa150143160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa156dee700 1 --2- 172.21.15.140:0/3763070646 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1500b0430 0x7fa15013cd60 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa156dee700 1 -- 172.21.15.140:0/3763070646 >> 172.21.15.140:0/3763070646 conn(0x7fa150063460 msgr2=0x7fa15005be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:23.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa156dee700 1 -- 172.21.15.140:0/3763070646 shutdown_connections 2022-07-21T01:57:23.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa156dee700 1 -- 172.21.15.140:0/3763070646 wait complete. 2022-07-21T01:57:23.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa156dee700 1 Processor -- start 2022-07-21T01:57:23.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.788+0000 7fa156dee700 1 -- start start 2022-07-21T01:57:23.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa156dee700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa15005c880 0x7fa15013e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa156dee700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1500b0430 0x7fa15010f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa156dee700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa150142cd0 0x7fa150139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa156dee700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15013e6d0 con 0x7fa1500b0430 2022-07-21T01:57:23.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa156dee700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa150140310 con 0x7fa150142cd0 2022-07-21T01:57:23.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa156dee700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1501405b0 con 0x7fa15005c880 2022-07-21T01:57:23.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1500b0430 0x7fa15010f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1500b0430 0x7fa15010f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40626/0 (socket says 172.21.15.140:40626) 2022-07-21T01:57:23.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14ffff700 1 -- 172.21.15.140:0/3439932259 learned_addr learned my addr 172.21.15.140:0/3439932259 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:23.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa154b8a700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa15005c880 0x7fa15013e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa15538b700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa150142cd0 0x7fa150139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14ffff700 1 -- 172.21.15.140:0/3439932259 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa15005c880 msgr2=0x7fa15013e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:23.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14ffff700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa15005c880 0x7fa15013e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14ffff700 1 -- 172.21.15.140:0/3439932259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa150142cd0 msgr2=0x7fa150139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:23.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14ffff700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa150142cd0 0x7fa150139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14ffff700 1 -- 172.21.15.140:0/3439932259 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa138009470 con 0x7fa1500b0430 2022-07-21T01:57:23.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa15538b700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa150142cd0 0x7fa150139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:23.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14ffff700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1500b0430 0x7fa15010f2f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7fa1380032f0 tx=0x7fa138016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:23.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14dffb700 1 -- 172.21.15.140:0/3439932259 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa13800b990 con 0x7fa1500b0430 2022-07-21T01:57:23.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14dffb700 1 -- 172.21.15.140:0/3439932259 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa138021630 con 0x7fa1500b0430 2022-07-21T01:57:23.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa156dee700 1 -- 172.21.15.140:0/3439932259 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15010f7e0 con 0x7fa1500b0430 2022-07-21T01:57:23.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14dffb700 1 -- 172.21.15.140:0/3439932259 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa13800b6c0 con 0x7fa1500b0430 2022-07-21T01:57:23.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa156dee700 1 -- 172.21.15.140:0/3439932259 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa150139c20 con 0x7fa1500b0430 2022-07-21T01:57:23.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa156dee700 1 -- 172.21.15.140:0/3439932259 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1500b2d80 con 0x7fa1500b0430 2022-07-21T01:57:23.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.792+0000 7fa14dffb700 1 -- 172.21.15.140:0/3439932259 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa13800f070 con 0x7fa1500b0430 2022-07-21T01:57:23.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.796+0000 7fa14dffb700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa128053950 0x7fa128055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:23.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.796+0000 7fa14dffb700 1 -- 172.21.15.140:0/3439932259 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa13806d350 con 0x7fa1500b0430 2022-07-21T01:57:23.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.796+0000 7fa154b8a700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa128053950 0x7fa128055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:23.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.796+0000 7fa154b8a700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa128053950 0x7fa128055e10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fa14800d000 tx=0x7fa1480017c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:23.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.800+0000 7fa14dffb700 1 -- 172.21.15.140:0/3439932259 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa138044210 con 0x7fa1500b0430 2022-07-21T01:57:23.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.920+0000 7fa156dee700 1 -- 172.21.15.140:0/3439932259 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa150068a70 con 0x7fa1500b0430 2022-07-21T01:57:23.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa14dffb700 1 -- 172.21.15.140:0/3439932259 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v132) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa1380150b0 con 0x7fa1500b0430 2022-07-21T01:57:23.925 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:57:23.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 -- 172.21.15.140:0/3439932259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa128053950 msgr2=0x7fa128055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:23.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa128053950 0x7fa128055e10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fa14800d000 tx=0x7fa1480017c0).stop 2022-07-21T01:57:23.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 -- 172.21.15.140:0/3439932259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1500b0430 msgr2=0x7fa15010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:23.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1500b0430 0x7fa15010f2f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7fa1380032f0 tx=0x7fa138016000).stop 2022-07-21T01:57:23.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 -- 172.21.15.140:0/3439932259 shutdown_connections 2022-07-21T01:57:23.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa128053950 0x7fa128055e10 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa150142cd0 0x7fa150139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa1500b0430 0x7fa15010f2f0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 --2- 172.21.15.140:0/3439932259 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa15005c880 0x7fa15013e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:23.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 -- 172.21.15.140:0/3439932259 >> 172.21.15.140:0/3439932259 conn(0x7fa150063460 msgr2=0x7fa15013a750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:23.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 -- 172.21.15.140:0/3439932259 shutdown_connections 2022-07-21T01:57:23.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:23.924+0000 7fa156dee700 1 -- 172.21.15.140:0/3439932259 wait complete. 2022-07-21T01:57:23.937 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:57:24.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.040+0000 7f714f59e700 1 Processor -- start 2022-07-21T01:57:24.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.040+0000 7f714f59e700 1 -- start start 2022-07-21T01:57:24.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f71500b0da0 0x7f71500b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f71500af0b0 0x7f715013f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71500af680 con 0x7f71500b14a0 2022-07-21T01:57:24.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7150140bf0 con 0x7f71500af0b0 2022-07-21T01:57:24.045 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71500af920 con 0x7f71500b0da0 2022-07-21T01:57:24.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f7147fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f71500b0da0 0x7f71500b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714ed9d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f71500af0b0 0x7f715013f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714ed9d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f71500af0b0 0x7f715013f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:46376/0 (socket says 172.21.15.140:46376) 2022-07-21T01:57:24.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714ed9d700 1 -- 172.21.15.140:0/59875886 learned_addr learned my addr 172.21.15.140:0/59875886 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:24.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714e59c700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f71500b14a0 0x7f7150067770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:55892/0 (socket says 172.21.15.140:55892) 2022-07-21T01:57:24.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714ed9d700 1 -- 172.21.15.140:0/59875886 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f71500b0da0 msgr2=0x7f71500b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:24.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714ed9d700 1 --2- 172.21.15.140:0/59875886 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f71500b0da0 0x7f71500b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714ed9d700 1 -- 172.21.15.140:0/59875886 >> v1:172.21.15.152:6789/0 conn(0x7f71500b14a0 legacy=0x7f7150067770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714ed9d700 1 -- 172.21.15.140:0/59875886 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71500afbc0 con 0x7f71500af0b0 2022-07-21T01:57:24.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714dd9b700 1 -- 172.21.15.140:0/59875886 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979671044 0 0) 0x7f71500af680 con 0x7f71500b14a0 2022-07-21T01:57:24.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714ed9d700 1 --2- 172.21.15.140:0/59875886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71500af0b0 0x7f715013f940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f71480032f0 tx=0x7f7148009290).ready entity=mon.1 client_cookie=cc82022724b1f689 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714dd9b700 1 -- 172.21.15.140:0/59875886 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f714800b8a0 con 0x7f71500af0b0 2022-07-21T01:57:24.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714dd9b700 1 -- 172.21.15.140:0/59875886 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714800ba00 con 0x7f71500af0b0 2022-07-21T01:57:24.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714dd9b700 1 -- 172.21.15.140:0/59875886 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f714800bb70 con 0x7f71500af0b0 2022-07-21T01:57:24.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 -- 172.21.15.140:0/59875886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71500af0b0 msgr2=0x7f715013f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 --2- 172.21.15.140:0/59875886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71500af0b0 0x7f715013f940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f71480032f0 tx=0x7f7148009290).stop 2022-07-21T01:57:24.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 -- 172.21.15.140:0/59875886 shutdown_connections 2022-07-21T01:57:24.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 --2- 172.21.15.140:0/59875886 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f71500b0da0 0x7f71500b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 --2- 172.21.15.140:0/59875886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71500af0b0 0x7f715013f940 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 -- 172.21.15.140:0/59875886 >> 172.21.15.140:0/59875886 conn(0x7f7150063200 msgr2=0x7f71500635e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:24.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 -- 172.21.15.140:0/59875886 shutdown_connections 2022-07-21T01:57:24.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.044+0000 7f714f59e700 1 -- 172.21.15.140:0/59875886 wait complete. 2022-07-21T01:57:24.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714f59e700 1 Processor -- start 2022-07-21T01:57:24.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714f59e700 1 -- start start 2022-07-21T01:57:24.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f71500af0b0 0x7f7150152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714f59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71500b0da0 0x7f71500b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714f59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f71500b14a0 0x7f71501107e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714f59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7150143610 con 0x7f71500b14a0 2022-07-21T01:57:24.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714f59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71501438b0 con 0x7f71500b0da0 2022-07-21T01:57:24.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7150143b50 con 0x7f71500af0b0 2022-07-21T01:57:24.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714e59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f71500af0b0 0x7f7150152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f71500b14a0 0x7f71501107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714e59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f71500af0b0 0x7f7150152b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:37368/0 (socket says 172.21.15.140:37368) 2022-07-21T01:57:24.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714e59c700 1 -- 172.21.15.140:0/713598471 learned_addr learned my addr 172.21.15.140:0/713598471 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:24.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714ed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f71500b14a0 0x7f71501107e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40642/0 (socket says 172.21.15.140:40642) 2022-07-21T01:57:24.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f7147fff700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71500b0da0 0x7f71500b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714ed9d700 1 -- 172.21.15.140:0/713598471 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f71500af0b0 msgr2=0x7f7150152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714ed9d700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f71500af0b0 0x7f7150152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714ed9d700 1 -- 172.21.15.140:0/713598471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71500b0da0 msgr2=0x7f71500b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714ed9d700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71500b0da0 0x7f71500b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.048+0000 7f714ed9d700 1 -- 172.21.15.140:0/713598471 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7148009530 con 0x7f71500b14a0 2022-07-21T01:57:24.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f714ed9d700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f71500b14a0 0x7f71501107e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f71480032f0 tx=0x7f7148016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f71477fe700 1 -- 172.21.15.140:0/713598471 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f714800b8a0 con 0x7f71500b14a0 2022-07-21T01:57:24.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f71477fe700 1 -- 172.21.15.140:0/713598471 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7148021630 con 0x7f71500b14a0 2022-07-21T01:57:24.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f714f59e700 1 -- 172.21.15.140:0/713598471 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7150152360 con 0x7f71500b14a0 2022-07-21T01:57:24.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f71477fe700 1 -- 172.21.15.140:0/713598471 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7148021990 con 0x7f71500b14a0 2022-07-21T01:57:24.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f714f59e700 1 -- 172.21.15.140:0/713598471 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715010f100 con 0x7f71500b14a0 2022-07-21T01:57:24.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f71477fe700 1 -- 172.21.15.140:0/713598471 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f714800f070 con 0x7f71500b14a0 2022-07-21T01:57:24.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f71477fe700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7128053b00 0x7f7128055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f71477fe700 1 -- 172.21.15.140:0/713598471 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f714806d200 con 0x7f71500b14a0 2022-07-21T01:57:24.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f714e59c700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7128053b00 0x7f7128055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f714f59e700 1 -- 172.21.15.140:0/713598471 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7150143d10 con 0x7f71500b14a0 2022-07-21T01:57:24.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.052+0000 7f714e59c700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7128053b00 0x7f7128055fc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f714000d000 tx=0x7f7140007c80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.060+0000 7f71477fe700 1 -- 172.21.15.140:0/713598471 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7148015080 con 0x7f71500b14a0 2022-07-21T01:57:24.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.184+0000 7f714f59e700 1 -- 172.21.15.140:0/713598471 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7150067e80 con 0x7f71500b14a0 2022-07-21T01:57:24.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.184+0000 7f71477fe700 1 -- 172.21.15.140:0/713598471 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v132) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7148044120 con 0x7f71500b14a0 2022-07-21T01:57:24.188 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:24.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 -- 172.21.15.140:0/713598471 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7128053b00 msgr2=0x7f7128055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7128053b00 0x7f7128055fc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f714000d000 tx=0x7f7140007c80).stop 2022-07-21T01:57:24.193 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 -- 172.21.15.140:0/713598471 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f71500b14a0 msgr2=0x7f71501107e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f71500b14a0 0x7f71501107e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f71480032f0 tx=0x7f7148016000).stop 2022-07-21T01:57:24.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 -- 172.21.15.140:0/713598471 shutdown_connections 2022-07-21T01:57:24.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7128053b00 0x7f7128055fc0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f71500b14a0 0x7f71501107e0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71500b0da0 0x7f71500b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 --2- 172.21.15.140:0/713598471 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f71500af0b0 0x7f7150152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 -- 172.21.15.140:0/713598471 >> 172.21.15.140:0/713598471 conn(0x7f7150063200 msgr2=0x7f715013b630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:24.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 -- 172.21.15.140:0/713598471 shutdown_connections 2022-07-21T01:57:24.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.188+0000 7f714f59e700 1 -- 172.21.15.140:0/713598471 wait complete. 2022-07-21T01:57:24.206 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:57:24.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.308+0000 7fb63b3fa700 1 Processor -- start 2022-07-21T01:57:24.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 -- start start 2022-07-21T01:57:24.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb63405c920 0x7fb6341477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6340ae890 con 0x7fb63405c540 2022-07-21T01:57:24.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6340aeac0 con 0x7fb6340affe0 2022-07-21T01:57:24.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6340b2be0 con 0x7fb63405c920 2022-07-21T01:57:24.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb639196700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fb63405c540 0x7fb634140e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54794/0 (socket says 172.21.15.140:54794) 2022-07-21T01:57:24.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb639196700 1 -- 172.21.15.140:0/260969842 learned_addr learned my addr 172.21.15.140:0/260969842 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:24.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb639997700 1 --2- 172.21.15.140:0/260969842 >> v2:172.21.15.152:3300/0 conn(0x7fb63405c920 0x7fb6341477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb627fff700 1 -- 172.21.15.140:0/260969842 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063629875 0 0) 0x7fb6340ae890 con 0x7fb63405c540 2022-07-21T01:57:24.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb627fff700 1 -- 172.21.15.140:0/260969842 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb618003260 con 0x7fb63405c540 2022-07-21T01:57:24.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb639997700 1 -- 172.21.15.140:0/260969842 >> v1:172.21.15.152:6789/0 conn(0x7fb6340affe0 legacy=0x7fb6341470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb639997700 1 -- 172.21.15.140:0/260969842 >> v1:172.21.15.140:6789/0 conn(0x7fb63405c540 legacy=0x7fb634140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb639997700 1 -- 172.21.15.140:0/260969842 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6340aed60 con 0x7fb63405c920 2022-07-21T01:57:24.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb639997700 1 --2- 172.21.15.140:0/260969842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63405c920 0x7fb6341477b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fb62c0032f0 tx=0x7fb62c0091d0).ready entity=mon.1 client_cookie=f629fcb62477fd3f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb627fff700 1 -- 172.21.15.140:0/260969842 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62c00b6c0 con 0x7fb63405c920 2022-07-21T01:57:24.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb627fff700 1 -- 172.21.15.140:0/260969842 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c00b820 con 0x7fb63405c920 2022-07-21T01:57:24.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb627fff700 1 -- 172.21.15.140:0/260969842 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62c00b990 con 0x7fb63405c920 2022-07-21T01:57:24.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 -- 172.21.15.140:0/260969842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63405c920 msgr2=0x7fb6341477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 --2- 172.21.15.140:0/260969842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63405c920 0x7fb6341477b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7fb62c0032f0 tx=0x7fb62c0091d0).stop 2022-07-21T01:57:24.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 -- 172.21.15.140:0/260969842 shutdown_connections 2022-07-21T01:57:24.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 --2- 172.21.15.140:0/260969842 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63405c920 0x7fb6341477b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 -- 172.21.15.140:0/260969842 >> 172.21.15.140:0/260969842 conn(0x7fb634063460 msgr2=0x7fb63405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:24.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 -- 172.21.15.140:0/260969842 shutdown_connections 2022-07-21T01:57:24.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.312+0000 7fb63b3fa700 1 -- 172.21.15.140:0/260969842 wait complete. 2022-07-21T01:57:24.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb63b3fa700 1 Processor -- start 2022-07-21T01:57:24.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb63b3fa700 1 -- start start 2022-07-21T01:57:24.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb63b3fa700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb63405c540 0x7fb634156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb63b3fa700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63405c920 0x7fb634143a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb63b3fa700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6340affe0 0x7fb634067210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb63b3fa700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb634144e40 con 0x7fb6340affe0 2022-07-21T01:57:24.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb63b3fa700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb634147220 con 0x7fb63405c920 2022-07-21T01:57:24.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb63b3fa700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6341442b0 con 0x7fb63405c540 2022-07-21T01:57:24.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb639997700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6340affe0 0x7fb634067210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb639997700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6340affe0 0x7fb634067210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40656/0 (socket says 172.21.15.140:40656) 2022-07-21T01:57:24.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb639997700 1 -- 172.21.15.140:0/29989583 learned_addr learned my addr 172.21.15.140:0/29989583 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:24.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb638995700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63405c920 0x7fb634143a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb639196700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb63405c540 0x7fb634156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb639997700 1 -- 172.21.15.140:0/29989583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb63405c540 msgr2=0x7fb634156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb639997700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb63405c540 0x7fb634156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb639997700 1 -- 172.21.15.140:0/29989583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63405c920 msgr2=0x7fb634143a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb639997700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63405c920 0x7fb634143a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb639997700 1 -- 172.21.15.140:0/29989583 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c009470 con 0x7fb6340affe0 2022-07-21T01:57:24.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb638995700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63405c920 0x7fb634143a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:24.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb639997700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6340affe0 0x7fb634067210 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7fb62c0032f0 tx=0x7fb62c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb6267fc700 1 -- 172.21.15.140:0/29989583 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62c00b6c0 con 0x7fb6340affe0 2022-07-21T01:57:24.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb6267fc700 1 -- 172.21.15.140:0/29989583 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c0107a0 con 0x7fb6340affe0 2022-07-21T01:57:24.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb63b3fa700 1 -- 172.21.15.140:0/29989583 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb634145df0 con 0x7fb6340affe0 2022-07-21T01:57:24.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb6267fc700 1 -- 172.21.15.140:0/29989583 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb62c010b00 con 0x7fb6340affe0 2022-07-21T01:57:24.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.316+0000 7fb63b3fa700 1 -- 172.21.15.140:0/29989583 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb634146390 con 0x7fb6340affe0 2022-07-21T01:57:24.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.320+0000 7fb63b3fa700 1 -- 172.21.15.140:0/29989583 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb634144fb0 con 0x7fb6340affe0 2022-07-21T01:57:24.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.320+0000 7fb6267fc700 1 -- 172.21.15.140:0/29989583 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb62c027020 con 0x7fb6340affe0 2022-07-21T01:57:24.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.320+0000 7fb6267fc700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb61c053a70 0x7fb61c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.320+0000 7fb6267fc700 1 -- 172.21.15.140:0/29989583 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb62c06d420 con 0x7fb6340affe0 2022-07-21T01:57:24.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.320+0000 7fb639196700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb61c053a70 0x7fb61c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.320+0000 7fb639196700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb61c053a70 0x7fb61c055f30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fb63000b000 tx=0x7fb630009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.328+0000 7fb6267fc700 1 -- 172.21.15.140:0/29989583 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb634144fb0 con 0x7fb6340affe0 2022-07-21T01:57:24.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.452+0000 7fb63b3fa700 1 -- 172.21.15.140:0/29989583 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb634144ca0 con 0x7fb6340affe0 2022-07-21T01:57:24.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.452+0000 7fb6267fc700 1 -- 172.21.15.140:0/29989583 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v132) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb634144ca0 con 0x7fb6340affe0 2022-07-21T01:57:24.455 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:24.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 -- 172.21.15.140:0/29989583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb61c053a70 msgr2=0x7fb61c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb61c053a70 0x7fb61c055f30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fb63000b000 tx=0x7fb630009040).stop 2022-07-21T01:57:24.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 -- 172.21.15.140:0/29989583 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6340affe0 msgr2=0x7fb634067210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6340affe0 0x7fb634067210 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7fb62c0032f0 tx=0x7fb62c016000).stop 2022-07-21T01:57:24.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 -- 172.21.15.140:0/29989583 shutdown_connections 2022-07-21T01:57:24.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb61c053a70 0x7fb61c055f30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb6340affe0 0x7fb634067210 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb63405c920 0x7fb634143a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 --2- 172.21.15.140:0/29989583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb63405c540 0x7fb634156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 -- 172.21.15.140:0/29989583 >> 172.21.15.140:0/29989583 conn(0x7fb634063460 msgr2=0x7fb634149160 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:24.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 -- 172.21.15.140:0/29989583 shutdown_connections 2022-07-21T01:57:24.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.456+0000 7fb63b3fa700 1 -- 172.21.15.140:0/29989583 wait complete. 2022-07-21T01:57:24.467 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:57:24.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.572+0000 7f89a359e700 1 Processor -- start 2022-07-21T01:57:24.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.572+0000 7f89a359e700 1 -- start start 2022-07-21T01:57:24.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.572+0000 7f89a359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a4060f90 0x7f89a40b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.572+0000 7f89a359e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a41464b0 con 0x7f89a40b5ed0 2022-07-21T01:57:24.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.572+0000 7f89a359e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a40b67a0 con 0x7f89a41422b0 2022-07-21T01:57:24.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.572+0000 7f89a359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40b4140 con 0x7f89a4060f90 2022-07-21T01:57:24.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a259c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f89a40b5ed0 0x7f89a4146b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54806/0 (socket says 172.21.15.140:54806) 2022-07-21T01:57:24.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a259c700 1 -- 172.21.15.140:0/617787557 learned_addr learned my addr 172.21.15.140:0/617787557 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:24.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a1d9b700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a4060f90 0x7f89a40b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860168638 0 0) 0x7f89a41464b0 con 0x7f89a40b5ed0 2022-07-21T01:57:24.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8988003260 con 0x7f89a40b5ed0 2022-07-21T01:57:24.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182263863 0 0) 0x7f89a40b67a0 con 0x7f89a41422b0 2022-07-21T01:57:24.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89a41464b0 con 0x7f89a41422b0 2022-07-21T01:57:24.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1345703795 0 0) 0x7f8988003260 con 0x7f89a40b5ed0 2022-07-21T01:57:24.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a4060f90 msgr2=0x7f89a40b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:24.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a4060f90 0x7f89a40b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 >> v1:172.21.15.152:6789/0 conn(0x7f89a41422b0 legacy=0x7f89a4142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40aff60 con 0x7f89a40b5ed0 2022-07-21T01:57:24.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f8998001660 con 0x7f89a40b5ed0 2022-07-21T01:57:24.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a1d9b700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a4060f90 0x7f89a40b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:24.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 >> v1:172.21.15.140:6789/0 conn(0x7f89a40b5ed0 legacy=0x7f89a4146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a4060f90 0x7f8988003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8988003e20 0x7f8988004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89880049f0 0x7f8988004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40b4140 con 0x7f8988003e20 2022-07-21T01:57:24.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898800b800 con 0x7f89880049f0 2022-07-21T01:57:24.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898800b9c0 con 0x7f89a4060f90 2022-07-21T01:57:24.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a1d9b700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8988003e20 0x7f8988004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a259c700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a4060f90 0x7f8988003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a2d9d700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89880049f0 0x7f8988004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a1d9b700 1 -- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a4060f90 msgr2=0x7f8988003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:24.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a1d9b700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a4060f90 0x7f8988003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a1d9b700 1 -- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89880049f0 msgr2=0x7f8988004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:24.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a1d9b700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89880049f0 0x7f8988004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a1d9b700 1 -- 172.21.15.140:0/617787557 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899400b180 con 0x7f8988003e20 2022-07-21T01:57:24.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a1d9b700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8988003e20 0x7f8988004500 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f899400c000 tx=0x7f8994006df0).ready entity=mon.0 client_cookie=4e903ded8563c656 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f899400e830 con 0x7f8988003e20 2022-07-21T01:57:24.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899400e990 con 0x7f8988003e20 2022-07-21T01:57:24.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a159a700 1 -- 172.21.15.140:0/617787557 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f899400ecf0 con 0x7f8988003e20 2022-07-21T01:57:24.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a359e700 1 -- 172.21.15.140:0/617787557 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8988003e20 msgr2=0x7f8988004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a359e700 1 --2- 172.21.15.140:0/617787557 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8988003e20 0x7f8988004500 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f899400c000 tx=0x7f8994006df0).stop 2022-07-21T01:57:24.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a2d9d700 1 -- 172.21.15.140:0/617787557 reap_dead start 2022-07-21T01:57:24.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a359e700 1 -- 172.21.15.140:0/617787557 shutdown_connections 2022-07-21T01:57:24.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a359e700 1 -- 172.21.15.140:0/617787557 >> 172.21.15.140:0/617787557 conn(0x7f89a405be30 msgr2=0x7f89a405c210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:24.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a359e700 1 -- 172.21.15.140:0/617787557 shutdown_connections 2022-07-21T01:57:24.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.576+0000 7f89a359e700 1 -- 172.21.15.140:0/617787557 wait complete. 2022-07-21T01:57:24.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a359e700 1 Processor -- start 2022-07-21T01:57:24.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a359e700 1 -- start start 2022-07-21T01:57:24.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a359e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8988003e20 0x7f89a4152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89a409a240 0x7f89a409a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a409e480 0x7f89a409e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a359e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a409ab90 con 0x7f89a409a240 2022-07-21T01:57:24.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a359e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40b0970 con 0x7f8988003e20 2022-07-21T01:57:24.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40b0c10 con 0x7f89a409e480 2022-07-21T01:57:24.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a1d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89a409a240 0x7f89a409a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a1d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89a409a240 0x7f89a409a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40676/0 (socket says 172.21.15.140:40676) 2022-07-21T01:57:24.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a1d9b700 1 -- 172.21.15.140:0/3170959216 learned_addr learned my addr 172.21.15.140:0/3170959216 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:24.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a259c700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8988003e20 0x7f89a4152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a2d9d700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a409e480 0x7f89a409e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a1d9b700 1 -- 172.21.15.140:0/3170959216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a409e480 msgr2=0x7f89a409e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a1d9b700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a409e480 0x7f89a409e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a1d9b700 1 -- 172.21.15.140:0/3170959216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8988003e20 msgr2=0x7f89a4152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a1d9b700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8988003e20 0x7f89a4152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a1d9b700 1 -- 172.21.15.140:0/3170959216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40b0eb0 con 0x7f89a409a240 2022-07-21T01:57:24.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a1d9b700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89a409a240 0x7f89a409a6a0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f899400c000 tx=0x7f8994019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89937fe700 1 -- 172.21.15.140:0/3170959216 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f899401c9d0 con 0x7f89a409a240 2022-07-21T01:57:24.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89937fe700 1 -- 172.21.15.140:0/3170959216 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8994000e90 con 0x7f89a409a240 2022-07-21T01:57:24.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89937fe700 1 -- 172.21.15.140:0/3170959216 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f899401cd20 con 0x7f89a409a240 2022-07-21T01:57:24.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a359e700 1 -- 172.21.15.140:0/3170959216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a409edf0 con 0x7f89a409a240 2022-07-21T01:57:24.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.580+0000 7f89a359e700 1 -- 172.21.15.140:0/3170959216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a4155c90 con 0x7f89a409a240 2022-07-21T01:57:24.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.584+0000 7f89a359e700 1 -- 172.21.15.140:0/3170959216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a409af10 con 0x7f89a409a240 2022-07-21T01:57:24.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.584+0000 7f89937fe700 1 -- 172.21.15.140:0/3170959216 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f89940243f0 con 0x7f89a409a240 2022-07-21T01:57:24.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.584+0000 7f89937fe700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89800539d0 0x7f8980055e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.584+0000 7f89937fe700 1 -- 172.21.15.140:0/3170959216 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f899406d4d0 con 0x7f89a409a240 2022-07-21T01:57:24.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.584+0000 7f89a259c700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89800539d0 0x7f8980055e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.584+0000 7f89a259c700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89800539d0 0x7f8980055e90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f8998005000 tx=0x7f89a40b0460).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.592+0000 7f89937fe700 1 -- 172.21.15.140:0/3170959216 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8994047dc0 con 0x7f89a409a240 2022-07-21T01:57:24.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.712+0000 7f89a359e700 1 -- 172.21.15.140:0/3170959216 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89a40574e0 con 0x7f89a409a240 2022-07-21T01:57:24.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89937fe700 1 -- 172.21.15.140:0/3170959216 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v132) v1 ==== 100+0+10 (secure 0 0 0) 0x7f89940479b0 con 0x7f89a409a240 2022-07-21T01:57:24.717 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:24.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 -- 172.21.15.140:0/3170959216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89800539d0 msgr2=0x7f8980055e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89800539d0 0x7f8980055e90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f8998005000 tx=0x7f89a40b0460).stop 2022-07-21T01:57:24.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 -- 172.21.15.140:0/3170959216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89a409a240 msgr2=0x7f89a409a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89a409a240 0x7f89a409a6a0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f899400c000 tx=0x7f8994019000).stop 2022-07-21T01:57:24.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 -- 172.21.15.140:0/3170959216 shutdown_connections 2022-07-21T01:57:24.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f89800539d0 0x7f8980055e90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f89a409e480 0x7f89a409e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f89a409a240 0x7f89a409a6a0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 --2- 172.21.15.140:0/3170959216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8988003e20 0x7f89a4152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 -- 172.21.15.140:0/3170959216 >> 172.21.15.140:0/3170959216 conn(0x7f89a405be30 msgr2=0x7f89a405c210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:24.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 -- 172.21.15.140:0/3170959216 shutdown_connections 2022-07-21T01:57:24.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.716+0000 7f89a359e700 1 -- 172.21.15.140:0/3170959216 wait complete. 2022-07-21T01:57:24.730 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:57:24.730 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:24.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 Processor -- start 2022-07-21T01:57:24.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 -- start start 2022-07-21T01:57:24.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5aa405c880 0x7f5aa40ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5aa40afd00 0x7f5aa4147880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa40adc90 con 0x7f5aa40af150 2022-07-21T01:57:24.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa405cc60 con 0x7f5aa405c880 2022-07-21T01:57:24.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa40adec0 con 0x7f5aa40afd00 2022-07-21T01:57:24.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aa8b9c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5aa405c880 0x7f5aa40ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aa8b9c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5aa405c880 0x7f5aa40ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40680/0 (socket says 172.21.15.140:40680) 2022-07-21T01:57:24.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aa8b9c700 1 -- 172.21.15.140:0/399860980 learned_addr learned my addr 172.21.15.140:0/399860980 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:24.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aa939d700 1 --2- 172.21.15.140:0/399860980 >> v2:172.21.15.152:3300/0 conn(0x7f5aa40afd00 0x7f5aa4147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aa8b9c700 1 -- 172.21.15.140:0/399860980 >> v2:172.21.15.152:3300/0 conn(0x7f5aa40afd00 msgr2=0x7f5aa4147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:24.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aa8b9c700 1 --2- 172.21.15.140:0/399860980 >> v2:172.21.15.152:3300/0 conn(0x7f5aa40afd00 0x7f5aa4147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aa8b9c700 1 -- 172.21.15.140:0/399860980 >> v1:172.21.15.152:6789/0 conn(0x7f5aa40af150 legacy=0x7f5aa4147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5a9b7fe700 1 -- 172.21.15.140:0/399860980 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859740857 0 0) 0x7f5aa40adc90 con 0x7f5aa40af150 2022-07-21T01:57:24.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aa8b9c700 1 -- 172.21.15.140:0/399860980 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa40aeaf0 con 0x7f5aa405c880 2022-07-21T01:57:24.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aa8b9c700 1 --2- 172.21.15.140:0/399860980 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5aa405c880 0x7f5aa40ae600 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f5aa40ad630 tx=0x7f5aa0006eb0).ready entity=mon.0 client_cookie=bcd4b0ae6be72cd3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5a9b7fe700 1 -- 172.21.15.140:0/399860980 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5aa000d610 con 0x7f5aa405c880 2022-07-21T01:57:24.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5a9b7fe700 1 -- 172.21.15.140:0/399860980 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa0000e90 con 0x7f5aa405c880 2022-07-21T01:57:24.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5a9b7fe700 1 -- 172.21.15.140:0/399860980 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5aa000d960 con 0x7f5aa405c880 2022-07-21T01:57:24.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 -- 172.21.15.140:0/399860980 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5aa405c880 msgr2=0x7f5aa40ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 --2- 172.21.15.140:0/399860980 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5aa405c880 0x7f5aa40ae600 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f5aa40ad630 tx=0x7f5aa0006eb0).stop 2022-07-21T01:57:24.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 -- 172.21.15.140:0/399860980 shutdown_connections 2022-07-21T01:57:24.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 --2- 172.21.15.140:0/399860980 >> v2:172.21.15.152:3300/0 conn(0x7f5aa40afd00 0x7f5aa4147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 --2- 172.21.15.140:0/399860980 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5aa405c880 0x7f5aa40ae600 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 -- 172.21.15.140:0/399860980 >> 172.21.15.140:0/399860980 conn(0x7f5aa4063460 msgr2=0x7f5aa405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:24.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 -- 172.21.15.140:0/399860980 shutdown_connections 2022-07-21T01:57:24.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.832+0000 7f5aaae00700 1 -- 172.21.15.140:0/399860980 wait complete. 2022-07-21T01:57:24.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aaae00700 1 Processor -- start 2022-07-21T01:57:24.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aaae00700 1 -- start start 2022-07-21T01:57:24.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aaae00700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5aa405c880 0x7f5aa4143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aaae00700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa40af150 0x7f5aa40b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aaae00700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5aa40afd00 0x7f5aa4141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aaae00700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41459a0 con 0x7f5aa40afd00 2022-07-21T01:57:24.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aaae00700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa4144370 con 0x7f5aa40af150 2022-07-21T01:57:24.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aaae00700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa4144610 con 0x7f5aa405c880 2022-07-21T01:57:24.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aa939d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5aa40afd00 0x7f5aa4141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aa939d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5aa40afd00 0x7f5aa4141c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40690/0 (socket says 172.21.15.140:40690) 2022-07-21T01:57:24.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aa939d700 1 -- 172.21.15.140:0/1865341916 learned_addr learned my addr 172.21.15.140:0/1865341916 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:24.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5a9bfff700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa40af150 0x7f5aa40b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aa8b9c700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5aa405c880 0x7f5aa4143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5a9bfff700 1 -- 172.21.15.140:0/1865341916 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5aa405c880 msgr2=0x7f5aa4143d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5a9bfff700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5aa405c880 0x7f5aa4143d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5a9bfff700 1 -- 172.21.15.140:0/1865341916 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5aa40afd00 msgr2=0x7f5aa4141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:24.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5a9bfff700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5aa40afd00 0x7f5aa4141c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:24.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5a9bfff700 1 -- 172.21.15.140:0/1865341916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa000b1e0 con 0x7f5aa40af150 2022-07-21T01:57:24.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aa939d700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5aa40afd00 0x7f5aa4141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:24.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aa8b9c700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5aa405c880 0x7f5aa4143d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:24.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5a9bfff700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa40af150 0x7f5aa40b32c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f5aa4156530 tx=0x7f5a94007a00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5a99ffb700 1 -- 172.21.15.140:0/1865341916 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a94012400 con 0x7f5aa40af150 2022-07-21T01:57:24.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5a99ffb700 1 -- 172.21.15.140:0/1865341916 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a94012560 con 0x7f5aa40af150 2022-07-21T01:57:24.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5aaae00700 1 -- 172.21.15.140:0/1865341916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa4156610 con 0x7f5aa40af150 2022-07-21T01:57:24.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.836+0000 7f5a99ffb700 1 -- 172.21.15.140:0/1865341916 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a940128c0 con 0x7f5aa40af150 2022-07-21T01:57:24.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.840+0000 7f5aaae00700 1 -- 172.21.15.140:0/1865341916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa414c720 con 0x7f5aa40af150 2022-07-21T01:57:24.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.840+0000 7f5aaae00700 1 -- 172.21.15.140:0/1865341916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa40624c0 con 0x7f5aa40af150 2022-07-21T01:57:24.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.840+0000 7f5a99ffb700 1 -- 172.21.15.140:0/1865341916 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5a94019030 con 0x7f5aa40af150 2022-07-21T01:57:24.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.840+0000 7f5a99ffb700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a84053a70 0x7f5a84055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:24.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.840+0000 7f5a99ffb700 1 -- 172.21.15.140:0/1865341916 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5a9406b190 con 0x7f5aa40af150 2022-07-21T01:57:24.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.840+0000 7f5aa8b9c700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a84053a70 0x7f5a84055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:24.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.844+0000 7f5aa8b9c700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a84053a70 0x7f5a84055f30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f5aa001c000 tx=0x7f5aa0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:24.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:24.848+0000 7f5a99ffb700 1 -- 172.21.15.140:0/1865341916 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5a94041ff0 con 0x7f5aa40af150 2022-07-21T01:57:25.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.012+0000 7f5aaae00700 1 -- 172.21.15.140:0/1865341916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5aa40af7f0 con 0x7f5aa40af150 2022-07-21T01:57:25.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.012+0000 7f5a99ffb700 1 -- 172.21.15.140:0/1865341916 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+2145 (secure 0 0 0) 0x7f5a94045610 con 0x7f5aa40af150 2022-07-21T01:57:25.015 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:25.015 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":64,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/558475048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":558475048},{"type":"v1","addr":"172.21.15.152:6833","nonce":558475048}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:14.423640+0000","modified":"2022-07-21T01:57:16.123489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2209871450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2209871450},{"type":"v1","addr":"172.21.15.140:6835","nonce":2209871450}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-07-21T01:57:25.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 -- 172.21.15.140:0/1865341916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a84053a70 msgr2=0x7f5a84055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:25.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a84053a70 0x7f5a84055f30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f5aa001c000 tx=0x7f5aa0015000).stop 2022-07-21T01:57:25.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 -- 172.21.15.140:0/1865341916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa40af150 msgr2=0x7f5aa40b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:25.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa40af150 0x7f5aa40b32c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f5aa4156530 tx=0x7f5a94007a00).stop 2022-07-21T01:57:25.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 -- 172.21.15.140:0/1865341916 shutdown_connections 2022-07-21T01:57:25.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5a84053a70 0x7f5a84055f30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:25.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5aa40afd00 0x7f5aa4141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:25.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5aa40af150 0x7f5aa40b32c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:25.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 --2- 172.21.15.140:0/1865341916 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5aa405c880 0x7f5aa4143d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:25.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 -- 172.21.15.140:0/1865341916 >> 172.21.15.140:0/1865341916 conn(0x7f5aa4063460 msgr2=0x7f5aa4147170 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:25.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 -- 172.21.15.140:0/1865341916 shutdown_connections 2022-07-21T01:57:25.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:25.016+0000 7f5aaae00700 1 -- 172.21.15.140:0/1865341916 wait complete. 2022-07-21T01:57:25.021 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 64 2022-07-21T01:57:25.234 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:57:14.423640+0000', 'modified': '2022-07-21T01:57:16.123489+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8061}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8061': {'gid': 8061, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/2209871450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 2209871450}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 2209871450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:57:25.235 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:57:25.235 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:57:25.351 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":8061,"want_state":"up:active","state":"up:active","rank_uptime":10.916575313999999,"mdsmap_epoch":64,"osdmap_epoch":132,"osdmap_epoch_barrier":132,"uptime":15.610551923999999} 2022-07-21T01:57:25.351 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 8061, "mdsmap_epoch": 64, "osdmap_epoch": 132, "osdmap_epoch_barrier": 132, "rank_uptime": 10.916575314, "state": "up:active", "uptime": 15.610551924, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:57:25.352 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:57:25.352 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-07-21T01:57:25.353 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-07-21T01:57:25.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:25.354 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:25.362 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:57:25.379 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:25.380 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:25.396 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:25.397 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:25.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:25.398 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:57:25.447 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:57:25.448 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:25.450 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:57:25.468 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:25.470 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:25.480 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:25.481 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:25.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:25.483 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:57:25.522 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2022-07-21T01:57:25.520+0000 7fa895f42080 -1 init, newargv = 0x56483f6c1390 newargc=9ceph-fuse[22900]: starting ceph client 2022-07-21T01:57:25.522 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr: 2022-07-21T01:57:25.531 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22900]: starting fuse 2022-07-21T01:57:26.630 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:57:26.649 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:26.649 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:26.708 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:26.710 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:26.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:26.712 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:57:26.720 INFO:teuthology.orchestra.run.smithi192.stdout:52 2022-07-21T01:57:26.721 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:57:26.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:26.722 DEBUG:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2022-07-21T01:57:26.722 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2022-07-21T01:57:26.723 DEBUG:teuthology.orchestra.run.smithi192:> import re 2022-07-21T01:57:26.723 DEBUG:teuthology.orchestra.run.smithi192:> import os 2022-07-21T01:57:26.724 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2022-07-21T01:57:26.724 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:26.725 DEBUG:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2022-07-21T01:57:26.725 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-07-21T01:57:26.726 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2022-07-21T01:57:26.726 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:26.726 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2022-07-21T01:57:26.727 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2022-07-21T01:57:26.727 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2022-07-21T01:57:26.728 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2022-07-21T01:57:26.728 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:26.728 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2022-07-21T01:57:26.729 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:57:26.729 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:57:26.730 DEBUG:teuthology.orchestra.run.smithi192:> return f 2022-07-21T01:57:26.730 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:57:26.731 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:26.731 DEBUG:teuthology.orchestra.run.smithi192:> print(find_socket("client.1")) 2022-07-21T01:57:26.732 DEBUG:teuthology.orchestra.run.smithi192:> ' 2022-07-21T01:57:26.775 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.22900.asok 2022-07-21T01:57:26.780 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22900.asok 2022-07-21T01:57:26.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:26.781 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22900.asok status 2022-07-21T01:57:26.918 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2022-07-21T01:57:26.919 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2022-07-21T01:57:26.919 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:57:26.919 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:57:26.919 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2022-07-21T01:57:26.920 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2022-07-21T01:57:26.920 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-07-21T01:57:26.920 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "22900", 2022-07-21T01:57:26.920 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2022-07-21T01:57:26.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:26.921 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2022-07-21T01:57:26.921 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2022-07-21T01:57:26.921 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 8322, 2022-07-21T01:57:26.922 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2022-07-21T01:57:26.922 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2022-07-21T01:57:26.922 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2022-07-21T01:57:26.922 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 8322 2022-07-21T01:57:26.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:26.923 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:57:26.923 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:57:26.923 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:57:26.924 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3748086433 2022-07-21T01:57:26.924 INFO:teuthology.orchestra.run.smithi192.stdout: } 2022-07-21T01:57:26.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:26.924 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:57:26.925 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:57:26.925 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:57:26.925 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3748086433 2022-07-21T01:57:26.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:26.925 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.8322 172.21.15.192:0/3748086433", 2022-07-21T01:57:26.926 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/3748086433", 2022-07-21T01:57:26.926 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2022-07-21T01:57:26.926 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 64, 2022-07-21T01:57:26.927 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 132, 2022-07-21T01:57:26.927 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:57:26.927 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false, 2022-07-21T01:57:26.927 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2022-07-21T01:57:26.927 INFO:teuthology.orchestra.run.smithi192.stdout:} 2022-07-21T01:57:26.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:26.929 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:26.930 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:26.937 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:57:26.939 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:57:26.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:26.940 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:26.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:26.958 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:26.959 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:26.967 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:57:26.968 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:57:26.969 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2022-07-21T01:57:26.969 DEBUG:teuthology.orchestra.run.smithi191:> import time 2022-07-21T01:57:26.969 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:57:26.970 DEBUG:teuthology.orchestra.run.smithi191:> with open("/home/ubuntu/cephtest/mnt.0/testfile", '"'"'w'"'"') as f: 2022-07-21T01:57:26.970 DEBUG:teuthology.orchestra.run.smithi191:> f.write('"'"'content'"'"') 2022-07-21T01:57:26.971 DEBUG:teuthology.orchestra.run.smithi191:> f.flush() 2022-07-21T01:57:26.971 DEBUG:teuthology.orchestra.run.smithi191:> f.write('"'"'content2'"'"') 2022-07-21T01:57:26.972 DEBUG:teuthology.orchestra.run.smithi191:> while True: 2022-07-21T01:57:26.972 DEBUG:teuthology.orchestra.run.smithi191:> time.sleep(1) 2022-07-21T01:57:26.972 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:57:27.014 DEBUG:teuthology.orchestra.run.smithi191:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2022-07-21T01:57:27.034 INFO:teuthology.orchestra.run.smithi191.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/testfile': No such file or directory 2022-07-21T01:57:27.035 DEBUG:teuthology.orchestra.run:got remote process result: 2 2022-07-21T01:57:28.037 DEBUG:teuthology.orchestra.run.smithi191:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2022-07-21T01:57:28.054 INFO:teuthology.orchestra.run.smithi191.stdout:/home/ubuntu/cephtest/mnt.0/testfile 2022-07-21T01:57:28.055 DEBUG:tasks.cephfs.mount:File testfile became visible from 0 after 1s 2022-07-21T01:57:28.056 DEBUG:teuthology.orchestra.run.smithi192:> sudo ls /home/ubuntu/cephtest/mnt.1/testfile 2022-07-21T01:57:28.094 INFO:teuthology.orchestra.run.smithi192.stdout:/home/ubuntu/cephtest/mnt.1/testfile 2022-07-21T01:57:28.096 DEBUG:tasks.cephfs.mount:File testfile became visible from 1 after 0s 2022-07-21T01:57:28.096 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-07-21T01:57:28.097 DEBUG:teuthology.orchestra.run.smithi192:> cd /home/ubuntu/cephtest/mnt.1 && sudo cat testfile 2022-07-21T01:57:28.143 INFO:teuthology.orchestra.run.smithi192.stdout:content 2022-07-21T01:57:28.158 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:57:28.159 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:28.181 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:28.200 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:28.222 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:28.243 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:28.264 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:57:28.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:28.265 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:28.266 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:28.274 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:57:28.275 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:57:28.276 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:57:28.276 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:57:28.277 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:28.291 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23325]: fuse finished with error 0 and tester_r 0 2022-07-21T01:57:28.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:28.294 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:28.295 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:28.342 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:57:28.343 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:57:28.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:57:28.395 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:28.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:28.404 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:28.404 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:28.412 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:57:28.414 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:57:28.414 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2022-07-21T01:57:28.415 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:57:28.415 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:28.430 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22900]: fuse finished with error 0 and tester_r 0 2022-07-21T01:57:28.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:28.434 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:28.435 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:28.442 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2022-07-21T01:57:28.444 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:57:28.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:57:28.522 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:28.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:28.531 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:28.531 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:57:28.539 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:57:28.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:28.540 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:57:28.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:57:28.542 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:57:28.548 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:57:28.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:28.550 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:57:28.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1f59e700 1 Processor -- start 2022-07-21T01:57:28.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1f59e700 1 -- start start 2022-07-21T01:57:28.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 0x7f9d200ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:28.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1f59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d200afbd0 0x7f9d200ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:28.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1f59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d200b5650 con 0x7f9d2014a530 2022-07-21T01:57:28.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1f59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d200ad9b0 con 0x7f9d200afbd0 2022-07-21T01:57:28.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d200adc50 con 0x7f9d200b5270 2022-07-21T01:57:28.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1e59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 0x7f9d200ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:28.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1dd9b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d200afbd0 0x7f9d200ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:28.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1ed9d700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9d2014a530 0x7f9d2014a960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:55958/0 (socket says 172.21.15.140:55958) 2022-07-21T01:57:28.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1ed9d700 1 -- 172.21.15.140:0/1989581834 learned_addr learned my addr 172.21.15.140:0/1989581834 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:28.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1dd9b700 1 -- 172.21.15.140:0/1989581834 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 msgr2=0x7f9d200ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:28.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1dd9b700 1 --2- 172.21.15.140:0/1989581834 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 0x7f9d200ae390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:28.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1dd9b700 1 -- 172.21.15.140:0/1989581834 >> v1:172.21.15.152:6789/0 conn(0x7f9d2014a530 legacy=0x7f9d2014a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:28.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1dd9b700 1 -- 172.21.15.140:0/1989581834 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d200adef0 con 0x7f9d200afbd0 2022-07-21T01:57:28.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1e59c700 1 --2- 172.21.15.140:0/1989581834 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 0x7f9d200ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:28.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.660+0000 7f9d1dd9b700 1 --2- 172.21.15.140:0/1989581834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d200afbd0 0x7f9d200ac590 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f9d100032f0 tx=0x7f9d100091d0).ready entity=mon.1 client_cookie=e7bed17fa2608d34 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:28.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.664+0000 7f9d1d59a700 1 -- 172.21.15.140:0/1989581834 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d1000b6c0 con 0x7f9d200afbd0 2022-07-21T01:57:28.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.664+0000 7f9d1d59a700 1 -- 172.21.15.140:0/1989581834 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1000b820 con 0x7f9d200afbd0 2022-07-21T01:57:28.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.664+0000 7f9d1d59a700 1 -- 172.21.15.140:0/1989581834 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d1000b990 con 0x7f9d200afbd0 2022-07-21T01:57:28.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.664+0000 7f9d1f59e700 1 -- 172.21.15.140:0/1989581834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d200afbd0 msgr2=0x7f9d200ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:28.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.664+0000 7f9d1f59e700 1 --2- 172.21.15.140:0/1989581834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d200afbd0 0x7f9d200ac590 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f9d100032f0 tx=0x7f9d100091d0).stop 2022-07-21T01:57:28.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.664+0000 7f9d1f59e700 1 -- 172.21.15.140:0/1989581834 shutdown_connections 2022-07-21T01:57:28.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.664+0000 7f9d1f59e700 1 --2- 172.21.15.140:0/1989581834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d200afbd0 0x7f9d200ac590 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:28.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.664+0000 7f9d1f59e700 1 --2- 172.21.15.140:0/1989581834 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 0x7f9d200ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:28.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.664+0000 7f9d1f59e700 1 -- 172.21.15.140:0/1989581834 >> 172.21.15.140:0/1989581834 conn(0x7f9d20062940 msgr2=0x7f9d20062d20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:28.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.664+0000 7f9d1f59e700 1 -- 172.21.15.140:0/1989581834 shutdown_connections 2022-07-21T01:57:28.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.664+0000 7f9d1f59e700 1 -- 172.21.15.140:0/1989581834 wait complete. 2022-07-21T01:57:28.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1f59e700 1 Processor -- start 2022-07-21T01:57:28.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1f59e700 1 -- start start 2022-07-21T01:57:28.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1f59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d200afbd0 0x7f9d200669d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:28.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1f59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 0x7f9d200aae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:28.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1f59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d2014a530 0x7f9d200ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:28.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1f59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d20067a00 con 0x7f9d200afbd0 2022-07-21T01:57:28.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1f59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d200a96f0 con 0x7f9d2014a530 2022-07-21T01:57:28.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1f59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d200a9990 con 0x7f9d200b5270 2022-07-21T01:57:28.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d200afbd0 0x7f9d200669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:28.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1dd9b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 0x7f9d200aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:28.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d200afbd0 0x7f9d200669d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40700/0 (socket says 172.21.15.140:40700) 2022-07-21T01:57:28.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1e59c700 1 -- 172.21.15.140:0/3105560181 learned_addr learned my addr 172.21.15.140:0/3105560181 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:28.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1ed9d700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d2014a530 0x7f9d200ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:28.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1e59c700 1 -- 172.21.15.140:0/3105560181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 msgr2=0x7f9d200aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:28.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1e59c700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 0x7f9d200aae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:28.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1e59c700 1 -- 172.21.15.140:0/3105560181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d2014a530 msgr2=0x7f9d200ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:28.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1e59c700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d2014a530 0x7f9d200ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:28.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1e59c700 1 -- 172.21.15.140:0/3105560181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1400b180 con 0x7f9d200afbd0 2022-07-21T01:57:28.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1dd9b700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 0x7f9d200aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:28.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1e59c700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d200afbd0 0x7f9d200669d0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f9d1400c000 tx=0x7f9d14006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:28.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d0f7fe700 1 -- 172.21.15.140:0/3105560181 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d1400c5f0 con 0x7f9d200afbd0 2022-07-21T01:57:28.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d0f7fe700 1 -- 172.21.15.140:0/3105560181 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1400c750 con 0x7f9d200afbd0 2022-07-21T01:57:28.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1f59e700 1 -- 172.21.15.140:0/3105560181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d10009470 con 0x7f9d200afbd0 2022-07-21T01:57:28.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d0f7fe700 1 -- 172.21.15.140:0/3105560181 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d1400cab0 con 0x7f9d200afbd0 2022-07-21T01:57:28.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.668+0000 7f9d1f59e700 1 -- 172.21.15.140:0/3105560181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d200676f0 con 0x7f9d200afbd0 2022-07-21T01:57:28.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.672+0000 7f9d1f59e700 1 -- 172.21.15.140:0/3105560181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d200574e0 con 0x7f9d200afbd0 2022-07-21T01:57:28.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.672+0000 7f9d0f7fe700 1 -- 172.21.15.140:0/3105560181 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9d1400cc10 con 0x7f9d200afbd0 2022-07-21T01:57:28.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.672+0000 7f9d0f7fe700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9cfc053b30 0x7f9cfc055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:28.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.672+0000 7f9d0f7fe700 1 -- 172.21.15.140:0/3105560181 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9d1406c4b0 con 0x7f9d200afbd0 2022-07-21T01:57:28.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.672+0000 7f9d1dd9b700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9cfc053b30 0x7f9cfc055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:28.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.672+0000 7f9d1dd9b700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9cfc053b30 0x7f9cfc055ff0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f9d1001b000 tx=0x7f9d10016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:28.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.680+0000 7f9d0f7fe700 1 -- 172.21.15.140:0/3105560181 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9d140432c0 con 0x7f9d200afbd0 2022-07-21T01:57:28.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.800+0000 7f9d1f59e700 1 -- 172.21.15.140:0/3105560181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9d20068a70 con 0x7f9d200afbd0 2022-07-21T01:57:28.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d0f7fe700 1 -- 172.21.15.140:0/3105560181 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v132) v1 ==== 81+0+33755 (secure 0 0 0) 0x7f9d140468e0 con 0x7f9d200afbd0 2022-07-21T01:57:28.806 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:28.807 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:57:28.807 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 132, 2022-07-21T01:57:28.807 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:57:28.807 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:57:28.808 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:57:15.426209+0000", 2022-07-21T01:57:28.808 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:28.808 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:28.808 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:57:28.808 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:57:28.809 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:57:28.809 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:57:28.809 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:57:28.809 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:57:28.810 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:57:28.810 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:28.810 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:57:28.810 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:57:28.811 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:57:28.811 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:57:28.811 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:57:28.811 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 26, 2022-07-21T01:57:28.811 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:57:28.812 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:57:28.812 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:57:28.812 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:57:28.812 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:57:28.813 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.813 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:57:28.813 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:57:28.813 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:57:28.814 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:28.814 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:28.814 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:28.814 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:28.815 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:28.815 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:28.815 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:28.815 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:28.815 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:57:28.816 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:57:28.816 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:57:28.816 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:57:28.816 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:57:28.817 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:28.817 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:28.817 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:28.817 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:28.818 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:28.818 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:28.818 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:28.818 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.818 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:57:28.819 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:28.819 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:28.819 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:28.819 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:28.820 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:28.820 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:28.820 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:28.820 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:28.821 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:28.821 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:28.821 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:28.821 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:28.822 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:28.822 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:28.822 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:28.822 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:28.823 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:28.823 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:28.823 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:28.823 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:28.824 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:28.824 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:28.824 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:28.824 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:28.825 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:28.825 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:28.825 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.825 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:28.825 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:28.826 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:28.826 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:28.826 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:28.827 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:28.827 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:28.828 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:28.828 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:28.828 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:28.828 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:28.828 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:57:28.829 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:57:28.829 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.829 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:28.829 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:57:28.830 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.830 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.830 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.830 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:57:28.831 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:57:28.831 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:57:28.831 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:28.831 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:28.832 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:28.832 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:28.832 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:28.832 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:28.832 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:28.833 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:28.833 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:28.833 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:28.833 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:28.834 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:28.834 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:28.834 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:28.834 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:28.835 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:28.835 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:28.835 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:28.835 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:28.835 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:28.836 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.836 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:57:28.836 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:28.836 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:28.837 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:28.837 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:28.837 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:28.837 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:28.838 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:28.838 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:28.838 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:28.838 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:28.839 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:28.839 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:28.839 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:28.839 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:28.839 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:28.840 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:28.840 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:28.840 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:28.840 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:28.841 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:28.841 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:28.841 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:28.841 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:28.841 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:28.842 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:28.842 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:28.842 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.842 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:28.843 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:28.843 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:28.843 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:28.843 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:28.844 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:28.844 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:28.844 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:28.844 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:28.845 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:28.845 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:28.845 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:57:28.845 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:28.845 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:57:28.846 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.846 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.846 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.846 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 25, 2022-07-21T01:57:28.847 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:57:28.847 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:57:12.443397+0000", 2022-07-21T01:57:28.847 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:28.847 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:28.847 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:28.848 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:28.848 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:28.848 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:28.848 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:28.849 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:28.849 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:28.849 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:28.849 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:28.849 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:28.850 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:28.850 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:28.850 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:28.850 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:28.851 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:28.851 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:28.851 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:28.851 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:28.852 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.852 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "131", 2022-07-21T01:57:28.852 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:28.852 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:28.852 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:28.853 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:28.853 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:28.853 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:28.853 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:28.853 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:28.854 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:28.854 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:28.854 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:28.854 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:28.855 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:28.855 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:28.855 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:28.855 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:28.856 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:28.856 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:28.856 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:28.856 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:28.857 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:28.857 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:28.857 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:28.857 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:28.858 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:28.858 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:28.858 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.858 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:28.859 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:28.859 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:28.859 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:28.859 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:28.859 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:28.860 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:28.860 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:28.860 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:28.860 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:28.861 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:28.861 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:57:28.861 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:57:28.861 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:57:28.862 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:57:28.862 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.862 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:28.862 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:57:28.863 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:57:28.863 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.863 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.863 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.864 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.864 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 26, 2022-07-21T01:57:28.864 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:57:28.864 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:57:13.427432+0000", 2022-07-21T01:57:28.864 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:28.865 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:28.865 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:28.865 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:28.866 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:28.866 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:28.866 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:28.867 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:28.867 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:28.867 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:28.867 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:28.868 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:28.868 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:28.868 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:28.868 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:28.868 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:28.869 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:28.869 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:28.869 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:28.869 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:28.870 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.870 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "132", 2022-07-21T01:57:28.870 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:28.870 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:28.871 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:28.871 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:28.871 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:28.871 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:28.871 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:28.872 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:28.872 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:28.872 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:28.872 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:28.873 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:28.873 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:28.873 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:28.873 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:28.874 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:28.874 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:28.874 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:28.874 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:28.874 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:28.875 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:28.875 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:28.875 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:28.875 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:28.876 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:28.876 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:28.876 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.876 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:28.877 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:28.877 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:28.877 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:28.877 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:28.877 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:28.878 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:28.878 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:28.878 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:28.878 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:28.879 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:28.879 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:57:28.879 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:28.879 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:57:28.880 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:57:28.880 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.880 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.880 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.880 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:28.881 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:57:28.881 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.881 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:57:28.881 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:57:28.882 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:28.882 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:28.882 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:28.882 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:28.883 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:28.883 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:28.883 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:28.883 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 129, 2022-07-21T01:57:28.883 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:28.884 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:28.884 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:28.884 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.884 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.885 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.885 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:57:28.885 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:28.885 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.886 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.886 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.886 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:57:28.886 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:28.886 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.887 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.887 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.887 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:28.887 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.888 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.888 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.888 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:57:28.888 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:28.888 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.889 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.889 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.889 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:57:28.889 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:28.890 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.890 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.890 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.890 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:28.890 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.891 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.891 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.891 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:57:28.891 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:28.892 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.892 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.892 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.892 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:57:28.892 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:28.893 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.893 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.893 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.893 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:28.894 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.894 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.894 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.894 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:57:28.894 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:28.895 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.895 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.895 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.895 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:57:28.896 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:28.896 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.896 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.896 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.896 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:57:28.897 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:57:28.897 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:57:28.897 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:57:28.897 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:28.898 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:28.898 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:28.898 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.898 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.899 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.899 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:57:28.899 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:57:28.899 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:28.899 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:28.900 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:28.900 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:28.900 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:28.900 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:28.900 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:28.901 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 130, 2022-07-21T01:57:28.901 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:28.901 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:28.901 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:28.902 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.902 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.902 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.902 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:57:28.903 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:28.903 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.903 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.903 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.903 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:57:28.904 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:28.904 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.904 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.904 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.905 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:28.905 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.905 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.905 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.906 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:57:28.906 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:28.906 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.906 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.907 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.907 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:57:28.907 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:28.907 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.908 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.908 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.908 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:28.908 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.909 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.909 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.909 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:57:28.909 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:28.909 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.910 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.910 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.910 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:57:28.910 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:28.911 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.911 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.911 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.911 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:28.911 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.912 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.912 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.912 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:57:28.912 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:28.913 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.913 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.913 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.913 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:57:28.913 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:28.914 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.914 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.914 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.914 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:57:28.915 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:57:28.915 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:57:28.915 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:57:28.915 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:28.916 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:28.916 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:28.916 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.916 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.917 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.917 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:57:28.917 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:57:28.918 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:28.918 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:28.918 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:28.918 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:28.918 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:28.919 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:28.919 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:28.919 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 130, 2022-07-21T01:57:28.919 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:28.920 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:28.920 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:28.920 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.920 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.921 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.922 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:57:28.922 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:28.922 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.922 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.923 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.923 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:57:28.923 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:28.924 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.924 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.924 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.924 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:28.925 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.925 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.925 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.926 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:57:28.926 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:28.926 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.927 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.927 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.927 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:57:28.928 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:28.928 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.928 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.928 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.929 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:28.929 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.929 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.929 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.930 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:57:28.930 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:28.930 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.930 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.931 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.931 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:57:28.931 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:28.932 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.932 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.932 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.932 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:28.933 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.933 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.933 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.933 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:57:28.934 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:28.934 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.934 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.934 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.935 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:57:28.935 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:28.935 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.935 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.936 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.936 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:57:28.936 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:57:28.936 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:57:28.937 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:57:28.937 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:28.937 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:28.938 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:28.938 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.938 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.939 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.939 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:57:28.939 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:57:28.939 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:28.940 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:28.940 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:28.940 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:28.940 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:28.941 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:28.941 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:28.941 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 129, 2022-07-21T01:57:28.941 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:28.942 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:28.942 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:28.942 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.943 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.943 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.943 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:57:28.944 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:28.944 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.945 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.945 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.946 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:57:28.946 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:28.947 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.947 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.947 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.948 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:28.948 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.948 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.949 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.949 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:57:28.950 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:28.950 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.950 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.951 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.951 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:57:28.951 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:28.952 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.952 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.952 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.952 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:28.952 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.953 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.953 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.953 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:57:28.953 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:28.954 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.954 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.954 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.955 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:57:28.955 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:28.955 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.955 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.956 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.956 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:28.956 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.957 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.957 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.958 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:57:28.958 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:28.958 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.959 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.959 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.959 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:57:28.960 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:28.960 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.960 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.960 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.961 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:57:28.961 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:57:28.961 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:57:28.961 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:57:28.961 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:28.962 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:28.962 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:28.962 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.962 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.963 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.963 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:57:28.963 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:57:28.963 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:28.964 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:28.964 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:28.964 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:28.964 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:28.964 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:28.965 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:28.965 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 130, 2022-07-21T01:57:28.965 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:28.965 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:28.966 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:28.966 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.966 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.966 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.966 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:57:28.967 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:28.967 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.967 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.967 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.968 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:57:28.968 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:28.968 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.968 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.969 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.969 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:28.969 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.969 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.969 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.970 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:57:28.970 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:28.970 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.970 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.971 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.971 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:57:28.971 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:28.971 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.972 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.972 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.972 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:28.972 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.972 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.973 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.973 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:57:28.973 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:28.973 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.974 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.974 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.974 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:57:28.974 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:28.975 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.975 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.975 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.975 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:28.976 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.976 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.976 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.976 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:57:28.976 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:28.977 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.977 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.977 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.977 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:57:28.978 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:28.978 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.978 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.978 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.978 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:57:28.979 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:57:28.979 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:57:28.979 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:57:28.979 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:28.980 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:28.980 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:28.980 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.980 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.981 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.981 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:57:28.981 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:57:28.981 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:28.982 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:28.982 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:28.982 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:28.982 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:28.983 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:28.983 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:28.983 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:28.983 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:28.984 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:28.984 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:28.984 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.984 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.984 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.985 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:57:28.985 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:28.985 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.985 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.986 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.986 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:57:28.986 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:28.986 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.987 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.987 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.987 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:28.987 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.988 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.988 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.988 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:57:28.989 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:28.989 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.989 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.990 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.990 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:57:28.990 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:28.990 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.991 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.991 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.991 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:28.992 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.992 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.992 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.992 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:57:28.993 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:28.993 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.993 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.993 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.994 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:57:28.994 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:28.994 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.994 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.994 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.995 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:28.995 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:28.995 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.996 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:28.996 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:57:28.996 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:28.996 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.997 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:28.997 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:28.997 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:57:28.997 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:28.998 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:28.998 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:28.998 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:28.998 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:57:28.998 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:57:28.999 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:57:28.999 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:57:28.999 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:28.999 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:29.000 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:29.000 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:29.000 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.000 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.001 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:57:29.001 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:57:29.001 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:29.001 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:29.001 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:29.002 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:29.002 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:29.002 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:29.002 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:29.003 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 130, 2022-07-21T01:57:29.003 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:29.003 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:29.003 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:29.003 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:29.004 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.004 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:29.004 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:57:29.004 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:29.005 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.005 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.005 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:29.005 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:57:29.005 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:29.006 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:29.006 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:29.006 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.006 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:29.007 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:29.007 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.007 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:29.007 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:57:29.008 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:29.008 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.008 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.008 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:29.009 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:57:29.009 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:29.009 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:29.009 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:29.010 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.010 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:29.010 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:29.010 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.011 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:29.011 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:57:29.011 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:29.012 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.012 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.012 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:29.012 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:57:29.012 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:29.013 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:29.013 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:29.013 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.013 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:29.014 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:29.014 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.014 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:29.014 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:57:29.015 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:29.015 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.015 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.016 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:29.016 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:57:29.016 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:29.016 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:29.017 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:29.017 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.017 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:57:29.017 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:57:29.018 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:57:29.018 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:57:29.018 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:29.019 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:29.019 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:29.019 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:29.019 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.020 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.020 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:57:29.020 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:57:29.020 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:29.021 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:29.021 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:29.021 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:29.021 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:29.021 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:29.022 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:29.022 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 130, 2022-07-21T01:57:29.022 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:29.022 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:29.023 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:29.023 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:29.023 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.024 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:29.024 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:57:29.024 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:29.024 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.025 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.025 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:29.025 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:57:29.025 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:29.026 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:29.026 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:29.026 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.027 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:29.027 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:29.027 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.027 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:29.028 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:57:29.028 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:29.028 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.028 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.029 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:29.029 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:57:29.029 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:29.030 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:29.030 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:29.030 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.030 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:29.031 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:29.031 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.031 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:29.031 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:57:29.032 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:29.032 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.032 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.032 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:29.033 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:57:29.033 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:29.033 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:29.033 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:29.034 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.034 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:29.034 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:29.034 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.035 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:29.035 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:57:29.035 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:29.035 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.036 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.036 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:29.036 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:57:29.036 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:29.037 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:29.037 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:29.037 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.037 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:57:29.037 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:57:29.038 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:57:29.038 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:57:29.038 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:29.038 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:29.039 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:29.039 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:29.039 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:29.039 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:29.040 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:57:29.040 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.040 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:57:29.040 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:29.041 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:29.041 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:29.041 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:29.041 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:29.042 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:57:29.042 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:29.042 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.042 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.043 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:57:29.043 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:29.043 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:29.043 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:29.044 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:29.044 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:29.044 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:57:29.044 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:29.045 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.045 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.045 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:57:29.045 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:29.045 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:29.046 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:29.046 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:29.046 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:29.047 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:57:29.047 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:29.047 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.047 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.048 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:57:29.048 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:29.048 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:29.048 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:29.048 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:29.049 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:29.049 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:57:29.049 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:29.050 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.050 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.050 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:57:29.050 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:29.051 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:29.051 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:29.051 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:29.051 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:29.052 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:57:29.052 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:29.052 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.052 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.053 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:57:29.053 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:29.053 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:29.053 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:29.054 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:29.054 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:29.054 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:57:29.055 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:29.055 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.055 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.055 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:57:29.056 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:29.056 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:29.056 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:29.056 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:29.057 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:29.057 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:57:29.057 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:29.057 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.058 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:29.058 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:57:29.058 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:29.058 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:29.059 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:29.059 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:29.059 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:29.059 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:57:29.059 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:29.060 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:29.060 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:29.060 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:57:29.061 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:57:29.061 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:57:29.061 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:57:29.061 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": {}, 2022-07-21T01:57:29.062 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:57:29.062 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:57:29.062 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:57:29.063 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:57:29.063 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:57:29.063 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:57:29.063 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:57:29.064 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:57:29.064 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:29.064 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:29.064 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:57:29.065 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:57:29.065 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:57:29.065 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:57:29.065 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:57:29.065 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:57:29.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d1f59e700 1 -- 172.21.15.140:0/3105560181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9cfc053b30 msgr2=0x7f9cfc055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d1f59e700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9cfc053b30 0x7f9cfc055ff0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f9d1001b000 tx=0x7f9d10016000).stop 2022-07-21T01:57:29.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d1f59e700 1 -- 172.21.15.140:0/3105560181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d200afbd0 msgr2=0x7f9d200669d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d1f59e700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d200afbd0 0x7f9d200669d0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f9d1400c000 tx=0x7f9d14006df0).stop 2022-07-21T01:57:29.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d1f59e700 1 -- 172.21.15.140:0/3105560181 shutdown_connections 2022-07-21T01:57:29.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d1f59e700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9cfc053b30 0x7f9cfc055ff0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d1f59e700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d2014a530 0x7f9d200ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d1f59e700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d200b5270 0x7f9d200aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d1f59e700 1 --2- 172.21.15.140:0/3105560181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9d200afbd0 0x7f9d200669d0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d1f59e700 1 -- 172.21.15.140:0/3105560181 >> 172.21.15.140:0/3105560181 conn(0x7f9d20062940 msgr2=0x7f9d20062d20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:29.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.804+0000 7f9d1f59e700 1 -- 172.21.15.140:0/3105560181 shutdown_connections 2022-07-21T01:57:29.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:28.808+0000 7f9d1f59e700 1 -- 172.21.15.140:0/3105560181 wait complete. 2022-07-21T01:57:29.072 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:29.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa02d2cd700 1 Processor -- start 2022-07-21T01:57:29.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa02d2cd700 1 -- start start 2022-07-21T01:57:29.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa02d2cd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa02805c880 0x7fa0280ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa02d2cd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0280afd00 0x7fa028147880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa02d2cd700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0280adc90 con 0x7fa0280af150 2022-07-21T01:57:29.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa02d2cd700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02805cc60 con 0x7fa02805c880 2022-07-21T01:57:29.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa02d2cd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280adec0 con 0x7fa0280afd00 2022-07-21T01:57:29.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa026ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa02805c880 0x7fa0280ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa0267fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa0280af150 0x7fa028147170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54858/0 (socket says 172.21.15.140:54858) 2022-07-21T01:57:29.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa0267fc700 1 -- 172.21.15.140:0/464358947 learned_addr learned my addr 172.21.15.140:0/464358947 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:29.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa0277fe700 1 --2- 172.21.15.140:0/464358947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0280afd00 0x7fa028147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa026ffd700 1 -- 172.21.15.140:0/464358947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0280afd00 msgr2=0x7fa028147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:29.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa026ffd700 1 --2- 172.21.15.140:0/464358947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0280afd00 0x7fa028147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa025ffb700 1 -- 172.21.15.140:0/464358947 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458789008 0 0) 0x7fa0280adc90 con 0x7fa0280af150 2022-07-21T01:57:29.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa026ffd700 1 -- 172.21.15.140:0/464358947 >> v1:172.21.15.140:6789/0 conn(0x7fa0280af150 legacy=0x7fa028147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa026ffd700 1 -- 172.21.15.140:0/464358947 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0280aeaf0 con 0x7fa02805c880 2022-07-21T01:57:29.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.180+0000 7fa026ffd700 1 --2- 172.21.15.140:0/464358947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa02805c880 0x7fa0280ae600 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7fa0280ad630 tx=0x7fa01c006df0).ready entity=mon.0 client_cookie=324a27ea7bdf8f1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:29.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.184+0000 7fa025ffb700 1 -- 172.21.15.140:0/464358947 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa01c00d570 con 0x7fa02805c880 2022-07-21T01:57:29.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.184+0000 7fa025ffb700 1 -- 172.21.15.140:0/464358947 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01c000e90 con 0x7fa02805c880 2022-07-21T01:57:29.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.184+0000 7fa025ffb700 1 -- 172.21.15.140:0/464358947 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa01c00d8c0 con 0x7fa02805c880 2022-07-21T01:57:29.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.184+0000 7fa02d2cd700 1 -- 172.21.15.140:0/464358947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa02805c880 msgr2=0x7fa0280ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.184+0000 7fa02d2cd700 1 --2- 172.21.15.140:0/464358947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa02805c880 0x7fa0280ae600 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7fa0280ad630 tx=0x7fa01c006df0).stop 2022-07-21T01:57:29.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.184+0000 7fa02d2cd700 1 -- 172.21.15.140:0/464358947 shutdown_connections 2022-07-21T01:57:29.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.184+0000 7fa02d2cd700 1 --2- 172.21.15.140:0/464358947 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0280afd00 0x7fa028147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.184+0000 7fa02d2cd700 1 --2- 172.21.15.140:0/464358947 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa02805c880 0x7fa0280ae600 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.184+0000 7fa02d2cd700 1 -- 172.21.15.140:0/464358947 >> 172.21.15.140:0/464358947 conn(0x7fa028063460 msgr2=0x7fa02805be30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:29.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.184+0000 7fa02d2cd700 1 -- 172.21.15.140:0/464358947 shutdown_connections 2022-07-21T01:57:29.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.184+0000 7fa02d2cd700 1 -- 172.21.15.140:0/464358947 wait complete. 2022-07-21T01:57:29.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa02d2cd700 1 Processor -- start 2022-07-21T01:57:29.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa02d2cd700 1 -- start start 2022-07-21T01:57:29.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa02d2cd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa02805c880 0x7fa028148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa02d2cd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0280af150 0x7fa028157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa02d2cd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0280afd00 0x7fa028144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa02d2cd700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028064860 con 0x7fa02805c880 2022-07-21T01:57:29.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa02d2cd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028156490 con 0x7fa0280af150 2022-07-21T01:57:29.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa02d2cd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa028156730 con 0x7fa0280afd00 2022-07-21T01:57:29.195 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa026ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa02805c880 0x7fa028148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa026ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa02805c880 0x7fa028148e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40714/0 (socket says 172.21.15.140:40714) 2022-07-21T01:57:29.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa026ffd700 1 -- 172.21.15.140:0/2161390843 learned_addr learned my addr 172.21.15.140:0/2161390843 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:29.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa0267fc700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0280af150 0x7fa028157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.196 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa0277fe700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0280afd00 0x7fa028144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa026ffd700 1 -- 172.21.15.140:0/2161390843 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0280afd00 msgr2=0x7fa028144ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa026ffd700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0280afd00 0x7fa028144ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa026ffd700 1 -- 172.21.15.140:0/2161390843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0280af150 msgr2=0x7fa028157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa026ffd700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0280af150 0x7fa028157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa026ffd700 1 -- 172.21.15.140:0/2161390843 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c00b180 con 0x7fa02805c880 2022-07-21T01:57:29.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa026ffd700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa02805c880 0x7fa028148e90 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7fa0280ad630 tx=0x7fa01c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:29.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa017fff700 1 -- 172.21.15.140:0/2161390843 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa01c016070 con 0x7fa02805c880 2022-07-21T01:57:29.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa017fff700 1 -- 172.21.15.140:0/2161390843 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01c002f70 con 0x7fa02805c880 2022-07-21T01:57:29.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa02d2cd700 1 -- 172.21.15.140:0/2161390843 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0281569d0 con 0x7fa02805c880 2022-07-21T01:57:29.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa017fff700 1 -- 172.21.15.140:0/2161390843 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa01c00de20 con 0x7fa02805c880 2022-07-21T01:57:29.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.188+0000 7fa02d2cd700 1 -- 172.21.15.140:0/2161390843 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa028157680 con 0x7fa02805c880 2022-07-21T01:57:29.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.192+0000 7fa02d2cd700 1 -- 172.21.15.140:0/2161390843 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02814a220 con 0x7fa02805c880 2022-07-21T01:57:29.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.192+0000 7fa017fff700 1 -- 172.21.15.140:0/2161390843 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa01c025030 con 0x7fa02805c880 2022-07-21T01:57:29.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.192+0000 7fa017fff700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa004053b00 0x7fa004055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.192+0000 7fa017fff700 1 -- 172.21.15.140:0/2161390843 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa01c06cee0 con 0x7fa02805c880 2022-07-21T01:57:29.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.192+0000 7fa0267fc700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa004053b00 0x7fa004055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.192+0000 7fa0267fc700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa004053b00 0x7fa004055fc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fa01800b000 tx=0x7fa018009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:29.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.196+0000 7fa017fff700 1 -- 172.21.15.140:0/2161390843 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa02814a220 con 0x7fa02805c880 2022-07-21T01:57:29.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 -- 172.21.15.140:0/2161390843 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa02814a220 con 0x7fa02805c880 2022-07-21T01:57:29.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa017fff700 1 -- 172.21.15.140:0/2161390843 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+2145 (secure 0 0 0) 0x7fa02814a220 con 0x7fa02805c880 2022-07-21T01:57:29.361 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:29.362 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":64,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/558475048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":558475048},{"type":"v1","addr":"172.21.15.152:6833","nonce":558475048}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:14.423640+0000","modified":"2022-07-21T01:57:16.123489+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8061":{"gid":8061,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/2209871450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2209871450},{"type":"v1","addr":"172.21.15.140:6835","nonce":2209871450}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-07-21T01:57:29.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 -- 172.21.15.140:0/2161390843 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa004053b00 msgr2=0x7fa004055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa004053b00 0x7fa004055fc0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fa01800b000 tx=0x7fa018009040).stop 2022-07-21T01:57:29.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 -- 172.21.15.140:0/2161390843 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa02805c880 msgr2=0x7fa028148e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa02805c880 0x7fa028148e90 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7fa0280ad630 tx=0x7fa01c014000).stop 2022-07-21T01:57:29.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 -- 172.21.15.140:0/2161390843 shutdown_connections 2022-07-21T01:57:29.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa004053b00 0x7fa004055fc0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0280afd00 0x7fa028144ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0280af150 0x7fa028157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 --2- 172.21.15.140:0/2161390843 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa02805c880 0x7fa028148e90 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 -- 172.21.15.140:0/2161390843 >> 172.21.15.140:0/2161390843 conn(0x7fa028063460 msgr2=0x7fa0280b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:29.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 -- 172.21.15.140:0/2161390843 shutdown_connections 2022-07-21T01:57:29.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.360+0000 7fa02d2cd700 1 -- 172.21.15.140:0/2161390843 wait complete. 2022-07-21T01:57:29.367 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 64 2022-07-21T01:57:29.375 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:57:29.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.476+0000 7f93a6bb5700 1 Processor -- start 2022-07-21T01:57:29.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.476+0000 7f93a6bb5700 1 -- start start 2022-07-21T01:57:29.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f93a00ad270 0x7f93a01437c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f93a01462a0 0x7f93a013fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a00b9c40 con 0x7f93a00b9860 2022-07-21T01:57:29.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a0143cb0 con 0x7f93a01462a0 2022-07-21T01:57:29.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a0142de0 con 0x7f93a00ad270 2022-07-21T01:57:29.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a5152700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f93a01462a0 0x7f93a013fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a4951700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f93a00b9860 0x7f93a00b4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54868/0 (socket says 172.21.15.140:54868) 2022-07-21T01:57:29.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a4951700 1 -- 172.21.15.140:0/3454424647 learned_addr learned my addr 172.21.15.140:0/3454424647 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:29.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f939ffff700 1 --2- 172.21.15.140:0/3454424647 >> v2:172.21.15.152:3300/0 conn(0x7f93a00ad270 0x7f93a01437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a5152700 1 -- 172.21.15.140:0/3454424647 >> v2:172.21.15.152:3300/0 conn(0x7f93a00ad270 msgr2=0x7f93a01437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:29.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a5152700 1 --2- 172.21.15.140:0/3454424647 >> v2:172.21.15.152:3300/0 conn(0x7f93a00ad270 0x7f93a01437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a5152700 1 -- 172.21.15.140:0/3454424647 >> v1:172.21.15.140:6789/0 conn(0x7f93a00b9860 legacy=0x7f93a00b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f939f7fe700 1 -- 172.21.15.140:0/3454424647 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881031725 0 0) 0x7f93a00b9c40 con 0x7f93a00b9860 2022-07-21T01:57:29.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a5152700 1 -- 172.21.15.140:0/3454424647 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0143080 con 0x7f93a01462a0 2022-07-21T01:57:29.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a5152700 1 --2- 172.21.15.140:0/3454424647 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93a01462a0 0x7f93a013fdb0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f93a00b9eb0 tx=0x7f93980091d0).ready entity=mon.0 client_cookie=f57f62e526719326 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:29.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f939f7fe700 1 -- 172.21.15.140:0/3454424647 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939800b640 con 0x7f93a01462a0 2022-07-21T01:57:29.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f939f7fe700 1 -- 172.21.15.140:0/3454424647 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939800b7a0 con 0x7f93a01462a0 2022-07-21T01:57:29.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f939f7fe700 1 -- 172.21.15.140:0/3454424647 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939800b910 con 0x7f93a01462a0 2022-07-21T01:57:29.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 -- 172.21.15.140:0/3454424647 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93a01462a0 msgr2=0x7f93a013fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 --2- 172.21.15.140:0/3454424647 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93a01462a0 0x7f93a013fdb0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f93a00b9eb0 tx=0x7f93980091d0).stop 2022-07-21T01:57:29.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 -- 172.21.15.140:0/3454424647 shutdown_connections 2022-07-21T01:57:29.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 --2- 172.21.15.140:0/3454424647 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93a01462a0 0x7f93a013fdb0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 --2- 172.21.15.140:0/3454424647 >> v2:172.21.15.152:3300/0 conn(0x7f93a00ad270 0x7f93a01437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 -- 172.21.15.140:0/3454424647 >> 172.21.15.140:0/3454424647 conn(0x7f93a0066e20 msgr2=0x7f93a0067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:29.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 -- 172.21.15.140:0/3454424647 shutdown_connections 2022-07-21T01:57:29.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.480+0000 7f93a6bb5700 1 -- 172.21.15.140:0/3454424647 wait complete. 2022-07-21T01:57:29.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a6bb5700 1 Processor -- start 2022-07-21T01:57:29.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a6bb5700 1 -- start start 2022-07-21T01:57:29.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a6bb5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93a00ad270 0x7f93a0112fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a6bb5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a00b9860 0x7f93a01134a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a6bb5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93a01462a0 0x7f93a013e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a6bb5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a00b20b0 con 0x7f93a01462a0 2022-07-21T01:57:29.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a6bb5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a0115970 con 0x7f93a00b9860 2022-07-21T01:57:29.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a6bb5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a00b30f0 con 0x7f93a00ad270 2022-07-21T01:57:29.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a5152700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93a01462a0 0x7f93a013e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a5152700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93a01462a0 0x7f93a013e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40732/0 (socket says 172.21.15.140:40732) 2022-07-21T01:57:29.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a5152700 1 -- 172.21.15.140:0/226906704 learned_addr learned my addr 172.21.15.140:0/226906704 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:29.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a4951700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93a00ad270 0x7f93a0112fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f939ffff700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a00b9860 0x7f93a01134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a5152700 1 -- 172.21.15.140:0/226906704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93a00ad270 msgr2=0x7f93a0112fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a5152700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93a00ad270 0x7f93a0112fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a5152700 1 -- 172.21.15.140:0/226906704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a00b9860 msgr2=0x7f93a01134a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:29.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a5152700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a00b9860 0x7f93a01134a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:29.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a5152700 1 -- 172.21.15.140:0/226906704 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9398009470 con 0x7f93a01462a0 2022-07-21T01:57:29.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a4951700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93a00ad270 0x7f93a0112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:29.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a5152700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93a01462a0 0x7f93a013e430 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f93a00b9eb0 tx=0x7f9398016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:29.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f939dffb700 1 -- 172.21.15.140:0/226906704 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939800b910 con 0x7f93a01462a0 2022-07-21T01:57:29.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a6bb5700 1 -- 172.21.15.140:0/226906704 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a0112780 con 0x7f93a01462a0 2022-07-21T01:57:29.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f939dffb700 1 -- 172.21.15.140:0/226906704 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93980032f0 con 0x7f93a01462a0 2022-07-21T01:57:29.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f93a6bb5700 1 -- 172.21.15.140:0/226906704 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a013eac0 con 0x7f93a01462a0 2022-07-21T01:57:29.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.484+0000 7f939dffb700 1 -- 172.21.15.140:0/226906704 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f939800b640 con 0x7f93a01462a0 2022-07-21T01:57:29.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.488+0000 7f93a6bb5700 1 -- 172.21.15.140:0/226906704 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a00b2220 con 0x7f93a01462a0 2022-07-21T01:57:29.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.488+0000 7f939dffb700 1 -- 172.21.15.140:0/226906704 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9398026030 con 0x7f93a01462a0 2022-07-21T01:57:29.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.488+0000 7f939dffb700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9380053b30 0x7f9380055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:29.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.488+0000 7f93a4951700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9380053b30 0x7f9380055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:29.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.488+0000 7f939dffb700 1 -- 172.21.15.140:0/226906704 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f939806ca10 con 0x7f93a01462a0 2022-07-21T01:57:29.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.488+0000 7f93a4951700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9380053b30 0x7f9380055ff0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f939000b000 tx=0x7f9390009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:29.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.496+0000 7f939dffb700 1 -- 172.21.15.140:0/226906704 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f93a00b2220 con 0x7f93a01462a0 2022-07-21T01:57:29.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:29.660+0000 7f93a6bb5700 1 -- 172.21.15.140:0/226906704 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f93a00b21f0 con 0x7f93a01462a0 2022-07-21T01:57:30.185 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:57:30.184+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:57:30.194 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.192+0000 7f939dffb700 1 -- 172.21.15.140:0/226906704 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9398024020 con 0x7f93a01462a0 2022-07-21T01:57:30.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.196+0000 7f93a6bb5700 1 -- 172.21.15.140:0/226906704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9380053b30 msgr2=0x7f9380055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.196+0000 7f93a6bb5700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9380053b30 0x7f9380055ff0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f939000b000 tx=0x7f9390009040).stop 2022-07-21T01:57:30.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.196+0000 7f93a6bb5700 1 -- 172.21.15.140:0/226906704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93a01462a0 msgr2=0x7f93a013e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.196+0000 7f93a6bb5700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93a01462a0 0x7f93a013e430 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f93a00b9eb0 tx=0x7f9398016000).stop 2022-07-21T01:57:30.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.196+0000 7f93a6bb5700 1 -- 172.21.15.140:0/226906704 shutdown_connections 2022-07-21T01:57:30.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.196+0000 7f93a6bb5700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9380053b30 0x7f9380055ff0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.196+0000 7f93a6bb5700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f93a01462a0 0x7f93a013e430 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.196+0000 7f93a6bb5700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93a00b9860 0x7f93a01134a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.196+0000 7f93a6bb5700 1 --2- 172.21.15.140:0/226906704 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93a00ad270 0x7f93a0112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.196+0000 7f93a6bb5700 1 -- 172.21.15.140:0/226906704 >> 172.21.15.140:0/226906704 conn(0x7f93a0066e20 msgr2=0x7f93a00b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:30.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.196+0000 7f93a6bb5700 1 -- 172.21.15.140:0/226906704 shutdown_connections 2022-07-21T01:57:30.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.200+0000 7f93a6bb5700 1 -- 172.21.15.140:0/226906704 wait complete. 2022-07-21T01:57:30.203 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:57:30.215 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:30.226 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:57:30.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f80715a2700 1 Processor -- start 2022-07-21T01:57:30.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f80715a2700 1 -- start start 2022-07-21T01:57:30.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f80715a2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 0x7f806c1460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:30.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f80715a2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806c05fae0 con 0x7f806c0b0e30 2022-07-21T01:57:30.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f80715a2700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f806c0b1210 con 0x7f806c13fa10 2022-07-21T01:57:30.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f80715a2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806c0607b0 con 0x7f806c0654b0 2022-07-21T01:57:30.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f806c0b0e30 0x7f806c064da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54894/0 (socket says 172.21.15.140:54894) 2022-07-21T01:57:30.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806bfff700 1 -- 172.21.15.140:0/389342611 learned_addr learned my addr 172.21.15.140:0/389342611 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:30.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806b7fe700 1 --2- 172.21.15.140:0/389342611 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 0x7f806c1460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:30.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806affd700 1 -- 172.21.15.140:0/389342611 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674652563 0 0) 0x7f806c05fae0 con 0x7f806c0b0e30 2022-07-21T01:57:30.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806affd700 1 -- 172.21.15.140:0/389342611 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8058003260 con 0x7f806c0b0e30 2022-07-21T01:57:30.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806affd700 1 -- 172.21.15.140:0/389342611 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622274875 0 0) 0x7f806c0b1210 con 0x7f806c13fa10 2022-07-21T01:57:30.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806affd700 1 -- 172.21.15.140:0/389342611 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f806c05fae0 con 0x7f806c13fa10 2022-07-21T01:57:30.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806affd700 1 -- 172.21.15.140:0/389342611 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2612904660 0 0) 0x7f8058003260 con 0x7f806c0b0e30 2022-07-21T01:57:30.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806b7fe700 1 -- 172.21.15.140:0/389342611 >> v1:172.21.15.152:6789/0 conn(0x7f806c13fa10 legacy=0x7f806c13fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806b7fe700 1 -- 172.21.15.140:0/389342611 >> v1:172.21.15.140:6789/0 conn(0x7f806c0b0e30 legacy=0x7f806c064da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806b7fe700 1 -- 172.21.15.140:0/389342611 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c0b25a0 con 0x7f806c0654b0 2022-07-21T01:57:30.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.308+0000 7f806b7fe700 1 --2- 172.21.15.140:0/389342611 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 0x7f806c1460f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f8064001490 tx=0x7f8060006df0).ready entity=mon.2 client_cookie=6bc854222af76ea1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:30.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f806affd700 1 -- 172.21.15.140:0/389342611 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80640019e0 con 0x7f806c0654b0 2022-07-21T01:57:30.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f806affd700 1 -- 172.21.15.140:0/389342611 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8060000e90 con 0x7f806c0654b0 2022-07-21T01:57:30.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f806affd700 1 -- 172.21.15.140:0/389342611 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8060012460 con 0x7f806c0654b0 2022-07-21T01:57:30.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 -- 172.21.15.140:0/389342611 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 msgr2=0x7f806c1460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 --2- 172.21.15.140:0/389342611 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 0x7f806c1460f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f8064001490 tx=0x7f8060006df0).stop 2022-07-21T01:57:30.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 -- 172.21.15.140:0/389342611 shutdown_connections 2022-07-21T01:57:30.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 --2- 172.21.15.140:0/389342611 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 0x7f806c1460f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 -- 172.21.15.140:0/389342611 >> 172.21.15.140:0/389342611 conn(0x7f806c061d70 msgr2=0x7f806c063a50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:30.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 -- 172.21.15.140:0/389342611 shutdown_connections 2022-07-21T01:57:30.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 -- 172.21.15.140:0/389342611 wait complete. 2022-07-21T01:57:30.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 Processor -- start 2022-07-21T01:57:30.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 -- start start 2022-07-21T01:57:30.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 0x7f806c0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:30.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f806c0b0e30 0x7f806c155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:30.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f806c13fa10 0x7f806c066f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:30.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c1429a0 con 0x7f806c13fa10 2022-07-21T01:57:30.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806c145b80 con 0x7f806c0b0e30 2022-07-21T01:57:30.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f80715a2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806c142700 con 0x7f806c0654b0 2022-07-21T01:57:30.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f806bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 0x7f806c0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:30.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.312+0000 7f8070da1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f806c13fa10 0x7f806c066f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:30.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f806bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 0x7f806c0ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:37484/0 (socket says 172.21.15.140:37484) 2022-07-21T01:57:30.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f806bfff700 1 -- 172.21.15.140:0/2585090082 learned_addr learned my addr 172.21.15.140:0/2585090082 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:30.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f806b7fe700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f806c0b0e30 0x7f806c155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:30.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f8070da1700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f806c13fa10 0x7f806c066f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40758/0 (socket says 172.21.15.140:40758) 2022-07-21T01:57:30.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f806b7fe700 1 -- 172.21.15.140:0/2585090082 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 msgr2=0x7f806c0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f806b7fe700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 0x7f806c0ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f806b7fe700 1 -- 172.21.15.140:0/2585090082 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f806c13fa10 msgr2=0x7f806c066f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f806b7fe700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f806c13fa10 0x7f806c066f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f806b7fe700 1 -- 172.21.15.140:0/2585090082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806000b180 con 0x7f806c0b0e30 2022-07-21T01:57:30.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f8070da1700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f806c13fa10 0x7f806c066f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:30.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f806b7fe700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f806c0b0e30 0x7f806c155670 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f8064001490 tx=0x7f8060014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:30.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f80697fa700 1 -- 172.21.15.140:0/2585090082 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8060016070 con 0x7f806c0b0e30 2022-07-21T01:57:30.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f80697fa700 1 -- 172.21.15.140:0/2585090082 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8060002f50 con 0x7f806c0b0e30 2022-07-21T01:57:30.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f80715a2700 1 -- 172.21.15.140:0/2585090082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f806c142410 con 0x7f806c0b0e30 2022-07-21T01:57:30.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f80697fa700 1 -- 172.21.15.140:0/2585090082 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8060012460 con 0x7f806c0b0e30 2022-07-21T01:57:30.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f80715a2700 1 -- 172.21.15.140:0/2585090082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806c140d60 con 0x7f806c0b0e30 2022-07-21T01:57:30.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f80715a2700 1 -- 172.21.15.140:0/2585090082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806c0574e0 con 0x7f806c0b0e30 2022-07-21T01:57:30.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f80697fa700 1 -- 172.21.15.140:0/2585090082 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8060025030 con 0x7f806c0b0e30 2022-07-21T01:57:30.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f80697fa700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f804c053b30 0x7f804c055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:30.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.316+0000 7f80697fa700 1 -- 172.21.15.140:0/2585090082 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f806006ccc0 con 0x7f806c0b0e30 2022-07-21T01:57:30.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.320+0000 7f806bfff700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f804c053b30 0x7f804c055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:30.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.320+0000 7f806bfff700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f804c053b30 0x7f804c055ff0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f806400d000 tx=0x7f8064000f40).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:30.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.324+0000 7f80697fa700 1 -- 172.21.15.140:0/2585090082 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8060043b90 con 0x7f806c0b0e30 2022-07-21T01:57:30.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.484+0000 7f80715a2700 1 -- 172.21.15.140:0/2585090082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f806c06a6e0 con 0x7f806c0b0e30 2022-07-21T01:57:30.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80697fa700 1 -- 172.21.15.140:0/2585090082 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+1782 (secure 0 0 0) 0x7f8060023020 con 0x7f806c0b0e30 2022-07-21T01:57:30.489 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:30.489 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":65,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/558475048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":558475048},{"type":"v1","addr":"172.21.15.152:6833","nonce":558475048}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:14.423640+0000","modified":"2022-07-21T01:57:30.185628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2022-07-21T01:57:30.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 -- 172.21.15.140:0/2585090082 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f804c053b30 msgr2=0x7f804c055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f804c053b30 0x7f804c055ff0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f806400d000 tx=0x7f8064000f40).stop 2022-07-21T01:57:30.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 -- 172.21.15.140:0/2585090082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f806c0b0e30 msgr2=0x7f806c155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f806c0b0e30 0x7f806c155670 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f8064001490 tx=0x7f8060014000).stop 2022-07-21T01:57:30.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 -- 172.21.15.140:0/2585090082 shutdown_connections 2022-07-21T01:57:30.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f804c053b30 0x7f804c055ff0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f806c13fa10 0x7f806c066f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f806c0b0e30 0x7f806c155670 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 --2- 172.21.15.140:0/2585090082 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f806c0654b0 0x7f806c0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 -- 172.21.15.140:0/2585090082 >> 172.21.15.140:0/2585090082 conn(0x7f806c061d70 msgr2=0x7f806c143660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:30.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 -- 172.21.15.140:0/2585090082 shutdown_connections 2022-07-21T01:57:30.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.488+0000 7f80715a2700 1 -- 172.21.15.140:0/2585090082 wait complete. 2022-07-21T01:57:30.509 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 65 2022-07-21T01:57:30.511 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:57:30.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.616+0000 7f0845c1e700 1 Processor -- start 2022-07-21T01:57:30.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.616+0000 7f0845c1e700 1 -- start start 2022-07-21T01:57:30.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f0845c1e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f08400664f0 0x7f084013f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:30.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f0845c1e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08400b2380 con 0x7f08400b2ea0 2022-07-21T01:57:30.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f0845c1e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08400659b0 con 0x7f0840064200 2022-07-21T01:57:30.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f0845c1e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08400b2850 con 0x7f08400664f0 2022-07-21T01:57:30.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083f7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f08400b2ea0 0x7f0840065de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54908/0 (socket says 172.21.15.140:54908) 2022-07-21T01:57:30.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083f7fe700 1 -- 172.21.15.140:0/1703436238 learned_addr learned my addr 172.21.15.140:0/1703436238 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:30.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083ffff700 1 --2- 172.21.15.140:0/1703436238 >> v2:172.21.15.152:3300/0 conn(0x7f08400664f0 0x7f084013f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:30.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083e7fc700 1 -- 172.21.15.140:0/1703436238 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248801425 0 0) 0x7f08400b2380 con 0x7f08400b2ea0 2022-07-21T01:57:30.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083e7fc700 1 -- 172.21.15.140:0/1703436238 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0820003260 con 0x7f08400b2ea0 2022-07-21T01:57:30.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083ffff700 1 -- 172.21.15.140:0/1703436238 >> v1:172.21.15.152:6789/0 conn(0x7f0840064200 legacy=0x7f084013e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083ffff700 1 -- 172.21.15.140:0/1703436238 >> v1:172.21.15.140:6789/0 conn(0x7f08400b2ea0 legacy=0x7f0840065de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083ffff700 1 -- 172.21.15.140:0/1703436238 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084013feb0 con 0x7f08400664f0 2022-07-21T01:57:30.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083ffff700 1 --2- 172.21.15.140:0/1703436238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08400664f0 0x7f084013f9c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f0840064cf0 tx=0x7f08380091d0).ready entity=mon.1 client_cookie=3b5f6515fa4e3a8f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:30.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083e7fc700 1 -- 172.21.15.140:0/1703436238 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f083800b680 con 0x7f08400664f0 2022-07-21T01:57:30.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083e7fc700 1 -- 172.21.15.140:0/1703436238 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083800b7e0 con 0x7f08400664f0 2022-07-21T01:57:30.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f083e7fc700 1 -- 172.21.15.140:0/1703436238 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f083800b950 con 0x7f08400664f0 2022-07-21T01:57:30.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f0845c1e700 1 -- 172.21.15.140:0/1703436238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08400664f0 msgr2=0x7f084013f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f0845c1e700 1 --2- 172.21.15.140:0/1703436238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08400664f0 0x7f084013f9c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f0840064cf0 tx=0x7f08380091d0).stop 2022-07-21T01:57:30.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f0845c1e700 1 -- 172.21.15.140:0/1703436238 shutdown_connections 2022-07-21T01:57:30.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f0845c1e700 1 --2- 172.21.15.140:0/1703436238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08400664f0 0x7f084013f9c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f0845c1e700 1 -- 172.21.15.140:0/1703436238 >> 172.21.15.140:0/1703436238 conn(0x7f0840069d80 msgr2=0x7f0840063af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:30.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f0845c1e700 1 -- 172.21.15.140:0/1703436238 shutdown_connections 2022-07-21T01:57:30.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.620+0000 7f0845c1e700 1 -- 172.21.15.140:0/1703436238 wait complete. 2022-07-21T01:57:30.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f0845c1e700 1 Processor -- start 2022-07-21T01:57:30.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f0845c1e700 1 -- start start 2022-07-21T01:57:30.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f0845c1e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0840064200 0x7f08400afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:30.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f0845c1e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08400664f0 0x7f08400af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:30.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f0845c1e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08400b2ea0 0x7f0840142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:30.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f0845c1e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08400afc80 con 0x7f08400b2ea0 2022-07-21T01:57:30.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f0845c1e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08400ae600 con 0x7f08400664f0 2022-07-21T01:57:30.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f0845c1e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08400ae8a0 con 0x7f0840064200 2022-07-21T01:57:30.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08400b2ea0 0x7f0840142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:30.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08400b2ea0 0x7f0840142fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40770/0 (socket says 172.21.15.140:40770) 2022-07-21T01:57:30.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083ffff700 1 -- 172.21.15.140:0/924352060 learned_addr learned my addr 172.21.15.140:0/924352060 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:30.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083f7fe700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0840064200 0x7f08400afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:30.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083effd700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08400664f0 0x7f08400af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:30.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083ffff700 1 -- 172.21.15.140:0/924352060 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0840064200 msgr2=0x7f08400afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083ffff700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0840064200 0x7f08400afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083ffff700 1 -- 172.21.15.140:0/924352060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08400664f0 msgr2=0x7f08400af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:30.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083ffff700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08400664f0 0x7f08400af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:30.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083ffff700 1 -- 172.21.15.140:0/924352060 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0838009470 con 0x7f08400b2ea0 2022-07-21T01:57:30.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083effd700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08400664f0 0x7f08400af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:57:30.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083ffff700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08400b2ea0 0x7f0840142fa0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f0840064cf0 tx=0x7f0838016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:30.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.624+0000 7f083cff9700 1 -- 172.21.15.140:0/924352060 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f083800b680 con 0x7f08400b2ea0 2022-07-21T01:57:30.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.628+0000 7f083cff9700 1 -- 172.21.15.140:0/924352060 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08380107a0 con 0x7f08400b2ea0 2022-07-21T01:57:30.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.628+0000 7f0845c1e700 1 -- 172.21.15.140:0/924352060 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08400aeb40 con 0x7f08400b2ea0 2022-07-21T01:57:30.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.628+0000 7f083cff9700 1 -- 172.21.15.140:0/924352060 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0838010b00 con 0x7f08400b2ea0 2022-07-21T01:57:30.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.628+0000 7f0845c1e700 1 -- 172.21.15.140:0/924352060 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0840143490 con 0x7f08400b2ea0 2022-07-21T01:57:30.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.628+0000 7f0845c1e700 1 -- 172.21.15.140:0/924352060 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0840143740 con 0x7f08400b2ea0 2022-07-21T01:57:30.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.628+0000 7f083cff9700 1 -- 172.21.15.140:0/924352060 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0838027020 con 0x7f08400b2ea0 2022-07-21T01:57:30.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.628+0000 7f083cff9700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0824053aa0 0x7f0824055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:30.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.628+0000 7f083f7fe700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0824053aa0 0x7f0824055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:30.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.628+0000 7f083cff9700 1 -- 172.21.15.140:0/924352060 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f083806d130 con 0x7f08400b2ea0 2022-07-21T01:57:30.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.628+0000 7f083f7fe700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0824053aa0 0x7f0824055f60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f083000b000 tx=0x7f0830009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:30.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.636+0000 7f083cff9700 1 -- 172.21.15.140:0/924352060 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0838044000 con 0x7f08400b2ea0 2022-07-21T01:57:30.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:30.800+0000 7f0845c1e700 1 -- 172.21.15.140:0/924352060 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f084013feb0 con 0x7f08400b2ea0 2022-07-21T01:57:31.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.211+0000 7f083cff9700 1 -- 172.21.15.140:0/924352060 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 104+0+0 (secure 0 0 0) 0x7f084013feb0 con 0x7f08400b2ea0 2022-07-21T01:57:31.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 -- 172.21.15.140:0/924352060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0824053aa0 msgr2=0x7f0824055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:31.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0824053aa0 0x7f0824055f60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f083000b000 tx=0x7f0830009040).stop 2022-07-21T01:57:31.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 -- 172.21.15.140:0/924352060 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08400b2ea0 msgr2=0x7f0840142fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:31.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08400b2ea0 0x7f0840142fa0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f0840064cf0 tx=0x7f0838016000).stop 2022-07-21T01:57:31.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 -- 172.21.15.140:0/924352060 shutdown_connections 2022-07-21T01:57:31.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0824053aa0 0x7f0824055f60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:31.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f08400b2ea0 0x7f0840142fa0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:31.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08400664f0 0x7f08400af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:31.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 --2- 172.21.15.140:0/924352060 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0840064200 0x7f08400afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:31.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 -- 172.21.15.140:0/924352060 >> 172.21.15.140:0/924352060 conn(0x7f0840069d80 msgr2=0x7f08400b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:31.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 -- 172.21.15.140:0/924352060 shutdown_connections 2022-07-21T01:57:31.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.215+0000 7f0845c1e700 1 -- 172.21.15.140:0/924352060 wait complete. 2022-07-21T01:57:31.235 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:57:31.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.347+0000 7fe454ee1700 1 Processor -- start 2022-07-21T01:57:31.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.347+0000 7fe454ee1700 1 -- start start 2022-07-21T01:57:31.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.347+0000 7fe454ee1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4500b5340 0x7fe4500b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:31.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.347+0000 7fe454ee1700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4500ba3a0 con 0x7fe4500b9fc0 2022-07-21T01:57:31.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.347+0000 7fe454ee1700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe450146e20 con 0x7fe4500b4b20 2022-07-21T01:57:31.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.347+0000 7fe454ee1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4500b1200 con 0x7fe4500b5340 2022-07-21T01:57:31.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.347+0000 7fe44e59c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe4500b9fc0 0x7fe4500b0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54924/0 (socket says 172.21.15.140:54924) 2022-07-21T01:57:31.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44e59c700 1 -- 172.21.15.140:0/3575985158 learned_addr learned my addr 172.21.15.140:0/3575985158 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:31.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44dd9b700 1 --2- 172.21.15.140:0/3575985158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4500b5340 0x7fe4500b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:31.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44d59a700 1 -- 172.21.15.140:0/3575985158 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438661425 0 0) 0x7fe4500ba3a0 con 0x7fe4500b9fc0 2022-07-21T01:57:31.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44d59a700 1 -- 172.21.15.140:0/3575985158 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe434003260 con 0x7fe4500b9fc0 2022-07-21T01:57:31.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44d59a700 1 -- 172.21.15.140:0/3575985158 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364733057 0 0) 0x7fe450146e20 con 0x7fe4500b4b20 2022-07-21T01:57:31.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44d59a700 1 -- 172.21.15.140:0/3575985158 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4500ba3a0 con 0x7fe4500b4b20 2022-07-21T01:57:31.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44dd9b700 1 -- 172.21.15.140:0/3575985158 >> v1:172.21.15.152:6789/0 conn(0x7fe4500b4b20 legacy=0x7fe4500b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:31.356 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44dd9b700 1 -- 172.21.15.140:0/3575985158 >> v1:172.21.15.140:6789/0 conn(0x7fe4500b9fc0 legacy=0x7fe4500b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:31.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44dd9b700 1 -- 172.21.15.140:0/3575985158 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4501470c0 con 0x7fe4500b5340 2022-07-21T01:57:31.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44dd9b700 1 --2- 172.21.15.140:0/3575985158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4500b5340 0x7fe4500b0030 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fe440002f70 tx=0x7fe440006df0).ready entity=mon.2 client_cookie=d8e6258677be1b5a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:31.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44d59a700 1 -- 172.21.15.140:0/3575985158 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe44000d5c0 con 0x7fe4500b5340 2022-07-21T01:57:31.357 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44d59a700 1 -- 172.21.15.140:0/3575985158 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe440000e90 con 0x7fe4500b5340 2022-07-21T01:57:31.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe44d59a700 1 -- 172.21.15.140:0/3575985158 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe44000d910 con 0x7fe4500b5340 2022-07-21T01:57:31.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe454ee1700 1 -- 172.21.15.140:0/3575985158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4500b5340 msgr2=0x7fe4500b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:31.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe454ee1700 1 --2- 172.21.15.140:0/3575985158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4500b5340 0x7fe4500b0030 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fe440002f70 tx=0x7fe440006df0).stop 2022-07-21T01:57:31.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe454ee1700 1 -- 172.21.15.140:0/3575985158 shutdown_connections 2022-07-21T01:57:31.358 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe454ee1700 1 --2- 172.21.15.140:0/3575985158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4500b5340 0x7fe4500b0030 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:31.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe454ee1700 1 -- 172.21.15.140:0/3575985158 >> 172.21.15.140:0/3575985158 conn(0x7fe4500678e0 msgr2=0x7fe4500b9570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:31.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe454ee1700 1 -- 172.21.15.140:0/3575985158 shutdown_connections 2022-07-21T01:57:31.359 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.351+0000 7fe454ee1700 1 -- 172.21.15.140:0/3575985158 wait complete. 2022-07-21T01:57:31.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe454ee1700 1 Processor -- start 2022-07-21T01:57:31.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe454ee1700 1 -- start start 2022-07-21T01:57:31.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe454ee1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4500b4b20 0x7fe450114730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:31.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe454ee1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4500b5340 0x7fe450152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:31.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe454ee1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4500b9fc0 0x7fe450152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:31.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe454ee1700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe450114080 con 0x7fe4500b4b20 2022-07-21T01:57:31.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe454ee1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe450114c20 con 0x7fe4500b5340 2022-07-21T01:57:31.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe454ee1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4500af620 con 0x7fe4500b9fc0 2022-07-21T01:57:31.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe44e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4500b4b20 0x7fe450114730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:31.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe44e59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4500b4b20 0x7fe450114730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40784/0 (socket says 172.21.15.140:40784) 2022-07-21T01:57:31.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe44e59c700 1 -- 172.21.15.140:0/2450637411 learned_addr learned my addr 172.21.15.140:0/2450637411 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:31.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe44dd9b700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4500b5340 0x7fe450152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:31.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe44ed9d700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4500b9fc0 0x7fe450152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:31.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe44e59c700 1 -- 172.21.15.140:0/2450637411 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4500b9fc0 msgr2=0x7fe450152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:31.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe44e59c700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4500b9fc0 0x7fe450152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:31.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe44e59c700 1 -- 172.21.15.140:0/2450637411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4500b5340 msgr2=0x7fe450152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:31.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe44e59c700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4500b5340 0x7fe450152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:31.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe44e59c700 1 -- 172.21.15.140:0/2450637411 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44000b180 con 0x7fe4500b4b20 2022-07-21T01:57:31.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe44e59c700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4500b4b20 0x7fe450114730 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7fe4440011d0 tx=0x7fe444007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:31.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe43f7fe700 1 -- 172.21.15.140:0/2450637411 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe44400ce70 con 0x7fe4500b4b20 2022-07-21T01:57:31.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe43f7fe700 1 -- 172.21.15.140:0/2450637411 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4440070d0 con 0x7fe4500b4b20 2022-07-21T01:57:31.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe454ee1700 1 -- 172.21.15.140:0/2450637411 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe450112b60 con 0x7fe4500b4b20 2022-07-21T01:57:31.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe43f7fe700 1 -- 172.21.15.140:0/2450637411 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe444007430 con 0x7fe4500b4b20 2022-07-21T01:57:31.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.355+0000 7fe454ee1700 1 -- 172.21.15.140:0/2450637411 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4501531d0 con 0x7fe4500b4b20 2022-07-21T01:57:31.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.359+0000 7fe454ee1700 1 -- 172.21.15.140:0/2450637411 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe450116fd0 con 0x7fe4500b4b20 2022-07-21T01:57:31.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.359+0000 7fe43f7fe700 1 -- 172.21.15.140:0/2450637411 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe444010070 con 0x7fe4500b4b20 2022-07-21T01:57:31.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.359+0000 7fe43f7fe700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe42c053a70 0x7fe42c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:31.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.359+0000 7fe43f7fe700 1 -- 172.21.15.140:0/2450637411 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe44406b560 con 0x7fe4500b4b20 2022-07-21T01:57:31.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.359+0000 7fe44dd9b700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe42c053a70 0x7fe42c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:31.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.359+0000 7fe44dd9b700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe42c053a70 0x7fe42c055f30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fe44001a000 tx=0x7fe440014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:31.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.363+0000 7fe43f7fe700 1 -- 172.21.15.140:0/2450637411 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe444042320 con 0x7fe4500b4b20 2022-07-21T01:57:31.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:31.491+0000 7fe454ee1700 1 -- 172.21.15.140:0/2450637411 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe450114fb0 con 0x7fe4500b4b20 2022-07-21T01:57:32.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.223+0000 7fe43f7fe700 1 -- 172.21.15.140:0/2450637411 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v134) v1 ==== 185+0+0 (secure 0 0 0) 0x7fe444045940 con 0x7fe4500b4b20 2022-07-21T01:57:32.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.227+0000 7fe454ee1700 1 -- 172.21.15.140:0/2450637411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe42c053a70 msgr2=0x7fe42c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:32.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.227+0000 7fe454ee1700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe42c053a70 0x7fe42c055f30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fe44001a000 tx=0x7fe440014000).stop 2022-07-21T01:57:32.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.227+0000 7fe454ee1700 1 -- 172.21.15.140:0/2450637411 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4500b4b20 msgr2=0x7fe450114730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:32.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.227+0000 7fe454ee1700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4500b4b20 0x7fe450114730 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7fe4440011d0 tx=0x7fe444007b50).stop 2022-07-21T01:57:32.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.227+0000 7fe454ee1700 1 -- 172.21.15.140:0/2450637411 shutdown_connections 2022-07-21T01:57:32.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.227+0000 7fe454ee1700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe42c053a70 0x7fe42c055f30 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:32.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.227+0000 7fe454ee1700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe4500b9fc0 0x7fe450152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:32.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.227+0000 7fe454ee1700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4500b5340 0x7fe450152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:32.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.227+0000 7fe454ee1700 1 --2- 172.21.15.140:0/2450637411 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe4500b4b20 0x7fe450114730 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:32.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.227+0000 7fe454ee1700 1 -- 172.21.15.140:0/2450637411 >> 172.21.15.140:0/2450637411 conn(0x7fe4500678e0 msgr2=0x7fe450141320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:32.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.227+0000 7fe454ee1700 1 -- 172.21.15.140:0/2450637411 shutdown_connections 2022-07-21T01:57:32.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.231+0000 7fe454ee1700 1 -- 172.21.15.140:0/2450637411 wait complete. 2022-07-21T01:57:32.235 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:57:32.245 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:57:32.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.359+0000 7f60e30c4700 1 Processor -- start 2022-07-21T01:57:32.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.359+0000 7f60e30c4700 1 -- start start 2022-07-21T01:57:32.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e30c4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f60dc0966e0 0x7f60dc0983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:32.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e30c4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60dc095c60 0x7f60dc060cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:32.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e30c4700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60dc0971d0 con 0x7f60dc0b1b80 2022-07-21T01:57:32.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e30c4700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60dc096f30 con 0x7f60dc0966e0 2022-07-21T01:57:32.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e30c4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc0b2280 con 0x7f60dc095c60 2022-07-21T01:57:32.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e20c2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f60dc0966e0 0x7f60dc0983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:32.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e20c2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f60dc0966e0 0x7f60dc0983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40792/0 (socket says 172.21.15.140:40792) 2022-07-21T01:57:32.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e20c2700 1 -- 172.21.15.140:0/1878366046 learned_addr learned my addr 172.21.15.140:0/1878366046 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:32.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e28c3700 1 --2- 172.21.15.140:0/1878366046 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60dc095c60 0x7f60dc060cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:32.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e20c2700 1 -- 172.21.15.140:0/1878366046 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60dc095c60 msgr2=0x7f60dc060cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:32.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e20c2700 1 --2- 172.21.15.140:0/1878366046 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60dc095c60 0x7f60dc060cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:32.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e20c2700 1 -- 172.21.15.140:0/1878366046 >> v1:172.21.15.152:6789/0 conn(0x7f60dc0b1b80 legacy=0x7f60dc0988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:32.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e20c2700 1 -- 172.21.15.140:0/1878366046 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc096ac0 con 0x7f60dc0966e0 2022-07-21T01:57:32.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e10c0700 1 -- 172.21.15.140:0/1878366046 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482782792 0 0) 0x7f60dc0971d0 con 0x7f60dc0b1b80 2022-07-21T01:57:32.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e20c2700 1 --2- 172.21.15.140:0/1878366046 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60dc0966e0 0x7f60dc0983c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f60d8002f70 tx=0x7f60d8006eb0).ready entity=mon.0 client_cookie=ef56c6369689042 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:32.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e10c0700 1 -- 172.21.15.140:0/1878366046 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60d800d570 con 0x7f60dc0966e0 2022-07-21T01:57:32.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e10c0700 1 -- 172.21.15.140:0/1878366046 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d8000e90 con 0x7f60dc0966e0 2022-07-21T01:57:32.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.363+0000 7f60e10c0700 1 -- 172.21.15.140:0/1878366046 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60d800d8c0 con 0x7f60dc0966e0 2022-07-21T01:57:32.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.367+0000 7f60e30c4700 1 -- 172.21.15.140:0/1878366046 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60dc0966e0 msgr2=0x7f60dc0983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:32.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.367+0000 7f60e30c4700 1 --2- 172.21.15.140:0/1878366046 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60dc0966e0 0x7f60dc0983c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f60d8002f70 tx=0x7f60d8006eb0).stop 2022-07-21T01:57:32.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.367+0000 7f60e30c4700 1 -- 172.21.15.140:0/1878366046 shutdown_connections 2022-07-21T01:57:32.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.367+0000 7f60e30c4700 1 --2- 172.21.15.140:0/1878366046 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60dc095c60 0x7f60dc060cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:32.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.367+0000 7f60e30c4700 1 --2- 172.21.15.140:0/1878366046 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60dc0966e0 0x7f60dc0983c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:32.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.367+0000 7f60e30c4700 1 -- 172.21.15.140:0/1878366046 >> 172.21.15.140:0/1878366046 conn(0x7f60dc0b2bd0 msgr2=0x7f60dc0b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:32.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.367+0000 7f60e30c4700 1 -- 172.21.15.140:0/1878366046 shutdown_connections 2022-07-21T01:57:32.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.367+0000 7f60e30c4700 1 -- 172.21.15.140:0/1878366046 wait complete. 2022-07-21T01:57:32.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.367+0000 7f60e30c4700 1 Processor -- start 2022-07-21T01:57:32.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.367+0000 7f60e30c4700 1 -- start start 2022-07-21T01:57:32.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e30c4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60dc095c60 0x7f60dc0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:32.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e30c4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60dc0966e0 0x7f60dc140770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:32.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e30c4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60dc0b1b80 0x7f60dc0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:32.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e30c4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc141a40 con 0x7f60dc095c60 2022-07-21T01:57:32.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e20c2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60dc095c60 0x7f60dc0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:32.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e20c2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60dc095c60 0x7f60dc0ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40798/0 (socket says 172.21.15.140:40798) 2022-07-21T01:57:32.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e20c2700 1 -- 172.21.15.140:0/115879883 learned_addr learned my addr 172.21.15.140:0/115879883 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:32.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc142240 con 0x7f60dc0966e0 2022-07-21T01:57:32.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc09b2c0 con 0x7f60dc0b1b80 2022-07-21T01:57:32.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e28c3700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60dc0b1b80 0x7f60dc0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:32.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e18c1700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60dc0966e0 0x7f60dc140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:32.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e20c2700 1 -- 172.21.15.140:0/115879883 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60dc0b1b80 msgr2=0x7f60dc0afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:32.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e20c2700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60dc0b1b80 0x7f60dc0afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:32.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e20c2700 1 -- 172.21.15.140:0/115879883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60dc0966e0 msgr2=0x7f60dc140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:32.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e20c2700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60dc0966e0 0x7f60dc140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:32.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e20c2700 1 -- 172.21.15.140:0/115879883 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d800b1e0 con 0x7f60dc095c60 2022-07-21T01:57:32.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e20c2700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60dc095c60 0x7f60dc0ad8a0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f60dc1423d0 tx=0x7f60d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:32.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60cb7fe700 1 -- 172.21.15.140:0/115879883 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60d8011860 con 0x7f60dc095c60 2022-07-21T01:57:32.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60cb7fe700 1 -- 172.21.15.140:0/115879883 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d80119c0 con 0x7f60dc095c60 2022-07-21T01:57:32.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60dc1417a0 con 0x7f60dc095c60 2022-07-21T01:57:32.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60cb7fe700 1 -- 172.21.15.140:0/115879883 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60d8011d20 con 0x7f60dc095c60 2022-07-21T01:57:32.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60dc143150 con 0x7f60dc095c60 2022-07-21T01:57:32.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60dc0574e0 con 0x7f60dc095c60 2022-07-21T01:57:32.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60cb7fe700 1 -- 172.21.15.140:0/115879883 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f60d8025030 con 0x7f60dc095c60 2022-07-21T01:57:32.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.371+0000 7f60cb7fe700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053980 0x7f60bc055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:32.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.375+0000 7f60cb7fe700 1 -- 172.21.15.140:0/115879883 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f60d806cff0 con 0x7f60dc095c60 2022-07-21T01:57:32.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.375+0000 7f60e18c1700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053980 0x7f60bc055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:32.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.375+0000 7f60e18c1700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053980 0x7f60bc055e40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f60cc00b000 tx=0x7f60cc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:32.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.379+0000 7f60cb7fe700 1 -- 172.21.15.140:0/115879883 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f60d8044070 con 0x7f60dc095c60 2022-07-21T01:57:32.452 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:57:32.455 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:57:32.458 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:57:32.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:57:32.465 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:57:32.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:32.507+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f60dc06a6e0 con 0x7f60dc095c60 2022-07-21T01:57:33.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.235+0000 7f60cb7fe700 1 -- 172.21.15.140:0/115879883 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v135) v1 ==== 173+0+0 (secure 0 0 0) 0x7f60d8023020 con 0x7f60dc095c60 2022-07-21T01:57:33.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053980 msgr2=0x7f60bc055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053980 0x7f60bc055e40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f60cc00b000 tx=0x7f60cc009040).stop 2022-07-21T01:57:33.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60dc095c60 msgr2=0x7f60dc0ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60dc095c60 0x7f60dc0ad8a0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f60dc1423d0 tx=0x7f60d8015000).stop 2022-07-21T01:57:33.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 shutdown_connections 2022-07-21T01:57:33.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f60bc053980 0x7f60bc055e40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60dc0b1b80 0x7f60dc0afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60dc0966e0 0x7f60dc140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 --2- 172.21.15.140:0/115879883 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f60dc095c60 0x7f60dc0ad8a0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 >> 172.21.15.140:0/115879883 conn(0x7f60dc0b2bd0 msgr2=0x7f60dc139cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:33.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 shutdown_connections 2022-07-21T01:57:33.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.239+0000 7f60e30c4700 1 -- 172.21.15.140:0/115879883 wait complete. 2022-07-21T01:57:33.244 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:57:33.254 DEBUG:teuthology.orchestra.run.smithi140:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:57:33.257 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:57:33.260 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:57:33.262 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:57:33.265 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-07-21T01:57:33.286 INFO:teuthology.orchestra.run.smithi140.stdout:[global] 2022-07-21T01:57:33.287 INFO:teuthology.orchestra.run.smithi140.stdout: chdir = "" 2022-07-21T01:57:33.287 INFO:teuthology.orchestra.run.smithi140.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:57:33.287 INFO:teuthology.orchestra.run.smithi140.stdout: auth supported = cephx 2022-07-21T01:57:33.288 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.288 INFO:teuthology.orchestra.run.smithi140.stdout: filestore xattr use omap = true 2022-07-21T01:57:33.288 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.289 INFO:teuthology.orchestra.run.smithi140.stdout: mon clock drift allowed = 1.000 2022-07-21T01:57:33.289 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.290 INFO:teuthology.orchestra.run.smithi140.stdout: osd crush chooseleaf type = 0 2022-07-21T01:57:33.290 INFO:teuthology.orchestra.run.smithi140.stdout: auth debug = true 2022-07-21T01:57:33.291 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.291 INFO:teuthology.orchestra.run.smithi140.stdout: ms die on old message = true 2022-07-21T01:57:33.291 INFO:teuthology.orchestra.run.smithi140.stdout: ms die on bug = true 2022-07-21T01:57:33.292 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.292 INFO:teuthology.orchestra.run.smithi140.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:57:33.293 INFO:teuthology.orchestra.run.smithi140.stdout: mon pg warn max object skew = 0 2022-07-21T01:57:33.293 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.294 INFO:teuthology.orchestra.run.smithi140.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:57:33.294 INFO:teuthology.orchestra.run.smithi140.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:57:33.294 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.295 INFO:teuthology.orchestra.run.smithi140.stdout: osd pool default size = 2 2022-07-21T01:57:33.295 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.295 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd allow primary affinity = true 2022-07-21T01:57:33.296 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd allow pg remap = true 2022-07-21T01:57:33.296 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:57:33.296 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:57:33.296 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on no sortbitwise = false 2022-07-21T01:57:33.297 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:57:33.297 INFO:teuthology.orchestra.run.smithi140.stdout: mon warn on too few osds = false 2022-07-21T01:57:33.297 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:57:33.297 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:57:33.297 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.298 INFO:teuthology.orchestra.run.smithi140.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:57:33.298 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.298 INFO:teuthology.orchestra.run.smithi140.stdout: osd default data pool replay window = 5 2022-07-21T01:57:33.298 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.299 INFO:teuthology.orchestra.run.smithi140.stdout: mon allow pool delete = true 2022-07-21T01:57:33.299 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.299 INFO:teuthology.orchestra.run.smithi140.stdout: mon cluster log file level = debug 2022-07-21T01:57:33.299 INFO:teuthology.orchestra.run.smithi140.stdout: debug asserts on shutdown = true 2022-07-21T01:57:33.300 INFO:teuthology.orchestra.run.smithi140.stdout: mon health detail to clog = false 2022-07-21T01:57:33.300 INFO:teuthology.orchestra.run.smithi140.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:57:33.300 INFO:teuthology.orchestra.run.smithi140.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:57:33.300 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.300 INFO:teuthology.orchestra.run.smithi140.stdout:[osd] 2022-07-21T01:57:33.301 INFO:teuthology.orchestra.run.smithi140.stdout: osd journal size = 100 2022-07-21T01:57:33.301 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.301 INFO:teuthology.orchestra.run.smithi140.stdout: osd scrub load threshold = 5.0 2022-07-21T01:57:33.301 INFO:teuthology.orchestra.run.smithi140.stdout: osd scrub max interval = 600 2022-07-21T01:57:33.301 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.302 INFO:teuthology.orchestra.run.smithi140.stdout: osd recover clone overlap = true 2022-07-21T01:57:33.302 INFO:teuthology.orchestra.run.smithi140.stdout: osd recovery max chunk = 1048576 2022-07-21T01:57:33.302 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.302 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug shutdown = true 2022-07-21T01:57:33.303 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug op order = true 2022-07-21T01:57:33.303 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug verify stray on activate = true 2022-07-21T01:57:33.303 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.303 INFO:teuthology.orchestra.run.smithi140.stdout: osd open classes on start = true 2022-07-21T01:57:33.304 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug pg log writeout = true 2022-07-21T01:57:33.304 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.304 INFO:teuthology.orchestra.run.smithi140.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:57:33.304 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.304 INFO:teuthology.orchestra.run.smithi140.stdout: osd map max advance = 10 2022-07-21T01:57:33.305 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.305 INFO:teuthology.orchestra.run.smithi140.stdout: journal zero on create = true 2022-07-21T01:57:33.305 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.305 INFO:teuthology.orchestra.run.smithi140.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:57:33.306 INFO:teuthology.orchestra.run.smithi140.stdout: filestore apply finisher threads = 3 2022-07-21T01:57:33.306 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.306 INFO:teuthology.orchestra.run.smithi140.stdout: bdev debug aio = true 2022-07-21T01:57:33.306 INFO:teuthology.orchestra.run.smithi140.stdout: osd debug misdirected ops = true 2022-07-21T01:57:33.307 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:57:33.307 INFO:teuthology.orchestra.run.smithi140.stdout: debug osd = 20 2022-07-21T01:57:33.307 INFO:teuthology.orchestra.run.smithi140.stdout: osd objectstore = filestore 2022-07-21T01:57:33.307 INFO:teuthology.orchestra.run.smithi140.stdout: osd op complaint time = 180 2022-07-21T01:57:33.307 INFO:teuthology.orchestra.run.smithi140.stdout: osd sloppy crc = True 2022-07-21T01:57:33.308 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.308 INFO:teuthology.orchestra.run.smithi140.stdout:[mgr] 2022-07-21T01:57:33.308 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:57:33.308 INFO:teuthology.orchestra.run.smithi140.stdout: debug mgr = 20 2022-07-21T01:57:33.309 INFO:teuthology.orchestra.run.smithi140.stdout: debug mon = 20 2022-07-21T01:57:33.309 INFO:teuthology.orchestra.run.smithi140.stdout: debug auth = 20 2022-07-21T01:57:33.309 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:33.309 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:33.310 INFO:teuthology.orchestra.run.smithi140.stdout: mgr/telemetry/nag = false 2022-07-21T01:57:33.310 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.310 INFO:teuthology.orchestra.run.smithi140.stdout:[mon] 2022-07-21T01:57:33.310 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:57:33.310 INFO:teuthology.orchestra.run.smithi140.stdout: debug mon = 20 2022-07-21T01:57:33.311 INFO:teuthology.orchestra.run.smithi140.stdout: debug paxos = 20 2022-07-21T01:57:33.311 INFO:teuthology.orchestra.run.smithi140.stdout: debug auth = 20 2022-07-21T01:57:33.311 INFO:teuthology.orchestra.run.smithi140.stdout: mon data avail warn = 5 2022-07-21T01:57:33.311 INFO:teuthology.orchestra.run.smithi140.stdout: mon mgr mkfs grace = 240 2022-07-21T01:57:33.312 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:33.312 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd reporter subtree level = osd 2022-07-21T01:57:33.312 INFO:teuthology.orchestra.run.smithi140.stdout: mon osd prime pg temp = true 2022-07-21T01:57:33.312 INFO:teuthology.orchestra.run.smithi140.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:33.312 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.313 INFO:teuthology.orchestra.run.smithi140.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:57:33.313 INFO:teuthology.orchestra.run.smithi140.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:57:33.313 INFO:teuthology.orchestra.run.smithi140.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:57:33.313 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.314 INFO:teuthology.orchestra.run.smithi140.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:57:33.314 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:57:33.314 INFO:teuthology.orchestra.run.smithi140.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:57:33.314 INFO:teuthology.orchestra.run.smithi140.stdout: mon op complaint time = 120 2022-07-21T01:57:33.315 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:33.315 INFO:teuthology.orchestra.run.smithi140.stdout:[client] 2022-07-21T01:57:33.315 INFO:teuthology.orchestra.run.smithi140.stdout: rgw cache enabled = true 2022-07-21T01:57:33.315 INFO:teuthology.orchestra.run.smithi140.stdout: rgw enable ops log = true 2022-07-21T01:57:33.315 INFO:teuthology.orchestra.run.smithi140.stdout: rgw enable usage log = true 2022-07-21T01:57:33.316 INFO:teuthology.orchestra.run.smithi140.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:57:33.316 INFO:teuthology.orchestra.run.smithi140.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:57:33.316 INFO:teuthology.orchestra.run.smithi140.stdout: client mount timeout = 600 2022-07-21T01:57:33.316 INFO:teuthology.orchestra.run.smithi140.stdout: debug client = 20 2022-07-21T01:57:33.317 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:57:33.317 INFO:teuthology.orchestra.run.smithi140.stdout: rados mon op timeout = 15m 2022-07-21T01:57:33.317 INFO:teuthology.orchestra.run.smithi140.stdout: rados osd op timeout = 15m 2022-07-21T01:57:33.317 INFO:teuthology.orchestra.run.smithi140.stdout:[mon.a] 2022-07-21T01:57:33.318 INFO:teuthology.orchestra.run.smithi140.stdout:[mon.b] 2022-07-21T01:57:33.318 INFO:teuthology.orchestra.run.smithi140.stdout:[mon.c] 2022-07-21T01:57:33.318 INFO:teuthology.orchestra.run.smithi140.stdout:[mds.a] 2022-07-21T01:57:33.319 INFO:teuthology.orchestra.run.smithi140.stdout:[mds.b] 2022-07-21T01:57:33.319 INFO:teuthology.orchestra.run.smithi140.stdout:[mds] 2022-07-21T01:57:33.319 INFO:teuthology.orchestra.run.smithi140.stdout: debug mds = 20 2022-07-21T01:57:33.319 INFO:teuthology.orchestra.run.smithi140.stdout: debug ms = 1 2022-07-21T01:57:33.319 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal frag = True 2022-07-21T01:57:33.320 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal fragment size max = 10000 2022-07-21T01:57:33.320 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal merge size = 5 2022-07-21T01:57:33.320 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal split bits = 3 2022-07-21T01:57:33.320 INFO:teuthology.orchestra.run.smithi140.stdout: mds bal split size = 100 2022-07-21T01:57:33.321 INFO:teuthology.orchestra.run.smithi140.stdout: mds debug frag = True 2022-07-21T01:57:33.322 INFO:teuthology.orchestra.run.smithi140.stdout: mds debug scatterstat = True 2022-07-21T01:57:33.322 INFO:teuthology.orchestra.run.smithi140.stdout: mds op complaint time = 180 2022-07-21T01:57:33.322 INFO:teuthology.orchestra.run.smithi140.stdout: mds verify scatter = True 2022-07-21T01:57:33.323 INFO:teuthology.orchestra.run.smithi140.stdout: osd op complaint time = 180 2022-07-21T01:57:33.323 INFO:teuthology.orchestra.run.smithi140.stdout: rados mon op timeout = 15m 2022-07-21T01:57:33.323 INFO:teuthology.orchestra.run.smithi140.stdout: rados osd op timeout = 15m 2022-07-21T01:57:33.327 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2022-07-21T01:57:33.327 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2022-07-21T01:57:33.328 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:57:33.328 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2022-07-21T01:57:33.328 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.328 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2022-07-21T01:57:33.329 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.329 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2022-07-21T01:57:33.329 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.329 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2022-07-21T01:57:33.329 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2022-07-21T01:57:33.330 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.330 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2022-07-21T01:57:33.330 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2022-07-21T01:57:33.331 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.331 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:57:33.331 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2022-07-21T01:57:33.332 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.332 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:57:33.332 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:57:33.332 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.333 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2022-07-21T01:57:33.333 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.333 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2022-07-21T01:57:33.333 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2022-07-21T01:57:33.334 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:57:33.334 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:57:33.334 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2022-07-21T01:57:33.334 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:57:33.335 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2022-07-21T01:57:33.335 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:57:33.335 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:57:33.335 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.335 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:57:33.336 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.336 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2022-07-21T01:57:33.336 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.336 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2022-07-21T01:57:33.337 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.337 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2022-07-21T01:57:33.337 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2022-07-21T01:57:33.337 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2022-07-21T01:57:33.337 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:57:33.338 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:57:33.338 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.338 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2022-07-21T01:57:33.338 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2022-07-21T01:57:33.339 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.339 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2022-07-21T01:57:33.339 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2022-07-21T01:57:33.339 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.340 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2022-07-21T01:57:33.340 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2022-07-21T01:57:33.340 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.341 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2022-07-21T01:57:33.341 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2022-07-21T01:57:33.341 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2022-07-21T01:57:33.341 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.341 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2022-07-21T01:57:33.342 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2022-07-21T01:57:33.342 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.342 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:57:33.342 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.343 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2022-07-21T01:57:33.343 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.343 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2022-07-21T01:57:33.344 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.344 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:57:33.344 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2022-07-21T01:57:33.344 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.345 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2022-07-21T01:57:33.345 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2022-07-21T01:57:33.345 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:57:33.345 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2022-07-21T01:57:33.345 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = filestore 2022-07-21T01:57:33.346 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2022-07-21T01:57:33.346 INFO:teuthology.orchestra.run.smithi195.stdout: osd sloppy crc = True 2022-07-21T01:57:33.346 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.346 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2022-07-21T01:57:33.347 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:57:33.347 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2022-07-21T01:57:33.347 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2022-07-21T01:57:33.347 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2022-07-21T01:57:33.347 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:33.348 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:33.348 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2022-07-21T01:57:33.348 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.348 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2022-07-21T01:57:33.349 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:57:33.349 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2022-07-21T01:57:33.349 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2022-07-21T01:57:33.349 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2022-07-21T01:57:33.350 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2022-07-21T01:57:33.350 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2022-07-21T01:57:33.350 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:33.350 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2022-07-21T01:57:33.350 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2022-07-21T01:57:33.351 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:33.351 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.351 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:57:33.351 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:57:33.352 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:57:33.352 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.352 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:57:33.353 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:57:33.353 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:57:33.353 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2022-07-21T01:57:33.353 INFO:teuthology.orchestra.run.smithi195.stdout: 2022-07-21T01:57:33.354 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2022-07-21T01:57:33.354 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2022-07-21T01:57:33.354 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2022-07-21T01:57:33.354 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2022-07-21T01:57:33.355 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:57:33.355 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:57:33.355 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2022-07-21T01:57:33.355 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2022-07-21T01:57:33.356 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:57:33.356 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 15m 2022-07-21T01:57:33.356 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 15m 2022-07-21T01:57:33.356 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2022-07-21T01:57:33.356 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2022-07-21T01:57:33.357 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2022-07-21T01:57:33.357 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2022-07-21T01:57:33.357 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2022-07-21T01:57:33.357 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2022-07-21T01:57:33.357 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2022-07-21T01:57:33.358 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2022-07-21T01:57:33.358 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal frag = True 2022-07-21T01:57:33.358 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal fragment size max = 10000 2022-07-21T01:57:33.358 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal merge size = 5 2022-07-21T01:57:33.359 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal split bits = 3 2022-07-21T01:57:33.359 INFO:teuthology.orchestra.run.smithi195.stdout: mds bal split size = 100 2022-07-21T01:57:33.359 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2022-07-21T01:57:33.359 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = True 2022-07-21T01:57:33.360 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2022-07-21T01:57:33.360 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = True 2022-07-21T01:57:33.360 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2022-07-21T01:57:33.360 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 15m 2022-07-21T01:57:33.360 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 15m 2022-07-21T01:57:33.361 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2022-07-21T01:57:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2022-07-21T01:57:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:57:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2022-07-21T01:57:33.362 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.362 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2022-07-21T01:57:33.362 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.362 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2022-07-21T01:57:33.363 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.363 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2022-07-21T01:57:33.363 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2022-07-21T01:57:33.363 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.364 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2022-07-21T01:57:33.364 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2022-07-21T01:57:33.364 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.364 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:57:33.365 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2022-07-21T01:57:33.365 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.365 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:57:33.365 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:57:33.365 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.366 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2022-07-21T01:57:33.366 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.366 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2022-07-21T01:57:33.366 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2022-07-21T01:57:33.367 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:57:33.367 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:57:33.367 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2022-07-21T01:57:33.367 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:57:33.367 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2022-07-21T01:57:33.368 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:57:33.368 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:57:33.368 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.368 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:57:33.369 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.369 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2022-07-21T01:57:33.369 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.369 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2022-07-21T01:57:33.369 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.370 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2022-07-21T01:57:33.370 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2022-07-21T01:57:33.370 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2022-07-21T01:57:33.370 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:57:33.371 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:57:33.371 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.371 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2022-07-21T01:57:33.371 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2022-07-21T01:57:33.372 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.372 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2022-07-21T01:57:33.372 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2022-07-21T01:57:33.372 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.372 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2022-07-21T01:57:33.373 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2022-07-21T01:57:33.373 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.373 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2022-07-21T01:57:33.374 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2022-07-21T01:57:33.374 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2022-07-21T01:57:33.374 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.374 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2022-07-21T01:57:33.375 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2022-07-21T01:57:33.375 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.375 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:57:33.375 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.376 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2022-07-21T01:57:33.376 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.376 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2022-07-21T01:57:33.376 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.376 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:57:33.377 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2022-07-21T01:57:33.377 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.377 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2022-07-21T01:57:33.377 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2022-07-21T01:57:33.378 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:57:33.378 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2022-07-21T01:57:33.378 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = filestore 2022-07-21T01:57:33.378 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2022-07-21T01:57:33.379 INFO:teuthology.orchestra.run.smithi192.stdout: osd sloppy crc = True 2022-07-21T01:57:33.379 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.379 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2022-07-21T01:57:33.379 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:57:33.380 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2022-07-21T01:57:33.380 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2022-07-21T01:57:33.380 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2022-07-21T01:57:33.380 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:33.381 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:33.381 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2022-07-21T01:57:33.381 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.381 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2022-07-21T01:57:33.381 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:57:33.382 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2022-07-21T01:57:33.382 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2022-07-21T01:57:33.382 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2022-07-21T01:57:33.382 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2022-07-21T01:57:33.383 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2022-07-21T01:57:33.383 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:33.383 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2022-07-21T01:57:33.383 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2022-07-21T01:57:33.384 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:33.384 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.384 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:57:33.384 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:57:33.384 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:57:33.385 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.385 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:57:33.385 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:57:33.385 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:57:33.386 INFO:teuthology.orchestra.run.smithi192.stdout: mon op complaint time = 120 2022-07-21T01:57:33.386 INFO:teuthology.orchestra.run.smithi192.stdout: 2022-07-21T01:57:33.386 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2022-07-21T01:57:33.387 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2022-07-21T01:57:33.387 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2022-07-21T01:57:33.387 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2022-07-21T01:57:33.388 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:57:33.388 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:57:33.388 INFO:teuthology.orchestra.run.smithi192.stdout: client mount timeout = 600 2022-07-21T01:57:33.388 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2022-07-21T01:57:33.388 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:57:33.389 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2022-07-21T01:57:33.389 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2022-07-21T01:57:33.389 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2022-07-21T01:57:33.389 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2022-07-21T01:57:33.390 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2022-07-21T01:57:33.390 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2022-07-21T01:57:33.390 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2022-07-21T01:57:33.390 INFO:teuthology.orchestra.run.smithi192.stdout:[mds] 2022-07-21T01:57:33.390 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds = 20 2022-07-21T01:57:33.391 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2022-07-21T01:57:33.391 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal frag = True 2022-07-21T01:57:33.391 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal fragment size max = 10000 2022-07-21T01:57:33.391 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal merge size = 5 2022-07-21T01:57:33.392 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split bits = 3 2022-07-21T01:57:33.392 INFO:teuthology.orchestra.run.smithi192.stdout: mds bal split size = 100 2022-07-21T01:57:33.392 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug frag = True 2022-07-21T01:57:33.392 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug scatterstat = True 2022-07-21T01:57:33.392 INFO:teuthology.orchestra.run.smithi192.stdout: mds op complaint time = 180 2022-07-21T01:57:33.393 INFO:teuthology.orchestra.run.smithi192.stdout: mds verify scatter = True 2022-07-21T01:57:33.393 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2022-07-21T01:57:33.393 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 15m 2022-07-21T01:57:33.393 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 15m 2022-07-21T01:57:33.394 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2022-07-21T01:57:33.394 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2022-07-21T01:57:33.394 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:57:33.394 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2022-07-21T01:57:33.395 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.395 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2022-07-21T01:57:33.395 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.395 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2022-07-21T01:57:33.396 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.396 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2022-07-21T01:57:33.396 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2022-07-21T01:57:33.396 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.397 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2022-07-21T01:57:33.397 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2022-07-21T01:57:33.397 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.397 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:57:33.397 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2022-07-21T01:57:33.398 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.398 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:57:33.398 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:57:33.398 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.399 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2022-07-21T01:57:33.399 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.399 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2022-07-21T01:57:33.399 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2022-07-21T01:57:33.399 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:57:33.400 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:57:33.400 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2022-07-21T01:57:33.400 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:57:33.400 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2022-07-21T01:57:33.401 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:57:33.401 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:57:33.401 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.401 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:57:33.401 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.402 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2022-07-21T01:57:33.402 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.402 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2022-07-21T01:57:33.402 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.403 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2022-07-21T01:57:33.403 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2022-07-21T01:57:33.403 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2022-07-21T01:57:33.403 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:57:33.403 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:57:33.404 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.404 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2022-07-21T01:57:33.404 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2022-07-21T01:57:33.404 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.405 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2022-07-21T01:57:33.405 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2022-07-21T01:57:33.405 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.405 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2022-07-21T01:57:33.405 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2022-07-21T01:57:33.406 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.406 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2022-07-21T01:57:33.406 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2022-07-21T01:57:33.406 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2022-07-21T01:57:33.407 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.407 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2022-07-21T01:57:33.407 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2022-07-21T01:57:33.407 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.407 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:57:33.408 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.408 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2022-07-21T01:57:33.408 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.408 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2022-07-21T01:57:33.409 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.409 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:57:33.409 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2022-07-21T01:57:33.409 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.409 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2022-07-21T01:57:33.410 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2022-07-21T01:57:33.410 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:57:33.410 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2022-07-21T01:57:33.410 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = filestore 2022-07-21T01:57:33.411 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2022-07-21T01:57:33.411 INFO:teuthology.orchestra.run.smithi191.stdout: osd sloppy crc = True 2022-07-21T01:57:33.411 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.411 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2022-07-21T01:57:33.411 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:57:33.412 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2022-07-21T01:57:33.412 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2022-07-21T01:57:33.412 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2022-07-21T01:57:33.412 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:33.413 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:33.413 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2022-07-21T01:57:33.413 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.413 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2022-07-21T01:57:33.413 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:57:33.414 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2022-07-21T01:57:33.414 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2022-07-21T01:57:33.414 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2022-07-21T01:57:33.414 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2022-07-21T01:57:33.415 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2022-07-21T01:57:33.415 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:33.415 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2022-07-21T01:57:33.415 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2022-07-21T01:57:33.415 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:33.416 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.416 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:57:33.416 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:57:33.416 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:57:33.417 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.417 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:57:33.417 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:57:33.417 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:57:33.418 INFO:teuthology.orchestra.run.smithi191.stdout: mon op complaint time = 120 2022-07-21T01:57:33.418 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-07-21T01:57:33.418 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2022-07-21T01:57:33.418 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2022-07-21T01:57:33.418 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2022-07-21T01:57:33.419 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2022-07-21T01:57:33.419 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:57:33.419 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:57:33.419 INFO:teuthology.orchestra.run.smithi191.stdout: client mount timeout = 600 2022-07-21T01:57:33.420 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2022-07-21T01:57:33.420 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:57:33.420 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 15m 2022-07-21T01:57:33.420 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 15m 2022-07-21T01:57:33.420 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2022-07-21T01:57:33.421 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.b] 2022-07-21T01:57:33.421 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.c] 2022-07-21T01:57:33.421 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2022-07-21T01:57:33.421 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2022-07-21T01:57:33.422 INFO:teuthology.orchestra.run.smithi191.stdout:[mds] 2022-07-21T01:57:33.422 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds = 20 2022-07-21T01:57:33.422 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-07-21T01:57:33.422 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal frag = True 2022-07-21T01:57:33.422 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal fragment size max = 10000 2022-07-21T01:57:33.423 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal merge size = 5 2022-07-21T01:57:33.423 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal split bits = 3 2022-07-21T01:57:33.423 INFO:teuthology.orchestra.run.smithi191.stdout: mds bal split size = 100 2022-07-21T01:57:33.423 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug frag = True 2022-07-21T01:57:33.424 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug scatterstat = True 2022-07-21T01:57:33.424 INFO:teuthology.orchestra.run.smithi191.stdout: mds op complaint time = 180 2022-07-21T01:57:33.424 INFO:teuthology.orchestra.run.smithi191.stdout: mds verify scatter = True 2022-07-21T01:57:33.424 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2022-07-21T01:57:33.424 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 15m 2022-07-21T01:57:33.425 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 15m 2022-07-21T01:57:33.425 INFO:teuthology.orchestra.run.smithi152.stdout:[global] 2022-07-21T01:57:33.425 INFO:teuthology.orchestra.run.smithi152.stdout: chdir = "" 2022-07-21T01:57:33.425 INFO:teuthology.orchestra.run.smithi152.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-07-21T01:57:33.426 INFO:teuthology.orchestra.run.smithi152.stdout: auth supported = cephx 2022-07-21T01:57:33.426 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.426 INFO:teuthology.orchestra.run.smithi152.stdout: filestore xattr use omap = true 2022-07-21T01:57:33.426 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.427 INFO:teuthology.orchestra.run.smithi152.stdout: mon clock drift allowed = 1.000 2022-07-21T01:57:33.427 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.427 INFO:teuthology.orchestra.run.smithi152.stdout: osd crush chooseleaf type = 0 2022-07-21T01:57:33.427 INFO:teuthology.orchestra.run.smithi152.stdout: auth debug = true 2022-07-21T01:57:33.428 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.428 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on old message = true 2022-07-21T01:57:33.428 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on bug = true 2022-07-21T01:57:33.428 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.428 INFO:teuthology.orchestra.run.smithi152.stdout: mon max pg per osd = 10000 # >= luminous 2022-07-21T01:57:33.429 INFO:teuthology.orchestra.run.smithi152.stdout: mon pg warn max object skew = 0 2022-07-21T01:57:33.429 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.429 INFO:teuthology.orchestra.run.smithi152.stdout: # disable pg_autoscaler by default for new pools 2022-07-21T01:57:33.429 INFO:teuthology.orchestra.run.smithi152.stdout: osd_pool_default_pg_autoscale_mode = off 2022-07-21T01:57:33.430 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.430 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default size = 2 2022-07-21T01:57:33.430 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.430 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow primary affinity = true 2022-07-21T01:57:33.430 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow pg remap = true 2022-07-21T01:57:33.431 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on legacy crush tunables = false 2022-07-21T01:57:33.431 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on crush straw calc version zero = false 2022-07-21T01:57:33.432 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on no sortbitwise = false 2022-07-21T01:57:33.432 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on osd down out interval zero = false 2022-07-21T01:57:33.432 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on too few osds = false 2022-07-21T01:57:33.432 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-07-21T01:57:33.433 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_no_redundancy = false 2022-07-21T01:57:33.433 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.433 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-07-21T01:57:33.433 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.433 INFO:teuthology.orchestra.run.smithi152.stdout: osd default data pool replay window = 5 2022-07-21T01:57:33.434 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.434 INFO:teuthology.orchestra.run.smithi152.stdout: mon allow pool delete = true 2022-07-21T01:57:33.434 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.434 INFO:teuthology.orchestra.run.smithi152.stdout: mon cluster log file level = debug 2022-07-21T01:57:33.435 INFO:teuthology.orchestra.run.smithi152.stdout: debug asserts on shutdown = true 2022-07-21T01:57:33.435 INFO:teuthology.orchestra.run.smithi152.stdout: mon health detail to clog = false 2022-07-21T01:57:33.435 INFO:teuthology.orchestra.run.smithi152.stdout: mon host = "172.21.15.140,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2022-07-21T01:57:33.435 INFO:teuthology.orchestra.run.smithi152.stdout: fsid = 87d2f829-64cd-4d49-9e42-a1dfafceefb8 2022-07-21T01:57:33.435 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.436 INFO:teuthology.orchestra.run.smithi152.stdout:[osd] 2022-07-21T01:57:33.436 INFO:teuthology.orchestra.run.smithi152.stdout: osd journal size = 100 2022-07-21T01:57:33.436 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.436 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub load threshold = 5.0 2022-07-21T01:57:33.437 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub max interval = 600 2022-07-21T01:57:33.437 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.437 INFO:teuthology.orchestra.run.smithi152.stdout: osd recover clone overlap = true 2022-07-21T01:57:33.437 INFO:teuthology.orchestra.run.smithi152.stdout: osd recovery max chunk = 1048576 2022-07-21T01:57:33.437 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.438 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug shutdown = true 2022-07-21T01:57:33.438 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug op order = true 2022-07-21T01:57:33.438 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify stray on activate = true 2022-07-21T01:57:33.438 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.439 INFO:teuthology.orchestra.run.smithi152.stdout: osd open classes on start = true 2022-07-21T01:57:33.439 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug pg log writeout = true 2022-07-21T01:57:33.439 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.439 INFO:teuthology.orchestra.run.smithi152.stdout: osd deep scrub update digest min age = 30 2022-07-21T01:57:33.439 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.440 INFO:teuthology.orchestra.run.smithi152.stdout: osd map max advance = 10 2022-07-21T01:57:33.440 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.440 INFO:teuthology.orchestra.run.smithi152.stdout: journal zero on create = true 2022-07-21T01:57:33.440 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.440 INFO:teuthology.orchestra.run.smithi152.stdout: filestore ondisk finisher threads = 3 2022-07-21T01:57:33.441 INFO:teuthology.orchestra.run.smithi152.stdout: filestore apply finisher threads = 3 2022-07-21T01:57:33.441 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.441 INFO:teuthology.orchestra.run.smithi152.stdout: bdev debug aio = true 2022-07-21T01:57:33.441 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug misdirected ops = true 2022-07-21T01:57:33.442 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:57:33.442 INFO:teuthology.orchestra.run.smithi152.stdout: debug osd = 20 2022-07-21T01:57:33.442 INFO:teuthology.orchestra.run.smithi152.stdout: osd objectstore = filestore 2022-07-21T01:57:33.442 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2022-07-21T01:57:33.442 INFO:teuthology.orchestra.run.smithi152.stdout: osd sloppy crc = True 2022-07-21T01:57:33.443 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.443 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr] 2022-07-21T01:57:33.443 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:57:33.443 INFO:teuthology.orchestra.run.smithi152.stdout: debug mgr = 20 2022-07-21T01:57:33.444 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2022-07-21T01:57:33.444 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2022-07-21T01:57:33.444 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:33.444 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:33.445 INFO:teuthology.orchestra.run.smithi152.stdout: mgr/telemetry/nag = false 2022-07-21T01:57:33.445 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.445 INFO:teuthology.orchestra.run.smithi152.stdout:[mon] 2022-07-21T01:57:33.445 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:57:33.445 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2022-07-21T01:57:33.446 INFO:teuthology.orchestra.run.smithi152.stdout: debug paxos = 20 2022-07-21T01:57:33.446 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2022-07-21T01:57:33.446 INFO:teuthology.orchestra.run.smithi152.stdout: mon data avail warn = 5 2022-07-21T01:57:33.446 INFO:teuthology.orchestra.run.smithi152.stdout: mon mgr mkfs grace = 240 2022-07-21T01:57:33.447 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2022-07-21T01:57:33.447 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd reporter subtree level = osd 2022-07-21T01:57:33.447 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd prime pg temp = true 2022-07-21T01:57:33.447 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2022-07-21T01:57:33.447 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-07-21T01:57:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: auth mon ticket ttl = 660 # 11m 2022-07-21T01:57:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: auth service ticket ttl = 240 # 4m 2022-07-21T01:57:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.448 INFO:teuthology.orchestra.run.smithi152.stdout: # don't complain about insecure global_id in the test suite 2022-07-21T01:57:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-07-21T01:57:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-07-21T01:57:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: mon op complaint time = 120 2022-07-21T01:57:33.449 INFO:teuthology.orchestra.run.smithi152.stdout: 2022-07-21T01:57:33.450 INFO:teuthology.orchestra.run.smithi152.stdout:[client] 2022-07-21T01:57:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: rgw cache enabled = true 2022-07-21T01:57:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable ops log = true 2022-07-21T01:57:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable usage log = true 2022-07-21T01:57:33.450 INFO:teuthology.orchestra.run.smithi152.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-07-21T01:57:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-07-21T01:57:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: client mount timeout = 600 2022-07-21T01:57:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: debug client = 20 2022-07-21T01:57:33.451 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:57:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 15m 2022-07-21T01:57:33.452 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 15m 2022-07-21T01:57:33.452 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.a] 2022-07-21T01:57:33.452 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.b] 2022-07-21T01:57:33.452 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.c] 2022-07-21T01:57:33.453 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.a] 2022-07-21T01:57:33.453 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.b] 2022-07-21T01:57:33.453 INFO:teuthology.orchestra.run.smithi152.stdout:[mds] 2022-07-21T01:57:33.453 INFO:teuthology.orchestra.run.smithi152.stdout: debug mds = 20 2022-07-21T01:57:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2022-07-21T01:57:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal frag = True 2022-07-21T01:57:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal fragment size max = 10000 2022-07-21T01:57:33.454 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal merge size = 5 2022-07-21T01:57:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal split bits = 3 2022-07-21T01:57:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: mds bal split size = 100 2022-07-21T01:57:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug frag = True 2022-07-21T01:57:33.455 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug scatterstat = True 2022-07-21T01:57:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: mds op complaint time = 180 2022-07-21T01:57:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: mds verify scatter = True 2022-07-21T01:57:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2022-07-21T01:57:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 15m 2022-07-21T01:57:33.456 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 15m 2022-07-21T01:57:33.457 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-07-21T01:57:33.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.567+0000 7f1d8b59e700 1 Processor -- start 2022-07-21T01:57:33.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.567+0000 7f1d8b59e700 1 -- start start 2022-07-21T01:57:33.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.567+0000 7f1d8b59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d8c0b5990 0x7f1d8c0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:33.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.567+0000 7f1d8b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d8c0af8e0 0x7f1d8c146960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:33.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.567+0000 7f1d8b59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d8c142a90 con 0x7f1d8c142ef0 2022-07-21T01:57:33.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.567+0000 7f1d8b59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8c146e50 con 0x7f1d8c0b5990 2022-07-21T01:57:33.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.567+0000 7f1d8b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c0b5d70 con 0x7f1d8c0af8e0 2022-07-21T01:57:33.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d89d9b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d8c0af8e0 0x7f1d8c146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:33.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8ad9d700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1d8c142ef0 0x7f1d8c1433a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:56070/0 (socket says 172.21.15.140:56070) 2022-07-21T01:57:33.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8a59c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1d8c0b5990 0x7f1d8c0ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:33.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8ad9d700 1 -- 172.21.15.140:0/3944425642 learned_addr learned my addr 172.21.15.140:0/3944425642 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:33.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8959a700 1 -- 172.21.15.140:0/3944425642 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515798416 0 0) 0x7f1d8c142a90 con 0x7f1d8c142ef0 2022-07-21T01:57:33.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8959a700 1 -- 172.21.15.140:0/3944425642 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d6c003260 con 0x7f1d8c142ef0 2022-07-21T01:57:33.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d89d9b700 1 -- 172.21.15.140:0/3944425642 >> v2:172.21.15.152:3300/0 conn(0x7f1d8c0b5990 msgr2=0x7f1d8c0ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:33.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d89d9b700 1 --2- 172.21.15.140:0/3944425642 >> v2:172.21.15.152:3300/0 conn(0x7f1d8c0b5990 0x7f1d8c0ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d89d9b700 1 -- 172.21.15.140:0/3944425642 >> v1:172.21.15.152:6789/0 conn(0x7f1d8c142ef0 legacy=0x7f1d8c1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d89d9b700 1 -- 172.21.15.140:0/3944425642 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c0afdf0 con 0x7f1d8c0af8e0 2022-07-21T01:57:33.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d89d9b700 1 --2- 172.21.15.140:0/3944425642 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d8c0af8e0 0x7f1d8c146960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f1d7c0032f0 tx=0x7f1d7c0091d0).ready entity=mon.2 client_cookie=a9fa795b4e8360d9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:33.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8959a700 1 -- 172.21.15.140:0/3944425642 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d7c00b600 con 0x7f1d8c0af8e0 2022-07-21T01:57:33.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8959a700 1 -- 172.21.15.140:0/3944425642 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c00b760 con 0x7f1d8c0af8e0 2022-07-21T01:57:33.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8959a700 1 -- 172.21.15.140:0/3944425642 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d7c00b8d0 con 0x7f1d8c0af8e0 2022-07-21T01:57:33.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8b59e700 1 -- 172.21.15.140:0/3944425642 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d8c0af8e0 msgr2=0x7f1d8c146960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8b59e700 1 --2- 172.21.15.140:0/3944425642 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d8c0af8e0 0x7f1d8c146960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f1d7c0032f0 tx=0x7f1d7c0091d0).stop 2022-07-21T01:57:33.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8b59e700 1 -- 172.21.15.140:0/3944425642 shutdown_connections 2022-07-21T01:57:33.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8b59e700 1 --2- 172.21.15.140:0/3944425642 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d8c0af8e0 0x7f1d8c146960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8b59e700 1 --2- 172.21.15.140:0/3944425642 >> v2:172.21.15.152:3300/0 conn(0x7f1d8c0b5990 0x7f1d8c0ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8b59e700 1 -- 172.21.15.140:0/3944425642 >> 172.21.15.140:0/3944425642 conn(0x7f1d8c063480 msgr2=0x7f1d8c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:33.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8b59e700 1 -- 172.21.15.140:0/3944425642 shutdown_connections 2022-07-21T01:57:33.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8b59e700 1 -- 172.21.15.140:0/3944425642 wait complete. 2022-07-21T01:57:33.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.571+0000 7f1d8b59e700 1 Processor -- start 2022-07-21T01:57:33.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8b59e700 1 -- start start 2022-07-21T01:57:33.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d8c0af8e0 0x7f1d8c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:33.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8c0b5990 0x7f1d8c05c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:33.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d8c142ef0 0x7f1d8c115650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:33.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c0676b0 con 0x7f1d8c0b5990 2022-07-21T01:57:33.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c113a20 con 0x7f1d8c142ef0 2022-07-21T01:57:33.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c113cc0 con 0x7f1d8c0af8e0 2022-07-21T01:57:33.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d89d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8c0b5990 0x7f1d8c05c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:33.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d89d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8c0b5990 0x7f1d8c05c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40814/0 (socket says 172.21.15.140:40814) 2022-07-21T01:57:33.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d89d9b700 1 -- 172.21.15.140:0/2866875740 learned_addr learned my addr 172.21.15.140:0/2866875740 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:33.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8a59c700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d8c0af8e0 0x7f1d8c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:33.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8ad9d700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d8c142ef0 0x7f1d8c115650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:33.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d89d9b700 1 -- 172.21.15.140:0/2866875740 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d8c0af8e0 msgr2=0x7f1d8c1558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d89d9b700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d8c0af8e0 0x7f1d8c1558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d89d9b700 1 -- 172.21.15.140:0/2866875740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d8c142ef0 msgr2=0x7f1d8c115650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d89d9b700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d8c142ef0 0x7f1d8c115650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d89d9b700 1 -- 172.21.15.140:0/2866875740 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c009470 con 0x7f1d8c0b5990 2022-07-21T01:57:33.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d89d9b700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8c0b5990 0x7f1d8c05c3d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f1d7c0032f0 tx=0x7f1d7c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:33.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d837fe700 1 -- 172.21.15.140:0/2866875740 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d7c011b60 con 0x7f1d8c0b5990 2022-07-21T01:57:33.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d837fe700 1 -- 172.21.15.140:0/2866875740 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c011cc0 con 0x7f1d8c0b5990 2022-07-21T01:57:33.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8b59e700 1 -- 172.21.15.140:0/2866875740 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8c112da0 con 0x7f1d8c0b5990 2022-07-21T01:57:33.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d837fe700 1 -- 172.21.15.140:0/2866875740 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d7c01f5e0 con 0x7f1d8c0b5990 2022-07-21T01:57:33.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8b59e700 1 -- 172.21.15.140:0/2866875740 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8c1134a0 con 0x7f1d8c0b5990 2022-07-21T01:57:33.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.575+0000 7f1d8b59e700 1 -- 172.21.15.140:0/2866875740 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8c113e90 con 0x7f1d8c0b5990 2022-07-21T01:57:33.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.579+0000 7f1d837fe700 1 -- 172.21.15.140:0/2866875740 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d7c026030 con 0x7f1d8c0b5990 2022-07-21T01:57:33.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.579+0000 7f1d837fe700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d70053a70 0x7f1d70055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:33.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.579+0000 7f1d837fe700 1 -- 172.21.15.140:0/2866875740 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f1d7c06c830 con 0x7f1d8c0b5990 2022-07-21T01:57:33.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.579+0000 7f1d8a59c700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d70053a70 0x7f1d70055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:33.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.579+0000 7f1d8a59c700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d70053a70 0x7f1d70055f30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f1d74006000 tx=0x7f1d74006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:33.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.587+0000 7f1d837fe700 1 -- 172.21.15.140:0/2866875740 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d7c043930 con 0x7f1d8c0b5990 2022-07-21T01:57:33.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.711+0000 7f1d8b59e700 1 -- 172.21.15.140:0/2866875740 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1d8c113830 con 0x7f1d8c0b5990 2022-07-21T01:57:33.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d837fe700 1 -- 172.21.15.140:0/2866875740 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1d7c0240a0 con 0x7f1d8c0b5990 2022-07-21T01:57:33.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 -- 172.21.15.140:0/2866875740 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d70053a70 msgr2=0x7f1d70055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d70053a70 0x7f1d70055f30 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f1d74006000 tx=0x7f1d74006d90).stop 2022-07-21T01:57:33.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 -- 172.21.15.140:0/2866875740 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8c0b5990 msgr2=0x7f1d8c05c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8c0b5990 0x7f1d8c05c3d0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f1d7c0032f0 tx=0x7f1d7c016000).stop 2022-07-21T01:57:33.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 -- 172.21.15.140:0/2866875740 shutdown_connections 2022-07-21T01:57:33.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1d70053a70 0x7f1d70055f30 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d8c142ef0 0x7f1d8c115650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d8c0b5990 0x7f1d8c05c3d0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 --2- 172.21.15.140:0/2866875740 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d8c0af8e0 0x7f1d8c1558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 -- 172.21.15.140:0/2866875740 >> 172.21.15.140:0/2866875740 conn(0x7f1d8c063480 msgr2=0x7f1d8c0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:33.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 -- 172.21.15.140:0/2866875740 shutdown_connections 2022-07-21T01:57:33.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.719+0000 7f1d8b59e700 1 -- 172.21.15.140:0/2866875740 wait complete. 2022-07-21T01:57:33.734 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2022-07-21T01:57:33.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3c4f14700 1 Processor -- start 2022-07-21T01:57:33.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3c4f14700 1 -- start start 2022-07-21T01:57:33.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3c4f14700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff3c00b5b30 0x7ff3c00aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:33.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3c4f14700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c00b5f10 con 0x7ff3c00ace90 2022-07-21T01:57:33.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3c4f14700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c00afc40 con 0x7ff3c00b13e0 2022-07-21T01:57:33.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3c4f14700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c00afee0 con 0x7ff3c00b5b30 2022-07-21T01:57:33.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3bdd9b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff3c00ace90 0x7ff3c00b0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54966/0 (socket says 172.21.15.140:54966) 2022-07-21T01:57:33.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3bdd9b700 1 -- 172.21.15.140:0/394136071 learned_addr learned my addr 172.21.15.140:0/394136071 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:33.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3be59c700 1 --2- 172.21.15.140:0/394136071 >> v2:172.21.15.152:3300/0 conn(0x7ff3c00b5b30 0x7ff3c00aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:33.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3bd59a700 1 -- 172.21.15.140:0/394136071 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287440204 0 0) 0x7ff3c00b5f10 con 0x7ff3c00ace90 2022-07-21T01:57:33.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3bd59a700 1 -- 172.21.15.140:0/394136071 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a4003260 con 0x7ff3c00ace90 2022-07-21T01:57:33.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3bd59a700 1 -- 172.21.15.140:0/394136071 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356775207 0 0) 0x7ff3c00afc40 con 0x7ff3c00b13e0 2022-07-21T01:57:33.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3bd59a700 1 -- 172.21.15.140:0/394136071 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c00b5f10 con 0x7ff3c00b13e0 2022-07-21T01:57:33.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3bd59a700 1 -- 172.21.15.140:0/394136071 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2696214659 0 0) 0x7ff3a4003260 con 0x7ff3c00ace90 2022-07-21T01:57:33.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3be59c700 1 -- 172.21.15.140:0/394136071 >> v1:172.21.15.152:6789/0 conn(0x7ff3c00b13e0 legacy=0x7ff3c01463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3be59c700 1 -- 172.21.15.140:0/394136071 >> v1:172.21.15.140:6789/0 conn(0x7ff3c00ace90 legacy=0x7ff3c00b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3be59c700 1 -- 172.21.15.140:0/394136071 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c00ad490 con 0x7ff3c00b5b30 2022-07-21T01:57:33.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.827+0000 7ff3be59c700 1 --2- 172.21.15.140:0/394136071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c00b5b30 0x7ff3c00aefb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7ff3a8001110 tx=0x7ff3b0006df0).ready entity=mon.1 client_cookie=2f43660ea6b59f78 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:33.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3bd59a700 1 -- 172.21.15.140:0/394136071 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a8001660 con 0x7ff3c00b5b30 2022-07-21T01:57:33.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3bd59a700 1 -- 172.21.15.140:0/394136071 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b0000e90 con 0x7ff3c00b5b30 2022-07-21T01:57:33.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3bd59a700 1 -- 172.21.15.140:0/394136071 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3b0012460 con 0x7ff3c00b5b30 2022-07-21T01:57:33.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3c4f14700 1 -- 172.21.15.140:0/394136071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c00b5b30 msgr2=0x7ff3c00aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3c4f14700 1 --2- 172.21.15.140:0/394136071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c00b5b30 0x7ff3c00aefb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7ff3a8001110 tx=0x7ff3b0006df0).stop 2022-07-21T01:57:33.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3c4f14700 1 -- 172.21.15.140:0/394136071 shutdown_connections 2022-07-21T01:57:33.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3c4f14700 1 --2- 172.21.15.140:0/394136071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c00b5b30 0x7ff3c00aefb0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3c4f14700 1 -- 172.21.15.140:0/394136071 >> 172.21.15.140:0/394136071 conn(0x7ff3c00635b0 msgr2=0x7ff3c00b50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:33.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3c4f14700 1 -- 172.21.15.140:0/394136071 shutdown_connections 2022-07-21T01:57:33.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3c4f14700 1 -- 172.21.15.140:0/394136071 wait complete. 2022-07-21T01:57:33.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3c4f14700 1 Processor -- start 2022-07-21T01:57:33.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.831+0000 7ff3c4f14700 1 -- start start 2022-07-21T01:57:33.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3c4f14700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c00ace90 0x7ff3c014d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:33.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3c4f14700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c00b13e0 0x7ff3c0144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:33.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3c4f14700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3c00b5b30 0x7ff3c0159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:33.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3c4f14700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c014c6b0 con 0x7ff3c00ace90 2022-07-21T01:57:33.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3c4f14700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c014c950 con 0x7ff3c00b13e0 2022-07-21T01:57:33.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3c4f14700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c014cbf0 con 0x7ff3c00b5b30 2022-07-21T01:57:33.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3be59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c00ace90 0x7ff3c014d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:33.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3be59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c00ace90 0x7ff3c014d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40828/0 (socket says 172.21.15.140:40828) 2022-07-21T01:57:33.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3be59c700 1 -- 172.21.15.140:0/2921836776 learned_addr learned my addr 172.21.15.140:0/2921836776 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:33.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3bdd9b700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c00b13e0 0x7ff3c0144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:33.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3bed9d700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3c00b5b30 0x7ff3c0159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:33.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3be59c700 1 -- 172.21.15.140:0/2921836776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3c00b5b30 msgr2=0x7ff3c0159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3be59c700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3c00b5b30 0x7ff3c0159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3be59c700 1 -- 172.21.15.140:0/2921836776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c00b13e0 msgr2=0x7ff3c0144e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:33.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3be59c700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c00b13e0 0x7ff3c0144e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:33.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3be59c700 1 -- 172.21.15.140:0/2921836776 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b000b180 con 0x7ff3c00ace90 2022-07-21T01:57:33.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3be59c700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c00ace90 0x7ff3c014d090 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7ff3a8001110 tx=0x7ff3b0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:33.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3b77fe700 1 -- 172.21.15.140:0/2921836776 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3b000d8a0 con 0x7ff3c00ace90 2022-07-21T01:57:33.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3b77fe700 1 -- 172.21.15.140:0/2921836776 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b000da00 con 0x7ff3c00ace90 2022-07-21T01:57:33.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3c4f14700 1 -- 172.21.15.140:0/2921836776 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c0143970 con 0x7ff3c00ace90 2022-07-21T01:57:33.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3b77fe700 1 -- 172.21.15.140:0/2921836776 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3b000dd60 con 0x7ff3c00ace90 2022-07-21T01:57:33.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3c4f14700 1 -- 172.21.15.140:0/2921836776 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c014bd90 con 0x7ff3c00ace90 2022-07-21T01:57:33.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.835+0000 7ff3c4f14700 1 -- 172.21.15.140:0/2921836776 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c0067910 con 0x7ff3c00ace90 2022-07-21T01:57:33.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.839+0000 7ff3b77fe700 1 -- 172.21.15.140:0/2921836776 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff3b0025030 con 0x7ff3c00ace90 2022-07-21T01:57:33.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.839+0000 7ff3b77fe700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff398053b30 0x7ff398055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:33.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.839+0000 7ff3b77fe700 1 -- 172.21.15.140:0/2921836776 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5339+0+0 (secure 0 0 0) 0x7ff3b006c780 con 0x7ff3c00ace90 2022-07-21T01:57:33.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.839+0000 7ff3bdd9b700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff398053b30 0x7ff398055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:33.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.839+0000 7ff3bdd9b700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff398053b30 0x7ff398055ff0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7ff3a800d000 tx=0x7ff3a8000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:33.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:33.843+0000 7ff3b77fe700 1 -- 172.21.15.140:0/2921836776 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff3c0067910 con 0x7ff3c00ace90 2022-07-21T01:57:34.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.011+0000 7ff3c4f14700 1 -- 172.21.15.140:0/2921836776 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7ff3c0067910 con 0x7ff3c00ace90 2022-07-21T01:57:34.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.235+0000 7ff3b77fe700 1 -- 172.21.15.140:0/2921836776 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v421) v1 ==== 128+0+0 (secure 0 0 0) 0x7ff3c0067910 con 0x7ff3c00ace90 2022-07-21T01:57:34.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 -- 172.21.15.140:0/2921836776 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff398053b30 msgr2=0x7ff398055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:34.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff398053b30 0x7ff398055ff0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7ff3a800d000 tx=0x7ff3a8000bc0).stop 2022-07-21T01:57:34.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 -- 172.21.15.140:0/2921836776 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c00ace90 msgr2=0x7ff3c014d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:34.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c00ace90 0x7ff3c014d090 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7ff3a8001110 tx=0x7ff3b0014000).stop 2022-07-21T01:57:34.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 -- 172.21.15.140:0/2921836776 shutdown_connections 2022-07-21T01:57:34.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff398053b30 0x7ff398055ff0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:34.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff3c00b5b30 0x7ff3c0159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:34.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff3c00b13e0 0x7ff3c0144e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:34.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 --2- 172.21.15.140:0/2921836776 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff3c00ace90 0x7ff3c014d090 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:34.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 -- 172.21.15.140:0/2921836776 >> 172.21.15.140:0/2921836776 conn(0x7ff3c00635b0 msgr2=0x7ff3c0148660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:34.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 -- 172.21.15.140:0/2921836776 shutdown_connections 2022-07-21T01:57:34.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.239+0000 7ff3c4f14700 1 -- 172.21.15.140:0/2921836776 wait complete. 2022-07-21T01:57:34.260 INFO:tasks.cephfs_test_runner:test_getattr_caps (tasks.cephfs.test_misc.TestMisc) ... ok 2022-07-21T01:57:34.261 INFO:tasks.cephfs_test_runner:Starting test: test_root_rctime (tasks.cephfs.test_misc.TestMisc) 2022-07-21T01:57:34.261 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2022-07-21T01:57:34.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 Processor -- start 2022-07-21T01:57:34.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 -- start start 2022-07-21T01:57:34.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40b1e00 0x7f28f414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:34.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f28f40acef0 0x7f28f4146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:34.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f405d480 con 0x7f28f405c3f0 2022-07-21T01:57:34.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f40b21e0 con 0x7f28f40acef0 2022-07-21T01:57:34.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f405cf40 con 0x7f28f40b1e00 2022-07-21T01:57:34.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f37fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40b1e00 0x7f28f414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:34.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f37fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40b1e00 0x7f28f414a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:37566/0 (socket says 172.21.15.140:37566) 2022-07-21T01:57:34.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f37fe700 1 -- 172.21.15.140:0/3098048241 learned_addr learned my addr 172.21.15.140:0/3098048241 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:34.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f2ffd700 1 --1- 172.21.15.140:0/3098048241 >> v1:172.21.15.152:6789/0 conn(0x7f28f405c3f0 0x7f28f40ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:56094/0 (socket says 172.21.15.140:56094) 2022-07-21T01:57:34.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f3fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f28f40acef0 0x7f28f4146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:34.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f37fe700 1 -- 172.21.15.140:0/3098048241 >> v2:172.21.15.152:3300/0 conn(0x7f28f40acef0 msgr2=0x7f28f4146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:34.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f37fe700 1 --2- 172.21.15.140:0/3098048241 >> v2:172.21.15.152:3300/0 conn(0x7f28f40acef0 0x7f28f4146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:34.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f37fe700 1 -- 172.21.15.140:0/3098048241 >> v1:172.21.15.152:6789/0 conn(0x7f28f405c3f0 legacy=0x7f28f40ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:34.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f27fc700 1 -- 172.21.15.140:0/3098048241 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4063555686 0 0) 0x7f28f405d480 con 0x7f28f405c3f0 2022-07-21T01:57:34.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f37fe700 1 -- 172.21.15.140:0/3098048241 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f405d1e0 con 0x7f28f40b1e00 2022-07-21T01:57:34.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f37fe700 1 --2- 172.21.15.140:0/3098048241 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40b1e00 0x7f28f414a6d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f28dc002f70 tx=0x7f28dc006eb0).ready entity=mon.2 client_cookie=b283134c2cc63c44 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:34.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f27fc700 1 -- 172.21.15.140:0/3098048241 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28dc00d570 con 0x7f28f40b1e00 2022-07-21T01:57:34.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f27fc700 1 -- 172.21.15.140:0/3098048241 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc000e90 con 0x7f28f40b1e00 2022-07-21T01:57:34.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28f27fc700 1 -- 172.21.15.140:0/3098048241 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28dc00d8c0 con 0x7f28f40b1e00 2022-07-21T01:57:34.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 -- 172.21.15.140:0/3098048241 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40b1e00 msgr2=0x7f28f414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:34.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 --2- 172.21.15.140:0/3098048241 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40b1e00 0x7f28f414a6d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f28dc002f70 tx=0x7f28dc006eb0).stop 2022-07-21T01:57:34.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 -- 172.21.15.140:0/3098048241 shutdown_connections 2022-07-21T01:57:34.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 --2- 172.21.15.140:0/3098048241 >> v2:172.21.15.152:3300/0 conn(0x7f28f40acef0 0x7f28f4146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:34.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 --2- 172.21.15.140:0/3098048241 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40b1e00 0x7f28f414a6d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:34.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 -- 172.21.15.140:0/3098048241 >> 172.21.15.140:0/3098048241 conn(0x7f28f40611c0 msgr2=0x7f28f4062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:34.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 -- 172.21.15.140:0/3098048241 shutdown_connections 2022-07-21T01:57:34.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.367+0000 7f28fa186700 1 -- 172.21.15.140:0/3098048241 wait complete. 2022-07-21T01:57:34.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28fa186700 1 Processor -- start 2022-07-21T01:57:34.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28fa186700 1 -- start start 2022-07-21T01:57:34.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28fa186700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40acef0 0x7f28f414c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:34.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28fa186700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f40b1e00 0x7f28f4159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:34.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28fa186700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f415a290 0x7f28f415a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:34.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28fa186700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f415ac30 con 0x7f28f415a290 2022-07-21T01:57:34.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28fa186700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f4149520 con 0x7f28f40b1e00 2022-07-21T01:57:34.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28fa186700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f4149b00 con 0x7f28f40acef0 2022-07-21T01:57:34.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28f37fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40acef0 0x7f28f414c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:34.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28f3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f415a290 0x7f28f415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:34.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28f2ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f40b1e00 0x7f28f4159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:34.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28f37fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40acef0 0x7f28f414c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:37570/0 (socket says 172.21.15.140:37570) 2022-07-21T01:57:34.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.371+0000 7f28f37fe700 1 -- 172.21.15.140:0/3727369525 learned_addr learned my addr 172.21.15.140:0/3727369525 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:34.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28f3fff700 1 -- 172.21.15.140:0/3727369525 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40acef0 msgr2=0x7f28f414c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:34.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28f3fff700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40acef0 0x7f28f414c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:34.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28f3fff700 1 -- 172.21.15.140:0/3727369525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f40b1e00 msgr2=0x7f28f4159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:34.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28f3fff700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f40b1e00 0x7f28f4159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:34.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28f3fff700 1 -- 172.21.15.140:0/3727369525 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc00b1e0 con 0x7f28f415a290 2022-07-21T01:57:34.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28f37fe700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40acef0 0x7f28f414c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:34.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28f3fff700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f415a290 0x7f28f415a740 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f28e400c000 tx=0x7f28e4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:34.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28f0ff9700 1 -- 172.21.15.140:0/3727369525 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28e4011750 con 0x7f28f415a290 2022-07-21T01:57:34.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28f0ff9700 1 -- 172.21.15.140:0/3727369525 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e40118b0 con 0x7f28f415a290 2022-07-21T01:57:34.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28fa186700 1 -- 172.21.15.140:0/3727369525 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f4149da0 con 0x7f28f415a290 2022-07-21T01:57:34.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28f0ff9700 1 -- 172.21.15.140:0/3727369525 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28e4011c10 con 0x7f28f415a290 2022-07-21T01:57:34.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28fa186700 1 -- 172.21.15.140:0/3727369525 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f41432a0 con 0x7f28f415a290 2022-07-21T01:57:34.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28fa186700 1 -- 172.21.15.140:0/3727369525 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f40b04b0 con 0x7f28f415a290 2022-07-21T01:57:34.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.375+0000 7f28f0ff9700 1 -- 172.21.15.140:0/3727369525 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f28e4011d70 con 0x7f28f415a290 2022-07-21T01:57:34.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.379+0000 7f28f0ff9700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28d8053b30 0x7f28d8055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:34.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.379+0000 7f28f37fe700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28d8053b30 0x7f28d8055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:34.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.379+0000 7f28f0ff9700 1 -- 172.21.15.140:0/3727369525 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f28e406bf60 con 0x7f28f415a290 2022-07-21T01:57:34.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.379+0000 7f28f37fe700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28d8053b30 0x7f28d8055ff0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f28dc01c000 tx=0x7f28dc015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:34.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.383+0000 7f28f0ff9700 1 -- 172.21.15.140:0/3727369525 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f28e4043050 con 0x7f28f415a290 2022-07-21T01:57:34.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:34.547+0000 7f28fa186700 1 -- 172.21.15.140:0/3727369525 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f28f40574e0 con 0x7f28f415a290 2022-07-21T01:57:35.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.239+0000 7f28f0ff9700 1 -- 172.21.15.140:0/3727369525 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v422) v1 ==== 130+0+0 (secure 0 0 0) 0x7f28e4046670 con 0x7f28f415a290 2022-07-21T01:57:35.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 -- 172.21.15.140:0/3727369525 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28d8053b30 msgr2=0x7f28d8055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28d8053b30 0x7f28d8055ff0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f28dc01c000 tx=0x7f28dc015000).stop 2022-07-21T01:57:35.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 -- 172.21.15.140:0/3727369525 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f415a290 msgr2=0x7f28f415a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f415a290 0x7f28f415a740 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f28e400c000 tx=0x7f28e4006df0).stop 2022-07-21T01:57:35.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 -- 172.21.15.140:0/3727369525 shutdown_connections 2022-07-21T01:57:35.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f28d8053b30 0x7f28d8055ff0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f28f415a290 0x7f28f415a740 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f28f40b1e00 0x7f28f4159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 --2- 172.21.15.140:0/3727369525 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f28f40acef0 0x7f28f414c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 -- 172.21.15.140:0/3727369525 >> 172.21.15.140:0/3727369525 conn(0x7f28f405c3f0 msgr2=0x7f28f4069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:35.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 -- 172.21.15.140:0/3727369525 shutdown_connections 2022-07-21T01:57:35.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.243+0000 7f28fa186700 1 -- 172.21.15.140:0/3727369525 wait complete. 2022-07-21T01:57:35.262 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:57:35.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.367+0000 7fe18d7f4700 1 Processor -- start 2022-07-21T01:57:35.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.367+0000 7fe18d7f4700 1 -- start start 2022-07-21T01:57:35.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe18d7f4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe18805cf40 0x7fe1880afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe18d7f4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe1880b1220 0x7fe18814a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe18d7f4700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1881472b0 con 0x7fe188147c90 2022-07-21T01:57:35.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe18d7f4700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1880b0920 con 0x7fe1880b1220 2022-07-21T01:57:35.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe18d7f4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe188147550 con 0x7fe18805cf40 2022-07-21T01:57:35.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1867fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe188147c90 0x7fe1880b0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:54994/0 (socket says 172.21.15.140:54994) 2022-07-21T01:57:35.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1867fc700 1 -- 172.21.15.140:0/1197297657 learned_addr learned my addr 172.21.15.140:0/1197297657 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:35.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe186ffd700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe18805cf40 0x7fe1880afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1877fe700 1 --2- 172.21.15.140:0/1197297657 >> v2:172.21.15.152:3300/0 conn(0x7fe1880b1220 0x7fe18814a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157364623 0 0) 0x7fe1881472b0 con 0x7fe188147c90 2022-07-21T01:57:35.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe16c003260 con 0x7fe188147c90 2022-07-21T01:57:35.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1856596213 0 0) 0x7fe16c003260 con 0x7fe188147c90 2022-07-21T01:57:35.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe18805cf40 msgr2=0x7fe1880afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:35.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe18805cf40 0x7fe1880afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 >> v2:172.21.15.152:3300/0 conn(0x7fe1880b1220 msgr2=0x7fe18814a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:35.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 --2- 172.21.15.140:0/1197297657 >> v2:172.21.15.152:3300/0 conn(0x7fe1880b1220 0x7fe18814a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1881477f0 con 0x7fe188147c90 2022-07-21T01:57:35.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fe1780019e0 con 0x7fe188147c90 2022-07-21T01:57:35.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 >> v1:172.21.15.140:6789/0 conn(0x7fe188147c90 legacy=0x7fe1880b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe18805cf40 0x7fe16c003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe16c003c80 0x7fe16c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 0x7fe16c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1880b0920 con 0x7fe16c0049f0 2022-07-21T01:57:35.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe188147550 con 0x7fe16c003c80 2022-07-21T01:57:35.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c00b750 con 0x7fe18805cf40 2022-07-21T01:57:35.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1877fe700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 0x7fe16c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1867fc700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe16c003c80 0x7fe16c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe186ffd700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe18805cf40 0x7fe16c003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1877fe700 1 -- 172.21.15.140:0/1197297657 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe18805cf40 msgr2=0x7fe16c003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:35.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1877fe700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe18805cf40 0x7fe16c003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1877fe700 1 -- 172.21.15.140:0/1197297657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe16c003c80 msgr2=0x7fe16c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:35.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1877fe700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe16c003c80 0x7fe16c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1877fe700 1 -- 172.21.15.140:0/1197297657 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c003260 con 0x7fe16c0049f0 2022-07-21T01:57:35.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1877fe700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 0x7fe16c004e70 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7fe16c0054d0 tx=0x7fe180006df0).ready entity=mon.0 client_cookie=3db4e8ed697e64c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:35.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe18000e800 con 0x7fe16c0049f0 2022-07-21T01:57:35.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18000e960 con 0x7fe16c0049f0 2022-07-21T01:57:35.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe185ffb700 1 -- 172.21.15.140:0/1197297657 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe18000ecc0 con 0x7fe16c0049f0 2022-07-21T01:57:35.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe18d7f4700 1 -- 172.21.15.140:0/1197297657 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 msgr2=0x7fe16c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe18d7f4700 1 --2- 172.21.15.140:0/1197297657 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 0x7fe16c004e70 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7fe16c0054d0 tx=0x7fe180006df0).stop 2022-07-21T01:57:35.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe1877fe700 1 -- 172.21.15.140:0/1197297657 reap_dead start 2022-07-21T01:57:35.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe18d7f4700 1 -- 172.21.15.140:0/1197297657 shutdown_connections 2022-07-21T01:57:35.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe18d7f4700 1 -- 172.21.15.140:0/1197297657 >> 172.21.15.140:0/1197297657 conn(0x7fe1880b5b70 msgr2=0x7fe1880b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:35.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe18d7f4700 1 -- 172.21.15.140:0/1197297657 shutdown_connections 2022-07-21T01:57:35.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.371+0000 7fe18d7f4700 1 -- 172.21.15.140:0/1197297657 wait complete. 2022-07-21T01:57:35.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe18d7f4700 1 Processor -- start 2022-07-21T01:57:35.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe18d7f4700 1 -- start start 2022-07-21T01:57:35.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe18d7f4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 0x7fe18814bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe18d7f4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe188063d30 0x7fe1880641b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe18d7f4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1880646a0 0x7fe188098fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe18d7f4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe188143780 con 0x7fe16c0049f0 2022-07-21T01:57:35.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe18d7f4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe188141660 con 0x7fe1880646a0 2022-07-21T01:57:35.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe18d7f4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe188141900 con 0x7fe188063d30 2022-07-21T01:57:35.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe186ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 0x7fe18814bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe186ffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 0x7fe18814bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40860/0 (socket says 172.21.15.140:40860) 2022-07-21T01:57:35.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe186ffd700 1 -- 172.21.15.140:0/2178447099 learned_addr learned my addr 172.21.15.140:0/2178447099 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:35.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe1867fc700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe188063d30 0x7fe1880641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe1877fe700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1880646a0 0x7fe188098fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe186ffd700 1 -- 172.21.15.140:0/2178447099 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe188063d30 msgr2=0x7fe1880641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe186ffd700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe188063d30 0x7fe1880641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe186ffd700 1 -- 172.21.15.140:0/2178447099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1880646a0 msgr2=0x7fe188098fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe186ffd700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1880646a0 0x7fe188098fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe186ffd700 1 -- 172.21.15.140:0/2178447099 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe188064b20 con 0x7fe16c0049f0 2022-07-21T01:57:35.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe186ffd700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 0x7fe18814bf80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7fe17c00b000 tx=0x7fe17c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:35.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe177fff700 1 -- 172.21.15.140:0/2178447099 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe17c012610 con 0x7fe16c0049f0 2022-07-21T01:57:35.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe177fff700 1 -- 172.21.15.140:0/2178447099 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c012770 con 0x7fe16c0049f0 2022-07-21T01:57:35.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe18d7f4700 1 -- 172.21.15.140:0/2178447099 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe188143470 con 0x7fe16c0049f0 2022-07-21T01:57:35.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe177fff700 1 -- 172.21.15.140:0/2178447099 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe17c012ad0 con 0x7fe16c0049f0 2022-07-21T01:57:35.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.375+0000 7fe18d7f4700 1 -- 172.21.15.140:0/2178447099 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18809afc0 con 0x7fe16c0049f0 2022-07-21T01:57:35.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.379+0000 7fe18d7f4700 1 -- 172.21.15.140:0/2178447099 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1881414c0 con 0x7fe16c0049f0 2022-07-21T01:57:35.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.379+0000 7fe177fff700 1 -- 172.21.15.140:0/2178447099 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe17c012c30 con 0x7fe16c0049f0 2022-07-21T01:57:35.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.379+0000 7fe177fff700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe164053b00 0x7fe164055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.379+0000 7fe177fff700 1 -- 172.21.15.140:0/2178447099 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fe17c06c430 con 0x7fe16c0049f0 2022-07-21T01:57:35.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.379+0000 7fe1867fc700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe164053b00 0x7fe164055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.379+0000 7fe1867fc700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe164053b00 0x7fe164055fc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fe178005000 tx=0x7fe1880b22c0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:35.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.383+0000 7fe177fff700 1 -- 172.21.15.140:0/2178447099 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe1881414c0 con 0x7fe16c0049f0 2022-07-21T01:57:35.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.511+0000 7fe18d7f4700 1 -- 172.21.15.140:0/2178447099 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe1881414c0 con 0x7fe16c0049f0 2022-07-21T01:57:35.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.519+0000 7fe177fff700 1 -- 172.21.15.140:0/2178447099 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe1881414c0 con 0x7fe16c0049f0 2022-07-21T01:57:35.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 -- 172.21.15.140:0/2178447099 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe164053b00 msgr2=0x7fe164055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe164053b00 0x7fe164055fc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fe178005000 tx=0x7fe1880b22c0).stop 2022-07-21T01:57:35.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 -- 172.21.15.140:0/2178447099 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 msgr2=0x7fe18814bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 0x7fe18814bf80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7fe17c00b000 tx=0x7fe17c008040).stop 2022-07-21T01:57:35.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 -- 172.21.15.140:0/2178447099 shutdown_connections 2022-07-21T01:57:35.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe164053b00 0x7fe164055fc0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe1880646a0 0x7fe188098fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe188063d30 0x7fe1880641b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 --2- 172.21.15.140:0/2178447099 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe16c0049f0 0x7fe18814bf80 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 -- 172.21.15.140:0/2178447099 >> 172.21.15.140:0/2178447099 conn(0x7fe1880b5b70 msgr2=0x7fe188145ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:35.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 -- 172.21.15.140:0/2178447099 shutdown_connections 2022-07-21T01:57:35.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.523+0000 7fe18d7f4700 1 -- 172.21.15.140:0/2178447099 wait complete. 2022-07-21T01:57:35.538 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:57:35.539 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:35.557 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:35.580 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:35.602 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:35.624 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:35.645 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:57:35.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:35.647 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:35.647 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:35.654 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:57:35.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:35.656 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:57:35.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:35.657 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:35.658 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:35.665 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:57:35.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:35.667 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:57:35.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:35.668 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:35.668 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:57:35.676 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:57:35.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:35.678 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:57:35.678 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:57:35.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.783+0000 7f8717066700 1 Processor -- start 2022-07-21T01:57:35.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.783+0000 7f8717066700 1 -- start start 2022-07-21T01:57:35.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8717066700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f87100b1e00 0x7f871014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8717066700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87100acef0 0x7f8710146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8717066700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871005d480 con 0x7f871005c3f0 2022-07-21T01:57:35.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8717066700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87100b21e0 con 0x7f87100b1e00 2022-07-21T01:57:35.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8717066700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871005cf40 con 0x7f87100acef0 2022-07-21T01:57:35.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8715603700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87100acef0 0x7f8710146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f870ffff700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f871005c3f0 0x7f87100ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:56128/0 (socket says 172.21.15.140:56128) 2022-07-21T01:57:35.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f870ffff700 1 -- 172.21.15.140:0/3225979551 learned_addr learned my addr 172.21.15.140:0/3225979551 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:35.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8714e02700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f87100b1e00 0x7f871014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f870f7fe700 1 -- 172.21.15.140:0/3225979551 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915247488 0 0) 0x7f871005d480 con 0x7f871005c3f0 2022-07-21T01:57:35.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f870f7fe700 1 -- 172.21.15.140:0/3225979551 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc003260 con 0x7f871005c3f0 2022-07-21T01:57:35.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8715603700 1 -- 172.21.15.140:0/3225979551 >> v2:172.21.15.152:3300/0 conn(0x7f87100b1e00 msgr2=0x7f871014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:35.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8715603700 1 --2- 172.21.15.140:0/3225979551 >> v2:172.21.15.152:3300/0 conn(0x7f87100b1e00 0x7f871014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8715603700 1 -- 172.21.15.140:0/3225979551 >> v1:172.21.15.152:6789/0 conn(0x7f871005c3f0 legacy=0x7f87100ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8715603700 1 -- 172.21.15.140:0/3225979551 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871005d1e0 con 0x7f87100acef0 2022-07-21T01:57:35.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8714e02700 1 --2- 172.21.15.140:0/3225979551 >> v2:172.21.15.152:3300/0 conn(0x7f87100b1e00 0x7f871014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:35.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8715603700 1 --2- 172.21.15.140:0/3225979551 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87100acef0 0x7f8710146410 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f8708002f70 tx=0x7f8708006df0).ready entity=mon.2 client_cookie=fd4cde588e03a6b0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:35.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f870f7fe700 1 -- 172.21.15.140:0/3225979551 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f870800d4a0 con 0x7f87100acef0 2022-07-21T01:57:35.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f870f7fe700 1 -- 172.21.15.140:0/3225979551 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8708000e90 con 0x7f87100acef0 2022-07-21T01:57:35.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f870f7fe700 1 -- 172.21.15.140:0/3225979551 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f870800d7f0 con 0x7f87100acef0 2022-07-21T01:57:35.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8717066700 1 -- 172.21.15.140:0/3225979551 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87100acef0 msgr2=0x7f8710146410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8717066700 1 --2- 172.21.15.140:0/3225979551 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87100acef0 0x7f8710146410 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f8708002f70 tx=0x7f8708006df0).stop 2022-07-21T01:57:35.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8717066700 1 -- 172.21.15.140:0/3225979551 shutdown_connections 2022-07-21T01:57:35.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8717066700 1 --2- 172.21.15.140:0/3225979551 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87100acef0 0x7f8710146410 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8717066700 1 --2- 172.21.15.140:0/3225979551 >> v2:172.21.15.152:3300/0 conn(0x7f87100b1e00 0x7f871014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.787+0000 7f8717066700 1 -- 172.21.15.140:0/3225979551 >> 172.21.15.140:0/3225979551 conn(0x7f87100611c0 msgr2=0x7f8710062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:35.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.791+0000 7f8717066700 1 -- 172.21.15.140:0/3225979551 shutdown_connections 2022-07-21T01:57:35.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.791+0000 7f8717066700 1 -- 172.21.15.140:0/3225979551 wait complete. 2022-07-21T01:57:35.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.791+0000 7f8717066700 1 Processor -- start 2022-07-21T01:57:35.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.791+0000 7f8717066700 1 -- start start 2022-07-21T01:57:35.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8717066700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87100acef0 0x7f871009e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8717066700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87100b1e00 0x7f8710159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8717066700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f871015a0c0 0x7f871015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8717066700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87101479f0 con 0x7f87100b1e00 2022-07-21T01:57:35.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8717066700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871015aa40 con 0x7f87100acef0 2022-07-21T01:57:35.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8717066700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87101569f0 con 0x7f871015a0c0 2022-07-21T01:57:35.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f870ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87100b1e00 0x7f8710159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8714e02700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87100acef0 0x7f871009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f870ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87100b1e00 0x7f8710159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40876/0 (socket says 172.21.15.140:40876) 2022-07-21T01:57:35.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f870ffff700 1 -- 172.21.15.140:0/1634509201 learned_addr learned my addr 172.21.15.140:0/1634509201 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:35.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8715603700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f871015a0c0 0x7f871015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8714e02700 1 -- 172.21.15.140:0/1634509201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f871015a0c0 msgr2=0x7f871015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8714e02700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f871015a0c0 0x7f871015a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8714e02700 1 -- 172.21.15.140:0/1634509201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87100b1e00 msgr2=0x7f8710159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:35.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8714e02700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87100b1e00 0x7f8710159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:35.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8714e02700 1 -- 172.21.15.140:0/1634509201 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870400b180 con 0x7f87100acef0 2022-07-21T01:57:35.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f870ffff700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87100b1e00 0x7f8710159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:35.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8714e02700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87100acef0 0x7f871009e220 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f870400c000 tx=0x7f8704006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:35.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f870dffb700 1 -- 172.21.15.140:0/1634509201 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8704010780 con 0x7f87100acef0 2022-07-21T01:57:35.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f870dffb700 1 -- 172.21.15.140:0/1634509201 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87040108e0 con 0x7f87100acef0 2022-07-21T01:57:35.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8717066700 1 -- 172.21.15.140:0/1634509201 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870800b180 con 0x7f87100acef0 2022-07-21T01:57:35.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f870dffb700 1 -- 172.21.15.140:0/1634509201 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8704010c40 con 0x7f87100acef0 2022-07-21T01:57:35.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8717066700 1 -- 172.21.15.140:0/1634509201 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871009e710 con 0x7f87100acef0 2022-07-21T01:57:35.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.795+0000 7f8717066700 1 -- 172.21.15.140:0/1634509201 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87100b04b0 con 0x7f87100acef0 2022-07-21T01:57:35.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.799+0000 7f870dffb700 1 -- 172.21.15.140:0/1634509201 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f870401c460 con 0x7f87100acef0 2022-07-21T01:57:35.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.799+0000 7f870dffb700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86f0053aa0 0x7f86f0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:35.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.799+0000 7f870dffb700 1 -- 172.21.15.140:0/1634509201 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f870406c070 con 0x7f87100acef0 2022-07-21T01:57:35.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.799+0000 7f870ffff700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86f0053aa0 0x7f86f0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:35.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.799+0000 7f870ffff700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86f0053aa0 0x7f86f0055f60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f870000d000 tx=0x7f8700009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:35.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.807+0000 7f870dffb700 1 -- 172.21.15.140:0/1634509201 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8704043170 con 0x7f87100acef0 2022-07-21T01:57:35.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.931+0000 7f8717066700 1 -- 172.21.15.140:0/1634509201 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f87100574e0 con 0x7f87100acef0 2022-07-21T01:57:35.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.935+0000 7f870dffb700 1 -- 172.21.15.140:0/1634509201 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v135) v1 ==== 81+0+28935 (secure 0 0 0) 0x7f8704046790 con 0x7f87100acef0 2022-07-21T01:57:35.938 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:35.938 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:57:35.939 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 135, 2022-07-21T01:57:35.939 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:57:35.939 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:57:35.939 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:57:33.222726+0000", 2022-07-21T01:57:35.940 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:35.940 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:35.940 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:57:35.940 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:57:35.941 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:57:35.941 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:57:35.941 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:57:35.941 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:57:35.942 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:57:35.942 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:35.942 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:57:35.942 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:57:35.942 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:57:35.943 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:57:35.943 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:57:35.943 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 26, 2022-07-21T01:57:35.943 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:57:35.944 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:57:35.944 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:57:35.944 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:57:35.944 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:57:35.945 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.945 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:57:35.945 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:57:35.945 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:57:35.946 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:35.946 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:35.946 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:35.946 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:35.946 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:35.947 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:35.947 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:35.947 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:35.947 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:57:35.948 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:57:35.948 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:57:35.948 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:57:35.948 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:57:35.949 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:35.949 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:35.949 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:35.949 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:35.950 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:35.950 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:35.950 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:35.950 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.950 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:57:35.951 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:35.951 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:35.951 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:35.951 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:35.952 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:35.952 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:35.952 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:35.952 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:35.953 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:35.953 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:35.953 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:35.953 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:35.953 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:35.954 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:35.954 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:35.954 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:35.954 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:35.955 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:35.955 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:35.955 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:35.955 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:35.956 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:35.956 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:35.956 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:35.956 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:35.957 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:35.957 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.957 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:35.957 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:35.957 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:35.958 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:35.958 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:35.958 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:35.958 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:35.959 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:35.959 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:35.959 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:35.959 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:35.960 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:57:35.960 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:57:35.960 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.960 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:35.960 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:57:35.961 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:35.961 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.961 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.961 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:57:35.962 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:57:35.962 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:57:35.962 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:35.962 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:35.963 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:35.963 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:35.963 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:35.963 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:35.963 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:35.964 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:35.964 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:35.964 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:35.964 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:35.965 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:35.965 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:35.965 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:35.965 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:35.965 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:35.966 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:35.966 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:35.966 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:35.966 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:35.967 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.967 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:57:35.967 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:35.967 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:35.968 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:35.968 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:35.968 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:35.968 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:35.968 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:35.969 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:35.969 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:35.969 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:35.969 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:35.970 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:35.970 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:35.970 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:35.970 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:35.971 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:35.971 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:35.971 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:35.971 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:35.971 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:35.972 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:35.972 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:35.972 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:35.972 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:35.973 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:35.973 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:35.973 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.973 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:35.974 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:35.974 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:35.974 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:35.974 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:35.975 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:35.975 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:35.975 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:35.975 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:35.976 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:35.976 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:35.976 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:57:35.976 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:35.977 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:57:35.977 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:35.977 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:35.977 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:35.977 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:57:35.978 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.978 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:57:35.978 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:57:35.978 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:35.979 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:35.979 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:35.979 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:35.979 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:35.980 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:35.980 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:35.980 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 129, 2022-07-21T01:57:35.980 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:35.980 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:35.981 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:35.981 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:35.981 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.981 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:35.982 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:57:35.982 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:35.982 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.982 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.983 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:35.983 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:57:35.983 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:35.983 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:35.984 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:35.984 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.984 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:35.984 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:35.984 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.985 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:35.985 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:57:35.985 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:35.985 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.986 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.986 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:35.986 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:57:35.986 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:35.987 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:35.987 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:35.987 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.987 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:35.987 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:35.988 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.988 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:35.988 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:57:35.988 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:35.989 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.989 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.989 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:35.989 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:57:35.989 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:35.990 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:35.990 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:35.990 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.990 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:35.991 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:35.991 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.991 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:35.991 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:57:35.992 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:35.992 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.992 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.992 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:35.993 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:57:35.993 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:35.993 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:35.993 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:35.993 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.994 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:57:35.994 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:57:35.994 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:57:35.994 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:57:35.995 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:35.995 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:35.995 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:35.995 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:35.995 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:35.996 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.996 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:57:35.996 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:57:35.996 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:35.997 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:35.997 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:35.997 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:35.997 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:35.997 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:35.998 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:35.998 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 130, 2022-07-21T01:57:35.998 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:35.998 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:35.999 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:35.999 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:35.999 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:35.999 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.000 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:57:36.000 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:36.000 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.000 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.000 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.001 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:57:36.001 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:36.001 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.001 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.002 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.002 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:36.002 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.002 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.003 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.003 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:57:36.003 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:36.003 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.003 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.004 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.004 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:57:36.004 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:36.004 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.005 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.005 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.005 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:36.005 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.006 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.006 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.006 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:57:36.006 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:36.007 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.007 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.007 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.007 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:57:36.008 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:36.008 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.008 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.008 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.008 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:36.009 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.009 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.009 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.009 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:57:36.010 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:36.010 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.010 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.010 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.010 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:57:36.011 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:36.011 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.011 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.011 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.012 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:57:36.012 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:57:36.012 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:57:36.012 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:57:36.013 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:36.013 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:36.013 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:36.013 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.013 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.014 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.014 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:57:36.014 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:57:36.014 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:36.015 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:36.015 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:36.015 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:36.015 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:36.015 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:36.016 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:36.016 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 130, 2022-07-21T01:57:36.016 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:36.016 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:36.017 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:36.017 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.017 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.017 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.017 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:57:36.018 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:36.018 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.018 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.018 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.019 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:57:36.019 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:36.019 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.019 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.020 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.020 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:36.020 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.020 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.020 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.021 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:57:36.021 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:36.021 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.021 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.022 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.022 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:57:36.022 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:36.022 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.023 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.023 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.023 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:36.023 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.024 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.024 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.024 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:57:36.024 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:36.024 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.025 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.025 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.025 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:57:36.025 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:36.025 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.026 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.026 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.026 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:36.027 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.027 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.027 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.027 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:57:36.027 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:36.028 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.028 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.028 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.028 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:57:36.029 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:36.029 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.029 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.029 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.030 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:57:36.030 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:57:36.030 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:57:36.030 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:57:36.030 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:36.031 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:36.031 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:36.031 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.031 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.032 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.032 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:57:36.032 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:57:36.032 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:36.033 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:36.033 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:36.033 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:36.033 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:36.034 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:36.034 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:36.034 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 129, 2022-07-21T01:57:36.034 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:36.035 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:36.035 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:36.035 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.035 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.035 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.036 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:57:36.036 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:36.036 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.036 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.037 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.037 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:57:36.037 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:36.037 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.037 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.038 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.038 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:36.038 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.038 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.039 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.039 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:57:36.039 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:36.039 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.039 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.040 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.040 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:57:36.040 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:36.040 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.041 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.041 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.041 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:36.041 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.041 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.042 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.042 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:57:36.042 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:36.042 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.043 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.043 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.043 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:57:36.043 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:36.043 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.044 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.044 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.044 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:36.044 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.045 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.045 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.045 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:57:36.045 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:36.046 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.046 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.046 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.046 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:57:36.046 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:36.047 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.047 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.047 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.047 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:57:36.048 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:57:36.048 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:57:36.048 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:57:36.048 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:36.049 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:36.049 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:36.049 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.049 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.049 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.050 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:57:36.050 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:57:36.050 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:36.050 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:36.051 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:36.051 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:36.051 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:36.051 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:36.051 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:36.052 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 130, 2022-07-21T01:57:36.052 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:36.052 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:36.052 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:36.053 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.053 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.053 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.053 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:57:36.053 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:36.054 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.054 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.054 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.054 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:57:36.055 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:36.055 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.055 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.055 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.056 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:36.056 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.056 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.056 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.057 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:57:36.057 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:36.057 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.057 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.057 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.058 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:57:36.058 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:36.058 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.058 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.059 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.059 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:36.059 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.059 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.060 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.060 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:57:36.060 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:36.060 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.061 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.061 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.061 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:57:36.061 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:36.061 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.062 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.062 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.062 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:36.063 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.063 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.063 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.063 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:57:36.063 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:36.064 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.064 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.064 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.064 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:57:36.065 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:36.065 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.065 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.065 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.065 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:57:36.066 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:57:36.066 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:57:36.066 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:57:36.066 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:36.067 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:36.067 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:36.067 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.067 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.068 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.068 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:57:36.068 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:57:36.068 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:36.068 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:36.069 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:36.069 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:36.069 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:36.069 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:36.070 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:36.070 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 122, 2022-07-21T01:57:36.070 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:36.070 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:36.071 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:36.071 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.071 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.071 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.071 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:57:36.072 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:36.072 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.072 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.072 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.073 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:57:36.073 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:36.073 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.073 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.073 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.074 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:36.074 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.074 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.074 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.075 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:57:36.075 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:36.075 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.075 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.075 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.076 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:57:36.076 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:36.076 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.076 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.076 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.077 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:36.077 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.077 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.077 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.078 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:57:36.078 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:36.078 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.078 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.078 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.079 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:57:36.079 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:36.079 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.079 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.080 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.080 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:36.080 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.080 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.081 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.081 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:57:36.081 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:36.081 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.082 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.082 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.082 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:57:36.082 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:36.083 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.083 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.083 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.083 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:57:36.083 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:57:36.084 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:57:36.084 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:57:36.084 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:36.084 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:36.085 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:36.085 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.085 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.085 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.086 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:57:36.086 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:57:36.086 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:36.086 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:36.087 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:36.087 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:36.087 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:36.087 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:36.087 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:36.088 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 130, 2022-07-21T01:57:36.088 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:36.088 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:36.088 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:36.089 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.089 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.089 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.089 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:57:36.089 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:36.090 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.090 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.090 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.090 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:57:36.091 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:36.091 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.091 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.091 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.091 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:36.092 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.092 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.092 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.092 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:57:36.093 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:36.093 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.093 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.093 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.093 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:57:36.094 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:36.094 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.094 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.094 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.095 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:36.095 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.095 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.095 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.096 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:57:36.096 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:36.096 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.096 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.096 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.097 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:57:36.097 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:36.097 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.097 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.097 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.098 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:36.098 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.098 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.098 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.099 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:57:36.099 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:36.099 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.099 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.100 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.100 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:57:36.100 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:36.100 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.100 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.101 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.101 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:57:36.101 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:57:36.101 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:57:36.102 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:57:36.102 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:36.102 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:36.102 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:36.102 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.103 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.103 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.103 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:57:36.103 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:57:36.104 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:36.104 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:36.104 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:36.104 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:36.104 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:36.105 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:36.105 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:36.105 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 130, 2022-07-21T01:57:36.105 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:36.106 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:36.106 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:36.106 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.106 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.107 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.107 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:57:36.107 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:36.107 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.107 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.108 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.108 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:57:36.108 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:36.108 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.108 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.109 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.109 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:36.109 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.109 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.110 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.110 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:57:36.110 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:36.110 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.111 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.111 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.111 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:57:36.111 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:36.111 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.112 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.112 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.112 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:36.112 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.113 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.113 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.113 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:57:36.113 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:36.114 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.114 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.114 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.114 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:57:36.114 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:36.115 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.115 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.115 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.115 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:36.116 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:36.116 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.116 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:36.116 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:57:36.116 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:36.117 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.117 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.117 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:36.117 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:57:36.118 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:36.118 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.118 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.118 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.118 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:57:36.119 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:57:36.119 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:57:36.119 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:57:36.119 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:36.120 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:36.120 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:36.120 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:36.120 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.121 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:36.121 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:57:36.121 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.121 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:57:36.121 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:36.122 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:36.122 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:36.122 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:36.122 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:36.123 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:57:36.123 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:36.123 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.123 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.123 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:57:36.124 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:36.124 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:36.124 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:36.124 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:36.125 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:36.125 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:57:36.125 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:36.125 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.125 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.126 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:57:36.126 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:36.126 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:36.126 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:36.127 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:36.127 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:36.127 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:57:36.127 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:36.128 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.128 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.128 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:57:36.128 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:36.128 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:36.129 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:36.129 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:36.129 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:36.129 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:57:36.130 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:36.130 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.130 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.130 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:57:36.131 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:36.131 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:36.131 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:36.131 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:36.131 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:36.132 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:57:36.132 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:36.132 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.132 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.133 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:57:36.133 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:36.133 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:36.133 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:36.134 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:36.134 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:36.134 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:57:36.134 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:36.135 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.135 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.135 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:57:36.135 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:36.135 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:36.136 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:36.136 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:36.136 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:36.136 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:57:36.137 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:36.137 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.137 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:36.137 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:57:36.138 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:36.138 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:36.138 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:36.138 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:36.139 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:36.139 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:57:36.139 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:36.139 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.139 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:36.140 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:57:36.140 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:57:36.140 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:57:36.140 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:57:36.141 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:57:36.141 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6835/2209871450": "2022-07-22T01:57:29.662540+0000", 2022-07-21T01:57:36.141 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6834/2209871450": "2022-07-22T01:57:29.662540+0000" 2022-07-21T01:57:36.141 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.142 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:57:36.142 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:57:36.142 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:57:36.142 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:57:36.143 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:57:36.143 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:57:36.143 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:57:36.143 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:57:36.143 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:36.144 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:36.144 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:57:36.144 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:57:36.144 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:57:36.145 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:57:36.145 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:57:36.145 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:57:36.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.935+0000 7f8717066700 1 -- 172.21.15.140:0/1634509201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86f0053aa0 msgr2=0x7f86f0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.935+0000 7f8717066700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86f0053aa0 0x7f86f0055f60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f870000d000 tx=0x7f8700009040).stop 2022-07-21T01:57:36.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.935+0000 7f8717066700 1 -- 172.21.15.140:0/1634509201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87100acef0 msgr2=0x7f871009e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.935+0000 7f8717066700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87100acef0 0x7f871009e220 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f870400c000 tx=0x7f8704006df0).stop 2022-07-21T01:57:36.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.935+0000 7f8717066700 1 -- 172.21.15.140:0/1634509201 shutdown_connections 2022-07-21T01:57:36.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.935+0000 7f8717066700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f86f0053aa0 0x7f86f0055f60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.935+0000 7f8717066700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f871015a0c0 0x7f871015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.935+0000 7f8717066700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f87100b1e00 0x7f8710159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.935+0000 7f8717066700 1 --2- 172.21.15.140:0/1634509201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87100acef0 0x7f871009e220 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.935+0000 7f8717066700 1 -- 172.21.15.140:0/1634509201 >> 172.21.15.140:0/1634509201 conn(0x7f871005c3f0 msgr2=0x7f87100afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:36.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.939+0000 7f8717066700 1 -- 172.21.15.140:0/1634509201 shutdown_connections 2022-07-21T01:57:36.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:35.939+0000 7f8717066700 1 -- 172.21.15.140:0/1634509201 wait complete. 2022-07-21T01:57:36.151 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:36.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7ba85700 1 Processor -- start 2022-07-21T01:57:36.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7ba85700 1 -- start start 2022-07-21T01:57:36.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7ba85700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0e740b3110 0x7f0e74061d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7ba85700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0e7413f940 0x7f0e7413fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7ba85700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e740acaf0 con 0x7f0e740b34f0 2022-07-21T01:57:36.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7ba85700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e740acd90 con 0x7f0e7413f940 2022-07-21T01:57:36.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7ba85700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e740b2190 con 0x7f0e740b3110 2022-07-21T01:57:36.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e79020700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f0e740b34f0 0x7f0e741414b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55030/0 (socket says 172.21.15.140:55030) 2022-07-21T01:57:36.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e79020700 1 -- 172.21.15.140:0/3183569921 learned_addr learned my addr 172.21.15.140:0/3183569921 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:36.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7a022700 1 --2- 172.21.15.140:0/3183569921 >> v2:172.21.15.140:3300/0 conn(0x7f0e7413f940 0x7f0e7413fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e79821700 1 --2- 172.21.15.140:0/3183569921 >> v2:172.21.15.152:3300/0 conn(0x7f0e740b3110 0x7f0e74061d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7881f700 1 -- 172.21.15.140:0/3183569921 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588174238 0 0) 0x7f0e740acaf0 con 0x7f0e740b34f0 2022-07-21T01:57:36.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7881f700 1 -- 172.21.15.140:0/3183569921 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e58003260 con 0x7f0e740b34f0 2022-07-21T01:57:36.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7a022700 1 -- 172.21.15.140:0/3183569921 >> v2:172.21.15.152:3300/0 conn(0x7f0e740b3110 msgr2=0x7f0e74061d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:36.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7a022700 1 --2- 172.21.15.140:0/3183569921 >> v2:172.21.15.152:3300/0 conn(0x7f0e740b3110 0x7f0e74061d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7a022700 1 -- 172.21.15.140:0/3183569921 >> v1:172.21.15.140:6789/0 conn(0x7f0e740b34f0 legacy=0x7f0e741414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7a022700 1 -- 172.21.15.140:0/3183569921 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e740b3940 con 0x7f0e7413f940 2022-07-21T01:57:36.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7a022700 1 --2- 172.21.15.140:0/3183569921 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e7413f940 0x7f0e7413fdb0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f0e740ad7d0 tx=0x7f0e68006df0).ready entity=mon.0 client_cookie=38ba4f8ebc91e9d3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:36.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7881f700 1 -- 172.21.15.140:0/3183569921 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e6800d4f0 con 0x7f0e7413f940 2022-07-21T01:57:36.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7881f700 1 -- 172.21.15.140:0/3183569921 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e68000e90 con 0x7f0e7413f940 2022-07-21T01:57:36.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.259+0000 7f0e7881f700 1 -- 172.21.15.140:0/3183569921 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e6800d840 con 0x7f0e7413f940 2022-07-21T01:57:36.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 -- 172.21.15.140:0/3183569921 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e7413f940 msgr2=0x7f0e7413fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 --2- 172.21.15.140:0/3183569921 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e7413f940 0x7f0e7413fdb0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f0e740ad7d0 tx=0x7f0e68006df0).stop 2022-07-21T01:57:36.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 -- 172.21.15.140:0/3183569921 shutdown_connections 2022-07-21T01:57:36.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 --2- 172.21.15.140:0/3183569921 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e7413f940 0x7f0e7413fdb0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 --2- 172.21.15.140:0/3183569921 >> v2:172.21.15.152:3300/0 conn(0x7f0e740b3110 0x7f0e74061d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 -- 172.21.15.140:0/3183569921 >> 172.21.15.140:0/3183569921 conn(0x7f0e740b47e0 msgr2=0x7f0e7405bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:36.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 -- 172.21.15.140:0/3183569921 shutdown_connections 2022-07-21T01:57:36.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 -- 172.21.15.140:0/3183569921 wait complete. 2022-07-21T01:57:36.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 Processor -- start 2022-07-21T01:57:36.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 -- start start 2022-07-21T01:57:36.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e740b3110 0x7f0e74066100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e740b34f0 0x7f0e74065050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e7413f940 0x7f0e740ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7409cfa0 con 0x7f0e740b3110 2022-07-21T01:57:36.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e740665f0 con 0x7f0e740b34f0 2022-07-21T01:57:36.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e740659c0 con 0x7f0e7413f940 2022-07-21T01:57:36.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e79821700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e740b3110 0x7f0e74066100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e79821700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e740b3110 0x7f0e74066100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40888/0 (socket says 172.21.15.140:40888) 2022-07-21T01:57:36.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e79821700 1 -- 172.21.15.140:0/1809853912 learned_addr learned my addr 172.21.15.140:0/1809853912 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:36.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e79020700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e740b34f0 0x7f0e74065050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7a022700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e7413f940 0x7f0e740ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e79821700 1 -- 172.21.15.140:0/1809853912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e7413f940 msgr2=0x7f0e740ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e79821700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e7413f940 0x7f0e740ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e79821700 1 -- 172.21.15.140:0/1809853912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e740b34f0 msgr2=0x7f0e74065050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e79821700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e740b34f0 0x7f0e74065050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e79821700 1 -- 172.21.15.140:0/1809853912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6800b180 con 0x7f0e740b3110 2022-07-21T01:57:36.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e79821700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e740b3110 0x7f0e74066100 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7f0e6c00c000 tx=0x7f0e6c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:36.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e66ffd700 1 -- 172.21.15.140:0/1809853912 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e6c00c850 con 0x7f0e740b3110 2022-07-21T01:57:36.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 -- 172.21.15.140:0/1809853912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e74065c60 con 0x7f0e740b3110 2022-07-21T01:57:36.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e7ba85700 1 -- 172.21.15.140:0/1809853912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e74067aa0 con 0x7f0e740b3110 2022-07-21T01:57:36.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e66ffd700 1 -- 172.21.15.140:0/1809853912 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c00c9b0 con 0x7f0e740b3110 2022-07-21T01:57:36.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.263+0000 7f0e66ffd700 1 -- 172.21.15.140:0/1809853912 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0e6c00cd10 con 0x7f0e740b3110 2022-07-21T01:57:36.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.267+0000 7f0e7ba85700 1 -- 172.21.15.140:0/1809853912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e50003410 con 0x7f0e740b3110 2022-07-21T01:57:36.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.267+0000 7f0e66ffd700 1 -- 172.21.15.140:0/1809853912 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0e6c022070 con 0x7f0e740b3110 2022-07-21T01:57:36.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.267+0000 7f0e66ffd700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e5c0539e0 0x7f0e5c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.267+0000 7f0e66ffd700 1 -- 172.21.15.140:0/1809853912 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f0e6c06c800 con 0x7f0e740b3110 2022-07-21T01:57:36.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.267+0000 7f0e79020700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e5c0539e0 0x7f0e5c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.267+0000 7f0e79020700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e5c0539e0 0x7f0e5c055ea0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f0e7000b000 tx=0x7f0e70009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:36.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.275+0000 7f0e66ffd700 1 -- 172.21.15.140:0/1809853912 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0e6c043a00 con 0x7f0e740b3110 2022-07-21T01:57:36.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.439+0000 7f0e7ba85700 1 -- 172.21.15.140:0/1809853912 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e50003990 con 0x7f0e740b3110 2022-07-21T01:57:36.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.439+0000 7f0e66ffd700 1 -- 172.21.15.140:0/1809853912 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+1212 (secure 0 0 0) 0x7f0e6c06c070 con 0x7f0e740b3110 2022-07-21T01:57:36.441 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:36.441 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/558475048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":558475048},{"type":"v1","addr":"172.21.15.152:6833","nonce":558475048}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1385837668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1385837668},{"type":"v1","addr":"172.21.15.140:6835","nonce":1385837668}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[]} 2022-07-21T01:57:36.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.439+0000 7f0e7ba85700 1 -- 172.21.15.140:0/1809853912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e5c0539e0 msgr2=0x7f0e5c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.439+0000 7f0e7ba85700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e5c0539e0 0x7f0e5c055ea0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f0e7000b000 tx=0x7f0e70009040).stop 2022-07-21T01:57:36.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.439+0000 7f0e7ba85700 1 -- 172.21.15.140:0/1809853912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e740b3110 msgr2=0x7f0e74066100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.439+0000 7f0e7ba85700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e740b3110 0x7f0e74066100 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7f0e6c00c000 tx=0x7f0e6c0091d0).stop 2022-07-21T01:57:36.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.443+0000 7f0e7ba85700 1 -- 172.21.15.140:0/1809853912 shutdown_connections 2022-07-21T01:57:36.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.443+0000 7f0e7ba85700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0e5c0539e0 0x7f0e5c055ea0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.443+0000 7f0e7ba85700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e7413f940 0x7f0e740ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.443+0000 7f0e7ba85700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e740b34f0 0x7f0e74065050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.443+0000 7f0e7ba85700 1 --2- 172.21.15.140:0/1809853912 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0e740b3110 0x7f0e74066100 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.443+0000 7f0e7ba85700 1 -- 172.21.15.140:0/1809853912 >> 172.21.15.140:0/1809853912 conn(0x7f0e740b47e0 msgr2=0x7f0e740625e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:36.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.443+0000 7f0e7ba85700 1 -- 172.21.15.140:0/1809853912 shutdown_connections 2022-07-21T01:57:36.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.443+0000 7f0e7ba85700 1 -- 172.21.15.140:0/1809853912 wait complete. 2022-07-21T01:57:36.448 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 66 2022-07-21T01:57:36.458 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:36.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.555+0000 7fc94a5b2700 1 Processor -- start 2022-07-21T01:57:36.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.555+0000 7fc94a5b2700 1 -- start start 2022-07-21T01:57:36.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc94a5b2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9440acef0 0x7fc94414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc94a5b2700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94405d480 con 0x7fc94405c3f0 2022-07-21T01:57:36.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc94a5b2700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9440b21e0 con 0x7fc9440b1e00 2022-07-21T01:57:36.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc94a5b2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94405cf40 con 0x7fc9440acef0 2022-07-21T01:57:36.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc9437fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc94405c3f0 0x7fc944146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55046/0 (socket says 172.21.15.140:55046) 2022-07-21T01:57:36.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc9437fe700 1 -- 172.21.15.140:0/1794161315 learned_addr learned my addr 172.21.15.140:0/1794161315 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:36.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc948b4f700 1 --2- 172.21.15.140:0/1794161315 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9440acef0 0x7fc94414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc942ffd700 1 -- 172.21.15.140:0/1794161315 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325204629 0 0) 0x7fc94405d480 con 0x7fc94405c3f0 2022-07-21T01:57:36.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc942ffd700 1 -- 172.21.15.140:0/1794161315 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc924003260 con 0x7fc94405c3f0 2022-07-21T01:57:36.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc948b4f700 1 -- 172.21.15.140:0/1794161315 >> v1:172.21.15.152:6789/0 conn(0x7fc9440b1e00 legacy=0x7fc9440ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc942ffd700 1 -- 172.21.15.140:0/1794161315 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74626752 0 0) 0x7fc9440b21e0 con 0x7fc9440b1e00 2022-07-21T01:57:36.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc948b4f700 1 -- 172.21.15.140:0/1794161315 >> v1:172.21.15.140:6789/0 conn(0x7fc94405c3f0 legacy=0x7fc944146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc948b4f700 1 -- 172.21.15.140:0/1794161315 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94405d1e0 con 0x7fc9440acef0 2022-07-21T01:57:36.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc948b4f700 1 --2- 172.21.15.140:0/1794161315 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9440acef0 0x7fc94414a6d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fc93c002f70 tx=0x7fc93c006df0).ready entity=mon.2 client_cookie=f1c1569ad13823ec server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:36.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc942ffd700 1 -- 172.21.15.140:0/1794161315 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93c00d570 con 0x7fc9440acef0 2022-07-21T01:57:36.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc942ffd700 1 -- 172.21.15.140:0/1794161315 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c000e90 con 0x7fc9440acef0 2022-07-21T01:57:36.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc942ffd700 1 -- 172.21.15.140:0/1794161315 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93c00d8c0 con 0x7fc9440acef0 2022-07-21T01:57:36.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc94a5b2700 1 -- 172.21.15.140:0/1794161315 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9440acef0 msgr2=0x7fc94414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc94a5b2700 1 --2- 172.21.15.140:0/1794161315 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9440acef0 0x7fc94414a6d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fc93c002f70 tx=0x7fc93c006df0).stop 2022-07-21T01:57:36.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc94a5b2700 1 -- 172.21.15.140:0/1794161315 shutdown_connections 2022-07-21T01:57:36.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc94a5b2700 1 --2- 172.21.15.140:0/1794161315 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9440acef0 0x7fc94414a6d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc94a5b2700 1 -- 172.21.15.140:0/1794161315 >> 172.21.15.140:0/1794161315 conn(0x7fc9440611c0 msgr2=0x7fc944062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:36.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc94a5b2700 1 -- 172.21.15.140:0/1794161315 shutdown_connections 2022-07-21T01:57:36.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.559+0000 7fc94a5b2700 1 -- 172.21.15.140:0/1794161315 wait complete. 2022-07-21T01:57:36.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc94a5b2700 1 Processor -- start 2022-07-21T01:57:36.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc94a5b2700 1 -- start start 2022-07-21T01:57:36.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc94a5b2700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9440acef0 0x7fc944149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc94a5b2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9440b1e00 0x7fc94414a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc94a5b2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc944066f50 0x7fc944142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc94a5b2700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9440666b0 con 0x7fc944066f50 2022-07-21T01:57:36.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc94a5b2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944149b80 con 0x7fc9440b1e00 2022-07-21T01:57:36.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc94a5b2700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94414a900 con 0x7fc9440acef0 2022-07-21T01:57:36.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc948b4f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc944066f50 0x7fc944142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc948b4f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc944066f50 0x7fc944142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40906/0 (socket says 172.21.15.140:40906) 2022-07-21T01:57:36.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc948b4f700 1 -- 172.21.15.140:0/867358164 learned_addr learned my addr 172.21.15.140:0/867358164 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:36.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc943fff700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9440acef0 0x7fc944149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc9437fe700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9440b1e00 0x7fc94414a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc948b4f700 1 -- 172.21.15.140:0/867358164 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9440acef0 msgr2=0x7fc944149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc948b4f700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9440acef0 0x7fc944149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc948b4f700 1 -- 172.21.15.140:0/867358164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9440b1e00 msgr2=0x7fc94414a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc948b4f700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9440b1e00 0x7fc94414a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc948b4f700 1 -- 172.21.15.140:0/867358164 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c00b180 con 0x7fc944066f50 2022-07-21T01:57:36.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc9437fe700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9440b1e00 0x7fc94414a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:36.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc948b4f700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc944066f50 0x7fc944142d60 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7fc93c002f70 tx=0x7fc93c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:36.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc9417fa700 1 -- 172.21.15.140:0/867358164 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93c00dea0 con 0x7fc944066f50 2022-07-21T01:57:36.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc9417fa700 1 -- 172.21.15.140:0/867358164 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c013e50 con 0x7fc944066f50 2022-07-21T01:57:36.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc9417fa700 1 -- 172.21.15.140:0/867358164 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc93c01e5f0 con 0x7fc944066f50 2022-07-21T01:57:36.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc94a5b2700 1 -- 172.21.15.140:0/867358164 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9440677a0 con 0x7fc944066f50 2022-07-21T01:57:36.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.563+0000 7fc94a5b2700 1 -- 172.21.15.140:0/867358164 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc944148740 con 0x7fc944066f50 2022-07-21T01:57:36.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.567+0000 7fc9417fa700 1 -- 172.21.15.140:0/867358164 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc93c01c070 con 0x7fc944066f50 2022-07-21T01:57:36.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.567+0000 7fc94a5b2700 1 -- 172.21.15.140:0/867358164 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc944066820 con 0x7fc944066f50 2022-07-21T01:57:36.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.567+0000 7fc9417fa700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc928053a70 0x7fc928055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.567+0000 7fc9417fa700 1 -- 172.21.15.140:0/867358164 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fc93c06c7e0 con 0x7fc944066f50 2022-07-21T01:57:36.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.567+0000 7fc943fff700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc928053a70 0x7fc928055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.567+0000 7fc943fff700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc928053a70 0x7fc928055f30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fc93400d000 tx=0x7fc934007f10).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:36.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.571+0000 7fc9417fa700 1 -- 172.21.15.140:0/867358164 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc944066820 con 0x7fc944066f50 2022-07-21T01:57:36.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.731+0000 7fc94a5b2700 1 -- 172.21.15.140:0/867358164 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9440667f0 con 0x7fc944066f50 2022-07-21T01:57:36.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.731+0000 7fc9417fa700 1 -- 172.21.15.140:0/867358164 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+1212 (secure 0 0 0) 0x7fc93c023070 con 0x7fc944066f50 2022-07-21T01:57:36.736 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:36.736 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7948,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/558475048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":558475048},{"type":"v1","addr":"172.21.15.152:6833","nonce":558475048}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":61},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/1385837668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":1385837668},{"type":"v1","addr":"172.21.15.140:6835","nonce":1385837668}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":66}],"filesystems":[]} 2022-07-21T01:57:36.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 -- 172.21.15.140:0/867358164 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc928053a70 msgr2=0x7fc928055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc928053a70 0x7fc928055f30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fc93400d000 tx=0x7fc934007f10).stop 2022-07-21T01:57:36.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 -- 172.21.15.140:0/867358164 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc944066f50 msgr2=0x7fc944142d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc944066f50 0x7fc944142d60 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7fc93c002f70 tx=0x7fc93c014000).stop 2022-07-21T01:57:36.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 -- 172.21.15.140:0/867358164 shutdown_connections 2022-07-21T01:57:36.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc928053a70 0x7fc928055f30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc944066f50 0x7fc944142d60 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9440b1e00 0x7fc94414a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 --2- 172.21.15.140:0/867358164 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9440acef0 0x7fc944149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 -- 172.21.15.140:0/867358164 >> 172.21.15.140:0/867358164 conn(0x7fc94405c3f0 msgr2=0x7fc9440aaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:36.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 -- 172.21.15.140:0/867358164 shutdown_connections 2022-07-21T01:57:36.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.735+0000 7fc94a5b2700 1 -- 172.21.15.140:0/867358164 wait complete. 2022-07-21T01:57:36.741 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 66 2022-07-21T01:57:36.748 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:57:36.748 INFO:tasks.ceph.mds.a:Stopping old one... 2022-07-21T01:57:36.749 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:57:36.749 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:57:36.749 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:57:36.750 INFO:tasks.ceph.mds.b:Stopping old one... 2022-07-21T01:57:36.750 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:57:36.750 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:57:36.758 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:57:36.759 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:57:36.761 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:57:36.762 DEBUG:teuthology.parallel:result is None 2022-07-21T01:57:36.766 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:57:36.766 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:57:36.769 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:57:36.769 DEBUG:teuthology.parallel:result is None 2022-07-21T01:57:36.770 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:57:36.830 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:57:36.844 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:57:36.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178bfff700 1 Processor -- start 2022-07-21T01:57:36.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178bfff700 1 -- start start 2022-07-21T01:57:36.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b6200 0x7f178c0ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178bfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f178c0b4af0 0x7f178c13f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f178c0afc90 0x7f178c13fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178bfff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c0677d0 con 0x7f178c0afc90 2022-07-21T01:57:36.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178bfff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c0b65e0 con 0x7f178c0b4af0 2022-07-21T01:57:36.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0aca50 con 0x7f178c0b6200 2022-07-21T01:57:36.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f178c0afc90 0x7f178c13fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178affd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b6200 0x7f178c0ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f178c0afc90 0x7f178c13fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40926/0 (socket says 172.21.15.140:40926) 2022-07-21T01:57:36.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178affd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b6200 0x7f178c0ac560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:37652/0 (socket says 172.21.15.140:37652) 2022-07-21T01:57:36.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f17827fc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f178c0b4af0 0x7f178c13f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.871+0000 7f178b7fe700 1 -- 172.21.15.140:0/3692155425 learned_addr learned my addr 172.21.15.140:0/3692155425 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:36.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178affd700 1 -- 172.21.15.140:0/3692155425 >> v2:172.21.15.152:3300/0 conn(0x7f178c0b4af0 msgr2=0x7f178c13f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:36.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178affd700 1 --2- 172.21.15.140:0/3692155425 >> v2:172.21.15.152:3300/0 conn(0x7f178c0b4af0 0x7f178c13f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178affd700 1 -- 172.21.15.140:0/3692155425 >> v2:172.21.15.140:3300/0 conn(0x7f178c0afc90 msgr2=0x7f178c13fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:36.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178affd700 1 --2- 172.21.15.140:0/3692155425 >> v2:172.21.15.140:3300/0 conn(0x7f178c0afc90 0x7f178c13fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178affd700 1 -- 172.21.15.140:0/3692155425 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c067530 con 0x7f178c0b6200 2022-07-21T01:57:36.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178b7fe700 1 --2- 172.21.15.140:0/3692155425 >> v2:172.21.15.140:3300/0 conn(0x7f178c0afc90 0x7f178c13fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:36.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178affd700 1 --2- 172.21.15.140:0/3692155425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b6200 0x7f178c0ac560 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f177c002f70 tx=0x7f177c006e80).ready entity=mon.2 client_cookie=5b2c6b4f95c2593f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:36.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178a7fc700 1 -- 172.21.15.140:0/3692155425 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f177c012070 con 0x7f178c0b6200 2022-07-21T01:57:36.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178a7fc700 1 -- 172.21.15.140:0/3692155425 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c000e90 con 0x7f178c0b6200 2022-07-21T01:57:36.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178a7fc700 1 -- 172.21.15.140:0/3692155425 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f177c017850 con 0x7f178c0b6200 2022-07-21T01:57:36.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178bfff700 1 -- 172.21.15.140:0/3692155425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b6200 msgr2=0x7f178c0ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178bfff700 1 --2- 172.21.15.140:0/3692155425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b6200 0x7f178c0ac560 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f177c002f70 tx=0x7f177c006e80).stop 2022-07-21T01:57:36.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178bfff700 1 -- 172.21.15.140:0/3692155425 shutdown_connections 2022-07-21T01:57:36.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178bfff700 1 --2- 172.21.15.140:0/3692155425 >> v2:172.21.15.140:3300/0 conn(0x7f178c0afc90 0x7f178c13fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178bfff700 1 --2- 172.21.15.140:0/3692155425 >> v2:172.21.15.152:3300/0 conn(0x7f178c0b4af0 0x7f178c13f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178bfff700 1 --2- 172.21.15.140:0/3692155425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b6200 0x7f178c0ac560 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178bfff700 1 -- 172.21.15.140:0/3692155425 >> 172.21.15.140:0/3692155425 conn(0x7f178c063d10 msgr2=0x7f178c0b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:36.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178bfff700 1 -- 172.21.15.140:0/3692155425 shutdown_connections 2022-07-21T01:57:36.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178bfff700 1 -- 172.21.15.140:0/3692155425 wait complete. 2022-07-21T01:57:36.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178bfff700 1 Processor -- start 2022-07-21T01:57:36.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.875+0000 7f178bfff700 1 -- start start 2022-07-21T01:57:36.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f178c0afc90 0x7f178c13c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b4af0 0x7f178c13a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178c0b6200 0x7f178c13ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178bfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c066350 con 0x7f178c0afc90 2022-07-21T01:57:36.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178bfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0665f0 con 0x7f178c0b6200 2022-07-21T01:57:36.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c066890 con 0x7f178c0b4af0 2022-07-21T01:57:36.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f178c0afc90 0x7f178c13c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f178c0afc90 0x7f178c13c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40928/0 (socket says 172.21.15.140:40928) 2022-07-21T01:57:36.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178affd700 1 -- 172.21.15.140:0/407176430 learned_addr learned my addr 172.21.15.140:0/407176430 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:36.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f17827fc700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b4af0 0x7f178c13a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.890 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178b7fe700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178c0b6200 0x7f178c13ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178affd700 1 -- 172.21.15.140:0/407176430 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b4af0 msgr2=0x7f178c13a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178affd700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b4af0 0x7f178c13a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178affd700 1 -- 172.21.15.140:0/407176430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178c0b6200 msgr2=0x7f178c13ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:36.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178affd700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178c0b6200 0x7f178c13ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:36.891 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178affd700 1 -- 172.21.15.140:0/407176430 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178400b5e0 con 0x7f178c0afc90 2022-07-21T01:57:36.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f17827fc700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b4af0 0x7f178c13a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:36.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178affd700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f178c0afc90 0x7f178c13c760 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f177c002f70 tx=0x7f177c006e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:36.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f1788ff9700 1 -- 172.21.15.140:0/407176430 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f177c00b920 con 0x7f178c0afc90 2022-07-21T01:57:36.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f1788ff9700 1 -- 172.21.15.140:0/407176430 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c00ba80 con 0x7f178c0afc90 2022-07-21T01:57:36.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178bfff700 1 -- 172.21.15.140:0/407176430 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177c00d590 con 0x7f178c0afc90 2022-07-21T01:57:36.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f1788ff9700 1 -- 172.21.15.140:0/407176430 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f177c00bde0 con 0x7f178c0afc90 2022-07-21T01:57:36.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178bfff700 1 -- 172.21.15.140:0/407176430 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c13b2f0 con 0x7f178c0afc90 2022-07-21T01:57:36.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f178bfff700 1 -- 172.21.15.140:0/407176430 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c066a50 con 0x7f178c0afc90 2022-07-21T01:57:36.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.879+0000 7f1788ff9700 1 -- 172.21.15.140:0/407176430 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f177c02a020 con 0x7f178c0afc90 2022-07-21T01:57:36.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.883+0000 7f1788ff9700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1770053b00 0x7f1770055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:36.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.883+0000 7f1788ff9700 1 -- 172.21.15.140:0/407176430 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f177c070ff0 con 0x7f178c0afc90 2022-07-21T01:57:36.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.883+0000 7f17827fc700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1770053b00 0x7f1770055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:36.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.883+0000 7f17827fc700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1770053b00 0x7f1770055fc0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f177400b000 tx=0x7f1774008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:36.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:36.887+0000 7f1788ff9700 1 -- 172.21.15.140:0/407176430 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f177c0481f0 con 0x7f178c0afc90 2022-07-21T01:57:37.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.023+0000 7f178bfff700 1 -- 172.21.15.140:0/407176430 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f178c0574e0 con 0x7f178c0afc90 2022-07-21T01:57:37.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.871+0000 7f1788ff9700 1 -- 172.21.15.140:0/407176430 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v137) v1 ==== 97+0+0 (secure 0 0 0) 0x7f177c04b810 con 0x7f178c0afc90 2022-07-21T01:57:37.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 -- 172.21.15.140:0/407176430 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1770053b00 msgr2=0x7f1770055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:37.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1770053b00 0x7f1770055fc0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f177400b000 tx=0x7f1774008040).stop 2022-07-21T01:57:37.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 -- 172.21.15.140:0/407176430 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f178c0afc90 msgr2=0x7f178c13c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:37.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f178c0afc90 0x7f178c13c760 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f177c002f70 tx=0x7f177c006e80).stop 2022-07-21T01:57:37.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 -- 172.21.15.140:0/407176430 shutdown_connections 2022-07-21T01:57:37.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1770053b00 0x7f1770055fc0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:37.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f178c0b6200 0x7f178c13ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:37.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f178c0b4af0 0x7f178c13a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:37.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 --2- 172.21.15.140:0/407176430 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f178c0afc90 0x7f178c13c760 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:37.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 -- 172.21.15.140:0/407176430 >> 172.21.15.140:0/407176430 conn(0x7f178c063d10 msgr2=0x7f178c05f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:37.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 -- 172.21.15.140:0/407176430 shutdown_connections 2022-07-21T01:57:37.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.875+0000 7f178bfff700 1 -- 172.21.15.140:0/407176430 wait complete. 2022-07-21T01:57:37.881 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:57:37.892 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:57:37.893 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:57:38.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.995+0000 7f26b3589700 1 Processor -- start 2022-07-21T01:57:38.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.995+0000 7f26b3589700 1 -- start start 2022-07-21T01:57:38.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b3589700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26ac0b1e00 0x7f26ac14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b3589700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac05d480 con 0x7f26ac0acef0 2022-07-21T01:57:38.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b3589700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac0b21e0 con 0x7f26ac05c3f0 2022-07-21T01:57:38.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b3589700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ac05cf40 con 0x7f26ac0b1e00 2022-07-21T01:57:38.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b1b26700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f26ac0acef0 0x7f26ac146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55088/0 (socket says 172.21.15.140:55088) 2022-07-21T01:57:38.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b1b26700 1 -- 172.21.15.140:0/3675364907 learned_addr learned my addr 172.21.15.140:0/3675364907 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:38.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b1325700 1 --2- 172.21.15.140:0/3675364907 >> v2:172.21.15.152:3300/0 conn(0x7f26ac0b1e00 0x7f26ac14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f269ffff700 1 -- 172.21.15.140:0/3675364907 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196940043 0 0) 0x7f26ac05d480 con 0x7f26ac0acef0 2022-07-21T01:57:38.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f269ffff700 1 -- 172.21.15.140:0/3675364907 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2690003260 con 0x7f26ac0acef0 2022-07-21T01:57:38.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f269ffff700 1 -- 172.21.15.140:0/3675364907 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531752900 0 0) 0x7f26ac0b21e0 con 0x7f26ac05c3f0 2022-07-21T01:57:38.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f269ffff700 1 -- 172.21.15.140:0/3675364907 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26ac05d480 con 0x7f26ac05c3f0 2022-07-21T01:57:38.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b1325700 1 -- 172.21.15.140:0/3675364907 >> v1:172.21.15.152:6789/0 conn(0x7f26ac05c3f0 legacy=0x7f26ac0ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f269ffff700 1 -- 172.21.15.140:0/3675364907 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1665627826 0 0) 0x7f2690003260 con 0x7f26ac0acef0 2022-07-21T01:57:38.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b1325700 1 -- 172.21.15.140:0/3675364907 >> v1:172.21.15.140:6789/0 conn(0x7f26ac0acef0 legacy=0x7f26ac146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b1325700 1 -- 172.21.15.140:0/3675364907 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac05d1e0 con 0x7f26ac0b1e00 2022-07-21T01:57:38.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b1325700 1 --2- 172.21.15.140:0/3675364907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26ac0b1e00 0x7f26ac14a6d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f26a4001040 tx=0x7f26a0006df0).ready entity=mon.1 client_cookie=294435d6d5450121 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f269ffff700 1 -- 172.21.15.140:0/3675364907 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26a4001590 con 0x7f26ac0b1e00 2022-07-21T01:57:38.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f269ffff700 1 -- 172.21.15.140:0/3675364907 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a0000e90 con 0x7f26ac0b1e00 2022-07-21T01:57:38.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f269ffff700 1 -- 172.21.15.140:0/3675364907 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26a0012460 con 0x7f26ac0b1e00 2022-07-21T01:57:38.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b3589700 1 -- 172.21.15.140:0/3675364907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26ac0b1e00 msgr2=0x7f26ac14a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b3589700 1 --2- 172.21.15.140:0/3675364907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26ac0b1e00 0x7f26ac14a6d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f26a4001040 tx=0x7f26a0006df0).stop 2022-07-21T01:57:38.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b3589700 1 -- 172.21.15.140:0/3675364907 shutdown_connections 2022-07-21T01:57:38.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b3589700 1 --2- 172.21.15.140:0/3675364907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26ac0b1e00 0x7f26ac14a6d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b3589700 1 -- 172.21.15.140:0/3675364907 >> 172.21.15.140:0/3675364907 conn(0x7f26ac0611c0 msgr2=0x7f26ac062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:38.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b3589700 1 -- 172.21.15.140:0/3675364907 shutdown_connections 2022-07-21T01:57:38.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:37.999+0000 7f26b3589700 1 -- 172.21.15.140:0/3675364907 wait complete. 2022-07-21T01:57:38.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b3589700 1 Processor -- start 2022-07-21T01:57:38.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b3589700 1 -- start start 2022-07-21T01:57:38.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b3589700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f26ac0acef0 0x7f26ac1474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b3589700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26ac0b1e00 0x7f26ac142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b3589700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26ac1435b0 0x7f26ac142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b3589700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac14a070 con 0x7f26ac0acef0 2022-07-21T01:57:38.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b3589700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac149e10 con 0x7f26ac0b1e00 2022-07-21T01:57:38.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b3589700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac143110 con 0x7f26ac1435b0 2022-07-21T01:57:38.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b1325700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f26ac0acef0 0x7f26ac1474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b1325700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f26ac0acef0 0x7f26ac1474b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40944/0 (socket says 172.21.15.140:40944) 2022-07-21T01:57:38.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b1325700 1 -- 172.21.15.140:0/915675850 learned_addr learned my addr 172.21.15.140:0/915675850 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:38.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b0b24700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26ac0b1e00 0x7f26ac142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b1b26700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26ac1435b0 0x7f26ac142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b1325700 1 -- 172.21.15.140:0/915675850 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26ac1435b0 msgr2=0x7f26ac142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b1325700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26ac1435b0 0x7f26ac142810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b1325700 1 -- 172.21.15.140:0/915675850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26ac0b1e00 msgr2=0x7f26ac142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b1325700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26ac0b1e00 0x7f26ac142320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b1325700 1 -- 172.21.15.140:0/915675850 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a000b180 con 0x7f26ac0acef0 2022-07-21T01:57:38.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b1b26700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26ac1435b0 0x7f26ac142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:38.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b1325700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f26ac0acef0 0x7f26ac1474b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f26a4001040 tx=0x7f26a0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f269e7fc700 1 -- 172.21.15.140:0/915675850 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26a000d7d0 con 0x7f26ac0acef0 2022-07-21T01:57:38.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f269e7fc700 1 -- 172.21.15.140:0/915675850 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a0017070 con 0x7f26ac0acef0 2022-07-21T01:57:38.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f269e7fc700 1 -- 172.21.15.140:0/915675850 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f26a000db20 con 0x7f26ac0acef0 2022-07-21T01:57:38.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b3589700 1 -- 172.21.15.140:0/915675850 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac142d00 con 0x7f26ac0acef0 2022-07-21T01:57:38.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.003+0000 7f26b3589700 1 -- 172.21.15.140:0/915675850 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac064e80 con 0x7f26ac0acef0 2022-07-21T01:57:38.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.007+0000 7f26b3589700 1 -- 172.21.15.140:0/915675850 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac143310 con 0x7f26ac0acef0 2022-07-21T01:57:38.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.007+0000 7f269e7fc700 1 -- 172.21.15.140:0/915675850 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f26a000f030 con 0x7f26ac0acef0 2022-07-21T01:57:38.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.007+0000 7f269e7fc700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f26940539e0 0x7f2694055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.007+0000 7f269e7fc700 1 -- 172.21.15.140:0/915675850 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f26a006c8b0 con 0x7f26ac0acef0 2022-07-21T01:57:38.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.007+0000 7f26b0b24700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f26940539e0 0x7f2694055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.007+0000 7f26b0b24700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f26940539e0 0x7f2694055ea0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f26a800d000 tx=0x7f26a8007da0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.011+0000 7f269e7fc700 1 -- 172.21.15.140:0/915675850 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f26ac143310 con 0x7f26ac0acef0 2022-07-21T01:57:38.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.175+0000 7f26b3589700 1 -- 172.21.15.140:0/915675850 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f26ac1432e0 con 0x7f26ac0acef0 2022-07-21T01:57:38.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.175+0000 7f269e7fc700 1 -- 172.21.15.140:0/915675850 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v45) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f26a0043b00 con 0x7f26ac0acef0 2022-07-21T01:57:38.180 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:38.180 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:57:38.180 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:57:38.180 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.181 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:57:38.181 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:57:38.181 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.181 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:57:38.182 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:38.182 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:38.182 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.182 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.183 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.183 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.183 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:57:38.183 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:57:38.183 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.184 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:57:38.184 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:38.184 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:38.184 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.185 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.185 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.185 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.185 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:57:38.185 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:57:38.186 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.186 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:38.186 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:38.186 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.187 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.187 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.187 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.187 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:57:38.187 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:57:38.188 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.188 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:38.188 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:38.188 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.188 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.189 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.189 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.189 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:57:38.189 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:57:38.190 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.190 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:38.190 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:38.190 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.191 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.191 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.191 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.191 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:57:38.191 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:57:38.192 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.192 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:38.192 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:38.192 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.193 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.193 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.193 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.193 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:57:38.193 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:57:38.194 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.194 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:38.194 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:38.194 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.195 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.195 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.195 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.195 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:57:38.196 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:57:38.196 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.196 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:38.196 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:38.196 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.197 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.197 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.197 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.197 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:57:38.197 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:57:38.198 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.198 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:38.198 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:38.198 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.199 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.199 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.199 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.199 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:57:38.199 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:57:38.200 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.200 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:38.200 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:38.200 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.201 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.201 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.201 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.201 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:57:38.201 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:57:38.202 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.202 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:57:38.202 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:57:38.202 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:57:38.203 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.203 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.203 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.203 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:57:38.203 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:57:38.204 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.204 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:57:38.204 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:57:38.204 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:57:38.205 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.205 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.205 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.205 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:57:38.205 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:57:38.206 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.206 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:57:38.206 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:57:38.206 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:57:38.207 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.207 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.207 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.207 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:57:38.208 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:57:38.208 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.208 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:57:38.208 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:57:38.208 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:57:38.209 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.209 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.209 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.209 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.209 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:57:38.210 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:57:38.210 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.210 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:57:38.210 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:57:38.211 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.211 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.211 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:38.211 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:38.211 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:57:38.212 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:57:38.212 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:57:38.212 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:57:38.212 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:57:38.213 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:57:38.213 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.213 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:38.213 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:38.214 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:57:38.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 -- 172.21.15.140:0/915675850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f26940539e0 msgr2=0x7f2694055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f26940539e0 0x7f2694055ea0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f26a800d000 tx=0x7f26a8007da0).stop 2022-07-21T01:57:38.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 -- 172.21.15.140:0/915675850 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f26ac0acef0 msgr2=0x7f26ac1474b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f26ac0acef0 0x7f26ac1474b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f26a4001040 tx=0x7f26a0014000).stop 2022-07-21T01:57:38.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 -- 172.21.15.140:0/915675850 shutdown_connections 2022-07-21T01:57:38.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f26940539e0 0x7f2694055ea0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26ac1435b0 0x7f26ac142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26ac0b1e00 0x7f26ac142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 --2- 172.21.15.140:0/915675850 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f26ac0acef0 0x7f26ac1474b0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 -- 172.21.15.140:0/915675850 >> 172.21.15.140:0/915675850 conn(0x7f26ac05c3f0 msgr2=0x7f26ac0ade90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:38.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 -- 172.21.15.140:0/915675850 shutdown_connections 2022-07-21T01:57:38.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.179+0000 7f26b3589700 1 -- 172.21.15.140:0/915675850 wait complete. 2022-07-21T01:57:38.219 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:38.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.315+0000 7f394947d700 1 Processor -- start 2022-07-21T01:57:38.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.315+0000 7f394947d700 1 -- start start 2022-07-21T01:57:38.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f39440b5990 0x7f394413fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39440aecf0 0x7f3944140020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f39440aff80 0x7f394405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39440b5d70 con 0x7f39440b5990 2022-07-21T01:57:38.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39440651c0 con 0x7f39440aff80 2022-07-21T01:57:38.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39440b0690 con 0x7f39440aecf0 2022-07-21T01:57:38.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3943fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f39440b5990 0x7f394413fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3943fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f39440b5990 0x7f394413fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40954/0 (socket says 172.21.15.140:40954) 2022-07-21T01:57:38.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3943fff700 1 -- 172.21.15.140:0/4001572665 learned_addr learned my addr 172.21.15.140:0/4001572665 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:38.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3948c7c700 1 --2- 172.21.15.140:0/4001572665 >> v2:172.21.15.152:3300/0 conn(0x7f39440aff80 0x7f394405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f39437fe700 1 --2- 172.21.15.140:0/4001572665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39440aecf0 0x7f3944140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3943fff700 1 -- 172.21.15.140:0/4001572665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39440aecf0 msgr2=0x7f3944140020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:38.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3943fff700 1 --2- 172.21.15.140:0/4001572665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39440aecf0 0x7f3944140020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3943fff700 1 -- 172.21.15.140:0/4001572665 >> v2:172.21.15.152:3300/0 conn(0x7f39440aff80 msgr2=0x7f394405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:38.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3943fff700 1 --2- 172.21.15.140:0/4001572665 >> v2:172.21.15.152:3300/0 conn(0x7f39440aff80 0x7f394405bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3943fff700 1 -- 172.21.15.140:0/4001572665 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39440b0930 con 0x7f39440b5990 2022-07-21T01:57:38.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f39437fe700 1 --2- 172.21.15.140:0/4001572665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39440aecf0 0x7f3944140020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:38.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3943fff700 1 --2- 172.21.15.140:0/4001572665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39440b5990 0x7f394413fb30 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f3934002f70 tx=0x7f3934006eb0).ready entity=mon.0 client_cookie=3fb46263387e1fe8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3942ffd700 1 -- 172.21.15.140:0/4001572665 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f393400d570 con 0x7f39440b5990 2022-07-21T01:57:38.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3942ffd700 1 -- 172.21.15.140:0/4001572665 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3934000e90 con 0x7f39440b5990 2022-07-21T01:57:38.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f3942ffd700 1 -- 172.21.15.140:0/4001572665 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f393400d900 con 0x7f39440b5990 2022-07-21T01:57:38.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 -- 172.21.15.140:0/4001572665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39440b5990 msgr2=0x7f394413fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 --2- 172.21.15.140:0/4001572665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39440b5990 0x7f394413fb30 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f3934002f70 tx=0x7f3934006eb0).stop 2022-07-21T01:57:38.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 -- 172.21.15.140:0/4001572665 shutdown_connections 2022-07-21T01:57:38.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 --2- 172.21.15.140:0/4001572665 >> v2:172.21.15.152:3300/0 conn(0x7f39440aff80 0x7f394405bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 --2- 172.21.15.140:0/4001572665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39440aecf0 0x7f3944140020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 --2- 172.21.15.140:0/4001572665 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39440b5990 0x7f394413fb30 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 -- 172.21.15.140:0/4001572665 >> 172.21.15.140:0/4001572665 conn(0x7f39440634e0 msgr2=0x7f39440b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:38.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 -- 172.21.15.140:0/4001572665 shutdown_connections 2022-07-21T01:57:38.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.319+0000 7f394947d700 1 -- 172.21.15.140:0/4001572665 wait complete. 2022-07-21T01:57:38.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f394947d700 1 Processor -- start 2022-07-21T01:57:38.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f394947d700 1 -- start start 2022-07-21T01:57:38.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f394947d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39440aecf0 0x7f39440ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f394947d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39440aff80 0x7f3944152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f394947d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39440b5990 0x7f3944112540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f394947d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39441446b0 con 0x7f39440b5990 2022-07-21T01:57:38.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f394947d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944152fe0 con 0x7f39440aff80 2022-07-21T01:57:38.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f394947d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39441523b0 con 0x7f39440aecf0 2022-07-21T01:57:38.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f3948c7c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39440b5990 0x7f3944112540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f3943fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39440aecf0 0x7f39440ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f3948c7c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39440b5990 0x7f3944112540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40962/0 (socket says 172.21.15.140:40962) 2022-07-21T01:57:38.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f3948c7c700 1 -- 172.21.15.140:0/3772911652 learned_addr learned my addr 172.21.15.140:0/3772911652 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:38.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f39437fe700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39440aff80 0x7f3944152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f3948c7c700 1 -- 172.21.15.140:0/3772911652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39440aecf0 msgr2=0x7f39440ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f3948c7c700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39440aecf0 0x7f39440ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f3948c7c700 1 -- 172.21.15.140:0/3772911652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39440aff80 msgr2=0x7f3944152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f3948c7c700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39440aff80 0x7f3944152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f3948c7c700 1 -- 172.21.15.140:0/3772911652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393400b1e0 con 0x7f39440b5990 2022-07-21T01:57:38.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f3948c7c700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39440b5990 0x7f3944112540 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f393c00c000 tx=0x7f393c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f39417fa700 1 -- 172.21.15.140:0/3772911652 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f393c012600 con 0x7f39440b5990 2022-07-21T01:57:38.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f39417fa700 1 -- 172.21.15.140:0/3772911652 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c012760 con 0x7f39440b5990 2022-07-21T01:57:38.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f394947d700 1 -- 172.21.15.140:0/3772911652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3944152650 con 0x7f39440b5990 2022-07-21T01:57:38.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f39417fa700 1 -- 172.21.15.140:0/3772911652 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f393c012ac0 con 0x7f39440b5990 2022-07-21T01:57:38.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.323+0000 7f394947d700 1 -- 172.21.15.140:0/3772911652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3944112a30 con 0x7f39440b5990 2022-07-21T01:57:38.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.327+0000 7f394947d700 1 -- 172.21.15.140:0/3772911652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3944152810 con 0x7f39440b5990 2022-07-21T01:57:38.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.327+0000 7f39417fa700 1 -- 172.21.15.140:0/3772911652 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f393c022070 con 0x7f39440b5990 2022-07-21T01:57:38.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.327+0000 7f39417fa700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3924053aa0 0x7f3924055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.327+0000 7f39417fa700 1 -- 172.21.15.140:0/3772911652 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f393c06c210 con 0x7f39440b5990 2022-07-21T01:57:38.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.327+0000 7f3943fff700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3924053aa0 0x7f3924055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.327+0000 7f3943fff700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3924053aa0 0x7f3924055f60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f393401c000 tx=0x7f3934015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.335+0000 7f39417fa700 1 -- 172.21.15.140:0/3772911652 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f393c043350 con 0x7f39440b5990 2022-07-21T01:57:38.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.459+0000 7f394947d700 1 -- 172.21.15.140:0/3772911652 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39440574e0 con 0x7f39440b5990 2022-07-21T01:57:38.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.459+0000 7f39417fa700 1 -- 172.21.15.140:0/3772911652 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+13287 (secure 0 0 0) 0x7f393c016090 con 0x7f39440b5990 2022-07-21T01:57:38.464 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:38.464 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":137,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:37.867174+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:38.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 -- 172.21.15.140:0/3772911652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3924053aa0 msgr2=0x7f3924055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3924053aa0 0x7f3924055f60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f393401c000 tx=0x7f3934015000).stop 2022-07-21T01:57:38.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 -- 172.21.15.140:0/3772911652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39440b5990 msgr2=0x7f3944112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39440b5990 0x7f3944112540 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f393c00c000 tx=0x7f393c0091d0).stop 2022-07-21T01:57:38.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 -- 172.21.15.140:0/3772911652 shutdown_connections 2022-07-21T01:57:38.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3924053aa0 0x7f3924055f60 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f39440b5990 0x7f3944112540 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39440aff80 0x7f3944152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 --2- 172.21.15.140:0/3772911652 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39440aecf0 0x7f39440ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 -- 172.21.15.140:0/3772911652 >> 172.21.15.140:0/3772911652 conn(0x7f39440634e0 msgr2=0x7f394405bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:38.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 -- 172.21.15.140:0/3772911652 shutdown_connections 2022-07-21T01:57:38.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.463+0000 7f394947d700 1 -- 172.21.15.140:0/3772911652 wait complete. 2022-07-21T01:57:38.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:57:38.478 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:57:38.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.583+0000 7fdd4161a700 1 Processor -- start 2022-07-21T01:57:38.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.583+0000 7fdd4161a700 1 -- start start 2022-07-21T01:57:38.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd4161a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd3c0b02e0 0x7fdd3c1431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd4161a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd3c0ad380 con 0x7fdd3c0acfa0 2022-07-21T01:57:38.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd4161a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd3c0af470 con 0x7fdd3c0b1030 2022-07-21T01:57:38.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd4161a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c13f030 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd3bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fdd3c0acfa0 0x7fdd3c142ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55112/0 (socket says 172.21.15.140:55112) 2022-07-21T01:57:38.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd3bfff700 1 -- 172.21.15.140:0/2054632837 learned_addr learned my addr 172.21.15.140:0/2054632837 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:38.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd40e19700 1 --2- 172.21.15.140:0/2054632837 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd3c0b02e0 0x7fdd3c1431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd3b7fe700 1 -- 172.21.15.140:0/2054632837 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832510603 0 0) 0x7fdd3c0ad380 con 0x7fdd3c0acfa0 2022-07-21T01:57:38.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd3b7fe700 1 -- 172.21.15.140:0/2054632837 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd1c004370 con 0x7fdd3c0acfa0 2022-07-21T01:57:38.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd3b7fe700 1 -- 172.21.15.140:0/2054632837 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141837355 0 0) 0x7fdd3c0af470 con 0x7fdd3c0b1030 2022-07-21T01:57:38.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd3b7fe700 1 -- 172.21.15.140:0/2054632837 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd3c0ad380 con 0x7fdd3c0b1030 2022-07-21T01:57:38.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd3b7fe700 1 -- 172.21.15.140:0/2054632837 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 440945848 0 0) 0x7fdd1c004370 con 0x7fdd3c0acfa0 2022-07-21T01:57:38.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd40e19700 1 -- 172.21.15.140:0/2054632837 >> v1:172.21.15.152:6789/0 conn(0x7fdd3c0b1030 legacy=0x7fdd3c0ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd40e19700 1 -- 172.21.15.140:0/2054632837 >> v1:172.21.15.140:6789/0 conn(0x7fdd3c0acfa0 legacy=0x7fdd3c142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd40e19700 1 -- 172.21.15.140:0/2054632837 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c0b1630 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd40e19700 1 --2- 172.21.15.140:0/2054632837 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd3c0b02e0 0x7fdd3c1431d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fdd2c001110 tx=0x7fdd340091d0).ready entity=mon.2 client_cookie=c277a591f052a885 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd3b7fe700 1 -- 172.21.15.140:0/2054632837 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd2c001680 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd3b7fe700 1 -- 172.21.15.140:0/2054632837 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3400f510 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd3b7fe700 1 -- 172.21.15.140:0/2054632837 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd3400f680 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd4161a700 1 -- 172.21.15.140:0/2054632837 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd3c0b02e0 msgr2=0x7fdd3c1431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd4161a700 1 --2- 172.21.15.140:0/2054632837 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd3c0b02e0 0x7fdd3c1431d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fdd2c001110 tx=0x7fdd340091d0).stop 2022-07-21T01:57:38.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd4161a700 1 -- 172.21.15.140:0/2054632837 shutdown_connections 2022-07-21T01:57:38.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd4161a700 1 --2- 172.21.15.140:0/2054632837 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd3c0b02e0 0x7fdd3c1431d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd4161a700 1 -- 172.21.15.140:0/2054632837 >> 172.21.15.140:0/2054632837 conn(0x7fdd3c0632d0 msgr2=0x7fdd3c0636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:38.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.587+0000 7fdd4161a700 1 -- 172.21.15.140:0/2054632837 shutdown_connections 2022-07-21T01:57:38.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd4161a700 1 -- 172.21.15.140:0/2054632837 wait complete. 2022-07-21T01:57:38.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd4161a700 1 Processor -- start 2022-07-21T01:57:38.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd4161a700 1 -- start start 2022-07-21T01:57:38.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd4161a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd3c0acfa0 0x7fdd3c066670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd4161a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd3c0b02e0 0x7fdd3c153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd4161a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd3c0b1030 0x7fdd3c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd4161a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c1131a0 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd4161a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c1133c0 con 0x7fdd3c0b1030 2022-07-21T01:57:38.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd4161a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c113660 con 0x7fdd3c0acfa0 2022-07-21T01:57:38.605 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd33fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd3c0b02e0 0x7fdd3c153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd33fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd3c0b02e0 0x7fdd3c153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40976/0 (socket says 172.21.15.140:40976) 2022-07-21T01:57:38.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd33fff700 1 -- 172.21.15.140:0/1016423830 learned_addr learned my addr 172.21.15.140:0/1016423830 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:38.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd3bfff700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd3c0acfa0 0x7fdd3c066670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.606 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd40e19700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd3c0b1030 0x7fdd3c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd33fff700 1 -- 172.21.15.140:0/1016423830 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd3c0acfa0 msgr2=0x7fdd3c066670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd33fff700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd3c0acfa0 0x7fdd3c066670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd33fff700 1 -- 172.21.15.140:0/1016423830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd3c0b1030 msgr2=0x7fdd3c1539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd33fff700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd3c0b1030 0x7fdd3c1539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.607 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd33fff700 1 -- 172.21.15.140:0/1016423830 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd34009470 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd33fff700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd3c0b02e0 0x7fdd3c153500 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fdd24001110 tx=0x7fdd24007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd39ffb700 1 -- 172.21.15.140:0/1016423830 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd24012400 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.608 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd39ffb700 1 -- 172.21.15.140:0/1016423830 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd24012560 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd4161a700 1 -- 172.21.15.140:0/1016423830 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3c153060 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.591+0000 7fdd39ffb700 1 -- 172.21.15.140:0/1016423830 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd240128c0 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.595+0000 7fdd4161a700 1 -- 172.21.15.140:0/1016423830 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3c153f70 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.609 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.595+0000 7fdd4161a700 1 -- 172.21.15.140:0/1016423830 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3c0b4800 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.595+0000 7fdd39ffb700 1 -- 172.21.15.140:0/1016423830 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdd24019030 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.595+0000 7fdd39ffb700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd20053aa0 0x7fdd20055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.610 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.595+0000 7fdd39ffb700 1 -- 172.21.15.140:0/1016423830 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fdd2406ad60 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.595+0000 7fdd3bfff700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd20053aa0 0x7fdd20055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.611 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.595+0000 7fdd3bfff700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd20053aa0 0x7fdd20055f60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fdd2c00d000 tx=0x7fdd2c000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.612 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.603+0000 7fdd39ffb700 1 -- 172.21.15.140:0/1016423830 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdd24041fb0 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.727+0000 7fdd4161a700 1 -- 172.21.15.140:0/1016423830 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdd3c1531f0 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.727+0000 7fdd39ffb700 1 -- 172.21.15.140:0/1016423830 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v137) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdd240455d0 con 0x7fdd3c0b02e0 2022-07-21T01:57:38.730 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:57:38.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 -- 172.21.15.140:0/1016423830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd20053aa0 msgr2=0x7fdd20055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd20053aa0 0x7fdd20055f60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7fdd2c00d000 tx=0x7fdd2c000bc0).stop 2022-07-21T01:57:38.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 -- 172.21.15.140:0/1016423830 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd3c0b02e0 msgr2=0x7fdd3c153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd3c0b02e0 0x7fdd3c153500 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fdd24001110 tx=0x7fdd24007ae0).stop 2022-07-21T01:57:38.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 -- 172.21.15.140:0/1016423830 shutdown_connections 2022-07-21T01:57:38.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd20053aa0 0x7fdd20055f60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd3c0b1030 0x7fdd3c1539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd3c0b02e0 0x7fdd3c153500 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 --2- 172.21.15.140:0/1016423830 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd3c0acfa0 0x7fdd3c066670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 -- 172.21.15.140:0/1016423830 >> 172.21.15.140:0/1016423830 conn(0x7fdd3c0632d0 msgr2=0x7fdd3c13fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:38.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 -- 172.21.15.140:0/1016423830 shutdown_connections 2022-07-21T01:57:38.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.731+0000 7fdd4161a700 1 -- 172.21.15.140:0/1016423830 wait complete. 2022-07-21T01:57:38.747 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:57:38.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.847+0000 7f2d9e706700 1 Processor -- start 2022-07-21T01:57:38.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.847+0000 7f2d9e706700 1 -- start start 2022-07-21T01:57:38.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.847+0000 7f2d9e706700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2d980b59d0 0x7f2d980b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.847+0000 7f2d9e706700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2d980af920 0x7f2d980ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.847+0000 7f2d9e706700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d9805d0c0 0x7f2d980ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.847+0000 7f2d9e706700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d980adf70 con 0x7f2d980af920 2022-07-21T01:57:38.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.847+0000 7f2d9e706700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d980b5db0 con 0x7f2d980b59d0 2022-07-21T01:57:38.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.847+0000 7f2d9e706700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d98146300 con 0x7f2d9805d0c0 2022-07-21T01:57:38.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.847+0000 7f2d9cf03700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2d980af920 0x7f2d980ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.847+0000 7f2d9cf03700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f2d980af920 0x7f2d980ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40980/0 (socket says 172.21.15.140:40980) 2022-07-21T01:57:38.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.847+0000 7f2d9cf03700 1 -- 172.21.15.140:0/460377320 learned_addr learned my addr 172.21.15.140:0/460377320 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:38.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9df05700 1 --2- 172.21.15.140:0/460377320 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d9805d0c0 0x7f2d980ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9d704700 1 --2- 172.21.15.140:0/460377320 >> v2:172.21.15.152:3300/0 conn(0x7f2d980b59d0 0x7f2d980b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9cf03700 1 -- 172.21.15.140:0/460377320 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d9805d0c0 msgr2=0x7f2d980ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:38.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9cf03700 1 --2- 172.21.15.140:0/460377320 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d9805d0c0 0x7f2d980ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9cf03700 1 -- 172.21.15.140:0/460377320 >> v2:172.21.15.152:3300/0 conn(0x7f2d980b59d0 msgr2=0x7f2d980b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:38.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9cf03700 1 --2- 172.21.15.140:0/460377320 >> v2:172.21.15.152:3300/0 conn(0x7f2d980b59d0 0x7f2d980b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9cf03700 1 -- 172.21.15.140:0/460377320 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d98146060 con 0x7f2d980af920 2022-07-21T01:57:38.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9d704700 1 --2- 172.21.15.140:0/460377320 >> v2:172.21.15.152:3300/0 conn(0x7f2d980b59d0 0x7f2d980b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:57:38.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9cf03700 1 --2- 172.21.15.140:0/460377320 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d980af920 0x7f2d980ae3d0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f2d940032f0 tx=0x7f2d94009290).ready entity=mon.0 client_cookie=b8a056946aa9ceff server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d8ffff700 1 -- 172.21.15.140:0/460377320 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d9400b870 con 0x7f2d980af920 2022-07-21T01:57:38.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d8ffff700 1 -- 172.21.15.140:0/460377320 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9400b9d0 con 0x7f2d980af920 2022-07-21T01:57:38.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d8ffff700 1 -- 172.21.15.140:0/460377320 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d9400bb40 con 0x7f2d980af920 2022-07-21T01:57:38.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 -- 172.21.15.140:0/460377320 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d980af920 msgr2=0x7f2d980ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 --2- 172.21.15.140:0/460377320 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d980af920 0x7f2d980ae3d0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f2d940032f0 tx=0x7f2d94009290).stop 2022-07-21T01:57:38.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 -- 172.21.15.140:0/460377320 shutdown_connections 2022-07-21T01:57:38.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 --2- 172.21.15.140:0/460377320 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d9805d0c0 0x7f2d980ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 --2- 172.21.15.140:0/460377320 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d980af920 0x7f2d980ae3d0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 --2- 172.21.15.140:0/460377320 >> v2:172.21.15.152:3300/0 conn(0x7f2d980b59d0 0x7f2d980b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 -- 172.21.15.140:0/460377320 >> 172.21.15.140:0/460377320 conn(0x7f2d980634c0 msgr2=0x7f2d980b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:38.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 -- 172.21.15.140:0/460377320 shutdown_connections 2022-07-21T01:57:38.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 -- 172.21.15.140:0/460377320 wait complete. 2022-07-21T01:57:38.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 Processor -- start 2022-07-21T01:57:38.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 -- start start 2022-07-21T01:57:38.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d9805d0c0 0x7f2d98065af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d980af920 0x7f2d9813ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d980b59d0 0x7f2d98142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d981521c0 con 0x7f2d980b59d0 2022-07-21T01:57:38.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d98112690 con 0x7f2d980af920 2022-07-21T01:57:38.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9e706700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d98112930 con 0x7f2d9805d0c0 2022-07-21T01:57:38.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9df05700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d980b59d0 0x7f2d98142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9df05700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d980b59d0 0x7f2d98142f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40990/0 (socket says 172.21.15.140:40990) 2022-07-21T01:57:38.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.851+0000 7f2d9df05700 1 -- 172.21.15.140:0/319358879 learned_addr learned my addr 172.21.15.140:0/319358879 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:38.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9d704700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d9805d0c0 0x7f2d98065af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9cf03700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d980af920 0x7f2d9813ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9df05700 1 -- 172.21.15.140:0/319358879 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d9805d0c0 msgr2=0x7f2d98065af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9df05700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d9805d0c0 0x7f2d98065af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9df05700 1 -- 172.21.15.140:0/319358879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d980af920 msgr2=0x7f2d9813ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9df05700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d980af920 0x7f2d9813ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9df05700 1 -- 172.21.15.140:0/319358879 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d94009530 con 0x7f2d980b59d0 2022-07-21T01:57:38.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9cf03700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d980af920 0x7f2d9813ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:38.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9df05700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d980b59d0 0x7f2d98142f50 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f2d9000a000 tx=0x7f2d90008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d8e7fc700 1 -- 172.21.15.140:0/319358879 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d90011780 con 0x7f2d980b59d0 2022-07-21T01:57:38.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d8e7fc700 1 -- 172.21.15.140:0/319358879 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d900118e0 con 0x7f2d980b59d0 2022-07-21T01:57:38.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9e706700 1 -- 172.21.15.140:0/319358879 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d98113890 con 0x7f2d980b59d0 2022-07-21T01:57:38.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d8e7fc700 1 -- 172.21.15.140:0/319358879 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d90011c40 con 0x7f2d980b59d0 2022-07-21T01:57:38.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9e706700 1 -- 172.21.15.140:0/319358879 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d98065fe0 con 0x7f2d980b59d0 2022-07-21T01:57:38.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9e706700 1 -- 172.21.15.140:0/319358879 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d98112af0 con 0x7f2d980b59d0 2022-07-21T01:57:38.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d8e7fc700 1 -- 172.21.15.140:0/319358879 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2d90012420 con 0x7f2d980b59d0 2022-07-21T01:57:38.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d8e7fc700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d780539e0 0x7f2d78055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:38.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d8e7fc700 1 -- 172.21.15.140:0/319358879 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2d9006b960 con 0x7f2d980b59d0 2022-07-21T01:57:38.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9d704700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d780539e0 0x7f2d78055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:38.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.855+0000 7f2d9d704700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d780539e0 0x7f2d78055ea0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f2d8800d000 tx=0x7f2d88008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:38.881 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.863+0000 7f2d8e7fc700 1 -- 172.21.15.140:0/319358879 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2d90042b50 con 0x7f2d980b59d0 2022-07-21T01:57:38.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.991+0000 7f2d9e706700 1 -- 172.21.15.140:0/319358879 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2d980aedb0 con 0x7f2d980b59d0 2022-07-21T01:57:38.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.991+0000 7f2d8e7fc700 1 -- 172.21.15.140:0/319358879 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2d90046170 con 0x7f2d980b59d0 2022-07-21T01:57:38.994 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:38.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.991+0000 7f2d9e706700 1 -- 172.21.15.140:0/319358879 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d780539e0 msgr2=0x7f2d78055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.991+0000 7f2d9e706700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d780539e0 0x7f2d78055ea0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f2d8800d000 tx=0x7f2d88008040).stop 2022-07-21T01:57:38.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.991+0000 7f2d9e706700 1 -- 172.21.15.140:0/319358879 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d980b59d0 msgr2=0x7f2d98142f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:38.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.991+0000 7f2d9e706700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d980b59d0 0x7f2d98142f50 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f2d9000a000 tx=0x7f2d90008040).stop 2022-07-21T01:57:38.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.995+0000 7f2d9e706700 1 -- 172.21.15.140:0/319358879 shutdown_connections 2022-07-21T01:57:38.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.995+0000 7f2d9e706700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d780539e0 0x7f2d78055ea0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.995+0000 7f2d9e706700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2d980b59d0 0x7f2d98142f50 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.995+0000 7f2d9e706700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d980af920 0x7f2d9813ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.995+0000 7f2d9e706700 1 --2- 172.21.15.140:0/319358879 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2d9805d0c0 0x7f2d98065af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:38.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.995+0000 7f2d9e706700 1 -- 172.21.15.140:0/319358879 >> 172.21.15.140:0/319358879 conn(0x7f2d980634c0 msgr2=0x7f2d980b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:39.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.995+0000 7f2d9e706700 1 -- 172.21.15.140:0/319358879 shutdown_connections 2022-07-21T01:57:39.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:38.995+0000 7f2d9e706700 1 -- 172.21.15.140:0/319358879 wait complete. 2022-07-21T01:57:39.008 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:57:39.108 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc587fff700 1 Processor -- start 2022-07-21T01:57:39.109 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc587fff700 1 -- start start 2022-07-21T01:57:39.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc587fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc5880b0570 0x7fc5880b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:39.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc587fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5881433d0 con 0x7fc5881424e0 2022-07-21T01:57:39.111 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc587fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc588143670 con 0x7fc5880aea10 2022-07-21T01:57:39.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc587fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc588143b10 con 0x7fc5880b0570 2022-07-21T01:57:39.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc586ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc5880b0570 0x7fc5880b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:39.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc5877fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc5881424e0 0x7fc5880ad350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55144/0 (socket says 172.21.15.140:55144) 2022-07-21T01:57:39.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc586ffd700 1 --2- 172.21.15.140:0/4151458479 >> v2:172.21.15.140:3300/0 conn(0x7fc5880b0570 0x7fc5880b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:40994/0 (socket says 172.21.15.140:40994) 2022-07-21T01:57:39.112 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc5877fe700 1 -- 172.21.15.140:0/4151458479 learned_addr learned my addr 172.21.15.140:0/4151458479 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:39.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc586ffd700 1 -- 172.21.15.140:0/4151458479 >> v1:172.21.15.152:6789/0 conn(0x7fc5880aea10 legacy=0x7fc588141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc586ffd700 1 -- 172.21.15.140:0/4151458479 >> v1:172.21.15.140:6789/0 conn(0x7fc5881424e0 legacy=0x7fc5880ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc586ffd700 1 -- 172.21.15.140:0/4151458479 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5880b1010 con 0x7fc5880b0570 2022-07-21T01:57:39.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc585ffb700 1 -- 172.21.15.140:0/4151458479 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854129281 0 0) 0x7fc5881433d0 con 0x7fc5881424e0 2022-07-21T01:57:39.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc586ffd700 1 --2- 172.21.15.140:0/4151458479 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5880b0570 0x7fc5880b0950 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7fc570002f70 tx=0x7fc570006df0).ready entity=mon.0 client_cookie=d22aabbd963b0558 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:39.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc585ffb700 1 -- 172.21.15.140:0/4151458479 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc57000d610 con 0x7fc5880b0570 2022-07-21T01:57:39.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc585ffb700 1 -- 172.21.15.140:0/4151458479 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc570000e90 con 0x7fc5880b0570 2022-07-21T01:57:39.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc585ffb700 1 -- 172.21.15.140:0/4151458479 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc57000d960 con 0x7fc5880b0570 2022-07-21T01:57:39.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc587fff700 1 -- 172.21.15.140:0/4151458479 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5880b0570 msgr2=0x7fc5880b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc587fff700 1 --2- 172.21.15.140:0/4151458479 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5880b0570 0x7fc5880b0950 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7fc570002f70 tx=0x7fc570006df0).stop 2022-07-21T01:57:39.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc587fff700 1 -- 172.21.15.140:0/4151458479 shutdown_connections 2022-07-21T01:57:39.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc587fff700 1 --2- 172.21.15.140:0/4151458479 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5880b0570 0x7fc5880b0950 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.107+0000 7fc587fff700 1 -- 172.21.15.140:0/4151458479 >> 172.21.15.140:0/4151458479 conn(0x7fc5880b9360 msgr2=0x7fc5880b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:39.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc587fff700 1 -- 172.21.15.140:0/4151458479 shutdown_connections 2022-07-21T01:57:39.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc587fff700 1 -- 172.21.15.140:0/4151458479 wait complete. 2022-07-21T01:57:39.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc587fff700 1 Processor -- start 2022-07-21T01:57:39.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc587fff700 1 -- start start 2022-07-21T01:57:39.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc587fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5880aea10 0x7fc5880b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:39.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc587fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5880b0570 0x7fc588112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:39.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc587fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5881424e0 0x7fc588114bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:39.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc587fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881166b0 con 0x7fc5881424e0 2022-07-21T01:57:39.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc587fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5881126d0 con 0x7fc5880aea10 2022-07-21T01:57:39.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc587fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc588112970 con 0x7fc5880b0570 2022-07-21T01:57:39.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc5877fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5881424e0 0x7fc588114bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:39.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc5877fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5881424e0 0x7fc588114bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41004/0 (socket says 172.21.15.140:41004) 2022-07-21T01:57:39.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc586ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5880aea10 0x7fc5880b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:39.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc5877fe700 1 -- 172.21.15.140:0/3921343873 learned_addr learned my addr 172.21.15.140:0/3921343873 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:39.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc5867fc700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5880b0570 0x7fc588112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:39.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc5877fe700 1 -- 172.21.15.140:0/3921343873 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5880b0570 msgr2=0x7fc588112e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc5877fe700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5880b0570 0x7fc588112e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc5877fe700 1 -- 172.21.15.140:0/3921343873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5880aea10 msgr2=0x7fc5880b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc5877fe700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5880aea10 0x7fc5880b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc5877fe700 1 -- 172.21.15.140:0/3921343873 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57000b180 con 0x7fc5881424e0 2022-07-21T01:57:39.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.111+0000 7fc5877fe700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5881424e0 0x7fc588114bd0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7fc578000bc0 tx=0x7fc578007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:39.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.115+0000 7fc57ffff700 1 -- 172.21.15.140:0/3921343873 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc578001d30 con 0x7fc5881424e0 2022-07-21T01:57:39.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.115+0000 7fc57ffff700 1 -- 172.21.15.140:0/3921343873 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5780070d0 con 0x7fc5881424e0 2022-07-21T01:57:39.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.115+0000 7fc587fff700 1 -- 172.21.15.140:0/3921343873 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc588114180 con 0x7fc5881424e0 2022-07-21T01:57:39.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.115+0000 7fc57ffff700 1 -- 172.21.15.140:0/3921343873 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc578007430 con 0x7fc5881424e0 2022-07-21T01:57:39.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.115+0000 7fc587fff700 1 -- 172.21.15.140:0/3921343873 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc588117b70 con 0x7fc5881424e0 2022-07-21T01:57:39.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.115+0000 7fc57ffff700 1 -- 172.21.15.140:0/3921343873 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc57801a060 con 0x7fc5881424e0 2022-07-21T01:57:39.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.115+0000 7fc587fff700 1 -- 172.21.15.140:0/3921343873 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5880ace30 con 0x7fc5881424e0 2022-07-21T01:57:39.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.115+0000 7fc57ffff700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc56c0539e0 0x7fc56c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:39.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.115+0000 7fc57ffff700 1 -- 172.21.15.140:0/3921343873 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc57806b140 con 0x7fc5881424e0 2022-07-21T01:57:39.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.115+0000 7fc586ffd700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc56c0539e0 0x7fc56c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:39.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.115+0000 7fc586ffd700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc56c0539e0 0x7fc56c055ea0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fc57001c000 tx=0x7fc570015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:39.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.119+0000 7fc57ffff700 1 -- 172.21.15.140:0/3921343873 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc578042390 con 0x7fc5881424e0 2022-07-21T01:57:39.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.239+0000 7fc587fff700 1 -- 172.21.15.140:0/3921343873 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc5880574e0 con 0x7fc5881424e0 2022-07-21T01:57:39.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.239+0000 7fc57ffff700 1 -- 172.21.15.140:0/3921343873 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v137) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc578018020 con 0x7fc5881424e0 2022-07-21T01:57:39.244 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:39.244 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:57:39.244 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:39.244 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:57:39.245 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:57:39.245 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:39.245 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:39.245 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:57:39.245 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:57:39.246 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:39.246 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:57:39.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 -- 172.21.15.140:0/3921343873 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc56c0539e0 msgr2=0x7fc56c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc56c0539e0 0x7fc56c055ea0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fc57001c000 tx=0x7fc570015000).stop 2022-07-21T01:57:39.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 -- 172.21.15.140:0/3921343873 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5881424e0 msgr2=0x7fc588114bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5881424e0 0x7fc588114bd0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7fc578000bc0 tx=0x7fc578007bc0).stop 2022-07-21T01:57:39.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 -- 172.21.15.140:0/3921343873 shutdown_connections 2022-07-21T01:57:39.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc56c0539e0 0x7fc56c055ea0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc5881424e0 0x7fc588114bd0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5880b0570 0x7fc588112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 --2- 172.21.15.140:0/3921343873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5880aea10 0x7fc5880b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 -- 172.21.15.140:0/3921343873 >> 172.21.15.140:0/3921343873 conn(0x7fc5880b9360 msgr2=0x7fc588140380 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:39.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 -- 172.21.15.140:0/3921343873 shutdown_connections 2022-07-21T01:57:39.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.243+0000 7fc587fff700 1 -- 172.21.15.140:0/3921343873 wait complete. 2022-07-21T01:57:39.256 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:57:39.257 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:57:39.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb176741700 1 Processor -- start 2022-07-21T01:57:39.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb176741700 1 -- start start 2022-07-21T01:57:39.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb176741700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb1700b3ee0 0x7fb170128a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:39.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb176741700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1700b1b10 0x7fb170128f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:39.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb176741700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1700b42c0 con 0x7fb17005d130 2022-07-21T01:57:39.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb176741700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17012a310 con 0x7fb1700b3ee0 2022-07-21T01:57:39.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb176741700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb170067c90 con 0x7fb1700b1b10 2022-07-21T01:57:39.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb175f40700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fb17005d130 0x7fb17005bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55158/0 (socket says 172.21.15.140:55158) 2022-07-21T01:57:39.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb175f40700 1 -- 172.21.15.140:0/4148208901 learned_addr learned my addr 172.21.15.140:0/4148208901 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:39.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb17573f700 1 --2- 172.21.15.140:0/4148208901 >> v2:172.21.15.140:3300/0 conn(0x7fb1700b3ee0 0x7fb170128a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:39.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb174f3e700 1 --2- 172.21.15.140:0/4148208901 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1700b1b10 0x7fb170128f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:39.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb16ffff700 1 -- 172.21.15.140:0/4148208901 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608670361 0 0) 0x7fb1700b42c0 con 0x7fb17005d130 2022-07-21T01:57:39.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb16ffff700 1 -- 172.21.15.140:0/4148208901 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb15c003260 con 0x7fb17005d130 2022-07-21T01:57:39.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb17573f700 1 -- 172.21.15.140:0/4148208901 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1700b1b10 msgr2=0x7fb170128f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:39.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb17573f700 1 --2- 172.21.15.140:0/4148208901 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1700b1b10 0x7fb170128f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb17573f700 1 -- 172.21.15.140:0/4148208901 >> v1:172.21.15.140:6789/0 conn(0x7fb17005d130 legacy=0x7fb17005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb17573f700 1 -- 172.21.15.140:0/4148208901 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1700672b0 con 0x7fb1700b3ee0 2022-07-21T01:57:39.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb17573f700 1 --2- 172.21.15.140:0/4148208901 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1700b3ee0 0x7fb170128a70 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7fb170066280 tx=0x7fb160006df0).ready entity=mon.0 client_cookie=643ec00c966b982f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:39.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb16ffff700 1 -- 172.21.15.140:0/4148208901 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb16000d4a0 con 0x7fb1700b3ee0 2022-07-21T01:57:39.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb16ffff700 1 -- 172.21.15.140:0/4148208901 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb160000e90 con 0x7fb1700b3ee0 2022-07-21T01:57:39.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.359+0000 7fb16ffff700 1 -- 172.21.15.140:0/4148208901 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb16000d7f0 con 0x7fb1700b3ee0 2022-07-21T01:57:39.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 -- 172.21.15.140:0/4148208901 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1700b3ee0 msgr2=0x7fb170128a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 --2- 172.21.15.140:0/4148208901 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1700b3ee0 0x7fb170128a70 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7fb170066280 tx=0x7fb160006df0).stop 2022-07-21T01:57:39.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 -- 172.21.15.140:0/4148208901 shutdown_connections 2022-07-21T01:57:39.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 --2- 172.21.15.140:0/4148208901 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb1700b1b10 0x7fb170128f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 --2- 172.21.15.140:0/4148208901 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1700b3ee0 0x7fb170128a70 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 -- 172.21.15.140:0/4148208901 >> 172.21.15.140:0/4148208901 conn(0x7fb170064c50 msgr2=0x7fb170065030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:39.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 -- 172.21.15.140:0/4148208901 shutdown_connections 2022-07-21T01:57:39.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 -- 172.21.15.140:0/4148208901 wait complete. 2022-07-21T01:57:39.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 Processor -- start 2022-07-21T01:57:39.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 -- start start 2022-07-21T01:57:39.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb17005d130 0x7fb1700614e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:39.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1700b1b10 0x7fb17012af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:39.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1700b3ee0 0x7fb1701552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:39.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17012c600 con 0x7fb1700b1b10 2022-07-21T01:57:39.375 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb170060120 con 0x7fb1700b3ee0 2022-07-21T01:57:39.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb176741700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb170060380 con 0x7fb17005d130 2022-07-21T01:57:39.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb174f3e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1700b1b10 0x7fb17012af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:39.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb174f3e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1700b1b10 0x7fb17012af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41016/0 (socket says 172.21.15.140:41016) 2022-07-21T01:57:39.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb174f3e700 1 -- 172.21.15.140:0/1944747724 learned_addr learned my addr 172.21.15.140:0/1944747724 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:39.376 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb175f40700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1700b3ee0 0x7fb1701552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:39.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.363+0000 7fb17573f700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb17005d130 0x7fb1700614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:39.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb174f3e700 1 -- 172.21.15.140:0/1944747724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb17005d130 msgr2=0x7fb1700614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb174f3e700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb17005d130 0x7fb1700614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.377 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb174f3e700 1 -- 172.21.15.140:0/1944747724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1700b3ee0 msgr2=0x7fb1701552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb174f3e700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1700b3ee0 0x7fb1701552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb174f3e700 1 -- 172.21.15.140:0/1944747724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16000b180 con 0x7fb1700b1b10 2022-07-21T01:57:39.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb174f3e700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1700b1b10 0x7fb17012af50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7fb16400c000 tx=0x7fb164006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:39.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb16e7fc700 1 -- 172.21.15.140:0/1944747724 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb164010590 con 0x7fb1700b1b10 2022-07-21T01:57:39.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb16e7fc700 1 -- 172.21.15.140:0/1944747724 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1640106f0 con 0x7fb1700b1b10 2022-07-21T01:57:39.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb176741700 1 -- 172.21.15.140:0/1944747724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb170060620 con 0x7fb1700b1b10 2022-07-21T01:57:39.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb16e7fc700 1 -- 172.21.15.140:0/1944747724 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb164010a50 con 0x7fb1700b1b10 2022-07-21T01:57:39.379 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb176741700 1 -- 172.21.15.140:0/1944747724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb170147ec0 con 0x7fb1700b1b10 2022-07-21T01:57:39.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb176741700 1 -- 172.21.15.140:0/1944747724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1700574e0 con 0x7fb1700b1b10 2022-07-21T01:57:39.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb16e7fc700 1 -- 172.21.15.140:0/1944747724 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb164010bb0 con 0x7fb1700b1b10 2022-07-21T01:57:39.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb16e7fc700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb150053aa0 0x7fb150055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:39.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb16e7fc700 1 -- 172.21.15.140:0/1944747724 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb16406bb10 con 0x7fb1700b1b10 2022-07-21T01:57:39.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.367+0000 7fb17573f700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb150053aa0 0x7fb150055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:39.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.371+0000 7fb17573f700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb150053aa0 0x7fb150055f60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fb16001a000 tx=0x7fb160014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:39.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.375+0000 7fb16e7fc700 1 -- 172.21.15.140:0/1944747724 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb164042d60 con 0x7fb1700b1b10 2022-07-21T01:57:39.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.499+0000 7fb176741700 1 -- 172.21.15.140:0/1944747724 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb1700686e0 con 0x7fb1700b1b10 2022-07-21T01:57:39.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.879+0000 7fb16e7fc700 1 -- 172.21.15.140:0/1944747724 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v138) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb164046380 con 0x7fb1700b1b10 2022-07-21T01:57:39.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.879+0000 7fb176741700 1 -- 172.21.15.140:0/1944747724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb150053aa0 msgr2=0x7fb150055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.879+0000 7fb176741700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb150053aa0 0x7fb150055f60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fb16001a000 tx=0x7fb160014000).stop 2022-07-21T01:57:39.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.883+0000 7fb176741700 1 -- 172.21.15.140:0/1944747724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1700b1b10 msgr2=0x7fb17012af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:39.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.883+0000 7fb176741700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1700b1b10 0x7fb17012af50 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7fb16400c000 tx=0x7fb164006df0).stop 2022-07-21T01:57:39.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.883+0000 7fb176741700 1 -- 172.21.15.140:0/1944747724 shutdown_connections 2022-07-21T01:57:39.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.883+0000 7fb176741700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb150053aa0 0x7fb150055f60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.883+0000 7fb176741700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1700b3ee0 0x7fb1701552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.883+0000 7fb176741700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb1700b1b10 0x7fb17012af50 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.883+0000 7fb176741700 1 --2- 172.21.15.140:0/1944747724 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb17005d130 0x7fb1700614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:39.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.883+0000 7fb176741700 1 -- 172.21.15.140:0/1944747724 >> 172.21.15.140:0/1944747724 conn(0x7fb170064c50 msgr2=0x7fb170140ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:39.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.883+0000 7fb176741700 1 -- 172.21.15.140:0/1944747724 shutdown_connections 2022-07-21T01:57:39.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:39.883+0000 7fb176741700 1 -- 172.21.15.140:0/1944747724 wait complete. 2022-07-21T01:57:39.888 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:57:39.900 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:57:40.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.015+0000 7f432e6c6700 1 Processor -- start 2022-07-21T01:57:40.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.015+0000 7f432e6c6700 1 -- start start 2022-07-21T01:57:40.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432e6c6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b0350 0x7f432814a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:40.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432e6c6700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43280aabe0 con 0x7f43280b4b10 2022-07-21T01:57:40.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432e6c6700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43280b4ef0 con 0x7f43280b0890 2022-07-21T01:57:40.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432e6c6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43280aae10 con 0x7f43280b0350 2022-07-21T01:57:40.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432d6c4700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f43280b4b10 0x7f43280af8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55168/0 (socket says 172.21.15.140:55168) 2022-07-21T01:57:40.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432d6c4700 1 -- 172.21.15.140:0/696950498 learned_addr learned my addr 172.21.15.140:0/696950498 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:40.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432dec5700 1 --2- 172.21.15.140:0/696950498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b0350 0x7f432814a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:40.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f4317fff700 1 -- 172.21.15.140:0/696950498 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164310845 0 0) 0x7f43280aabe0 con 0x7f43280b4b10 2022-07-21T01:57:40.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f4317fff700 1 -- 172.21.15.140:0/696950498 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4310003260 con 0x7f43280b4b10 2022-07-21T01:57:40.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f4317fff700 1 -- 172.21.15.140:0/696950498 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107077033 0 0) 0x7f43280b4ef0 con 0x7f43280b0890 2022-07-21T01:57:40.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f4317fff700 1 -- 172.21.15.140:0/696950498 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43280aabe0 con 0x7f43280b0890 2022-07-21T01:57:40.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432dec5700 1 -- 172.21.15.140:0/696950498 >> v1:172.21.15.152:6789/0 conn(0x7f43280b0890 legacy=0x7f43280ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:40.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432dec5700 1 -- 172.21.15.140:0/696950498 >> v1:172.21.15.140:6789/0 conn(0x7f43280b4b10 legacy=0x7f43280af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:40.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432dec5700 1 -- 172.21.15.140:0/696950498 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280ab0b0 con 0x7f43280b0350 2022-07-21T01:57:40.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432dec5700 1 --2- 172.21.15.140:0/696950498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b0350 0x7f432814a3d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f43200032f0 tx=0x7f43200091d0).ready entity=mon.2 client_cookie=fa750bcdcae3a697 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:40.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f4317fff700 1 -- 172.21.15.140:0/696950498 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432000b640 con 0x7f43280b0350 2022-07-21T01:57:40.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f4317fff700 1 -- 172.21.15.140:0/696950498 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432000b7a0 con 0x7f43280b0350 2022-07-21T01:57:40.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f4317fff700 1 -- 172.21.15.140:0/696950498 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f432000b910 con 0x7f43280b0350 2022-07-21T01:57:40.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432e6c6700 1 -- 172.21.15.140:0/696950498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b0350 msgr2=0x7f432814a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:40.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432e6c6700 1 --2- 172.21.15.140:0/696950498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b0350 0x7f432814a3d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f43200032f0 tx=0x7f43200091d0).stop 2022-07-21T01:57:40.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432e6c6700 1 -- 172.21.15.140:0/696950498 shutdown_connections 2022-07-21T01:57:40.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432e6c6700 1 --2- 172.21.15.140:0/696950498 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b0350 0x7f432814a3d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:40.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432e6c6700 1 -- 172.21.15.140:0/696950498 >> 172.21.15.140:0/696950498 conn(0x7f43280632b0 msgr2=0x7f4328063690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:40.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432e6c6700 1 -- 172.21.15.140:0/696950498 shutdown_connections 2022-07-21T01:57:40.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.019+0000 7f432e6c6700 1 -- 172.21.15.140:0/696950498 wait complete. 2022-07-21T01:57:40.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432e6c6700 1 Processor -- start 2022-07-21T01:57:40.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432e6c6700 1 -- start start 2022-07-21T01:57:40.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432e6c6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43280b0350 0x7f4328145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:40.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432e6c6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43280b0890 0x7f43281422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:40.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432e6c6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b4b10 0x7f4328141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:40.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432e6c6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432809b910 con 0x7f43280b0350 2022-07-21T01:57:40.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432e6c6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4328067120 con 0x7f43280b0890 2022-07-21T01:57:40.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432e6c6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4328140fa0 con 0x7f43280b4b10 2022-07-21T01:57:40.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432d6c4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43280b0350 0x7f4328145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:40.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432d6c4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43280b0350 0x7f4328145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41028/0 (socket says 172.21.15.140:41028) 2022-07-21T01:57:40.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432d6c4700 1 -- 172.21.15.140:0/2231720068 learned_addr learned my addr 172.21.15.140:0/2231720068 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:40.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432dec5700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b4b10 0x7f4328141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:40.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432cec3700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43280b0890 0x7f43281422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:40.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432d6c4700 1 -- 172.21.15.140:0/2231720068 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b4b10 msgr2=0x7f4328141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:40.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432d6c4700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b4b10 0x7f4328141940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:40.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432d6c4700 1 -- 172.21.15.140:0/2231720068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43280b0890 msgr2=0x7f43281422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:40.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432d6c4700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43280b0890 0x7f43281422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:40.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432d6c4700 1 -- 172.21.15.140:0/2231720068 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4320009470 con 0x7f43280b0350 2022-07-21T01:57:40.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432cec3700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43280b0890 0x7f43281422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:40.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432dec5700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b4b10 0x7f4328141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:40.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432d6c4700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43280b0350 0x7f4328145970 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f4328067260 tx=0x7f4324007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:40.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f43167fc700 1 -- 172.21.15.140:0/2231720068 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43240070d0 con 0x7f43280b0350 2022-07-21T01:57:40.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f43167fc700 1 -- 172.21.15.140:0/2231720068 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4324007230 con 0x7f43280b0350 2022-07-21T01:57:40.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432e6c6700 1 -- 172.21.15.140:0/2231720068 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43281427c0 con 0x7f43280b0350 2022-07-21T01:57:40.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f43167fc700 1 -- 172.21.15.140:0/2231720068 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4324005610 con 0x7f43280b0350 2022-07-21T01:57:40.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432e6c6700 1 -- 172.21.15.140:0/2231720068 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4328140d00 con 0x7f43280b0350 2022-07-21T01:57:40.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f432e6c6700 1 -- 172.21.15.140:0/2231720068 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43281413b0 con 0x7f43280b0350 2022-07-21T01:57:40.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.023+0000 7f43167fc700 1 -- 172.21.15.140:0/2231720068 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4324010070 con 0x7f43280b0350 2022-07-21T01:57:40.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.027+0000 7f43167fc700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4308053950 0x7f4308055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:40.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.027+0000 7f43167fc700 1 -- 172.21.15.140:0/2231720068 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f432406b500 con 0x7f43280b0350 2022-07-21T01:57:40.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.027+0000 7f432cec3700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4308053950 0x7f4308055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:40.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.027+0000 7f432cec3700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4308053950 0x7f4308055e10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f431800b000 tx=0x7f4318009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:40.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.031+0000 7f43167fc700 1 -- 172.21.15.140:0/2231720068 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4324042550 con 0x7f43280b0350 2022-07-21T01:57:40.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.155+0000 7f432e6c6700 1 -- 172.21.15.140:0/2231720068 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f43280574e0 con 0x7f43280b0350 2022-07-21T01:57:40.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.883+0000 7f43167fc700 1 -- 172.21.15.140:0/2231720068 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v139) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4324045b70 con 0x7f43280b0350 2022-07-21T01:57:40.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 -- 172.21.15.140:0/2231720068 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4308053950 msgr2=0x7f4308055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:40.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4308053950 0x7f4308055e10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f431800b000 tx=0x7f4318009040).stop 2022-07-21T01:57:40.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 -- 172.21.15.140:0/2231720068 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43280b0350 msgr2=0x7f4328145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:40.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43280b0350 0x7f4328145970 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f4328067260 tx=0x7f4324007b50).stop 2022-07-21T01:57:40.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 -- 172.21.15.140:0/2231720068 shutdown_connections 2022-07-21T01:57:40.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4308053950 0x7f4308055e10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:40.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f43280b4b10 0x7f4328141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:40.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f43280b0890 0x7f43281422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:40.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 --2- 172.21.15.140:0/2231720068 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f43280b0350 0x7f4328145970 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:40.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 -- 172.21.15.140:0/2231720068 >> 172.21.15.140:0/2231720068 conn(0x7f43280632b0 msgr2=0x7f43280b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:40.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 -- 172.21.15.140:0/2231720068 shutdown_connections 2022-07-21T01:57:40.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:40.891+0000 7f432e6c6700 1 -- 172.21.15.140:0/2231720068 wait complete. 2022-07-21T01:57:40.898 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:57:40.908 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:57:41.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c3a417700 1 Processor -- start 2022-07-21T01:57:41.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c3a417700 1 -- start start 2022-07-21T01:57:41.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c3a417700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5c340b5990 0x7f5c34146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c3a417700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5c340af3e0 0x7f5c34146560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c3a417700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c3405d210 0x7f5c3414a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c3a417700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c340ad050 con 0x7f5c340af3e0 2022-07-21T01:57:41.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c3a417700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c340b17e0 con 0x7f5c340b5990 2022-07-21T01:57:41.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c3a417700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c340b5d70 con 0x7f5c3405d210 2022-07-21T01:57:41.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c38c14700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5c340af3e0 0x7f5c34146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c38c14700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5c340af3e0 0x7f5c34146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41038/0 (socket says 172.21.15.140:41038) 2022-07-21T01:57:41.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c38c14700 1 -- 172.21.15.140:0/2782623240 learned_addr learned my addr 172.21.15.140:0/2782623240 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:41.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.011+0000 7f5c39c16700 1 --2- 172.21.15.140:0/2782623240 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c3405d210 0x7f5c3414a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c39415700 1 --2- 172.21.15.140:0/2782623240 >> v2:172.21.15.152:3300/0 conn(0x7f5c340b5990 0x7f5c34146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c38c14700 1 -- 172.21.15.140:0/2782623240 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c3405d210 msgr2=0x7f5c3414a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:41.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c38c14700 1 --2- 172.21.15.140:0/2782623240 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c3405d210 0x7f5c3414a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c38c14700 1 -- 172.21.15.140:0/2782623240 >> v2:172.21.15.152:3300/0 conn(0x7f5c340b5990 msgr2=0x7f5c34146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:41.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c38c14700 1 --2- 172.21.15.140:0/2782623240 >> v2:172.21.15.152:3300/0 conn(0x7f5c340b5990 0x7f5c34146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c38c14700 1 -- 172.21.15.140:0/2782623240 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c340b1010 con 0x7f5c340af3e0 2022-07-21T01:57:41.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c38c14700 1 --2- 172.21.15.140:0/2782623240 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c340af3e0 0x7f5c34146560 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f5c240032f0 tx=0x7f5c24009290).ready entity=mon.0 client_cookie=531d78009aee4d7f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c2bfff700 1 -- 172.21.15.140:0/2782623240 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c2400b680 con 0x7f5c340af3e0 2022-07-21T01:57:41.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c2bfff700 1 -- 172.21.15.140:0/2782623240 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2400b7e0 con 0x7f5c340af3e0 2022-07-21T01:57:41.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c2bfff700 1 -- 172.21.15.140:0/2782623240 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c2400b950 con 0x7f5c340af3e0 2022-07-21T01:57:41.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c3a417700 1 -- 172.21.15.140:0/2782623240 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c340af3e0 msgr2=0x7f5c34146560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c3a417700 1 --2- 172.21.15.140:0/2782623240 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c340af3e0 0x7f5c34146560 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f5c240032f0 tx=0x7f5c24009290).stop 2022-07-21T01:57:41.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c3a417700 1 -- 172.21.15.140:0/2782623240 shutdown_connections 2022-07-21T01:57:41.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c3a417700 1 --2- 172.21.15.140:0/2782623240 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c3405d210 0x7f5c3414a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c3a417700 1 --2- 172.21.15.140:0/2782623240 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c340af3e0 0x7f5c34146560 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c3a417700 1 --2- 172.21.15.140:0/2782623240 >> v2:172.21.15.152:3300/0 conn(0x7f5c340b5990 0x7f5c34146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c3a417700 1 -- 172.21.15.140:0/2782623240 >> 172.21.15.140:0/2782623240 conn(0x7f5c34063480 msgr2=0x7f5c340b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:41.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c3a417700 1 -- 172.21.15.140:0/2782623240 shutdown_connections 2022-07-21T01:57:41.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c3a417700 1 -- 172.21.15.140:0/2782623240 wait complete. 2022-07-21T01:57:41.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c3a417700 1 Processor -- start 2022-07-21T01:57:41.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.015+0000 7f5c3a417700 1 -- start start 2022-07-21T01:57:41.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c3a417700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c3405d210 0x7f5c34111720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c3a417700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c340af3e0 0x7f5c3414a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c3a417700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c340b5990 0x7f5c3410eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c3a417700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34110000 con 0x7f5c3405d210 2022-07-21T01:57:41.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c3a417700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3410e5b0 con 0x7f5c340b5990 2022-07-21T01:57:41.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c3a417700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34142eb0 con 0x7f5c340af3e0 2022-07-21T01:57:41.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c39415700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c3405d210 0x7f5c34111720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c39415700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c3405d210 0x7f5c34111720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41042/0 (socket says 172.21.15.140:41042) 2022-07-21T01:57:41.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c39415700 1 -- 172.21.15.140:0/1016849486 learned_addr learned my addr 172.21.15.140:0/1016849486 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:41.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c38c14700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c340af3e0 0x7f5c3414a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c39c16700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c340b5990 0x7f5c3410eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c39415700 1 -- 172.21.15.140:0/1016849486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c340af3e0 msgr2=0x7f5c3414a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c39415700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c340af3e0 0x7f5c3414a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c39415700 1 -- 172.21.15.140:0/1016849486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c340b5990 msgr2=0x7f5c3410eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c39415700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c340b5990 0x7f5c3410eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c39415700 1 -- 172.21.15.140:0/1016849486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24009530 con 0x7f5c3405d210 2022-07-21T01:57:41.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c39c16700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c340b5990 0x7f5c3410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:41.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c39415700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c3405d210 0x7f5c34111720 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f5c34144160 tx=0x7f5c30006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c2a7fc700 1 -- 172.21.15.140:0/1016849486 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c30011750 con 0x7f5c3405d210 2022-07-21T01:57:41.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c2a7fc700 1 -- 172.21.15.140:0/1016849486 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c300118b0 con 0x7f5c3405d210 2022-07-21T01:57:41.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c3a417700 1 -- 172.21.15.140:0/1016849486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c34144b50 con 0x7f5c3405d210 2022-07-21T01:57:41.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c2a7fc700 1 -- 172.21.15.140:0/1016849486 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c30011c10 con 0x7f5c3405d210 2022-07-21T01:57:41.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c3a417700 1 -- 172.21.15.140:0/1016849486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3410e780 con 0x7f5c3405d210 2022-07-21T01:57:41.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c2a7fc700 1 -- 172.21.15.140:0/1016849486 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5c30011d70 con 0x7f5c3405d210 2022-07-21T01:57:41.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c3a417700 1 -- 172.21.15.140:0/1016849486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c340afa60 con 0x7f5c3405d210 2022-07-21T01:57:41.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.019+0000 7f5c2a7fc700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5c14053a70 0x7f5c14055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.023+0000 7f5c2a7fc700 1 -- 172.21.15.140:0/1016849486 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f5c3006bd20 con 0x7f5c3405d210 2022-07-21T01:57:41.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.023+0000 7f5c38c14700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5c14053a70 0x7f5c14055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.023+0000 7f5c38c14700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5c14053a70 0x7f5c14055f30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f5c2401a000 tx=0x7f5c24017000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.027+0000 7f5c2a7fc700 1 -- 172.21.15.140:0/1016849486 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5c30042ac0 con 0x7f5c3405d210 2022-07-21T01:57:41.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.187+0000 7f5c3a417700 1 -- 172.21.15.140:0/1016849486 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f5c34142ff0 con 0x7f5c3405d210 2022-07-21T01:57:41.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c2a7fc700 1 -- 172.21.15.140:0/1016849486 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 27 and data pool 28 v68) v1 ==== 189+0+0 (secure 0 0 0) 0x7f5c300460e0 con 0x7f5c3405d210 2022-07-21T01:57:41.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 -- 172.21.15.140:0/1016849486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5c14053a70 msgr2=0x7f5c14055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5c14053a70 0x7f5c14055f30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f5c2401a000 tx=0x7f5c24017000).stop 2022-07-21T01:57:41.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 -- 172.21.15.140:0/1016849486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c3405d210 msgr2=0x7f5c34111720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c3405d210 0x7f5c34111720 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f5c34144160 tx=0x7f5c30006d90).stop 2022-07-21T01:57:41.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 -- 172.21.15.140:0/1016849486 shutdown_connections 2022-07-21T01:57:41.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5c14053a70 0x7f5c14055f30 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c340b5990 0x7f5c3410eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5c340af3e0 0x7f5c3414a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 --2- 172.21.15.140:0/1016849486 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5c3405d210 0x7f5c34111720 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 -- 172.21.15.140:0/1016849486 >> 172.21.15.140:0/1016849486 conn(0x7f5c34063480 msgr2=0x7f5c340b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:41.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 -- 172.21.15.140:0/1016849486 shutdown_connections 2022-07-21T01:57:41.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.199+0000 7f5c3a417700 1 -- 172.21.15.140:0/1016849486 wait complete. 2022-07-21T01:57:41.216 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 27 and data pool 28 2022-07-21T01:57:41.218 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:41.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4184c700 1 Processor -- start 2022-07-21T01:57:41.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4184c700 1 -- start start 2022-07-21T01:57:41.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4184c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec3c0b61d0 0x7fec3c0b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4184c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fec3c0af120 0x7fec3c067260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4184c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3c0b20f0 con 0x7fec3c0b13e0 2022-07-21T01:57:41.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4184c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3c13efd0 con 0x7fec3c0af120 2022-07-21T01:57:41.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4184c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0b65b0 con 0x7fec3c0b61d0 2022-07-21T01:57:41.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec3bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fec3c0b13e0 0x7fec3c0aea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55206/0 (socket says 172.21.15.140:55206) 2022-07-21T01:57:41.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec3bfff700 1 -- 172.21.15.140:0/842193967 learned_addr learned my addr 172.21.15.140:0/842193967 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:41.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4084a700 1 --2- 172.21.15.140:0/842193967 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec3c0b61d0 0x7fec3c0b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4104b700 1 --2- 172.21.15.140:0/842193967 >> v2:172.21.15.152:3300/0 conn(0x7fec3c0af120 0x7fec3c067260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec3b7fe700 1 -- 172.21.15.140:0/842193967 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161304652 0 0) 0x7fec3c0b20f0 con 0x7fec3c0b13e0 2022-07-21T01:57:41.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec3b7fe700 1 -- 172.21.15.140:0/842193967 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec1c003260 con 0x7fec3c0b13e0 2022-07-21T01:57:41.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4084a700 1 -- 172.21.15.140:0/842193967 >> v2:172.21.15.152:3300/0 conn(0x7fec3c0af120 msgr2=0x7fec3c067260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:41.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4084a700 1 --2- 172.21.15.140:0/842193967 >> v2:172.21.15.152:3300/0 conn(0x7fec3c0af120 0x7fec3c067260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4084a700 1 -- 172.21.15.140:0/842193967 >> v1:172.21.15.140:6789/0 conn(0x7fec3c0b13e0 legacy=0x7fec3c0aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4084a700 1 -- 172.21.15.140:0/842193967 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c0b1ad0 con 0x7fec3c0b61d0 2022-07-21T01:57:41.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec4084a700 1 --2- 172.21.15.140:0/842193967 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec3c0b61d0 0x7fec3c0b0460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fec2c002f70 tx=0x7fec2c006df0).ready entity=mon.2 client_cookie=faecc0212ec481b5 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec3b7fe700 1 -- 172.21.15.140:0/842193967 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec2c00d4a0 con 0x7fec3c0b61d0 2022-07-21T01:57:41.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec3b7fe700 1 -- 172.21.15.140:0/842193967 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c000e90 con 0x7fec3c0b61d0 2022-07-21T01:57:41.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.307+0000 7fec3b7fe700 1 -- 172.21.15.140:0/842193967 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec2c00d7f0 con 0x7fec3c0b61d0 2022-07-21T01:57:41.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 -- 172.21.15.140:0/842193967 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec3c0b61d0 msgr2=0x7fec3c0b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 --2- 172.21.15.140:0/842193967 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec3c0b61d0 0x7fec3c0b0460 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fec2c002f70 tx=0x7fec2c006df0).stop 2022-07-21T01:57:41.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 -- 172.21.15.140:0/842193967 shutdown_connections 2022-07-21T01:57:41.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 --2- 172.21.15.140:0/842193967 >> v2:172.21.15.152:3300/0 conn(0x7fec3c0af120 0x7fec3c067260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 --2- 172.21.15.140:0/842193967 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec3c0b61d0 0x7fec3c0b0460 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 -- 172.21.15.140:0/842193967 >> 172.21.15.140:0/842193967 conn(0x7fec3c063cf0 msgr2=0x7fec3c0b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:41.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 -- 172.21.15.140:0/842193967 shutdown_connections 2022-07-21T01:57:41.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 -- 172.21.15.140:0/842193967 wait complete. 2022-07-21T01:57:41.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 Processor -- start 2022-07-21T01:57:41.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 -- start start 2022-07-21T01:57:41.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec3c0af120 0x7fec3c10fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec3c0b13e0 0x7fec3c13c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec3c0b61d0 0x7fec3c13c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c112640 con 0x7fec3c0b61d0 2022-07-21T01:57:41.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c13cd20 con 0x7fec3c0b13e0 2022-07-21T01:57:41.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4184c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1123a0 con 0x7fec3c0af120 2022-07-21T01:57:41.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4104b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec3c0b61d0 0x7fec3c13c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4104b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec3c0b61d0 0x7fec3c13c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41070/0 (socket says 172.21.15.140:41070) 2022-07-21T01:57:41.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4084a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec3c0af120 0x7fec3c10fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec3bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec3c0b13e0 0x7fec3c13c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4104b700 1 -- 172.21.15.140:0/2737048533 learned_addr learned my addr 172.21.15.140:0/2737048533 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:41.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4104b700 1 -- 172.21.15.140:0/2737048533 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec3c0af120 msgr2=0x7fec3c10fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.311+0000 7fec4104b700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec3c0af120 0x7fec3c10fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec4104b700 1 -- 172.21.15.140:0/2737048533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec3c0b13e0 msgr2=0x7fec3c13c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec4104b700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec3c0b13e0 0x7fec3c13c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec4104b700 1 -- 172.21.15.140:0/2737048533 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c00b180 con 0x7fec3c0b61d0 2022-07-21T01:57:41.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec4104b700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec3c0b61d0 0x7fec3c13c830 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7fec34006e00 tx=0x7fec340091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec39ffb700 1 -- 172.21.15.140:0/2737048533 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec34012600 con 0x7fec3c0b61d0 2022-07-21T01:57:41.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec39ffb700 1 -- 172.21.15.140:0/2737048533 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec34012760 con 0x7fec3c0b61d0 2022-07-21T01:57:41.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec4184c700 1 -- 172.21.15.140:0/2737048533 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c066be0 con 0x7fec3c0b61d0 2022-07-21T01:57:41.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec39ffb700 1 -- 172.21.15.140:0/2737048533 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec34012ac0 con 0x7fec3c0b61d0 2022-07-21T01:57:41.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec4184c700 1 -- 172.21.15.140:0/2737048533 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c067080 con 0x7fec3c0b61d0 2022-07-21T01:57:41.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec4184c700 1 -- 172.21.15.140:0/2737048533 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3c0b1d70 con 0x7fec3c0b61d0 2022-07-21T01:57:41.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec39ffb700 1 -- 172.21.15.140:0/2737048533 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fec34022070 con 0x7fec3c0b61d0 2022-07-21T01:57:41.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec39ffb700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec20053a70 0x7fec20055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec39ffb700 1 -- 172.21.15.140:0/2737048533 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec3406c440 con 0x7fec3c0b61d0 2022-07-21T01:57:41.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.315+0000 7fec4084a700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec20053a70 0x7fec20055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.319+0000 7fec4084a700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec20053a70 0x7fec20055f30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fec2c01a000 tx=0x7fec2c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.323+0000 7fec39ffb700 1 -- 172.21.15.140:0/2737048533 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fec340432c0 con 0x7fec3c0b61d0 2022-07-21T01:57:41.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.451+0000 7fec4184c700 1 -- 172.21.15.140:0/2737048533 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec3c0574e0 con 0x7fec3c0b61d0 2022-07-21T01:57:41.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec39ffb700 1 -- 172.21.15.140:0/2737048533 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16161 (secure 0 0 0) 0x7fec34016090 con 0x7fec3c0b61d0 2022-07-21T01:57:41.458 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:41.458 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":141,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:41.192239+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:57:39.555617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-07-21T01:57:40.211694+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:41.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 -- 172.21.15.140:0/2737048533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec20053a70 msgr2=0x7fec20055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec20053a70 0x7fec20055f30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7fec2c01a000 tx=0x7fec2c014000).stop 2022-07-21T01:57:41.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 -- 172.21.15.140:0/2737048533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec3c0b61d0 msgr2=0x7fec3c13c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec3c0b61d0 0x7fec3c13c830 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7fec34006e00 tx=0x7fec340091d0).stop 2022-07-21T01:57:41.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 -- 172.21.15.140:0/2737048533 shutdown_connections 2022-07-21T01:57:41.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fec20053a70 0x7fec20055f30 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fec3c0b61d0 0x7fec3c13c830 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec3c0b13e0 0x7fec3c13c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 --2- 172.21.15.140:0/2737048533 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fec3c0af120 0x7fec3c10fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 -- 172.21.15.140:0/2737048533 >> 172.21.15.140:0/2737048533 conn(0x7fec3c063cf0 msgr2=0x7fec3c0b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:41.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 -- 172.21.15.140:0/2737048533 shutdown_connections 2022-07-21T01:57:41.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.455+0000 7fec4184c700 1 -- 172.21.15.140:0/2737048533 wait complete. 2022-07-21T01:57:41.471 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:41.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ee81700 1 Processor -- start 2022-07-21T01:57:41.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ee81700 1 -- start start 2022-07-21T01:57:41.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ee81700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb3880b5650 0x7fb3880b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ee81700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb3880af8e0 0x7fb38805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.585 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ee81700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb388147010 con 0x7fb3880adff0 2022-07-21T01:57:41.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ee81700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3881472b0 con 0x7fb3880b5650 2022-07-21T01:57:41.586 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ee81700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb388146d40 con 0x7fb3880af8e0 2022-07-21T01:57:41.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38de7f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb3880b5650 0x7fb3880b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38de7f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fb3880b5650 0x7fb3880b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41074/0 (socket says 172.21.15.140:41074) 2022-07-21T01:57:41.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38de7f700 1 -- 172.21.15.140:0/3357919869 learned_addr learned my addr 172.21.15.140:0/3357919869 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:41.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38e680700 1 --2- 172.21.15.140:0/3357919869 >> v2:172.21.15.152:3300/0 conn(0x7fb3880af8e0 0x7fb38805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ce7d700 1 -- 172.21.15.140:0/3357919869 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159128298 0 0) 0x7fb388147010 con 0x7fb3880adff0 2022-07-21T01:57:41.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38de7f700 1 -- 172.21.15.140:0/3357919869 >> v2:172.21.15.152:3300/0 conn(0x7fb3880af8e0 msgr2=0x7fb38805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:41.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38de7f700 1 --2- 172.21.15.140:0/3357919869 >> v2:172.21.15.152:3300/0 conn(0x7fb3880af8e0 0x7fb38805bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38de7f700 1 -- 172.21.15.140:0/3357919869 >> v1:172.21.15.152:6789/0 conn(0x7fb3880adff0 legacy=0x7fb388147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38de7f700 1 -- 172.21.15.140:0/3357919869 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880acae0 con 0x7fb3880b5650 2022-07-21T01:57:41.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38de7f700 1 --2- 172.21.15.140:0/3357919869 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880b5650 0x7fb3880b5a30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7fb378002f70 tx=0x7fb378006eb0).ready entity=mon.0 client_cookie=96044dd11112d6d6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ce7d700 1 -- 172.21.15.140:0/3357919869 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb37800d5c0 con 0x7fb3880b5650 2022-07-21T01:57:41.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ce7d700 1 -- 172.21.15.140:0/3357919869 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb378000e90 con 0x7fb3880b5650 2022-07-21T01:57:41.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ce7d700 1 -- 172.21.15.140:0/3357919869 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb37800d910 con 0x7fb3880b5650 2022-07-21T01:57:41.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ee81700 1 -- 172.21.15.140:0/3357919869 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880b5650 msgr2=0x7fb3880b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.571+0000 7fb38ee81700 1 --2- 172.21.15.140:0/3357919869 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880b5650 0x7fb3880b5a30 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7fb378002f70 tx=0x7fb378006eb0).stop 2022-07-21T01:57:41.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 -- 172.21.15.140:0/3357919869 shutdown_connections 2022-07-21T01:57:41.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 --2- 172.21.15.140:0/3357919869 >> v2:172.21.15.152:3300/0 conn(0x7fb3880af8e0 0x7fb38805bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 --2- 172.21.15.140:0/3357919869 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880b5650 0x7fb3880b5a30 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 -- 172.21.15.140:0/3357919869 >> 172.21.15.140:0/3357919869 conn(0x7fb3880634e0 msgr2=0x7fb3880b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:41.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 -- 172.21.15.140:0/3357919869 shutdown_connections 2022-07-21T01:57:41.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 -- 172.21.15.140:0/3357919869 wait complete. 2022-07-21T01:57:41.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 Processor -- start 2022-07-21T01:57:41.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 -- start start 2022-07-21T01:57:41.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3880adff0 0x7fb3881111c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880af8e0 0x7fb388064100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3880b5650 0x7fb3880645f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb388067700 con 0x7fb3880af8e0 2022-07-21T01:57:41.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880679a0 con 0x7fb3880b5650 2022-07-21T01:57:41.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb388063c60 con 0x7fb3880adff0 2022-07-21T01:57:41.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38d67e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880af8e0 0x7fb388064100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38d67e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880af8e0 0x7fb388064100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41082/0 (socket says 172.21.15.140:41082) 2022-07-21T01:57:41.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38d67e700 1 -- 172.21.15.140:0/1022215509 learned_addr learned my addr 172.21.15.140:0/1022215509 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:41.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38de7f700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3880adff0 0x7fb3881111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38e680700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3880b5650 0x7fb3880645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38d67e700 1 -- 172.21.15.140:0/1022215509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3880adff0 msgr2=0x7fb3881111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38d67e700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3880adff0 0x7fb3881111c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38d67e700 1 -- 172.21.15.140:0/1022215509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3880b5650 msgr2=0x7fb3880645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38d67e700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3880b5650 0x7fb3880645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38d67e700 1 -- 172.21.15.140:0/1022215509 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37800b1e0 con 0x7fb3880af8e0 2022-07-21T01:57:41.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38d67e700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880af8e0 0x7fb388064100 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7fb380001450 tx=0x7fb380007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb37effd700 1 -- 172.21.15.140:0/1022215509 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb38000e4a0 con 0x7fb3880af8e0 2022-07-21T01:57:41.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb37effd700 1 -- 172.21.15.140:0/1022215509 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38000e600 con 0x7fb3880af8e0 2022-07-21T01:57:41.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb38ee81700 1 -- 172.21.15.140:0/1022215509 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb388067380 con 0x7fb3880af8e0 2022-07-21T01:57:41.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.575+0000 7fb37effd700 1 -- 172.21.15.140:0/1022215509 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb38000e960 con 0x7fb3880af8e0 2022-07-21T01:57:41.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.579+0000 7fb38ee81700 1 -- 172.21.15.140:0/1022215509 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb388064c80 con 0x7fb3880af8e0 2022-07-21T01:57:41.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.579+0000 7fb38ee81700 1 -- 172.21.15.140:0/1022215509 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3880574e0 con 0x7fb3880af8e0 2022-07-21T01:57:41.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.579+0000 7fb37effd700 1 -- 172.21.15.140:0/1022215509 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb380019030 con 0x7fb3880af8e0 2022-07-21T01:57:41.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.579+0000 7fb37effd700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb370053b30 0x7fb370055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.579+0000 7fb37effd700 1 -- 172.21.15.140:0/1022215509 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb38006b590 con 0x7fb3880af8e0 2022-07-21T01:57:41.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.579+0000 7fb38de7f700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb370053b30 0x7fb370055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.579+0000 7fb38de7f700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb370053b30 0x7fb370055ff0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fb37801a000 tx=0x7fb378015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.604 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.587+0000 7fb37effd700 1 -- 172.21.15.140:0/1022215509 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fb380042470 con 0x7fb3880af8e0 2022-07-21T01:57:41.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.707+0000 7fb38ee81700 1 -- 172.21.15.140:0/1022215509 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb388067ea0 con 0x7fb3880af8e0 2022-07-21T01:57:41.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.711+0000 7fb37effd700 1 -- 172.21.15.140:0/1022215509 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+16161 (secure 0 0 0) 0x7fb380024f40 con 0x7fb3880af8e0 2022-07-21T01:57:41.714 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:41.714 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":141,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:41.192239+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:57:39.555617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-07-21T01:57:40.211694+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:41.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.711+0000 7fb38ee81700 1 -- 172.21.15.140:0/1022215509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb370053b30 msgr2=0x7fb370055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.711+0000 7fb38ee81700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb370053b30 0x7fb370055ff0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fb37801a000 tx=0x7fb378015000).stop 2022-07-21T01:57:41.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.711+0000 7fb38ee81700 1 -- 172.21.15.140:0/1022215509 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880af8e0 msgr2=0x7fb388064100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.711+0000 7fb38ee81700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880af8e0 0x7fb388064100 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7fb380001450 tx=0x7fb380007e10).stop 2022-07-21T01:57:41.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.711+0000 7fb38ee81700 1 -- 172.21.15.140:0/1022215509 shutdown_connections 2022-07-21T01:57:41.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.711+0000 7fb38ee81700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb370053b30 0x7fb370055ff0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.711+0000 7fb38ee81700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3880b5650 0x7fb3880645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.711+0000 7fb38ee81700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb3880af8e0 0x7fb388064100 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.711+0000 7fb38ee81700 1 --2- 172.21.15.140:0/1022215509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3880adff0 0x7fb3881111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.711+0000 7fb38ee81700 1 -- 172.21.15.140:0/1022215509 >> 172.21.15.140:0/1022215509 conn(0x7fb3880634e0 msgr2=0x7fb3880ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:41.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.715+0000 7fb38ee81700 1 -- 172.21.15.140:0/1022215509 shutdown_connections 2022-07-21T01:57:41.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.715+0000 7fb38ee81700 1 -- 172.21.15.140:0/1022215509 wait complete. 2022-07-21T01:57:41.728 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:57:41.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.831+0000 7f332ba14700 1 Processor -- start 2022-07-21T01:57:41.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.831+0000 7f332ba14700 1 -- start start 2022-07-21T01:57:41.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.831+0000 7f332ba14700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f332409de90 0x7f33240b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.831+0000 7f332ba14700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33240607e0 0x7f33240b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.831+0000 7f332ba14700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332409d8d0 con 0x7f33240b4a20 2022-07-21T01:57:41.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.831+0000 7f332ba14700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332405ce80 con 0x7f332409de90 2022-07-21T01:57:41.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.831+0000 7f332ba14700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3324061400 con 0x7f33240607e0 2022-07-21T01:57:41.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f33297b0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f332409de90 0x7f33240b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f33297b0700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f332409de90 0x7f33240b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41088/0 (socket says 172.21.15.140:41088) 2022-07-21T01:57:41.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f33297b0700 1 -- 172.21.15.140:0/2894434775 learned_addr learned my addr 172.21.15.140:0/2894434775 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:41.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f3328faf700 1 --2- 172.21.15.140:0/2894434775 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33240607e0 0x7f33240b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f33297b0700 1 -- 172.21.15.140:0/2894434775 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33240607e0 msgr2=0x7f33240b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:41.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f33297b0700 1 --2- 172.21.15.140:0/2894434775 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33240607e0 0x7f33240b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f33297b0700 1 -- 172.21.15.140:0/2894434775 >> v1:172.21.15.152:6789/0 conn(0x7f33240b4a20 legacy=0x7f3324146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:41.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f33297b0700 1 -- 172.21.15.140:0/2894434775 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33240b4e50 con 0x7f332409de90 2022-07-21T01:57:41.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f3328faf700 1 --2- 172.21.15.140:0/2894434775 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33240607e0 0x7f33240b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:57:41.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f33297b0700 1 --2- 172.21.15.140:0/2894434775 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f332409de90 0x7f33240b4040 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f332405f350 tx=0x7f331c009290).ready entity=mon.0 client_cookie=b09ef9276775fe47 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f3323fff700 1 -- 172.21.15.140:0/2894434775 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f331c00b860 con 0x7f332409de90 2022-07-21T01:57:41.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f3323fff700 1 -- 172.21.15.140:0/2894434775 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331c00b9c0 con 0x7f332409de90 2022-07-21T01:57:41.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f3323fff700 1 -- 172.21.15.140:0/2894434775 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f331c00bb70 con 0x7f332409de90 2022-07-21T01:57:41.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 -- 172.21.15.140:0/2894434775 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f332409de90 msgr2=0x7f33240b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 --2- 172.21.15.140:0/2894434775 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f332409de90 0x7f33240b4040 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f332405f350 tx=0x7f331c009290).stop 2022-07-21T01:57:41.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 -- 172.21.15.140:0/2894434775 shutdown_connections 2022-07-21T01:57:41.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 --2- 172.21.15.140:0/2894434775 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33240607e0 0x7f33240b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 --2- 172.21.15.140:0/2894434775 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f332409de90 0x7f33240b4040 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 -- 172.21.15.140:0/2894434775 >> 172.21.15.140:0/2894434775 conn(0x7f33240b5f30 msgr2=0x7f33240b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:41.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 -- 172.21.15.140:0/2894434775 shutdown_connections 2022-07-21T01:57:41.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 -- 172.21.15.140:0/2894434775 wait complete. 2022-07-21T01:57:41.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 Processor -- start 2022-07-21T01:57:41.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 -- start start 2022-07-21T01:57:41.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33240607e0 0x7f3324148a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332409de90 0x7f3324090970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33240b4a20 0x7f332408fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33240900d0 con 0x7f33240b4a20 2022-07-21T01:57:41.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3324090360 con 0x7f332409de90 2022-07-21T01:57:41.856 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f332ba14700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3324090600 con 0x7f33240607e0 2022-07-21T01:57:41.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f3329fb1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33240b4a20 0x7f332408fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.857 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f3329fb1700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33240b4a20 0x7f332408fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41098/0 (socket says 172.21.15.140:41098) 2022-07-21T01:57:41.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f33297b0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33240607e0 0x7f3324148a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f3329fb1700 1 -- 172.21.15.140:0/1328844050 learned_addr learned my addr 172.21.15.140:0/1328844050 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:41.858 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.835+0000 7f3328faf700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332409de90 0x7f3324090970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f3329fb1700 1 -- 172.21.15.140:0/1328844050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33240607e0 msgr2=0x7f3324148a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.859 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f3329fb1700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33240607e0 0x7f3324148a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f3329fb1700 1 -- 172.21.15.140:0/1328844050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332409de90 msgr2=0x7f3324090970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:41.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f3329fb1700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332409de90 0x7f3324090970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:41.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f3329fb1700 1 -- 172.21.15.140:0/1328844050 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c009530 con 0x7f33240b4a20 2022-07-21T01:57:41.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f3329fb1700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33240b4a20 0x7f332408fbe0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f3314002f30 tx=0x7f3314006cf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f33227fc700 1 -- 172.21.15.140:0/1328844050 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3314009e60 con 0x7f33240b4a20 2022-07-21T01:57:41.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f33227fc700 1 -- 172.21.15.140:0/1328844050 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3314012410 con 0x7f33240b4a20 2022-07-21T01:57:41.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f332ba14700 1 -- 172.21.15.140:0/1328844050 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3324155290 con 0x7f33240b4a20 2022-07-21T01:57:41.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f33227fc700 1 -- 172.21.15.140:0/1328844050 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3314012770 con 0x7f33240b4a20 2022-07-21T01:57:41.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f332ba14700 1 -- 172.21.15.140:0/1328844050 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3324094360 con 0x7f33240b4a20 2022-07-21T01:57:41.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f332ba14700 1 -- 172.21.15.140:0/1328844050 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3324155420 con 0x7f33240b4a20 2022-07-21T01:57:41.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f33227fc700 1 -- 172.21.15.140:0/1328844050 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f331400f030 con 0x7f33240b4a20 2022-07-21T01:57:41.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f33227fc700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3304053aa0 0x7f3304055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:41.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.839+0000 7f33227fc700 1 -- 172.21.15.140:0/1328844050 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f331406b9c0 con 0x7f33240b4a20 2022-07-21T01:57:41.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.843+0000 7f33297b0700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3304053aa0 0x7f3304055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:41.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.843+0000 7f33297b0700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3304053aa0 0x7f3304055f60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f331c01b000 tx=0x7f331c016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:41.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:41.847+0000 7f33227fc700 1 -- 172.21.15.140:0/1328844050 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3314042880 con 0x7f33240b4a20 2022-07-21T01:57:42.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.007+0000 7f332ba14700 1 -- 172.21.15.140:0/1328844050 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f33240574e0 con 0x7f33240b4a20 2022-07-21T01:57:42.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.199+0000 7f33227fc700 1 -- 172.21.15.140:0/1328844050 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3314045ea0 con 0x7f33240b4a20 2022-07-21T01:57:42.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 -- 172.21.15.140:0/1328844050 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3304053aa0 msgr2=0x7f3304055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3304053aa0 0x7f3304055f60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f331c01b000 tx=0x7f331c016000).stop 2022-07-21T01:57:42.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 -- 172.21.15.140:0/1328844050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33240b4a20 msgr2=0x7f332408fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33240b4a20 0x7f332408fbe0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7f3314002f30 tx=0x7f3314006cf0).stop 2022-07-21T01:57:42.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 -- 172.21.15.140:0/1328844050 shutdown_connections 2022-07-21T01:57:42.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3304053aa0 0x7f3304055f60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f33240b4a20 0x7f332408fbe0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332409de90 0x7f3324090970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 --2- 172.21.15.140:0/1328844050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f33240607e0 0x7f3324148a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 -- 172.21.15.140:0/1328844050 >> 172.21.15.140:0/1328844050 conn(0x7f33240b5f30 msgr2=0x7f332405fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:42.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 -- 172.21.15.140:0/1328844050 shutdown_connections 2022-07-21T01:57:42.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.203+0000 7f332ba14700 1 -- 172.21.15.140:0/1328844050 wait complete. 2022-07-21T01:57:42.223 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:42.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58b1abc700 1 Processor -- start 2022-07-21T01:57:42.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58b1abc700 1 -- start start 2022-07-21T01:57:42.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58b1abc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f58ac0acef0 0x7f58ac14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58b1abc700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac05d480 con 0x7f58ac0b1e00 2022-07-21T01:57:42.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58b1abc700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac0b21e0 con 0x7f58ac05c3f0 2022-07-21T01:57:42.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58b1abc700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ac05cf40 con 0x7f58ac0acef0 2022-07-21T01:57:42.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58ab7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f58ac0b1e00 0x7f58ac0ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55248/0 (socket says 172.21.15.140:55248) 2022-07-21T01:57:42.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58ab7fe700 1 -- 172.21.15.140:0/3102548737 learned_addr learned my addr 172.21.15.140:0/3102548737 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:42.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58abfff700 1 --2- 172.21.15.140:0/3102548737 >> v2:172.21.15.152:3300/0 conn(0x7f58ac0acef0 0x7f58ac14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056775786 0 0) 0x7f58ac05d480 con 0x7f58ac0b1e00 2022-07-21T01:57:42.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f588c003260 con 0x7f58ac0b1e00 2022-07-21T01:57:42.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 165605983 0 0) 0x7f588c003260 con 0x7f58ac0b1e00 2022-07-21T01:57:42.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 >> v2:172.21.15.152:3300/0 conn(0x7f58ac0acef0 msgr2=0x7f58ac14a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:42.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 --2- 172.21.15.140:0/3102548737 >> v2:172.21.15.152:3300/0 conn(0x7f58ac0acef0 0x7f58ac14a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 >> v1:172.21.15.152:6789/0 conn(0x7f58ac05c3f0 legacy=0x7f58ac146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac05d1e0 con 0x7f58ac0b1e00 2022-07-21T01:57:42.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f5894001660 con 0x7f58ac0b1e00 2022-07-21T01:57:42.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 >> v1:172.21.15.140:6789/0 conn(0x7f58ac0b1e00 legacy=0x7f58ac0ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f588c003750 0x7f588c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f588c004500 0x7f588c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f588c004e70 0x7f588c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac05cf40 con 0x7f588c003750 2022-07-21T01:57:42.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c00bda0 con 0x7f588c004e70 2022-07-21T01:57:42.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58ab7fe700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f588c003750 0x7f588c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c00bf60 con 0x7f588c004500 2022-07-21T01:57:42.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aaffd700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f588c004500 0x7f588c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58abfff700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f588c004e70 0x7f588c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58ab7fe700 1 -- 172.21.15.140:0/3102548737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f588c004500 msgr2=0x7f588c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:42.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58ab7fe700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f588c004500 0x7f588c004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58ab7fe700 1 -- 172.21.15.140:0/3102548737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f588c004e70 msgr2=0x7f588c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:42.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58ab7fe700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f588c004e70 0x7f588c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58ab7fe700 1 -- 172.21.15.140:0/3102548737 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c003260 con 0x7f588c003750 2022-07-21T01:57:42.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58abfff700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f588c004e70 0x7f588c00b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:57:42.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58abfff700 1 -- 172.21.15.140:0/3102548737 reap_dead start 2022-07-21T01:57:42.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58ab7fe700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f588c003750 0x7f588c003b30 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f5894001370 tx=0x7f58ac0ac4d0).ready entity=mon.0 client_cookie=765115c8b197e8b6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:42.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58940070d0 con 0x7f588c003750 2022-07-21T01:57:42.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5894007230 con 0x7f588c003750 2022-07-21T01:57:42.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.327+0000 7f58aa7fc700 1 -- 172.21.15.140:0/3102548737 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f589400c9c0 con 0x7f588c003750 2022-07-21T01:57:42.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 -- 172.21.15.140:0/3102548737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f588c003750 msgr2=0x7f588c003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f588c003750 0x7f588c003b30 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f5894001370 tx=0x7f58ac0ac4d0).stop 2022-07-21T01:57:42.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 -- 172.21.15.140:0/3102548737 shutdown_connections 2022-07-21T01:57:42.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 --2- 172.21.15.140:0/3102548737 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f588c003750 0x7f588c003b30 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 -- 172.21.15.140:0/3102548737 >> 172.21.15.140:0/3102548737 conn(0x7f58ac0611c0 msgr2=0x7f58ac062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:42.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 -- 172.21.15.140:0/3102548737 shutdown_connections 2022-07-21T01:57:42.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 -- 172.21.15.140:0/3102548737 wait complete. 2022-07-21T01:57:42.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 Processor -- start 2022-07-21T01:57:42.355 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 -- start start 2022-07-21T01:57:42.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58ac0a12b0 0x7f58ac09ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.361 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac1428f0 0x7f58ac142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58ac143260 0x7f58ac1485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac146970 con 0x7f58ac143260 2022-07-21T01:57:42.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac146ba0 con 0x7f58ac1428f0 2022-07-21T01:57:42.363 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58b1abc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0a1690 con 0x7f58ac0a12b0 2022-07-21T01:57:42.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58abfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58ac143260 0x7f58ac1485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58ab7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58ac0a12b0 0x7f58ac09ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58aaffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac1428f0 0x7f58ac142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58abfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58ac143260 0x7f58ac1485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41118/0 (socket says 172.21.15.140:41118) 2022-07-21T01:57:42.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.331+0000 7f58abfff700 1 -- 172.21.15.140:0/3152943863 learned_addr learned my addr 172.21.15.140:0/3152943863 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:42.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58abfff700 1 -- 172.21.15.140:0/3152943863 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58ac0a12b0 msgr2=0x7f58ac09ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58abfff700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58ac0a12b0 0x7f58ac09ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58abfff700 1 -- 172.21.15.140:0/3152943863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac1428f0 msgr2=0x7f58ac142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58abfff700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac1428f0 0x7f58ac142d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58abfff700 1 -- 172.21.15.140:0/3152943863 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac0a0450 con 0x7f58ac143260 2022-07-21T01:57:42.368 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58abfff700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58ac143260 0x7f58ac1485d0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f589c00b000 tx=0x7f589c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:42.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58a8ff9700 1 -- 172.21.15.140:0/3152943863 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f589c017070 con 0x7f58ac143260 2022-07-21T01:57:42.369 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58a8ff9700 1 -- 172.21.15.140:0/3152943863 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c013400 con 0x7f58ac143260 2022-07-21T01:57:42.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58b1abc700 1 -- 172.21.15.140:0/3152943863 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac146e60 con 0x7f58ac143260 2022-07-21T01:57:42.370 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58a8ff9700 1 -- 172.21.15.140:0/3152943863 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f589c013760 con 0x7f58ac143260 2022-07-21T01:57:42.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58b1abc700 1 -- 172.21.15.140:0/3152943863 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac0a1fa0 con 0x7f58ac143260 2022-07-21T01:57:42.371 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58a8ff9700 1 -- 172.21.15.140:0/3152943863 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f589c0138c0 con 0x7f58ac143260 2022-07-21T01:57:42.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58b1abc700 1 -- 172.21.15.140:0/3152943863 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac146ff0 con 0x7f58ac143260 2022-07-21T01:57:42.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58a8ff9700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5890053aa0 0x7f5890055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.372 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58a8ff9700 1 -- 172.21.15.140:0/3152943863 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f589c06bb40 con 0x7f58ac143260 2022-07-21T01:57:42.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.335+0000 7f58ab7fe700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5890053aa0 0x7f5890055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.373 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.339+0000 7f58ab7fe700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5890053aa0 0x7f5890055f60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f5894019000 tx=0x7f5894013000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:42.374 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.343+0000 7f58a8ff9700 1 -- 172.21.15.140:0/3152943863 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f589c042a60 con 0x7f58ac143260 2022-07-21T01:57:42.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.499+0000 7f58b1abc700 1 -- 172.21.15.140:0/3152943863 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58ac0b04b0 con 0x7f58ac143260 2022-07-21T01:57:42.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.499+0000 7f58a8ff9700 1 -- 172.21.15.140:0/3152943863 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f589c046080 con 0x7f58ac143260 2022-07-21T01:57:42.505 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:42.505 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":69,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/148832193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":148832193},{"type":"v1","addr":"172.21.15.152:6833","nonce":148832193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:41.192528+0000","modified":"2022-07-21T01:57:42.195096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8308":{"gid":8308,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/961657495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":961657495},{"type":"v1","addr":"172.21.15.140:6835","nonce":961657495}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-07-21T01:57:42.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 -- 172.21.15.140:0/3152943863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5890053aa0 msgr2=0x7f5890055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5890053aa0 0x7f5890055f60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f5894019000 tx=0x7f5894013000).stop 2022-07-21T01:57:42.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 -- 172.21.15.140:0/3152943863 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58ac143260 msgr2=0x7f58ac1485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58ac143260 0x7f58ac1485d0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f589c00b000 tx=0x7f589c008040).stop 2022-07-21T01:57:42.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 -- 172.21.15.140:0/3152943863 shutdown_connections 2022-07-21T01:57:42.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5890053aa0 0x7f5890055f60 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f58ac143260 0x7f58ac1485d0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58ac1428f0 0x7f58ac142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 --2- 172.21.15.140:0/3152943863 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f58ac0a12b0 0x7f58ac09ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 -- 172.21.15.140:0/3152943863 >> 172.21.15.140:0/3152943863 conn(0x7f588c003750 msgr2=0x7f58ac0ae750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:42.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 -- 172.21.15.140:0/3152943863 shutdown_connections 2022-07-21T01:57:42.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.503+0000 7f58b1abc700 1 -- 172.21.15.140:0/3152943863 wait complete. 2022-07-21T01:57:42.511 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 69 2022-07-21T01:57:42.516 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:42.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 Processor -- start 2022-07-21T01:57:42.616 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 -- start start 2022-07-21T01:57:42.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f78240b5650 0x7f78240b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78240af8e0 0x7f782405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.619 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7824147010 con 0x7f78240adff0 2022-07-21T01:57:42.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78241472b0 con 0x7f78240b5650 2022-07-21T01:57:42.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7824146d40 con 0x7f78240af8e0 2022-07-21T01:57:42.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782259c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f78240b5650 0x7f78240b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.620 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f781bfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f78240adff0 0x7f7824147750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55272/0 (socket says 172.21.15.140:55272) 2022-07-21T01:57:42.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f781bfff700 1 -- 172.21.15.140:0/1499139738 learned_addr learned my addr 172.21.15.140:0/1499139738 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:42.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782259c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f78240b5650 0x7f78240b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41122/0 (socket says 172.21.15.140:41122) 2022-07-21T01:57:42.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f7822d9d700 1 --2- 172.21.15.140:0/1499139738 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78240af8e0 0x7f782405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.621 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782259c700 1 -- 172.21.15.140:0/1499139738 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78240af8e0 msgr2=0x7f782405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:42.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782259c700 1 --2- 172.21.15.140:0/1499139738 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78240af8e0 0x7f782405bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f7821d9b700 1 -- 172.21.15.140:0/1499139738 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946524186 0 0) 0x7f7824147010 con 0x7f78240adff0 2022-07-21T01:57:42.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782259c700 1 -- 172.21.15.140:0/1499139738 >> v1:172.21.15.140:6789/0 conn(0x7f78240adff0 legacy=0x7f7824147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782259c700 1 -- 172.21.15.140:0/1499139738 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78240acae0 con 0x7f78240b5650 2022-07-21T01:57:42.622 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f7822d9d700 1 --2- 172.21.15.140:0/1499139738 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78240af8e0 0x7f782405bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:42.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782259c700 1 --2- 172.21.15.140:0/1499139738 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78240b5650 0x7f78240b5a30 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f7814002f70 tx=0x7f7814006df0).ready entity=mon.0 client_cookie=909349b5294d0b9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:42.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f7821d9b700 1 -- 172.21.15.140:0/1499139738 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f781400d570 con 0x7f78240b5650 2022-07-21T01:57:42.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f7821d9b700 1 -- 172.21.15.140:0/1499139738 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7814000e90 con 0x7f78240b5650 2022-07-21T01:57:42.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f7821d9b700 1 -- 172.21.15.140:0/1499139738 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f781400d8c0 con 0x7f78240b5650 2022-07-21T01:57:42.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 -- 172.21.15.140:0/1499139738 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78240b5650 msgr2=0x7f78240b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 --2- 172.21.15.140:0/1499139738 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78240b5650 0x7f78240b5a30 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f7814002f70 tx=0x7f7814006df0).stop 2022-07-21T01:57:42.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 -- 172.21.15.140:0/1499139738 shutdown_connections 2022-07-21T01:57:42.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 --2- 172.21.15.140:0/1499139738 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78240af8e0 0x7f782405bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.624 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 --2- 172.21.15.140:0/1499139738 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78240b5650 0x7f78240b5a30 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 -- 172.21.15.140:0/1499139738 >> 172.21.15.140:0/1499139738 conn(0x7f78240634e0 msgr2=0x7f78240b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:42.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 -- 172.21.15.140:0/1499139738 shutdown_connections 2022-07-21T01:57:42.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.615+0000 7f782359e700 1 -- 172.21.15.140:0/1499139738 wait complete. 2022-07-21T01:57:42.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782359e700 1 Processor -- start 2022-07-21T01:57:42.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782359e700 1 -- start start 2022-07-21T01:57:42.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78240adff0 0x7f78240b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782359e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78240af8e0 0x7f7824111100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78240b5650 0x7f78240660f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782359e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7824067470 con 0x7f78240adff0 2022-07-21T01:57:42.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782359e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78240676a0 con 0x7f78240af8e0 2022-07-21T01:57:42.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7824067940 con 0x7f78240b5650 2022-07-21T01:57:42.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78240adff0 0x7f78240b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78240adff0 0x7f78240b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41128/0 (socket says 172.21.15.140:41128) 2022-07-21T01:57:42.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782259c700 1 -- 172.21.15.140:0/886455193 learned_addr learned my addr 172.21.15.140:0/886455193 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:42.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f781bfff700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78240af8e0 0x7f7824111100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f7822d9d700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78240b5650 0x7f78240660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782259c700 1 -- 172.21.15.140:0/886455193 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78240b5650 msgr2=0x7f78240660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782259c700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78240b5650 0x7f78240660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782259c700 1 -- 172.21.15.140:0/886455193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78240af8e0 msgr2=0x7f7824111100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782259c700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78240af8e0 0x7f7824111100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782259c700 1 -- 172.21.15.140:0/886455193 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c009470 con 0x7f78240adff0 2022-07-21T01:57:42.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782259c700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78240adff0 0x7f78240b3c60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f7814002f70 tx=0x7f7814014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:42.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f781b7fe700 1 -- 172.21.15.140:0/886455193 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7814016070 con 0x7f78240adff0 2022-07-21T01:57:42.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f781b7fe700 1 -- 172.21.15.140:0/886455193 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78140138d0 con 0x7f78240adff0 2022-07-21T01:57:42.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782359e700 1 -- 172.21.15.140:0/886455193 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781400b180 con 0x7f78240adff0 2022-07-21T01:57:42.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f781b7fe700 1 -- 172.21.15.140:0/886455193 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7814013c30 con 0x7f78240adff0 2022-07-21T01:57:42.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.619+0000 7f782359e700 1 -- 172.21.15.140:0/886455193 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7824064120 con 0x7f78240adff0 2022-07-21T01:57:42.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.623+0000 7f782359e700 1 -- 172.21.15.140:0/886455193 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78240574e0 con 0x7f78240adff0 2022-07-21T01:57:42.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.623+0000 7f781b7fe700 1 -- 172.21.15.140:0/886455193 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7814025030 con 0x7f78240adff0 2022-07-21T01:57:42.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.623+0000 7f781b7fe700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7808053a70 0x7f7808055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.623+0000 7f781b7fe700 1 -- 172.21.15.140:0/886455193 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f781406ce40 con 0x7f78240adff0 2022-07-21T01:57:42.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.623+0000 7f781bfff700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7808053a70 0x7f7808055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.623+0000 7f781bfff700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7808053a70 0x7f7808055f30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f780c00d000 tx=0x7f780c0017a0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:42.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.631+0000 7f781b7fe700 1 -- 172.21.15.140:0/886455193 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7814043c50 con 0x7f78240adff0 2022-07-21T01:57:42.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.755+0000 7f782359e700 1 -- 172.21.15.140:0/886455193 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7824067ea0 con 0x7f78240adff0 2022-07-21T01:57:42.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.755+0000 7f781b7fe700 1 -- 172.21.15.140:0/886455193 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f7814023070 con 0x7f78240adff0 2022-07-21T01:57:42.759 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:42.759 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":142,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:42.195796+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:57:39.555617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-07-21T01:57:40.211694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:42.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 -- 172.21.15.140:0/886455193 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7808053a70 msgr2=0x7f7808055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7808053a70 0x7f7808055f30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f780c00d000 tx=0x7f780c0017a0).stop 2022-07-21T01:57:42.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 -- 172.21.15.140:0/886455193 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78240adff0 msgr2=0x7f78240b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.762 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78240adff0 0x7f78240b3c60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7f7814002f70 tx=0x7f7814014000).stop 2022-07-21T01:57:42.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 -- 172.21.15.140:0/886455193 shutdown_connections 2022-07-21T01:57:42.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7808053a70 0x7f7808055f30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f78240b5650 0x7f78240660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.763 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78240af8e0 0x7f7824111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 --2- 172.21.15.140:0/886455193 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f78240adff0 0x7f78240b3c60 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 -- 172.21.15.140:0/886455193 >> 172.21.15.140:0/886455193 conn(0x7f78240634e0 msgr2=0x7f7824144ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:42.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 -- 172.21.15.140:0/886455193 shutdown_connections 2022-07-21T01:57:42.764 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.759+0000 7f782359e700 1 -- 172.21.15.140:0/886455193 wait complete. 2022-07-21T01:57:42.777 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:57:42.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 Processor -- start 2022-07-21T01:57:42.892 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 -- start start 2022-07-21T01:57:42.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f21f00b6800 0x7f21f00aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f21f00b3cb0 0x7f21f0145f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f21f00ad0e0 0x7f21f0146430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f0146920 con 0x7f21f00b6800 2022-07-21T01:57:42.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f00ae280 con 0x7f21f00b3cb0 2022-07-21T01:57:42.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f00b6be0 con 0x7f21f00ad0e0 2022-07-21T01:57:42.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f63f4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f21f00b6800 0x7f21f00aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f63f4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f21f00b6800 0x7f21f00aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41136/0 (socket says 172.21.15.140:41136) 2022-07-21T01:57:42.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f63f4700 1 -- 172.21.15.140:0/388377273 learned_addr learned my addr 172.21.15.140:0/388377273 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:42.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f5bf3700 1 --2- 172.21.15.140:0/388377273 >> v2:172.21.15.152:3300/0 conn(0x7f21f00b3cb0 0x7f21f0145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f6bf5700 1 --2- 172.21.15.140:0/388377273 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f21f00ad0e0 0x7f21f0146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f63f4700 1 -- 172.21.15.140:0/388377273 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f21f00ad0e0 msgr2=0x7f21f0146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:42.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f63f4700 1 --2- 172.21.15.140:0/388377273 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f21f00ad0e0 0x7f21f0146430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f63f4700 1 -- 172.21.15.140:0/388377273 >> v2:172.21.15.152:3300/0 conn(0x7f21f00b3cb0 msgr2=0x7f21f0145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:42.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f63f4700 1 --2- 172.21.15.140:0/388377273 >> v2:172.21.15.152:3300/0 conn(0x7f21f00b3cb0 0x7f21f0145f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.899 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f63f4700 1 -- 172.21.15.140:0/388377273 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f00ae4b0 con 0x7f21f00b6800 2022-07-21T01:57:42.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f63f4700 1 --2- 172.21.15.140:0/388377273 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f21f00b6800 0x7f21f00aebf0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f21f0146a60 tx=0x7f21e0006eb0).ready entity=mon.0 client_cookie=d5880b6a572e37ce server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:42.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f53f2700 1 -- 172.21.15.140:0/388377273 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21e000d570 con 0x7f21f00b6800 2022-07-21T01:57:42.900 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f53f2700 1 -- 172.21.15.140:0/388377273 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e0000e90 con 0x7f21f00b6800 2022-07-21T01:57:42.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f53f2700 1 -- 172.21.15.140:0/388377273 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21e000d900 con 0x7f21f00b6800 2022-07-21T01:57:42.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 -- 172.21.15.140:0/388377273 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f21f00b6800 msgr2=0x7f21f00aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 --2- 172.21.15.140:0/388377273 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f21f00b6800 0x7f21f00aebf0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f21f0146a60 tx=0x7f21e0006eb0).stop 2022-07-21T01:57:42.901 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 -- 172.21.15.140:0/388377273 shutdown_connections 2022-07-21T01:57:42.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 --2- 172.21.15.140:0/388377273 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f21f00ad0e0 0x7f21f0146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.891+0000 7f21f73f6700 1 --2- 172.21.15.140:0/388377273 >> v2:172.21.15.152:3300/0 conn(0x7f21f00b3cb0 0x7f21f0145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 --2- 172.21.15.140:0/388377273 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f21f00b6800 0x7f21f00aebf0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.902 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 -- 172.21.15.140:0/388377273 >> 172.21.15.140:0/388377273 conn(0x7f21f00b9dd0 msgr2=0x7f21f00b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:42.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 -- 172.21.15.140:0/388377273 shutdown_connections 2022-07-21T01:57:42.903 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 -- 172.21.15.140:0/388377273 wait complete. 2022-07-21T01:57:42.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 Processor -- start 2022-07-21T01:57:42.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 -- start start 2022-07-21T01:57:42.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f00ad0e0 0x7f21f00b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f21f00b3cb0 0x7f21f0113420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f21f00b6800 0x7f21f0151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f0113910 con 0x7f21f00b6800 2022-07-21T01:57:42.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f01454b0 con 0x7f21f00ad0e0 2022-07-21T01:57:42.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f73f6700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f0112a40 con 0x7f21f00b3cb0 2022-07-21T01:57:42.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f6bf5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f21f00b6800 0x7f21f0151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f6bf5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f21f00b6800 0x7f21f0151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41146/0 (socket says 172.21.15.140:41146) 2022-07-21T01:57:42.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f6bf5700 1 -- 172.21.15.140:0/2757943552 learned_addr learned my addr 172.21.15.140:0/2757943552 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:42.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f5bf3700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f21f00b3cb0 0x7f21f0113420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.895+0000 7f21f63f4700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f00ad0e0 0x7f21f00b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21f6bf5700 1 -- 172.21.15.140:0/2757943552 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f21f00b3cb0 msgr2=0x7f21f0113420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.911 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21f6bf5700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f21f00b3cb0 0x7f21f0113420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21f6bf5700 1 -- 172.21.15.140:0/2757943552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f00ad0e0 msgr2=0x7f21f00b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:42.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21f6bf5700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f00ad0e0 0x7f21f00b8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:42.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21f6bf5700 1 -- 172.21.15.140:0/2757943552 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e000b1e0 con 0x7f21f00b6800 2022-07-21T01:57:42.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21f63f4700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f00ad0e0 0x7f21f00b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:42.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21f6bf5700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f21f00b6800 0x7f21f0151df0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f21e800a000 tx=0x7f21e8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:42.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21df7fe700 1 -- 172.21.15.140:0/2757943552 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21e8011780 con 0x7f21f00b6800 2022-07-21T01:57:42.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21df7fe700 1 -- 172.21.15.140:0/2757943552 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e80118e0 con 0x7f21f00b6800 2022-07-21T01:57:42.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21df7fe700 1 -- 172.21.15.140:0/2757943552 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21e8011c40 con 0x7f21f00b6800 2022-07-21T01:57:42.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21f73f6700 1 -- 172.21.15.140:0/2757943552 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f0112ce0 con 0x7f21f00b6800 2022-07-21T01:57:42.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21f73f6700 1 -- 172.21.15.140:0/2757943552 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f00b24b0 con 0x7f21f00b6800 2022-07-21T01:57:42.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21f73f6700 1 -- 172.21.15.140:0/2757943552 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f00b1650 con 0x7f21f00b6800 2022-07-21T01:57:42.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.899+0000 7f21df7fe700 1 -- 172.21.15.140:0/2757943552 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f21e8012420 con 0x7f21f00b6800 2022-07-21T01:57:42.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.903+0000 7f21df7fe700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f21d0053a70 0x7f21d0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:42.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.903+0000 7f21df7fe700 1 -- 172.21.15.140:0/2757943552 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f21e806bdf0 con 0x7f21f00b6800 2022-07-21T01:57:42.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.903+0000 7f21f63f4700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f21d0053a70 0x7f21d0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:42.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.903+0000 7f21f63f4700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f21d0053a70 0x7f21d0055f30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f21e0019000 tx=0x7f21e0015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:42.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:42.907+0000 7f21df7fe700 1 -- 172.21.15.140:0/2757943552 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f21e8042c30 con 0x7f21f00b6800 2022-07-21T01:57:43.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.067+0000 7f21f73f6700 1 -- 172.21.15.140:0/2757943552 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f21f0112e70 con 0x7f21f00b6800 2022-07-21T01:57:43.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21df7fe700 1 -- 172.21.15.140:0/2757943552 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v46) v1 ==== 184+0+0 (secure 0 0 0) 0x7f21e8046250 con 0x7f21f00b6800 2022-07-21T01:57:43.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 -- 172.21.15.140:0/2757943552 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f21d0053a70 msgr2=0x7f21d0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f21d0053a70 0x7f21d0055f30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f21e0019000 tx=0x7f21e0015000).stop 2022-07-21T01:57:43.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 -- 172.21.15.140:0/2757943552 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f21f00b6800 msgr2=0x7f21f0151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f21f00b6800 0x7f21f0151df0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f21e800a000 tx=0x7f21e8008040).stop 2022-07-21T01:57:43.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 -- 172.21.15.140:0/2757943552 shutdown_connections 2022-07-21T01:57:43.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f21d0053a70 0x7f21d0055f30 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f21f00b6800 0x7f21f0151df0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f21f00b3cb0 0x7f21f0113420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 --2- 172.21.15.140:0/2757943552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21f00ad0e0 0x7f21f00b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 -- 172.21.15.140:0/2757943552 >> 172.21.15.140:0/2757943552 conn(0x7f21f00b9dd0 msgr2=0x7f21f0141310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:43.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 -- 172.21.15.140:0/2757943552 shutdown_connections 2022-07-21T01:57:43.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.079+0000 7f21f73f6700 1 -- 172.21.15.140:0/2757943552 wait complete. 2022-07-21T01:57:43.085 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:57:43.095 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:57:43.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb33fff700 1 Processor -- start 2022-07-21T01:57:43.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb33fff700 1 -- start start 2022-07-21T01:57:43.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb33fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdb340b6800 0x7fdb340aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb33fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdb340b3cb0 0x7fdb34145f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb33fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb340ae280 con 0x7fdb340ad0e0 2022-07-21T01:57:43.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb33fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb340b6be0 con 0x7fdb340b6800 2022-07-21T01:57:43.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb33fff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb340ae4b0 con 0x7fdb340b3cb0 2022-07-21T01:57:43.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb32ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdb340b6800 0x7fdb340aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb32ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fdb340b6800 0x7fdb340aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41150/0 (socket says 172.21.15.140:41150) 2022-07-21T01:57:43.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb32ffd700 1 -- 172.21.15.140:0/3475193027 learned_addr learned my addr 172.21.15.140:0/3475193027 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:43.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb327fc700 1 --2- 172.21.15.140:0/3475193027 >> v2:172.21.15.152:3300/0 conn(0x7fdb340b3cb0 0x7fdb34145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb32ffd700 1 -- 172.21.15.140:0/3475193027 >> v2:172.21.15.152:3300/0 conn(0x7fdb340b3cb0 msgr2=0x7fdb34145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:43.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb32ffd700 1 --2- 172.21.15.140:0/3475193027 >> v2:172.21.15.152:3300/0 conn(0x7fdb340b3cb0 0x7fdb34145f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb32ffd700 1 -- 172.21.15.140:0/3475193027 >> v1:172.21.15.152:6789/0 conn(0x7fdb340ad0e0 legacy=0x7fdb3405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb32ffd700 1 -- 172.21.15.140:0/3475193027 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb340ae750 con 0x7fdb340b6800 2022-07-21T01:57:43.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb32ffd700 1 --2- 172.21.15.140:0/3475193027 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb340b6800 0x7fdb340aebf0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7fdb28002f70 tx=0x7fdb28006eb0).ready entity=mon.0 client_cookie=b8e5c3070539bfdf server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:43.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb31ffb700 1 -- 172.21.15.140:0/3475193027 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb2800d5c0 con 0x7fdb340b6800 2022-07-21T01:57:43.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb31ffb700 1 -- 172.21.15.140:0/3475193027 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb28000e90 con 0x7fdb340b6800 2022-07-21T01:57:43.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.203+0000 7fdb31ffb700 1 -- 172.21.15.140:0/3475193027 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb2800d950 con 0x7fdb340b6800 2022-07-21T01:57:43.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 -- 172.21.15.140:0/3475193027 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb340b6800 msgr2=0x7fdb340aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 --2- 172.21.15.140:0/3475193027 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb340b6800 0x7fdb340aebf0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7fdb28002f70 tx=0x7fdb28006eb0).stop 2022-07-21T01:57:43.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 -- 172.21.15.140:0/3475193027 shutdown_connections 2022-07-21T01:57:43.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 --2- 172.21.15.140:0/3475193027 >> v2:172.21.15.152:3300/0 conn(0x7fdb340b3cb0 0x7fdb34145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 --2- 172.21.15.140:0/3475193027 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb340b6800 0x7fdb340aebf0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 -- 172.21.15.140:0/3475193027 >> 172.21.15.140:0/3475193027 conn(0x7fdb340b9dd0 msgr2=0x7fdb340b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:43.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 -- 172.21.15.140:0/3475193027 shutdown_connections 2022-07-21T01:57:43.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 -- 172.21.15.140:0/3475193027 wait complete. 2022-07-21T01:57:43.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 Processor -- start 2022-07-21T01:57:43.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 -- start start 2022-07-21T01:57:43.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb340ad0e0 0x7fdb34112b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb340b3cb0 0x7fdb34115ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb340b6800 0x7fdb340b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb34115390 con 0x7fdb340b3cb0 2022-07-21T01:57:43.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb34115630 con 0x7fdb340ad0e0 2022-07-21T01:57:43.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb34115fc0 con 0x7fdb340b6800 2022-07-21T01:57:43.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb327fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb340b3cb0 0x7fdb34115ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb327fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb340b3cb0 0x7fdb34115ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41158/0 (socket says 172.21.15.140:41158) 2022-07-21T01:57:43.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb327fc700 1 -- 172.21.15.140:0/1869432392 learned_addr learned my addr 172.21.15.140:0/1869432392 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:43.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb32ffd700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb340ad0e0 0x7fdb34112b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb337fe700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb340b6800 0x7fdb340b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb327fc700 1 -- 172.21.15.140:0/1869432392 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb340b6800 msgr2=0x7fdb340b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb327fc700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb340b6800 0x7fdb340b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb327fc700 1 -- 172.21.15.140:0/1869432392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb340ad0e0 msgr2=0x7fdb34112b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb327fc700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb340ad0e0 0x7fdb34112b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb327fc700 1 -- 172.21.15.140:0/1869432392 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2800b1e0 con 0x7fdb340b3cb0 2022-07-21T01:57:43.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb327fc700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb340b3cb0 0x7fdb34115ad0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7fdb2400c000 tx=0x7fdb240091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:43.224 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb23fff700 1 -- 172.21.15.140:0/1869432392 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb2400c670 con 0x7fdb340b3cb0 2022-07-21T01:57:43.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb23fff700 1 -- 172.21.15.140:0/1869432392 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2400c7d0 con 0x7fdb340b3cb0 2022-07-21T01:57:43.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb23fff700 1 -- 172.21.15.140:0/1869432392 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdb2400cb30 con 0x7fdb340b3cb0 2022-07-21T01:57:43.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 -- 172.21.15.140:0/1869432392 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb340b2e60 con 0x7fdb340b3cb0 2022-07-21T01:57:43.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.207+0000 7fdb33fff700 1 -- 172.21.15.140:0/1869432392 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb34113a60 con 0x7fdb340b3cb0 2022-07-21T01:57:43.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.211+0000 7fdb33fff700 1 -- 172.21.15.140:0/1869432392 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb340b5530 con 0x7fdb340b3cb0 2022-07-21T01:57:43.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.211+0000 7fdb23fff700 1 -- 172.21.15.140:0/1869432392 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdb24022070 con 0x7fdb340b3cb0 2022-07-21T01:57:43.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.211+0000 7fdb23fff700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdb10053b30 0x7fdb10055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.211+0000 7fdb23fff700 1 -- 172.21.15.140:0/1869432392 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdb2406c480 con 0x7fdb340b3cb0 2022-07-21T01:57:43.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.211+0000 7fdb32ffd700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdb10053b30 0x7fdb10055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.215+0000 7fdb32ffd700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdb10053b30 0x7fdb10055ff0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fdb2801c000 tx=0x7fdb28015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:43.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.219+0000 7fdb23fff700 1 -- 172.21.15.140:0/1869432392 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdb240433a0 con 0x7fdb340b3cb0 2022-07-21T01:57:43.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.387+0000 7fdb33fff700 1 -- 172.21.15.140:0/1869432392 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fdb340574e0 con 0x7fdb340b3cb0 2022-07-21T01:57:43.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.395+0000 7fdb23fff700 1 -- 172.21.15.140:0/1869432392 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v47) v1 ==== 184+0+0 (secure 0 0 0) 0x7fdb240469c0 con 0x7fdb340b3cb0 2022-07-21T01:57:43.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.395+0000 7fdb33fff700 1 -- 172.21.15.140:0/1869432392 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdb10053b30 msgr2=0x7fdb10055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.395+0000 7fdb33fff700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdb10053b30 0x7fdb10055ff0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fdb2801c000 tx=0x7fdb28015000).stop 2022-07-21T01:57:43.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.395+0000 7fdb33fff700 1 -- 172.21.15.140:0/1869432392 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb340b3cb0 msgr2=0x7fdb34115ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.395+0000 7fdb33fff700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb340b3cb0 0x7fdb34115ad0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7fdb2400c000 tx=0x7fdb240091d0).stop 2022-07-21T01:57:43.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.395+0000 7fdb33fff700 1 -- 172.21.15.140:0/1869432392 shutdown_connections 2022-07-21T01:57:43.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.395+0000 7fdb33fff700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdb10053b30 0x7fdb10055ff0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.395+0000 7fdb33fff700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb340b6800 0x7fdb340b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.395+0000 7fdb33fff700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb340b3cb0 0x7fdb34115ad0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.395+0000 7fdb33fff700 1 --2- 172.21.15.140:0/1869432392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb340ad0e0 0x7fdb34112b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.395+0000 7fdb33fff700 1 -- 172.21.15.140:0/1869432392 >> 172.21.15.140:0/1869432392 conn(0x7fdb340b9dd0 msgr2=0x7fdb340b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:43.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.399+0000 7fdb33fff700 1 -- 172.21.15.140:0/1869432392 shutdown_connections 2022-07-21T01:57:43.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.399+0000 7fdb33fff700 1 -- 172.21.15.140:0/1869432392 wait complete. 2022-07-21T01:57:43.404 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:57:43.413 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:57:43.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 Processor -- start 2022-07-21T01:57:43.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 -- start start 2022-07-21T01:57:43.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f840c0b3810 0x7f840c140c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f840c141130 0x7f840c0ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c05d4c0 con 0x7f840c05d0e0 2022-07-21T01:57:43.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c1443d0 con 0x7f840c141130 2022-07-21T01:57:43.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c144670 con 0x7f840c0b3810 2022-07-21T01:57:43.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f840ad9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f840c141130 0x7f840c0ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f840ad9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f840c141130 0x7f840c0ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41168/0 (socket says 172.21.15.140:41168) 2022-07-21T01:57:43.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f840ad9d700 1 -- 172.21.15.140:0/3348183604 learned_addr learned my addr 172.21.15.140:0/3348183604 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:43.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8409d9b700 1 --2- 172.21.15.140:0/3348183604 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f840c0b3810 0x7f840c140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f840ad9d700 1 -- 172.21.15.140:0/3348183604 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f840c0b3810 msgr2=0x7f840c140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:43.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f840ad9d700 1 --2- 172.21.15.140:0/3348183604 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f840c0b3810 0x7f840c140c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f840ad9d700 1 -- 172.21.15.140:0/3348183604 >> v1:172.21.15.152:6789/0 conn(0x7f840c05d0e0 legacy=0x7f840c144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f840ad9d700 1 -- 172.21.15.140:0/3348183604 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c063490 con 0x7f840c141130 2022-07-21T01:57:43.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f840ad9d700 1 --2- 172.21.15.140:0/3348183604 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f840c141130 0x7f840c0ad1e0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f84040032f0 tx=0x7f8404009290).ready entity=mon.0 client_cookie=9e9a5550d2478aa5 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:43.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f840959a700 1 -- 172.21.15.140:0/3348183604 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840400b8a0 con 0x7f840c141130 2022-07-21T01:57:43.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f840959a700 1 -- 172.21.15.140:0/3348183604 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840400ba00 con 0x7f840c141130 2022-07-21T01:57:43.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f840959a700 1 -- 172.21.15.140:0/3348183604 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f840400bb70 con 0x7f840c141130 2022-07-21T01:57:43.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 -- 172.21.15.140:0/3348183604 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f840c141130 msgr2=0x7f840c0ad1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 --2- 172.21.15.140:0/3348183604 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f840c141130 0x7f840c0ad1e0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f84040032f0 tx=0x7f8404009290).stop 2022-07-21T01:57:43.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 -- 172.21.15.140:0/3348183604 shutdown_connections 2022-07-21T01:57:43.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 --2- 172.21.15.140:0/3348183604 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f840c141130 0x7f840c0ad1e0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 --2- 172.21.15.140:0/3348183604 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f840c0b3810 0x7f840c140c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.519+0000 7f8410bda700 1 -- 172.21.15.140:0/3348183604 >> 172.21.15.140:0/3348183604 conn(0x7f840c0b7d60 msgr2=0x7f840c0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:43.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f8410bda700 1 -- 172.21.15.140:0/3348183604 shutdown_connections 2022-07-21T01:57:43.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f8410bda700 1 -- 172.21.15.140:0/3348183604 wait complete. 2022-07-21T01:57:43.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f8410bda700 1 Processor -- start 2022-07-21T01:57:43.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f8410bda700 1 -- start start 2022-07-21T01:57:43.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f8410bda700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f840c05d0e0 0x7f840c143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f8410bda700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f840c0b3810 0x7f840c153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f8410bda700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f840c141130 0x7f840c14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f8410bda700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c154350 con 0x7f840c141130 2022-07-21T01:57:43.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f8410bda700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c153480 con 0x7f840c0b3810 2022-07-21T01:57:43.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f8410bda700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c153720 con 0x7f840c05d0e0 2022-07-21T01:57:43.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f840ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f840c141130 0x7f840c14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f840ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f840c141130 0x7f840c14feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41174/0 (socket says 172.21.15.140:41174) 2022-07-21T01:57:43.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f840a59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f840c05d0e0 0x7f840c143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f840ad9d700 1 -- 172.21.15.140:0/1292208726 learned_addr learned my addr 172.21.15.140:0/1292208726 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:43.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f8409d9b700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f840c0b3810 0x7f840c153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f840ad9d700 1 -- 172.21.15.140:0/1292208726 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f840c05d0e0 msgr2=0x7f840c143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f840ad9d700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f840c05d0e0 0x7f840c143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f840ad9d700 1 -- 172.21.15.140:0/1292208726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f840c0b3810 msgr2=0x7f840c153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f840ad9d700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f840c0b3810 0x7f840c153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.523+0000 7f840ad9d700 1 -- 172.21.15.140:0/1292208726 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8404009530 con 0x7f840c141130 2022-07-21T01:57:43.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f840ad9d700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f840c141130 0x7f840c14feb0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f84040032f0 tx=0x7f8404016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:43.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f84037fe700 1 -- 172.21.15.140:0/1292208726 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8404021670 con 0x7f840c141130 2022-07-21T01:57:43.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f84037fe700 1 -- 172.21.15.140:0/1292208726 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84040217d0 con 0x7f840c141130 2022-07-21T01:57:43.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f8410bda700 1 -- 172.21.15.140:0/1292208726 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c1539c0 con 0x7f840c141130 2022-07-21T01:57:43.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f84037fe700 1 -- 172.21.15.140:0/1292208726 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8404021b30 con 0x7f840c141130 2022-07-21T01:57:43.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f8410bda700 1 -- 172.21.15.140:0/1292208726 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c0ae6d0 con 0x7f840c141130 2022-07-21T01:57:43.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f84037fe700 1 -- 172.21.15.140:0/1292208726 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f840400f070 con 0x7f840c141130 2022-07-21T01:57:43.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f84037fe700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f0053b00 0x7f83f0055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f84037fe700 1 -- 172.21.15.140:0/1292208726 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f840406d190 con 0x7f840c141130 2022-07-21T01:57:43.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f8410bda700 1 -- 172.21.15.140:0/1292208726 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c153b50 con 0x7f840c141130 2022-07-21T01:57:43.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f840a59c700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f0053b00 0x7f83f0055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.527+0000 7f840a59c700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f0053b00 0x7f83f0055fc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f83fc00d000 tx=0x7f83fc007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:43.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.535+0000 7f84037fe700 1 -- 172.21.15.140:0/1292208726 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8404015070 con 0x7f840c141130 2022-07-21T01:57:43.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.695+0000 7f8410bda700 1 -- 172.21.15.140:0/1292208726 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f840c097270 con 0x7f840c141130 2022-07-21T01:57:43.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.703+0000 7f84037fe700 1 -- 172.21.15.140:0/1292208726 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v48) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8404026030 con 0x7f840c141130 2022-07-21T01:57:43.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 -- 172.21.15.140:0/1292208726 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f0053b00 msgr2=0x7f83f0055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f0053b00 0x7f83f0055fc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f83fc00d000 tx=0x7f83fc007b90).stop 2022-07-21T01:57:43.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 -- 172.21.15.140:0/1292208726 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f840c141130 msgr2=0x7f840c14feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f840c141130 0x7f840c14feb0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7f84040032f0 tx=0x7f8404016000).stop 2022-07-21T01:57:43.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 -- 172.21.15.140:0/1292208726 shutdown_connections 2022-07-21T01:57:43.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83f0053b00 0x7f83f0055fc0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f840c141130 0x7f840c14feb0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f840c0b3810 0x7f840c153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 --2- 172.21.15.140:0/1292208726 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f840c05d0e0 0x7f840c143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 -- 172.21.15.140:0/1292208726 >> 172.21.15.140:0/1292208726 conn(0x7f840c0b7d60 msgr2=0x7f840c062130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:43.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 -- 172.21.15.140:0/1292208726 shutdown_connections 2022-07-21T01:57:43.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.707+0000 7f8410bda700 1 -- 172.21.15.140:0/1292208726 wait complete. 2022-07-21T01:57:43.714 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:57:43.726 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:43.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.823+0000 7f5372a03700 1 Processor -- start 2022-07-21T01:57:43.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.823+0000 7f5372a03700 1 -- start start 2022-07-21T01:57:43.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f536c0b1e00 0x7f536c14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f536c05c3f0 0x7f536c0ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536c05d480 con 0x7f536c05bd90 2022-07-21T01:57:43.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c0b21e0 con 0x7f536c05c3f0 2022-07-21T01:57:43.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c05cf40 con 0x7f536c0b1e00 2022-07-21T01:57:43.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5370fa0700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f536c05bd90 0x7f536c146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55328/0 (socket says 172.21.15.140:55328) 2022-07-21T01:57:43.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5370fa0700 1 -- 172.21.15.140:0/4184482039 learned_addr learned my addr 172.21.15.140:0/4184482039 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:43.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536b7fe700 1 --2- 172.21.15.140:0/4184482039 >> v2:172.21.15.152:3300/0 conn(0x7f536c05c3f0 0x7f536c0ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536bfff700 1 --2- 172.21.15.140:0/4184482039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f536c0b1e00 0x7f536c14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536affd700 1 -- 172.21.15.140:0/4184482039 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411468779 0 0) 0x7f536c05d480 con 0x7f536c05bd90 2022-07-21T01:57:43.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536affd700 1 -- 172.21.15.140:0/4184482039 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5358003260 con 0x7f536c05bd90 2022-07-21T01:57:43.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536b7fe700 1 -- 172.21.15.140:0/4184482039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f536c0b1e00 msgr2=0x7f536c14a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:43.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536b7fe700 1 --2- 172.21.15.140:0/4184482039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f536c0b1e00 0x7f536c14a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536affd700 1 -- 172.21.15.140:0/4184482039 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 306391162 0 0) 0x7f5358003260 con 0x7f536c05bd90 2022-07-21T01:57:43.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536b7fe700 1 -- 172.21.15.140:0/4184482039 >> v1:172.21.15.140:6789/0 conn(0x7f536c05bd90 legacy=0x7f536c146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536b7fe700 1 -- 172.21.15.140:0/4184482039 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c05d1e0 con 0x7f536c05c3f0 2022-07-21T01:57:43.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536b7fe700 1 --2- 172.21.15.140:0/4184482039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f536c05c3f0 0x7f536c0ac7e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f535c001800 tx=0x7f53640091d0).ready entity=mon.1 client_cookie=f277b2ba5712e442 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:43.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536affd700 1 -- 172.21.15.140:0/4184482039 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f535c001660 con 0x7f536c05c3f0 2022-07-21T01:57:43.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536affd700 1 -- 172.21.15.140:0/4184482039 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536400f510 con 0x7f536c05c3f0 2022-07-21T01:57:43.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f536affd700 1 -- 172.21.15.140:0/4184482039 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f536400f680 con 0x7f536c05c3f0 2022-07-21T01:57:43.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 -- 172.21.15.140:0/4184482039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f536c05c3f0 msgr2=0x7f536c0ac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 --2- 172.21.15.140:0/4184482039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f536c05c3f0 0x7f536c0ac7e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f535c001800 tx=0x7f53640091d0).stop 2022-07-21T01:57:43.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 -- 172.21.15.140:0/4184482039 shutdown_connections 2022-07-21T01:57:43.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 --2- 172.21.15.140:0/4184482039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f536c05c3f0 0x7f536c0ac7e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 --2- 172.21.15.140:0/4184482039 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f536c0b1e00 0x7f536c14a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 -- 172.21.15.140:0/4184482039 >> 172.21.15.140:0/4184482039 conn(0x7f536c0611c0 msgr2=0x7f536c062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:43.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 -- 172.21.15.140:0/4184482039 shutdown_connections 2022-07-21T01:57:43.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.827+0000 7f5372a03700 1 -- 172.21.15.140:0/4184482039 wait complete. 2022-07-21T01:57:43.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f5372a03700 1 Processor -- start 2022-07-21T01:57:43.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f5372a03700 1 -- start start 2022-07-21T01:57:43.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f5372a03700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f536c05c3f0 0x7f536c063ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f5372a03700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f536c0b1e00 0x7f536c0656b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f5372a03700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f536c0b4400 0x7f536c156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f5372a03700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c065ba0 con 0x7f536c05c3f0 2022-07-21T01:57:43.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f5372a03700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c065dc0 con 0x7f536c0b1e00 2022-07-21T01:57:43.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f5372a03700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c066060 con 0x7f536c0b4400 2022-07-21T01:57:43.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f536bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f536c05c3f0 0x7f536c063ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f536bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f536c05c3f0 0x7f536c063ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41186/0 (socket says 172.21.15.140:41186) 2022-07-21T01:57:43.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f536bfff700 1 -- 172.21.15.140:0/2117990229 learned_addr learned my addr 172.21.15.140:0/2117990229 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:43.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f536b7fe700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f536c0b1e00 0x7f536c0656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f5370fa0700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f536c0b4400 0x7f536c156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f536bfff700 1 -- 172.21.15.140:0/2117990229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f536c0b4400 msgr2=0x7f536c156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f536bfff700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f536c0b4400 0x7f536c156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f536bfff700 1 -- 172.21.15.140:0/2117990229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f536c0b1e00 msgr2=0x7f536c0656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:43.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f536bfff700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f536c0b1e00 0x7f536c0656b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:43.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f536bfff700 1 -- 172.21.15.140:0/2117990229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5364009470 con 0x7f536c05c3f0 2022-07-21T01:57:43.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f536bfff700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f536c05c3f0 0x7f536c063ca0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f536000c000 tx=0x7f5360006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:43.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f53697fa700 1 -- 172.21.15.140:0/2117990229 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5360011750 con 0x7f536c05c3f0 2022-07-21T01:57:43.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f53697fa700 1 -- 172.21.15.140:0/2117990229 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53600118b0 con 0x7f536c05c3f0 2022-07-21T01:57:43.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f5372a03700 1 -- 172.21.15.140:0/2117990229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c066300 con 0x7f536c05c3f0 2022-07-21T01:57:43.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f53697fa700 1 -- 172.21.15.140:0/2117990229 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5360011c10 con 0x7f536c05c3f0 2022-07-21T01:57:43.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.831+0000 7f5372a03700 1 -- 172.21.15.140:0/2117990229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c157230 con 0x7f536c05c3f0 2022-07-21T01:57:43.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.835+0000 7f53697fa700 1 -- 172.21.15.140:0/2117990229 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5360011d70 con 0x7f536c05c3f0 2022-07-21T01:57:43.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.835+0000 7f5372a03700 1 -- 172.21.15.140:0/2117990229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c0b04b0 con 0x7f536c05c3f0 2022-07-21T01:57:43.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.835+0000 7f53697fa700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f534c053b00 0x7f534c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:43.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.835+0000 7f536b7fe700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f534c053b00 0x7f534c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:43.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.835+0000 7f53697fa700 1 -- 172.21.15.140:0/2117990229 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f536006c490 con 0x7f536c05c3f0 2022-07-21T01:57:43.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.835+0000 7f536b7fe700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f534c053b00 0x7f534c055fc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f536401b000 tx=0x7f5364016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:43.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.839+0000 7f53697fa700 1 -- 172.21.15.140:0/2117990229 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f53600432a0 con 0x7f536c05c3f0 2022-07-21T01:57:44.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.999+0000 7f5372a03700 1 -- 172.21.15.140:0/2117990229 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f536c0574e0 con 0x7f536c05c3f0 2022-07-21T01:57:44.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.999+0000 7f53697fa700 1 -- 172.21.15.140:0/2117990229 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f53600468c0 con 0x7f536c05c3f0 2022-07-21T01:57:44.002 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:44.002 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":69,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/148832193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":148832193},{"type":"v1","addr":"172.21.15.152:6833","nonce":148832193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:41.192528+0000","modified":"2022-07-21T01:57:42.195096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8308":{"gid":8308,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/961657495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":961657495},{"type":"v1","addr":"172.21.15.140:6835","nonce":961657495}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-07-21T01:57:44.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.999+0000 7f5372a03700 1 -- 172.21.15.140:0/2117990229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f534c053b00 msgr2=0x7f534c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.999+0000 7f5372a03700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f534c053b00 0x7f534c055fc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f536401b000 tx=0x7f5364016000).stop 2022-07-21T01:57:44.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.999+0000 7f5372a03700 1 -- 172.21.15.140:0/2117990229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f536c05c3f0 msgr2=0x7f536c063ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.999+0000 7f5372a03700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f536c05c3f0 0x7f536c063ca0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f536000c000 tx=0x7f5360006df0).stop 2022-07-21T01:57:44.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.999+0000 7f5372a03700 1 -- 172.21.15.140:0/2117990229 shutdown_connections 2022-07-21T01:57:44.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.999+0000 7f5372a03700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f534c053b00 0x7f534c055fc0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.999+0000 7f5372a03700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f536c0b4400 0x7f536c156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.999+0000 7f5372a03700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f536c0b1e00 0x7f536c0656b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:43.999+0000 7f5372a03700 1 --2- 172.21.15.140:0/2117990229 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f536c05c3f0 0x7f536c063ca0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.003+0000 7f5372a03700 1 -- 172.21.15.140:0/2117990229 >> 172.21.15.140:0/2117990229 conn(0x7f536c05bd90 msgr2=0x7f536c14a280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:44.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.003+0000 7f5372a03700 1 -- 172.21.15.140:0/2117990229 shutdown_connections 2022-07-21T01:57:44.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.003+0000 7f5372a03700 1 -- 172.21.15.140:0/2117990229 wait complete. 2022-07-21T01:57:44.007 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 69 2022-07-21T01:57:44.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:57:41.192528+0000', 'modified': '2022-07-21T01:57:42.195096+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8308}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8308': {'gid': 8308, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/961657495', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 961657495}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 961657495}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:57:44.217 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:57:44.217 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:57:44.332 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":8308,"want_state":"up:active","state":"up:active","rank_uptime":3.125948722,"mdsmap_epoch":69,"osdmap_epoch":142,"osdmap_epoch_barrier":141,"uptime":7.4901614250000002} 2022-07-21T01:57:44.332 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 8308, "mdsmap_epoch": 69, "osdmap_epoch": 142, "osdmap_epoch_barrier": 141, "rank_uptime": 3.125948722, "state": "up:active", "uptime": 7.490161425, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:57:44.333 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:44.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd19956e700 1 Processor -- start 2022-07-21T01:57:44.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd19956e700 1 -- start start 2022-07-21T01:57:44.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd19956e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1940acef0 0x7fd19414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd19956e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19405d480 con 0x7fd19405c3f0 2022-07-21T01:57:44.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd19956e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1940b21e0 con 0x7fd1940b1e00 2022-07-21T01:57:44.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd19956e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19405cf40 con 0x7fd1940acef0 2022-07-21T01:57:44.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd1927fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd19405c3f0 0x7fd194146410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55340/0 (socket says 172.21.15.140:55340) 2022-07-21T01:57:44.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd1927fc700 1 -- 172.21.15.140:0/1193072716 learned_addr learned my addr 172.21.15.140:0/1193072716 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:44.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd1937fe700 1 --2- 172.21.15.140:0/1193072716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1940acef0 0x7fd19414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd191ffb700 1 -- 172.21.15.140:0/1193072716 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400415543 0 0) 0x7fd19405d480 con 0x7fd19405c3f0 2022-07-21T01:57:44.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd191ffb700 1 -- 172.21.15.140:0/1193072716 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd174003260 con 0x7fd19405c3f0 2022-07-21T01:57:44.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd191ffb700 1 -- 172.21.15.140:0/1193072716 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900887744 0 0) 0x7fd1940b21e0 con 0x7fd1940b1e00 2022-07-21T01:57:44.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd191ffb700 1 -- 172.21.15.140:0/1193072716 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd19405d480 con 0x7fd1940b1e00 2022-07-21T01:57:44.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd1937fe700 1 -- 172.21.15.140:0/1193072716 >> v1:172.21.15.152:6789/0 conn(0x7fd1940b1e00 legacy=0x7fd1940ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd1937fe700 1 -- 172.21.15.140:0/1193072716 >> v1:172.21.15.140:6789/0 conn(0x7fd19405c3f0 legacy=0x7fd194146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd1937fe700 1 -- 172.21.15.140:0/1193072716 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19405d1e0 con 0x7fd1940acef0 2022-07-21T01:57:44.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd1937fe700 1 --2- 172.21.15.140:0/1193072716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1940acef0 0x7fd19414a6d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fd18c002f70 tx=0x7fd18c006df0).ready entity=mon.2 client_cookie=2ca5d615c47e58d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:44.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd191ffb700 1 -- 172.21.15.140:0/1193072716 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd18c00d5c0 con 0x7fd1940acef0 2022-07-21T01:57:44.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd191ffb700 1 -- 172.21.15.140:0/1193072716 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c000e90 con 0x7fd1940acef0 2022-07-21T01:57:44.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.435+0000 7fd191ffb700 1 -- 172.21.15.140:0/1193072716 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd18c00d910 con 0x7fd1940acef0 2022-07-21T01:57:44.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 -- 172.21.15.140:0/1193072716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1940acef0 msgr2=0x7fd19414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 --2- 172.21.15.140:0/1193072716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1940acef0 0x7fd19414a6d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fd18c002f70 tx=0x7fd18c006df0).stop 2022-07-21T01:57:44.453 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 -- 172.21.15.140:0/1193072716 shutdown_connections 2022-07-21T01:57:44.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 --2- 172.21.15.140:0/1193072716 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1940acef0 0x7fd19414a6d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 -- 172.21.15.140:0/1193072716 >> 172.21.15.140:0/1193072716 conn(0x7fd1940611c0 msgr2=0x7fd194062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:44.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 -- 172.21.15.140:0/1193072716 shutdown_connections 2022-07-21T01:57:44.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 -- 172.21.15.140:0/1193072716 wait complete. 2022-07-21T01:57:44.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 Processor -- start 2022-07-21T01:57:44.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 -- start start 2022-07-21T01:57:44.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1940acef0 0x7fd194143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1940b1e00 0x7fd1940ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1940ae830 0x7fd194156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd194156570 con 0x7fd1940b1e00 2022-07-21T01:57:44.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19409f820 con 0x7fd1940acef0 2022-07-21T01:57:44.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd19956e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19409fa50 con 0x7fd1940ae830 2022-07-21T01:57:44.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd1927fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1940b1e00 0x7fd1940ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd1927fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1940b1e00 0x7fd1940ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41200/0 (socket says 172.21.15.140:41200) 2022-07-21T01:57:44.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd1927fc700 1 -- 172.21.15.140:0/3668905826 learned_addr learned my addr 172.21.15.140:0/3668905826 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:44.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd192ffd700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1940acef0 0x7fd194143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd1937fe700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1940ae830 0x7fd194156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd1927fc700 1 -- 172.21.15.140:0/3668905826 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1940ae830 msgr2=0x7fd194156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd1927fc700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1940ae830 0x7fd194156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd1927fc700 1 -- 172.21.15.140:0/3668905826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1940acef0 msgr2=0x7fd194143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd1927fc700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1940acef0 0x7fd194143380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.439+0000 7fd1927fc700 1 -- 172.21.15.140:0/3668905826 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c00b180 con 0x7fd1940b1e00 2022-07-21T01:57:44.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.443+0000 7fd1927fc700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1940b1e00 0x7fd1940ae340 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7fd184001490 tx=0x7fd184007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:44.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.443+0000 7fd18bfff700 1 -- 172.21.15.140:0/3668905826 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd184005ba0 con 0x7fd1940b1e00 2022-07-21T01:57:44.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.443+0000 7fd18bfff700 1 -- 172.21.15.140:0/3668905826 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184005d00 con 0x7fd1940b1e00 2022-07-21T01:57:44.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.443+0000 7fd19956e700 1 -- 172.21.15.140:0/3668905826 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1941567c0 con 0x7fd1940b1e00 2022-07-21T01:57:44.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.443+0000 7fd18bfff700 1 -- 172.21.15.140:0/3668905826 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1840072c0 con 0x7fd1940b1e00 2022-07-21T01:57:44.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.443+0000 7fd19956e700 1 -- 172.21.15.140:0/3668905826 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1941570f0 con 0x7fd1940b1e00 2022-07-21T01:57:44.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.443+0000 7fd19956e700 1 -- 172.21.15.140:0/3668905826 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1940b04b0 con 0x7fd1940b1e00 2022-07-21T01:57:44.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.443+0000 7fd18bfff700 1 -- 172.21.15.140:0/3668905826 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd184010070 con 0x7fd1940b1e00 2022-07-21T01:57:44.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.443+0000 7fd18bfff700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd178053aa0 0x7fd178055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.443+0000 7fd18bfff700 1 -- 172.21.15.140:0/3668905826 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd18406b700 con 0x7fd1940b1e00 2022-07-21T01:57:44.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.443+0000 7fd192ffd700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd178053aa0 0x7fd178055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.447+0000 7fd192ffd700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd178053aa0 0x7fd178055f60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fd17c00d000 tx=0x7fd17c007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:44.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.451+0000 7fd18bfff700 1 -- 172.21.15.140:0/3668905826 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd184042580 con 0x7fd1940b1e00 2022-07-21T01:57:44.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd19956e700 1 -- 172.21.15.140:0/3668905826 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd194069a60 con 0x7fd1940b1e00 2022-07-21T01:57:44.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd18bfff700 1 -- 172.21.15.140:0/3668905826 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+16148 (secure 0 0 0) 0x7fd184045ba0 con 0x7fd1940b1e00 2022-07-21T01:57:44.578 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:44.578 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":142,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:42.195796+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:57:39.555617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-07-21T01:57:40.211694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:44.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd19956e700 1 -- 172.21.15.140:0/3668905826 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd178053aa0 msgr2=0x7fd178055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd19956e700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd178053aa0 0x7fd178055f60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fd17c00d000 tx=0x7fd17c007a20).stop 2022-07-21T01:57:44.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd19956e700 1 -- 172.21.15.140:0/3668905826 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1940b1e00 msgr2=0x7fd1940ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd19956e700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1940b1e00 0x7fd1940ae340 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7fd184001490 tx=0x7fd184007e20).stop 2022-07-21T01:57:44.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd19956e700 1 -- 172.21.15.140:0/3668905826 shutdown_connections 2022-07-21T01:57:44.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd19956e700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd178053aa0 0x7fd178055f60 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd19956e700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1940ae830 0x7fd194156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd19956e700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd1940b1e00 0x7fd1940ae340 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd19956e700 1 --2- 172.21.15.140:0/3668905826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1940acef0 0x7fd194143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.575+0000 7fd19956e700 1 -- 172.21.15.140:0/3668905826 >> 172.21.15.140:0/3668905826 conn(0x7fd19405c3f0 msgr2=0x7fd1940afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:44.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.579+0000 7fd19956e700 1 -- 172.21.15.140:0/3668905826 shutdown_connections 2022-07-21T01:57:44.583 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.579+0000 7fd19956e700 1 -- 172.21.15.140:0/3668905826 wait complete. 2022-07-21T01:57:44.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:57:39.555617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:57:40.211694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:57:44.592 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:57:44.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f900a78c700 1 Processor -- start 2022-07-21T01:57:44.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f900a78c700 1 -- start start 2022-07-21T01:57:44.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f900a78c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f90040b23b0 0x7f900405c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f900a78c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90041468a0 0x7f90040b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f900a78c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90040b16a0 con 0x7f90040b0580 2022-07-21T01:57:44.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f900a78c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9004067f70 con 0x7f90040b23b0 2022-07-21T01:57:44.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f900a78c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90040ad0e0 con 0x7f90041468a0 2022-07-21T01:57:44.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f9008f89700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f90040b0580 0x7f9004146190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55354/0 (socket says 172.21.15.140:55354) 2022-07-21T01:57:44.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f9008f89700 1 -- 172.21.15.140:0/2988894128 learned_addr learned my addr 172.21.15.140:0/2988894128 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:44.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f900978a700 1 --2- 172.21.15.140:0/2988894128 >> v2:172.21.15.152:3300/0 conn(0x7f90040b23b0 0x7f900405c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f9009f8b700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90041468a0 0x7f90040b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233127258 0 0) 0x7f90040b16a0 con 0x7f90040b0580 2022-07-21T01:57:44.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fec003260 con 0x7f90040b0580 2022-07-21T01:57:44.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1750323870 0 0) 0x7f8fec003260 con 0x7f90040b0580 2022-07-21T01:57:44.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90041468a0 msgr2=0x7f90040b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:44.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f8ff3fff700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90041468a0 0x7f90040b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 >> v2:172.21.15.152:3300/0 conn(0x7f90040b23b0 msgr2=0x7f900405c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:44.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f8ff3fff700 1 --2- 172.21.15.140:0/2988894128 >> v2:172.21.15.152:3300/0 conn(0x7f90040b23b0 0x7f900405c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90040ae660 con 0x7f90040b0580 2022-07-21T01:57:44.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.699+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f9000001660 con 0x7f90040b0580 2022-07-21T01:57:44.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 >> v1:172.21.15.140:6789/0 conn(0x7f90040b0580 legacy=0x7f9004146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f8ff3fff700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90041468a0 0x7f8fec003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f8ff3fff700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fec003c80 0x7f8fec004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f8ff3fff700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8fec0049f0 0x7f8fec004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90040ad0e0 con 0x7f8fec0049f0 2022-07-21T01:57:44.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec00b750 con 0x7f8fec003c80 2022-07-21T01:57:44.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec00b890 con 0x7f90041468a0 2022-07-21T01:57:44.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f9009f8b700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8fec0049f0 0x7f8fec004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900978a700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90041468a0 0x7f8fec003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f9008f89700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fec003c80 0x7f8fec004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900978a700 1 -- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fec003c80 msgr2=0x7f8fec004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:44.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900978a700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fec003c80 0x7f8fec004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900978a700 1 -- 172.21.15.140:0/2988894128 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8fec0049f0 msgr2=0x7f8fec004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:44.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900978a700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8fec0049f0 0x7f8fec004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900978a700 1 -- 172.21.15.140:0/2988894128 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec003260 con 0x7f90041468a0 2022-07-21T01:57:44.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f9009f8b700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8fec0049f0 0x7f8fec004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:44.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900978a700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90041468a0 0x7f8fec003750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f8ff400c000 tx=0x7f8ff4006df0).ready entity=mon.2 client_cookie=c584158d2f38bcd9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:44.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ff400e830 con 0x7f90041468a0 2022-07-21T01:57:44.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff400e990 con 0x7f90041468a0 2022-07-21T01:57:44.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f8ff3fff700 1 -- 172.21.15.140:0/2988894128 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ff400ecf0 con 0x7f90041468a0 2022-07-21T01:57:44.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900a78c700 1 -- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90041468a0 msgr2=0x7f8fec003750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900a78c700 1 --2- 172.21.15.140:0/2988894128 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90041468a0 0x7f8fec003750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f8ff400c000 tx=0x7f8ff4006df0).stop 2022-07-21T01:57:44.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f9009f8b700 1 -- 172.21.15.140:0/2988894128 reap_dead start 2022-07-21T01:57:44.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900a78c700 1 -- 172.21.15.140:0/2988894128 shutdown_connections 2022-07-21T01:57:44.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900a78c700 1 -- 172.21.15.140:0/2988894128 >> 172.21.15.140:0/2988894128 conn(0x7f90040632d0 msgr2=0x7f90040636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:44.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900a78c700 1 -- 172.21.15.140:0/2988894128 shutdown_connections 2022-07-21T01:57:44.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.703+0000 7f900a78c700 1 -- 172.21.15.140:0/2988894128 wait complete. 2022-07-21T01:57:44.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900a78c700 1 Processor -- start 2022-07-21T01:57:44.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900a78c700 1 -- start start 2022-07-21T01:57:44.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900a78c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90041468a0 0x7f90041161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900a78c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9004151b60 0x7f9004066250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900a78c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9004066740 0x7f90041521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900a78c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90041166d0 con 0x7f90041468a0 2022-07-21T01:57:44.728 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900a78c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9004114370 con 0x7f9004151b60 2022-07-21T01:57:44.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900a78c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9004114610 con 0x7f9004066740 2022-07-21T01:57:44.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900978a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90041468a0 0x7f90041161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900978a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90041468a0 0x7f90041161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41218/0 (socket says 172.21.15.140:41218) 2022-07-21T01:57:44.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900978a700 1 -- 172.21.15.140:0/558781806 learned_addr learned my addr 172.21.15.140:0/558781806 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:44.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f9008f89700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9004151b60 0x7f9004066250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f9009f8b700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9004066740 0x7f90041521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900978a700 1 -- 172.21.15.140:0/558781806 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9004066740 msgr2=0x7f90041521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900978a700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9004066740 0x7f90041521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900978a700 1 -- 172.21.15.140:0/558781806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9004151b60 msgr2=0x7f9004066250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900978a700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9004151b60 0x7f9004066250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900978a700 1 -- 172.21.15.140:0/558781806 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90041148b0 con 0x7f90041468a0 2022-07-21T01:57:44.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900978a700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90041468a0 0x7f90041161e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f8ff400c000 tx=0x7f8ff4019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:44.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f8ff27fc700 1 -- 172.21.15.140:0/558781806 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ff401ddb0 con 0x7f90041468a0 2022-07-21T01:57:44.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f8ff27fc700 1 -- 172.21.15.140:0/558781806 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff4000e90 con 0x7f90041468a0 2022-07-21T01:57:44.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f8ff27fc700 1 -- 172.21.15.140:0/558781806 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ff401b610 con 0x7f90041468a0 2022-07-21T01:57:44.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900a78c700 1 -- 172.21.15.140:0/558781806 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9004115560 con 0x7f90041468a0 2022-07-21T01:57:44.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.707+0000 7f900a78c700 1 -- 172.21.15.140:0/558781806 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90041526a0 con 0x7f90041468a0 2022-07-21T01:57:44.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.711+0000 7f900a78c700 1 -- 172.21.15.140:0/558781806 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90041141a0 con 0x7f90041468a0 2022-07-21T01:57:44.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.711+0000 7f8ff27fc700 1 -- 172.21.15.140:0/558781806 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8ff401b770 con 0x7f90041468a0 2022-07-21T01:57:44.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.711+0000 7f8ff27fc700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8fe4053b00 0x7f8fe4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.711+0000 7f8ff27fc700 1 -- 172.21.15.140:0/558781806 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8ff406d610 con 0x7f90041468a0 2022-07-21T01:57:44.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.711+0000 7f9008f89700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8fe4053b00 0x7f8fe4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.711+0000 7f9008f89700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8fe4053b00 0x7f8fe4055fc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f9000005000 tx=0x7f9000001e50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:44.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.719+0000 7f8ff27fc700 1 -- 172.21.15.140:0/558781806 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8ff4044530 con 0x7f90041468a0 2022-07-21T01:57:44.845 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.843+0000 7f900a78c700 1 -- 172.21.15.140:0/558781806 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f90040b1390 con 0x7f90041468a0 2022-07-21T01:57:44.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.843+0000 7f8ff27fc700 1 -- 172.21.15.140:0/558781806 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v142) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8ff402a070 con 0x7f90041468a0 2022-07-21T01:57:44.847 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:57:44.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 -- 172.21.15.140:0/558781806 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8fe4053b00 msgr2=0x7f8fe4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8fe4053b00 0x7f8fe4055fc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f9000005000 tx=0x7f9000001e50).stop 2022-07-21T01:57:44.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 -- 172.21.15.140:0/558781806 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90041468a0 msgr2=0x7f90041161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90041468a0 0x7f90041161e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f8ff400c000 tx=0x7f8ff4019000).stop 2022-07-21T01:57:44.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 -- 172.21.15.140:0/558781806 shutdown_connections 2022-07-21T01:57:44.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8fe4053b00 0x7f8fe4055fc0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9004066740 0x7f90041521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9004151b60 0x7f9004066250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 --2- 172.21.15.140:0/558781806 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90041468a0 0x7f90041161e0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 -- 172.21.15.140:0/558781806 >> 172.21.15.140:0/558781806 conn(0x7f90040632d0 msgr2=0x7f9004142c70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:44.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 -- 172.21.15.140:0/558781806 shutdown_connections 2022-07-21T01:57:44.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.847+0000 7f900a78c700 1 -- 172.21.15.140:0/558781806 wait complete. 2022-07-21T01:57:44.864 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:57:44.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.971+0000 7ff48759e700 1 Processor -- start 2022-07-21T01:57:44.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.971+0000 7ff48759e700 1 -- start start 2022-07-21T01:57:44.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff4880b59d0 0x7ff4880b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4880af920 0x7ff4880ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:44.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4880adf70 con 0x7ff48805d0c0 2022-07-21T01:57:44.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4880b5db0 con 0x7ff4880b59d0 2022-07-21T01:57:44.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff488146300 con 0x7ff4880af920 2022-07-21T01:57:44.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff486d9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff48805d0c0 0x7ff4880ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55374/0 (socket says 172.21.15.140:55374) 2022-07-21T01:57:44.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff486d9d700 1 -- 172.21.15.140:0/2179304223 learned_addr learned my addr 172.21.15.140:0/2179304223 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:44.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48659c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7ff4880b59d0 0x7ff4880b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff485d9b700 1 --2- 172.21.15.140:0/2179304223 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4880af920 0x7ff4880ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:44.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48559a700 1 -- 172.21.15.140:0/2179304223 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123422 0 0) 0x7ff4880adf70 con 0x7ff48805d0c0 2022-07-21T01:57:44.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48559a700 1 -- 172.21.15.140:0/2179304223 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff468003260 con 0x7ff48805d0c0 2022-07-21T01:57:44.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48559a700 1 -- 172.21.15.140:0/2179304223 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1858265232 0 0) 0x7ff468003260 con 0x7ff48805d0c0 2022-07-21T01:57:44.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48659c700 1 -- 172.21.15.140:0/2179304223 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4880af920 msgr2=0x7ff4880ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:44.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48659c700 1 --2- 172.21.15.140:0/2179304223 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4880af920 0x7ff4880ae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48659c700 1 -- 172.21.15.140:0/2179304223 >> v1:172.21.15.140:6789/0 conn(0x7ff48805d0c0 legacy=0x7ff4880ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48659c700 1 -- 172.21.15.140:0/2179304223 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488146060 con 0x7ff4880b59d0 2022-07-21T01:57:44.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff485d9b700 1 --2- 172.21.15.140:0/2179304223 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4880af920 0x7ff4880ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:44.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48659c700 1 --2- 172.21.15.140:0/2179304223 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4880b59d0 0x7ff4880b0040 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7ff4880add70 tx=0x7ff480006df0).ready entity=mon.0 client_cookie=76f634615ffacaf4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:44.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48559a700 1 -- 172.21.15.140:0/2179304223 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff478001a00 con 0x7ff4880b59d0 2022-07-21T01:57:44.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48559a700 1 -- 172.21.15.140:0/2179304223 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff480000e90 con 0x7ff4880b59d0 2022-07-21T01:57:44.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48559a700 1 -- 172.21.15.140:0/2179304223 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff480012460 con 0x7ff4880b59d0 2022-07-21T01:57:44.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 -- 172.21.15.140:0/2179304223 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4880b59d0 msgr2=0x7ff4880b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:44.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 --2- 172.21.15.140:0/2179304223 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4880b59d0 0x7ff4880b0040 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7ff4880add70 tx=0x7ff480006df0).stop 2022-07-21T01:57:44.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 -- 172.21.15.140:0/2179304223 shutdown_connections 2022-07-21T01:57:44.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 --2- 172.21.15.140:0/2179304223 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4880af920 0x7ff4880ae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 --2- 172.21.15.140:0/2179304223 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4880b59d0 0x7ff4880b0040 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:44.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 -- 172.21.15.140:0/2179304223 >> 172.21.15.140:0/2179304223 conn(0x7ff4880634c0 msgr2=0x7ff4880b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:45.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 -- 172.21.15.140:0/2179304223 shutdown_connections 2022-07-21T01:57:45.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.975+0000 7ff48759e700 1 -- 172.21.15.140:0/2179304223 wait complete. 2022-07-21T01:57:45.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48759e700 1 Processor -- start 2022-07-21T01:57:45.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48759e700 1 -- start start 2022-07-21T01:57:45.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff48805d0c0 0x7ff488142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4880af920 0x7ff488065640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4880b59d0 0x7ff48813fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4880671b0 con 0x7ff48805d0c0 2022-07-21T01:57:45.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff488064330 con 0x7ff4880af920 2022-07-21T01:57:45.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4880645d0 con 0x7ff4880b59d0 2022-07-21T01:57:45.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff48805d0c0 0x7ff488142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48659c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff48805d0c0 0x7ff488142cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41232/0 (socket says 172.21.15.140:41232) 2022-07-21T01:57:45.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48659c700 1 -- 172.21.15.140:0/2516379572 learned_addr learned my addr 172.21.15.140:0/2516379572 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:45.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff485d9b700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4880af920 0x7ff488065640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff486d9d700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4880b59d0 0x7ff48813fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48659c700 1 -- 172.21.15.140:0/2516379572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4880b59d0 msgr2=0x7ff48813fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48659c700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4880b59d0 0x7ff48813fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48659c700 1 -- 172.21.15.140:0/2516379572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4880af920 msgr2=0x7ff488065640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48659c700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4880af920 0x7ff488065640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48659c700 1 -- 172.21.15.140:0/2516379572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47000b180 con 0x7ff48805d0c0 2022-07-21T01:57:45.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48659c700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff48805d0c0 0x7ff488142cf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7ff4880add70 tx=0x7ff480014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:45.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff47f7fe700 1 -- 172.21.15.140:0/2516379572 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff480016070 con 0x7ff48805d0c0 2022-07-21T01:57:45.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff47f7fe700 1 -- 172.21.15.140:0/2516379572 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff480013e40 con 0x7ff48805d0c0 2022-07-21T01:57:45.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff47f7fe700 1 -- 172.21.15.140:0/2516379572 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff48000dcc0 con 0x7ff48805d0c0 2022-07-21T01:57:45.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48759e700 1 -- 172.21.15.140:0/2516379572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48000b180 con 0x7ff48805d0c0 2022-07-21T01:57:45.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.979+0000 7ff48759e700 1 -- 172.21.15.140:0/2516379572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff488064870 con 0x7ff48805d0c0 2022-07-21T01:57:45.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.983+0000 7ff48759e700 1 -- 172.21.15.140:0/2516379572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff488155240 con 0x7ff48805d0c0 2022-07-21T01:57:45.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.983+0000 7ff47f7fe700 1 -- 172.21.15.140:0/2516379572 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff480024020 con 0x7ff48805d0c0 2022-07-21T01:57:45.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.983+0000 7ff47f7fe700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff46c053a70 0x7ff46c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.983+0000 7ff47f7fe700 1 -- 172.21.15.140:0/2516379572 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff48006cd90 con 0x7ff48805d0c0 2022-07-21T01:57:45.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.983+0000 7ff485d9b700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff46c053a70 0x7ff46c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.983+0000 7ff485d9b700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff46c053a70 0x7ff46c055f30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7ff47000b000 tx=0x7ff470006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:45.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:44.991+0000 7ff47f7fe700 1 -- 172.21.15.140:0/2516379572 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff480043cc0 con 0x7ff48805d0c0 2022-07-21T01:57:45.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.115+0000 7ff48759e700 1 -- 172.21.15.140:0/2516379572 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff4880af2d0 con 0x7ff48805d0c0 2022-07-21T01:57:45.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.115+0000 7ff47f7fe700 1 -- 172.21.15.140:0/2516379572 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v142) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff4800472e0 con 0x7ff48805d0c0 2022-07-21T01:57:45.118 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:45.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 -- 172.21.15.140:0/2516379572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff46c053a70 msgr2=0x7ff46c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff46c053a70 0x7ff46c055f30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7ff47000b000 tx=0x7ff470006df0).stop 2022-07-21T01:57:45.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 -- 172.21.15.140:0/2516379572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff48805d0c0 msgr2=0x7ff488142cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff48805d0c0 0x7ff488142cf0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7ff4880add70 tx=0x7ff480014000).stop 2022-07-21T01:57:45.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 -- 172.21.15.140:0/2516379572 shutdown_connections 2022-07-21T01:57:45.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff46c053a70 0x7ff46c055f30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4880b59d0 0x7ff48813fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4880af920 0x7ff488065640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 --2- 172.21.15.140:0/2516379572 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff48805d0c0 0x7ff488142cf0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 -- 172.21.15.140:0/2516379572 >> 172.21.15.140:0/2516379572 conn(0x7ff4880634c0 msgr2=0x7ff4880b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:45.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 -- 172.21.15.140:0/2516379572 shutdown_connections 2022-07-21T01:57:45.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.119+0000 7ff48759e700 1 -- 172.21.15.140:0/2516379572 wait complete. 2022-07-21T01:57:45.131 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:57:45.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.243+0000 7f8b058dd700 1 Processor -- start 2022-07-21T01:57:45.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.243+0000 7f8b058dd700 1 -- start start 2022-07-21T01:57:45.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.243+0000 7f8b058dd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8b0005cf40 0x7f8b000678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.243+0000 7f8b058dd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b000b2e10 0x7f8b00061d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.243+0000 7f8b058dd700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b000b3670 con 0x7f8b000b1bf0 2022-07-21T01:57:45.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.243+0000 7f8b058dd700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b000b2310 con 0x7f8b0005cf40 2022-07-21T01:57:45.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.243+0000 7f8b058dd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000b3910 con 0x7f8b000b2e10 2022-07-21T01:57:45.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8afeffd700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8b0005cf40 0x7f8b000678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8aff7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b000b2e10 0x7f8b00061d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8afe7fc700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f8b000b1bf0 0x7f8b000b2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:56500/0 (socket says 172.21.15.140:56500) 2022-07-21T01:57:45.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8afe7fc700 1 -- 172.21.15.140:0/3012612583 learned_addr learned my addr 172.21.15.140:0/3012612583 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:45.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8afdffb700 1 -- 172.21.15.140:0/3012612583 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071970570 0 0) 0x7f8b000b3670 con 0x7f8b000b1bf0 2022-07-21T01:57:45.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8afdffb700 1 -- 172.21.15.140:0/3012612583 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae4003260 con 0x7f8b000b1bf0 2022-07-21T01:57:45.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8aff7fe700 1 -- 172.21.15.140:0/3012612583 >> v2:172.21.15.152:3300/0 conn(0x7f8b0005cf40 msgr2=0x7f8b000678f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:45.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8aff7fe700 1 --2- 172.21.15.140:0/3012612583 >> v2:172.21.15.152:3300/0 conn(0x7f8b0005cf40 0x7f8b000678f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8aff7fe700 1 -- 172.21.15.140:0/3012612583 >> v1:172.21.15.152:6789/0 conn(0x7f8b000b1bf0 legacy=0x7f8b000b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8aff7fe700 1 -- 172.21.15.140:0/3012612583 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0005d320 con 0x7f8b000b2e10 2022-07-21T01:57:45.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8aff7fe700 1 --2- 172.21.15.140:0/3012612583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b000b2e10 0x7f8b00061d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f8af80032f0 tx=0x7f8af80091d0).ready entity=mon.2 client_cookie=b2576c7ac9cb10ae server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:45.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8afdffb700 1 -- 172.21.15.140:0/3012612583 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af800b600 con 0x7f8b000b2e10 2022-07-21T01:57:45.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8afdffb700 1 -- 172.21.15.140:0/3012612583 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af800b760 con 0x7f8b000b2e10 2022-07-21T01:57:45.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8afdffb700 1 -- 172.21.15.140:0/3012612583 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af800b8d0 con 0x7f8b000b2e10 2022-07-21T01:57:45.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8b058dd700 1 -- 172.21.15.140:0/3012612583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b000b2e10 msgr2=0x7f8b00061d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8b058dd700 1 --2- 172.21.15.140:0/3012612583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b000b2e10 0x7f8b00061d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f8af80032f0 tx=0x7f8af80091d0).stop 2022-07-21T01:57:45.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8b058dd700 1 -- 172.21.15.140:0/3012612583 shutdown_connections 2022-07-21T01:57:45.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8b058dd700 1 --2- 172.21.15.140:0/3012612583 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b000b2e10 0x7f8b00061d00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8b058dd700 1 --2- 172.21.15.140:0/3012612583 >> v2:172.21.15.152:3300/0 conn(0x7f8b0005cf40 0x7f8b000678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8b058dd700 1 -- 172.21.15.140:0/3012612583 >> 172.21.15.140:0/3012612583 conn(0x7f8b000b7d20 msgr2=0x7f8b000b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:45.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8b058dd700 1 -- 172.21.15.140:0/3012612583 shutdown_connections 2022-07-21T01:57:45.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.247+0000 7f8b058dd700 1 -- 172.21.15.140:0/3012612583 wait complete. 2022-07-21T01:57:45.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8b058dd700 1 Processor -- start 2022-07-21T01:57:45.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8b058dd700 1 -- start start 2022-07-21T01:57:45.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8b058dd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b0005cf40 0x7f8b00144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8b058dd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b000b1bf0 0x7f8b0013f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8b058dd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b000b2e10 0x7f8b0014fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8b058dd700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b00153060 con 0x7f8b000b2e10 2022-07-21T01:57:45.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8b058dd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b001532c0 con 0x7f8b000b1bf0 2022-07-21T01:57:45.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8b058dd700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b00141eb0 con 0x7f8b0005cf40 2022-07-21T01:57:45.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8afeffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b0005cf40 0x7f8b00144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aff7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b000b2e10 0x7f8b0014fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8afe7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b000b1bf0 0x7f8b0013f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aff7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b000b2e10 0x7f8b0014fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41250/0 (socket says 172.21.15.140:41250) 2022-07-21T01:57:45.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aff7fe700 1 -- 172.21.15.140:0/819659516 learned_addr learned my addr 172.21.15.140:0/819659516 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:45.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8afeffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b0005cf40 0x7f8b00144ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:37976/0 (socket says 172.21.15.140:37976) 2022-07-21T01:57:45.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aff7fe700 1 -- 172.21.15.140:0/819659516 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b0005cf40 msgr2=0x7f8b00144ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aff7fe700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b0005cf40 0x7f8b00144ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aff7fe700 1 -- 172.21.15.140:0/819659516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b000b1bf0 msgr2=0x7f8b0013f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aff7fe700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b000b1bf0 0x7f8b0013f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aff7fe700 1 -- 172.21.15.140:0/819659516 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af8009470 con 0x7f8b000b2e10 2022-07-21T01:57:45.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8afe7fc700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b000b1bf0 0x7f8b0013f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:45.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aff7fe700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b000b2e10 0x7f8b0014fa10 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f8af80032f0 tx=0x7f8af8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:45.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aeffff700 1 -- 172.21.15.140:0/819659516 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af800b8d0 con 0x7f8b000b2e10 2022-07-21T01:57:45.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aeffff700 1 -- 172.21.15.140:0/819659516 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af8021630 con 0x7f8b000b2e10 2022-07-21T01:57:45.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8b058dd700 1 -- 172.21.15.140:0/819659516 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b00142180 con 0x7f8b000b2e10 2022-07-21T01:57:45.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8aeffff700 1 -- 172.21.15.140:0/819659516 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8af800b600 con 0x7f8b000b2e10 2022-07-21T01:57:45.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.251+0000 7f8b058dd700 1 -- 172.21.15.140:0/819659516 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0014ff00 con 0x7f8b000b2e10 2022-07-21T01:57:45.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.255+0000 7f8b058dd700 1 -- 172.21.15.140:0/819659516 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b000574e0 con 0x7f8b000b2e10 2022-07-21T01:57:45.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.255+0000 7f8aeffff700 1 -- 172.21.15.140:0/819659516 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8af800f070 con 0x7f8b000b2e10 2022-07-21T01:57:45.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.255+0000 7f8aeffff700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8adc053aa0 0x7f8adc055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.255+0000 7f8aeffff700 1 -- 172.21.15.140:0/819659516 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8af806d720 con 0x7f8b000b2e10 2022-07-21T01:57:45.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.255+0000 7f8afeffd700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8adc053aa0 0x7f8adc055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.255+0000 7f8afeffd700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8adc053aa0 0x7f8adc055f60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f8af400b000 tx=0x7f8af4006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:45.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.259+0000 7f8aeffff700 1 -- 172.21.15.140:0/819659516 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8af8044640 con 0x7f8b000b2e10 2022-07-21T01:57:45.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.387+0000 7f8b058dd700 1 -- 172.21.15.140:0/819659516 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b000008d0 con 0x7f8b000b2e10 2022-07-21T01:57:45.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.387+0000 7f8aeffff700 1 -- 172.21.15.140:0/819659516 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v142) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8af8047c60 con 0x7f8b000b2e10 2022-07-21T01:57:45.390 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:45.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 -- 172.21.15.140:0/819659516 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8adc053aa0 msgr2=0x7f8adc055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8adc053aa0 0x7f8adc055f60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f8af400b000 tx=0x7f8af4006df0).stop 2022-07-21T01:57:45.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 -- 172.21.15.140:0/819659516 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b000b2e10 msgr2=0x7f8b0014fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b000b2e10 0x7f8b0014fa10 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f8af80032f0 tx=0x7f8af8016000).stop 2022-07-21T01:57:45.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 -- 172.21.15.140:0/819659516 shutdown_connections 2022-07-21T01:57:45.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8adc053aa0 0x7f8adc055f60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8b000b2e10 0x7f8b0014fa10 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b000b1bf0 0x7f8b0013f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 --2- 172.21.15.140:0/819659516 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8b0005cf40 0x7f8b00144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 -- 172.21.15.140:0/819659516 >> 172.21.15.140:0/819659516 conn(0x7f8b000b7d20 msgr2=0x7f8b00066a70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:45.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 -- 172.21.15.140:0/819659516 shutdown_connections 2022-07-21T01:57:45.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.391+0000 7f8b058dd700 1 -- 172.21.15.140:0/819659516 wait complete. 2022-07-21T01:57:45.407 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:57:45.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafffff700 1 Processor -- start 2022-07-21T01:57:45.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafffff700 1 -- start start 2022-07-21T01:57:45.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdb000b59d0 0x7fdb000b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb000b14f0 0x7fdb00147050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb000afbc0 con 0x7fdb0014ada0 2022-07-21T01:57:45.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafffff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb00148080 con 0x7fdb000b59d0 2022-07-21T01:57:45.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb000b00e0 con 0x7fdb000b14f0 2022-07-21T01:57:45.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafe7fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fdb0014ada0 0x7fdb0005bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55402/0 (socket says 172.21.15.140:55402) 2022-07-21T01:57:45.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafe7fc700 1 -- 172.21.15.140:0/1075414325 learned_addr learned my addr 172.21.15.140:0/1075414325 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:45.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafeffd700 1 --2- 172.21.15.140:0/1075414325 >> v2:172.21.15.152:3300/0 conn(0x7fdb000b59d0 0x7fdb000b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdaff7fe700 1 --2- 172.21.15.140:0/1075414325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb000b14f0 0x7fdb00147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafdffb700 1 -- 172.21.15.140:0/1075414325 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829003399 0 0) 0x7fdb000afbc0 con 0x7fdb0014ada0 2022-07-21T01:57:45.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafdffb700 1 -- 172.21.15.140:0/1075414325 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdae0003260 con 0x7fdb0014ada0 2022-07-21T01:57:45.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafeffd700 1 -- 172.21.15.140:0/1075414325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb000b14f0 msgr2=0x7fdb00147050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:45.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafeffd700 1 --2- 172.21.15.140:0/1075414325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb000b14f0 0x7fdb00147050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafeffd700 1 -- 172.21.15.140:0/1075414325 >> v1:172.21.15.140:6789/0 conn(0x7fdb0014ada0 legacy=0x7fdb0005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafeffd700 1 -- 172.21.15.140:0/1075414325 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb000b5db0 con 0x7fdb000b59d0 2022-07-21T01:57:45.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafeffd700 1 --2- 172.21.15.140:0/1075414325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb000b59d0 0x7fdb000b07e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fdaf0002f70 tx=0x7fdaf0006df0).ready entity=mon.1 client_cookie=ff8b306f1808e6fe server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:45.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafdffb700 1 -- 172.21.15.140:0/1075414325 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf000d4a0 con 0x7fdb000b59d0 2022-07-21T01:57:45.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafdffb700 1 -- 172.21.15.140:0/1075414325 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf0000e90 con 0x7fdb000b59d0 2022-07-21T01:57:45.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.511+0000 7fdafdffb700 1 -- 172.21.15.140:0/1075414325 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf000d830 con 0x7fdb000b59d0 2022-07-21T01:57:45.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 -- 172.21.15.140:0/1075414325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb000b59d0 msgr2=0x7fdb000b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 --2- 172.21.15.140:0/1075414325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb000b59d0 0x7fdb000b07e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fdaf0002f70 tx=0x7fdaf0006df0).stop 2022-07-21T01:57:45.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 -- 172.21.15.140:0/1075414325 shutdown_connections 2022-07-21T01:57:45.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 --2- 172.21.15.140:0/1075414325 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb000b14f0 0x7fdb00147050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 --2- 172.21.15.140:0/1075414325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb000b59d0 0x7fdb000b07e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 -- 172.21.15.140:0/1075414325 >> 172.21.15.140:0/1075414325 conn(0x7fdb000634c0 msgr2=0x7fdb000b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:45.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 -- 172.21.15.140:0/1075414325 shutdown_connections 2022-07-21T01:57:45.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 -- 172.21.15.140:0/1075414325 wait complete. 2022-07-21T01:57:45.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 Processor -- start 2022-07-21T01:57:45.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 -- start start 2022-07-21T01:57:45.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb000b14f0 0x7fdb000aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb000b59d0 0x7fdb001117a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb0014ada0 0x7fdb00111c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb000642a0 con 0x7fdb0014ada0 2022-07-21T01:57:45.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb00064810 con 0x7fdb000b14f0 2022-07-21T01:57:45.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb00064000 con 0x7fdb000b59d0 2022-07-21T01:57:45.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdaff7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb0014ada0 0x7fdb00111c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdaff7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb0014ada0 0x7fdb00111c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41264/0 (socket says 172.21.15.140:41264) 2022-07-21T01:57:45.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdaff7fe700 1 -- 172.21.15.140:0/4249281645 learned_addr learned my addr 172.21.15.140:0/4249281645 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:45.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafeffd700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb000b14f0 0x7fdb000aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdafe7fc700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb000b59d0 0x7fdb001117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdaff7fe700 1 -- 172.21.15.140:0/4249281645 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb000b59d0 msgr2=0x7fdb001117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.515+0000 7fdaff7fe700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb000b59d0 0x7fdb001117a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.519+0000 7fdaff7fe700 1 -- 172.21.15.140:0/4249281645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb000b14f0 msgr2=0x7fdb000aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.519+0000 7fdaff7fe700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb000b14f0 0x7fdb000aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.519+0000 7fdaff7fe700 1 -- 172.21.15.140:0/4249281645 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf000b180 con 0x7fdb0014ada0 2022-07-21T01:57:45.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.519+0000 7fdaff7fe700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb0014ada0 0x7fdb00111c90 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7fdaf8006e00 tx=0x7fdaf80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:45.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.519+0000 7fdaf7fff700 1 -- 172.21.15.140:0/4249281645 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf8012600 con 0x7fdb0014ada0 2022-07-21T01:57:45.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.519+0000 7fdaf7fff700 1 -- 172.21.15.140:0/4249281645 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf8012760 con 0x7fdb0014ada0 2022-07-21T01:57:45.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.519+0000 7fdafffff700 1 -- 172.21.15.140:0/4249281645 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb00112180 con 0x7fdb0014ada0 2022-07-21T01:57:45.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.519+0000 7fdaf7fff700 1 -- 172.21.15.140:0/4249281645 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdaf8012ac0 con 0x7fdb0014ada0 2022-07-21T01:57:45.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.519+0000 7fdafffff700 1 -- 172.21.15.140:0/4249281645 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0010e170 con 0x7fdb0014ada0 2022-07-21T01:57:45.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.519+0000 7fdaf7fff700 1 -- 172.21.15.140:0/4249281645 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdaf8022070 con 0x7fdb0014ada0 2022-07-21T01:57:45.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.519+0000 7fdafffff700 1 -- 172.21.15.140:0/4249281645 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb000b0340 con 0x7fdb0014ada0 2022-07-21T01:57:45.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.523+0000 7fdaf7fff700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdae4053b00 0x7fdae4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.523+0000 7fdafeffd700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdae4053b00 0x7fdae4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.523+0000 7fdaf7fff700 1 -- 172.21.15.140:0/4249281645 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdaf806c590 con 0x7fdb0014ada0 2022-07-21T01:57:45.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.523+0000 7fdafeffd700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdae4053b00 0x7fdae4055fc0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fdaf001a000 tx=0x7fdaf0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:45.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.527+0000 7fdaf7fff700 1 -- 172.21.15.140:0/4249281645 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdaf8043410 con 0x7fdb0014ada0 2022-07-21T01:57:45.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.651+0000 7fdafffff700 1 -- 172.21.15.140:0/4249281645 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb00112310 con 0x7fdb0014ada0 2022-07-21T01:57:45.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.651+0000 7fdaf7fff700 1 -- 172.21.15.140:0/4249281645 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v142) v1 ==== 100+0+10 (secure 0 0 0) 0x7fdb00112310 con 0x7fdb0014ada0 2022-07-21T01:57:45.656 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:45.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 -- 172.21.15.140:0/4249281645 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdae4053b00 msgr2=0x7fdae4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdae4053b00 0x7fdae4055fc0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fdaf001a000 tx=0x7fdaf0014000).stop 2022-07-21T01:57:45.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 -- 172.21.15.140:0/4249281645 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb0014ada0 msgr2=0x7fdb00111c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb0014ada0 0x7fdb00111c90 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7fdaf8006e00 tx=0x7fdaf80091d0).stop 2022-07-21T01:57:45.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 -- 172.21.15.140:0/4249281645 shutdown_connections 2022-07-21T01:57:45.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdae4053b00 0x7fdae4055fc0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdb0014ada0 0x7fdb00111c90 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdb000b59d0 0x7fdb001117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 --2- 172.21.15.140:0/4249281645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdb000b14f0 0x7fdb000aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 -- 172.21.15.140:0/4249281645 >> 172.21.15.140:0/4249281645 conn(0x7fdb000634c0 msgr2=0x7fdb000b4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:45.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 -- 172.21.15.140:0/4249281645 shutdown_connections 2022-07-21T01:57:45.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.655+0000 7fdafffff700 1 -- 172.21.15.140:0/4249281645 wait complete. 2022-07-21T01:57:45.669 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:57:45.669 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:45.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2c1ecc700 1 Processor -- start 2022-07-21T01:57:45.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2c1ecc700 1 -- start start 2022-07-21T01:57:45.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2c1ecc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa2bc0adff0 0x7fa2bc0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2c1ecc700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2bc147010 con 0x7fa2bc0af8e0 2022-07-21T01:57:45.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2c1ecc700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2bc1472b0 con 0x7fa2bc0b5650 2022-07-21T01:57:45.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2c1ecc700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2bc146d40 con 0x7fa2bc0adff0 2022-07-21T01:57:45.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2c16cb700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fa2bc0af8e0 0x7fa2bc05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55416/0 (socket says 172.21.15.140:55416) 2022-07-21T01:57:45.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2c16cb700 1 -- 172.21.15.140:0/3305761656 learned_addr learned my addr 172.21.15.140:0/3305761656 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:45.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2bbfff700 1 --2- 172.21.15.140:0/3305761656 >> v2:172.21.15.152:3300/0 conn(0x7fa2bc0adff0 0x7fa2bc0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2bb7fe700 1 -- 172.21.15.140:0/3305761656 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083758334 0 0) 0x7fa2bc147010 con 0x7fa2bc0af8e0 2022-07-21T01:57:45.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2bb7fe700 1 -- 172.21.15.140:0/3305761656 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a0003260 con 0x7fa2bc0af8e0 2022-07-21T01:57:45.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2bbfff700 1 -- 172.21.15.140:0/3305761656 >> v1:172.21.15.152:6789/0 conn(0x7fa2bc0b5650 legacy=0x7fa2bc147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2bbfff700 1 -- 172.21.15.140:0/3305761656 >> v1:172.21.15.140:6789/0 conn(0x7fa2bc0af8e0 legacy=0x7fa2bc05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2bbfff700 1 -- 172.21.15.140:0/3305761656 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc05c4a0 con 0x7fa2bc0adff0 2022-07-21T01:57:45.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.767+0000 7fa2bbfff700 1 --2- 172.21.15.140:0/3305761656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2bc0adff0 0x7fa2bc0b5a30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fa2a4002f70 tx=0x7fa2a4006d90).ready entity=mon.1 client_cookie=44e3da7002a5c1a3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:45.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2bb7fe700 1 -- 172.21.15.140:0/3305761656 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2a400d4a0 con 0x7fa2bc0adff0 2022-07-21T01:57:45.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2bb7fe700 1 -- 172.21.15.140:0/3305761656 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a4000e90 con 0x7fa2bc0adff0 2022-07-21T01:57:45.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2bb7fe700 1 -- 172.21.15.140:0/3305761656 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2a400d7f0 con 0x7fa2bc0adff0 2022-07-21T01:57:45.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/3305761656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2bc0adff0 msgr2=0x7fa2bc0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2c1ecc700 1 --2- 172.21.15.140:0/3305761656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2bc0adff0 0x7fa2bc0b5a30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fa2a4002f70 tx=0x7fa2a4006d90).stop 2022-07-21T01:57:45.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/3305761656 shutdown_connections 2022-07-21T01:57:45.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2c1ecc700 1 --2- 172.21.15.140:0/3305761656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2bc0adff0 0x7fa2bc0b5a30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/3305761656 >> 172.21.15.140:0/3305761656 conn(0x7fa2bc0634e0 msgr2=0x7fa2bc0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:45.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/3305761656 shutdown_connections 2022-07-21T01:57:45.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/3305761656 wait complete. 2022-07-21T01:57:45.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2c1ecc700 1 Processor -- start 2022-07-21T01:57:45.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.771+0000 7fa2c1ecc700 1 -- start start 2022-07-21T01:57:45.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c1ecc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa2bc0adff0 0x7fa2bc0669b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c1ecc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2bc0af8e0 0x7fa2bc1100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c1ecc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa2bc0b5650 0x7fa2bc1105c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c1ecc700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc157040 con 0x7fa2bc0b5650 2022-07-21T01:57:45.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c1ecc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc110ab0 con 0x7fa2bc0af8e0 2022-07-21T01:57:45.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c1ecc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc1574c0 con 0x7fa2bc0adff0 2022-07-21T01:57:45.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c16cb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa2bc0b5650 0x7fa2bc1105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c16cb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa2bc0b5650 0x7fa2bc1105c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41276/0 (socket says 172.21.15.140:41276) 2022-07-21T01:57:45.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c16cb700 1 -- 172.21.15.140:0/1343923998 learned_addr learned my addr 172.21.15.140:0/1343923998 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:45.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2bbfff700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2bc0af8e0 0x7fa2bc1100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c0eca700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa2bc0adff0 0x7fa2bc0669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c16cb700 1 -- 172.21.15.140:0/1343923998 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa2bc0adff0 msgr2=0x7fa2bc0669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c16cb700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa2bc0adff0 0x7fa2bc0669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c16cb700 1 -- 172.21.15.140:0/1343923998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2bc0af8e0 msgr2=0x7fa2bc1100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c16cb700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2bc0af8e0 0x7fa2bc1100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c16cb700 1 -- 172.21.15.140:0/1343923998 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a400b120 con 0x7fa2bc0b5650 2022-07-21T01:57:45.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c16cb700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa2bc0b5650 0x7fa2bc1105c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7fa2b4001490 tx=0x7fa2b4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:45.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2b9ffb700 1 -- 172.21.15.140:0/1343923998 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2b4005ba0 con 0x7fa2bc0b5650 2022-07-21T01:57:45.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2b9ffb700 1 -- 172.21.15.140:0/1343923998 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b4005d00 con 0x7fa2bc0b5650 2022-07-21T01:57:45.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2b9ffb700 1 -- 172.21.15.140:0/1343923998 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2b40072c0 con 0x7fa2bc0b5650 2022-07-21T01:57:45.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/1343923998 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2bc156c90 con 0x7fa2bc0b5650 2022-07-21T01:57:45.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/1343923998 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2bc067410 con 0x7fa2bc0b5650 2022-07-21T01:57:45.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/1343923998 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2bc0574e0 con 0x7fa2bc0b5650 2022-07-21T01:57:45.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.775+0000 7fa2b9ffb700 1 -- 172.21.15.140:0/1343923998 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa2b4010070 con 0x7fa2bc0b5650 2022-07-21T01:57:45.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.779+0000 7fa2b9ffb700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa294053a70 0x7fa294055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:45.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.779+0000 7fa2b9ffb700 1 -- 172.21.15.140:0/1343923998 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa2b406b9b0 con 0x7fa2bc0b5650 2022-07-21T01:57:45.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.779+0000 7fa2c0eca700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa294053a70 0x7fa294055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:45.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.779+0000 7fa2c0eca700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa294053a70 0x7fa294055f30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fa2ac00b000 tx=0x7fa2ac009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:45.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.783+0000 7fa2b9ffb700 1 -- 172.21.15.140:0/1343923998 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa2b4042830 con 0x7fa2bc0b5650 2022-07-21T01:57:45.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.943+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/1343923998 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2bc067ea0 con 0x7fa2bc0b5650 2022-07-21T01:57:45.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.943+0000 7fa2b9ffb700 1 -- 172.21.15.140:0/1343923998 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2142 (secure 0 0 0) 0x7fa2b4045e50 con 0x7fa2bc0b5650 2022-07-21T01:57:45.948 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:45.948 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":69,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/148832193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":148832193},{"type":"v1","addr":"172.21.15.152:6833","nonce":148832193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:41.192528+0000","modified":"2022-07-21T01:57:42.195096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8308":{"gid":8308,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/961657495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":961657495},{"type":"v1","addr":"172.21.15.140:6835","nonce":961657495}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-07-21T01:57:45.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/1343923998 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa294053a70 msgr2=0x7fa294055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa294053a70 0x7fa294055f30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fa2ac00b000 tx=0x7fa2ac009040).stop 2022-07-21T01:57:45.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/1343923998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa2bc0b5650 msgr2=0x7fa2bc1105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:45.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa2bc0b5650 0x7fa2bc1105c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7fa2b4001490 tx=0x7fa2b4007e00).stop 2022-07-21T01:57:45.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/1343923998 shutdown_connections 2022-07-21T01:57:45.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa294053a70 0x7fa294055f30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa2bc0b5650 0x7fa2bc1105c0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa2bc0af8e0 0x7fa2bc1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 --2- 172.21.15.140:0/1343923998 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa2bc0adff0 0x7fa2bc0669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:45.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/1343923998 >> 172.21.15.140:0/1343923998 conn(0x7fa2bc0634e0 msgr2=0x7fa2bc14a250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:45.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/1343923998 shutdown_connections 2022-07-21T01:57:45.956 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:45.947+0000 7fa2c1ecc700 1 -- 172.21.15.140:0/1343923998 wait complete. 2022-07-21T01:57:45.957 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 69 2022-07-21T01:57:46.162 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:57:41.192528+0000', 'modified': '2022-07-21T01:57:42.195096+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8308}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8308': {'gid': 8308, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/961657495', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 961657495}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 961657495}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:57:46.163 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:57:46.164 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:57:46.281 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":8308,"want_state":"up:active","state":"up:active","rank_uptime":5.0750696020000001,"mdsmap_epoch":69,"osdmap_epoch":142,"osdmap_epoch_barrier":141,"uptime":9.4392719189999994} 2022-07-21T01:57:46.282 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 8308, "mdsmap_epoch": 69, "osdmap_epoch": 142, "osdmap_epoch_barrier": 141, "rank_uptime": 5.075069602, "state": "up:active", "uptime": 9.439271919, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:57:46.282 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:57:46.283 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:57:46.283 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:57:46.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:46.284 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:46.293 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:57:46.353 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:46.354 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:46.365 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:46.366 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:46.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:46.368 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:57:46.376 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:57:46.377 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:46.422 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:57:46.438 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:46.439 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:46.450 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:46.450 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:46.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:46.452 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:57:46.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:57:46.490+0000 7f73d37d1080 -1 init, newargv = 0x561d7462e070 newargc=9ceph-fuse[23438]: starting ceph client 2022-07-21T01:57:46.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr: 2022-07-21T01:57:46.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23438]: starting fuse 2022-07-21T01:57:47.559 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:57:47.578 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:47.578 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:47.635 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:47.637 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:47.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:47.639 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:57:47.646 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:57:47.648 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:57:47.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:47.649 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:57:47.650 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:57:47.650 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:57:47.651 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:57:47.651 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:57:47.652 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:57:47.652 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:57:47.652 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:57:47.653 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:57:47.653 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:57:47.654 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:57:47.654 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:57:47.655 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:57:47.655 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:57:47.655 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:57:47.656 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:57:47.656 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:57:47.657 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:57:47.657 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:57:47.657 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:57:47.658 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:57:47.658 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:57:47.659 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:57:47.703 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.23438.asok 2022-07-21T01:57:47.708 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23438.asok 2022-07-21T01:57:47.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:47.710 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23438.asok status 2022-07-21T01:57:47.849 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:57:47.849 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:57:47.850 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:57:47.850 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:57:47.850 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:57:47.850 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:57:47.851 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:57:47.851 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "23438", 2022-07-21T01:57:47.851 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:57:47.851 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:57:47.851 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:57:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:57:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 8583, 2022-07-21T01:57:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:57:47.852 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:57:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:57:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 8583 2022-07-21T01:57:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:57:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:57:47.853 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:57:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:57:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 2100711643 2022-07-21T01:57:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:57:47.854 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:57:47.855 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:57:47.855 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:57:47.855 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:57:47.855 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 2100711643 2022-07-21T01:57:47.856 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:57:47.856 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.8583 172.21.15.191:0/2100711643", 2022-07-21T01:57:47.856 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/2100711643", 2022-07-21T01:57:47.856 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:57:47.856 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 69, 2022-07-21T01:57:47.857 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 142, 2022-07-21T01:57:47.857 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:57:47.857 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:57:47.857 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:57:47.858 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:57:47.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:47.860 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:47.860 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:47.867 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:57:47.868 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:57:47.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:47.869 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:47.887 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:57:48.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 Processor -- start 2022-07-21T01:57:48.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 -- start start 2022-07-21T01:57:48.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3fec0b5650 0x7f3fec0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3fec0adff0 0x7f3fec147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3fec0af8e0 0x7f3fec05bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fec147010 con 0x7f3fec0b5650 2022-07-21T01:57:48.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fec1472b0 con 0x7f3fec0adff0 2022-07-21T01:57:48.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec146d40 con 0x7f3fec0af8e0 2022-07-21T01:57:48.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fead9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3fec0b5650 0x7f3fec0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fead9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3fec0b5650 0x7f3fec0b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41280/0 (socket says 172.21.15.140:41280) 2022-07-21T01:57:48.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fead9d700 1 -- 172.21.15.140:0/4150990299 learned_addr learned my addr 172.21.15.140:0/4150990299 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:48.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3feb59e700 1 --2- 172.21.15.140:0/4150990299 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3fec0af8e0 0x7f3fec05bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fea59c700 1 --2- 172.21.15.140:0/4150990299 >> v2:172.21.15.152:3300/0 conn(0x7f3fec0adff0 0x7f3fec147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fead9d700 1 -- 172.21.15.140:0/4150990299 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3fec0af8e0 msgr2=0x7f3fec05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:48.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fead9d700 1 --2- 172.21.15.140:0/4150990299 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3fec0af8e0 0x7f3fec05bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fead9d700 1 -- 172.21.15.140:0/4150990299 >> v2:172.21.15.152:3300/0 conn(0x7f3fec0adff0 msgr2=0x7f3fec147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:48.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fead9d700 1 --2- 172.21.15.140:0/4150990299 >> v2:172.21.15.152:3300/0 conn(0x7f3fec0adff0 0x7f3fec147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fead9d700 1 -- 172.21.15.140:0/4150990299 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec147c40 con 0x7f3fec0b5650 2022-07-21T01:57:48.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3feb59e700 1 --2- 172.21.15.140:0/4150990299 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3fec0af8e0 0x7f3fec05bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:48.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fead9d700 1 --2- 172.21.15.140:0/4150990299 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3fec0b5650 0x7f3fec0b5a30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f3fe40032f0 tx=0x7f3fe4009290).ready entity=mon.0 client_cookie=6f506c8a54cbf9f6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fe9d9b700 1 -- 172.21.15.140:0/4150990299 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fe400b680 con 0x7f3fec0b5650 2022-07-21T01:57:48.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fe9d9b700 1 -- 172.21.15.140:0/4150990299 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe400b7e0 con 0x7f3fec0b5650 2022-07-21T01:57:48.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3fe9d9b700 1 -- 172.21.15.140:0/4150990299 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fe400b950 con 0x7f3fec0b5650 2022-07-21T01:57:48.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 -- 172.21.15.140:0/4150990299 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3fec0b5650 msgr2=0x7f3fec0b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 --2- 172.21.15.140:0/4150990299 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3fec0b5650 0x7f3fec0b5a30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f3fe40032f0 tx=0x7f3fe4009290).stop 2022-07-21T01:57:48.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 -- 172.21.15.140:0/4150990299 shutdown_connections 2022-07-21T01:57:48.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 --2- 172.21.15.140:0/4150990299 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3fec0af8e0 0x7f3fec05bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 --2- 172.21.15.140:0/4150990299 >> v2:172.21.15.152:3300/0 conn(0x7f3fec0adff0 0x7f3fec147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 --2- 172.21.15.140:0/4150990299 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3fec0b5650 0x7f3fec0b5a30 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 -- 172.21.15.140:0/4150990299 >> 172.21.15.140:0/4150990299 conn(0x7f3fec0634e0 msgr2=0x7f3fec0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:48.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:47.999+0000 7f3ff082c700 1 -- 172.21.15.140:0/4150990299 shutdown_connections 2022-07-21T01:57:48.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3ff082c700 1 -- 172.21.15.140:0/4150990299 wait complete. 2022-07-21T01:57:48.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3ff082c700 1 Processor -- start 2022-07-21T01:57:48.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3ff082c700 1 -- start start 2022-07-21T01:57:48.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3ff082c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3fec0adff0 0x7f3fec066f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3ff082c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3fec0af8e0 0x7f3fec067480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3ff082c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fec0b5650 0x7f3fec065110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3ff082c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec1100b0 con 0x7f3fec0adff0 2022-07-21T01:57:48.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3ff082c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec156d30 con 0x7f3fec0b5650 2022-07-21T01:57:48.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3ff082c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fec156fd0 con 0x7f3fec0af8e0 2022-07-21T01:57:48.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fead9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3fec0adff0 0x7f3fec066f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fead9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3fec0adff0 0x7f3fec066f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41284/0 (socket says 172.21.15.140:41284) 2022-07-21T01:57:48.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fead9d700 1 -- 172.21.15.140:0/2570962487 learned_addr learned my addr 172.21.15.140:0/2570962487 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:48.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fea59c700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3fec0af8e0 0x7f3fec067480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3feb59e700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fec0b5650 0x7f3fec065110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fead9d700 1 -- 172.21.15.140:0/2570962487 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3fec0af8e0 msgr2=0x7f3fec067480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fead9d700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3fec0af8e0 0x7f3fec067480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fead9d700 1 -- 172.21.15.140:0/2570962487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fec0b5650 msgr2=0x7f3fec065110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fead9d700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fec0b5650 0x7f3fec065110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fead9d700 1 -- 172.21.15.140:0/2570962487 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc00b180 con 0x7f3fec0adff0 2022-07-21T01:57:48.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fead9d700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3fec0adff0 0x7f3fec066f90 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f3fe40032f0 tx=0x7f3fe4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fdbfff700 1 -- 172.21.15.140:0/2570962487 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fe401f070 con 0x7f3fec0adff0 2022-07-21T01:57:48.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fdbfff700 1 -- 172.21.15.140:0/2570962487 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe401b440 con 0x7f3fec0adff0 2022-07-21T01:57:48.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3ff082c700 1 -- 172.21.15.140:0/2570962487 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe4009530 con 0x7f3fec0adff0 2022-07-21T01:57:48.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3fdbfff700 1 -- 172.21.15.140:0/2570962487 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fe400b680 con 0x7f3fec0adff0 2022-07-21T01:57:48.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.003+0000 7f3ff082c700 1 -- 172.21.15.140:0/2570962487 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec10fda0 con 0x7f3fec0adff0 2022-07-21T01:57:48.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.007+0000 7f3fdbfff700 1 -- 172.21.15.140:0/2570962487 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3fe400f070 con 0x7f3fec0adff0 2022-07-21T01:57:48.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.007+0000 7f3ff082c700 1 -- 172.21.15.140:0/2570962487 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fec0574e0 con 0x7f3fec0adff0 2022-07-21T01:57:48.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.007+0000 7f3fdbfff700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3fc8053950 0x7f3fc8055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.007+0000 7f3fdbfff700 1 -- 172.21.15.140:0/2570962487 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3fe406d250 con 0x7f3fec0adff0 2022-07-21T01:57:48.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.007+0000 7f3fea59c700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3fc8053950 0x7f3fc8055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.007+0000 7f3fea59c700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3fc8053950 0x7f3fc8055e10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f3fe0006000 tx=0x7f3fe0006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.011+0000 7f3fdbfff700 1 -- 172.21.15.140:0/2570962487 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3fe4044170 con 0x7f3fec0adff0 2022-07-21T01:57:48.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.135+0000 7f3ff082c700 1 -- 172.21.15.140:0/2570962487 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fec067ea0 con 0x7f3fec0adff0 2022-07-21T01:57:48.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3fdbfff700 1 -- 172.21.15.140:0/2570962487 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f3fe4047790 con 0x7f3fec0adff0 2022-07-21T01:57:48.140 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:48.140 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":142,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:57:42.195796+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:57:39.555617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2022-07-21T01:57:40.211694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:57:48.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 -- 172.21.15.140:0/2570962487 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3fc8053950 msgr2=0x7f3fc8055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3fc8053950 0x7f3fc8055e10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f3fe0006000 tx=0x7f3fe0006d90).stop 2022-07-21T01:57:48.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 -- 172.21.15.140:0/2570962487 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3fec0adff0 msgr2=0x7f3fec066f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3fec0adff0 0x7f3fec066f90 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7f3fe40032f0 tx=0x7f3fe4017000).stop 2022-07-21T01:57:48.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 -- 172.21.15.140:0/2570962487 shutdown_connections 2022-07-21T01:57:48.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3fc8053950 0x7f3fc8055e10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3fec0b5650 0x7f3fec065110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3fec0af8e0 0x7f3fec067480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 --2- 172.21.15.140:0/2570962487 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3fec0adff0 0x7f3fec066f90 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 -- 172.21.15.140:0/2570962487 >> 172.21.15.140:0/2570962487 conn(0x7f3fec0634e0 msgr2=0x7f3fec0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:48.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 -- 172.21.15.140:0/2570962487 shutdown_connections 2022-07-21T01:57:48.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.139+0000 7f3ff082c700 1 -- 172.21.15.140:0/2570962487 wait complete. 2022-07-21T01:57:48.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:57:39.555617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:57:40.211694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:57:48.157 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:57:48.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f6908ff7700 1 Processor -- start 2022-07-21T01:57:48.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f6908ff7700 1 -- start start 2022-07-21T01:57:48.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f6908ff7700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f69040b59f0 0x7f69041473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f6908ff7700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690405cc70 con 0x7f69040af940 2022-07-21T01:57:48.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f6908ff7700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69040b5dd0 con 0x7f69040b1400 2022-07-21T01:57:48.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f6908ff7700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69040b1070 con 0x7f69040b59f0 2022-07-21T01:57:48.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f6902ffd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f69040af940 0x7f690414a3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55440/0 (socket says 172.21.15.140:55440) 2022-07-21T01:57:48.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f6902ffd700 1 -- 172.21.15.140:0/4121425143 learned_addr learned my addr 172.21.15.140:0/4121425143 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:48.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69037fe700 1 --2- 172.21.15.140:0/4121425143 >> v2:172.21.15.152:3300/0 conn(0x7f69040b59f0 0x7f69041473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294081549 0 0) 0x7f690405cc70 con 0x7f69040af940 2022-07-21T01:57:48.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f0003260 con 0x7f69040af940 2022-07-21T01:57:48.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632988457 0 0) 0x7f69040b5dd0 con 0x7f69040b1400 2022-07-21T01:57:48.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f690405cc70 con 0x7f69040b1400 2022-07-21T01:57:48.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3943166615 0 0) 0x7f68f0003260 con 0x7f69040af940 2022-07-21T01:57:48.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 >> v2:172.21.15.152:3300/0 conn(0x7f69040b59f0 msgr2=0x7f69041473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:48.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 --2- 172.21.15.140:0/4121425143 >> v2:172.21.15.152:3300/0 conn(0x7f69040b59f0 0x7f69041473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 >> v1:172.21.15.152:6789/0 conn(0x7f69040b1400 legacy=0x7f690414aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690405c760 con 0x7f69040af940 2022-07-21T01:57:48.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f68f4001660 con 0x7f69040af940 2022-07-21T01:57:48.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 >> v1:172.21.15.140:6789/0 conn(0x7f69040af940 legacy=0x7f690414a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 --2- 172.21.15.140:0/4121425143 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 0x7f68f0003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 --2- 172.21.15.140:0/4121425143 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68f0004500 0x7f68f0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 --2- 172.21.15.140:0/4121425143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68f0004e70 0x7f68f000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040b1070 con 0x7f68f0003930 2022-07-21T01:57:48.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f000bdd0 con 0x7f68f0004e70 2022-07-21T01:57:48.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f000bf90 con 0x7f68f0004500 2022-07-21T01:57:48.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69037fe700 1 --2- 172.21.15.140:0/4121425143 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 0x7f68f0003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f6902ffd700 1 --2- 172.21.15.140:0/4121425143 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68f0004500 0x7f68f0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f6903fff700 1 --2- 172.21.15.140:0/4121425143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68f0004e70 0x7f68f000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69037fe700 1 -- 172.21.15.140:0/4121425143 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68f0004500 msgr2=0x7f68f0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:48.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69037fe700 1 --2- 172.21.15.140:0/4121425143 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f68f0004500 0x7f68f0004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69037fe700 1 -- 172.21.15.140:0/4121425143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68f0004e70 msgr2=0x7f68f000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69037fe700 1 --2- 172.21.15.140:0/4121425143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68f0004e70 0x7f68f000b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f6903fff700 1 -- 172.21.15.140:0/4121425143 reap_dead start 2022-07-21T01:57:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.267+0000 7f69037fe700 1 -- 172.21.15.140:0/4121425143 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f0003260 con 0x7f68f0003930 2022-07-21T01:57:48.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f69037fe700 1 --2- 172.21.15.140:0/4121425143 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 0x7f68f0003d30 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f68f800c000 tx=0x7f68f8006df0).ready entity=mon.0 client_cookie=cb4db219869be8e6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68f800e830 con 0x7f68f0003930 2022-07-21T01:57:48.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f800e990 con 0x7f68f0003930 2022-07-21T01:57:48.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f69027fc700 1 -- 172.21.15.140:0/4121425143 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68f800ecf0 con 0x7f68f0003930 2022-07-21T01:57:48.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f6908ff7700 1 -- 172.21.15.140:0/4121425143 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 msgr2=0x7f68f0003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f6908ff7700 1 --2- 172.21.15.140:0/4121425143 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 0x7f68f0003d30 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f68f800c000 tx=0x7f68f8006df0).stop 2022-07-21T01:57:48.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f6908ff7700 1 -- 172.21.15.140:0/4121425143 shutdown_connections 2022-07-21T01:57:48.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f6908ff7700 1 --2- 172.21.15.140:0/4121425143 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 0x7f68f0003d30 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f6908ff7700 1 -- 172.21.15.140:0/4121425143 >> 172.21.15.140:0/4121425143 conn(0x7f6904063480 msgr2=0x7f69040b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:48.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f6908ff7700 1 -- 172.21.15.140:0/4121425143 shutdown_connections 2022-07-21T01:57:48.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f6908ff7700 1 -- 172.21.15.140:0/4121425143 wait complete. 2022-07-21T01:57:48.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f6908ff7700 1 Processor -- start 2022-07-21T01:57:48.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.271+0000 7f6908ff7700 1 -- start start 2022-07-21T01:57:48.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6908ff7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 0x7f6904142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6908ff7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6904141dd0 0x7f6904143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6908ff7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6904149250 0x7f69041496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6908ff7700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69041432a0 con 0x7f68f0003930 2022-07-21T01:57:48.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6908ff7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69041572f0 con 0x7f6904141dd0 2022-07-21T01:57:48.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6908ff7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6904157050 con 0x7f6904149250 2022-07-21T01:57:48.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f69037fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 0x7f6904142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f69037fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 0x7f6904142b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41304/0 (socket says 172.21.15.140:41304) 2022-07-21T01:57:48.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f69037fe700 1 -- 172.21.15.140:0/121986970 learned_addr learned my addr 172.21.15.140:0/121986970 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:48.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6903fff700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6904149250 0x7f69041496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6902ffd700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6904141dd0 0x7f6904143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f69037fe700 1 -- 172.21.15.140:0/121986970 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6904149250 msgr2=0x7f69041496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f69037fe700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6904149250 0x7f69041496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f69037fe700 1 -- 172.21.15.140:0/121986970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6904141dd0 msgr2=0x7f6904143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f69037fe700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6904141dd0 0x7f6904143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f69037fe700 1 -- 172.21.15.140:0/121986970 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69041435d0 con 0x7f68f0003930 2022-07-21T01:57:48.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6903fff700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6904149250 0x7f69041496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:48.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6902ffd700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6904141dd0 0x7f6904143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:48.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f69037fe700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 0x7f6904142b30 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f6904157430 tx=0x7f68f8019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6900ff9700 1 -- 172.21.15.140:0/121986970 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68f801cdf0 con 0x7f68f0003930 2022-07-21T01:57:48.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6908ff7700 1 -- 172.21.15.140:0/121986970 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6904143f60 con 0x7f68f0003930 2022-07-21T01:57:48.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6900ff9700 1 -- 172.21.15.140:0/121986970 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f8000e90 con 0x7f68f0003930 2022-07-21T01:57:48.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6900ff9700 1 -- 172.21.15.140:0/121986970 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f68f80245e0 con 0x7f68f0003930 2022-07-21T01:57:48.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6908ff7700 1 -- 172.21.15.140:0/121986970 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69041419d0 con 0x7f68f0003930 2022-07-21T01:57:48.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6900ff9700 1 -- 172.21.15.140:0/121986970 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f68f802a020 con 0x7f68f0003930 2022-07-21T01:57:48.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6908ff7700 1 -- 172.21.15.140:0/121986970 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6904143760 con 0x7f68f0003930 2022-07-21T01:57:48.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6900ff9700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68e4053b00 0x7f68e4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6902ffd700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68e4053b00 0x7f68e4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.275+0000 7f6900ff9700 1 -- 172.21.15.140:0/121986970 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f68f806d580 con 0x7f68f0003930 2022-07-21T01:57:48.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.279+0000 7f6902ffd700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68e4053b00 0x7f68e4055fc0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f68f4005000 tx=0x7f68f4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.283+0000 7f6900ff9700 1 -- 172.21.15.140:0/121986970 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f68f8044400 con 0x7f68f0003930 2022-07-21T01:57:48.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.407+0000 7f6908ff7700 1 -- 172.21.15.140:0/121986970 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f690405c9d0 con 0x7f68f0003930 2022-07-21T01:57:48.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.407+0000 7f6900ff9700 1 -- 172.21.15.140:0/121986970 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v142) v1 ==== 110+0+10 (secure 0 0 0) 0x7f68f8047a20 con 0x7f68f0003930 2022-07-21T01:57:48.410 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:57:48.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 -- 172.21.15.140:0/121986970 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68e4053b00 msgr2=0x7f68e4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68e4053b00 0x7f68e4055fc0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f68f4005000 tx=0x7f68f4009040).stop 2022-07-21T01:57:48.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 -- 172.21.15.140:0/121986970 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 msgr2=0x7f6904142b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 0x7f6904142b30 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f6904157430 tx=0x7f68f8019000).stop 2022-07-21T01:57:48.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 -- 172.21.15.140:0/121986970 shutdown_connections 2022-07-21T01:57:48.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f68e4053b00 0x7f68e4055fc0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6904149250 0x7f69041496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6904141dd0 0x7f6904143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 --2- 172.21.15.140:0/121986970 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f68f0003930 0x7f6904142b30 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 -- 172.21.15.140:0/121986970 >> 172.21.15.140:0/121986970 conn(0x7f6904063480 msgr2=0x7f69040add40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:48.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 -- 172.21.15.140:0/121986970 shutdown_connections 2022-07-21T01:57:48.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.411+0000 7f6908ff7700 1 -- 172.21.15.140:0/121986970 wait complete. 2022-07-21T01:57:48.426 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:57:48.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f678cc9e700 1 Processor -- start 2022-07-21T01:57:48.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f678cc9e700 1 -- start start 2022-07-21T01:57:48.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f678cc9e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f678814ada0 0x7f6788146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f678cc9e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f67880b14f0 0x7f6788146530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f678cc9e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67880afbc0 con 0x7f67880b59d0 2022-07-21T01:57:48.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f678cc9e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6788148080 con 0x7f67880b14f0 2022-07-21T01:57:48.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f678cc9e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67880b00e0 con 0x7f678814ada0 2022-07-21T01:57:48.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f6787fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f67880b14f0 0x7f6788146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f6787fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f67880b14f0 0x7f6788146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41316/0 (socket says 172.21.15.140:41316) 2022-07-21T01:57:48.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f6787fff700 1 -- 172.21.15.140:0/411554878 learned_addr learned my addr 172.21.15.140:0/411554878 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:48.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f6786ffd700 1 --2- 172.21.15.140:0/411554878 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f678814ada0 0x7f6788146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f6787fff700 1 -- 172.21.15.140:0/411554878 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f678814ada0 msgr2=0x7f6788146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:48.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f6787fff700 1 --2- 172.21.15.140:0/411554878 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f678814ada0 0x7f6788146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f6787fff700 1 -- 172.21.15.140:0/411554878 >> v1:172.21.15.152:6789/0 conn(0x7f67880b59d0 legacy=0x7f67880b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f6787fff700 1 -- 172.21.15.140:0/411554878 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67880b5db0 con 0x7f67880b14f0 2022-07-21T01:57:48.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f6787fff700 1 --2- 172.21.15.140:0/411554878 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67880b14f0 0x7f6788146530 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f67800032f0 tx=0x7f6780009290).ready entity=mon.0 client_cookie=be7dc991ee84caca server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f67867fc700 1 -- 172.21.15.140:0/411554878 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678000b8a0 con 0x7f67880b14f0 2022-07-21T01:57:48.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f67867fc700 1 -- 172.21.15.140:0/411554878 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678000ba00 con 0x7f67880b14f0 2022-07-21T01:57:48.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.535+0000 7f67867fc700 1 -- 172.21.15.140:0/411554878 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678000bb70 con 0x7f67880b14f0 2022-07-21T01:57:48.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 -- 172.21.15.140:0/411554878 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67880b14f0 msgr2=0x7f6788146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 --2- 172.21.15.140:0/411554878 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67880b14f0 0x7f6788146530 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f67800032f0 tx=0x7f6780009290).stop 2022-07-21T01:57:48.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 -- 172.21.15.140:0/411554878 shutdown_connections 2022-07-21T01:57:48.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 --2- 172.21.15.140:0/411554878 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f67880b14f0 0x7f6788146530 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 --2- 172.21.15.140:0/411554878 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f678814ada0 0x7f6788146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 -- 172.21.15.140:0/411554878 >> 172.21.15.140:0/411554878 conn(0x7f67880634c0 msgr2=0x7f67880b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:48.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 -- 172.21.15.140:0/411554878 shutdown_connections 2022-07-21T01:57:48.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 -- 172.21.15.140:0/411554878 wait complete. 2022-07-21T01:57:48.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 Processor -- start 2022-07-21T01:57:48.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 -- start start 2022-07-21T01:57:48.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67880b14f0 0x7f67880b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67880b59d0 0x7f6788148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f678814ada0 0x7f6788159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678814a1a0 con 0x7f678814ada0 2022-07-21T01:57:48.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678814c3d0 con 0x7f67880b14f0 2022-07-21T01:57:48.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f678cc9e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6788159690 con 0x7f67880b59d0 2022-07-21T01:57:48.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f6787fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f678814ada0 0x7f6788159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f6787fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f678814ada0 0x7f6788159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41322/0 (socket says 172.21.15.140:41322) 2022-07-21T01:57:48.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f6787fff700 1 -- 172.21.15.140:0/841125743 learned_addr learned my addr 172.21.15.140:0/841125743 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:48.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f6786ffd700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67880b59d0 0x7f6788148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f67877fe700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67880b14f0 0x7f67880b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f6787fff700 1 -- 172.21.15.140:0/841125743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67880b59d0 msgr2=0x7f6788148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f6787fff700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67880b59d0 0x7f6788148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f6787fff700 1 -- 172.21.15.140:0/841125743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67880b14f0 msgr2=0x7f67880b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f6787fff700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67880b14f0 0x7f67880b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.539+0000 7f6787fff700 1 -- 172.21.15.140:0/841125743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6780009530 con 0x7f678814ada0 2022-07-21T01:57:48.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f6787fff700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f678814ada0 0x7f6788159af0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f67800032f0 tx=0x7f6780016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f6784ff9700 1 -- 172.21.15.140:0/841125743 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678000bb70 con 0x7f678814ada0 2022-07-21T01:57:48.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f6784ff9700 1 -- 172.21.15.140:0/841125743 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6780021670 con 0x7f678814ada0 2022-07-21T01:57:48.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f678cc9e700 1 -- 172.21.15.140:0/841125743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678814a670 con 0x7f678814ada0 2022-07-21T01:57:48.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f6784ff9700 1 -- 172.21.15.140:0/841125743 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f678000b8a0 con 0x7f678814ada0 2022-07-21T01:57:48.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f678cc9e700 1 -- 172.21.15.140:0/841125743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678810ed50 con 0x7f678814ada0 2022-07-21T01:57:48.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f678cc9e700 1 -- 172.21.15.140:0/841125743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6788149d80 con 0x7f678814ada0 2022-07-21T01:57:48.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f6784ff9700 1 -- 172.21.15.140:0/841125743 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f678000f070 con 0x7f678814ada0 2022-07-21T01:57:48.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f6784ff9700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67680539e0 0x7f6768055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f6784ff9700 1 -- 172.21.15.140:0/841125743 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f678006cfd0 con 0x7f678814ada0 2022-07-21T01:57:48.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f67877fe700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67680539e0 0x7f6768055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.543+0000 7f67877fe700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67680539e0 0x7f6768055ea0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f677800d000 tx=0x7f6778007b50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.547+0000 7f6784ff9700 1 -- 172.21.15.140:0/841125743 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6788149d80 con 0x7f678814ada0 2022-07-21T01:57:48.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 -- 172.21.15.140:0/841125743 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6788149d80 con 0x7f678814ada0 2022-07-21T01:57:48.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f6784ff9700 1 -- 172.21.15.140:0/841125743 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v142) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6788149d80 con 0x7f678814ada0 2022-07-21T01:57:48.677 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:48.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 -- 172.21.15.140:0/841125743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67680539e0 msgr2=0x7f6768055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67680539e0 0x7f6768055ea0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f677800d000 tx=0x7f6778007b50).stop 2022-07-21T01:57:48.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 -- 172.21.15.140:0/841125743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f678814ada0 msgr2=0x7f6788159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f678814ada0 0x7f6788159af0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f67800032f0 tx=0x7f6780016000).stop 2022-07-21T01:57:48.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 -- 172.21.15.140:0/841125743 shutdown_connections 2022-07-21T01:57:48.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f67680539e0 0x7f6768055ea0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f678814ada0 0x7f6788159af0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f67880b59d0 0x7f6788148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 --2- 172.21.15.140:0/841125743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f67880b14f0 0x7f67880b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 -- 172.21.15.140:0/841125743 >> 172.21.15.140:0/841125743 conn(0x7f67880634c0 msgr2=0x7f67880b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:48.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 -- 172.21.15.140:0/841125743 shutdown_connections 2022-07-21T01:57:48.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.675+0000 7f678cc9e700 1 -- 172.21.15.140:0/841125743 wait complete. 2022-07-21T01:57:48.690 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:57:48.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.799+0000 7f23c2051700 1 Processor -- start 2022-07-21T01:57:48.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.799+0000 7f23c2051700 1 -- start start 2022-07-21T01:57:48.805 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.799+0000 7f23c2051700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f23bc0b1080 0x7f23bc0b1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.799+0000 7f23c2051700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23bc13f970 con 0x7f23bc0ae720 2022-07-21T01:57:48.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.799+0000 7f23c2051700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23bc13fbe0 con 0x7f23bc13e410 2022-07-21T01:57:48.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.799+0000 7f23c2051700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23bc13fe80 con 0x7f23bc0b1080 2022-07-21T01:57:48.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c1850700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f23bc0ae720 0x7f23bc0aeb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55476/0 (socket says 172.21.15.140:55476) 2022-07-21T01:57:48.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c1850700 1 -- 172.21.15.140:0/3568726940 learned_addr learned my addr 172.21.15.140:0/3568726940 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:48.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c104f700 1 --2- 172.21.15.140:0/3568726940 >> v2:172.21.15.152:3300/0 conn(0x7f23bc0b1080 0x7f23bc0b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23bbfff700 1 -- 172.21.15.140:0/3568726940 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762867918 0 0) 0x7f23bc13f970 con 0x7f23bc0ae720 2022-07-21T01:57:48.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23bbfff700 1 -- 172.21.15.140:0/3568726940 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a8003260 con 0x7f23bc0ae720 2022-07-21T01:57:48.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c104f700 1 -- 172.21.15.140:0/3568726940 >> v1:172.21.15.152:6789/0 conn(0x7f23bc13e410 legacy=0x7f23bc13e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23bbfff700 1 -- 172.21.15.140:0/3568726940 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168615236 0 0) 0x7f23bc13fbe0 con 0x7f23bc13e410 2022-07-21T01:57:48.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c104f700 1 -- 172.21.15.140:0/3568726940 >> v1:172.21.15.140:6789/0 conn(0x7f23bc0ae720 legacy=0x7f23bc0aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c104f700 1 -- 172.21.15.140:0/3568726940 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc0af240 con 0x7f23bc0b1080 2022-07-21T01:57:48.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c104f700 1 --2- 172.21.15.140:0/3568726940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23bc0b1080 0x7f23bc0b1460 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f23ac002f70 tx=0x7f23ac006df0).ready entity=mon.1 client_cookie=3bd1143247c66ec1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23bbfff700 1 -- 172.21.15.140:0/3568726940 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23ac00d610 con 0x7f23bc0b1080 2022-07-21T01:57:48.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23bbfff700 1 -- 172.21.15.140:0/3568726940 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac000e90 con 0x7f23bc0b1080 2022-07-21T01:57:48.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23bbfff700 1 -- 172.21.15.140:0/3568726940 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23ac00d960 con 0x7f23bc0b1080 2022-07-21T01:57:48.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c2051700 1 -- 172.21.15.140:0/3568726940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23bc0b1080 msgr2=0x7f23bc0b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c2051700 1 --2- 172.21.15.140:0/3568726940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23bc0b1080 0x7f23bc0b1460 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f23ac002f70 tx=0x7f23ac006df0).stop 2022-07-21T01:57:48.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c2051700 1 -- 172.21.15.140:0/3568726940 shutdown_connections 2022-07-21T01:57:48.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c2051700 1 --2- 172.21.15.140:0/3568726940 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23bc0b1080 0x7f23bc0b1460 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c2051700 1 -- 172.21.15.140:0/3568726940 >> 172.21.15.140:0/3568726940 conn(0x7f23bc063480 msgr2=0x7f23bc0b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:48.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c2051700 1 -- 172.21.15.140:0/3568726940 shutdown_connections 2022-07-21T01:57:48.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.803+0000 7f23c2051700 1 -- 172.21.15.140:0/3568726940 wait complete. 2022-07-21T01:57:48.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c2051700 1 Processor -- start 2022-07-21T01:57:48.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c2051700 1 -- start start 2022-07-21T01:57:48.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c2051700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23bc0ae720 0x7f23bc063ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c2051700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23bc0b1080 0x7f23bc065a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c2051700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23bc13e410 0x7f23bc14ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c2051700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc067a80 con 0x7f23bc13e410 2022-07-21T01:57:48.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c2051700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0b42c0 con 0x7f23bc0b1080 2022-07-21T01:57:48.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c2051700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0652e0 con 0x7f23bc0ae720 2022-07-21T01:57:48.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c1850700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23bc13e410 0x7f23bc14ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c1850700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23bc13e410 0x7f23bc14ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41336/0 (socket says 172.21.15.140:41336) 2022-07-21T01:57:48.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c1850700 1 -- 172.21.15.140:0/2796826279 learned_addr learned my addr 172.21.15.140:0/2796826279 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:48.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c104f700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23bc0ae720 0x7f23bc063ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c084e700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23bc0b1080 0x7f23bc065a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c1850700 1 -- 172.21.15.140:0/2796826279 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23bc0ae720 msgr2=0x7f23bc063ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c1850700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23bc0ae720 0x7f23bc063ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c1850700 1 -- 172.21.15.140:0/2796826279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23bc0b1080 msgr2=0x7f23bc065a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c1850700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23bc0b1080 0x7f23bc065a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c1850700 1 -- 172.21.15.140:0/2796826279 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac00b180 con 0x7f23bc13e410 2022-07-21T01:57:48.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c084e700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23bc0b1080 0x7f23bc065a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:57:48.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c104f700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23bc0ae720 0x7f23bc063ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:57:48.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c1850700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23bc13e410 0x7f23bc14ea50 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f23b4001490 tx=0x7f23b4007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23ba7fc700 1 -- 172.21.15.140:0/2796826279 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23b4005d90 con 0x7f23bc13e410 2022-07-21T01:57:48.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23ba7fc700 1 -- 172.21.15.140:0/2796826279 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b400c820 con 0x7f23bc13e410 2022-07-21T01:57:48.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c2051700 1 -- 172.21.15.140:0/2796826279 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc065580 con 0x7f23bc13e410 2022-07-21T01:57:48.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23ba7fc700 1 -- 172.21.15.140:0/2796826279 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23b400cb80 con 0x7f23bc13e410 2022-07-21T01:57:48.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23c2051700 1 -- 172.21.15.140:0/2796826279 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc065f10 con 0x7f23bc13e410 2022-07-21T01:57:48.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.807+0000 7f23ba7fc700 1 -- 172.21.15.140:0/2796826279 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f23b400cce0 con 0x7f23bc13e410 2022-07-21T01:57:48.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.811+0000 7f23c2051700 1 -- 172.21.15.140:0/2796826279 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23bc0b40e0 con 0x7f23bc13e410 2022-07-21T01:57:48.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.811+0000 7f23ba7fc700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f239c0539e0 0x7f239c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:48.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.811+0000 7f23c104f700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f239c0539e0 0x7f239c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:48.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.811+0000 7f23ba7fc700 1 -- 172.21.15.140:0/2796826279 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f23b406e8e0 con 0x7f23bc13e410 2022-07-21T01:57:48.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.811+0000 7f23c104f700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f239c0539e0 0x7f239c055ea0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f23ac015000 tx=0x7f23ac014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:48.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.815+0000 7f23ba7fc700 1 -- 172.21.15.140:0/2796826279 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f23b4045800 con 0x7f23bc13e410 2022-07-21T01:57:48.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.939+0000 7f23c2051700 1 -- 172.21.15.140:0/2796826279 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23bc05bd90 con 0x7f23bc13e410 2022-07-21T01:57:48.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.939+0000 7f23ba7fc700 1 -- 172.21.15.140:0/2796826279 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v142) v1 ==== 104+0+10 (secure 0 0 0) 0x7f23b4048e20 con 0x7f23bc13e410 2022-07-21T01:57:48.943 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:48.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 -- 172.21.15.140:0/2796826279 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f239c0539e0 msgr2=0x7f239c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f239c0539e0 0x7f239c055ea0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f23ac015000 tx=0x7f23ac014000).stop 2022-07-21T01:57:48.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 -- 172.21.15.140:0/2796826279 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23bc13e410 msgr2=0x7f23bc14ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:48.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23bc13e410 0x7f23bc14ea50 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f23b4001490 tx=0x7f23b4007e50).stop 2022-07-21T01:57:48.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 -- 172.21.15.140:0/2796826279 shutdown_connections 2022-07-21T01:57:48.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f239c0539e0 0x7f239c055ea0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f23bc13e410 0x7f23bc14ea50 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23bc0b1080 0x7f23bc065a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 --2- 172.21.15.140:0/2796826279 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23bc0ae720 0x7f23bc063ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:48.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 -- 172.21.15.140:0/2796826279 >> 172.21.15.140:0/2796826279 conn(0x7f23bc063480 msgr2=0x7f23bc1415e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:48.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 -- 172.21.15.140:0/2796826279 shutdown_connections 2022-07-21T01:57:48.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:48.943+0000 7f23c2051700 1 -- 172.21.15.140:0/2796826279 wait complete. 2022-07-21T01:57:48.961 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:57:49.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f548752a700 1 Processor -- start 2022-07-21T01:57:49.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f548752a700 1 -- start start 2022-07-21T01:57:49.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f548752a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f54800b5960 0x7f5480142b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f548752a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5480143060 0x7f548009b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f548752a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5480147030 con 0x7f548005d0e0 2022-07-21T01:57:49.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f548752a700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548005d4c0 con 0x7f5480143060 2022-07-21T01:57:49.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f548752a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54801461e0 con 0x7f54800b5960 2022-07-21T01:57:49.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f5485ac7700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5480143060 0x7f548009b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f5485ac7700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5480143060 0x7f548009b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41344/0 (socket says 172.21.15.140:41344) 2022-07-21T01:57:49.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f5485ac7700 1 -- 172.21.15.140:0/2553456923 learned_addr learned my addr 172.21.15.140:0/2553456923 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:49.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f5484ac5700 1 --2- 172.21.15.140:0/2553456923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f54800b5960 0x7f5480142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f5485ac7700 1 -- 172.21.15.140:0/2553456923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f54800b5960 msgr2=0x7f5480142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:49.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f5485ac7700 1 --2- 172.21.15.140:0/2553456923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f54800b5960 0x7f5480142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f5485ac7700 1 -- 172.21.15.140:0/2553456923 >> v1:172.21.15.152:6789/0 conn(0x7f548005d0e0 legacy=0x7f5480146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f5485ac7700 1 -- 172.21.15.140:0/2553456923 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480146480 con 0x7f5480143060 2022-07-21T01:57:49.075 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.067+0000 7f5485ac7700 1 --2- 172.21.15.140:0/2553456923 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5480143060 0x7f548009b6d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f5478002f70 tx=0x7f5478006eb0).ready entity=mon.0 client_cookie=296af6af3527895e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:49.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f5477fff700 1 -- 172.21.15.140:0/2553456923 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f547800d570 con 0x7f5480143060 2022-07-21T01:57:49.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f5477fff700 1 -- 172.21.15.140:0/2553456923 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5478000e90 con 0x7f5480143060 2022-07-21T01:57:49.076 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f5477fff700 1 -- 172.21.15.140:0/2553456923 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f547800d900 con 0x7f5480143060 2022-07-21T01:57:49.079 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f548752a700 1 -- 172.21.15.140:0/2553456923 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5480143060 msgr2=0x7f548009b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f548752a700 1 --2- 172.21.15.140:0/2553456923 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5480143060 0x7f548009b6d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f5478002f70 tx=0x7f5478006eb0).stop 2022-07-21T01:57:49.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f548752a700 1 -- 172.21.15.140:0/2553456923 shutdown_connections 2022-07-21T01:57:49.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f548752a700 1 --2- 172.21.15.140:0/2553456923 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5480143060 0x7f548009b6d0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.080 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f548752a700 1 --2- 172.21.15.140:0/2553456923 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f54800b5960 0x7f5480142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f548752a700 1 -- 172.21.15.140:0/2553456923 >> 172.21.15.140:0/2553456923 conn(0x7f54800b6a20 msgr2=0x7f5480066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:49.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f548752a700 1 -- 172.21.15.140:0/2553456923 shutdown_connections 2022-07-21T01:57:49.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f548752a700 1 -- 172.21.15.140:0/2553456923 wait complete. 2022-07-21T01:57:49.081 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f548752a700 1 Processor -- start 2022-07-21T01:57:49.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.071+0000 7f548752a700 1 -- start start 2022-07-21T01:57:49.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f548752a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f548005d0e0 0x7f5480155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.082 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f548752a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54800b5960 0x7f54800b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f548752a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5480143060 0x7f548008f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f548752a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548009dea0 con 0x7f5480143060 2022-07-21T01:57:49.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f548752a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480153040 con 0x7f54800b5960 2022-07-21T01:57:49.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f548752a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5480155dd0 con 0x7f548005d0e0 2022-07-21T01:57:49.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f5485ac7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5480143060 0x7f548008f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f5485ac7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5480143060 0x7f548008f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41350/0 (socket says 172.21.15.140:41350) 2022-07-21T01:57:49.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f5485ac7700 1 -- 172.21.15.140:0/1461305327 learned_addr learned my addr 172.21.15.140:0/1461305327 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:49.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f54852c6700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f548005d0e0 0x7f5480155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f5484ac5700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54800b5960 0x7f54800b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f5485ac7700 1 -- 172.21.15.140:0/1461305327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f548005d0e0 msgr2=0x7f5480155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f5485ac7700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f548005d0e0 0x7f5480155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f5485ac7700 1 -- 172.21.15.140:0/1461305327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54800b5960 msgr2=0x7f54800b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f5485ac7700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54800b5960 0x7f54800b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f5485ac7700 1 -- 172.21.15.140:0/1461305327 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547800b1e0 con 0x7f5480143060 2022-07-21T01:57:49.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f54852c6700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f548005d0e0 0x7f5480155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:49.086 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f5485ac7700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5480143060 0x7f548008f610 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f5478002f70 tx=0x7f5478015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:49.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f54767fc700 1 -- 172.21.15.140:0/1461305327 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54780116e0 con 0x7f5480143060 2022-07-21T01:57:49.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f54767fc700 1 -- 172.21.15.140:0/1461305327 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5478011840 con 0x7f5480143060 2022-07-21T01:57:49.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f548752a700 1 -- 172.21.15.140:0/1461305327 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54800b3400 con 0x7f5480143060 2022-07-21T01:57:49.087 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f548752a700 1 -- 172.21.15.140:0/1461305327 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54800b3160 con 0x7f5480143060 2022-07-21T01:57:49.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f54767fc700 1 -- 172.21.15.140:0/1461305327 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5478011be0 con 0x7f5480143060 2022-07-21T01:57:49.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.075+0000 7f548752a700 1 -- 172.21.15.140:0/1461305327 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5480155f90 con 0x7f5480143060 2022-07-21T01:57:49.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.079+0000 7f54767fc700 1 -- 172.21.15.140:0/1461305327 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5478025030 con 0x7f5480143060 2022-07-21T01:57:49.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.079+0000 7f54767fc700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5468053a70 0x7f5468055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.079+0000 7f54767fc700 1 -- 172.21.15.140:0/1461305327 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f547806d050 con 0x7f5480143060 2022-07-21T01:57:49.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.079+0000 7f54852c6700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5468053a70 0x7f5468055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.079+0000 7f54852c6700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5468053a70 0x7f5468055f30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f547c00d000 tx=0x7f547c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:49.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.083+0000 7f54767fc700 1 -- 172.21.15.140:0/1461305327 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5478047e60 con 0x7f5480143060 2022-07-21T01:57:49.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 -- 172.21.15.140:0/1461305327 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54800b01d0 con 0x7f5480143060 2022-07-21T01:57:49.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f54767fc700 1 -- 172.21.15.140:0/1461305327 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v142) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5478023020 con 0x7f5480143060 2022-07-21T01:57:49.205 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:57:49.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 -- 172.21.15.140:0/1461305327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5468053a70 msgr2=0x7f5468055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5468053a70 0x7f5468055f30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f547c00d000 tx=0x7f547c009040).stop 2022-07-21T01:57:49.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 -- 172.21.15.140:0/1461305327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5480143060 msgr2=0x7f548008f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5480143060 0x7f548008f610 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f5478002f70 tx=0x7f5478015000).stop 2022-07-21T01:57:49.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 -- 172.21.15.140:0/1461305327 shutdown_connections 2022-07-21T01:57:49.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5468053a70 0x7f5468055f30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5480143060 0x7f548008f610 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f54800b5960 0x7f54800b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 --2- 172.21.15.140:0/1461305327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f548005d0e0 0x7f5480155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 -- 172.21.15.140:0/1461305327 >> 172.21.15.140:0/1461305327 conn(0x7f54800b6a20 msgr2=0x7f54800b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:49.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 -- 172.21.15.140:0/1461305327 shutdown_connections 2022-07-21T01:57:49.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.203+0000 7f548752a700 1 -- 172.21.15.140:0/1461305327 wait complete. 2022-07-21T01:57:49.218 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:57:49.219 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:49.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.315+0000 7f65ab59e700 1 Processor -- start 2022-07-21T01:57:49.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.315+0000 7f65ab59e700 1 -- start start 2022-07-21T01:57:49.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.315+0000 7f65ab59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f65ac13c790 0x7f65ac0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.315+0000 7f65ab59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f65ac0b3ca0 0x7f65ac0b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.315+0000 7f65ab59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ac142ba0 con 0x7f65ac0b9f00 2022-07-21T01:57:49.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.315+0000 7f65ab59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ac1429b0 con 0x7f65ac13c790 2022-07-21T01:57:49.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.315+0000 7f65ab59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ac146330 con 0x7f65ac0b3ca0 2022-07-21T01:57:49.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.315+0000 7f65aa59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f65ac13c790 0x7f65ac0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.315+0000 7f65aad9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f65ac0b9f00 0x7f65ac143000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55504/0 (socket says 172.21.15.140:55504) 2022-07-21T01:57:49.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.315+0000 7f65aad9d700 1 -- 172.21.15.140:0/4270264523 learned_addr learned my addr 172.21.15.140:0/4270264523 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:49.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65a9d9b700 1 --2- 172.21.15.140:0/4270264523 >> v2:172.21.15.152:3300/0 conn(0x7f65ac0b3ca0 0x7f65ac0b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65aa59c700 1 -- 172.21.15.140:0/4270264523 >> v2:172.21.15.152:3300/0 conn(0x7f65ac0b3ca0 msgr2=0x7f65ac0b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:49.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65aa59c700 1 --2- 172.21.15.140:0/4270264523 >> v2:172.21.15.152:3300/0 conn(0x7f65ac0b3ca0 0x7f65ac0b9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65aa59c700 1 -- 172.21.15.140:0/4270264523 >> v1:172.21.15.140:6789/0 conn(0x7f65ac0b9f00 legacy=0x7f65ac143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65aa59c700 1 -- 172.21.15.140:0/4270264523 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac0b4c00 con 0x7f65ac13c790 2022-07-21T01:57:49.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65a959a700 1 -- 172.21.15.140:0/4270264523 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647628260 0 0) 0x7f65ac142ba0 con 0x7f65ac0b9f00 2022-07-21T01:57:49.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65aa59c700 1 --2- 172.21.15.140:0/4270264523 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f65ac13c790 0x7f65ac0b4710 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f65ac146fe0 tx=0x7f659c006df0).ready entity=mon.0 client_cookie=7d1d652a357059 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:49.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65a959a700 1 -- 172.21.15.140:0/4270264523 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f659c00d5c0 con 0x7f65ac13c790 2022-07-21T01:57:49.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65a959a700 1 -- 172.21.15.140:0/4270264523 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c000e90 con 0x7f65ac13c790 2022-07-21T01:57:49.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65a959a700 1 -- 172.21.15.140:0/4270264523 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f659c00d910 con 0x7f65ac13c790 2022-07-21T01:57:49.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65ab59e700 1 -- 172.21.15.140:0/4270264523 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f65ac13c790 msgr2=0x7f65ac0b4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65ab59e700 1 --2- 172.21.15.140:0/4270264523 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f65ac13c790 0x7f65ac0b4710 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f65ac146fe0 tx=0x7f659c006df0).stop 2022-07-21T01:57:49.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65ab59e700 1 -- 172.21.15.140:0/4270264523 shutdown_connections 2022-07-21T01:57:49.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65ab59e700 1 --2- 172.21.15.140:0/4270264523 >> v2:172.21.15.152:3300/0 conn(0x7f65ac0b3ca0 0x7f65ac0b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65ab59e700 1 --2- 172.21.15.140:0/4270264523 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f65ac13c790 0x7f65ac0b4710 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65ab59e700 1 -- 172.21.15.140:0/4270264523 >> 172.21.15.140:0/4270264523 conn(0x7f65ac0678b0 msgr2=0x7f65ac0b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:49.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65ab59e700 1 -- 172.21.15.140:0/4270264523 shutdown_connections 2022-07-21T01:57:49.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.319+0000 7f65ab59e700 1 -- 172.21.15.140:0/4270264523 wait complete. 2022-07-21T01:57:49.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65ab59e700 1 Processor -- start 2022-07-21T01:57:49.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65ab59e700 1 -- start start 2022-07-21T01:57:49.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65ab59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65ac0b3ca0 0x7f65ac0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65ab59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ac0b9f00 0x7f65ac0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65ab59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f65ac13c790 0x7f65ac0b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65ab59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0af970 con 0x7f65ac13c790 2022-07-21T01:57:49.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65ab59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0af6d0 con 0x7f65ac0b9f00 2022-07-21T01:57:49.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65ab59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0b1cb0 con 0x7f65ac0b3ca0 2022-07-21T01:57:49.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65aad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f65ac13c790 0x7f65ac0b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65aad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f65ac13c790 0x7f65ac0b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41364/0 (socket says 172.21.15.140:41364) 2022-07-21T01:57:49.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65aad9d700 1 -- 172.21.15.140:0/1057891057 learned_addr learned my addr 172.21.15.140:0/1057891057 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:49.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65a9d9b700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ac0b9f00 0x7f65ac0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65aa59c700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65ac0b3ca0 0x7f65ac0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65aad9d700 1 -- 172.21.15.140:0/1057891057 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65ac0b3ca0 msgr2=0x7f65ac0ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65aad9d700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65ac0b3ca0 0x7f65ac0ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65aad9d700 1 -- 172.21.15.140:0/1057891057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ac0b9f00 msgr2=0x7f65ac0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65aad9d700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ac0b9f00 0x7f65ac0b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65aad9d700 1 -- 172.21.15.140:0/1057891057 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c00b180 con 0x7f65ac13c790 2022-07-21T01:57:49.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65aad9d700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f65ac13c790 0x7f65ac0b2640 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f65a4000bc0 tx=0x7f65a40079e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:49.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65a37fe700 1 -- 172.21.15.140:0/1057891057 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65a40070d0 con 0x7f65ac13c790 2022-07-21T01:57:49.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65a37fe700 1 -- 172.21.15.140:0/1057891057 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a4007230 con 0x7f65ac13c790 2022-07-21T01:57:49.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65ab59e700 1 -- 172.21.15.140:0/1057891057 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ac0b06c0 con 0x7f65ac13c790 2022-07-21T01:57:49.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65a37fe700 1 -- 172.21.15.140:0/1057891057 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65a40055f0 con 0x7f65ac13c790 2022-07-21T01:57:49.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.323+0000 7f65ab59e700 1 -- 172.21.15.140:0/1057891057 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ac0af450 con 0x7f65ac13c790 2022-07-21T01:57:49.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.327+0000 7f65ab59e700 1 -- 172.21.15.140:0/1057891057 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ac0b2ef0 con 0x7f65ac13c790 2022-07-21T01:57:49.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.327+0000 7f65a37fe700 1 -- 172.21.15.140:0/1057891057 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f65a401a060 con 0x7f65ac13c790 2022-07-21T01:57:49.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.327+0000 7f65a37fe700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65840539e0 0x7f6584055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:49.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.327+0000 7f65a37fe700 1 -- 172.21.15.140:0/1057891057 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f65a406b610 con 0x7f65ac13c790 2022-07-21T01:57:49.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.327+0000 7f65aa59c700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65840539e0 0x7f6584055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:49.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.327+0000 7f65aa59c700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65840539e0 0x7f6584055ea0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f659c018000 tx=0x7f659c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:49.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.331+0000 7f65a37fe700 1 -- 172.21.15.140:0/1057891057 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f65ac0b2ef0 con 0x7f65ac13c790 2022-07-21T01:57:49.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.495+0000 7f65ab59e700 1 -- 172.21.15.140:0/1057891057 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65ac0b2ef0 con 0x7f65ac13c790 2022-07-21T01:57:49.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.495+0000 7f65a37fe700 1 -- 172.21.15.140:0/1057891057 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f65ac0b2ef0 con 0x7f65ac13c790 2022-07-21T01:57:49.498 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:49.499 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":69,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/148832193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":148832193},{"type":"v1","addr":"172.21.15.152:6833","nonce":148832193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:41.192528+0000","modified":"2022-07-21T01:57:42.195096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8308":{"gid":8308,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/961657495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":961657495},{"type":"v1","addr":"172.21.15.140:6835","nonce":961657495}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-07-21T01:57:49.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.495+0000 7f65ab59e700 1 -- 172.21.15.140:0/1057891057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65840539e0 msgr2=0x7f6584055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.495+0000 7f65ab59e700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65840539e0 0x7f6584055ea0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f659c018000 tx=0x7f659c014000).stop 2022-07-21T01:57:49.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.495+0000 7f65ab59e700 1 -- 172.21.15.140:0/1057891057 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f65ac13c790 msgr2=0x7f65ac0b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:49.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.495+0000 7f65ab59e700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f65ac13c790 0x7f65ac0b2640 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f65a4000bc0 tx=0x7f65a40079e0).stop 2022-07-21T01:57:49.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.499+0000 7f65ab59e700 1 -- 172.21.15.140:0/1057891057 shutdown_connections 2022-07-21T01:57:49.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.499+0000 7f65ab59e700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f65840539e0 0x7f6584055ea0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.499+0000 7f65ab59e700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f65ac13c790 0x7f65ac0b2640 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.499+0000 7f65ab59e700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ac0b9f00 0x7f65ac0b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.499+0000 7f65ab59e700 1 --2- 172.21.15.140:0/1057891057 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f65ac0b3ca0 0x7f65ac0ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:49.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.499+0000 7f65ab59e700 1 -- 172.21.15.140:0/1057891057 >> 172.21.15.140:0/1057891057 conn(0x7f65ac0678b0 msgr2=0x7f65ac144db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:49.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.499+0000 7f65ab59e700 1 -- 172.21.15.140:0/1057891057 shutdown_connections 2022-07-21T01:57:49.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:49.499+0000 7f65ab59e700 1 -- 172.21.15.140:0/1057891057 wait complete. 2022-07-21T01:57:49.504 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 69 2022-07-21T01:57:49.713 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:57:41.192528+0000', 'modified': '2022-07-21T01:57:42.195096+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8308}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8308': {'gid': 8308, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/961657495', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 961657495}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 961657495}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:57:49.714 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:57:49.715 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:57:49.831 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":8308,"want_state":"up:active","state":"up:active","rank_uptime":8.6249512500000005,"mdsmap_epoch":69,"osdmap_epoch":142,"osdmap_epoch_barrier":141,"uptime":12.989163019999999} 2022-07-21T01:57:49.831 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 8308, "mdsmap_epoch": 69, "osdmap_epoch": 142, "osdmap_epoch_barrier": 141, "rank_uptime": 8.62495125, "state": "up:active", "uptime": 12.98916302, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:57:49.832 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:57:49.832 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-07-21T01:57:49.833 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-07-21T01:57:49.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:49.834 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:49.843 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:57:49.860 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:49.861 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:49.871 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:49.872 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:49.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:49.875 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:57:49.923 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:57:49.923 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:49.926 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:57:49.941 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:49.941 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:49.956 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:49.959 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:49.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:49.961 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:57:50.000 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2022-07-21T01:57:49.996+0000 7fb9d7b91080 -1 init, newargv = 0x55660c733350 newargc=9ceph-fuse[22998 2022-07-21T01:57:50.001 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:]: starting ceph client 2022-07-21T01:57:50.008 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22998]: starting fuse 2022-07-21T01:57:51.108 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:57:51.127 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:57:51.128 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:57:51.184 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:57:51.184 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:57:51.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:51.186 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:57:51.193 INFO:teuthology.orchestra.run.smithi192.stdout:52 2022-07-21T01:57:51.194 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:57:51.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:51.195 DEBUG:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2022-07-21T01:57:51.196 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2022-07-21T01:57:51.197 DEBUG:teuthology.orchestra.run.smithi192:> import re 2022-07-21T01:57:51.197 DEBUG:teuthology.orchestra.run.smithi192:> import os 2022-07-21T01:57:51.198 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2022-07-21T01:57:51.199 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:51.199 DEBUG:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2022-07-21T01:57:51.200 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-07-21T01:57:51.200 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2022-07-21T01:57:51.200 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:51.201 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2022-07-21T01:57:51.201 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2022-07-21T01:57:51.202 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2022-07-21T01:57:51.202 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2022-07-21T01:57:51.203 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:51.203 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2022-07-21T01:57:51.203 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:57:51.204 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:57:51.204 DEBUG:teuthology.orchestra.run.smithi192:> return f 2022-07-21T01:57:51.205 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:57:51.206 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:57:51.206 DEBUG:teuthology.orchestra.run.smithi192:> print(find_socket("client.1")) 2022-07-21T01:57:51.207 DEBUG:teuthology.orchestra.run.smithi192:> ' 2022-07-21T01:57:51.251 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.22998.asok 2022-07-21T01:57:51.256 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.22998.asok 2022-07-21T01:57:51.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:51.257 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.22998.asok status 2022-07-21T01:57:51.345 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2022-07-21T01:57:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2022-07-21T01:57:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:57:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:57:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2022-07-21T01:57:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2022-07-21T01:57:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-07-21T01:57:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "22998", 2022-07-21T01:57:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2022-07-21T01:57:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2022-07-21T01:57:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2022-07-21T01:57:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 7430, 2022-07-21T01:57:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2022-07-21T01:57:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2022-07-21T01:57:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2022-07-21T01:57:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 7430 2022-07-21T01:57:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:57:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:57:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:57:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3888981513 2022-07-21T01:57:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: } 2022-07-21T01:57:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:57:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:57:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:57:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3888981513 2022-07-21T01:57:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:57:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.7430 172.21.15.192:0/3888981513", 2022-07-21T01:57:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/3888981513", 2022-07-21T01:57:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2022-07-21T01:57:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 69, 2022-07-21T01:57:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 142, 2022-07-21T01:57:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:57:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false, 2022-07-21T01:57:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2022-07-21T01:57:51.359 INFO:teuthology.orchestra.run.smithi192.stdout:} 2022-07-21T01:57:51.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:51.361 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:51.362 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:51.369 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:57:51.370 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:57:51.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:51.371 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:51.387 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-07-21T01:57:51.388 DEBUG:teuthology.orchestra.run.smithi191:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.rctime . 2022-07-21T01:57:51.433 INFO:teuthology.orchestra.run.smithi191.stdout:1658368671.385706741 2022-07-21T01:57:51.434 INFO:tasks.cephfs.test_misc:rctime = 1658368671.385706741 2022-07-21T01:57:51.434 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:57:51.435 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:51.458 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:51.484 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:51.508 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:51.542 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:51.585 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:57:51.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:51.587 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:51.588 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:51.598 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:57:51.599 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:57:51.600 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi191.front.sepia.ceph.com... 2022-07-21T01:57:51.600 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:57:51.600 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:51.618 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23438]: fuse finished with error 0 and tester_r 0 2022-07-21T01:57:51.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:51.620 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:51.620 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:51.682 INFO:teuthology.orchestra.run.smithi191.stdout:ext2/ext3 2022-07-21T01:57:51.688 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:57:51.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:57:51.701 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:51.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:51.763 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:51.764 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:51.776 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:57:51.776 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:57:51.777 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2022-07-21T01:57:51.777 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:57:51.778 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:51.796 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[22998]: fuse finished with error 0 and tester_r 0 2022-07-21T01:57:51.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:51.800 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:51.800 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:51.847 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2022-07-21T01:57:51.849 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:57:51.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:57:51.967 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:51.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:51.978 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:51.979 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:57:51.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:51.993 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:57:51.993 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:57:51.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:57:51.995 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:57:52.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:52.006 INFO:teuthology.orchestra.run.smithi195.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:57:52.006 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:57:52.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3b0513e700 1 Processor -- start 2022-07-21T01:57:52.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3b0513e700 1 -- start start 2022-07-21T01:57:52.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3b0513e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3b000b5650 0x7f3b000b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3b0513e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3b000adff0 0x7f3b00147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3b0513e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b000af8e0 0x7f3b0005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3b0513e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b00147010 con 0x7f3b000adff0 2022-07-21T01:57:52.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3b0513e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b001472b0 con 0x7f3b000b5650 2022-07-21T01:57:52.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3b0513e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00146d40 con 0x7f3b000af8e0 2022-07-21T01:57:52.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3aff7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3b000adff0 0x7f3b00147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3aff7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3b000adff0 0x7f3b00147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41368/0 (socket says 172.21.15.140:41368) 2022-07-21T01:57:52.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3aff7fe700 1 -- 172.21.15.140:0/512595971 learned_addr learned my addr 172.21.15.140:0/512595971 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3afffff700 1 --2- 172.21.15.140:0/512595971 >> v2:172.21.15.152:3300/0 conn(0x7f3b000b5650 0x7f3b000b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3b0493d700 1 --2- 172.21.15.140:0/512595971 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b000af8e0 0x7f3b0005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3aff7fe700 1 -- 172.21.15.140:0/512595971 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b000af8e0 msgr2=0x7f3b0005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3aff7fe700 1 --2- 172.21.15.140:0/512595971 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b000af8e0 0x7f3b0005bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3aff7fe700 1 -- 172.21.15.140:0/512595971 >> v2:172.21.15.152:3300/0 conn(0x7f3b000b5650 msgr2=0x7f3b000b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3aff7fe700 1 --2- 172.21.15.140:0/512595971 >> v2:172.21.15.152:3300/0 conn(0x7f3b000b5650 0x7f3b000b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3aff7fe700 1 -- 172.21.15.140:0/512595971 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b00147c40 con 0x7f3b000adff0 2022-07-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3afffff700 1 --2- 172.21.15.140:0/512595971 >> v2:172.21.15.152:3300/0 conn(0x7f3b000b5650 0x7f3b000b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3b0493d700 1 --2- 172.21.15.140:0/512595971 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b000af8e0 0x7f3b0005bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.115+0000 7f3aff7fe700 1 --2- 172.21.15.140:0/512595971 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b000adff0 0x7f3b00147750 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f3af0002f70 tx=0x7f3af0006e50).ready entity=mon.0 client_cookie=94fa36541a2314a3 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:52.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3afeffd700 1 -- 172.21.15.140:0/512595971 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af000d570 con 0x7f3b000adff0 2022-07-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3afeffd700 1 -- 172.21.15.140:0/512595971 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af0000e90 con 0x7f3b000adff0 2022-07-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3afeffd700 1 -- 172.21.15.140:0/512595971 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af000d900 con 0x7f3b000adff0 2022-07-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 -- 172.21.15.140:0/512595971 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b000adff0 msgr2=0x7f3b00147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 --2- 172.21.15.140:0/512595971 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b000adff0 0x7f3b00147750 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7f3af0002f70 tx=0x7f3af0006e50).stop 2022-07-21T01:57:52.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 -- 172.21.15.140:0/512595971 shutdown_connections 2022-07-21T01:57:52.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 --2- 172.21.15.140:0/512595971 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b000af8e0 0x7f3b0005bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 --2- 172.21.15.140:0/512595971 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b000adff0 0x7f3b00147750 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 --2- 172.21.15.140:0/512595971 >> v2:172.21.15.152:3300/0 conn(0x7f3b000b5650 0x7f3b000b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 -- 172.21.15.140:0/512595971 >> 172.21.15.140:0/512595971 conn(0x7f3b000634e0 msgr2=0x7f3b000b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:52.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 -- 172.21.15.140:0/512595971 shutdown_connections 2022-07-21T01:57:52.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 -- 172.21.15.140:0/512595971 wait complete. 2022-07-21T01:57:52.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 Processor -- start 2022-07-21T01:57:52.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 -- start start 2022-07-21T01:57:52.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b000adff0 0x7f3b00066d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b000af8e0 0x7f3b001433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b000b5650 0x7f3b00111190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00064480 con 0x7f3b000b5650 2022-07-21T01:57:52.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000677f0 con 0x7f3b000af8e0 2022-07-21T01:57:52.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0513e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b00110cf0 con 0x7f3b000adff0 2022-07-21T01:57:52.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0493d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b000b5650 0x7f3b00111190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3afffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b000adff0 0x7f3b00066d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0493d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b000b5650 0x7f3b00111190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41378/0 (socket says 172.21.15.140:41378) 2022-07-21T01:57:52.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3aff7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b000af8e0 0x7f3b001433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.119+0000 7f3b0493d700 1 -- 172.21.15.140:0/2817756821 learned_addr learned my addr 172.21.15.140:0/2817756821 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:52.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3b0493d700 1 -- 172.21.15.140:0/2817756821 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b000adff0 msgr2=0x7f3b00066d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3b0493d700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b000adff0 0x7f3b00066d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3b0493d700 1 -- 172.21.15.140:0/2817756821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b000af8e0 msgr2=0x7f3b001433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3b0493d700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b000af8e0 0x7f3b001433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3b0493d700 1 -- 172.21.15.140:0/2817756821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af000b1e0 con 0x7f3b000b5650 2022-07-21T01:57:52.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3b0493d700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b000b5650 0x7f3b00111190 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f3af8007000 tx=0x7f3af8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:52.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3afd7fa700 1 -- 172.21.15.140:0/2817756821 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af8010580 con 0x7f3b000b5650 2022-07-21T01:57:52.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3afd7fa700 1 -- 172.21.15.140:0/2817756821 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af80106e0 con 0x7f3b000b5650 2022-07-21T01:57:52.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3b0513e700 1 -- 172.21.15.140:0/2817756821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b000b4000 con 0x7f3b000b5650 2022-07-21T01:57:52.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3afd7fa700 1 -- 172.21.15.140:0/2817756821 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3af8010a40 con 0x7f3b000b5650 2022-07-21T01:57:52.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3b0513e700 1 -- 172.21.15.140:0/2817756821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b00111680 con 0x7f3b000b5650 2022-07-21T01:57:52.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3afd7fa700 1 -- 172.21.15.140:0/2817756821 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3af8010ba0 con 0x7f3b000b5650 2022-07-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3b0513e700 1 -- 172.21.15.140:0/2817756821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b00110b50 con 0x7f3b000b5650 2022-07-21T01:57:52.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3afd7fa700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053aa0 0x7f3ae0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3afd7fa700 1 -- 172.21.15.140:0/2817756821 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3af806c200 con 0x7f3b000b5650 2022-07-21T01:57:52.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3afffff700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053aa0 0x7f3ae0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.123+0000 7f3afffff700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053aa0 0x7f3ae0055f60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f3af400d000 tx=0x7f3af4008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:52.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.127+0000 7f3afd7fa700 1 -- 172.21.15.140:0/2817756821 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3b00110b50 con 0x7f3b000b5650 2022-07-21T01:57:52.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.255+0000 7f3b0513e700 1 -- 172.21.15.140:0/2817756821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3b00110b50 con 0x7f3b000b5650 2022-07-21T01:57:52.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3afd7fa700 1 -- 172.21.15.140:0/2817756821 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v142) v1 ==== 81+0+33755 (secure 0 0 0) 0x7f3b00110b50 con 0x7f3b000b5650 2022-07-21T01:57:52.261 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:52.262 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:57:52.262 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 142, 2022-07-21T01:57:52.263 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:57:52.263 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:57:52.264 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:57:42.195796+0000", 2022-07-21T01:57:52.264 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:52.264 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:52.264 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:57:52.265 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:57:52.265 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:57:52.265 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:57:52.265 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:57:52.266 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:57:52.266 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:57:52.266 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:52.266 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:57:52.267 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:57:52.267 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:57:52.267 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:57:52.267 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:57:52.268 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 28, 2022-07-21T01:57:52.268 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:57:52.268 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:57:52.268 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:57:52.269 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:57:52.269 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:57:52.269 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.269 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:57:52.269 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:57:52.270 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:57:52.270 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:52.270 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:52.270 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:52.271 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:52.271 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:52.271 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:52.271 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:52.272 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:52.272 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:57:52.272 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:57:52.272 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:57:52.273 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:57:52.273 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:57:52.273 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:52.273 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:52.274 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:52.274 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:52.274 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:52.274 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:52.275 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:52.275 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.275 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:57:52.275 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:52.276 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:52.276 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:52.276 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:52.276 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:52.276 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:52.277 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:52.277 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:52.277 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:52.277 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:52.278 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:52.278 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:52.278 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:52.278 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:52.278 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:52.279 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:52.279 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:52.279 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:52.279 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:52.280 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:52.280 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:52.280 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:52.280 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:52.280 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:52.281 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:52.281 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:52.281 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.281 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:52.282 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:52.282 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:52.282 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:52.282 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:52.282 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:52.283 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:52.283 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:52.283 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:52.283 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:52.283 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:52.284 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:57:52.284 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:57:52.284 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.285 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:52.285 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:57:52.285 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.285 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.285 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.286 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:57:52.286 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:57:52.286 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:57:52.286 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:52.287 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:52.287 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:52.287 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:52.287 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:52.288 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:52.288 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:52.288 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:52.288 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:52.289 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:52.289 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:52.289 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:52.289 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:52.290 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:52.290 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:52.290 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:52.291 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:52.291 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:52.291 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:52.291 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:52.292 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.292 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:57:52.292 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:52.292 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:52.292 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:52.293 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:52.293 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:52.293 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:52.293 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:52.293 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:52.294 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:52.294 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:52.294 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:52.294 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:52.295 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:52.295 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:52.295 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:52.295 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:52.296 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:52.296 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:52.296 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:52.296 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:52.296 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:52.297 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:52.297 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:52.297 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:52.298 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:52.298 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:52.298 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.298 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:52.299 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:52.299 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:52.299 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:52.300 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:52.300 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:52.300 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:52.301 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:52.301 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:52.301 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:52.301 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:52.302 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:57:52.302 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:52.303 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:57:52.303 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.303 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.303 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.304 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 27, 2022-07-21T01:57:52.304 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:57:52.304 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:57:39.555617+0000", 2022-07-21T01:57:52.304 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:52.305 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:52.305 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:52.305 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:52.306 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:52.306 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:52.307 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:52.307 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:52.307 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:52.307 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:52.307 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:52.308 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:52.308 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:52.308 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:52.308 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:52.309 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:52.309 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:52.309 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:52.309 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:52.309 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:52.310 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.310 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "141", 2022-07-21T01:57:52.310 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:52.310 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:52.311 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:52.311 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:52.311 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:52.311 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:52.312 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:52.312 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:52.312 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:52.313 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:52.313 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:52.313 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:52.313 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:52.313 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:52.314 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:52.314 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:52.314 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:52.315 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:52.315 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:52.315 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:52.316 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:52.316 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:52.316 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:52.317 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:52.317 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:52.317 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:52.317 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.318 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:52.318 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:52.318 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:52.318 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:52.319 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:52.319 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:52.319 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:52.319 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:52.319 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:52.320 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:52.320 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:52.320 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:57:52.320 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:57:52.321 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:57:52.321 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:57:52.321 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.322 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:52.322 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:57:52.322 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:57:52.322 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.323 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.323 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.323 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.323 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 28, 2022-07-21T01:57:52.324 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:57:52.324 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:57:40.211694+0000", 2022-07-21T01:57:52.324 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:52.324 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:52.325 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:52.325 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:52.325 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:52.325 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:52.325 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:52.326 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:52.326 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:52.326 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:52.327 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:52.327 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:52.327 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:52.327 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:52.328 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:52.328 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:52.328 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:52.328 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:52.328 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:52.329 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:52.329 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.329 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "142", 2022-07-21T01:57:52.329 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:52.329 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:52.330 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:52.330 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:52.330 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:52.330 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:52.331 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:52.331 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:52.331 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:52.331 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:52.332 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:52.332 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:52.332 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:52.332 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:52.332 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:52.333 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:52.333 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:52.333 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:52.333 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:52.334 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:52.334 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:52.334 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:52.334 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:52.335 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:52.335 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:52.335 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:52.335 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.335 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:52.336 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:52.336 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:52.336 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:52.336 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:52.337 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:52.337 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:52.337 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:52.337 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:52.338 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:52.338 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:52.338 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:57:52.338 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:52.339 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:57:52.339 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:57:52.339 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.339 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.340 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.340 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:52.340 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:57:52.340 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.341 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:57:52.341 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:57:52.341 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:52.341 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:52.342 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:52.342 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:52.342 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:52.342 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:52.343 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:52.343 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 139, 2022-07-21T01:57:52.343 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:52.343 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:52.344 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:52.344 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.344 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.344 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.345 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:57:52.345 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:52.345 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.346 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.346 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.346 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:57:52.346 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:52.347 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.347 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.347 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.347 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:52.347 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.348 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.348 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.348 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:57:52.348 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:52.348 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.349 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.349 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.349 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:57:52.349 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:52.350 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.350 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.350 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.350 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:52.350 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.351 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.351 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.351 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:57:52.351 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:52.352 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.352 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.352 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.352 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:57:52.352 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:52.353 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.353 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.353 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.353 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:52.354 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.354 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.355 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.355 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:57:52.355 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:52.355 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.356 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.356 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.356 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:57:52.357 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:52.357 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.357 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.357 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.357 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:57:52.358 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:57:52.358 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:57:52.358 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:57:52.358 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:52.359 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:52.359 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:52.359 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.359 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.360 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.360 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:57:52.360 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:57:52.360 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:52.360 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:52.361 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:52.361 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:52.362 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:52.362 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:52.362 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:52.362 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 139, 2022-07-21T01:57:52.363 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:52.363 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:52.363 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:52.363 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.364 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.364 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.364 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:57:52.364 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:52.364 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.365 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.365 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.365 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:57:52.366 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:52.366 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.366 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.367 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.367 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:52.367 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.367 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.367 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.368 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:57:52.368 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:52.368 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.368 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.369 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.369 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:57:52.369 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:52.369 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.370 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.370 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.370 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:52.370 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.371 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.371 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.371 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:57:52.371 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:52.371 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.372 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.372 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.372 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:57:52.372 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:52.372 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.373 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.373 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.373 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:52.373 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.374 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.374 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.374 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:57:52.374 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:52.374 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.375 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.375 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.375 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:57:52.375 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:52.376 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.376 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.376 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.376 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:57:52.376 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:57:52.377 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:57:52.377 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:57:52.377 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:52.377 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:52.377 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:52.378 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.378 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.378 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.378 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:57:52.379 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:57:52.379 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:52.379 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:52.379 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:52.379 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:52.380 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:52.380 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:52.380 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:52.380 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 138, 2022-07-21T01:57:52.380 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:52.381 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:52.381 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:52.381 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.381 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.382 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.382 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:57:52.382 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:52.382 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.382 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.383 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.383 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:57:52.383 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:52.383 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.384 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.384 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.384 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:52.384 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.385 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.385 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.385 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:57:52.385 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:52.385 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.386 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.386 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.386 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:57:52.386 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:52.387 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.387 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.387 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.387 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:52.388 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.388 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.388 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.388 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:57:52.388 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:52.389 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.389 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.389 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.389 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:57:52.390 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:52.390 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.390 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.390 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.390 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:52.391 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.391 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.391 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.391 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:57:52.392 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:52.392 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.392 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.392 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.392 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:57:52.393 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:52.393 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.393 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.393 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.393 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:57:52.394 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:57:52.394 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:57:52.394 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:57:52.394 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:52.395 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:52.395 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:52.395 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.395 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.395 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.396 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:57:52.396 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:57:52.396 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:52.396 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:52.397 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:52.397 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:52.397 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:52.397 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:52.398 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:52.398 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 129, 2022-07-21T01:57:52.398 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:52.398 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:52.399 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:52.399 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.399 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.399 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.399 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:57:52.400 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:52.400 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.400 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.400 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.400 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:57:52.401 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:52.401 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.401 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.401 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.402 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:52.402 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.402 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.402 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.403 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:57:52.403 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:52.403 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.403 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.403 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.404 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:57:52.404 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:52.404 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.404 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.404 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.405 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:52.405 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.405 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.405 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.406 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:57:52.406 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:52.406 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.406 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.407 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.407 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:57:52.407 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:52.407 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.407 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.408 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.408 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:52.408 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.408 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.409 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.409 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:57:52.409 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:52.409 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.409 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.410 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.410 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:57:52.410 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:52.410 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.411 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.411 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.411 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:57:52.411 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:57:52.411 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:57:52.412 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:57:52.412 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:52.412 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:52.412 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:52.412 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.413 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.413 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.413 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:57:52.414 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:57:52.414 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:52.414 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:52.414 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:52.415 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:52.415 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:52.415 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:52.415 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:52.415 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 138, 2022-07-21T01:57:52.416 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:52.416 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:52.416 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:52.416 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.417 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.417 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.418 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:57:52.418 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:52.418 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.418 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.419 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.419 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:57:52.419 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:52.419 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.419 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.420 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.420 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:52.420 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.420 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.420 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.421 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:57:52.421 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:52.421 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.421 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.422 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.422 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:57:52.422 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:52.422 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.422 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.423 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.423 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:52.423 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.423 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.424 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.424 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:57:52.424 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:52.424 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.424 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.425 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.425 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:57:52.425 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:52.426 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.426 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.426 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.427 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:52.427 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.427 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.427 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.428 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:57:52.428 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:52.428 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.428 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.429 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.429 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:57:52.429 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:52.429 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.429 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.430 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.430 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:57:52.430 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:57:52.430 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:57:52.431 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:57:52.431 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:52.431 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:52.431 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:52.432 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.432 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.432 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.432 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:57:52.432 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:57:52.433 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:52.433 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:52.433 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:52.433 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:52.433 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:52.434 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:52.434 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:52.434 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 139, 2022-07-21T01:57:52.434 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:52.435 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:52.435 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:52.435 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.435 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.436 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.436 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:57:52.436 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:52.436 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.436 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.437 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.437 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:57:52.437 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:52.437 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.438 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.438 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.438 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:52.438 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.439 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.439 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.439 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:57:52.439 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:52.440 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.440 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.440 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.440 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:57:52.440 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:52.441 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.441 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.441 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.441 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:52.442 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.442 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.442 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.442 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:57:52.442 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:52.443 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.443 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.443 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.443 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:57:52.444 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:52.444 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.444 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.444 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.444 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:52.445 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.445 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.445 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.445 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:57:52.446 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:52.446 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.446 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.446 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.446 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:57:52.447 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:52.447 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.447 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.447 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.448 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:57:52.448 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:57:52.448 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:57:52.448 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:57:52.448 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:52.449 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:52.449 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:52.449 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.449 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.449 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.450 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:57:52.450 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:57:52.450 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:52.450 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:52.451 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:52.451 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:52.451 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:52.451 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:52.451 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:52.452 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 139, 2022-07-21T01:57:52.452 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:52.452 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:52.452 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:52.453 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.453 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.453 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.453 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:57:52.453 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:52.454 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.454 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.454 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.454 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:57:52.455 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:52.455 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.455 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.455 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.456 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:52.456 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.456 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.456 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.456 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:57:52.457 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:52.457 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.457 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.457 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.458 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:57:52.458 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:52.458 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.458 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.458 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.459 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:52.459 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.459 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.459 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.460 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:57:52.460 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:52.460 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.460 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.460 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.461 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:57:52.461 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:52.461 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.461 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.462 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.462 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:52.462 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.462 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.462 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.463 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:57:52.463 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:52.463 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.463 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.464 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.464 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:57:52.464 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:52.464 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.464 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.465 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.465 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:57:52.465 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:57:52.465 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:57:52.465 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:57:52.466 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:52.466 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:52.466 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:52.466 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.467 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.467 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.467 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:57:52.467 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:57:52.467 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:52.468 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:52.468 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:52.468 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:52.468 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:52.469 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:52.469 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:52.469 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 139, 2022-07-21T01:57:52.469 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:52.469 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:52.470 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:52.470 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.470 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.470 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.471 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:57:52.471 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:52.471 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.471 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.471 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.472 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:57:52.472 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:52.472 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.472 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.473 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.473 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:52.473 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.473 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.473 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.474 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:57:52.474 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:52.474 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.474 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.475 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.475 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:57:52.475 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:52.475 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.475 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.476 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.476 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:52.476 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.476 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.477 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.477 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:57:52.477 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:52.477 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.477 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.478 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.478 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:57:52.478 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:52.478 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.479 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.479 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.479 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:52.479 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:52.479 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.480 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:52.480 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:57:52.480 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:52.480 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.481 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.481 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:52.481 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:57:52.482 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:57:52.482 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.482 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.482 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.482 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:57:52.483 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:57:52.483 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:57:52.483 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:57:52.483 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:52.483 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:52.484 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:52.484 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:52.484 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.484 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:52.485 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:57:52.485 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.485 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:57:52.485 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:52.486 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:52.486 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:52.486 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:52.486 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:52.486 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:57:52.487 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:52.487 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.487 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.487 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:57:52.488 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:52.488 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:52.488 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:52.488 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:52.488 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:52.489 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:57:52.489 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:52.489 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.489 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.489 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:57:52.490 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:52.490 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:52.490 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:52.490 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:52.491 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:52.491 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:57:52.491 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:52.491 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.491 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.492 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:57:52.492 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:52.492 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:52.492 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:52.493 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:52.493 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:52.493 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:57:52.493 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:52.493 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.494 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.494 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:57:52.494 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:52.494 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:52.495 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:52.495 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:52.495 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:52.495 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:57:52.495 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:52.496 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.496 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.496 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:57:52.497 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:52.497 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:52.497 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:52.497 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:52.498 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:52.498 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:57:52.498 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:52.498 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.499 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.499 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:57:52.499 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:52.499 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:52.499 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:52.500 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:52.500 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:52.500 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:57:52.500 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:52.500 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.501 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:52.501 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:57:52.501 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:57:52.501 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:57:52.502 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:57:52.502 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:57:52.502 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:57:52.502 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:57:52.502 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:57:52.503 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.503 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:52.503 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:57:52.503 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:57:52.504 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:57:52.504 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:57:52.504 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": {}, 2022-07-21T01:57:52.504 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:57:52.504 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:57:52.505 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:57:52.505 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:57:52.505 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:57:52.505 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:57:52.506 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:57:52.506 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:57:52.506 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:52.506 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:52.507 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:57:52.507 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:57:52.507 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:57:52.507 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:57:52.507 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:57:52.508 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:57:52.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 -- 172.21.15.140:0/2817756821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053aa0 msgr2=0x7f3ae0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053aa0 0x7f3ae0055f60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f3af400d000 tx=0x7f3af4008040).stop 2022-07-21T01:57:52.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 -- 172.21.15.140:0/2817756821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b000b5650 msgr2=0x7f3b00111190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b000b5650 0x7f3b00111190 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7f3af8007000 tx=0x7f3af8008040).stop 2022-07-21T01:57:52.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 -- 172.21.15.140:0/2817756821 shutdown_connections 2022-07-21T01:57:52.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3ae0053aa0 0x7f3ae0055f60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b000b5650 0x7f3b00111190 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b000af8e0 0x7f3b001433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 --2- 172.21.15.140:0/2817756821 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b000adff0 0x7f3b00066d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 -- 172.21.15.140:0/2817756821 >> 172.21.15.140:0/2817756821 conn(0x7f3b000634e0 msgr2=0x7f3b0005bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:52.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 -- 172.21.15.140:0/2817756821 shutdown_connections 2022-07-21T01:57:52.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.259+0000 7f3b0513e700 1 -- 172.21.15.140:0/2817756821 wait complete. 2022-07-21T01:57:52.513 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:52.626 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.623+0000 7f3bee3b4700 1 Processor -- start 2022-07-21T01:57:52.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.623+0000 7f3bee3b4700 1 -- start start 2022-07-21T01:57:52.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.623+0000 7f3bee3b4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3be80b5650 0x7f3be80b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.623+0000 7f3bee3b4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3be80adff0 0x7f3be8147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.623+0000 7f3bee3b4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3be80af8e0 0x7f3be805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.623+0000 7f3bee3b4700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be8147010 con 0x7f3be80b5650 2022-07-21T01:57:52.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.623+0000 7f3bee3b4700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be81472b0 con 0x7f3be80af8e0 2022-07-21T01:57:52.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.623+0000 7f3bee3b4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be8146d40 con 0x7f3be80adff0 2022-07-21T01:57:52.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bed3b2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3be80b5650 0x7f3be80b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bed3b2700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f3be80b5650 0x7f3be80b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41382/0 (socket says 172.21.15.140:41382) 2022-07-21T01:57:52.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bed3b2700 1 -- 172.21.15.140:0/512723920 learned_addr learned my addr 172.21.15.140:0/512723920 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:52.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bedbb3700 1 --2- 172.21.15.140:0/512723920 >> v2:172.21.15.152:3300/0 conn(0x7f3be80af8e0 0x7f3be805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.632 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3becbb1700 1 --2- 172.21.15.140:0/512723920 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3be80adff0 0x7f3be8147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bedbb3700 1 -- 172.21.15.140:0/512723920 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3be80adff0 msgr2=0x7f3be8147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:52.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bedbb3700 1 --2- 172.21.15.140:0/512723920 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3be80adff0 0x7f3be8147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bedbb3700 1 -- 172.21.15.140:0/512723920 >> v2:172.21.15.140:3300/0 conn(0x7f3be80b5650 msgr2=0x7f3be80b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:52.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bedbb3700 1 --2- 172.21.15.140:0/512723920 >> v2:172.21.15.140:3300/0 conn(0x7f3be80b5650 0x7f3be80b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bedbb3700 1 -- 172.21.15.140:0/512723920 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be8147c40 con 0x7f3be80af8e0 2022-07-21T01:57:52.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bed3b2700 1 --2- 172.21.15.140:0/512723920 >> v2:172.21.15.140:3300/0 conn(0x7f3be80b5650 0x7f3be80b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:52.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bedbb3700 1 --2- 172.21.15.140:0/512723920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3be80af8e0 0x7f3be805bd90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f3be4002f70 tx=0x7f3be4006df0).ready entity=mon.1 client_cookie=a03799e6c3d621a9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:52.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bdffff700 1 -- 172.21.15.140:0/512723920 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be400d4a0 con 0x7f3be80af8e0 2022-07-21T01:57:52.634 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bdffff700 1 -- 172.21.15.140:0/512723920 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be4000e90 con 0x7f3be80af8e0 2022-07-21T01:57:52.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bdffff700 1 -- 172.21.15.140:0/512723920 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be400d7f0 con 0x7f3be80af8e0 2022-07-21T01:57:52.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bee3b4700 1 -- 172.21.15.140:0/512723920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3be80af8e0 msgr2=0x7f3be805bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bee3b4700 1 --2- 172.21.15.140:0/512723920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3be80af8e0 0x7f3be805bd90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f3be4002f70 tx=0x7f3be4006df0).stop 2022-07-21T01:57:52.635 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bee3b4700 1 -- 172.21.15.140:0/512723920 shutdown_connections 2022-07-21T01:57:52.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bee3b4700 1 --2- 172.21.15.140:0/512723920 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3be80af8e0 0x7f3be805bd90 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bee3b4700 1 --2- 172.21.15.140:0/512723920 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3be80adff0 0x7f3be8147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bee3b4700 1 --2- 172.21.15.140:0/512723920 >> v2:172.21.15.140:3300/0 conn(0x7f3be80b5650 0x7f3be80b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.636 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bee3b4700 1 -- 172.21.15.140:0/512723920 >> 172.21.15.140:0/512723920 conn(0x7f3be80634e0 msgr2=0x7f3be80b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:52.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bee3b4700 1 -- 172.21.15.140:0/512723920 shutdown_connections 2022-07-21T01:57:52.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bee3b4700 1 -- 172.21.15.140:0/512723920 wait complete. 2022-07-21T01:57:52.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bee3b4700 1 Processor -- start 2022-07-21T01:57:52.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.627+0000 7f3bee3b4700 1 -- start start 2022-07-21T01:57:52.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bee3b4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3be80adff0 0x7f3be8148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bee3b4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3be80af8e0 0x7f3be8064d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bee3b4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3be80b5650 0x7f3be80651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bee3b4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be8111180 con 0x7f3be80adff0 2022-07-21T01:57:52.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bee3b4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be8111510 con 0x7f3be80af8e0 2022-07-21T01:57:52.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bee3b4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be81117b0 con 0x7f3be80b5650 2022-07-21T01:57:52.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bed3b2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3be80adff0 0x7f3be8148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bed3b2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3be80adff0 0x7f3be8148580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41388/0 (socket says 172.21.15.140:41388) 2022-07-21T01:57:52.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bed3b2700 1 -- 172.21.15.140:0/2555990736 learned_addr learned my addr 172.21.15.140:0/2555990736 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:52.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bedbb3700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3be80b5650 0x7f3be80651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3becbb1700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3be80af8e0 0x7f3be8064d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bed3b2700 1 -- 172.21.15.140:0/2555990736 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3be80b5650 msgr2=0x7f3be80651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bed3b2700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3be80b5650 0x7f3be80651f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bed3b2700 1 -- 172.21.15.140:0/2555990736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3be80af8e0 msgr2=0x7f3be8064d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bed3b2700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3be80af8e0 0x7f3be8064d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bed3b2700 1 -- 172.21.15.140:0/2555990736 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be0009530 con 0x7f3be80adff0 2022-07-21T01:57:52.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bed3b2700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3be80adff0 0x7f3be8148580 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f3be0006b60 tx=0x7f3be0009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:52.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bde7fc700 1 -- 172.21.15.140:0/2555990736 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be000c850 con 0x7f3be80adff0 2022-07-21T01:57:52.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bde7fc700 1 -- 172.21.15.140:0/2555990736 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000c9b0 con 0x7f3be80adff0 2022-07-21T01:57:52.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bee3b4700 1 -- 172.21.15.140:0/2555990736 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be400b180 con 0x7f3be80adff0 2022-07-21T01:57:52.645 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bde7fc700 1 -- 172.21.15.140:0/2555990736 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3be000cd10 con 0x7f3be80adff0 2022-07-21T01:57:52.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.631+0000 7f3bee3b4700 1 -- 172.21.15.140:0/2555990736 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be810f910 con 0x7f3be80adff0 2022-07-21T01:57:52.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.635+0000 7f3bee3b4700 1 -- 172.21.15.140:0/2555990736 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be810fc20 con 0x7f3be80adff0 2022-07-21T01:57:52.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.635+0000 7f3bde7fc700 1 -- 172.21.15.140:0/2555990736 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3be0022070 con 0x7f3be80adff0 2022-07-21T01:57:52.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.635+0000 7f3bde7fc700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bd0053aa0 0x7f3bd0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.635+0000 7f3bde7fc700 1 -- 172.21.15.140:0/2555990736 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3be006c750 con 0x7f3be80adff0 2022-07-21T01:57:52.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.635+0000 7f3becbb1700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bd0053aa0 0x7f3bd0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.635+0000 7f3becbb1700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bd0053aa0 0x7f3bd0055f60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f3bd8006000 tx=0x7f3bd8006d90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:52.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.639+0000 7f3bde7fc700 1 -- 172.21.15.140:0/2555990736 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3be0043560 con 0x7f3be80adff0 2022-07-21T01:57:52.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.799+0000 7f3bee3b4700 1 -- 172.21.15.140:0/2555990736 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3be80649f0 con 0x7f3be80adff0 2022-07-21T01:57:52.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bde7fc700 1 -- 172.21.15.140:0/2555990736 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2142 (secure 0 0 0) 0x7f3be0046b80 con 0x7f3be80adff0 2022-07-21T01:57:52.804 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:52.804 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":69,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/148832193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":148832193},{"type":"v1","addr":"172.21.15.152:6833","nonce":148832193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:41.192528+0000","modified":"2022-07-21T01:57:42.195096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8308},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8308":{"gid":8308,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/961657495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":961657495},{"type":"v1","addr":"172.21.15.140:6835","nonce":961657495}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-07-21T01:57:52.807 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 -- 172.21.15.140:0/2555990736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bd0053aa0 msgr2=0x7f3bd0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bd0053aa0 0x7f3bd0055f60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f3bd8006000 tx=0x7f3bd8006d90).stop 2022-07-21T01:57:52.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 -- 172.21.15.140:0/2555990736 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3be80adff0 msgr2=0x7f3be8148580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3be80adff0 0x7f3be8148580 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f3be0006b60 tx=0x7f3be0009290).stop 2022-07-21T01:57:52.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 -- 172.21.15.140:0/2555990736 shutdown_connections 2022-07-21T01:57:52.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3bd0053aa0 0x7f3bd0055f60 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3be80b5650 0x7f3be80651f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3be80af8e0 0x7f3be8064d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 --2- 172.21.15.140:0/2555990736 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3be80adff0 0x7f3be8148580 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 -- 172.21.15.140:0/2555990736 >> 172.21.15.140:0/2555990736 conn(0x7f3be80634e0 msgr2=0x7f3be80afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:52.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 -- 172.21.15.140:0/2555990736 shutdown_connections 2022-07-21T01:57:52.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.803+0000 7f3bee3b4700 1 -- 172.21.15.140:0/2555990736 wait complete. 2022-07-21T01:57:52.811 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 69 2022-07-21T01:57:52.823 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:57:52.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.927+0000 7f86b72c4700 1 Processor -- start 2022-07-21T01:57:52.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.927+0000 7f86b72c4700 1 -- start start 2022-07-21T01:57:52.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b72c4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f86b00b5860 0x7f86b00acae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b72c4700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b00b46e0 con 0x7f86b0146770 2022-07-21T01:57:52.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b72c4700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b00b4b10 con 0x7f86b00b4f80 2022-07-21T01:57:52.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b72c4700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b00ad990 con 0x7f86b00b5860 2022-07-21T01:57:52.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b6ac3700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f86b0146770 0x7f86b014c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55544/0 (socket says 172.21.15.140:55544) 2022-07-21T01:57:52.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b6ac3700 1 -- 172.21.15.140:0/2839218625 learned_addr learned my addr 172.21.15.140:0/2839218625 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:52.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b62c2700 1 --2- 172.21.15.140:0/2839218625 >> v2:172.21.15.152:3300/0 conn(0x7f86b00b5860 0x7f86b00acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b52c0700 1 -- 172.21.15.140:0/2839218625 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587069648 0 0) 0x7f86b00b46e0 con 0x7f86b0146770 2022-07-21T01:57:52.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b52c0700 1 -- 172.21.15.140:0/2839218625 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8698003260 con 0x7f86b0146770 2022-07-21T01:57:52.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b62c2700 1 -- 172.21.15.140:0/2839218625 >> v1:172.21.15.152:6789/0 conn(0x7f86b00b4f80 legacy=0x7f86b0146060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b62c2700 1 -- 172.21.15.140:0/2839218625 >> v1:172.21.15.140:6789/0 conn(0x7f86b0146770 legacy=0x7f86b014c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b62c2700 1 -- 172.21.15.140:0/2839218625 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b00ad3d0 con 0x7f86b00b5860 2022-07-21T01:57:52.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b62c2700 1 --2- 172.21.15.140:0/2839218625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86b00b5860 0x7f86b00acae0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f86ac002f70 tx=0x7f86ac006df0).ready entity=mon.1 client_cookie=5939cdca5f0749c server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:52.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b52c0700 1 -- 172.21.15.140:0/2839218625 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86ac00d5c0 con 0x7f86b00b5860 2022-07-21T01:57:52.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b52c0700 1 -- 172.21.15.140:0/2839218625 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac000e90 con 0x7f86b00b5860 2022-07-21T01:57:52.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b52c0700 1 -- 172.21.15.140:0/2839218625 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86ac00d910 con 0x7f86b00b5860 2022-07-21T01:57:52.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b72c4700 1 -- 172.21.15.140:0/2839218625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86b00b5860 msgr2=0x7f86b00acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b72c4700 1 --2- 172.21.15.140:0/2839218625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86b00b5860 0x7f86b00acae0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f86ac002f70 tx=0x7f86ac006df0).stop 2022-07-21T01:57:52.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b72c4700 1 -- 172.21.15.140:0/2839218625 shutdown_connections 2022-07-21T01:57:52.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b72c4700 1 --2- 172.21.15.140:0/2839218625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86b00b5860 0x7f86b00acae0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b72c4700 1 -- 172.21.15.140:0/2839218625 >> 172.21.15.140:0/2839218625 conn(0x7f86b00b5e10 msgr2=0x7f86b00b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:52.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b72c4700 1 -- 172.21.15.140:0/2839218625 shutdown_connections 2022-07-21T01:57:52.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.931+0000 7f86b72c4700 1 -- 172.21.15.140:0/2839218625 wait complete. 2022-07-21T01:57:52.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b72c4700 1 Processor -- start 2022-07-21T01:57:52.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b72c4700 1 -- start start 2022-07-21T01:57:52.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b72c4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86b00b4f80 0x7f86b00b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b72c4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86b00b5860 0x7f86b013ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b72c4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86b0146770 0x7f86b013bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b72c4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00b0620 con 0x7f86b0146770 2022-07-21T01:57:52.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b72c4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b013c410 con 0x7f86b00b4f80 2022-07-21T01:57:52.943 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b72c4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00afc40 con 0x7f86b00b5860 2022-07-21T01:57:52.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b6ac3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86b0146770 0x7f86b013bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b6ac3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86b0146770 0x7f86b013bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41406/0 (socket says 172.21.15.140:41406) 2022-07-21T01:57:52.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b6ac3700 1 -- 172.21.15.140:0/841023943 learned_addr learned my addr 172.21.15.140:0/841023943 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:52.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b62c2700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86b00b4f80 0x7f86b00b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b5ac1700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86b00b5860 0x7f86b013ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b6ac3700 1 -- 172.21.15.140:0/841023943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86b00b5860 msgr2=0x7f86b013ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b6ac3700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86b00b5860 0x7f86b013ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b6ac3700 1 -- 172.21.15.140:0/841023943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86b00b4f80 msgr2=0x7f86b00b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:52.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b6ac3700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86b00b4f80 0x7f86b00b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:52.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b6ac3700 1 -- 172.21.15.140:0/841023943 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac00b180 con 0x7f86b0146770 2022-07-21T01:57:52.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b6ac3700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86b0146770 0x7f86b013bf20 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7f86b013c5a0 tx=0x7f86a8007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:52.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f869f7fe700 1 -- 172.21.15.140:0/841023943 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86a8005ba0 con 0x7f86b0146770 2022-07-21T01:57:52.946 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f869f7fe700 1 -- 172.21.15.140:0/841023943 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a8005d00 con 0x7f86b0146770 2022-07-21T01:57:52.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b72c4700 1 -- 172.21.15.140:0/841023943 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b00afee0 con 0x7f86b0146770 2022-07-21T01:57:52.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f869f7fe700 1 -- 172.21.15.140:0/841023943 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86a80072c0 con 0x7f86b0146770 2022-07-21T01:57:52.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.935+0000 7f86b72c4700 1 -- 172.21.15.140:0/841023943 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b0112f40 con 0x7f86b0146770 2022-07-21T01:57:52.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.939+0000 7f86b72c4700 1 -- 172.21.15.140:0/841023943 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b00af0f0 con 0x7f86b0146770 2022-07-21T01:57:52.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.939+0000 7f869f7fe700 1 -- 172.21.15.140:0/841023943 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f86a8010070 con 0x7f86b0146770 2022-07-21T01:57:52.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.939+0000 7f869f7fe700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8690053b00 0x7f8690055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:52.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.939+0000 7f869f7fe700 1 -- 172.21.15.140:0/841023943 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f86a806b920 con 0x7f86b0146770 2022-07-21T01:57:52.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.939+0000 7f86b62c2700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8690053b00 0x7f8690055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:52.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.939+0000 7f86b62c2700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8690053b00 0x7f8690055fc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f86ac01a000 tx=0x7f86ac014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:52.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:52.943+0000 7f869f7fe700 1 -- 172.21.15.140:0/841023943 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f86a80426b0 con 0x7f86b0146770 2022-07-21T01:57:53.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:53.103+0000 7f86b72c4700 1 -- 172.21.15.140:0/841023943 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f86b00574e0 con 0x7f86b0146770 2022-07-21T01:57:54.085 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:57:54.083+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:57:54.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.095+0000 7f869f7fe700 1 -- 172.21.15.140:0/841023943 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7f86a8045cd0 con 0x7f86b0146770 2022-07-21T01:57:54.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 -- 172.21.15.140:0/841023943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8690053b00 msgr2=0x7f8690055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8690053b00 0x7f8690055fc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f86ac01a000 tx=0x7f86ac014000).stop 2022-07-21T01:57:54.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 -- 172.21.15.140:0/841023943 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86b0146770 msgr2=0x7f86b013bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86b0146770 0x7f86b013bf20 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7f86b013c5a0 tx=0x7f86a8007e00).stop 2022-07-21T01:57:54.103 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 -- 172.21.15.140:0/841023943 shutdown_connections 2022-07-21T01:57:54.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8690053b00 0x7f8690055fc0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86b0146770 0x7f86b013bf20 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86b00b5860 0x7f86b013ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.104 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 --2- 172.21.15.140:0/841023943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86b00b4f80 0x7f86b00b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 -- 172.21.15.140:0/841023943 >> 172.21.15.140:0/841023943 conn(0x7f86b00b5e10 msgr2=0x7f86b005c770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:54.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 -- 172.21.15.140:0/841023943 shutdown_connections 2022-07-21T01:57:54.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.099+0000 7f86b72c4700 1 -- 172.21.15.140:0/841023943 wait complete. 2022-07-21T01:57:54.106 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:57:54.117 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:57:54.128 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:57:54.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626bfff700 1 Processor -- start 2022-07-21T01:57:54.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626bfff700 1 -- start start 2022-07-21T01:57:54.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6264026cf0 0x7f6264029370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626bfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f62640d4230 0x7f6264025a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626bfff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62640270d0 con 0x7f6264025f70 2022-07-21T01:57:54.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626bfff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6264028990 con 0x7f6264026cf0 2022-07-21T01:57:54.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626bfff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6264028c30 con 0x7f62640d4230 2022-07-21T01:57:54.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626affd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6264026cf0 0x7f6264029370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626affd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6264026cf0 0x7f6264029370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41422/0 (socket says 172.21.15.140:41422) 2022-07-21T01:57:54.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626affd700 1 -- 172.21.15.140:0/2772448739 learned_addr learned my addr 172.21.15.140:0/2772448739 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:54.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626a7fc700 1 --2- 172.21.15.140:0/2772448739 >> v2:172.21.15.152:3300/0 conn(0x7f62640d4230 0x7f6264025a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626affd700 1 -- 172.21.15.140:0/2772448739 >> v2:172.21.15.152:3300/0 conn(0x7f62640d4230 msgr2=0x7f6264025a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:54.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626affd700 1 --2- 172.21.15.140:0/2772448739 >> v2:172.21.15.152:3300/0 conn(0x7f62640d4230 0x7f6264025a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626affd700 1 -- 172.21.15.140:0/2772448739 >> v1:172.21.15.152:6789/0 conn(0x7f6264025f70 legacy=0x7f62640d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626affd700 1 -- 172.21.15.140:0/2772448739 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6264028ed0 con 0x7f6264026cf0 2022-07-21T01:57:54.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f626affd700 1 --2- 172.21.15.140:0/2772448739 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6264026cf0 0x7f6264029370 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f625c002f70 tx=0x7f625c006eb0).ready entity=mon.0 client_cookie=b98fbb72307b572 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:54.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.211+0000 7f6269ffb700 1 -- 172.21.15.140:0/2772448739 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f625c00d610 con 0x7f6264026cf0 2022-07-21T01:57:54.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f6269ffb700 1 -- 172.21.15.140:0/2772448739 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c000e90 con 0x7f6264026cf0 2022-07-21T01:57:54.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f6269ffb700 1 -- 172.21.15.140:0/2772448739 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f625c00d9a0 con 0x7f6264026cf0 2022-07-21T01:57:54.225 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 -- 172.21.15.140:0/2772448739 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6264026cf0 msgr2=0x7f6264029370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 --2- 172.21.15.140:0/2772448739 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6264026cf0 0x7f6264029370 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f625c002f70 tx=0x7f625c006eb0).stop 2022-07-21T01:57:54.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 -- 172.21.15.140:0/2772448739 shutdown_connections 2022-07-21T01:57:54.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 --2- 172.21.15.140:0/2772448739 >> v2:172.21.15.152:3300/0 conn(0x7f62640d4230 0x7f6264025a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 --2- 172.21.15.140:0/2772448739 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6264026cf0 0x7f6264029370 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 -- 172.21.15.140:0/2772448739 >> 172.21.15.140:0/2772448739 conn(0x7f6264046bf0 msgr2=0x7f6264046fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:54.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 -- 172.21.15.140:0/2772448739 shutdown_connections 2022-07-21T01:57:54.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 -- 172.21.15.140:0/2772448739 wait complete. 2022-07-21T01:57:54.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 Processor -- start 2022-07-21T01:57:54.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 -- start start 2022-07-21T01:57:54.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6264025f70 0x7f62640d6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6264026cf0 0x7f626402b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62640d4230 0x7f626402b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.215+0000 7f626bfff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626403da30 con 0x7f6264025f70 2022-07-21T01:57:54.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626bfff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626403dc60 con 0x7f62640d4230 2022-07-21T01:57:54.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626bfff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626402a8e0 con 0x7f6264026cf0 2022-07-21T01:57:54.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6264025f70 0x7f62640d6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626affd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6264025f70 0x7f62640d6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41428/0 (socket says 172.21.15.140:41428) 2022-07-21T01:57:54.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626affd700 1 -- 172.21.15.140:0/1457899369 learned_addr learned my addr 172.21.15.140:0/1457899369 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:54.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626b7fe700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62640d4230 0x7f626402b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626a7fc700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6264026cf0 0x7f626402b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626affd700 1 -- 172.21.15.140:0/1457899369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6264026cf0 msgr2=0x7f626402b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626affd700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6264026cf0 0x7f626402b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626affd700 1 -- 172.21.15.140:0/1457899369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62640d4230 msgr2=0x7f626402b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626affd700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62640d4230 0x7f626402b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626affd700 1 -- 172.21.15.140:0/1457899369 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c00b1e0 con 0x7f6264025f70 2022-07-21T01:57:54.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626affd700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6264025f70 0x7f62640d6ac0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f625c002f70 tx=0x7f625c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:54.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f625bfff700 1 -- 172.21.15.140:0/1457899369 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f625c011710 con 0x7f6264025f70 2022-07-21T01:57:54.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f625bfff700 1 -- 172.21.15.140:0/1457899369 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c011870 con 0x7f6264025f70 2022-07-21T01:57:54.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626bfff700 1 -- 172.21.15.140:0/1457899369 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6264029cf0 con 0x7f6264025f70 2022-07-21T01:57:54.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f625bfff700 1 -- 172.21.15.140:0/1457899369 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f625c011bd0 con 0x7f6264025f70 2022-07-21T01:57:54.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626bfff700 1 -- 172.21.15.140:0/1457899369 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62640403f0 con 0x7f6264025f70 2022-07-21T01:57:54.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626bfff700 1 -- 172.21.15.140:0/1457899369 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6264005270 con 0x7f6264025f70 2022-07-21T01:57:54.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f625bfff700 1 -- 172.21.15.140:0/1457899369 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f625c025030 con 0x7f6264025f70 2022-07-21T01:57:54.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f625bfff700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6248053aa0 0x7f6248055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f625bfff700 1 -- 172.21.15.140:0/1457899369 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f625c06cff0 con 0x7f6264025f70 2022-07-21T01:57:54.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.219+0000 7f626a7fc700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6248053aa0 0x7f6248055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.223+0000 7f626a7fc700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6248053aa0 0x7f6248055f60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f626000c000 tx=0x7f6260008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:54.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.227+0000 7f625bfff700 1 -- 172.21.15.140:0/1457899369 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f625c043ec0 con 0x7f6264025f70 2022-07-21T01:57:54.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.383+0000 7f626bfff700 1 -- 172.21.15.140:0/1457899369 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f626403de20 con 0x7f6264025f70 2022-07-21T01:57:54.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.387+0000 7f625bfff700 1 -- 172.21.15.140:0/1457899369 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+1782 (secure 0 0 0) 0x7f625c0474e0 con 0x7f6264025f70 2022-07-21T01:57:54.389 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:54.389 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":70,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/148832193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":148832193},{"type":"v1","addr":"172.21.15.152:6833","nonce":148832193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68}],"filesystems":[{"mdsmap":{"epoch":70,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:57:41.192528+0000","modified":"2022-07-21T01:57:54.086346+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":143,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2022-07-21T01:57:54.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.387+0000 7f626bfff700 1 -- 172.21.15.140:0/1457899369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6248053aa0 msgr2=0x7f6248055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.387+0000 7f626bfff700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6248053aa0 0x7f6248055f60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f626000c000 tx=0x7f6260008040).stop 2022-07-21T01:57:54.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.391+0000 7f626bfff700 1 -- 172.21.15.140:0/1457899369 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6264025f70 msgr2=0x7f62640d6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.391+0000 7f626bfff700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6264025f70 0x7f62640d6ac0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f625c002f70 tx=0x7f625c015000).stop 2022-07-21T01:57:54.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.391+0000 7f626bfff700 1 -- 172.21.15.140:0/1457899369 shutdown_connections 2022-07-21T01:57:54.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.391+0000 7f626bfff700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6248053aa0 0x7f6248055f60 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.391+0000 7f626bfff700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62640d4230 0x7f626402b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.391+0000 7f626bfff700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6264026cf0 0x7f626402b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.391+0000 7f626bfff700 1 --2- 172.21.15.140:0/1457899369 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6264025f70 0x7f62640d6ac0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.391+0000 7f626bfff700 1 -- 172.21.15.140:0/1457899369 >> 172.21.15.140:0/1457899369 conn(0x7f6264046bf0 msgr2=0x7f6264041590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:54.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.391+0000 7f626bfff700 1 -- 172.21.15.140:0/1457899369 shutdown_connections 2022-07-21T01:57:54.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.391+0000 7f626bfff700 1 -- 172.21.15.140:0/1457899369 wait complete. 2022-07-21T01:57:54.396 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 70 2022-07-21T01:57:54.408 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:57:54.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8617fff700 1 Processor -- start 2022-07-21T01:57:54.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8617fff700 1 -- start start 2022-07-21T01:57:54.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8617fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86180b5970 0x7f86180ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8617fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f86180af8c0 0x7f86180ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8617fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86180b0b80 con 0x7f86180aeae0 2022-07-21T01:57:54.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8617fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f861805d4d0 con 0x7f86180af8c0 2022-07-21T01:57:54.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8617fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86180b5d50 con 0x7f86180b5970 2022-07-21T01:57:54.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f86167fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f86180af8c0 0x7f86180ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f86167fc700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f86180af8c0 0x7f86180ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41436/0 (socket says 172.21.15.140:41436) 2022-07-21T01:57:54.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f86167fc700 1 -- 172.21.15.140:0/3105955861 learned_addr learned my addr 172.21.15.140:0/3105955861 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:54.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8616ffd700 1 --2- 172.21.15.140:0/3105955861 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86180b5970 0x7f86180ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f86167fc700 1 -- 172.21.15.140:0/3105955861 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86180b5970 msgr2=0x7f86180ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:54.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f86167fc700 1 --2- 172.21.15.140:0/3105955861 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86180b5970 0x7f86180ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f86167fc700 1 -- 172.21.15.140:0/3105955861 >> v1:172.21.15.152:6789/0 conn(0x7f86180aeae0 legacy=0x7f861805bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f86167fc700 1 -- 172.21.15.140:0/3105955861 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86180b0db0 con 0x7f86180af8c0 2022-07-21T01:57:54.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8616ffd700 1 --2- 172.21.15.140:0/3105955861 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86180b5970 0x7f86180ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:57:54.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f86167fc700 1 --2- 172.21.15.140:0/3105955861 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86180af8c0 0x7f86180ae5f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f86080032f0 tx=0x7f8608009290).ready entity=mon.0 client_cookie=cb3467dc0e4efa85 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:54.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8615ffb700 1 -- 172.21.15.140:0/3105955861 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f860800b8a0 con 0x7f86180af8c0 2022-07-21T01:57:54.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8615ffb700 1 -- 172.21.15.140:0/3105955861 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860800ba00 con 0x7f86180af8c0 2022-07-21T01:57:54.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.511+0000 7f8615ffb700 1 -- 172.21.15.140:0/3105955861 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f860800bb70 con 0x7f86180af8c0 2022-07-21T01:57:54.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 -- 172.21.15.140:0/3105955861 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86180af8c0 msgr2=0x7f86180ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 --2- 172.21.15.140:0/3105955861 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86180af8c0 0x7f86180ae5f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f86080032f0 tx=0x7f8608009290).stop 2022-07-21T01:57:54.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 -- 172.21.15.140:0/3105955861 shutdown_connections 2022-07-21T01:57:54.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 --2- 172.21.15.140:0/3105955861 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86180af8c0 0x7f86180ae5f0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 --2- 172.21.15.140:0/3105955861 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86180b5970 0x7f86180ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 -- 172.21.15.140:0/3105955861 >> 172.21.15.140:0/3105955861 conn(0x7f8618063490 msgr2=0x7f86180b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:54.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 -- 172.21.15.140:0/3105955861 shutdown_connections 2022-07-21T01:57:54.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 -- 172.21.15.140:0/3105955861 wait complete. 2022-07-21T01:57:54.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 Processor -- start 2022-07-21T01:57:54.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 -- start start 2022-07-21T01:57:54.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86180aeae0 0x7f8618149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86180af8c0 0x7f8618159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86180b5970 0x7f8618111410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8618142230 con 0x7f86180af8c0 2022-07-21T01:57:54.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86181402f0 con 0x7f86180b5970 2022-07-21T01:57:54.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8618159a30 con 0x7f86180aeae0 2022-07-21T01:57:54.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f86167fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86180af8c0 0x7f8618159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f86167fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86180af8c0 0x7f8618159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41444/0 (socket says 172.21.15.140:41444) 2022-07-21T01:57:54.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f86167fc700 1 -- 172.21.15.140:0/1589947515 learned_addr learned my addr 172.21.15.140:0/1589947515 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:54.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f86177fe700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86180b5970 0x7f8618111410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8616ffd700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86180aeae0 0x7f8618149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f86167fc700 1 -- 172.21.15.140:0/1589947515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86180aeae0 msgr2=0x7f8618149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f86167fc700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86180aeae0 0x7f8618149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f86167fc700 1 -- 172.21.15.140:0/1589947515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86180b5970 msgr2=0x7f8618111410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:54.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f86167fc700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86180b5970 0x7f8618111410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:54.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f86167fc700 1 -- 172.21.15.140:0/1589947515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8608009530 con 0x7f86180af8c0 2022-07-21T01:57:54.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f86167fc700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86180af8c0 0x7f8618159540 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f8618159b70 tx=0x7f8608016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:54.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f860ffff700 1 -- 172.21.15.140:0/1589947515 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8608021630 con 0x7f86180af8c0 2022-07-21T01:57:54.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f860ffff700 1 -- 172.21.15.140:0/1589947515 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8608021790 con 0x7f86180af8c0 2022-07-21T01:57:54.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f8617fff700 1 -- 172.21.15.140:0/1589947515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8618159c50 con 0x7f86180af8c0 2022-07-21T01:57:54.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.515+0000 7f860ffff700 1 -- 172.21.15.140:0/1589947515 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8608021af0 con 0x7f86180af8c0 2022-07-21T01:57:54.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.519+0000 7f8617fff700 1 -- 172.21.15.140:0/1589947515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861810e1f0 con 0x7f86180af8c0 2022-07-21T01:57:54.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.519+0000 7f8617fff700 1 -- 172.21.15.140:0/1589947515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8618159de0 con 0x7f86180af8c0 2022-07-21T01:57:54.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.519+0000 7f860ffff700 1 -- 172.21.15.140:0/1589947515 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8608027020 con 0x7f86180af8c0 2022-07-21T01:57:54.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.519+0000 7f860ffff700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85fc053b00 0x7f85fc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:54.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.519+0000 7f860ffff700 1 -- 172.21.15.140:0/1589947515 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f860806d860 con 0x7f86180af8c0 2022-07-21T01:57:54.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.519+0000 7f8616ffd700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85fc053b00 0x7f85fc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:54.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.523+0000 7f8616ffd700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85fc053b00 0x7f85fc055fc0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f860000b000 tx=0x7f8600008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:54.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.527+0000 7f860ffff700 1 -- 172.21.15.140:0/1589947515 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f86080447e0 con 0x7f86180af8c0 2022-07-21T01:57:54.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:54.691+0000 7f8617fff700 1 -- 172.21.15.140:0/1589947515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f86180acae0 con 0x7f86180af8c0 2022-07-21T01:57:55.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.115+0000 7f860ffff700 1 -- 172.21.15.140:0/1589947515 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8608047e00 con 0x7f86180af8c0 2022-07-21T01:57:55.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 -- 172.21.15.140:0/1589947515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85fc053b00 msgr2=0x7f85fc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:55.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85fc053b00 0x7f85fc055fc0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f860000b000 tx=0x7f8600008040).stop 2022-07-21T01:57:55.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 -- 172.21.15.140:0/1589947515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86180af8c0 msgr2=0x7f8618159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:55.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86180af8c0 0x7f8618159540 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f8618159b70 tx=0x7f8608016000).stop 2022-07-21T01:57:55.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 -- 172.21.15.140:0/1589947515 shutdown_connections 2022-07-21T01:57:55.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85fc053b00 0x7f85fc055fc0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:55.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86180b5970 0x7f8618111410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:55.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f86180af8c0 0x7f8618159540 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:55.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 --2- 172.21.15.140:0/1589947515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f86180aeae0 0x7f8618149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:55.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 -- 172.21.15.140:0/1589947515 >> 172.21.15.140:0/1589947515 conn(0x7f8618063490 msgr2=0x7f8618147300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:55.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 -- 172.21.15.140:0/1589947515 shutdown_connections 2022-07-21T01:57:55.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.123+0000 7f8617fff700 1 -- 172.21.15.140:0/1589947515 wait complete. 2022-07-21T01:57:55.140 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:57:55.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.255+0000 7f7e0a7b8700 1 Processor -- start 2022-07-21T01:57:55.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.255+0000 7f7e0a7b8700 1 -- start start 2022-07-21T01:57:55.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b3cf0 0x7f7e04146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:55.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7e04146530 0x7f7e04141a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:55.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e04066b10 con 0x7f7e040b6840 2022-07-21T01:57:55.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e040b5c90 con 0x7f7e04146530 2022-07-21T01:57:55.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040b6c20 con 0x7f7e040b3cf0 2022-07-21T01:57:55.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e097b6700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f7e040b6840 0x7f7e040b5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55598/0 (socket says 172.21.15.140:55598) 2022-07-21T01:57:55.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e097b6700 1 -- 172.21.15.140:0/2417460933 learned_addr learned my addr 172.21.15.140:0/2417460933 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:55.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e09fb7700 1 --2- 172.21.15.140:0/2417460933 >> v2:172.21.15.152:3300/0 conn(0x7f7e04146530 0x7f7e04141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:55.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e08fb5700 1 --2- 172.21.15.140:0/2417460933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b3cf0 0x7f7e04146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:55.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7dfbfff700 1 -- 172.21.15.140:0/2417460933 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926349027 0 0) 0x7f7e04066b10 con 0x7f7e040b6840 2022-07-21T01:57:55.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7dfbfff700 1 -- 172.21.15.140:0/2417460933 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de8003260 con 0x7f7e040b6840 2022-07-21T01:57:55.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e09fb7700 1 -- 172.21.15.140:0/2417460933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b3cf0 msgr2=0x7f7e04146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:55.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e09fb7700 1 --2- 172.21.15.140:0/2417460933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b3cf0 0x7f7e04146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:55.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e09fb7700 1 -- 172.21.15.140:0/2417460933 >> v1:172.21.15.140:6789/0 conn(0x7f7e040b6840 legacy=0x7f7e040b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:55.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e09fb7700 1 -- 172.21.15.140:0/2417460933 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e040aead0 con 0x7f7e04146530 2022-07-21T01:57:55.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e08fb5700 1 --2- 172.21.15.140:0/2417460933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b3cf0 0x7f7e04146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:55.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e09fb7700 1 --2- 172.21.15.140:0/2417460933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e04146530 0x7f7e04141a90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f7dfc0032f0 tx=0x7f7dfc0091d0).ready entity=mon.1 client_cookie=d6950f89ce0d887d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:55.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7dfbfff700 1 -- 172.21.15.140:0/2417460933 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dfc00b600 con 0x7f7e04146530 2022-07-21T01:57:55.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7dfbfff700 1 -- 172.21.15.140:0/2417460933 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc00b760 con 0x7f7e04146530 2022-07-21T01:57:55.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7dfbfff700 1 -- 172.21.15.140:0/2417460933 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dfc00b8d0 con 0x7f7e04146530 2022-07-21T01:57:55.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/2417460933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e04146530 msgr2=0x7f7e04141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:55.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 --2- 172.21.15.140:0/2417460933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e04146530 0x7f7e04141a90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f7dfc0032f0 tx=0x7f7dfc0091d0).stop 2022-07-21T01:57:55.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/2417460933 shutdown_connections 2022-07-21T01:57:55.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 --2- 172.21.15.140:0/2417460933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e04146530 0x7f7e04141a90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:55.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 --2- 172.21.15.140:0/2417460933 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b3cf0 0x7f7e04146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:55.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/2417460933 >> 172.21.15.140:0/2417460933 conn(0x7f7e040b9e10 msgr2=0x7f7e040b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:55.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/2417460933 shutdown_connections 2022-07-21T01:57:55.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/2417460933 wait complete. 2022-07-21T01:57:55.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 Processor -- start 2022-07-21T01:57:55.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.259+0000 7f7e0a7b8700 1 -- start start 2022-07-21T01:57:55.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e0a7b8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e040b3cf0 0x7f7e04112910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:55.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e0a7b8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b6840 0x7f7e040aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:55.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e0a7b8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7e04146530 0x7f7e040b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:55.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e0a7b8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e041525b0 con 0x7f7e04146530 2022-07-21T01:57:55.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e0a7b8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040b3180 con 0x7f7e040b3cf0 2022-07-21T01:57:55.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e0a7b8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040b3420 con 0x7f7e040b6840 2022-07-21T01:57:55.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e09fb7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7e04146530 0x7f7e040b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:55.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e09fb7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7e04146530 0x7f7e040b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41462/0 (socket says 172.21.15.140:41462) 2022-07-21T01:57:55.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e09fb7700 1 -- 172.21.15.140:0/3473162705 learned_addr learned my addr 172.21.15.140:0/3473162705 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:55.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e097b6700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e040b3cf0 0x7f7e04112910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:55.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e08fb5700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b6840 0x7f7e040aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:55.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e09fb7700 1 -- 172.21.15.140:0/3473162705 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b6840 msgr2=0x7f7e040aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:55.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e09fb7700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b6840 0x7f7e040aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:55.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e09fb7700 1 -- 172.21.15.140:0/3473162705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e040b3cf0 msgr2=0x7f7e04112910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:55.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e09fb7700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e040b3cf0 0x7f7e04112910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:55.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e09fb7700 1 -- 172.21.15.140:0/3473162705 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0000b180 con 0x7f7e04146530 2022-07-21T01:57:55.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e08fb5700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b6840 0x7f7e040aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:55.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e097b6700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e040b3cf0 0x7f7e04112910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:55.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e09fb7700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7e04146530 0x7f7e040b03f0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f7dfc0032f0 tx=0x7f7dfc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:55.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7dfa7fc700 1 -- 172.21.15.140:0/3473162705 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dfc00b8d0 con 0x7f7e04146530 2022-07-21T01:57:55.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7dfa7fc700 1 -- 172.21.15.140:0/3473162705 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc021670 con 0x7f7e04146530 2022-07-21T01:57:55.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7dfa7fc700 1 -- 172.21.15.140:0/3473162705 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7dfc00b600 con 0x7f7e04146530 2022-07-21T01:57:55.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/3473162705 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dfc009470 con 0x7f7e04146530 2022-07-21T01:57:55.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/3473162705 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e040b08e0 con 0x7f7e04146530 2022-07-21T01:57:55.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7dfa7fc700 1 -- 172.21.15.140:0/3473162705 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7dfc028070 con 0x7f7e04146530 2022-07-21T01:57:55.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.263+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/3473162705 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e040aed70 con 0x7f7e04146530 2022-07-21T01:57:55.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.267+0000 7f7dfa7fc700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7dec053a70 0x7f7dec055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:55.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.267+0000 7f7dfa7fc700 1 -- 172.21.15.140:0/3473162705 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7dfc06d080 con 0x7f7e04146530 2022-07-21T01:57:55.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.267+0000 7f7e097b6700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7dec053a70 0x7f7dec055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:55.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.267+0000 7f7e097b6700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7dec053a70 0x7f7dec055f30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f7df400d000 tx=0x7f7df4009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:55.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.271+0000 7f7dfa7fc700 1 -- 172.21.15.140:0/3473162705 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7dfc043dc0 con 0x7f7e04146530 2022-07-21T01:57:55.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:55.395+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/3473162705 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7e040574e0 con 0x7f7e04146530 2022-07-21T01:57:56.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.127+0000 7f7dfa7fc700 1 -- 172.21.15.140:0/3473162705 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v144) v1 ==== 185+0+0 (secure 0 0 0) 0x7f7dfc015070 con 0x7f7e04146530 2022-07-21T01:57:56.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.131+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/3473162705 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7dec053a70 msgr2=0x7f7dec055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:56.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.131+0000 7f7e0a7b8700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7dec053a70 0x7f7dec055f30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f7df400d000 tx=0x7f7df4009040).stop 2022-07-21T01:57:56.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.131+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/3473162705 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7e04146530 msgr2=0x7f7e040b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:56.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.131+0000 7f7e0a7b8700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7e04146530 0x7f7e040b03f0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7f7dfc0032f0 tx=0x7f7dfc016000).stop 2022-07-21T01:57:56.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.131+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/3473162705 shutdown_connections 2022-07-21T01:57:56.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.131+0000 7f7e0a7b8700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7dec053a70 0x7f7dec055f30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:56.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.131+0000 7f7e0a7b8700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7e04146530 0x7f7e040b03f0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:56.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.131+0000 7f7e0a7b8700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e040b6840 0x7f7e040aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:56.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.131+0000 7f7e0a7b8700 1 --2- 172.21.15.140:0/3473162705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e040b3cf0 0x7f7e04112910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:56.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.131+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/3473162705 >> 172.21.15.140:0/3473162705 conn(0x7f7e040b9e10 msgr2=0x7f7e04141a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:56.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.131+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/3473162705 shutdown_connections 2022-07-21T01:57:56.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.135+0000 7f7e0a7b8700 1 -- 172.21.15.140:0/3473162705 wait complete. 2022-07-21T01:57:56.139 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:57:56.149 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:57:56.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.255+0000 7f993c98a700 1 Processor -- start 2022-07-21T01:57:56.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.255+0000 7f993c98a700 1 -- start start 2022-07-21T01:57:56.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f993c98a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f99380aecf0 0x7f99380af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:56.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f993c98a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9938065040 con 0x7f99380b57f0 2022-07-21T01:57:56.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f993c98a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993805bf10 con 0x7f993805c3b0 2022-07-21T01:57:56.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f993c98a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9938064d70 con 0x7f99380aecf0 2022-07-21T01:57:56.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f99377fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f99380b57f0 0x7f9938143d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55612/0 (socket says 172.21.15.140:55612) 2022-07-21T01:57:56.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f99377fe700 1 -- 172.21.15.140:0/2591494710 learned_addr learned my addr 172.21.15.140:0/2591494710 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:56.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f992ffff700 1 --2- 172.21.15.140:0/2591494710 >> v2:172.21.15.152:3300/0 conn(0x7f99380aecf0 0x7f99380af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:56.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f9936ffd700 1 -- 172.21.15.140:0/2591494710 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821631660 0 0) 0x7f9938065040 con 0x7f99380b57f0 2022-07-21T01:57:56.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f9936ffd700 1 -- 172.21.15.140:0/2591494710 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9918003260 con 0x7f99380b57f0 2022-07-21T01:57:56.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f992ffff700 1 -- 172.21.15.140:0/2591494710 >> v1:172.21.15.152:6789/0 conn(0x7f993805c3b0 legacy=0x7f993813a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:56.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f992ffff700 1 -- 172.21.15.140:0/2591494710 >> v1:172.21.15.140:6789/0 conn(0x7f99380b57f0 legacy=0x7f9938143d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:56.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f992ffff700 1 -- 172.21.15.140:0/2591494710 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99380b5bd0 con 0x7f99380aecf0 2022-07-21T01:57:56.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f992ffff700 1 --2- 172.21.15.140:0/2591494710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99380aecf0 0x7f99380af270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f9920002f70 tx=0x7f9920006df0).ready entity=mon.1 client_cookie=6e1c2d73bf68f713 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:56.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f9936ffd700 1 -- 172.21.15.140:0/2591494710 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f992000d610 con 0x7f99380aecf0 2022-07-21T01:57:56.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f9936ffd700 1 -- 172.21.15.140:0/2591494710 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9920000e90 con 0x7f99380aecf0 2022-07-21T01:57:56.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f9936ffd700 1 -- 172.21.15.140:0/2591494710 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f992000d960 con 0x7f99380aecf0 2022-07-21T01:57:56.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.259+0000 7f993c98a700 1 -- 172.21.15.140:0/2591494710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99380aecf0 msgr2=0x7f99380af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:56.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.263+0000 7f993c98a700 1 --2- 172.21.15.140:0/2591494710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99380aecf0 0x7f99380af270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f9920002f70 tx=0x7f9920006df0).stop 2022-07-21T01:57:56.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.263+0000 7f993c98a700 1 -- 172.21.15.140:0/2591494710 shutdown_connections 2022-07-21T01:57:56.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.263+0000 7f993c98a700 1 --2- 172.21.15.140:0/2591494710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99380aecf0 0x7f99380af270 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:56.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.263+0000 7f993c98a700 1 -- 172.21.15.140:0/2591494710 >> 172.21.15.140:0/2591494710 conn(0x7f9938063480 msgr2=0x7f99380b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:56.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.263+0000 7f993c98a700 1 -- 172.21.15.140:0/2591494710 shutdown_connections 2022-07-21T01:57:56.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.263+0000 7f993c98a700 1 -- 172.21.15.140:0/2591494710 wait complete. 2022-07-21T01:57:56.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.263+0000 7f993c98a700 1 Processor -- start 2022-07-21T01:57:56.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.263+0000 7f993c98a700 1 -- start start 2022-07-21T01:57:56.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.263+0000 7f993c98a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f993805c3b0 0x7f993813d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:56.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.263+0000 7f993c98a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99380aecf0 0x7f9938150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:56.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f993c98a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99380b57f0 0x7f9938153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:56.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f993c98a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9938143ce0 con 0x7f993805c3b0 2022-07-21T01:57:56.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f993c98a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9938144010 con 0x7f99380aecf0 2022-07-21T01:57:56.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f993c98a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9938153ad0 con 0x7f99380b57f0 2022-07-21T01:57:56.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99377fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f993805c3b0 0x7f993813d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:56.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99377fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f993805c3b0 0x7f993813d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41470/0 (socket says 172.21.15.140:41470) 2022-07-21T01:57:56.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99377fe700 1 -- 172.21.15.140:0/106210745 learned_addr learned my addr 172.21.15.140:0/106210745 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:56.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f992ffff700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99380aecf0 0x7f9938150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:56.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f9937fff700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99380b57f0 0x7f9938153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:56.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99377fe700 1 -- 172.21.15.140:0/106210745 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99380b57f0 msgr2=0x7f9938153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:56.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99377fe700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99380b57f0 0x7f9938153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:56.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99377fe700 1 -- 172.21.15.140:0/106210745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99380aecf0 msgr2=0x7f9938150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:56.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99377fe700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99380aecf0 0x7f9938150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:56.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99377fe700 1 -- 172.21.15.140:0/106210745 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992000b180 con 0x7f993805c3b0 2022-07-21T01:57:56.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f992ffff700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99380aecf0 0x7f9938150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:56.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99377fe700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f993805c3b0 0x7f993813d520 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f9928004870 tx=0x7f9928007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:56.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99357fa700 1 -- 172.21.15.140:0/106210745 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f992800ce20 con 0x7f993805c3b0 2022-07-21T01:57:56.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99357fa700 1 -- 172.21.15.140:0/106210745 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9928000e90 con 0x7f993805c3b0 2022-07-21T01:57:56.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f993c98a700 1 -- 172.21.15.140:0/106210745 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9938150840 con 0x7f993805c3b0 2022-07-21T01:57:56.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f99357fa700 1 -- 172.21.15.140:0/106210745 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f99280072c0 con 0x7f993805c3b0 2022-07-21T01:57:56.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.267+0000 7f993c98a700 1 -- 172.21.15.140:0/106210745 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9938144a60 con 0x7f993805c3b0 2022-07-21T01:57:56.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.271+0000 7f993c98a700 1 -- 172.21.15.140:0/106210745 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99381538e0 con 0x7f993805c3b0 2022-07-21T01:57:56.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.271+0000 7f99357fa700 1 -- 172.21.15.140:0/106210745 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9928005610 con 0x7f993805c3b0 2022-07-21T01:57:56.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.271+0000 7f99357fa700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f991c053aa0 0x7f991c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:56.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.271+0000 7f99357fa700 1 -- 172.21.15.140:0/106210745 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f992806b580 con 0x7f993805c3b0 2022-07-21T01:57:56.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.271+0000 7f992ffff700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f991c053aa0 0x7f991c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:56.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.271+0000 7f992ffff700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f991c053aa0 0x7f991c055f60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f992001a000 tx=0x7f9920014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:56.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.279+0000 7f99357fa700 1 -- 172.21.15.140:0/106210745 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f99280425a0 con 0x7f993805c3b0 2022-07-21T01:57:56.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:56.403+0000 7f993c98a700 1 -- 172.21.15.140:0/106210745 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99381441d0 con 0x7f993805c3b0 2022-07-21T01:57:57.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.135+0000 7f99357fa700 1 -- 172.21.15.140:0/106210745 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v145) v1 ==== 173+0+0 (secure 0 0 0) 0x7f9928007bc0 con 0x7f993805c3b0 2022-07-21T01:57:57.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 -- 172.21.15.140:0/106210745 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f991c053aa0 msgr2=0x7f991c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f991c053aa0 0x7f991c055f60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f992001a000 tx=0x7f9920014000).stop 2022-07-21T01:57:57.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 -- 172.21.15.140:0/106210745 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f993805c3b0 msgr2=0x7f993813d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f993805c3b0 0x7f993813d520 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f9928004870 tx=0x7f9928007a80).stop 2022-07-21T01:57:57.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 -- 172.21.15.140:0/106210745 shutdown_connections 2022-07-21T01:57:57.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f991c053aa0 0x7f991c055f60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f99380b57f0 0x7f9938153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99380aecf0 0x7f9938150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 --2- 172.21.15.140:0/106210745 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f993805c3b0 0x7f993813d520 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 -- 172.21.15.140:0/106210745 >> 172.21.15.140:0/106210745 conn(0x7f9938063480 msgr2=0x7f9938139f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:57.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 -- 172.21.15.140:0/106210745 shutdown_connections 2022-07-21T01:57:57.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.139+0000 7f993c98a700 1 -- 172.21.15.140:0/106210745 wait complete. 2022-07-21T01:57:57.145 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:57:57.152 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2022-07-21T01:57:57.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.251+0000 7f878ffff700 1 Processor -- start 2022-07-21T01:57:57.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.251+0000 7f878ffff700 1 -- start start 2022-07-21T01:57:57.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8790140060 0x7f87900ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87900b4af0 0x7f87900b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878ffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87900677d0 con 0x7f87900b6200 2022-07-21T01:57:57.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878ffff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87900b65e0 con 0x7f8790140060 2022-07-21T01:57:57.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87900aca50 con 0x7f87900b4af0 2022-07-21T01:57:57.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878effd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f87900b6200 0x7f879013f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55624/0 (socket says 172.21.15.140:55624) 2022-07-21T01:57:57.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878effd700 1 -- 172.21.15.140:0/3236626678 learned_addr learned my addr 172.21.15.140:0/3236626678 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:57.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f87867fc700 1 --2- 172.21.15.140:0/3236626678 >> v2:172.21.15.152:3300/0 conn(0x7f8790140060 0x7f87900ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878f7fe700 1 --2- 172.21.15.140:0/3236626678 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87900b4af0 0x7f87900b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878e7fc700 1 -- 172.21.15.140:0/3236626678 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377994416 0 0) 0x7f87900677d0 con 0x7f87900b6200 2022-07-21T01:57:57.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878e7fc700 1 -- 172.21.15.140:0/3236626678 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8770003260 con 0x7f87900b6200 2022-07-21T01:57:57.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f87867fc700 1 -- 172.21.15.140:0/3236626678 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87900b4af0 msgr2=0x7f87900b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:57.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f87867fc700 1 --2- 172.21.15.140:0/3236626678 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87900b4af0 0x7f87900b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f87867fc700 1 -- 172.21.15.140:0/3236626678 >> v1:172.21.15.140:6789/0 conn(0x7f87900b6200 legacy=0x7f879013f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f87867fc700 1 -- 172.21.15.140:0/3236626678 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8790067530 con 0x7f8790140060 2022-07-21T01:57:57.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f87867fc700 1 --2- 172.21.15.140:0/3236626678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8790140060 0x7f87900ac560 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f8778002f70 tx=0x7f8778006df0).ready entity=mon.1 client_cookie=6937f2060b067a5 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:57.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878e7fc700 1 -- 172.21.15.140:0/3236626678 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f877800d4a0 con 0x7f8790140060 2022-07-21T01:57:57.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878e7fc700 1 -- 172.21.15.140:0/3236626678 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8778000e90 con 0x7f8790140060 2022-07-21T01:57:57.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878e7fc700 1 -- 172.21.15.140:0/3236626678 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f877800d7f0 con 0x7f8790140060 2022-07-21T01:57:57.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.255+0000 7f878ffff700 1 -- 172.21.15.140:0/3236626678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8790140060 msgr2=0x7f87900ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.259+0000 7f878ffff700 1 --2- 172.21.15.140:0/3236626678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8790140060 0x7f87900ac560 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f8778002f70 tx=0x7f8778006df0).stop 2022-07-21T01:57:57.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.259+0000 7f878ffff700 1 -- 172.21.15.140:0/3236626678 shutdown_connections 2022-07-21T01:57:57.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.259+0000 7f878ffff700 1 --2- 172.21.15.140:0/3236626678 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87900b4af0 0x7f87900b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.259+0000 7f878ffff700 1 --2- 172.21.15.140:0/3236626678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8790140060 0x7f87900ac560 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.259+0000 7f878ffff700 1 -- 172.21.15.140:0/3236626678 >> 172.21.15.140:0/3236626678 conn(0x7f8790063d10 msgr2=0x7f87900b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:57.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.259+0000 7f878ffff700 1 -- 172.21.15.140:0/3236626678 shutdown_connections 2022-07-21T01:57:57.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.259+0000 7f878ffff700 1 -- 172.21.15.140:0/3236626678 wait complete. 2022-07-21T01:57:57.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.259+0000 7f878ffff700 1 Processor -- start 2022-07-21T01:57:57.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.259+0000 7f878ffff700 1 -- start start 2022-07-21T01:57:57.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b4af0 0x7f87900b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87900b6200 0x7f879014db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8790140060 0x7f879014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87900b0e20 con 0x7f8790140060 2022-07-21T01:57:57.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8790151fa0 con 0x7f87900b4af0 2022-07-21T01:57:57.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87900641a0 con 0x7f87900b6200 2022-07-21T01:57:57.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8790140060 0x7f879014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8790140060 0x7f879014e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41488/0 (socket says 172.21.15.140:41488) 2022-07-21T01:57:57.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878f7fe700 1 -- 172.21.15.140:0/3266645245 learned_addr learned my addr 172.21.15.140:0/3266645245 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:57.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878effd700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b4af0 0x7f87900b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f87867fc700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87900b6200 0x7f879014db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878f7fe700 1 -- 172.21.15.140:0/3266645245 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87900b6200 msgr2=0x7f879014db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878f7fe700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87900b6200 0x7f879014db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878f7fe700 1 -- 172.21.15.140:0/3266645245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b4af0 msgr2=0x7f87900b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878f7fe700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b4af0 0x7f87900b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878f7fe700 1 -- 172.21.15.140:0/3266645245 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877800b180 con 0x7f8790140060 2022-07-21T01:57:57.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878f7fe700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8790140060 0x7f879014e080 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f8788006e00 tx=0x7f878800b470).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:57.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878cff9700 1 -- 172.21.15.140:0/3266645245 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f878801a470 con 0x7f8790140060 2022-07-21T01:57:57.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878cff9700 1 -- 172.21.15.140:0/3266645245 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878801a5d0 con 0x7f8790140060 2022-07-21T01:57:57.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878ffff700 1 -- 172.21.15.140:0/3266645245 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8790064440 con 0x7f8790140060 2022-07-21T01:57:57.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878cff9700 1 -- 172.21.15.140:0/3266645245 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f878801a930 con 0x7f8790140060 2022-07-21T01:57:57.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878ffff700 1 -- 172.21.15.140:0/3266645245 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8790141ed0 con 0x7f8790140060 2022-07-21T01:57:57.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878ffff700 1 -- 172.21.15.140:0/3266645245 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87900645d0 con 0x7f8790140060 2022-07-21T01:57:57.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878cff9700 1 -- 172.21.15.140:0/3266645245 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8788021060 con 0x7f8790140060 2022-07-21T01:57:57.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878cff9700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87740539e0 0x7f8774055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878cff9700 1 -- 172.21.15.140:0/3266645245 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f878806ee90 con 0x7f8790140060 2022-07-21T01:57:57.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.263+0000 7f878effd700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87740539e0 0x7f8774055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.267+0000 7f878effd700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87740539e0 0x7f8774055ea0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f878000d000 tx=0x7f8780009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:57.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.271+0000 7f878cff9700 1 -- 172.21.15.140:0/3266645245 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8788046090 con 0x7f8790140060 2022-07-21T01:57:57.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.395+0000 7f878ffff700 1 -- 172.21.15.140:0/3266645245 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f87900b5220 con 0x7f8790140060 2022-07-21T01:57:57.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.403+0000 7f878cff9700 1 -- 172.21.15.140:0/3266645245 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f878801f020 con 0x7f8790140060 2022-07-21T01:57:57.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 -- 172.21.15.140:0/3266645245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87740539e0 msgr2=0x7f8774055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87740539e0 0x7f8774055ea0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f878000d000 tx=0x7f8780009040).stop 2022-07-21T01:57:57.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 -- 172.21.15.140:0/3266645245 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8790140060 msgr2=0x7f879014e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8790140060 0x7f879014e080 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f8788006e00 tx=0x7f878800b470).stop 2022-07-21T01:57:57.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 -- 172.21.15.140:0/3266645245 shutdown_connections 2022-07-21T01:57:57.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f87740539e0 0x7f8774055ea0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8790140060 0x7f879014e080 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f87900b6200 0x7f879014db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 --2- 172.21.15.140:0/3266645245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87900b4af0 0x7f87900b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 -- 172.21.15.140:0/3266645245 >> 172.21.15.140:0/3266645245 conn(0x7f8790063d10 msgr2=0x7f879013f950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:57.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 -- 172.21.15.140:0/3266645245 shutdown_connections 2022-07-21T01:57:57.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.407+0000 7f878ffff700 1 -- 172.21.15.140:0/3266645245 wait complete. 2022-07-21T01:57:57.421 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2022-07-21T01:57:57.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.519+0000 7f052359e700 1 Processor -- start 2022-07-21T01:57:57.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.519+0000 7f052359e700 1 -- start start 2022-07-21T01:57:57.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.519+0000 7f052359e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f05240b4440 0x7f0524143590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.519+0000 7f052359e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0524145f90 0x7f052413fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.519+0000 7f052359e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05240b6c60 con 0x7f05240b6880 2022-07-21T01:57:57.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.519+0000 7f052359e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05241429c0 con 0x7f0524145f90 2022-07-21T01:57:57.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.519+0000 7f052359e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0524142bb0 con 0x7f05240b4440 2022-07-21T01:57:57.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.519+0000 7f0522d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f0524145f90 0x7f052413fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.519+0000 7f052259c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f05240b6880 0x7f05240b3d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55638/0 (socket says 172.21.15.140:55638) 2022-07-21T01:57:57.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.519+0000 7f052259c700 1 -- 172.21.15.140:0/2373414816 learned_addr learned my addr 172.21.15.140:0/2373414816 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:57.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.519+0000 7f0519d9b700 1 --2- 172.21.15.140:0/2373414816 >> v2:172.21.15.152:3300/0 conn(0x7f05240b4440 0x7f0524143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f0522d9d700 1 -- 172.21.15.140:0/2373414816 >> v2:172.21.15.152:3300/0 conn(0x7f05240b4440 msgr2=0x7f0524143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:57.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f0522d9d700 1 --2- 172.21.15.140:0/2373414816 >> v2:172.21.15.152:3300/0 conn(0x7f05240b4440 0x7f0524143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f0522d9d700 1 -- 172.21.15.140:0/2373414816 >> v1:172.21.15.140:6789/0 conn(0x7f05240b6880 legacy=0x7f05240b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f0522d9d700 1 -- 172.21.15.140:0/2373414816 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524142e50 con 0x7f0524145f90 2022-07-21T01:57:57.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f0522d9d700 1 --2- 172.21.15.140:0/2373414816 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0524145f90 0x7f052413fa00 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f051c0032f0 tx=0x7f051c0091d0).ready entity=mon.0 client_cookie=d5e0e28566ef36f7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:57.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f0521d9b700 1 -- 172.21.15.140:0/2373414816 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f051c00b680 con 0x7f0524145f90 2022-07-21T01:57:57.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f0521d9b700 1 -- 172.21.15.140:0/2373414816 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c00b7e0 con 0x7f0524145f90 2022-07-21T01:57:57.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f0521d9b700 1 -- 172.21.15.140:0/2373414816 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f051c00b950 con 0x7f0524145f90 2022-07-21T01:57:57.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f052359e700 1 -- 172.21.15.140:0/2373414816 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0524145f90 msgr2=0x7f052413fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f052359e700 1 --2- 172.21.15.140:0/2373414816 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0524145f90 0x7f052413fa00 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f051c0032f0 tx=0x7f051c0091d0).stop 2022-07-21T01:57:57.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f052359e700 1 -- 172.21.15.140:0/2373414816 shutdown_connections 2022-07-21T01:57:57.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f052359e700 1 --2- 172.21.15.140:0/2373414816 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0524145f90 0x7f052413fa00 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f052359e700 1 --2- 172.21.15.140:0/2373414816 >> v2:172.21.15.152:3300/0 conn(0x7f05240b4440 0x7f0524143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f052359e700 1 -- 172.21.15.140:0/2373414816 >> 172.21.15.140:0/2373414816 conn(0x7f05240b9e60 msgr2=0x7f05240b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:57.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f052359e700 1 -- 172.21.15.140:0/2373414816 shutdown_connections 2022-07-21T01:57:57.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.523+0000 7f052359e700 1 -- 172.21.15.140:0/2373414816 wait complete. 2022-07-21T01:57:57.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052359e700 1 Processor -- start 2022-07-21T01:57:57.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052359e700 1 -- start start 2022-07-21T01:57:57.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05240b4440 0x7f05241529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052359e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05240b6880 0x7f052405c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0524145f90 0x7f05240b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052359e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240aedb0 con 0x7f05240b4440 2022-07-21T01:57:57.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052359e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0524151e10 con 0x7f05240b6880 2022-07-21T01:57:57.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0524152000 con 0x7f0524145f90 2022-07-21T01:57:57.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05240b4440 0x7f05241529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05240b4440 0x7f05241529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41498/0 (socket says 172.21.15.140:41498) 2022-07-21T01:57:57.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052259c700 1 -- 172.21.15.140:0/4065170047 learned_addr learned my addr 172.21.15.140:0/4065170047 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:57.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f0519d9b700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05240b6880 0x7f052405c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f0522d9d700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0524145f90 0x7f05240b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052259c700 1 -- 172.21.15.140:0/4065170047 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0524145f90 msgr2=0x7f05240b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052259c700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0524145f90 0x7f05240b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052259c700 1 -- 172.21.15.140:0/4065170047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05240b6880 msgr2=0x7f052405c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:57.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052259c700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05240b6880 0x7f052405c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:57.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052259c700 1 -- 172.21.15.140:0/4065170047 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c009470 con 0x7f05240b4440 2022-07-21T01:57:57.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f0522d9d700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0524145f90 0x7f05240b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:57:57.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052259c700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05240b4440 0x7f05241529e0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f0524066470 tx=0x7f0514007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:57.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f051bfff700 1 -- 172.21.15.140:0/4065170047 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f05140070d0 con 0x7f05240b4440 2022-07-21T01:57:57.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f051bfff700 1 -- 172.21.15.140:0/4065170047 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0514007230 con 0x7f05240b4440 2022-07-21T01:57:57.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052359e700 1 -- 172.21.15.140:0/4065170047 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05241522a0 con 0x7f05240b4440 2022-07-21T01:57:57.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f051bfff700 1 -- 172.21.15.140:0/4065170047 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0514005610 con 0x7f05240b4440 2022-07-21T01:57:57.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.527+0000 7f052359e700 1 -- 172.21.15.140:0/4065170047 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052405c6d0 con 0x7f05240b4440 2022-07-21T01:57:57.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.531+0000 7f052359e700 1 -- 172.21.15.140:0/4065170047 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05241430f0 con 0x7f05240b4440 2022-07-21T01:57:57.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.531+0000 7f051bfff700 1 -- 172.21.15.140:0/4065170047 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f0514010070 con 0x7f05240b4440 2022-07-21T01:57:57.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.531+0000 7f051bfff700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0508053b00 0x7f0508055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:57.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.531+0000 7f051bfff700 1 -- 172.21.15.140:0/4065170047 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f051406b130 con 0x7f05240b4440 2022-07-21T01:57:57.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.531+0000 7f0519d9b700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0508053b00 0x7f0508055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:57.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.531+0000 7f0519d9b700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0508053b00 0x7f0508055fc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f050c00b000 tx=0x7f050c006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:57.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.535+0000 7f051bfff700 1 -- 172.21.15.140:0/4065170047 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0514042330 con 0x7f05240b4440 2022-07-21T01:57:57.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:57.699+0000 7f052359e700 1 -- 172.21.15.140:0/4065170047 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f05240574e0 con 0x7f05240b4440 2022-07-21T01:57:58.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.143+0000 7f051bfff700 1 -- 172.21.15.140:0/4065170047 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v442) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0514045950 con 0x7f05240b4440 2022-07-21T01:57:58.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.147+0000 7f052359e700 1 -- 172.21.15.140:0/4065170047 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0508053b00 msgr2=0x7f0508055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:58.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.147+0000 7f052359e700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0508053b00 0x7f0508055fc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f050c00b000 tx=0x7f050c006df0).stop 2022-07-21T01:57:58.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.147+0000 7f052359e700 1 -- 172.21.15.140:0/4065170047 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05240b4440 msgr2=0x7f05241529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:58.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.147+0000 7f052359e700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05240b4440 0x7f05241529e0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f0524066470 tx=0x7f0514007c00).stop 2022-07-21T01:57:58.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.147+0000 7f052359e700 1 -- 172.21.15.140:0/4065170047 shutdown_connections 2022-07-21T01:57:58.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.147+0000 7f052359e700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0508053b00 0x7f0508055fc0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:58.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.147+0000 7f052359e700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0524145f90 0x7f05240b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:58.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.147+0000 7f052359e700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05240b6880 0x7f052405c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:58.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.147+0000 7f052359e700 1 --2- 172.21.15.140:0/4065170047 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05240b4440 0x7f05241529e0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:58.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.147+0000 7f052359e700 1 -- 172.21.15.140:0/4065170047 >> 172.21.15.140:0/4065170047 conn(0x7f05240b9e60 msgr2=0x7f052413aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:58.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.147+0000 7f052359e700 1 -- 172.21.15.140:0/4065170047 shutdown_connections 2022-07-21T01:57:58.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.151+0000 7f052359e700 1 -- 172.21.15.140:0/4065170047 wait complete. 2022-07-21T01:57:58.168 INFO:tasks.cephfs_test_runner:test_root_rctime (tasks.cephfs.test_misc.TestMisc) ... ok 2022-07-21T01:57:58.169 INFO:tasks.cephfs_test_runner:Starting test: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-07-21T01:57:58.169 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs' 2022-07-21T01:57:58.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.271+0000 7fc0b34ea700 1 Processor -- start 2022-07-21T01:57:58.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.271+0000 7fc0b34ea700 1 -- start start 2022-07-21T01:57:58.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc0ac0b2ec0 0x7fc0ac0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:58.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc0ac1461c0 0x7fc0ac146630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:58.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ac0b3f10 con 0x7fc0ac05d160 2022-07-21T01:57:58.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ac0b4ea0 con 0x7fc0ac0b2ec0 2022-07-21T01:57:58.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0b3790 con 0x7fc0ac1461c0 2022-07-21T01:57:58.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b1ce7700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc0ac05d160 0x7fc0ac142210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55652/0 (socket says 172.21.15.140:55652) 2022-07-21T01:57:58.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b1ce7700 1 -- 172.21.15.140:0/3118512973 learned_addr learned my addr 172.21.15.140:0/3118512973 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:58.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b24e8700 1 --2- 172.21.15.140:0/3118512973 >> v2:172.21.15.140:3300/0 conn(0x7fc0ac0b2ec0 0x7fc0ac0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:58.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b2ce9700 1 --2- 172.21.15.140:0/3118512973 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc0ac1461c0 0x7fc0ac146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:58.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b14e6700 1 -- 172.21.15.140:0/3118512973 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241764816 0 0) 0x7fc0ac0b3f10 con 0x7fc0ac05d160 2022-07-21T01:57:58.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b14e6700 1 -- 172.21.15.140:0/3118512973 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc094003260 con 0x7fc0ac05d160 2022-07-21T01:57:58.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b24e8700 1 -- 172.21.15.140:0/3118512973 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc0ac1461c0 msgr2=0x7fc0ac146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:58.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b24e8700 1 --2- 172.21.15.140:0/3118512973 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc0ac1461c0 0x7fc0ac146630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:58.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b24e8700 1 -- 172.21.15.140:0/3118512973 >> v1:172.21.15.140:6789/0 conn(0x7fc0ac05d160 legacy=0x7fc0ac142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:58.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b14e6700 1 -- 172.21.15.140:0/3118512973 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1123248835 0 0) 0x7fc094003260 con 0x7fc0ac05d160 2022-07-21T01:57:58.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b24e8700 1 -- 172.21.15.140:0/3118512973 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac0b2620 con 0x7fc0ac0b2ec0 2022-07-21T01:57:58.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b24e8700 1 --2- 172.21.15.140:0/3118512973 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0ac0b2ec0 0x7fc0ac0b32a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7fc09c001800 tx=0x7fc0a8006df0).ready entity=mon.0 client_cookie=263dd49131a64f70 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:58.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b14e6700 1 -- 172.21.15.140:0/3118512973 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc09c001660 con 0x7fc0ac0b2ec0 2022-07-21T01:57:58.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b14e6700 1 -- 172.21.15.140:0/3118512973 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a8000e90 con 0x7fc0ac0b2ec0 2022-07-21T01:57:58.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b14e6700 1 -- 172.21.15.140:0/3118512973 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0a8012460 con 0x7fc0ac0b2ec0 2022-07-21T01:57:58.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 -- 172.21.15.140:0/3118512973 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0ac0b2ec0 msgr2=0x7fc0ac0b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:58.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 --2- 172.21.15.140:0/3118512973 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0ac0b2ec0 0x7fc0ac0b32a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7fc09c001800 tx=0x7fc0a8006df0).stop 2022-07-21T01:57:58.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 -- 172.21.15.140:0/3118512973 shutdown_connections 2022-07-21T01:57:58.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 --2- 172.21.15.140:0/3118512973 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc0ac1461c0 0x7fc0ac146630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:58.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 --2- 172.21.15.140:0/3118512973 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0ac0b2ec0 0x7fc0ac0b32a0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:58.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 -- 172.21.15.140:0/3118512973 >> 172.21.15.140:0/3118512973 conn(0x7fc0ac066e20 msgr2=0x7fc0ac067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:58.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 -- 172.21.15.140:0/3118512973 shutdown_connections 2022-07-21T01:57:58.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.275+0000 7fc0b34ea700 1 -- 172.21.15.140:0/3118512973 wait complete. 2022-07-21T01:57:58.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b34ea700 1 Processor -- start 2022-07-21T01:57:58.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b34ea700 1 -- start start 2022-07-21T01:57:58.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b34ea700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc0ac05d160 0x7fc0ac141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:58.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b34ea700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0ac0b2ec0 0x7fc0ac13b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:58.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b34ea700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0ac1461c0 0x7fc0ac13b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:58.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b34ea700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac13bb70 con 0x7fc0ac1461c0 2022-07-21T01:57:58.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b34ea700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac145f30 con 0x7fc0ac0b2ec0 2022-07-21T01:57:58.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b34ea700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0aee90 con 0x7fc0ac05d160 2022-07-21T01:57:58.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b2ce9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0ac1461c0 0x7fc0ac13b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:58.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b2ce9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0ac1461c0 0x7fc0ac13b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41516/0 (socket says 172.21.15.140:41516) 2022-07-21T01:57:58.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b2ce9700 1 -- 172.21.15.140:0/2527594136 learned_addr learned my addr 172.21.15.140:0/2527594136 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:58.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b24e8700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc0ac05d160 0x7fc0ac141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:58.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b1ce7700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0ac0b2ec0 0x7fc0ac13b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:58.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b2ce9700 1 -- 172.21.15.140:0/2527594136 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc0ac05d160 msgr2=0x7fc0ac141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:58.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b2ce9700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc0ac05d160 0x7fc0ac141060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:58.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b2ce9700 1 -- 172.21.15.140:0/2527594136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0ac0b2ec0 msgr2=0x7fc0ac13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:58.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b2ce9700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0ac0b2ec0 0x7fc0ac13b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:58.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b2ce9700 1 -- 172.21.15.140:0/2527594136 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a800b180 con 0x7fc0ac1461c0 2022-07-21T01:57:58.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b2ce9700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0ac1461c0 0x7fc0ac13b680 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7fc0a4006e00 tx=0x7fc0a40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:58.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc09b7fe700 1 -- 172.21.15.140:0/2527594136 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0a400c640 con 0x7fc0ac1461c0 2022-07-21T01:57:58.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc09b7fe700 1 -- 172.21.15.140:0/2527594136 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a400c7a0 con 0x7fc0ac1461c0 2022-07-21T01:57:58.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc0b34ea700 1 -- 172.21.15.140:0/2527594136 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac0af130 con 0x7fc0ac1461c0 2022-07-21T01:57:58.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.279+0000 7fc09b7fe700 1 -- 172.21.15.140:0/2527594136 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0a400cb00 con 0x7fc0ac1461c0 2022-07-21T01:57:58.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.283+0000 7fc0b34ea700 1 -- 172.21.15.140:0/2527594136 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac0af870 con 0x7fc0ac1461c0 2022-07-21T01:57:58.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.283+0000 7fc0b34ea700 1 -- 172.21.15.140:0/2527594136 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ac0b28c0 con 0x7fc0ac1461c0 2022-07-21T01:57:58.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.283+0000 7fc09b7fe700 1 -- 172.21.15.140:0/2527594136 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc0a4022070 con 0x7fc0ac1461c0 2022-07-21T01:57:58.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.283+0000 7fc09b7fe700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc08c053a70 0x7fc08c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:58.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.283+0000 7fc09b7fe700 1 -- 172.21.15.140:0/2527594136 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fc0a406c2f0 con 0x7fc0ac1461c0 2022-07-21T01:57:58.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.283+0000 7fc0b24e8700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc08c053a70 0x7fc08c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:58.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.283+0000 7fc0b24e8700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc08c053a70 0x7fc08c055f30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fc0a801a000 tx=0x7fc0a8014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:58.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.287+0000 7fc09b7fe700 1 -- 172.21.15.140:0/2527594136 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc0a40433e0 con 0x7fc0ac1461c0 2022-07-21T01:57:58.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:58.451+0000 7fc0b34ea700 1 -- 172.21.15.140:0/2527594136 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]} v 0) v1 -- 0x7fc0ac0574e0 con 0x7fc0ac1461c0 2022-07-21T01:57:59.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.147+0000 7fc09b7fe700 1 -- 172.21.15.140:0/2527594136 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]}]=0 v443) v1 ==== 139+0+0 (secure 0 0 0) 0x7fc0a4046a00 con 0x7fc0ac1461c0 2022-07-21T01:57:59.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 -- 172.21.15.140:0/2527594136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc08c053a70 msgr2=0x7fc08c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc08c053a70 0x7fc08c055f30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fc0a801a000 tx=0x7fc0a8014000).stop 2022-07-21T01:57:59.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 -- 172.21.15.140:0/2527594136 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0ac1461c0 msgr2=0x7fc0ac13b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0ac1461c0 0x7fc0ac13b680 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7fc0a4006e00 tx=0x7fc0a40091d0).stop 2022-07-21T01:57:59.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 -- 172.21.15.140:0/2527594136 shutdown_connections 2022-07-21T01:57:59.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc08c053a70 0x7fc08c055f30 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc0ac1461c0 0x7fc0ac13b680 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0ac0b2ec0 0x7fc0ac13b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 --2- 172.21.15.140:0/2527594136 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc0ac05d160 0x7fc0ac141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 -- 172.21.15.140:0/2527594136 >> 172.21.15.140:0/2527594136 conn(0x7fc0ac066e20 msgr2=0x7fc0ac13e280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:59.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 -- 172.21.15.140:0/2527594136 shutdown_connections 2022-07-21T01:57:59.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.151+0000 7fc0b34ea700 1 -- 172.21.15.140:0/2527594136 wait complete. 2022-07-21T01:57:59.170 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2022-07-21T01:57:59.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.283+0000 7eff8cde3700 1 Processor -- start 2022-07-21T01:57:59.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.283+0000 7eff8cde3700 1 -- start start 2022-07-21T01:57:59.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8cde3700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7eff8805d0e0 0x7eff88146920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8cde3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff8809b6d0 0x7eff8809bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8cde3700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8805d4c0 con 0x7eff880b5960 2022-07-21T01:57:59.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8cde3700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff881461e0 con 0x7eff8805d0e0 2022-07-21T01:57:59.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8cde3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff88146480 con 0x7eff8809b6d0 2022-07-21T01:57:59.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff85d9b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7eff880b5960 0x7eff88142b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55670/0 (socket says 172.21.15.140:55670) 2022-07-21T01:57:59.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff85d9b700 1 -- 172.21.15.140:0/182240386 learned_addr learned my addr 172.21.15.140:0/182240386 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:59.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8659c700 1 --2- 172.21.15.140:0/182240386 >> v2:172.21.15.152:3300/0 conn(0x7eff8805d0e0 0x7eff88146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff86d9d700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff8809b6d0 0x7eff8809bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905374383 0 0) 0x7eff8805d4c0 con 0x7eff880b5960 2022-07-21T01:57:59.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff68003260 con 0x7eff880b5960 2022-07-21T01:57:59.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 761316323 0 0) 0x7eff68003260 con 0x7eff880b5960 2022-07-21T01:57:59.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff8809b6d0 msgr2=0x7eff8809bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:59.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff8809b6d0 0x7eff8809bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 >> v2:172.21.15.152:3300/0 conn(0x7eff8805d0e0 msgr2=0x7eff88146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:59.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 --2- 172.21.15.140:0/182240386 >> v2:172.21.15.152:3300/0 conn(0x7eff8805d0e0 0x7eff88146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff88143280 con 0x7eff880b5960 2022-07-21T01:57:59.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7eff70001660 con 0x7eff880b5960 2022-07-21T01:57:59.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 >> v1:172.21.15.140:6789/0 conn(0x7eff880b5960 legacy=0x7eff88142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff68003750 0x7eff68003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff8809b6d0 0x7eff68004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff680049f0 0x7eff68004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff881461e0 con 0x7eff680049f0 2022-07-21T01:57:59.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff88146480 con 0x7eff68003750 2022-07-21T01:57:59.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6800b750 con 0x7eff8809b6d0 2022-07-21T01:57:59.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff86d9d700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff680049f0 0x7eff68004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8659c700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff68003750 0x7eff68003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff85d9b700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff8809b6d0 0x7eff68004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff86d9d700 1 -- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff8809b6d0 msgr2=0x7eff68004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:59.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff86d9d700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff8809b6d0 0x7eff68004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff86d9d700 1 -- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff68003750 msgr2=0x7eff68003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:57:59.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff86d9d700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff68003750 0x7eff68003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff86d9d700 1 -- 172.21.15.140:0/182240386 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff68003260 con 0x7eff680049f0 2022-07-21T01:57:59.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8659c700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff68003750 0x7eff68003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:59.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff85d9b700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff8809b6d0 0x7eff68004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:59.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff86d9d700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff680049f0 0x7eff68004e70 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7eff8000c000 tx=0x7eff800091d0).ready entity=mon.0 client_cookie=3b261a14185cc766 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:59.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff8000e7e0 con 0x7eff680049f0 2022-07-21T01:57:59.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8000e940 con 0x7eff680049f0 2022-07-21T01:57:59.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.287+0000 7eff8559a700 1 -- 172.21.15.140:0/182240386 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff8000eca0 con 0x7eff680049f0 2022-07-21T01:57:59.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 -- 172.21.15.140:0/182240386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff680049f0 msgr2=0x7eff68004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 --2- 172.21.15.140:0/182240386 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff680049f0 0x7eff68004e70 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7eff8000c000 tx=0x7eff800091d0).stop 2022-07-21T01:57:59.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff86d9d700 1 -- 172.21.15.140:0/182240386 reap_dead start 2022-07-21T01:57:59.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 -- 172.21.15.140:0/182240386 shutdown_connections 2022-07-21T01:57:59.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 -- 172.21.15.140:0/182240386 >> 172.21.15.140:0/182240386 conn(0x7eff880b6a20 msgr2=0x7eff88066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:59.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 -- 172.21.15.140:0/182240386 shutdown_connections 2022-07-21T01:57:59.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 -- 172.21.15.140:0/182240386 wait complete. 2022-07-21T01:57:59.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 Processor -- start 2022-07-21T01:57:59.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 -- start start 2022-07-21T01:57:59.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff880b12b0 0x7eff880b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff88149d90 0x7eff8814a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff88152210 0x7eff88152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff88147b40 con 0x7eff88152210 2022-07-21T01:57:59.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff88147d70 con 0x7eff880b12b0 2022-07-21T01:57:59.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8cde3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff88148010 con 0x7eff88149d90 2022-07-21T01:57:59.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff86d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff88152210 0x7eff88152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff86d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff88152210 0x7eff88152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41536/0 (socket says 172.21.15.140:41536) 2022-07-21T01:57:59.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff8659c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff880b12b0 0x7eff880b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff86d9d700 1 -- 172.21.15.140:0/2296232505 learned_addr learned my addr 172.21.15.140:0/2296232505 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:59.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.291+0000 7eff85d9b700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff88149d90 0x7eff8814a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff8659c700 1 -- 172.21.15.140:0/2296232505 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff88149d90 msgr2=0x7eff8814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff8659c700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff88149d90 0x7eff8814a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff8659c700 1 -- 172.21.15.140:0/2296232505 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff88152210 msgr2=0x7eff88152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff8659c700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff88152210 0x7eff88152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff8659c700 1 -- 172.21.15.140:0/2296232505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff881482b0 con 0x7eff880b12b0 2022-07-21T01:57:59.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff86d9d700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff88152210 0x7eff88152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:57:59.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff8659c700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff880b12b0 0x7eff880b2420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7eff88153580 tx=0x7eff78006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:59.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff7f7fe700 1 -- 172.21.15.140:0/2296232505 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff7800f6e0 con 0x7eff880b12b0 2022-07-21T01:57:59.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff8cde3700 1 -- 172.21.15.140:0/2296232505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8814a6e0 con 0x7eff880b12b0 2022-07-21T01:57:59.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff7f7fe700 1 -- 172.21.15.140:0/2296232505 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7800f840 con 0x7eff880b12b0 2022-07-21T01:57:59.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff7f7fe700 1 -- 172.21.15.140:0/2296232505 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff7800f9b0 con 0x7eff880b12b0 2022-07-21T01:57:59.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff8cde3700 1 -- 172.21.15.140:0/2296232505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff880b3380 con 0x7eff880b12b0 2022-07-21T01:57:59.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff8cde3700 1 -- 172.21.15.140:0/2296232505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff88148440 con 0x7eff880b12b0 2022-07-21T01:57:59.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff7f7fe700 1 -- 172.21.15.140:0/2296232505 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7eff7801a020 con 0x7eff880b12b0 2022-07-21T01:57:59.322 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff7f7fe700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7eff6c053aa0 0x7eff6c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff7f7fe700 1 -- 172.21.15.140:0/2296232505 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5339+0+0 (secure 0 0 0) 0x7eff7806c210 con 0x7eff880b12b0 2022-07-21T01:57:59.323 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.295+0000 7eff85d9b700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7eff6c053aa0 0x7eff6c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.299+0000 7eff85d9b700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7eff6c053aa0 0x7eff6c055f60 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7eff70005000 tx=0x7eff70001e50).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:59.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.303+0000 7eff7f7fe700 1 -- 172.21.15.140:0/2296232505 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7eff78043370 con 0x7eff880b12b0 2022-07-21T01:57:59.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.431+0000 7eff8cde3700 1 -- 172.21.15.140:0/2296232505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7eff880b01d0 con 0x7eff880b12b0 2022-07-21T01:57:59.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff7f7fe700 1 -- 172.21.15.140:0/2296232505 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7eff78046990 con 0x7eff880b12b0 2022-07-21T01:57:59.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 -- 172.21.15.140:0/2296232505 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7eff6c053aa0 msgr2=0x7eff6c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7eff6c053aa0 0x7eff6c055f60 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7eff70005000 tx=0x7eff70001e50).stop 2022-07-21T01:57:59.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 -- 172.21.15.140:0/2296232505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff880b12b0 msgr2=0x7eff880b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff880b12b0 0x7eff880b2420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7eff88153580 tx=0x7eff78006df0).stop 2022-07-21T01:57:59.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 -- 172.21.15.140:0/2296232505 shutdown_connections 2022-07-21T01:57:59.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7eff6c053aa0 0x7eff6c055f60 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7eff88152210 0x7eff88152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff88149d90 0x7eff8814a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 --2- 172.21.15.140:0/2296232505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff880b12b0 0x7eff880b2420 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 -- 172.21.15.140:0/2296232505 >> 172.21.15.140:0/2296232505 conn(0x7eff880b6a20 msgr2=0x7eff880b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:59.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 -- 172.21.15.140:0/2296232505 shutdown_connections 2022-07-21T01:57:59.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.443+0000 7eff8cde3700 1 -- 172.21.15.140:0/2296232505 wait complete. 2022-07-21T01:57:59.459 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T01:57:59.460 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:59.475 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:59.496 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:59.517 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:59.538 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T01:57:59.559 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T01:57:59.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:59.560 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:59.561 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:57:59.568 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2022-07-21T01:57:59.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:59.569 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2022-07-21T01:57:59.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:59.571 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:59.571 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:57:59.578 INFO:teuthology.orchestra.run.smithi192.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2022-07-21T01:57:59.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:59.580 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2022-07-21T01:57:59.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:57:59.581 DEBUG:teuthology.orchestra.run.smithi195:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:57:59.581 DEBUG:teuthology.orchestra.run.smithi195:> ' -- /home/ubuntu/cephtest/mnt.2) 2022-07-21T01:57:59.588 INFO:teuthology.orchestra.run.smithi195.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.2': No such file or directory 2022-07-21T01:57:59.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T01:57:59.590 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.2 2022-07-21T01:57:59.590 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:57:59.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 Processor -- start 2022-07-21T01:57:59.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 -- start start 2022-07-21T01:57:59.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88e40adff0 0x7f88e40b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e4147010 con 0x7f88e40af8e0 2022-07-21T01:57:59.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e41472b0 con 0x7f88e40b5650 2022-07-21T01:57:59.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4146d40 con 0x7f88e40adff0 2022-07-21T01:57:59.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88e9adb700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f88e40af8e0 0x7f88e405bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55690/0 (socket says 172.21.15.140:55690) 2022-07-21T01:57:59.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88e9adb700 1 -- 172.21.15.140:0/3915617800 learned_addr learned my addr 172.21.15.140:0/3915617800 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:57:59.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88e8ad9700 1 --2- 172.21.15.140:0/3915617800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88e40adff0 0x7f88e40b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88dbfff700 1 -- 172.21.15.140:0/3915617800 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091933574 0 0) 0x7f88e4147010 con 0x7f88e40af8e0 2022-07-21T01:57:59.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88dbfff700 1 -- 172.21.15.140:0/3915617800 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88c8003260 con 0x7f88e40af8e0 2022-07-21T01:57:59.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88e8ad9700 1 -- 172.21.15.140:0/3915617800 >> v1:172.21.15.152:6789/0 conn(0x7f88e40b5650 legacy=0x7f88e4147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88e8ad9700 1 -- 172.21.15.140:0/3915617800 >> v1:172.21.15.140:6789/0 conn(0x7f88e40af8e0 legacy=0x7f88e405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88e8ad9700 1 -- 172.21.15.140:0/3915617800 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e405c4a0 con 0x7f88e40adff0 2022-07-21T01:57:59.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88e8ad9700 1 --2- 172.21.15.140:0/3915617800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88e40adff0 0x7f88e40b5a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f88d4002f70 tx=0x7f88d4006d90).ready entity=mon.2 client_cookie=c4be4831bdf58d1b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:59.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88dbfff700 1 -- 172.21.15.140:0/3915617800 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88d400d4a0 con 0x7f88e40adff0 2022-07-21T01:57:59.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88dbfff700 1 -- 172.21.15.140:0/3915617800 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d4000e90 con 0x7f88e40adff0 2022-07-21T01:57:59.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88dbfff700 1 -- 172.21.15.140:0/3915617800 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88d400d7f0 con 0x7f88e40adff0 2022-07-21T01:57:59.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 -- 172.21.15.140:0/3915617800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88e40adff0 msgr2=0x7f88e40b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.707 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 --2- 172.21.15.140:0/3915617800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88e40adff0 0x7f88e40b5a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f88d4002f70 tx=0x7f88d4006d90).stop 2022-07-21T01:57:59.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 -- 172.21.15.140:0/3915617800 shutdown_connections 2022-07-21T01:57:59.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 --2- 172.21.15.140:0/3915617800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88e40adff0 0x7f88e40b5a30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.708 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 -- 172.21.15.140:0/3915617800 >> 172.21.15.140:0/3915617800 conn(0x7f88e40634e0 msgr2=0x7f88e40b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:57:59.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 -- 172.21.15.140:0/3915617800 shutdown_connections 2022-07-21T01:57:59.709 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.699+0000 7f88ea2dc700 1 -- 172.21.15.140:0/3915617800 wait complete. 2022-07-21T01:57:59.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88ea2dc700 1 Processor -- start 2022-07-21T01:57:59.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88ea2dc700 1 -- start start 2022-07-21T01:57:59.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88ea2dc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88e40adff0 0x7f88e414a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88ea2dc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88e40af8e0 0x7f88e4064a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88ea2dc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88e40b5650 0x7f88e4064f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88ea2dc700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e4063cd0 con 0x7f88e40af8e0 2022-07-21T01:57:59.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88ea2dc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e410f8d0 con 0x7f88e40b5650 2022-07-21T01:57:59.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88ea2dc700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e410fb70 con 0x7f88e40adff0 2022-07-21T01:57:59.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88e8ad9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88e40af8e0 0x7f88e4064a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88e8ad9700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88e40af8e0 0x7f88e4064a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41546/0 (socket says 172.21.15.140:41546) 2022-07-21T01:57:59.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88e8ad9700 1 -- 172.21.15.140:0/728396152 learned_addr learned my addr 172.21.15.140:0/728396152 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:57:59.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88e92da700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88e40adff0 0x7f88e414a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88e9adb700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88e40b5650 0x7f88e4064f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88e8ad9700 1 -- 172.21.15.140:0/728396152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88e40adff0 msgr2=0x7f88e414a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88e8ad9700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88e40adff0 0x7f88e414a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88e8ad9700 1 -- 172.21.15.140:0/728396152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88e40b5650 msgr2=0x7f88e4064f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:57:59.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88e8ad9700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88e40b5650 0x7f88e4064f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:57:59.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88e8ad9700 1 -- 172.21.15.140:0/728396152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d400b120 con 0x7f88e40af8e0 2022-07-21T01:57:59.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88e8ad9700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88e40af8e0 0x7f88e4064a30 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f88d4002f70 tx=0x7f88d4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:59.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88da7fc700 1 -- 172.21.15.140:0/728396152 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88d40117c0 con 0x7f88e40af8e0 2022-07-21T01:57:59.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88da7fc700 1 -- 172.21.15.140:0/728396152 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d4011920 con 0x7f88e40af8e0 2022-07-21T01:57:59.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88ea2dc700 1 -- 172.21.15.140:0/728396152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e410fe10 con 0x7f88e40af8e0 2022-07-21T01:57:59.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88da7fc700 1 -- 172.21.15.140:0/728396152 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88d4011c80 con 0x7f88e40af8e0 2022-07-21T01:57:59.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.703+0000 7f88ea2dc700 1 -- 172.21.15.140:0/728396152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e410f5b0 con 0x7f88e40af8e0 2022-07-21T01:57:59.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.707+0000 7f88ea2dc700 1 -- 172.21.15.140:0/728396152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e40574e0 con 0x7f88e40af8e0 2022-07-21T01:57:59.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.707+0000 7f88da7fc700 1 -- 172.21.15.140:0/728396152 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f88d400f070 con 0x7f88e40af8e0 2022-07-21T01:57:59.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.707+0000 7f88da7fc700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88cc053a70 0x7f88cc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:57:59.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.707+0000 7f88da7fc700 1 -- 172.21.15.140:0/728396152 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f88d406cee0 con 0x7f88e40af8e0 2022-07-21T01:57:59.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.707+0000 7f88e92da700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88cc053a70 0x7f88cc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:57:59.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.707+0000 7f88e92da700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88cc053a70 0x7f88cc055f30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f88e000d000 tx=0x7f88e0007b90).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:57:59.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.711+0000 7f88da7fc700 1 -- 172.21.15.140:0/728396152 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f88d4044040 con 0x7f88e40af8e0 2022-07-21T01:57:59.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.835+0000 7f88ea2dc700 1 -- 172.21.15.140:0/728396152 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f88e4067ea0 con 0x7f88e40af8e0 2022-07-21T01:57:59.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.835+0000 7f88da7fc700 1 -- 172.21.15.140:0/728396152 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v145) v1 ==== 81+0+28933 (secure 0 0 0) 0x7f88d4047660 con 0x7f88e40af8e0 2022-07-21T01:57:59.840 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:57:59.840 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:57:59.840 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 145, 2022-07-21T01:57:59.840 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:57:59.841 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:57:59.841 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:57:57.126000+0000", 2022-07-21T01:57:59.841 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:59.841 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:57:59.842 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:57:59.842 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:57:59.842 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:57:59.843 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:57:59.843 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:57:59.844 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:57:59.844 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:57:59.844 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:59.844 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:57:59.845 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:57:59.845 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:57:59.845 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:57:59.845 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:57:59.845 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 28, 2022-07-21T01:57:59.846 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:57:59.846 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:57:59.846 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:57:59.846 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:57:59.847 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:57:59.847 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.847 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:57:59.847 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:57:59.847 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:57:59.848 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:59.848 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:59.848 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:59.848 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:59.849 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:59.849 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:59.849 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:59.849 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:59.849 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:57:59.850 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:57:59.850 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:57:59.850 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:57:59.850 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:57:59.851 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:59.851 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:59.851 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:59.851 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:59.852 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:59.852 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:59.853 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:59.853 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.853 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:57:59.853 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:59.854 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:59.854 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:59.854 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:59.854 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:59.855 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:59.855 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:59.855 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:59.855 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:59.856 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:59.856 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:59.856 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:59.856 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:59.857 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:59.857 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:59.857 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:59.857 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:59.857 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:59.858 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:59.858 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:59.858 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:59.858 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:59.859 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:59.859 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:59.859 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:59.859 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:59.859 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.860 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:59.860 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:59.860 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:59.860 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:59.861 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:59.861 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:59.861 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:59.861 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:59.862 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:59.862 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:59.862 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:59.862 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:57:59.863 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:57:59.863 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.863 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:59.864 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:57:59.864 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.864 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.865 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.865 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:57:59.865 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:57:59.865 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:57:59.866 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:57:59.866 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:57:59.866 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:57:59.866 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:57:59.866 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:57:59.867 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:57:59.867 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:57:59.867 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:57:59.867 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:57:59.868 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:57:59.868 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:57:59.868 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:57:59.868 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:57:59.868 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:57:59.869 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:57:59.869 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:57:59.869 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:57:59.869 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:57:59.869 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:57:59.870 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:57:59.870 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.870 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:57:59.870 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:57:59.871 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:57:59.871 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:57:59.871 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:57:59.871 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:57:59.871 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:57:59.872 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:57:59.872 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:57:59.872 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:57:59.872 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:57:59.873 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:57:59.873 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:57:59.873 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:57:59.874 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:57:59.874 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:57:59.875 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:57:59.875 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:57:59.875 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:57:59.875 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:57:59.876 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:57:59.876 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:57:59.876 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:57:59.876 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:57:59.877 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:57:59.877 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:57:59.877 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:57:59.877 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.878 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:57:59.878 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:57:59.878 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:57:59.878 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:57:59.878 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:57:59.879 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:57:59.879 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:57:59.879 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:57:59.879 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:57:59.880 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:57:59.880 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:57:59.880 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:57:59.880 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:57:59.880 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:57:59.881 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.881 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.881 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:57:59.881 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:57:59.881 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.882 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:57:59.882 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:57:59.882 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:59.882 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:59.883 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:59.883 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:59.883 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:59.883 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:59.884 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:59.884 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 139, 2022-07-21T01:57:59.884 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:59.885 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:59.885 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:59.885 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.886 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.886 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.886 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:57:59.886 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:59.887 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.887 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.887 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.887 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:57:59.888 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:59.888 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.888 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.888 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.889 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:59.889 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.889 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.889 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.890 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:57:59.890 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:59.890 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.890 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.890 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.891 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:57:59.891 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:59.891 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.891 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.892 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.892 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:59.892 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.892 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.892 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.893 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:57:59.893 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:59.893 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.893 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.894 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.894 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:57:59.894 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:59.895 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.895 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.895 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.896 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:59.896 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.896 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.896 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.897 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:57:59.897 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:59.897 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.897 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.897 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.898 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:57:59.898 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:57:59.898 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.899 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.899 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.899 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:57:59.899 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:57:59.900 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:57:59.900 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:57:59.900 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:59.900 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:59.900 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:59.901 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.901 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.902 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.902 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:57:59.902 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:57:59.903 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:59.903 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:59.903 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:59.904 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:59.904 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:59.904 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:59.904 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:59.905 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 139, 2022-07-21T01:57:59.905 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:59.905 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:59.906 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:59.906 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.906 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.907 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.907 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:57:59.907 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:59.907 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.907 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.908 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.908 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:57:59.908 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:59.908 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.909 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.909 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.909 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:59.909 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.909 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.910 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.910 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:57:59.910 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:59.910 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.911 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.911 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.911 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:57:59.911 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:59.911 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.912 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.912 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.912 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:59.912 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.912 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.913 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.913 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:57:59.913 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:59.913 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.914 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.914 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.914 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:57:59.915 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:59.915 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.915 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.916 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.916 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:59.916 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.917 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.917 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.917 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:57:59.917 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:59.917 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.918 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.918 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.918 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:57:59.918 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:57:59.919 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.919 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.919 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.919 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:57:59.920 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:57:59.920 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:57:59.920 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:57:59.920 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:59.921 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:59.921 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:59.921 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.922 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.922 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.923 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:57:59.923 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:57:59.924 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:59.924 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:59.925 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:59.925 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:59.925 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:59.926 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:59.926 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:59.927 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 138, 2022-07-21T01:57:59.927 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:59.927 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:59.928 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:59.928 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.928 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.929 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.929 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:57:59.930 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:59.930 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.930 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.931 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.931 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:57:59.931 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:59.932 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.932 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.932 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.932 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:59.933 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.933 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.933 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.933 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:57:59.933 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:59.934 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.934 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.934 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.935 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:57:59.935 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:59.935 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.936 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.936 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.936 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:59.936 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.937 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.937 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.937 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:57:59.937 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:59.937 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.938 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.938 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.938 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:57:59.938 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:59.939 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.939 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.939 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.939 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:59.940 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.940 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.940 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.940 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:57:59.940 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:59.941 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.941 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.941 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.941 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:57:59.941 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:57:59.942 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.942 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.942 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.942 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:57:59.943 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:57:59.943 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:57:59.943 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:57:59.943 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:59.943 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:59.944 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:59.944 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.944 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.944 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.945 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:57:59.945 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:57:59.945 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:59.945 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:59.945 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:59.946 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:59.946 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:59.946 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:59.946 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:59.947 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 129, 2022-07-21T01:57:59.947 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:59.947 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:59.947 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:59.947 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.948 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.948 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.948 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:57:59.948 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:59.948 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.949 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.949 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.949 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:57:59.949 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:59.949 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.950 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.950 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.950 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:59.950 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.951 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.951 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.951 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:57:59.951 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:59.951 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.952 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.952 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.952 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:57:59.952 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:59.953 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.953 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.953 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.953 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:59.953 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.954 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.954 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.954 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:57:59.954 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:59.955 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.955 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.955 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.955 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:57:59.955 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:59.956 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.956 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.956 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.956 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:59.957 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.957 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.957 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.957 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:57:59.957 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:59.958 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.958 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.958 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.958 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:57:59.958 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:57:59.959 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.959 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.959 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.959 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:57:59.960 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:57:59.960 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:57:59.960 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:57:59.960 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:59.960 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:59.961 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:59.961 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.961 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.961 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.962 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:57:59.962 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:57:59.962 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:59.962 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:59.962 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:59.963 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:59.963 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:59.963 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:59.963 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:59.964 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 138, 2022-07-21T01:57:59.964 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:59.964 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:59.964 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:59.964 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.965 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.965 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.965 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:57:59.965 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:59.965 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.966 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.966 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.966 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:57:59.966 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:59.967 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.967 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.967 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.967 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:59.967 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.968 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.968 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.968 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:57:59.968 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:59.969 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.969 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.969 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.969 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:57:59.969 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:59.970 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.970 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.970 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.970 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:59.971 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.971 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.971 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.971 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:57:59.971 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:59.972 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.972 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.972 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.972 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:57:59.972 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:59.973 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.973 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.973 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.973 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:59.974 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.974 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.974 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.974 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:57:59.974 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:59.975 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.975 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.975 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.975 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:57:59.976 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:57:59.976 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.976 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.976 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.976 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:57:59.977 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:57:59.977 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:57:59.977 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:57:59.977 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:59.977 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:59.978 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:59.978 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.978 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.978 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.979 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:57:59.979 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:57:59.979 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:59.979 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:59.979 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:59.980 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:59.980 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:59.980 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:59.980 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:59.981 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 139, 2022-07-21T01:57:59.981 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:59.981 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:59.981 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:59.981 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.982 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.982 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.982 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:57:59.982 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:59.982 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.983 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.983 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.983 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:57:59.983 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:59.984 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.984 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.984 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.984 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:57:59.984 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.985 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.985 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.985 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:57:59.985 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:59.986 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.986 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.986 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.986 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:57:59.986 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:59.987 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.987 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.987 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.987 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:57:59.987 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.988 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.988 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.988 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:57:59.988 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:59.989 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.989 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.989 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.989 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:57:59.989 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:59.990 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.990 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.990 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.990 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:57:59.990 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.991 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.991 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.991 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:57:59.991 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:59.992 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.992 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.992 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:57:59.992 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:57:59.992 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:57:59.993 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:57:59.993 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.993 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.993 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:57:59.993 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:57:59.994 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:57:59.994 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:57:59.994 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:57:59.994 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:57:59.995 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:57:59.995 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:57:59.995 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:57:59.995 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.996 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:57:59.996 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:57:59.996 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:57:59.996 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:57:59.996 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:57:59.997 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:57:59.997 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:57:59.997 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:57:59.997 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:57:59.997 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 139, 2022-07-21T01:57:59.998 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:57:59.998 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:57:59.998 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:57:59.998 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:57:59.999 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:57:59.999 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:57:59.999 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:57:59.999 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:57:59.999 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.000 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.000 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:00.000 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:58:00.000 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:00.000 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:00.001 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:00.001 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.001 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:58:00.001 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:00.002 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.002 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:00.002 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:58:00.002 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:00.002 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.003 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.003 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:00.003 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:58:00.003 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:00.004 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:00.004 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:00.004 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.004 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:58:00.004 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:00.005 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.005 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:00.005 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:58:00.005 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:00.005 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.006 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.006 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:00.006 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:58:00.006 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:00.007 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:00.007 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:00.007 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.007 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:58:00.007 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:00.008 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.008 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:00.008 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:58:00.008 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:00.009 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.009 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.009 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:00.009 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:58:00.009 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:00.010 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:00.010 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:00.010 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.010 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:58:00.010 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:58:00.011 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:58:00.011 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:58:00.011 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:58:00.011 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:58:00.012 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:58:00.012 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:00.012 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.012 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.012 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:58:00.013 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:58:00.013 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:58:00.013 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:58:00.013 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:58:00.013 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:58:00.014 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:58:00.014 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:58:00.014 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:58:00.014 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 139, 2022-07-21T01:58:00.015 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:58:00.015 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:58:00.015 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:58:00.015 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:00.016 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.016 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:00.016 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:58:00.016 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:00.016 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.017 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.017 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:00.017 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:58:00.017 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:00.018 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:00.018 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:00.018 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.018 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:58:00.018 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:00.019 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.019 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:00.019 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:58:00.019 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:00.019 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.020 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.020 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:00.020 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:58:00.020 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:00.021 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:00.021 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:00.021 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.021 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:58:00.021 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:00.022 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.022 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:00.022 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:58:00.022 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:00.023 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.023 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.023 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:00.023 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:58:00.023 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:00.024 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:00.024 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:00.024 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.024 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:58:00.025 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:00.025 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.025 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:00.025 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:58:00.025 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:00.026 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.026 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.026 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:00.026 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:58:00.027 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:00.027 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:00.027 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:00.027 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.028 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:58:00.028 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:58:00.028 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:58:00.028 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:58:00.028 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:58:00.029 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:58:00.029 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:58:00.029 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:00.029 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:00.030 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:58:00.030 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:58:00.030 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.030 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:58:00.030 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:00.031 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:00.031 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:00.031 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:00.031 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:00.032 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:58:00.032 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:00.032 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.032 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.033 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:58:00.033 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:00.033 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:00.033 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:00.033 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:00.034 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:00.034 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:58:00.034 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:00.034 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.035 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.035 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:58:00.035 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:00.035 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:00.035 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:00.036 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:00.036 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:00.036 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:58:00.036 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:00.037 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.037 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.037 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:58:00.037 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:00.037 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:00.038 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:00.038 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:00.038 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:00.038 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:58:00.039 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:00.039 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.039 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.039 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:58:00.039 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:00.040 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:00.040 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:00.040 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:00.040 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:00.040 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:58:00.041 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:00.041 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.041 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.041 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:58:00.042 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:00.042 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:00.042 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:00.042 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:00.042 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:00.043 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:58:00.043 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:00.043 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.043 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.044 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:58:00.044 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:00.044 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:00.044 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:00.044 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:00.045 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:00.045 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:58:00.045 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:00.045 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.045 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:00.046 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:58:00.046 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:00.046 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:00.046 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:00.047 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:00.047 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:00.047 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:58:00.047 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:00.047 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:00.048 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:58:00.048 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:58:00.048 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:58:00.048 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:58:00.048 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:58:00.049 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": { 2022-07-21T01:58:00.049 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6835/961657495": "2022-07-22T01:57:53.109276+0000", 2022-07-21T01:58:00.049 INFO:teuthology.orchestra.run.smithi140.stdout: "172.21.15.140:6834/961657495": "2022-07-22T01:57:53.109276+0000" 2022-07-21T01:58:00.049 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.050 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:58:00.050 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:58:00.050 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:58:00.050 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:58:00.050 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:58:00.051 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:58:00.051 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:58:00.051 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:58:00.051 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:00.052 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:00.052 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:58:00.052 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:58:00.052 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:58:00.052 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:58:00.053 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:58:00.053 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:58:00.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 -- 172.21.15.140:0/728396152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88cc053a70 msgr2=0x7f88cc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88cc053a70 0x7f88cc055f30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f88e000d000 tx=0x7f88e0007b90).stop 2022-07-21T01:58:00.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 -- 172.21.15.140:0/728396152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88e40af8e0 msgr2=0x7f88e4064a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88e40af8e0 0x7f88e4064a30 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f88d4002f70 tx=0x7f88d4014000).stop 2022-07-21T01:58:00.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 -- 172.21.15.140:0/728396152 shutdown_connections 2022-07-21T01:58:00.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f88cc053a70 0x7f88cc055f30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88e40b5650 0x7f88e4064f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f88e40af8e0 0x7f88e4064a30 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 --2- 172.21.15.140:0/728396152 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f88e40adff0 0x7f88e414a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 -- 172.21.15.140:0/728396152 >> 172.21.15.140:0/728396152 conn(0x7f88e40634e0 msgr2=0x7f88e41460c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:00.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 -- 172.21.15.140:0/728396152 shutdown_connections 2022-07-21T01:58:00.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:57:59.839+0000 7f88ea2dc700 1 -- 172.21.15.140:0/728396152 wait complete. 2022-07-21T01:58:00.058 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:58:00.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 Processor -- start 2022-07-21T01:58:00.168 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 -- start start 2022-07-21T01:58:00.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45c00b0570 0x7f45c00b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.170 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f45c00aea10 0x7f45c0141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c0142750 con 0x7f45c01422c0 2022-07-21T01:58:00.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c0143b10 con 0x7f45c00aea10 2022-07-21T01:58:00.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c01433d0 con 0x7f45c00b0570 2022-07-21T01:58:00.171 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45be59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45c00b0570 0x7f45c00b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bdd9b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f45c00aea10 0x7f45c0141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bed9d700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f45c01422c0 0x7f45c00ad350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:56816/0 (socket says 172.21.15.140:56816) 2022-07-21T01:58:00.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bed9d700 1 -- 172.21.15.140:0/2724642425 learned_addr learned my addr 172.21.15.140:0/2724642425 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:00.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45be59c700 1 -- 172.21.15.140:0/2724642425 >> v2:172.21.15.152:3300/0 conn(0x7f45c00aea10 msgr2=0x7f45c0141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:00.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45be59c700 1 --2- 172.21.15.140:0/2724642425 >> v2:172.21.15.152:3300/0 conn(0x7f45c00aea10 0x7f45c0141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45be59c700 1 -- 172.21.15.140:0/2724642425 >> v1:172.21.15.152:6789/0 conn(0x7f45c01422c0 legacy=0x7f45c00ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45be59c700 1 -- 172.21.15.140:0/2724642425 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0143670 con 0x7f45c00b0570 2022-07-21T01:58:00.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bdd9b700 1 --2- 172.21.15.140:0/2724642425 >> v2:172.21.15.152:3300/0 conn(0x7f45c00aea10 0x7f45c0141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:58:00.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45be59c700 1 --2- 172.21.15.140:0/2724642425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45c00b0570 0x7f45c00b0950 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f45b0002f70 tx=0x7f45b0006df0).ready entity=mon.2 client_cookie=17532d1ace203462 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:00.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bd59a700 1 -- 172.21.15.140:0/2724642425 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b000d570 con 0x7f45c00b0570 2022-07-21T01:58:00.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bd59a700 1 -- 172.21.15.140:0/2724642425 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b0000e90 con 0x7f45c00b0570 2022-07-21T01:58:00.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bd59a700 1 -- 172.21.15.140:0/2724642425 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b000d8c0 con 0x7f45c00b0570 2022-07-21T01:58:00.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 -- 172.21.15.140:0/2724642425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45c00b0570 msgr2=0x7f45c00b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 --2- 172.21.15.140:0/2724642425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45c00b0570 0x7f45c00b0950 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f45b0002f70 tx=0x7f45b0006df0).stop 2022-07-21T01:58:00.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 -- 172.21.15.140:0/2724642425 shutdown_connections 2022-07-21T01:58:00.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 --2- 172.21.15.140:0/2724642425 >> v2:172.21.15.152:3300/0 conn(0x7f45c00aea10 0x7f45c0141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 --2- 172.21.15.140:0/2724642425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45c00b0570 0x7f45c00b0950 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 -- 172.21.15.140:0/2724642425 >> 172.21.15.140:0/2724642425 conn(0x7f45c00b9360 msgr2=0x7f45c00b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:00.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 -- 172.21.15.140:0/2724642425 shutdown_connections 2022-07-21T01:58:00.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.167+0000 7f45bf59e700 1 -- 172.21.15.140:0/2724642425 wait complete. 2022-07-21T01:58:00.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bf59e700 1 Processor -- start 2022-07-21T01:58:00.180 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bf59e700 1 -- start start 2022-07-21T01:58:00.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bf59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45c00aea10 0x7f45c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bf59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45c00b0570 0x7f45c0151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bf59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45c01422c0 0x7f45c0152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.181 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bf59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c0145bf0 con 0x7f45c01422c0 2022-07-21T01:58:00.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bf59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c0149770 con 0x7f45c00b0570 2022-07-21T01:58:00.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bf59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c0156090 con 0x7f45c00aea10 2022-07-21T01:58:00.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45c01422c0 0x7f45c0152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.182 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45c01422c0 0x7f45c0152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41564/0 (socket says 172.21.15.140:41564) 2022-07-21T01:58:00.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45be59c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45c00aea10 0x7f45c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bed9d700 1 -- 172.21.15.140:0/3929695678 learned_addr learned my addr 172.21.15.140:0/3929695678 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:00.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bdd9b700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45c00b0570 0x7f45c0151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.183 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bed9d700 1 -- 172.21.15.140:0/3929695678 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45c00aea10 msgr2=0x7f45c00b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bed9d700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45c00aea10 0x7f45c00b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bed9d700 1 -- 172.21.15.140:0/3929695678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45c00b0570 msgr2=0x7f45c0151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bed9d700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45c00b0570 0x7f45c0151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bed9d700 1 -- 172.21.15.140:0/3929695678 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b400b180 con 0x7f45c01422c0 2022-07-21T01:58:00.184 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bed9d700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45c01422c0 0x7f45c0152440 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f45c00afd00 tx=0x7f45b80013c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:00.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45af7fe700 1 -- 172.21.15.140:0/3929695678 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b8005ba0 con 0x7f45c01422c0 2022-07-21T01:58:00.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45af7fe700 1 -- 172.21.15.140:0/3929695678 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b8005d00 con 0x7f45c01422c0 2022-07-21T01:58:00.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45af7fe700 1 -- 172.21.15.140:0/3929695678 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45b80072c0 con 0x7f45c01422c0 2022-07-21T01:58:00.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bf59e700 1 -- 172.21.15.140:0/3929695678 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b000b180 con 0x7f45c01422c0 2022-07-21T01:58:00.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.171+0000 7f45bf59e700 1 -- 172.21.15.140:0/3929695678 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c0152930 con 0x7f45c01422c0 2022-07-21T01:58:00.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.175+0000 7f45af7fe700 1 -- 172.21.15.140:0/3929695678 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f45b8010070 con 0x7f45c01422c0 2022-07-21T01:58:00.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.175+0000 7f45bf59e700 1 -- 172.21.15.140:0/3929695678 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c01498e0 con 0x7f45c01422c0 2022-07-21T01:58:00.186 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.175+0000 7f45af7fe700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f459c053b00 0x7f459c055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.175+0000 7f45be59c700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f459c053b00 0x7f459c055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.175+0000 7f45af7fe700 1 -- 172.21.15.140:0/3929695678 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f45b806b4c0 con 0x7f45c01422c0 2022-07-21T01:58:00.187 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.175+0000 7f45be59c700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f459c053b00 0x7f459c055fc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f45b001a000 tx=0x7f45b0014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:00.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.183+0000 7f45af7fe700 1 -- 172.21.15.140:0/3929695678 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f45c01498e0 con 0x7f45c01422c0 2022-07-21T01:58:00.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.335+0000 7f45bf59e700 1 -- 172.21.15.140:0/3929695678 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45c01498e0 con 0x7f45c01422c0 2022-07-21T01:58:00.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45af7fe700 1 -- 172.21.15.140:0/3929695678 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+1212 (secure 0 0 0) 0x7f45c01498e0 con 0x7f45c01422c0 2022-07-21T01:58:00.342 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:00.343 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/148832193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":148832193},{"type":"v1","addr":"172.21.15.152:6833","nonce":148832193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":8536,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/2784313377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2784313377},{"type":"v1","addr":"172.21.15.140:6835","nonce":2784313377}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[]} 2022-07-21T01:58:00.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 -- 172.21.15.140:0/3929695678 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f459c053b00 msgr2=0x7f459c055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f459c053b00 0x7f459c055fc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f45b001a000 tx=0x7f45b0014000).stop 2022-07-21T01:58:00.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 -- 172.21.15.140:0/3929695678 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45c01422c0 msgr2=0x7f45c0152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45c01422c0 0x7f45c0152440 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f45c00afd00 tx=0x7f45b80013c0).stop 2022-07-21T01:58:00.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 -- 172.21.15.140:0/3929695678 shutdown_connections 2022-07-21T01:58:00.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f459c053b00 0x7f459c055fc0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f45c01422c0 0x7f45c0152440 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45c00b0570 0x7f45c0151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.351 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 --2- 172.21.15.140:0/3929695678 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f45c00aea10 0x7f45c00b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 -- 172.21.15.140:0/3929695678 >> 172.21.15.140:0/3929695678 conn(0x7f45c00b9360 msgr2=0x7f45c0140530 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:00.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 -- 172.21.15.140:0/3929695678 shutdown_connections 2022-07-21T01:58:00.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.339+0000 7f45bf59e700 1 -- 172.21.15.140:0/3929695678 wait complete. 2022-07-21T01:58:00.352 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 71 2022-07-21T01:58:00.354 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:58:00.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.451+0000 7f189e9ff700 1 Processor -- start 2022-07-21T01:58:00.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.451+0000 7f189e9ff700 1 -- start start 2022-07-21T01:58:00.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189e9ff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18981424e0 0x7f18980b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189e9ff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18981433d0 con 0x7f18980b0570 2022-07-21T01:58:00.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189e9ff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1898143670 con 0x7f18980aea10 2022-07-21T01:58:00.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189e9ff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1898143b10 con 0x7f18981424e0 2022-07-21T01:58:00.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189d9fd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f18980b0570 0x7f1898141dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55716/0 (socket says 172.21.15.140:55716) 2022-07-21T01:58:00.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189d9fd700 1 -- 172.21.15.140:0/2549247786 learned_addr learned my addr 172.21.15.140:0/2549247786 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:00.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189d1fc700 1 --2- 172.21.15.140:0/2549247786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18981424e0 0x7f18980b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189c9fb700 1 -- 172.21.15.140:0/2549247786 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885904988 0 0) 0x7f18981433d0 con 0x7f18980b0570 2022-07-21T01:58:00.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189c9fb700 1 -- 172.21.15.140:0/2549247786 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1880003260 con 0x7f18980b0570 2022-07-21T01:58:00.471 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189c9fb700 1 -- 172.21.15.140:0/2549247786 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741632293 0 0) 0x7f1898143670 con 0x7f18980aea10 2022-07-21T01:58:00.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189c9fb700 1 -- 172.21.15.140:0/2549247786 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18981433d0 con 0x7f18980aea10 2022-07-21T01:58:00.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189d1fc700 1 -- 172.21.15.140:0/2549247786 >> v1:172.21.15.152:6789/0 conn(0x7f18980aea10 legacy=0x7f18980ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189d1fc700 1 -- 172.21.15.140:0/2549247786 >> v1:172.21.15.140:6789/0 conn(0x7f18980b0570 legacy=0x7f1898141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.472 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189d1fc700 1 -- 172.21.15.140:0/2549247786 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18980b1010 con 0x7f18981424e0 2022-07-21T01:58:00.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189d1fc700 1 --2- 172.21.15.140:0/2549247786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18981424e0 0x7f18980b0950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1894002f70 tx=0x7f1894006df0).ready entity=mon.2 client_cookie=35f8d08eeca630ff server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:00.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189c9fb700 1 -- 172.21.15.140:0/2549247786 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f189400d5c0 con 0x7f18981424e0 2022-07-21T01:58:00.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189c9fb700 1 -- 172.21.15.140:0/2549247786 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1894000e90 con 0x7f18981424e0 2022-07-21T01:58:00.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189c9fb700 1 -- 172.21.15.140:0/2549247786 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f189400d910 con 0x7f18981424e0 2022-07-21T01:58:00.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189e9ff700 1 -- 172.21.15.140:0/2549247786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18981424e0 msgr2=0x7f18980b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189e9ff700 1 --2- 172.21.15.140:0/2549247786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18981424e0 0x7f18980b0950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1894002f70 tx=0x7f1894006df0).stop 2022-07-21T01:58:00.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189e9ff700 1 -- 172.21.15.140:0/2549247786 shutdown_connections 2022-07-21T01:58:00.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189e9ff700 1 --2- 172.21.15.140:0/2549247786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18981424e0 0x7f18980b0950 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189e9ff700 1 -- 172.21.15.140:0/2549247786 >> 172.21.15.140:0/2549247786 conn(0x7f18980b9360 msgr2=0x7f18980b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:00.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189e9ff700 1 -- 172.21.15.140:0/2549247786 shutdown_connections 2022-07-21T01:58:00.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.455+0000 7f189e9ff700 1 -- 172.21.15.140:0/2549247786 wait complete. 2022-07-21T01:58:00.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189e9ff700 1 Processor -- start 2022-07-21T01:58:00.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189e9ff700 1 -- start start 2022-07-21T01:58:00.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189e9ff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18980aea10 0x7f18980b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189e9ff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18980b0570 0x7f189813ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189e9ff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18981424e0 0x7f189813f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189e9ff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18980b4fe0 con 0x7f18980b0570 2022-07-21T01:58:00.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189e9ff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189813f6e0 con 0x7f18981424e0 2022-07-21T01:58:00.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189e9ff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18981469f0 con 0x7f18980aea10 2022-07-21T01:58:00.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189d1fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18980b0570 0x7f189813ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189d1fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18980b0570 0x7f189813ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41576/0 (socket says 172.21.15.140:41576) 2022-07-21T01:58:00.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189d1fc700 1 -- 172.21.15.140:0/2923638531 learned_addr learned my addr 172.21.15.140:0/2923638531 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:00.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189d9fd700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18980aea10 0x7f18980b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189e1fe700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18981424e0 0x7f189813f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189d1fc700 1 -- 172.21.15.140:0/2923638531 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18980aea10 msgr2=0x7f18980b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189d1fc700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18980aea10 0x7f18980b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189d1fc700 1 -- 172.21.15.140:0/2923638531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18981424e0 msgr2=0x7f189813f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189d1fc700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18981424e0 0x7f189813f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189d1fc700 1 -- 172.21.15.140:0/2923638531 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189400b180 con 0x7f18980b0570 2022-07-21T01:58:00.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189d1fc700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18980b0570 0x7f189813ed00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f1894002f70 tx=0x7f1894014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:00.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f1886ffd700 1 -- 172.21.15.140:0/2923638531 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1894013de0 con 0x7f18980b0570 2022-07-21T01:58:00.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f1886ffd700 1 -- 172.21.15.140:0/2923638531 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189401e400 con 0x7f18980b0570 2022-07-21T01:58:00.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189e9ff700 1 -- 172.21.15.140:0/2923638531 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18980b51f0 con 0x7f18980b0570 2022-07-21T01:58:00.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f1886ffd700 1 -- 172.21.15.140:0/2923638531 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f189401e760 con 0x7f18980b0570 2022-07-21T01:58:00.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.459+0000 7f189e9ff700 1 -- 172.21.15.140:0/2923638531 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1898145dc0 con 0x7f18980b0570 2022-07-21T01:58:00.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.463+0000 7f189e9ff700 1 -- 172.21.15.140:0/2923638531 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18980b14a0 con 0x7f18980b0570 2022-07-21T01:58:00.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.463+0000 7f1886ffd700 1 -- 172.21.15.140:0/2923638531 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f189401c070 con 0x7f18980b0570 2022-07-21T01:58:00.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.463+0000 7f1886ffd700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18780539e0 0x7f1878055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.463+0000 7f1886ffd700 1 -- 172.21.15.140:0/2923638531 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f189406c510 con 0x7f18980b0570 2022-07-21T01:58:00.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.463+0000 7f189d9fd700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18780539e0 0x7f1878055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.463+0000 7f189d9fd700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18780539e0 0x7f1878055ea0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f188800d000 tx=0x7f1888009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:00.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.471+0000 7f1886ffd700 1 -- 172.21.15.140:0/2923638531 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1894043670 con 0x7f18980b0570 2022-07-21T01:58:00.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 -- 172.21.15.140:0/2923638531 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18980574e0 con 0x7f18980b0570 2022-07-21T01:58:00.633 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f1886ffd700 1 -- 172.21.15.140:0/2923638531 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+1212 (secure 0 0 0) 0x7f1894023020 con 0x7f18980b0570 2022-07-21T01:58:00.634 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:00.635 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7274,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/148832193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":148832193},{"type":"v1","addr":"172.21.15.152:6833","nonce":148832193}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":68},{"gid":8536,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.140:6835/2784313377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":2784313377},{"type":"v1","addr":"172.21.15.140:6835","nonce":2784313377}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[]} 2022-07-21T01:58:00.637 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 -- 172.21.15.140:0/2923638531 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18780539e0 msgr2=0x7f1878055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18780539e0 0x7f1878055ea0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f188800d000 tx=0x7f1888009040).stop 2022-07-21T01:58:00.638 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 -- 172.21.15.140:0/2923638531 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18980b0570 msgr2=0x7f189813ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18980b0570 0x7f189813ed00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f1894002f70 tx=0x7f1894014000).stop 2022-07-21T01:58:00.639 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 -- 172.21.15.140:0/2923638531 shutdown_connections 2022-07-21T01:58:00.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f18780539e0 0x7f1878055ea0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f18981424e0 0x7f189813f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.640 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f18980b0570 0x7f189813ed00 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 --2- 172.21.15.140:0/2923638531 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18980aea10 0x7f18980b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 -- 172.21.15.140:0/2923638531 >> 172.21.15.140:0/2923638531 conn(0x7f18980b9360 msgr2=0x7f18980b7260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:00.641 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 -- 172.21.15.140:0/2923638531 shutdown_connections 2022-07-21T01:58:00.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.631+0000 7f189e9ff700 1 -- 172.21.15.140:0/2923638531 wait complete. 2022-07-21T01:58:00.642 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 71 2022-07-21T01:58:00.647 INFO:tasks.ceph.mds.a:Restarting daemon 2022-07-21T01:58:00.647 INFO:tasks.ceph.mds.a:Stopping old one... 2022-07-21T01:58:00.648 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T01:58:00.649 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:58:00.649 INFO:tasks.ceph.mds.b:Restarting daemon 2022-07-21T01:58:00.650 INFO:tasks.ceph.mds.b:Stopping old one... 2022-07-21T01:58:00.651 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T01:58:00.651 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T01:58:00.661 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T01:58:00.662 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-07-21T01:58:00.664 INFO:tasks.ceph.mds.a:Started 2022-07-21T01:58:00.664 DEBUG:teuthology.parallel:result is None 2022-07-21T01:58:00.667 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T01:58:00.667 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-07-21T01:58:00.669 INFO:tasks.ceph.mds.b:Started 2022-07-21T01:58:00.670 DEBUG:teuthology.parallel:result is None 2022-07-21T01:58:00.670 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2022-07-21T01:58:00.743 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:58:00.744 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2022-07-21T01:58:00.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.767+0000 7f29ac85a700 1 Processor -- start 2022-07-21T01:58:00.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.767+0000 7f29ac85a700 1 -- start start 2022-07-21T01:58:00.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29ac85a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f29a80b6200 0x7f29a80ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29ac85a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a80677d0 con 0x7f29a80b4af0 2022-07-21T01:58:00.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29ac85a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a80b65e0 con 0x7f29a8140060 2022-07-21T01:58:00.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29ac85a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a80aca50 con 0x7f29a80b6200 2022-07-21T01:58:00.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a659c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f29a80b4af0 0x7f29a813f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55750/0 (socket says 172.21.15.140:55750) 2022-07-21T01:58:00.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a659c700 1 -- 172.21.15.140:0/186048641 learned_addr learned my addr 172.21.15.140:0/186048641 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:00.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a6d9d700 1 --2- 172.21.15.140:0/186048641 >> v2:172.21.15.152:3300/0 conn(0x7f29a80b6200 0x7f29a80ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a5d9b700 1 -- 172.21.15.140:0/186048641 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265689370 0 0) 0x7f29a80677d0 con 0x7f29a80b4af0 2022-07-21T01:58:00.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a5d9b700 1 -- 172.21.15.140:0/186048641 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2988003260 con 0x7f29a80b4af0 2022-07-21T01:58:00.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a6d9d700 1 -- 172.21.15.140:0/186048641 >> v1:172.21.15.152:6789/0 conn(0x7f29a8140060 legacy=0x7f29a80b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a6d9d700 1 -- 172.21.15.140:0/186048641 >> v1:172.21.15.140:6789/0 conn(0x7f29a80b4af0 legacy=0x7f29a813f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a6d9d700 1 -- 172.21.15.140:0/186048641 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a8067530 con 0x7f29a80b6200 2022-07-21T01:58:00.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a6d9d700 1 --2- 172.21.15.140:0/186048641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29a80b6200 0x7f29a80ac560 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f2990002f70 tx=0x7f2990006df0).ready entity=mon.1 client_cookie=178408dcdcb400a8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:00.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a5d9b700 1 -- 172.21.15.140:0/186048641 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f299000d610 con 0x7f29a80b6200 2022-07-21T01:58:00.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a5d9b700 1 -- 172.21.15.140:0/186048641 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2990000e90 con 0x7f29a80b6200 2022-07-21T01:58:00.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.771+0000 7f29a5d9b700 1 -- 172.21.15.140:0/186048641 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f299000d960 con 0x7f29a80b6200 2022-07-21T01:58:00.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 -- 172.21.15.140:0/186048641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29a80b6200 msgr2=0x7f29a80ac560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 --2- 172.21.15.140:0/186048641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29a80b6200 0x7f29a80ac560 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f2990002f70 tx=0x7f2990006df0).stop 2022-07-21T01:58:00.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 -- 172.21.15.140:0/186048641 shutdown_connections 2022-07-21T01:58:00.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 --2- 172.21.15.140:0/186048641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29a80b6200 0x7f29a80ac560 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 -- 172.21.15.140:0/186048641 >> 172.21.15.140:0/186048641 conn(0x7f29a8063d10 msgr2=0x7f29a80b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:00.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 -- 172.21.15.140:0/186048641 shutdown_connections 2022-07-21T01:58:00.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 -- 172.21.15.140:0/186048641 wait complete. 2022-07-21T01:58:00.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 Processor -- start 2022-07-21T01:58:00.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 -- start start 2022-07-21T01:58:00.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29a80b4af0 0x7f29a8066680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29a80b6200 0x7f29a8155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29a8140060 0x7f29a8141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8155f20 con 0x7f29a8140060 2022-07-21T01:58:00.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8066b70 con 0x7f29a80b6200 2022-07-21T01:58:00.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29ac85a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a8064a10 con 0x7f29a80b4af0 2022-07-21T01:58:00.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29a759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29a8140060 0x7f29a8141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.775+0000 7f29a759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29a8140060 0x7f29a8141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41610/0 (socket says 172.21.15.140:41610) 2022-07-21T01:58:00.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29a759e700 1 -- 172.21.15.140:0/3898847673 learned_addr learned my addr 172.21.15.140:0/3898847673 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:00.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29a6d9d700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29a80b4af0 0x7f29a8066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29a659c700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29a80b6200 0x7f29a8155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29a759e700 1 -- 172.21.15.140:0/3898847673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29a80b4af0 msgr2=0x7f29a8066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29a759e700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29a80b4af0 0x7f29a8066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29a759e700 1 -- 172.21.15.140:0/3898847673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29a80b6200 msgr2=0x7f29a8155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:00.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29a759e700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29a80b6200 0x7f29a8155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:00.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29a759e700 1 -- 172.21.15.140:0/3898847673 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299000b180 con 0x7f29a8140060 2022-07-21T01:58:00.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29a759e700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29a8140060 0x7f29a8141f20 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f29a806ad30 tx=0x7f29a0007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:00.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f299ffff700 1 -- 172.21.15.140:0/3898847673 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29a0012400 con 0x7f29a8140060 2022-07-21T01:58:00.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f299ffff700 1 -- 172.21.15.140:0/3898847673 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a0012560 con 0x7f29a8140060 2022-07-21T01:58:00.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29ac85a700 1 -- 172.21.15.140:0/3898847673 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a8153010 con 0x7f29a8140060 2022-07-21T01:58:00.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f299ffff700 1 -- 172.21.15.140:0/3898847673 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29a00128c0 con 0x7f29a8140060 2022-07-21T01:58:00.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29ac85a700 1 -- 172.21.15.140:0/3898847673 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a8064610 con 0x7f29a8140060 2022-07-21T01:58:00.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29ac85a700 1 -- 172.21.15.140:0/3898847673 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a80b5220 con 0x7f29a8140060 2022-07-21T01:58:00.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f299ffff700 1 -- 172.21.15.140:0/3898847673 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f29a0019030 con 0x7f29a8140060 2022-07-21T01:58:00.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f299ffff700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f298c053950 0x7f298c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:00.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f299ffff700 1 -- 172.21.15.140:0/3898847673 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f29a006ae90 con 0x7f29a8140060 2022-07-21T01:58:00.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.779+0000 7f29a6d9d700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f298c053950 0x7f298c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:00.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.783+0000 7f29a6d9d700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f298c053950 0x7f298c055e10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f299001a000 tx=0x7f2990014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:00.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.783+0000 7f299ffff700 1 -- 172.21.15.140:0/3898847673 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f29a0042090 con 0x7f29a8140060 2022-07-21T01:58:00.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:00.919+0000 7f29ac85a700 1 -- 172.21.15.140:0/3898847673 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f29a80574e0 con 0x7f29a8140060 2022-07-21T01:58:01.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.783+0000 7f299ffff700 1 -- 172.21.15.140:0/3898847673 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v147) v1 ==== 97+0+0 (secure 0 0 0) 0x7f29a00456b0 con 0x7f29a8140060 2022-07-21T01:58:01.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 -- 172.21.15.140:0/3898847673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f298c053950 msgr2=0x7f298c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:01.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f298c053950 0x7f298c055e10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f299001a000 tx=0x7f2990014000).stop 2022-07-21T01:58:01.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 -- 172.21.15.140:0/3898847673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29a8140060 msgr2=0x7f29a8141f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:01.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29a8140060 0x7f29a8141f20 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7f29a806ad30 tx=0x7f29a0007b90).stop 2022-07-21T01:58:01.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 -- 172.21.15.140:0/3898847673 shutdown_connections 2022-07-21T01:58:01.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f298c053950 0x7f298c055e10 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:01.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f29a8140060 0x7f29a8141f20 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:01.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29a80b6200 0x7f29a8155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:01.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 --2- 172.21.15.140:0/3898847673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f29a80b4af0 0x7f29a8066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:01.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 -- 172.21.15.140:0/3898847673 >> 172.21.15.140:0/3898847673 conn(0x7f29a8063d10 msgr2=0x7f29a80b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:01.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 -- 172.21.15.140:0/3898847673 shutdown_connections 2022-07-21T01:58:01.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.787+0000 7f29ac85a700 1 -- 172.21.15.140:0/3898847673 wait complete. 2022-07-21T01:58:01.793 INFO:teuthology.orchestra.run.smithi140.stderr: removed all blacklist entries 2022-07-21T01:58:01.808 INFO:tasks.cephfs.cephfs_test_case:['0', '1', '2'] 2022-07-21T01:58:01.809 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2022-07-21T01:58:01.916 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f7b75b700 1 Processor -- start 2022-07-21T01:58:01.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f7b75b700 1 -- start start 2022-07-21T01:58:01.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f7b75b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f740b1e00 0x7f5f7414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:01.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f7b75b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5f740acef0 0x7f5f74146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:01.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f7b75b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7405d480 con 0x7f5f7405c3f0 2022-07-21T01:58:01.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f7b75b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f740b21e0 con 0x7f5f740acef0 2022-07-21T01:58:01.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f7b75b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7405cf40 con 0x7f5f740b1e00 2022-07-21T01:58:01.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f78cf6700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f5f7405c3f0 0x7f5f740ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55762/0 (socket says 172.21.15.140:55762) 2022-07-21T01:58:01.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f78cf6700 1 -- 172.21.15.140:0/2464143514 learned_addr learned my addr 172.21.15.140:0/2464143514 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:01.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f794f7700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f740b1e00 0x7f5f7414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:01.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f79cf8700 1 --2- 172.21.15.140:0/2464143514 >> v2:172.21.15.152:3300/0 conn(0x7f5f740acef0 0x7f5f74146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:01.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504145595 0 0) 0x7f5f7405d480 con 0x7f5f7405c3f0 2022-07-21T01:58:01.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f58003260 con 0x7f5f7405c3f0 2022-07-21T01:58:01.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2616087197 0 0) 0x7f5f58003260 con 0x7f5f7405c3f0 2022-07-21T01:58:01.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f740b1e00 msgr2=0x7f5f7414a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:01.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f740b1e00 0x7f5f7414a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:01.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 >> v2:172.21.15.152:3300/0 conn(0x7f5f740acef0 msgr2=0x7f5f74146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:01.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 --2- 172.21.15.140:0/2464143514 >> v2:172.21.15.152:3300/0 conn(0x7f5f740acef0 0x7f5f74146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:01.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7405d1e0 con 0x7f5f7405c3f0 2022-07-21T01:58:01.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f5f6c0019e0 con 0x7f5f7405c3f0 2022-07-21T01:58:01.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 >> v1:172.21.15.140:6789/0 conn(0x7f5f7405c3f0 legacy=0x7f5f740ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:01.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f58003750 0x7f5f58003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:01.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f740b1e00 0x7f5f58004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:01.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f580049f0 0x7f5f58004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:01.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740b21e0 con 0x7f5f58003750 2022-07-21T01:58:01.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7405cf40 con 0x7f5f580049f0 2022-07-21T01:58:01.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5800b750 con 0x7f5f740b1e00 2022-07-21T01:58:01.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f794f7700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f58003750 0x7f5f58003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:01.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f78cf6700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f740b1e00 0x7f5f58004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:01.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f79cf8700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f580049f0 0x7f5f58004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:01.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f794f7700 1 -- 172.21.15.140:0/2464143514 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f740b1e00 msgr2=0x7f5f58004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:01.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f794f7700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f740b1e00 0x7f5f58004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:01.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f794f7700 1 -- 172.21.15.140:0/2464143514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f580049f0 msgr2=0x7f5f58004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:01.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f794f7700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f580049f0 0x7f5f58004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:01.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f794f7700 1 -- 172.21.15.140:0/2464143514 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f58003260 con 0x7f5f58003750 2022-07-21T01:58:01.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.915+0000 7f5f794f7700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f58003750 0x7f5f58003b70 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f5f7000c000 tx=0x7f5f70006df0).ready entity=mon.0 client_cookie=96b2ab497d4e85de server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:01.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.919+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f7000e830 con 0x7f5f58003750 2022-07-21T01:58:01.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.919+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7000e990 con 0x7f5f58003750 2022-07-21T01:58:01.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.919+0000 7f5f67fff700 1 -- 172.21.15.140:0/2464143514 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f7000ecf0 con 0x7f5f58003750 2022-07-21T01:58:01.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.919+0000 7f5f7b75b700 1 -- 172.21.15.140:0/2464143514 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f58003750 msgr2=0x7f5f58003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:01.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.919+0000 7f5f7b75b700 1 --2- 172.21.15.140:0/2464143514 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f58003750 0x7f5f58003b70 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f5f7000c000 tx=0x7f5f70006df0).stop 2022-07-21T01:58:01.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.919+0000 7f5f79cf8700 1 -- 172.21.15.140:0/2464143514 reap_dead start 2022-07-21T01:58:01.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.919+0000 7f5f7b75b700 1 -- 172.21.15.140:0/2464143514 shutdown_connections 2022-07-21T01:58:01.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.919+0000 7f5f7b75b700 1 -- 172.21.15.140:0/2464143514 >> 172.21.15.140:0/2464143514 conn(0x7f5f740611c0 msgr2=0x7f5f74062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:01.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.919+0000 7f5f7b75b700 1 -- 172.21.15.140:0/2464143514 shutdown_connections 2022-07-21T01:58:01.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.919+0000 7f5f7b75b700 1 -- 172.21.15.140:0/2464143514 wait complete. 2022-07-21T01:58:01.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.919+0000 7f5f7b75b700 1 Processor -- start 2022-07-21T01:58:01.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f7b75b700 1 -- start start 2022-07-21T01:58:01.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f7b75b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f74063d20 0x7f5f740654f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:01.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f7b75b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f74145b60 0x7f5f740659e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:01.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f7b75b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f74159f50 0x7f5f7415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:01.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f7b75b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f74067200 con 0x7f5f74063d20 2022-07-21T01:58:01.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f7b75b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7414ad50 con 0x7f5f74145b60 2022-07-21T01:58:01.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f7b75b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7409e080 con 0x7f5f74159f50 2022-07-21T01:58:01.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f794f7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f74063d20 0x7f5f740654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:01.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f794f7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f74063d20 0x7f5f740654f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41628/0 (socket says 172.21.15.140:41628) 2022-07-21T01:58:01.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f794f7700 1 -- 172.21.15.140:0/4171687175 learned_addr learned my addr 172.21.15.140:0/4171687175 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:01.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f78cf6700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f74145b60 0x7f5f740659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:01.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f79cf8700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f74159f50 0x7f5f7415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:01.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f794f7700 1 -- 172.21.15.140:0/4171687175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f74159f50 msgr2=0x7f5f7415a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:01.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f794f7700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f74159f50 0x7f5f7415a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:01.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f794f7700 1 -- 172.21.15.140:0/4171687175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f74145b60 msgr2=0x7f5f740659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:01.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f794f7700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f74145b60 0x7f5f740659e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:01.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f794f7700 1 -- 172.21.15.140:0/4171687175 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f74064100 con 0x7f5f74063d20 2022-07-21T01:58:01.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f79cf8700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f74159f50 0x7f5f7415a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:58:01.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f78cf6700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f74145b60 0x7f5f740659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:58:01.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f794f7700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f74063d20 0x7f5f740654f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f5f7000c000 tx=0x7f5f70019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:01.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f667fc700 1 -- 172.21.15.140:0/4171687175 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f7001b820 con 0x7f5f74063d20 2022-07-21T01:58:01.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f667fc700 1 -- 172.21.15.140:0/4171687175 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f70000e90 con 0x7f5f74063d20 2022-07-21T01:58:01.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f7b75b700 1 -- 172.21.15.140:0/4171687175 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f74067430 con 0x7f5f74063d20 2022-07-21T01:58:01.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f667fc700 1 -- 172.21.15.140:0/4171687175 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5f7001bb70 con 0x7f5f74063d20 2022-07-21T01:58:01.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.923+0000 7f5f7b75b700 1 -- 172.21.15.140:0/4171687175 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f74064550 con 0x7f5f74063d20 2022-07-21T01:58:01.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.927+0000 7f5f7b75b700 1 -- 172.21.15.140:0/4171687175 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7409e1f0 con 0x7f5f74063d20 2022-07-21T01:58:01.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.927+0000 7f5f667fc700 1 -- 172.21.15.140:0/4171687175 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5f7002d070 con 0x7f5f74063d20 2022-07-21T01:58:01.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.927+0000 7f5f667fc700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f5c053a10 0x7f5f5c055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:01.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.927+0000 7f5f667fc700 1 -- 172.21.15.140:0/4171687175 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f5f7006d520 con 0x7f5f74063d20 2022-07-21T01:58:01.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.927+0000 7f5f78cf6700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f5c053a10 0x7f5f5c055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:01.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.927+0000 7f5f78cf6700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f5c053a10 0x7f5f5c055ed0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f5f6c005000 tx=0x7f5f6c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:01.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:01.931+0000 7f5f667fc700 1 -- 172.21.15.140:0/4171687175 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5f700446a0 con 0x7f5f74063d20 2022-07-21T01:58:02.094 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.091+0000 7f5f7b75b700 1 -- 172.21.15.140:0/4171687175 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5f740b04b0 con 0x7f5f74063d20 2022-07-21T01:58:02.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.091+0000 7f5f667fc700 1 -- 172.21.15.140:0/4171687175 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v48) v1 ==== 80+0+4242 (secure 0 0 0) 0x7f5f70047c80 con 0x7f5f74063d20 2022-07-21T01:58:02.096 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:02.096 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:58:02.097 INFO:teuthology.orchestra.run.smithi140.stdout: "auth_dump": [ 2022-07-21T01:58:02.097 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.097 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.a", 2022-07-21T01:58:02.098 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiB28DFRAAUqIriDRoITv5f/Og3rw22w==", 2022-07-21T01:58:02.098 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.099 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:58:02.099 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:58:02.100 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:58:02.100 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.100 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.100 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.101 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.101 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mds.b", 2022-07-21T01:58:02.101 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhihL9AGRAAeudVyIxdq2RuhzidzCYcmw==", 2022-07-21T01:58:02.101 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.102 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:58:02.102 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:58:02.102 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:58:02.103 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.103 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.103 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.103 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.104 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.0", 2022-07-21T01:58:02.105 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhi3uW5GRAAO4proS7v4uslN/TrqprwyA==", 2022-07-21T01:58:02.106 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.106 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:58:02.107 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:58:02.107 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.107 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.107 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.108 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.108 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.1", 2022-07-21T01:58:02.108 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBVsNhiEu8jORAAVCS9O/S17c5A8zZI9ws9XQ==", 2022-07-21T01:58:02.108 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.108 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:58:02.109 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:58:02.109 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.109 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.109 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.110 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.110 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.2", 2022-07-21T01:58:02.110 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhiEbJJGBAA4rfeLxDVKe+IGmCinRBWIg==", 2022-07-21T01:58:02.110 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.111 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:58:02.111 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:58:02.111 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.111 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.111 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.112 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.112 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.3", 2022-07-21T01:58:02.112 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBWsNhi4nftKRAAFeRssKnWUWsHT3aFVewK9w==", 2022-07-21T01:58:02.112 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.112 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:58:02.113 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:58:02.113 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.113 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.113 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.114 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.114 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.4", 2022-07-21T01:58:02.114 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBZsNhisjvMJxAAdXoTxEBhKB35CcBX7S1hXw==", 2022-07-21T01:58:02.114 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.114 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:58:02.115 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:58:02.115 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.115 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.115 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.116 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.116 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.5", 2022-07-21T01:58:02.116 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhidUlvBRAAk4zdSnQAA4nKoMaCGjfWfQ==", 2022-07-21T01:58:02.116 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.116 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:58:02.117 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:58:02.117 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.117 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.117 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.118 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.118 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.6", 2022-07-21T01:58:02.118 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhi20TyHxAA82faVooURqH43KLpZOvE5Q==", 2022-07-21T01:58:02.118 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.118 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:58:02.119 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:58:02.119 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.119 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.119 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.120 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.120 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "osd.7", 2022-07-21T01:58:02.120 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBasNhiJlP6LRAAvFYgnNlhzU3bhA8IaWadmw==", 2022-07-21T01:58:02.120 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.120 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:58:02.121 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:58:02.121 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.121 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.121 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.121 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.122 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.0", 2022-07-21T01:58:02.122 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhifICMHxAAv6zo7o3LHTN/5xO4EZXdvw==", 2022-07-21T01:58:02.122 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.122 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:58:02.123 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:58:02.123 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:58:02.123 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.123 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.123 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.124 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.1", 2022-07-21T01:58:02.124 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiFgC0IhAAGyZ5cvrZYfk2eYFoNyTs3g==", 2022-07-21T01:58:02.124 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.124 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:58:02.125 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:58:02.125 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:58:02.125 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.125 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.126 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.126 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.2", 2022-07-21T01:58:02.126 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiWVhJJhAAn45WA1uunp87m+5ySDFN3g==", 2022-07-21T01:58:02.126 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.127 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow", 2022-07-21T01:58:02.127 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow r", 2022-07-21T01:58:02.127 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow rw pool=cephfs_data" 2022-07-21T01:58:02.127 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.127 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.128 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.128 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "client.admin", 2022-07-21T01:58:02.128 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBRsNhixdCtKBAAYbJ+71tB/qy/7kv8PVDbBQ==", 2022-07-21T01:58:02.128 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.129 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:58:02.129 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr": "allow *", 2022-07-21T01:58:02.129 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow *", 2022-07-21T01:58:02.129 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.129 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.130 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.130 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.130 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.x", 2022-07-21T01:58:02.130 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhiZZAjEhAAUHYzy/BLT7IHGh5O8eirgw==", 2022-07-21T01:58:02.130 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.131 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:58:02.131 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:58:02.131 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.131 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.132 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:02.132 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:02.132 INFO:teuthology.orchestra.run.smithi140.stdout: "entity": "mgr.y", 2022-07-21T01:58:02.132 INFO:teuthology.orchestra.run.smithi140.stdout: "key": "AQBSsNhik3mtDhAAGZi2WSU/PKt2YC4GDs42NQ==", 2022-07-21T01:58:02.132 INFO:teuthology.orchestra.run.smithi140.stdout: "caps": { 2022-07-21T01:58:02.133 INFO:teuthology.orchestra.run.smithi140.stdout: "mds": "allow *", 2022-07-21T01:58:02.133 INFO:teuthology.orchestra.run.smithi140.stdout: "mon": "allow profile mgr", 2022-07-21T01:58:02.133 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": "allow *" 2022-07-21T01:58:02.133 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.134 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:02.134 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:02.134 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:58:02.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 -- 172.21.15.140:0/4171687175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f5c053a10 msgr2=0x7f5f5c055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f5c053a10 0x7f5f5c055ed0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f5f6c005000 tx=0x7f5f6c009040).stop 2022-07-21T01:58:02.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 -- 172.21.15.140:0/4171687175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f74063d20 msgr2=0x7f5f740654f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f74063d20 0x7f5f740654f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f5f7000c000 tx=0x7f5f70019000).stop 2022-07-21T01:58:02.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 -- 172.21.15.140:0/4171687175 shutdown_connections 2022-07-21T01:58:02.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5f5c053a10 0x7f5f5c055ed0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f74159f50 0x7f5f7415a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f74145b60 0x7f5f740659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 --2- 172.21.15.140:0/4171687175 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5f74063d20 0x7f5f740654f0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 -- 172.21.15.140:0/4171687175 >> 172.21.15.140:0/4171687175 conn(0x7f5f58003750 msgr2=0x7f5f741461e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:02.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 -- 172.21.15.140:0/4171687175 shutdown_connections 2022-07-21T01:58:02.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.095+0000 7f5f7b75b700 1 -- 172.21.15.140:0/4171687175 wait complete. 2022-07-21T01:58:02.140 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:58:02.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd30ca7700 1 Processor -- start 2022-07-21T01:58:02.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd30ca7700 1 -- start start 2022-07-21T01:58:02.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd30ca7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c05c3f0 0x7efd2c14a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd30ca7700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efd2c0acef0 0x7efd2c146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd30ca7700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2c05d480 con 0x7efd2c0b1e00 2022-07-21T01:58:02.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd30ca7700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd2c0b21e0 con 0x7efd2c0acef0 2022-07-21T01:58:02.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd30ca7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c05cf40 con 0x7efd2c05c3f0 2022-07-21T01:58:02.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd29d9b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c05c3f0 0x7efd2c14a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd2ad9d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efd2c0acef0 0x7efd2c146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd29d9b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c05c3f0 0x7efd2c14a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:38368/0 (socket says 172.21.15.140:38368) 2022-07-21T01:58:02.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd29d9b700 1 -- 172.21.15.140:0/4052140779 learned_addr learned my addr 172.21.15.140:0/4052140779 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:02.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd2ad9d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efd2c0acef0 0x7efd2c146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:47380/0 (socket says 172.21.15.140:47380) 2022-07-21T01:58:02.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd2a59c700 1 --1- 172.21.15.140:0/4052140779 >> v1:172.21.15.152:6789/0 conn(0x7efd2c0b1e00 0x7efd2c0ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:56896/0 (socket says 172.21.15.140:56896) 2022-07-21T01:58:02.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd2ad9d700 1 -- 172.21.15.140:0/4052140779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c05c3f0 msgr2=0x7efd2c14a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:02.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd2ad9d700 1 --2- 172.21.15.140:0/4052140779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c05c3f0 0x7efd2c14a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd2ad9d700 1 -- 172.21.15.140:0/4052140779 >> v1:172.21.15.152:6789/0 conn(0x7efd2c0b1e00 legacy=0x7efd2c0ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd2ad9d700 1 -- 172.21.15.140:0/4052140779 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c05d1e0 con 0x7efd2c0acef0 2022-07-21T01:58:02.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd29d9b700 1 --2- 172.21.15.140:0/4052140779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c05c3f0 0x7efd2c14a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:58:02.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd2ad9d700 1 --2- 172.21.15.140:0/4052140779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd2c0acef0 0x7efd2c146410 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7efd14002f70 tx=0x7efd14006df0).ready entity=mon.1 client_cookie=9dd649e5c7c47bee server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:02.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd2959a700 1 -- 172.21.15.140:0/4052140779 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd1400d570 con 0x7efd2c0acef0 2022-07-21T01:58:02.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd2959a700 1 -- 172.21.15.140:0/4052140779 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd14000e90 con 0x7efd2c0acef0 2022-07-21T01:58:02.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd2959a700 1 -- 172.21.15.140:0/4052140779 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd1400d8c0 con 0x7efd2c0acef0 2022-07-21T01:58:02.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd30ca7700 1 -- 172.21.15.140:0/4052140779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd2c0acef0 msgr2=0x7efd2c146410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.243+0000 7efd30ca7700 1 --2- 172.21.15.140:0/4052140779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd2c0acef0 0x7efd2c146410 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7efd14002f70 tx=0x7efd14006df0).stop 2022-07-21T01:58:02.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 -- 172.21.15.140:0/4052140779 shutdown_connections 2022-07-21T01:58:02.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 --2- 172.21.15.140:0/4052140779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c05c3f0 0x7efd2c14a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 --2- 172.21.15.140:0/4052140779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd2c0acef0 0x7efd2c146410 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 -- 172.21.15.140:0/4052140779 >> 172.21.15.140:0/4052140779 conn(0x7efd2c0611c0 msgr2=0x7efd2c062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:02.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 -- 172.21.15.140:0/4052140779 shutdown_connections 2022-07-21T01:58:02.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 -- 172.21.15.140:0/4052140779 wait complete. 2022-07-21T01:58:02.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 Processor -- start 2022-07-21T01:58:02.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 -- start start 2022-07-21T01:58:02.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c0acef0 0x7efd2c159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd2c0b1e00 0x7efd2c1485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efd2c065890 0x7efd2c065d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c0649e0 con 0x7efd2c065890 2022-07-21T01:58:02.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c15a400 con 0x7efd2c0b1e00 2022-07-21T01:58:02.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd30ca7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c15a640 con 0x7efd2c0acef0 2022-07-21T01:58:02.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd2ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efd2c065890 0x7efd2c065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd29d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd2c0b1e00 0x7efd2c1485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd2ad9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efd2c065890 0x7efd2c065d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41646/0 (socket says 172.21.15.140:41646) 2022-07-21T01:58:02.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd2ad9d700 1 -- 172.21.15.140:0/862180723 learned_addr learned my addr 172.21.15.140:0/862180723 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:02.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd2a59c700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c0acef0 0x7efd2c159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd2ad9d700 1 -- 172.21.15.140:0/862180723 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c0acef0 msgr2=0x7efd2c159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd2ad9d700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c0acef0 0x7efd2c159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd2ad9d700 1 -- 172.21.15.140:0/862180723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd2c0b1e00 msgr2=0x7efd2c1485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd2ad9d700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd2c0b1e00 0x7efd2c1485d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd2ad9d700 1 -- 172.21.15.140:0/862180723 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd24009530 con 0x7efd2c065890 2022-07-21T01:58:02.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd2a59c700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c0acef0 0x7efd2c159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:58:02.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd2ad9d700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efd2c065890 0x7efd2c065d40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7efd14002f70 tx=0x7efd14014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:02.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd237fe700 1 -- 172.21.15.140:0/862180723 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd14013e90 con 0x7efd2c065890 2022-07-21T01:58:02.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.247+0000 7efd237fe700 1 -- 172.21.15.140:0/862180723 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1400de20 con 0x7efd2c065890 2022-07-21T01:58:02.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.251+0000 7efd30ca7700 1 -- 172.21.15.140:0/862180723 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1400b180 con 0x7efd2c065890 2022-07-21T01:58:02.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.251+0000 7efd237fe700 1 -- 172.21.15.140:0/862180723 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd1401e5f0 con 0x7efd2c065890 2022-07-21T01:58:02.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.251+0000 7efd30ca7700 1 -- 172.21.15.140:0/862180723 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2c063c80 con 0x7efd2c065890 2022-07-21T01:58:02.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.251+0000 7efd30ca7700 1 -- 172.21.15.140:0/862180723 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2c0b04b0 con 0x7efd2c065890 2022-07-21T01:58:02.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.251+0000 7efd237fe700 1 -- 172.21.15.140:0/862180723 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efd14025030 con 0x7efd2c065890 2022-07-21T01:58:02.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.251+0000 7efd237fe700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd100539e0 0x7efd10055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.251+0000 7efd237fe700 1 -- 172.21.15.140:0/862180723 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5253+0+0 (secure 0 0 0) 0x7efd1406c860 con 0x7efd2c065890 2022-07-21T01:58:02.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.251+0000 7efd2a59c700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd100539e0 0x7efd10055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.251+0000 7efd2a59c700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd100539e0 0x7efd10055ea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7efd1c00b000 tx=0x7efd1c007100).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:02.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.259+0000 7efd237fe700 1 -- 172.21.15.140:0/862180723 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efd14043ab0 con 0x7efd2c065890 2022-07-21T01:58:02.382 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.379+0000 7efd30ca7700 1 -- 172.21.15.140:0/862180723 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd2c069a60 con 0x7efd2c065890 2022-07-21T01:58:02.383 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.379+0000 7efd237fe700 1 -- 172.21.15.140:0/862180723 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+13287 (secure 0 0 0) 0x7efd14023020 con 0x7efd2c065890 2022-07-21T01:58:02.383 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:02.384 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":147,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:58:01.776852+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:58:02.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 -- 172.21.15.140:0/862180723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd100539e0 msgr2=0x7efd10055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd100539e0 0x7efd10055ea0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7efd1c00b000 tx=0x7efd1c007100).stop 2022-07-21T01:58:02.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 -- 172.21.15.140:0/862180723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efd2c065890 msgr2=0x7efd2c065d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efd2c065890 0x7efd2c065d40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7efd14002f70 tx=0x7efd14014000).stop 2022-07-21T01:58:02.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 -- 172.21.15.140:0/862180723 shutdown_connections 2022-07-21T01:58:02.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efd100539e0 0x7efd10055ea0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efd2c065890 0x7efd2c065d40 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd2c0b1e00 0x7efd2c1485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 --2- 172.21.15.140:0/862180723 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd2c0acef0 0x7efd2c159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 -- 172.21.15.140:0/862180723 >> 172.21.15.140:0/862180723 conn(0x7efd2c05c3f0 msgr2=0x7efd2c143280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:02.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 -- 172.21.15.140:0/862180723 shutdown_connections 2022-07-21T01:58:02.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.383+0000 7efd30ca7700 1 -- 172.21.15.140:0/862180723 wait complete. 2022-07-21T01:58:02.396 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-07-21T01:58:02.397 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:58:02.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31ffff700 1 Processor -- start 2022-07-21T01:58:02.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31ffff700 1 -- start start 2022-07-21T01:58:02.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3200aef30 0x7fd3201445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd320144a90 0x7fd320129340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31ffff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3200b5dd0 con 0x7fd3200b59f0 2022-07-21T01:58:02.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31ffff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3200b1750 con 0x7fd320144a90 2022-07-21T01:58:02.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd320140ef0 con 0x7fd3200aef30 2022-07-21T01:58:02.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31e7fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3200aef30 0x7fd3201445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31f7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd320144a90 0x7fd320129340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31effd700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd3200b59f0 0x7fd3201407e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:56910/0 (socket says 172.21.15.140:56910) 2022-07-21T01:58:02.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31f7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd320144a90 0x7fd320129340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:47394/0 (socket says 172.21.15.140:47394) 2022-07-21T01:58:02.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31effd700 1 -- 172.21.15.140:0/202884646 learned_addr learned my addr 172.21.15.140:0/202884646 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:02.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31dffb700 1 -- 172.21.15.140:0/202884646 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775888630 0 0) 0x7fd3200b5dd0 con 0x7fd3200b59f0 2022-07-21T01:58:02.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31dffb700 1 -- 172.21.15.140:0/202884646 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd300003260 con 0x7fd3200b59f0 2022-07-21T01:58:02.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31e7fc700 1 -- 172.21.15.140:0/202884646 >> v2:172.21.15.152:3300/0 conn(0x7fd320144a90 msgr2=0x7fd320129340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:02.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31e7fc700 1 --2- 172.21.15.140:0/202884646 >> v2:172.21.15.152:3300/0 conn(0x7fd320144a90 0x7fd320129340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31e7fc700 1 -- 172.21.15.140:0/202884646 >> v1:172.21.15.152:6789/0 conn(0x7fd3200b59f0 legacy=0x7fd3201407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31e7fc700 1 -- 172.21.15.140:0/202884646 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3200652a0 con 0x7fd3200aef30 2022-07-21T01:58:02.512 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31e7fc700 1 --2- 172.21.15.140:0/202884646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3200aef30 0x7fd3201445a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fd3100032f0 tx=0x7fd3100091d0).ready entity=mon.2 client_cookie=3a79cb713a50a297 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:02.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31dffb700 1 -- 172.21.15.140:0/202884646 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd31000b600 con 0x7fd3200aef30 2022-07-21T01:58:02.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31dffb700 1 -- 172.21.15.140:0/202884646 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31000b760 con 0x7fd3200aef30 2022-07-21T01:58:02.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.503+0000 7fd31dffb700 1 -- 172.21.15.140:0/202884646 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd31000b8d0 con 0x7fd3200aef30 2022-07-21T01:58:02.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 -- 172.21.15.140:0/202884646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3200aef30 msgr2=0x7fd3201445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 --2- 172.21.15.140:0/202884646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3200aef30 0x7fd3201445a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fd3100032f0 tx=0x7fd3100091d0).stop 2022-07-21T01:58:02.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 -- 172.21.15.140:0/202884646 shutdown_connections 2022-07-21T01:58:02.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 --2- 172.21.15.140:0/202884646 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3200aef30 0x7fd3201445a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 --2- 172.21.15.140:0/202884646 >> v2:172.21.15.152:3300/0 conn(0x7fd320144a90 0x7fd320129340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 -- 172.21.15.140:0/202884646 >> 172.21.15.140:0/202884646 conn(0x7fd320063480 msgr2=0x7fd3200b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:02.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 -- 172.21.15.140:0/202884646 shutdown_connections 2022-07-21T01:58:02.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 -- 172.21.15.140:0/202884646 wait complete. 2022-07-21T01:58:02.516 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:58:02.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 Processor -- start 2022-07-21T01:58:02.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 -- start start 2022-07-21T01:58:02.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3200aef30 0x7fd32012dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3200b59f0 0x7fd320131f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd320144a90 0x7fd32012f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd320130320 con 0x7fd320144a90 2022-07-21T01:58:02.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32012e0b0 con 0x7fd3200b59f0 2022-07-21T01:58:02.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd320131810 con 0x7fd3200aef30 2022-07-21T01:58:02.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd320144a90 0x7fd32012f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd320144a90 0x7fd32012f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41660/0 (socket says 172.21.15.140:41660) 2022-07-21T01:58:02.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31e7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3200b59f0 0x7fd320131f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31f7fe700 1 -- 172.21.15.140:0/3240139134 learned_addr learned my addr 172.21.15.140:0/3240139134 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:02.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.507+0000 7fd31effd700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3200aef30 0x7fd32012dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd31f7fe700 1 -- 172.21.15.140:0/3240139134 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3200aef30 msgr2=0x7fd32012dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd31f7fe700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3200aef30 0x7fd32012dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd31f7fe700 1 -- 172.21.15.140:0/3240139134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3200b59f0 msgr2=0x7fd320131f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd31f7fe700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3200b59f0 0x7fd320131f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd31f7fe700 1 -- 172.21.15.140:0/3240139134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd310009470 con 0x7fd320144a90 2022-07-21T01:58:02.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd31f7fe700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd320144a90 0x7fd32012f9a0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7fd31800c000 tx=0x7fd318006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:02.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd317fff700 1 -- 172.21.15.140:0/3240139134 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd31800eea0 con 0x7fd320144a90 2022-07-21T01:58:02.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd317fff700 1 -- 172.21.15.140:0/3240139134 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd318011750 con 0x7fd320144a90 2022-07-21T01:58:02.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd317fff700 1 -- 172.21.15.140:0/3240139134 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd318011ab0 con 0x7fd320144a90 2022-07-21T01:58:02.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd31ffff700 1 -- 172.21.15.140:0/3240139134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd320131ab0 con 0x7fd320144a90 2022-07-21T01:58:02.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd31ffff700 1 -- 172.21.15.140:0/3240139134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32012fe90 con 0x7fd320144a90 2022-07-21T01:58:02.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd31ffff700 1 -- 172.21.15.140:0/3240139134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd320131660 con 0x7fd320144a90 2022-07-21T01:58:02.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd317fff700 1 -- 172.21.15.140:0/3240139134 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd318011c10 con 0x7fd320144a90 2022-07-21T01:58:02.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd317fff700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd304053b30 0x7fd304055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.511+0000 7fd317fff700 1 -- 172.21.15.140:0/3240139134 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd31806bf60 con 0x7fd320144a90 2022-07-21T01:58:02.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.515+0000 7fd31effd700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd304053b30 0x7fd304055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.515+0000 7fd31effd700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd304053b30 0x7fd304055ff0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fd30800d000 tx=0x7fd308009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:02.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.519+0000 7fd317fff700 1 -- 172.21.15.140:0/3240139134 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd318043040 con 0x7fd320144a90 2022-07-21T01:58:02.529 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:58:02.532 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:58:02.535 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:58:02.538 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:58:02.643 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.639+0000 7fd31ffff700 1 -- 172.21.15.140:0/3240139134 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd320143e60 con 0x7fd320144a90 2022-07-21T01:58:02.644 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd317fff700 1 -- 172.21.15.140:0/3240139134 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v147) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd318046660 con 0x7fd320144a90 2022-07-21T01:58:02.645 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:58:02.646 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 -- 172.21.15.140:0/3240139134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd304053b30 msgr2=0x7fd304055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd304053b30 0x7fd304055ff0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fd30800d000 tx=0x7fd308009040).stop 2022-07-21T01:58:02.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 -- 172.21.15.140:0/3240139134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd320144a90 msgr2=0x7fd32012f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd320144a90 0x7fd32012f9a0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7fd31800c000 tx=0x7fd318006df0).stop 2022-07-21T01:58:02.647 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 -- 172.21.15.140:0/3240139134 shutdown_connections 2022-07-21T01:58:02.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd304053b30 0x7fd304055ff0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd320144a90 0x7fd32012f9a0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3200b59f0 0x7fd320131f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.648 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 --2- 172.21.15.140:0/3240139134 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd3200aef30 0x7fd32012dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 -- 172.21.15.140:0/3240139134 >> 172.21.15.140:0/3240139134 conn(0x7fd320063480 msgr2=0x7fd320129c20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:02.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 -- 172.21.15.140:0/3240139134 shutdown_connections 2022-07-21T01:58:02.649 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.643+0000 7fd31ffff700 1 -- 172.21.15.140:0/3240139134 wait complete. 2022-07-21T01:58:02.663 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:58:02.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.767+0000 7f900ffff700 1 Processor -- start 2022-07-21T01:58:02.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.767+0000 7f900ffff700 1 -- start start 2022-07-21T01:58:02.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.767+0000 7f900ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f90100aea20 0x7f90100b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.767+0000 7f900ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f90100acae0 0x7f90100afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.767+0000 7f900ffff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90100af3f0 con 0x7f90100b5690 2022-07-21T01:58:02.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.767+0000 7f900ffff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901013fff0 con 0x7f90100acae0 2022-07-21T01:58:02.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.767+0000 7f900ffff700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90100af150 con 0x7f90100aea20 2022-07-21T01:58:02.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.767+0000 7f900f7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f90100b5690 0x7f901013f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55812/0 (socket says 172.21.15.140:55812) 2022-07-21T01:58:02.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.767+0000 7f9007fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f90100acae0 0x7f90100afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.767+0000 7f900f7fe700 1 -- 172.21.15.140:0/1969820488 learned_addr learned my addr 172.21.15.140:0/1969820488 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:02.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900effd700 1 --2- 172.21.15.140:0/1969820488 >> v2:172.21.15.152:3300/0 conn(0x7f90100aea20 0x7f90100b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900e7fc700 1 -- 172.21.15.140:0/1969820488 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953422632 0 0) 0x7f90100af3f0 con 0x7f90100b5690 2022-07-21T01:58:02.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900e7fc700 1 -- 172.21.15.140:0/1969820488 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ff0003260 con 0x7f90100b5690 2022-07-21T01:58:02.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900effd700 1 -- 172.21.15.140:0/1969820488 >> v2:172.21.15.140:3300/0 conn(0x7f90100acae0 msgr2=0x7f90100afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:02.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900effd700 1 --2- 172.21.15.140:0/1969820488 >> v2:172.21.15.140:3300/0 conn(0x7f90100acae0 0x7f90100afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900effd700 1 -- 172.21.15.140:0/1969820488 >> v1:172.21.15.140:6789/0 conn(0x7f90100b5690 legacy=0x7f901013f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900effd700 1 -- 172.21.15.140:0/1969820488 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90100b0720 con 0x7f90100aea20 2022-07-21T01:58:02.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f9007fff700 1 --2- 172.21.15.140:0/1969820488 >> v2:172.21.15.140:3300/0 conn(0x7f90100acae0 0x7f90100afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:58:02.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900effd700 1 --2- 172.21.15.140:0/1969820488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90100aea20 0x7f90100b1480 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f9000002f70 tx=0x7f9000006df0).ready entity=mon.1 client_cookie=3ee5450e809d2581 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:02.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900e7fc700 1 -- 172.21.15.140:0/1969820488 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f900000d4f0 con 0x7f90100aea20 2022-07-21T01:58:02.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900e7fc700 1 -- 172.21.15.140:0/1969820488 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9000000e90 con 0x7f90100aea20 2022-07-21T01:58:02.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900e7fc700 1 -- 172.21.15.140:0/1969820488 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f900000d840 con 0x7f90100aea20 2022-07-21T01:58:02.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 -- 172.21.15.140:0/1969820488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90100aea20 msgr2=0x7f90100b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 --2- 172.21.15.140:0/1969820488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90100aea20 0x7f90100b1480 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f9000002f70 tx=0x7f9000006df0).stop 2022-07-21T01:58:02.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 -- 172.21.15.140:0/1969820488 shutdown_connections 2022-07-21T01:58:02.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 --2- 172.21.15.140:0/1969820488 >> v2:172.21.15.140:3300/0 conn(0x7f90100acae0 0x7f90100afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 --2- 172.21.15.140:0/1969820488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90100aea20 0x7f90100b1480 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 -- 172.21.15.140:0/1969820488 >> 172.21.15.140:0/1969820488 conn(0x7f90100634c0 msgr2=0x7f90100b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:02.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 -- 172.21.15.140:0/1969820488 shutdown_connections 2022-07-21T01:58:02.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 -- 172.21.15.140:0/1969820488 wait complete. 2022-07-21T01:58:02.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 Processor -- start 2022-07-21T01:58:02.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 -- start start 2022-07-21T01:58:02.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90100acae0 0x7f90100661a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90100aea20 0x7f9010151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90100b5690 0x7f9010151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901010f250 con 0x7f90100b5690 2022-07-21T01:58:02.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901010f480 con 0x7f90100acae0 2022-07-21T01:58:02.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900ffff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9010064670 con 0x7f90100aea20 2022-07-21T01:58:02.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90100b5690 0x7f9010151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90100b5690 0x7f9010151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41674/0 (socket says 172.21.15.140:41674) 2022-07-21T01:58:02.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900f7fe700 1 -- 172.21.15.140:0/2030584013 learned_addr learned my addr 172.21.15.140:0/2030584013 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:02.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f900effd700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90100acae0 0x7f90100661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.771+0000 7f9007fff700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90100aea20 0x7f9010151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900f7fe700 1 -- 172.21.15.140:0/2030584013 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90100aea20 msgr2=0x7f9010151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900f7fe700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90100aea20 0x7f9010151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900f7fe700 1 -- 172.21.15.140:0/2030584013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90100acae0 msgr2=0x7f90100661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900f7fe700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90100acae0 0x7f90100661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900f7fe700 1 -- 172.21.15.140:0/2030584013 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff800b180 con 0x7f90100b5690 2022-07-21T01:58:02.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900effd700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90100acae0 0x7f90100661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:58:02.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900f7fe700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90100b5690 0x7f9010151ef0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f9008001490 tx=0x7f9008007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:02.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900cff9700 1 -- 172.21.15.140:0/2030584013 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9008005ba0 con 0x7f90100b5690 2022-07-21T01:58:02.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900cff9700 1 -- 172.21.15.140:0/2030584013 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9008005d00 con 0x7f90100b5690 2022-07-21T01:58:02.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900cff9700 1 -- 172.21.15.140:0/2030584013 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90080072c0 con 0x7f90100b5690 2022-07-21T01:58:02.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900ffff700 1 -- 172.21.15.140:0/2030584013 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900000b180 con 0x7f90100b5690 2022-07-21T01:58:02.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900ffff700 1 -- 172.21.15.140:0/2030584013 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9010152600 con 0x7f90100b5690 2022-07-21T01:58:02.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900ffff700 1 -- 172.21.15.140:0/2030584013 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9010064ac0 con 0x7f90100b5690 2022-07-21T01:58:02.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.775+0000 7f900cff9700 1 -- 172.21.15.140:0/2030584013 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9008010070 con 0x7f90100b5690 2022-07-21T01:58:02.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.779+0000 7f900cff9700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 0x7f8ff4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:02.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.779+0000 7f900cff9700 1 -- 172.21.15.140:0/2030584013 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f900806b470 con 0x7f90100b5690 2022-07-21T01:58:02.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.779+0000 7f900effd700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 0x7f8ff4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:02.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.779+0000 7f900effd700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 0x7f8ff4055f30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f900001a000 tx=0x7f9000014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:02.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.783+0000 7f900cff9700 1 -- 172.21.15.140:0/2030584013 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9010064ac0 con 0x7f90100b5690 2022-07-21T01:58:02.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.907+0000 7f900ffff700 1 -- 172.21.15.140:0/2030584013 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9010064ac0 con 0x7f90100b5690 2022-07-21T01:58:02.910 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.907+0000 7f900cff9700 1 -- 172.21.15.140:0/2030584013 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v147) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9010064ac0 con 0x7f90100b5690 2022-07-21T01:58:02.910 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:58:02.912 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 -- 172.21.15.140:0/2030584013 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 msgr2=0x7f8ff4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 0x7f8ff4055f30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f900001a000 tx=0x7f9000014000).stop 2022-07-21T01:58:02.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 -- 172.21.15.140:0/2030584013 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90100b5690 msgr2=0x7f9010151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:02.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90100b5690 0x7f9010151ef0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f9008001490 tx=0x7f9008007e00).stop 2022-07-21T01:58:02.913 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 -- 172.21.15.140:0/2030584013 shutdown_connections 2022-07-21T01:58:02.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8ff4053a70 0x7f8ff4055f30 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f90100b5690 0x7f9010151ef0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f90100aea20 0x7f9010151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.914 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 --2- 172.21.15.140:0/2030584013 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90100acae0 0x7f90100661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:02.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 -- 172.21.15.140:0/2030584013 >> 172.21.15.140:0/2030584013 conn(0x7f90100634c0 msgr2=0x7f90100b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:02.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 -- 172.21.15.140:0/2030584013 shutdown_connections 2022-07-21T01:58:02.915 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:02.911+0000 7f900ffff700 1 -- 172.21.15.140:0/2030584013 wait complete. 2022-07-21T01:58:02.923 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-07-21T01:58:03.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.019+0000 7f40a869d700 1 Processor -- start 2022-07-21T01:58:03.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.019+0000 7f40a869d700 1 -- start start 2022-07-21T01:58:03.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f40a00b1e00 0x7f40a014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f40a005c3f0 0x7f40a00ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a005d480 con 0x7f40a005bd90 2022-07-21T01:58:03.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a00b21e0 con 0x7f40a005c3f0 2022-07-21T01:58:03.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a005cf40 con 0x7f40a00b1e00 2022-07-21T01:58:03.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a5c38700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f40a005c3f0 0x7f40a00ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a5c38700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f40a005c3f0 0x7f40a00ac7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41678/0 (socket says 172.21.15.140:41678) 2022-07-21T01:58:03.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a5c38700 1 -- 172.21.15.140:0/2671671817 learned_addr learned my addr 172.21.15.140:0/2671671817 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:03.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a6439700 1 --2- 172.21.15.140:0/2671671817 >> v2:172.21.15.152:3300/0 conn(0x7f40a00b1e00 0x7f40a014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a5c38700 1 -- 172.21.15.140:0/2671671817 >> v2:172.21.15.152:3300/0 conn(0x7f40a00b1e00 msgr2=0x7f40a014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:03.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a5c38700 1 --2- 172.21.15.140:0/2671671817 >> v2:172.21.15.152:3300/0 conn(0x7f40a00b1e00 0x7f40a014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a5c38700 1 -- 172.21.15.140:0/2671671817 >> v1:172.21.15.152:6789/0 conn(0x7f40a005bd90 legacy=0x7f40a0146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:03.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a5c38700 1 -- 172.21.15.140:0/2671671817 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a005d1e0 con 0x7f40a005c3f0 2022-07-21T01:58:03.030 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a6439700 1 --2- 172.21.15.140:0/2671671817 >> v2:172.21.15.152:3300/0 conn(0x7f40a00b1e00 0x7f40a014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:58:03.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a5c38700 1 --2- 172.21.15.140:0/2671671817 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40a005c3f0 0x7f40a00ac7e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f40a005f3a0 tx=0x7f409c006eb0).ready entity=mon.0 client_cookie=37440b7dcb8d8869 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:03.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a5437700 1 -- 172.21.15.140:0/2671671817 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409c00d5c0 con 0x7f40a005c3f0 2022-07-21T01:58:03.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a5437700 1 -- 172.21.15.140:0/2671671817 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c000e90 con 0x7f40a005c3f0 2022-07-21T01:58:03.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a5437700 1 -- 172.21.15.140:0/2671671817 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f409c00d910 con 0x7f40a005c3f0 2022-07-21T01:58:03.031 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 -- 172.21.15.140:0/2671671817 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40a005c3f0 msgr2=0x7f40a00ac7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:03.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 --2- 172.21.15.140:0/2671671817 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40a005c3f0 0x7f40a00ac7e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f40a005f3a0 tx=0x7f409c006eb0).stop 2022-07-21T01:58:03.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 -- 172.21.15.140:0/2671671817 shutdown_connections 2022-07-21T01:58:03.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 --2- 172.21.15.140:0/2671671817 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40a005c3f0 0x7f40a00ac7e0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 --2- 172.21.15.140:0/2671671817 >> v2:172.21.15.152:3300/0 conn(0x7f40a00b1e00 0x7f40a014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 -- 172.21.15.140:0/2671671817 >> 172.21.15.140:0/2671671817 conn(0x7f40a00611c0 msgr2=0x7f40a0062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:03.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 -- 172.21.15.140:0/2671671817 shutdown_connections 2022-07-21T01:58:03.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 -- 172.21.15.140:0/2671671817 wait complete. 2022-07-21T01:58:03.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.023+0000 7f40a869d700 1 Processor -- start 2022-07-21T01:58:03.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a869d700 1 -- start start 2022-07-21T01:58:03.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a869d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40a005c3f0 0x7f40a0066fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a869d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40a00b1e00 0x7f40a0063a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a869d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40a00b47f0 0x7f40a0063f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a869d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a013de00 con 0x7f40a00b47f0 2022-07-21T01:58:03.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a869d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a0149a50 con 0x7f40a005c3f0 2022-07-21T01:58:03.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a869d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00b4d70 con 0x7f40a00b1e00 2022-07-21T01:58:03.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a6c3a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40a00b47f0 0x7f40a0063f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a6c3a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40a00b47f0 0x7f40a0063f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41686/0 (socket says 172.21.15.140:41686) 2022-07-21T01:58:03.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a6c3a700 1 -- 172.21.15.140:0/1133058830 learned_addr learned my addr 172.21.15.140:0/1133058830 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:03.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a6439700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40a005c3f0 0x7f40a0066fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a5c38700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40a00b1e00 0x7f40a0063a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a6c3a700 1 -- 172.21.15.140:0/1133058830 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40a00b1e00 msgr2=0x7f40a0063a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:03.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a6c3a700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40a00b1e00 0x7f40a0063a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a6c3a700 1 -- 172.21.15.140:0/1133058830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40a005c3f0 msgr2=0x7f40a0066fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:03.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a6c3a700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40a005c3f0 0x7f40a0066fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a6c3a700 1 -- 172.21.15.140:0/1133058830 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c00b1e0 con 0x7f40a00b47f0 2022-07-21T01:58:03.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a5c38700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40a00b1e00 0x7f40a0063a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:58:03.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a6c3a700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40a00b47f0 0x7f40a0063f70 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f4098002320 tx=0x7f40980023f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:03.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f408f7fe700 1 -- 172.21.15.140:0/1133058830 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4098006aa0 con 0x7f40a00b47f0 2022-07-21T01:58:03.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f408f7fe700 1 -- 172.21.15.140:0/1133058830 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098006c00 con 0x7f40a00b47f0 2022-07-21T01:58:03.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a869d700 1 -- 172.21.15.140:0/1133058830 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a0149c80 con 0x7f40a00b47f0 2022-07-21T01:58:03.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f408f7fe700 1 -- 172.21.15.140:0/1133058830 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4098007030 con 0x7f40a00b47f0 2022-07-21T01:58:03.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a869d700 1 -- 172.21.15.140:0/1133058830 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a00b5e10 con 0x7f40a00b47f0 2022-07-21T01:58:03.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.027+0000 7f40a869d700 1 -- 172.21.15.140:0/1133058830 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a0149e10 con 0x7f40a00b47f0 2022-07-21T01:58:03.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.031+0000 7f408f7fe700 1 -- 172.21.15.140:0/1133058830 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f4098010030 con 0x7f40a00b47f0 2022-07-21T01:58:03.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.031+0000 7f408f7fe700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4080053a70 0x7f4080055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.031+0000 7f408f7fe700 1 -- 172.21.15.140:0/1133058830 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f409806bde0 con 0x7f40a00b47f0 2022-07-21T01:58:03.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.031+0000 7f40a6439700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4080053a70 0x7f4080055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.043 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.031+0000 7f40a6439700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4080053a70 0x7f4080055f30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f409000b000 tx=0x7f4090008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:03.044 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.035+0000 7f408f7fe700 1 -- 172.21.15.140:0/1133058830 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f4098042ea0 con 0x7f40a00b47f0 2022-07-21T01:58:03.163 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.159+0000 7f40a869d700 1 -- 172.21.15.140:0/1133058830 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f40a00b04b0 con 0x7f40a00b47f0 2022-07-21T01:58:03.164 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f408f7fe700 1 -- 172.21.15.140:0/1133058830 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v147) v1 ==== 84+0+143 (secure 0 0 0) 0x7f40980464c0 con 0x7f40a00b47f0 2022-07-21T01:58:03.165 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:03.165 INFO:teuthology.orchestra.run.smithi140.stdout:[ 2022-07-21T01:58:03.165 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:03.166 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 1, 2022-07-21T01:58:03.166 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "device_health_metrics" 2022-07-21T01:58:03.166 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:03.167 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:03.167 INFO:teuthology.orchestra.run.smithi140.stdout: "poolnum": 2, 2022-07-21T01:58:03.168 INFO:teuthology.orchestra.run.smithi140.stdout: "poolname": "rbd" 2022-07-21T01:58:03.168 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:03.169 INFO:teuthology.orchestra.run.smithi140.stdout:] 2022-07-21T01:58:03.172 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 -- 172.21.15.140:0/1133058830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4080053a70 msgr2=0x7f4080055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:03.173 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4080053a70 0x7f4080055f30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f409000b000 tx=0x7f4090008040).stop 2022-07-21T01:58:03.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 -- 172.21.15.140:0/1133058830 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40a00b47f0 msgr2=0x7f40a0063f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:03.174 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40a00b47f0 0x7f40a0063f70 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f4098002320 tx=0x7f40980023f0).stop 2022-07-21T01:58:03.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 -- 172.21.15.140:0/1133058830 shutdown_connections 2022-07-21T01:58:03.175 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f4080053a70 0x7f4080055f30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.176 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f40a00b47f0 0x7f40a0063f70 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f40a00b1e00 0x7f40a0063a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.177 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 --2- 172.21.15.140:0/1133058830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40a005c3f0 0x7f40a0066fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 -- 172.21.15.140:0/1133058830 >> 172.21.15.140:0/1133058830 conn(0x7f40a005bd90 msgr2=0x7f40a0144320 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:03.178 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 -- 172.21.15.140:0/1133058830 shutdown_connections 2022-07-21T01:58:03.179 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.163+0000 7f40a869d700 1 -- 172.21.15.140:0/1133058830 wait complete. 2022-07-21T01:58:03.182 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-07-21T01:58:03.183 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2022-07-21T01:58:03.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.287+0000 7f3227426700 1 Processor -- start 2022-07-21T01:58:03.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.287+0000 7f3227426700 1 -- start start 2022-07-21T01:58:03.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3227426700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f32200b1fa0 0x7f322013fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3227426700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322005bd90 con 0x7f32200b7bb0 2022-07-21T01:58:03.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3227426700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322005c820 con 0x7f32200b3770 2022-07-21T01:58:03.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3227426700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32201467d0 con 0x7f32200b1fa0 2022-07-21T01:58:03.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3226424700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f32200b7bb0 0x7f32201460c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55838/0 (socket says 172.21.15.140:55838) 2022-07-21T01:58:03.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3226424700 1 -- 172.21.15.140:0/280652339 learned_addr learned my addr 172.21.15.140:0/280652339 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:03.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3226c25700 1 --2- 172.21.15.140:0/280652339 >> v2:172.21.15.152:3300/0 conn(0x7f32200b1fa0 0x7f322013fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 53931110 0 0) 0x7f322005bd90 con 0x7f32200b7bb0 2022-07-21T01:58:03.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f320c003260 con 0x7f32200b7bb0 2022-07-21T01:58:03.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4012071652 0 0) 0x7f320c003260 con 0x7f32200b7bb0 2022-07-21T01:58:03.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 >> v2:172.21.15.152:3300/0 conn(0x7f32200b1fa0 msgr2=0x7f322013fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:03.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 --2- 172.21.15.140:0/280652339 >> v2:172.21.15.152:3300/0 conn(0x7f32200b1fa0 0x7f322013fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 >> v1:172.21.15.152:6789/0 conn(0x7f32200b3770 legacy=0x7f322005cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:03.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200b7f90 con 0x7f32200b7bb0 2022-07-21T01:58:03.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f3210001660 con 0x7f32200b7bb0 2022-07-21T01:58:03.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 >> v1:172.21.15.140:6789/0 conn(0x7f32200b7bb0 legacy=0x7f32201460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:03.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f320c003750 0x7f320c003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 0x7f320c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c004e70 0x7f320c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32201467d0 con 0x7f320c004500 2022-07-21T01:58:03.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c00bc70 con 0x7f320c004e70 2022-07-21T01:58:03.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320c00be30 con 0x7f320c003750 2022-07-21T01:58:03.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225c23700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 0x7f320c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3226424700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f320c003750 0x7f320c003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3226c25700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c004e70 0x7f320c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225c23700 1 -- 172.21.15.140:0/280652339 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f320c003750 msgr2=0x7f320c003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:03.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225c23700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f320c003750 0x7f320c003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225c23700 1 -- 172.21.15.140:0/280652339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c004e70 msgr2=0x7f320c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:03.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225c23700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c004e70 0x7f320c00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225c23700 1 -- 172.21.15.140:0/280652339 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c003260 con 0x7f320c004500 2022-07-21T01:58:03.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3226c25700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f320c004e70 0x7f320c00b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:58:03.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3226c25700 1 -- 172.21.15.140:0/280652339 reap_dead start 2022-07-21T01:58:03.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225c23700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 0x7f320c004980 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f3214001110 tx=0x7f3214007a00).ready entity=mon.0 client_cookie=b79d7ee5ab792a05 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:03.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3214007d10 con 0x7f320c004500 2022-07-21T01:58:03.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3214007030 con 0x7f320c004500 2022-07-21T01:58:03.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.291+0000 7f3225422700 1 -- 172.21.15.140:0/280652339 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3214007390 con 0x7f320c004500 2022-07-21T01:58:03.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.295+0000 7f3227426700 1 -- 172.21.15.140:0/280652339 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 msgr2=0x7f320c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:03.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.295+0000 7f3227426700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 0x7f320c004980 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f3214001110 tx=0x7f3214007a00).stop 2022-07-21T01:58:03.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.295+0000 7f3227426700 1 -- 172.21.15.140:0/280652339 shutdown_connections 2022-07-21T01:58:03.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.295+0000 7f3227426700 1 --2- 172.21.15.140:0/280652339 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 0x7f320c004980 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:03.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.295+0000 7f3227426700 1 -- 172.21.15.140:0/280652339 >> 172.21.15.140:0/280652339 conn(0x7f3220065720 msgr2=0x7f32200b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:03.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.295+0000 7f3227426700 1 -- 172.21.15.140:0/280652339 shutdown_connections 2022-07-21T01:58:03.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.295+0000 7f3227426700 1 -- 172.21.15.140:0/280652339 wait complete. 2022-07-21T01:58:03.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.295+0000 7f3227426700 1 Processor -- start 2022-07-21T01:58:03.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.295+0000 7f3227426700 1 -- start start 2022-07-21T01:58:03.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3227426700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 0x7f3220145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3227426700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3220116870 0x7f3220116cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3227426700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f32201171e0 0x7f322013f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3227426700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200afdf0 con 0x7f320c004500 2022-07-21T01:58:03.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3227426700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200affe0 con 0x7f3220116870 2022-07-21T01:58:03.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3227426700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32200b0cd0 con 0x7f32201171e0 2022-07-21T01:58:03.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3226424700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 0x7f3220145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3226424700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 0x7f3220145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41704/0 (socket says 172.21.15.140:41704) 2022-07-21T01:58:03.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3226424700 1 -- 172.21.15.140:0/1028282759 learned_addr learned my addr 172.21.15.140:0/1028282759 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:03.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3226424700 1 -- 172.21.15.140:0/1028282759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f32201171e0 msgr2=0x7f322013f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:03.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3226424700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f32201171e0 0x7f322013f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:58:03.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3226424700 1 -- 172.21.15.140:0/1028282759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3220116870 msgr2=0x7f3220116cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:03.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3226424700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3220116870 0x7f3220116cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:58:03.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3226424700 1 -- 172.21.15.140:0/1028282759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200b0f70 con 0x7f320c004500 2022-07-21T01:58:03.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3226424700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 0x7f3220145480 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f321000d000 tx=0x7f3210001e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:03.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f321f7fe700 1 -- 172.21.15.140:0/1028282759 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3210011780 con 0x7f320c004500 2022-07-21T01:58:03.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f321f7fe700 1 -- 172.21.15.140:0/1028282759 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32100118e0 con 0x7f320c004500 2022-07-21T01:58:03.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3227426700 1 -- 172.21.15.140:0/1028282759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3220146070 con 0x7f320c004500 2022-07-21T01:58:03.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f321f7fe700 1 -- 172.21.15.140:0/1028282759 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3210011c40 con 0x7f320c004500 2022-07-21T01:58:03.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3227426700 1 -- 172.21.15.140:0/1028282759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3220155250 con 0x7f320c004500 2022-07-21T01:58:03.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f3227426700 1 -- 172.21.15.140:0/1028282759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322005bfc0 con 0x7f320c004500 2022-07-21T01:58:03.315 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.299+0000 7f321f7fe700 1 -- 172.21.15.140:0/1028282759 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f321001b030 con 0x7f320c004500 2022-07-21T01:58:03.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.303+0000 7f321f7fe700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32000539e0 0x7f3200055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:03.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.303+0000 7f321f7fe700 1 -- 172.21.15.140:0/1028282759 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f321006b7f0 con 0x7f320c004500 2022-07-21T01:58:03.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.303+0000 7f3225c23700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32000539e0 0x7f3200055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:03.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.303+0000 7f3225c23700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32000539e0 0x7f3200055ea0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f3214001110 tx=0x7f3214015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:03.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.307+0000 7f321f7fe700 1 -- 172.21.15.140:0/1028282759 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f32100429a0 con 0x7f320c004500 2022-07-21T01:58:03.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:03.431+0000 7f3227426700 1 -- 172.21.15.140:0/1028282759 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f32200699b0 con 0x7f320c004500 2022-07-21T01:58:04.185 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.183+0000 7f321f7fe700 1 -- 172.21.15.140:0/1028282759 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v148) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3210019030 con 0x7f320c004500 2022-07-21T01:58:04.188 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 -- 172.21.15.140:0/1028282759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32000539e0 msgr2=0x7f3200055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32000539e0 0x7f3200055ea0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f3214001110 tx=0x7f3214015000).stop 2022-07-21T01:58:04.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 -- 172.21.15.140:0/1028282759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 msgr2=0x7f3220145480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 0x7f3220145480 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f321000d000 tx=0x7f3210001e30).stop 2022-07-21T01:58:04.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 -- 172.21.15.140:0/1028282759 shutdown_connections 2022-07-21T01:58:04.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f32000539e0 0x7f3200055ea0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f32201171e0 0x7f322013f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:58:04.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3220116870 0x7f3220116cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2022-07-21T01:58:04.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 --2- 172.21.15.140:0/1028282759 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f320c004500 0x7f3220145480 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 -- 172.21.15.140:0/1028282759 >> 172.21.15.140:0/1028282759 conn(0x7f3220065720 msgr2=0x7f3220143330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:04.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 -- 172.21.15.140:0/1028282759 shutdown_connections 2022-07-21T01:58:04.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.187+0000 7f3227426700 1 -- 172.21.15.140:0/1028282759 wait complete. 2022-07-21T01:58:04.192 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' created 2022-07-21T01:58:04.203 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2022-07-21T01:58:04.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f38454c4700 1 Processor -- start 2022-07-21T01:58:04.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f38454c4700 1 -- start start 2022-07-21T01:58:04.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f38454c4700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f384005d160 0x7f38400b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:04.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f38454c4700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38400b3f10 con 0x7f38401461c0 2022-07-21T01:58:04.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f38454c4700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38400b3790 con 0x7f38400b2ec0 2022-07-21T01:58:04.316 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f38454c4700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38400b4ea0 con 0x7f384005d160 2022-07-21T01:58:04.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f383f7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f384005d160 0x7f38400b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:04.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f3844cc3700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f38401461c0 0x7f38401465f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55862/0 (socket says 172.21.15.140:55862) 2022-07-21T01:58:04.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f3844cc3700 1 -- 172.21.15.140:0/139217997 learned_addr learned my addr 172.21.15.140:0/139217997 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:04.317 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f383effd700 1 -- 172.21.15.140:0/139217997 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205618326 0 0) 0x7f38400b3f10 con 0x7f38401461c0 2022-07-21T01:58:04.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f383effd700 1 -- 172.21.15.140:0/139217997 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f382c003260 con 0x7f38401461c0 2022-07-21T01:58:04.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f383effd700 1 -- 172.21.15.140:0/139217997 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298038285 0 0) 0x7f38400b3790 con 0x7f38400b2ec0 2022-07-21T01:58:04.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f383effd700 1 -- 172.21.15.140:0/139217997 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38400b3f10 con 0x7f38400b2ec0 2022-07-21T01:58:04.318 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f383f7fe700 1 -- 172.21.15.140:0/139217997 >> v1:172.21.15.152:6789/0 conn(0x7f38400b2ec0 legacy=0x7f3840142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f383f7fe700 1 -- 172.21.15.140:0/139217997 >> v1:172.21.15.140:6789/0 conn(0x7f38401461c0 legacy=0x7f38401465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.311+0000 7f383f7fe700 1 -- 172.21.15.140:0/139217997 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38400b2620 con 0x7f384005d160 2022-07-21T01:58:04.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f383f7fe700 1 --2- 172.21.15.140:0/139217997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f384005d160 0x7f38400b32a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7f3834002f70 tx=0x7f3834006df0).ready entity=mon.0 client_cookie=de59d3cdbd5d181d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:04.319 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f383effd700 1 -- 172.21.15.140:0/139217997 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f383400d540 con 0x7f384005d160 2022-07-21T01:58:04.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f383effd700 1 -- 172.21.15.140:0/139217997 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834000e90 con 0x7f384005d160 2022-07-21T01:58:04.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f383effd700 1 -- 172.21.15.140:0/139217997 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f383400d890 con 0x7f384005d160 2022-07-21T01:58:04.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f38454c4700 1 -- 172.21.15.140:0/139217997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f384005d160 msgr2=0x7f38400b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f38454c4700 1 --2- 172.21.15.140:0/139217997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f384005d160 0x7f38400b32a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7f3834002f70 tx=0x7f3834006df0).stop 2022-07-21T01:58:04.324 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f38454c4700 1 -- 172.21.15.140:0/139217997 shutdown_connections 2022-07-21T01:58:04.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f38454c4700 1 --2- 172.21.15.140:0/139217997 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f384005d160 0x7f38400b32a0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f38454c4700 1 -- 172.21.15.140:0/139217997 >> 172.21.15.140:0/139217997 conn(0x7f3840066e20 msgr2=0x7f3840067200 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:04.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f38454c4700 1 -- 172.21.15.140:0/139217997 shutdown_connections 2022-07-21T01:58:04.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f38454c4700 1 -- 172.21.15.140:0/139217997 wait complete. 2022-07-21T01:58:04.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f38454c4700 1 Processor -- start 2022-07-21T01:58:04.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.315+0000 7f38454c4700 1 -- start start 2022-07-21T01:58:04.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f38454c4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f384005d160 0x7f3840140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:04.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f38454c4700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f38400b2ec0 0x7f38400b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:04.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f38454c4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38401461c0 0x7f38401523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:04.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f38454c4700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400b8200 con 0x7f38400b2ec0 2022-07-21T01:58:04.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f38454c4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400b84a0 con 0x7f384005d160 2022-07-21T01:58:04.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f38454c4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38401413b0 con 0x7f38401461c0 2022-07-21T01:58:04.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f38400b2ec0 0x7f38400b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:04.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f38400b2ec0 0x7f38400b8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41720/0 (socket says 172.21.15.140:41720) 2022-07-21T01:58:04.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383f7fe700 1 -- 172.21.15.140:0/4128492111 learned_addr learned my addr 172.21.15.140:0/4128492111 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:04.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f3844cc3700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38401461c0 0x7f38401523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:04.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383ffff700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f384005d160 0x7f3840140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:04.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383f7fe700 1 -- 172.21.15.140:0/4128492111 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38401461c0 msgr2=0x7f38401523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383f7fe700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38401461c0 0x7f38401523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.329 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383f7fe700 1 -- 172.21.15.140:0/4128492111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f384005d160 msgr2=0x7f3840140ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383f7fe700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f384005d160 0x7f3840140ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383f7fe700 1 -- 172.21.15.140:0/4128492111 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383400b180 con 0x7f38400b2ec0 2022-07-21T01:58:04.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383f7fe700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f38400b2ec0 0x7f38400b8940 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f3834002f70 tx=0x7f3834014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:04.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383d7fa700 1 -- 172.21.15.140:0/4128492111 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f383400d1c0 con 0x7f38400b2ec0 2022-07-21T01:58:04.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383d7fa700 1 -- 172.21.15.140:0/4128492111 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834013bf0 con 0x7f38400b2ec0 2022-07-21T01:58:04.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f38454c4700 1 -- 172.21.15.140:0/4128492111 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3840155980 con 0x7f38400b2ec0 2022-07-21T01:58:04.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f383d7fa700 1 -- 172.21.15.140:0/4128492111 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f383401e400 con 0x7f38400b2ec0 2022-07-21T01:58:04.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f38454c4700 1 -- 172.21.15.140:0/4128492111 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38400b9050 con 0x7f38400b2ec0 2022-07-21T01:58:04.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.319+0000 7f38454c4700 1 -- 172.21.15.140:0/4128492111 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38400b28c0 con 0x7f38400b2ec0 2022-07-21T01:58:04.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.323+0000 7f383d7fa700 1 -- 172.21.15.140:0/4128492111 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3834025030 con 0x7f38400b2ec0 2022-07-21T01:58:04.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.323+0000 7f383d7fa700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3820053a70 0x7f3820055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:04.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.323+0000 7f383d7fa700 1 -- 172.21.15.140:0/4128492111 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f383406ca70 con 0x7f38400b2ec0 2022-07-21T01:58:04.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.323+0000 7f383ffff700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3820053a70 0x7f3820055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:04.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.323+0000 7f383ffff700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3820053a70 0x7f3820055f30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f383000d000 tx=0x7f3830007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:04.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.327+0000 7f383d7fa700 1 -- 172.21.15.140:0/4128492111 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3834043ba0 con 0x7f38400b2ec0 2022-07-21T01:58:04.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.447+0000 7f38454c4700 1 -- 172.21.15.140:0/4128492111 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f38400574e0 con 0x7f38400b2ec0 2022-07-21T01:58:04.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.799+0000 7f383d7fa700 1 -- 172.21.15.140:0/4128492111 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v149) v1 ==== 125+0+0 (secure 0 0 0) 0x7f383406ce60 con 0x7f38400b2ec0 2022-07-21T01:58:04.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 -- 172.21.15.140:0/4128492111 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3820053a70 msgr2=0x7f3820055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3820053a70 0x7f3820055f30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f383000d000 tx=0x7f3830007a20).stop 2022-07-21T01:58:04.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 -- 172.21.15.140:0/4128492111 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f38400b2ec0 msgr2=0x7f38400b8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f38400b2ec0 0x7f38400b8940 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f3834002f70 tx=0x7f3834014000).stop 2022-07-21T01:58:04.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 -- 172.21.15.140:0/4128492111 shutdown_connections 2022-07-21T01:58:04.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3820053a70 0x7f3820055f30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38401461c0 0x7f38401523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f38400b2ec0 0x7f38400b8940 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 --2- 172.21.15.140:0/4128492111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f384005d160 0x7f3840140ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 -- 172.21.15.140:0/4128492111 >> 172.21.15.140:0/4128492111 conn(0x7f3840066e20 msgr2=0x7f3840143550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:04.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 -- 172.21.15.140:0/4128492111 shutdown_connections 2022-07-21T01:58:04.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.807+0000 7f38454c4700 1 -- 172.21.15.140:0/4128492111 wait complete. 2022-07-21T01:58:04.815 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' created 2022-07-21T01:58:04.823 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2022-07-21T01:58:04.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.919+0000 7fcaf4eec700 1 Processor -- start 2022-07-21T01:58:04.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.919+0000 7fcaf4eec700 1 -- start start 2022-07-21T01:58:04.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaf005f9e0 0x7fcaf00616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:04.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcaf005fdc0 0x7fcaf013eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:04.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf013f920 con 0x7fcaf013f3a0 2022-07-21T01:58:04.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf00ab9d0 con 0x7fcaf005fdc0 2022-07-21T01:58:04.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00abc70 con 0x7fcaf005f9e0 2022-07-21T01:58:04.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaeeffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcaf005fdc0 0x7fcaf013eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:04.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaeffff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fcaf013f3a0 0x7fcaf013fd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55876/0 (socket says 172.21.15.140:55876) 2022-07-21T01:58:04.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaeffff700 1 -- 172.21.15.140:0/3701007693 learned_addr learned my addr 172.21.15.140:0/3701007693 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:04.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaef7fe700 1 --2- 172.21.15.140:0/3701007693 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaf005f9e0 0x7fcaf00616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:04.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaeeffd700 1 -- 172.21.15.140:0/3701007693 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaf005f9e0 msgr2=0x7fcaf00616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:04.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaeeffd700 1 --2- 172.21.15.140:0/3701007693 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaf005f9e0 0x7fcaf00616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaeeffd700 1 -- 172.21.15.140:0/3701007693 >> v1:172.21.15.140:6789/0 conn(0x7fcaf013f3a0 legacy=0x7fcaf013fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaeeffd700 1 -- 172.21.15.140:0/3701007693 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf00ad110 con 0x7fcaf005fdc0 2022-07-21T01:58:04.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaeeffd700 1 --2- 172.21.15.140:0/3701007693 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaf005fdc0 0x7fcaf013eeb0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fcad8002f70 tx=0x7fcad8006df0).ready entity=mon.0 client_cookie=ce92beaece020ff8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:04.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaee7fc700 1 -- 172.21.15.140:0/3701007693 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcad800d570 con 0x7fcaf005fdc0 2022-07-21T01:58:04.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaee7fc700 1 -- 172.21.15.140:0/3701007693 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad8000e90 con 0x7fcaf005fdc0 2022-07-21T01:58:04.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaee7fc700 1 -- 172.21.15.140:0/3701007693 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcad800d8c0 con 0x7fcaf005fdc0 2022-07-21T01:58:04.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 -- 172.21.15.140:0/3701007693 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaf005fdc0 msgr2=0x7fcaf013eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 --2- 172.21.15.140:0/3701007693 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaf005fdc0 0x7fcaf013eeb0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fcad8002f70 tx=0x7fcad8006df0).stop 2022-07-21T01:58:04.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 -- 172.21.15.140:0/3701007693 shutdown_connections 2022-07-21T01:58:04.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 --2- 172.21.15.140:0/3701007693 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaf005fdc0 0x7fcaf013eeb0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 --2- 172.21.15.140:0/3701007693 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaf005f9e0 0x7fcaf00616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 -- 172.21.15.140:0/3701007693 >> 172.21.15.140:0/3701007693 conn(0x7fcaf00b6050 msgr2=0x7fcaf00b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:04.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 -- 172.21.15.140:0/3701007693 shutdown_connections 2022-07-21T01:58:04.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.923+0000 7fcaf4eec700 1 -- 172.21.15.140:0/3701007693 wait complete. 2022-07-21T01:58:04.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.927+0000 7fcaf4eec700 1 Processor -- start 2022-07-21T01:58:04.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.927+0000 7fcaf4eec700 1 -- start start 2022-07-21T01:58:04.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.927+0000 7fcaf4eec700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaf005f9e0 0x7fcaf0065830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:04.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.927+0000 7fcaf4eec700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf005fdc0 0x7fcaf0152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:04.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.927+0000 7fcaf4eec700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaf013f3a0 0x7fcaf0152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:04.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaf4eec700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0099350 con 0x7fcaf013f3a0 2022-07-21T01:58:04.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaf4eec700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00aed70 con 0x7fcaf005fdc0 2022-07-21T01:58:04.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaf4eec700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00af010 con 0x7fcaf005f9e0 2022-07-21T01:58:04.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaeffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaf013f3a0 0x7fcaf0152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:04.938 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaeffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaf013f3a0 0x7fcaf0152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41736/0 (socket says 172.21.15.140:41736) 2022-07-21T01:58:04.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaeffff700 1 -- 172.21.15.140:0/2204014800 learned_addr learned my addr 172.21.15.140:0/2204014800 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:04.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaef7fe700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaf005f9e0 0x7fcaf0065830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:04.939 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaeeffd700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf005fdc0 0x7fcaf0152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:04.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaeffff700 1 -- 172.21.15.140:0/2204014800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaf005f9e0 msgr2=0x7fcaf0065830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaeffff700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaf005f9e0 0x7fcaf0065830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaeffff700 1 -- 172.21.15.140:0/2204014800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf005fdc0 msgr2=0x7fcaf0152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:04.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaeffff700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf005fdc0 0x7fcaf0152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:04.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaeffff700 1 -- 172.21.15.140:0/2204014800 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad800b180 con 0x7fcaf013f3a0 2022-07-21T01:58:04.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaeeffd700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf005fdc0 0x7fcaf0152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:58:04.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaeffff700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaf013f3a0 0x7fcaf0152930 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fcaf013fc30 tx=0x7fcae8007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:04.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaecff9700 1 -- 172.21.15.140:0/2204014800 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcae8005610 con 0x7fcaf013f3a0 2022-07-21T01:58:04.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaecff9700 1 -- 172.21.15.140:0/2204014800 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae8005770 con 0x7fcaf013f3a0 2022-07-21T01:58:04.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaecff9700 1 -- 172.21.15.140:0/2204014800 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcae8005ad0 con 0x7fcaf013f3a0 2022-07-21T01:58:04.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaf4eec700 1 -- 172.21.15.140:0/2204014800 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf00af2b0 con 0x7fcaf013f3a0 2022-07-21T01:58:04.942 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaf4eec700 1 -- 172.21.15.140:0/2204014800 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf00b38f0 con 0x7fcaf013f3a0 2022-07-21T01:58:04.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaecff9700 1 -- 172.21.15.140:0/2204014800 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fcae8005c30 con 0x7fcaf013f3a0 2022-07-21T01:58:04.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.931+0000 7fcaf4eec700 1 -- 172.21.15.140:0/2204014800 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf00574e0 con 0x7fcaf013f3a0 2022-07-21T01:58:04.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.935+0000 7fcaecff9700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcad4053a70 0x7fcad4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:04.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.935+0000 7fcaecff9700 1 -- 172.21.15.140:0/2204014800 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fcae806e3e0 con 0x7fcaf013f3a0 2022-07-21T01:58:04.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.935+0000 7fcaef7fe700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcad4053a70 0x7fcad4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:04.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.935+0000 7fcaef7fe700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcad4053a70 0x7fcad4055f30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fcae000b000 tx=0x7fcae00091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:04.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:04.939+0000 7fcaecff9700 1 -- 172.21.15.140:0/2204014800 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fcae80452f0 con 0x7fcaf013f3a0 2022-07-21T01:58:05.105 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.103+0000 7fcaf4eec700 1 -- 172.21.15.140:0/2204014800 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fcaf005db10 con 0x7fcaf013f3a0 2022-07-21T01:58:05.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.111+0000 7fcaecff9700 1 -- 172.21.15.140:0/2204014800 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 29 and data pool 30 v73) v1 ==== 189+0+0 (secure 0 0 0) 0x7fcae8048910 con 0x7fcaf013f3a0 2022-07-21T01:58:05.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.115+0000 7fcaf4eec700 1 -- 172.21.15.140:0/2204014800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcad4053a70 msgr2=0x7fcad4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.115+0000 7fcaf4eec700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcad4053a70 0x7fcad4055f30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fcae000b000 tx=0x7fcae00091d0).stop 2022-07-21T01:58:05.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.115+0000 7fcaf4eec700 1 -- 172.21.15.140:0/2204014800 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaf013f3a0 msgr2=0x7fcaf0152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.115+0000 7fcaf4eec700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaf013f3a0 0x7fcaf0152930 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fcaf013fc30 tx=0x7fcae8007c70).stop 2022-07-21T01:58:05.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.115+0000 7fcaf4eec700 1 -- 172.21.15.140:0/2204014800 shutdown_connections 2022-07-21T01:58:05.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.115+0000 7fcaf4eec700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fcad4053a70 0x7fcad4055f30 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.115+0000 7fcaf4eec700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaf013f3a0 0x7fcaf0152930 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.115+0000 7fcaf4eec700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaf005fdc0 0x7fcaf0152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.115+0000 7fcaf4eec700 1 --2- 172.21.15.140:0/2204014800 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaf005f9e0 0x7fcaf0065830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.115+0000 7fcaf4eec700 1 -- 172.21.15.140:0/2204014800 >> 172.21.15.140:0/2204014800 conn(0x7fcaf00b6050 msgr2=0x7fcaf00ac550 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:05.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.119+0000 7fcaf4eec700 1 -- 172.21.15.140:0/2204014800 shutdown_connections 2022-07-21T01:58:05.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.119+0000 7fcaf4eec700 1 -- 172.21.15.140:0/2204014800 wait complete. 2022-07-21T01:58:05.126 INFO:teuthology.orchestra.run.smithi140.stderr:new fs with metadata pool 29 and data pool 30 2022-07-21T01:58:05.133 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:58:05.222 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.219+0000 7f1d0ec3c700 1 Processor -- start 2022-07-21T01:58:05.223 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.219+0000 7f1d0ec3c700 1 -- start start 2022-07-21T01:58:05.226 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.219+0000 7f1d0ec3c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1d080b1ef0 0x7f1d080ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.219+0000 7f1d0ec3c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080b22d0 con 0x7f1d08061020 2022-07-21T01:58:05.227 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.219+0000 7f1d0ec3c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d080ac1c0 con 0x7f1d08141f80 2022-07-21T01:58:05.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.219+0000 7f1d0ec3c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d080ac460 con 0x7f1d080b1ef0 2022-07-21T01:58:05.228 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.219+0000 7f1d0dc3a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1d080b1ef0 0x7f1d080ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.219+0000 7f1d0e43b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1d08061020 0x7f1d08061450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55894/0 (socket says 172.21.15.140:55894) 2022-07-21T01:58:05.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.219+0000 7f1d0e43b700 1 -- 172.21.15.140:0/1648238346 learned_addr learned my addr 172.21.15.140:0/1648238346 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:05.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.219+0000 7f1d0dc3a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f1d080b1ef0 0x7f1d080ab3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41746/0 (socket says 172.21.15.140:41746) 2022-07-21T01:58:05.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0dc3a700 1 -- 172.21.15.140:0/1648238346 >> v1:172.21.15.152:6789/0 conn(0x7f1d08141f80 legacy=0x7f1d081423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0dc3a700 1 -- 172.21.15.140:0/1648238346 >> v1:172.21.15.140:6789/0 conn(0x7f1d08061020 legacy=0x7f1d08061450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0dc3a700 1 -- 172.21.15.140:0/1648238346 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080ac700 con 0x7f1d080b1ef0 2022-07-21T01:58:05.231 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0dc3a700 1 --2- 172.21.15.140:0/1648238346 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d080b1ef0 0x7f1d080ab3b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f1d04002f70 tx=0x7f1d04006df0).ready entity=mon.0 client_cookie=e243d1cf1bfb4d10 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:05.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0cc38700 1 -- 172.21.15.140:0/1648238346 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0400d610 con 0x7f1d080b1ef0 2022-07-21T01:58:05.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0cc38700 1 -- 172.21.15.140:0/1648238346 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d04000e90 con 0x7f1d080b1ef0 2022-07-21T01:58:05.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0cc38700 1 -- 172.21.15.140:0/1648238346 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0400d960 con 0x7f1d080b1ef0 2022-07-21T01:58:05.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/1648238346 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d080b1ef0 msgr2=0x7f1d080ab3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0ec3c700 1 --2- 172.21.15.140:0/1648238346 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d080b1ef0 0x7f1d080ab3b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f1d04002f70 tx=0x7f1d04006df0).stop 2022-07-21T01:58:05.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/1648238346 shutdown_connections 2022-07-21T01:58:05.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0ec3c700 1 --2- 172.21.15.140:0/1648238346 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d080b1ef0 0x7f1d080ab3b0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/1648238346 >> 172.21.15.140:0/1648238346 conn(0x7f1d080b2ca0 msgr2=0x7f1d080b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:05.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/1648238346 shutdown_connections 2022-07-21T01:58:05.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/1648238346 wait complete. 2022-07-21T01:58:05.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0ec3c700 1 Processor -- start 2022-07-21T01:58:05.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.223+0000 7f1d0ec3c700 1 -- start start 2022-07-21T01:58:05.240 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0ec3c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d08061020 0x7f1d080b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0ec3c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d080b1ef0 0x7f1d0813b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.241 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0ec3c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d08141f80 0x7f1d0813ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.242 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0ec3c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d080b0240 con 0x7f1d08141f80 2022-07-21T01:58:05.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0ec3c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0813bf30 con 0x7f1d08061020 2022-07-21T01:58:05.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0ec3c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d08152ae0 con 0x7f1d080b1ef0 2022-07-21T01:58:05.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0e43b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d08141f80 0x7f1d0813ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0e43b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d08141f80 0x7f1d0813ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41756/0 (socket says 172.21.15.140:41756) 2022-07-21T01:58:05.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0e43b700 1 -- 172.21.15.140:0/3147894181 learned_addr learned my addr 172.21.15.140:0/3147894181 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:05.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0dc3a700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d08061020 0x7f1d080b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0d439700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d080b1ef0 0x7f1d0813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0e43b700 1 -- 172.21.15.140:0/3147894181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d080b1ef0 msgr2=0x7f1d0813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0e43b700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d080b1ef0 0x7f1d0813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0e43b700 1 -- 172.21.15.140:0/3147894181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d08061020 msgr2=0x7f1d080b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0e43b700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d08061020 0x7f1d080b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0e43b700 1 -- 172.21.15.140:0/3147894181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0400b180 con 0x7f1d08141f80 2022-07-21T01:58:05.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0d439700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d080b1ef0 0x7f1d0813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:58:05.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0e43b700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d08141f80 0x7f1d0813ba40 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f1d080b3f60 tx=0x7f1d00007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:05.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1cf6ffd700 1 -- 172.21.15.140:0/3147894181 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d00005ba0 con 0x7f1d08141f80 2022-07-21T01:58:05.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1cf6ffd700 1 -- 172.21.15.140:0/3147894181 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d00005d00 con 0x7f1d08141f80 2022-07-21T01:58:05.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/3147894181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d08152d80 con 0x7f1d08141f80 2022-07-21T01:58:05.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1cf6ffd700 1 -- 172.21.15.140:0/3147894181 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d000072c0 con 0x7f1d08141f80 2022-07-21T01:58:05.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/3147894181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d080ad640 con 0x7f1d08141f80 2022-07-21T01:58:05.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/3147894181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d08152920 con 0x7f1d08141f80 2022-07-21T01:58:05.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.227+0000 7f1cf6ffd700 1 -- 172.21.15.140:0/3147894181 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1d00010070 con 0x7f1d08141f80 2022-07-21T01:58:05.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.231+0000 7f1cf6ffd700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ce8053b00 0x7f1ce8055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.231+0000 7f1cf6ffd700 1 -- 172.21.15.140:0/3147894181 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1d0006b810 con 0x7f1d08141f80 2022-07-21T01:58:05.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.231+0000 7f1d0dc3a700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ce8053b00 0x7f1ce8055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.231+0000 7f1d0dc3a700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ce8053b00 0x7f1ce8055fc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f1d0401c000 tx=0x7f1d04015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:05.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.235+0000 7f1cf6ffd700 1 -- 172.21.15.140:0/3147894181 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1d00042690 con 0x7f1d08141f80 2022-07-21T01:58:05.360 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.359+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/3147894181 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d08142af0 con 0x7f1d08141f80 2022-07-21T01:58:05.362 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.359+0000 7f1cf6ffd700 1 -- 172.21.15.140:0/3147894181 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16161 (secure 0 0 0) 0x7f1d00018020 con 0x7f1d08141f80 2022-07-21T01:58:05.363 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:05.363 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":150,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:58:05.106735+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:58:03.485327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-07-21T01:58:04.503066+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:58:05.364 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/3147894181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ce8053b00 msgr2=0x7f1ce8055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ce8053b00 0x7f1ce8055fc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f1d0401c000 tx=0x7f1d04015000).stop 2022-07-21T01:58:05.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/3147894181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d08141f80 msgr2=0x7f1d0813ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d08141f80 0x7f1d0813ba40 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f1d080b3f60 tx=0x7f1d00007f40).stop 2022-07-21T01:58:05.365 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/3147894181 shutdown_connections 2022-07-21T01:58:05.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1ce8053b00 0x7f1ce8055fc0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1d08141f80 0x7f1d0813ba40 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.366 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1d080b1ef0 0x7f1d0813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 --2- 172.21.15.140:0/3147894181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d08061020 0x7f1d080b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/3147894181 >> 172.21.15.140:0/3147894181 conn(0x7f1d080b2ca0 msgr2=0x7f1d08148ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:05.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/3147894181 shutdown_connections 2022-07-21T01:58:05.367 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.363+0000 7f1d0ec3c700 1 -- 172.21.15.140:0/3147894181 wait complete. 2022-07-21T01:58:05.376 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:58:05.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.475+0000 7fdd5b1b3700 1 Processor -- start 2022-07-21T01:58:05.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.475+0000 7fdd5b1b3700 1 -- start start 2022-07-21T01:58:05.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.475+0000 7fdd5b1b3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 0x7fdd540b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.475+0000 7fdd5b1b3700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd54147010 con 0x7fdd540adff0 2022-07-21T01:58:05.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.475+0000 7fdd5b1b3700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd541472b0 con 0x7fdd540b5650 2022-07-21T01:58:05.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.475+0000 7fdd5b1b3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54146d40 con 0x7fdd540af8e0 2022-07-21T01:58:05.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.475+0000 7fdd599b0700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fdd540adff0 0x7fdd5405bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55906/0 (socket says 172.21.15.140:55906) 2022-07-21T01:58:05.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.475+0000 7fdd599b0700 1 -- 172.21.15.140:0/314185161 learned_addr learned my addr 172.21.15.140:0/314185161 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:05.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.475+0000 7fdd5a9b2700 1 --2- 172.21.15.140:0/314185161 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 0x7fdd540b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.475+0000 7fdd591af700 1 -- 172.21.15.140:0/314185161 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518327047 0 0) 0x7fdd54147010 con 0x7fdd540adff0 2022-07-21T01:58:05.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.475+0000 7fdd591af700 1 -- 172.21.15.140:0/314185161 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd40003260 con 0x7fdd540adff0 2022-07-21T01:58:05.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd591af700 1 -- 172.21.15.140:0/314185161 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019956681 0 0) 0x7fdd541472b0 con 0x7fdd540b5650 2022-07-21T01:58:05.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd591af700 1 -- 172.21.15.140:0/314185161 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd54147010 con 0x7fdd540b5650 2022-07-21T01:58:05.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5a9b2700 1 -- 172.21.15.140:0/314185161 >> v1:172.21.15.152:6789/0 conn(0x7fdd540b5650 legacy=0x7fdd54147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5a9b2700 1 -- 172.21.15.140:0/314185161 >> v1:172.21.15.140:6789/0 conn(0x7fdd540adff0 legacy=0x7fdd5405bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5a9b2700 1 -- 172.21.15.140:0/314185161 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5405c4a0 con 0x7fdd540af8e0 2022-07-21T01:58:05.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5a9b2700 1 --2- 172.21.15.140:0/314185161 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 0x7fdd540b5a30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fdd50001d00 tx=0x7fdd4c006df0).ready entity=mon.2 client_cookie=e2fcf70d8f8f0d5f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:05.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd591af700 1 -- 172.21.15.140:0/314185161 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd50001b60 con 0x7fdd540af8e0 2022-07-21T01:58:05.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd591af700 1 -- 172.21.15.140:0/314185161 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c000e90 con 0x7fdd540af8e0 2022-07-21T01:58:05.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd591af700 1 -- 172.21.15.140:0/314185161 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd4c011400 con 0x7fdd540af8e0 2022-07-21T01:58:05.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/314185161 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 msgr2=0x7fdd540b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 --2- 172.21.15.140:0/314185161 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 0x7fdd540b5a30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fdd50001d00 tx=0x7fdd4c006df0).stop 2022-07-21T01:58:05.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/314185161 shutdown_connections 2022-07-21T01:58:05.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 --2- 172.21.15.140:0/314185161 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 0x7fdd540b5a30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/314185161 >> 172.21.15.140:0/314185161 conn(0x7fdd540634e0 msgr2=0x7fdd540b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:05.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/314185161 shutdown_connections 2022-07-21T01:58:05.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/314185161 wait complete. 2022-07-21T01:58:05.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 Processor -- start 2022-07-21T01:58:05.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 -- start start 2022-07-21T01:58:05.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd540adff0 0x7fdd5414a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 0x7fdd5410e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd540b5650 0x7fdd5410f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5410e620 con 0x7fdd540b5650 2022-07-21T01:58:05.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd54064c50 con 0x7fdd540adff0 2022-07-21T01:58:05.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.479+0000 7fdd5b1b3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5410fe30 con 0x7fdd540af8e0 2022-07-21T01:58:05.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd5a9b2700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd540b5650 0x7fdd5410f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd599b0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 0x7fdd5410e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd5a1b1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd540adff0 0x7fdd5414a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd5a1b1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd540adff0 0x7fdd5414a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:47506/0 (socket says 172.21.15.140:47506) 2022-07-21T01:58:05.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd5a1b1700 1 -- 172.21.15.140:0/1724700576 learned_addr learned my addr 172.21.15.140:0/1724700576 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:05.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd599b0700 1 -- 172.21.15.140:0/1724700576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd540adff0 msgr2=0x7fdd5414a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd599b0700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd540adff0 0x7fdd5414a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd599b0700 1 -- 172.21.15.140:0/1724700576 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd540b5650 msgr2=0x7fdd5410f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd599b0700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd540b5650 0x7fdd5410f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd599b0700 1 -- 172.21.15.140:0/1724700576 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c00b180 con 0x7fdd540af8e0 2022-07-21T01:58:05.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd5a9b2700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd540b5650 0x7fdd5410f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:58:05.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd599b0700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 0x7fdd5410e130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fdd50001870 tx=0x7fdd50002e30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:05.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd4b7fe700 1 -- 172.21.15.140:0/1724700576 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd50007110 con 0x7fdd540af8e0 2022-07-21T01:58:05.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd4b7fe700 1 -- 172.21.15.140:0/1724700576 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50007270 con 0x7fdd540af8e0 2022-07-21T01:58:05.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/1724700576 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd541100c0 con 0x7fdd540af8e0 2022-07-21T01:58:05.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd4b7fe700 1 -- 172.21.15.140:0/1724700576 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd500075d0 con 0x7fdd540af8e0 2022-07-21T01:58:05.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/1724700576 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd54110d60 con 0x7fdd540af8e0 2022-07-21T01:58:05.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/1724700576 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd54064df0 con 0x7fdd540af8e0 2022-07-21T01:58:05.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd4b7fe700 1 -- 172.21.15.140:0/1724700576 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fdd50010030 con 0x7fdd540af8e0 2022-07-21T01:58:05.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.483+0000 7fdd4b7fe700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd340539e0 0x7fdd34055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.487+0000 7fdd4b7fe700 1 -- 172.21.15.140:0/1724700576 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdd5006c020 con 0x7fdd540af8e0 2022-07-21T01:58:05.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.487+0000 7fdd5a1b1700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd340539e0 0x7fdd34055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.487+0000 7fdd5a1b1700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd340539e0 0x7fdd34055ea0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fdd4400d000 tx=0x7fdd44007ae0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:05.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.495+0000 7fdd4b7fe700 1 -- 172.21.15.140:0/1724700576 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fdd54064df0 con 0x7fdd540af8e0 2022-07-21T01:58:05.623 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.619+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/1724700576 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd54064d90 con 0x7fdd540af8e0 2022-07-21T01:58:05.625 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.623+0000 7fdd4b7fe700 1 -- 172.21.15.140:0/1724700576 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16161 (secure 0 0 0) 0x7fdd50042ea0 con 0x7fdd540af8e0 2022-07-21T01:58:05.625 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:05.626 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":150,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:58:05.106735+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:58:03.485327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-07-21T01:58:04.503066+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:58:05.628 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/1724700576 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd340539e0 msgr2=0x7fdd34055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd340539e0 0x7fdd34055ea0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fdd4400d000 tx=0x7fdd44007ae0).stop 2022-07-21T01:58:05.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/1724700576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 msgr2=0x7fdd5410e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 0x7fdd5410e130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fdd50001870 tx=0x7fdd50002e30).stop 2022-07-21T01:58:05.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/1724700576 shutdown_connections 2022-07-21T01:58:05.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fdd340539e0 0x7fdd34055ea0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fdd540b5650 0x7fdd5410f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdd540af8e0 0x7fdd5410e130 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.630 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 --2- 172.21.15.140:0/1724700576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd540adff0 0x7fdd5414a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/1724700576 >> 172.21.15.140:0/1724700576 conn(0x7fdd540634e0 msgr2=0x7fdd54146070 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:05.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/1724700576 shutdown_connections 2022-07-21T01:58:05.631 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.627+0000 7fdd5b1b3700 1 -- 172.21.15.140:0/1724700576 wait complete. 2022-07-21T01:58:05.642 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-07-21T01:58:05.742 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.739+0000 7f1185216700 1 Processor -- start 2022-07-21T01:58:05.743 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.739+0000 7f1185216700 1 -- start start 2022-07-21T01:58:05.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.739+0000 7f1185216700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f11800addd0 0x7f118005c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.739+0000 7f1185216700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118005d2f0 con 0x7f11800af940 2022-07-21T01:58:05.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.739+0000 7f1185216700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11800b5dd0 con 0x7f11800b59f0 2022-07-21T01:58:05.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.739+0000 7f1185216700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118005ccd0 con 0x7f11800addd0 2022-07-21T01:58:05.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.739+0000 7f117f7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f11800af940 0x7f1180142d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55922/0 (socket says 172.21.15.140:55922) 2022-07-21T01:58:05.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.739+0000 7f117f7fe700 1 -- 172.21.15.140:0/669970409 learned_addr learned my addr 172.21.15.140:0/669970409 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:05.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.739+0000 7f1184a15700 1 --2- 172.21.15.140:0/669970409 >> v2:172.21.15.140:3300/0 conn(0x7f11800addd0 0x7f118005c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f117effd700 1 -- 172.21.15.140:0/669970409 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145851158 0 0) 0x7f118005d2f0 con 0x7f11800af940 2022-07-21T01:58:05.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f117effd700 1 -- 172.21.15.140:0/669970409 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1160003260 con 0x7f11800af940 2022-07-21T01:58:05.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f117effd700 1 -- 172.21.15.140:0/669970409 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470528699 0 0) 0x7f11800b5dd0 con 0x7f11800b59f0 2022-07-21T01:58:05.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f117effd700 1 -- 172.21.15.140:0/669970409 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f118005d2f0 con 0x7f11800b59f0 2022-07-21T01:58:05.753 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f1184a15700 1 -- 172.21.15.140:0/669970409 >> v1:172.21.15.152:6789/0 conn(0x7f11800b59f0 legacy=0x7f118005c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f1184a15700 1 -- 172.21.15.140:0/669970409 >> v1:172.21.15.140:6789/0 conn(0x7f11800af940 legacy=0x7f1180142d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.754 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f1184a15700 1 -- 172.21.15.140:0/669970409 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11801460b0 con 0x7f11800addd0 2022-07-21T01:58:05.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f1184a15700 1 --2- 172.21.15.140:0/669970409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11800addd0 0x7f118005c7e0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f1178002f70 tx=0x7f1178006df0).ready entity=mon.0 client_cookie=6c8bfe0d837a0ffd server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:05.755 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f117effd700 1 -- 172.21.15.140:0/669970409 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117800d540 con 0x7f11800addd0 2022-07-21T01:58:05.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f117effd700 1 -- 172.21.15.140:0/669970409 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1178000e90 con 0x7f11800addd0 2022-07-21T01:58:05.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f117effd700 1 -- 172.21.15.140:0/669970409 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f117800d890 con 0x7f11800addd0 2022-07-21T01:58:05.756 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f1185216700 1 -- 172.21.15.140:0/669970409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11800addd0 msgr2=0x7f118005c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f1185216700 1 --2- 172.21.15.140:0/669970409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11800addd0 0x7f118005c7e0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f1178002f70 tx=0x7f1178006df0).stop 2022-07-21T01:58:05.757 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f1185216700 1 -- 172.21.15.140:0/669970409 shutdown_connections 2022-07-21T01:58:05.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f1185216700 1 --2- 172.21.15.140:0/669970409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11800addd0 0x7f118005c7e0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.758 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f1185216700 1 -- 172.21.15.140:0/669970409 >> 172.21.15.140:0/669970409 conn(0x7f1180063480 msgr2=0x7f11800b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:05.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f1185216700 1 -- 172.21.15.140:0/669970409 shutdown_connections 2022-07-21T01:58:05.759 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.743+0000 7f1185216700 1 -- 172.21.15.140:0/669970409 wait complete. 2022-07-21T01:58:05.767 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f1185216700 1 Processor -- start 2022-07-21T01:58:05.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f1185216700 1 -- start start 2022-07-21T01:58:05.768 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f1185216700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11800addd0 0x7f1180067450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f1185216700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11800af940 0x7f118013de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.769 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f1185216700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11800b59f0 0x7f1180155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f1185216700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1180152f30 con 0x7f11800af940 2022-07-21T01:58:05.770 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f1185216700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118013f5f0 con 0x7f11800b59f0 2022-07-21T01:58:05.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f1185216700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118013f890 con 0x7f11800addd0 2022-07-21T01:58:05.771 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f117f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11800af940 0x7f118013de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f117f7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11800af940 0x7f118013de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41782/0 (socket says 172.21.15.140:41782) 2022-07-21T01:58:05.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f117f7fe700 1 -- 172.21.15.140:0/520491085 learned_addr learned my addr 172.21.15.140:0/520491085 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:05.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f117ffff700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11800addd0 0x7f1180067450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f1184a15700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11800b59f0 0x7f1180155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f117f7fe700 1 -- 172.21.15.140:0/520491085 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11800addd0 msgr2=0x7f1180067450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.774 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f117f7fe700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11800addd0 0x7f1180067450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f117f7fe700 1 -- 172.21.15.140:0/520491085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11800b59f0 msgr2=0x7f1180155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:05.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f117f7fe700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11800b59f0 0x7f1180155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:05.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f117f7fe700 1 -- 172.21.15.140:0/520491085 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117800b180 con 0x7f11800af940 2022-07-21T01:58:05.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.747+0000 7f117f7fe700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11800af940 0x7f118013de90 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f1168001550 tx=0x7f1168007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:05.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.751+0000 7f117d7fa700 1 -- 172.21.15.140:0/520491085 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1168005ba0 con 0x7f11800af940 2022-07-21T01:58:05.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.751+0000 7f117d7fa700 1 -- 172.21.15.140:0/520491085 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1168005d00 con 0x7f11800af940 2022-07-21T01:58:05.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.751+0000 7f1185216700 1 -- 172.21.15.140:0/520491085 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118013fb30 con 0x7f11800af940 2022-07-21T01:58:05.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.751+0000 7f117d7fa700 1 -- 172.21.15.140:0/520491085 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11680072c0 con 0x7f11800af940 2022-07-21T01:58:05.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.751+0000 7f1185216700 1 -- 172.21.15.140:0/520491085 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1180067940 con 0x7f11800af940 2022-07-21T01:58:05.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.751+0000 7f1185216700 1 -- 172.21.15.140:0/520491085 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1180152150 con 0x7f11800af940 2022-07-21T01:58:05.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.751+0000 7f117d7fa700 1 -- 172.21.15.140:0/520491085 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f116801a060 con 0x7f11800af940 2022-07-21T01:58:05.780 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.751+0000 7f117d7fa700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1164053a10 0x7f1164055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:05.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.751+0000 7f117d7fa700 1 -- 172.21.15.140:0/520491085 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f116806b9c0 con 0x7f11800af940 2022-07-21T01:58:05.781 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.751+0000 7f117ffff700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1164053a10 0x7f1164055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:05.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.751+0000 7f117ffff700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1164053a10 0x7f1164055ed0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f117000d000 tx=0x7f1170007a20).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:05.782 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.759+0000 7f117d7fa700 1 -- 172.21.15.140:0/520491085 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1168042860 con 0x7f11800af940 2022-07-21T01:58:05.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:05.923+0000 7f1185216700 1 -- 172.21.15.140:0/520491085 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f118005cfe0 con 0x7f11800af940 2022-07-21T01:58:06.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.111+0000 7f117d7fa700 1 -- 172.21.15.140:0/520491085 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1168018070 con 0x7f11800af940 2022-07-21T01:58:06.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 -- 172.21.15.140:0/520491085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1164053a10 msgr2=0x7f1164055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1164053a10 0x7f1164055ed0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f117000d000 tx=0x7f1170007a20).stop 2022-07-21T01:58:06.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 -- 172.21.15.140:0/520491085 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11800af940 msgr2=0x7f118013de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11800af940 0x7f118013de90 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7f1168001550 tx=0x7f1168007ed0).stop 2022-07-21T01:58:06.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 -- 172.21.15.140:0/520491085 shutdown_connections 2022-07-21T01:58:06.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1164053a10 0x7f1164055ed0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f11800b59f0 0x7f1180155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f11800af940 0x7f118013de90 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 --2- 172.21.15.140:0/520491085 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f11800addd0 0x7f1180067450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 -- 172.21.15.140:0/520491085 >> 172.21.15.140:0/520491085 conn(0x7f1180063480 msgr2=0x7f11801464b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:06.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 -- 172.21.15.140:0/520491085 shutdown_connections 2022-07-21T01:58:06.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.115+0000 7f1185216700 1 -- 172.21.15.140:0/520491085 wait complete. 2022-07-21T01:58:06.135 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:58:06.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03b59e700 1 Processor -- start 2022-07-21T01:58:06.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03b59e700 1 -- start start 2022-07-21T01:58:06.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b1ef0 0x7fc03c061a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03b59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc03c098aa0 0x7fc03c05fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03b59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc03c0ab9e0 con 0x7fc03c0abd80 2022-07-21T01:58:06.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03b59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c147000 con 0x7fc03c098aa0 2022-07-21T01:58:06.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0b22d0 con 0x7fc03c0b1ef0 2022-07-21T01:58:06.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc039d9b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc03c098aa0 0x7fc03c05fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03ad9d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc03c0abd80 0x7fc03c05c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55938/0 (socket says 172.21.15.140:55938) 2022-07-21T01:58:06.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03ad9d700 1 -- 172.21.15.140:0/2070943626 learned_addr learned my addr 172.21.15.140:0/2070943626 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:06.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc039d9b700 1 --2- 172.21.15.140:0/2070943626 >> v2:172.21.15.140:3300/0 conn(0x7fc03c098aa0 0x7fc03c05fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41790/0 (socket says 172.21.15.140:41790) 2022-07-21T01:58:06.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03a59c700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b1ef0 0x7fc03c061a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749906432 0 0) 0x7fc03c0ab9e0 con 0x7fc03c0abd80 2022-07-21T01:58:06.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc020003260 con 0x7fc03c0abd80 2022-07-21T01:58:06.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 934176993 0 0) 0x7fc020003260 con 0x7fc03c0abd80 2022-07-21T01:58:06.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b1ef0 msgr2=0x7fc03c061a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:06.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b1ef0 0x7fc03c061a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 >> v2:172.21.15.140:3300/0 conn(0x7fc03c098aa0 msgr2=0x7fc03c05fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:06.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 --2- 172.21.15.140:0/2070943626 >> v2:172.21.15.140:3300/0 conn(0x7fc03c098aa0 0x7fc03c05fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c060200 con 0x7fc03c0abd80 2022-07-21T01:58:06.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7fc0340019e0 con 0x7fc03c0abd80 2022-07-21T01:58:06.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc039d9b700 1 --2- 172.21.15.140:0/2070943626 >> v2:172.21.15.140:3300/0 conn(0x7fc03c098aa0 0x7fc03c05fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:58:06.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 >> v1:172.21.15.140:6789/0 conn(0x7fc03c0abd80 legacy=0x7fc03c05c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b1ef0 0x7fc020003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 0x7fc020004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0200049f0 0x7fc020004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c147000 con 0x7fc020003c80 2022-07-21T01:58:06.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0b22d0 con 0x7fc0200049f0 2022-07-21T01:58:06.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02000b750 con 0x7fc03c0b1ef0 2022-07-21T01:58:06.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc039d9b700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 0x7fc020004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03a59c700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b1ef0 0x7fc020003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03ad9d700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0200049f0 0x7fc020004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc039d9b700 1 -- 172.21.15.140:0/2070943626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b1ef0 msgr2=0x7fc020003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:06.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc039d9b700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b1ef0 0x7fc020003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc039d9b700 1 -- 172.21.15.140:0/2070943626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0200049f0 msgr2=0x7fc020004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:06.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc039d9b700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0200049f0 0x7fc020004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc039d9b700 1 -- 172.21.15.140:0/2070943626 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020003260 con 0x7fc020003c80 2022-07-21T01:58:06.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc039d9b700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 0x7fc020004500 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7fc0200054d0 tx=0x7fc02c006df0).ready entity=mon.0 client_cookie=e7ca9753ac033888 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:06.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.239+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc02c00e830 con 0x7fc020003c80 2022-07-21T01:58:06.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.243+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c00e990 con 0x7fc020003c80 2022-07-21T01:58:06.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.243+0000 7fc03959a700 1 -- 172.21.15.140:0/2070943626 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc02c00ecf0 con 0x7fc020003c80 2022-07-21T01:58:06.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.243+0000 7fc03b59e700 1 -- 172.21.15.140:0/2070943626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 msgr2=0x7fc020004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.243+0000 7fc03b59e700 1 --2- 172.21.15.140:0/2070943626 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 0x7fc020004500 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7fc0200054d0 tx=0x7fc02c006df0).stop 2022-07-21T01:58:06.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.243+0000 7fc03ad9d700 1 -- 172.21.15.140:0/2070943626 reap_dead start 2022-07-21T01:58:06.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.243+0000 7fc03b59e700 1 -- 172.21.15.140:0/2070943626 shutdown_connections 2022-07-21T01:58:06.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.243+0000 7fc03b59e700 1 -- 172.21.15.140:0/2070943626 >> 172.21.15.140:0/2070943626 conn(0x7fc03c0b2ca0 msgr2=0x7fc03c0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:06.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.243+0000 7fc03b59e700 1 -- 172.21.15.140:0/2070943626 shutdown_connections 2022-07-21T01:58:06.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.243+0000 7fc03b59e700 1 -- 172.21.15.140:0/2070943626 wait complete. 2022-07-21T01:58:06.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.243+0000 7fc03b59e700 1 Processor -- start 2022-07-21T01:58:06.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.243+0000 7fc03b59e700 1 -- start start 2022-07-21T01:58:06.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 0x7fc03c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc03c09a5b0 0x7fc03c144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b0310 0x7fc03c1451f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0af2b0 con 0x7fc020003c80 2022-07-21T01:58:06.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0ae5f0 con 0x7fc03c09a5b0 2022-07-21T01:58:06.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c1456e0 con 0x7fc03c0b0310 2022-07-21T01:58:06.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 0x7fc03c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03ad9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b0310 0x7fc03c1451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03a59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 0x7fc03c0b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41800/0 (socket says 172.21.15.140:41800) 2022-07-21T01:58:06.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03a59c700 1 -- 172.21.15.140:0/1035609249 learned_addr learned my addr 172.21.15.140:0/1035609249 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:06.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc039d9b700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc03c09a5b0 0x7fc03c144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03a59c700 1 -- 172.21.15.140:0/1035609249 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b0310 msgr2=0x7fc03c1451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03a59c700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b0310 0x7fc03c1451f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03a59c700 1 -- 172.21.15.140:0/1035609249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc03c09a5b0 msgr2=0x7fc03c144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03a59c700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc03c09a5b0 0x7fc03c144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03a59c700 1 -- 172.21.15.140:0/1035609249 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c0ae070 con 0x7fc020003c80 2022-07-21T01:58:06.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03a59c700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 0x7fc03c0b4de0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7fc030006000 tx=0x7fc030008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:06.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc02b7fe700 1 -- 172.21.15.140:0/1035609249 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0300135e0 con 0x7fc020003c80 2022-07-21T01:58:06.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc02b7fe700 1 -- 172.21.15.140:0/1035609249 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc030013740 con 0x7fc020003c80 2022-07-21T01:58:06.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03b59e700 1 -- 172.21.15.140:0/1035609249 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c0ad640 con 0x7fc020003c80 2022-07-21T01:58:06.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc02b7fe700 1 -- 172.21.15.140:0/1035609249 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc030013aa0 con 0x7fc020003c80 2022-07-21T01:58:06.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.247+0000 7fc03b59e700 1 -- 172.21.15.140:0/1035609249 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c09b8f0 con 0x7fc020003c80 2022-07-21T01:58:06.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.251+0000 7fc03b59e700 1 -- 172.21.15.140:0/1035609249 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c0af580 con 0x7fc020003c80 2022-07-21T01:58:06.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.251+0000 7fc02b7fe700 1 -- 172.21.15.140:0/1035609249 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc030013c00 con 0x7fc020003c80 2022-07-21T01:58:06.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.251+0000 7fc02b7fe700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc018053a10 0x7fc018055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.251+0000 7fc02b7fe700 1 -- 172.21.15.140:0/1035609249 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc03006c5f0 con 0x7fc020003c80 2022-07-21T01:58:06.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.251+0000 7fc039d9b700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc018053a10 0x7fc018055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.251+0000 7fc039d9b700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc018053a10 0x7fc018055ed0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7fc02c01e000 tx=0x7fc02c019000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:06.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.255+0000 7fc02b7fe700 1 -- 172.21.15.140:0/1035609249 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc030043490 con 0x7fc020003c80 2022-07-21T01:58:06.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.419+0000 7fc03b59e700 1 -- 172.21.15.140:0/1035609249 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc03c0ab4a0 con 0x7fc020003c80 2022-07-21T01:58:06.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.419+0000 7fc02b7fe700 1 -- 172.21.15.140:0/1035609249 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2148 (secure 0 0 0) 0x7fc030046ab0 con 0x7fc020003c80 2022-07-21T01:58:06.423 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:06.423 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":74,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/3591375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591375250},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591375250}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:58:05.107197+0000","modified":"2022-07-21T01:58:06.110113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8727":{"gid":8727,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/4013573612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":4013573612},{"type":"v1","addr":"172.21.15.140:6835","nonce":4013573612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-07-21T01:58:06.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.419+0000 7fc03b59e700 1 -- 172.21.15.140:0/1035609249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc018053a10 msgr2=0x7fc018055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.419+0000 7fc03b59e700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc018053a10 0x7fc018055ed0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7fc02c01e000 tx=0x7fc02c019000).stop 2022-07-21T01:58:06.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.419+0000 7fc03b59e700 1 -- 172.21.15.140:0/1035609249 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 msgr2=0x7fc03c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.419+0000 7fc03b59e700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 0x7fc03c0b4de0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7fc030006000 tx=0x7fc030008040).stop 2022-07-21T01:58:06.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.419+0000 7fc03b59e700 1 -- 172.21.15.140:0/1035609249 shutdown_connections 2022-07-21T01:58:06.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.419+0000 7fc03b59e700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc018053a10 0x7fc018055ed0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.423+0000 7fc03b59e700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc03c0b0310 0x7fc03c1451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.423+0000 7fc03b59e700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc03c09a5b0 0x7fc03c144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.423+0000 7fc03b59e700 1 --2- 172.21.15.140:0/1035609249 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc020003c80 0x7fc03c0b4de0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.423+0000 7fc03b59e700 1 -- 172.21.15.140:0/1035609249 >> 172.21.15.140:0/1035609249 conn(0x7fc03c0b2ca0 msgr2=0x7fc03c14ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:06.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.423+0000 7fc03b59e700 1 -- 172.21.15.140:0/1035609249 shutdown_connections 2022-07-21T01:58:06.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.423+0000 7fc03b59e700 1 -- 172.21.15.140:0/1035609249 wait complete. 2022-07-21T01:58:06.428 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 74 2022-07-21T01:58:06.435 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:58:06.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.535+0000 7f24d4a58700 1 Processor -- start 2022-07-21T01:58:06.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.535+0000 7f24d4a58700 1 -- start start 2022-07-21T01:58:06.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.535+0000 7f24d4a58700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f24d00ac2d0 0x7f24d00ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.535+0000 7f24d4a58700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24d0062490 0x7f24d00b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.535+0000 7f24d4a58700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d0147110 con 0x7f24d0146c40 2022-07-21T01:58:06.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.535+0000 7f24d4a58700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d00b3b40 con 0x7f24d00ac2d0 2022-07-21T01:58:06.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.535+0000 7f24d4a58700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00b2f10 con 0x7f24d0062490 2022-07-21T01:58:06.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.535+0000 7f24ce59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f24d00ac2d0 0x7f24d00ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24ce59c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f24d00ac2d0 0x7f24d00ac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41808/0 (socket says 172.21.15.140:41808) 2022-07-21T01:58:06.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24ce59c700 1 -- 172.21.15.140:0/2836788565 learned_addr learned my addr 172.21.15.140:0/2836788565 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:06.549 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24ced9d700 1 --2- 172.21.15.140:0/2836788565 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24d0062490 0x7f24d00b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24ce59c700 1 -- 172.21.15.140:0/2836788565 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24d0062490 msgr2=0x7f24d00b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:06.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24ce59c700 1 --2- 172.21.15.140:0/2836788565 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24d0062490 0x7f24d00b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24ce59c700 1 -- 172.21.15.140:0/2836788565 >> v1:172.21.15.152:6789/0 conn(0x7f24d0146c40 legacy=0x7f24d0061d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.550 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24ce59c700 1 -- 172.21.15.140:0/2836788565 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00b31b0 con 0x7f24d00ac2d0 2022-07-21T01:58:06.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24ce59c700 1 --2- 172.21.15.140:0/2836788565 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24d00ac2d0 0x7f24d00ac6b0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f24c0002f70 tx=0x7f24c0006eb0).ready entity=mon.0 client_cookie=bf3910d87a20796b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:06.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24cdd9b700 1 -- 172.21.15.140:0/2836788565 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24c000d570 con 0x7f24d00ac2d0 2022-07-21T01:58:06.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24cdd9b700 1 -- 172.21.15.140:0/2836788565 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c0000e90 con 0x7f24d00ac2d0 2022-07-21T01:58:06.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24cdd9b700 1 -- 172.21.15.140:0/2836788565 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24c000d900 con 0x7f24d00ac2d0 2022-07-21T01:58:06.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24d4a58700 1 -- 172.21.15.140:0/2836788565 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24d00ac2d0 msgr2=0x7f24d00ac6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24d4a58700 1 --2- 172.21.15.140:0/2836788565 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24d00ac2d0 0x7f24d00ac6b0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f24c0002f70 tx=0x7f24c0006eb0).stop 2022-07-21T01:58:06.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24d4a58700 1 -- 172.21.15.140:0/2836788565 shutdown_connections 2022-07-21T01:58:06.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24d4a58700 1 --2- 172.21.15.140:0/2836788565 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24d0062490 0x7f24d00b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24d4a58700 1 --2- 172.21.15.140:0/2836788565 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24d00ac2d0 0x7f24d00ac6b0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24d4a58700 1 -- 172.21.15.140:0/2836788565 >> 172.21.15.140:0/2836788565 conn(0x7f24d00b47e0 msgr2=0x7f24d005bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:06.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24d4a58700 1 -- 172.21.15.140:0/2836788565 shutdown_connections 2022-07-21T01:58:06.553 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.539+0000 7f24d4a58700 1 -- 172.21.15.140:0/2836788565 wait complete. 2022-07-21T01:58:06.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24d4a58700 1 Processor -- start 2022-07-21T01:58:06.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24d4a58700 1 -- start start 2022-07-21T01:58:06.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24d4a58700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24d0062490 0x7f24d00b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24d4a58700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24d00ac2d0 0x7f24d0149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24d4a58700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24d0146c40 0x7f24d01553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24d4a58700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0099000 con 0x7f24d0062490 2022-07-21T01:58:06.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24d4a58700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00b1130 con 0x7f24d0146c40 2022-07-21T01:58:06.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24d4a58700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00b13d0 con 0x7f24d00ac2d0 2022-07-21T01:58:06.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24ce59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24d0062490 0x7f24d00b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24ce59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24d0062490 0x7f24d00b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41814/0 (socket says 172.21.15.140:41814) 2022-07-21T01:58:06.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24ce59c700 1 -- 172.21.15.140:0/1360623461 learned_addr learned my addr 172.21.15.140:0/1360623461 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:06.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24ced9d700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24d0146c40 0x7f24d01553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24c5d9b700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24d00ac2d0 0x7f24d0149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24ce59c700 1 -- 172.21.15.140:0/1360623461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24d00ac2d0 msgr2=0x7f24d0149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24ce59c700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24d00ac2d0 0x7f24d0149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24ce59c700 1 -- 172.21.15.140:0/1360623461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24d0146c40 msgr2=0x7f24d01553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24ce59c700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24d0146c40 0x7f24d01553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24ce59c700 1 -- 172.21.15.140:0/1360623461 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c000b1e0 con 0x7f24d0062490 2022-07-21T01:58:06.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24c5d9b700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24d00ac2d0 0x7f24d0149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:58:06.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24ce59c700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24d0062490 0x7f24d00b0290 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f24c0002f70 tx=0x7f24c0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:06.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24c7fff700 1 -- 172.21.15.140:0/1360623461 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24c00117e0 con 0x7f24d0062490 2022-07-21T01:58:06.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24d4a58700 1 -- 172.21.15.140:0/1360623461 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d00978a0 con 0x7f24d0062490 2022-07-21T01:58:06.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24c7fff700 1 -- 172.21.15.140:0/1360623461 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c0011940 con 0x7f24d0062490 2022-07-21T01:58:06.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24c7fff700 1 -- 172.21.15.140:0/1360623461 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f24c0011ca0 con 0x7f24d0062490 2022-07-21T01:58:06.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.543+0000 7f24d4a58700 1 -- 172.21.15.140:0/1360623461 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d01468d0 con 0x7f24d0062490 2022-07-21T01:58:06.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.547+0000 7f24d4a58700 1 -- 172.21.15.140:0/1360623461 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d00574e0 con 0x7f24d0062490 2022-07-21T01:58:06.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.547+0000 7f24c7fff700 1 -- 172.21.15.140:0/1360623461 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f24c0025030 con 0x7f24d0062490 2022-07-21T01:58:06.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.547+0000 7f24c7fff700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24b4053950 0x7f24b4055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.547+0000 7f24c7fff700 1 -- 172.21.15.140:0/1360623461 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f24c006d2d0 con 0x7f24d0062490 2022-07-21T01:58:06.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.547+0000 7f24c5d9b700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24b4053950 0x7f24b4055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.547+0000 7f24c5d9b700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24b4053950 0x7f24b4055e10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f24b800b000 tx=0x7f24b8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:06.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.551+0000 7f24c7fff700 1 -- 172.21.15.140:0/1360623461 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f24c0044060 con 0x7f24d0062490 2022-07-21T01:58:06.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.679+0000 7f24d4a58700 1 -- 172.21.15.140:0/1360623461 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24d0069a60 con 0x7f24d0062490 2022-07-21T01:58:06.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.679+0000 7f24c7fff700 1 -- 172.21.15.140:0/1360623461 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f24c0047680 con 0x7f24d0062490 2022-07-21T01:58:06.683 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:06.683 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":151,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:58:06.110718+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:58:03.485327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-07-21T01:58:04.503066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:58:06.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 -- 172.21.15.140:0/1360623461 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24b4053950 msgr2=0x7f24b4055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24b4053950 0x7f24b4055e10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f24b800b000 tx=0x7f24b8009040).stop 2022-07-21T01:58:06.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 -- 172.21.15.140:0/1360623461 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24d0062490 msgr2=0x7f24d00b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24d0062490 0x7f24d00b0290 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f24c0002f70 tx=0x7f24c0015000).stop 2022-07-21T01:58:06.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 -- 172.21.15.140:0/1360623461 shutdown_connections 2022-07-21T01:58:06.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f24b4053950 0x7f24b4055e10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24d0146c40 0x7f24d01553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f24d00ac2d0 0x7f24d0149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 --2- 172.21.15.140:0/1360623461 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f24d0062490 0x7f24d00b0290 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 -- 172.21.15.140:0/1360623461 >> 172.21.15.140:0/1360623461 conn(0x7f24d00b47e0 msgr2=0x7f24d0143310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:06.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 -- 172.21.15.140:0/1360623461 shutdown_connections 2022-07-21T01:58:06.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.683+0000 7f24d4a58700 1 -- 172.21.15.140:0/1360623461 wait complete. 2022-07-21T01:58:06.696 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:58:06.808 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f168730c700 1 Processor -- start 2022-07-21T01:58:06.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f168730c700 1 -- start start 2022-07-21T01:58:06.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f168730c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f168005cf40 0x7f1680146a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f168730c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16800b0130 con 0x7f16800b5830 2022-07-21T01:58:06.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f168730c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16801462d0 con 0x7f16800af2c0 2022-07-21T01:58:06.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f168730c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1680146570 con 0x7f168005cf40 2022-07-21T01:58:06.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f16848a7700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f16800b5830 0x7f1680142c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55970/0 (socket says 172.21.15.140:55970) 2022-07-21T01:58:06.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f16848a7700 1 -- 172.21.15.140:0/1436122721 learned_addr learned my addr 172.21.15.140:0/1436122721 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:06.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f16850a8700 1 --2- 172.21.15.140:0/1436122721 >> v2:172.21.15.152:3300/0 conn(0x7f168005cf40 0x7f1680146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f1673fff700 1 -- 172.21.15.140:0/1436122721 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659130493 0 0) 0x7f16800b0130 con 0x7f16800b5830 2022-07-21T01:58:06.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f1673fff700 1 -- 172.21.15.140:0/1436122721 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1664003260 con 0x7f16800b5830 2022-07-21T01:58:06.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f16850a8700 1 -- 172.21.15.140:0/1436122721 >> v1:172.21.15.152:6789/0 conn(0x7f16800af2c0 legacy=0x7f16800af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f16850a8700 1 -- 172.21.15.140:0/1436122721 >> v1:172.21.15.140:6789/0 conn(0x7f16800b5830 legacy=0x7f1680142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f16850a8700 1 -- 172.21.15.140:0/1436122721 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1680143340 con 0x7f168005cf40 2022-07-21T01:58:06.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f16850a8700 1 --2- 172.21.15.140:0/1436122721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f168005cf40 0x7f1680146a10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f1674002f70 tx=0x7f1674006df0).ready entity=mon.1 client_cookie=709fba7aca9a9453 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:06.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f1673fff700 1 -- 172.21.15.140:0/1436122721 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f167400d610 con 0x7f168005cf40 2022-07-21T01:58:06.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f1673fff700 1 -- 172.21.15.140:0/1436122721 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1674000e90 con 0x7f168005cf40 2022-07-21T01:58:06.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.807+0000 7f1673fff700 1 -- 172.21.15.140:0/1436122721 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f167400d960 con 0x7f168005cf40 2022-07-21T01:58:06.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.811+0000 7f168730c700 1 -- 172.21.15.140:0/1436122721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f168005cf40 msgr2=0x7f1680146a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.811+0000 7f168730c700 1 --2- 172.21.15.140:0/1436122721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f168005cf40 0x7f1680146a10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f1674002f70 tx=0x7f1674006df0).stop 2022-07-21T01:58:06.820 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.811+0000 7f168730c700 1 -- 172.21.15.140:0/1436122721 shutdown_connections 2022-07-21T01:58:06.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.811+0000 7f168730c700 1 --2- 172.21.15.140:0/1436122721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f168005cf40 0x7f1680146a10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.821 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.811+0000 7f168730c700 1 -- 172.21.15.140:0/1436122721 >> 172.21.15.140:0/1436122721 conn(0x7f16800b69e0 msgr2=0x7f1680066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:06.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.811+0000 7f168730c700 1 -- 172.21.15.140:0/1436122721 shutdown_connections 2022-07-21T01:58:06.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.811+0000 7f168730c700 1 -- 172.21.15.140:0/1436122721 wait complete. 2022-07-21T01:58:06.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.811+0000 7f168730c700 1 Processor -- start 2022-07-21T01:58:06.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.811+0000 7f168730c700 1 -- start start 2022-07-21T01:58:06.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f168730c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f168005cf40 0x7f1680155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f168730c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16800af2c0 0x7f168009d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f168730c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16800b5830 0x7f168009dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f168730c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16800b0dd0 con 0x7f16800af2c0 2022-07-21T01:58:06.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f168730c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16800b1070 con 0x7f168005cf40 2022-07-21T01:58:06.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16848a7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16800af2c0 0x7f168009d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f168730c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16800b1310 con 0x7f16800b5830 2022-07-21T01:58:06.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16848a7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16800af2c0 0x7f168009d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41830/0 (socket says 172.21.15.140:41830) 2022-07-21T01:58:06.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16848a7700 1 -- 172.21.15.140:0/2101813080 learned_addr learned my addr 172.21.15.140:0/2101813080 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:06.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16858a9700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16800b5830 0x7f168009dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16850a8700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f168005cf40 0x7f1680155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16848a7700 1 -- 172.21.15.140:0/2101813080 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16800b5830 msgr2=0x7f168009dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16848a7700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16800b5830 0x7f168009dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16848a7700 1 -- 172.21.15.140:0/2101813080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f168005cf40 msgr2=0x7f1680155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:06.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16848a7700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f168005cf40 0x7f1680155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:06.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16848a7700 1 -- 172.21.15.140:0/2101813080 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167400b180 con 0x7f16800af2c0 2022-07-21T01:58:06.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16850a8700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f168005cf40 0x7f1680155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:58:06.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16848a7700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16800af2c0 0x7f168009d720 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f167c001110 tx=0x7f167c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:06.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16727fc700 1 -- 172.21.15.140:0/2101813080 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f167c00ce20 con 0x7f16800af2c0 2022-07-21T01:58:06.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16727fc700 1 -- 172.21.15.140:0/2101813080 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167c0072e0 con 0x7f16800af2c0 2022-07-21T01:58:06.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f16727fc700 1 -- 172.21.15.140:0/2101813080 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f167c005800 con 0x7f16800af2c0 2022-07-21T01:58:06.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f168730c700 1 -- 172.21.15.140:0/2101813080 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1680156460 con 0x7f16800af2c0 2022-07-21T01:58:06.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.815+0000 7f168730c700 1 -- 172.21.15.140:0/2101813080 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16800b2b40 con 0x7f16800af2c0 2022-07-21T01:58:06.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.819+0000 7f168730c700 1 -- 172.21.15.140:0/2101813080 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16800afe00 con 0x7f16800af2c0 2022-07-21T01:58:06.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.819+0000 7f16727fc700 1 -- 172.21.15.140:0/2101813080 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f167c010070 con 0x7f16800af2c0 2022-07-21T01:58:06.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.819+0000 7f16727fc700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1668053a70 0x7f1668055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:06.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.819+0000 7f16727fc700 1 -- 172.21.15.140:0/2101813080 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f167c06b620 con 0x7f16800af2c0 2022-07-21T01:58:06.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.819+0000 7f16850a8700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1668053a70 0x7f1668055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:06.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.819+0000 7f16850a8700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1668053a70 0x7f1668055f30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f1674018000 tx=0x7f1674014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:06.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.827+0000 7f16727fc700 1 -- 172.21.15.140:0/2101813080 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f167c042540 con 0x7f16800af2c0 2022-07-21T01:58:06.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.991+0000 7f168730c700 1 -- 172.21.15.140:0/2101813080 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f16800574e0 con 0x7f16800af2c0 2022-07-21T01:58:07.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:06.999+0000 7f16727fc700 1 -- 172.21.15.140:0/2101813080 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v49) v1 ==== 184+0+0 (secure 0 0 0) 0x7f167c045b60 con 0x7f16800af2c0 2022-07-21T01:58:07.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 -- 172.21.15.140:0/2101813080 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1668053a70 msgr2=0x7f1668055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1668053a70 0x7f1668055f30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f1674018000 tx=0x7f1674014000).stop 2022-07-21T01:58:07.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 -- 172.21.15.140:0/2101813080 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16800af2c0 msgr2=0x7f168009d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16800af2c0 0x7f168009d720 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f167c001110 tx=0x7f167c007ad0).stop 2022-07-21T01:58:07.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 -- 172.21.15.140:0/2101813080 shutdown_connections 2022-07-21T01:58:07.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1668053a70 0x7f1668055f30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16800b5830 0x7f168009dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16800af2c0 0x7f168009d720 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 --2- 172.21.15.140:0/2101813080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f168005cf40 0x7f1680155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 -- 172.21.15.140:0/2101813080 >> 172.21.15.140:0/2101813080 conn(0x7f16800b69e0 msgr2=0x7f1680144130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:07.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 -- 172.21.15.140:0/2101813080 shutdown_connections 2022-07-21T01:58:07.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.003+0000 7f168730c700 1 -- 172.21.15.140:0/2101813080 wait complete. 2022-07-21T01:58:07.009 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.0 2022-07-21T01:58:07.018 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:58:07.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.131+0000 7f9c79e44700 1 Processor -- start 2022-07-21T01:58:07.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.131+0000 7f9c79e44700 1 -- start start 2022-07-21T01:58:07.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.131+0000 7f9c79e44700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c740b3cb0 0x7f9c740aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.131+0000 7f9c79e44700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9c740ad0e0 0x7f9c741418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.131+0000 7f9c79e44700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c740ae280 con 0x7f9c740b6800 2022-07-21T01:58:07.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.131+0000 7f9c79e44700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c740b6be0 con 0x7f9c740ad0e0 2022-07-21T01:58:07.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.131+0000 7f9c79e44700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740ae4b0 con 0x7f9c740b3cb0 2022-07-21T01:58:07.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79643700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9c740ad0e0 0x7f9c741418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79643700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f9c740ad0e0 0x7f9c741418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41838/0 (socket says 172.21.15.140:41838) 2022-07-21T01:58:07.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79643700 1 -- 172.21.15.140:0/1443289419 learned_addr learned my addr 172.21.15.140:0/1443289419 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:07.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c78e42700 1 --1- 172.21.15.140:0/1443289419 >> v1:172.21.15.140:6789/0 conn(0x7f9c740b6800 0x7f9c74145f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:55982/0 (socket says 172.21.15.140:55982) 2022-07-21T01:58:07.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c73fff700 1 --2- 172.21.15.140:0/1443289419 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c740b3cb0 0x7f9c740aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79643700 1 -- 172.21.15.140:0/1443289419 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c740b3cb0 msgr2=0x7f9c740aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:07.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79643700 1 --2- 172.21.15.140:0/1443289419 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c740b3cb0 0x7f9c740aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79643700 1 -- 172.21.15.140:0/1443289419 >> v1:172.21.15.140:6789/0 conn(0x7f9c740b6800 legacy=0x7f9c74145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c737fe700 1 -- 172.21.15.140:0/1443289419 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59613342 0 0) 0x7f9c740ae280 con 0x7f9c740b6800 2022-07-21T01:58:07.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79643700 1 -- 172.21.15.140:0/1443289419 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c740ae750 con 0x7f9c740ad0e0 2022-07-21T01:58:07.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79643700 1 --2- 172.21.15.140:0/1443289419 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c740ad0e0 0x7f9c741418f0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f9c6c0032f0 tx=0x7f9c6c009290).ready entity=mon.0 client_cookie=8794c75ad3074646 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:07.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c737fe700 1 -- 172.21.15.140:0/1443289419 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c6c00b8a0 con 0x7f9c740ad0e0 2022-07-21T01:58:07.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c737fe700 1 -- 172.21.15.140:0/1443289419 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c00ba00 con 0x7f9c740ad0e0 2022-07-21T01:58:07.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c737fe700 1 -- 172.21.15.140:0/1443289419 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c6c00bb70 con 0x7f9c740ad0e0 2022-07-21T01:58:07.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79e44700 1 -- 172.21.15.140:0/1443289419 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c740ad0e0 msgr2=0x7f9c741418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79e44700 1 --2- 172.21.15.140:0/1443289419 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c740ad0e0 0x7f9c741418f0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f9c6c0032f0 tx=0x7f9c6c009290).stop 2022-07-21T01:58:07.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79e44700 1 -- 172.21.15.140:0/1443289419 shutdown_connections 2022-07-21T01:58:07.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79e44700 1 --2- 172.21.15.140:0/1443289419 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c740ad0e0 0x7f9c741418f0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79e44700 1 --2- 172.21.15.140:0/1443289419 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c740b3cb0 0x7f9c740aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79e44700 1 -- 172.21.15.140:0/1443289419 >> 172.21.15.140:0/1443289419 conn(0x7f9c740b9dd0 msgr2=0x7f9c740b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:07.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79e44700 1 -- 172.21.15.140:0/1443289419 shutdown_connections 2022-07-21T01:58:07.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79e44700 1 -- 172.21.15.140:0/1443289419 wait complete. 2022-07-21T01:58:07.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79e44700 1 Processor -- start 2022-07-21T01:58:07.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.135+0000 7f9c79e44700 1 -- start start 2022-07-21T01:58:07.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79e44700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c740ad0e0 0x7f9c740af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79e44700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c740b3cb0 0x7f9c74151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79e44700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c740b6800 0x7f9c74144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79e44700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7413d970 con 0x7f9c740b6800 2022-07-21T01:58:07.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79e44700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c74145cc0 con 0x7f9c740b3cb0 2022-07-21T01:58:07.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79e44700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c74152120 con 0x7f9c740ad0e0 2022-07-21T01:58:07.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79643700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c740b6800 0x7f9c74144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79643700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c740b6800 0x7f9c74144ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41846/0 (socket says 172.21.15.140:41846) 2022-07-21T01:58:07.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79643700 1 -- 172.21.15.140:0/1473497418 learned_addr learned my addr 172.21.15.140:0/1473497418 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:07.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c78e42700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c740ad0e0 0x7f9c740af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c73fff700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c740b3cb0 0x7f9c74151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79643700 1 -- 172.21.15.140:0/1473497418 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c740ad0e0 msgr2=0x7f9c740af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79643700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c740ad0e0 0x7f9c740af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79643700 1 -- 172.21.15.140:0/1473497418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c740b3cb0 msgr2=0x7f9c74151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79643700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c740b3cb0 0x7f9c74151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79643700 1 -- 172.21.15.140:0/1473497418 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c009530 con 0x7f9c740b6800 2022-07-21T01:58:07.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79643700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c740b6800 0x7f9c74144ef0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f9c74145e50 tx=0x7f9c6c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:07.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c71ffb700 1 -- 172.21.15.140:0/1473497418 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c6c00b8a0 con 0x7f9c740b6800 2022-07-21T01:58:07.158 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c71ffb700 1 -- 172.21.15.140:0/1473497418 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c021630 con 0x7f9c740b6800 2022-07-21T01:58:07.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79e44700 1 -- 172.21.15.140:0/1473497418 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c740b3330 con 0x7f9c740b6800 2022-07-21T01:58:07.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c71ffb700 1 -- 172.21.15.140:0/1473497418 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9c6c021990 con 0x7f9c740b6800 2022-07-21T01:58:07.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c79e44700 1 -- 172.21.15.140:0/1473497418 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7413db10 con 0x7f9c740b6800 2022-07-21T01:58:07.159 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.139+0000 7f9c71ffb700 1 -- 172.21.15.140:0/1473497418 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9c6c029070 con 0x7f9c740b6800 2022-07-21T01:58:07.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.143+0000 7f9c71ffb700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c58053b30 0x7f9c58055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.143+0000 7f9c71ffb700 1 -- 172.21.15.140:0/1473497418 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9c6c06cd20 con 0x7f9c740b6800 2022-07-21T01:58:07.160 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.143+0000 7f9c78e42700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c58053b30 0x7f9c58055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.143+0000 7f9c79e44700 1 -- 172.21.15.140:0/1473497418 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c740b1b50 con 0x7f9c740b6800 2022-07-21T01:58:07.161 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.143+0000 7f9c78e42700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c58053b30 0x7f9c58055ff0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f9c6400d000 tx=0x7f9c64007bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:07.162 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.147+0000 7f9c71ffb700 1 -- 172.21.15.140:0/1473497418 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9c6c015020 con 0x7f9c740b6800 2022-07-21T01:58:07.314 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.311+0000 7f9c79e44700 1 -- 172.21.15.140:0/1473497418 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f9c740b5530 con 0x7f9c740b6800 2022-07-21T01:58:07.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.319+0000 7f9c71ffb700 1 -- 172.21.15.140:0/1473497418 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v50) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9c6c015020 con 0x7f9c740b6800 2022-07-21T01:58:07.325 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 -- 172.21.15.140:0/1473497418 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c58053b30 msgr2=0x7f9c58055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c58053b30 0x7f9c58055ff0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f9c6400d000 tx=0x7f9c64007bc0).stop 2022-07-21T01:58:07.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 -- 172.21.15.140:0/1473497418 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c740b6800 msgr2=0x7f9c74144ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.326 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c740b6800 0x7f9c74144ef0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f9c74145e50 tx=0x7f9c6c016000).stop 2022-07-21T01:58:07.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 -- 172.21.15.140:0/1473497418 shutdown_connections 2022-07-21T01:58:07.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9c58053b30 0x7f9c58055ff0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9c740b6800 0x7f9c74144ef0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9c740b3cb0 0x7f9c74151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.327 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 --2- 172.21.15.140:0/1473497418 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9c740ad0e0 0x7f9c740af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 -- 172.21.15.140:0/1473497418 >> 172.21.15.140:0/1473497418 conn(0x7f9c740b9dd0 msgr2=0x7f9c74142f90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:07.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 -- 172.21.15.140:0/1473497418 shutdown_connections 2022-07-21T01:58:07.328 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.323+0000 7f9c79e44700 1 -- 172.21.15.140:0/1473497418 wait complete. 2022-07-21T01:58:07.329 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.1 2022-07-21T01:58:07.341 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.2 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2022-07-21T01:58:07.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc2f08700 1 Processor -- start 2022-07-21T01:58:07.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc2f08700 1 -- start start 2022-07-21T01:58:07.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc2f08700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc0b5830 0x7f2dbc142c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc2f08700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dbc143120 0x7f2dbc0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc2f08700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc147120 con 0x7f2dbc05cf40 2022-07-21T01:58:07.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc2f08700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dbc0b0130 con 0x7f2dbc143120 2022-07-21T01:58:07.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc2f08700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc1462d0 con 0x7f2dbc0b5830 2022-07-21T01:58:07.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc14a5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dbc143120 0x7f2dbc0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dbbfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc0b5830 0x7f2dbc142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc14a5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f2dbc143120 0x7f2dbc0af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:47594/0 (socket says 172.21.15.140:47594) 2022-07-21T01:58:07.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc14a5700 1 -- 172.21.15.140:0/3532953990 learned_addr learned my addr 172.21.15.140:0/3532953990 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:07.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc0ca4700 1 --1- 172.21.15.140:0/3532953990 >> v1:172.21.15.152:6789/0 conn(0x7f2dbc05cf40 0x7f2dbc146a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:57110/0 (socket says 172.21.15.140:57110) 2022-07-21T01:58:07.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc14a5700 1 -- 172.21.15.140:0/3532953990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc0b5830 msgr2=0x7f2dbc142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:07.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc14a5700 1 --2- 172.21.15.140:0/3532953990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc0b5830 0x7f2dbc142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc14a5700 1 -- 172.21.15.140:0/3532953990 >> v1:172.21.15.152:6789/0 conn(0x7f2dbc05cf40 legacy=0x7f2dbc146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc14a5700 1 -- 172.21.15.140:0/3532953990 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc146570 con 0x7f2dbc143120 2022-07-21T01:58:07.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dc14a5700 1 --2- 172.21.15.140:0/3532953990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dbc143120 0x7f2dbc0af2c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f2db40032f0 tx=0x7f2db4009290).ready entity=mon.1 client_cookie=fd62358a7fcd2f94 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:07.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dbb7fe700 1 -- 172.21.15.140:0/3532953990 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db400b8a0 con 0x7f2dbc143120 2022-07-21T01:58:07.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dbb7fe700 1 -- 172.21.15.140:0/3532953990 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db400ba00 con 0x7f2dbc143120 2022-07-21T01:58:07.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.459+0000 7f2dbb7fe700 1 -- 172.21.15.140:0/3532953990 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db400bb70 con 0x7f2dbc143120 2022-07-21T01:58:07.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.463+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3532953990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dbc143120 msgr2=0x7f2dbc0af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.463+0000 7f2dc2f08700 1 --2- 172.21.15.140:0/3532953990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dbc143120 0x7f2dbc0af2c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f2db40032f0 tx=0x7f2db4009290).stop 2022-07-21T01:58:07.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.463+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3532953990 shutdown_connections 2022-07-21T01:58:07.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.463+0000 7f2dc2f08700 1 --2- 172.21.15.140:0/3532953990 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc0b5830 0x7f2dbc142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.463+0000 7f2dc2f08700 1 --2- 172.21.15.140:0/3532953990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dbc143120 0x7f2dbc0af2c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.463+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3532953990 >> 172.21.15.140:0/3532953990 conn(0x7f2dbc0b69e0 msgr2=0x7f2dbc066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:07.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.463+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3532953990 shutdown_connections 2022-07-21T01:58:07.470 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.463+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3532953990 wait complete. 2022-07-21T01:58:07.473 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.463+0000 7f2dc2f08700 1 Processor -- start 2022-07-21T01:58:07.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.463+0000 7f2dc2f08700 1 -- start start 2022-07-21T01:58:07.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc2f08700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc05cf40 0x7f2dbc0600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc2f08700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dbc0b5830 0x7f2dbc1527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.474 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc2f08700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2dbc143120 0x7f2dbc0b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc2f08700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc13f700 con 0x7f2dbc143120 2022-07-21T01:58:07.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc2f08700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc13f9a0 con 0x7f2dbc0b5830 2022-07-21T01:58:07.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc2f08700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc13fc40 con 0x7f2dbc05cf40 2022-07-21T01:58:07.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc14a5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2dbc143120 0x7f2dbc0b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.475 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc0ca4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc05cf40 0x7f2dbc0600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc14a5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2dbc143120 0x7f2dbc0b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41860/0 (socket says 172.21.15.140:41860) 2022-07-21T01:58:07.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc14a5700 1 -- 172.21.15.140:0/3393235408 learned_addr learned my addr 172.21.15.140:0/3393235408 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:07.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc0ca4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc05cf40 0x7f2dbc0600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:38586/0 (socket says 172.21.15.140:38586) 2022-07-21T01:58:07.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dbbfff700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dbc0b5830 0x7f2dbc1527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc14a5700 1 -- 172.21.15.140:0/3393235408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc05cf40 msgr2=0x7f2dbc0600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc14a5700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc05cf40 0x7f2dbc0600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc14a5700 1 -- 172.21.15.140:0/3393235408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dbc0b5830 msgr2=0x7f2dbc1527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc14a5700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dbc0b5830 0x7f2dbc1527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc14a5700 1 -- 172.21.15.140:0/3393235408 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db4009530 con 0x7f2dbc143120 2022-07-21T01:58:07.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc0ca4700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc05cf40 0x7f2dbc0600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:58:07.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc14a5700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2dbc143120 0x7f2dbc0b2b00 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f2db40032f0 tx=0x7f2db4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:07.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2db9ffb700 1 -- 172.21.15.140:0/3393235408 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db400bb70 con 0x7f2dbc143120 2022-07-21T01:58:07.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2db9ffb700 1 -- 172.21.15.140:0/3393235408 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4021630 con 0x7f2dbc143120 2022-07-21T01:58:07.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3393235408 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc152cb0 con 0x7f2dbc143120 2022-07-21T01:58:07.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2db9ffb700 1 -- 172.21.15.140:0/3393235408 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2db400b8a0 con 0x7f2dbc143120 2022-07-21T01:58:07.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3393235408 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc09f3f0 con 0x7f2dbc143120 2022-07-21T01:58:07.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3393235408 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc061280 con 0x7f2dbc143120 2022-07-21T01:58:07.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.467+0000 7f2db9ffb700 1 -- 172.21.15.140:0/3393235408 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2db400f070 con 0x7f2dbc143120 2022-07-21T01:58:07.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.471+0000 7f2db9ffb700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d9c0539e0 0x7f2d9c055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.471+0000 7f2db9ffb700 1 -- 172.21.15.140:0/3393235408 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2db406cf90 con 0x7f2dbc143120 2022-07-21T01:58:07.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.471+0000 7f2dc0ca4700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d9c0539e0 0x7f2d9c055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.471+0000 7f2dc0ca4700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d9c0539e0 0x7f2d9c055ea0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f2dac00d000 tx=0x7f2dac007b80).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:07.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.475+0000 7f2db9ffb700 1 -- 172.21.15.140:0/3393235408 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2db4043da0 con 0x7f2dbc143120 2022-07-21T01:58:07.642 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.639+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3393235408 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2dbc0574e0 con 0x7f2dbc143120 2022-07-21T01:58:07.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.647+0000 7f2db9ffb700 1 -- 172.21.15.140:0/3393235408 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.2", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.2 v51) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2db4015030 con 0x7f2dbc143120 2022-07-21T01:58:07.652 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3393235408 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d9c0539e0 msgr2=0x7f2d9c055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d9c0539e0 0x7f2d9c055ea0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f2dac00d000 tx=0x7f2dac007b80).stop 2022-07-21T01:58:07.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3393235408 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2dbc143120 msgr2=0x7f2dbc0b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2dbc143120 0x7f2dbc0b2b00 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f2db40032f0 tx=0x7f2db4016000).stop 2022-07-21T01:58:07.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3393235408 shutdown_connections 2022-07-21T01:58:07.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f2d9c0539e0 0x7f2d9c055ea0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f2dbc143120 0x7f2dbc0b2b00 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2dbc0b5830 0x7f2dbc1527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 --2- 172.21.15.140:0/3393235408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2dbc05cf40 0x7f2dbc0600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3393235408 >> 172.21.15.140:0/3393235408 conn(0x7f2dbc0b69e0 msgr2=0x7f2dbc1428c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:07.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3393235408 shutdown_connections 2022-07-21T01:58:07.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.651+0000 7f2dc2f08700 1 -- 172.21.15.140:0/3393235408 wait complete. 2022-07-21T01:58:07.656 INFO:teuthology.orchestra.run.smithi140.stderr:updated caps for client.2 2022-07-21T01:58:07.666 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:58:07.772 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 Processor -- start 2022-07-21T01:58:07.773 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 -- start start 2022-07-21T01:58:07.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f31400adff0 0x7f31400b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3140147010 con 0x7f31400b5650 2022-07-21T01:58:07.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31401472b0 con 0x7f31400af8e0 2022-07-21T01:58:07.775 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3140146d40 con 0x7f31400adff0 2022-07-21T01:58:07.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f313ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f31400adff0 0x7f31400b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f313ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f31400adff0 0x7f31400b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41866/0 (socket says 172.21.15.140:41866) 2022-07-21T01:58:07.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f313ffff700 1 -- 172.21.15.140:0/848733183 learned_addr learned my addr 172.21.15.140:0/848733183 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:07.776 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f313ffff700 1 -- 172.21.15.140:0/848733183 >> v1:172.21.15.152:6789/0 conn(0x7f31400af8e0 legacy=0x7f314005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:07.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f313ffff700 1 -- 172.21.15.140:0/848733183 >> v1:172.21.15.140:6789/0 conn(0x7f31400b5650 legacy=0x7f3140147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f313ffff700 1 -- 172.21.15.140:0/848733183 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314005c4a0 con 0x7f31400adff0 2022-07-21T01:58:07.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f313ffff700 1 --2- 172.21.15.140:0/848733183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31400adff0 0x7f31400b5a30 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f31400622d0 tx=0x7f3138009250).ready entity=mon.0 client_cookie=2d20753b275fc5a4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:07.777 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f313f7fe700 1 -- 172.21.15.140:0/848733183 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f313800b860 con 0x7f31400adff0 2022-07-21T01:58:07.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f313f7fe700 1 -- 172.21.15.140:0/848733183 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313800b9c0 con 0x7f31400adff0 2022-07-21T01:58:07.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f313f7fe700 1 -- 172.21.15.140:0/848733183 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f313800bb30 con 0x7f31400adff0 2022-07-21T01:58:07.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 -- 172.21.15.140:0/848733183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31400adff0 msgr2=0x7f31400b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 --2- 172.21.15.140:0/848733183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31400adff0 0x7f31400b5a30 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f31400622d0 tx=0x7f3138009250).stop 2022-07-21T01:58:07.778 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 -- 172.21.15.140:0/848733183 shutdown_connections 2022-07-21T01:58:07.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 --2- 172.21.15.140:0/848733183 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31400adff0 0x7f31400b5a30 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 -- 172.21.15.140:0/848733183 >> 172.21.15.140:0/848733183 conn(0x7f31400634e0 msgr2=0x7f31400b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:07.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 -- 172.21.15.140:0/848733183 shutdown_connections 2022-07-21T01:58:07.779 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.771+0000 7f3145bda700 1 -- 172.21.15.140:0/848733183 wait complete. 2022-07-21T01:58:07.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3145bda700 1 Processor -- start 2022-07-21T01:58:07.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3145bda700 1 -- start start 2022-07-21T01:58:07.783 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3145bda700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31400adff0 0x7f31401573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3145bda700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31400af8e0 0x7f31401105c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3145bda700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31400b5650 0x7f31401491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3145bda700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314010f430 con 0x7f31400adff0 2022-07-21T01:58:07.784 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3145bda700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314010e980 con 0x7f31400af8e0 2022-07-21T01:58:07.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3145bda700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314010f190 con 0x7f31400b5650 2022-07-21T01:58:07.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3144bd8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31400adff0 0x7f31401573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3144bd8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31400adff0 0x7f31401573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41870/0 (socket says 172.21.15.140:41870) 2022-07-21T01:58:07.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3144bd8700 1 -- 172.21.15.140:0/4003449121 learned_addr learned my addr 172.21.15.140:0/4003449121 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:07.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f313ffff700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31400af8e0 0x7f31401105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f31453d9700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31400b5650 0x7f31401491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3144bd8700 1 -- 172.21.15.140:0/4003449121 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31400b5650 msgr2=0x7f31401491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3144bd8700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31400b5650 0x7f31401491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3144bd8700 1 -- 172.21.15.140:0/4003449121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31400af8e0 msgr2=0x7f31401105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3144bd8700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31400af8e0 0x7f31401105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3144bd8700 1 -- 172.21.15.140:0/4003449121 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31380094d0 con 0x7f31400adff0 2022-07-21T01:58:07.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f31453d9700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31400b5650 0x7f31401491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:58:07.787 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3144bd8700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31400adff0 0x7f31401573c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f314010f0b0 tx=0x7f3128007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:07.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f313dffb700 1 -- 172.21.15.140:0/4003449121 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f312800ce20 con 0x7f31400adff0 2022-07-21T01:58:07.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f313dffb700 1 -- 172.21.15.140:0/4003449121 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312800f470 con 0x7f31400adff0 2022-07-21T01:58:07.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3145bda700 1 -- 172.21.15.140:0/4003449121 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314010eb50 con 0x7f31400adff0 2022-07-21T01:58:07.788 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f313dffb700 1 -- 172.21.15.140:0/4003449121 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f312800f7d0 con 0x7f31400adff0 2022-07-21T01:58:07.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.775+0000 7f3145bda700 1 -- 172.21.15.140:0/4003449121 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314010fec0 con 0x7f31400adff0 2022-07-21T01:58:07.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.779+0000 7f3145bda700 1 -- 172.21.15.140:0/4003449121 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31400574e0 con 0x7f31400adff0 2022-07-21T01:58:07.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.779+0000 7f313dffb700 1 -- 172.21.15.140:0/4003449121 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3128019030 con 0x7f31400adff0 2022-07-21T01:58:07.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.779+0000 7f313dffb700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f31240539e0 0x7f3124055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:07.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.779+0000 7f313dffb700 1 -- 172.21.15.140:0/4003449121 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f312806b2d0 con 0x7f31400adff0 2022-07-21T01:58:07.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.779+0000 7f313ffff700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f31240539e0 0x7f3124055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:07.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.779+0000 7f313ffff700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f31240539e0 0x7f3124055ea0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f3138019000 tx=0x7f3138016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:07.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.783+0000 7f313dffb700 1 -- 172.21.15.140:0/4003449121 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f31280421f0 con 0x7f31400adff0 2022-07-21T01:58:07.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 -- 172.21.15.140:0/4003449121 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3140067ea0 con 0x7f31400adff0 2022-07-21T01:58:07.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f313dffb700 1 -- 172.21.15.140:0/4003449121 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2148 (secure 0 0 0) 0x7f3128045810 con 0x7f31400adff0 2022-07-21T01:58:07.946 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:07.946 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":74,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/3591375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591375250},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591375250}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:58:05.107197+0000","modified":"2022-07-21T01:58:06.110113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8727":{"gid":8727,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/4013573612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":4013573612},{"type":"v1","addr":"172.21.15.140:6835","nonce":4013573612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-07-21T01:58:07.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 -- 172.21.15.140:0/4003449121 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f31240539e0 msgr2=0x7f3124055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f31240539e0 0x7f3124055ea0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f3138019000 tx=0x7f3138016000).stop 2022-07-21T01:58:07.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 -- 172.21.15.140:0/4003449121 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31400adff0 msgr2=0x7f31401573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:07.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31400adff0 0x7f31401573c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f314010f0b0 tx=0x7f3128007c50).stop 2022-07-21T01:58:07.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 -- 172.21.15.140:0/4003449121 shutdown_connections 2022-07-21T01:58:07.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f31240539e0 0x7f3124055ea0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31400b5650 0x7f31401491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f31400af8e0 0x7f31401105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 --2- 172.21.15.140:0/4003449121 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f31400adff0 0x7f31401573c0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:07.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 -- 172.21.15.140:0/4003449121 >> 172.21.15.140:0/4003449121 conn(0x7f31400634e0 msgr2=0x7f31400b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:07.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 -- 172.21.15.140:0/4003449121 shutdown_connections 2022-07-21T01:58:07.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:07.943+0000 7f3145bda700 1 -- 172.21.15.140:0/4003449121 wait complete. 2022-07-21T01:58:07.951 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 74 2022-07-21T01:58:08.160 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:58:05.107197+0000', 'modified': '2022-07-21T01:58:06.110113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8727}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8727': {'gid': 8727, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/4013573612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 4013573612}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 4013573612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:58:08.161 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:58:08.161 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:58:08.281 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":8727,"want_state":"up:active","state":"up:active","rank_uptime":3.162511249,"mdsmap_epoch":74,"osdmap_epoch":151,"osdmap_epoch_barrier":150,"uptime":7.5356497979999997} 2022-07-21T01:58:08.282 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 8727, "mdsmap_epoch": 74, "osdmap_epoch": 151, "osdmap_epoch_barrier": 150, "rank_uptime": 3.162511249, "state": "up:active", "uptime": 7.535649798, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:58:08.283 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:58:08.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c37518700 1 Processor -- start 2022-07-21T01:58:08.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c37518700 1 -- start start 2022-07-21T01:58:08.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c37518700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c300b5650 0x7f8c300b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c37518700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8c300adff0 0x7f8c30147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c37518700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c30147010 con 0x7f8c300af8e0 2022-07-21T01:58:08.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c37518700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c301472b0 con 0x7f8c300b5650 2022-07-21T01:58:08.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c37518700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c30146d40 con 0x7f8c300adff0 2022-07-21T01:58:08.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c36516700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c300b5650 0x7f8c300b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c36516700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8c300b5650 0x7f8c300b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41878/0 (socket says 172.21.15.140:41878) 2022-07-21T01:58:08.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c36516700 1 -- 172.21.15.140:0/547565927 learned_addr learned my addr 172.21.15.140:0/547565927 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:08.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c35d15700 1 --2- 172.21.15.140:0/547565927 >> v2:172.21.15.152:3300/0 conn(0x7f8c300adff0 0x7f8c30147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c36516700 1 -- 172.21.15.140:0/547565927 >> v2:172.21.15.152:3300/0 conn(0x7f8c300adff0 msgr2=0x7f8c30147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:08.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c36516700 1 --2- 172.21.15.140:0/547565927 >> v2:172.21.15.152:3300/0 conn(0x7f8c300adff0 0x7f8c30147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c36516700 1 -- 172.21.15.140:0/547565927 >> v1:172.21.15.152:6789/0 conn(0x7f8c300af8e0 legacy=0x7f8c3005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:08.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c36516700 1 -- 172.21.15.140:0/547565927 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30147c40 con 0x7f8c300b5650 2022-07-21T01:58:08.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c35d15700 1 --2- 172.21.15.140:0/547565927 >> v2:172.21.15.152:3300/0 conn(0x7f8c300adff0 0x7f8c30147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:58:08.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c36516700 1 --2- 172.21.15.140:0/547565927 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c300b5650 0x7f8c300b5a30 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f8c30060b70 tx=0x7f8c28009290).ready entity=mon.0 client_cookie=13d8773601936d9e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:08.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c35514700 1 -- 172.21.15.140:0/547565927 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c2800b830 con 0x7f8c300b5650 2022-07-21T01:58:08.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c35514700 1 -- 172.21.15.140:0/547565927 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2800b990 con 0x7f8c300b5650 2022-07-21T01:58:08.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.383+0000 7f8c35514700 1 -- 172.21.15.140:0/547565927 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c2800bb40 con 0x7f8c300b5650 2022-07-21T01:58:08.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 -- 172.21.15.140:0/547565927 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c300b5650 msgr2=0x7f8c300b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 --2- 172.21.15.140:0/547565927 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c300b5650 0x7f8c300b5a30 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f8c30060b70 tx=0x7f8c28009290).stop 2022-07-21T01:58:08.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 -- 172.21.15.140:0/547565927 shutdown_connections 2022-07-21T01:58:08.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 --2- 172.21.15.140:0/547565927 >> v2:172.21.15.152:3300/0 conn(0x7f8c300adff0 0x7f8c30147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 --2- 172.21.15.140:0/547565927 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c300b5650 0x7f8c300b5a30 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 -- 172.21.15.140:0/547565927 >> 172.21.15.140:0/547565927 conn(0x7f8c300634e0 msgr2=0x7f8c300b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:08.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 -- 172.21.15.140:0/547565927 shutdown_connections 2022-07-21T01:58:08.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 -- 172.21.15.140:0/547565927 wait complete. 2022-07-21T01:58:08.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 Processor -- start 2022-07-21T01:58:08.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 -- start start 2022-07-21T01:58:08.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c300adff0 0x7f8c300648a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c300af8e0 0x7f8c301435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c300b5650 0x7f8c30143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c30067500 con 0x7f8c300af8e0 2022-07-21T01:58:08.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c30067730 con 0x7f8c300b5650 2022-07-21T01:58:08.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c37518700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c30065500 con 0x7f8c300adff0 2022-07-21T01:58:08.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c35d15700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c300af8e0 0x7f8c301435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c35d15700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c300af8e0 0x7f8c301435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41884/0 (socket says 172.21.15.140:41884) 2022-07-21T01:58:08.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c35d15700 1 -- 172.21.15.140:0/1213301074 learned_addr learned my addr 172.21.15.140:0/1213301074 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:08.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c36516700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c300adff0 0x7f8c300648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.387+0000 7f8c36d17700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c300b5650 0x7f8c30143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c35d15700 1 -- 172.21.15.140:0/1213301074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c300adff0 msgr2=0x7f8c300648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c35d15700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c300adff0 0x7f8c300648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c35d15700 1 -- 172.21.15.140:0/1213301074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c300b5650 msgr2=0x7f8c30143ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c35d15700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c300b5650 0x7f8c30143ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c35d15700 1 -- 172.21.15.140:0/1213301074 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c28009530 con 0x7f8c300af8e0 2022-07-21T01:58:08.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c35d15700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c300af8e0 0x7f8c301435d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f8c20007000 tx=0x7f8c20008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:08.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c1f7fe700 1 -- 172.21.15.140:0/1213301074 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c20010580 con 0x7f8c300af8e0 2022-07-21T01:58:08.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c1f7fe700 1 -- 172.21.15.140:0/1213301074 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c200106e0 con 0x7f8c300af8e0 2022-07-21T01:58:08.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c37518700 1 -- 172.21.15.140:0/1213301074 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c30143fb0 con 0x7f8c300af8e0 2022-07-21T01:58:08.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c1f7fe700 1 -- 172.21.15.140:0/1213301074 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c20010a40 con 0x7f8c300af8e0 2022-07-21T01:58:08.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c37518700 1 -- 172.21.15.140:0/1213301074 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c30063a40 con 0x7f8c300af8e0 2022-07-21T01:58:08.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c37518700 1 -- 172.21.15.140:0/1213301074 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c30111f80 con 0x7f8c300af8e0 2022-07-21T01:58:08.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.391+0000 7f8c1f7fe700 1 -- 172.21.15.140:0/1213301074 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8c20010ba0 con 0x7f8c300af8e0 2022-07-21T01:58:08.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.395+0000 7f8c1f7fe700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c10053a70 0x7f8c10055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.395+0000 7f8c1f7fe700 1 -- 172.21.15.140:0/1213301074 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8c2006b940 con 0x7f8c300af8e0 2022-07-21T01:58:08.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.395+0000 7f8c36516700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c10053a70 0x7f8c10055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.395+0000 7f8c36516700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c10053a70 0x7f8c10055f30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f8c2801b000 tx=0x7f8c28016000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:08.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.395+0000 7f8c1f7fe700 1 -- 172.21.15.140:0/1213301074 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8c20042800 con 0x7f8c300af8e0 2022-07-21T01:58:08.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.523+0000 7f8c37518700 1 -- 172.21.15.140:0/1213301074 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c3005c4a0 con 0x7f8c300af8e0 2022-07-21T01:58:08.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.523+0000 7f8c1f7fe700 1 -- 172.21.15.140:0/1213301074 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f8c20045e20 con 0x7f8c300af8e0 2022-07-21T01:58:08.528 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:08.528 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":151,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:58:06.110718+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:58:03.485327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-07-21T01:58:04.503066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:58:08.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 -- 172.21.15.140:0/1213301074 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c10053a70 msgr2=0x7f8c10055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c10053a70 0x7f8c10055f30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f8c2801b000 tx=0x7f8c28016000).stop 2022-07-21T01:58:08.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 -- 172.21.15.140:0/1213301074 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c300af8e0 msgr2=0x7f8c301435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c300af8e0 0x7f8c301435d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f8c20007000 tx=0x7f8c20008040).stop 2022-07-21T01:58:08.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 -- 172.21.15.140:0/1213301074 shutdown_connections 2022-07-21T01:58:08.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8c10053a70 0x7f8c10055f30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c300b5650 0x7f8c30143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8c300af8e0 0x7f8c301435d0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 --2- 172.21.15.140:0/1213301074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8c300adff0 0x7f8c300648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 -- 172.21.15.140:0/1213301074 >> 172.21.15.140:0/1213301074 conn(0x7f8c300634e0 msgr2=0x7f8c300b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:08.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 -- 172.21.15.140:0/1213301074 shutdown_connections 2022-07-21T01:58:08.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.527+0000 7f8c37518700 1 -- 172.21.15.140:0/1213301074 wait complete. 2022-07-21T01:58:08.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:58:03.485327+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:58:04.503066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:58:08.542 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:58:08.650 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.647+0000 7fcaaa8ee700 1 Processor -- start 2022-07-21T01:58:08.651 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.647+0000 7fcaaa8ee700 1 -- start start 2022-07-21T01:58:08.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaa40b0480 0x7fcaa413f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.654 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcaa40b1660 0x7fcaa413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa40ac900 con 0x7fcaa40b1280 2022-07-21T01:58:08.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40b09b0 con 0x7fcaa40b1660 2022-07-21T01:58:08.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40aef30 con 0x7fcaa40b0480 2022-07-21T01:58:08.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa8e8b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcaa40b1660 0x7fcaa413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa8e8b700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fcaa40b1660 0x7fcaa413f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41894/0 (socket says 172.21.15.140:41894) 2022-07-21T01:58:08.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa8e8b700 1 -- 172.21.15.140:0/2962727186 learned_addr learned my addr 172.21.15.140:0/2962727186 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:08.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa37fe700 1 --2- 172.21.15.140:0/2962727186 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaa40b0480 0x7fcaa413f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa8e8b700 1 -- 172.21.15.140:0/2962727186 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaa40b0480 msgr2=0x7fcaa413f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:08.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa8e8b700 1 --2- 172.21.15.140:0/2962727186 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaa40b0480 0x7fcaa413f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa8e8b700 1 -- 172.21.15.140:0/2962727186 >> v1:172.21.15.152:6789/0 conn(0x7fcaa40b1280 legacy=0x7fcaa413c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa8e8b700 1 -- 172.21.15.140:0/2962727186 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40af1d0 con 0x7fcaa40b1660 2022-07-21T01:58:08.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa8e8b700 1 --2- 172.21.15.140:0/2962727186 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaa40b1660 0x7fcaa413f5d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7fca98002f70 tx=0x7fca98006eb0).ready entity=mon.0 client_cookie=26918beca0ad22b9 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:08.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa2ffd700 1 -- 172.21.15.140:0/2962727186 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca98014070 con 0x7fcaa40b1660 2022-07-21T01:58:08.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa2ffd700 1 -- 172.21.15.140:0/2962727186 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca98000e90 con 0x7fcaa40b1660 2022-07-21T01:58:08.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaa2ffd700 1 -- 172.21.15.140:0/2962727186 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca9800eaa0 con 0x7fcaa40b1660 2022-07-21T01:58:08.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/2962727186 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaa40b1660 msgr2=0x7fcaa413f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 --2- 172.21.15.140:0/2962727186 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaa40b1660 0x7fcaa413f5d0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7fca98002f70 tx=0x7fca98006eb0).stop 2022-07-21T01:58:08.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/2962727186 shutdown_connections 2022-07-21T01:58:08.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 --2- 172.21.15.140:0/2962727186 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaa40b1660 0x7fcaa413f5d0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 --2- 172.21.15.140:0/2962727186 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaa40b0480 0x7fcaa413f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.659 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/2962727186 >> 172.21.15.140:0/2962727186 conn(0x7fcaa40635e0 msgr2=0x7fcaa40b5100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:08.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/2962727186 shutdown_connections 2022-07-21T01:58:08.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.651+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/2962727186 wait complete. 2022-07-21T01:58:08.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaaa8ee700 1 Processor -- start 2022-07-21T01:58:08.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaaa8ee700 1 -- start start 2022-07-21T01:58:08.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaaa8ee700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaa40b0480 0x7fcaa413f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaaa8ee700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaa40b1280 0x7fcaa40671b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaaa8ee700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa40b1660 0x7fcaa41520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaaa8ee700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa4156140 con 0x7fcaa40b0480 2022-07-21T01:58:08.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaaa8ee700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa41563e0 con 0x7fcaa40b1660 2022-07-21T01:58:08.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaaa8ee700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40b4970 con 0x7fcaa40b1280 2022-07-21T01:58:08.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaa40b0480 0x7fcaa413f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaa40b0480 0x7fcaa413f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41896/0 (socket says 172.21.15.140:41896) 2022-07-21T01:58:08.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa3fff700 1 -- 172.21.15.140:0/3001024467 learned_addr learned my addr 172.21.15.140:0/3001024467 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:08.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa8e8b700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa40b1660 0x7fcaa41520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa37fe700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaa40b1280 0x7fcaa40671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa3fff700 1 -- 172.21.15.140:0/3001024467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaa40b1280 msgr2=0x7fcaa40671b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa3fff700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaa40b1280 0x7fcaa40671b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa3fff700 1 -- 172.21.15.140:0/3001024467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa40b1660 msgr2=0x7fcaa41520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa3fff700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa40b1660 0x7fcaa41520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa3fff700 1 -- 172.21.15.140:0/3001024467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9800b1e0 con 0x7fcaa40b0480 2022-07-21T01:58:08.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa37fe700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaa40b1280 0x7fcaa40671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:58:08.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa8e8b700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa40b1660 0x7fcaa41520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:58:08.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa3fff700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaa40b0480 0x7fcaa413f7a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7fcaa4062710 tx=0x7fca9c0014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:08.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa17fa700 1 -- 172.21.15.140:0/3001024467 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca9c010510 con 0x7fcaa40b0480 2022-07-21T01:58:08.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa17fa700 1 -- 172.21.15.140:0/3001024467 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c010670 con 0x7fcaa40b0480 2022-07-21T01:58:08.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/3001024467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa40676a0 con 0x7fcaa40b0480 2022-07-21T01:58:08.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaa17fa700 1 -- 172.21.15.140:0/3001024467 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca9c0107e0 con 0x7fcaa40b0480 2022-07-21T01:58:08.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.655+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/3001024467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa40a2630 con 0x7fcaa40b0480 2022-07-21T01:58:08.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.659+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/3001024467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa41565a0 con 0x7fcaa40b0480 2022-07-21T01:58:08.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.659+0000 7fcaa17fa700 1 -- 172.21.15.140:0/3001024467 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fca9c010070 con 0x7fcaa40b0480 2022-07-21T01:58:08.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.659+0000 7fcaa17fa700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fca84053a70 0x7fca84055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.659+0000 7fcaa17fa700 1 -- 172.21.15.140:0/3001024467 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fca9c06bb10 con 0x7fcaa40b0480 2022-07-21T01:58:08.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.659+0000 7fcaa37fe700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fca84053a70 0x7fca84055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.659+0000 7fcaa37fe700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fca84053a70 0x7fca84055f30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fca9400b000 tx=0x7fca94008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:08.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.663+0000 7fcaa17fa700 1 -- 172.21.15.140:0/3001024467 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fca9c042a30 con 0x7fcaa40b0480 2022-07-21T01:58:08.789 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.787+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/3001024467 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcaa40574e0 con 0x7fcaa40b0480 2022-07-21T01:58:08.790 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.787+0000 7fcaa17fa700 1 -- 172.21.15.140:0/3001024467 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v151) v1 ==== 110+0+10 (secure 0 0 0) 0x7fca9c046050 con 0x7fcaa40b0480 2022-07-21T01:58:08.791 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:58:08.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/3001024467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fca84053a70 msgr2=0x7fca84055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fca84053a70 0x7fca84055f30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7fca9400b000 tx=0x7fca94008040).stop 2022-07-21T01:58:08.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/3001024467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaa40b0480 msgr2=0x7fcaa413f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaa40b0480 0x7fcaa413f7a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7fcaa4062710 tx=0x7fca9c0014a0).stop 2022-07-21T01:58:08.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/3001024467 shutdown_connections 2022-07-21T01:58:08.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fca84053a70 0x7fca84055f30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcaa40b1660 0x7fcaa41520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcaa40b1280 0x7fcaa40671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 --2- 172.21.15.140:0/3001024467 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fcaa40b0480 0x7fcaa413f7a0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/3001024467 >> 172.21.15.140:0/3001024467 conn(0x7fcaa40635e0 msgr2=0x7fcaa4142c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:08.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/3001024467 shutdown_connections 2022-07-21T01:58:08.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.791+0000 7fcaaa8ee700 1 -- 172.21.15.140:0/3001024467 wait complete. 2022-07-21T01:58:08.809 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:58:08.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6feb48a700 1 Processor -- start 2022-07-21T01:58:08.917 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6feb48a700 1 -- start start 2022-07-21T01:58:08.919 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6feb48a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fe40b14f0 0x7f6fe4146750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6feb48a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe40afbc0 con 0x7f6fe414ada0 2022-07-21T01:58:08.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6feb48a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe40b00e0 con 0x7f6fe40b59d0 2022-07-21T01:58:08.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6feb48a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe4148080 con 0x7f6fe40b14f0 2022-07-21T01:58:08.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6fe9c87700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f6fe414ada0 0x7f6fe4146040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56050/0 (socket says 172.21.15.140:56050) 2022-07-21T01:58:08.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6fe9c87700 1 -- 172.21.15.140:0/2183077425 learned_addr learned my addr 172.21.15.140:0/2183077425 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:08.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6feac89700 1 --2- 172.21.15.140:0/2183077425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fe40b14f0 0x7f6fe4146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6fe9486700 1 -- 172.21.15.140:0/2183077425 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405035642 0 0) 0x7f6fe40afbc0 con 0x7f6fe414ada0 2022-07-21T01:58:08.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6fe9486700 1 -- 172.21.15.140:0/2183077425 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fcc003260 con 0x7f6fe414ada0 2022-07-21T01:58:08.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6fe9486700 1 -- 172.21.15.140:0/2183077425 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157527278 0 0) 0x7f6fe40b00e0 con 0x7f6fe40b59d0 2022-07-21T01:58:08.922 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6fe9486700 1 -- 172.21.15.140:0/2183077425 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fe40afbc0 con 0x7f6fe40b59d0 2022-07-21T01:58:08.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6feac89700 1 -- 172.21.15.140:0/2183077425 >> v1:172.21.15.152:6789/0 conn(0x7f6fe40b59d0 legacy=0x7f6fe40b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6feac89700 1 -- 172.21.15.140:0/2183077425 >> v1:172.21.15.140:6789/0 conn(0x7f6fe414ada0 legacy=0x7f6fe4146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6feac89700 1 -- 172.21.15.140:0/2183077425 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40b5db0 con 0x7f6fe40b14f0 2022-07-21T01:58:08.923 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.915+0000 7f6feac89700 1 --2- 172.21.15.140:0/2183077425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fe40b14f0 0x7f6fe4146750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f6fdc0032f0 tx=0x7f6fdc0091d0).ready entity=mon.2 client_cookie=bcbe6e718192d01a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:08.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9486700 1 -- 172.21.15.140:0/2183077425 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fdc00b640 con 0x7f6fe40b14f0 2022-07-21T01:58:08.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9486700 1 -- 172.21.15.140:0/2183077425 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc00b7a0 con 0x7f6fe40b14f0 2022-07-21T01:58:08.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9486700 1 -- 172.21.15.140:0/2183077425 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fdc00b910 con 0x7f6fe40b14f0 2022-07-21T01:58:08.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 -- 172.21.15.140:0/2183077425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fe40b14f0 msgr2=0x7f6fe4146750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 --2- 172.21.15.140:0/2183077425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fe40b14f0 0x7f6fe4146750 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f6fdc0032f0 tx=0x7f6fdc0091d0).stop 2022-07-21T01:58:08.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 -- 172.21.15.140:0/2183077425 shutdown_connections 2022-07-21T01:58:08.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 --2- 172.21.15.140:0/2183077425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fe40b14f0 0x7f6fe4146750 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 -- 172.21.15.140:0/2183077425 >> 172.21.15.140:0/2183077425 conn(0x7f6fe40634c0 msgr2=0x7f6fe40b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:08.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 -- 172.21.15.140:0/2183077425 shutdown_connections 2022-07-21T01:58:08.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 -- 172.21.15.140:0/2183077425 wait complete. 2022-07-21T01:58:08.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 Processor -- start 2022-07-21T01:58:08.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 -- start start 2022-07-21T01:58:08.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fe40b14f0 0x7f6fe40ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fe40b59d0 0x7f6fe41597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.930 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fe414ada0 0x7f6fe4159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe414a7f0 con 0x7f6fe40b59d0 2022-07-21T01:58:08.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe4066ba0 con 0x7f6fe414ada0 2022-07-21T01:58:08.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe4066e40 con 0x7f6fe40b14f0 2022-07-21T01:58:08.931 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9c87700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fe40b59d0 0x7f6fe41597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9c87700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fe40b59d0 0x7f6fe41597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41910/0 (socket says 172.21.15.140:41910) 2022-07-21T01:58:08.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9c87700 1 -- 172.21.15.140:0/3254197887 learned_addr learned my addr 172.21.15.140:0/3254197887 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:08.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feac89700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fe414ada0 0x7f6fe4159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.932 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fea488700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fe40b14f0 0x7f6fe40ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9c87700 1 -- 172.21.15.140:0/3254197887 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fe40b14f0 msgr2=0x7f6fe40ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9c87700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fe40b14f0 0x7f6fe40ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9c87700 1 -- 172.21.15.140:0/3254197887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fe414ada0 msgr2=0x7f6fe4159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:08.933 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9c87700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fe414ada0 0x7f6fe4159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:08.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9c87700 1 -- 172.21.15.140:0/3254197887 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc009470 con 0x7f6fe40b59d0 2022-07-21T01:58:08.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fe9c87700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fe40b59d0 0x7f6fe41597e0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f6fe0001110 tx=0x7f6fe0007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:08.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fd77fe700 1 -- 172.21.15.140:0/3254197887 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe000cd80 con 0x7f6fe40b59d0 2022-07-21T01:58:08.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fd77fe700 1 -- 172.21.15.140:0/3254197887 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe00070d0 con 0x7f6fe40b59d0 2022-07-21T01:58:08.934 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6feb48a700 1 -- 172.21.15.140:0/3254197887 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe4149a90 con 0x7f6fe40b59d0 2022-07-21T01:58:08.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.919+0000 7f6fd77fe700 1 -- 172.21.15.140:0/3254197887 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe0007470 con 0x7f6fe40b59d0 2022-07-21T01:58:08.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.923+0000 7f6feb48a700 1 -- 172.21.15.140:0/3254197887 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe415a3e0 con 0x7f6fe40b59d0 2022-07-21T01:58:08.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.923+0000 7f6feb48a700 1 -- 172.21.15.140:0/3254197887 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe4149c20 con 0x7f6fe40b59d0 2022-07-21T01:58:08.935 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.923+0000 7f6fd77fe700 1 -- 172.21.15.140:0/3254197887 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6fe0010070 con 0x7f6fe40b59d0 2022-07-21T01:58:08.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.923+0000 7f6fd77fe700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fc4053b00 0x7f6fc4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:08.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.923+0000 7f6fd77fe700 1 -- 172.21.15.140:0/3254197887 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6fe006b2f0 con 0x7f6fe40b59d0 2022-07-21T01:58:08.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.923+0000 7f6fea488700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fc4053b00 0x7f6fc4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:08.936 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.923+0000 7f6fea488700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fc4053b00 0x7f6fc4055fc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f6fd800d000 tx=0x7f6fd8007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:08.937 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:08.935+0000 7f6fd77fe700 1 -- 172.21.15.140:0/3254197887 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6fe00421b0 con 0x7f6fe40b59d0 2022-07-21T01:58:09.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 -- 172.21.15.140:0/3254197887 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6fe40b0340 con 0x7f6fe40b59d0 2022-07-21T01:58:09.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6fd77fe700 1 -- 172.21.15.140:0/3254197887 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6fe00457d0 con 0x7f6fe40b59d0 2022-07-21T01:58:09.066 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:58:09.068 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 -- 172.21.15.140:0/3254197887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fc4053b00 msgr2=0x7f6fc4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fc4053b00 0x7f6fc4055fc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f6fd800d000 tx=0x7f6fd8007a00).stop 2022-07-21T01:58:09.069 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 -- 172.21.15.140:0/3254197887 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fe40b59d0 msgr2=0x7f6fe41597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fe40b59d0 0x7f6fe41597e0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f6fe0001110 tx=0x7f6fe0007a60).stop 2022-07-21T01:58:09.070 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 -- 172.21.15.140:0/3254197887 shutdown_connections 2022-07-21T01:58:09.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6fc4053b00 0x7f6fc4055fc0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.071 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6fe414ada0 0x7f6fe4159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6fe40b59d0 0x7f6fe41597e0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 --2- 172.21.15.140:0/3254197887 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6fe40b14f0 0x7f6fe40ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.072 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 -- 172.21.15.140:0/3254197887 >> 172.21.15.140:0/3254197887 conn(0x7f6fe40634c0 msgr2=0x7f6fe40b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:09.073 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 -- 172.21.15.140:0/3254197887 shutdown_connections 2022-07-21T01:58:09.074 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.063+0000 7f6feb48a700 1 -- 172.21.15.140:0/3254197887 wait complete. 2022-07-21T01:58:09.078 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:58:09.189 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f053aed5700 1 Processor -- start 2022-07-21T01:58:09.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f053aed5700 1 -- start start 2022-07-21T01:58:09.190 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f053aed5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05340b59f0 0x7f05341473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f053aed5700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053405cc70 con 0x7f05340b1400 2022-07-21T01:58:09.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f053aed5700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05340b5dd0 con 0x7f05340af940 2022-07-21T01:58:09.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f053aed5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05340b1070 con 0x7f05340b59f0 2022-07-21T01:58:09.191 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f053a6d4700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f05340b1400 0x7f053414aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56068/0 (socket says 172.21.15.140:56068) 2022-07-21T01:58:09.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f053a6d4700 1 -- 172.21.15.140:0/2949993061 learned_addr learned my addr 172.21.15.140:0/2949993061 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:09.192 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0539ed3700 1 --2- 172.21.15.140:0/2949993061 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05340b59f0 0x7f05341473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0538ed1700 1 -- 172.21.15.140:0/2949993061 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838237074 0 0) 0x7f053405cc70 con 0x7f05340b1400 2022-07-21T01:58:09.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0538ed1700 1 -- 172.21.15.140:0/2949993061 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0518003260 con 0x7f05340b1400 2022-07-21T01:58:09.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0538ed1700 1 -- 172.21.15.140:0/2949993061 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685436228 0 0) 0x7f05340b5dd0 con 0x7f05340af940 2022-07-21T01:58:09.197 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0538ed1700 1 -- 172.21.15.140:0/2949993061 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f053405cc70 con 0x7f05340af940 2022-07-21T01:58:09.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0539ed3700 1 -- 172.21.15.140:0/2949993061 >> v1:172.21.15.152:6789/0 conn(0x7f05340af940 legacy=0x7f053414a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0539ed3700 1 -- 172.21.15.140:0/2949993061 >> v1:172.21.15.140:6789/0 conn(0x7f05340b1400 legacy=0x7f053414aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0539ed3700 1 -- 172.21.15.140:0/2949993061 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053405c760 con 0x7f05340b59f0 2022-07-21T01:58:09.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0539ed3700 1 --2- 172.21.15.140:0/2949993061 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05340b59f0 0x7f05341473a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f0524002f70 tx=0x7f0524006df0).ready entity=mon.2 client_cookie=c69fdf2bb96d214d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:09.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0538ed1700 1 -- 172.21.15.140:0/2949993061 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f052400d5c0 con 0x7f05340b59f0 2022-07-21T01:58:09.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0538ed1700 1 -- 172.21.15.140:0/2949993061 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0524000e90 con 0x7f05340b59f0 2022-07-21T01:58:09.199 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.187+0000 7f0538ed1700 1 -- 172.21.15.140:0/2949993061 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f052400d910 con 0x7f05340b59f0 2022-07-21T01:58:09.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 -- 172.21.15.140:0/2949993061 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05340b59f0 msgr2=0x7f05341473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 --2- 172.21.15.140:0/2949993061 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05340b59f0 0x7f05341473a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f0524002f70 tx=0x7f0524006df0).stop 2022-07-21T01:58:09.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 -- 172.21.15.140:0/2949993061 shutdown_connections 2022-07-21T01:58:09.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 --2- 172.21.15.140:0/2949993061 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05340b59f0 0x7f05341473a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.200 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 -- 172.21.15.140:0/2949993061 >> 172.21.15.140:0/2949993061 conn(0x7f0534063480 msgr2=0x7f05340b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:09.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 -- 172.21.15.140:0/2949993061 shutdown_connections 2022-07-21T01:58:09.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 -- 172.21.15.140:0/2949993061 wait complete. 2022-07-21T01:58:09.201 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 Processor -- start 2022-07-21T01:58:09.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 -- start start 2022-07-21T01:58:09.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05340af940 0x7f05340b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05340b1400 0x7f0534156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05340b59f0 0x7f0534156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0534157310 con 0x7f05340af940 2022-07-21T01:58:09.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053414ab40 con 0x7f05340b59f0 2022-07-21T01:58:09.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f053aed5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0534111e80 con 0x7f05340b1400 2022-07-21T01:58:09.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f0539ed3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05340af940 0x7f05340b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f0539ed3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05340af940 0x7f05340b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41924/0 (socket says 172.21.15.140:41924) 2022-07-21T01:58:09.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f0539ed3700 1 -- 172.21.15.140:0/75800754 learned_addr learned my addr 172.21.15.140:0/75800754 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:09.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.191+0000 7f05396d2700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05340b1400 0x7f0534156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f053a6d4700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05340b59f0 0x7f0534156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f0539ed3700 1 -- 172.21.15.140:0/75800754 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05340b1400 msgr2=0x7f0534156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f0539ed3700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05340b1400 0x7f0534156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f0539ed3700 1 -- 172.21.15.140:0/75800754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05340b59f0 msgr2=0x7f0534156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f0539ed3700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05340b59f0 0x7f0534156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f0539ed3700 1 -- 172.21.15.140:0/75800754 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052400b180 con 0x7f05340af940 2022-07-21T01:58:09.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f0539ed3700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05340af940 0x7f05340b2f70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f0524002f70 tx=0x7f0524014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:09.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f052affd700 1 -- 172.21.15.140:0/75800754 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0524016070 con 0x7f05340af940 2022-07-21T01:58:09.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f053aed5700 1 -- 172.21.15.140:0/75800754 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0534112120 con 0x7f05340af940 2022-07-21T01:58:09.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f052affd700 1 -- 172.21.15.140:0/75800754 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05240137d0 con 0x7f05340af940 2022-07-21T01:58:09.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f053aed5700 1 -- 172.21.15.140:0/75800754 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05341562d0 con 0x7f05340af940 2022-07-21T01:58:09.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f052affd700 1 -- 172.21.15.140:0/75800754 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0524013b30 con 0x7f05340af940 2022-07-21T01:58:09.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f053aed5700 1 -- 172.21.15.140:0/75800754 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053405c9d0 con 0x7f05340af940 2022-07-21T01:58:09.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f052affd700 1 -- 172.21.15.140:0/75800754 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f052400f030 con 0x7f05340af940 2022-07-21T01:58:09.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f052affd700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f051c053a70 0x7f051c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.195+0000 7f052affd700 1 -- 172.21.15.140:0/75800754 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f052406cdf0 con 0x7f05340af940 2022-07-21T01:58:09.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.199+0000 7f05396d2700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f051c053a70 0x7f051c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.199+0000 7f05396d2700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f051c053a70 0x7f051c055f30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f053000d000 tx=0x7f0530007a00).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:09.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.203+0000 7f052affd700 1 -- 172.21.15.140:0/75800754 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f0524043b80 con 0x7f05340af940 2022-07-21T01:58:09.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 -- 172.21.15.140:0/75800754 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05340574e0 con 0x7f05340af940 2022-07-21T01:58:09.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f052affd700 1 -- 172.21.15.140:0/75800754 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+10 (secure 0 0 0) 0x7f05240230a0 con 0x7f05340af940 2022-07-21T01:58:09.333 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:58:09.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 -- 172.21.15.140:0/75800754 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f051c053a70 msgr2=0x7f051c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f051c053a70 0x7f051c055f30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f053000d000 tx=0x7f0530007a00).stop 2022-07-21T01:58:09.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 -- 172.21.15.140:0/75800754 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05340af940 msgr2=0x7f05340b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05340af940 0x7f05340b2f70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f0524002f70 tx=0x7f0524014000).stop 2022-07-21T01:58:09.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 -- 172.21.15.140:0/75800754 shutdown_connections 2022-07-21T01:58:09.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f051c053a70 0x7f051c055f30 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05340b59f0 0x7f0534156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f05340b1400 0x7f0534156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 --2- 172.21.15.140:0/75800754 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f05340af940 0x7f05340b2f70 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 -- 172.21.15.140:0/75800754 >> 172.21.15.140:0/75800754 conn(0x7f0534063480 msgr2=0x7f05340b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:09.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 -- 172.21.15.140:0/75800754 shutdown_connections 2022-07-21T01:58:09.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.331+0000 7f053aed5700 1 -- 172.21.15.140:0/75800754 wait complete. 2022-07-21T01:58:09.352 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:58:09.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.455+0000 7f3631251700 1 Processor -- start 2022-07-21T01:58:09.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.455+0000 7f3631251700 1 -- start start 2022-07-21T01:58:09.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f362c0af030 0x7f362c05c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f362c0b5690 0x7f362c140730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0af410 con 0x7f362c140c20 2022-07-21T01:58:09.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c144050 con 0x7f362c0b5690 2022-07-21T01:58:09.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c05be30 con 0x7f362c0af030 2022-07-21T01:58:09.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f362c0b5690 0x7f362c140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362b7fe700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f362c0b5690 0x7f362c140730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41934/0 (socket says 172.21.15.140:41934) 2022-07-21T01:58:09.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362b7fe700 1 -- 172.21.15.140:0/1869880263 learned_addr learned my addr 172.21.15.140:0/1869880263 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:09.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362bfff700 1 --2- 172.21.15.140:0/1869880263 >> v2:172.21.15.152:3300/0 conn(0x7f362c0af030 0x7f362c05c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362b7fe700 1 -- 172.21.15.140:0/1869880263 >> v2:172.21.15.152:3300/0 conn(0x7f362c0af030 msgr2=0x7f362c05c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:09.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362b7fe700 1 --2- 172.21.15.140:0/1869880263 >> v2:172.21.15.152:3300/0 conn(0x7f362c0af030 0x7f362c05c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362b7fe700 1 -- 172.21.15.140:0/1869880263 >> v1:172.21.15.152:6789/0 conn(0x7f362c140c20 legacy=0x7f362c1296f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362b7fe700 1 -- 172.21.15.140:0/1869880263 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c05c0d0 con 0x7f362c0b5690 2022-07-21T01:58:09.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362b7fe700 1 --2- 172.21.15.140:0/1869880263 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f362c0b5690 0x7f362c140730 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f36140032f0 tx=0x7f3614009290).ready entity=mon.0 client_cookie=997fbfe13e2044ae server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:09.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362affd700 1 -- 172.21.15.140:0/1869880263 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361400b870 con 0x7f362c0b5690 2022-07-21T01:58:09.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362affd700 1 -- 172.21.15.140:0/1869880263 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361400b9d0 con 0x7f362c0b5690 2022-07-21T01:58:09.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f362affd700 1 -- 172.21.15.140:0/1869880263 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361400bb40 con 0x7f362c0b5690 2022-07-21T01:58:09.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 -- 172.21.15.140:0/1869880263 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f362c0b5690 msgr2=0x7f362c140730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.476 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 --2- 172.21.15.140:0/1869880263 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f362c0b5690 0x7f362c140730 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f36140032f0 tx=0x7f3614009290).stop 2022-07-21T01:58:09.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 -- 172.21.15.140:0/1869880263 shutdown_connections 2022-07-21T01:58:09.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 --2- 172.21.15.140:0/1869880263 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f362c0b5690 0x7f362c140730 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.477 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 --2- 172.21.15.140:0/1869880263 >> v2:172.21.15.152:3300/0 conn(0x7f362c0af030 0x7f362c05c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 -- 172.21.15.140:0/1869880263 >> 172.21.15.140:0/1869880263 conn(0x7f362c0634c0 msgr2=0x7f362c0b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:09.478 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 -- 172.21.15.140:0/1869880263 shutdown_connections 2022-07-21T01:58:09.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.459+0000 7f3631251700 1 -- 172.21.15.140:0/1869880263 wait complete. 2022-07-21T01:58:09.479 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3631251700 1 Processor -- start 2022-07-21T01:58:09.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3631251700 1 -- start start 2022-07-21T01:58:09.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3631251700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f362c0af030 0x7f362c0b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3631251700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f362c0b5690 0x7f362c066690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3631251700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f362c140c20 0x7f362c064950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3631251700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c12c530 con 0x7f362c140c20 2022-07-21T01:58:09.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3631251700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c064210 con 0x7f362c0af030 2022-07-21T01:58:09.482 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3631251700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0644b0 con 0x7f362c0b5690 2022-07-21T01:58:09.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3630a50700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f362c140c20 0x7f362c064950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.483 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3630a50700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f362c140c20 0x7f362c064950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41942/0 (socket says 172.21.15.140:41942) 2022-07-21T01:58:09.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3630a50700 1 -- 172.21.15.140:0/2170171961 learned_addr learned my addr 172.21.15.140:0/2170171961 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:09.484 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f362bfff700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f362c0af030 0x7f362c0b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f362b7fe700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f362c0b5690 0x7f362c066690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.485 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3630a50700 1 -- 172.21.15.140:0/2170171961 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f362c0b5690 msgr2=0x7f362c066690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3630a50700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f362c0b5690 0x7f362c066690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3630a50700 1 -- 172.21.15.140:0/2170171961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f362c0af030 msgr2=0x7f362c0b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.486 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3630a50700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f362c0af030 0x7f362c0b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3630a50700 1 -- 172.21.15.140:0/2170171961 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614009530 con 0x7f362c140c20 2022-07-21T01:58:09.487 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3630a50700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f362c140c20 0x7f362c064950 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f362c064130 tx=0x7f3624007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:09.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f36297fa700 1 -- 172.21.15.140:0/2170171961 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3624001d50 con 0x7f362c140c20 2022-07-21T01:58:09.488 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f36297fa700 1 -- 172.21.15.140:0/2170171961 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3624007030 con 0x7f362c140c20 2022-07-21T01:58:09.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3631251700 1 -- 172.21.15.140:0/2170171961 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c066b80 con 0x7f362c140c20 2022-07-21T01:58:09.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f36297fa700 1 -- 172.21.15.140:0/2170171961 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3624007390 con 0x7f362c140c20 2022-07-21T01:58:09.489 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.463+0000 7f3631251700 1 -- 172.21.15.140:0/2170171961 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c065dd0 con 0x7f362c140c20 2022-07-21T01:58:09.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.467+0000 7f36297fa700 1 -- 172.21.15.140:0/2170171961 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3624010030 con 0x7f362c140c20 2022-07-21T01:58:09.490 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.467+0000 7f3631251700 1 -- 172.21.15.140:0/2170171961 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c064670 con 0x7f362c140c20 2022-07-21T01:58:09.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.467+0000 7f36297fa700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36100539e0 0x7f3610055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.467+0000 7f36297fa700 1 -- 172.21.15.140:0/2170171961 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f362406b750 con 0x7f362c140c20 2022-07-21T01:58:09.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.467+0000 7f362bfff700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36100539e0 0x7f3610055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.467+0000 7f362bfff700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36100539e0 0x7f3610055ea0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f361c00c000 tx=0x7f361c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:09.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.471+0000 7f36297fa700 1 -- 172.21.15.140:0/2170171961 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3624042670 con 0x7f362c140c20 2022-07-21T01:58:09.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.595+0000 7f3631251700 1 -- 172.21.15.140:0/2170171961 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f362c065dd0 con 0x7f362c140c20 2022-07-21T01:58:09.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.595+0000 7f36297fa700 1 -- 172.21.15.140:0/2170171961 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+10 (secure 0 0 0) 0x7f362c065dd0 con 0x7f362c140c20 2022-07-21T01:58:09.599 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:58:09.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 -- 172.21.15.140:0/2170171961 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36100539e0 msgr2=0x7f3610055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36100539e0 0x7f3610055ea0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f361c00c000 tx=0x7f361c008040).stop 2022-07-21T01:58:09.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 -- 172.21.15.140:0/2170171961 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f362c140c20 msgr2=0x7f362c064950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f362c140c20 0x7f362c064950 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f362c064130 tx=0x7f3624007bf0).stop 2022-07-21T01:58:09.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 -- 172.21.15.140:0/2170171961 shutdown_connections 2022-07-21T01:58:09.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f36100539e0 0x7f3610055ea0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f362c140c20 0x7f362c064950 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f362c0b5690 0x7f362c066690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 --2- 172.21.15.140:0/2170171961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f362c0af030 0x7f362c0b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 -- 172.21.15.140:0/2170171961 >> 172.21.15.140:0/2170171961 conn(0x7f362c0634c0 msgr2=0x7f362c142c50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:09.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 -- 172.21.15.140:0/2170171961 shutdown_connections 2022-07-21T01:58:09.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.599+0000 7f3631251700 1 -- 172.21.15.140:0/2170171961 wait complete. 2022-07-21T01:58:09.611 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:58:09.612 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:58:09.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f25998de700 1 Processor -- start 2022-07-21T01:58:09.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f25998de700 1 -- start start 2022-07-21T01:58:09.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f25998de700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f25940b1200 0x7f25940b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f25998de700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f259405bd90 0x7f2594142d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f25998de700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25940aea30 con 0x7f25940b5650 2022-07-21T01:58:09.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f25998de700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2594142970 con 0x7f25940b1200 2022-07-21T01:58:09.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f25998de700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25940aec60 con 0x7f259405bd90 2022-07-21T01:58:09.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f2593fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f25940b1200 0x7f25940b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f25988dc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f25940b5650 0x7f259405c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56094/0 (socket says 172.21.15.140:56094) 2022-07-21T01:58:09.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f25988dc700 1 -- 172.21.15.140:0/2368507157 learned_addr learned my addr 172.21.15.140:0/2368507157 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:09.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f25990dd700 1 --2- 172.21.15.140:0/2368507157 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f259405bd90 0x7f2594142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f2593fff700 1 -- 172.21.15.140:0/2368507157 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f259405bd90 msgr2=0x7f2594142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:09.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.711+0000 7f2593fff700 1 --2- 172.21.15.140:0/2368507157 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f259405bd90 0x7f2594142d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f2593fff700 1 -- 172.21.15.140:0/2368507157 >> v1:172.21.15.140:6789/0 conn(0x7f25940b5650 legacy=0x7f259405c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25937fe700 1 -- 172.21.15.140:0/2368507157 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849765374 0 0) 0x7f25940aea30 con 0x7f25940b5650 2022-07-21T01:58:09.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f2593fff700 1 -- 172.21.15.140:0/2368507157 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25940af180 con 0x7f25940b1200 2022-07-21T01:58:09.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f2593fff700 1 --2- 172.21.15.140:0/2368507157 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f25940b1200 0x7f25940b5a30 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f257c0032f0 tx=0x7f257c0091a0).ready entity=mon.0 client_cookie=b30b6956183639b4 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:09.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25937fe700 1 -- 172.21.15.140:0/2368507157 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f257c00b680 con 0x7f25940b1200 2022-07-21T01:58:09.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25937fe700 1 -- 172.21.15.140:0/2368507157 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c00b7e0 con 0x7f25940b1200 2022-07-21T01:58:09.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25937fe700 1 -- 172.21.15.140:0/2368507157 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f257c00b950 con 0x7f25940b1200 2022-07-21T01:58:09.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 -- 172.21.15.140:0/2368507157 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f25940b1200 msgr2=0x7f25940b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 --2- 172.21.15.140:0/2368507157 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f25940b1200 0x7f25940b5a30 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f257c0032f0 tx=0x7f257c0091a0).stop 2022-07-21T01:58:09.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 -- 172.21.15.140:0/2368507157 shutdown_connections 2022-07-21T01:58:09.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 --2- 172.21.15.140:0/2368507157 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f259405bd90 0x7f2594142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 --2- 172.21.15.140:0/2368507157 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f25940b1200 0x7f25940b5a30 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 -- 172.21.15.140:0/2368507157 >> 172.21.15.140:0/2368507157 conn(0x7f25940634e0 msgr2=0x7f25940b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:09.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 -- 172.21.15.140:0/2368507157 shutdown_connections 2022-07-21T01:58:09.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 -- 172.21.15.140:0/2368507157 wait complete. 2022-07-21T01:58:09.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 Processor -- start 2022-07-21T01:58:09.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 -- start start 2022-07-21T01:58:09.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f259405bd90 0x7f259413d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f25940b1200 0x7f25940b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f25940b5650 0x7f2594155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.729 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2594064870 con 0x7f25940b5650 2022-07-21T01:58:09.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2594155450 con 0x7f259405bd90 2022-07-21T01:58:09.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25998de700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25941556f0 con 0x7f25940b1200 2022-07-21T01:58:09.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25990dd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f25940b5650 0x7f2594155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.730 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25990dd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f25940b5650 0x7f2594155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41956/0 (socket says 172.21.15.140:41956) 2022-07-21T01:58:09.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.715+0000 7f25990dd700 1 -- 172.21.15.140:0/4145016954 learned_addr learned my addr 172.21.15.140:0/4145016954 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:09.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f2593fff700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f25940b1200 0x7f25940b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f25988dc700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f259405bd90 0x7f259413d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.731 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f25990dd700 1 -- 172.21.15.140:0/4145016954 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f25940b1200 msgr2=0x7f25940b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f25990dd700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f25940b1200 0x7f25940b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f25990dd700 1 -- 172.21.15.140:0/4145016954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f259405bd90 msgr2=0x7f259413d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f25990dd700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f259405bd90 0x7f259413d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f25990dd700 1 -- 172.21.15.140:0/4145016954 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c009420 con 0x7f25940b5650 2022-07-21T01:58:09.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f25990dd700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f25940b5650 0x7f2594155b90 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f258400c000 tx=0x7f2584006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:09.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f2591ffb700 1 -- 172.21.15.140:0/4145016954 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2584011750 con 0x7f25940b5650 2022-07-21T01:58:09.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f2591ffb700 1 -- 172.21.15.140:0/4145016954 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25840118b0 con 0x7f25940b5650 2022-07-21T01:58:09.733 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f2591ffb700 1 -- 172.21.15.140:0/4145016954 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2584011c10 con 0x7f25940b5650 2022-07-21T01:58:09.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f25998de700 1 -- 172.21.15.140:0/4145016954 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259413d0a0 con 0x7f25940b5650 2022-07-21T01:58:09.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f25998de700 1 -- 172.21.15.140:0/4145016954 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2594064400 con 0x7f25940b5650 2022-07-21T01:58:09.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f25998de700 1 -- 172.21.15.140:0/4145016954 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25940646b0 con 0x7f25940b5650 2022-07-21T01:58:09.734 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f2591ffb700 1 -- 172.21.15.140:0/4145016954 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f2584011d70 con 0x7f25940b5650 2022-07-21T01:58:09.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f2591ffb700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f256c053a70 0x7f256c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:09.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f25988dc700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f256c053a70 0x7f256c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:09.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.719+0000 7f2591ffb700 1 -- 172.21.15.140:0/4145016954 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f258406c2e0 con 0x7f25940b5650 2022-07-21T01:58:09.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.723+0000 7f25988dc700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f256c053a70 0x7f256c055f30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f258c00b000 tx=0x7f258c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:09.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.723+0000 7f2591ffb700 1 -- 172.21.15.140:0/4145016954 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f2584043200 con 0x7f25940b5650 2022-07-21T01:58:09.888 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.883+0000 7f25998de700 1 -- 172.21.15.140:0/4145016954 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25941558b0 con 0x7f25940b5650 2022-07-21T01:58:09.889 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f2591ffb700 1 -- 172.21.15.140:0/4145016954 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2148 (secure 0 0 0) 0x7f2584046820 con 0x7f25940b5650 2022-07-21T01:58:09.890 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:09.890 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":74,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/3591375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591375250},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591375250}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:58:05.107197+0000","modified":"2022-07-21T01:58:06.110113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8727":{"gid":8727,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/4013573612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":4013573612},{"type":"v1","addr":"172.21.15.140:6835","nonce":4013573612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-07-21T01:58:09.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 -- 172.21.15.140:0/4145016954 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f256c053a70 msgr2=0x7f256c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.893 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f256c053a70 0x7f256c055f30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f258c00b000 tx=0x7f258c009040).stop 2022-07-21T01:58:09.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 -- 172.21.15.140:0/4145016954 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f25940b5650 msgr2=0x7f2594155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:09.894 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f25940b5650 0x7f2594155b90 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7f258400c000 tx=0x7f2584006df0).stop 2022-07-21T01:58:09.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 -- 172.21.15.140:0/4145016954 shutdown_connections 2022-07-21T01:58:09.895 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f256c053a70 0x7f256c055f30 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f25940b5650 0x7f2594155b90 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.896 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f25940b1200 0x7f25940b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 --2- 172.21.15.140:0/4145016954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f259405bd90 0x7f259413d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:09.897 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 -- 172.21.15.140:0/4145016954 >> 172.21.15.140:0/4145016954 conn(0x7f25940634e0 msgr2=0x7f25940afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:09.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 -- 172.21.15.140:0/4145016954 shutdown_connections 2022-07-21T01:58:09.898 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:09.887+0000 7f25998de700 1 -- 172.21.15.140:0/4145016954 wait complete. 2022-07-21T01:58:09.899 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 74 2022-07-21T01:58:10.103 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:58:05.107197+0000', 'modified': '2022-07-21T01:58:06.110113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8727}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8727': {'gid': 8727, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/4013573612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 4013573612}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 4013573612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:58:10.104 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:58:10.105 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:58:10.220 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":8727,"want_state":"up:active","state":"up:active","rank_uptime":5.1003637460000002,"mdsmap_epoch":74,"osdmap_epoch":151,"osdmap_epoch_barrier":150,"uptime":9.4735146639999996} 2022-07-21T01:58:10.220 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 8727, "mdsmap_epoch": 74, "osdmap_epoch": 151, "osdmap_epoch_barrier": 150, "rank_uptime": 5.100363746, "state": "up:active", "uptime": 9.473514664, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:58:10.221 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:58:10.222 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2022-07-21T01:58:10.222 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@smithi191.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2022-07-21T01:58:10.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:10.223 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:58:10.232 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:58:10.249 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:58:10.250 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:58:10.261 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:58:10.262 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:58:10.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:10.264 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:58:10.270 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:58:10.270 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:58:10.273 DEBUG:teuthology.orchestra.run.smithi191:> sudo modprobe fuse 2022-07-21T01:58:10.288 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:58:10.289 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:58:10.346 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:2022-07-21T01:58:10.339+0000 7fd30a948080 -1 init, newargv = 0x562229e1b2f0 newargc=9 2022-07-21T01:58:10.346 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23523]: starting ceph client 2022-07-21T01:58:10.347 INFO:teuthology.orchestra.run.smithi191.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:58:10.348 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:58:10.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:10.349 DEBUG:teuthology.orchestra.run.smithi191:> ls /sys/fs/fuse/connections 2022-07-21T01:58:10.355 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi191.stderr:ceph-fuse[23523]: starting fuse 2022-07-21T01:58:10.356 INFO:teuthology.orchestra.run.smithi191.stdout:52 2022-07-21T01:58:10.357 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:58:10.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:10.357 DEBUG:teuthology.orchestra.run.smithi191:> sudo python3 -c ' 2022-07-21T01:58:10.358 DEBUG:teuthology.orchestra.run.smithi191:> import glob 2022-07-21T01:58:10.359 DEBUG:teuthology.orchestra.run.smithi191:> import re 2022-07-21T01:58:10.359 DEBUG:teuthology.orchestra.run.smithi191:> import os 2022-07-21T01:58:10.359 DEBUG:teuthology.orchestra.run.smithi191:> import subprocess 2022-07-21T01:58:10.360 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:58:10.360 DEBUG:teuthology.orchestra.run.smithi191:> def find_socket(client_name): 2022-07-21T01:58:10.361 DEBUG:teuthology.orchestra.run.smithi191:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2022-07-21T01:58:10.361 DEBUG:teuthology.orchestra.run.smithi191:> files = glob.glob(asok_path) 2022-07-21T01:58:10.362 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:58:10.362 DEBUG:teuthology.orchestra.run.smithi191:> # Given a non-glob path, it better be there 2022-07-21T01:58:10.362 DEBUG:teuthology.orchestra.run.smithi191:> if "*" not in asok_path: 2022-07-21T01:58:10.363 DEBUG:teuthology.orchestra.run.smithi191:> assert(len(files) == 1) 2022-07-21T01:58:10.363 DEBUG:teuthology.orchestra.run.smithi191:> return files[0] 2022-07-21T01:58:10.364 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:58:10.364 DEBUG:teuthology.orchestra.run.smithi191:> for f in files: 2022-07-21T01:58:10.364 DEBUG:teuthology.orchestra.run.smithi191:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:58:10.365 DEBUG:teuthology.orchestra.run.smithi191:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:58:10.365 DEBUG:teuthology.orchestra.run.smithi191:> return f 2022-07-21T01:58:10.366 DEBUG:teuthology.orchestra.run.smithi191:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:58:10.366 DEBUG:teuthology.orchestra.run.smithi191:> 2022-07-21T01:58:10.366 DEBUG:teuthology.orchestra.run.smithi191:> print(find_socket("client.0")) 2022-07-21T01:58:10.367 DEBUG:teuthology.orchestra.run.smithi191:> ' 2022-07-21T01:58:10.404 INFO:teuthology.orchestra.run.smithi191.stdout:/var/run/ceph/ceph-client.0.23523.asok 2022-07-21T01:58:10.409 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.23523.asok 2022-07-21T01:58:10.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:10.410 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.23523.asok status 2022-07-21T01:58:10.544 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2022-07-21T01:58:10.545 INFO:teuthology.orchestra.run.smithi191.stdout: "metadata": { 2022-07-21T01:58:10.545 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:58:10.545 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:58:10.545 INFO:teuthology.orchestra.run.smithi191.stdout: "entity_id": "0", 2022-07-21T01:58:10.546 INFO:teuthology.orchestra.run.smithi191.stdout: "hostname": "smithi191", 2022-07-21T01:58:10.546 INFO:teuthology.orchestra.run.smithi191.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2022-07-21T01:58:10.546 INFO:teuthology.orchestra.run.smithi191.stdout: "pid": "23523", 2022-07-21T01:58:10.546 INFO:teuthology.orchestra.run.smithi191.stdout: "root": "/" 2022-07-21T01:58:10.547 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:58:10.547 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_count": 0, 2022-07-21T01:58:10.547 INFO:teuthology.orchestra.run.smithi191.stdout: "dentry_pinned_count": 0, 2022-07-21T01:58:10.547 INFO:teuthology.orchestra.run.smithi191.stdout: "id": 8904, 2022-07-21T01:58:10.548 INFO:teuthology.orchestra.run.smithi191.stdout: "inst": { 2022-07-21T01:58:10.548 INFO:teuthology.orchestra.run.smithi191.stdout: "name": { 2022-07-21T01:58:10.548 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "client", 2022-07-21T01:58:10.548 INFO:teuthology.orchestra.run.smithi191.stdout: "num": 8904 2022-07-21T01:58:10.549 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:58:10.549 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:58:10.549 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:58:10.550 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:58:10.550 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 2978025797 2022-07-21T01:58:10.550 INFO:teuthology.orchestra.run.smithi191.stdout: } 2022-07-21T01:58:10.550 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:58:10.551 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": { 2022-07-21T01:58:10.551 INFO:teuthology.orchestra.run.smithi191.stdout: "type": "v1", 2022-07-21T01:58:10.551 INFO:teuthology.orchestra.run.smithi191.stdout: "addr": "172.21.15.191:0", 2022-07-21T01:58:10.551 INFO:teuthology.orchestra.run.smithi191.stdout: "nonce": 2978025797 2022-07-21T01:58:10.552 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2022-07-21T01:58:10.552 INFO:teuthology.orchestra.run.smithi191.stdout: "inst_str": "client.8904 172.21.15.191:0/2978025797", 2022-07-21T01:58:10.552 INFO:teuthology.orchestra.run.smithi191.stdout: "addr_str": "172.21.15.191:0/2978025797", 2022-07-21T01:58:10.552 INFO:teuthology.orchestra.run.smithi191.stdout: "inode_count": 1, 2022-07-21T01:58:10.553 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_epoch": 74, 2022-07-21T01:58:10.553 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch": 151, 2022-07-21T01:58:10.553 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:58:10.553 INFO:teuthology.orchestra.run.smithi191.stdout: "blacklisted": false, 2022-07-21T01:58:10.553 INFO:teuthology.orchestra.run.smithi191.stdout: "fs_name": "cephfs" 2022-07-21T01:58:10.554 INFO:teuthology.orchestra.run.smithi191.stdout:} 2022-07-21T01:58:10.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:10.556 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:58:10.556 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:58:10.563 INFO:teuthology.orchestra.run.smithi191.stdout:fuseblk 2022-07-21T01:58:10.565 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2022-07-21T01:58:10.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:10.566 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:58:10.624 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T01:58:10.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 Processor -- start 2022-07-21T01:58:10.732 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 -- start start 2022-07-21T01:58:10.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8ac00b5650 0x7f8ac00b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:10.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8ac00adff0 0x7f8ac0147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:10.735 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac0147010 con 0x7f8ac00af8e0 2022-07-21T01:58:10.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac01472b0 con 0x7f8ac00adff0 2022-07-21T01:58:10.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac0146d40 con 0x7f8ac00b5650 2022-07-21T01:58:10.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8ab7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8ac00adff0 0x7f8ac0147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:10.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8ab7fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f8ac00adff0 0x7f8ac0147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41960/0 (socket says 172.21.15.140:41960) 2022-07-21T01:58:10.736 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8ab7fff700 1 -- 172.21.15.140:0/283615042 learned_addr learned my addr 172.21.15.140:0/283615042 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:10.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abed9d700 1 --1- 172.21.15.140:0/283615042 >> v1:172.21.15.140:6789/0 conn(0x7f8ac00af8e0 0x7f8ac005bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56110/0 (socket says 172.21.15.140:56110) 2022-07-21T01:58:10.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abe59c700 1 --2- 172.21.15.140:0/283615042 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8ac00b5650 0x7f8ac00b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:10.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8ab7fff700 1 -- 172.21.15.140:0/283615042 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8ac00b5650 msgr2=0x7f8ac00b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:10.737 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8ab7fff700 1 --2- 172.21.15.140:0/283615042 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8ac00b5650 0x7f8ac00b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:10.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8ab7fff700 1 -- 172.21.15.140:0/283615042 >> v1:172.21.15.140:6789/0 conn(0x7f8ac00af8e0 legacy=0x7f8ac005bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:10.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8ab7fff700 1 -- 172.21.15.140:0/283615042 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac0147c40 con 0x7f8ac00adff0 2022-07-21T01:58:10.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8ab7fff700 1 --2- 172.21.15.140:0/283615042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8ac00adff0 0x7f8ac0147750 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f8aa8002f70 tx=0x7f8aa8006e50).ready entity=mon.0 client_cookie=230326df3bce00ac server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:10.738 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abdd9b700 1 -- 172.21.15.140:0/283615042 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8aa800d5c0 con 0x7f8ac00adff0 2022-07-21T01:58:10.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abdd9b700 1 -- 172.21.15.140:0/283615042 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa8000e90 con 0x7f8ac00adff0 2022-07-21T01:58:10.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abdd9b700 1 -- 172.21.15.140:0/283615042 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8aa800d950 con 0x7f8ac00adff0 2022-07-21T01:58:10.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 -- 172.21.15.140:0/283615042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8ac00adff0 msgr2=0x7f8ac0147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:10.739 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 --2- 172.21.15.140:0/283615042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8ac00adff0 0x7f8ac0147750 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f8aa8002f70 tx=0x7f8aa8006e50).stop 2022-07-21T01:58:10.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 -- 172.21.15.140:0/283615042 shutdown_connections 2022-07-21T01:58:10.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 --2- 172.21.15.140:0/283615042 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8ac00adff0 0x7f8ac0147750 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:10.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 --2- 172.21.15.140:0/283615042 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8ac00b5650 0x7f8ac00b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:10.740 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 -- 172.21.15.140:0/283615042 >> 172.21.15.140:0/283615042 conn(0x7f8ac00634e0 msgr2=0x7f8ac00b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:10.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 -- 172.21.15.140:0/283615042 shutdown_connections 2022-07-21T01:58:10.741 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.731+0000 7f8abf59e700 1 -- 172.21.15.140:0/283615042 wait complete. 2022-07-21T01:58:10.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abf59e700 1 Processor -- start 2022-07-21T01:58:10.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abf59e700 1 -- start start 2022-07-21T01:58:10.744 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abf59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ac00adff0 0x7f8ac010f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:10.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abf59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8ac00af8e0 0x7f8ac0066900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:10.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abf59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8ac00b5650 0x7f8ac0064480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:10.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abf59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac01119d0 con 0x7f8ac00b5650 2022-07-21T01:58:10.745 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abf59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac0066f50 con 0x7f8ac00adff0 2022-07-21T01:58:10.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abf59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00671f0 con 0x7f8ac00af8e0 2022-07-21T01:58:10.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8ac00b5650 0x7f8ac0064480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:10.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abed9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8ac00b5650 0x7f8ac0064480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41970/0 (socket says 172.21.15.140:41970) 2022-07-21T01:58:10.746 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abed9d700 1 -- 172.21.15.140:0/836058294 learned_addr learned my addr 172.21.15.140:0/836058294 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:10.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abe59c700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ac00adff0 0x7f8ac010f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:10.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8ab7fff700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8ac00af8e0 0x7f8ac0066900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:10.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abed9d700 1 -- 172.21.15.140:0/836058294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8ac00af8e0 msgr2=0x7f8ac0066900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:10.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abed9d700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8ac00af8e0 0x7f8ac0066900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:10.747 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abed9d700 1 -- 172.21.15.140:0/836058294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ac00adff0 msgr2=0x7f8ac010f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:10.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abed9d700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ac00adff0 0x7f8ac010f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:10.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abed9d700 1 -- 172.21.15.140:0/836058294 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa800b1e0 con 0x7f8ac00b5650 2022-07-21T01:58:10.748 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abe59c700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ac00adff0 0x7f8ac010f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:58:10.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abed9d700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8ac00b5650 0x7f8ac0064480 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f8ac0060b70 tx=0x7f8ab8007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:10.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8ab77fe700 1 -- 172.21.15.140:0/836058294 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab8005ae0 con 0x7f8ac00b5650 2022-07-21T01:58:10.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8ab77fe700 1 -- 172.21.15.140:0/836058294 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab8005c40 con 0x7f8ac00b5650 2022-07-21T01:58:10.749 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abf59e700 1 -- 172.21.15.140:0/836058294 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac01116c0 con 0x7f8ac00b5650 2022-07-21T01:58:10.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8ab77fe700 1 -- 172.21.15.140:0/836058294 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ab80070d0 con 0x7f8ac00b5650 2022-07-21T01:58:10.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.735+0000 7f8abf59e700 1 -- 172.21.15.140:0/836058294 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac00675d0 con 0x7f8ac00b5650 2022-07-21T01:58:10.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.739+0000 7f8abf59e700 1 -- 172.21.15.140:0/836058294 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac00673b0 con 0x7f8ac00b5650 2022-07-21T01:58:10.750 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.739+0000 7f8ab77fe700 1 -- 172.21.15.140:0/836058294 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8ab801a060 con 0x7f8ac00b5650 2022-07-21T01:58:10.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.739+0000 7f8ab77fe700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8aa4053b00 0x7f8aa4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:10.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.739+0000 7f8ab77fe700 1 -- 172.21.15.140:0/836058294 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8ab806b9c0 con 0x7f8ac00b5650 2022-07-21T01:58:10.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.739+0000 7f8abe59c700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8aa4053b00 0x7f8aa4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:10.751 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.739+0000 7f8abe59c700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8aa4053b00 0x7f8aa4055fc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f8ab000c000 tx=0x7f8ab0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:10.752 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.747+0000 7f8ab77fe700 1 -- 172.21.15.140:0/836058294 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8ab80428e0 con 0x7f8ac00b5650 2022-07-21T01:58:10.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.871+0000 7f8abf59e700 1 -- 172.21.15.140:0/836058294 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ac005c4a0 con 0x7f8ac00b5650 2022-07-21T01:58:10.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.871+0000 7f8ab77fe700 1 -- 172.21.15.140:0/836058294 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16148 (secure 0 0 0) 0x7f8ab8018070 con 0x7f8ac00b5650 2022-07-21T01:58:10.875 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:10.875 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":151,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:58:06.110718+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2022-07-21T01:58:03.485327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2022-07-21T01:58:04.503066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T01:58:10.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 -- 172.21.15.140:0/836058294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8aa4053b00 msgr2=0x7f8aa4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:10.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8aa4053b00 0x7f8aa4055fc0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f8ab000c000 tx=0x7f8ab0008040).stop 2022-07-21T01:58:10.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 -- 172.21.15.140:0/836058294 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8ac00b5650 msgr2=0x7f8ac0064480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:10.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8ac00b5650 0x7f8ac0064480 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7f8ac0060b70 tx=0x7f8ab8007f40).stop 2022-07-21T01:58:10.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 -- 172.21.15.140:0/836058294 shutdown_connections 2022-07-21T01:58:10.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8aa4053b00 0x7f8aa4055fc0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:10.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f8ac00b5650 0x7f8ac0064480 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:10.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8ac00af8e0 0x7f8ac0066900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:10.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 --2- 172.21.15.140:0/836058294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ac00adff0 0x7f8ac010f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:10.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 -- 172.21.15.140:0/836058294 >> 172.21.15.140:0/836058294 conn(0x7f8ac00634e0 msgr2=0x7f8ac00afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:10.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 -- 172.21.15.140:0/836058294 shutdown_connections 2022-07-21T01:58:10.880 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.875+0000 7f8abf59e700 1 -- 172.21.15.140:0/836058294 wait complete. 2022-07-21T01:58:10.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2022-07-21T01:48:17.946023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-07-21T01:48:26.620359+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2022-07-21T01:58:03.485327+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2022-07-21T01:58:04.503066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2022-07-21T01:58:10.894 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-07-21T01:58:10.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.995+0000 7f85d759e700 1 Processor -- start 2022-07-21T01:58:10.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.995+0000 7f85d759e700 1 -- start start 2022-07-21T01:58:11.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.995+0000 7f85d759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85d80b59f0 0x7f85d81473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.995+0000 7f85d759e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f85d80b1400 0x7f85d8147890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.995+0000 7f85d759e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d805cc70 con 0x7f85d80af940 2022-07-21T01:58:11.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.995+0000 7f85d759e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d80b1070 con 0x7f85d80b1400 2022-07-21T01:58:11.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.995+0000 7f85d759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d80b5dd0 con 0x7f85d80b59f0 2022-07-21T01:58:11.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d6d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f85d80b1400 0x7f85d8147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d6d9d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f85d80b1400 0x7f85d8147890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41978/0 (socket says 172.21.15.140:41978) 2022-07-21T01:58:11.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d6d9d700 1 -- 172.21.15.140:0/2937256988 learned_addr learned my addr 172.21.15.140:0/2937256988 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:11.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d659c700 1 --2- 172.21.15.140:0/2937256988 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85d80b59f0 0x7f85d81473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d6d9d700 1 -- 172.21.15.140:0/2937256988 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85d80b59f0 msgr2=0x7f85d81473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:11.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d6d9d700 1 --2- 172.21.15.140:0/2937256988 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85d80b59f0 0x7f85d81473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d6d9d700 1 -- 172.21.15.140:0/2937256988 >> v1:172.21.15.152:6789/0 conn(0x7f85d80af940 legacy=0x7f85d814a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d6d9d700 1 -- 172.21.15.140:0/2937256988 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d805c760 con 0x7f85d80b1400 2022-07-21T01:58:11.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d6d9d700 1 --2- 172.21.15.140:0/2937256988 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85d80b1400 0x7f85d8147890 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f85d00032f0 tx=0x7f85d0009290).ready entity=mon.0 client_cookie=8091c61dc286ac73 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d559a700 1 -- 172.21.15.140:0/2937256988 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85d000b8a0 con 0x7f85d80b1400 2022-07-21T01:58:11.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d559a700 1 -- 172.21.15.140:0/2937256988 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d000ba00 con 0x7f85d80b1400 2022-07-21T01:58:11.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d559a700 1 -- 172.21.15.140:0/2937256988 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85d000bb70 con 0x7f85d80b1400 2022-07-21T01:58:11.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 -- 172.21.15.140:0/2937256988 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85d80b1400 msgr2=0x7f85d8147890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 --2- 172.21.15.140:0/2937256988 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85d80b1400 0x7f85d8147890 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7f85d00032f0 tx=0x7f85d0009290).stop 2022-07-21T01:58:11.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 -- 172.21.15.140:0/2937256988 shutdown_connections 2022-07-21T01:58:11.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 --2- 172.21.15.140:0/2937256988 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85d80b1400 0x7f85d8147890 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 --2- 172.21.15.140:0/2937256988 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85d80b59f0 0x7f85d81473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 -- 172.21.15.140:0/2937256988 >> 172.21.15.140:0/2937256988 conn(0x7f85d8063480 msgr2=0x7f85d80b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:11.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 -- 172.21.15.140:0/2937256988 shutdown_connections 2022-07-21T01:58:11.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 -- 172.21.15.140:0/2937256988 wait complete. 2022-07-21T01:58:11.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 Processor -- start 2022-07-21T01:58:11.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 -- start start 2022-07-21T01:58:11.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85d80af940 0x7f85d81117a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85d80b1400 0x7f85d813e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85d80b59f0 0x7f85d813e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d81109a0 con 0x7f85d80b59f0 2022-07-21T01:58:11.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d810f9d0 con 0x7f85d80b1400 2022-07-21T01:58:11.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d759e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d8112090 con 0x7f85d80af940 2022-07-21T01:58:11.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d6d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85d80b59f0 0x7f85d813e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d6d9d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85d80b59f0 0x7f85d813e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41984/0 (socket says 172.21.15.140:41984) 2022-07-21T01:58:11.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d6d9d700 1 -- 172.21.15.140:0/2692345168 learned_addr learned my addr 172.21.15.140:0/2692345168 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:11.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:10.999+0000 7f85d5d9b700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85d80b1400 0x7f85d813e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d659c700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85d80af940 0x7f85d81117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d6d9d700 1 -- 172.21.15.140:0/2692345168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85d80af940 msgr2=0x7f85d81117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d6d9d700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85d80af940 0x7f85d81117a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d6d9d700 1 -- 172.21.15.140:0/2692345168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85d80b1400 msgr2=0x7f85d813e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d6d9d700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85d80b1400 0x7f85d813e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d6d9d700 1 -- 172.21.15.140:0/2692345168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d0009530 con 0x7f85d80b59f0 2022-07-21T01:58:11.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d659c700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85d80af940 0x7f85d81117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T01:58:11.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d6d9d700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85d80b59f0 0x7f85d813e700 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f85d00032f0 tx=0x7f85d0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85cf7fe700 1 -- 172.21.15.140:0/2692345168 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85d000b8a0 con 0x7f85d80b59f0 2022-07-21T01:58:11.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85cf7fe700 1 -- 172.21.15.140:0/2692345168 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d0021440 con 0x7f85d80b59f0 2022-07-21T01:58:11.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d759e700 1 -- 172.21.15.140:0/2692345168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d8111df0 con 0x7f85d80b59f0 2022-07-21T01:58:11.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85cf7fe700 1 -- 172.21.15.140:0/2692345168 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85d00217a0 con 0x7f85d80b59f0 2022-07-21T01:58:11.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d759e700 1 -- 172.21.15.140:0/2692345168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d810eb20 con 0x7f85d80b59f0 2022-07-21T01:58:11.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d759e700 1 -- 172.21.15.140:0/2692345168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d810fb40 con 0x7f85d80b59f0 2022-07-21T01:58:11.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85cf7fe700 1 -- 172.21.15.140:0/2692345168 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f85d0027020 con 0x7f85d80b59f0 2022-07-21T01:58:11.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85cf7fe700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85bc0539e0 0x7f85bc055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85cf7fe700 1 -- 172.21.15.140:0/2692345168 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f85d006d540 con 0x7f85d80b59f0 2022-07-21T01:58:11.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d659c700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85bc0539e0 0x7f85bc055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.003+0000 7f85d659c700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85bc0539e0 0x7f85bc055ea0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f85c800b000 tx=0x7f85c8008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.011+0000 7f85cf7fe700 1 -- 172.21.15.140:0/2692345168 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f85d0044460 con 0x7f85d80b59f0 2022-07-21T01:58:11.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 -- 172.21.15.140:0/2692345168 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f85d805c9d0 con 0x7f85d80b59f0 2022-07-21T01:58:11.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85cf7fe700 1 -- 172.21.15.140:0/2692345168 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v151) v1 ==== 110+0+10 (secure 0 0 0) 0x7f85d0015020 con 0x7f85d80b59f0 2022-07-21T01:58:11.134 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 1 2022-07-21T01:58:11.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 -- 172.21.15.140:0/2692345168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85bc0539e0 msgr2=0x7f85bc055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85bc0539e0 0x7f85bc055ea0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f85c800b000 tx=0x7f85c8008040).stop 2022-07-21T01:58:11.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 -- 172.21.15.140:0/2692345168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85d80b59f0 msgr2=0x7f85d813e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85d80b59f0 0x7f85d813e700 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7f85d00032f0 tx=0x7f85d0016000).stop 2022-07-21T01:58:11.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 -- 172.21.15.140:0/2692345168 shutdown_connections 2022-07-21T01:58:11.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f85bc0539e0 0x7f85bc055ea0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f85d80b59f0 0x7f85d813e700 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85d80b1400 0x7f85d813e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 --2- 172.21.15.140:0/2692345168 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f85d80af940 0x7f85d81117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 -- 172.21.15.140:0/2692345168 >> 172.21.15.140:0/2692345168 conn(0x7f85d8063480 msgr2=0x7f85d813f010 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:11.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 -- 172.21.15.140:0/2692345168 shutdown_connections 2022-07-21T01:58:11.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.131+0000 7f85d759e700 1 -- 172.21.15.140:0/2692345168 wait complete. 2022-07-21T01:58:11.147 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-07-21T01:58:11.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30f2b1b700 1 Processor -- start 2022-07-21T01:58:11.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30f2b1b700 1 -- start start 2022-07-21T01:58:11.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30f2b1b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f30ec05d0e0 0x7f30ec146920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30f2b1b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ec05d4c0 con 0x7f30ec09b6d0 2022-07-21T01:58:11.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30f2b1b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ec1461e0 con 0x7f30ec0b5960 2022-07-21T01:58:11.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30f2b1b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ec146480 con 0x7f30ec05d0e0 2022-07-21T01:58:11.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30f10b8700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f30ec09b6d0 0x7f30ec09bb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56138/0 (socket says 172.21.15.140:56138) 2022-07-21T01:58:11.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30f10b8700 1 -- 172.21.15.140:0/2505527551 learned_addr learned my addr 172.21.15.140:0/2505527551 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:11.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30f08b7700 1 --2- 172.21.15.140:0/2505527551 >> v2:172.21.15.152:3300/0 conn(0x7f30ec05d0e0 0x7f30ec146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30eb7fe700 1 -- 172.21.15.140:0/2505527551 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393869778 0 0) 0x7f30ec05d4c0 con 0x7f30ec09b6d0 2022-07-21T01:58:11.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30eb7fe700 1 -- 172.21.15.140:0/2505527551 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30d8003260 con 0x7f30ec09b6d0 2022-07-21T01:58:11.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30eb7fe700 1 -- 172.21.15.140:0/2505527551 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70881084 0 0) 0x7f30ec1461e0 con 0x7f30ec0b5960 2022-07-21T01:58:11.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30eb7fe700 1 -- 172.21.15.140:0/2505527551 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30ec05d4c0 con 0x7f30ec0b5960 2022-07-21T01:58:11.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.255+0000 7f30f08b7700 1 -- 172.21.15.140:0/2505527551 >> v1:172.21.15.152:6789/0 conn(0x7f30ec0b5960 legacy=0x7f30ec142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f08b7700 1 -- 172.21.15.140:0/2505527551 >> v1:172.21.15.140:6789/0 conn(0x7f30ec09b6d0 legacy=0x7f30ec09bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f08b7700 1 -- 172.21.15.140:0/2505527551 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec143280 con 0x7f30ec05d0e0 2022-07-21T01:58:11.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f08b7700 1 --2- 172.21.15.140:0/2505527551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ec05d0e0 0x7f30ec146920 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f30dc002f70 tx=0x7f30dc006df0).ready entity=mon.1 client_cookie=74ca48bac5b27719 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30eb7fe700 1 -- 172.21.15.140:0/2505527551 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30dc00d540 con 0x7f30ec05d0e0 2022-07-21T01:58:11.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30eb7fe700 1 -- 172.21.15.140:0/2505527551 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc000e90 con 0x7f30ec05d0e0 2022-07-21T01:58:11.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30eb7fe700 1 -- 172.21.15.140:0/2505527551 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30dc00d890 con 0x7f30ec05d0e0 2022-07-21T01:58:11.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f2b1b700 1 -- 172.21.15.140:0/2505527551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ec05d0e0 msgr2=0x7f30ec146920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f2b1b700 1 --2- 172.21.15.140:0/2505527551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ec05d0e0 0x7f30ec146920 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f30dc002f70 tx=0x7f30dc006df0).stop 2022-07-21T01:58:11.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f2b1b700 1 -- 172.21.15.140:0/2505527551 shutdown_connections 2022-07-21T01:58:11.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f2b1b700 1 --2- 172.21.15.140:0/2505527551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ec05d0e0 0x7f30ec146920 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.292 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f2b1b700 1 -- 172.21.15.140:0/2505527551 >> 172.21.15.140:0/2505527551 conn(0x7f30ec0b6a20 msgr2=0x7f30ec066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:11.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f2b1b700 1 -- 172.21.15.140:0/2505527551 shutdown_connections 2022-07-21T01:58:11.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f2b1b700 1 -- 172.21.15.140:0/2505527551 wait complete. 2022-07-21T01:58:11.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f2b1b700 1 Processor -- start 2022-07-21T01:58:11.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.259+0000 7f30f2b1b700 1 -- start start 2022-07-21T01:58:11.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f2b1b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30ec05d0e0 0x7f30ec09c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f2b1b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30ec09b6d0 0x7f30ec145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f2b1b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ec0b5960 0x7f30ec09cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f2b1b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec142e10 con 0x7f30ec05d0e0 2022-07-21T01:58:11.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f2b1b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0b0470 con 0x7f30ec0b5960 2022-07-21T01:58:11.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f2b1b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0b0710 con 0x7f30ec09b6d0 2022-07-21T01:58:11.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f08b7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30ec05d0e0 0x7f30ec09c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f08b7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30ec05d0e0 0x7f30ec09c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:41994/0 (socket says 172.21.15.140:41994) 2022-07-21T01:58:11.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f08b7700 1 -- 172.21.15.140:0/4001649037 learned_addr learned my addr 172.21.15.140:0/4001649037 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:11.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f10b8700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ec0b5960 0x7f30ec09cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30ebfff700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30ec09b6d0 0x7f30ec145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f08b7700 1 -- 172.21.15.140:0/4001649037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30ec09b6d0 msgr2=0x7f30ec145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f08b7700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30ec09b6d0 0x7f30ec145310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f08b7700 1 -- 172.21.15.140:0/4001649037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ec0b5960 msgr2=0x7f30ec09cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f08b7700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ec0b5960 0x7f30ec09cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f08b7700 1 -- 172.21.15.140:0/4001649037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc00b180 con 0x7f30ec05d0e0 2022-07-21T01:58:11.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f08b7700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30ec05d0e0 0x7f30ec09c7c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7f30dc002f70 tx=0x7f30dc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30e9ffb700 1 -- 172.21.15.140:0/4001649037 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30dc016070 con 0x7f30ec05d0e0 2022-07-21T01:58:11.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30e9ffb700 1 -- 172.21.15.140:0/4001649037 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc0138d0 con 0x7f30ec05d0e0 2022-07-21T01:58:11.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f2b1b700 1 -- 172.21.15.140:0/4001649037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ec09f290 con 0x7f30ec05d0e0 2022-07-21T01:58:11.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30e9ffb700 1 -- 172.21.15.140:0/4001649037 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f30dc013c30 con 0x7f30ec05d0e0 2022-07-21T01:58:11.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.263+0000 7f30f2b1b700 1 -- 172.21.15.140:0/4001649037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ec09ef20 con 0x7f30ec05d0e0 2022-07-21T01:58:11.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.267+0000 7f30f2b1b700 1 -- 172.21.15.140:0/4001649037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ec0b08d0 con 0x7f30ec05d0e0 2022-07-21T01:58:11.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.267+0000 7f30e9ffb700 1 -- 172.21.15.140:0/4001649037 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f30dc025030 con 0x7f30ec05d0e0 2022-07-21T01:58:11.305 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.267+0000 7f30e9ffb700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f30cc053a70 0x7f30cc055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.267+0000 7f30e9ffb700 1 -- 172.21.15.140:0/4001649037 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f30dc06ce60 con 0x7f30ec05d0e0 2022-07-21T01:58:11.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.267+0000 7f30ebfff700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f30cc053a70 0x7f30cc055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.306 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.267+0000 7f30ebfff700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f30cc053a70 0x7f30cc055f30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f30e000b000 tx=0x7f30e0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.307 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.271+0000 7f30e9ffb700 1 -- 172.21.15.140:0/4001649037 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f30dc043d80 con 0x7f30ec05d0e0 2022-07-21T01:58:11.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 -- 172.21.15.140:0/4001649037 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f30ec0574e0 con 0x7f30ec05d0e0 2022-07-21T01:58:11.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30e9ffb700 1 -- 172.21.15.140:0/4001649037 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7f30dc023070 con 0x7f30ec05d0e0 2022-07-21T01:58:11.402 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:58:11.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 -- 172.21.15.140:0/4001649037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f30cc053a70 msgr2=0x7f30cc055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f30cc053a70 0x7f30cc055f30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f30e000b000 tx=0x7f30e0009040).stop 2022-07-21T01:58:11.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 -- 172.21.15.140:0/4001649037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30ec05d0e0 msgr2=0x7f30ec09c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30ec05d0e0 0x7f30ec09c7c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7f30dc002f70 tx=0x7f30dc014000).stop 2022-07-21T01:58:11.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 -- 172.21.15.140:0/4001649037 shutdown_connections 2022-07-21T01:58:11.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f30cc053a70 0x7f30cc055f30 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ec0b5960 0x7f30ec09cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f30ec09b6d0 0x7f30ec145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 --2- 172.21.15.140:0/4001649037 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f30ec05d0e0 0x7f30ec09c7c0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 -- 172.21.15.140:0/4001649037 >> 172.21.15.140:0/4001649037 conn(0x7f30ec0b6a20 msgr2=0x7f30ec13f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:11.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 -- 172.21.15.140:0/4001649037 shutdown_connections 2022-07-21T01:58:11.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.399+0000 7f30f2b1b700 1 -- 172.21.15.140:0/4001649037 wait complete. 2022-07-21T01:58:11.414 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2022-07-21T01:58:11.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.523+0000 7efef4eeb700 1 Processor -- start 2022-07-21T01:58:11.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.523+0000 7efef4eeb700 1 -- start start 2022-07-21T01:58:11.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.523+0000 7efef4eeb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef00b3cf0 0x7efef0142a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.523+0000 7efef4eeb700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7efef0142f40 0x7efef01419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.523+0000 7efef4eeb700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef00ba180 con 0x7efef00b9da0 2022-07-21T01:58:11.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.523+0000 7efef4eeb700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef00ad5b0 con 0x7efef0142f40 2022-07-21T01:58:11.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.523+0000 7efef4eeb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef01461b0 con 0x7efef00b3cf0 2022-07-21T01:58:11.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.523+0000 7efeef7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7efef00b9da0 0x7efef01468f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56148/0 (socket says 172.21.15.140:56148) 2022-07-21T01:58:11.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.523+0000 7efeef7fe700 1 -- 172.21.15.140:0/2990533409 learned_addr learned my addr 172.21.15.140:0/2990533409 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:11.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.523+0000 7efeeffff700 1 --2- 172.21.15.140:0/2990533409 >> v2:172.21.15.140:3300/0 conn(0x7efef0142f40 0x7efef01419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efeeeffd700 1 --2- 172.21.15.140:0/2990533409 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef00b3cf0 0x7efef0142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efeeffff700 1 -- 172.21.15.140:0/2990533409 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef00b3cf0 msgr2=0x7efef0142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:11.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efeeffff700 1 --2- 172.21.15.140:0/2990533409 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef00b3cf0 0x7efef0142a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efeeffff700 1 -- 172.21.15.140:0/2990533409 >> v1:172.21.15.140:6789/0 conn(0x7efef00b9da0 legacy=0x7efef01468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efeee7fc700 1 -- 172.21.15.140:0/2990533409 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742274264 0 0) 0x7efef00ba180 con 0x7efef00b9da0 2022-07-21T01:58:11.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efeeffff700 1 -- 172.21.15.140:0/2990533409 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef0146450 con 0x7efef0142f40 2022-07-21T01:58:11.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efeeffff700 1 --2- 172.21.15.140:0/2990533409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef0142f40 0x7efef01419a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7efee80032f0 tx=0x7efee80091d0).ready entity=mon.0 client_cookie=20fb9cee4a09cbed server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efeee7fc700 1 -- 172.21.15.140:0/2990533409 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee800b6c0 con 0x7efef0142f40 2022-07-21T01:58:11.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efeee7fc700 1 -- 172.21.15.140:0/2990533409 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee800b820 con 0x7efef0142f40 2022-07-21T01:58:11.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efeee7fc700 1 -- 172.21.15.140:0/2990533409 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee800b990 con 0x7efef0142f40 2022-07-21T01:58:11.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efef4eeb700 1 -- 172.21.15.140:0/2990533409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef0142f40 msgr2=0x7efef01419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efef4eeb700 1 --2- 172.21.15.140:0/2990533409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef0142f40 0x7efef01419a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7efee80032f0 tx=0x7efee80091d0).stop 2022-07-21T01:58:11.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efef4eeb700 1 -- 172.21.15.140:0/2990533409 shutdown_connections 2022-07-21T01:58:11.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efef4eeb700 1 --2- 172.21.15.140:0/2990533409 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef0142f40 0x7efef01419a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efef4eeb700 1 --2- 172.21.15.140:0/2990533409 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef00b3cf0 0x7efef0142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efef4eeb700 1 -- 172.21.15.140:0/2990533409 >> 172.21.15.140:0/2990533409 conn(0x7efef00678c0 msgr2=0x7efef00b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:11.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efef4eeb700 1 -- 172.21.15.140:0/2990533409 shutdown_connections 2022-07-21T01:58:11.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efef4eeb700 1 -- 172.21.15.140:0/2990533409 wait complete. 2022-07-21T01:58:11.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efef4eeb700 1 Processor -- start 2022-07-21T01:58:11.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.527+0000 7efef4eeb700 1 -- start start 2022-07-21T01:58:11.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efef4eeb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef00b3cf0 0x7efef013e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efef4eeb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef00b9da0 0x7efef013d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efef4eeb700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef0142f40 0x7efef00b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efef4eeb700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0148000 con 0x7efef0142f40 2022-07-21T01:58:11.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efef4eeb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef01482a0 con 0x7efef00b3cf0 2022-07-21T01:58:11.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efef4eeb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef0148540 con 0x7efef00b9da0 2022-07-21T01:58:11.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeeffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef0142f40 0x7efef00b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeeffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef0142f40 0x7efef00b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42012/0 (socket says 172.21.15.140:42012) 2022-07-21T01:58:11.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeeffff700 1 -- 172.21.15.140:0/1806650394 learned_addr learned my addr 172.21.15.140:0/1806650394 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:11.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeef7fe700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef00b3cf0 0x7efef013e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeeeffd700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef00b9da0 0x7efef013d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeeffff700 1 -- 172.21.15.140:0/1806650394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef00b9da0 msgr2=0x7efef013d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeeffff700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef00b9da0 0x7efef013d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeeffff700 1 -- 172.21.15.140:0/1806650394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef00b3cf0 msgr2=0x7efef013e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeeffff700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef00b3cf0 0x7efef013e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeeffff700 1 -- 172.21.15.140:0/1806650394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee8009470 con 0x7efef0142f40 2022-07-21T01:58:11.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeeffff700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef0142f40 0x7efef00b0b70 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7efee80032f0 tx=0x7efee8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeecff9700 1 -- 172.21.15.140:0/1806650394 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee8011e30 con 0x7efef0142f40 2022-07-21T01:58:11.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeecff9700 1 -- 172.21.15.140:0/1806650394 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee8024a00 con 0x7efef0142f40 2022-07-21T01:58:11.545 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efef4eeb700 1 -- 172.21.15.140:0/1806650394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef01487e0 con 0x7efef0142f40 2022-07-21T01:58:11.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efeecff9700 1 -- 172.21.15.140:0/1806650394 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efee8024d60 con 0x7efef0142f40 2022-07-21T01:58:11.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efef4eeb700 1 -- 172.21.15.140:0/1806650394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef01471b0 con 0x7efef0142f40 2022-07-21T01:58:11.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.531+0000 7efef4eeb700 1 -- 172.21.15.140:0/1806650394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef0148970 con 0x7efef0142f40 2022-07-21T01:58:11.546 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.535+0000 7efeecff9700 1 -- 172.21.15.140:0/1806650394 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7efee8034070 con 0x7efef0142f40 2022-07-21T01:58:11.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.535+0000 7efeecff9700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed4053a70 0x7efed4055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.535+0000 7efeecff9700 1 -- 172.21.15.140:0/1806650394 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efee8075b30 con 0x7efef0142f40 2022-07-21T01:58:11.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.535+0000 7efeef7fe700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed4053a70 0x7efed4055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.547 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.535+0000 7efeef7fe700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed4053a70 0x7efed4055f30 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7efed800b000 tx=0x7efed8009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.548 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.539+0000 7efeecff9700 1 -- 172.21.15.140:0/1806650394 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7efee804c950 con 0x7efef0142f40 2022-07-21T01:58:11.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.663+0000 7efef4eeb700 1 -- 172.21.15.140:0/1806650394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efef00667f0 con 0x7efef0142f40 2022-07-21T01:58:11.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.663+0000 7efeecff9700 1 -- 172.21.15.140:0/1806650394 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+10 (secure 0 0 0) 0x7efee80150a0 con 0x7efef0142f40 2022-07-21T01:58:11.668 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:58:11.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 -- 172.21.15.140:0/1806650394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed4053a70 msgr2=0x7efed4055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed4053a70 0x7efed4055f30 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7efed800b000 tx=0x7efed8009040).stop 2022-07-21T01:58:11.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 -- 172.21.15.140:0/1806650394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef0142f40 msgr2=0x7efef00b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef0142f40 0x7efef00b0b70 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7efee80032f0 tx=0x7efee8016000).stop 2022-07-21T01:58:11.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 -- 172.21.15.140:0/1806650394 shutdown_connections 2022-07-21T01:58:11.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7efed4053a70 0x7efed4055f30 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7efef0142f40 0x7efef00b0b70 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efef00b9da0 0x7efef013d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 --2- 172.21.15.140:0/1806650394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efef00b3cf0 0x7efef013e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 -- 172.21.15.140:0/1806650394 >> 172.21.15.140:0/1806650394 conn(0x7efef00678c0 msgr2=0x7efef00b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:11.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 -- 172.21.15.140:0/1806650394 shutdown_connections 2022-07-21T01:58:11.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.667+0000 7efef4eeb700 1 -- 172.21.15.140:0/1806650394 wait complete. 2022-07-21T01:58:11.680 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2022-07-21T01:58:11.785 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f04381ef700 1 Processor -- start 2022-07-21T01:58:11.786 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f04381ef700 1 -- start start 2022-07-21T01:58:11.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f04381ef700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04300ac680 0x7f0430140860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f04381ef700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043013fe80 con 0x7f04300b2210 2022-07-21T01:58:11.791 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f04381ef700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0430140120 con 0x7f04300967e0 2022-07-21T01:58:11.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f04381ef700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04301403c0 con 0x7f04300ac680 2022-07-21T01:58:11.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f0435f8b700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f04300b2210 0x7f04300ab0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56162/0 (socket says 172.21.15.140:56162) 2022-07-21T01:58:11.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f0435f8b700 1 -- 172.21.15.140:0/4223693950 learned_addr learned my addr 172.21.15.140:0/4223693950 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:11.792 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f043678c700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04300ac680 0x7f0430140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017269166 0 0) 0x7f043013fe80 con 0x7f04300b2210 2022-07-21T01:58:11.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0418003260 con 0x7f04300b2210 2022-07-21T01:58:11.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275861456 0 0) 0x7f0430140120 con 0x7f04300967e0 2022-07-21T01:58:11.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f043013fe80 con 0x7f04300967e0 2022-07-21T01:58:11.793 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.783+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3536144565 0 0) 0x7f0418003260 con 0x7f04300b2210 2022-07-21T01:58:11.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04300ac680 msgr2=0x7f0430140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:11.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04300ac680 0x7f0430140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 >> v1:172.21.15.152:6789/0 conn(0x7f04300967e0 legacy=0x7f043013efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.794 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04300b25f0 con 0x7f04300b2210 2022-07-21T01:58:11.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f042c001660 con 0x7f04300b2210 2022-07-21T01:58:11.795 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 >> v1:172.21.15.140:6789/0 conn(0x7f04300b2210 legacy=0x7f04300ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04300ac680 0x7f0418003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0418003e70 0x7f0418004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.796 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04180049f0 0x7f0418004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04301403c0 con 0x7f0418003e70 2022-07-21T01:58:11.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041800b830 con 0x7f04180049f0 2022-07-21T01:58:11.797 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041800b9f0 con 0x7f04300ac680 2022-07-21T01:58:11.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0435f8b700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04300ac680 0x7f0418003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f043678c700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04180049f0 0x7f0418004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f043578a700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0418003e70 0x7f0418004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.798 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0435f8b700 1 -- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04180049f0 msgr2=0x7f0418004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:11.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0435f8b700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04180049f0 0x7f0418004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0435f8b700 1 -- 172.21.15.140:0/4223693950 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0418003e70 msgr2=0x7f0418004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:11.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0435f8b700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0418003e70 0x7f0418004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.799 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0435f8b700 1 -- 172.21.15.140:0/4223693950 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0418003260 con 0x7f04300ac680 2022-07-21T01:58:11.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f043578a700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f0418003e70 0x7f0418004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:58:11.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0435f8b700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04300ac680 0x7f0418003910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f042c001430 tx=0x7f042c001d00).ready entity=mon.2 client_cookie=d0773cd4469bb940 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f042c007e70 con 0x7f04300ac680 2022-07-21T01:58:11.800 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c0070d0 con 0x7f04300ac680 2022-07-21T01:58:11.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f0434f89700 1 -- 172.21.15.140:0/4223693950 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f042c007430 con 0x7f04300ac680 2022-07-21T01:58:11.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f04381ef700 1 -- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04300ac680 msgr2=0x7f0418003910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f04381ef700 1 --2- 172.21.15.140:0/4223693950 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04300ac680 0x7f0418003910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f042c001430 tx=0x7f042c001d00).stop 2022-07-21T01:58:11.801 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f043678c700 1 -- 172.21.15.140:0/4223693950 reap_dead start 2022-07-21T01:58:11.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f04381ef700 1 -- 172.21.15.140:0/4223693950 shutdown_connections 2022-07-21T01:58:11.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f04381ef700 1 -- 172.21.15.140:0/4223693950 >> 172.21.15.140:0/4223693950 conn(0x7f04300b36c0 msgr2=0x7f04300b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:11.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f04381ef700 1 -- 172.21.15.140:0/4223693950 shutdown_connections 2022-07-21T01:58:11.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f04381ef700 1 -- 172.21.15.140:0/4223693950 wait complete. 2022-07-21T01:58:11.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f04381ef700 1 Processor -- start 2022-07-21T01:58:11.804 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.787+0000 7f04381ef700 1 -- start start 2022-07-21T01:58:11.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f04381ef700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04300ac680 0x7f0430151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.809 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f04381ef700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0430094da0 0x7f04301468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f04381ef700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0430146db0 0x7f0430155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f04381ef700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04300aed20 con 0x7f04300ac680 2022-07-21T01:58:11.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f04381ef700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0430141fe0 con 0x7f0430094da0 2022-07-21T01:58:11.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f04381ef700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0430155600 con 0x7f0430146db0 2022-07-21T01:58:11.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0435f8b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04300ac680 0x7f0430151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0435f8b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04300ac680 0x7f0430151e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42028/0 (socket says 172.21.15.140:42028) 2022-07-21T01:58:11.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0435f8b700 1 -- 172.21.15.140:0/4062431322 learned_addr learned my addr 172.21.15.140:0/4062431322 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:11.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f043678c700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0430146db0 0x7f0430155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f043578a700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0430094da0 0x7f04301468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0435f8b700 1 -- 172.21.15.140:0/4062431322 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0430146db0 msgr2=0x7f0430155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.812 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0435f8b700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0430146db0 0x7f0430155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0435f8b700 1 -- 172.21.15.140:0/4062431322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0430094da0 msgr2=0x7f04301468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0435f8b700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0430094da0 0x7f04301468c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0435f8b700 1 -- 172.21.15.140:0/4062431322 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0430152350 con 0x7f04300ac680 2022-07-21T01:58:11.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0435f8b700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04300ac680 0x7f0430151e60 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7f042c001430 tx=0x7f042c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.813 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0422ffd700 1 -- 172.21.15.140:0/4062431322 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f042c00ce40 con 0x7f04300ac680 2022-07-21T01:58:11.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0422ffd700 1 -- 172.21.15.140:0/4062431322 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c004520 con 0x7f04300ac680 2022-07-21T01:58:11.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f04381ef700 1 -- 172.21.15.140:0/4062431322 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0430141b90 con 0x7f04300ac680 2022-07-21T01:58:11.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0422ffd700 1 -- 172.21.15.140:0/4062431322 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f042c004880 con 0x7f04300ac680 2022-07-21T01:58:11.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f04381ef700 1 -- 172.21.15.140:0/4062431322 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04300949c0 con 0x7f04300ac680 2022-07-21T01:58:11.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f04381ef700 1 -- 172.21.15.140:0/4062431322 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04301421a0 con 0x7f04300ac680 2022-07-21T01:58:11.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0422ffd700 1 -- 172.21.15.140:0/4062431322 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f042c010070 con 0x7f04300ac680 2022-07-21T01:58:11.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.791+0000 7f0422ffd700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0410053a70 0x7f0410055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:11.815 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.795+0000 7f0422ffd700 1 -- 172.21.15.140:0/4062431322 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f042c06ccb0 con 0x7f04300ac680 2022-07-21T01:58:11.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.795+0000 7f043578a700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0410053a70 0x7f0410055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:11.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.795+0000 7f043578a700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0410053a70 0x7f0410055f30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f0424003000 tx=0x7f0424009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:11.816 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.799+0000 7f0422ffd700 1 -- 172.21.15.140:0/4062431322 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f042c043bd0 con 0x7f04300ac680 2022-07-21T01:58:11.920 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 -- 172.21.15.140:0/4062431322 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04300b2840 con 0x7f04300ac680 2022-07-21T01:58:11.921 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f0422ffd700 1 -- 172.21.15.140:0/4062431322 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+10 (secure 0 0 0) 0x7f042c0471f0 con 0x7f04300ac680 2022-07-21T01:58:11.922 INFO:teuthology.orchestra.run.smithi140.stdout:pg_num: 8 2022-07-21T01:58:11.924 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 -- 172.21.15.140:0/4062431322 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0410053a70 msgr2=0x7f0410055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0410053a70 0x7f0410055f30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f0424003000 tx=0x7f0424009040).stop 2022-07-21T01:58:11.925 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 -- 172.21.15.140:0/4062431322 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04300ac680 msgr2=0x7f0430151e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:11.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04300ac680 0x7f0430151e60 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7f042c001430 tx=0x7f042c015000).stop 2022-07-21T01:58:11.926 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 -- 172.21.15.140:0/4062431322 shutdown_connections 2022-07-21T01:58:11.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f0410053a70 0x7f0410055f30 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.927 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0430146db0 0x7f0430155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0430094da0 0x7f04301468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 --2- 172.21.15.140:0/4062431322 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f04300ac680 0x7f0430151e60 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:11.928 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 -- 172.21.15.140:0/4062431322 >> 172.21.15.140:0/4062431322 conn(0x7f04300b36c0 msgr2=0x7f043005f840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:11.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 -- 172.21.15.140:0/4062431322 shutdown_connections 2022-07-21T01:58:11.929 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:11.919+0000 7f04381ef700 1 -- 172.21.15.140:0/4062431322 wait complete. 2022-07-21T01:58:11.934 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2022-07-21T01:58:11.935 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:58:12.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57d1bed700 1 Processor -- start 2022-07-21T01:58:12.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57d1bed700 1 -- start start 2022-07-21T01:58:12.046 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57d1bed700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57cc0b1620 0x7f57cc142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:12.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57d1bed700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f57cc142fd0 0x7f57cc13e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:12.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57d1bed700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57cc1426b0 con 0x7f57cc13ec20 2022-07-21T01:58:12.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57d1bed700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57cc060b50 con 0x7f57cc142fd0 2022-07-21T01:58:12.047 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57d1bed700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0b1a00 con 0x7f57cc0b1620 2022-07-21T01:58:12.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57cbfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f57cc13ec20 0x7f57cc0aa9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56186/0 (socket says 172.21.15.140:56186) 2022-07-21T01:58:12.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57cbfff700 1 -- 172.21.15.140:0/743523358 learned_addr learned my addr 172.21.15.140:0/743523358 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:12.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57caffd700 1 --2- 172.21.15.140:0/743523358 >> v2:172.21.15.152:3300/0 conn(0x7f57cc142fd0 0x7f57cc13e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:12.048 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57cb7fe700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57cc0b1620 0x7f57cc142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:12.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136205193 0 0) 0x7f57cc1426b0 con 0x7f57cc13ec20 2022-07-21T01:58:12.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b8003260 con 0x7f57cc13ec20 2022-07-21T01:58:12.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3458453909 0 0) 0x7f57b8003260 con 0x7f57cc13ec20 2022-07-21T01:58:12.049 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57cc0b1620 msgr2=0x7f57cc142ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:12.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57cc0b1620 0x7f57cc142ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:12.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 >> v2:172.21.15.152:3300/0 conn(0x7f57cc142fd0 msgr2=0x7f57cc13e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:12.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 --2- 172.21.15.140:0/743523358 >> v2:172.21.15.152:3300/0 conn(0x7f57cc142fd0 0x7f57cc13e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:12.050 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 --> v1:172.21.15.140:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc05f840 con 0x7f57cc13ec20 2022-07-21T01:58:12.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 <== mon.0 v1:172.21.15.140:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 372377358 0 0) 0x7f57c40019e0 con 0x7f57cc13ec20 2022-07-21T01:58:12.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 >> v1:172.21.15.140:6789/0 conn(0x7f57cc13ec20 legacy=0x7f57cc0aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:12.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57b8003750 0x7f57b8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:12.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.035+0000 7f57ca7fc700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57b8004500 0x7f57b8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:12.051 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57ca7fc700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b8004e70 0x7f57b8005720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:12.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc060b50 con 0x7f57b8003750 2022-07-21T01:58:12.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0b1a00 con 0x7f57b8004e70 2022-07-21T01:58:12.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b8005c10 con 0x7f57b8004500 2022-07-21T01:58:12.052 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57cb7fe700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57cc0b1620 0x7f57cc142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:58:12.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57cb7fe700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57b8003750 0x7f57b8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:12.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57cbfff700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b8004e70 0x7f57b8005720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:12.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57caffd700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57b8004500 0x7f57b8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:12.053 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57cb7fe700 1 -- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57b8004500 msgr2=0x7f57b8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:12.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57cb7fe700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57b8004500 0x7f57b8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:12.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57cb7fe700 1 -- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b8004e70 msgr2=0x7f57b8005720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:12.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57cb7fe700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b8004e70 0x7f57b8005720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:12.054 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57cb7fe700 1 -- 172.21.15.140:0/743523358 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8003260 con 0x7f57b8003750 2022-07-21T01:58:12.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57cb7fe700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57b8003750 0x7f57b8003b70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f57bc00a000 tx=0x7f57bc008040).ready entity=mon.0 client_cookie=204b966f83d52f28 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:12.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57bc013670 con 0x7f57b8003750 2022-07-21T01:58:12.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc0137d0 con 0x7f57b8003750 2022-07-21T01:58:12.055 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57ca7fc700 1 -- 172.21.15.140:0/743523358 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57bc013b30 con 0x7f57b8003750 2022-07-21T01:58:12.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57d1bed700 1 -- 172.21.15.140:0/743523358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57b8003750 msgr2=0x7f57b8003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:12.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57d1bed700 1 --2- 172.21.15.140:0/743523358 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57b8003750 0x7f57b8003b70 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f57bc00a000 tx=0x7f57bc008040).stop 2022-07-21T01:58:12.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57cbfff700 1 -- 172.21.15.140:0/743523358 reap_dead start 2022-07-21T01:58:12.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57d1bed700 1 -- 172.21.15.140:0/743523358 shutdown_connections 2022-07-21T01:58:12.056 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57d1bed700 1 -- 172.21.15.140:0/743523358 >> 172.21.15.140:0/743523358 conn(0x7f57cc0b23d0 msgr2=0x7f57cc0b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:12.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57d1bed700 1 -- 172.21.15.140:0/743523358 shutdown_connections 2022-07-21T01:58:12.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57d1bed700 1 -- 172.21.15.140:0/743523358 wait complete. 2022-07-21T01:58:12.057 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57d1bed700 1 Processor -- start 2022-07-21T01:58:12.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.039+0000 7f57d1bed700 1 -- start start 2022-07-21T01:58:12.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57d1bed700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b8003750 0x7f57cc063e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:12.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57d1bed700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc144fb0 0x7f57cc0aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:12.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57d1bed700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57cc13aa50 0x7f57cc13aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:12.058 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57d1bed700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0af360 con 0x7f57cc144fb0 2022-07-21T01:58:12.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57d1bed700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0ada40 con 0x7f57b8003750 2022-07-21T01:58:12.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57d1bed700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0643e0 con 0x7f57cc13aa50 2022-07-21T01:58:12.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57caffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc144fb0 0x7f57cc0aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:12.059 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57caffd700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc144fb0 0x7f57cc0aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42050/0 (socket says 172.21.15.140:42050) 2022-07-21T01:58:12.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57caffd700 1 -- 172.21.15.140:0/1986572312 learned_addr learned my addr 172.21.15.140:0/1986572312 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:12.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57cb7fe700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b8003750 0x7f57cc063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:12.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57cbfff700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57cc13aa50 0x7f57cc13aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:12.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57caffd700 1 -- 172.21.15.140:0/1986572312 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57cc13aa50 msgr2=0x7f57cc13aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:12.060 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57caffd700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57cc13aa50 0x7f57cc13aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:12.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57caffd700 1 -- 172.21.15.140:0/1986572312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b8003750 msgr2=0x7f57cc063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:12.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57caffd700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b8003750 0x7f57cc063e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:12.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57caffd700 1 -- 172.21.15.140:0/1986572312 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc0acf70 con 0x7f57cc144fb0 2022-07-21T01:58:12.061 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57cbfff700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57cc13aa50 0x7f57cc13aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T01:58:12.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57caffd700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc144fb0 0x7f57cc0aad10 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f57c000b000 tx=0x7f57c0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:12.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57c8ff9700 1 -- 172.21.15.140:0/1986572312 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57c000f940 con 0x7f57cc144fb0 2022-07-21T01:58:12.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57c8ff9700 1 -- 172.21.15.140:0/1986572312 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c000faa0 con 0x7f57cc144fb0 2022-07-21T01:58:12.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57d1bed700 1 -- 172.21.15.140:0/1986572312 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc0ad210 con 0x7f57cc144fb0 2022-07-21T01:58:12.062 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57c8ff9700 1 -- 172.21.15.140:0/1986572312 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f57c000fe00 con 0x7f57cc144fb0 2022-07-21T01:58:12.063 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57d1bed700 1 -- 172.21.15.140:0/1986572312 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc13b560 con 0x7f57cc144fb0 2022-07-21T01:58:12.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57d1bed700 1 -- 172.21.15.140:0/1986572312 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc0574e0 con 0x7f57cc144fb0 2022-07-21T01:58:12.064 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.043+0000 7f57c8ff9700 1 -- 172.21.15.140:0/1986572312 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f57c0014440 con 0x7f57cc144fb0 2022-07-21T01:58:12.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.047+0000 7f57c8ff9700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57ac053980 0x7f57ac055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:12.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.047+0000 7f57c8ff9700 1 -- 172.21.15.140:0/1986572312 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f57c006c680 con 0x7f57cc144fb0 2022-07-21T01:58:12.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.047+0000 7f57cb7fe700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57ac053980 0x7f57ac055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:12.065 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.047+0000 7f57cb7fe700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57ac053980 0x7f57ac055e40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f57bc01b000 tx=0x7f57bc008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:12.066 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.051+0000 7f57c8ff9700 1 -- 172.21.15.140:0/1986572312 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f57c0043550 con 0x7f57cc144fb0 2022-07-21T01:58:12.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.215+0000 7f57d1bed700 1 -- 172.21.15.140:0/1986572312 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57cc13b560 con 0x7f57cc144fb0 2022-07-21T01:58:12.220 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57c8ff9700 1 -- 172.21.15.140:0/1986572312 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2148 (secure 0 0 0) 0x7f57cc13b560 con 0x7f57cc144fb0 2022-07-21T01:58:12.266 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:12.267 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":74,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/3591375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591375250},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591375250}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:58:05.107197+0000","modified":"2022-07-21T01:58:06.110113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8727":{"gid":8727,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/4013573612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":4013573612},{"type":"v1","addr":"172.21.15.140:6835","nonce":4013573612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-07-21T01:58:12.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 -- 172.21.15.140:0/1986572312 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57ac053980 msgr2=0x7f57ac055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:12.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57ac053980 0x7f57ac055e40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f57bc01b000 tx=0x7f57bc008040).stop 2022-07-21T01:58:12.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 -- 172.21.15.140:0/1986572312 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc144fb0 msgr2=0x7f57cc0aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:12.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc144fb0 0x7f57cc0aad10 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f57c000b000 tx=0x7f57c0006df0).stop 2022-07-21T01:58:12.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 -- 172.21.15.140:0/1986572312 shutdown_connections 2022-07-21T01:58:12.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f57ac053980 0x7f57ac055e40 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:12.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f57cc13aa50 0x7f57cc13aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:12.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f57cc144fb0 0x7f57cc0aad10 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:12.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 --2- 172.21.15.140:0/1986572312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57b8003750 0x7f57cc063e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:12.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 -- 172.21.15.140:0/1986572312 >> 172.21.15.140:0/1986572312 conn(0x7f57cc0b23d0 msgr2=0x7f57cc0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:12.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 -- 172.21.15.140:0/1986572312 shutdown_connections 2022-07-21T01:58:12.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:12.219+0000 7f57d1bed700 1 -- 172.21.15.140:0/1986572312 wait complete. 2022-07-21T01:58:12.276 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 74 2022-07-21T01:58:12.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2022-07-21T01:58:05.107197+0000', 'modified': '2022-07-21T01:58:06.110113+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8727}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8727': {'gid': 8727, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.140:6835/4013573612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.140:6834', 'nonce': 4013573612}, {'type': 'v1', 'addr': '172.21.15.140:6835', 'nonce': 4013573612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2022-07-21T01:58:12.479 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2022-07-21T01:58:12.480 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2022-07-21T01:58:12.597 INFO:teuthology.orchestra.run.smithi140.stdout:{"cluster_fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","whoami":0,"id":8727,"want_state":"up:active","state":"up:active","rank_uptime":7.4772278649999997,"mdsmap_epoch":74,"osdmap_epoch":151,"osdmap_epoch_barrier":150,"uptime":11.850368573000001} 2022-07-21T01:58:12.598 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", "id": 8727, "mdsmap_epoch": 74, "osdmap_epoch": 151, "osdmap_epoch_barrier": 150, "rank_uptime": 7.477227865, "state": "up:active", "uptime": 11.850368573, "want_state": "up:active", "whoami": 0 } 2022-07-21T01:58:12.599 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2022-07-21T01:58:12.599 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2022-07-21T01:58:12.600 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@smithi192.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2022-07-21T01:58:12.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:12.601 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:58:12.610 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:58:12.630 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:58:12.630 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:58:12.647 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:58:12.647 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:58:12.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:12.649 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:58:12.699 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2022-07-21T01:58:12.699 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:58:12.702 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:58:12.719 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:58:12.720 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:58:12.729 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:58:12.730 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:58:12.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:12.732 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:58:12.777 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2022-07-21T01:58:12.775+0000 7f41f3c4f080 -1 init, newargv = 0x564ba5c2b240 newargc=9 2022-07-21T01:58:12.778 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[23084]: starting ceph client 2022-07-21T01:58:12.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[23084]: starting fuse 2022-07-21T01:58:13.881 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2022-07-21T01:58:13.900 INFO:teuthology.orchestra.run:Running command with timeout 30 2022-07-21T01:58:13.900 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2022-07-21T01:58:13.956 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2022-07-21T01:58:13.957 DEBUG:teuthology.orchestra.run:got remote process result: 32 2022-07-21T01:58:13.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:13.958 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2022-07-21T01:58:13.963 INFO:teuthology.orchestra.run.smithi192.stdout:52 2022-07-21T01:58:13.963 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2022-07-21T01:58:13.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:13.964 DEBUG:teuthology.orchestra.run.smithi192:> sudo python3 -c ' 2022-07-21T01:58:13.964 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2022-07-21T01:58:13.965 DEBUG:teuthology.orchestra.run.smithi192:> import re 2022-07-21T01:58:13.965 DEBUG:teuthology.orchestra.run.smithi192:> import os 2022-07-21T01:58:13.966 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2022-07-21T01:58:13.966 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:58:13.966 DEBUG:teuthology.orchestra.run.smithi192:> def find_socket(client_name): 2022-07-21T01:58:13.967 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2022-07-21T01:58:13.967 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2022-07-21T01:58:13.968 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:58:13.968 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2022-07-21T01:58:13.968 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2022-07-21T01:58:13.969 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2022-07-21T01:58:13.969 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2022-07-21T01:58:13.970 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:58:13.970 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2022-07-21T01:58:13.970 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2022-07-21T01:58:13.971 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2022-07-21T01:58:13.971 DEBUG:teuthology.orchestra.run.smithi192:> return f 2022-07-21T01:58:13.972 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2022-07-21T01:58:13.972 DEBUG:teuthology.orchestra.run.smithi192:> 2022-07-21T01:58:13.972 DEBUG:teuthology.orchestra.run.smithi192:> print(find_socket("client.1")) 2022-07-21T01:58:13.973 DEBUG:teuthology.orchestra.run.smithi192:> ' 2022-07-21T01:58:14.015 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.23084.asok 2022-07-21T01:58:14.020 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.23084.asok 2022-07-21T01:58:14.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:14.021 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.23084.asok status 2022-07-21T01:58:14.103 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2022-07-21T01:58:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2022-07-21T01:58:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "57544da114dbbe1c44f27b2ea18e3233e4e917e0", 2022-07-21T01:58:14.104 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 15.2.16-336-g57544da1 (57544da114dbbe1c44f27b2ea18e3233e4e917e0) octopus (stable)", 2022-07-21T01:58:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2022-07-21T01:58:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2022-07-21T01:58:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2022-07-21T01:58:14.105 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "23084", 2022-07-21T01:58:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2022-07-21T01:58:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:58:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2022-07-21T01:58:14.106 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2022-07-21T01:58:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 8952, 2022-07-21T01:58:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2022-07-21T01:58:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2022-07-21T01:58:14.107 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2022-07-21T01:58:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 8952 2022-07-21T01:58:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:58:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:58:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:58:14.108 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:58:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2498838996 2022-07-21T01:58:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2022-07-21T01:58:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:58:14.109 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2022-07-21T01:58:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2022-07-21T01:58:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "172.21.15.192:0", 2022-07-21T01:58:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2498838996 2022-07-21T01:58:14.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2022-07-21T01:58:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.8952 172.21.15.192:0/2498838996", 2022-07-21T01:58:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "172.21.15.192:0/2498838996", 2022-07-21T01:58:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2022-07-21T01:58:14.111 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 74, 2022-07-21T01:58:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 151, 2022-07-21T01:58:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2022-07-21T01:58:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "blacklisted": false, 2022-07-21T01:58:14.112 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2022-07-21T01:58:14.112 INFO:teuthology.orchestra.run.smithi192.stdout:} 2022-07-21T01:58:14.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:14.118 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:58:14.118 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:58:14.126 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:58:14.127 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:58:14.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:14.128 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:58:14.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:14.148 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:58:14.148 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:58:14.156 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:58:14.157 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:58:14.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:14.158 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:58:14.158 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:58:14.165 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2022-07-21T01:58:14.167 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2022-07-21T01:58:14.167 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2022-07-21T01:58:14.167 INFO:teuthology.orchestra.run:Running command with timeout 1800 2022-07-21T01:58:14.168 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:58:14.183 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[23084]: fuse finished with error 0 and tester_r 0 2022-07-21T01:58:14.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:14.187 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:58:14.187 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:58:14.235 INFO:teuthology.orchestra.run.smithi192.stdout:ext2/ext3 2022-07-21T01:58:14.236 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2022-07-21T01:58:14.237 INFO:teuthology.orchestra.run:waiting for 900 2022-07-21T01:58:20.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2022-07-21T01:58:20.339 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2022-07-21T01:58:20.348 INFO:teuthology.orchestra.run:Running command with timeout 10800 2022-07-21T01:58:20.348 DEBUG:teuthology.orchestra.run.smithi191:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2022-07-21T01:58:20.368 INFO:teuthology.orchestra.run.smithi191.stdout: File: "." 2022-07-21T01:58:20.368 INFO:teuthology.orchestra.run.smithi191.stdout: ID: 0 Namelen: 255 Type: fuseblk 2022-07-21T01:58:20.368 INFO:teuthology.orchestra.run.smithi191.stdout:Block size: 4194304 Fundamental block size: 4194304 2022-07-21T01:58:20.368 INFO:teuthology.orchestra.run.smithi191.stdout:Blocks: Total: 86148 Free: 86148 Available: 86148 2022-07-21T01:58:20.369 INFO:teuthology.orchestra.run.smithi191.stdout:Inodes: Total: 1 Free: 0 2022-07-21T01:58:20.370 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2022-07-21T01:58:20.480 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 Processor -- start 2022-07-21T01:58:20.481 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 -- start start 2022-07-21T01:58:20.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50ec0b1ef0 0x7f50ec0605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:20.491 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f50ec13f980 0x7f50ec0abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:20.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec140940 con 0x7f50ec0ac4e0 2022-07-21T01:58:20.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ec0b22d0 con 0x7f50ec13f980 2022-07-21T01:58:20.492 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec05fbc0 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f0940700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f50ec13f980 0x7f50ec0abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:20.493 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f0940700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f50ec13f980 0x7f50ec0abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42056/0 (socket says 172.21.15.140:42056) 2022-07-21T01:58:20.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f0940700 1 -- 172.21.15.140:0/1414975353 learned_addr learned my addr 172.21.15.140:0/1414975353 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:20.494 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f1141700 1 --2- 172.21.15.140:0/1414975353 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50ec0b1ef0 0x7f50ec0605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:20.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f0940700 1 -- 172.21.15.140:0/1414975353 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50ec0b1ef0 msgr2=0x7f50ec0605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:20.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f0940700 1 --2- 172.21.15.140:0/1414975353 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50ec0b1ef0 0x7f50ec0605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:20.495 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f0940700 1 -- 172.21.15.140:0/1414975353 >> v1:172.21.15.152:6789/0 conn(0x7f50ec0ac4e0 legacy=0x7f50ec0ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:20.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f0940700 1 -- 172.21.15.140:0/1414975353 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec05fe60 con 0x7f50ec13f980 2022-07-21T01:58:20.496 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f0940700 1 --2- 172.21.15.140:0/1414975353 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50ec13f980 0x7f50ec0abff0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f50ec062670 tx=0x7f50dc009290).ready entity=mon.0 client_cookie=23c4b8fe81885d0e server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:20.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50ebfff700 1 -- 172.21.15.140:0/1414975353 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50dc00b860 con 0x7f50ec13f980 2022-07-21T01:58:20.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50ebfff700 1 -- 172.21.15.140:0/1414975353 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc00b9c0 con 0x7f50ec13f980 2022-07-21T01:58:20.497 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50ebfff700 1 -- 172.21.15.140:0/1414975353 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50dc00bb70 con 0x7f50ec13f980 2022-07-21T01:58:20.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 -- 172.21.15.140:0/1414975353 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50ec13f980 msgr2=0x7f50ec0abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:20.498 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 --2- 172.21.15.140:0/1414975353 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50ec13f980 0x7f50ec0abff0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f50ec062670 tx=0x7f50dc009290).stop 2022-07-21T01:58:20.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 -- 172.21.15.140:0/1414975353 shutdown_connections 2022-07-21T01:58:20.499 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 --2- 172.21.15.140:0/1414975353 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50ec13f980 0x7f50ec0abff0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:20.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 --2- 172.21.15.140:0/1414975353 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50ec0b1ef0 0x7f50ec0605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:20.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 -- 172.21.15.140:0/1414975353 >> 172.21.15.140:0/1414975353 conn(0x7f50ec0b2ca0 msgr2=0x7f50ec0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:20.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 -- 172.21.15.140:0/1414975353 shutdown_connections 2022-07-21T01:58:20.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.479+0000 7f50f2143700 1 -- 172.21.15.140:0/1414975353 wait complete. 2022-07-21T01:58:20.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f2143700 1 Processor -- start 2022-07-21T01:58:20.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f2143700 1 -- start start 2022-07-21T01:58:20.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f2143700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50ec0ac4e0 0x7f50ec154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:20.503 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f2143700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50ec0b1ef0 0x7f50ec09a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:20.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f2143700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50ec13f980 0x7f50ec133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:20.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f2143700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0afc20 con 0x7f50ec13f980 2022-07-21T01:58:20.504 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f2143700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec09bdb0 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f2143700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0afe50 con 0x7f50ec0ac4e0 2022-07-21T01:58:20.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f0940700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50ec0b1ef0 0x7f50ec09a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:20.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f1942700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50ec13f980 0x7f50ec133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:20.505 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f0940700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50ec0b1ef0 0x7f50ec09a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:47802/0 (socket says 172.21.15.140:47802) 2022-07-21T01:58:20.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f0940700 1 -- 172.21.15.140:0/2932126135 learned_addr learned my addr 172.21.15.140:0/2932126135 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:20.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.483+0000 7f50f1141700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50ec0ac4e0 0x7f50ec154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:20.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50f0940700 1 -- 172.21.15.140:0/2932126135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50ec0ac4e0 msgr2=0x7f50ec154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:20.506 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50f0940700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50ec0ac4e0 0x7f50ec154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:20.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50f0940700 1 -- 172.21.15.140:0/2932126135 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50ec13f980 msgr2=0x7f50ec133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:20.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50f0940700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50ec13f980 0x7f50ec133ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:20.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50f0940700 1 -- 172.21.15.140:0/2932126135 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc009530 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.507 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50f0940700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50ec0b1ef0 0x7f50ec09a3e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f50ec062670 tx=0x7f50dc016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:20.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50ea7fc700 1 -- 172.21.15.140:0/2932126135 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50dc0032f0 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50ea7fc700 1 -- 172.21.15.140:0/2932126135 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc021440 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50f2143700 1 -- 172.21.15.140:0/2932126135 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec09c040 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.508 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50ea7fc700 1 -- 172.21.15.140:0/2932126135 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50dc0217a0 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50f2143700 1 -- 172.21.15.140:0/2932126135 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ec09aa70 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50f2143700 1 -- 172.21.15.140:0/2932126135 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ec0aff90 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.509 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.487+0000 7f50ea7fc700 1 -- 172.21.15.140:0/2932126135 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f50dc00f070 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.491+0000 7f50ea7fc700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50cc053980 0x7f50cc055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:20.510 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.491+0000 7f50f1141700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50cc053980 0x7f50cc055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:20.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.491+0000 7f50ea7fc700 1 -- 172.21.15.140:0/2932126135 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f50dc06d380 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.491+0000 7f50f1141700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50cc053980 0x7f50cc055e40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f50e000c000 tx=0x7f50e0008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:20.511 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.499+0000 7f50ea7fc700 1 -- 172.21.15.140:0/2932126135 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f50ec0aff90 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.627 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.623+0000 7f50f2143700 1 -- 172.21.15.140:0/2932126135 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f50ec0aff90 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.629 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.627+0000 7f50ea7fc700 1 -- 172.21.15.140:0/2932126135 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v151) v1 ==== 81+0+33755 (secure 0 0 0) 0x7f50ec0aff90 con 0x7f50ec0b1ef0 2022-07-21T01:58:20.630 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:20.631 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T01:58:20.631 INFO:teuthology.orchestra.run.smithi140.stdout: "epoch": 151, 2022-07-21T01:58:20.632 INFO:teuthology.orchestra.run.smithi140.stdout: "fsid": "87d2f829-64cd-4d49-9e42-a1dfafceefb8", 2022-07-21T01:58:20.632 INFO:teuthology.orchestra.run.smithi140.stdout: "created": "2022-07-21T01:48:15.114780+0000", 2022-07-21T01:58:20.632 INFO:teuthology.orchestra.run.smithi140.stdout: "modified": "2022-07-21T01:58:06.110718+0000", 2022-07-21T01:58:20.632 INFO:teuthology.orchestra.run.smithi140.stdout: "last_up_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:58:20.632 INFO:teuthology.orchestra.run.smithi140.stdout: "last_in_change": "2022-07-21T01:48:21.170967+0000", 2022-07-21T01:58:20.633 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2022-07-21T01:58:20.633 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_num": 5799936, 2022-07-21T01:58:20.633 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_set": [ 2022-07-21T01:58:20.633 INFO:teuthology.orchestra.run.smithi140.stdout: "pglog_hardlimit", 2022-07-21T01:58:20.634 INFO:teuthology.orchestra.run.smithi140.stdout: "purged_snapdirs", 2022-07-21T01:58:20.634 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_deletes", 2022-07-21T01:58:20.634 INFO:teuthology.orchestra.run.smithi140.stdout: "sortbitwise" 2022-07-21T01:58:20.634 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:58:20.635 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_version": 4, 2022-07-21T01:58:20.635 INFO:teuthology.orchestra.run.smithi140.stdout: "full_ratio": 0.94999998807907104, 2022-07-21T01:58:20.635 INFO:teuthology.orchestra.run.smithi140.stdout: "backfillfull_ratio": 0.89999997615814209, 2022-07-21T01:58:20.635 INFO:teuthology.orchestra.run.smithi140.stdout: "nearfull_ratio": 0.85000002384185791, 2022-07-21T01:58:20.635 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_snapshot": "", 2022-07-21T01:58:20.636 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_max": 30, 2022-07-21T01:58:20.636 INFO:teuthology.orchestra.run.smithi140.stdout: "max_osd": 8, 2022-07-21T01:58:20.636 INFO:teuthology.orchestra.run.smithi140.stdout: "require_min_compat_client": "jewel", 2022-07-21T01:58:20.636 INFO:teuthology.orchestra.run.smithi140.stdout: "min_compat_client": "jewel", 2022-07-21T01:58:20.636 INFO:teuthology.orchestra.run.smithi140.stdout: "require_osd_release": "octopus", 2022-07-21T01:58:20.637 INFO:teuthology.orchestra.run.smithi140.stdout: "pools": [ 2022-07-21T01:58:20.637 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.637 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 1, 2022-07-21T01:58:20.637 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "device_health_metrics", 2022-07-21T01:58:20.638 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:17.946023+0000", 2022-07-21T01:58:20.638 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:58:20.638 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:58:20.638 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:58:20.639 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:58:20.639 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:58:20.639 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:58:20.639 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:58:20.640 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:58:20.640 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 1, 2022-07-21T01:58:20.640 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 1, 2022-07-21T01:58:20.640 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 1, 2022-07-21T01:58:20.640 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 1, 2022-07-21T01:58:20.641 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 1, 2022-07-21T01:58:20.641 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:58:20.641 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:58:20.641 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:58:20.642 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:58:20.642 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:58:20.642 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:58:20.642 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:58:20.642 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.643 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "15", 2022-07-21T01:58:20.643 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:58:20.643 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:58:20.643 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:58:20.644 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:58:20.644 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:58:20.644 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:58:20.644 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:58:20.644 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:58:20.645 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:58:20.645 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:58:20.645 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:58:20.645 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:58:20.646 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:58:20.646 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:58:20.646 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:58:20.646 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:58:20.646 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:58:20.647 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:58:20.647 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:58:20.647 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:58:20.647 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:58:20.648 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:58:20.648 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:58:20.648 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:58:20.648 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:58:20.648 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:58:20.649 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.649 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:58:20.649 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:58:20.649 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:58:20.650 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:58:20.650 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:58:20.650 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:58:20.650 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:58:20.650 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:58:20.651 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:58:20.651 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:58:20.651 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:58:20.651 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:58:20.652 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 1 2022-07-21T01:58:20.652 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.652 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:58:20.652 INFO:teuthology.orchestra.run.smithi140.stdout: "mgr_devicehealth": {} 2022-07-21T01:58:20.652 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.653 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.653 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.653 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 2, 2022-07-21T01:58:20.653 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "rbd", 2022-07-21T01:58:20.654 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:48:26.620359+0000", 2022-07-21T01:58:20.654 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:58:20.654 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:58:20.654 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:58:20.654 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:58:20.655 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:58:20.655 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:58:20.655 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:58:20.655 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:58:20.656 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:58:20.656 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:58:20.656 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:58:20.656 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:58:20.656 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:58:20.657 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:58:20.657 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:58:20.657 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:58:20.657 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:58:20.658 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:58:20.658 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:58:20.658 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:58:20.658 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.658 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "18", 2022-07-21T01:58:20.659 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:58:20.659 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:58:20.659 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:58:20.659 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:58:20.660 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:58:20.660 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:58:20.660 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:58:20.660 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:58:20.660 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:58:20.661 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:58:20.661 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:58:20.661 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:58:20.661 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:58:20.662 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:58:20.662 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:58:20.662 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:58:20.662 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:58:20.662 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:58:20.663 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:58:20.663 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:58:20.663 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:58:20.663 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:58:20.663 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:58:20.664 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:58:20.664 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:58:20.664 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:58:20.664 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.665 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:58:20.665 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:58:20.665 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:58:20.665 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:58:20.666 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:58:20.666 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:58:20.666 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:58:20.666 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:58:20.667 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:58:20.667 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:58:20.667 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:58:20.667 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:58:20.667 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:58:20.668 INFO:teuthology.orchestra.run.smithi140.stdout: "rbd": {} 2022-07-21T01:58:20.668 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.668 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.668 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.669 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 29, 2022-07-21T01:58:20.669 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_metadata", 2022-07-21T01:58:20.669 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:58:03.485327+0000", 2022-07-21T01:58:20.669 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:58:20.669 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:58:20.670 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:58:20.670 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:58:20.670 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:58:20.670 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:58:20.671 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:58:20.671 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:58:20.671 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:58:20.671 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:58:20.671 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:58:20.672 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:58:20.672 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:58:20.672 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:58:20.672 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:58:20.673 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:58:20.673 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:58:20.673 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:58:20.673 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:58:20.673 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:58:20.674 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.674 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "150", 2022-07-21T01:58:20.674 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:58:20.674 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:58:20.675 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:58:20.675 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:58:20.675 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:58:20.675 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:58:20.675 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:58:20.676 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:58:20.676 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:58:20.676 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:58:20.676 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:58:20.677 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:58:20.677 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:58:20.677 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:58:20.677 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:58:20.677 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:58:20.678 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:58:20.678 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:58:20.678 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:58:20.678 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:58:20.679 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:58:20.679 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:58:20.679 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:58:20.679 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:58:20.679 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:58:20.680 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:58:20.680 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.680 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:58:20.680 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:58:20.680 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:58:20.681 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:58:20.681 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:58:20.681 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:58:20.681 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:58:20.682 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:58:20.682 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:58:20.682 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:58:20.682 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:58:20.682 INFO:teuthology.orchestra.run.smithi140.stdout: "options": { 2022-07-21T01:58:20.683 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_bias": 4, 2022-07-21T01:58:20.683 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_min": 16, 2022-07-21T01:58:20.683 INFO:teuthology.orchestra.run.smithi140.stdout: "recovery_priority": 5 2022-07-21T01:58:20.683 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.684 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:58:20.684 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:58:20.684 INFO:teuthology.orchestra.run.smithi140.stdout: "metadata": "cephfs" 2022-07-21T01:58:20.684 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.684 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.685 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.685 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.685 INFO:teuthology.orchestra.run.smithi140.stdout: "pool": 30, 2022-07-21T01:58:20.685 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_name": "cephfs_data", 2022-07-21T01:58:20.686 INFO:teuthology.orchestra.run.smithi140.stdout: "create_time": "2022-07-21T01:58:04.503066+0000", 2022-07-21T01:58:20.686 INFO:teuthology.orchestra.run.smithi140.stdout: "flags": 1, 2022-07-21T01:58:20.686 INFO:teuthology.orchestra.run.smithi140.stdout: "flags_names": "hashpspool", 2022-07-21T01:58:20.686 INFO:teuthology.orchestra.run.smithi140.stdout: "type": 1, 2022-07-21T01:58:20.686 INFO:teuthology.orchestra.run.smithi140.stdout: "size": 2, 2022-07-21T01:58:20.687 INFO:teuthology.orchestra.run.smithi140.stdout: "min_size": 1, 2022-07-21T01:58:20.687 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_rule": 0, 2022-07-21T01:58:20.687 INFO:teuthology.orchestra.run.smithi140.stdout: "object_hash": 2, 2022-07-21T01:58:20.687 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_autoscale_mode": "off", 2022-07-21T01:58:20.688 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num": 8, 2022-07-21T01:58:20.688 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num": 8, 2022-07-21T01:58:20.688 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_placement_num_target": 8, 2022-07-21T01:58:20.688 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_target": 8, 2022-07-21T01:58:20.688 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_num_pending": 8, 2022-07-21T01:58:20.689 INFO:teuthology.orchestra.run.smithi140.stdout: "last_pg_merge_meta": { 2022-07-21T01:58:20.689 INFO:teuthology.orchestra.run.smithi140.stdout: "source_pgid": "0.0", 2022-07-21T01:58:20.689 INFO:teuthology.orchestra.run.smithi140.stdout: "ready_epoch": 0, 2022-07-21T01:58:20.689 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_started": 0, 2022-07-21T01:58:20.689 INFO:teuthology.orchestra.run.smithi140.stdout: "last_epoch_clean": 0, 2022-07-21T01:58:20.690 INFO:teuthology.orchestra.run.smithi140.stdout: "source_version": "0'0", 2022-07-21T01:58:20.690 INFO:teuthology.orchestra.run.smithi140.stdout: "target_version": "0'0" 2022-07-21T01:58:20.691 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.691 INFO:teuthology.orchestra.run.smithi140.stdout: "last_change": "151", 2022-07-21T01:58:20.692 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend": "0", 2022-07-21T01:58:20.692 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_prenautilus": "0", 2022-07-21T01:58:20.693 INFO:teuthology.orchestra.run.smithi140.stdout: "last_force_op_resend_preluminous": "0", 2022-07-21T01:58:20.693 INFO:teuthology.orchestra.run.smithi140.stdout: "auid": 0, 2022-07-21T01:58:20.694 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_mode": "selfmanaged", 2022-07-21T01:58:20.694 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_seq": 0, 2022-07-21T01:58:20.694 INFO:teuthology.orchestra.run.smithi140.stdout: "snap_epoch": 0, 2022-07-21T01:58:20.695 INFO:teuthology.orchestra.run.smithi140.stdout: "pool_snaps": [], 2022-07-21T01:58:20.695 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps": "[]", 2022-07-21T01:58:20.696 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_bytes": 0, 2022-07-21T01:58:20.696 INFO:teuthology.orchestra.run.smithi140.stdout: "quota_max_objects": 0, 2022-07-21T01:58:20.696 INFO:teuthology.orchestra.run.smithi140.stdout: "tiers": [], 2022-07-21T01:58:20.697 INFO:teuthology.orchestra.run.smithi140.stdout: "tier_of": -1, 2022-07-21T01:58:20.697 INFO:teuthology.orchestra.run.smithi140.stdout: "read_tier": -1, 2022-07-21T01:58:20.697 INFO:teuthology.orchestra.run.smithi140.stdout: "write_tier": -1, 2022-07-21T01:58:20.698 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_mode": "none", 2022-07-21T01:58:20.698 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_bytes": 0, 2022-07-21T01:58:20.698 INFO:teuthology.orchestra.run.smithi140.stdout: "target_max_objects": 0, 2022-07-21T01:58:20.698 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_ratio_micro": 400000, 2022-07-21T01:58:20.699 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2022-07-21T01:58:20.699 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_target_full_ratio_micro": 800000, 2022-07-21T01:58:20.699 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_flush_age": 0, 2022-07-21T01:58:20.700 INFO:teuthology.orchestra.run.smithi140.stdout: "cache_min_evict_age": 0, 2022-07-21T01:58:20.700 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profile": "", 2022-07-21T01:58:20.700 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_params": { 2022-07-21T01:58:20.701 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "none" 2022-07-21T01:58:20.701 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.701 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_period": 0, 2022-07-21T01:58:20.702 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_count": 0, 2022-07-21T01:58:20.702 INFO:teuthology.orchestra.run.smithi140.stdout: "use_gmt_hitset": true, 2022-07-21T01:58:20.703 INFO:teuthology.orchestra.run.smithi140.stdout: "min_read_recency_for_promote": 0, 2022-07-21T01:58:20.703 INFO:teuthology.orchestra.run.smithi140.stdout: "min_write_recency_for_promote": 0, 2022-07-21T01:58:20.703 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_grade_decay_rate": 0, 2022-07-21T01:58:20.704 INFO:teuthology.orchestra.run.smithi140.stdout: "hit_set_search_last_n": 0, 2022-07-21T01:58:20.704 INFO:teuthology.orchestra.run.smithi140.stdout: "grade_table": [], 2022-07-21T01:58:20.705 INFO:teuthology.orchestra.run.smithi140.stdout: "stripe_width": 0, 2022-07-21T01:58:20.705 INFO:teuthology.orchestra.run.smithi140.stdout: "expected_num_objects": 0, 2022-07-21T01:58:20.706 INFO:teuthology.orchestra.run.smithi140.stdout: "fast_read": false, 2022-07-21T01:58:20.706 INFO:teuthology.orchestra.run.smithi140.stdout: "options": {}, 2022-07-21T01:58:20.707 INFO:teuthology.orchestra.run.smithi140.stdout: "application_metadata": { 2022-07-21T01:58:20.707 INFO:teuthology.orchestra.run.smithi140.stdout: "cephfs": { 2022-07-21T01:58:20.708 INFO:teuthology.orchestra.run.smithi140.stdout: "data": "cephfs" 2022-07-21T01:58:20.708 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.709 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.709 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.710 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:58:20.710 INFO:teuthology.orchestra.run.smithi140.stdout: "osds": [ 2022-07-21T01:58:20.711 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.711 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:58:20.712 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "f32f4b5f-243e-4377-b3e7-6d2876095aec", 2022-07-21T01:58:20.712 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:58:20.712 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:58:20.713 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:58:20.713 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:58:20.714 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:58:20.714 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:58:20.715 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:58:20.715 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 149, 2022-07-21T01:58:20.716 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:58:20.716 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:58:20.717 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:58:20.717 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.718 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.718 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.719 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6802", 2022-07-21T01:58:20.719 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:58:20.720 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.720 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.721 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.721 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6803", 2022-07-21T01:58:20.722 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:58:20.722 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.723 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.723 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.723 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:58:20.724 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.724 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.725 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.725 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6804", 2022-07-21T01:58:20.726 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:58:20.726 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.727 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.727 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.728 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6805", 2022-07-21T01:58:20.728 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:58:20.729 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.729 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.730 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.730 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:58:20.731 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.731 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.732 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.732 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6808", 2022-07-21T01:58:20.733 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:58:20.733 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.734 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.734 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.735 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6809", 2022-07-21T01:58:20.735 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:58:20.735 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.735 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.736 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.736 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:58:20.736 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.736 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.737 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.737 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6806", 2022-07-21T01:58:20.737 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:58:20.737 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.738 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.738 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.738 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6807", 2022-07-21T01:58:20.738 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22710 2022-07-21T01:58:20.739 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.739 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.739 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.739 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6803/22710", 2022-07-21T01:58:20.740 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6805/22710", 2022-07-21T01:58:20.740 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6809/22710", 2022-07-21T01:58:20.740 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6807/22710", 2022-07-21T01:58:20.740 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:58:20.741 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:58:20.741 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:58:20.741 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.741 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.742 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.742 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:58:20.742 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "50f0b7fd-5258-4d4d-8075-828f0b34d254", 2022-07-21T01:58:20.742 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:58:20.743 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:58:20.743 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:58:20.743 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:58:20.743 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:58:20.744 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:58:20.744 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:58:20.744 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 148, 2022-07-21T01:58:20.744 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:58:20.744 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:58:20.745 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:58:20.745 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.745 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.745 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.746 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6826", 2022-07-21T01:58:20.746 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:58:20.746 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.746 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.747 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.747 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6827", 2022-07-21T01:58:20.747 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:58:20.747 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.748 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.748 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.748 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:58:20.748 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.749 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.749 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.749 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6828", 2022-07-21T01:58:20.749 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:58:20.750 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.750 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.750 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.750 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6829", 2022-07-21T01:58:20.751 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:58:20.751 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.751 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.751 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.751 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:58:20.752 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.752 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.752 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.752 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6832", 2022-07-21T01:58:20.753 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:58:20.753 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.753 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.753 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.754 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6833", 2022-07-21T01:58:20.754 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:58:20.754 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.754 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.755 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.755 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:58:20.755 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.755 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.756 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.756 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6830", 2022-07-21T01:58:20.756 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:58:20.756 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.757 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.757 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.757 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6831", 2022-07-21T01:58:20.757 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22712 2022-07-21T01:58:20.758 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.758 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.758 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.758 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6827/22712", 2022-07-21T01:58:20.759 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6829/22712", 2022-07-21T01:58:20.759 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6833/22712", 2022-07-21T01:58:20.759 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6831/22712", 2022-07-21T01:58:20.759 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:58:20.759 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:58:20.760 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:58:20.760 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.760 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.760 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.761 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:58:20.761 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "0c4190c0-0c1b-49da-ba40-93dab22993a2", 2022-07-21T01:58:20.761 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:58:20.761 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:58:20.762 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:58:20.762 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:58:20.762 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:58:20.762 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:58:20.763 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:58:20.763 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 149, 2022-07-21T01:58:20.763 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:58:20.763 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:58:20.764 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:58:20.764 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.764 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.764 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.765 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6810", 2022-07-21T01:58:20.765 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:58:20.765 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.765 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.765 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.766 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6811", 2022-07-21T01:58:20.766 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:58:20.766 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.766 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.767 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.767 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:58:20.767 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.767 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.768 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.768 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6812", 2022-07-21T01:58:20.768 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:58:20.768 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.769 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.769 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.769 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6813", 2022-07-21T01:58:20.769 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:58:20.770 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.770 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.770 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.770 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:58:20.771 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.771 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.771 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.771 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6816", 2022-07-21T01:58:20.772 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:58:20.772 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.772 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.772 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.772 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6817", 2022-07-21T01:58:20.773 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:58:20.773 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.773 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.773 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.774 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:58:20.774 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.774 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.774 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.775 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6814", 2022-07-21T01:58:20.775 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:58:20.775 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.775 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.776 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.776 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6815", 2022-07-21T01:58:20.776 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22724 2022-07-21T01:58:20.776 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.777 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.777 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.777 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6811/22724", 2022-07-21T01:58:20.777 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6813/22724", 2022-07-21T01:58:20.778 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6817/22724", 2022-07-21T01:58:20.778 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6815/22724", 2022-07-21T01:58:20.778 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:58:20.778 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:58:20.779 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:58:20.779 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.779 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.779 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.779 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:58:20.780 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "6318d81c-39e1-4e05-8ada-4bb0536c536f", 2022-07-21T01:58:20.780 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:58:20.780 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:58:20.780 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:58:20.781 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:58:20.781 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:58:20.781 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:58:20.781 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:58:20.782 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 149, 2022-07-21T01:58:20.782 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:58:20.782 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:58:20.782 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:58:20.783 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.783 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.783 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.783 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6818", 2022-07-21T01:58:20.784 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:58:20.784 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.784 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.784 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.785 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6819", 2022-07-21T01:58:20.785 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:58:20.785 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.785 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.786 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.786 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:58:20.786 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.786 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.787 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.787 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6820", 2022-07-21T01:58:20.787 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:58:20.787 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.788 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.788 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.788 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6821", 2022-07-21T01:58:20.788 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:58:20.788 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.789 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.789 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.789 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:58:20.789 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.790 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.790 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.790 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6824", 2022-07-21T01:58:20.790 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:58:20.791 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.791 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.791 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.791 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6825", 2022-07-21T01:58:20.792 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:58:20.792 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.792 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.792 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.793 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:58:20.793 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.793 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.793 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.794 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6822", 2022-07-21T01:58:20.794 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:58:20.794 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.794 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.794 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.795 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.140:6823", 2022-07-21T01:58:20.795 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 22721 2022-07-21T01:58:20.795 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.795 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.796 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.796 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.140:6819/22721", 2022-07-21T01:58:20.796 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.140:6821/22721", 2022-07-21T01:58:20.797 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.140:6825/22721", 2022-07-21T01:58:20.797 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.140:6823/22721", 2022-07-21T01:58:20.797 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:58:20.797 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:58:20.798 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:58:20.798 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.798 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.798 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.799 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:58:20.799 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "fc54a43f-8a54-40e1-9db5-e4bb3c84ed26", 2022-07-21T01:58:20.799 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:58:20.799 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:58:20.800 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:58:20.800 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:58:20.800 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:58:20.800 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:58:20.800 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:58:20.801 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 148, 2022-07-21T01:58:20.801 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:58:20.801 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:58:20.801 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:58:20.802 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.802 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.802 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.802 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6800", 2022-07-21T01:58:20.803 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:58:20.803 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.803 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.803 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.804 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6801", 2022-07-21T01:58:20.804 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:58:20.804 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.804 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.805 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.805 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:58:20.805 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.805 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.805 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.806 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6802", 2022-07-21T01:58:20.806 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:58:20.806 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.806 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.807 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.807 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6803", 2022-07-21T01:58:20.807 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:58:20.807 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.808 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.808 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.808 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:58:20.808 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.809 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.809 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.809 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6806", 2022-07-21T01:58:20.809 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:58:20.810 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.810 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.810 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.810 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6807", 2022-07-21T01:58:20.810 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:58:20.811 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.811 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.811 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.811 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:58:20.812 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.812 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.812 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.812 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6804", 2022-07-21T01:58:20.813 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:58:20.813 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.813 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.813 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.814 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6805", 2022-07-21T01:58:20.814 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23141 2022-07-21T01:58:20.814 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.814 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.815 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.815 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6801/23141", 2022-07-21T01:58:20.815 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6803/23141", 2022-07-21T01:58:20.815 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6807/23141", 2022-07-21T01:58:20.816 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6805/23141", 2022-07-21T01:58:20.816 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:58:20.816 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:58:20.816 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:58:20.816 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.817 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.817 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.817 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:58:20.817 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "d63edac8-54f9-468a-8748-faf4231a7cb8", 2022-07-21T01:58:20.818 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:58:20.818 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:58:20.818 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:58:20.818 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:58:20.819 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:58:20.819 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:58:20.819 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:58:20.819 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 149, 2022-07-21T01:58:20.820 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:58:20.820 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:58:20.820 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:58:20.820 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.821 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.821 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.821 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6810", 2022-07-21T01:58:20.821 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:58:20.822 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.822 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.822 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.822 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6812", 2022-07-21T01:58:20.823 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:58:20.823 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.823 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.823 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.823 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:58:20.824 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.824 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.824 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.824 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6814", 2022-07-21T01:58:20.825 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:58:20.825 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.825 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.825 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.826 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6816", 2022-07-21T01:58:20.826 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:58:20.826 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.826 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.827 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.827 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:58:20.827 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.827 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.828 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.828 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6822", 2022-07-21T01:58:20.828 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:58:20.828 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.829 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.829 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.829 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6823", 2022-07-21T01:58:20.829 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:58:20.830 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.830 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.830 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.830 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:58:20.831 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.831 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.831 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.831 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6818", 2022-07-21T01:58:20.831 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:58:20.832 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.832 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.832 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.832 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6820", 2022-07-21T01:58:20.833 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23143 2022-07-21T01:58:20.833 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.833 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.833 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.834 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6812/23143", 2022-07-21T01:58:20.834 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6816/23143", 2022-07-21T01:58:20.834 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6823/23143", 2022-07-21T01:58:20.834 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6820/23143", 2022-07-21T01:58:20.835 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:58:20.835 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:58:20.835 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:58:20.835 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.836 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.836 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.836 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:58:20.836 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "81228326-ddc7-4683-83f7-c1c6eb0232d4", 2022-07-21T01:58:20.837 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:58:20.837 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:58:20.837 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:58:20.837 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:58:20.837 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:58:20.838 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:58:20.838 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:58:20.838 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 149, 2022-07-21T01:58:20.838 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:58:20.839 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:58:20.839 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:58:20.839 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.839 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.840 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.840 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6808", 2022-07-21T01:58:20.840 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:20.840 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.841 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.841 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.841 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6809", 2022-07-21T01:58:20.841 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:20.842 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.842 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.842 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.842 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:58:20.843 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.843 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.843 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.843 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6811", 2022-07-21T01:58:20.843 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:20.844 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.844 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.844 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.844 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6813", 2022-07-21T01:58:20.845 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:20.845 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.845 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.845 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.846 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:58:20.846 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.846 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.846 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.847 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6819", 2022-07-21T01:58:20.847 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:20.847 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.847 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.848 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.848 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6821", 2022-07-21T01:58:20.848 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:20.848 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.848 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.849 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.849 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:58:20.849 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.849 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.850 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.850 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6815", 2022-07-21T01:58:20.850 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:20.850 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.851 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.851 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.851 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6817", 2022-07-21T01:58:20.851 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23144 2022-07-21T01:58:20.852 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.852 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.852 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.852 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6809/23144", 2022-07-21T01:58:20.853 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6813/23144", 2022-07-21T01:58:20.853 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6821/23144", 2022-07-21T01:58:20.853 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6817/23144", 2022-07-21T01:58:20.853 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:58:20.854 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:58:20.854 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:58:20.854 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.854 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.854 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.855 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:58:20.855 INFO:teuthology.orchestra.run.smithi140.stdout: "uuid": "4efb2119-4ea0-4c46-95e2-662416bc7990", 2022-07-21T01:58:20.856 INFO:teuthology.orchestra.run.smithi140.stdout: "up": 1, 2022-07-21T01:58:20.857 INFO:teuthology.orchestra.run.smithi140.stdout: "in": 1, 2022-07-21T01:58:20.857 INFO:teuthology.orchestra.run.smithi140.stdout: "weight": 1, 2022-07-21T01:58:20.857 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_affinity": 1, 2022-07-21T01:58:20.858 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_begin": 0, 2022-07-21T01:58:20.858 INFO:teuthology.orchestra.run.smithi140.stdout: "last_clean_end": 0, 2022-07-21T01:58:20.858 INFO:teuthology.orchestra.run.smithi140.stdout: "up_from": 13, 2022-07-21T01:58:20.858 INFO:teuthology.orchestra.run.smithi140.stdout: "up_thru": 149, 2022-07-21T01:58:20.859 INFO:teuthology.orchestra.run.smithi140.stdout: "down_at": 0, 2022-07-21T01:58:20.859 INFO:teuthology.orchestra.run.smithi140.stdout: "lost_at": 0, 2022-07-21T01:58:20.859 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addrs": { 2022-07-21T01:58:20.859 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.860 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.860 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.860 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6824", 2022-07-21T01:58:20.860 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:20.861 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.861 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.861 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.861 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6825", 2022-07-21T01:58:20.862 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:20.862 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.862 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.862 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.863 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addrs": { 2022-07-21T01:58:20.863 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.863 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.863 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.864 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6826", 2022-07-21T01:58:20.864 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:20.864 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.864 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.864 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.865 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6827", 2022-07-21T01:58:20.865 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:20.865 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.865 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.866 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.866 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addrs": { 2022-07-21T01:58:20.866 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.866 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.867 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.867 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6830", 2022-07-21T01:58:20.867 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:20.867 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.868 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.868 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.868 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6831", 2022-07-21T01:58:20.868 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:20.869 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.869 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.869 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.869 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addrs": { 2022-07-21T01:58:20.870 INFO:teuthology.orchestra.run.smithi140.stdout: "addrvec": [ 2022-07-21T01:58:20.870 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.870 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v2", 2022-07-21T01:58:20.870 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6828", 2022-07-21T01:58:20.871 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:20.871 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.871 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.871 INFO:teuthology.orchestra.run.smithi140.stdout: "type": "v1", 2022-07-21T01:58:20.871 INFO:teuthology.orchestra.run.smithi140.stdout: "addr": "172.21.15.152:6829", 2022-07-21T01:58:20.872 INFO:teuthology.orchestra.run.smithi140.stdout: "nonce": 23142 2022-07-21T01:58:20.872 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.872 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.872 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.873 INFO:teuthology.orchestra.run.smithi140.stdout: "public_addr": "172.21.15.152:6825/23142", 2022-07-21T01:58:20.873 INFO:teuthology.orchestra.run.smithi140.stdout: "cluster_addr": "172.21.15.152:6827/23142", 2022-07-21T01:58:20.873 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_back_addr": "172.21.15.152:6831/23142", 2022-07-21T01:58:20.873 INFO:teuthology.orchestra.run.smithi140.stdout: "heartbeat_front_addr": "172.21.15.152:6829/23142", 2022-07-21T01:58:20.874 INFO:teuthology.orchestra.run.smithi140.stdout: "state": [ 2022-07-21T01:58:20.874 INFO:teuthology.orchestra.run.smithi140.stdout: "exists", 2022-07-21T01:58:20.874 INFO:teuthology.orchestra.run.smithi140.stdout: "up" 2022-07-21T01:58:20.874 INFO:teuthology.orchestra.run.smithi140.stdout: ] 2022-07-21T01:58:20.875 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.875 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:58:20.875 INFO:teuthology.orchestra.run.smithi140.stdout: "osd_xinfo": [ 2022-07-21T01:58:20.875 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.876 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 0, 2022-07-21T01:58:20.876 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:20.876 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:20.876 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:20.877 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:20.877 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:20.877 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.223940+0000", 2022-07-21T01:58:20.877 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:20.878 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.878 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.878 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 1, 2022-07-21T01:58:20.878 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:20.879 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:20.879 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:20.879 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:20.879 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:20.880 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.315596+0000", 2022-07-21T01:58:20.880 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:20.880 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.880 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.881 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 2, 2022-07-21T01:58:20.881 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:20.881 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:20.881 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:20.882 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:20.882 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:20.882 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.275895+0000", 2022-07-21T01:58:20.882 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:20.882 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.883 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.883 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 3, 2022-07-21T01:58:20.883 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:20.883 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:20.884 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:20.884 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:20.884 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:20.884 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.318162+0000", 2022-07-21T01:58:20.885 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:20.885 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.885 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.885 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 4, 2022-07-21T01:58:20.886 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:20.886 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:20.886 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:20.886 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:20.887 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:20.887 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.311667+0000", 2022-07-21T01:58:20.887 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:20.887 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.888 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.888 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 5, 2022-07-21T01:58:20.888 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:20.888 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:20.889 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:20.889 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:20.889 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:20.889 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.338645+0000", 2022-07-21T01:58:20.889 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:20.890 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.890 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.890 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 6, 2022-07-21T01:58:20.890 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:20.891 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:20.891 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:20.891 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:20.891 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:20.892 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.347179+0000", 2022-07-21T01:58:20.892 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:20.892 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.892 INFO:teuthology.orchestra.run.smithi140.stdout: { 2022-07-21T01:58:20.893 INFO:teuthology.orchestra.run.smithi140.stdout: "osd": 7, 2022-07-21T01:58:20.893 INFO:teuthology.orchestra.run.smithi140.stdout: "down_stamp": "0.000000", 2022-07-21T01:58:20.893 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_probability": 0, 2022-07-21T01:58:20.893 INFO:teuthology.orchestra.run.smithi140.stdout: "laggy_interval": 0, 2022-07-21T01:58:20.894 INFO:teuthology.orchestra.run.smithi140.stdout: "features": 4540138292840890367, 2022-07-21T01:58:20.894 INFO:teuthology.orchestra.run.smithi140.stdout: "old_weight": 0, 2022-07-21T01:58:20.894 INFO:teuthology.orchestra.run.smithi140.stdout: "last_purged_snaps_scrub": "2022-07-21T01:48:19.335541+0000", 2022-07-21T01:58:20.894 INFO:teuthology.orchestra.run.smithi140.stdout: "dead_epoch": 0 2022-07-21T01:58:20.895 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.895 INFO:teuthology.orchestra.run.smithi140.stdout: ], 2022-07-21T01:58:20.895 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap": [], 2022-07-21T01:58:20.895 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_upmap_items": [], 2022-07-21T01:58:20.895 INFO:teuthology.orchestra.run.smithi140.stdout: "pg_temp": [], 2022-07-21T01:58:20.896 INFO:teuthology.orchestra.run.smithi140.stdout: "primary_temp": [], 2022-07-21T01:58:20.896 INFO:teuthology.orchestra.run.smithi140.stdout: "blacklist": {}, 2022-07-21T01:58:20.896 INFO:teuthology.orchestra.run.smithi140.stdout: "erasure_code_profiles": { 2022-07-21T01:58:20.896 INFO:teuthology.orchestra.run.smithi140.stdout: "default": { 2022-07-21T01:58:20.897 INFO:teuthology.orchestra.run.smithi140.stdout: "crush-failure-domain": "osd", 2022-07-21T01:58:20.897 INFO:teuthology.orchestra.run.smithi140.stdout: "k": "2", 2022-07-21T01:58:20.897 INFO:teuthology.orchestra.run.smithi140.stdout: "m": "1", 2022-07-21T01:58:20.897 INFO:teuthology.orchestra.run.smithi140.stdout: "plugin": "jerasure", 2022-07-21T01:58:20.898 INFO:teuthology.orchestra.run.smithi140.stdout: "ruleset-failure-domain": "osd", 2022-07-21T01:58:20.898 INFO:teuthology.orchestra.run.smithi140.stdout: "technique": "reed_sol_van" 2022-07-21T01:58:20.898 INFO:teuthology.orchestra.run.smithi140.stdout: } 2022-07-21T01:58:20.898 INFO:teuthology.orchestra.run.smithi140.stdout: }, 2022-07-21T01:58:20.899 INFO:teuthology.orchestra.run.smithi140.stdout: "removed_snaps_queue": [], 2022-07-21T01:58:20.899 INFO:teuthology.orchestra.run.smithi140.stdout: "new_removed_snaps": [], 2022-07-21T01:58:20.899 INFO:teuthology.orchestra.run.smithi140.stdout: "new_purged_snaps": [], 2022-07-21T01:58:20.899 INFO:teuthology.orchestra.run.smithi140.stdout: "crush_node_flags": {}, 2022-07-21T01:58:20.900 INFO:teuthology.orchestra.run.smithi140.stdout: "device_class_flags": {} 2022-07-21T01:58:20.900 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T01:58:20.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 -- 172.21.15.140:0/2932126135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50cc053980 msgr2=0x7f50cc055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:20.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50cc053980 0x7f50cc055e40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f50e000c000 tx=0x7f50e0008040).stop 2022-07-21T01:58:20.906 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 -- 172.21.15.140:0/2932126135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50ec0b1ef0 msgr2=0x7f50ec09a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:20.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50ec0b1ef0 0x7f50ec09a3e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f50ec062670 tx=0x7f50dc016000).stop 2022-07-21T01:58:20.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 -- 172.21.15.140:0/2932126135 shutdown_connections 2022-07-21T01:58:20.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f50cc053980 0x7f50cc055e40 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:20.907 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f50ec13f980 0x7f50ec133ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:20.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f50ec0b1ef0 0x7f50ec09a3e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:20.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 --2- 172.21.15.140:0/2932126135 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f50ec0ac4e0 0x7f50ec154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:20.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 -- 172.21.15.140:0/2932126135 >> 172.21.15.140:0/2932126135 conn(0x7f50ec0b2ca0 msgr2=0x7f50ec05cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:20.908 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 -- 172.21.15.140:0/2932126135 shutdown_connections 2022-07-21T01:58:20.909 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:20.631+0000 7f50f2143700 1 -- 172.21.15.140:0/2932126135 wait complete. 2022-07-21T01:58:20.909 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:58:21.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.015+0000 7fa089653700 1 Processor -- start 2022-07-21T01:58:21.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.015+0000 7fa089653700 1 -- start start 2022-07-21T01:58:21.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa0840b5650 0x7fa0840b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0840adff0 0x7fa084147750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa0840af8e0 0x7fa08405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa084147010 con 0x7fa0840af8e0 2022-07-21T01:58:21.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0841472b0 con 0x7fa0840b5650 2022-07-21T01:58:21.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa084146d40 con 0x7fa0840adff0 2022-07-21T01:58:21.023 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa088e52700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa0840af8e0 0x7fa08405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa088e52700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa0840af8e0 0x7fa08405bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42074/0 (socket says 172.21.15.140:42074) 2022-07-21T01:58:21.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa088e52700 1 -- 172.21.15.140:0/5463972 learned_addr learned my addr 172.21.15.140:0/5463972 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:21.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa0837fe700 1 --2- 172.21.15.140:0/5463972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0840adff0 0x7fa084147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa083fff700 1 --2- 172.21.15.140:0/5463972 >> v2:172.21.15.152:3300/0 conn(0x7fa0840b5650 0x7fa0840b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa088e52700 1 -- 172.21.15.140:0/5463972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0840adff0 msgr2=0x7fa084147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:21.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa088e52700 1 --2- 172.21.15.140:0/5463972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0840adff0 0x7fa084147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa088e52700 1 -- 172.21.15.140:0/5463972 >> v2:172.21.15.152:3300/0 conn(0x7fa0840b5650 msgr2=0x7fa0840b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:21.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa088e52700 1 --2- 172.21.15.140:0/5463972 >> v2:172.21.15.152:3300/0 conn(0x7fa0840b5650 0x7fa0840b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa088e52700 1 -- 172.21.15.140:0/5463972 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa084147c40 con 0x7fa0840af8e0 2022-07-21T01:58:21.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa088e52700 1 --2- 172.21.15.140:0/5463972 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0840af8e0 0x7fa08405bd90 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7fa07c002f70 tx=0x7fa07c006eb0).ready entity=mon.0 client_cookie=f765e936a9d05ece server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:21.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa082ffd700 1 -- 172.21.15.140:0/5463972 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa07c00d570 con 0x7fa0840af8e0 2022-07-21T01:58:21.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa082ffd700 1 -- 172.21.15.140:0/5463972 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07c000e90 con 0x7fa0840af8e0 2022-07-21T01:58:21.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa082ffd700 1 -- 172.21.15.140:0/5463972 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa07c00d900 con 0x7fa0840af8e0 2022-07-21T01:58:21.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 -- 172.21.15.140:0/5463972 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0840af8e0 msgr2=0x7fa08405bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:21.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 --2- 172.21.15.140:0/5463972 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0840af8e0 0x7fa08405bd90 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7fa07c002f70 tx=0x7fa07c006eb0).stop 2022-07-21T01:58:21.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 -- 172.21.15.140:0/5463972 shutdown_connections 2022-07-21T01:58:21.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 --2- 172.21.15.140:0/5463972 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0840af8e0 0x7fa08405bd90 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 --2- 172.21.15.140:0/5463972 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0840adff0 0x7fa084147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 --2- 172.21.15.140:0/5463972 >> v2:172.21.15.152:3300/0 conn(0x7fa0840b5650 0x7fa0840b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 -- 172.21.15.140:0/5463972 >> 172.21.15.140:0/5463972 conn(0x7fa0840634e0 msgr2=0x7fa0840b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:21.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 -- 172.21.15.140:0/5463972 shutdown_connections 2022-07-21T01:58:21.029 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 -- 172.21.15.140:0/5463972 wait complete. 2022-07-21T01:58:21.032 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 Processor -- start 2022-07-21T01:58:21.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.019+0000 7fa089653700 1 -- start start 2022-07-21T01:58:21.033 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa089653700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0840adff0 0x7fa084110cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa089653700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0840af8e0 0x7fa0841598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.034 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa089653700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0840b5650 0x7fa08410f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa089653700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08410f9c0 con 0x7fa0840af8e0 2022-07-21T01:58:21.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa089653700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa084064c20 con 0x7fa0840adff0 2022-07-21T01:58:21.035 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa089653700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08410fbe0 con 0x7fa0840b5650 2022-07-21T01:58:21.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0840af8e0 0x7fa0841598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0837fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0840af8e0 0x7fa0841598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42076/0 (socket says 172.21.15.140:42076) 2022-07-21T01:58:21.036 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0837fe700 1 -- 172.21.15.140:0/3725820156 learned_addr learned my addr 172.21.15.140:0/3725820156 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:21.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa083fff700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0840adff0 0x7fa084110cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa088e52700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0840b5650 0x7fa08410f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.037 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0837fe700 1 -- 172.21.15.140:0/3725820156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0840b5650 msgr2=0x7fa08410f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:21.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0837fe700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0840b5650 0x7fa08410f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0837fe700 1 -- 172.21.15.140:0/3725820156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0840adff0 msgr2=0x7fa084110cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:21.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0837fe700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0840adff0 0x7fa084110cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.038 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0837fe700 1 -- 172.21.15.140:0/3725820156 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c00b1e0 con 0x7fa0840af8e0 2022-07-21T01:58:21.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0837fe700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0840af8e0 0x7fa0841598b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fa07400c000 tx=0x7fa0740091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:21.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0817fa700 1 -- 172.21.15.140:0/3725820156 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa07400c670 con 0x7fa0840af8e0 2022-07-21T01:58:21.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0817fa700 1 -- 172.21.15.140:0/3725820156 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07400c7d0 con 0x7fa0840af8e0 2022-07-21T01:58:21.039 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa089653700 1 -- 172.21.15.140:0/3725820156 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08410fe80 con 0x7fa0840af8e0 2022-07-21T01:58:21.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0817fa700 1 -- 172.21.15.140:0/3725820156 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa07400cb30 con 0x7fa0840af8e0 2022-07-21T01:58:21.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa089653700 1 -- 172.21.15.140:0/3725820156 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa084111780 con 0x7fa0840af8e0 2022-07-21T01:58:21.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa089653700 1 -- 172.21.15.140:0/3725820156 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0840574e0 con 0x7fa0840af8e0 2022-07-21T01:58:21.040 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.023+0000 7fa0817fa700 1 -- 172.21.15.140:0/3725820156 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa074022070 con 0x7fa0840af8e0 2022-07-21T01:58:21.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.027+0000 7fa0817fa700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa05c053950 0x7fa05c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.027+0000 7fa083fff700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa05c053950 0x7fa05c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.027+0000 7fa0817fa700 1 -- 172.21.15.140:0/3725820156 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa07406c590 con 0x7fa0840af8e0 2022-07-21T01:58:21.041 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.027+0000 7fa083fff700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa05c053950 0x7fa05c055e10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fa06c00c000 tx=0x7fa06c008040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:21.042 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.035+0000 7fa0817fa700 1 -- 172.21.15.140:0/3725820156 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa074043450 con 0x7fa0840af8e0 2022-07-21T01:58:21.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.199+0000 7fa089653700 1 -- 172.21.15.140:0/3725820156 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa084067ea0 con 0x7fa0840af8e0 2022-07-21T01:58:21.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa0817fa700 1 -- 172.21.15.140:0/3725820156 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2148 (secure 0 0 0) 0x7fa074016070 con 0x7fa0840af8e0 2022-07-21T01:58:21.204 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:21.205 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":74,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/3591375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591375250},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591375250}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:58:05.107197+0000","modified":"2022-07-21T01:58:06.110113+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8727":{"gid":8727,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.140:6835/4013573612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6834","nonce":4013573612},{"type":"v1","addr":"172.21.15.140:6835","nonce":4013573612}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-07-21T01:58:21.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 -- 172.21.15.140:0/3725820156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa05c053950 msgr2=0x7fa05c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:21.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa05c053950 0x7fa05c055e10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fa06c00c000 tx=0x7fa06c008040).stop 2022-07-21T01:58:21.207 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 -- 172.21.15.140:0/3725820156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0840af8e0 msgr2=0x7fa0841598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:21.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0840af8e0 0x7fa0841598b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fa07400c000 tx=0x7fa0740091a0).stop 2022-07-21T01:58:21.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 -- 172.21.15.140:0/3725820156 shutdown_connections 2022-07-21T01:58:21.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa05c053950 0x7fa05c055e10 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.208 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa0840b5650 0x7fa08410f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa0840af8e0 0x7fa0841598b0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 --2- 172.21.15.140:0/3725820156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0840adff0 0x7fa084110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 -- 172.21.15.140:0/3725820156 >> 172.21.15.140:0/3725820156 conn(0x7fa0840634e0 msgr2=0x7fa0840b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:21.209 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 -- 172.21.15.140:0/3725820156 shutdown_connections 2022-07-21T01:58:21.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.203+0000 7fa089653700 1 -- 172.21.15.140:0/3725820156 wait complete. 2022-07-21T01:58:21.210 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 74 2022-07-21T01:58:21.217 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2022-07-21T01:58:21.320 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 Processor -- start 2022-07-21T01:58:21.321 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 -- start start 2022-07-21T01:58:21.330 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6a340676f0 0x7f6a3405fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a34067ad0 con 0x7f6a340611f0 2022-07-21T01:58:21.331 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a34060330 con 0x7f6a340b1b00 2022-07-21T01:58:21.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a34061690 con 0x7f6a340676f0 2022-07-21T01:58:21.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a32ffd700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f6a340676f0 0x7f6a3405fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.332 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a327fc700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f6a340611f0 0x7f6a3414aa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56232/0 (socket says 172.21.15.140:56232) 2022-07-21T01:58:21.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a327fc700 1 -- 172.21.15.140:0/3305180806 learned_addr learned my addr 172.21.15.140:0/3305180806 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:21.333 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a32ffd700 1 -- 172.21.15.140:0/3305180806 >> v1:172.21.15.152:6789/0 conn(0x7f6a340b1b00 legacy=0x7f6a340b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:21.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a32ffd700 1 -- 172.21.15.140:0/3305180806 >> v1:172.21.15.140:6789/0 conn(0x7f6a340611f0 legacy=0x7f6a3414aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:21.334 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a32ffd700 1 -- 172.21.15.140:0/3305180806 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3414a580 con 0x7f6a340676f0 2022-07-21T01:58:21.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a32ffd700 1 --2- 172.21.15.140:0/3305180806 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a340676f0 0x7f6a3405fe40 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f6a28002f70 tx=0x7f6a28006df0).ready entity=mon.0 client_cookie=7004d9215c84a355 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:21.335 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a31ffb700 1 -- 172.21.15.140:0/3305180806 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a2800d610 con 0x7f6a340676f0 2022-07-21T01:58:21.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a31ffb700 1 -- 172.21.15.140:0/3305180806 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a28000e90 con 0x7f6a340676f0 2022-07-21T01:58:21.336 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a31ffb700 1 -- 172.21.15.140:0/3305180806 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a2800d960 con 0x7f6a340676f0 2022-07-21T01:58:21.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 -- 172.21.15.140:0/3305180806 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a340676f0 msgr2=0x7f6a3405fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:21.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 --2- 172.21.15.140:0/3305180806 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a340676f0 0x7f6a3405fe40 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7f6a28002f70 tx=0x7f6a28006df0).stop 2022-07-21T01:58:21.337 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 -- 172.21.15.140:0/3305180806 shutdown_connections 2022-07-21T01:58:21.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 --2- 172.21.15.140:0/3305180806 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a340676f0 0x7f6a3405fe40 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.338 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 -- 172.21.15.140:0/3305180806 >> 172.21.15.140:0/3305180806 conn(0x7f6a34064c00 msgr2=0x7f6a34064fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:21.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 -- 172.21.15.140:0/3305180806 shutdown_connections 2022-07-21T01:58:21.339 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.319+0000 7f6a33fff700 1 -- 172.21.15.140:0/3305180806 wait complete. 2022-07-21T01:58:21.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a33fff700 1 Processor -- start 2022-07-21T01:58:21.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a33fff700 1 -- start start 2022-07-21T01:58:21.340 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a33fff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6a340611f0 0x7f6a34145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a33fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a340676f0 0x7f6a34136770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.341 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a33fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a340b1b00 0x7f6a3414cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a33fff700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a34138110 con 0x7f6a340676f0 2022-07-21T01:58:21.342 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a33fff700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a34137e70 con 0x7f6a340b1b00 2022-07-21T01:58:21.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a33fff700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3414c7c0 con 0x7f6a340611f0 2022-07-21T01:58:21.343 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a327fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a340676f0 0x7f6a34136770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a327fc700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a340676f0 0x7f6a34136770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42092/0 (socket says 172.21.15.140:42092) 2022-07-21T01:58:21.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a32ffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6a340611f0 0x7f6a34145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.344 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a327fc700 1 -- 172.21.15.140:0/1638971870 learned_addr learned my addr 172.21.15.140:0/1638971870 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:21.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a337fe700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a340b1b00 0x7f6a3414cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.345 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a327fc700 1 -- 172.21.15.140:0/1638971870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6a340611f0 msgr2=0x7f6a34145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:21.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a327fc700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6a340611f0 0x7f6a34145ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.346 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a327fc700 1 -- 172.21.15.140:0/1638971870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a340b1b00 msgr2=0x7f6a3414cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:21.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a327fc700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a340b1b00 0x7f6a3414cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:21.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a327fc700 1 -- 172.21.15.140:0/1638971870 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2800b180 con 0x7f6a340676f0 2022-07-21T01:58:21.347 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a327fc700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a340676f0 0x7f6a34136770 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f6a3405cfb0 tx=0x7f6a24007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:21.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.323+0000 7f6a23fff700 1 -- 172.21.15.140:0/1638971870 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a240070d0 con 0x7f6a340676f0 2022-07-21T01:58:21.348 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.327+0000 7f6a23fff700 1 -- 172.21.15.140:0/1638971870 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a24007230 con 0x7f6a340676f0 2022-07-21T01:58:21.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.327+0000 7f6a33fff700 1 -- 172.21.15.140:0/1638971870 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a34136c60 con 0x7f6a340676f0 2022-07-21T01:58:21.349 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.327+0000 7f6a23fff700 1 -- 172.21.15.140:0/1638971870 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a24005610 con 0x7f6a340676f0 2022-07-21T01:58:21.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.327+0000 7f6a33fff700 1 -- 172.21.15.140:0/1638971870 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a34137a40 con 0x7f6a340676f0 2022-07-21T01:58:21.350 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.327+0000 7f6a33fff700 1 -- 172.21.15.140:0/1638971870 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3414d490 con 0x7f6a340676f0 2022-07-21T01:58:21.352 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.327+0000 7f6a23fff700 1 -- 172.21.15.140:0/1638971870 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f6a24010070 con 0x7f6a340676f0 2022-07-21T01:58:21.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.327+0000 7f6a23fff700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a10053a70 0x7f6a10055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:21.353 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.327+0000 7f6a23fff700 1 -- 172.21.15.140:0/1638971870 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6a2406b550 con 0x7f6a340676f0 2022-07-21T01:58:21.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.327+0000 7f6a32ffd700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a10053a70 0x7f6a10055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:21.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.331+0000 7f6a32ffd700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a10053a70 0x7f6a10055f30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f6a2801c000 tx=0x7f6a28015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:21.354 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.335+0000 7f6a23fff700 1 -- 172.21.15.140:0/1638971870 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f6a3414d490 con 0x7f6a340676f0 2022-07-21T01:58:21.500 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:21.499+0000 7f6a33fff700 1 -- 172.21.15.140:0/1638971870 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6a3414d490 con 0x7f6a340676f0 2022-07-21T01:58:22.076 INFO:tasks.ceph.mon.a.smithi140.stderr:2022-07-21T01:58:22.071+0000 7f72a45f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2022-07-21T01:58:22.083 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.079+0000 7f6a23fff700 1 -- 172.21.15.140:0/1638971870 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6a3414d490 con 0x7f6a340676f0 2022-07-21T01:58:22.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.083+0000 7f6a33fff700 1 -- 172.21.15.140:0/1638971870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a10053a70 msgr2=0x7f6a10055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.083+0000 7f6a33fff700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a10053a70 0x7f6a10055f30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f6a2801c000 tx=0x7f6a28015000).stop 2022-07-21T01:58:22.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.083+0000 7f6a33fff700 1 -- 172.21.15.140:0/1638971870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a340676f0 msgr2=0x7f6a34136770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.088 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.083+0000 7f6a33fff700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a340676f0 0x7f6a34136770 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f6a3405cfb0 tx=0x7f6a24007c40).stop 2022-07-21T01:58:22.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.083+0000 7f6a33fff700 1 -- 172.21.15.140:0/1638971870 shutdown_connections 2022-07-21T01:58:22.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.083+0000 7f6a33fff700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f6a10053a70 0x7f6a10055f30 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.083+0000 7f6a33fff700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6a340b1b00 0x7f6a3414cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.083+0000 7f6a33fff700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6a340676f0 0x7f6a34136770 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.089 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.083+0000 7f6a33fff700 1 --2- 172.21.15.140:0/1638971870 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6a340611f0 0x7f6a34145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.083+0000 7f6a33fff700 1 -- 172.21.15.140:0/1638971870 >> 172.21.15.140:0/1638971870 conn(0x7f6a34064c00 msgr2=0x7f6a340b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:22.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.083+0000 7f6a33fff700 1 -- 172.21.15.140:0/1638971870 shutdown_connections 2022-07-21T01:58:22.090 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.087+0000 7f6a33fff700 1 -- 172.21.15.140:0/1638971870 wait complete. 2022-07-21T01:58:22.091 INFO:teuthology.orchestra.run.smithi140.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2022-07-21T01:58:22.103 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-07-21T01:58:22.109 INFO:tasks.ceph.mds.a.smithi140.stdout:starting mds.a at 2022-07-21T01:58:22.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.195+0000 7f130359e700 1 Processor -- start 2022-07-21T01:58:22.198 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.195+0000 7f130359e700 1 -- start start 2022-07-21T01:58:22.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f13040b61d0 0x7f13040adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1304060d80 0x7f1304145b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13040b1d10 con 0x7f13040b08c0 2022-07-21T01:58:22.202 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13040ad480 con 0x7f13040b61d0 2022-07-21T01:58:22.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13040b65b0 con 0x7f1304060d80 2022-07-21T01:58:22.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f12fbfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f13040b08c0 0x7f1304145400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56256/0 (socket says 172.21.15.140:56256) 2022-07-21T01:58:22.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f12fbfff700 1 -- 172.21.15.140:0/115670440 learned_addr learned my addr 172.21.15.140:0/115670440 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:22.203 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f1302d9d700 1 --2- 172.21.15.140:0/115670440 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1304060d80 0x7f1304145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130259c700 1 --2- 172.21.15.140:0/115670440 >> v2:172.21.15.152:3300/0 conn(0x7f13040b61d0 0x7f13040adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f1301d9b700 1 -- 172.21.15.140:0/115670440 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767308481 0 0) 0x7f13040b1d10 con 0x7f13040b08c0 2022-07-21T01:58:22.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f1301d9b700 1 -- 172.21.15.140:0/115670440 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e4003260 con 0x7f13040b08c0 2022-07-21T01:58:22.204 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f1302d9d700 1 -- 172.21.15.140:0/115670440 >> v2:172.21.15.152:3300/0 conn(0x7f13040b61d0 msgr2=0x7f13040adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:22.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f1302d9d700 1 --2- 172.21.15.140:0/115670440 >> v2:172.21.15.152:3300/0 conn(0x7f13040b61d0 0x7f13040adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f1302d9d700 1 -- 172.21.15.140:0/115670440 >> v1:172.21.15.140:6789/0 conn(0x7f13040b08c0 legacy=0x7f1304145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f1302d9d700 1 -- 172.21.15.140:0/115670440 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13040b1f70 con 0x7f1304060d80 2022-07-21T01:58:22.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130259c700 1 --2- 172.21.15.140:0/115670440 >> v2:172.21.15.152:3300/0 conn(0x7f13040b61d0 0x7f13040adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T01:58:22.205 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f1302d9d700 1 --2- 172.21.15.140:0/115670440 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1304060d80 0x7f1304145b10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f12fc0032f0 tx=0x7f12fc0091d0).ready entity=mon.2 client_cookie=b305d5273bc598be server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:22.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f1301d9b700 1 -- 172.21.15.140:0/115670440 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12fc00b600 con 0x7f1304060d80 2022-07-21T01:58:22.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f1301d9b700 1 -- 172.21.15.140:0/115670440 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc00b760 con 0x7f1304060d80 2022-07-21T01:58:22.206 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f1301d9b700 1 -- 172.21.15.140:0/115670440 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12fc00b8d0 con 0x7f1304060d80 2022-07-21T01:58:22.210 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 -- 172.21.15.140:0/115670440 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1304060d80 msgr2=0x7f1304145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 --2- 172.21.15.140:0/115670440 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1304060d80 0x7f1304145b10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f12fc0032f0 tx=0x7f12fc0091d0).stop 2022-07-21T01:58:22.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 -- 172.21.15.140:0/115670440 shutdown_connections 2022-07-21T01:58:22.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 --2- 172.21.15.140:0/115670440 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1304060d80 0x7f1304145b10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.211 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 --2- 172.21.15.140:0/115670440 >> v2:172.21.15.152:3300/0 conn(0x7f13040b61d0 0x7f13040adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 -- 172.21.15.140:0/115670440 >> 172.21.15.140:0/115670440 conn(0x7f1304063cf0 msgr2=0x7f13040b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:22.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 -- 172.21.15.140:0/115670440 shutdown_connections 2022-07-21T01:58:22.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.199+0000 7f130359e700 1 -- 172.21.15.140:0/115670440 wait complete. 2022-07-21T01:58:22.212 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f130359e700 1 Processor -- start 2022-07-21T01:58:22.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f130359e700 1 -- start start 2022-07-21T01:58:22.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f130359e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1304060d80 0x7f1304144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f130359e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f13040b08c0 0x7f130412fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.213 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f130359e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13040b61d0 0x7f13041301f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f130359e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13040668d0 con 0x7f1304060d80 2022-07-21T01:58:22.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f130359e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13041314d0 con 0x7f13040b61d0 2022-07-21T01:58:22.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f130359e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1304066b50 con 0x7f13040b08c0 2022-07-21T01:58:22.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f130259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1304060d80 0x7f1304144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.214 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f130259c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1304060d80 0x7f1304144130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42114/0 (socket says 172.21.15.140:42114) 2022-07-21T01:58:22.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f130259c700 1 -- 172.21.15.140:0/3282617821 learned_addr learned my addr 172.21.15.140:0/3282617821 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:22.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f12fbfff700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f13040b08c0 0x7f130412fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.215 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.203+0000 7f1302d9d700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13040b61d0 0x7f13041301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f130259c700 1 -- 172.21.15.140:0/3282617821 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f13040b08c0 msgr2=0x7f130412fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f130259c700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f13040b08c0 0x7f130412fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f130259c700 1 -- 172.21.15.140:0/3282617821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13040b61d0 msgr2=0x7f13041301f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.216 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f130259c700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13040b61d0 0x7f13041301f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f130259c700 1 -- 172.21.15.140:0/3282617821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f400b180 con 0x7f1304060d80 2022-07-21T01:58:22.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f130259c700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1304060d80 0x7f1304144130 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f12f400c000 tx=0x7f12f4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:22.217 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f12fb7fe700 1 -- 172.21.15.140:0/3282617821 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f4010780 con 0x7f1304060d80 2022-07-21T01:58:22.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f12fb7fe700 1 -- 172.21.15.140:0/3282617821 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f40108e0 con 0x7f1304060d80 2022-07-21T01:58:22.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f130359e700 1 -- 172.21.15.140:0/3282617821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc009470 con 0x7f1304060d80 2022-07-21T01:58:22.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f12fb7fe700 1 -- 172.21.15.140:0/3282617821 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12f4010c40 con 0x7f1304060d80 2022-07-21T01:58:22.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f130359e700 1 -- 172.21.15.140:0/3282617821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1304067090 con 0x7f1304060d80 2022-07-21T01:58:22.218 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f130359e700 1 -- 172.21.15.140:0/3282617821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1304151d50 con 0x7f1304060d80 2022-07-21T01:58:22.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f12fb7fe700 1 -- 172.21.15.140:0/3282617821 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f12f401c460 con 0x7f1304060d80 2022-07-21T01:58:22.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f12fb7fe700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f12e80539e0 0x7f12e8055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f12fb7fe700 1 -- 172.21.15.140:0/3282617821 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f12f406c4c0 con 0x7f1304060d80 2022-07-21T01:58:22.219 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.207+0000 7f12fbfff700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f12e80539e0 0x7f12e8055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.211+0000 7f12fbfff700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f12e80539e0 0x7f12e8055ea0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f12ec00d000 tx=0x7f12ec009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:22.221 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.215+0000 7f12fb7fe700 1 -- 172.21.15.140:0/3282617821 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f12f4043290 con 0x7f1304060d80 2022-07-21T01:58:22.380 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.379+0000 7f130359e700 1 -- 172.21.15.140:0/3282617821 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13040574e0 con 0x7f1304060d80 2022-07-21T01:58:22.381 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.379+0000 7f12fb7fe700 1 -- 172.21.15.140:0/3282617821 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+1785 (secure 0 0 0) 0x7f12f40468b0 con 0x7f1304060d80 2022-07-21T01:58:22.382 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T01:58:22.382 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":75,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8617,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.152:6833/3591375250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591375250},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591375250}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73}],"filesystems":[{"mdsmap":{"epoch":75,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-07-21T01:58:05.107197+0000","modified":"2022-07-21T01:58:22.076691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":152,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2022-07-21T01:58:22.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 -- 172.21.15.140:0/3282617821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f12e80539e0 msgr2=0x7f12e8055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f12e80539e0 0x7f12e8055ea0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f12ec00d000 tx=0x7f12ec009040).stop 2022-07-21T01:58:22.386 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 -- 172.21.15.140:0/3282617821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1304060d80 msgr2=0x7f1304144130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1304060d80 0x7f1304144130 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f12f400c000 tx=0x7f12f4006df0).stop 2022-07-21T01:58:22.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 -- 172.21.15.140:0/3282617821 shutdown_connections 2022-07-21T01:58:22.387 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f12e80539e0 0x7f12e8055ea0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13040b61d0 0x7f13041301f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f13040b08c0 0x7f130412fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 --2- 172.21.15.140:0/3282617821 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1304060d80 0x7f1304144130 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 -- 172.21.15.140:0/3282617821 >> 172.21.15.140:0/3282617821 conn(0x7f1304063cf0 msgr2=0x7f13041451d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:22.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 -- 172.21.15.140:0/3282617821 shutdown_connections 2022-07-21T01:58:22.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.383+0000 7f130359e700 1 -- 172.21.15.140:0/3282617821 wait complete. 2022-07-21T01:58:22.390 INFO:teuthology.orchestra.run.smithi140.stderr:dumped fsmap epoch 75 2022-07-21T01:58:22.399 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2022-07-21T01:58:22.501 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809f81700 1 Processor -- start 2022-07-21T01:58:22.502 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809f81700 1 -- start start 2022-07-21T01:58:22.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809f81700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd804146650 0x7fd8040b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.513 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809f81700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8040acf80 0x7fd8041417f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809f81700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8041428a0 con 0x7fd8040b6800 2022-07-21T01:58:22.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809f81700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8040b4120 con 0x7fd804146650 2022-07-21T01:58:22.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809f81700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040b6be0 con 0x7fd8040acf80 2022-07-21T01:58:22.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd808f7f700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fd8040b6800 0x7fd804145f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56270/0 (socket says 172.21.15.140:56270) 2022-07-21T01:58:22.515 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd808f7f700 1 -- 172.21.15.140:0/570102640 learned_addr learned my addr 172.21.15.140:0/570102640 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:22.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809780700 1 --2- 172.21.15.140:0/570102640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8040acf80 0x7fd8041417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.516 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd803fff700 1 --2- 172.21.15.140:0/570102640 >> v2:172.21.15.152:3300/0 conn(0x7fd804146650 0x7fd8040b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd8037fe700 1 -- 172.21.15.140:0/570102640 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964264718 0 0) 0x7fd7f4001110 con 0x7fd8040b6800 2022-07-21T01:58:22.517 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd8037fe700 1 -- 172.21.15.140:0/570102640 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e8003260 con 0x7fd8040b6800 2022-07-21T01:58:22.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809780700 1 -- 172.21.15.140:0/570102640 >> v2:172.21.15.152:3300/0 conn(0x7fd804146650 msgr2=0x7fd8040b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T01:58:22.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809780700 1 --2- 172.21.15.140:0/570102640 >> v2:172.21.15.152:3300/0 conn(0x7fd804146650 0x7fd8040b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809780700 1 -- 172.21.15.140:0/570102640 >> v1:172.21.15.140:6789/0 conn(0x7fd8040b6800 legacy=0x7fd804145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.499+0000 7fd809780700 1 -- 172.21.15.140:0/570102640 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8040b4800 con 0x7fd8040acf80 2022-07-21T01:58:22.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd809780700 1 --2- 172.21.15.140:0/570102640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8040acf80 0x7fd8041417f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fd7fc002f70 tx=0x7fd7fc006df0).ready entity=mon.2 client_cookie=45cf1e8faa83922f server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:22.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd8037fe700 1 -- 172.21.15.140:0/570102640 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7fc00d4a0 con 0x7fd8040acf80 2022-07-21T01:58:22.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd8037fe700 1 -- 172.21.15.140:0/570102640 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc000e90 con 0x7fd8040acf80 2022-07-21T01:58:22.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd8037fe700 1 -- 172.21.15.140:0/570102640 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7fc00d7f0 con 0x7fd8040acf80 2022-07-21T01:58:22.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd809f81700 1 -- 172.21.15.140:0/570102640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8040acf80 msgr2=0x7fd8041417f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd809f81700 1 --2- 172.21.15.140:0/570102640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8040acf80 0x7fd8041417f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fd7fc002f70 tx=0x7fd7fc006df0).stop 2022-07-21T01:58:22.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd809f81700 1 -- 172.21.15.140:0/570102640 shutdown_connections 2022-07-21T01:58:22.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd809f81700 1 --2- 172.21.15.140:0/570102640 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8040acf80 0x7fd8041417f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd809f81700 1 --2- 172.21.15.140:0/570102640 >> v2:172.21.15.152:3300/0 conn(0x7fd804146650 0x7fd8040b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd809f81700 1 -- 172.21.15.140:0/570102640 >> 172.21.15.140:0/570102640 conn(0x7fd8040b9de0 msgr2=0x7fd8040b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:22.523 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd809f81700 1 -- 172.21.15.140:0/570102640 shutdown_connections 2022-07-21T01:58:22.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd809f81700 1 -- 172.21.15.140:0/570102640 wait complete. 2022-07-21T01:58:22.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.503+0000 7fd809f81700 1 Processor -- start 2022-07-21T01:58:22.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd809f81700 1 -- start start 2022-07-21T01:58:22.524 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd809f81700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8040acf80 0x7fd8040b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd809f81700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8040b6800 0x7fd804155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd809f81700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd804146650 0x7fd804113890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd809f81700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd804114ba0 con 0x7fd8040acf80 2022-07-21T01:58:22.525 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd809f81700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd804116020 con 0x7fd8040b6800 2022-07-21T01:58:22.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd809f81700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040b3810 con 0x7fd804146650 2022-07-21T01:58:22.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd808f7f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8040acf80 0x7fd8040b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.526 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd808f7f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8040acf80 0x7fd8040b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42128/0 (socket says 172.21.15.140:42128) 2022-07-21T01:58:22.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd808f7f700 1 -- 172.21.15.140:0/3012843394 learned_addr learned my addr 172.21.15.140:0/3012843394 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:22.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd809780700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd804146650 0x7fd804113890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.528 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd803fff700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8040b6800 0x7fd804155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd808f7f700 1 -- 172.21.15.140:0/3012843394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd804146650 msgr2=0x7fd804113890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.529 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd808f7f700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd804146650 0x7fd804113890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd808f7f700 1 -- 172.21.15.140:0/3012843394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8040b6800 msgr2=0x7fd804155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:22.530 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd808f7f700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8040b6800 0x7fd804155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:22.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd808f7f700 1 -- 172.21.15.140:0/3012843394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc00b180 con 0x7fd8040acf80 2022-07-21T01:58:22.531 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd808f7f700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8040acf80 0x7fd8040b09f0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7fd7f40012a0 tx=0x7fd7f4007940).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:22.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd801ffb700 1 -- 172.21.15.140:0/3012843394 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7f40070d0 con 0x7fd8040acf80 2022-07-21T01:58:22.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd801ffb700 1 -- 172.21.15.140:0/3012843394 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f4007230 con 0x7fd8040acf80 2022-07-21T01:58:22.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd809f81700 1 -- 172.21.15.140:0/3012843394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd804116250 con 0x7fd8040acf80 2022-07-21T01:58:22.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd801ffb700 1 -- 172.21.15.140:0/3012843394 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7f4005610 con 0x7fd8040acf80 2022-07-21T01:58:22.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.507+0000 7fd809f81700 1 -- 172.21.15.140:0/3012843394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8040b32d0 con 0x7fd8040acf80 2022-07-21T01:58:22.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.511+0000 7fd809f81700 1 -- 172.21.15.140:0/3012843394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8040b4aa0 con 0x7fd8040acf80 2022-07-21T01:58:22.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.511+0000 7fd801ffb700 1 -- 172.21.15.140:0/3012843394 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fd7f401a060 con 0x7fd8040acf80 2022-07-21T01:58:22.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.511+0000 7fd801ffb700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7dc053b00 0x7fd7dc055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:22.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.511+0000 7fd801ffb700 1 -- 172.21.15.140:0/3012843394 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd7f406b4c0 con 0x7fd8040acf80 2022-07-21T01:58:22.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.511+0000 7fd803fff700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7dc053b00 0x7fd7dc055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:22.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.511+0000 7fd803fff700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7dc053b00 0x7fd7dc055fc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fd7ec00b000 tx=0x7fd7ec0091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:22.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.519+0000 7fd801ffb700 1 -- 172.21.15.140:0/3012843394 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fd7f4042390 con 0x7fd8040acf80 2022-07-21T01:58:22.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:22.675+0000 7fd809f81700 1 -- 172.21.15.140:0/3012843394 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd804142880 con 0x7fd8040acf80 2022-07-21T01:58:23.514 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.511+0000 7fd801ffb700 1 -- 172.21.15.140:0/3012843394 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd7f4018020 con 0x7fd8040acf80 2022-07-21T01:58:23.518 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 -- 172.21.15.140:0/3012843394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7dc053b00 msgr2=0x7fd7dc055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:23.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7dc053b00 0x7fd7dc055fc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fd7ec00b000 tx=0x7fd7ec0091d0).stop 2022-07-21T01:58:23.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 -- 172.21.15.140:0/3012843394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8040acf80 msgr2=0x7fd8040b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:23.519 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8040acf80 0x7fd8040b09f0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7fd7f40012a0 tx=0x7fd7f4007940).stop 2022-07-21T01:58:23.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 -- 172.21.15.140:0/3012843394 shutdown_connections 2022-07-21T01:58:23.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fd7dc053b00 0x7fd7dc055fc0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:23.520 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd804146650 0x7fd804113890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:23.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8040b6800 0x7fd804155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:23.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 --2- 172.21.15.140:0/3012843394 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fd8040acf80 0x7fd8040b09f0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:23.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 -- 172.21.15.140:0/3012843394 >> 172.21.15.140:0/3012843394 conn(0x7fd8040b9de0 msgr2=0x7fd8040b7210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:23.521 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 -- 172.21.15.140:0/3012843394 shutdown_connections 2022-07-21T01:58:23.522 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.515+0000 7fd809f81700 1 -- 172.21.15.140:0/3012843394 wait complete. 2022-07-21T01:58:23.534 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2022-07-21T01:58:23.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.651+0000 7f872b59e700 1 Processor -- start 2022-07-21T01:58:23.653 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.651+0000 7f872b59e700 1 -- start start 2022-07-21T01:58:23.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.651+0000 7f872b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f872c0b1500 0x7f872c05fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:23.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.651+0000 7f872b59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872c067ad0 con 0x7f872c0676f0 2022-07-21T01:58:23.655 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.651+0000 7f872b59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872c060330 con 0x7f872c0611f0 2022-07-21T01:58:23.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.651+0000 7f872b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c061690 con 0x7f872c0b1500 2022-07-21T01:58:23.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.651+0000 7f872a59c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f872c0676f0 0x7f872c14aa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56282/0 (socket says 172.21.15.140:56282) 2022-07-21T01:58:23.656 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.651+0000 7f872a59c700 1 -- 172.21.15.140:0/3796628076 learned_addr learned my addr 172.21.15.140:0/3796628076 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:23.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.651+0000 7f872ad9d700 1 --2- 172.21.15.140:0/3796628076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f872c0b1500 0x7f872c05fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:23.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.651+0000 7f872959a700 1 -- 172.21.15.140:0/3796628076 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729044513 0 0) 0x7f872c067ad0 con 0x7f872c0676f0 2022-07-21T01:58:23.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.651+0000 7f872959a700 1 -- 172.21.15.140:0/3796628076 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f870c003260 con 0x7f872c0676f0 2022-07-21T01:58:23.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872959a700 1 -- 172.21.15.140:0/3796628076 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903783801 0 0) 0x7f872c060330 con 0x7f872c0611f0 2022-07-21T01:58:23.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872959a700 1 -- 172.21.15.140:0/3796628076 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f872c067ad0 con 0x7f872c0611f0 2022-07-21T01:58:23.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872959a700 1 -- 172.21.15.140:0/3796628076 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2209269714 0 0) 0x7f870c003260 con 0x7f872c0676f0 2022-07-21T01:58:23.660 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872ad9d700 1 -- 172.21.15.140:0/3796628076 >> v1:172.21.15.152:6789/0 conn(0x7f872c0611f0 legacy=0x7f872c0b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:23.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872ad9d700 1 -- 172.21.15.140:0/3796628076 >> v1:172.21.15.140:6789/0 conn(0x7f872c0676f0 legacy=0x7f872c14aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:23.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872ad9d700 1 -- 172.21.15.140:0/3796628076 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c14a580 con 0x7f872c0b1500 2022-07-21T01:58:23.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872ad9d700 1 --2- 172.21.15.140:0/3796628076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f872c0b1500 0x7f872c05fe40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f871c001110 tx=0x7f87240091d0).ready entity=mon.2 client_cookie=bd71222e27347c74 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:23.661 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872959a700 1 -- 172.21.15.140:0/3796628076 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f871c001660 con 0x7f872c0b1500 2022-07-21T01:58:23.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872959a700 1 -- 172.21.15.140:0/3796628076 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872400f510 con 0x7f872c0b1500 2022-07-21T01:58:23.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872959a700 1 -- 172.21.15.140:0/3796628076 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f872400f680 con 0x7f872c0b1500 2022-07-21T01:58:23.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872b59e700 1 -- 172.21.15.140:0/3796628076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f872c0b1500 msgr2=0x7f872c05fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:23.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872b59e700 1 --2- 172.21.15.140:0/3796628076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f872c0b1500 0x7f872c05fe40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f871c001110 tx=0x7f87240091d0).stop 2022-07-21T01:58:23.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872b59e700 1 -- 172.21.15.140:0/3796628076 shutdown_connections 2022-07-21T01:58:23.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872b59e700 1 --2- 172.21.15.140:0/3796628076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f872c0b1500 0x7f872c05fe40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:23.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872b59e700 1 -- 172.21.15.140:0/3796628076 >> 172.21.15.140:0/3796628076 conn(0x7f872c064c00 msgr2=0x7f872c064fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:23.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872b59e700 1 -- 172.21.15.140:0/3796628076 shutdown_connections 2022-07-21T01:58:23.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872b59e700 1 -- 172.21.15.140:0/3796628076 wait complete. 2022-07-21T01:58:23.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872b59e700 1 Processor -- start 2022-07-21T01:58:23.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.655+0000 7f872b59e700 1 -- start start 2022-07-21T01:58:23.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f872b59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f872c0611f0 0x7f872c0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:23.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f872b59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f872c0676f0 0x7f872c14b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:23.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f872b59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f872c0b1500 0x7f872c1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:23.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f872b59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c0acae0 con 0x7f872c0676f0 2022-07-21T01:58:23.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f872b59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c14ad40 con 0x7f872c0b1500 2022-07-21T01:58:23.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f872b59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c0acd10 con 0x7f872c0611f0 2022-07-21T01:58:23.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f8729d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f872c0676f0 0x7f872c14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:23.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f8729d9b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f872c0676f0 0x7f872c14b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42142/0 (socket says 172.21.15.140:42142) 2022-07-21T01:58:23.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f8729d9b700 1 -- 172.21.15.140:0/663638075 learned_addr learned my addr 172.21.15.140:0/663638075 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:23.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f872a59c700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f872c0611f0 0x7f872c0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:23.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f872ad9d700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f872c0b1500 0x7f872c1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:23.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f8729d9b700 1 -- 172.21.15.140:0/663638075 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f872c0611f0 msgr2=0x7f872c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:23.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f8729d9b700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f872c0611f0 0x7f872c0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:23.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f8729d9b700 1 -- 172.21.15.140:0/663638075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f872c0b1500 msgr2=0x7f872c1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:23.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f8729d9b700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f872c0b1500 0x7f872c1564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:23.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f8729d9b700 1 -- 172.21.15.140:0/663638075 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8724009470 con 0x7f872c0676f0 2022-07-21T01:58:23.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f8729d9b700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f872c0676f0 0x7f872c14b440 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f8714001110 tx=0x7f8714007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:23.671 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f87237fe700 1 -- 172.21.15.140:0/663638075 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f871400f470 con 0x7f872c0676f0 2022-07-21T01:58:23.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f87237fe700 1 -- 172.21.15.140:0/663638075 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871400f5d0 con 0x7f872c0676f0 2022-07-21T01:58:23.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f872b59e700 1 -- 172.21.15.140:0/663638075 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872c14afa0 con 0x7f872c0676f0 2022-07-21T01:58:23.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f87237fe700 1 -- 172.21.15.140:0/663638075 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f871400f930 con 0x7f872c0676f0 2022-07-21T01:58:23.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.659+0000 7f872b59e700 1 -- 172.21.15.140:0/663638075 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872c145450 con 0x7f872c0676f0 2022-07-21T01:58:23.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.663+0000 7f87237fe700 1 -- 172.21.15.140:0/663638075 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f8714019030 con 0x7f872c0676f0 2022-07-21T01:58:23.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.663+0000 7f872b59e700 1 -- 172.21.15.140:0/663638075 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872c14aae0 con 0x7f872c0676f0 2022-07-21T01:58:23.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.663+0000 7f87237fe700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8710053a70 0x7f8710055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:23.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.663+0000 7f87237fe700 1 -- 172.21.15.140:0/663638075 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f871406b5e0 con 0x7f872c0676f0 2022-07-21T01:58:23.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.663+0000 7f872a59c700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8710053a70 0x7f8710055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:23.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.663+0000 7f872a59c700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8710053a70 0x7f8710055f30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f871c00b000 tx=0x7f871c000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:23.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.671+0000 7f87237fe700 1 -- 172.21.15.140:0/663638075 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f8714042320 con 0x7f872c0676f0 2022-07-21T01:58:23.803 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:23.798+0000 7f872b59e700 1 -- 172.21.15.140:0/663638075 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f872c0574e0 con 0x7f872c0676f0 2022-07-21T01:58:24.527 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.522+0000 7f87237fe700 1 -- 172.21.15.140:0/663638075 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8714045940 con 0x7f872c0676f0 2022-07-21T01:58:24.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 -- 172.21.15.140:0/663638075 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8710053a70 msgr2=0x7f8710055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:24.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8710053a70 0x7f8710055f30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7f871c00b000 tx=0x7f871c000bc0).stop 2022-07-21T01:58:24.532 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 -- 172.21.15.140:0/663638075 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f872c0676f0 msgr2=0x7f872c14b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:24.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f872c0676f0 0x7f872c14b440 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f8714001110 tx=0x7f8714007ae0).stop 2022-07-21T01:58:24.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 -- 172.21.15.140:0/663638075 shutdown_connections 2022-07-21T01:58:24.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f8710053a70 0x7f8710055f30 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:24.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f872c0b1500 0x7f872c1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:24.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f872c0676f0 0x7f872c14b440 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:24.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 --2- 172.21.15.140:0/663638075 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f872c0611f0 0x7f872c0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:24.534 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 -- 172.21.15.140:0/663638075 >> 172.21.15.140:0/663638075 conn(0x7f872c064c00 msgr2=0x7f872c05bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:24.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 -- 172.21.15.140:0/663638075 shutdown_connections 2022-07-21T01:58:24.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.526+0000 7f872b59e700 1 -- 172.21.15.140:0/663638075 wait complete. 2022-07-21T01:58:24.535 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_metadata' removed 2022-07-21T01:58:24.544 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2022-07-21T01:58:24.662 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.658+0000 7f5be128a700 1 Processor -- start 2022-07-21T01:58:24.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.658+0000 7f5be128a700 1 -- start start 2022-07-21T01:58:24.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.658+0000 7f5be128a700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5bdc098ae0 0x7f5bdc0abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:24.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.658+0000 7f5be128a700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0aba30 con 0x7f5bdc0ac470 2022-07-21T01:58:24.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.658+0000 7f5be128a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0acbc0 con 0x7f5bdc0b35c0 2022-07-21T01:58:24.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.658+0000 7f5be128a700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc0b39a0 con 0x7f5bdc098ae0 2022-07-21T01:58:24.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.658+0000 7f5bd3fff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f5bdc098ae0 0x7f5bdc0abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:24.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.658+0000 7f5be0a89700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f5bdc0ac470 0x7f5bdc143120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56300/0 (socket says 172.21.15.140:56300) 2022-07-21T01:58:24.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.658+0000 7f5be0a89700 1 -- 172.21.15.140:0/829952495 learned_addr learned my addr 172.21.15.140:0/829952495 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T01:58:24.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bd3fff700 1 -- 172.21.15.140:0/829952495 >> v1:172.21.15.152:6789/0 conn(0x7f5bdc0b35c0 legacy=0x7f5bdc061440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:24.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bd3fff700 1 -- 172.21.15.140:0/829952495 >> v1:172.21.15.140:6789/0 conn(0x7f5bdc0ac470 legacy=0x7f5bdc143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:24.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bd3fff700 1 -- 172.21.15.140:0/829952495 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0ace60 con 0x7f5bdc098ae0 2022-07-21T01:58:24.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bd3fff700 1 --2- 172.21.15.140:0/829952495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5bdc098ae0 0x7f5bdc0abe60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f5bdc0602b0 tx=0x7f5bc4006df0).ready entity=mon.0 client_cookie=ba77e22f3f687cb1 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:24.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bdb7fe700 1 -- 172.21.15.140:0/829952495 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bc400fa10 con 0x7f5bdc098ae0 2022-07-21T01:58:24.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bdb7fe700 1 -- 172.21.15.140:0/829952495 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4000e90 con 0x7f5bdc098ae0 2022-07-21T01:58:24.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bdb7fe700 1 -- 172.21.15.140:0/829952495 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bc400fd60 con 0x7f5bdc098ae0 2022-07-21T01:58:24.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 -- 172.21.15.140:0/829952495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5bdc098ae0 msgr2=0x7f5bdc0abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:24.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 --2- 172.21.15.140:0/829952495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5bdc098ae0 0x7f5bdc0abe60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f5bdc0602b0 tx=0x7f5bc4006df0).stop 2022-07-21T01:58:24.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 -- 172.21.15.140:0/829952495 shutdown_connections 2022-07-21T01:58:24.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 --2- 172.21.15.140:0/829952495 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5bdc098ae0 0x7f5bdc0abe60 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:24.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 -- 172.21.15.140:0/829952495 >> 172.21.15.140:0/829952495 conn(0x7f5bdc0b6b20 msgr2=0x7f5bdc0b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:24.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 -- 172.21.15.140:0/829952495 shutdown_connections 2022-07-21T01:58:24.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 -- 172.21.15.140:0/829952495 wait complete. 2022-07-21T01:58:24.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 Processor -- start 2022-07-21T01:58:24.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 -- start start 2022-07-21T01:58:24.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bdc098ae0 0x7f5bdc066b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:24.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5bdc0ac470 0x7f5bdc097fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:24.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5bdc0b35c0 0x7f5bdc155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:24.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0adb00 con 0x7f5bdc0ac470 2022-07-21T01:58:24.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0ae350 con 0x7f5bdc098ae0 2022-07-21T01:58:24.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be128a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc067060 con 0x7f5bdc0b35c0 2022-07-21T01:58:24.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bd3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5bdc0ac470 0x7f5bdc097fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:24.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bd3fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5bdc0ac470 0x7f5bdc097fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42158/0 (socket says 172.21.15.140:42158) 2022-07-21T01:58:24.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bd3fff700 1 -- 172.21.15.140:0/3367635584 learned_addr learned my addr 172.21.15.140:0/3367635584 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T01:58:24.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5be0a89700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5bdc0b35c0 0x7f5bdc155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:24.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bdbfff700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bdc098ae0 0x7f5bdc066b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:24.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bd3fff700 1 -- 172.21.15.140:0/3367635584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5bdc0b35c0 msgr2=0x7f5bdc155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:24.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.662+0000 7f5bd3fff700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5bdc0b35c0 0x7f5bdc155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:24.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bd3fff700 1 -- 172.21.15.140:0/3367635584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bdc098ae0 msgr2=0x7f5bdc066b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:24.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bd3fff700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bdc098ae0 0x7f5bdc066b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:24.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bd3fff700 1 -- 172.21.15.140:0/3367635584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc400b180 con 0x7f5bdc0ac470 2022-07-21T01:58:24.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bd3fff700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5bdc0ac470 0x7f5bdc097fb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f5bdc0602b0 tx=0x7f5bc4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:24.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bd9ffb700 1 -- 172.21.15.140:0/3367635584 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bc40185f0 con 0x7f5bdc0ac470 2022-07-21T01:58:24.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bd9ffb700 1 -- 172.21.15.140:0/3367635584 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc400f4a0 con 0x7f5bdc0ac470 2022-07-21T01:58:24.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bd9ffb700 1 -- 172.21.15.140:0/3367635584 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bc4018940 con 0x7f5bdc0ac470 2022-07-21T01:58:24.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5be128a700 1 -- 172.21.15.140:0/3367635584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc0b02f0 con 0x7f5bdc0ac470 2022-07-21T01:58:24.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5be128a700 1 -- 172.21.15.140:0/3367635584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc0aef60 con 0x7f5bdc0ac470 2022-07-21T01:58:24.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5be128a700 1 -- 172.21.15.140:0/3367635584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc0adc70 con 0x7f5bdc0ac470 2022-07-21T01:58:24.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bd9ffb700 1 -- 172.21.15.140:0/3367635584 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5bc402b070 con 0x7f5bdc0ac470 2022-07-21T01:58:24.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bd9ffb700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bc0053aa0 0x7f5bc0055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T01:58:24.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bd9ffb700 1 -- 172.21.15.140:0/3367635584 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f5bc406d230 con 0x7f5bdc0ac470 2022-07-21T01:58:24.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bdbfff700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bc0053aa0 0x7f5bc0055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T01:58:24.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.666+0000 7f5bdbfff700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bc0053aa0 0x7f5bc0055f60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f5bcc00d000 tx=0x7f5bcc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T01:58:24.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.674+0000 7f5bd9ffb700 1 -- 172.21.15.140:0/3367635584 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5bc40442b0 con 0x7f5bdc0ac470 2022-07-21T01:58:24.802 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:24.798+0000 7f5be128a700 1 -- 172.21.15.140:0/3367635584 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5bdc0ad100 con 0x7f5bdc0ac470 2022-07-21T01:58:25.533 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.526+0000 7f5bd9ffb700 1 -- 172.21.15.140:0/3367635584 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) v1 ==== 173+0+0 (secure 0 0 0) 0x7f5bc4002f50 con 0x7f5bdc0ac470 2022-07-21T01:58:25.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 -- 172.21.15.140:0/3367635584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bc0053aa0 msgr2=0x7f5bc0055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:25.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bc0053aa0 0x7f5bc0055f60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f5bcc00d000 tx=0x7f5bcc009040).stop 2022-07-21T01:58:25.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 -- 172.21.15.140:0/3367635584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5bdc0ac470 msgr2=0x7f5bdc097fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T01:58:25.535 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5bdc0ac470 0x7f5bdc097fb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f5bdc0602b0 tx=0x7f5bc4017000).stop 2022-07-21T01:58:25.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 -- 172.21.15.140:0/3367635584 shutdown_connections 2022-07-21T01:58:25.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5bc0053aa0 0x7f5bc0055f60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:25.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5bdc0b35c0 0x7f5bdc155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:25.536 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f5bdc0ac470 0x7f5bdc097fb0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:25.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 --2- 172.21.15.140:0/3367635584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bdc098ae0 0x7f5bdc066b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T01:58:25.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 -- 172.21.15.140:0/3367635584 >> 172.21.15.140:0/3367635584 conn(0x7f5bdc0b6b20 msgr2=0x7f5bdc09b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T01:58:25.537 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 -- 172.21.15.140:0/3367635584 shutdown_connections 2022-07-21T01:58:25.538 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T01:58:25.530+0000 7f5be128a700 1 -- 172.21.15.140:0/3367635584 wait complete. 2022-07-21T01:58:25.538 INFO:teuthology.orchestra.run.smithi140.stderr:pool 'cephfs_data' removed 2022-07-21T01:58:25.549 INFO:teuthology.orchestra.run:Running command with timeout 60 2022-07-21T01:58:25.550 DEBUG:teuthology.orchestra.run.smithi191:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2022-07-21T01:58:32.612 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:58:32.615 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:58:32.618 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:58:32.621 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:58:32.626 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:58:55.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T01:58:55.655 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T01:58:55.656 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T01:59:02.674 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:59:02.677 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:59:02.680 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:59:02.683 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:59:02.685 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:59:25.558 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-07-21T01:59:25.566 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-07-21T01:59:32.751 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:59:32.755 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:59:32.757 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:59:32.760 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T01:59:32.763 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:00:02.817 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:00:02.821 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:00:02.826 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:00:02.830 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:00:02.832 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:00:32.930 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:00:32.934 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:00:32.937 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:00:32.940 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:00:32.942 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:01:02.994 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:01:02.998 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:01:03.001 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:01:03.003 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:01:03.006 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:01:33.151 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:01:33.155 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:01:33.159 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:01:33.162 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:01:33.165 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:02:03.213 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:02:03.217 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:02:03.220 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:02:03.223 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:02:03.225 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:02:33.281 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:02:33.288 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:02:33.292 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:02:33.298 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:02:33.300 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:03:03.367 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:03:03.376 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:03:03.379 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:03:03.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:03:03.384 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:03:33.437 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:03:33.442 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:03:33.449 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:03:33.454 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:03:33.459 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:04:03.514 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:04:03.518 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:04:03.521 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:04:03.526 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:04:03.529 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:04:33.587 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:04:33.591 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:04:33.595 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:04:33.599 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:04:33.601 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:05:03.660 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:05:03.666 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:05:03.670 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:05:03.674 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:05:03.677 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:05:33.744 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:05:33.749 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:05:33.759 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:05:33.762 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:05:33.766 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:06:03.830 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:06:03.834 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:06:03.837 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:06:03.841 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:06:03.845 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:06:33.897 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:06:33.902 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:06:33.908 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:06:33.914 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:06:33.918 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:07:03.970 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:07:03.974 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:07:03.977 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:07:03.980 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:07:03.982 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:07:34.033 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:07:34.037 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:07:34.039 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:07:34.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:07:34.046 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:08:04.104 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:08:04.108 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:08:04.111 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:08:04.114 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:08:04.116 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:08:34.167 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:08:34.175 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:08:34.178 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:08:34.181 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:08:34.184 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:09:04.268 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:09:04.274 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:09:04.278 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:09:04.282 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:09:04.286 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:09:34.360 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:09:34.364 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:09:34.367 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:09:34.369 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:09:34.372 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:10:04.570 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:10:04.575 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:10:04.578 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:10:04.581 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:10:04.584 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:10:34.650 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:10:34.654 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:10:34.658 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:10:34.661 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:10:34.664 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:11:04.742 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:11:04.752 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:11:04.761 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:11:04.778 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:11:04.786 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:11:34.877 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:11:34.880 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:11:34.884 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:11:34.886 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:11:34.889 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:12:04.961 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:12:04.966 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:12:04.970 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:12:04.974 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:12:04.978 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:12:35.073 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:12:35.077 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:12:35.080 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:12:35.083 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:12:35.086 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:13:05.137 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:13:05.142 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:13:05.146 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:13:05.149 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:13:05.153 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:13:35.213 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:13:35.217 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:13:35.220 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:13:35.224 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:13:35.226 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:13:55.661 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-07-21T02:13:55.664 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-07-21T02:13:55.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T02:13:55.676 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2022-07-21T02:13:55.677 DEBUG:teuthology.orchestra.run.smithi140:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T02:13:55.700 DEBUG:teuthology.orchestra.run.smithi152:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T02:13:55.722 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T02:13:55.744 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T02:13:55.766 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2022-07-21T02:13:55.788 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2022-07-21T02:13:55.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T02:13:55.789 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T02:13:55.790 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T02:14:05.337 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:14:05.341 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:14:05.343 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:14:05.346 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:14:05.349 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:14:35.399 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:14:35.403 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:14:35.405 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:14:35.408 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:14:35.411 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:15:05.461 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:15:05.465 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:15:05.467 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:15:05.470 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:15:05.472 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:15:35.548 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:15:35.562 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:15:35.567 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:15:35.572 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:15:35.576 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:16:05.633 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:16:05.637 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:16:05.640 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:16:05.643 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:16:05.646 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:16:35.699 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:16:35.703 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:16:35.705 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:16:35.708 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:16:35.710 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:17:05.761 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:17:05.765 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:17:05.768 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:17:05.772 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:17:05.775 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:17:35.820 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:17:35.827 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:17:35.830 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:17:35.834 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:17:35.837 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:18:05.907 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:18:05.919 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:18:05.923 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:18:05.935 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:18:05.945 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:18:36.093 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:18:36.106 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:18:36.110 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:18:36.113 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:18:36.120 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:19:06.174 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:19:06.180 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:19:06.184 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:19:06.188 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:19:06.194 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:19:36.244 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:19:36.248 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:19:36.251 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:19:36.253 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:19:36.256 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:20:06.341 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:20:06.345 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:20:06.347 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:20:06.350 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:20:06.353 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:20:36.431 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:20:36.438 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:20:36.442 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:20:36.446 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:20:36.456 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:21:06.619 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:21:06.622 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:21:06.625 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:21:06.629 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:21:06.633 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:21:36.687 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:21:36.691 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:21:36.694 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:21:36.696 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:21:36.699 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:22:06.755 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:22:06.759 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:22:06.761 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:22:06.764 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:22:06.766 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:22:36.824 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:22:36.831 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:22:36.835 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:22:36.840 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:22:36.844 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:23:06.929 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:23:06.933 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:23:06.935 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:23:06.938 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:23:06.942 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:23:36.998 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:23:37.002 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:23:37.006 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:23:37.009 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:23:37.011 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:24:07.062 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:24:07.069 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:24:07.072 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:24:07.076 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:24:07.079 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:24:37.246 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:24:37.257 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:24:37.265 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:24:37.273 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:24:37.280 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:25:07.355 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:25:07.362 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:25:07.366 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:25:07.369 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:25:07.374 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:25:37.504 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:25:37.515 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:25:37.518 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:25:37.522 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:25:37.525 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:26:07.583 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:26:07.588 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:26:07.590 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:26:07.599 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:26:07.602 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:26:37.720 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:26:37.728 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:26:37.733 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:26:37.737 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:26:37.746 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:27:07.828 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:27:07.845 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:27:07.848 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:27:07.852 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:27:07.855 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:27:37.958 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:27:37.971 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:27:37.977 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:27:37.980 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:27:37.999 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:28:08.108 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:28:08.125 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:28:08.129 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:28:08.135 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:28:08.138 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:28:38.239 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:28:38.244 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:28:38.247 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:28:38.254 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:28:38.257 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:28:55.805 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-07-21T02:28:55.808 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-07-21T02:28:55.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T02:28:55.818 INFO:tasks.cephfs_test_runner:test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) ... ERROR 2022-07-21T02:28:55.820 INFO:tasks.cephfs_test_runner:ERROR 2022-07-21T02:28:55.821 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.821 INFO:tasks.cephfs_test_runner:====================================================================== 2022-07-21T02:28:55.822 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-07-21T02:28:55.823 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-07-21T02:28:55.826 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-07-21T02:28:55.827 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-07-21T02:28:55.828 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-07-21T02:28:55.828 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-07-21T02:28:55.829 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-07-21T02:28:55.829 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-07-21T02:28:55.830 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.831 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-07-21T02:28:55.831 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.832 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-07-21T02:28:55.833 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2022-07-21T02:28:55.833 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2022-07-21T02:28:55.833 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2022-07-21T02:28:55.834 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2022-07-21T02:28:55.835 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-07-21T02:28:55.835 INFO:tasks.cephfs_test_runner: proc.wait() 2022-07-21T02:28:55.836 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 148, in wait 2022-07-21T02:28:55.837 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-07-21T02:28:55.837 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-07-21T02:28:55.838 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-07-21T02:28:55.838 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-07-21T02:28:55.839 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-07-21T02:28:55.839 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-07-21T02:28:55.840 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-07-21T02:28:55.840 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-07-21T02:28:55.841 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-07-21T02:28:55.841 INFO:tasks.cephfs_test_runner: for line in f: 2022-07-21T02:28:55.842 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-07-21T02:28:55.842 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-07-21T02:28:55.843 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-07-21T02:28:55.843 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-07-21T02:28:55.843 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-07-21T02:28:55.844 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-07-21T02:28:55.844 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-07-21T02:28:55.844 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-07-21T02:28:55.845 INFO:tasks.cephfs_test_runner:socket.timeout 2022-07-21T02:28:55.845 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.845 INFO:tasks.cephfs_test_runner:====================================================================== 2022-07-21T02:28:55.846 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-07-21T02:28:55.846 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-07-21T02:28:55.847 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-07-21T02:28:55.847 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-07-21T02:28:55.848 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-07-21T02:28:55.848 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-07-21T02:28:55.849 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-07-21T02:28:55.849 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-07-21T02:28:55.850 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.851 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-07-21T02:28:55.851 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.852 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-07-21T02:28:55.853 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2022-07-21T02:28:55.853 INFO:tasks.cephfs_test_runner: m.teardown() 2022-07-21T02:28:55.853 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2022-07-21T02:28:55.854 INFO:tasks.cephfs_test_runner: self.umount() 2022-07-21T02:28:55.854 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2022-07-21T02:28:55.855 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2022-07-21T02:28:55.855 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-07-21T02:28:55.855 INFO:tasks.cephfs_test_runner: proc.wait() 2022-07-21T02:28:55.855 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 148, in wait 2022-07-21T02:28:55.856 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-07-21T02:28:55.856 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-07-21T02:28:55.856 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-07-21T02:28:55.857 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-07-21T02:28:55.857 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-07-21T02:28:55.857 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-07-21T02:28:55.858 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-07-21T02:28:55.858 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-07-21T02:28:55.858 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-07-21T02:28:55.859 INFO:tasks.cephfs_test_runner: for line in f: 2022-07-21T02:28:55.859 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-07-21T02:28:55.859 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-07-21T02:28:55.860 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-07-21T02:28:55.860 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-07-21T02:28:55.860 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-07-21T02:28:55.861 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-07-21T02:28:55.861 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-07-21T02:28:55.861 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-07-21T02:28:55.861 INFO:tasks.cephfs_test_runner:socket.timeout 2022-07-21T02:28:55.864 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.864 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-07-21T02:28:55.865 INFO:tasks.cephfs_test_runner:Ran 12 tests in 2396.802s 2022-07-21T02:28:55.865 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.865 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2022-07-21T02:28:55.866 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.866 INFO:tasks.cephfs_test_runner:====================================================================== 2022-07-21T02:28:55.866 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-07-21T02:28:55.867 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-07-21T02:28:55.867 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-07-21T02:28:55.867 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-07-21T02:28:55.867 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-07-21T02:28:55.868 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-07-21T02:28:55.868 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-07-21T02:28:55.868 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-07-21T02:28:55.869 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.869 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-07-21T02:28:55.869 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.870 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-07-21T02:28:55.870 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2022-07-21T02:28:55.871 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2022-07-21T02:28:55.871 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2022-07-21T02:28:55.871 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2022-07-21T02:28:55.872 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-07-21T02:28:55.872 INFO:tasks.cephfs_test_runner: proc.wait() 2022-07-21T02:28:55.873 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 148, in wait 2022-07-21T02:28:55.873 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-07-21T02:28:55.873 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-07-21T02:28:55.874 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-07-21T02:28:55.874 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-07-21T02:28:55.874 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-07-21T02:28:55.875 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-07-21T02:28:55.875 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-07-21T02:28:55.876 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-07-21T02:28:55.880 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-07-21T02:28:55.881 INFO:tasks.cephfs_test_runner: for line in f: 2022-07-21T02:28:55.881 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-07-21T02:28:55.882 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-07-21T02:28:55.882 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-07-21T02:28:55.883 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-07-21T02:28:55.884 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-07-21T02:28:55.884 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-07-21T02:28:55.885 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-07-21T02:28:55.885 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-07-21T02:28:55.886 INFO:tasks.cephfs_test_runner:socket.timeout 2022-07-21T02:28:55.888 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.888 INFO:tasks.cephfs_test_runner:====================================================================== 2022-07-21T02:28:55.889 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-07-21T02:28:55.889 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2022-07-21T02:28:55.890 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-07-21T02:28:55.890 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2022-07-21T02:28:55.891 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2022-07-21T02:28:55.891 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2022-07-21T02:28:55.892 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2022-07-21T02:28:55.892 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2022-07-21T02:28:55.893 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.894 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2022-07-21T02:28:55.895 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.895 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2022-07-21T02:28:55.896 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2022-07-21T02:28:55.896 INFO:tasks.cephfs_test_runner: m.teardown() 2022-07-21T02:28:55.897 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2022-07-21T02:28:55.897 INFO:tasks.cephfs_test_runner: self.umount() 2022-07-21T02:28:55.898 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2022-07-21T02:28:55.899 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2022-07-21T02:28:55.900 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2022-07-21T02:28:55.901 INFO:tasks.cephfs_test_runner: proc.wait() 2022-07-21T02:28:55.902 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 148, in wait 2022-07-21T02:28:55.903 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2022-07-21T02:28:55.903 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2022-07-21T02:28:55.904 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2022-07-21T02:28:55.904 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2022-07-21T02:28:55.905 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2022-07-21T02:28:55.906 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2022-07-21T02:28:55.906 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to 2022-07-21T02:28:55.906 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2022-07-21T02:28:55.907 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log 2022-07-21T02:28:55.907 INFO:tasks.cephfs_test_runner: for line in f: 2022-07-21T02:28:55.908 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2022-07-21T02:28:55.908 INFO:tasks.cephfs_test_runner: line = self.readline() 2022-07-21T02:28:55.909 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2022-07-21T02:28:55.909 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2022-07-21T02:28:55.910 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2022-07-21T02:28:55.911 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2022-07-21T02:28:55.911 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2022-07-21T02:28:55.912 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2022-07-21T02:28:55.912 INFO:tasks.cephfs_test_runner:socket.timeout 2022-07-21T02:28:55.913 INFO:tasks.cephfs_test_runner: 2022-07-21T02:28:55.913 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/run_tasks.py", line 106, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-07-21T02:28:56.118 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=11f4ca151b464641ac7dd728a9bb47ed Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/run_tasks.py", line 106, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2022-07-21T02:28:56.121 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2022-07-21T02:28:56.133 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2022-07-21T02:28:56.168 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2022-07-21T02:28:56.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T02:28:56.170 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T02:28:56.170 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T02:29:08.313 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:29:08.317 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:29:08.322 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:29:08.336 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:29:08.340 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:29:38.450 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:29:38.487 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:29:38.499 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:29:38.504 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:29:38.522 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:30:08.569 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:30:08.573 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:30:08.577 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:30:08.582 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:30:08.588 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:30:38.701 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:30:38.706 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:30:38.709 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:30:38.712 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:30:38.715 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:31:08.770 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:31:08.775 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:31:08.778 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:31:08.786 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:31:08.789 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:31:38.892 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:31:38.896 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:31:38.900 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:31:38.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:31:38.907 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:32:09.032 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:32:09.038 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:32:09.041 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:32:09.044 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:32:09.046 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:32:39.107 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:32:39.112 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:32:39.115 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:32:39.117 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:32:39.120 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:33:09.240 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:33:09.245 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:33:09.249 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:33:09.252 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:33:09.255 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:33:39.383 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:33:39.387 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:33:39.407 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:33:39.412 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:33:39.424 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:34:09.500 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:34:09.506 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:34:09.510 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:34:09.513 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:34:09.516 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:34:39.835 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:34:39.839 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:34:39.841 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:34:39.844 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:34:39.847 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:35:09.901 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:35:09.907 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:35:09.911 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:35:09.914 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:35:09.917 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:35:39.987 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:35:39.993 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:35:39.999 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:35:40.003 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:35:40.009 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:36:10.065 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:36:10.070 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:36:10.074 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:36:10.078 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:36:10.081 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:36:40.166 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:36:40.169 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:36:40.172 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:36:40.174 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:36:40.177 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:37:10.393 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:37:10.398 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:37:10.401 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:37:10.405 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:37:10.409 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:37:40.501 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:37:40.506 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:37:40.510 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:37:40.514 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:37:40.517 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:38:10.633 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:38:10.639 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:38:10.645 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:38:10.649 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:38:10.654 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:38:40.761 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:38:40.780 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:38:40.784 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:38:40.788 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:38:40.795 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:39:10.922 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:39:10.929 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:39:10.932 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:39:10.936 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:39:10.940 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:39:40.999 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:39:41.003 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:39:41.006 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:39:41.010 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:39:41.012 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:40:11.322 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:40:11.330 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:40:11.334 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:40:11.347 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:40:11.355 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:40:41.460 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:40:41.493 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:40:41.519 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:40:41.537 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:40:41.580 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:41:11.648 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:41:11.656 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:41:11.661 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:41:11.665 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:41:11.668 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:41:41.768 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:41:41.773 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:41:41.777 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:41:41.783 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:41:41.787 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:42:11.892 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:42:11.896 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:42:11.898 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:42:11.901 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:42:11.904 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:42:42.015 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:42:42.020 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:42:42.024 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:42:42.030 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:42:42.034 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:43:12.136 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:43:12.141 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:43:12.151 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:43:12.156 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:43:12.160 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:43:42.220 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:43:42.247 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:43:42.251 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:43:42.257 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:43:42.261 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:43:56.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T02:43:56.190 INFO:teuthology.orchestra.run.smithi191.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Connection timed out 2022-07-21T02:43:56.191 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2022-07-21T02:43:56.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2022-07-21T02:43:56.193 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2022-07-21T02:43:56.193 DEBUG:teuthology.orchestra.run.smithi191:> ' -- /home/ubuntu/cephtest/mnt.0) 2022-07-21T02:44:12.386 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:44:12.399 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:44:12.405 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:44:12.409 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:44:12.412 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:44:42.535 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:44:42.539 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:44:42.542 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:44:42.544 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:44:42.548 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:45:12.674 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:45:12.680 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:45:12.683 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:45:12.687 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:45:12.695 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:45:42.838 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:45:42.843 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:45:42.847 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:45:42.850 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:45:42.853 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:46:13.065 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:46:13.069 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:46:13.072 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:46:13.075 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:46:13.077 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:46:43.179 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:46:43.182 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:46:43.185 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:46:43.187 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:46:43.190 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:47:13.292 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:47:13.297 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:47:13.300 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:47:13.303 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:47:13.305 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:47:43.381 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:47:43.387 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:47:43.391 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:47:43.396 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:47:43.400 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:48:13.495 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:48:13.505 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:48:13.511 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:48:13.515 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:48:13.520 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:48:43.624 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:48:43.634 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:48:43.638 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:48:43.643 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:48:43.648 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:49:13.754 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:49:13.764 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:49:13.775 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:49:13.780 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:49:13.783 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:49:43.896 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:49:43.901 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:49:43.905 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:49:43.908 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:49:43.913 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:50:13.995 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:50:14.012 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:50:14.046 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:50:14.051 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:50:14.055 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:50:44.156 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:50:44.171 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:50:44.183 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:50:44.201 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:50:44.207 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:51:14.270 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:51:14.279 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:51:14.291 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:51:14.294 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:51:14.300 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:51:44.425 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:51:44.436 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:51:44.452 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:51:44.464 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:51:44.468 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:52:14.538 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:52:14.550 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:52:14.556 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:52:14.563 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:52:14.587 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:52:44.652 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:52:44.663 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:52:44.668 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:52:44.671 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:52:44.678 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:53:14.759 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:53:14.766 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:53:14.773 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:53:14.776 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:53:14.779 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:53:44.990 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:53:45.005 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:53:45.018 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:53:45.026 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:53:45.029 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:54:15.101 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:54:15.105 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:54:15.109 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:54:15.116 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:54:15.121 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:54:45.234 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:54:45.240 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:54:45.245 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:54:45.249 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:54:45.254 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:55:15.368 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:55:15.376 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:55:15.384 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:55:15.387 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:55:15.390 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:55:45.496 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:55:45.770 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:55:45.789 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:55:45.797 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:55:45.814 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:56:15.937 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:56:15.941 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:56:15.944 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:56:15.947 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:56:15.950 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:56:46.007 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:56:46.011 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:56:46.014 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:56:46.017 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:56:46.020 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:57:16.110 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:57:16.116 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:57:16.120 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:57:16.124 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:57:16.127 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:57:46.266 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:57:46.269 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:57:46.272 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:57:46.275 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:57:46.279 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:58:16.387 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:58:16.392 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:58:16.395 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:58:16.398 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:58:16.402 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:58:46.510 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:58:46.515 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:58:46.518 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:58:46.522 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:58:46.526 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:58:56.198 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-07-21T02:58:56.216 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2022-07-21T02:58:56.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T02:58:56.250 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-07-21T02:58:56.262 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2022-07-21T02:58:56.273 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2022-07-21T02:58:56.276 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T02:58:56.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.399+0000 7f1628aba700 1 Processor -- start 2022-07-21T02:58:56.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.399+0000 7f1628aba700 1 -- start start 2022-07-21T02:58:56.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1628aba700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f16240b1ef0 0x7f1624142f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1628aba700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1624060980 con 0x7f16240ac3a0 2022-07-21T02:58:56.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1628aba700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16240b22d0 con 0x7f162405fa50 2022-07-21T02:58:56.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1628aba700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1624098810 con 0x7f16240b1ef0 2022-07-21T02:58:56.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f161affd700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f16240ac3a0 0x7f1624143470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56394/0 (socket says 172.21.15.140:56394) 2022-07-21T02:58:56.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f161affd700 1 -- 172.21.15.140:0/3771061408 learned_addr learned my addr 172.21.15.140:0/3771061408 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:58:56.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f16237fe700 1 --2- 172.21.15.140:0/3771061408 >> v2:172.21.15.152:3300/0 conn(0x7f16240b1ef0 0x7f1624142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1622ffd700 1 -- 172.21.15.140:0/3771061408 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376939790 0 0) 0x7f1624060980 con 0x7f16240ac3a0 2022-07-21T02:58:56.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1622ffd700 1 -- 172.21.15.140:0/3771061408 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1604003260 con 0x7f16240ac3a0 2022-07-21T02:58:56.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1622ffd700 1 -- 172.21.15.140:0/3771061408 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732213473 0 0) 0x7f16240b22d0 con 0x7f162405fa50 2022-07-21T02:58:56.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1622ffd700 1 -- 172.21.15.140:0/3771061408 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1624060980 con 0x7f162405fa50 2022-07-21T02:58:56.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f16237fe700 1 -- 172.21.15.140:0/3771061408 >> v1:172.21.15.152:6789/0 conn(0x7f162405fa50 legacy=0x7f162413e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f16237fe700 1 -- 172.21.15.140:0/3771061408 >> v1:172.21.15.140:6789/0 conn(0x7f16240ac3a0 legacy=0x7f1624143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f16237fe700 1 -- 172.21.15.140:0/3771061408 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1624060bb0 con 0x7f16240b1ef0 2022-07-21T02:58:56.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f16237fe700 1 --2- 172.21.15.140:0/3771061408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16240b1ef0 0x7f1624142f80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f1614002f70 tx=0x7f1614006d90).ready entity=mon.1 client_cookie=ac23e5a0a0e43fa8 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:56.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1622ffd700 1 -- 172.21.15.140:0/3771061408 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f161400e630 con 0x7f16240b1ef0 2022-07-21T02:58:56.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1622ffd700 1 -- 172.21.15.140:0/3771061408 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614000e90 con 0x7f16240b1ef0 2022-07-21T02:58:56.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1622ffd700 1 -- 172.21.15.140:0/3771061408 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f161400e980 con 0x7f16240b1ef0 2022-07-21T02:58:56.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1628aba700 1 -- 172.21.15.140:0/3771061408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16240b1ef0 msgr2=0x7f1624142f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1628aba700 1 --2- 172.21.15.140:0/3771061408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16240b1ef0 0x7f1624142f80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f1614002f70 tx=0x7f1614006d90).stop 2022-07-21T02:58:56.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1628aba700 1 -- 172.21.15.140:0/3771061408 shutdown_connections 2022-07-21T02:58:56.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1628aba700 1 --2- 172.21.15.140:0/3771061408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16240b1ef0 0x7f1624142f80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1628aba700 1 -- 172.21.15.140:0/3771061408 >> 172.21.15.140:0/3771061408 conn(0x7f16240b2ca0 msgr2=0x7f16240b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:56.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1628aba700 1 -- 172.21.15.140:0/3771061408 shutdown_connections 2022-07-21T02:58:56.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.403+0000 7f1628aba700 1 -- 172.21.15.140:0/3771061408 wait complete. 2022-07-21T02:58:56.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.407+0000 7f1628aba700 1 Processor -- start 2022-07-21T02:58:56.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.407+0000 7f1628aba700 1 -- start start 2022-07-21T02:58:56.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1628aba700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f162405fa50 0x7f16241431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1628aba700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16240ac3a0 0x7f162409ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1628aba700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16240b1ef0 0x7f162409b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1628aba700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16240ab6d0 con 0x7f16240b1ef0 2022-07-21T02:58:56.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1628aba700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162413cda0 con 0x7f162405fa50 2022-07-21T02:58:56.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1628aba700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16240b0210 con 0x7f16240ac3a0 2022-07-21T02:58:56.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1623fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16240b1ef0 0x7f162409b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1623fff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16240b1ef0 0x7f162409b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42254/0 (socket says 172.21.15.140:42254) 2022-07-21T02:58:56.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1623fff700 1 -- 172.21.15.140:0/2548363792 learned_addr learned my addr 172.21.15.140:0/2548363792 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:56.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f161affd700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16240ac3a0 0x7f162409ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f16237fe700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f162405fa50 0x7f16241431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1623fff700 1 -- 172.21.15.140:0/2548363792 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16240ac3a0 msgr2=0x7f162409ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1623fff700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16240ac3a0 0x7f162409ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1623fff700 1 -- 172.21.15.140:0/2548363792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f162405fa50 msgr2=0x7f16241431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1623fff700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f162405fa50 0x7f16241431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1623fff700 1 -- 172.21.15.140:0/2548363792 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161400b120 con 0x7f16240b1ef0 2022-07-21T02:58:56.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1623fff700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16240b1ef0 0x7f162409b110 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f161c001490 tx=0x7f161c007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:56.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f16217fa700 1 -- 172.21.15.140:0/2548363792 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f161c002240 con 0x7f16240b1ef0 2022-07-21T02:58:56.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f16217fa700 1 -- 172.21.15.140:0/2548363792 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c0023a0 con 0x7f16240b1ef0 2022-07-21T02:58:56.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1628aba700 1 -- 172.21.15.140:0/2548363792 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16240b04b0 con 0x7f16240b1ef0 2022-07-21T02:58:56.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f16217fa700 1 -- 172.21.15.140:0/2548363792 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f161c001650 con 0x7f16240b1ef0 2022-07-21T02:58:56.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1628aba700 1 -- 172.21.15.140:0/2548363792 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162409c020 con 0x7f16240b1ef0 2022-07-21T02:58:56.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.411+0000 7f1628aba700 1 -- 172.21.15.140:0/2548363792 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162413cbc0 con 0x7f16240b1ef0 2022-07-21T02:58:56.448 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.415+0000 7f16217fa700 1 -- 172.21.15.140:0/2548363792 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f161c010030 con 0x7f16240b1ef0 2022-07-21T02:58:56.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.415+0000 7f16217fa700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1608053a70 0x7f1608055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.415+0000 7f16217fa700 1 -- 172.21.15.140:0/2548363792 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f161c06bb30 con 0x7f16240b1ef0 2022-07-21T02:58:56.450 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.415+0000 7f16237fe700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1608053a70 0x7f1608055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.451 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.415+0000 7f16237fe700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1608053a70 0x7f1608055f30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f161401a000 tx=0x7f1614014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:56.452 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.423+0000 7f16217fa700 1 -- 172.21.15.140:0/2548363792 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f161c042c90 con 0x7f16240b1ef0 2022-07-21T02:58:56.551 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.547+0000 7f1628aba700 1 -- 172.21.15.140:0/2548363792 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1624062270 con 0x7f16240b1ef0 2022-07-21T02:58:56.552 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.547+0000 7f16217fa700 1 -- 172.21.15.140:0/2548363792 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+13418 (secure 0 0 0) 0x7f161c0462b0 con 0x7f16240b1ef0 2022-07-21T02:58:56.553 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T02:58:56.554 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":154,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:58:25.522770+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.140:6835/4013573612":"2022-07-22T01:58:21.502611+0000","172.21.15.140:6834/4013573612":"2022-07-22T01:58:21.502611+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T02:58:56.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.547+0000 7f1628aba700 1 -- 172.21.15.140:0/2548363792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1608053a70 msgr2=0x7f1608055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.547+0000 7f1628aba700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1608053a70 0x7f1608055f30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f161401a000 tx=0x7f1614014000).stop 2022-07-21T02:58:56.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.547+0000 7f1628aba700 1 -- 172.21.15.140:0/2548363792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16240b1ef0 msgr2=0x7f162409b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.547+0000 7f1628aba700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16240b1ef0 0x7f162409b110 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f161c001490 tx=0x7f161c007dd0).stop 2022-07-21T02:58:56.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.547+0000 7f1628aba700 1 -- 172.21.15.140:0/2548363792 shutdown_connections 2022-07-21T02:58:56.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.547+0000 7f1628aba700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1608053a70 0x7f1608055f30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.547+0000 7f1628aba700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f16240b1ef0 0x7f162409b110 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.547+0000 7f1628aba700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16240ac3a0 0x7f162409ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.547+0000 7f1628aba700 1 --2- 172.21.15.140:0/2548363792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f162405fa50 0x7f16241431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.551+0000 7f1628aba700 1 -- 172.21.15.140:0/2548363792 >> 172.21.15.140:0/2548363792 conn(0x7f16240b2ca0 msgr2=0x7f16240b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:56.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.551+0000 7f1628aba700 1 -- 172.21.15.140:0/2548363792 shutdown_connections 2022-07-21T02:58:56.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.551+0000 7f1628aba700 1 -- 172.21.15.140:0/2548363792 wait complete. 2022-07-21T02:58:56.564 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-07-21T02:58:56.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.663+0000 7fea2a72b700 1 Processor -- start 2022-07-21T02:58:56.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.663+0000 7fea2a72b700 1 -- start start 2022-07-21T02:58:56.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fea2405c3f0 0x7fea2414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea240acef0 0x7fea24146410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2405d480 con 0x7fea240b1e00 2022-07-21T02:58:56.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea240b21e0 con 0x7fea2405c3f0 2022-07-21T02:58:56.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2405cf40 con 0x7fea240acef0 2022-07-21T02:58:56.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea237fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fea2405c3f0 0x7fea2414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.676 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea28cc8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea240acef0 0x7fea24146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea23fff700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fea240b1e00 0x7fea240ac7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:57520/0 (socket says 172.21.15.140:57520) 2022-07-21T02:58:56.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea23fff700 1 -- 172.21.15.140:0/3463950727 learned_addr learned my addr 172.21.15.140:0/3463950727 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:58:56.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea237fe700 1 -- 172.21.15.140:0/3463950727 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea240acef0 msgr2=0x7fea24146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:58:56.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea237fe700 1 --2- 172.21.15.140:0/3463950727 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea240acef0 0x7fea24146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea237fe700 1 -- 172.21.15.140:0/3463950727 >> v1:172.21.15.152:6789/0 conn(0x7fea240b1e00 legacy=0x7fea240ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea237fe700 1 -- 172.21.15.140:0/3463950727 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2405d1e0 con 0x7fea2405c3f0 2022-07-21T02:58:56.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea237fe700 1 --2- 172.21.15.140:0/3463950727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea2405c3f0 0x7fea2414a6d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fea140032f0 tx=0x7fea140091d0).ready entity=mon.1 client_cookie=1b8fc0b9872d000b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:56.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea22ffd700 1 -- 172.21.15.140:0/3463950727 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea1400b6c0 con 0x7fea2405c3f0 2022-07-21T02:58:56.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea22ffd700 1 -- 172.21.15.140:0/3463950727 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1400b820 con 0x7fea2405c3f0 2022-07-21T02:58:56.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea22ffd700 1 -- 172.21.15.140:0/3463950727 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea1400b990 con 0x7fea2405c3f0 2022-07-21T02:58:56.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 -- 172.21.15.140:0/3463950727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea2405c3f0 msgr2=0x7fea2414a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 --2- 172.21.15.140:0/3463950727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea2405c3f0 0x7fea2414a6d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7fea140032f0 tx=0x7fea140091d0).stop 2022-07-21T02:58:56.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 -- 172.21.15.140:0/3463950727 shutdown_connections 2022-07-21T02:58:56.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 --2- 172.21.15.140:0/3463950727 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea240acef0 0x7fea24146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 --2- 172.21.15.140:0/3463950727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea2405c3f0 0x7fea2414a6d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 -- 172.21.15.140:0/3463950727 >> 172.21.15.140:0/3463950727 conn(0x7fea240611c0 msgr2=0x7fea24062af0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:56.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 -- 172.21.15.140:0/3463950727 shutdown_connections 2022-07-21T02:58:56.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.667+0000 7fea2a72b700 1 -- 172.21.15.140:0/3463950727 wait complete. 2022-07-21T02:58:56.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea2a72b700 1 Processor -- start 2022-07-21T02:58:56.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea2a72b700 1 -- start start 2022-07-21T02:58:56.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea2a72b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea240acef0 0x7fea241443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea2a72b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea240b1e00 0x7fea24143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea2a72b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea240b3ff0 0x7fea24143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea2a72b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2409eb40 con 0x7fea240b3ff0 2022-07-21T02:58:56.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea2a72b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2409f890 con 0x7fea240acef0 2022-07-21T02:58:56.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea2a72b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2409fb30 con 0x7fea240b1e00 2022-07-21T02:58:56.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea28cc8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea240b3ff0 0x7fea24143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea28cc8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea240b3ff0 0x7fea24143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42270/0 (socket says 172.21.15.140:42270) 2022-07-21T02:58:56.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea237fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea240b1e00 0x7fea24143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea28cc8700 1 -- 172.21.15.140:0/3979968669 learned_addr learned my addr 172.21.15.140:0/3979968669 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:56.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.671+0000 7fea23fff700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea240acef0 0x7fea241443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea28cc8700 1 -- 172.21.15.140:0/3979968669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea240b1e00 msgr2=0x7fea24143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea28cc8700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea240b1e00 0x7fea24143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea28cc8700 1 -- 172.21.15.140:0/3979968669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea240acef0 msgr2=0x7fea241443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea28cc8700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea240acef0 0x7fea241443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea28cc8700 1 -- 172.21.15.140:0/3979968669 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea14009470 con 0x7fea240b3ff0 2022-07-21T02:58:56.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea28cc8700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea240b3ff0 0x7fea24143960 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7fea1c00c000 tx=0x7fea1c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:56.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea217fa700 1 -- 172.21.15.140:0/3979968669 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea1c011750 con 0x7fea240b3ff0 2022-07-21T02:58:56.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea217fa700 1 -- 172.21.15.140:0/3979968669 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c0118b0 con 0x7fea240b3ff0 2022-07-21T02:58:56.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea2a72b700 1 -- 172.21.15.140:0/3979968669 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2409fdd0 con 0x7fea240b3ff0 2022-07-21T02:58:56.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea217fa700 1 -- 172.21.15.140:0/3979968669 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea1c011c10 con 0x7fea240b3ff0 2022-07-21T02:58:56.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea2a72b700 1 -- 172.21.15.140:0/3979968669 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea240a0270 con 0x7fea240b3ff0 2022-07-21T02:58:56.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea2a72b700 1 -- 172.21.15.140:0/3979968669 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2409f6e0 con 0x7fea240b3ff0 2022-07-21T02:58:56.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea217fa700 1 -- 172.21.15.140:0/3979968669 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fea1c011d70 con 0x7fea240b3ff0 2022-07-21T02:58:56.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea217fa700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea08053b00 0x7fea08055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea217fa700 1 -- 172.21.15.140:0/3979968669 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fea1c06bfb0 con 0x7fea240b3ff0 2022-07-21T02:58:56.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.675+0000 7fea23fff700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea08053b00 0x7fea08055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.679+0000 7fea23fff700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea08053b00 0x7fea08055fc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fea0c00d000 tx=0x7fea0c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:56.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.683+0000 7fea217fa700 1 -- 172.21.15.140:0/3979968669 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fea1c043150 con 0x7fea240b3ff0 2022-07-21T02:58:56.806 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.807+0000 7fea2a72b700 1 -- 172.21.15.140:0/3979968669 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fea240b04b0 con 0x7fea08053b00 2022-07-21T02:58:56.814 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.811+0000 7fea217fa700 1 -- 172.21.15.140:0/3979968669 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68378 (secure 0 0 0) 0x7fea240b04b0 con 0x7fea08053b00 2022-07-21T02:58:56.816 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T02:58:56.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 -- 172.21.15.140:0/3979968669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea08053b00 msgr2=0x7fea08055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea08053b00 0x7fea08055fc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7fea0c00d000 tx=0x7fea0c009040).stop 2022-07-21T02:58:56.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 -- 172.21.15.140:0/3979968669 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea240b3ff0 msgr2=0x7fea24143960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea240b3ff0 0x7fea24143960 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7fea1c00c000 tx=0x7fea1c006df0).stop 2022-07-21T02:58:56.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 -- 172.21.15.140:0/3979968669 shutdown_connections 2022-07-21T02:58:56.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fea08053b00 0x7fea08055fc0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fea240b3ff0 0x7fea24143960 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fea240b1e00 0x7fea24143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 --2- 172.21.15.140:0/3979968669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fea240acef0 0x7fea241443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 -- 172.21.15.140:0/3979968669 >> 172.21.15.140:0/3979968669 conn(0x7fea2405c3f0 msgr2=0x7fea240afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:56.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 -- 172.21.15.140:0/3979968669 shutdown_connections 2022-07-21T02:58:56.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.815+0000 7fea2a72b700 1 -- 172.21.15.140:0/3979968669 wait complete. 2022-07-21T02:58:56.830 INFO:teuthology.orchestra.run.smithi140.stderr:dumped all 2022-07-21T02:58:56.832 INFO:teuthology.orchestra.run.smithi140.stdout:{"pg_ready":true,"pg_map":{"version":2265,"stamp":"2022-07-21T02:58:55.059915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":6385408,"kb_used_data":743209632,"kb_used_omap":27966,"kb_used_meta":0,"kb_avail":743209632,"statfs":{"total":767585320960,"available":761046663168,"internally_reserved":0,"allocated":761046663168,"data_stored":761046663168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28638144,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003294"},"pg_stats":[{"pgid":"1.0","version":"14'2","reported_seq":137,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.525923+0000","last_change":"2022-07-21T01:48:22.241647+0000","last_active":"2022-07-21T01:58:24.525923+0000","last_peered":"2022-07-21T01:58:24.525923+0000","last_clean":"2022-07-21T01:58:24.525923+0000","last_became_active":"2022-07-21T01:48:22.240745+0000","last_became_peered":"2022-07-21T01:48:22.240745+0000","last_unstale":"2022-07-21T01:58:24.525923+0000","last_undegraded":"2022-07-21T01:58:24.525923+0000","last_fullsized":"2022-07-21T01:58:24.525923+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:17.950031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:17.950031+0000","last_clean_scrub_stamp":"2022-07-21T01:48:17.950031+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.652618+0000","last_change":"2022-07-21T01:48:27.974374+0000","last_active":"2022-07-21T01:58:24.652618+0000","last_peered":"2022-07-21T01:58:24.652618+0000","last_clean":"2022-07-21T01:58:24.652618+0000","last_became_active":"2022-07-21T01:48:27.973977+0000","last_became_peered":"2022-07-21T01:48:27.973977+0000","last_unstale":"2022-07-21T01:58:24.652618+0000","last_undegraded":"2022-07-21T01:58:24.652618+0000","last_fullsized":"2022-07-21T01:58:24.652618+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.525740+0000","last_change":"2022-07-21T01:48:27.974429+0000","last_active":"2022-07-21T01:58:24.525740+0000","last_peered":"2022-07-21T01:58:24.525740+0000","last_clean":"2022-07-21T01:58:24.525740+0000","last_became_active":"2022-07-21T01:48:27.973957+0000","last_became_peered":"2022-07-21T01:48:27.973957+0000","last_unstale":"2022-07-21T01:58:24.525740+0000","last_undegraded":"2022-07-21T01:58:24.525740+0000","last_fullsized":"2022-07-21T01:58:24.525740+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.652197+0000","last_change":"2022-07-21T01:48:27.968845+0000","last_active":"2022-07-21T01:58:24.652197+0000","last_peered":"2022-07-21T01:58:24.652197+0000","last_clean":"2022-07-21T01:58:24.652197+0000","last_became_active":"2022-07-21T01:48:27.967411+0000","last_became_peered":"2022-07-21T01:48:27.967411+0000","last_unstale":"2022-07-21T01:58:24.652197+0000","last_undegraded":"2022-07-21T01:58:24.652197+0000","last_fullsized":"2022-07-21T01:58:24.652197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.652301+0000","last_change":"2022-07-21T01:48:27.974179+0000","last_active":"2022-07-21T01:58:24.652301+0000","last_peered":"2022-07-21T01:58:24.652301+0000","last_clean":"2022-07-21T01:58:24.652301+0000","last_became_active":"2022-07-21T01:48:27.973744+0000","last_became_peered":"2022-07-21T01:48:27.973744+0000","last_unstale":"2022-07-21T01:58:24.652301+0000","last_undegraded":"2022-07-21T01:58:24.652301+0000","last_fullsized":"2022-07-21T01:58:24.652301+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.653584+0000","last_change":"2022-07-21T01:48:27.992744+0000","last_active":"2022-07-21T01:58:24.653584+0000","last_peered":"2022-07-21T01:58:24.653584+0000","last_clean":"2022-07-21T01:58:24.653584+0000","last_became_active":"2022-07-21T01:48:27.992216+0000","last_became_peered":"2022-07-21T01:48:27.992216+0000","last_unstale":"2022-07-21T01:58:24.653584+0000","last_undegraded":"2022-07-21T01:58:24.653584+0000","last_fullsized":"2022-07-21T01:58:24.653584+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.526165+0000","last_change":"2022-07-21T01:48:27.996870+0000","last_active":"2022-07-21T01:58:24.526165+0000","last_peered":"2022-07-21T01:58:24.526165+0000","last_clean":"2022-07-21T01:58:24.526165+0000","last_became_active":"2022-07-21T01:48:27.996456+0000","last_became_peered":"2022-07-21T01:48:27.996456+0000","last_unstale":"2022-07-21T01:58:24.526165+0000","last_undegraded":"2022-07-21T01:58:24.526165+0000","last_fullsized":"2022-07-21T01:58:24.526165+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.653792+0000","last_change":"2022-07-21T01:48:27.970841+0000","last_active":"2022-07-21T01:58:24.653792+0000","last_peered":"2022-07-21T01:58:24.653792+0000","last_clean":"2022-07-21T01:58:24.653792+0000","last_became_active":"2022-07-21T01:48:27.970498+0000","last_became_peered":"2022-07-21T01:48:27.970498+0000","last_unstale":"2022-07-21T01:58:24.653792+0000","last_undegraded":"2022-07-21T01:58:24.653792+0000","last_fullsized":"2022-07-21T01:58:24.653792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":145,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.527727+0000","last_change":"2022-07-21T01:48:27.969947+0000","last_active":"2022-07-21T01:58:24.527727+0000","last_peered":"2022-07-21T01:58:24.527727+0000","last_clean":"2022-07-21T01:58:24.527727+0000","last_became_active":"2022-07-21T01:48:27.968884+0000","last_became_peered":"2022-07-21T01:48:27.968884+0000","last_unstale":"2022-07-21T01:58:24.527727+0000","last_undegraded":"2022-07-21T01:58:24.527727+0000","last_fullsized":"2022-07-21T01:58:24.527727+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":1,"up_from":13,"seq":55834575697,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3974,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4069881,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.623,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.41799999999999998},"max":{"1min":0.83699999999999997,"5min":0.96199999999999997,"15min":1.131},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.59699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95699999999999996,"5min":0.96899999999999997,"15min":1.018},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.622},"min":{"1min":0.52200000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.79300000000000004,"5min":0.83399999999999996,"15min":1.069},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.501,"15min":0.496},"max":{"1min":0.90600000000000003,"5min":1.105,"15min":1.3779999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.52300000000000002},"max":{"1min":0.93200000000000005,"5min":1.006,"15min":1.3240000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93000000000000005,"5min":0.98099999999999998,"15min":1.2290000000000001},"last":0.76500000000000001}]},{"osd":4,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.90500000000000003,"15min":0.89000000000000001},"min":{"1min":0.85399999999999998,"5min":0.67100000000000004,"15min":0.63300000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.583},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.86899999999999999,"15min":0.876},"min":{"1min":0.63900000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.008,"5min":1.0960000000000001,"15min":1.335},"last":0.69199999999999995}]},{"osd":5,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.90600000000000003,"15min":0.89900000000000002},"min":{"1min":0.79500000000000004,"5min":0.63800000000000001,"15min":0.59699999999999998},"max":{"1min":1.0960000000000001,"5min":1.2010000000000001,"15min":1.4970000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.999,"15min":0.98599999999999999},"min":{"1min":0.872,"5min":0.73899999999999999,"15min":0.56100000000000005},"max":{"1min":1.147,"5min":1.3400000000000001,"15min":1.569},"last":0.88}]},{"osd":6,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.95299999999999996,"15min":0.95399999999999996},"min":{"1min":0.78900000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.054,"5min":1.2969999999999999,"15min":1.4099999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.94499999999999995,"15min":0.95799999999999996},"min":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.67700000000000005},"max":{"1min":1.1020000000000001,"5min":1.1930000000000001,"15min":1.548},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0269999999999999,"15min":1.014},"min":{"1min":0.79700000000000004,"5min":0.746,"15min":0.72899999999999998},"max":{"1min":1.1579999999999999,"5min":1.3879999999999999,"15min":1.613},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0249999999999999,"15min":1.0229999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.69599999999999995},"max":{"1min":1.159,"5min":1.425,"15min":1.522},"last":1.2350000000000001}]}]},{"osd":7,"up_from":13,"seq":55834575696,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3677,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3765344,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.80800000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1679999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.80600000000000005,"15min":0.80900000000000005},"min":{"1min":0.58599999999999997,"5min":0.54400000000000004,"15min":0.52500000000000002},"max":{"1min":1.0389999999999999,"5min":1.085,"15min":1.181},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.871,"15min":0.88400000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.54700000000000004},"max":{"1min":1.133,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.91300000000000003,"15min":0.91800000000000004},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.61099999999999999},"max":{"1min":1.0700000000000001,"5min":1.1140000000000001,"15min":1.2569999999999999},"last":0.88300000000000001}]},{"osd":2,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.88100000000000001},"min":{"1min":0.79200000000000004,"5min":0.58799999999999997,"15min":0.51700000000000002},"max":{"1min":1.0369999999999999,"5min":1.119,"15min":1.153},"last":1.01},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.92000000000000004,"15min":0.92300000000000004},"min":{"1min":0.79800000000000004,"5min":0.65700000000000003,"15min":0.59899999999999998},"max":{"1min":1.1100000000000001,"5min":1.1519999999999999,"15min":1.1879999999999999},"last":0.95699999999999996}]},{"osd":3,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83799999999999997,"15min":0.84799999999999998},"min":{"1min":0.66100000000000003,"5min":0.65900000000000003,"15min":0.57999999999999996},"max":{"1min":1.069,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.048},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89800000000000002,"15min":0.88500000000000001},"min":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.083,"5min":1.238,"15min":1.238},"last":0.81200000000000006}]},{"osd":4,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.77800000000000002,"15min":0.79400000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.008,"5min":1.1399999999999999,"15min":1.256},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.83299999999999996,"15min":0.85999999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.52200000000000002},"max":{"1min":1.016,"5min":1.22,"15min":1.2669999999999999},"last":1.0840000000000001}]},{"osd":5,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.874,"15min":0.88100000000000001},"min":{"1min":0.64500000000000002,"5min":0.58699999999999997,"15min":0.47399999999999998},"max":{"1min":1.1479999999999999,"5min":1.276,"15min":1.276},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"max":{"1min":1.111,"5min":1.1799999999999999,"15min":1.2649999999999999},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.59899999999999998},"max":{"1min":1.1200000000000001,"5min":1.2290000000000001,"15min":1.4570000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95499999999999996,"15min":0.95899999999999996},"min":{"1min":0.70799999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2010000000000001,"5min":1.3680000000000001,"15min":1.546},"last":1.1200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575698,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":4010,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4106826,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76100000000000001,"15min":0.79700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0980000000000001,"5min":1.1399999999999999,"15min":1.1639999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80800000000000005,"15min":0.82299999999999995},"min":{"1min":0.65900000000000003,"5min":0.59499999999999997,"15min":0.54500000000000004},"max":{"1min":1.016,"5min":1.101,"15min":1.1870000000000001},"last":0.68100000000000005}]},{"osd":1,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.629,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.0409999999999999,"5min":1.085,"15min":1.1399999999999999},"last":1.002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.56999999999999995},"max":{"1min":1.075,"5min":1.177,"15min":1.177},"last":0.60999999999999999}]},{"osd":2,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89400000000000002,"15min":0.88800000000000001},"min":{"1min":0.72299999999999998,"5min":0.57799999999999996,"15min":0.56200000000000006},"max":{"1min":1.1439999999999999,"5min":1.153,"15min":1.2490000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.93899999999999995,"15min":0.94799999999999995},"min":{"1min":0.76800000000000002,"5min":0.624,"15min":0.59399999999999997},"max":{"1min":1.097,"5min":1.1799999999999999,"15min":1.2549999999999999},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.89400000000000002,"15min":0.88100000000000001},"min":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1819999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.65900000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1339999999999999,"5min":1.1830000000000001,"15min":1.2250000000000001},"last":0.92900000000000005}]},{"osd":4,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.52400000000000002},"max":{"1min":1.1259999999999999,"5min":1.1399999999999999,"15min":1.236},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.86599999999999999,"15min":0.85899999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.57299999999999995},"max":{"1min":1.0720000000000001,"5min":1.173,"15min":1.2629999999999999},"last":0.877}]},{"osd":5,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90800000000000003,"15min":0.89800000000000002},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.61299999999999999},"max":{"1min":1.206,"5min":1.2230000000000001,"15min":1.3009999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88400000000000001,"15min":0.89700000000000002},"min":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.57999999999999996},"max":{"1min":1.159,"5min":1.206,"15min":1.4490000000000001},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.95399999999999996,"15min":0.94899999999999995},"min":{"1min":0.67300000000000004,"5min":0.66000000000000003,"15min":0.60599999999999998},"max":{"1min":1.133,"5min":1.3069999999999999,"15min":1.329},"last":1.028},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.97999999999999998,"15min":0.98499999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.2609999999999999,"5min":1.407,"15min":1.4199999999999999},"last":1.056}]}]},{"osd":5,"up_from":13,"seq":55834575697,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":2756,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2822493,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81399999999999995,"15min":0.81499999999999995},"min":{"1min":0.55400000000000005,"5min":0.54500000000000004,"15min":0.505},"max":{"1min":1.006,"5min":1.0229999999999999,"15min":1.113},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92100000000000004,"15min":0.91400000000000003},"min":{"1min":0.73299999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.169},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90400000000000003,"15min":0.89400000000000002},"min":{"1min":0.65200000000000002,"5min":0.64000000000000001,"15min":0.57299999999999995},"max":{"1min":1.1719999999999999,"5min":1.1899999999999999,"15min":1.1970000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86399999999999999,"15min":0.87},"min":{"1min":0.77900000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.96799999999999997,"5min":1.091,"15min":1.1220000000000001},"last":0.59099999999999997}]},{"osd":2,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.59899999999999998,"5min":0.56499999999999995,"15min":0.49199999999999999},"max":{"1min":0.999,"5min":1.1950000000000001,"15min":1.7809999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.83299999999999996,"15min":0.83999999999999997},"min":{"1min":0.68400000000000005,"5min":0.61599999999999999,"15min":0.59099999999999997},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.198},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.874,"15min":0.872},"min":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61699999999999999},"max":{"1min":1.091,"5min":1.1419999999999999,"15min":1.2130000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.89000000000000001,"15min":0.88200000000000001},"min":{"1min":0.66800000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.131,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.95499999999999996}]},{"osd":4,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.753,"15min":0.745},"min":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.44800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0669999999999999,"15min":1.079},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.53700000000000003},"max":{"1min":1.198,"5min":1.234,"15min":1.3149999999999999},"last":0.97099999999999997}]},{"osd":6,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89900000000000002},"min":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.53600000000000003},"max":{"1min":1.089,"5min":1.1259999999999999,"15min":1.26},"last":1.024},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.56200000000000006},"max":{"1min":1.1060000000000001,"5min":1.1539999999999999,"15min":1.196},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.96299999999999997,"15min":0.93600000000000005},"min":{"1min":0.752,"5min":0.63600000000000001,"15min":0.58199999999999996},"max":{"1min":1.246,"5min":1.464,"15min":1.464},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97599999999999998,"15min":0.96399999999999997},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.56999999999999995},"max":{"1min":1.335,"5min":1.335,"15min":1.413},"last":1.0649999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575698,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798064,"kb_used_data":92901316,"kb_used_omap":3556,"kb_used_meta":0,"kb_avail":92901316,"statfs":{"total":95948165120,"available":95130947584,"internally_reserved":0,"allocated":95130947584,"data_stored":95130947584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3641905,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.024,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.83599999999999997,"15min":0.82699999999999996},"min":{"1min":0.67400000000000004,"5min":0.51300000000000001,"15min":0.46200000000000002},"max":{"1min":1.048,"5min":1.1519999999999999,"15min":1.26},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81899999999999995,"15min":0.83599999999999997},"min":{"1min":0.61399999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.111,"5min":1.1930000000000001,"15min":1.206},"last":0.749},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.65500000000000003,"5min":0.61699999999999999,"15min":0.58599999999999997},"max":{"1min":0.95199999999999996,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.83199999999999996}]},{"osd":2,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.65000000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.042,"5min":1.151,"15min":1.151},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.872,"15min":0.872},"min":{"1min":0.75800000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.159},"last":0.79000000000000004}]},{"osd":3,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.63700000000000001},"max":{"1min":1.0049999999999999,"5min":1.0640000000000001,"15min":1.1639999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.88300000000000001,"15min":0.89300000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.026,"5min":1.131,"15min":1.1970000000000001},"last":0.879}]},{"osd":5,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.876},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.54700000000000004},"max":{"1min":1.147,"5min":1.1579999999999999,"15min":1.24},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92300000000000004,"15min":0.93400000000000005},"min":{"1min":0.72899999999999998,"5min":0.59299999999999997,"15min":0.53800000000000003},"max":{"1min":1.0629999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.90500000000000003}]},{"osd":6,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.878,"15min":0.86499999999999999},"min":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54600000000000004},"max":{"1min":1.1479999999999999,"5min":1.2649999999999999,"15min":1.3400000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93700000000000006,"15min":0.92900000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61299999999999999},"max":{"1min":1.1479999999999999,"5min":1.226,"15min":1.226},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.90200000000000002,"15min":0.89100000000000001},"min":{"1min":0.64700000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2350000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.97199999999999998,"15min":0.96399999999999997},"min":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.139,"5min":1.3240000000000001,"15min":1.3859999999999999},"last":0.70399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575697,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3358,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3439467,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.46800000000000003,"5min":0.438,"15min":0.433},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.079},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.65600000000000003,"5min":0.58399999999999996,"15min":0.438},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.1439999999999999},"last":0.92500000000000004}]},{"osd":1,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47799999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.0820000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.60999999999999999,"5min":0.53700000000000003,"15min":0.51400000000000001},"max":{"1min":0.98499999999999999,"5min":1.0329999999999999,"15min":1.0860000000000001},"last":0.83799999999999997}]},{"osd":2,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.5},"max":{"1min":0.85299999999999998,"5min":0.93200000000000005,"15min":1.0609999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.52100000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1559999999999999},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.88800000000000001,"15min":0.88300000000000001},"min":{"1min":0.76700000000000002,"5min":0.65700000000000003,"15min":0.61899999999999999},"max":{"1min":1.034,"5min":1.127,"15min":1.1639999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.89800000000000002,"15min":0.91000000000000003},"min":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.68200000000000005},"max":{"1min":1.0620000000000001,"5min":1.1240000000000001,"15min":1.1890000000000001},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.89900000000000002,"15min":0.90800000000000003},"min":{"1min":0.77500000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.153},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90400000000000003,"15min":0.92700000000000005},"min":{"1min":0.73299999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.143,"5min":1.2030000000000001,"15min":1.349},"last":1.0580000000000001}]},{"osd":6,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.92700000000000005,"15min":0.94699999999999995},"min":{"1min":0.70599999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1160000000000001,"5min":1.2589999999999999,"15min":1.4159999999999999},"last":1.133},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.96899999999999997,"15min":0.96999999999999997},"min":{"1min":0.79100000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3799999999999999},"last":1.21}]},{"osd":7,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.83699999999999997,"5min":0.746,"15min":0.74099999999999999},"max":{"1min":1.2649999999999999,"5min":1.3100000000000001,"15min":1.3280000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.012,"15min":1.018},"min":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.0860000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575698,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":2831,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2899071,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.98399999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68600000000000005,"15min":0.69799999999999995},"min":{"1min":0.54300000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.026},"last":0.66000000000000003}]},{"osd":1,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.503,"5min":0.48699999999999999,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.012,"15min":1.04},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.433},"max":{"1min":0.89300000000000002,"5min":1.046,"15min":1.077},"last":0.89800000000000002}]},{"osd":3,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.50900000000000001,"5min":0.46999999999999997,"15min":0.43099999999999999},"max":{"1min":0.92900000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.623},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.48899999999999999},"max":{"1min":0.96599999999999997,"5min":1.052,"15min":1.0640000000000001},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.76600000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.0649999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.097},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.877,"15min":0.88200000000000001},"min":{"1min":0.71999999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.083,"5min":1.101,"15min":1.2050000000000001},"last":0.96999999999999997}]},{"osd":5,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.93600000000000005,"15min":0.93300000000000005},"min":{"1min":0.77300000000000002,"5min":0.61899999999999999,"15min":0.59499999999999997},"max":{"1min":1.0960000000000001,"5min":1.202,"15min":1.202},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.91800000000000004},"min":{"1min":0.78900000000000003,"5min":0.69099999999999995,"15min":0.58499999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.387},"last":0.83699999999999997}]},{"osd":6,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.85499999999999998,"15min":0.85899999999999999},"min":{"1min":0.71399999999999997,"5min":0.65000000000000002,"15min":0.64400000000000002},"max":{"1min":1.0549999999999999,"5min":1.1739999999999999,"15min":1.2989999999999999},"last":1.079},{"interface":"front","average":{"1min":0.998,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.78500000000000003,"5min":0.72599999999999998,"15min":0.68799999999999994},"max":{"1min":1.2310000000000001,"5min":1.3919999999999999,"15min":1.571},"last":1.0029999999999999}]},{"osd":7,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.95199999999999996,"15min":0.95999999999999996},"min":{"1min":0.76700000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.2829999999999999,"5min":1.4299999999999999,"15min":1.5660000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.024,"5min":0.95799999999999996,"15min":0.95099999999999996},"min":{"1min":0.81299999999999994,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.335,"5min":1.381,"15min":1.4299999999999999},"last":0.94899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575698,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3801,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3893157,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.47399999999999998,"5min":0.443,"15min":0.442},"max":{"1min":0.92500000000000004,"5min":0.93899999999999995,"15min":1.089},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.55200000000000005,"5min":0.48499999999999999,"15min":0.45400000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.085},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69699999999999995,"15min":0.68000000000000005},"min":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.46300000000000002},"max":{"1min":0.89000000000000001,"5min":1.048,"15min":1.048},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.59099999999999997,"5min":0.52200000000000002,"15min":0.5},"max":{"1min":0.92900000000000005,"5min":1.0469999999999999,"15min":1.0820000000000001},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.53500000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.84099999999999997,"5min":1.155,"15min":1.155},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.85699999999999998,"15min":0.84299999999999997},"min":{"1min":0.65200000000000002,"5min":0.63500000000000001,"15min":0.57199999999999995},"max":{"1min":1.0109999999999999,"5min":1.167,"15min":1.167},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89000000000000001,"15min":0.89200000000000002},"min":{"1min":0.73099999999999998,"5min":0.66100000000000003,"15min":0.60099999999999998},"max":{"1min":1.1599999999999999,"5min":1.282,"15min":1.3600000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88400000000000001,"15min":0.877},"min":{"1min":0.70699999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"max":{"1min":1.226,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.0640000000000001}]},{"osd":5,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94799999999999995,"15min":0.93400000000000005},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.66000000000000003},"max":{"1min":1.3779999999999999,"5min":1.474,"15min":1.474},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.68200000000000005},"max":{"1min":1.153,"5min":1.55,"15min":1.55},"last":1.036}]},{"osd":6,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.96299999999999997,"15min":0.93899999999999995},"min":{"1min":0.78500000000000003,"5min":0.72699999999999998,"15min":0.64700000000000002},"max":{"1min":1.298,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.177},{"interface":"front","average":{"1min":0.999,"5min":1.03,"15min":1.0049999999999999},"min":{"1min":0.877,"5min":0.78400000000000003,"15min":0.70699999999999996},"max":{"1min":1.3260000000000001,"5min":1.637,"15min":1.637},"last":1.0229999999999999}]},{"osd":7,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0289999999999999,"15min":1.018},"min":{"1min":0.90300000000000002,"5min":0.72299999999999998,"15min":0.71799999999999997},"max":{"1min":1.2110000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.036,"15min":1.02},"min":{"1min":0.879,"5min":0.751,"15min":0.72599999999999998},"max":{"1min":1.353,"5min":1.3879999999999999,"15min":1.397},"last":1.127}]}]}]}} 2022-07-21T02:58:56.836 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-07-21T02:58:56.940 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f30317b7700 1 Processor -- start 2022-07-21T02:58:56.941 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f30317b7700 1 -- start start 2022-07-21T02:58:56.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f30317b7700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f302c0b7b70 0x7f302c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.944 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f30317b7700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f302c0b31e0 0x7f302c13d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f30317b7700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302c0b7f50 con 0x7f302c066160 2022-07-21T02:58:56.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f30317b7700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302c0b2800 con 0x7f302c0b7b70 2022-07-21T02:58:56.945 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f30317b7700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302c0b2aa0 con 0x7f302c0b31e0 2022-07-21T02:58:56.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f302bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f302c0b7b70 0x7f302c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f3030fb6700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f302c066160 0x7f302c13dd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56424/0 (socket says 172.21.15.140:56424) 2022-07-21T02:58:56.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f3030fb6700 1 -- 172.21.15.140:0/2379863974 learned_addr learned my addr 172.21.15.140:0/2379863974 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:58:56.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f302b7fe700 1 --2- 172.21.15.140:0/2379863974 >> v2:172.21.15.152:3300/0 conn(0x7f302c0b31e0 0x7f302c13d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f302affd700 1 -- 172.21.15.140:0/2379863974 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828912837 0 0) 0x7f302c0b7f50 con 0x7f302c066160 2022-07-21T02:58:56.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.939+0000 7f302affd700 1 -- 172.21.15.140:0/2379863974 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3018003260 con 0x7f302c066160 2022-07-21T02:58:56.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f302bfff700 1 -- 172.21.15.140:0/2379863974 >> v2:172.21.15.152:3300/0 conn(0x7f302c0b31e0 msgr2=0x7f302c13d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:58:56.949 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f302bfff700 1 --2- 172.21.15.140:0/2379863974 >> v2:172.21.15.152:3300/0 conn(0x7f302c0b31e0 0x7f302c13d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f302bfff700 1 -- 172.21.15.140:0/2379863974 >> v1:172.21.15.140:6789/0 conn(0x7f302c066160 legacy=0x7f302c13dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f302bfff700 1 -- 172.21.15.140:0/2379863974 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c0b2d40 con 0x7f302c0b7b70 2022-07-21T02:58:56.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f302bfff700 1 --2- 172.21.15.140:0/2379863974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f302c0b7b70 0x7f302c0b1ac0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f301c002f70 tx=0x7f301c006df0).ready entity=mon.0 client_cookie=be8f8039841b4dc7 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:56.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f302affd700 1 -- 172.21.15.140:0/2379863974 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f301c00d4f0 con 0x7f302c0b7b70 2022-07-21T02:58:56.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f302affd700 1 -- 172.21.15.140:0/2379863974 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301c000e90 con 0x7f302c0b7b70 2022-07-21T02:58:56.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f302affd700 1 -- 172.21.15.140:0/2379863974 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f301c00d840 con 0x7f302c0b7b70 2022-07-21T02:58:56.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 -- 172.21.15.140:0/2379863974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f302c0b7b70 msgr2=0x7f302c0b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 --2- 172.21.15.140:0/2379863974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f302c0b7b70 0x7f302c0b1ac0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f301c002f70 tx=0x7f301c006df0).stop 2022-07-21T02:58:56.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 -- 172.21.15.140:0/2379863974 shutdown_connections 2022-07-21T02:58:56.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 --2- 172.21.15.140:0/2379863974 >> v2:172.21.15.152:3300/0 conn(0x7f302c0b31e0 0x7f302c13d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 --2- 172.21.15.140:0/2379863974 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f302c0b7b70 0x7f302c0b1ac0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.954 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 -- 172.21.15.140:0/2379863974 >> 172.21.15.140:0/2379863974 conn(0x7f302c0656b0 msgr2=0x7f302c0b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:56.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 -- 172.21.15.140:0/2379863974 shutdown_connections 2022-07-21T02:58:56.955 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 -- 172.21.15.140:0/2379863974 wait complete. 2022-07-21T02:58:56.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 Processor -- start 2022-07-21T02:58:56.959 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 -- start start 2022-07-21T02:58:56.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f302c066160 0x7f302c13d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f302c0b31e0 0x7f302c136c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.960 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f302c0b7b70 0x7f302c1370f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c14ff80 con 0x7f302c0b31e0 2022-07-21T02:58:56.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c1375e0 con 0x7f302c0b7b70 2022-07-21T02:58:56.961 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f30317b7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c13fa10 con 0x7f302c066160 2022-07-21T02:58:56.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.943+0000 7f302b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f302c0b31e0 0x7f302c136c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f302b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f302c0b31e0 0x7f302c136c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42282/0 (socket says 172.21.15.140:42282) 2022-07-21T02:58:56.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f302b7fe700 1 -- 172.21.15.140:0/3361638528 learned_addr learned my addr 172.21.15.140:0/3361638528 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:56.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f302bfff700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f302c066160 0x7f302c13d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f3030fb6700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f302c0b7b70 0x7f302c1370f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f302b7fe700 1 -- 172.21.15.140:0/3361638528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f302c066160 msgr2=0x7f302c13d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f302b7fe700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f302c066160 0x7f302c13d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f302b7fe700 1 -- 172.21.15.140:0/3361638528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f302c0b7b70 msgr2=0x7f302c1370f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:56.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f302b7fe700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f302c0b7b70 0x7f302c1370f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:56.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f302b7fe700 1 -- 172.21.15.140:0/3361638528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c00b180 con 0x7f302c0b31e0 2022-07-21T02:58:56.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f302b7fe700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f302c0b31e0 0x7f302c136c00 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f302c1500c0 tx=0x7f3020006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:56.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f30297fa700 1 -- 172.21.15.140:0/3361638528 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3020010780 con 0x7f302c0b31e0 2022-07-21T02:58:56.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f30297fa700 1 -- 172.21.15.140:0/3361638528 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302000b3b0 con 0x7f302c0b31e0 2022-07-21T02:58:56.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f30317b7700 1 -- 172.21.15.140:0/3361638528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c0b03a0 con 0x7f302c0b31e0 2022-07-21T02:58:56.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f30297fa700 1 -- 172.21.15.140:0/3361638528 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3020010ad0 con 0x7f302c0b31e0 2022-07-21T02:58:56.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f30317b7700 1 -- 172.21.15.140:0/3361638528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c0b07d0 con 0x7f302c0b31e0 2022-07-21T02:58:56.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f30317b7700 1 -- 172.21.15.140:0/3361638528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302c0b01d0 con 0x7f302c0b31e0 2022-07-21T02:58:56.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f30297fa700 1 -- 172.21.15.140:0/3361638528 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3020010c30 con 0x7f302c0b31e0 2022-07-21T02:58:56.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.947+0000 7f30297fa700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f300c053a70 0x7f300c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:56.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.951+0000 7f30297fa700 1 -- 172.21.15.140:0/3361638528 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f302006c040 con 0x7f302c0b31e0 2022-07-21T02:58:56.968 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.951+0000 7f302bfff700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f300c053a70 0x7f300c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:56.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.951+0000 7f302bfff700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f300c053a70 0x7f300c055f30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f301c01a000 tx=0x7f301c014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:56.969 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:56.955+0000 7f30297fa700 1 -- 172.21.15.140:0/3361638528 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f3020043140 con 0x7f302c0b31e0 2022-07-21T02:58:57.084 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.083+0000 7f30317b7700 1 -- 172.21.15.140:0/3361638528 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f302c0b0530 con 0x7f302c0b31e0 2022-07-21T02:58:57.085 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.083+0000 7f30297fa700 1 -- 172.21.15.140:0/3361638528 <== mon.0 v2:172.21.15.140:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+13418 (secure 0 0 0) 0x7f3020046760 con 0x7f302c0b31e0 2022-07-21T02:58:57.086 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T02:58:57.087 INFO:teuthology.orchestra.run.smithi140.stdout:{"epoch":154,"fsid":"87d2f829-64cd-4d49-9e42-a1dfafceefb8","created":"2022-07-21T01:48:15.114780+0000","modified":"2022-07-21T01:58:25.522770+0000","last_up_change":"2022-07-21T01:48:21.170967+0000","last_in_change":"2022-07-21T01:48:21.170967+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2022-07-21T01:48:17.946023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-07-21T01:48:26.620359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"f32f4b5f-243e-4377-b3e7-6d2876095aec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6802","nonce":22710},{"type":"v1","addr":"172.21.15.140:6803","nonce":22710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6804","nonce":22710},{"type":"v1","addr":"172.21.15.140:6805","nonce":22710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6808","nonce":22710},{"type":"v1","addr":"172.21.15.140:6809","nonce":22710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6806","nonce":22710},{"type":"v1","addr":"172.21.15.140:6807","nonce":22710}]},"public_addr":"172.21.15.140:6803/22710","cluster_addr":"172.21.15.140:6805/22710","heartbeat_back_addr":"172.21.15.140:6809/22710","heartbeat_front_addr":"172.21.15.140:6807/22710","state":["exists","up"]},{"osd":1,"uuid":"50f0b7fd-5258-4d4d-8075-828f0b34d254","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6826","nonce":22712},{"type":"v1","addr":"172.21.15.140:6827","nonce":22712}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6828","nonce":22712},{"type":"v1","addr":"172.21.15.140:6829","nonce":22712}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6832","nonce":22712},{"type":"v1","addr":"172.21.15.140:6833","nonce":22712}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6830","nonce":22712},{"type":"v1","addr":"172.21.15.140:6831","nonce":22712}]},"public_addr":"172.21.15.140:6827/22712","cluster_addr":"172.21.15.140:6829/22712","heartbeat_back_addr":"172.21.15.140:6833/22712","heartbeat_front_addr":"172.21.15.140:6831/22712","state":["exists","up"]},{"osd":2,"uuid":"0c4190c0-0c1b-49da-ba40-93dab22993a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6810","nonce":22724},{"type":"v1","addr":"172.21.15.140:6811","nonce":22724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6812","nonce":22724},{"type":"v1","addr":"172.21.15.140:6813","nonce":22724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6816","nonce":22724},{"type":"v1","addr":"172.21.15.140:6817","nonce":22724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6814","nonce":22724},{"type":"v1","addr":"172.21.15.140:6815","nonce":22724}]},"public_addr":"172.21.15.140:6811/22724","cluster_addr":"172.21.15.140:6813/22724","heartbeat_back_addr":"172.21.15.140:6817/22724","heartbeat_front_addr":"172.21.15.140:6815/22724","state":["exists","up"]},{"osd":3,"uuid":"6318d81c-39e1-4e05-8ada-4bb0536c536f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6818","nonce":22721},{"type":"v1","addr":"172.21.15.140:6819","nonce":22721}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6820","nonce":22721},{"type":"v1","addr":"172.21.15.140:6821","nonce":22721}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6824","nonce":22721},{"type":"v1","addr":"172.21.15.140:6825","nonce":22721}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.140:6822","nonce":22721},{"type":"v1","addr":"172.21.15.140:6823","nonce":22721}]},"public_addr":"172.21.15.140:6819/22721","cluster_addr":"172.21.15.140:6821/22721","heartbeat_back_addr":"172.21.15.140:6825/22721","heartbeat_front_addr":"172.21.15.140:6823/22721","state":["exists","up"]},{"osd":4,"uuid":"fc54a43f-8a54-40e1-9db5-e4bb3c84ed26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":23141},{"type":"v1","addr":"172.21.15.152:6801","nonce":23141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":23141},{"type":"v1","addr":"172.21.15.152:6803","nonce":23141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":23141},{"type":"v1","addr":"172.21.15.152:6807","nonce":23141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":23141},{"type":"v1","addr":"172.21.15.152:6805","nonce":23141}]},"public_addr":"172.21.15.152:6801/23141","cluster_addr":"172.21.15.152:6803/23141","heartbeat_back_addr":"172.21.15.152:6807/23141","heartbeat_front_addr":"172.21.15.152:6805/23141","state":["exists","up"]},{"osd":5,"uuid":"d63edac8-54f9-468a-8748-faf4231a7cb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":23143},{"type":"v1","addr":"172.21.15.152:6812","nonce":23143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":23143},{"type":"v1","addr":"172.21.15.152:6816","nonce":23143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":23143},{"type":"v1","addr":"172.21.15.152:6823","nonce":23143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":23143},{"type":"v1","addr":"172.21.15.152:6820","nonce":23143}]},"public_addr":"172.21.15.152:6812/23143","cluster_addr":"172.21.15.152:6816/23143","heartbeat_back_addr":"172.21.15.152:6823/23143","heartbeat_front_addr":"172.21.15.152:6820/23143","state":["exists","up"]},{"osd":6,"uuid":"81228326-ddc7-4683-83f7-c1c6eb0232d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":23144},{"type":"v1","addr":"172.21.15.152:6809","nonce":23144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":23144},{"type":"v1","addr":"172.21.15.152:6813","nonce":23144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6819","nonce":23144},{"type":"v1","addr":"172.21.15.152:6821","nonce":23144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":23144},{"type":"v1","addr":"172.21.15.152:6817","nonce":23144}]},"public_addr":"172.21.15.152:6809/23144","cluster_addr":"172.21.15.152:6813/23144","heartbeat_back_addr":"172.21.15.152:6821/23144","heartbeat_front_addr":"172.21.15.152:6817/23144","state":["exists","up"]},{"osd":7,"uuid":"4efb2119-4ea0-4c46-95e2-662416bc7990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":23142},{"type":"v1","addr":"172.21.15.152:6825","nonce":23142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":23142},{"type":"v1","addr":"172.21.15.152:6827","nonce":23142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":23142},{"type":"v1","addr":"172.21.15.152:6831","nonce":23142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":23142},{"type":"v1","addr":"172.21.15.152:6829","nonce":23142}]},"public_addr":"172.21.15.152:6825/23142","cluster_addr":"172.21.15.152:6827/23142","heartbeat_back_addr":"172.21.15.152:6831/23142","heartbeat_front_addr":"172.21.15.152:6829/23142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.223940+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.315596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.275895+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.318162+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.311667+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.338645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.347179+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2022-07-21T01:48:19.335541+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.140:6835/4013573612":"2022-07-22T01:58:21.502611+0000","172.21.15.140:6834/4013573612":"2022-07-22T01:58:21.502611+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2022-07-21T02:58:57.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 -- 172.21.15.140:0/3361638528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f300c053a70 msgr2=0x7f300c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:57.092 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f300c053a70 0x7f300c055f30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f301c01a000 tx=0x7f301c014000).stop 2022-07-21T02:58:57.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 -- 172.21.15.140:0/3361638528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f302c0b31e0 msgr2=0x7f302c136c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:57.096 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f302c0b31e0 0x7f302c136c00 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f302c1500c0 tx=0x7f3020006df0).stop 2022-07-21T02:58:57.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 -- 172.21.15.140:0/3361638528 shutdown_connections 2022-07-21T02:58:57.097 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f300c053a70 0x7f300c055f30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:57.098 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f302c0b7b70 0x7f302c1370f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:57.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f302c0b31e0 0x7f302c136c00 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:57.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 --2- 172.21.15.140:0/3361638528 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f302c066160 0x7f302c13d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:57.099 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 -- 172.21.15.140:0/3361638528 >> 172.21.15.140:0/3361638528 conn(0x7f302c0656b0 msgr2=0x7f302c0b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:57.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 -- 172.21.15.140:0/3361638528 shutdown_connections 2022-07-21T02:58:57.100 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:57.087+0000 7f30317b7700 1 -- 172.21.15.140:0/3361638528 wait complete. 2022-07-21T02:58:58.106 INFO:tasks.ceph:Scrubbing osd.0 2022-07-21T02:58:58.107 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2022-07-21T02:58:58.229 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.227+0000 7fb905cd5700 1 Processor -- start 2022-07-21T02:58:58.230 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.227+0000 7fb905cd5700 1 -- start start 2022-07-21T02:58:58.232 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9000b4040 0x7fb90005ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9000acae0 0x7fb9000acf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9000b4420 con 0x7fb900060090 2022-07-21T02:58:58.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb90005c320 con 0x7fb9000b4040 2022-07-21T02:58:58.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90005f950 con 0x7fb9000acae0 2022-07-21T02:58:58.234 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8ff7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb9000b4040 0x7fb90005ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8fffff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9000acae0 0x7fb9000acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8feffd700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fb900060090 0x7fb9001462a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:57548/0 (socket says 172.21.15.140:57548) 2022-07-21T02:58:58.235 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8feffd700 1 -- 172.21.15.140:0/3972997976 learned_addr learned my addr 172.21.15.140:0/3972997976 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:58:58.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8fffff700 1 -- 172.21.15.140:0/3972997976 >> v2:172.21.15.152:3300/0 conn(0x7fb9000b4040 msgr2=0x7fb90005ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:58:58.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8fffff700 1 --2- 172.21.15.140:0/3972997976 >> v2:172.21.15.152:3300/0 conn(0x7fb9000b4040 0x7fb90005ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8fffff700 1 -- 172.21.15.140:0/3972997976 >> v1:172.21.15.152:6789/0 conn(0x7fb900060090 legacy=0x7fb9001462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8fffff700 1 -- 172.21.15.140:0/3972997976 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90005fbf0 con 0x7fb9000acae0 2022-07-21T02:58:58.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8ff7fe700 1 --2- 172.21.15.140:0/3972997976 >> v2:172.21.15.152:3300/0 conn(0x7fb9000b4040 0x7fb90005ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T02:58:58.237 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8fffff700 1 --2- 172.21.15.140:0/3972997976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9000acae0 0x7fb9000acf50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fb8f80032f0 tx=0x7fb8f80091d0).ready entity=mon.2 client_cookie=4b0731eb5ea14422 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8fe7fc700 1 -- 172.21.15.140:0/3972997976 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8f800b6c0 con 0x7fb9000acae0 2022-07-21T02:58:58.238 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8fe7fc700 1 -- 172.21.15.140:0/3972997976 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f800b820 con 0x7fb9000acae0 2022-07-21T02:58:58.239 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb8fe7fc700 1 -- 172.21.15.140:0/3972997976 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8f800b990 con 0x7fb9000acae0 2022-07-21T02:58:58.243 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 -- 172.21.15.140:0/3972997976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9000acae0 msgr2=0x7fb9000acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 --2- 172.21.15.140:0/3972997976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9000acae0 0x7fb9000acf50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fb8f80032f0 tx=0x7fb8f80091d0).stop 2022-07-21T02:58:58.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 -- 172.21.15.140:0/3972997976 shutdown_connections 2022-07-21T02:58:58.244 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 --2- 172.21.15.140:0/3972997976 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb9000acae0 0x7fb9000acf50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 --2- 172.21.15.140:0/3972997976 >> v2:172.21.15.152:3300/0 conn(0x7fb9000b4040 0x7fb90005ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 -- 172.21.15.140:0/3972997976 >> 172.21.15.140:0/3972997976 conn(0x7fb900064d20 msgr2=0x7fb900065100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:58.245 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 -- 172.21.15.140:0/3972997976 shutdown_connections 2022-07-21T02:58:58.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.231+0000 7fb905cd5700 1 -- 172.21.15.140:0/3972997976 wait complete. 2022-07-21T02:58:58.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb905cd5700 1 Processor -- start 2022-07-21T02:58:58.246 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb905cd5700 1 -- start start 2022-07-21T02:58:58.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb905cd5700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb900060090 0x7fb9001562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb905cd5700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9000acae0 0x7fb900152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.247 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb905cd5700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9000b4040 0x7fb900152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb905cd5700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb900155900 con 0x7fb9000b4040 2022-07-21T02:58:58.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb905cd5700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb900155ba0 con 0x7fb9000acae0 2022-07-21T02:58:58.248 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb905cd5700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb900155e40 con 0x7fb900060090 2022-07-21T02:58:58.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8ff7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb900060090 0x7fb9001562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8fffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9000b4040 0x7fb900152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8ff7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb900060090 0x7fb9001562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:39024/0 (socket says 172.21.15.140:39024) 2022-07-21T02:58:58.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8ff7fe700 1 -- 172.21.15.140:0/3133003561 learned_addr learned my addr 172.21.15.140:0/3133003561 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:58.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8feffd700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9000acae0 0x7fb900152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8ff7fe700 1 -- 172.21.15.140:0/3133003561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9000acae0 msgr2=0x7fb900152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8ff7fe700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9000acae0 0x7fb900152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8ff7fe700 1 -- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9000b4040 msgr2=0x7fb900152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8ff7fe700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9000b4040 0x7fb900152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8ff7fe700 1 -- 172.21.15.140:0/3133003561 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e800b120 con 0x7fb900060090 2022-07-21T02:58:58.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8fffff700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fb9000b4040 0x7fb900152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T02:58:58.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.235+0000 7fb8ff7fe700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb900060090 0x7fb9001562e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fb8e800c000 tx=0x7fb8e8008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.239+0000 7fb8fcff9700 1 -- 172.21.15.140:0/3133003561 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8e8012600 con 0x7fb900060090 2022-07-21T02:58:58.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.239+0000 7fb8fcff9700 1 -- 172.21.15.140:0/3133003561 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e8012760 con 0x7fb900060090 2022-07-21T02:58:58.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.239+0000 7fb905cd5700 1 -- 172.21.15.140:0/3133003561 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f8009470 con 0x7fb900060090 2022-07-21T02:58:58.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.239+0000 7fb8fcff9700 1 -- 172.21.15.140:0/3133003561 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8e8012ac0 con 0x7fb900060090 2022-07-21T02:58:58.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.239+0000 7fb905cd5700 1 -- 172.21.15.140:0/3133003561 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9000ae730 con 0x7fb900060090 2022-07-21T02:58:58.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.239+0000 7fb905cd5700 1 -- 172.21.15.140:0/3133003561 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb900156000 con 0x7fb900060090 2022-07-21T02:58:58.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.239+0000 7fb8fcff9700 1 -- 172.21.15.140:0/3133003561 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fb8e8022070 con 0x7fb900060090 2022-07-21T02:58:58.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.239+0000 7fb8fcff9700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb8e4053b00 0x7fb8e4055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.243+0000 7fb8fcff9700 1 -- 172.21.15.140:0/3133003561 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fb8e806c0e0 con 0x7fb900060090 2022-07-21T02:58:58.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.243+0000 7fb8feffd700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb8e4053b00 0x7fb8e4055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.243+0000 7fb8fcff9700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x7fb8e405a1f0 0x7fb8e405c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.243+0000 7fb8fcff9700 1 -- 172.21.15.140:0/3133003561 --> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8e405cc30 con 0x7fb8e405a1f0 2022-07-21T02:58:58.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.243+0000 7fb8fcff9700 1 -- 172.21.15.140:0/3133003561 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8e806e660 con 0x7fb900060090 2022-07-21T02:58:58.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.243+0000 7fb8fffff700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x7fb8e405a1f0 0x7fb8e405c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.243+0000 7fb8feffd700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb8e4053b00 0x7fb8e4055fc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7fb8f000d000 tx=0x7fb8f0009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.243+0000 7fb8fffff700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x7fb8e405a1f0 0x7fb8e405c650 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.243+0000 7fb8fcff9700 1 -- 172.21.15.140:0/3133003561 <== osd.0 v2:172.21.15.140:6802/22710 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7fb8e405cc30 con 0x7fb8e405a1f0 2022-07-21T02:58:58.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb905cd5700 1 -- 172.21.15.140:0/3133003561 --> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb9000af040 con 0x7fb8e405a1f0 2022-07-21T02:58:58.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb8fcff9700 1 -- 172.21.15.140:0/3133003561 <== osd.0 v2:172.21.15.140:6802/22710 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb9000af040 con 0x7fb8e405a1f0 2022-07-21T02:58:58.264 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T02:58:58.264 INFO:teuthology.orchestra.run.smithi140.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-07-21T02:58:58.265 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T02:58:58.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb905cd5700 1 -- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x7fb8e405a1f0 msgr2=0x7fb8e405c650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb905cd5700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:6802/22710,v1:172.21.15.140:6803/22710] conn(0x7fb8e405a1f0 0x7fb8e405c650 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb905cd5700 1 -- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb8e4053b00 msgr2=0x7fb8e4055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb905cd5700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fb8e4053b00 0x7fb8e4055fc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7fb8f000d000 tx=0x7fb8f0009040).stop 2022-07-21T02:58:58.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb905cd5700 1 -- 172.21.15.140:0/3133003561 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb900060090 msgr2=0x7fb9001562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb905cd5700 1 --2- 172.21.15.140:0/3133003561 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb900060090 0x7fb9001562e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fb8e800c000 tx=0x7fb8e8008040).stop 2022-07-21T02:58:58.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb8fffff700 1 -- 172.21.15.140:0/3133003561 reap_dead start 2022-07-21T02:58:58.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb905cd5700 1 -- 172.21.15.140:0/3133003561 shutdown_connections 2022-07-21T02:58:58.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb905cd5700 1 -- 172.21.15.140:0/3133003561 >> 172.21.15.140:0/3133003561 conn(0x7fb900064d20 msgr2=0x7fb9001404f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:58.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb905cd5700 1 -- 172.21.15.140:0/3133003561 shutdown_connections 2022-07-21T02:58:58.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.263+0000 7fb905cd5700 1 -- 172.21.15.140:0/3133003561 wait complete. 2022-07-21T02:58:58.277 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2022-07-21T02:58:58.384 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a6719e700 1 Processor -- start 2022-07-21T02:58:58.385 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a6719e700 1 -- start start 2022-07-21T02:58:58.388 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a6719e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a600b7cd0 0x7f9a60065fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a6719e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9a600adff0 0x7f9a600664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a6719e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a60143f00 con 0x7f9a600ad4a0 2022-07-21T02:58:58.389 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a6719e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a60143020 con 0x7f9a600adff0 2022-07-21T02:58:58.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a6719e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a600b80b0 con 0x7f9a600b7cd0 2022-07-21T02:58:58.390 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a5ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9a600adff0 0x7f9a600664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.392 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a64f3a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a600b7cd0 0x7f9a60065fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a6573b700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9a600ad4a0 0x7f9a601462c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:57566/0 (socket says 172.21.15.140:57566) 2022-07-21T02:58:58.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a6573b700 1 -- 172.21.15.140:0/1841165864 learned_addr learned my addr 172.21.15.140:0/1841165864 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:58:58.394 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.383+0000 7f9a5ffff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9a600adff0 0x7f9a600664a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:48048/0 (socket says 172.21.15.140:48048) 2022-07-21T02:58:58.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a5ffff700 1 -- 172.21.15.140:0/1841165864 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a600b7cd0 msgr2=0x7f9a60065fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:58:58.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a5ffff700 1 --2- 172.21.15.140:0/1841165864 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a600b7cd0 0x7f9a60065fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.395 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a5ffff700 1 -- 172.21.15.140:0/1841165864 >> v1:172.21.15.152:6789/0 conn(0x7f9a600ad4a0 legacy=0x7f9a601462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a5ffff700 1 -- 172.21.15.140:0/1841165864 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a600acf20 con 0x7f9a600adff0 2022-07-21T02:58:58.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a5ffff700 1 --2- 172.21.15.140:0/1841165864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a600adff0 0x7f9a600664a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f9a4c002f70 tx=0x7f9a4c006df0).ready entity=mon.1 client_cookie=6f5fb220a9d1d961 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.396 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a5f7fe700 1 -- 172.21.15.140:0/1841165864 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a4c00d570 con 0x7f9a600adff0 2022-07-21T02:58:58.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a5f7fe700 1 -- 172.21.15.140:0/1841165864 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c000e90 con 0x7f9a600adff0 2022-07-21T02:58:58.397 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a5f7fe700 1 -- 172.21.15.140:0/1841165864 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a4c00d8c0 con 0x7f9a600adff0 2022-07-21T02:58:58.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a6719e700 1 -- 172.21.15.140:0/1841165864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a600adff0 msgr2=0x7f9a600664a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a6719e700 1 --2- 172.21.15.140:0/1841165864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a600adff0 0x7f9a600664a0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f9a4c002f70 tx=0x7f9a4c006df0).stop 2022-07-21T02:58:58.398 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a6719e700 1 -- 172.21.15.140:0/1841165864 shutdown_connections 2022-07-21T02:58:58.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a6719e700 1 --2- 172.21.15.140:0/1841165864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a600adff0 0x7f9a600664a0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a6719e700 1 --2- 172.21.15.140:0/1841165864 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a600b7cd0 0x7f9a60065fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.399 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a6719e700 1 -- 172.21.15.140:0/1841165864 >> 172.21.15.140:0/1841165864 conn(0x7f9a600657d0 msgr2=0x7f9a600b7280 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:58.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a6719e700 1 -- 172.21.15.140:0/1841165864 shutdown_connections 2022-07-21T02:58:58.400 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a6719e700 1 -- 172.21.15.140:0/1841165864 wait complete. 2022-07-21T02:58:58.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a6719e700 1 Processor -- start 2022-07-21T02:58:58.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.387+0000 7f9a6719e700 1 -- start start 2022-07-21T02:58:58.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6719e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a600ad4a0 0x7f9a60146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6719e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a600adff0 0x7f9a600b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6719e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a600b7cd0 0x7f9a601454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6719e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a600af9c0 con 0x7f9a600b7cd0 2022-07-21T02:58:58.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6719e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a600b0740 con 0x7f9a600adff0 2022-07-21T02:58:58.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6719e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a600b0c90 con 0x7f9a600ad4a0 2022-07-21T02:58:58.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6573b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a600b7cd0 0x7f9a601454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6573b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a600b7cd0 0x7f9a601454c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42314/0 (socket says 172.21.15.140:42314) 2022-07-21T02:58:58.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6573b700 1 -- 172.21.15.140:0/796729499 learned_addr learned my addr 172.21.15.140:0/796729499 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:58.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a64f3a700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a600ad4a0 0x7f9a60146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a5ffff700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a600adff0 0x7f9a600b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6573b700 1 -- 172.21.15.140:0/796729499 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a600ad4a0 msgr2=0x7f9a60146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6573b700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a600ad4a0 0x7f9a60146810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6573b700 1 -- 172.21.15.140:0/796729499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a600adff0 msgr2=0x7f9a600b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6573b700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a600adff0 0x7f9a600b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6573b700 1 -- 172.21.15.140:0/796729499 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c00b180 con 0x7f9a600b7cd0 2022-07-21T02:58:58.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6573b700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a600b7cd0 0x7f9a601454c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f9a60143dc0 tx=0x7f9a58007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a5dffb700 1 -- 172.21.15.140:0/796729499 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a58005ba0 con 0x7f9a600b7cd0 2022-07-21T02:58:58.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a5dffb700 1 -- 172.21.15.140:0/796729499 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a58005d00 con 0x7f9a600b7cd0 2022-07-21T02:58:58.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6719e700 1 -- 172.21.15.140:0/796729499 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a600b0f30 con 0x7f9a600b7cd0 2022-07-21T02:58:58.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a5dffb700 1 -- 172.21.15.140:0/796729499 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a580072c0 con 0x7f9a600b7cd0 2022-07-21T02:58:58.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6719e700 1 -- 172.21.15.140:0/796729499 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a600b13d0 con 0x7f9a600b7cd0 2022-07-21T02:58:58.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a6719e700 1 -- 172.21.15.140:0/796729499 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a600b0930 con 0x7f9a600b7cd0 2022-07-21T02:58:58.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.391+0000 7f9a5dffb700 1 -- 172.21.15.140:0/796729499 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f9a5801a060 con 0x7f9a600b7cd0 2022-07-21T02:58:58.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.395+0000 7f9a5dffb700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a40053b00 0x7f9a40055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.395+0000 7f9a5dffb700 1 -- 172.21.15.140:0/796729499 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f9a5806b5d0 con 0x7f9a600b7cd0 2022-07-21T02:58:58.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.395+0000 7f9a64f3a700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a40053b00 0x7f9a40055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.395+0000 7f9a64f3a700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a40053b00 0x7f9a40055fc0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f9a5400b000 tx=0x7f9a54006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.403+0000 7f9a5dffb700 1 -- 172.21.15.140:0/796729499 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f9a580427d0 con 0x7f9a600b7cd0 2022-07-21T02:58:58.539 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.531+0000 7f9a6719e700 1 -- 172.21.15.140:0/796729499 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a600b0b10 con 0x7f9a40053b00 2022-07-21T02:58:58.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a5dffb700 1 -- 172.21.15.140:0/796729499 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9a54000e90 con 0x7f9a40053b00 2022-07-21T02:58:58.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 -- 172.21.15.140:0/796729499 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a40053b00 msgr2=0x7f9a40055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a40053b00 0x7f9a40055fc0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f9a5400b000 tx=0x7f9a54006df0).stop 2022-07-21T02:58:58.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 -- 172.21.15.140:0/796729499 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a600b7cd0 msgr2=0x7f9a601454c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a600b7cd0 0x7f9a601454c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f9a60143dc0 tx=0x7f9a58007f10).stop 2022-07-21T02:58:58.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 -- 172.21.15.140:0/796729499 shutdown_connections 2022-07-21T02:58:58.542 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f9a40053b00 0x7f9a40055fc0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f9a600b7cd0 0x7f9a601454c0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9a600adff0 0x7f9a600b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.543 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 --2- 172.21.15.140:0/796729499 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9a600ad4a0 0x7f9a60146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 -- 172.21.15.140:0/796729499 >> 172.21.15.140:0/796729499 conn(0x7f9a600657d0 msgr2=0x7f9a600b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:58.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 -- 172.21.15.140:0/796729499 shutdown_connections 2022-07-21T02:58:58.544 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.535+0000 7f9a6719e700 1 -- 172.21.15.140:0/796729499 wait complete. 2022-07-21T02:58:58.545 INFO:teuthology.orchestra.run.smithi140.stderr:instructed osd(s) 0 to deep-scrub 2022-07-21T02:58:58.547 INFO:tasks.ceph:Scrubbing osd.1 2022-07-21T02:58:58.548 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2022-07-21T02:58:58.657 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.655+0000 7f83d2202700 1 Processor -- start 2022-07-21T02:58:58.658 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.655+0000 7f83d2202700 1 -- start start 2022-07-21T02:58:58.663 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83cc0b3cf0 0x7f83cc146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f83cc146530 0x7f83cc141a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83cc066b10 con 0x7f83cc0b6840 2022-07-21T02:58:58.664 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83cc0b5c90 con 0x7f83cc146530 2022-07-21T02:58:58.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0b6c20 con 0x7f83cc0b3cf0 2022-07-21T02:58:58.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d09ff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83cc0b3cf0 0x7f83cc146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.665 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d1a01700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f83cc146530 0x7f83cc141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d09ff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83cc0b3cf0 0x7f83cc146040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:39050/0 (socket says 172.21.15.140:39050) 2022-07-21T02:58:58.666 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d09ff700 1 -- 172.21.15.140:0/620551774 learned_addr learned my addr 172.21.15.140:0/620551774 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:58.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d1200700 1 --1- 172.21.15.140:0/620551774 >> v1:172.21.15.152:6789/0 conn(0x7f83cc0b6840 0x7f83cc0b5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:57578/0 (socket says 172.21.15.140:57578) 2022-07-21T02:58:58.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d1a01700 1 -- 172.21.15.140:0/620551774 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83cc0b3cf0 msgr2=0x7f83cc146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:58:58.667 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d1a01700 1 --2- 172.21.15.140:0/620551774 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83cc0b3cf0 0x7f83cc146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d1a01700 1 -- 172.21.15.140:0/620551774 >> v1:172.21.15.152:6789/0 conn(0x7f83cc0b6840 legacy=0x7f83cc0b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.668 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d1a01700 1 -- 172.21.15.140:0/620551774 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc0aead0 con 0x7f83cc146530 2022-07-21T02:58:58.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d1a01700 1 --2- 172.21.15.140:0/620551774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83cc146530 0x7f83cc141a90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f83c40032f0 tx=0x7f83c40091d0).ready entity=mon.1 client_cookie=4ba8cf5040e33154 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.669 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83c3fff700 1 -- 172.21.15.140:0/620551774 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83c400b6c0 con 0x7f83cc146530 2022-07-21T02:58:58.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83c3fff700 1 -- 172.21.15.140:0/620551774 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c400b820 con 0x7f83cc146530 2022-07-21T02:58:58.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83c3fff700 1 -- 172.21.15.140:0/620551774 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83c400b990 con 0x7f83cc146530 2022-07-21T02:58:58.670 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 -- 172.21.15.140:0/620551774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83cc146530 msgr2=0x7f83cc141a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 --2- 172.21.15.140:0/620551774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83cc146530 0x7f83cc141a90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f83c40032f0 tx=0x7f83c40091d0).stop 2022-07-21T02:58:58.672 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 -- 172.21.15.140:0/620551774 shutdown_connections 2022-07-21T02:58:58.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 --2- 172.21.15.140:0/620551774 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83cc0b3cf0 0x7f83cc146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 --2- 172.21.15.140:0/620551774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83cc146530 0x7f83cc141a90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.673 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 -- 172.21.15.140:0/620551774 >> 172.21.15.140:0/620551774 conn(0x7f83cc0b9e10 msgr2=0x7f83cc0b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:58.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 -- 172.21.15.140:0/620551774 shutdown_connections 2022-07-21T02:58:58.674 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.659+0000 7f83d2202700 1 -- 172.21.15.140:0/620551774 wait complete. 2022-07-21T02:58:58.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d2202700 1 Processor -- start 2022-07-21T02:58:58.675 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d2202700 1 -- start start 2022-07-21T02:58:58.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d2202700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83cc0b3cf0 0x7f83cc0b8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.679 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d2202700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83cc0b6840 0x7f83cc155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d2202700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83cc146530 0x7f83cc1151f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.680 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d2202700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc114dc0 con 0x7f83cc146530 2022-07-21T02:58:58.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d2202700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0b2110 con 0x7f83cc0b6840 2022-07-21T02:58:58.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d2202700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0b23b0 con 0x7f83cc0b3cf0 2022-07-21T02:58:58.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d1a01700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83cc146530 0x7f83cc1151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d1a01700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83cc146530 0x7f83cc1151f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42328/0 (socket says 172.21.15.140:42328) 2022-07-21T02:58:58.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d1a01700 1 -- 172.21.15.140:0/2811271399 learned_addr learned my addr 172.21.15.140:0/2811271399 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:58.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d1200700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83cc0b3cf0 0x7f83cc0b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d09ff700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83cc0b6840 0x7f83cc155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d1a01700 1 -- 172.21.15.140:0/2811271399 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83cc0b3cf0 msgr2=0x7f83cc0b8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d1a01700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83cc0b3cf0 0x7f83cc0b8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d1a01700 1 -- 172.21.15.140:0/2811271399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83cc0b6840 msgr2=0x7f83cc155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d1a01700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83cc0b6840 0x7f83cc155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d1a01700 1 -- 172.21.15.140:0/2811271399 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c4009470 con 0x7f83cc146530 2022-07-21T02:58:58.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d1200700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83cc0b3cf0 0x7f83cc0b8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T02:58:58.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d1a01700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83cc146530 0x7f83cc1151f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f83c40032f0 tx=0x7f83c4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83c27fc700 1 -- 172.21.15.140:0/2811271399 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83c4021440 con 0x7f83cc146530 2022-07-21T02:58:58.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d2202700 1 -- 172.21.15.140:0/2811271399 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83cc0b1d90 con 0x7f83cc146530 2022-07-21T02:58:58.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83c27fc700 1 -- 172.21.15.140:0/2811271399 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c40215a0 con 0x7f83cc146530 2022-07-21T02:58:58.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83c27fc700 1 -- 172.21.15.140:0/2811271399 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83c4021900 con 0x7f83cc146530 2022-07-21T02:58:58.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.663+0000 7f83d2202700 1 -- 172.21.15.140:0/2811271399 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83cc112b30 con 0x7f83cc146530 2022-07-21T02:58:58.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83d2202700 1 -- 172.21.15.140:0/2811271399 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83cc114f30 con 0x7f83cc146530 2022-07-21T02:58:58.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83c27fc700 1 -- 172.21.15.140:0/2811271399 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f83c400f070 con 0x7f83cc146530 2022-07-21T02:58:58.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83c27fc700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83b4053980 0x7f83b4055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83c27fc700 1 -- 172.21.15.140:0/2811271399 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f83c406c760 con 0x7f83cc146530 2022-07-21T02:58:58.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83c27fc700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] conn(0x7f83b405a110 0x7f83b405c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83d1200700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83b4053980 0x7f83b4055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83c27fc700 1 -- 172.21.15.140:0/2811271399 --> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83b405cbf0 con 0x7f83b405a110 2022-07-21T02:58:58.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83c27fc700 1 -- 172.21.15.140:0/2811271399 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83c406edd0 con 0x7f83cc146530 2022-07-21T02:58:58.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83d09ff700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] conn(0x7f83b405a110 0x7f83b405c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.691 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83d09ff700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] conn(0x7f83b405a110 0x7f83b405c570 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83d1200700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83b4053980 0x7f83b4055e40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f83bc00b000 tx=0x7f83bc0070e0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.692 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.667+0000 7f83c27fc700 1 -- 172.21.15.140:0/2811271399 <== osd.1 v2:172.21.15.140:6826/22712 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f83b405cbf0 con 0x7f83b405a110 2022-07-21T02:58:58.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d2202700 1 -- 172.21.15.140:0/2811271399 --> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f83cc1159a0 con 0x7f83b405a110 2022-07-21T02:58:58.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83c27fc700 1 -- 172.21.15.140:0/2811271399 <== osd.1 v2:172.21.15.140:6826/22712 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f83cc1159a0 con 0x7f83b405a110 2022-07-21T02:58:58.694 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T02:58:58.695 INFO:teuthology.orchestra.run.smithi140.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-07-21T02:58:58.695 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T02:58:58.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d2202700 1 -- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] conn(0x7f83b405a110 msgr2=0x7f83b405c570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d2202700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:6826/22712,v1:172.21.15.140:6827/22712] conn(0x7f83b405a110 0x7f83b405c570 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d2202700 1 -- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83b4053980 msgr2=0x7f83b4055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d2202700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f83b4053980 0x7f83b4055e40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f83bc00b000 tx=0x7f83bc0070e0).stop 2022-07-21T02:58:58.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d2202700 1 -- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83cc146530 msgr2=0x7f83cc1151f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d2202700 1 --2- 172.21.15.140:0/2811271399 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f83cc146530 0x7f83cc1151f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7f83c40032f0 tx=0x7f83c4016000).stop 2022-07-21T02:58:58.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d1a01700 1 -- 172.21.15.140:0/2811271399 reap_dead start 2022-07-21T02:58:58.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d2202700 1 -- 172.21.15.140:0/2811271399 shutdown_connections 2022-07-21T02:58:58.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d2202700 1 -- 172.21.15.140:0/2811271399 >> 172.21.15.140:0/2811271399 conn(0x7f83cc0b9e10 msgr2=0x7f83cc05bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:58.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d2202700 1 -- 172.21.15.140:0/2811271399 shutdown_connections 2022-07-21T02:58:58.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.683+0000 7f83d2202700 1 -- 172.21.15.140:0/2811271399 wait complete. 2022-07-21T02:58:58.701 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2022-07-21T02:58:58.810 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 Processor -- start 2022-07-21T02:58:58.811 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 -- start start 2022-07-21T02:58:58.818 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0b5a10 0x7f7b9c0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.819 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7b9c0ae780 0x7f7b9c0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c13ced0 con 0x7f7b9c0b5630 2022-07-21T02:58:58.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9c0b0a10 con 0x7f7b9c0ae780 2022-07-21T02:58:58.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9c0b0cb0 con 0x7f7b9c0b5a10 2022-07-21T02:58:58.823 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7b9c0ae780 0x7f7b9c0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7b9c0ae780 0x7f7b9c0aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42338/0 (socket says 172.21.15.140:42338) 2022-07-21T02:58:58.824 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9bfff700 1 -- 172.21.15.140:0/3587058046 learned_addr learned my addr 172.21.15.140:0/3587058046 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:58.825 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9affd700 1 --2- 172.21.15.140:0/3587058046 >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0b5a10 0x7f7b9c0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9bfff700 1 -- 172.21.15.140:0/3587058046 >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0b5a10 msgr2=0x7f7b9c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:58:58.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9bfff700 1 --2- 172.21.15.140:0/3587058046 >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0b5a10 0x7f7b9c0ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9bfff700 1 -- 172.21.15.140:0/3587058046 >> v1:172.21.15.140:6789/0 conn(0x7f7b9c0b5630 legacy=0x7f7b9c13c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9bfff700 1 -- 172.21.15.140:0/3587058046 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c0b0f50 con 0x7f7b9c0ae780 2022-07-21T02:58:58.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9bfff700 1 --2- 172.21.15.140:0/3587058046 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0ae780 0x7f7b9c0aebf0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f7b94002f70 tx=0x7f7b94006eb0).ready entity=mon.0 client_cookie=fa75ff1047a4ff51 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9a7fc700 1 -- 172.21.15.140:0/3587058046 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b9400d5c0 con 0x7f7b9c0ae780 2022-07-21T02:58:58.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9a7fc700 1 -- 172.21.15.140:0/3587058046 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b94000e90 con 0x7f7b9c0ae780 2022-07-21T02:58:58.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7b9a7fc700 1 -- 172.21.15.140:0/3587058046 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b9400d950 con 0x7f7b9c0ae780 2022-07-21T02:58:58.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 -- 172.21.15.140:0/3587058046 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0ae780 msgr2=0x7f7b9c0aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 --2- 172.21.15.140:0/3587058046 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0ae780 0x7f7b9c0aebf0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f7b94002f70 tx=0x7f7b94006eb0).stop 2022-07-21T02:58:58.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 -- 172.21.15.140:0/3587058046 shutdown_connections 2022-07-21T02:58:58.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 --2- 172.21.15.140:0/3587058046 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0ae780 0x7f7b9c0aebf0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 --2- 172.21.15.140:0/3587058046 >> v2:172.21.15.152:3300/0 conn(0x7f7b9c0b5a10 0x7f7b9c0ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 -- 172.21.15.140:0/3587058046 >> 172.21.15.140:0/3587058046 conn(0x7f7b9c063490 msgr2=0x7f7b9c0b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:58.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 -- 172.21.15.140:0/3587058046 shutdown_connections 2022-07-21T02:58:58.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.811+0000 7f7ba098d700 1 -- 172.21.15.140:0/3587058046 wait complete. 2022-07-21T02:58:58.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7ba098d700 1 Processor -- start 2022-07-21T02:58:58.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7ba098d700 1 -- start start 2022-07-21T02:58:58.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7ba098d700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0ae780 0x7f7b9c151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.834 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7ba098d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c0b5630 0x7f7b9c13eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7ba098d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c0b5a10 0x7f7b9c13d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7ba098d700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c13de90 con 0x7f7b9c0ae780 2022-07-21T02:58:58.835 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7ba098d700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c13e0c0 con 0x7f7b9c0b5630 2022-07-21T02:58:58.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7ba098d700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c13e360 con 0x7f7b9c0b5a10 2022-07-21T02:58:58.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0ae780 0x7f7b9c151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.836 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0ae780 0x7f7b9c151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42340/0 (socket says 172.21.15.140:42340) 2022-07-21T02:58:58.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9b7fe700 1 -- 172.21.15.140:0/2772141682 learned_addr learned my addr 172.21.15.140:0/2772141682 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:58.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9bfff700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c0b5a10 0x7f7b9c13d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.837 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9affd700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c0b5630 0x7f7b9c13eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9b7fe700 1 -- 172.21.15.140:0/2772141682 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c0b5a10 msgr2=0x7f7b9c13d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9b7fe700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c0b5a10 0x7f7b9c13d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.838 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9b7fe700 1 -- 172.21.15.140:0/2772141682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c0b5630 msgr2=0x7f7b9c13eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9b7fe700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c0b5630 0x7f7b9c13eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.839 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9b7fe700 1 -- 172.21.15.140:0/2772141682 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9400b1e0 con 0x7f7b9c0ae780 2022-07-21T02:58:58.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9bfff700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c0b5a10 0x7f7b9c13d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T02:58:58.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b9b7fe700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0ae780 0x7f7b9c151dd0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7f7b9c0ae170 tx=0x7f7b8c0014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.841 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b98ff9700 1 -- 172.21.15.140:0/2772141682 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8c00e4a0 con 0x7f7b9c0ae780 2022-07-21T02:58:58.842 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b98ff9700 1 -- 172.21.15.140:0/2772141682 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00e600 con 0x7f7b9c0ae780 2022-07-21T02:58:58.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7ba098d700 1 -- 172.21.15.140:0/2772141682 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c13e600 con 0x7f7b9c0ae780 2022-07-21T02:58:58.843 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7b98ff9700 1 -- 172.21.15.140:0/2772141682 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8c00e960 con 0x7f7b9c0ae780 2022-07-21T02:58:58.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.815+0000 7f7ba098d700 1 -- 172.21.15.140:0/2772141682 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c13f350 con 0x7f7b9c0ae780 2022-07-21T02:58:58.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.819+0000 7f7ba098d700 1 -- 172.21.15.140:0/2772141682 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c0574e0 con 0x7f7b9c0ae780 2022-07-21T02:58:58.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.819+0000 7f7b98ff9700 1 -- 172.21.15.140:0/2772141682 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7b8c019030 con 0x7f7b9c0ae780 2022-07-21T02:58:58.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.819+0000 7f7b98ff9700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 0x7f7b80055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:58.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.819+0000 7f7b98ff9700 1 -- 172.21.15.140:0/2772141682 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f7b8c06b8a0 con 0x7f7b9c0ae780 2022-07-21T02:58:58.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.819+0000 7f7b9affd700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 0x7f7b80055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:58.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.819+0000 7f7b9affd700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 0x7f7b80055fc0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f7b8400b000 tx=0x7f7b84006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:58.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.823+0000 7f7b98ff9700 1 -- 172.21.15.140:0/2772141682 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7b8c042930 con 0x7f7b9c0ae780 2022-07-21T02:58:58.947 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 -- 172.21.15.140:0/2772141682 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b9c05db10 con 0x7f7b80053b00 2022-07-21T02:58:58.948 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7b98ff9700 1 -- 172.21.15.140:0/2772141682 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7b9c05db10 con 0x7f7b80053b00 2022-07-21T02:58:58.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 -- 172.21.15.140:0/2772141682 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 msgr2=0x7f7b80055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.950 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 0x7f7b80055fc0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f7b8400b000 tx=0x7f7b84006df0).stop 2022-07-21T02:58:58.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 -- 172.21.15.140:0/2772141682 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0ae780 msgr2=0x7f7b9c151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:58.951 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0ae780 0x7f7b9c151dd0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7f7b9c0ae170 tx=0x7f7b8c0014a0).stop 2022-07-21T02:58:58.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 -- 172.21.15.140:0/2772141682 shutdown_connections 2022-07-21T02:58:58.952 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7b80053b00 0x7f7b80055fc0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.953 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7b9c0b5a10 0x7f7b9c13d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.962 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b9c0b5630 0x7f7b9c13eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 --2- 172.21.15.140:0/2772141682 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7b9c0ae780 0x7f7b9c151dd0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 -- 172.21.15.140:0/2772141682 >> 172.21.15.140:0/2772141682 conn(0x7f7b9c063490 msgr2=0x7f7b9c13a310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:58.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 -- 172.21.15.140:0/2772141682 shutdown_connections 2022-07-21T02:58:58.965 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:58.946+0000 7f7ba098d700 1 -- 172.21.15.140:0/2772141682 wait complete. 2022-07-21T02:58:58.975 INFO:teuthology.orchestra.run.smithi140.stderr:instructed osd(s) 1 to deep-scrub 2022-07-21T02:58:58.976 INFO:tasks.ceph:Scrubbing osd.2 2022-07-21T02:58:58.977 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2022-07-21T02:58:59.101 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.098+0000 7f97eb59e700 1 Processor -- start 2022-07-21T02:58:59.102 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.098+0000 7f97eb59e700 1 -- start start 2022-07-21T02:58:59.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f97ec0b3cf0 0x7f97ec146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec066b10 con 0x7f97ec0b6840 2022-07-21T02:58:59.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec0b6c20 con 0x7f97ec146530 2022-07-21T02:58:59.118 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ec0b5c90 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97ea59c700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f97ec0b6840 0x7f97ec0b5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56494/0 (socket says 172.21.15.140:56494) 2022-07-21T02:58:59.119 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97ea59c700 1 -- 172.21.15.140:0/2672804371 learned_addr learned my addr 172.21.15.140:0/2672804371 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:58:59.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e9d9b700 1 --2- 172.21.15.140:0/2672804371 >> v2:172.21.15.152:3300/0 conn(0x7f97ec0b3cf0 0x7f97ec146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.120 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e959a700 1 -- 172.21.15.140:0/2672804371 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176209609 0 0) 0x7f97ec066b10 con 0x7f97ec0b6840 2022-07-21T02:58:59.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e959a700 1 -- 172.21.15.140:0/2672804371 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97cc003260 con 0x7f97ec0b6840 2022-07-21T02:58:59.121 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e959a700 1 -- 172.21.15.140:0/2672804371 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531807473 0 0) 0x7f97ec0b6c20 con 0x7f97ec146530 2022-07-21T02:58:59.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e959a700 1 -- 172.21.15.140:0/2672804371 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97ec066b10 con 0x7f97ec146530 2022-07-21T02:58:59.122 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e9d9b700 1 -- 172.21.15.140:0/2672804371 >> v1:172.21.15.152:6789/0 conn(0x7f97ec146530 legacy=0x7f97ec141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e9d9b700 1 -- 172.21.15.140:0/2672804371 >> v1:172.21.15.140:6789/0 conn(0x7f97ec0b6840 legacy=0x7f97ec0b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e959a700 1 -- 172.21.15.140:0/2672804371 <== mon.0 v1:172.21.15.140:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641217958 0 0) 0x7f97cc003260 con 0x7f97ec0b6840 2022-07-21T02:58:59.123 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e9d9b700 1 -- 172.21.15.140:0/2672804371 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec0aead0 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e9d9b700 1 --2- 172.21.15.140:0/2672804371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec0b3cf0 0x7f97ec146040 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f97dc001110 tx=0x7f97d4006df0).ready entity=mon.1 client_cookie=e19d0ed816028bff server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e959a700 1 -- 172.21.15.140:0/2672804371 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97dc001680 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.124 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e959a700 1 -- 172.21.15.140:0/2672804371 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d4000e90 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97e959a700 1 -- 172.21.15.140:0/2672804371 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97d4012460 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 -- 172.21.15.140:0/2672804371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec0b3cf0 msgr2=0x7f97ec146040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.125 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 --2- 172.21.15.140:0/2672804371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec0b3cf0 0x7f97ec146040 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f97dc001110 tx=0x7f97d4006df0).stop 2022-07-21T02:58:59.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 -- 172.21.15.140:0/2672804371 shutdown_connections 2022-07-21T02:58:59.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 --2- 172.21.15.140:0/2672804371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec0b3cf0 0x7f97ec146040 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.126 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 -- 172.21.15.140:0/2672804371 >> 172.21.15.140:0/2672804371 conn(0x7f97ec0b9e10 msgr2=0x7f97ec0b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:59.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 -- 172.21.15.140:0/2672804371 shutdown_connections 2022-07-21T02:58:59.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 -- 172.21.15.140:0/2672804371 wait complete. 2022-07-21T02:58:59.127 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.102+0000 7f97eb59e700 1 Processor -- start 2022-07-21T02:58:59.128 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97eb59e700 1 -- start start 2022-07-21T02:58:59.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97eb59e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97ec0b3cf0 0x7f97ec1134d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.129 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97eb59e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97ec0b6840 0x7f97ec0aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97eb59e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec146530 0x7f97ec0b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97eb59e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0b18a0 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97eb59e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec152a50 con 0x7f97ec146530 2022-07-21T02:58:59.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97eb59e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0b20b0 con 0x7f97ec0b6840 2022-07-21T02:58:59.131 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97ea59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97ec0b3cf0 0x7f97ec1134d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97ea59c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97ec0b3cf0 0x7f97ec1134d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42354/0 (socket says 172.21.15.140:42354) 2022-07-21T02:58:59.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97ea59c700 1 -- 172.21.15.140:0/3392311826 learned_addr learned my addr 172.21.15.140:0/3392311826 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:59.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97e9d9b700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97ec0b6840 0x7f97ec0aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97ead9d700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec146530 0x7f97ec0b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97ea59c700 1 -- 172.21.15.140:0/3392311826 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97ec0b6840 msgr2=0x7f97ec0aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97ea59c700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97ec0b6840 0x7f97ec0aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97ea59c700 1 -- 172.21.15.140:0/3392311826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec146530 msgr2=0x7f97ec0b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97ea59c700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec146530 0x7f97ec0b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97ea59c700 1 -- 172.21.15.140:0/3392311826 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d400b180 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97ead9d700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97ec146530 0x7f97ec0b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T02:58:59.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97ea59c700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97ec0b3cf0 0x7f97ec1134d0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f97dc001860 tx=0x7f97dc001e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97e37fe700 1 -- 172.21.15.140:0/3392311826 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97dc007170 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97e37fe700 1 -- 172.21.15.140:0/3392311826 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc0072d0 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97e37fe700 1 -- 172.21.15.140:0/3392311826 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97dc004920 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97eb59e700 1 -- 172.21.15.140:0/3392311826 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec0b2350 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97eb59e700 1 -- 172.21.15.140:0/3392311826 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec0b08e0 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.106+0000 7f97eb59e700 1 -- 172.21.15.140:0/3392311826 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97ec0aed70 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.110+0000 7f97e37fe700 1 -- 172.21.15.140:0/3392311826 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f97dc01b070 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.110+0000 7f97e37fe700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f97d00539e0 0x7f97d0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.110+0000 7f97e37fe700 1 -- 172.21.15.140:0/3392311826 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f97dc06b490 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.110+0000 7f97e9d9b700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f97d00539e0 0x7f97d0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.110+0000 7f97e37fe700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] conn(0x7f97d005a0d0 0x7f97d005c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.110+0000 7f97e37fe700 1 -- 172.21.15.140:0/3392311826 --> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97d005cbb0 con 0x7f97d005a0d0 2022-07-21T02:58:59.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.110+0000 7f97e37fe700 1 -- 172.21.15.140:0/3392311826 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97dc06da10 con 0x7f97ec0b3cf0 2022-07-21T02:58:59.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.110+0000 7f97ead9d700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] conn(0x7f97d005a0d0 0x7f97d005c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.110+0000 7f97e9d9b700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f97d00539e0 0x7f97d0055ea0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f97d4018000 tx=0x7f97d4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.110+0000 7f97ead9d700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] conn(0x7f97d005a0d0 0x7f97d005c530 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.110+0000 7f97e37fe700 1 -- 172.21.15.140:0/3392311826 <== osd.2 v2:172.21.15.140:6810/22724 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f97d005cbb0 con 0x7f97d005a0d0 2022-07-21T02:58:59.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97eb59e700 1 -- 172.21.15.140:0/3392311826 --> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f97ec153140 con 0x7f97d005a0d0 2022-07-21T02:58:59.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97e37fe700 1 -- 172.21.15.140:0/3392311826 <== osd.2 v2:172.21.15.140:6810/22724 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f97ec153140 con 0x7f97d005a0d0 2022-07-21T02:58:59.144 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T02:58:59.144 INFO:teuthology.orchestra.run.smithi140.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-07-21T02:58:59.145 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T02:58:59.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97eb59e700 1 -- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] conn(0x7f97d005a0d0 msgr2=0x7f97d005c530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97eb59e700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:6810/22724,v1:172.21.15.140:6811/22724] conn(0x7f97d005a0d0 0x7f97d005c530 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97eb59e700 1 -- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f97d00539e0 msgr2=0x7f97d0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97eb59e700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f97d00539e0 0x7f97d0055ea0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7f97d4018000 tx=0x7f97d4014000).stop 2022-07-21T02:58:59.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97eb59e700 1 -- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97ec0b3cf0 msgr2=0x7f97ec1134d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97eb59e700 1 --2- 172.21.15.140:0/3392311826 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f97ec0b3cf0 0x7f97ec1134d0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7f97dc001860 tx=0x7f97dc001e20).stop 2022-07-21T02:58:59.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97ead9d700 1 -- 172.21.15.140:0/3392311826 reap_dead start 2022-07-21T02:58:59.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97eb59e700 1 -- 172.21.15.140:0/3392311826 shutdown_connections 2022-07-21T02:58:59.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97eb59e700 1 -- 172.21.15.140:0/3392311826 >> 172.21.15.140:0/3392311826 conn(0x7f97ec0b9e10 msgr2=0x7f97ec141a00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:59.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97eb59e700 1 -- 172.21.15.140:0/3392311826 shutdown_connections 2022-07-21T02:58:59.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.126+0000 7f97eb59e700 1 -- 172.21.15.140:0/3392311826 wait complete. 2022-07-21T02:58:59.152 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2022-07-21T02:58:59.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7aa305e700 1 Processor -- start 2022-07-21T02:58:59.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7aa305e700 1 -- start start 2022-07-21T02:58:59.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7aa305e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7a9c097a80 0x7f7a9c061080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7aa305e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7a9c0afb80 0x7f7a9c14b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7aa305e700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9c14a750 con 0x7f7a9c060ca0 2022-07-21T02:58:59.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7aa305e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9c148430 con 0x7f7a9c097a80 2022-07-21T02:58:59.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7aa305e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9c0b2840 con 0x7f7a9c0afb80 2022-07-21T02:58:59.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7a9bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7a9c097a80 0x7f7a9c061080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7a9bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f7a9c097a80 0x7f7a9c061080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42364/0 (socket says 172.21.15.140:42364) 2022-07-21T02:58:59.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7a9bfff700 1 -- 172.21.15.140:0/104035400 learned_addr learned my addr 172.21.15.140:0/104035400 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:59.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7aa15fb700 1 --2- 172.21.15.140:0/104035400 >> v2:172.21.15.152:3300/0 conn(0x7f7a9c0afb80 0x7f7a9c14b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7a9bfff700 1 -- 172.21.15.140:0/104035400 >> v2:172.21.15.152:3300/0 conn(0x7f7a9c0afb80 msgr2=0x7f7a9c14b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:58:59.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7a9bfff700 1 --2- 172.21.15.140:0/104035400 >> v2:172.21.15.152:3300/0 conn(0x7f7a9c0afb80 0x7f7a9c14b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7a9bfff700 1 -- 172.21.15.140:0/104035400 >> v1:172.21.15.140:6789/0 conn(0x7f7a9c060ca0 legacy=0x7f7a9c14ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7a9bfff700 1 -- 172.21.15.140:0/104035400 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c0ac110 con 0x7f7a9c097a80 2022-07-21T02:58:59.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7aa15fb700 1 --2- 172.21.15.140:0/104035400 >> v2:172.21.15.152:3300/0 conn(0x7f7a9c0afb80 0x7f7a9c14b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T02:58:59.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7a9bfff700 1 --2- 172.21.15.140:0/104035400 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a9c097a80 0x7f7a9c061080 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f7a940032f0 tx=0x7f7a94009290).ready entity=mon.0 client_cookie=bc015e328b9be022 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.258+0000 7f7a9b7fe700 1 -- 172.21.15.140:0/104035400 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a9400b870 con 0x7f7a9c097a80 2022-07-21T02:58:59.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7a9b7fe700 1 -- 172.21.15.140:0/104035400 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9400b9d0 con 0x7f7a9c097a80 2022-07-21T02:58:59.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7a9b7fe700 1 -- 172.21.15.140:0/104035400 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a9400bb40 con 0x7f7a9c097a80 2022-07-21T02:58:59.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 -- 172.21.15.140:0/104035400 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a9c097a80 msgr2=0x7f7a9c061080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 --2- 172.21.15.140:0/104035400 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a9c097a80 0x7f7a9c061080 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 rx=0x7f7a940032f0 tx=0x7f7a94009290).stop 2022-07-21T02:58:59.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 -- 172.21.15.140:0/104035400 shutdown_connections 2022-07-21T02:58:59.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 --2- 172.21.15.140:0/104035400 >> v2:172.21.15.152:3300/0 conn(0x7f7a9c0afb80 0x7f7a9c14b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 --2- 172.21.15.140:0/104035400 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a9c097a80 0x7f7a9c061080 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 -- 172.21.15.140:0/104035400 >> 172.21.15.140:0/104035400 conn(0x7f7a9c0b5cb0 msgr2=0x7f7a9c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:59.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 -- 172.21.15.140:0/104035400 shutdown_connections 2022-07-21T02:58:59.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 -- 172.21.15.140:0/104035400 wait complete. 2022-07-21T02:58:59.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 Processor -- start 2022-07-21T02:58:59.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 -- start start 2022-07-21T02:58:59.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a9c060ca0 0x7f7a9c05c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a9c097a80 0x7f7a9c140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a9c0afb80 0x7f7a9c141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c14cbd0 con 0x7f7a9c060ca0 2022-07-21T02:58:59.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c14bf50 con 0x7f7a9c0afb80 2022-07-21T02:58:59.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa305e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c14c1f0 con 0x7f7a9c097a80 2022-07-21T02:58:59.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa0dfa700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a9c060ca0 0x7f7a9c05c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa0dfa700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a9c060ca0 0x7f7a9c05c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42370/0 (socket says 172.21.15.140:42370) 2022-07-21T02:58:59.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa0dfa700 1 -- 172.21.15.140:0/2488208880 learned_addr learned my addr 172.21.15.140:0/2488208880 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:59.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7a9bfff700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a9c097a80 0x7f7a9c140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.262+0000 7f7aa15fb700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a9c0afb80 0x7f7a9c141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7a9bfff700 1 -- 172.21.15.140:0/2488208880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a9c0afb80 msgr2=0x7f7a9c141300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7a9bfff700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a9c0afb80 0x7f7a9c141300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7a9bfff700 1 -- 172.21.15.140:0/2488208880 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a9c060ca0 msgr2=0x7f7a9c05c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7a9bfff700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a9c060ca0 0x7f7a9c05c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7a9bfff700 1 -- 172.21.15.140:0/2488208880 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a94009530 con 0x7f7a9c097a80 2022-07-21T02:58:59.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7a9bfff700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a9c097a80 0x7f7a9c140e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f7a940032f0 tx=0x7f7a94016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7a99ffb700 1 -- 172.21.15.140:0/2488208880 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a94021440 con 0x7f7a9c097a80 2022-07-21T02:58:59.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7a99ffb700 1 -- 172.21.15.140:0/2488208880 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a940215a0 con 0x7f7a9c097a80 2022-07-21T02:58:59.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7aa305e700 1 -- 172.21.15.140:0/2488208880 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c14c490 con 0x7f7a9c097a80 2022-07-21T02:58:59.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7a99ffb700 1 -- 172.21.15.140:0/2488208880 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a94021900 con 0x7f7a9c097a80 2022-07-21T02:58:59.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7aa305e700 1 -- 172.21.15.140:0/2488208880 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c1417f0 con 0x7f7a9c097a80 2022-07-21T02:58:59.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7aa305e700 1 -- 172.21.15.140:0/2488208880 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9c0abe40 con 0x7f7a9c097a80 2022-07-21T02:58:59.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7a99ffb700 1 -- 172.21.15.140:0/2488208880 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f7a94027020 con 0x7f7a9c097a80 2022-07-21T02:58:59.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.266+0000 7f7a99ffb700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a7c053aa0 0x7f7a7c055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.270+0000 7f7a99ffb700 1 -- 172.21.15.140:0/2488208880 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f7a9406d6a0 con 0x7f7a9c097a80 2022-07-21T02:58:59.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.270+0000 7f7aa0dfa700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a7c053aa0 0x7f7a7c055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.270+0000 7f7aa0dfa700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a7c053aa0 0x7f7a7c055f60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f7a9000d000 tx=0x7f7a90000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.290 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.278+0000 7f7a99ffb700 1 -- 172.21.15.140:0/2488208880 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f7a94044710 con 0x7f7a9c097a80 2022-07-21T02:58:59.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.402+0000 7f7aa305e700 1 -- 172.21.15.140:0/2488208880 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a9c14c930 con 0x7f7a7c053aa0 2022-07-21T02:58:59.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.402+0000 7f7a99ffb700 1 -- 172.21.15.140:0/2488208880 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7a90015410 con 0x7f7a7c053aa0 2022-07-21T02:58:59.410 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 -- 172.21.15.140:0/2488208880 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a7c053aa0 msgr2=0x7f7a7c055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a7c053aa0 0x7f7a7c055f60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f7a9000d000 tx=0x7f7a90000bc0).stop 2022-07-21T02:58:59.411 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 -- 172.21.15.140:0/2488208880 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a9c097a80 msgr2=0x7f7a9c140e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.412 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a9c097a80 0x7f7a9c140e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f7a940032f0 tx=0x7f7a94016000).stop 2022-07-21T02:58:59.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 -- 172.21.15.140:0/2488208880 shutdown_connections 2022-07-21T02:58:59.413 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f7a7c053aa0 0x7f7a7c055f60 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.414 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a9c0afb80 0x7f7a9c141300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7a9c097a80 0x7f7a9c140e10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.415 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 --2- 172.21.15.140:0/2488208880 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f7a9c060ca0 0x7f7a9c05c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 -- 172.21.15.140:0/2488208880 >> 172.21.15.140:0/2488208880 conn(0x7f7a9c0b5cb0 msgr2=0x7f7a9c146b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:59.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 -- 172.21.15.140:0/2488208880 shutdown_connections 2022-07-21T02:58:59.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.406+0000 7f7aa305e700 1 -- 172.21.15.140:0/2488208880 wait complete. 2022-07-21T02:58:59.421 INFO:teuthology.orchestra.run.smithi140.stderr:instructed osd(s) 2 to deep-scrub 2022-07-21T02:58:59.425 INFO:tasks.ceph:Scrubbing osd.3 2022-07-21T02:58:59.426 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2022-07-21T02:58:59.540 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.538+0000 7fc55446f700 1 Processor -- start 2022-07-21T02:58:59.541 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.538+0000 7fc55446f700 1 -- start start 2022-07-21T02:58:59.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.538+0000 7fc55446f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc54c062f30 0x7fc54c0adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.554 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.538+0000 7fc55446f700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc54c142b30 0x7fc54c142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.555 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.538+0000 7fc55446f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c1462c0 con 0x7fc54c05cf40 2022-07-21T02:58:59.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.538+0000 7fc55446f700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c147110 con 0x7fc54c142b30 2022-07-21T02:58:59.556 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.538+0000 7fc55446f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0ae040 con 0x7fc54c062f30 2022-07-21T02:58:59.557 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.538+0000 7fc552a0c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc54c142b30 0x7fc54c142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.538+0000 7fc552a0c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc54c142b30 0x7fc54c142fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42382/0 (socket says 172.21.15.140:42382) 2022-07-21T02:58:59.558 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.538+0000 7fc552a0c700 1 -- 172.21.15.140:0/2675371630 learned_addr learned my addr 172.21.15.140:0/2675371630 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:59.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.538+0000 7fc551a0a700 1 --2- 172.21.15.140:0/2675371630 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc54c062f30 0x7fc54c0adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.559 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc552a0c700 1 -- 172.21.15.140:0/2675371630 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc54c062f30 msgr2=0x7fc54c0adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:58:59.560 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc552a0c700 1 --2- 172.21.15.140:0/2675371630 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc54c062f30 0x7fc54c0adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc552a0c700 1 -- 172.21.15.140:0/2675371630 >> v1:172.21.15.152:6789/0 conn(0x7fc54c05cf40 legacy=0x7fc54c146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.561 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc552a0c700 1 -- 172.21.15.140:0/2675371630 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c146560 con 0x7fc54c142b30 2022-07-21T02:58:59.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc552a0c700 1 --2- 172.21.15.140:0/2675371630 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc54c142b30 0x7fc54c142fa0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7fc5440032f0 tx=0x7fc544009290).ready entity=mon.0 client_cookie=be21794396bda9e0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.562 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc551209700 1 -- 172.21.15.140:0/2675371630 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc54400b8a0 con 0x7fc54c142b30 2022-07-21T02:58:59.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc551209700 1 -- 172.21.15.140:0/2675371630 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54400ba00 con 0x7fc54c142b30 2022-07-21T02:58:59.563 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc551209700 1 -- 172.21.15.140:0/2675371630 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc54400bb70 con 0x7fc54c142b30 2022-07-21T02:58:59.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc55446f700 1 -- 172.21.15.140:0/2675371630 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc54c142b30 msgr2=0x7fc54c142fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc55446f700 1 --2- 172.21.15.140:0/2675371630 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc54c142b30 0x7fc54c142fa0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7fc5440032f0 tx=0x7fc544009290).stop 2022-07-21T02:58:59.565 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc55446f700 1 -- 172.21.15.140:0/2675371630 shutdown_connections 2022-07-21T02:58:59.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc55446f700 1 --2- 172.21.15.140:0/2675371630 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc54c142b30 0x7fc54c142fa0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc55446f700 1 --2- 172.21.15.140:0/2675371630 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc54c062f30 0x7fc54c0adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc55446f700 1 -- 172.21.15.140:0/2675371630 >> 172.21.15.140:0/2675371630 conn(0x7fc54c0b7d00 msgr2=0x7fc54c0b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:59.567 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc55446f700 1 -- 172.21.15.140:0/2675371630 shutdown_connections 2022-07-21T02:58:59.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc55446f700 1 -- 172.21.15.140:0/2675371630 wait complete. 2022-07-21T02:58:59.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc55446f700 1 Processor -- start 2022-07-21T02:58:59.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.542+0000 7fc55446f700 1 -- start start 2022-07-21T02:58:59.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc55446f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc54c05cf40 0x7fc54c155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc55446f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc54c062f30 0x7fc54c0660a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc55446f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc54c142b30 0x7fc54c0ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc55446f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c09ae60 con 0x7fc54c062f30 2022-07-21T02:58:59.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc55446f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c09b100 con 0x7fc54c05cf40 2022-07-21T02:58:59.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc55446f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c066590 con 0x7fc54c142b30 2022-07-21T02:58:59.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc551a0a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc54c062f30 0x7fc54c0660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc551a0a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc54c062f30 0x7fc54c0660a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42386/0 (socket says 172.21.15.140:42386) 2022-07-21T02:58:59.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc551a0a700 1 -- 172.21.15.140:0/308209851 learned_addr learned my addr 172.21.15.140:0/308209851 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:59.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc55220b700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc54c05cf40 0x7fc54c155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc552a0c700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc54c142b30 0x7fc54c0ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc551a0a700 1 -- 172.21.15.140:0/308209851 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc54c142b30 msgr2=0x7fc54c0ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc551a0a700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc54c142b30 0x7fc54c0ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc551a0a700 1 -- 172.21.15.140:0/308209851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc54c05cf40 msgr2=0x7fc54c155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc551a0a700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc54c05cf40 0x7fc54c155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc551a0a700 1 -- 172.21.15.140:0/308209851 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc544009530 con 0x7fc54c062f30 2022-07-21T02:58:59.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc551a0a700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc54c062f30 0x7fc54c0660a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7fc54800c000 tx=0x7fc548006d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc53b7fe700 1 -- 172.21.15.140:0/308209851 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5480104c0 con 0x7fc54c062f30 2022-07-21T02:58:59.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc53b7fe700 1 -- 172.21.15.140:0/308209851 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc548010620 con 0x7fc54c062f30 2022-07-21T02:58:59.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc55446f700 1 -- 172.21.15.140:0/308209851 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c09bfb0 con 0x7fc54c062f30 2022-07-21T02:58:59.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc53b7fe700 1 -- 172.21.15.140:0/308209851 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc548010980 con 0x7fc54c062f30 2022-07-21T02:58:59.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc55446f700 1 -- 172.21.15.140:0/308209851 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c09abe0 con 0x7fc54c062f30 2022-07-21T02:58:59.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.546+0000 7fc55446f700 1 -- 172.21.15.140:0/308209851 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc54c09b300 con 0x7fc54c062f30 2022-07-21T02:58:59.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.550+0000 7fc53b7fe700 1 -- 172.21.15.140:0/308209851 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc54801a030 con 0x7fc54c062f30 2022-07-21T02:58:59.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.550+0000 7fc53b7fe700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc52c053950 0x7fc52c055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.550+0000 7fc53b7fe700 1 -- 172.21.15.140:0/308209851 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fc54806baf0 con 0x7fc54c062f30 2022-07-21T02:58:59.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.550+0000 7fc55220b700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc52c053950 0x7fc52c055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.550+0000 7fc53b7fe700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] conn(0x7fc52c05a0e0 0x7fc52c05c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.550+0000 7fc53b7fe700 1 -- 172.21.15.140:0/308209851 --> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc52c05cbc0 con 0x7fc52c05a0e0 2022-07-21T02:58:59.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.550+0000 7fc53b7fe700 1 -- 172.21.15.140:0/308209851 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc548070240 con 0x7fc54c062f30 2022-07-21T02:58:59.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.550+0000 7fc552a0c700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] conn(0x7fc52c05a0e0 0x7fc52c05c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.550+0000 7fc55220b700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc52c053950 0x7fc52c055e10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fc53c00d000 tx=0x7fc53c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.550+0000 7fc552a0c700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] conn(0x7fc52c05a0e0 0x7fc52c05c540 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.550+0000 7fc53b7fe700 1 -- 172.21.15.140:0/308209851 <== osd.3 v2:172.21.15.140:6818/22721 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7fc52c05cbc0 con 0x7fc52c05a0e0 2022-07-21T02:58:59.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.562+0000 7fc55446f700 1 -- 172.21.15.140:0/308209851 --> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc54c0574e0 con 0x7fc52c05a0e0 2022-07-21T02:58:59.584 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc53b7fe700 1 -- 172.21.15.140:0/308209851 <== osd.3 v2:172.21.15.140:6818/22721 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc54c0574e0 con 0x7fc52c05a0e0 2022-07-21T02:58:59.585 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T02:58:59.585 INFO:teuthology.orchestra.run.smithi140.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-07-21T02:58:59.585 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T02:58:59.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc55446f700 1 -- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] conn(0x7fc52c05a0e0 msgr2=0x7fc52c05c540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc55446f700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:6818/22721,v1:172.21.15.140:6819/22721] conn(0x7fc52c05a0e0 0x7fc52c05c540 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc55446f700 1 -- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc52c053950 msgr2=0x7fc52c055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc55446f700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc52c053950 0x7fc52c055e10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fc53c00d000 tx=0x7fc53c009040).stop 2022-07-21T02:58:59.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc55446f700 1 -- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc54c062f30 msgr2=0x7fc54c0660a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc55446f700 1 --2- 172.21.15.140:0/308209851 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc54c062f30 0x7fc54c0660a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7fc54800c000 tx=0x7fc548006d90).stop 2022-07-21T02:58:59.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc552a0c700 1 -- 172.21.15.140:0/308209851 reap_dead start 2022-07-21T02:58:59.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc55446f700 1 -- 172.21.15.140:0/308209851 shutdown_connections 2022-07-21T02:58:59.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc55446f700 1 -- 172.21.15.140:0/308209851 >> 172.21.15.140:0/308209851 conn(0x7fc54c0b7d00 msgr2=0x7fc54c1456d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:59.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc55446f700 1 -- 172.21.15.140:0/308209851 shutdown_connections 2022-07-21T02:58:59.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.566+0000 7fc55446f700 1 -- 172.21.15.140:0/308209851 wait complete. 2022-07-21T02:58:59.592 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2022-07-21T02:58:59.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.698+0000 7f6006a59700 1 Processor -- start 2022-07-21T02:58:59.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.698+0000 7f6006a59700 1 -- start start 2022-07-21T02:58:59.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006a59700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f60000b5970 0x7f60000ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006a59700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60000b0b80 con 0x7f6000143a10 2022-07-21T02:58:59.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006a59700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600005d4d0 con 0x7f60000af8c0 2022-07-21T02:58:59.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006a59700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60000b5d50 con 0x7f60000b5970 2022-07-21T02:58:59.705 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6005a57700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f60000b5970 0x7f60000ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006258700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f6000143a10 0x7f6000143e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56544/0 (socket says 172.21.15.140:56544) 2022-07-21T02:58:59.706 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006258700 1 -- 172.21.15.140:0/2811346082 learned_addr learned my addr 172.21.15.140:0/2811346082 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:58:59.710 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6004a55700 1 -- 172.21.15.140:0/2811346082 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847590383 0 0) 0x7f60000b0b80 con 0x7f6000143a10 2022-07-21T02:58:59.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6004a55700 1 -- 172.21.15.140:0/2811346082 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fec003260 con 0x7f6000143a10 2022-07-21T02:58:59.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6005a57700 1 -- 172.21.15.140:0/2811346082 >> v1:172.21.15.152:6789/0 conn(0x7f60000af8c0 legacy=0x7f60000ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.711 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6005a57700 1 -- 172.21.15.140:0/2811346082 >> v1:172.21.15.140:6789/0 conn(0x7f6000143a10 legacy=0x7f6000143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6005a57700 1 -- 172.21.15.140:0/2811346082 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60000b0db0 con 0x7f60000b5970 2022-07-21T02:58:59.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6005a57700 1 --2- 172.21.15.140:0/2811346082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60000b5970 0x7f60000ae100 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f5ff4002f70 tx=0x7f5ff4006df0).ready entity=mon.1 client_cookie=8019e902c9bbf12a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.712 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6004a55700 1 -- 172.21.15.140:0/2811346082 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ff400d610 con 0x7f60000b5970 2022-07-21T02:58:59.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6004a55700 1 -- 172.21.15.140:0/2811346082 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff4000e90 con 0x7f60000b5970 2022-07-21T02:58:59.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6004a55700 1 -- 172.21.15.140:0/2811346082 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ff400d960 con 0x7f60000b5970 2022-07-21T02:58:59.713 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006a59700 1 -- 172.21.15.140:0/2811346082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60000b5970 msgr2=0x7f60000ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006a59700 1 --2- 172.21.15.140:0/2811346082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60000b5970 0x7f60000ae100 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f5ff4002f70 tx=0x7f5ff4006df0).stop 2022-07-21T02:58:59.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006a59700 1 -- 172.21.15.140:0/2811346082 shutdown_connections 2022-07-21T02:58:59.714 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006a59700 1 --2- 172.21.15.140:0/2811346082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60000b5970 0x7f60000ae100 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006a59700 1 -- 172.21.15.140:0/2811346082 >> 172.21.15.140:0/2811346082 conn(0x7f6000063490 msgr2=0x7f60000b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:59.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006a59700 1 -- 172.21.15.140:0/2811346082 shutdown_connections 2022-07-21T02:58:59.715 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.702+0000 7f6006a59700 1 -- 172.21.15.140:0/2811346082 wait complete. 2022-07-21T02:58:59.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006a59700 1 Processor -- start 2022-07-21T02:58:59.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006a59700 1 -- start start 2022-07-21T02:58:59.716 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006a59700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60000af8c0 0x7f6000141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006a59700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60000b5970 0x7f60001598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006a59700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6000143a10 0x7f6000159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006a59700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6000065010 con 0x7f6000143a10 2022-07-21T02:58:59.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006a59700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60000652b0 con 0x7f60000b5970 2022-07-21T02:58:59.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006a59700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6000065550 con 0x7f60000af8c0 2022-07-21T02:58:59.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006258700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6000143a10 0x7f6000159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006258700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6000143a10 0x7f6000159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42402/0 (socket says 172.21.15.140:42402) 2022-07-21T02:58:59.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006258700 1 -- 172.21.15.140:0/3773718195 learned_addr learned my addr 172.21.15.140:0/3773718195 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:58:59.719 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6005a57700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60000af8c0 0x7f6000141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6005256700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60000b5970 0x7f60001598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006258700 1 -- 172.21.15.140:0/3773718195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60000af8c0 msgr2=0x7f6000141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.720 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006258700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60000af8c0 0x7f6000141470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006258700 1 -- 172.21.15.140:0/3773718195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60000b5970 msgr2=0x7f60001598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006258700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60000b5970 0x7f60001598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006258700 1 -- 172.21.15.140:0/3773718195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff400b180 con 0x7f6000143a10 2022-07-21T02:58:59.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6005256700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60000b5970 0x7f60001598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T02:58:59.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006258700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6000143a10 0x7f6000159db0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f5ff8001110 tx=0x7f5ff8007a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f5ffeffd700 1 -- 172.21.15.140:0/3773718195 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ff80070d0 con 0x7f6000143a10 2022-07-21T02:58:59.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f5ffeffd700 1 -- 172.21.15.140:0/3773718195 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff8007230 con 0x7f6000143a10 2022-07-21T02:58:59.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006a59700 1 -- 172.21.15.140:0/3773718195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6000063e50 con 0x7f6000143a10 2022-07-21T02:58:59.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f5ffeffd700 1 -- 172.21.15.140:0/3773718195 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ff8005610 con 0x7f6000143a10 2022-07-21T02:58:59.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.706+0000 7f6006a59700 1 -- 172.21.15.140:0/3773718195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6000156220 con 0x7f6000143a10 2022-07-21T02:58:59.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.710+0000 7f6006a59700 1 -- 172.21.15.140:0/3773718195 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60000acae0 con 0x7f6000143a10 2022-07-21T02:58:59.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.710+0000 7f5ffeffd700 1 -- 172.21.15.140:0/3773718195 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f5ff8010070 con 0x7f6000143a10 2022-07-21T02:58:59.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.710+0000 7f5ffeffd700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fe00539e0 0x7f5fe0055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.710+0000 7f5ffeffd700 1 -- 172.21.15.140:0/3773718195 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f5ff806b120 con 0x7f6000143a10 2022-07-21T02:58:59.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.710+0000 7f6005a57700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fe00539e0 0x7f5fe0055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.710+0000 7f6005a57700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fe00539e0 0x7f5fe0055ea0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f5ff4018000 tx=0x7f5ff4014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.727 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.714+0000 7f5ffeffd700 1 -- 172.21.15.140:0/3773718195 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f5ff80422a0 con 0x7f6000143a10 2022-07-21T02:58:59.840 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f6006a59700 1 -- 172.21.15.140:0/3773718195 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6000159620 con 0x7f5fe00539e0 2022-07-21T02:58:59.844 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f5ffeffd700 1 -- 172.21.15.140:0/3773718195 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5ff400d7f0 con 0x7f5fe00539e0 2022-07-21T02:58:59.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f6006a59700 1 -- 172.21.15.140:0/3773718195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fe00539e0 msgr2=0x7f5fe0055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.846 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f6006a59700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fe00539e0 0x7f5fe0055ea0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f5ff4018000 tx=0x7f5ff4014000).stop 2022-07-21T02:58:59.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f6006a59700 1 -- 172.21.15.140:0/3773718195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6000143a10 msgr2=0x7f6000159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f6006a59700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6000143a10 0x7f6000159db0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7f5ff8001110 tx=0x7f5ff8007a20).stop 2022-07-21T02:58:59.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f6006a59700 1 -- 172.21.15.140:0/3773718195 shutdown_connections 2022-07-21T02:58:59.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f6006a59700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f5fe00539e0 0x7f5fe0055ea0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f6006a59700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f6000143a10 0x7f6000159db0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f6006a59700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60000b5970 0x7f60001598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f6006a59700 1 --2- 172.21.15.140:0/3773718195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f60000af8c0 0x7f6000141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.838+0000 7f6006a59700 1 -- 172.21.15.140:0/3773718195 >> 172.21.15.140:0/3773718195 conn(0x7f6000063490 msgr2=0x7f60000b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:59.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.842+0000 7f6006a59700 1 -- 172.21.15.140:0/3773718195 shutdown_connections 2022-07-21T02:58:59.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.842+0000 7f6006a59700 1 -- 172.21.15.140:0/3773718195 wait complete. 2022-07-21T02:58:59.853 INFO:teuthology.orchestra.run.smithi140.stderr:instructed osd(s) 3 to deep-scrub 2022-07-21T02:58:59.857 INFO:tasks.ceph:Scrubbing osd.4 2022-07-21T02:58:59.859 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2022-07-21T02:58:59.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.970+0000 7f09c17f3700 1 Processor -- start 2022-07-21T02:58:59.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.970+0000 7f09c17f3700 1 -- start start 2022-07-21T02:58:59.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.970+0000 7f09c17f3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09bc0b6840 0x7f09bc0b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.970+0000 7f09c17f3700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f09bc0b3cf0 0x7f09bc146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:58:59.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.970+0000 7f09c17f3700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09bc066b10 con 0x7f09bc146530 2022-07-21T02:58:59.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.970+0000 7f09c17f3700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09bc0b6c20 con 0x7f09bc0b3cf0 2022-07-21T02:58:59.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.970+0000 7f09c17f3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc0aead0 con 0x7f09bc0b6840 2022-07-21T02:58:59.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09bbfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09bc0b6840 0x7f09bc0b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09b37fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f09bc0b3cf0 0x7f09bc146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:58:59.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09c0ff2700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f09bc146530 0x7f09bc05bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.140:57672/0 (socket says 172.21.15.140:57672) 2022-07-21T02:58:59.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09c0ff2700 1 -- 172.21.15.140:0/1897516875 learned_addr learned my addr 172.21.15.140:0/1897516875 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:58:59.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09bb7fe700 1 -- 172.21.15.140:0/1897516875 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595780063 0 0) 0x7f09bc066b10 con 0x7f09bc146530 2022-07-21T02:58:59.987 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09bb7fe700 1 -- 172.21.15.140:0/1897516875 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f099c003260 con 0x7f09bc146530 2022-07-21T02:58:59.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09b37fe700 1 -- 172.21.15.140:0/1897516875 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09bc0b6840 msgr2=0x7f09bc0b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:58:59.988 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09b37fe700 1 --2- 172.21.15.140:0/1897516875 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09bc0b6840 0x7f09bc0b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.989 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09b37fe700 1 -- 172.21.15.140:0/1897516875 >> v1:172.21.15.152:6789/0 conn(0x7f09bc146530 legacy=0x7f09bc05bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09b37fe700 1 -- 172.21.15.140:0/1897516875 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc0aed70 con 0x7f09bc0b3cf0 2022-07-21T02:58:59.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09bbfff700 1 --2- 172.21.15.140:0/1897516875 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09bc0b6840 0x7f09bc0b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T02:58:59.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09b37fe700 1 --2- 172.21.15.140:0/1897516875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09bc0b3cf0 0x7f09bc146040 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f09a4002f70 tx=0x7f09a4006df0).ready entity=mon.1 client_cookie=2508a1b51f3f069b server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:58:59.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09bb7fe700 1 -- 172.21.15.140:0/1897516875 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a400d4a0 con 0x7f09bc0b3cf0 2022-07-21T02:58:59.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09bb7fe700 1 -- 172.21.15.140:0/1897516875 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a4000e90 con 0x7f09bc0b3cf0 2022-07-21T02:58:59.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09bb7fe700 1 -- 172.21.15.140:0/1897516875 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a400d7f0 con 0x7f09bc0b3cf0 2022-07-21T02:58:59.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09c17f3700 1 -- 172.21.15.140:0/1897516875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09bc0b3cf0 msgr2=0x7f09bc146040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:58:59.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09c17f3700 1 --2- 172.21.15.140:0/1897516875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09bc0b3cf0 0x7f09bc146040 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f09a4002f70 tx=0x7f09a4006df0).stop 2022-07-21T02:58:59.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09c17f3700 1 -- 172.21.15.140:0/1897516875 shutdown_connections 2022-07-21T02:58:59.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09c17f3700 1 --2- 172.21.15.140:0/1897516875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09bc0b3cf0 0x7f09bc146040 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09c17f3700 1 --2- 172.21.15.140:0/1897516875 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09bc0b6840 0x7f09bc0b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:58:59.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09c17f3700 1 -- 172.21.15.140:0/1897516875 >> 172.21.15.140:0/1897516875 conn(0x7f09bc0b9e10 msgr2=0x7f09bc0b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:58:59.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09c17f3700 1 -- 172.21.15.140:0/1897516875 shutdown_connections 2022-07-21T02:58:59.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.974+0000 7f09c17f3700 1 -- 172.21.15.140:0/1897516875 wait complete. 2022-07-21T02:59:00.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c17f3700 1 Processor -- start 2022-07-21T02:59:00.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c17f3700 1 -- start start 2022-07-21T02:59:00.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c17f3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09bc0b3cf0 0x7f09bc155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c17f3700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f09bc0b6840 0x7f09bc0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c17f3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09bc146530 0x7f09bc114e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c17f3700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc113d80 con 0x7f09bc0b6840 2022-07-21T02:59:00.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c17f3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc113730 con 0x7f09bc0b3cf0 2022-07-21T02:59:00.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c17f3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc1126c0 con 0x7f09bc146530 2022-07-21T02:59:00.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09b37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f09bc0b6840 0x7f09bc0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09b37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f09bc0b6840 0x7f09bc0af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42416/0 (socket says 172.21.15.140:42416) 2022-07-21T02:59:00.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09b37fe700 1 -- 172.21.15.140:0/956089497 learned_addr learned my addr 172.21.15.140:0/956089497 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:00.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c0ff2700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09bc146530 0x7f09bc114e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09bbfff700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09bc0b3cf0 0x7f09bc155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c0ff2700 1 -- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09bc0b3cf0 msgr2=0x7f09bc155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c0ff2700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09bc0b3cf0 0x7f09bc155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c0ff2700 1 -- 172.21.15.140:0/956089497 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f09bc0b6840 msgr2=0x7f09bc0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c0ff2700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f09bc0b6840 0x7f09bc0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c0ff2700 1 -- 172.21.15.140:0/956089497 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac00b180 con 0x7f09bc146530 2022-07-21T02:59:00.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09b37fe700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f09bc0b6840 0x7f09bc0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T02:59:00.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09c0ff2700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09bc146530 0x7f09bc114e90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f09b4001490 tx=0x7f09b4007d20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.978+0000 7f09b9ffb700 1 -- 172.21.15.140:0/956089497 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09b400ce20 con 0x7f09bc146530 2022-07-21T02:59:00.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09b9ffb700 1 -- 172.21.15.140:0/956089497 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b4005ba0 con 0x7f09bc146530 2022-07-21T02:59:00.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09c17f3700 1 -- 172.21.15.140:0/956089497 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a400b180 con 0x7f09bc146530 2022-07-21T02:59:00.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09b9ffb700 1 -- 172.21.15.140:0/956089497 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09b40070d0 con 0x7f09bc146530 2022-07-21T02:59:00.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09c17f3700 1 -- 172.21.15.140:0/956089497 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc0b1c60 con 0x7f09bc146530 2022-07-21T02:59:00.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09c17f3700 1 -- 172.21.15.140:0/956089497 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09bc112540 con 0x7f09bc146530 2022-07-21T02:59:00.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09b9ffb700 1 -- 172.21.15.140:0/956089497 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f09b4010070 con 0x7f09bc146530 2022-07-21T02:59:00.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09b9ffb700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f09a0053a70 0x7f09a0055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09b9ffb700 1 -- 172.21.15.140:0/956089497 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f09b406bc60 con 0x7f09bc146530 2022-07-21T02:59:00.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09b9ffb700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] conn(0x7f09a005a160 0x7f09a005c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09b9ffb700 1 -- 172.21.15.140:0/956089497 --> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09a005cba0 con 0x7f09a005a160 2022-07-21T02:59:00.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09b9ffb700 1 -- 172.21.15.140:0/956089497 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09b406c040 con 0x7f09bc146530 2022-07-21T02:59:00.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09bbfff700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f09a0053a70 0x7f09a0055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09b37fe700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] conn(0x7f09a005a160 0x7f09a005c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09bbfff700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f09a0053a70 0x7f09a0055f30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f09ac00b000 tx=0x7f09ac006df0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.982+0000 7f09b37fe700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] conn(0x7f09a005a160 0x7f09a005c5c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:58:59.986+0000 7f09b9ffb700 1 -- 172.21.15.140:0/956089497 <== osd.4 v2:172.21.15.152:6800/23141 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f09a005cba0 con 0x7f09a005a160 2022-07-21T02:59:00.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.002+0000 7f09c17f3700 1 -- 172.21.15.140:0/956089497 --> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f09bc0b1eb0 con 0x7f09a005a160 2022-07-21T02:59:00.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09b9ffb700 1 -- 172.21.15.140:0/956089497 <== osd.4 v2:172.21.15.152:6800/23141 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f09a4017080 con 0x7f09a005a160 2022-07-21T02:59:00.020 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T02:59:00.021 INFO:teuthology.orchestra.run.smithi140.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-07-21T02:59:00.021 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T02:59:00.024 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09c17f3700 1 -- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] conn(0x7f09a005a160 msgr2=0x7f09a005c5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09c17f3700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:6800/23141,v1:172.21.15.152:6801/23141] conn(0x7f09a005a160 0x7f09a005c5c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09c17f3700 1 -- 172.21.15.140:0/956089497 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f09a0053a70 msgr2=0x7f09a0055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.025 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09c17f3700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f09a0053a70 0x7f09a0055f30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f09ac00b000 tx=0x7f09ac006df0).stop 2022-07-21T02:59:00.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09c17f3700 1 -- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09bc146530 msgr2=0x7f09bc114e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.026 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09c17f3700 1 --2- 172.21.15.140:0/956089497 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09bc146530 0x7f09bc114e90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f09b4001490 tx=0x7f09b4007d20).stop 2022-07-21T02:59:00.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09c0ff2700 1 -- 172.21.15.140:0/956089497 reap_dead start 2022-07-21T02:59:00.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09c17f3700 1 -- 172.21.15.140:0/956089497 shutdown_connections 2022-07-21T02:59:00.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09c17f3700 1 -- 172.21.15.140:0/956089497 >> 172.21.15.140:0/956089497 conn(0x7f09bc0b9e10 msgr2=0x7f09bc0b7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:00.027 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09c17f3700 1 -- 172.21.15.140:0/956089497 shutdown_connections 2022-07-21T02:59:00.028 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.006+0000 7f09c17f3700 1 -- 172.21.15.140:0/956089497 wait complete. 2022-07-21T02:59:00.028 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2022-07-21T02:59:00.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.126+0000 7fc2425c8700 1 Processor -- start 2022-07-21T02:59:00.130 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.126+0000 7fc2425c8700 1 -- start start 2022-07-21T02:59:00.132 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc2425c8700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc23c060ca0 0x7fc23c061080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc2425c8700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc23c14a750 con 0x7fc23c097a80 2022-07-21T02:59:00.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc2425c8700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc23c0b2840 con 0x7fc23c0afb80 2022-07-21T02:59:00.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc2425c8700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23c148430 con 0x7fc23c060ca0 2022-07-21T02:59:00.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc23bfff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fc23c060ca0 0x7fc23c061080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc23b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fc23c097a80 0x7fc23c14ab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56570/0 (socket says 172.21.15.140:56570) 2022-07-21T02:59:00.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc23b7fe700 1 -- 172.21.15.140:0/3771322155 learned_addr learned my addr 172.21.15.140:0/3771322155 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:59:00.135 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc23bfff700 1 -- 172.21.15.140:0/3771322155 >> v1:172.21.15.152:6789/0 conn(0x7fc23c0afb80 legacy=0x7fc23c146850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc23bfff700 1 -- 172.21.15.140:0/3771322155 >> v1:172.21.15.140:6789/0 conn(0x7fc23c097a80 legacy=0x7fc23c14ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.136 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc23bfff700 1 -- 172.21.15.140:0/3771322155 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c0ac110 con 0x7fc23c060ca0 2022-07-21T02:59:00.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc23bfff700 1 --2- 172.21.15.140:0/3771322155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc23c060ca0 0x7fc23c061080 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7fc224002f70 tx=0x7fc224006df0).ready entity=mon.0 client_cookie=69106cbec6669e94 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc23affd700 1 -- 172.21.15.140:0/3771322155 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc22400d610 con 0x7fc23c060ca0 2022-07-21T02:59:00.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc23affd700 1 -- 172.21.15.140:0/3771322155 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc224000e90 con 0x7fc23c060ca0 2022-07-21T02:59:00.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc23affd700 1 -- 172.21.15.140:0/3771322155 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc22400d960 con 0x7fc23c060ca0 2022-07-21T02:59:00.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc2425c8700 1 -- 172.21.15.140:0/3771322155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc23c060ca0 msgr2=0x7fc23c061080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc2425c8700 1 --2- 172.21.15.140:0/3771322155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc23c060ca0 0x7fc23c061080 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7fc224002f70 tx=0x7fc224006df0).stop 2022-07-21T02:59:00.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc2425c8700 1 -- 172.21.15.140:0/3771322155 shutdown_connections 2022-07-21T02:59:00.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc2425c8700 1 --2- 172.21.15.140:0/3771322155 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc23c060ca0 0x7fc23c061080 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc2425c8700 1 -- 172.21.15.140:0/3771322155 >> 172.21.15.140:0/3771322155 conn(0x7fc23c0b5cb0 msgr2=0x7fc23c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:00.144 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc2425c8700 1 -- 172.21.15.140:0/3771322155 shutdown_connections 2022-07-21T02:59:00.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.130+0000 7fc2425c8700 1 -- 172.21.15.140:0/3771322155 wait complete. 2022-07-21T02:59:00.145 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.134+0000 7fc2425c8700 1 Processor -- start 2022-07-21T02:59:00.146 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.134+0000 7fc2425c8700 1 -- start start 2022-07-21T02:59:00.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.134+0000 7fc2425c8700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc23c060ca0 0x7fc23c0ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.134+0000 7fc2425c8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc23c097a80 0x7fc23c067690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.147 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.134+0000 7fc2425c8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc23c0afb80 0x7fc23c066000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.134+0000 7fc2425c8700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c09a180 con 0x7fc23c060ca0 2022-07-21T02:59:00.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.134+0000 7fc2425c8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c067b80 con 0x7fc23c0afb80 2022-07-21T02:59:00.148 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.134+0000 7fc2425c8700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c067190 con 0x7fc23c097a80 2022-07-21T02:59:00.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.134+0000 7fc23bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc23c060ca0 0x7fc23c0ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc23bfff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc23c060ca0 0x7fc23c0ae960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42428/0 (socket says 172.21.15.140:42428) 2022-07-21T02:59:00.149 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc23bfff700 1 -- 172.21.15.140:0/1639015515 learned_addr learned my addr 172.21.15.140:0/1639015515 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:00.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc23b7fe700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc23c097a80 0x7fc23c067690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc240b65700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc23c0afb80 0x7fc23c066000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.150 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc23bfff700 1 -- 172.21.15.140:0/1639015515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc23c097a80 msgr2=0x7fc23c067690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc23bfff700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc23c097a80 0x7fc23c067690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc23bfff700 1 -- 172.21.15.140:0/1639015515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc23c0afb80 msgr2=0x7fc23c066000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.151 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc23bfff700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc23c0afb80 0x7fc23c066000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc23bfff700 1 -- 172.21.15.140:0/1639015515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22400b180 con 0x7fc23c060ca0 2022-07-21T02:59:00.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc23bfff700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc23c060ca0 0x7fc23c0ae960 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7fc23c099b40 tx=0x7fc224015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc2397fa700 1 -- 172.21.15.140:0/1639015515 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc22400d990 con 0x7fc23c060ca0 2022-07-21T02:59:00.152 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc2397fa700 1 -- 172.21.15.140:0/1639015515 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc224011b70 con 0x7fc23c060ca0 2022-07-21T02:59:00.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc2425c8700 1 -- 172.21.15.140:0/1639015515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23c099d80 con 0x7fc23c060ca0 2022-07-21T02:59:00.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc2397fa700 1 -- 172.21.15.140:0/1639015515 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc22401e460 con 0x7fc23c060ca0 2022-07-21T02:59:00.153 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc2425c8700 1 -- 172.21.15.140:0/1639015515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23c09aa40 con 0x7fc23c060ca0 2022-07-21T02:59:00.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc2425c8700 1 -- 172.21.15.140:0/1639015515 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23c09a450 con 0x7fc23c060ca0 2022-07-21T02:59:00.154 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc2397fa700 1 -- 172.21.15.140:0/1639015515 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fc224025030 con 0x7fc23c060ca0 2022-07-21T02:59:00.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc2397fa700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc220053950 0x7fc220055e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.155 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc2397fa700 1 -- 172.21.15.140:0/1639015515 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fc22406cf90 con 0x7fc23c060ca0 2022-07-21T02:59:00.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.138+0000 7fc23b7fe700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc220053950 0x7fc220055e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.156 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.142+0000 7fc23b7fe700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc220053950 0x7fc220055e10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fc22c00d000 tx=0x7fc22c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.157 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.142+0000 7fc2397fa700 1 -- 172.21.15.140:0/1639015515 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fc2240441d0 con 0x7fc23c060ca0 2022-07-21T02:59:00.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 -- 172.21.15.140:0/1639015515 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc23c067010 con 0x7fc220053950 2022-07-21T02:59:00.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2397fa700 1 -- 172.21.15.140:0/1639015515 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc22c00f680 con 0x7fc220053950 2022-07-21T02:59:00.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 -- 172.21.15.140:0/1639015515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc220053950 msgr2=0x7fc220055e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc220053950 0x7fc220055e10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fc22c00d000 tx=0x7fc22c009040).stop 2022-07-21T02:59:00.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 -- 172.21.15.140:0/1639015515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc23c060ca0 msgr2=0x7fc23c0ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc23c060ca0 0x7fc23c0ae960 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7fc23c099b40 tx=0x7fc224015000).stop 2022-07-21T02:59:00.277 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 -- 172.21.15.140:0/1639015515 shutdown_connections 2022-07-21T02:59:00.278 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fc220053950 0x7fc220055e10 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc23c0afb80 0x7fc23c066000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.279 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc23c097a80 0x7fc23c067690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.280 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 --2- 172.21.15.140:0/1639015515 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fc23c060ca0 0x7fc23c0ae960 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 -- 172.21.15.140:0/1639015515 >> 172.21.15.140:0/1639015515 conn(0x7fc23c0b5cb0 msgr2=0x7fc23c147350 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:00.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.270+0000 7fc2425c8700 1 -- 172.21.15.140:0/1639015515 shutdown_connections 2022-07-21T02:59:00.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.274+0000 7fc2425c8700 1 -- 172.21.15.140:0/1639015515 wait complete. 2022-07-21T02:59:00.287 INFO:teuthology.orchestra.run.smithi140.stderr:instructed osd(s) 4 to deep-scrub 2022-07-21T02:59:00.288 INFO:tasks.ceph:Scrubbing osd.5 2022-07-21T02:59:00.289 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2022-07-21T02:59:00.416 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe66318c700 1 Processor -- start 2022-07-21T02:59:00.417 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe66318c700 1 -- start start 2022-07-21T02:59:00.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe66318c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe65c0b5990 0x7fe65c0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.418 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe66318c700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe65c0af8e0 0x7fe65c146960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe66318c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe65c142ef0 0x7fe65c1433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe66318c700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65c146e50 con 0x7fe65c0af8e0 2022-07-21T02:59:00.419 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe66318c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65c142a90 con 0x7fe65c142ef0 2022-07-21T02:59:00.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe66318c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0b5d70 con 0x7fe65c0b5990 2022-07-21T02:59:00.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe661989700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe65c0af8e0 0x7fe65c146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.420 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe661989700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fe65c0af8e0 0x7fe65c146960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42438/0 (socket says 172.21.15.140:42438) 2022-07-21T02:59:00.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe661989700 1 -- 172.21.15.140:0/2012420123 learned_addr learned my addr 172.21.15.140:0/2012420123 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:00.421 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe66218a700 1 --2- 172.21.15.140:0/2012420123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe65c0b5990 0x7fe65c0ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe66298b700 1 --2- 172.21.15.140:0/2012420123 >> v2:172.21.15.152:3300/0 conn(0x7fe65c142ef0 0x7fe65c1433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe661989700 1 -- 172.21.15.140:0/2012420123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe65c0b5990 msgr2=0x7fe65c0ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:59:00.422 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe661989700 1 --2- 172.21.15.140:0/2012420123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe65c0b5990 0x7fe65c0ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe661989700 1 -- 172.21.15.140:0/2012420123 >> v2:172.21.15.152:3300/0 conn(0x7fe65c142ef0 msgr2=0x7fe65c1433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:59:00.423 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe661989700 1 --2- 172.21.15.140:0/2012420123 >> v2:172.21.15.152:3300/0 conn(0x7fe65c142ef0 0x7fe65c1433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe661989700 1 -- 172.21.15.140:0/2012420123 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0afdf0 con 0x7fe65c0af8e0 2022-07-21T02:59:00.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe66218a700 1 --2- 172.21.15.140:0/2012420123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe65c0b5990 0x7fe65c0ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T02:59:00.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.402+0000 7fe661989700 1 --2- 172.21.15.140:0/2012420123 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe65c0af8e0 0x7fe65c146960 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7fe644002f70 tx=0x7fe644006eb0).ready entity=mon.0 client_cookie=5dabe61f8a744bef server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe661188700 1 -- 172.21.15.140:0/2012420123 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe64400d570 con 0x7fe65c0af8e0 2022-07-21T02:59:00.425 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe661188700 1 -- 172.21.15.140:0/2012420123 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe644000e90 con 0x7fe65c0af8e0 2022-07-21T02:59:00.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe661188700 1 -- 172.21.15.140:0/2012420123 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe64400d900 con 0x7fe65c0af8e0 2022-07-21T02:59:00.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 -- 172.21.15.140:0/2012420123 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe65c0af8e0 msgr2=0x7fe65c146960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.426 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 --2- 172.21.15.140:0/2012420123 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe65c0af8e0 0x7fe65c146960 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7fe644002f70 tx=0x7fe644006eb0).stop 2022-07-21T02:59:00.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 -- 172.21.15.140:0/2012420123 shutdown_connections 2022-07-21T02:59:00.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 --2- 172.21.15.140:0/2012420123 >> v2:172.21.15.152:3300/0 conn(0x7fe65c142ef0 0x7fe65c1433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 --2- 172.21.15.140:0/2012420123 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe65c0af8e0 0x7fe65c146960 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 --2- 172.21.15.140:0/2012420123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe65c0b5990 0x7fe65c0ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 -- 172.21.15.140:0/2012420123 >> 172.21.15.140:0/2012420123 conn(0x7fe65c063480 msgr2=0x7fe65c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:00.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 -- 172.21.15.140:0/2012420123 shutdown_connections 2022-07-21T02:59:00.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 -- 172.21.15.140:0/2012420123 wait complete. 2022-07-21T02:59:00.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 Processor -- start 2022-07-21T02:59:00.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 -- start start 2022-07-21T02:59:00.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe65c0af8e0 0x7fe65c0664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe65c0b5990 0x7fe65c115250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe65c142ef0 0x7fe65c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c117970 con 0x7fe65c0af8e0 2022-07-21T02:59:00.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c1176d0 con 0x7fe65c0b5990 2022-07-21T02:59:00.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c114db0 con 0x7fe65c142ef0 2022-07-21T02:59:00.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66218a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe65c0af8e0 0x7fe65c0664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66218a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe65c0af8e0 0x7fe65c0664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42444/0 (socket says 172.21.15.140:42444) 2022-07-21T02:59:00.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66218a700 1 -- 172.21.15.140:0/2500460019 learned_addr learned my addr 172.21.15.140:0/2500460019 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:00.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe661989700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe65c0b5990 0x7fe65c115250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66298b700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe65c142ef0 0x7fe65c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66218a700 1 -- 172.21.15.140:0/2500460019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe65c142ef0 msgr2=0x7fe65c1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66218a700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe65c142ef0 0x7fe65c1524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66218a700 1 -- 172.21.15.140:0/2500460019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe65c0b5990 msgr2=0x7fe65c115250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66218a700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe65c0b5990 0x7fe65c115250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66218a700 1 -- 172.21.15.140:0/2500460019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64400b1e0 con 0x7fe65c0af8e0 2022-07-21T02:59:00.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66218a700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe65c0af8e0 0x7fe65c0664c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7fe654006c60 tx=0x7fe654008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe65b7fe700 1 -- 172.21.15.140:0/2500460019 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe654013620 con 0x7fe65c0af8e0 2022-07-21T02:59:00.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 -- 172.21.15.140:0/2500460019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c114a50 con 0x7fe65c0af8e0 2022-07-21T02:59:00.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.406+0000 7fe66318c700 1 -- 172.21.15.140:0/2500460019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c114500 con 0x7fe65c0af8e0 2022-07-21T02:59:00.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe65b7fe700 1 -- 172.21.15.140:0/2500460019 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe654005a70 con 0x7fe65c0af8e0 2022-07-21T02:59:00.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe65b7fe700 1 -- 172.21.15.140:0/2500460019 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe654005e10 con 0x7fe65c0af8e0 2022-07-21T02:59:00.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe66318c700 1 -- 172.21.15.140:0/2500460019 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe634003660 con 0x7fe65c0af8e0 2022-07-21T02:59:00.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe65b7fe700 1 -- 172.21.15.140:0/2500460019 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe654024070 con 0x7fe65c0af8e0 2022-07-21T02:59:00.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe65b7fe700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe640053aa0 0x7fe640055f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.441 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe65b7fe700 1 -- 172.21.15.140:0/2500460019 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fe65406c2d0 con 0x7fe65c0af8e0 2022-07-21T02:59:00.442 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe65b7fe700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] conn(0x7fe64005a190 0x7fe64005c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe65b7fe700 1 -- 172.21.15.140:0/2500460019 --> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe64005cbd0 con 0x7fe64005a190 2022-07-21T02:59:00.443 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe65b7fe700 1 -- 172.21.15.140:0/2500460019 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe65406c5a0 con 0x7fe65c0af8e0 2022-07-21T02:59:00.444 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe661989700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe640053aa0 0x7fe640055f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe66298b700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] conn(0x7fe64005a190 0x7fe64005c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe661989700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe640053aa0 0x7fe640055f60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fe64401c000 tx=0x7fe644015000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.410+0000 7fe66298b700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] conn(0x7fe64005a190 0x7fe64005c5f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.414+0000 7fe65b7fe700 1 -- 172.21.15.140:0/2500460019 <== osd.5 v2:172.21.15.152:6810/23143 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7fe64005cbd0 con 0x7fe64005a190 2022-07-21T02:59:00.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66318c700 1 -- 172.21.15.140:0/2500460019 --> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe634002870 con 0x7fe64005a190 2022-07-21T02:59:00.449 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe65b7fe700 1 -- 172.21.15.140:0/2500460019 <== osd.5 v2:172.21.15.152:6810/23143 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe634002870 con 0x7fe64005a190 2022-07-21T02:59:00.450 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T02:59:00.451 INFO:teuthology.orchestra.run.smithi140.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-07-21T02:59:00.452 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T02:59:00.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66318c700 1 -- 172.21.15.140:0/2500460019 >> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] conn(0x7fe64005a190 msgr2=0x7fe64005c5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66318c700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.152:6810/23143,v1:172.21.15.152:6812/23143] conn(0x7fe64005a190 0x7fe64005c5f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66318c700 1 -- 172.21.15.140:0/2500460019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe640053aa0 msgr2=0x7fe640055f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66318c700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe640053aa0 0x7fe640055f60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fe64401c000 tx=0x7fe644015000).stop 2022-07-21T02:59:00.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66318c700 1 -- 172.21.15.140:0/2500460019 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe65c0af8e0 msgr2=0x7fe65c0664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66318c700 1 --2- 172.21.15.140:0/2500460019 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe65c0af8e0 0x7fe65c0664c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7fe654006c60 tx=0x7fe654008040).stop 2022-07-21T02:59:00.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66298b700 1 -- 172.21.15.140:0/2500460019 reap_dead start 2022-07-21T02:59:00.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66318c700 1 -- 172.21.15.140:0/2500460019 shutdown_connections 2022-07-21T02:59:00.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66318c700 1 -- 172.21.15.140:0/2500460019 >> 172.21.15.140:0/2500460019 conn(0x7fe65c063480 msgr2=0x7fe65c0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:00.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66318c700 1 -- 172.21.15.140:0/2500460019 shutdown_connections 2022-07-21T02:59:00.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.426+0000 7fe66318c700 1 -- 172.21.15.140:0/2500460019 wait complete. 2022-07-21T02:59:00.462 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2022-07-21T02:59:00.573 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 Processor -- start 2022-07-21T02:59:00.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 -- start start 2022-07-21T02:59:00.574 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c400af8c0 0x7f1c400ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1c40143a10 0x7f1c40143e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.575 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c400b0b80 con 0x7f1c400b5970 2022-07-21T02:59:00.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4005d4d0 con 0x7f1c40143a10 2022-07-21T02:59:00.576 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400b5d50 con 0x7f1c400af8c0 2022-07-21T02:59:00.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c44c2d700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f1c400b5970 0x7f1c400ae100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56598/0 (socket says 172.21.15.140:56598) 2022-07-21T02:59:00.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c44c2d700 1 -- 172.21.15.140:0/1485222159 learned_addr learned my addr 172.21.15.140:0/1485222159 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:59:00.577 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c3ffff700 1 --2- 172.21.15.140:0/1485222159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c400af8c0 0x7f1c400ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c4542e700 1 --2- 172.21.15.140:0/1485222159 >> v2:172.21.15.152:3300/0 conn(0x7f1c40143a10 0x7f1c40143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.578 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c3f7fe700 1 -- 172.21.15.140:0/1485222159 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541226033 0 0) 0x7f1c400b0b80 con 0x7f1c400b5970 2022-07-21T02:59:00.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c3f7fe700 1 -- 172.21.15.140:0/1485222159 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c2c003260 con 0x7f1c400b5970 2022-07-21T02:59:00.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c3ffff700 1 -- 172.21.15.140:0/1485222159 >> v2:172.21.15.152:3300/0 conn(0x7f1c40143a10 msgr2=0x7f1c40143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:59:00.579 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c3ffff700 1 --2- 172.21.15.140:0/1485222159 >> v2:172.21.15.152:3300/0 conn(0x7f1c40143a10 0x7f1c40143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c3ffff700 1 -- 172.21.15.140:0/1485222159 >> v1:172.21.15.140:6789/0 conn(0x7f1c400b5970 legacy=0x7f1c400ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c3ffff700 1 -- 172.21.15.140:0/1485222159 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400b0db0 con 0x7f1c400af8c0 2022-07-21T02:59:00.580 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c3ffff700 1 --2- 172.21.15.140:0/1485222159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c400af8c0 0x7f1c400ae810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f1c380032f0 tx=0x7f1c380091d0).ready entity=mon.2 client_cookie=1ac3eca97ebd829a server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c3f7fe700 1 -- 172.21.15.140:0/1485222159 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3800b600 con 0x7f1c400af8c0 2022-07-21T02:59:00.581 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c3f7fe700 1 -- 172.21.15.140:0/1485222159 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3800b760 con 0x7f1c400af8c0 2022-07-21T02:59:00.582 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c3f7fe700 1 -- 172.21.15.140:0/1485222159 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c3800b8d0 con 0x7f1c400af8c0 2022-07-21T02:59:00.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1485222159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c400af8c0 msgr2=0x7f1c400ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.587 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 --2- 172.21.15.140:0/1485222159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c400af8c0 0x7f1c400ae810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f1c380032f0 tx=0x7f1c380091d0).stop 2022-07-21T02:59:00.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1485222159 shutdown_connections 2022-07-21T02:59:00.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 --2- 172.21.15.140:0/1485222159 >> v2:172.21.15.152:3300/0 conn(0x7f1c40143a10 0x7f1c40143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.588 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 --2- 172.21.15.140:0/1485222159 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c400af8c0 0x7f1c400ae810 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1485222159 >> 172.21.15.140:0/1485222159 conn(0x7f1c40063490 msgr2=0x7f1c400b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:00.589 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.570+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1485222159 shutdown_connections 2022-07-21T02:59:00.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1485222159 wait complete. 2022-07-21T02:59:00.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c45c2f700 1 Processor -- start 2022-07-21T02:59:00.590 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c45c2f700 1 -- start start 2022-07-21T02:59:00.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c45c2f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c400af8c0 0x7f1c4010e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c45c2f700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c400b5970 0x7f1c40156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.591 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c45c2f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c40143a10 0x7f1c4010f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c45c2f700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c401561a0 con 0x7f1c400b5970 2022-07-21T02:59:00.592 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c45c2f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c40156440 con 0x7f1c400af8c0 2022-07-21T02:59:00.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c45c2f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c401566e0 con 0x7f1c40143a10 2022-07-21T02:59:00.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c3ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c400b5970 0x7f1c40156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.593 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c3ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c400b5970 0x7f1c40156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42458/0 (socket says 172.21.15.140:42458) 2022-07-21T02:59:00.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c3ffff700 1 -- 172.21.15.140:0/1536700440 learned_addr learned my addr 172.21.15.140:0/1536700440 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:00.594 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c4542e700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c40143a10 0x7f1c4010f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c44c2d700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c400af8c0 0x7f1c4010e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c4542e700 1 -- 172.21.15.140:0/1536700440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c400af8c0 msgr2=0x7f1c4010e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.595 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c4542e700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c400af8c0 0x7f1c4010e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c4542e700 1 -- 172.21.15.140:0/1536700440 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c400b5970 msgr2=0x7f1c40156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.596 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c4542e700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c400b5970 0x7f1c40156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c4542e700 1 -- 172.21.15.140:0/1536700440 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c38009470 con 0x7f1c40143a10 2022-07-21T02:59:00.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c3ffff700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c400b5970 0x7f1c40156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T02:59:00.597 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c4542e700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c40143a10 0x7f1c4010f020 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f1c3000c000 tx=0x7f1c30006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.574+0000 7f1c3dffb700 1 -- 172.21.15.140:0/1536700440 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c30016070 con 0x7f1c40143a10 2022-07-21T02:59:00.598 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.578+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1536700440 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c40148db0 con 0x7f1c40143a10 2022-07-21T02:59:00.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.578+0000 7f1c3dffb700 1 -- 172.21.15.140:0/1536700440 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c30011750 con 0x7f1c40143a10 2022-07-21T02:59:00.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.578+0000 7f1c3dffb700 1 -- 172.21.15.140:0/1536700440 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1c30011ab0 con 0x7f1c40143a10 2022-07-21T02:59:00.599 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.578+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1536700440 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4010f510 con 0x7f1c40143a10 2022-07-21T02:59:00.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.578+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1536700440 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c401568a0 con 0x7f1c40143a10 2022-07-21T02:59:00.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.578+0000 7f1c3dffb700 1 -- 172.21.15.140:0/1536700440 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1c30011c10 con 0x7f1c40143a10 2022-07-21T02:59:00.600 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.578+0000 7f1c3dffb700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c20053a10 0x7f1c20055ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.578+0000 7f1c3dffb700 1 -- 172.21.15.140:0/1536700440 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f1c3006bbc0 con 0x7f1c40143a10 2022-07-21T02:59:00.601 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.578+0000 7f1c44c2d700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c20053a10 0x7f1c20055ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.602 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.578+0000 7f1c44c2d700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c20053a10 0x7f1c20055ed0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f1c3400d000 tx=0x7f1c34009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.603 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.586+0000 7f1c3dffb700 1 -- 172.21.15.140:0/1536700440 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1c30042d20 con 0x7f1c40143a10 2022-07-21T02:59:00.717 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.714+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1536700440 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c40157290 con 0x7f1c20053a10 2022-07-21T02:59:00.718 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.714+0000 7f1c3dffb700 1 -- 172.21.15.140:0/1536700440 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1c340133f0 con 0x7f1c20053a10 2022-07-21T02:59:00.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1536700440 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c20053a10 msgr2=0x7f1c20055ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.721 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c20053a10 0x7f1c20055ed0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f1c3400d000 tx=0x7f1c34009040).stop 2022-07-21T02:59:00.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1536700440 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c40143a10 msgr2=0x7f1c4010f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.722 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c40143a10 0x7f1c4010f020 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f1c3000c000 tx=0x7f1c30006df0).stop 2022-07-21T02:59:00.723 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1536700440 shutdown_connections 2022-07-21T02:59:00.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f1c20053a10 0x7f1c20055ed0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1c40143a10 0x7f1c4010f020 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.724 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f1c400b5970 0x7f1c40156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 --2- 172.21.15.140:0/1536700440 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c400af8c0 0x7f1c4010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.725 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1536700440 >> 172.21.15.140:0/1536700440 conn(0x7f1c40063490 msgr2=0x7f1c400b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:00.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1536700440 shutdown_connections 2022-07-21T02:59:00.726 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.718+0000 7f1c45c2f700 1 -- 172.21.15.140:0/1536700440 wait complete. 2022-07-21T02:59:00.726 INFO:teuthology.orchestra.run.smithi140.stderr:instructed osd(s) 5 to deep-scrub 2022-07-21T02:59:00.731 INFO:tasks.ceph:Scrubbing osd.6 2022-07-21T02:59:00.734 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2022-07-21T02:59:00.847 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f339396a700 1 Processor -- start 2022-07-21T02:59:00.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f339396a700 1 -- start start 2022-07-21T02:59:00.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f339396a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c062f30 0x7f338c0adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.848 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f339396a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f338c142b30 0x7f338c142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f339396a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338c1462c0 con 0x7f338c05cf40 2022-07-21T02:59:00.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f339396a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338c147110 con 0x7f338c142b30 2022-07-21T02:59:00.849 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f339396a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0ae040 con 0x7f338c062f30 2022-07-21T02:59:00.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f3390f05700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c062f30 0x7f338c0adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f3391f07700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f338c142b30 0x7f338c142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.850 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f3391f07700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f338c142b30 0x7f338c142fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.140:48210/0 (socket says 172.21.15.140:48210) 2022-07-21T02:59:00.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f3390f05700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c062f30 0x7f338c0adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:39198/0 (socket says 172.21.15.140:39198) 2022-07-21T02:59:00.851 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.842+0000 7f3391f07700 1 -- 172.21.15.140:0/919996230 learned_addr learned my addr 172.21.15.140:0/919996230 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:00.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f3390f05700 1 -- 172.21.15.140:0/919996230 >> v2:172.21.15.152:3300/0 conn(0x7f338c142b30 msgr2=0x7f338c142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:59:00.852 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f3390f05700 1 --2- 172.21.15.140:0/919996230 >> v2:172.21.15.152:3300/0 conn(0x7f338c142b30 0x7f338c142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f3390f05700 1 -- 172.21.15.140:0/919996230 >> v1:172.21.15.152:6789/0 conn(0x7f338c05cf40 legacy=0x7f338c146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f3390f05700 1 -- 172.21.15.140:0/919996230 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c146560 con 0x7f338c062f30 2022-07-21T02:59:00.853 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f3391f07700 1 --2- 172.21.15.140:0/919996230 >> v2:172.21.15.152:3300/0 conn(0x7f338c142b30 0x7f338c142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T02:59:00.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f3390f05700 1 --2- 172.21.15.140:0/919996230 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c062f30 0x7f338c0adb50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f3388002f70 tx=0x7f3388006d90).ready entity=mon.2 client_cookie=43260432dd245992 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.854 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f3383fff700 1 -- 172.21.15.140:0/919996230 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338800d4a0 con 0x7f338c062f30 2022-07-21T02:59:00.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f3383fff700 1 -- 172.21.15.140:0/919996230 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3388000e90 con 0x7f338c062f30 2022-07-21T02:59:00.855 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f3383fff700 1 -- 172.21.15.140:0/919996230 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338800d7f0 con 0x7f338c062f30 2022-07-21T02:59:00.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f339396a700 1 -- 172.21.15.140:0/919996230 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c062f30 msgr2=0x7f338c0adb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.860 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f339396a700 1 --2- 172.21.15.140:0/919996230 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c062f30 0x7f338c0adb50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f3388002f70 tx=0x7f3388006d90).stop 2022-07-21T02:59:00.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f339396a700 1 -- 172.21.15.140:0/919996230 shutdown_connections 2022-07-21T02:59:00.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f339396a700 1 --2- 172.21.15.140:0/919996230 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c062f30 0x7f338c0adb50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.861 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f339396a700 1 --2- 172.21.15.140:0/919996230 >> v2:172.21.15.152:3300/0 conn(0x7f338c142b30 0x7f338c142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f339396a700 1 -- 172.21.15.140:0/919996230 >> 172.21.15.140:0/919996230 conn(0x7f338c0b7d00 msgr2=0x7f338c0b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:00.862 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f339396a700 1 -- 172.21.15.140:0/919996230 shutdown_connections 2022-07-21T02:59:00.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f339396a700 1 -- 172.21.15.140:0/919996230 wait complete. 2022-07-21T02:59:00.863 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f339396a700 1 Processor -- start 2022-07-21T02:59:00.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.846+0000 7f339396a700 1 -- start start 2022-07-21T02:59:00.864 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f339396a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c05cf40 0x7f338c0b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f339396a700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f338c062f30 0x7f338c0607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.865 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f339396a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f338c142b30 0x7f338c060cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f339396a700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c1566b0 con 0x7f338c062f30 2022-07-21T02:59:00.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f339396a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c155720 con 0x7f338c142b30 2022-07-21T02:59:00.866 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f339396a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0af190 con 0x7f338c05cf40 2022-07-21T02:59:00.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3390f05700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f338c062f30 0x7f338c0607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3390f05700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f338c062f30 0x7f338c0607d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42472/0 (socket says 172.21.15.140:42472) 2022-07-21T02:59:00.867 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3390f05700 1 -- 172.21.15.140:0/3169057415 learned_addr learned my addr 172.21.15.140:0/3169057415 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:00.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3391f07700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f338c142b30 0x7f338c060cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3391706700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c05cf40 0x7f338c0b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.868 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3390f05700 1 -- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c05cf40 msgr2=0x7f338c0b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3390f05700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c05cf40 0x7f338c0b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3390f05700 1 -- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f338c142b30 msgr2=0x7f338c060cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.869 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3390f05700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f338c142b30 0x7f338c060cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3390f05700 1 -- 172.21.15.140:0/3169057415 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3384009530 con 0x7f338c062f30 2022-07-21T02:59:00.870 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3391706700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f338c05cf40 0x7f338c0b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-07-21T02:59:00.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f3390f05700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f338c062f30 0x7f338c0607d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f338c09aac0 tx=0x7f3388014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f33827fc700 1 -- 172.21.15.140:0/3169057415 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33880117c0 con 0x7f338c062f30 2022-07-21T02:59:00.871 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f33827fc700 1 -- 172.21.15.140:0/3169057415 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3388011920 con 0x7f338c062f30 2022-07-21T02:59:00.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f339396a700 1 -- 172.21.15.140:0/3169057415 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338800b120 con 0x7f338c062f30 2022-07-21T02:59:00.872 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f33827fc700 1 -- 172.21.15.140:0/3169057415 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3388011c80 con 0x7f338c062f30 2022-07-21T02:59:00.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f339396a700 1 -- 172.21.15.140:0/3169057415 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c1426f0 con 0x7f338c062f30 2022-07-21T02:59:00.873 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.850+0000 7f339396a700 1 -- 172.21.15.140:0/3169057415 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f338c0aedb0 con 0x7f338c062f30 2022-07-21T02:59:00.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.854+0000 7f33827fc700 1 -- 172.21.15.140:0/3169057415 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f338800f070 con 0x7f338c062f30 2022-07-21T02:59:00.874 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.854+0000 7f33827fc700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3374053980 0x7f3374055e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.854+0000 7f33827fc700 1 -- 172.21.15.140:0/3169057415 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f338806cff0 con 0x7f338c062f30 2022-07-21T02:59:00.875 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.854+0000 7f33827fc700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] conn(0x7f337405a110 0x7f337405c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.854+0000 7f33827fc700 1 -- 172.21.15.140:0/3169057415 --> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f337405cbf0 con 0x7f337405a110 2022-07-21T02:59:00.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.854+0000 7f33827fc700 1 -- 172.21.15.140:0/3169057415 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f338806f570 con 0x7f338c062f30 2022-07-21T02:59:00.876 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.854+0000 7f3391706700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3374053980 0x7f3374055e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.854+0000 7f3391f07700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] conn(0x7f337405a110 0x7f337405c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.877 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.854+0000 7f3391706700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3374053980 0x7f3374055e40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f337c00b000 tx=0x7f337c009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.854+0000 7f3391f07700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] conn(0x7f337405a110 0x7f337405c570 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:00.878 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.854+0000 7f33827fc700 1 -- 172.21.15.140:0/3169057415 <== osd.6 v2:172.21.15.152:6808/23144 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f337405cbf0 con 0x7f337405a110 2022-07-21T02:59:00.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.870+0000 7f339396a700 1 -- 172.21.15.140:0/3169057415 --> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f338c0574e0 con 0x7f337405a110 2022-07-21T02:59:00.879 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.870+0000 7f33827fc700 1 -- 172.21.15.140:0/3169057415 <== osd.6 v2:172.21.15.152:6808/23144 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f338c0574e0 con 0x7f337405a110 2022-07-21T02:59:00.880 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T02:59:00.881 INFO:teuthology.orchestra.run.smithi140.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-07-21T02:59:00.881 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T02:59:00.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.870+0000 7f339396a700 1 -- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] conn(0x7f337405a110 msgr2=0x7f337405c570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.870+0000 7f339396a700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.152:6808/23144,v1:172.21.15.152:6809/23144] conn(0x7f337405a110 0x7f337405c570 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.883 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.874+0000 7f339396a700 1 -- 172.21.15.140:0/3169057415 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3374053980 msgr2=0x7f3374055e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.874+0000 7f339396a700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3374053980 0x7f3374055e40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f337c00b000 tx=0x7f337c009040).stop 2022-07-21T02:59:00.884 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.874+0000 7f339396a700 1 -- 172.21.15.140:0/3169057415 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f338c062f30 msgr2=0x7f338c0607d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.874+0000 7f339396a700 1 --2- 172.21.15.140:0/3169057415 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f338c062f30 0x7f338c0607d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f338c09aac0 tx=0x7f3388014000).stop 2022-07-21T02:59:00.885 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.874+0000 7f3391f07700 1 -- 172.21.15.140:0/3169057415 reap_dead start 2022-07-21T02:59:00.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.874+0000 7f339396a700 1 -- 172.21.15.140:0/3169057415 shutdown_connections 2022-07-21T02:59:00.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.874+0000 7f339396a700 1 -- 172.21.15.140:0/3169057415 >> 172.21.15.140:0/3169057415 conn(0x7f338c0b7d00 msgr2=0x7f338c144040 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:00.886 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.874+0000 7f339396a700 1 -- 172.21.15.140:0/3169057415 shutdown_connections 2022-07-21T02:59:00.887 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.874+0000 7f339396a700 1 -- 172.21.15.140:0/3169057415 wait complete. 2022-07-21T02:59:00.888 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2022-07-21T02:59:00.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa122495700 1 Processor -- start 2022-07-21T02:59:00.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa122495700 1 -- start start 2022-07-21T02:59:00.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa122495700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa11c0af8c0 0x7fa11c0ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa122495700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa11c143a10 0x7fa11c143e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:00.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa122495700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c0b0b80 con 0x7fa11c0b5970 2022-07-21T02:59:00.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa122495700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c05d4d0 con 0x7fa11c0af8c0 2022-07-21T02:59:00.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa122495700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c0b5d50 con 0x7fa11c143a10 2022-07-21T02:59:00.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa120c92700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa11c0af8c0 0x7fa11c0ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa120c92700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fa11c0af8c0 0x7fa11c0ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42482/0 (socket says 172.21.15.140:42482) 2022-07-21T02:59:00.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa120c92700 1 -- 172.21.15.140:0/4017443476 learned_addr learned my addr 172.21.15.140:0/4017443476 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:00.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa121c94700 1 --2- 172.21.15.140:0/4017443476 >> v2:172.21.15.152:3300/0 conn(0x7fa11c143a10 0x7fa11c143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:00.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa120c92700 1 -- 172.21.15.140:0/4017443476 >> v2:172.21.15.152:3300/0 conn(0x7fa11c143a10 msgr2=0x7fa11c143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:59:00.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa120c92700 1 --2- 172.21.15.140:0/4017443476 >> v2:172.21.15.152:3300/0 conn(0x7fa11c143a10 0x7fa11c143e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:00.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa120c92700 1 -- 172.21.15.140:0/4017443476 >> v1:172.21.15.152:6789/0 conn(0x7fa11c0b5970 legacy=0x7fa11c0ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:00.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa120c92700 1 -- 172.21.15.140:0/4017443476 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0b0db0 con 0x7fa11c0af8c0 2022-07-21T02:59:01.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa120c92700 1 --2- 172.21.15.140:0/4017443476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa11c0af8c0 0x7fa11c0ae810 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7fa1180032f0 tx=0x7fa118009290).ready entity=mon.0 client_cookie=f7589ee8d8f2fa34 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa113fff700 1 -- 172.21.15.140:0/4017443476 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa11800b870 con 0x7fa11c0af8c0 2022-07-21T02:59:01.000 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa113fff700 1 -- 172.21.15.140:0/4017443476 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11800b9d0 con 0x7fa11c0af8c0 2022-07-21T02:59:01.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.990+0000 7fa113fff700 1 -- 172.21.15.140:0/4017443476 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa11800bb40 con 0x7fa11c0af8c0 2022-07-21T02:59:01.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 -- 172.21.15.140:0/4017443476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa11c0af8c0 msgr2=0x7fa11c0ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 --2- 172.21.15.140:0/4017443476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa11c0af8c0 0x7fa11c0ae810 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7fa1180032f0 tx=0x7fa118009290).stop 2022-07-21T02:59:01.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 -- 172.21.15.140:0/4017443476 shutdown_connections 2022-07-21T02:59:01.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 --2- 172.21.15.140:0/4017443476 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa11c0af8c0 0x7fa11c0ae810 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 --2- 172.21.15.140:0/4017443476 >> v2:172.21.15.152:3300/0 conn(0x7fa11c143a10 0x7fa11c143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 -- 172.21.15.140:0/4017443476 >> 172.21.15.140:0/4017443476 conn(0x7fa11c063490 msgr2=0x7fa11c0b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:01.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 -- 172.21.15.140:0/4017443476 shutdown_connections 2022-07-21T02:59:01.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 -- 172.21.15.140:0/4017443476 wait complete. 2022-07-21T02:59:01.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 Processor -- start 2022-07-21T02:59:01.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 -- start start 2022-07-21T02:59:01.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa11c0af8c0 0x7fa11c145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa11c0b5970 0x7fa11c143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c143a10 0x7fa11c14c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1450a0 con 0x7fa11c0af8c0 2022-07-21T02:59:01.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1452d0 con 0x7fa11c143a10 2022-07-21T02:59:01.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa122495700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c145570 con 0x7fa11c0b5970 2022-07-21T02:59:01.010 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa121493700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa11c0af8c0 0x7fa11c145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa121c94700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c143a10 0x7fa11c14c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa121493700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa11c0af8c0 0x7fa11c145ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42488/0 (socket says 172.21.15.140:42488) 2022-07-21T02:59:01.011 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa121493700 1 -- 172.21.15.140:0/3852750985 learned_addr learned my addr 172.21.15.140:0/3852750985 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:01.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa120c92700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa11c0b5970 0x7fa11c143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa121c94700 1 -- 172.21.15.140:0/3852750985 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa11c0b5970 msgr2=0x7fa11c143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.012 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa121c94700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa11c0b5970 0x7fa11c143e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa121c94700 1 -- 172.21.15.140:0/3852750985 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa11c0af8c0 msgr2=0x7fa11c145ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.013 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa121c94700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa11c0af8c0 0x7fa11c145ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa121c94700 1 -- 172.21.15.140:0/3852750985 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa118009530 con 0x7fa11c143a10 2022-07-21T02:59:01.014 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.994+0000 7fa121c94700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c143a10 0x7fa11c14c850 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fa11400c000 tx=0x7fa114006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.015 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.998+0000 7fa1127fc700 1 -- 172.21.15.140:0/3852750985 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa114016070 con 0x7fa11c143a10 2022-07-21T02:59:01.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.998+0000 7fa1127fc700 1 -- 172.21.15.140:0/3852750985 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa114011750 con 0x7fa11c143a10 2022-07-21T02:59:01.016 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.998+0000 7fa122495700 1 -- 172.21.15.140:0/3852750985 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c14cd40 con 0x7fa11c143a10 2022-07-21T02:59:01.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.998+0000 7fa1127fc700 1 -- 172.21.15.140:0/3852750985 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa114011ab0 con 0x7fa11c143a10 2022-07-21T02:59:01.017 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.998+0000 7fa122495700 1 -- 172.21.15.140:0/3852750985 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c1406f0 con 0x7fa11c143a10 2022-07-21T02:59:01.018 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.998+0000 7fa122495700 1 -- 172.21.15.140:0/3852750985 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c145730 con 0x7fa11c143a10 2022-07-21T02:59:01.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.998+0000 7fa1127fc700 1 -- 172.21.15.140:0/3852750985 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fa114011c10 con 0x7fa11c143a10 2022-07-21T02:59:01.019 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.998+0000 7fa1127fc700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa0fc053b30 0x7fa0fc055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.998+0000 7fa121493700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa0fc053b30 0x7fa0fc055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.020 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.998+0000 7fa1127fc700 1 -- 172.21.15.140:0/3852750985 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fa11406bba0 con 0x7fa11c143a10 2022-07-21T02:59:01.021 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:00.998+0000 7fa121493700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa0fc053b30 0x7fa0fc055ff0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fa10c00d000 tx=0x7fa10c000bc0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.022 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.006+0000 7fa1127fc700 1 -- 172.21.15.140:0/3852750985 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fa114042c10 con 0x7fa11c143a10 2022-07-21T02:59:01.133 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.130+0000 7fa122495700 1 -- 172.21.15.140:0/3852750985 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fa11c0acae0 con 0x7fa0fc053b30 2022-07-21T02:59:01.134 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.130+0000 7fa1127fc700 1 -- 172.21.15.140:0/3852750985 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa11c0acae0 con 0x7fa0fc053b30 2022-07-21T02:59:01.137 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 -- 172.21.15.140:0/3852750985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa0fc053b30 msgr2=0x7fa0fc055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa0fc053b30 0x7fa0fc055ff0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fa10c00d000 tx=0x7fa10c000bc0).stop 2022-07-21T02:59:01.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 -- 172.21.15.140:0/3852750985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c143a10 msgr2=0x7fa11c14c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.138 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c143a10 0x7fa11c14c850 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fa11400c000 tx=0x7fa114006df0).stop 2022-07-21T02:59:01.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 -- 172.21.15.140:0/3852750985 shutdown_connections 2022-07-21T02:59:01.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fa0fc053b30 0x7fa0fc055ff0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.139 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c143a10 0x7fa11c14c850 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.140 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa11c0b5970 0x7fa11c143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.141 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 --2- 172.21.15.140:0/3852750985 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fa11c0af8c0 0x7fa11c145ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 -- 172.21.15.140:0/3852750985 >> 172.21.15.140:0/3852750985 conn(0x7fa11c063490 msgr2=0x7fa11c1478a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:01.142 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 -- 172.21.15.140:0/3852750985 shutdown_connections 2022-07-21T02:59:01.143 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.134+0000 7fa122495700 1 -- 172.21.15.140:0/3852750985 wait complete. 2022-07-21T02:59:01.143 INFO:teuthology.orchestra.run.smithi140.stderr:instructed osd(s) 6 to deep-scrub 2022-07-21T02:59:01.152 INFO:tasks.ceph:Scrubbing osd.7 2022-07-21T02:59:01.153 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2022-07-21T02:59:01.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.262+0000 7f3b8634c700 1 Processor -- start 2022-07-21T02:59:01.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.262+0000 7f3b8634c700 1 -- start start 2022-07-21T02:59:01.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.262+0000 7f3b8634c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3b800b4bc0 0x7f3b8014d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.262+0000 7f3b8634c700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b80066b10 con 0x7f3b800b6840 2022-07-21T02:59:01.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.262+0000 7f3b8634c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b800b6c20 con 0x7f3b800b3cf0 2022-07-21T02:59:01.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.262+0000 7f3b8634c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b800b5c90 con 0x7f3b800b4bc0 2022-07-21T02:59:01.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.262+0000 7f3b8534a700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7f3b800b6840 0x7f3b800b5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56642/0 (socket says 172.21.15.140:56642) 2022-07-21T02:59:01.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.262+0000 7f3b8534a700 1 -- 172.21.15.140:0/2268169198 learned_addr learned my addr 172.21.15.140:0/2268169198 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:59:01.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b85b4b700 1 --2- 172.21.15.140:0/2268169198 >> v2:172.21.15.152:3300/0 conn(0x7f3b800b4bc0 0x7f3b8014d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b77fff700 1 -- 172.21.15.140:0/2268169198 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763745884 0 0) 0x7f3b80066b10 con 0x7f3b800b6840 2022-07-21T02:59:01.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b77fff700 1 -- 172.21.15.140:0/2268169198 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b64003260 con 0x7f3b800b6840 2022-07-21T02:59:01.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b85b4b700 1 -- 172.21.15.140:0/2268169198 >> v1:172.21.15.152:6789/0 conn(0x7f3b800b3cf0 legacy=0x7f3b80146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b85b4b700 1 -- 172.21.15.140:0/2268169198 >> v1:172.21.15.140:6789/0 conn(0x7f3b800b6840 legacy=0x7f3b800b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b85b4b700 1 -- 172.21.15.140:0/2268169198 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b800aead0 con 0x7f3b800b4bc0 2022-07-21T02:59:01.273 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b85b4b700 1 --2- 172.21.15.140:0/2268169198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b800b4bc0 0x7f3b8014d610 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f3b780032f0 tx=0x7f3b780091d0).ready entity=mon.1 client_cookie=2d7aace3ebef57db server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.274 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b77fff700 1 -- 172.21.15.140:0/2268169198 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b7800b6c0 con 0x7f3b800b4bc0 2022-07-21T02:59:01.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b77fff700 1 -- 172.21.15.140:0/2268169198 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7800b820 con 0x7f3b800b4bc0 2022-07-21T02:59:01.275 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b77fff700 1 -- 172.21.15.140:0/2268169198 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b7800b990 con 0x7f3b800b4bc0 2022-07-21T02:59:01.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b8634c700 1 -- 172.21.15.140:0/2268169198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b800b4bc0 msgr2=0x7f3b8014d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.276 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b8634c700 1 --2- 172.21.15.140:0/2268169198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b800b4bc0 0x7f3b8014d610 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f3b780032f0 tx=0x7f3b780091d0).stop 2022-07-21T02:59:01.281 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b8634c700 1 -- 172.21.15.140:0/2268169198 shutdown_connections 2022-07-21T02:59:01.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b8634c700 1 --2- 172.21.15.140:0/2268169198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b800b4bc0 0x7f3b8014d610 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.282 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b8634c700 1 -- 172.21.15.140:0/2268169198 >> 172.21.15.140:0/2268169198 conn(0x7f3b800b9e10 msgr2=0x7f3b800b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:01.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b8634c700 1 -- 172.21.15.140:0/2268169198 shutdown_connections 2022-07-21T02:59:01.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.266+0000 7f3b8634c700 1 -- 172.21.15.140:0/2268169198 wait complete. 2022-07-21T02:59:01.283 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b8634c700 1 Processor -- start 2022-07-21T02:59:01.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b8634c700 1 -- start start 2022-07-21T02:59:01.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b8634c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b800b3cf0 0x7f3b800b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.284 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b8634c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b800b4bc0 0x7f3b80113660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b8634c700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b800b6840 0x7f3b8013ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b8634c700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b800b0820 con 0x7f3b800b6840 2022-07-21T02:59:01.285 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b8634c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b80112f20 con 0x7f3b800b4bc0 2022-07-21T02:59:01.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b8634c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b801131c0 con 0x7f3b800b3cf0 2022-07-21T02:59:01.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b85b4b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b800b6840 0x7f3b8013ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.286 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b85b4b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b800b6840 0x7f3b8013ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42506/0 (socket says 172.21.15.140:42506) 2022-07-21T02:59:01.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b85b4b700 1 -- 172.21.15.140:0/2440463710 learned_addr learned my addr 172.21.15.140:0/2440463710 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:01.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b8534a700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b800b3cf0 0x7f3b800b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.287 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b84b49700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b800b4bc0 0x7f3b80113660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b84b49700 1 -- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b800b3cf0 msgr2=0x7f3b800b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b84b49700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3b800b3cf0 0x7f3b800b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.288 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b84b49700 1 -- 172.21.15.140:0/2440463710 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b800b6840 msgr2=0x7f3b8013ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b84b49700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b800b6840 0x7f3b8013ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b84b49700 1 -- 172.21.15.140:0/2440463710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b78009470 con 0x7f3b800b4bc0 2022-07-21T02:59:01.289 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b85b4b700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f3b800b6840 0x7f3b8013ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-07-21T02:59:01.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b84b49700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b800b4bc0 0x7f3b80113660 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f3b80067e80 tx=0x7f3b7c007b70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.291 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b767fc700 1 -- 172.21.15.140:0/2440463710 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b7c00cc70 con 0x7f3b800b4bc0 2022-07-21T02:59:01.293 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b767fc700 1 -- 172.21.15.140:0/2440463710 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c00cdd0 con 0x7f3b800b4bc0 2022-07-21T02:59:01.294 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b8634c700 1 -- 172.21.15.140:0/2440463710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b80112c50 con 0x7f3b800b4bc0 2022-07-21T02:59:01.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b767fc700 1 -- 172.21.15.140:0/2440463710 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b7c007220 con 0x7f3b800b4bc0 2022-07-21T02:59:01.295 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.270+0000 7f3b8634c700 1 -- 172.21.15.140:0/2440463710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b800b1360 con 0x7f3b800b4bc0 2022-07-21T02:59:01.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.274+0000 7f3b8634c700 1 -- 172.21.15.140:0/2440463710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3b801133c0 con 0x7f3b800b4bc0 2022-07-21T02:59:01.296 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.274+0000 7f3b767fc700 1 -- 172.21.15.140:0/2440463710 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f3b7c010030 con 0x7f3b800b4bc0 2022-07-21T02:59:01.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.274+0000 7f3b767fc700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b680539e0 0x7f3b68055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.297 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.274+0000 7f3b767fc700 1 -- 172.21.15.140:0/2440463710 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f3b7c06c100 con 0x7f3b800b4bc0 2022-07-21T02:59:01.298 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.274+0000 7f3b767fc700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] conn(0x7f3b6805a0d0 0x7f3b6805c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.274+0000 7f3b767fc700 1 -- 172.21.15.140:0/2440463710 --> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b6805cbb0 con 0x7f3b6805a0d0 2022-07-21T02:59:01.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.274+0000 7f3b767fc700 1 -- 172.21.15.140:0/2440463710 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b7c06c460 con 0x7f3b800b4bc0 2022-07-21T02:59:01.299 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.274+0000 7f3b8534a700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b680539e0 0x7f3b68055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.300 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.274+0000 7f3b85b4b700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] conn(0x7f3b6805a0d0 0x7f3b6805c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.274+0000 7f3b8534a700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b680539e0 0x7f3b68055ea0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f3b7000d000 tx=0x7f3b70009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.301 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.274+0000 7f3b85b4b700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] conn(0x7f3b6805a0d0 0x7f3b6805c530 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.302 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.278+0000 7f3b767fc700 1 -- 172.21.15.140:0/2440463710 <== osd.7 v2:172.21.15.152:6824/23142 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+21404 (crc 0 0 0) 0x7f3b6805cbb0 con 0x7f3b6805a0d0 2022-07-21T02:59:01.303 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b8634c700 1 -- 172.21.15.140:0/2440463710 --> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3b80142ef0 con 0x7f3b6805a0d0 2022-07-21T02:59:01.304 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b767fc700 1 -- 172.21.15.140:0/2440463710 <== osd.7 v2:172.21.15.152:6824/23142 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3b80142ef0 con 0x7f3b6805a0d0 2022-07-21T02:59:01.305 INFO:teuthology.orchestra.run.smithi140.stdout:{ 2022-07-21T02:59:01.305 INFO:teuthology.orchestra.run.smithi140.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2022-07-21T02:59:01.305 INFO:teuthology.orchestra.run.smithi140.stdout:} 2022-07-21T02:59:01.308 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b8634c700 1 -- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] conn(0x7f3b6805a0d0 msgr2=0x7f3b6805c530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b8634c700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:6824/23142,v1:172.21.15.152:6825/23142] conn(0x7f3b6805a0d0 0x7f3b6805c530 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b8634c700 1 -- 172.21.15.140:0/2440463710 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b680539e0 msgr2=0x7f3b68055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.309 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b8634c700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f3b680539e0 0x7f3b68055ea0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f3b7000d000 tx=0x7f3b70009040).stop 2022-07-21T02:59:01.310 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b8634c700 1 -- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b800b4bc0 msgr2=0x7f3b80113660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b8634c700 1 --2- 172.21.15.140:0/2440463710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b800b4bc0 0x7f3b80113660 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f3b80067e80 tx=0x7f3b7c007b70).stop 2022-07-21T02:59:01.311 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b85b4b700 1 -- 172.21.15.140:0/2440463710 reap_dead start 2022-07-21T02:59:01.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b8634c700 1 -- 172.21.15.140:0/2440463710 shutdown_connections 2022-07-21T02:59:01.312 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b8634c700 1 -- 172.21.15.140:0/2440463710 >> 172.21.15.140:0/2440463710 conn(0x7f3b800b9e10 msgr2=0x7f3b8005c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:01.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b8634c700 1 -- 172.21.15.140:0/2440463710 shutdown_connections 2022-07-21T02:59:01.313 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.290+0000 7f3b8634c700 1 -- 172.21.15.140:0/2440463710 wait complete. 2022-07-21T02:59:01.314 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2022-07-21T02:59:01.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba7528e700 1 Processor -- start 2022-07-21T02:59:01.424 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba7528e700 1 -- start start 2022-07-21T02:59:01.427 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba7528e700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fba700acc10 0x7fba700acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba7528e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba70095b10 0x7fba70095f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.428 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba7528e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba700abde0 con 0x7fba70096460 2022-07-21T02:59:01.429 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba7528e700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba70096c80 con 0x7fba700acc10 2022-07-21T02:59:01.430 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba7528e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba70098910 con 0x7fba70095b10 2022-07-21T02:59:01.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fba700acc10 0x7fba700acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.431 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6ffff700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7fba700acc10 0x7fba700acff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42512/0 (socket says 172.21.15.140:42512) 2022-07-21T02:59:01.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6ffff700 1 -- 172.21.15.140:0/2958232337 learned_addr learned my addr 172.21.15.140:0/2958232337 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:01.432 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6f7fe700 1 --2- 172.21.15.140:0/2958232337 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba70095b10 0x7fba70095f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.433 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6ffff700 1 -- 172.21.15.140:0/2958232337 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba70095b10 msgr2=0x7fba70095f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:59:01.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6ffff700 1 --2- 172.21.15.140:0/2958232337 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba70095b10 0x7fba70095f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.434 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6ffff700 1 -- 172.21.15.140:0/2958232337 >> v1:172.21.15.152:6789/0 conn(0x7fba70096460 legacy=0x7fba701430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.435 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6ffff700 1 -- 172.21.15.140:0/2958232337 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba70098b20 con 0x7fba700acc10 2022-07-21T02:59:01.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6ffff700 1 --2- 172.21.15.140:0/2958232337 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba700acc10 0x7fba700acff0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7fba58002f70 tx=0x7fba58006eb0).ready entity=mon.0 client_cookie=a22db33943eead05 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.436 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6effd700 1 -- 172.21.15.140:0/2958232337 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba5800d570 con 0x7fba700acc10 2022-07-21T02:59:01.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6effd700 1 -- 172.21.15.140:0/2958232337 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba58000e90 con 0x7fba700acc10 2022-07-21T02:59:01.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.422+0000 7fba6effd700 1 -- 172.21.15.140:0/2958232337 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba5800d900 con 0x7fba700acc10 2022-07-21T02:59:01.437 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 -- 172.21.15.140:0/2958232337 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba700acc10 msgr2=0x7fba700acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 --2- 172.21.15.140:0/2958232337 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba700acc10 0x7fba700acff0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7fba58002f70 tx=0x7fba58006eb0).stop 2022-07-21T02:59:01.438 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 -- 172.21.15.140:0/2958232337 shutdown_connections 2022-07-21T02:59:01.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 --2- 172.21.15.140:0/2958232337 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba70095b10 0x7fba70095f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.439 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 --2- 172.21.15.140:0/2958232337 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba700acc10 0x7fba700acff0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 -- 172.21.15.140:0/2958232337 >> 172.21.15.140:0/2958232337 conn(0x7fba700b6210 msgr2=0x7fba700b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:01.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 -- 172.21.15.140:0/2958232337 shutdown_connections 2022-07-21T02:59:01.440 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 -- 172.21.15.140:0/2958232337 wait complete. 2022-07-21T02:59:01.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 Processor -- start 2022-07-21T02:59:01.445 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 -- start start 2022-07-21T02:59:01.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba70095b10 0x7fba700b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.446 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba70096460 0x7fba7009a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba700acc10 0x7fba700afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.447 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba700af7c0 con 0x7fba70095b10 2022-07-21T02:59:01.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7009c110 con 0x7fba700acc10 2022-07-21T02:59:01.454 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba7528e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba700adf30 con 0x7fba70096460 2022-07-21T02:59:01.455 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba6ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba70095b10 0x7fba700b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba74a8d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba700acc10 0x7fba700afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.456 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba6ffff700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba70095b10 0x7fba700b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42518/0 (socket says 172.21.15.140:42518) 2022-07-21T02:59:01.457 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba6ffff700 1 -- 172.21.15.140:0/3469131993 learned_addr learned my addr 172.21.15.140:0/3469131993 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:01.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.426+0000 7fba6f7fe700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba70096460 0x7fba7009a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.458 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6ffff700 1 -- 172.21.15.140:0/3469131993 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba70096460 msgr2=0x7fba7009a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.459 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6ffff700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba70096460 0x7fba7009a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6ffff700 1 -- 172.21.15.140:0/3469131993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba700acc10 msgr2=0x7fba700afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.460 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6ffff700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba700acc10 0x7fba700afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6ffff700 1 -- 172.21.15.140:0/3469131993 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5800b1e0 con 0x7fba70095b10 2022-07-21T02:59:01.461 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6ffff700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba70095b10 0x7fba700b5870 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7fba58002f70 tx=0x7fba58015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.462 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6d7fa700 1 -- 172.21.15.140:0/3469131993 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba58011760 con 0x7fba70095b10 2022-07-21T02:59:01.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6d7fa700 1 -- 172.21.15.140:0/3469131993 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba580118c0 con 0x7fba70095b10 2022-07-21T02:59:01.463 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba7528e700 1 -- 172.21.15.140:0/3469131993 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba700ae1d0 con 0x7fba70095b10 2022-07-21T02:59:01.464 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6d7fa700 1 -- 172.21.15.140:0/3469131993 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba58011c20 con 0x7fba70095b10 2022-07-21T02:59:01.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba7528e700 1 -- 172.21.15.140:0/3469131993 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7009ab70 con 0x7fba70095b10 2022-07-21T02:59:01.465 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba7528e700 1 -- 172.21.15.140:0/3469131993 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba700574e0 con 0x7fba70095b10 2022-07-21T02:59:01.466 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6d7fa700 1 -- 172.21.15.140:0/3469131993 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fba58025030 con 0x7fba70095b10 2022-07-21T02:59:01.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6d7fa700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba54053b00 0x7fba54055fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.467 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6d7fa700 1 -- 172.21.15.140:0/3469131993 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fba5806ce00 con 0x7fba70095b10 2022-07-21T02:59:01.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.430+0000 7fba6f7fe700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba54053b00 0x7fba54055fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.468 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.434+0000 7fba6f7fe700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba54053b00 0x7fba54055fc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fba6000b000 tx=0x7fba600091d0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.469 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.434+0000 7fba6d7fa700 1 -- 172.21.15.140:0/3469131993 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fba58043ef0 con 0x7fba70095b10 2022-07-21T02:59:01.564 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.562+0000 7fba7528e700 1 -- 172.21.15.140:0/3469131993 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fba7009c330 con 0x7fba54053b00 2022-07-21T02:59:01.566 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.562+0000 7fba6d7fa700 1 -- 172.21.15.140:0/3469131993 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fba7009c330 con 0x7fba54053b00 2022-07-21T02:59:01.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 -- 172.21.15.140:0/3469131993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba54053b00 msgr2=0x7fba54055fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba54053b00 0x7fba54055fc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7fba6000b000 tx=0x7fba600091d0).stop 2022-07-21T02:59:01.568 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 -- 172.21.15.140:0/3469131993 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba70095b10 msgr2=0x7fba700b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba70095b10 0x7fba700b5870 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7fba58002f70 tx=0x7fba58015000).stop 2022-07-21T02:59:01.569 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 -- 172.21.15.140:0/3469131993 shutdown_connections 2022-07-21T02:59:01.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fba54053b00 0x7fba54055fc0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fba700acc10 0x7fba700afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.570 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fba70096460 0x7fba7009a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 --2- 172.21.15.140:0/3469131993 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fba70095b10 0x7fba700b5870 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 -- 172.21.15.140:0/3469131993 >> 172.21.15.140:0/3469131993 conn(0x7fba700b6210 msgr2=0x7fba70061e80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:01.571 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 -- 172.21.15.140:0/3469131993 shutdown_connections 2022-07-21T02:59:01.572 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.566+0000 7fba7528e700 1 -- 172.21.15.140:0/3469131993 wait complete. 2022-07-21T02:59:01.574 INFO:teuthology.orchestra.run.smithi140.stderr:instructed osd(s) 7 to deep-scrub 2022-07-21T02:59:01.578 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-07-21T02:59:01.677 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.674+0000 7f172da18700 1 Processor -- start 2022-07-21T02:59:01.678 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.674+0000 7f172da18700 1 -- start start 2022-07-21T02:59:01.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.674+0000 7f172da18700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17280adff0 0x7f17280b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.681 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.674+0000 7f172da18700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f17280af8e0 0x7f172805bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.674+0000 7f172da18700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1728147010 con 0x7f17280b5650 2022-07-21T02:59:01.682 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.674+0000 7f172da18700 1 -- --> v2:172.21.15.140:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17281472b0 con 0x7f17280af8e0 2022-07-21T02:59:01.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.674+0000 7f172da18700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1728146d40 con 0x7f17280adff0 2022-07-21T02:59:01.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.674+0000 7f172d217700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f17280af8e0 0x7f172805bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.683 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.674+0000 7f172d217700 1 --2- >> v2:172.21.15.140:3300/0 conn(0x7f17280af8e0 0x7f172805bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42530/0 (socket says 172.21.15.140:42530) 2022-07-21T02:59:01.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.674+0000 7f172d217700 1 -- 172.21.15.140:0/2192998760 learned_addr learned my addr 172.21.15.140:0/2192998760 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:01.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f1727fff700 1 --2- 172.21.15.140:0/2192998760 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17280adff0 0x7f17280b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.684 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172d217700 1 -- 172.21.15.140:0/2192998760 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17280adff0 msgr2=0x7f17280b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-07-21T02:59:01.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172d217700 1 --2- 172.21.15.140:0/2192998760 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17280adff0 0x7f17280b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172d217700 1 -- 172.21.15.140:0/2192998760 >> v1:172.21.15.152:6789/0 conn(0x7f17280b5650 legacy=0x7f1728147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.685 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172d217700 1 -- 172.21.15.140:0/2192998760 --> v2:172.21.15.140:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17280acae0 con 0x7f17280af8e0 2022-07-21T02:59:01.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f17277fe700 1 -- 172.21.15.140:0/2192998760 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182712522 0 0) 0x7f1728147010 con 0x7f17280b5650 2022-07-21T02:59:01.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172d217700 1 --2- 172.21.15.140:0/2192998760 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17280af8e0 0x7f172805bd90 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f1718002f70 tx=0x7f1718006eb0).ready entity=mon.0 client_cookie=acdda0f9e17334c6 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.686 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f17277fe700 1 -- 172.21.15.140:0/2192998760 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171800d570 con 0x7f17280af8e0 2022-07-21T02:59:01.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f17277fe700 1 -- 172.21.15.140:0/2192998760 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1718000e90 con 0x7f17280af8e0 2022-07-21T02:59:01.687 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f17277fe700 1 -- 172.21.15.140:0/2192998760 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f171800d8c0 con 0x7f17280af8e0 2022-07-21T02:59:01.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172da18700 1 -- 172.21.15.140:0/2192998760 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17280af8e0 msgr2=0x7f172805bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172da18700 1 --2- 172.21.15.140:0/2192998760 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17280af8e0 0x7f172805bd90 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f1718002f70 tx=0x7f1718006eb0).stop 2022-07-21T02:59:01.688 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172da18700 1 -- 172.21.15.140:0/2192998760 shutdown_connections 2022-07-21T02:59:01.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172da18700 1 --2- 172.21.15.140:0/2192998760 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17280af8e0 0x7f172805bd90 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172da18700 1 --2- 172.21.15.140:0/2192998760 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17280adff0 0x7f17280b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.689 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172da18700 1 -- 172.21.15.140:0/2192998760 >> 172.21.15.140:0/2192998760 conn(0x7f17280634e0 msgr2=0x7f17280b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:01.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172da18700 1 -- 172.21.15.140:0/2192998760 shutdown_connections 2022-07-21T02:59:01.690 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.678+0000 7f172da18700 1 -- 172.21.15.140:0/2192998760 wait complete. 2022-07-21T02:59:01.693 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172da18700 1 Processor -- start 2022-07-21T02:59:01.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172da18700 1 -- start start 2022-07-21T02:59:01.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172da18700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17280adff0 0x7f17280b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.694 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172da18700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17280af8e0 0x7f17281566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172da18700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17280b5650 0x7f1728156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172da18700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728063b50 con 0x7f17280b5650 2022-07-21T02:59:01.695 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172da18700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728063df0 con 0x7f17280af8e0 2022-07-21T02:59:01.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172da18700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1728064090 con 0x7f17280adff0 2022-07-21T02:59:01.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172d217700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17280b5650 0x7f1728156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.696 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172d217700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17280b5650 0x7f1728156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42536/0 (socket says 172.21.15.140:42536) 2022-07-21T02:59:01.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172d217700 1 -- 172.21.15.140:0/1590804939 learned_addr learned my addr 172.21.15.140:0/1590804939 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:01.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172ca16700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17280adff0 0x7f17280b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.697 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f1727fff700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17280af8e0 0x7f17281566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172d217700 1 -- 172.21.15.140:0/1590804939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17280adff0 msgr2=0x7f17280b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172d217700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17280adff0 0x7f17280b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.698 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172d217700 1 -- 172.21.15.140:0/1590804939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17280af8e0 msgr2=0x7f17281566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172d217700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17280af8e0 0x7f17281566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172d217700 1 -- 172.21.15.140:0/1590804939 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171800b1e0 con 0x7f17280b5650 2022-07-21T02:59:01.699 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.682+0000 7f172d217700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17280b5650 0x7f1728156bd0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f1718002f70 tx=0x7f1718015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.686+0000 7f1725ffb700 1 -- 172.21.15.140:0/1590804939 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1718011790 con 0x7f17280b5650 2022-07-21T02:59:01.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.686+0000 7f1725ffb700 1 -- 172.21.15.140:0/1590804939 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17180118f0 con 0x7f17280b5650 2022-07-21T02:59:01.700 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.686+0000 7f172da18700 1 -- 172.21.15.140:0/1590804939 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17280b3400 con 0x7f17280b5650 2022-07-21T02:59:01.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.686+0000 7f1725ffb700 1 -- 172.21.15.140:0/1590804939 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1718011c50 con 0x7f17280b5650 2022-07-21T02:59:01.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.686+0000 7f172da18700 1 -- 172.21.15.140:0/1590804939 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1728157370 con 0x7f17280b5650 2022-07-21T02:59:01.701 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.686+0000 7f172da18700 1 -- 172.21.15.140:0/1590804939 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1728065080 con 0x7f17280b5650 2022-07-21T02:59:01.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.686+0000 7f1725ffb700 1 -- 172.21.15.140:0/1590804939 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7f1718025030 con 0x7f17280b5650 2022-07-21T02:59:01.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.686+0000 7f1725ffb700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f17000539e0 0x7f1700055ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:01.702 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.686+0000 7f1725ffb700 1 -- 172.21.15.140:0/1590804939 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f171806cdf0 con 0x7f17280b5650 2022-07-21T02:59:01.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.686+0000 7f172ca16700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f17000539e0 0x7f1700055ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:01.703 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.686+0000 7f172ca16700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f17000539e0 0x7f1700055ea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f172000d000 tx=0x7f1720009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:01.704 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.694+0000 7f1725ffb700 1 -- 172.21.15.140:0/1590804939 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7f1718043ff0 con 0x7f17280b5650 2022-07-21T02:59:01.817 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.814+0000 7f172da18700 1 -- 172.21.15.140:0/1590804939 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1728063990 con 0x7f17000539e0 2022-07-21T02:59:01.822 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.818+0000 7f1725ffb700 1 -- 172.21.15.140:0/1590804939 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68484 (secure 0 0 0) 0x7f1728063990 con 0x7f17000539e0 2022-07-21T02:59:01.824 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T02:59:01.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 -- 172.21.15.140:0/1590804939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f17000539e0 msgr2=0x7f1700055ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.826 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f17000539e0 0x7f1700055ea0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f172000d000 tx=0x7f1720009040).stop 2022-07-21T02:59:01.827 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 -- 172.21.15.140:0/1590804939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17280b5650 msgr2=0x7f1728156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:01.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17280b5650 0x7f1728156bd0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f1718002f70 tx=0x7f1718015000).stop 2022-07-21T02:59:01.828 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 -- 172.21.15.140:0/1590804939 shutdown_connections 2022-07-21T02:59:01.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7f17000539e0 0x7f1700055ea0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.829 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7f17280b5650 0x7f1728156bd0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.830 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17280af8e0 0x7f17281566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.831 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 --2- 172.21.15.140:0/1590804939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f17280adff0 0x7f17280b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:01.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 -- 172.21.15.140:0/1590804939 >> 172.21.15.140:0/1590804939 conn(0x7f17280634e0 msgr2=0x7f172814a250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:01.832 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 -- 172.21.15.140:0/1590804939 shutdown_connections 2022-07-21T02:59:01.833 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:01.822+0000 7f172da18700 1 -- 172.21.15.140:0/1590804939 wait complete. 2022-07-21T02:59:01.834 INFO:teuthology.orchestra.run.smithi140.stderr:dumped all 2022-07-21T02:59:01.836 INFO:teuthology.orchestra.run.smithi140.stdout:{"pg_ready":true,"pg_map":{"version":2268,"stamp":"2022-07-21T02:59:01.061475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":6385408,"kb_used_data":743209632,"kb_used_omap":27966,"kb_used_meta":0,"kb_avail":743209632,"statfs":{"total":767585320960,"available":761046663168,"internally_reserved":0,"allocated":761046663168,"data_stored":761046663168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28638144,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003248"},"pg_stats":[{"pgid":"1.0","version":"14'2","reported_seq":137,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.525923+0000","last_change":"2022-07-21T01:48:22.241647+0000","last_active":"2022-07-21T01:58:24.525923+0000","last_peered":"2022-07-21T01:58:24.525923+0000","last_clean":"2022-07-21T01:58:24.525923+0000","last_became_active":"2022-07-21T01:48:22.240745+0000","last_became_peered":"2022-07-21T01:48:22.240745+0000","last_unstale":"2022-07-21T01:58:24.525923+0000","last_undegraded":"2022-07-21T01:58:24.525923+0000","last_fullsized":"2022-07-21T01:58:24.525923+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:17.950031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:17.950031+0000","last_clean_scrub_stamp":"2022-07-21T01:48:17.950031+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.652618+0000","last_change":"2022-07-21T01:48:27.974374+0000","last_active":"2022-07-21T01:58:24.652618+0000","last_peered":"2022-07-21T01:58:24.652618+0000","last_clean":"2022-07-21T01:58:24.652618+0000","last_became_active":"2022-07-21T01:48:27.973977+0000","last_became_peered":"2022-07-21T01:48:27.973977+0000","last_unstale":"2022-07-21T01:58:24.652618+0000","last_undegraded":"2022-07-21T01:58:24.652618+0000","last_fullsized":"2022-07-21T01:58:24.652618+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.525740+0000","last_change":"2022-07-21T01:48:27.974429+0000","last_active":"2022-07-21T01:58:24.525740+0000","last_peered":"2022-07-21T01:58:24.525740+0000","last_clean":"2022-07-21T01:58:24.525740+0000","last_became_active":"2022-07-21T01:48:27.973957+0000","last_became_peered":"2022-07-21T01:48:27.973957+0000","last_unstale":"2022-07-21T01:58:24.525740+0000","last_undegraded":"2022-07-21T01:58:24.525740+0000","last_fullsized":"2022-07-21T01:58:24.525740+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":141,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.652197+0000","last_change":"2022-07-21T01:48:27.968845+0000","last_active":"2022-07-21T01:58:24.652197+0000","last_peered":"2022-07-21T01:58:24.652197+0000","last_clean":"2022-07-21T01:58:24.652197+0000","last_became_active":"2022-07-21T01:48:27.967411+0000","last_became_peered":"2022-07-21T01:48:27.967411+0000","last_unstale":"2022-07-21T01:58:24.652197+0000","last_undegraded":"2022-07-21T01:58:24.652197+0000","last_fullsized":"2022-07-21T01:58:24.652197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.652301+0000","last_change":"2022-07-21T01:48:27.974179+0000","last_active":"2022-07-21T01:58:24.652301+0000","last_peered":"2022-07-21T01:58:24.652301+0000","last_clean":"2022-07-21T01:58:24.652301+0000","last_became_active":"2022-07-21T01:48:27.973744+0000","last_became_peered":"2022-07-21T01:48:27.973744+0000","last_unstale":"2022-07-21T01:58:24.652301+0000","last_undegraded":"2022-07-21T01:58:24.652301+0000","last_fullsized":"2022-07-21T01:58:24.652301+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.653584+0000","last_change":"2022-07-21T01:48:27.992744+0000","last_active":"2022-07-21T01:58:24.653584+0000","last_peered":"2022-07-21T01:58:24.653584+0000","last_clean":"2022-07-21T01:58:24.653584+0000","last_became_active":"2022-07-21T01:48:27.992216+0000","last_became_peered":"2022-07-21T01:48:27.992216+0000","last_unstale":"2022-07-21T01:58:24.653584+0000","last_undegraded":"2022-07-21T01:58:24.653584+0000","last_fullsized":"2022-07-21T01:58:24.653584+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":133,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.526165+0000","last_change":"2022-07-21T01:48:27.996870+0000","last_active":"2022-07-21T01:58:24.526165+0000","last_peered":"2022-07-21T01:58:24.526165+0000","last_clean":"2022-07-21T01:58:24.526165+0000","last_became_active":"2022-07-21T01:48:27.996456+0000","last_became_peered":"2022-07-21T01:48:27.996456+0000","last_unstale":"2022-07-21T01:58:24.526165+0000","last_undegraded":"2022-07-21T01:58:24.526165+0000","last_fullsized":"2022-07-21T01:58:24.526165+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.653792+0000","last_change":"2022-07-21T01:48:27.970841+0000","last_active":"2022-07-21T01:58:24.653792+0000","last_peered":"2022-07-21T01:58:24.653792+0000","last_clean":"2022-07-21T01:58:24.653792+0000","last_became_active":"2022-07-21T01:48:27.970498+0000","last_became_peered":"2022-07-21T01:48:27.970498+0000","last_unstale":"2022-07-21T01:58:24.653792+0000","last_undegraded":"2022-07-21T01:58:24.653792+0000","last_fullsized":"2022-07-21T01:58:24.653792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":145,"reported_epoch":153,"state":"active+clean","last_fresh":"2022-07-21T01:58:24.527727+0000","last_change":"2022-07-21T01:48:27.969947+0000","last_active":"2022-07-21T01:58:24.527727+0000","last_peered":"2022-07-21T01:58:24.527727+0000","last_clean":"2022-07-21T01:58:24.527727+0000","last_became_active":"2022-07-21T01:48:27.968884+0000","last_became_peered":"2022-07-21T01:48:27.968884+0000","last_unstale":"2022-07-21T01:58:24.527727+0000","last_undegraded":"2022-07-21T01:58:24.527727+0000","last_fullsized":"2022-07-21T01:58:24.527727+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T01:48:26.950679+0000","last_clean_scrub_stamp":"2022-07-21T01:48:26.950679+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":1,"up_from":13,"seq":55834575698,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3974,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4069881,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.623,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.41799999999999998},"max":{"1min":0.83699999999999997,"5min":0.96199999999999997,"15min":1.131},"last":0.501},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.59699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95699999999999996,"5min":0.96899999999999997,"15min":1.018},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.622},"min":{"1min":0.52200000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.79300000000000004,"5min":0.83399999999999996,"15min":1.069},"last":0.623},{"interface":"front","average":{"1min":0.75,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.501,"15min":0.496},"max":{"1min":0.90600000000000003,"5min":1.105,"15min":1.3779999999999999},"last":0.69999999999999996}]},{"osd":3,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.52300000000000002},"max":{"1min":0.93200000000000005,"5min":1.006,"15min":1.3240000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93000000000000005,"5min":0.98099999999999998,"15min":1.2290000000000001},"last":0.79400000000000004}]},{"osd":4,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.90500000000000003,"15min":0.89000000000000001},"min":{"1min":0.85399999999999998,"5min":0.67100000000000004,"15min":0.63300000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.583},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.86899999999999999,"15min":0.876},"min":{"1min":0.63900000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.008,"5min":1.0960000000000001,"15min":1.335},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.90600000000000003,"15min":0.89900000000000002},"min":{"1min":0.79500000000000004,"5min":0.63800000000000001,"15min":0.59699999999999998},"max":{"1min":1.0960000000000001,"5min":1.2010000000000001,"15min":1.4970000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.999,"15min":0.98599999999999999},"min":{"1min":0.872,"5min":0.73899999999999999,"15min":0.56100000000000005},"max":{"1min":1.147,"5min":1.3400000000000001,"15min":1.569},"last":0.92900000000000005}]},{"osd":6,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.95299999999999996,"15min":0.95399999999999996},"min":{"1min":0.78900000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.054,"5min":1.2969999999999999,"15min":1.4099999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.94499999999999995,"15min":0.95799999999999996},"min":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.67700000000000005},"max":{"1min":1.1020000000000001,"5min":1.1930000000000001,"15min":1.548},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0269999999999999,"15min":1.014},"min":{"1min":0.79700000000000004,"5min":0.746,"15min":0.72899999999999998},"max":{"1min":1.1579999999999999,"5min":1.3879999999999999,"15min":1.613},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0249999999999999,"15min":1.0229999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.69599999999999995},"max":{"1min":1.159,"5min":1.425,"15min":1.522},"last":0.95999999999999996}]}]},{"osd":7,"up_from":13,"seq":55834575697,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3677,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3765344,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.80800000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1679999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.80600000000000005,"15min":0.80900000000000005},"min":{"1min":0.58599999999999997,"5min":0.54400000000000004,"15min":0.52500000000000002},"max":{"1min":1.0389999999999999,"5min":1.085,"15min":1.181},"last":0.85599999999999998}]},{"osd":1,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.871,"15min":0.88400000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.54700000000000004},"max":{"1min":1.133,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.91300000000000003,"15min":0.91800000000000004},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.61099999999999999},"max":{"1min":1.0700000000000001,"5min":1.1140000000000001,"15min":1.2569999999999999},"last":0.97899999999999998}]},{"osd":2,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.88100000000000001},"min":{"1min":0.79200000000000004,"5min":0.58799999999999997,"15min":0.51700000000000002},"max":{"1min":1.0369999999999999,"5min":1.119,"15min":1.153},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.92000000000000004,"15min":0.92300000000000004},"min":{"1min":0.79800000000000004,"5min":0.65700000000000003,"15min":0.59899999999999998},"max":{"1min":1.1100000000000001,"5min":1.1519999999999999,"15min":1.1879999999999999},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83799999999999997,"15min":0.84799999999999998},"min":{"1min":0.66100000000000003,"5min":0.65900000000000003,"15min":0.57999999999999996},"max":{"1min":1.069,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89800000000000002,"15min":0.88500000000000001},"min":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.083,"5min":1.238,"15min":1.238},"last":0.67100000000000004}]},{"osd":4,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.77800000000000002,"15min":0.79400000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.008,"5min":1.1399999999999999,"15min":1.256},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.83299999999999996,"15min":0.85999999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.52200000000000002},"max":{"1min":1.016,"5min":1.22,"15min":1.2669999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.874,"15min":0.88100000000000001},"min":{"1min":0.64500000000000002,"5min":0.58699999999999997,"15min":0.47399999999999998},"max":{"1min":1.1479999999999999,"5min":1.276,"15min":1.276},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"max":{"1min":1.111,"5min":1.1799999999999999,"15min":1.2649999999999999},"last":0.94299999999999995}]},{"osd":6,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.59899999999999998},"max":{"1min":1.1200000000000001,"5min":1.2290000000000001,"15min":1.4570000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95499999999999996,"15min":0.95899999999999996},"min":{"1min":0.70799999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2010000000000001,"5min":1.3680000000000001,"15min":1.546},"last":0.79500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575699,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":4010,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4106826,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76100000000000001,"15min":0.79700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0980000000000001,"5min":1.1399999999999999,"15min":1.1639999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80800000000000005,"15min":0.82299999999999995},"min":{"1min":0.65900000000000003,"5min":0.59499999999999997,"15min":0.54500000000000004},"max":{"1min":1.016,"5min":1.101,"15min":1.1870000000000001},"last":0.752}]},{"osd":1,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.629,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.0409999999999999,"5min":1.085,"15min":1.1399999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.56999999999999995},"max":{"1min":1.075,"5min":1.177,"15min":1.177},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89400000000000002,"15min":0.88800000000000001},"min":{"1min":0.72299999999999998,"5min":0.57799999999999996,"15min":0.56200000000000006},"max":{"1min":1.1439999999999999,"5min":1.153,"15min":1.2490000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.93899999999999995,"15min":0.94799999999999995},"min":{"1min":0.76800000000000002,"5min":0.624,"15min":0.59399999999999997},"max":{"1min":1.097,"5min":1.1799999999999999,"15min":1.2549999999999999},"last":0.82699999999999996}]},{"osd":3,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.89400000000000002,"15min":0.88100000000000001},"min":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1819999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.65900000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1339999999999999,"5min":1.1830000000000001,"15min":1.2250000000000001},"last":0.98299999999999998}]},{"osd":4,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.52400000000000002},"max":{"1min":1.1259999999999999,"5min":1.1399999999999999,"15min":1.236},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.86599999999999999,"15min":0.85899999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.57299999999999995},"max":{"1min":1.0720000000000001,"5min":1.173,"15min":1.2629999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90800000000000003,"15min":0.89800000000000002},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.61299999999999999},"max":{"1min":1.206,"5min":1.2230000000000001,"15min":1.3009999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88400000000000001,"15min":0.89700000000000002},"min":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.57999999999999996},"max":{"1min":1.159,"5min":1.206,"15min":1.4490000000000001},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.95399999999999996,"15min":0.94899999999999995},"min":{"1min":0.67300000000000004,"5min":0.66000000000000003,"15min":0.60599999999999998},"max":{"1min":1.133,"5min":1.3069999999999999,"15min":1.329},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.97999999999999998,"15min":0.98499999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.2609999999999999,"5min":1.407,"15min":1.4199999999999999},"last":0.81299999999999994}]}]},{"osd":5,"up_from":13,"seq":55834575698,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":2756,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2822493,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81399999999999995,"15min":0.81499999999999995},"min":{"1min":0.55400000000000005,"5min":0.54500000000000004,"15min":0.505},"max":{"1min":1.006,"5min":1.0229999999999999,"15min":1.113},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92100000000000004,"15min":0.91400000000000003},"min":{"1min":0.73299999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.169},"last":1.042}]},{"osd":1,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90400000000000003,"15min":0.89400000000000002},"min":{"1min":0.65200000000000002,"5min":0.64000000000000001,"15min":0.57299999999999995},"max":{"1min":1.1719999999999999,"5min":1.1899999999999999,"15min":1.1970000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86399999999999999,"15min":0.87},"min":{"1min":0.77900000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.96799999999999997,"5min":1.091,"15min":1.1220000000000001},"last":0.85999999999999999}]},{"osd":2,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.59899999999999998,"5min":0.56499999999999995,"15min":0.49199999999999999},"max":{"1min":0.999,"5min":1.1950000000000001,"15min":1.7809999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.83299999999999996,"15min":0.83999999999999997},"min":{"1min":0.68400000000000005,"5min":0.61599999999999999,"15min":0.59099999999999997},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.198},"last":0.89600000000000002}]},{"osd":3,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.874,"15min":0.872},"min":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61699999999999999},"max":{"1min":1.091,"5min":1.1419999999999999,"15min":1.2130000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.89000000000000001,"15min":0.88200000000000001},"min":{"1min":0.66800000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.131,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.1220000000000001}]},{"osd":4,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.753,"15min":0.745},"min":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.44800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0669999999999999,"15min":1.079},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.53700000000000003},"max":{"1min":1.198,"5min":1.234,"15min":1.3149999999999999},"last":1.1379999999999999}]},{"osd":6,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89900000000000002},"min":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.53600000000000003},"max":{"1min":1.089,"5min":1.1259999999999999,"15min":1.26},"last":1.079},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.56200000000000006},"max":{"1min":1.1060000000000001,"5min":1.1539999999999999,"15min":1.196},"last":0.98899999999999999}]},{"osd":7,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.96299999999999997,"15min":0.93600000000000005},"min":{"1min":0.752,"5min":0.63600000000000001,"15min":0.58199999999999996},"max":{"1min":1.246,"5min":1.464,"15min":1.464},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97599999999999998,"15min":0.96399999999999997},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.56999999999999995},"max":{"1min":1.335,"5min":1.335,"15min":1.413},"last":1.159}]}]},{"osd":4,"up_from":13,"seq":55834575699,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798064,"kb_used_data":92901316,"kb_used_omap":3556,"kb_used_meta":0,"kb_avail":92901316,"statfs":{"total":95948165120,"available":95130947584,"internally_reserved":0,"allocated":95130947584,"data_stored":95130947584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3641905,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.024,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.83599999999999997,"15min":0.82699999999999996},"min":{"1min":0.67400000000000004,"5min":0.51300000000000001,"15min":0.46200000000000002},"max":{"1min":1.048,"5min":1.1519999999999999,"15min":1.26},"last":1.0549999999999999}]},{"osd":1,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81899999999999995,"15min":0.83599999999999997},"min":{"1min":0.61399999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.111,"5min":1.1930000000000001,"15min":1.206},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.65500000000000003,"5min":0.61699999999999999,"15min":0.58599999999999997},"max":{"1min":0.95199999999999996,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.97699999999999998}]},{"osd":2,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.65000000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.042,"5min":1.151,"15min":1.151},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.872,"15min":0.872},"min":{"1min":0.75800000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.159},"last":0.89800000000000002}]},{"osd":3,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.63700000000000001},"max":{"1min":1.0049999999999999,"5min":1.0640000000000001,"15min":1.1639999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.88300000000000001,"15min":0.89300000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.026,"5min":1.131,"15min":1.1970000000000001},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.876},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.54700000000000004},"max":{"1min":1.147,"5min":1.1579999999999999,"15min":1.24},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92300000000000004,"15min":0.93400000000000005},"min":{"1min":0.72899999999999998,"5min":0.59299999999999997,"15min":0.53800000000000003},"max":{"1min":1.0629999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.1100000000000001}]},{"osd":6,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.878,"15min":0.86499999999999999},"min":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54600000000000004},"max":{"1min":1.1479999999999999,"5min":1.2649999999999999,"15min":1.3400000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93700000000000006,"15min":0.92900000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61299999999999999},"max":{"1min":1.1479999999999999,"5min":1.226,"15min":1.226},"last":1.1439999999999999}]},{"osd":7,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.90200000000000002,"15min":0.89100000000000001},"min":{"1min":0.64700000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2350000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.97199999999999998,"15min":0.96399999999999997},"min":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.139,"5min":1.3240000000000001,"15min":1.3859999999999999},"last":1.1639999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575698,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3358,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3439467,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.46800000000000003,"5min":0.438,"15min":0.433},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.079},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.65600000000000003,"5min":0.58399999999999996,"15min":0.438},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.1439999999999999},"last":0.68899999999999995}]},{"osd":1,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47799999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.0820000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.60999999999999999,"5min":0.53700000000000003,"15min":0.51400000000000001},"max":{"1min":0.98499999999999999,"5min":1.0329999999999999,"15min":1.0860000000000001},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.5},"max":{"1min":0.85299999999999998,"5min":0.93200000000000005,"15min":1.0609999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.52100000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1559999999999999},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.88800000000000001,"15min":0.88300000000000001},"min":{"1min":0.76700000000000002,"5min":0.65700000000000003,"15min":0.61899999999999999},"max":{"1min":1.034,"5min":1.127,"15min":1.1639999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.89800000000000002,"15min":0.91000000000000003},"min":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.68200000000000005},"max":{"1min":1.0620000000000001,"5min":1.1240000000000001,"15min":1.1890000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.89900000000000002,"15min":0.90800000000000003},"min":{"1min":0.77500000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.153},"last":1.004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90400000000000003,"15min":0.92700000000000005},"min":{"1min":0.73299999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.143,"5min":1.2030000000000001,"15min":1.349},"last":0.91200000000000003}]},{"osd":6,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.92700000000000005,"15min":0.94699999999999995},"min":{"1min":0.70599999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1160000000000001,"5min":1.2589999999999999,"15min":1.4159999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.96899999999999997,"15min":0.96999999999999997},"min":{"1min":0.79100000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3799999999999999},"last":1.046}]},{"osd":7,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.83699999999999997,"5min":0.746,"15min":0.74099999999999999},"max":{"1min":1.2649999999999999,"5min":1.3100000000000001,"15min":1.3280000000000001},"last":1.125},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.012,"15min":1.018},"min":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.079}]}]},{"osd":2,"up_from":13,"seq":55834575699,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":2831,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2899071,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.98399999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68600000000000005,"15min":0.69799999999999995},"min":{"1min":0.54300000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.026},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.503,"5min":0.48699999999999999,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.012,"15min":1.04},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.433},"max":{"1min":0.89300000000000002,"5min":1.046,"15min":1.077},"last":0.73099999999999998}]},{"osd":3,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.50900000000000001,"5min":0.46999999999999997,"15min":0.43099999999999999},"max":{"1min":0.92900000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.48899999999999999},"max":{"1min":0.96599999999999997,"5min":1.052,"15min":1.0640000000000001},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.76600000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.0649999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.877,"15min":0.88200000000000001},"min":{"1min":0.71999999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.083,"5min":1.101,"15min":1.2050000000000001},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.93600000000000005,"15min":0.93300000000000005},"min":{"1min":0.77300000000000002,"5min":0.61899999999999999,"15min":0.59499999999999997},"max":{"1min":1.0960000000000001,"5min":1.202,"15min":1.202},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.91800000000000004},"min":{"1min":0.78900000000000003,"5min":0.69099999999999995,"15min":0.58499999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.387},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.85499999999999998,"15min":0.85899999999999999},"min":{"1min":0.71399999999999997,"5min":0.65000000000000002,"15min":0.64400000000000002},"max":{"1min":1.0549999999999999,"5min":1.1739999999999999,"15min":1.2989999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.998,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.78500000000000003,"5min":0.72599999999999998,"15min":0.68799999999999994},"max":{"1min":1.2310000000000001,"5min":1.3919999999999999,"15min":1.571},"last":0.73899999999999999}]},{"osd":7,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.95199999999999996,"15min":0.95999999999999996},"min":{"1min":0.76700000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.2829999999999999,"5min":1.4299999999999999,"15min":1.5660000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.024,"5min":0.95799999999999996,"15min":0.95099999999999996},"min":{"1min":0.81299999999999994,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.335,"5min":1.381,"15min":1.4299999999999999},"last":0.87}]}]},{"osd":0,"up_from":13,"seq":55834575699,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3801,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3893157,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.47399999999999998,"5min":0.443,"15min":0.442},"max":{"1min":0.92500000000000004,"5min":0.93899999999999995,"15min":1.089},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.55200000000000005,"5min":0.48499999999999999,"15min":0.45400000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.085},"last":0.78300000000000003}]},{"osd":2,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69699999999999995,"15min":0.68000000000000005},"min":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.46300000000000002},"max":{"1min":0.89000000000000001,"5min":1.048,"15min":1.048},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.59099999999999997,"5min":0.52200000000000002,"15min":0.5},"max":{"1min":0.92900000000000005,"5min":1.0469999999999999,"15min":1.0820000000000001},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.53500000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.84099999999999997,"5min":1.155,"15min":1.155},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.85699999999999998,"15min":0.84299999999999997},"min":{"1min":0.65200000000000002,"5min":0.63500000000000001,"15min":0.57199999999999995},"max":{"1min":1.0109999999999999,"5min":1.167,"15min":1.167},"last":0.84199999999999997}]},{"osd":4,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89000000000000001,"15min":0.89200000000000002},"min":{"1min":0.73099999999999998,"5min":0.66100000000000003,"15min":0.60099999999999998},"max":{"1min":1.1599999999999999,"5min":1.282,"15min":1.3600000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88400000000000001,"15min":0.877},"min":{"1min":0.70699999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"max":{"1min":1.226,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94799999999999995,"15min":0.93400000000000005},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.66000000000000003},"max":{"1min":1.3779999999999999,"5min":1.474,"15min":1.474},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.68200000000000005},"max":{"1min":1.153,"5min":1.55,"15min":1.55},"last":0.92700000000000005}]},{"osd":6,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.96299999999999997,"15min":0.93899999999999995},"min":{"1min":0.78500000000000003,"5min":0.72699999999999998,"15min":0.64700000000000002},"max":{"1min":1.298,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.999,"5min":1.03,"15min":1.0049999999999999},"min":{"1min":0.877,"5min":0.78400000000000003,"15min":0.70699999999999996},"max":{"1min":1.3260000000000001,"5min":1.637,"15min":1.637},"last":0.95399999999999996}]},{"osd":7,"last update":"Thu Jul 21 02:58:13 2022","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0289999999999999,"15min":1.018},"min":{"1min":0.90300000000000002,"5min":0.72299999999999998,"15min":0.71799999999999997},"max":{"1min":1.2110000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.036,"15min":1.02},"min":{"1min":0.879,"5min":0.751,"15min":0.72599999999999998},"max":{"1min":1.353,"5min":1.3879999999999999,"15min":1.397},"last":0.81899999999999995}]}]}]}} 2022-07-21T02:59:01.845 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2022-07-21T01:48:17.950031+0000 time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=1, tm_min=48, tm_sec=17, tm_wday=3, tm_yday=202, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=58, tm_sec=57, tm_wday=3, tm_yday=202, tm_isdst=0) 2022-07-21T02:59:01.846 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2022-07-21T01:48:26.950679+0000 time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=1, tm_min=48, tm_sec=26, tm_wday=3, tm_yday=202, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=58, tm_sec=57, tm_wday=3, tm_yday=202, tm_isdst=0) 2022-07-21T02:59:01.846 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2022-07-21T01:48:26.950679+0000 time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=1, tm_min=48, tm_sec=26, tm_wday=3, tm_yday=202, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=58, tm_sec=57, tm_wday=3, tm_yday=202, tm_isdst=0) 2022-07-21T02:59:01.847 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2022-07-21T01:48:26.950679+0000 time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=1, tm_min=48, tm_sec=26, tm_wday=3, tm_yday=202, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=58, tm_sec=57, tm_wday=3, tm_yday=202, tm_isdst=0) 2022-07-21T02:59:01.848 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2022-07-21T01:48:26.950679+0000 time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=1, tm_min=48, tm_sec=26, tm_wday=3, tm_yday=202, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=58, tm_sec=57, tm_wday=3, tm_yday=202, tm_isdst=0) 2022-07-21T02:59:01.849 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2022-07-21T01:48:26.950679+0000 time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=1, tm_min=48, tm_sec=26, tm_wday=3, tm_yday=202, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=58, tm_sec=57, tm_wday=3, tm_yday=202, tm_isdst=0) 2022-07-21T02:59:01.849 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2022-07-21T01:48:26.950679+0000 time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=1, tm_min=48, tm_sec=26, tm_wday=3, tm_yday=202, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=58, tm_sec=57, tm_wday=3, tm_yday=202, tm_isdst=0) 2022-07-21T02:59:01.850 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2022-07-21T01:48:26.950679+0000 time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=1, tm_min=48, tm_sec=26, tm_wday=3, tm_yday=202, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=58, tm_sec=57, tm_wday=3, tm_yday=202, tm_isdst=0) 2022-07-21T02:59:01.851 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2022-07-21T01:48:26.950679+0000 time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=1, tm_min=48, tm_sec=26, tm_wday=3, tm_yday=202, tm_isdst=-1) <= time.struct_time(tm_year=2022, tm_mon=7, tm_mday=21, tm_hour=2, tm_min=58, tm_sec=57, tm_wday=3, tm_yday=202, tm_isdst=0) 2022-07-21T02:59:01.851 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2022-07-21T02:59:16.608 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:16.613 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:16.617 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:16.620 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:16.623 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:21.853 DEBUG:teuthology.orchestra.run.smithi140:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2022-07-21T02:59:21.966 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.962+0000 7ff4e16ca700 1 Processor -- start 2022-07-21T02:59:21.967 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.962+0000 7ff4e16ca700 1 -- start start 2022-07-21T02:59:21.972 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4e16ca700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff4dc0ac3a0 0x7ff4dc143690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:21.973 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4e16ca700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4dc060980 con 0x7ff4dc0b1ef0 2022-07-21T02:59:21.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4e16ca700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4dc0b22d0 con 0x7ff4dc05fa50 2022-07-21T02:59:21.975 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4e16ca700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4dc098810 con 0x7ff4dc0ac3a0 2022-07-21T02:59:21.976 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4dbfff700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7ff4dc0b1ef0 0x7ff4dc142f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56686/0 (socket says 172.21.15.140:56686) 2022-07-21T02:59:21.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4dbfff700 1 -- 172.21.15.140:0/2123316374 learned_addr learned my addr 172.21.15.140:0/2123316374 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:59:21.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4d37fe700 1 --2- 172.21.15.140:0/2123316374 >> v2:172.21.15.152:3300/0 conn(0x7ff4dc0ac3a0 0x7ff4dc143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:21.977 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4db7fe700 1 -- 172.21.15.140:0/2123316374 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981555080 0 0) 0x7ff4dc060980 con 0x7ff4dc0b1ef0 2022-07-21T02:59:21.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4db7fe700 1 -- 172.21.15.140:0/2123316374 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4bc003260 con 0x7ff4dc0b1ef0 2022-07-21T02:59:21.978 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4db7fe700 1 -- 172.21.15.140:0/2123316374 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206999131 0 0) 0x7ff4dc0b22d0 con 0x7ff4dc05fa50 2022-07-21T02:59:21.979 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4db7fe700 1 -- 172.21.15.140:0/2123316374 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4dc060980 con 0x7ff4dc05fa50 2022-07-21T02:59:21.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4d37fe700 1 -- 172.21.15.140:0/2123316374 >> v1:172.21.15.152:6789/0 conn(0x7ff4dc05fa50 legacy=0x7ff4dc13e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:21.980 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4db7fe700 1 -- 172.21.15.140:0/2123316374 <== mon.1 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2048297016 0 0) 0x7ff4dc060980 con 0x7ff4dc05fa50 2022-07-21T02:59:21.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4d37fe700 1 -- 172.21.15.140:0/2123316374 >> v1:172.21.15.140:6789/0 conn(0x7ff4dc0b1ef0 legacy=0x7ff4dc142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:21.981 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4d37fe700 1 -- 172.21.15.140:0/2123316374 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc060bb0 con 0x7ff4dc0ac3a0 2022-07-21T02:59:21.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4d37fe700 1 --2- 172.21.15.140:0/2123316374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4dc0ac3a0 0x7ff4dc143690 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7ff4d4001490 tx=0x7ff4c4006df0).ready entity=mon.1 client_cookie=909786a655b67f7d server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:21.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4db7fe700 1 -- 172.21.15.140:0/2123316374 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4d40019e0 con 0x7ff4dc0ac3a0 2022-07-21T02:59:21.982 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4db7fe700 1 -- 172.21.15.140:0/2123316374 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c4000e90 con 0x7ff4dc0ac3a0 2022-07-21T02:59:21.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.966+0000 7ff4db7fe700 1 -- 172.21.15.140:0/2123316374 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4c4012460 con 0x7ff4dc0ac3a0 2022-07-21T02:59:21.983 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.970+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2123316374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4dc0ac3a0 msgr2=0x7ff4dc143690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:21.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.970+0000 7ff4e16ca700 1 --2- 172.21.15.140:0/2123316374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4dc0ac3a0 0x7ff4dc143690 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7ff4d4001490 tx=0x7ff4c4006df0).stop 2022-07-21T02:59:21.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.970+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2123316374 shutdown_connections 2022-07-21T02:59:21.984 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.970+0000 7ff4e16ca700 1 --2- 172.21.15.140:0/2123316374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4dc0ac3a0 0x7ff4dc143690 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:21.985 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.970+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2123316374 >> 172.21.15.140:0/2123316374 conn(0x7ff4dc0b2ca0 msgr2=0x7ff4dc0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:21.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.970+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2123316374 shutdown_connections 2022-07-21T02:59:21.986 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.970+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2123316374 wait complete. 2022-07-21T02:59:21.990 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.970+0000 7ff4e16ca700 1 Processor -- start 2022-07-21T02:59:21.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.970+0000 7ff4e16ca700 1 -- start start 2022-07-21T02:59:21.991 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4e16ca700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4dc05fa50 0x7ff4dc13c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:21.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4e16ca700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4dc0ac3a0 0x7ff4dc097c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:21.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4e16ca700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4dc0b1ef0 0x7ff4dc098180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:21.992 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4e16ca700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc13a140 con 0x7ff4dc0ac3a0 2022-07-21T02:59:21.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4e16ca700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc098670 con 0x7ff4dc05fa50 2022-07-21T02:59:21.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4e16ca700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc13cbb0 con 0x7ff4dc0b1ef0 2022-07-21T02:59:21.993 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4dc0ac3a0 0x7ff4dc097c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:21.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d37fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4dc0ac3a0 0x7ff4dc097c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42546/0 (socket says 172.21.15.140:42546) 2022-07-21T02:59:21.994 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d37fe700 1 -- 172.21.15.140:0/2834217056 learned_addr learned my addr 172.21.15.140:0/2834217056 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:21.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4e0ec9700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4dc0b1ef0 0x7ff4dc098180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:21.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4dbfff700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4dc05fa50 0x7ff4dc13c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:21.995 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d37fe700 1 -- 172.21.15.140:0/2834217056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4dc0b1ef0 msgr2=0x7ff4dc098180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:21.996 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d37fe700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4dc0b1ef0 0x7ff4dc098180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:21.997 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d37fe700 1 -- 172.21.15.140:0/2834217056 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4dc05fa50 msgr2=0x7ff4dc13c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:21.998 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d37fe700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4dc05fa50 0x7ff4dc13c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:21.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d37fe700 1 -- 172.21.15.140:0/2834217056 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c400b180 con 0x7ff4dc0ac3a0 2022-07-21T02:59:21.999 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4dbfff700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4dc05fa50 0x7ff4dc13c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-07-21T02:59:22.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d37fe700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4dc0ac3a0 0x7ff4dc097c90 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7ff4d4001490 tx=0x7ff4c4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:22.001 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d9ffb700 1 -- 172.21.15.140:0/2834217056 <== mon.0 v2:172.21.15.140:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4c400d8d0 con 0x7ff4dc0ac3a0 2022-07-21T02:59:22.002 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d9ffb700 1 -- 172.21.15.140:0/2834217056 <== mon.0 v2:172.21.15.140:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c400da30 con 0x7ff4dc0ac3a0 2022-07-21T02:59:22.003 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2834217056 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4dc13ce20 con 0x7ff4dc0ac3a0 2022-07-21T02:59:22.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d9ffb700 1 -- 172.21.15.140:0/2834217056 <== mon.0 v2:172.21.15.140:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4c400dd90 con 0x7ff4dc0ac3a0 2022-07-21T02:59:22.004 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2834217056 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc153230 con 0x7ff4dc0ac3a0 2022-07-21T02:59:22.005 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4d9ffb700 1 -- 172.21.15.140:0/2834217056 <== mon.0 v2:172.21.15.140:3300/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7ff4c4025030 con 0x7ff4dc0ac3a0 2022-07-21T02:59:22.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.974+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2834217056 --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4dc0ae9b0 con 0x7ff4dc0ac3a0 2022-07-21T02:59:22.006 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.978+0000 7ff4d9ffb700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4c0053b30 0x7ff4c0055ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:22.007 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.978+0000 7ff4dbfff700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4c0053b30 0x7ff4c0055ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:22.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.978+0000 7ff4d9ffb700 1 -- 172.21.15.140:0/2834217056 <== mon.0 v2:172.21.15.140:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7ff4c406c960 con 0x7ff4dc0ac3a0 2022-07-21T02:59:22.008 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.978+0000 7ff4dbfff700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4c0053b30 0x7ff4c0055ff0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7ff4cc008000 tx=0x7ff4cc009040).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:22.009 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:21.986+0000 7ff4d9ffb700 1 -- 172.21.15.140:0/2834217056 <== mon.0 v2:172.21.15.140:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7ff4c4043a50 con 0x7ff4dc0ac3a0 2022-07-21T02:59:22.107 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.106+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2834217056 --> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4dc13cfb0 con 0x7ff4c0053b30 2022-07-21T02:59:22.110 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.106+0000 7ff4d9ffb700 1 -- 172.21.15.140:0/2834217056 <== mgr.4104 v2:172.21.15.140:6800/22519 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68450 (secure 0 0 0) 0x7ff4cc007110 con 0x7ff4c0053b30 2022-07-21T02:59:22.112 INFO:teuthology.orchestra.run.smithi140.stdout: 2022-07-21T02:59:22.113 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2834217056 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4c0053b30 msgr2=0x7ff4c0055ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:22.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4c0053b30 0x7ff4c0055ff0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7ff4cc008000 tx=0x7ff4cc009040).stop 2022-07-21T02:59:22.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2834217056 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4dc0ac3a0 msgr2=0x7ff4dc097c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:22.114 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4dc0ac3a0 0x7ff4dc097c90 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7ff4d4001490 tx=0x7ff4c4014000).stop 2022-07-21T02:59:22.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2834217056 shutdown_connections 2022-07-21T02:59:22.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7ff4c0053b30 0x7ff4c0055ff0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:22.115 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff4dc0b1ef0 0x7ff4dc098180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:22.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7ff4dc0ac3a0 0x7ff4dc097c90 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:22.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 --2- 172.21.15.140:0/2834217056 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff4dc05fa50 0x7ff4dc13c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:22.116 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2834217056 >> 172.21.15.140:0/2834217056 conn(0x7ff4dc0b2ca0 msgr2=0x7ff4dc05ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:22.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2834217056 shutdown_connections 2022-07-21T02:59:22.117 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.110+0000 7ff4e16ca700 1 -- 172.21.15.140:0/2834217056 wait complete. 2022-07-21T02:59:22.118 INFO:teuthology.orchestra.run.smithi140.stderr:dumped all 2022-07-21T02:59:22.124 INFO:teuthology.orchestra.run.smithi140.stdout:{"pg_ready":true,"pg_map":{"version":2278,"stamp":"2022-07-21T02:59:21.067587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9703,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":6385408,"kb_used_data":743209632,"kb_used_omap":27991,"kb_used_meta":0,"kb_avail":743209632,"statfs":{"total":767585320960,"available":761046663168,"internally_reserved":0,"allocated":761046663168,"data_stored":761046663168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28663005,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003841"},"pg_stats":[{"pgid":"1.0","version":"154'4","reported_seq":142,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-07-21T02:59:03.158199+0000","last_change":"2022-07-21T02:59:03.158306+0000","last_active":"2022-07-21T02:59:03.158199+0000","last_peered":"2022-07-21T02:59:03.158199+0000","last_clean":"2022-07-21T02:59:03.158199+0000","last_became_active":"2022-07-21T01:48:22.240745+0000","last_became_peered":"2022-07-21T01:48:22.240745+0000","last_unstale":"2022-07-21T02:59:03.158199+0000","last_undegraded":"2022-07-21T02:59:03.158199+0000","last_fullsized":"2022-07-21T02:59:03.158199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'4","last_scrub_stamp":"2022-07-21T02:59:03.158198+0000","last_deep_scrub":"154'4","last_deep_scrub_stamp":"2022-07-21T02:59:03.158198+0000","last_clean_scrub_stamp":"2022-07-21T02:59:03.158198+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9703,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-07-21T02:59:04.590518+0000","last_change":"2022-07-21T02:59:04.590655+0000","last_active":"2022-07-21T02:59:04.590518+0000","last_peered":"2022-07-21T02:59:04.590518+0000","last_clean":"2022-07-21T02:59:04.590518+0000","last_became_active":"2022-07-21T01:48:27.973977+0000","last_became_peered":"2022-07-21T01:48:27.973977+0000","last_unstale":"2022-07-21T02:59:04.590518+0000","last_undegraded":"2022-07-21T02:59:04.590518+0000","last_fullsized":"2022-07-21T02:59:04.590518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T02:59:04.590517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T02:59:04.590517+0000","last_clean_scrub_stamp":"2022-07-21T02:59:04.590517+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-07-21T02:59:04.197787+0000","last_change":"2022-07-21T02:59:04.197889+0000","last_active":"2022-07-21T02:59:04.197787+0000","last_peered":"2022-07-21T02:59:04.197787+0000","last_clean":"2022-07-21T02:59:04.197787+0000","last_became_active":"2022-07-21T01:48:27.973957+0000","last_became_peered":"2022-07-21T01:48:27.973957+0000","last_unstale":"2022-07-21T02:59:04.197787+0000","last_undegraded":"2022-07-21T02:59:04.197787+0000","last_fullsized":"2022-07-21T02:59:04.197787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T02:59:04.197786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T02:59:04.197786+0000","last_clean_scrub_stamp":"2022-07-21T02:59:04.197786+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":144,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-07-21T02:58:59.774062+0000","last_change":"2022-07-21T02:58:59.774147+0000","last_active":"2022-07-21T02:58:59.774062+0000","last_peered":"2022-07-21T02:58:59.774062+0000","last_clean":"2022-07-21T02:58:59.774062+0000","last_became_active":"2022-07-21T01:48:27.967411+0000","last_became_peered":"2022-07-21T01:48:27.967411+0000","last_unstale":"2022-07-21T02:58:59.774062+0000","last_undegraded":"2022-07-21T02:58:59.774062+0000","last_fullsized":"2022-07-21T02:58:59.774062+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T02:58:59.774061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T02:58:59.774061+0000","last_clean_scrub_stamp":"2022-07-21T02:58:59.774061+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":140,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-07-21T02:59:01.522166+0000","last_change":"2022-07-21T02:59:01.522262+0000","last_active":"2022-07-21T02:59:01.522166+0000","last_peered":"2022-07-21T02:59:01.522166+0000","last_clean":"2022-07-21T02:59:01.522166+0000","last_became_active":"2022-07-21T01:48:27.973744+0000","last_became_peered":"2022-07-21T01:48:27.973744+0000","last_unstale":"2022-07-21T02:59:01.522166+0000","last_undegraded":"2022-07-21T02:59:01.522166+0000","last_fullsized":"2022-07-21T02:59:01.522166+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T02:59:01.522165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T02:59:01.522165+0000","last_clean_scrub_stamp":"2022-07-21T02:59:01.522165+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":139,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-07-21T02:59:00.359493+0000","last_change":"2022-07-21T02:59:00.359584+0000","last_active":"2022-07-21T02:59:00.359493+0000","last_peered":"2022-07-21T02:59:00.359493+0000","last_clean":"2022-07-21T02:59:00.359493+0000","last_became_active":"2022-07-21T01:48:27.992216+0000","last_became_peered":"2022-07-21T01:48:27.992216+0000","last_unstale":"2022-07-21T02:59:00.359493+0000","last_undegraded":"2022-07-21T02:59:00.359493+0000","last_fullsized":"2022-07-21T02:59:00.359493+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T02:59:00.359492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T02:59:00.359492+0000","last_clean_scrub_stamp":"2022-07-21T02:59:00.359492+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-07-21T02:59:09.175530+0000","last_change":"2022-07-21T02:59:09.175672+0000","last_active":"2022-07-21T02:59:09.175530+0000","last_peered":"2022-07-21T02:59:09.175530+0000","last_clean":"2022-07-21T02:59:09.175530+0000","last_became_active":"2022-07-21T01:48:27.996456+0000","last_became_peered":"2022-07-21T01:48:27.996456+0000","last_unstale":"2022-07-21T02:59:09.175530+0000","last_undegraded":"2022-07-21T02:59:09.175530+0000","last_fullsized":"2022-07-21T02:59:09.175530+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T02:59:09.175529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T02:59:09.175529+0000","last_clean_scrub_stamp":"2022-07-21T02:59:09.175529+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":139,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-07-21T02:59:02.363408+0000","last_change":"2022-07-21T02:59:02.363505+0000","last_active":"2022-07-21T02:59:02.363408+0000","last_peered":"2022-07-21T02:59:02.363408+0000","last_clean":"2022-07-21T02:59:02.363408+0000","last_became_active":"2022-07-21T01:48:27.970498+0000","last_became_peered":"2022-07-21T01:48:27.970498+0000","last_unstale":"2022-07-21T02:59:02.363408+0000","last_undegraded":"2022-07-21T02:59:02.363408+0000","last_fullsized":"2022-07-21T02:59:02.363408+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T02:59:02.363407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T02:59:02.363407+0000","last_clean_scrub_stamp":"2022-07-21T02:59:02.363407+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":154,"state":"active+clean","last_fresh":"2022-07-21T02:59:02.954963+0000","last_change":"2022-07-21T02:59:02.955062+0000","last_active":"2022-07-21T02:59:02.954963+0000","last_peered":"2022-07-21T02:59:02.954963+0000","last_clean":"2022-07-21T02:59:02.954963+0000","last_became_active":"2022-07-21T01:48:27.968884+0000","last_became_peered":"2022-07-21T01:48:27.968884+0000","last_unstale":"2022-07-21T02:59:02.954963+0000","last_undegraded":"2022-07-21T02:59:02.954963+0000","last_fullsized":"2022-07-21T02:59:02.954963+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2022-07-21T02:59:02.954962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2022-07-21T02:59:02.954962+0000","last_clean_scrub_stamp":"2022-07-21T02:59:02.954962+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9703,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":1,"up_from":13,"seq":55834575702,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3979,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4074551,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.623,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.41799999999999998},"max":{"1min":0.83699999999999997,"5min":0.96199999999999997,"15min":1.131},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.59699999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95699999999999996,"5min":0.96899999999999997,"15min":1.018},"last":0.749}]},{"osd":2,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.622},"min":{"1min":0.52200000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.79300000000000004,"5min":0.83399999999999996,"15min":1.069},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.53600000000000003,"5min":0.501,"15min":0.496},"max":{"1min":0.90600000000000003,"5min":1.105,"15min":1.3779999999999999},"last":0.84099999999999997}]},{"osd":3,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.52300000000000002},"max":{"1min":0.93200000000000005,"5min":1.006,"15min":1.3240000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93000000000000005,"5min":0.98099999999999998,"15min":1.2290000000000001},"last":0.626}]},{"osd":4,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.90500000000000003,"15min":0.89000000000000001},"min":{"1min":0.85399999999999998,"5min":0.67100000000000004,"15min":0.63300000000000001},"max":{"1min":1.179,"5min":1.179,"15min":1.583},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.86899999999999999,"15min":0.876},"min":{"1min":0.63900000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.008,"5min":1.0960000000000001,"15min":1.335},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.90600000000000003,"15min":0.89900000000000002},"min":{"1min":0.79500000000000004,"5min":0.63800000000000001,"15min":0.59699999999999998},"max":{"1min":1.0960000000000001,"5min":1.2010000000000001,"15min":1.4970000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.999,"15min":0.98599999999999999},"min":{"1min":0.872,"5min":0.73899999999999999,"15min":0.56100000000000005},"max":{"1min":1.147,"5min":1.3400000000000001,"15min":1.569},"last":0.997}]},{"osd":6,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.95299999999999996,"15min":0.95399999999999996},"min":{"1min":0.78900000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.054,"5min":1.2969999999999999,"15min":1.4099999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.94499999999999995,"15min":0.95799999999999996},"min":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.67700000000000005},"max":{"1min":1.1020000000000001,"5min":1.1930000000000001,"15min":1.548},"last":0.96699999999999997}]},{"osd":7,"last update":"Thu Jul 21 02:58:25 2022","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0269999999999999,"15min":1.014},"min":{"1min":0.79700000000000004,"5min":0.746,"15min":0.72899999999999998},"max":{"1min":1.1579999999999999,"5min":1.3879999999999999,"15min":1.613},"last":1.093},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0249999999999999,"15min":1.0229999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.69599999999999995},"max":{"1min":1.159,"5min":1.425,"15min":1.522},"last":1.0660000000000001}]}]},{"osd":7,"up_from":13,"seq":55834575701,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3683,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3771807,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.80800000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1679999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":1.28},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.80600000000000005,"15min":0.80900000000000005},"min":{"1min":0.58599999999999997,"5min":0.54400000000000004,"15min":0.52500000000000002},"max":{"1min":1.0389999999999999,"5min":1.085,"15min":1.181},"last":0.56899999999999995}]},{"osd":1,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.871,"15min":0.88400000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.54700000000000004},"max":{"1min":1.133,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.91300000000000003,"15min":0.91800000000000004},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.61099999999999999},"max":{"1min":1.0700000000000001,"5min":1.1140000000000001,"15min":1.2569999999999999},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.88100000000000001},"min":{"1min":0.79200000000000004,"5min":0.58799999999999997,"15min":0.51700000000000002},"max":{"1min":1.0369999999999999,"5min":1.119,"15min":1.153},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.92000000000000004,"15min":0.92300000000000004},"min":{"1min":0.79800000000000004,"5min":0.65700000000000003,"15min":0.59899999999999998},"max":{"1min":1.1100000000000001,"5min":1.1519999999999999,"15min":1.1879999999999999},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83799999999999997,"15min":0.84799999999999998},"min":{"1min":0.66100000000000003,"5min":0.65900000000000003,"15min":0.57999999999999996},"max":{"1min":1.069,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89800000000000002,"15min":0.88500000000000001},"min":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.083,"5min":1.238,"15min":1.238},"last":1.0289999999999999}]},{"osd":4,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.77800000000000002,"15min":0.79400000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.008,"5min":1.1399999999999999,"15min":1.256},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.83299999999999996,"15min":0.85999999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.52200000000000002},"max":{"1min":1.016,"5min":1.22,"15min":1.2669999999999999},"last":1.143}]},{"osd":5,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.874,"15min":0.88100000000000001},"min":{"1min":0.64500000000000002,"5min":0.58699999999999997,"15min":0.47399999999999998},"max":{"1min":1.1479999999999999,"5min":1.276,"15min":1.276},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.59499999999999997},"max":{"1min":1.111,"5min":1.1799999999999999,"15min":1.2649999999999999},"last":1.2270000000000001}]},{"osd":6,"last update":"Thu Jul 21 02:58:37 2022","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.96099999999999997,"15min":0.95399999999999996},"min":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.59899999999999998},"max":{"1min":1.1200000000000001,"5min":1.2290000000000001,"15min":1.4570000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.95499999999999996,"15min":0.95899999999999996},"min":{"1min":0.70799999999999996,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2010000000000001,"5min":1.3680000000000001,"15min":1.546},"last":1.006}]}]},{"osd":6,"up_from":13,"seq":55834575703,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":4012,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4109161,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76100000000000001,"15min":0.79700000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0980000000000001,"5min":1.1399999999999999,"15min":1.1639999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80800000000000005,"15min":0.82299999999999995},"min":{"1min":0.65900000000000003,"5min":0.59499999999999997,"15min":0.54500000000000004},"max":{"1min":1.016,"5min":1.101,"15min":1.1870000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.629,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.0409999999999999,"5min":1.085,"15min":1.1399999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.56999999999999995},"max":{"1min":1.075,"5min":1.177,"15min":1.177},"last":0.747}]},{"osd":2,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89400000000000002,"15min":0.88800000000000001},"min":{"1min":0.72299999999999998,"5min":0.57799999999999996,"15min":0.56200000000000006},"max":{"1min":1.1439999999999999,"5min":1.153,"15min":1.2490000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.93899999999999995,"15min":0.94799999999999995},"min":{"1min":0.76800000000000002,"5min":0.624,"15min":0.59399999999999997},"max":{"1min":1.097,"5min":1.1799999999999999,"15min":1.2549999999999999},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.89400000000000002,"15min":0.88100000000000001},"min":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1819999999999999},"last":0.877},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.65900000000000003,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1339999999999999,"5min":1.1830000000000001,"15min":1.2250000000000001},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.52400000000000002},"max":{"1min":1.1259999999999999,"5min":1.1399999999999999,"15min":1.236},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.86599999999999999,"15min":0.85899999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.57299999999999995},"max":{"1min":1.0720000000000001,"5min":1.173,"15min":1.2629999999999999},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.90800000000000003,"15min":0.89800000000000002},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.61299999999999999},"max":{"1min":1.206,"5min":1.2230000000000001,"15min":1.3009999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.88400000000000001,"15min":0.89700000000000002},"min":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.57999999999999996},"max":{"1min":1.159,"5min":1.206,"15min":1.4490000000000001},"last":1.046}]},{"osd":7,"last update":"Thu Jul 21 02:58:49 2022","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.95399999999999996,"15min":0.94899999999999995},"min":{"1min":0.67300000000000004,"5min":0.66000000000000003,"15min":0.60599999999999998},"max":{"1min":1.133,"5min":1.3069999999999999,"15min":1.329},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.97999999999999998,"15min":0.98499999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.2609999999999999,"5min":1.407,"15min":1.4199999999999999},"last":1.0209999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575702,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":2758,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2824805,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.81399999999999995,"15min":0.81499999999999995},"min":{"1min":0.55400000000000005,"5min":0.54500000000000004,"15min":0.505},"max":{"1min":1.006,"5min":1.0229999999999999,"15min":1.113},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.92100000000000004,"15min":0.91400000000000003},"min":{"1min":0.73299999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.169},"last":0.79000000000000004}]},{"osd":1,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90400000000000003,"15min":0.89400000000000002},"min":{"1min":0.65200000000000002,"5min":0.64000000000000001,"15min":0.57299999999999995},"max":{"1min":1.1719999999999999,"5min":1.1899999999999999,"15min":1.1970000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86399999999999999,"15min":0.87},"min":{"1min":0.77900000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.96799999999999997,"5min":1.091,"15min":1.1220000000000001},"last":0.92800000000000005}]},{"osd":2,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.59899999999999998,"5min":0.56499999999999995,"15min":0.49199999999999999},"max":{"1min":0.999,"5min":1.1950000000000001,"15min":1.7809999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.83299999999999996,"15min":0.83999999999999997},"min":{"1min":0.68400000000000005,"5min":0.61599999999999999,"15min":0.59099999999999997},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.198},"last":1.042}]},{"osd":3,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.874,"15min":0.872},"min":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61699999999999999},"max":{"1min":1.091,"5min":1.1419999999999999,"15min":1.2130000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.89000000000000001,"15min":0.88200000000000001},"min":{"1min":0.66800000000000004,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.131,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.753,"15min":0.745},"min":{"1min":0.52700000000000002,"5min":0.52000000000000002,"15min":0.44800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0669999999999999,"15min":1.079},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84599999999999997,"15min":0.86799999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.53700000000000003},"max":{"1min":1.198,"5min":1.234,"15min":1.3149999999999999},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.89900000000000002},"min":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.53600000000000003},"max":{"1min":1.089,"5min":1.1259999999999999,"15min":1.26},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.56200000000000006},"max":{"1min":1.1060000000000001,"5min":1.1539999999999999,"15min":1.196},"last":0.83199999999999996}]},{"osd":7,"last update":"Thu Jul 21 02:58:33 2022","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.96299999999999997,"15min":0.93600000000000005},"min":{"1min":0.752,"5min":0.63600000000000001,"15min":0.58199999999999996},"max":{"1min":1.246,"5min":1.464,"15min":1.464},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97599999999999998,"15min":0.96399999999999997},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.56999999999999995},"max":{"1min":1.335,"5min":1.335,"15min":1.413},"last":1.0049999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575703,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798064,"kb_used_data":92901316,"kb_used_omap":3556,"kb_used_meta":0,"kb_avail":92901316,"statfs":{"total":95948165120,"available":95130947584,"internally_reserved":0,"allocated":95130947584,"data_stored":95130947584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3641905,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.024,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.83599999999999997,"15min":0.82699999999999996},"min":{"1min":0.67400000000000004,"5min":0.51300000000000001,"15min":0.46200000000000002},"max":{"1min":1.048,"5min":1.1519999999999999,"15min":1.26},"last":0.69299999999999995}]},{"osd":1,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81899999999999995,"15min":0.83599999999999997},"min":{"1min":0.61399999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.111,"5min":1.1930000000000001,"15min":1.206},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.65500000000000003,"5min":0.61699999999999999,"15min":0.58599999999999997},"max":{"1min":0.95199999999999996,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.92600000000000005}]},{"osd":2,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.65000000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.042,"5min":1.151,"15min":1.151},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.872,"15min":0.872},"min":{"1min":0.75800000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.159},"last":0.88700000000000001}]},{"osd":3,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.63700000000000001},"max":{"1min":1.0049999999999999,"5min":1.0640000000000001,"15min":1.1639999999999999},"last":1.034},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.88300000000000001,"15min":0.89300000000000002},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.026,"5min":1.131,"15min":1.1970000000000001},"last":1.131}]},{"osd":5,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.876},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.54700000000000004},"max":{"1min":1.147,"5min":1.1579999999999999,"15min":1.24},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.92300000000000004,"15min":0.93400000000000005},"min":{"1min":0.72899999999999998,"5min":0.59299999999999997,"15min":0.53800000000000003},"max":{"1min":1.0629999999999999,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.1060000000000001}]},{"osd":6,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.878,"15min":0.86499999999999999},"min":{"1min":0.58799999999999997,"5min":0.54700000000000004,"15min":0.54600000000000004},"max":{"1min":1.1479999999999999,"5min":1.2649999999999999,"15min":1.3400000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93700000000000006,"15min":0.92900000000000005},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61299999999999999},"max":{"1min":1.1479999999999999,"5min":1.226,"15min":1.226},"last":0.96999999999999997}]},{"osd":7,"last update":"Thu Jul 21 02:58:41 2022","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.90200000000000002,"15min":0.89100000000000001},"min":{"1min":0.64700000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2350000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.163},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.97199999999999998,"15min":0.96399999999999997},"min":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.139,"5min":1.3240000000000001,"15min":1.3859999999999999},"last":1.0880000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3360,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3440703,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.46800000000000003,"5min":0.438,"15min":0.433},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.079},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.65600000000000003,"5min":0.58399999999999996,"15min":0.438},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.1439999999999999},"last":0.60399999999999998}]},{"osd":1,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47799999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.0820000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.60999999999999999,"5min":0.53700000000000003,"15min":0.51400000000000001},"max":{"1min":0.98499999999999999,"5min":1.0329999999999999,"15min":1.0860000000000001},"last":0.91100000000000003}]},{"osd":2,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.5},"max":{"1min":0.85299999999999998,"5min":0.93200000000000005,"15min":1.0609999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.53900000000000003,"5min":0.53400000000000003,"15min":0.52100000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1559999999999999},"last":0.93600000000000005}]},{"osd":4,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.88800000000000001,"15min":0.88300000000000001},"min":{"1min":0.76700000000000002,"5min":0.65700000000000003,"15min":0.61899999999999999},"max":{"1min":1.034,"5min":1.127,"15min":1.1639999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.89800000000000002,"15min":0.91000000000000003},"min":{"1min":0.71499999999999997,"5min":0.69799999999999995,"15min":0.68200000000000005},"max":{"1min":1.0620000000000001,"5min":1.1240000000000001,"15min":1.1890000000000001},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.89900000000000002,"15min":0.90800000000000003},"min":{"1min":0.77500000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.153},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90400000000000003,"15min":0.92700000000000005},"min":{"1min":0.73299999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.143,"5min":1.2030000000000001,"15min":1.349},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.92700000000000005,"15min":0.94699999999999995},"min":{"1min":0.70599999999999996,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.1160000000000001,"5min":1.2589999999999999,"15min":1.4159999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.96899999999999997,"15min":0.96999999999999997},"min":{"1min":0.79100000000000004,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3799999999999999},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.83699999999999997,"5min":0.746,"15min":0.74099999999999999},"max":{"1min":1.2649999999999999,"5min":1.3100000000000001,"15min":1.3280000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.012,"15min":1.018},"min":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.0149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575703,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":2833,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2901456,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60699999999999998,"15min":0.61099999999999999},"min":{"1min":0.46700000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.98399999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68600000000000005,"15min":0.69799999999999995},"min":{"1min":0.54300000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.026},"last":0.65800000000000003}]},{"osd":1,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.503,"5min":0.48699999999999999,"15min":0.47699999999999998},"max":{"1min":1.0009999999999999,"5min":1.012,"15min":1.04},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.754,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.433},"max":{"1min":0.89300000000000002,"5min":1.046,"15min":1.077},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.50900000000000001,"5min":0.46999999999999997,"15min":0.43099999999999999},"max":{"1min":0.92900000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72199999999999998,"15min":0.72799999999999998},"min":{"1min":0.57999999999999996,"5min":0.53700000000000003,"15min":0.48899999999999999},"max":{"1min":0.96599999999999997,"5min":1.052,"15min":1.0640000000000001},"last":0.63}]},{"osd":4,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.76600000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.0649999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.877,"15min":0.88200000000000001},"min":{"1min":0.71999999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.083,"5min":1.101,"15min":1.2050000000000001},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.93600000000000005,"15min":0.93300000000000005},"min":{"1min":0.77300000000000002,"5min":0.61899999999999999,"15min":0.59499999999999997},"max":{"1min":1.0960000000000001,"5min":1.202,"15min":1.202},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91600000000000004,"15min":0.91800000000000004},"min":{"1min":0.78900000000000003,"5min":0.69099999999999995,"15min":0.58499999999999996},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.387},"last":1.087}]},{"osd":6,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.85499999999999998,"15min":0.85899999999999999},"min":{"1min":0.71399999999999997,"5min":0.65000000000000002,"15min":0.64400000000000002},"max":{"1min":1.0549999999999999,"5min":1.1739999999999999,"15min":1.2989999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.998,"5min":0.93899999999999995,"15min":0.95099999999999996},"min":{"1min":0.78500000000000003,"5min":0.72599999999999998,"15min":0.68799999999999994},"max":{"1min":1.2310000000000001,"5min":1.3919999999999999,"15min":1.571},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jul 21 02:58:32 2022","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.95199999999999996,"15min":0.95999999999999996},"min":{"1min":0.76700000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.2829999999999999,"5min":1.4299999999999999,"15min":1.5660000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.024,"5min":0.95799999999999996,"15min":0.95099999999999996},"min":{"1min":0.81299999999999994,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.335,"5min":1.381,"15min":1.4299999999999999},"last":1.1220000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575703,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":798192,"kb_used_data":92901188,"kb_used_omap":3807,"kb_used_meta":0,"kb_avail":92901188,"statfs":{"total":95948165120,"available":95130816512,"internally_reserved":0,"allocated":95130816512,"data_stored":95130816512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3898617,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 21 02:59:13 2022","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59699999999999998,"15min":0.59799999999999998},"min":{"1min":0.50600000000000001,"5min":0.443,"15min":0.442},"max":{"1min":0.82299999999999995,"5min":0.92500000000000004,"15min":1.089},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.45400000000000001},"max":{"1min":0.82799999999999996,"5min":0.98799999999999999,"15min":1.044},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jul 21 02:59:13 2022","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.53500000000000003,"5min":0.53000000000000003,"15min":0.46300000000000002},"max":{"1min":0.746,"5min":0.99099999999999999,"15min":1.048},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.81999999999999995,"5min":1.0469999999999999,"15min":1.0820000000000001},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jul 21 02:59:13 2022","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97899999999999998,"5min":1.155,"15min":1.155},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.84599999999999997,"15min":0.84199999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97399999999999998,"5min":1.121,"15min":1.167},"last":0.80000000000000004}]},{"osd":4,"last update":"Thu Jul 21 02:59:13 2022","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.89000000000000001,"15min":0.88900000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.026,"5min":1.282,"15min":1.3600000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.88600000000000001,"15min":0.88},"min":{"1min":0.70399999999999996,"5min":0.63600000000000001,"15min":0.63100000000000001},"max":{"1min":1.157,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jul 21 02:59:13 2022","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.93400000000000005,"15min":0.93500000000000005},"min":{"1min":0.77500000000000002,"5min":0.68400000000000005,"15min":0.66000000000000003},"max":{"1min":1.2470000000000001,"5min":1.474,"15min":1.474},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.94899999999999995,"15min":0.95299999999999996},"min":{"1min":0.79000000000000004,"5min":0.75600000000000001,"15min":0.70699999999999996},"max":{"1min":1.0740000000000001,"5min":1.55,"15min":1.55},"last":0.99199999999999999}]},{"osd":6,"last update":"Thu Jul 21 02:59:13 2022","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.94599999999999995,"15min":0.93899999999999995},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.196,"5min":1.298,"15min":1.4510000000000001},"last":1.004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.006,"15min":1.0049999999999999},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.70699999999999996},"max":{"1min":1.2,"5min":1.534,"15min":1.637},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jul 21 02:59:13 2022","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.73999999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.1180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0269999999999999,"15min":1.0209999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5860000000000001,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.0960000000000001}]}]}]}} 2022-07-21T02:59:22.127 DEBUG:teuthology.orchestra.run.smithi140:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2022-07-21T02:59:22.233 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.230+0000 7fe07290b700 1 Processor -- start 2022-07-21T02:59:22.236 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.230+0000 7fe07290b700 1 -- start start 2022-07-21T02:59:22.249 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.230+0000 7fe07290b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 0x7fe06c0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:22.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.230+0000 7fe07290b700 1 -- --> v1:172.21.15.140:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe06c0b6680 con 0x7fe06c0b3ed0 2022-07-21T02:59:22.250 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe07290b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe06c1465c0 con 0x7fe06c0b01e0 2022-07-21T02:59:22.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe07290b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c142df0 con 0x7fe06c0b62a0 2022-07-21T02:59:22.251 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06b7fe700 1 --1- >> v1:172.21.15.140:6789/0 conn(0x7fe06c0b3ed0 0x7fe06c143530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.140:6789/0 says I am v1:172.21.15.140:56700/0 (socket says 172.21.15.140:56700) 2022-07-21T02:59:22.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06b7fe700 1 -- 172.21.15.140:0/3410616448 learned_addr learned my addr 172.21.15.140:0/3410616448 (peer_addr_for_me v1:172.21.15.140:0/0) 2022-07-21T02:59:22.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06bfff700 1 --2- 172.21.15.140:0/3410616448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 0x7fe06c0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:22.252 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06affd700 1 -- 172.21.15.140:0/3410616448 <== mon.0 v1:172.21.15.140:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221036381 0 0) 0x7fe06c0b6680 con 0x7fe06c0b3ed0 2022-07-21T02:59:22.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06affd700 1 -- 172.21.15.140:0/3410616448 --> v1:172.21.15.140:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe058003260 con 0x7fe06c0b3ed0 2022-07-21T02:59:22.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06affd700 1 -- 172.21.15.140:0/3410616448 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796636295 0 0) 0x7fe06c1465c0 con 0x7fe06c0b01e0 2022-07-21T02:59:22.253 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06affd700 1 -- 172.21.15.140:0/3410616448 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe06c0b6680 con 0x7fe06c0b01e0 2022-07-21T02:59:22.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06bfff700 1 -- 172.21.15.140:0/3410616448 >> v1:172.21.15.152:6789/0 conn(0x7fe06c0b01e0 legacy=0x7fe06c13fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:22.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06bfff700 1 -- 172.21.15.140:0/3410616448 >> v1:172.21.15.140:6789/0 conn(0x7fe06c0b3ed0 legacy=0x7fe06c143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:22.254 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06bfff700 1 -- 172.21.15.140:0/3410616448 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c143090 con 0x7fe06c0b62a0 2022-07-21T02:59:22.255 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06bfff700 1 --2- 172.21.15.140:0/3410616448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 0x7fe06c0b43e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fe060002f70 tx=0x7fe060006df0).ready entity=mon.2 client_cookie=917f394b291a5177 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:22.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06affd700 1 -- 172.21.15.140:0/3410616448 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe06000d5c0 con 0x7fe06c0b62a0 2022-07-21T02:59:22.256 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06affd700 1 -- 172.21.15.140:0/3410616448 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe060000e90 con 0x7fe06c0b62a0 2022-07-21T02:59:22.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe06affd700 1 -- 172.21.15.140:0/3410616448 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe06000d910 con 0x7fe06c0b62a0 2022-07-21T02:59:22.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe07290b700 1 -- 172.21.15.140:0/3410616448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 msgr2=0x7fe06c0b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:22.257 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe07290b700 1 --2- 172.21.15.140:0/3410616448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 0x7fe06c0b43e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fe060002f70 tx=0x7fe060006df0).stop 2022-07-21T02:59:22.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe07290b700 1 -- 172.21.15.140:0/3410616448 shutdown_connections 2022-07-21T02:59:22.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe07290b700 1 --2- 172.21.15.140:0/3410616448 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 0x7fe06c0b43e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:22.258 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe07290b700 1 -- 172.21.15.140:0/3410616448 >> 172.21.15.140:0/3410616448 conn(0x7fe06c066f00 msgr2=0x7fe06c0672e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:22.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe07290b700 1 -- 172.21.15.140:0/3410616448 shutdown_connections 2022-07-21T02:59:22.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe07290b700 1 -- 172.21.15.140:0/3410616448 wait complete. 2022-07-21T02:59:22.259 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe07290b700 1 Processor -- start 2022-07-21T02:59:22.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.234+0000 7fe07290b700 1 -- start start 2022-07-21T02:59:22.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe07290b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe06c0b01e0 0x7fe06c0ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:22.260 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe07290b700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe06c0b3ed0 0x7fe06c1553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:22.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe07290b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 0x7fe06c0ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:22.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe07290b700 1 -- --> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c0aed20 con 0x7fe06c0b3ed0 2022-07-21T02:59:22.261 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe07290b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c13db00 con 0x7fe06c0b01e0 2022-07-21T02:59:22.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe07290b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c146980 con 0x7fe06c0b62a0 2022-07-21T02:59:22.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe06b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe06c0b3ed0 0x7fe06c1553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:22.262 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe070ea8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 0x7fe06c0ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:22.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe06bfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe06c0b01e0 0x7fe06c0ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:22.263 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe070ea8700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 0x7fe06c0ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.140:39292/0 (socket says 172.21.15.140:39292) 2022-07-21T02:59:22.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe06b7fe700 1 --2- >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe06c0b3ed0 0x7fe06c1553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.140:3300/0 says I am v2:172.21.15.140:42564/0 (socket says 172.21.15.140:42564) 2022-07-21T02:59:22.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe070ea8700 1 -- 172.21.15.140:0/3882443904 learned_addr learned my addr 172.21.15.140:0/3882443904 (peer_addr_for_me v2:172.21.15.140:0/0) 2022-07-21T02:59:22.264 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe070ea8700 1 -- 172.21.15.140:0/3882443904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe06c0b01e0 msgr2=0x7fe06c0ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:22.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe070ea8700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe06c0b01e0 0x7fe06c0ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:22.265 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe070ea8700 1 -- 172.21.15.140:0/3882443904 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe06c0b3ed0 msgr2=0x7fe06c1553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:22.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe070ea8700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe06c0b3ed0 0x7fe06c1553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:22.266 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe070ea8700 1 -- 172.21.15.140:0/3882443904 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06000b180 con 0x7fe06c0b62a0 2022-07-21T02:59:22.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe070ea8700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 0x7fe06c0ae830 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fe064001490 tx=0x7fe064007dd0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:22.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe0697fa700 1 -- 172.21.15.140:0/3882443904 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe064002240 con 0x7fe06c0b62a0 2022-07-21T02:59:22.267 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe0697fa700 1 -- 172.21.15.140:0/3882443904 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0640023a0 con 0x7fe06c0b62a0 2022-07-21T02:59:22.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe07290b700 1 -- 172.21.15.140:0/3882443904 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06c146c20 con 0x7fe06c0b62a0 2022-07-21T02:59:22.268 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe0697fa700 1 -- 172.21.15.140:0/3882443904 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe064001650 con 0x7fe06c0b62a0 2022-07-21T02:59:22.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe07290b700 1 -- 172.21.15.140:0/3882443904 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06c0adb70 con 0x7fe06c0b62a0 2022-07-21T02:59:22.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe07290b700 1 -- 172.21.15.140:0/3882443904 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06c140080 con 0x7fe06c0b62a0 2022-07-21T02:59:22.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.238+0000 7fe0697fa700 1 -- 172.21.15.140:0/3882443904 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 4) v1 ==== 66004+0+0 (secure 0 0 0) 0x7fe064010030 con 0x7fe06c0b62a0 2022-07-21T02:59:22.269 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.242+0000 7fe0697fa700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe04c053a70 0x7fe04c055f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-07-21T02:59:22.270 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.242+0000 7fe0697fa700 1 -- 172.21.15.140:0/3882443904 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fe06406bb00 con 0x7fe06c0b62a0 2022-07-21T02:59:22.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.242+0000 7fe06bfff700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe04c053a70 0x7fe04c055f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-07-21T02:59:22.271 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.242+0000 7fe06bfff700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe04c053a70 0x7fe04c055f30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7fe06001a000 tx=0x7fe060014000).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-07-21T02:59:22.272 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.250+0000 7fe0697fa700 1 -- 172.21.15.140:0/3882443904 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138877 (secure 0 0 0) 0x7fe06c140080 con 0x7fe06c0b62a0 2022-07-21T02:59:22.378 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.374+0000 7fe07290b700 1 -- 172.21.15.140:0/3882443904 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fe06c140080 con 0x7fe06c0b62a0 2022-07-21T02:59:22.391 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.390+0000 7fe0697fa700 1 -- 172.21.15.140:0/3882443904 <== mon.2 v2:172.21.15.152:3301/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe064042c60 con 0x7fe06c0b62a0 2022-07-21T02:59:22.393 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.390+0000 7fe0697fa700 1 -- 172.21.15.140:0/3882443904 <== mon.2 v2:172.21.15.152:3301/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v25)=0 v25) v1 ==== 123+0+0 (secure 0 0 0) 0x7fe06c140080 con 0x7fe06c0b62a0 2022-07-21T02:59:22.401 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 -- 172.21.15.140:0/3882443904 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe04c053a70 msgr2=0x7fe04c055f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:22.402 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe04c053a70 0x7fe04c055f30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7fe06001a000 tx=0x7fe060014000).stop 2022-07-21T02:59:22.403 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 -- 172.21.15.140:0/3882443904 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 msgr2=0x7fe06c0ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-07-21T02:59:22.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 0x7fe06c0ae830 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fe064001490 tx=0x7fe064007dd0).stop 2022-07-21T02:59:22.404 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 -- 172.21.15.140:0/3882443904 shutdown_connections 2022-07-21T02:59:22.405 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.140:6800/22519,v1:172.21.15.140:6801/22519] conn(0x7fe04c053a70 0x7fe04c055f30 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:22.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe06c0b62a0 0x7fe06c0ae830 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:22.406 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.140:3300/0,v1:172.21.15.140:6789/0] conn(0x7fe06c0b3ed0 0x7fe06c1553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:22.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 --2- 172.21.15.140:0/3882443904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe06c0b01e0 0x7fe06c0ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-07-21T02:59:22.407 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 -- 172.21.15.140:0/3882443904 >> 172.21.15.140:0/3882443904 conn(0x7fe06c066f00 msgr2=0x7fe06c0b8460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-07-21T02:59:22.408 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 -- 172.21.15.140:0/3882443904 shutdown_connections 2022-07-21T02:59:22.409 INFO:teuthology.orchestra.run.smithi140.stderr:2022-07-21T02:59:22.394+0000 7fe07290b700 1 -- 172.21.15.140:0/3882443904 wait complete. 2022-07-21T02:59:22.411 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/ceph.py", line 1858, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-07-21T02:59:22.418 INFO:teuthology.misc:Shutting down mds daemons... 2022-07-21T02:59:22.422 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2022-07-21T02:59:22.423 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.440 INFO:tasks.ceph.mds.a:Stopped 2022-07-21T02:59:22.442 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2022-07-21T02:59:22.443 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.459 INFO:tasks.ceph.mds.b:Stopped 2022-07-21T02:59:22.460 INFO:teuthology.misc:Shutting down osd daemons... 2022-07-21T02:59:22.462 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2022-07-21T02:59:22.464 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.503 INFO:tasks.ceph.osd.0:Stopped 2022-07-21T02:59:22.504 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2022-07-21T02:59:22.505 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.527 INFO:tasks.ceph.osd.1:Stopped 2022-07-21T02:59:22.531 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2022-07-21T02:59:22.532 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.546 INFO:tasks.ceph.osd.2:Stopped 2022-07-21T02:59:22.548 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2022-07-21T02:59:22.548 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.609 INFO:tasks.ceph.osd.3:Stopped 2022-07-21T02:59:22.611 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2022-07-21T02:59:22.612 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.629 INFO:tasks.ceph.osd.4:Stopped 2022-07-21T02:59:22.630 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2022-07-21T02:59:22.631 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.648 INFO:tasks.ceph.osd.5:Stopped 2022-07-21T02:59:22.649 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2022-07-21T02:59:22.650 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.669 INFO:tasks.ceph.osd.6:Stopped 2022-07-21T02:59:22.670 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2022-07-21T02:59:22.671 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.732 INFO:tasks.ceph.osd.7:Stopped 2022-07-21T02:59:22.733 INFO:teuthology.misc:Shutting down mgr daemons... 2022-07-21T02:59:22.734 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2022-07-21T02:59:22.735 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.765 INFO:tasks.ceph.mgr.y:Stopped 2022-07-21T02:59:22.767 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2022-07-21T02:59:22.768 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.791 INFO:tasks.ceph.mgr.x:Stopped 2022-07-21T02:59:22.792 INFO:teuthology.misc:Shutting down mon daemons... 2022-07-21T02:59:22.794 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2022-07-21T02:59:22.796 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.826 INFO:tasks.ceph.mon.a:Stopped 2022-07-21T02:59:22.829 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2022-07-21T02:59:22.830 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.852 INFO:tasks.ceph.mon.b:Stopped 2022-07-21T02:59:22.853 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2022-07-21T02:59:22.854 INFO:teuthology.orchestra.run:waiting for 300 2022-07-21T02:59:22.915 INFO:tasks.ceph.mon.c:Stopped 2022-07-21T02:59:22.917 INFO:tasks.ceph:Checking cluster log for badness... 2022-07-21T02:59:22.918 DEBUG:teuthology.orchestra.run.smithi140:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'has not responded to cap revoke by MDS for over' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 'responding to mclientcaps' | head -n 1 2022-07-21T02:59:22.942 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi140.front.sepia.ceph.com 2022-07-21T02:59:22.955 DEBUG:teuthology.orchestra.run.smithi140:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2022-07-21T02:59:23.389 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi140.front.sepia.ceph.com 2022-07-21T02:59:23.390 DEBUG:teuthology.orchestra.run.smithi140:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2022-07-21T02:59:23.653 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi140.front.sepia.ceph.com 2022-07-21T02:59:23.655 DEBUG:teuthology.orchestra.run.smithi140:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2022-07-21T02:59:23.885 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi140.front.sepia.ceph.com 2022-07-21T02:59:23.886 DEBUG:teuthology.orchestra.run.smithi140:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2022-07-21T02:59:24.061 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi152.front.sepia.ceph.com 2022-07-21T02:59:24.062 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2022-07-21T02:59:24.385 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi152.front.sepia.ceph.com 2022-07-21T02:59:24.386 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2022-07-21T02:59:24.646 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi152.front.sepia.ceph.com 2022-07-21T02:59:24.647 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2022-07-21T02:59:24.854 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi152.front.sepia.ceph.com 2022-07-21T02:59:24.856 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2022-07-21T02:59:25.041 INFO:tasks.ceph:Archiving mon data... 2022-07-21T02:59:25.046 DEBUG:teuthology.misc:Transferring archived files from smithi140:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/data/mon.a.tgz 2022-07-21T02:59:25.047 DEBUG:teuthology.orchestra.run.smithi140:> mktemp 2022-07-21T02:59:25.059 INFO:teuthology.orchestra.run.smithi140.stdout:/tmp/tmp.RZVYMdiohT 2022-07-21T02:59:25.060 DEBUG:teuthology.orchestra.run.smithi140:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.RZVYMdiohT 2022-07-21T02:59:25.303 DEBUG:teuthology.orchestra.run.smithi140:> sudo chmod 0666 /tmp/tmp.RZVYMdiohT 2022-07-21T02:59:25.332 DEBUG:teuthology.orchestra.remote:smithi140:/tmp/tmp.RZVYMdiohT is 469KB 2022-07-21T02:59:25.545 DEBUG:teuthology.orchestra.run.smithi140:> rm -fr /tmp/tmp.RZVYMdiohT 2022-07-21T02:59:25.567 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/data/mon.b.tgz 2022-07-21T02:59:25.570 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T02:59:25.586 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.Qom7gfIiTo 2022-07-21T02:59:25.587 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Qom7gfIiTo 2022-07-21T02:59:25.862 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.Qom7gfIiTo 2022-07-21T02:59:25.953 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.Qom7gfIiTo is 593KB 2022-07-21T02:59:26.104 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.Qom7gfIiTo 2022-07-21T02:59:26.127 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/data/mon.c.tgz 2022-07-21T02:59:26.129 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2022-07-21T02:59:26.155 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.VbztKQmpz4 2022-07-21T02:59:26.156 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.VbztKQmpz4 2022-07-21T02:59:26.420 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.VbztKQmpz4 2022-07-21T02:59:26.471 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.VbztKQmpz4 is 593KB 2022-07-21T02:59:26.537 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.VbztKQmpz4 2022-07-21T02:59:26.554 INFO:tasks.ceph:Cleaning ceph cluster... 2022-07-21T02:59:26.556 DEBUG:teuthology.orchestra.run.smithi140:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-07-21T02:59:26.562 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-07-21T02:59:26.603 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-07-21T02:59:26.612 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-07-21T02:59:26.616 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2022-07-21T02:59:26.634 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2022-07-21T02:59:26.635 DEBUG:teuthology.orchestra.run.smithi140:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-07-21T02:59:26.639 DEBUG:teuthology.orchestra.run.smithi152:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-07-21T02:59:26.675 DEBUG:teuthology.orchestra.run.smithi191:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-07-21T02:59:26.687 DEBUG:teuthology.orchestra.run.smithi192:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-07-21T02:59:26.692 DEBUG:teuthology.orchestra.run.smithi195:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2022-07-21T02:59:26.698 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-07-21T02:59:26.718 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-07-21T02:59:26.723 INFO:teuthology.orchestra.run.smithi140.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-07-21T02:59:26.725 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-07-21T02:59:26.735 INFO:teuthology.orchestra.run.smithi195.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2022-07-21T02:59:26.738 INFO:tasks.ceph:Archiving crash dumps... 2022-07-21T02:59:26.741 DEBUG:teuthology.misc:Transferring archived files from smithi140:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi140/crash 2022-07-21T02:59:26.777 DEBUG:teuthology.orchestra.run.smithi140:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-07-21T02:59:26.800 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi152/crash 2022-07-21T02:59:26.807 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-07-21T02:59:26.842 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi191/crash 2022-07-21T02:59:26.845 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-07-21T02:59:26.874 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi192/crash 2022-07-21T02:59:26.879 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-07-21T02:59:26.905 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi195/crash 2022-07-21T02:59:26.908 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2022-07-21T02:59:26.928 INFO:tasks.ceph:Shutting down logrotate 2022-07-21T02:59:26.929 DEBUG:teuthology.orchestra.run.smithi140:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:26.932 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:26.937 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:26.943 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:26.947 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:27.000 DEBUG:teuthology.orchestra.run.smithi140:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:27.047 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:27.084 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:27.129 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:27.152 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm /etc/logrotate.d/ceph-test.conf 2022-07-21T02:59:27.182 INFO:tasks.ceph:Compressing logs... 2022-07-21T02:59:27.183 DEBUG:teuthology.orchestra.run.smithi140:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-07-21T02:59:27.187 DEBUG:teuthology.orchestra.run.smithi152:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-07-21T02:59:27.193 DEBUG:teuthology.orchestra.run.smithi191:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-07-21T02:59:27.204 DEBUG:teuthology.orchestra.run.smithi192:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-07-21T02:59:27.210 DEBUG:teuthology.orchestra.run.smithi195:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-07-21T02:59:36.509 INFO:tasks.ceph:Archiving logs... 2022-07-21T02:59:36.510 DEBUG:teuthology.misc:Transferring archived files from smithi140:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi140/log 2022-07-21T02:59:36.512 DEBUG:teuthology.orchestra.run.smithi140:> sudo tar cz -f - -C /var/log/ceph -- . 2022-07-21T02:59:39.712 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi152/log 2022-07-21T02:59:39.718 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar cz -f - -C /var/log/ceph -- . 2022-07-21T02:59:41.992 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi191/log 2022-07-21T02:59:42.010 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /var/log/ceph -- . 2022-07-21T02:59:42.513 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi192/log 2022-07-21T02:59:42.530 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/log/ceph -- . 2022-07-21T02:59:42.599 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi195/log 2022-07-21T02:59:42.601 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/log/ceph -- . 2022-07-21T02:59:42.627 DEBUG:teuthology.run_tasks:Unwinding manager install 2022-07-21T02:59:42.642 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/task/install/__init__.py", line 619, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/ceph_fuse.py", line 160, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait if not (self.is_mounted() and self.fuse_daemon): File "/home/teuthworker/src/github.com_ceph_ceph-c_57544da114dbbe1c44f27b2ea18e3233e4e917e0/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_89d874d555f56b1c1f383df72487aecec0ee6cf9/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2022-07-21T02:59:42.646 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2022-07-21T02:59:42.647 DEBUG:teuthology.orchestra.run.smithi140:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-07-21T02:59:42.653 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-07-21T02:59:42.671 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-07-21T02:59:42.680 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-07-21T02:59:42.686 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2022-07-21T02:59:42.880 DEBUG:teuthology.run_tasks:Unwinding manager clock 2022-07-21T02:59:42.918 INFO:teuthology.task.clock:Checking final clock skew... 2022-07-21T02:59:42.920 DEBUG:teuthology.orchestra.run.smithi140:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-07-21T02:59:42.934 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-07-21T02:59:42.951 INFO:teuthology.orchestra.run.smithi140.stdout: remote refid st t when poll reach delay offset jitter 2022-07-21T02:59:42.952 INFO:teuthology.orchestra.run.smithi140.stdout:============================================================================== 2022-07-21T02:59:42.953 INFO:teuthology.orchestra.run.smithi140.stdout:+hv01.front.sepi 162.248.241.94 3 u 95 128 377 0.141 4.090 0.426 2022-07-21T02:59:42.954 INFO:teuthology.orchestra.run.smithi140.stdout:*hv02.front.sepi 163.237.218.18 2 u 52 128 377 0.181 -2.881 0.252 2022-07-21T02:59:42.954 INFO:teuthology.orchestra.run.smithi140.stdout:+hv03.front.sepi 216.229.4.69 3 u 101 128 377 0.162 -0.627 0.298 2022-07-21T02:59:42.955 INFO:teuthology.orchestra.run.smithi140.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-07-21T02:59:42.956 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-07-21T02:59:42.960 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-07-21T02:59:42.973 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-07-21T02:59:42.978 INFO:teuthology.orchestra.run.smithi152.stdout: remote refid st t when poll reach delay offset jitter 2022-07-21T02:59:42.979 INFO:teuthology.orchestra.run.smithi152.stdout:============================================================================== 2022-07-21T02:59:42.980 INFO:teuthology.orchestra.run.smithi152.stdout:+hv01.front.sepi 162.248.241.94 3 u 24 256 377 0.126 2.835 0.473 2022-07-21T02:59:42.980 INFO:teuthology.orchestra.run.smithi152.stdout:*hv02.front.sepi 163.237.218.18 2 u 39 256 377 0.185 -3.982 0.336 2022-07-21T02:59:42.981 INFO:teuthology.orchestra.run.smithi152.stdout:+hv03.front.sepi 216.229.4.69 3 u 103 128 377 0.176 -1.700 0.230 2022-07-21T02:59:42.983 INFO:teuthology.orchestra.run.smithi152.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-07-21T02:59:42.987 INFO:teuthology.orchestra.run.smithi191.stdout: remote refid st t when poll reach delay offset jitter 2022-07-21T02:59:42.988 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================== 2022-07-21T02:59:42.989 INFO:teuthology.orchestra.run.smithi191.stdout:+hv01.front.sepi 162.248.241.94 3 u 24 128 377 0.170 10.693 0.662 2022-07-21T02:59:42.989 INFO:teuthology.orchestra.run.smithi191.stdout:*hv02.front.sepi 163.237.218.18 2 u 47 128 377 0.188 4.500 0.544 2022-07-21T02:59:42.990 INFO:teuthology.orchestra.run.smithi191.stdout:+hv03.front.sepi 216.229.4.69 3 u 115 128 377 0.210 6.091 0.613 2022-07-21T02:59:42.990 INFO:teuthology.orchestra.run.smithi191.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-07-21T02:59:42.994 INFO:teuthology.orchestra.run.smithi192.stdout: remote refid st t when poll reach delay offset jitter 2022-07-21T02:59:42.995 INFO:teuthology.orchestra.run.smithi192.stdout:============================================================================== 2022-07-21T02:59:42.995 INFO:teuthology.orchestra.run.smithi192.stdout:+hv01.front.sepi 162.248.241.94 3 u 42 256 377 0.182 2.742 1.038 2022-07-21T02:59:42.996 INFO:teuthology.orchestra.run.smithi192.stdout:*hv02.front.sepi 163.237.218.18 2 u 52 256 377 0.197 -3.689 0.600 2022-07-21T02:59:42.997 INFO:teuthology.orchestra.run.smithi192.stdout:+hv03.front.sepi 216.229.4.69 3 u 230 256 377 0.160 -0.781 0.516 2022-07-21T02:59:42.998 INFO:teuthology.orchestra.run.smithi192.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-07-21T02:59:43.000 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2022-07-21T02:59:43.001 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2022-07-21T02:59:43.002 INFO:teuthology.orchestra.run.smithi195.stdout:+hv01.front.sepi 162.248.241.94 3 u 102 128 377 0.147 4.129 0.448 2022-07-21T02:59:43.003 INFO:teuthology.orchestra.run.smithi195.stdout:*hv02.front.sepi 163.237.218.18 2 u 34 128 377 0.162 -2.735 0.375 2022-07-21T02:59:43.004 INFO:teuthology.orchestra.run.smithi195.stdout:+hv03.front.sepi 216.229.4.69 3 u 99 128 377 0.136 -0.766 0.208 2022-07-21T02:59:43.005 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2022-07-21T02:59:43.006 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2022-07-21T02:59:43.032 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2022-07-21T02:59:43.034 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2022-07-21T02:59:43.046 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2022-07-21T02:59:43.084 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2022-07-21T02:59:43.124 INFO:teuthology.task.internal:Duration was 4726.428793 seconds 2022-07-21T02:59:43.126 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2022-07-21T02:59:43.160 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2022-07-21T02:59:43.161 DEBUG:teuthology.orchestra.run.smithi140:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-07-21T02:59:43.165 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-07-21T02:59:43.172 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-07-21T02:59:43.176 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-07-21T02:59:43.180 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2022-07-21T02:59:43.244 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2022-07-21T02:59:43.246 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi140.front.sepia.ceph.com 2022-07-21T02:59:43.248 DEBUG:teuthology.orchestra.run.smithi140:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-07-21T02:59:43.268 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2022-07-21T02:59:43.269 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-07-21T02:59:43.289 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2022-07-21T02:59:43.293 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-07-21T02:59:43.316 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2022-07-21T02:59:43.318 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-07-21T02:59:43.335 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2022-07-21T02:59:43.336 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2022-07-21T02:59:43.358 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2022-07-21T02:59:43.360 DEBUG:teuthology.orchestra.run.smithi140:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-07-21T02:59:43.374 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-07-21T02:59:43.389 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-07-21T02:59:43.399 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-07-21T02:59:43.427 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2022-07-21T02:59:43.467 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2022-07-21T02:59:43.469 DEBUG:teuthology.orchestra.run.smithi140:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-07-21T02:59:43.479 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-07-21T02:59:43.489 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-07-21T02:59:43.502 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-07-21T02:59:43.509 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2022-07-21T02:59:43.612 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2022-07-21T02:59:43.628 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2022-07-21T02:59:43.629 DEBUG:teuthology.orchestra.run.smithi140:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-07-21T02:59:43.651 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-07-21T02:59:43.671 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-07-21T02:59:43.692 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-07-21T02:59:43.716 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2022-07-21T02:59:43.736 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2022-07-21T02:59:43.773 DEBUG:teuthology.orchestra.run.smithi140:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-07-21T02:59:43.780 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-07-21T02:59:43.785 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-07-21T02:59:43.789 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-07-21T02:59:43.791 INFO:teuthology.orchestra.run.smithi140.stdout:kernel.core_pattern = core 2022-07-21T02:59:43.795 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2022-07-21T02:59:43.797 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2022-07-21T02:59:43.802 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2022-07-21T02:59:43.812 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2022-07-21T02:59:43.813 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2022-07-21T02:59:43.832 DEBUG:teuthology.orchestra.run.smithi140:> test -e /home/ubuntu/cephtest/archive/coredump 2022-07-21T02:59:43.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T02:59:43.866 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2022-07-21T02:59:43.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T02:59:43.881 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2022-07-21T02:59:43.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T02:59:43.889 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2022-07-21T02:59:43.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T02:59:43.898 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2022-07-21T02:59:43.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-07-21T02:59:43.916 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2022-07-21T02:59:43.938 INFO:teuthology.task.internal:Transferring archived files... 2022-07-21T02:59:43.941 DEBUG:teuthology.misc:Transferring archived files from smithi140:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi140 2022-07-21T02:59:43.942 DEBUG:teuthology.orchestra.run.smithi140:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-07-21T02:59:43.997 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi152 2022-07-21T02:59:43.998 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-07-21T02:59:44.046 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi191 2022-07-21T02:59:44.048 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-07-21T02:59:44.125 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi192 2022-07-21T02:59:44.145 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-07-21T02:59:44.212 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2022-07-20_20:56:09-fs-octopus-release-distro-default-smithi/6941365/remote/smithi195 2022-07-21T02:59:44.213 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2022-07-21T02:59:44.263 INFO:teuthology.task.internal:Removing archive directory... 2022-07-21T02:59:44.264 DEBUG:teuthology.orchestra.run.smithi140:> rm -rf -- /home/ubuntu/cephtest/archive 2022-07-21T02:59:44.275 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2022-07-21T02:59:44.281 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2022-07-21T02:59:44.286 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2022-07-21T02:59:44.298 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2022-07-21T02:59:44.313 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2022-07-21T02:59:44.336 INFO:teuthology.task.internal:Not uploading archives. 2022-07-21T02:59:44.337 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2022-07-21T02:59:44.380 INFO:teuthology.task.internal:Tidying up after the test... 2022-07-21T02:59:44.390 DEBUG:teuthology.orchestra.run.smithi140:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-07-21T02:59:44.406 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-07-21T02:59:44.418 INFO:teuthology.orchestra.run.smithi140.stdout: 524316 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 02:59 /home/ubuntu/cephtest 2022-07-21T02:59:44.428 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-07-21T02:59:44.435 INFO:teuthology.orchestra.run.smithi152.stdout: 524316 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 02:59 /home/ubuntu/cephtest 2022-07-21T02:59:44.457 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-07-21T02:59:44.469 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2022-07-21T02:59:44.476 INFO:teuthology.orchestra.run.smithi192.stdout: 524316 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 02:59 /home/ubuntu/cephtest 2022-07-21T02:59:44.492 INFO:teuthology.orchestra.run.smithi195.stdout: 524316 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 02:59 /home/ubuntu/cephtest 2022-07-21T13:41:14.096 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2022-07-21T13:41:14.148 DEBUG:teuthology.task.console_log:Killing console logger for smithi140 2022-07-21T13:41:14.152 DEBUG:teuthology.task.console_log:Killing console logger for smithi152 2022-07-21T13:41:14.153 DEBUG:teuthology.task.console_log:Killing console logger for smithi191 2022-07-21T13:41:14.165 DEBUG:teuthology.task.console_log:Killing console logger for smithi192 2022-07-21T13:41:14.166 DEBUG:teuthology.task.console_log:Killing console logger for smithi195 2022-07-21T13:41:14.169 DEBUG:teuthology.exit:Finished running handlers